The EMH backup module was a 24th century computer memory storage device carried on Starfleet vessels equipped with the Emergency Medical Hologram program. It was capable of storing data too complex or too large for contemporary systems. As such, it was used to store backup versions of the EMH, on a frequent basis, in case it was lost or damaged.
The USS Voyager's backup module was stolen during an attack and recovered centuries later by the Kyrians. This version of The Doctor helped to reshape Kyrian and Vaskan history by debunking the Kyrians' warped view of Voyager's role in the Great War and served as the Kyrian / Vaskan Surgical Chancellor for many years before finally taking a small vessel and setting course for the Alpha Quadrant in an attempt to trace the path of Voyager. He claimed to have "a longing for home." (VOY: "Living Witness")
The existence of a backup module would seem to contradict "Message in a Bottle", in which Tom Paris and Harry Kim tried and failed to create another version of The Doctor. However, since no date or stardate was given in "Living Witness", the only things that date when the module was stolen is Seven of Nine and the state of Voyager's distance from home being 60,000 light-years, placing the episode sometime after "The Gift" but before "Timeless". This means that the module could have been stolen before the events in "Message in a Bottle", or it could have been created afterward and then stolen. The determinative question never directly answered by any episode of the series is whether Voyager launched with this technology on board. Kim's ability to create the Crell Moset program in "Nothing Human" seems to suggest the backup program was created after "Message in a Bottle" and stolen before "Timeless". Voyager's presence in the Vaskan sector in "Demon" as revealed in "Course: Oblivion" also seems to suggest that the module was stolen around the end of Voyager's fourth season.
In the String Theory trilogy of novels, it was revealed that there were several backup EMHs, most of them based on different versions of The Doctor according to when the information was "saved." In the second novel in the trilogy, Fusion, a Nacene who had infiltrated the crew by altering their memories- creating the illusion that she was Janeway's sister, Phoebe Janeway, who had accompanied Voyager on their original mission to create a portrait of the Badlands and remained with the crew ever since-, was forced to shut down The Doctor and activate the first backup version – essentially the "original" EMH activated in "Caretaker" – as The Doctor knew that she was not who she seemed (Significantly, 'Phoebe' had initially contemplated simply altering his programmed memories, but her analysis of The Doctor showed some undefined quality within him that made him more than a simple computer program, suggesting that The Doctor's development at this point in his existence had advanced to a point where he had acquired some kind of 'soul'). Despite his lack of practical experience, the new EMH was still able to confirm that the crew's memories had been altered, as well as explain why Harry Kim and Naomi Wildman had not been affected by the alteration, as the two were slightly 'out of phase' with the rest of the crew as a result of them being duplicates created during the events of VOY: "Deadlock", before B'Elanna Torres was able to restore the original Doctor.