The EMH backup module was a 24th century computer memory storage device carried on Starfleet vessels equipped with the Emergency Medical Hologram program. It was capable of storing data too complex or too large for contemporary systems. As such, it was used to store backup versions of the EMH, on a frequent basis, in case it was lost or damaged.
The USS Voyager's backup module was stolen during an attack and recovered centuries later by the Kyrians. This version of The Doctor helped to reshape Kyrian and Vaskan history and served as the Kyrian surgical chancellor for many years before finally taking a small vessel and setting course for the Alpha Quadrant in an attempt to trace the path of Voyager. (VOY: "Living Witness")
Apocrypha
In the String Theory trilogy of novels, it was revealed that there were several backup EMHs, most of them based on different versions of The Doctor according to when the information was "saved." In the second novel in the trilogy, Fusion, a Nacene who had infiltrated the crew by altering their memories was forced to shut down the Doctor and activate the first backup version (essentially the "original" EMH), as the Doctor knew that she was not who she seemed. Despite his lack of practical experience, the new EMH was still able to confirm that the crew's memories had been altered, as well as explain why Harry Kim and Naomi Wildman had not been affected due to the events of VOY: "Deadlock".