Mount Olympus was a mountain in the ancient nation-state of Greece, located in Europe's Mediterranean region on Earth. It figured heavily in the universe of Greek mythology as the home of the gods.
In reality, these gods were aliens with psychokinetic abilities that made them god-like to ancient Greeks. After Humans outgrew the need for such gods, they fled Olympus to Pollux IV.
In 2267, attempting to bring Carolyn Palamas back from her reverie regarding Apollo, James T. Kirk remarked, "All right, lieutenant, you can come down from Mount Olympus now. You've got work to do." (TOS: "Who Mourns for Adonais?")
In the script for DS9: "Distant Voices", for Dr. Julian Bashir's surprise birthday party in 2371, the crew of Deep Space 9 had the choices narrowed down to two different holosuite programs: a Greek palace on Mount Olympus or Louis Pasteur's house in Paris. This scene does not appear in the aired episode.