(covers information from several alternate timelines)
A witch was a woman thought to possess magical powers granted to her by a compact with the Devil, a powerful and evil being, through the use of witchcraft and black magic. They were similar in nature to wizards and warlocks.
The witch's traditional connection to evil has been used in many plays, novels, and stories for centuries on Earth; it has essentially been woven into the Human racial subconscious. And this was where the alien explorer Korob, living on Pyris VII, found it when he telepathically scanned a USS Enterprise landing party in 2267. Witches were among the eerie manifestations of his power; he created an illusion of three witches either to dissuade Captain Kirk or to test his character.
During Kirk and the landing party's encounter with the three witches, the spectral hags warned Kirk to go back, reminded him of the curse laid on his ship, and used doggerel to advise him that the consequence of remaining was death:
Following this, the shades faded from view (their cackling the last to go); soon after, gusts of wind battered the party and the fog thickened. Spock speculated later that the imagery, including the witches, was possibly calculated to frighten visitors, and might be the result of an attempt to probe Human minds that missed, reaching only the fears normally bottled up in the subconscious.
Korob and his colleague Sylvia, presented themselves as a wizard and witch, complete with ancient, fog-bound castles, magical implements and accouterments, and displays of power intended to suggest magic. Korob later told Kirk that all of it was created as a test of loyalty, bravery and character. (TOS: "Catspaw")
After Admiral Norah Satie's failed attempt to expose a non-existent conspiracy within the crew of the USS Enterprise-D, Captain Jean-Luc Picard compared the situation to the, "torture of heretics and the burning of witches." (TNG: "The Drumhead")