Heresy was a term for beliefs contrary to established doctrine, also known as dogma. This dogma might be religious in nature. Some cultures viewed heresy as a crime. Someone espousing heresy was called a heretic.
In 2153, a group of Triannons seized control of Enterprise NX-01, hoping to eliminate a rival faction. Believing that the Delphic Expanse, or "Chosen Realm," was created in nine days, they considered the rival faction's belief that it was created in ten days to be heresy. Furthermore, their leader, Pri'Nam D'Jamat, believed that any questioning of his word by his followers could be considered heresy. (ENT: "Chosen Realm")
In 2373, Doctor Julian Bashir caused great commotion at a medical conference with a paper that stepped on some toes. He later explained to Jake Sisko that it was heresy to even consider the possibility that prion replication could be inhibited by quantum resonance effects. (DS9: "Nor the Battle to the Strong")
Late that year, the Voth Ministry of Elders charged Professor Forra Gegen with the crime of heresy for teaching Distant Origin Theory, a scientific theory that challenged Voth Doctrine. Despite the evidence standing before them, Professor Gegen was still found guilty. (VOY: "Distant Origin")
In 2374, Miles O'Brien confided to Lisa Cusak that saying out loud that ship's counselors sometimes just got in the way was almost heresy. (DS9: "The Sound of Her Voice")
In 2375, upon discovering that Dukat (posing as Anjohl Tennan) worshiped the Pah-wraiths, Kai Winn angrily accused him of being a heretic. (DS9: "Strange Bedfellows")