Exotic matter referred to hypothetical particles, states of matter, materials, or substances that exhibited "exotic" properties that were not normally possible in a standard model of physics, such as negative mass.
In 2383, the Protostar-class of Federation starship utilized exotic matter with a dilithium matrix to power the proto-drive and generate a protostar. This was required for an enhanced form of warp travel, known as proto-warp. When the exotic matter dilithium matrix was removed, the protostar drive would not function. (PRO: "A Moral Star, Part 1")