Serration was a series of points similar to teeth that ran along the edge of a blade.
In 2327, a young Jean-Luc Picard was in shock at the sight of a serrated Nausicaan weapon sticking out of his chest. (TNG: "Samaritan Snare")
In 2370, Data dreamed that he cut a cellular peptide cake resembling Deanna Troi, using a serrated knife. (TNG: "Phantasms")