A mystery novel was a type of novel where the plot focused on a crime committed by an unknown person or persons.
Julian Bashir preferred Human mystery novels to Cardassian enigma tales, because unlike the Cardassian stories, the endings varied. He explained this to Elim Garak in 2371 after the tailor gave him a holoprogram adaptation of one of Shoggoth's enigma tales. (DS9: "Distant Voices")