A boundary was a dividing line. It could be a metaphorical line separating culturally acceptable actions and/or beliefs from technically possible ones, or an actual line demarcating the physical end of something.
In 2367, while Captain Jean-Luc Picard and Worf had a talk in the ready room about the Klingon High Council, Worf said, "I have acted within the boundaries of Klingon law and tradition." (TNG: "Reunion")