The kai was the religious leader of the Bajorans, elected from among and by the Vedek Assembly to a life term. A symbol of strength and unity to the Bajorans for centuries, the kai's religious authority was rivaled only by that of the Emissary of the Prophets. Besides religious power, the kai also had a great deal of political influence in the Bajoran government. Disputes between the kai and the first minister of the government, while rare, could be extremely divisive. While it was not forbidden for the kai to serve as first minister, such involvement in political matters was discouraged. The kai was properly addressed as "Eminence". (DS9: "Emissary", "The Collaborator", "Shakaar", "In the Cards")
When the Cardassian Union began its occupation of the planet, the Bajoran religion was outlawed. Power passed from then-Kai Meressa to Opaka, who protected her people as best she could and became revered. However, she went into isolation during the last years of the occupation. With the conclusion of the Cardassian Withdrawal in 2369, it was expected that Opaka was the only one who could unite the Bajorans, yet she remained isolated. The arrival of Benjamin Sisko, the Emissary, and Opaka's disappearance that same year significantly decreased the influence of the kai on Bajoran life. (DS9: "Emissary", "Battle Lines")
Bajor was spiritually and politically leaderless as the Bajoran Provisional Government struggled to maintain order, and the Vedek Assembly did not elect a new kai until 2370. The election was largely a choice between Vedeks Bareil Antos and Winn Adami. As common knowledge held that Bareil was Opaka's hand-picked successor, his election seemed inevitable, but the threat of a scandal involving the Kendra Valley Massacre led him to drop out of the race in order to keep the certain secrets about Kai Opaka's involvement from coming to light. Winn was thus assured victory and easily won the election. (DS9: "The Collaborator")
Winn served as kai for over five years, during which time Bajor saw significant changes. Relations with the United Federation of Planets were tentative, yet despite Winn's opposition to the Federation's presence the authority of the kai was superseded by that of the Emissary. She eventually made amends with Captain Sisko but was never satisfied with his status as Emissary. The signing of the Bajoran-Cardassian Treaty took place in 2371, and as kai, Winn took credit for the move even though Vedek Bareil was largely responsible for the negotiations, sacrificing his health (to the point of giving his life) to ensure the treaty would work out so Bajor could begin healing. (DS9: "Life Support", "The Reckoning")
Popularity from the signing of the treaty with the Cardassians allowed Winn to briefly ascend to the position of first minister, essentially giving her complete control of Bajor. Although she was defeated in a formal election by Shakaar Edon, she remained popular on Bajor despite her opportunistic nature. The position of kai suffered a tremendous blow when Winn betrayed the Prophets and aided Dukat in his attempt to release the Pah-wraiths from the Fire Caves, however. When Winn was killed in 2375, Quark, who was taking bets on who the next kai would be, told Colonel Kira Nerys that "the smart money was on Vedek Ungtae." (DS9: "Shakaar", "The Reckoning", "What You Leave Behind")
According to the script for "Battle Lines", this title's name was pronounced as "KYE". 
The term "kai" was likely derived by the writers from the Welsh name which means "keeper of the keys" or "kai", the Persian title for kings in the Shanameh following the reign of Kai Kavus.
The position of kai (at least Kai Winn's interpretation of it) can be likened to that of the pope in Catholicism. Winn actress Louise Fletcher herself made the analogy, remarking, "I think therefore that when you're elected kai, whatever you say is the law… the religious law." (Star Trek Monthly issue 34, p. 50) Also, the kai was explicitly compared with the pope in the script of "The Reckoning"  (though the Emissary was also once compared to the pope, in the script of "Rapture").  As with the kai, the pope was elected from among the senior ranks of the clergy, and the members of both the Bajoran religion and Catholicism were supposed to follow the leader's spiritual guidance. However, Winn's brief term as first minister of Bajor and her active role in Bajoran political affairs thereafter was akin to the papacy of Medieval Europe; see Pope Innocent III at Wikipedia. Innocent ascended to the papacy shortly before a vacancy in rule of the Holy Roman Empire, which was headquartered in Germany, and he used the opening as a chance to gain both spiritual and political control over much of what was now Italy. Any direct similarities are probably coincidental, however.
It should be noted that the true nature of the role of kai was somewhat uncertain. The only characters to have the title were Opaka and Winn, and while the former went into isolation during the Cardassian Occupation, the latter (as mentioned above) took the opposite direction and used her role to gain an increasing level of power. We therefore have no experience with a "normal" kai and any assumptions about the nature of the role may be based on skewed information.
Unlike Catholicism, which can only elevate men to hold the position of pope, selection of the kai (and Bajoran clergy in general) appeared to be gender-neutral. Also, it appeared that the kai could be married and have children – Kai Opaka had at least one son.
It is interesting to note that the clothes of the kai were very similar to the traditional garments of the Doge of Venice; Kai Winn's cap was similar to the traditional cap of the Doge and her garments were almost identical to Doge Lenardo Loredan as painted by Bellini.