In 2293, the location of the First Federation in the Milky Way Galaxy was labeled in a star chart that was in Captain James T. Kirk's quarters aboard the USS Enterprise-A. (Star Trek VI: The Undiscovered Country, okudagram)
First contact between the United Federation of Planets and the First Federation occurred in 2266, when the USS Enterprise was conducting a star mapping mission and unknowingly crossed into First Federation space, destroying a marker buoy in the process.
Shortly thereafter, the Enterprise encountered the massive flagship of the First Federation, the Fesarius. Charged with violating their territory, the Enterprise and its crew were threatened with destruction. However, after a series of bluffs and counter-bluffs, it was discovered that the encounter had only been a test, designed by the Fesarius' captain, Balok. A cultural exchange was proposed, with Enterprise crewman Dave Bailey remaining on board the Fesarius. (TOS: "The Corbomite Maneuver")
According to Star Trek: Star Charts (p. 36 & "United Federation of Planets I") and Stellar Cartography: The Starfleet Reference Library ("Stellar Cartography", p. 10; "Federation Historical Highlights, 2161-2385"), the First Federation's homeworld and capital system were named Fesarius. The primary of this system was a K-class star. There were warning buoys encircling First Federation territory in the late 24th century.
The Pocket TOS novel The Face of the Unknown centers around a visit to the First Federation. It is depicted as a series of domes within a Jupiter-like planet that house species and natural environments similar to several destroyed planets.