Gençlerbirliği was founded in a slightly different way to many other football clubs, having been established by students from a high school, Ankara Erkek Lisesi (Sultani Mektebi). The students in question (Ramiz Eren, Mennan İz, Mazhar Atacanlı, Sait, Kenan, Nuri, Namık Katoğlu, Namık Ambarcıoğlu, Rıdvan Kırmacı, Hafi Araç, Ruhi, Sarı Ziya and Hakkı) had failed to be selected for their own school team, and asked one of their student friends, Asim, to talk with his father about setting up a football team for them to show how well they could play.
On 14 March 1923, Asim's father, the member of parliament from Muş Province established the team for his son and his friends. Since all the members of the teams were students he chose the name 'Youth Union'. They elected Sarı Ziya's father Faik Bey as their first chairman.
Subsequently these young students wanted to play against the school team which had not selected them. Gençlerbirliği won the game, played at "Hamit tarlası", 3–0. According to these young rebel players this victory forged the identity of their new team.
They won Ankara Football League 10 times between 1923–59. They entered the Turkish National League in 1959 and played 12 seasons until relegation in 1969–70 season. They played in the second level until relegation to third level in 1978–79 season. They came back to second level after merging third level with second level after finishing 1979–80 season. They finished second from last in A Group of Second League, Sakaryaspor won championship, and had to relegation to regional league. But, because of increasing teams of the second level, Gençlerbirliği was readmitted. Gençlerbirliği won D Group of it in 1982–83 season and returned to the top-flight after 13 years. They relegated again in 1987–88 season. Finally, Gençlerbirliği returned to the top-flight in 1988–89 season, in first return attempt. Gençlerbirliği's the most successful seasons were 1965–66 and 2002–03. They had 3rd in these seasons.
There are two rumors about how Gençlerbirliği got its team colors, black and red:
The young students went to a local haberdashers and the only available colors were black and red. They purchased these colors to sew their own kit. 
The outskirts of Ankara during spring time sees plenty of Poppy flowers (Gelincik) blooming and Gençlerbirliği selected the red and black of this flower for its colors. Gençlerbirliği merchandise in its store is usually called Gelincik by its supporters.
Gençlerbirliği has always been one of the teams in Turkey most notable for its talented young footballers that it procures through scouting both in Turkey and throughout Europe and Africa. For example, Geremi was scouted and raised by the club, transferred to Real Madrid and later Chelsea. Another of Gençlerbirliği's young stars Isaac Promise received the 2005–06 Turkcell Super League Individual Youth player of the year award.
Gençlerbirliği's state-of-the-art youth academy is in Beştepe, Ankara with 50 acres (200,000 m2) of training facilities (Beştepe İlhan Cavcav Tesisleri).
Gençlerbirliği has been run by the current president İlhan Cavcav since 1977. With Cavcav's help, Gençlerbirliği have become one of the more stable clubs in Turkey. This has been achieved mainly through the departure of the most talented players every season to the other clubs in Turkey. Departing players are usually replaced with much cheaper imports from Turkey, Europe or Africa.