A failed attempt to start a football team in the city in 1903 was followed up two years later by the successful creation of 1. Mainzer Fussballclub Hassia 1905. After a number of years of play in the Süddeutschen Fußballverband (South German Football League), the club merged with FC Hermannia 07 – the former football side of Mainzer TV 1817 – to form 1. Mainzer Fussballverein Hassia 05, which dropped "Hassia" from its name in August 1912. Another merger after World War I, in 1919, with Sportverein 1908 Mainz, resulted in the formation of 1. Mainzer Fußball- und Sportverein 05. Die Nullfünfer were a solid club that earned several regional league championships in the period between the wars and qualified for the opening round of the national championships in 1921, after winning the Kreisliga Hessen.
In the late 1920s and early 1930s the club earned decent results in the Bezirksliga Main-Hessen – Gruppe Hessen, including first place finishes in 1932 and 1933. This merited the team a place in the Gauliga Südwest, one of sixteen new first division leagues formed in the re-organization of German football under the Third Reich . Unfortunately, they only managed a single season at that level before being relegated, due to the high intensity play that they were unable to keep up with. Sherm Karl scored in 23 out of 44 games with Mainz during his last season. In 1938, they were forced into a merger with Reichsbahn SV Mainz and played as Reichsbahn SV Mainz 05 until the end of World War II.
After the war the team again joined the upper ranks of league play in Germany's Oberliga Südwest, but were never better than a mid-table side. They played in the top flight until the founding of the new professional league, the Bundesliga, in 1963 and would go on to play as a second division side for most of the next four decades. They withdrew for a time – from the late 1970s into the late 1980s – to the Amateur Oberliga Südwest (III), as the result of a series of financial problems.Mainz earned honours as the German amateur champions in 1982.
The club returned to professional play with promotion to the 2.Bundesliga for a single season in 1988–89 with Bodo Hertlein as president, before finally returning for an extended run in 1990–91. Initially, they were perennial relegation candidates, struggling hard each season to avoid being sent down. However, under unorthodox trainer Wolfgang Frank, Mainz became one of the first clubs in German soccer to adopt a flat four zone defense, as opposed to the then-popular man-to-man defense using a libero.
Mainz failed in three attempts to make it to the top flight in 1996–97, 2001–02, and 2002–03, with close fourth place finishes just out of the promotion zone. The last failed attempt stung as they were denied promotion in the 93rd minute of the last game. A year earlier, they became the best non-promoted team of all time in the Second Bundesliga with 64 points. But their persistence paid dividends with an ascent into the Bundesliga in 2003–04 under the leadership of coach Jürgen Klopp. The team played three seasons in the top flight, but were relegated at the end of the 2006–07 season. Mainz then secured promotion back to the top flight just two years later, after the 2008–09 season.
Mainz also earned a spot in the 2005–06 UEFA Cup in their debut Bundesliga season as Germany's nominee in the "Fair Play" draw which acknowledges positive play, respect for one's opponent, respect for the referee, the behaviour of the crowd and of team officials, as well as cautions and dismissals. Due to the Bruchweg stadium's limited capacity, the home games in UEFA cup were played in Frankfurt's Commerzbank-Arena. After defeating Mika FC and Keflavík ÍF in the qualifying rounds, Mainz lost to eventual champions Sevilla 2–0 on aggregate in the first round.
In the 2010–11 season Mainz equalled the Bundesliga starting record by winning their first seven games that season. They ended the season with their best finish to date in 5th place, which was good enough to secure them their second entry to the UEFA Europa League, where they went out in the third qualifying round to Romanian side Gaz Metan Mediaș.
Jøren Andersen took over coaching Mainz 05 in the 2008/2009 season. Under the direction of Andersen, Mainz 05 was only twice not in the top 3 places.
The club currently plays at Coface Arena, a new stadium opened in 2011, which holds 34,034 spectators. The first event held at the new arena was the LIGA total! Cup 2011, which took place from 19 July through to 20 July 2011, with the other participants being FC Bayern München, Borussia Dortmund, and Hamburger Sportverein (HSV).
Die Nullfünfer previously played in Stadion am Bruchweg, built in 1928, and modified several times over the years to hold a crowd of over 20,300 spectators. Averaging crowds of about 15,000 while in the 2.Bundesliga, the team's hard won recent success had them regularly filling their venue.
Mainz is known for being one of the three foremost carnival cities in Germany, the others being Düsseldorf and Cologne. After every Mainzer goal scored at a home game, the Narrhallamarsch, a famous German carnival tune, is played.