Immediately after the end of the Second World War, the sports activities in Zemun were restored. By early 1945, a club FK Maksim Divnić was formed, named in honour of a player with that name that played before the war and who died as Partisan fighter during the war. After just a couple of played matches the club changed its name into FK Sremac Zemun. By the end of that year another club is also formed in Zemun, FK Sparta Zemun. On October 20, 1946, the two clubs were merged to form a new sports society whose football section was FK Jedinstvo Zemun.
By 1962 the club got promotion into the group East of the Yugoslav Second League, that was divided back then into two geographical groups, East and West. It stayed at Second national level for two seasons, after which it returned into the Serbian republic league, Yugoslav third tier. In the late 1960s, the club struggled financially, and on February 23, 1969, it is merged with FK Galenika, a club sponsored by the Galenika a.d., a pharmaceutical company located in Zemun. Under the new name FK Galenika Zemun, and with new financial backing, the club soon returned to the higher leagues, winning promotion as soon as 1970 to the Yugoslav Second League.
Coached by Ivan Čabrinović, the club became one of the strongest teams in the second league, and after several times being close, it finally reached promotion to the Yugoslav First League in 1982. In that same season, the club reached the semi-finals of the 1981–82 Yugoslav Cup where they lost against the cup winners Red Star Belgrade. The club played in the 1982–83 Yugoslav First League however it finished bottom of the table subsequently relegated back to the Second League. On January 1, 1985, the club changed its name into the one that still carries nowadays, FK Zemun.
FK Zemun competed in the top league all the way until 2007. In the season 2004-05 they managed to achieve 5th place once more.
In 2007, Zemun had one of the most memorable seasons in recent history.[according to whom?] They competed in the Serbian First League and came away with the 15th position and an automatic relegation to the Srpska Liga, third tier. However FK Zemun enjoyed an unbelievable[according to whom?] run in the 2007–08 Serbian Cup and found its way to the final. The club won three matches after penalties, defeated two first league sides and goalie Nemanja Jovšić was voted most valuable player. They lost the final 3-0 to Serbian team Partizan but since Partizan had already qualified for the Champions League Zemun were given the UEFA Cup spot as they were the runners-up. During the summer however, Zemun was not able to obtain a license from UEFA and had to give their spot to fourth-placed side Borac Čačak. Borac then gave their Intertoto Cup spot to OFK Beograd.