The club was founded in 1907 as FC Germania Elversberg. It was dissolved in 1914, but then re-constituted in 1918 as Sportvereinigung VfB Elversberg, spending a season in the tier-one Kreisliga Saar in 1921–22. After World War II a number of local associations came together to form Sportgemeinde Elversberg. The current club became independent of SG as SV Elversberg VfB 07 in 1952. From 1951 to 1960 SV played in the Amateurliga Saarland (III), but then disappeared into tier IV and V football until 1980 when they re-emerged in the Amateur Oberliga Südwest (III) for a span of seven seasons. After nearly another decade spent between the Verbandsliga Saarland (IV) and the Landesliga Saarland/Nordost (V) the club returned to tier III football in the Regionalliga West/Südwest.
Since 1998, SV Elversberg has played in the Regionalliga Süd (III) as a lower table side. For the 2008–09 season, the club belonged to the new Regionalliga West where it stayed until 2012 when it became part of the new Regionalliga Südwest. In 2012–13 the club finished runners-up in the new league and qualified for the promotion play-off to the 3. Liga where it overcame the reserve team of TSV 1860 München to earn promotion to this league.
SV Elversberg lasted for only one season in the 3. Liga, suffering relegation alongside fellow Saar side 1. FC Saarbrücken back to the Regionalliga. It finished runners-up in the league once more in 2015–16 but lost to FSV Zwickau in the promotion play-off and had to remain in the league.