The club was founded 1 September 1903 as 1. FC Arminia Rheingönheim and enjoyed some success in local level competition prior to the outbreak of World War I. Under the Nazis it was forced in to a merger with TSG Rheingönheim as part of a process of political consolidation that eliminated worker's clubs like 1. FC, as well as faith-based clubs, as being ideologically unacceptable to the regime. The club was reestablished as VfL Rheingönheim after World War II before once again becoming FC Arminia Rheingönheim on 22 October 1949. In 1969, the association adopted the name of the city of Ludwigshafen.
FC first advanced to the Amateurliga Südwest (III) in 1966 where they would compete for nine seasons with their best result coming as a fourth-place finish in 1969–70. In 1988, after several years in the Bezirksliga Vorderpfalz, Arminia slipped to the Kreisliga and then bounced back and forth between these two levels of play for several seasons. A string of three consecutive promotions saw the club return to the Verbandsliga Südwest (VI) in 2005, and in 2011, they won their way into the Oberliga Südwest (V) for the first time, a league renamed to Oberliga Rheinland-Pfalz/Saar in 2012.