Varbergs Gymnastik- och Idrottsförening was formed on 28 May 1905 and shortly afterwards included football in their programme. In the same year the club became founder members of the Hallands Idrottsförbund (Halland Sports Federation) at the invitation of IF Kamraterna, Halmstad.
Varbergs GIF FF were declared bankrupt in December 2007 when their tax liabilities could not be met. As a consequence the club dropped out of Division 4 Halland and reformed in the bottom tier in Division 6 Halland Norra.
Varbergs GIF FK currently plays in Division 4 Halland Elit which is the sixth tier of Swedish football. They play their home matches at the Påskbergsvallen in Varberg.