The club was founded in 2004 as Virumaa Jalgpalliklubi Rakvere (English: Virumaa Football Club Rakvere). From 2008, Rakvere became affiliated with Flora and became Rakvere FC Flora. In 2011, the team name was changed to Tarvas (English: Aurochs), which is the symbol of Rakvere and is derived from the historic name of the town, Tarvanpea (Aurochs' head). In 2012, Tarvas was promoted to the Esiliiga and finished the 2012 season in third place. In promotion games against JK Tallinna Kalev Tarvas lost 1-3 on aggregate. Tarvas finished the 2013 Esiliiga season in 4th place but were beaten in the promotion play-offs by Tammeka 2–6 on aggregate. In July 2014, Valeri Bondarenko was hired as manager. Tarvas finished the 2015 Esiliiga season in 4th place and was promoted to the Meistriliiga for the first time in history. Tarvas earned just 1 point in first 14 Meistriliiga games. On 4 June 2016, Valeri Bondarenko was sacked and replaced as manager by Urmas Kirs, who coached the team in 2008. With new coach Tarvas earned another two draws and finished season at the bottom of the table.