Ethnic Estonians are a minority in Jõhvi, as about 55% of the town's population are either ethnic Russians who arrived in Estonia during the period of Soviet rule of Estonia (1944 - 1991) or their descendants.
Jõhvi was first mentioned as a village in 1241 in Liber Census Daniae when it was ruled by Denmark. Historical names of Jõhvi were Gewi and Jewi. In the 13th century a church was built here and Jõhvi became the centre of the local church parish.
On 1 May 1938 Konstantin Päts renamed the Jõhvi borough a town along with nearly all Estonian boroughs. Up to 1991, Jõhvi was a district of Kohtla-Järve. In 2005 the town of Jõhvi was united with the parish of Jõhvi.