The community was originally known as Hobart, possibly named after Garrett Hobart, a New Jersey legislator who would eventually become vice-president under William McKinley. A post office was established in 1895, but it was closed in 1903. The name was changed around 1914 to honor the capital of Germany, the former homeland of many of the town's residents. The new name was named by John (Kritner) Kreitner.
The historic Lidy Walker Covered Bridge was located just north of Berlin at the farm of Winford I. "Lidy" Walker on Lidy's Lake from 1958 until its collapse in 2001. Originally called the Big Branch Covered Bridge, it was built near Blountsville in Blount County in 1926 and moved to Berlin 32 years later.