The castle of Lihula (German: Leal) was first mentioned in 1211, but it appears the site was used as a fortress since the Iron Age. In 1220, a Swedish army started constructing a castle here, but were defeated by Estonian forces on 8 August 1220, in the Battle of Lihula.
Despite the fact that the castle was destroyed during the Livonian War, a new manorial estate grew up right next to the original location thereafter. In the 1630s, it was recorded as the property of the Swedish general Åke Tott. The manor stayed in the Tott family until 1684, after which it belonged to various aristocratic families. The present neoclassicist manor house was built in the early 19th century.