The crusaders of Sword Brethren conquered the hill fort at the place of later main castle in 1223. A year later, construction of stone fortifications started. Viljandi was chosen as the high seat of the order.
The convent house, a typical form of castle of Teutonic Knights, was erected late 13th – early 14th century. In the following centuries the castle was extended and fortified further. It was badly damaged in the Polish-Swedish wars in early 17th century and not repaired any more. In 18th century, the ruins were used for quarrying stones for construction work in Viljandi.
The first excavations in the castle were performed in 1878–1879. In recent decades, these have turned to almost yearly events.
Currently the ruins form a popular resort area just outside of central Viljandi. An open-air stage is located in the former central courtyard.