In 1184 Baldwin V, Count of Hainaut, built the first castle on this site. The lords of Écaussinnes and their heirs the de Lalaings altered the original structure several times over the centuries.
In 1450 Marie de Lalaing married Jean II de Croÿ, descendant of the Counts (later Princes) of Chimay. By a quirk of inheritance, the castle later became the property of the Lalaing family again. They sold it in 1642 to the van der Burcht family.
In the 18th century the interior was redecorated with Rococo elements in the first floor of the Gothic hall.