The fortified castle was erected around 1240 to defend the Duchy of Brabant against the County of Flanders. The castle was destroyed however by Brussels city troops in revenge for the assassination of Everard t'Serclaes, which was commanded by the Lord of Gaasbeek.
At the beginning of the 16th century the Horne family constructed a brick castle on the ruins of the medieval fortress. In 1565 Lamoral, Count of Egmont, acquired the castle and its domain, including feudal rights in 17 surrounding villages. Accused of high treason by Philip II of Spain, the Count of Egmont was beheaded three years later.
In the following centuries the castle was inhabited by several noble families. It obtained its pseudo-medieval appearance as the result of a renovation during the years 1887-1898. The works were executed by the architect Charles Albert and ordered by the Marquis d'Arconati Visconti who owned the castle at that time. His widow Marie Peyrat (d. 1922) donated the castle to the Belgian state, including the art collection and the grounds.
Since 1980 the castle has been owned by the Flemish Community (Vlaamse Gemeenschap). The castle contains impressive art collections displayed in lavishly decorated historical rooms. A remarkable collection piece is the authentic testament of the famous painter Peter Paul Rubens The castle and its grounds (a park of 50 hectares or 124 acres) are open to the public.