It was founded in 1128 in honour of the Blessed Virgin Mary, by Giselbert of Kasterlee, who not only gave the land, but also himself became a lay brother in the new community. The first monks were sent from St. Michael's Abbey, Antwerp, under Henry, who had come with Saint Norbert, founder of the Premonstratensian Order, to Antwerp to extirpate the Tanchelmite heresies. The charter of its foundation was signed, amongst others, by Bernard of Clairvaux and by Waltman, first abbot of Antwerp. The Bishop of Cambrai granted synodal rights to the abbots.
From small beginnings the abbey became influential in the district called Campine, now in north-east Belgium and the south of the Netherlands, then a wild area. The bishops of Cambrai, the chapters of Liège and Maastricht, and several landowners entrusted the charge of parishes, with the right of patronage, to the abbey. In time the abbey had to provide priests for some forty parishes in these parts.
With the erection of new dioceses (1559–60) in Belgium and the Netherlands, heavy burdens were cast on the abbey, for not only had it to provide funds for the new diocese of 's-Hertogenbosch, but the new bishop was put at its head as abbot. This state of affairs lasted until 1590, when, to obtain its independence, the abbey had to give up much property in support of the new diocese. The abbey was a centre of education. It possessed one of the largest libraries, and was able to take up the work of the Bollandists.
The rise of Calvinism in the Netherlands caused conflict. Three monks of Tongerlo became Catholic martyrs: Arnold Vessem and Henry Bosch in 1557, and Peter Janssens in 1572. In the seventeenth century Francis Wichmans of the abbey rallied local Catholics.
The abbey's property was confiscated and sold by the French occupying forces in 1796, but in 1840 was bought back under its post-revolutionary re-founder, Peter Hubert Evermode Backx.
The abbey is also the site of a Leonardo da Vinci Museum, which contains a 16th-century copy on canvas of Leonardo's Last Supper, in approximately original size. The copy reveals many details that are no longer visible in the original fresco due to deterioration.