Sometime before 1321, Grand Duke Gediminas transferred the capital of Lithuania from Kernavė to Trakai (today's Senieji Trakai) and erected his brick castle. In 1337 it became a seat of the newly established Duchy of Trakai. Gediminas' son Kęstutis erected a new castle in New Trakai (today's Trakai). Kęstutis' son Vytautas was born in Old Trakai ca. 1350.
The castle in Senieji Trakai was destroyed by the Teutonic Order in 1391. The ruins of the castle were gifted to Benedictine monks by Vytautas in 1405. It is supposed that the present monastery building dating from the 15th century incorporates the remains of the Gediminas' castle.