Aberdyfi Castle was founded by the Lord Rhys, Rhys ap Gruffydd in 1156, shortly after he had become ruler of Deheubarth. Rhys had heard rumours that Owain Gwynedd was planning to invade Ceredigion in order to reclaim it for Gwynedd. Rhys responded by building a castle at Aberdyfi. However, the threatened invasion did not take place, and Turvey claims that Owain's intention may have been to test the resolve of the new ruler. Rhys "raised a ditch to give battle", according to Brut y Tywysogion, known as the 'Chronicles of the Princes' in English.
Although the threat had not materialised, an earthwork topped by a timber castle was built on the site. A motte was constructed at the end of a low ridge running across marshy ground, surrounded on two sides by the confluence of the Dyfi and Einion rivers. The mound is still around 20 feet high with a diameter at the top of about 30 feet and it was originally surrounded by a deep ditch.
Roger de Clare, a Norman lord, attacked and captured the castle in about 1158. However, the Lord Rhys recaptured the castle the same year. Llywelyn the Great held an assembly at Aberdyfi Castle in 1216 where he supervised the parcelling-out of land in south Wales to the minor rulers of Deheubarth in return for their homage. Hitherto Llywelyn had ruled over Gwynedd only; however, the Aberdyfi assembly is seen as a crucial point in his reign, marking the moment at which the minor rulers of Wales submitted to Llywelyn's overlordship. In 1230 he began using the style 'prince of Aberffraw, lord of Snowdon' - in essence, a confirmation of his de facto position as prince of Wales.