It was built by the Hacketts, a Norman family. The Kirwans, one of the tribes of Galway, settled there in the 15th century. The Castle Hackett branch of the family was established in the mid-17th century by Sir John Kirwan. The castle was abandoned in the 18th century and the Kirwans built a new three-story house called Castlehacket which was burned in 1923 during the Civil War but rebuilt and still stands today.
In the introduction to his Fairy and Folk Tales of the Irish Peasantry (1888), William Butler Yeats mentions the family and castle Hackett, writing, "Each county has usually some family, or personage, supposed to have been favoured or plagued [with fairy-seeing abilities], especially by the phantoms, as the Hackets of Castle Hacket, Galway, who had for their ancestor a fairy…"