The castle was founded by King David I. It was destroyed in 1276, rebuilt in 1530 by Dame Ker, and destroyed again by the 3rd Duke of Roxburghe in 1760. Very little of it now remains. This Norman castle was named after the Lords Holydean, who were originally the deans of Kelso Abbey: monks who held great power in what was one of the largest feudal territories and most profitable regions of Scotland. The peerage title, barony, and castle eventually went to the Kerrs who were made Earls of Roxburghe, and later Dukes.
Holydean Farm stands on the site of the old Holydean Castle. A stone block rescued from the castle now forms the lintel of the farmhouse doorway, and the castle well is still preserved. The castle's alternative name is Hobbie Ker's Well.
- A historical perspective, drawn from the Ordnance Gazetteer of Scotland: A Survey of Scottish Topography, Statistical, Biographical and Historical, edited by Francis H. Groome, originally published in parts by Thomas C. Jack, Grange Publishing Works, Edinburgh between 1882 and 1885.
- Geograph image: Farmland at Holydean, viewed from a track between Holydean Farm and Over Whitlaw Farm
- Alexander Jeffrey, The history and antiquities of Roxburghshire and adjacent districts (Volume Four, pages 28 of 30