Castle was born on 14 September 1698 near Canterbury in Kent, where he received the most of his early education. He was admitted to Corpus Christi College, Cambridge, in 1716, graduating B.A. in 1719, and being made a Fellow in 1722.
Castle was appointed public orator for the University of Cambridge in 1727;, he gave up the office in 1729, on being appointed to the vicarages of Elm and Emneth. He was then moved to Barley, Hertfordshire. In 1744 he was made rector of St Paul's School, London, in 1745 master of Corpus Christi College, and in 1746 vice-chancellor. In 1747 he was promoted to a prebend at Lincoln Cathedral, and in 1749 to the deanery of Hereford.
Castle died at Bath on 6 June 1750. He was buried at Barley, Hertfordshire, where there was a Latin inscription to his memory.