He was born in Kedleston in Derbyshire, England, the son of Thomas Middleton, Rector of Kedleston and educated at Christs Hospital. He then went up to Pembroke College, Cambridge, and on graduation was ordained in the Church of England. He was appointed curate of Gainsborough (1792), Rector of Tansor (1795), Rector of Bytham (1802), Prebendary of Lincoln (1809), Archdeacon of Huntingdon and Vicar of St Pancras.
In 1814, he became the first Bishop of Calcutta. This diocese included not just India, but the entire territory of the East India Company. While a Bishop, he founded Bishop's College in Calcutta. In May, 1814 he was elected a Fellow of the Royal Society on the basis of being "a Gentleman well known to the literary world as the author of several classical works, and conversant with various departments of science" 
He died in Calcutta of sunstroke on July 8, 1822 and is buried in Calcutta Cathedral.