Holmes was born in Woodstock, Connecticut. He was the son of David Holmes and Temperance Bishop. He graduated from Yale College in 1783. In 1784, while ministering in South Carolina, he was recruited to be the minister at the Congregational Church in Midway, Georgia. He returned to New England to be ordained in 1785 and once for health reasons between 1786 and 1787, but returned to Midway and remained there until 1791. Holmes married Mary Stiles, the daughter of Ezra Stiles, the president of Yale. Mr. Stiles was the subject of a laudatory biography penned by Holmes.
In 1792, Rev. Holmes became the minister at First Church in Cambridge, Massachusetts. He was elected a Fellow of the American Academy of Arts and Sciences in 1803. In 1805, he published a history entitled American Annals. Amid a theological controversy between Calvinism and Arminianism, Holmes resigned from the ministry in 1831 and the church chose a Unitarian minister to replace him. Holmes died June 4, 1837.
Holmes married Mary Stiles, daughter of Ezra Stiles, in 1790 and second to Sarah Wendell. By the second marriage, Abiel was the father of Oliver Wendell Holmes, Sr. and grandfather of Oliver Wendell Holmes, Jr.
- "Book of Members, 1780–2010: Chapter H". American Academy of Arts and Sciences. Retrieved August 7, 2014.
- Gray, George Arthur (1908), Descendants of George Holmes of Roxbury 1594-1908, Boston: David Clapp and Son.
- White, G. Edward (1993), Justice Oliver Wendell Holmes: Law and the Inner Self.
- Open Library. Abiel Holmes
- The Oliver Wendell Holmes Library at the Library of Congress contains his paternal grandfather, the Rev. Abiel Holmes books.