Holmes was born in Woodstock, Connecticut. He was the son of David Holmes and Temperance Bishop. He graduated from Yale College in 1783. In 1784, while ministering in South Carolina, he was recruited to be the minister at the Congregational Church in Midway, Georgia. He returned to New England to be ordained in 1785 and once for health reasons between 1786 and 1787, but returned to Midway and remained there until 1791. Holmes married Mary Stiles, the daughter of Ezra Stiles, the president of Yale. Mr. Stiles was the subject of a laudatory biography penned by Holmes.
In 1792, Rev. Holmes became the minister at First Church in Cambridge, Massachusetts. He was elected a Fellow of the American Academy of Arts and Sciences in 1803. In 1805, he published a history entitled American Annals. Holmes was elected a member of the American Antiquarian Society in 1813, and also served as its corresponding secretary from 1816-1828. Amid a theological controversy between Calvinism and Arminianism, Holmes resigned from the ministry in 1831 and the church chose a Unitarian minister to replace him. Holmes died June 4, 1837.
Holmes married Mary Stiles, daughter of Ezra Stiles, in 1790 and second to Sarah Wendell. By the second marriage, Abiel was the father of Oliver Wendell Holmes, Sr. and grandfather of Oliver Wendell Holmes, Jr.
- The Life of Ezra Stiles (1798)
- The History of Cambridge (1801)
- A Memoir of the Mohegan Indians (1804)
- American Annals (1805)
- Gray, George Arthur (1908), Descendants of George Holmes of Roxbury 1594-1908, Boston: David Clapp and Son.
- White, G. Edward (1993), Justice Oliver Wendell Holmes: Law and the Inner Self.