Rich was the son of Rev. Elisha Rich and Mary Davis. His father being a Baptist and his mother an adherent of the "standing order,"[disambiguation needed] he probably became accustomed to hearing discussions of religious questions in his boyhood. He had a speculative and inquiring mind, and as appears readily accepted the views of the opposers of orthodoxy of the day in general, although on special points of doctrine he held views peculiar to himself. He was nominally a Baptist, and retained his connection with that denomination until after his removal to Warwick at about twenty-one years of age. There his peculiar notions became so offensive that he and also his brother Nathaniel were brought before the church "for errors in doctrine," and on the charge, were set aside. Soon after, in 1773, these brothers with one or two associates formed the new society at Warwick, and Caleb became the minister.
In 1775, at the "Lexington alarm," he marched as a minuteman, under Capt. Wright, for Cambridge, and after being there a few weeks obtained a furlough and went to visit among his relatives at Sutton and Oxford. His friends would hear him explain his new doctrine; and therefore from house to house in Sutton, Oxford, Charlton, Thompson and Douglas, he opened and expounded his theories, until on his return to the camp, he says, "the brethren numbered forty or fifty." On his return to Warwick he renewed his labors with increased zeal. There his ordination as a minister occurred in 1781. After having remained in Warwick about thirty years, he removed to Shoreham, Vt, and later to New Haven, Vt, in which place and vicinity he continued to preach, as his health permitted. He was familiar with the scriptures, delighted in arguments of doctrine subjects, and was more than an ordinary man as a thinker, but as a preacher he did not excel. Rev. W. S. Balch, from whose sketch of him these notes are in part compiled, says: "He was the real founder of American Universalism." (now known as the Universalist Church)
- "Samuel Davis, of Oxford MA and Joseph Davis of Dudley, MA, and their Descendants", authored by George C. Davis, 1884, North Andover, MA