Reynolds was born 1854 in Lyons, Illinois. He was converted at age twenty-two and began preaching the following year in the Methodist church in New England. Reynolds was ordained in 1862 by Methodist Episcopal Bishop John Fletcher Hurst. In 1895, Dr. Reynolds joined the Association of Pentecostal Churches of America. He was elected Home and Foreign Missionary Secretary two years later. When the Association merged with the Church of the Nazarene to form the Pentecostal Church of the Nazarene in 1907, Reynolds was elected general superintendent along with Phineas F. Bresee. Re-elected the following year during the Second General Assembly at Pilot Point, he continued in that office until his retirement in 1932. During most of his superintendency, he also served as Secretary for Foreign Missions and in 1896 began the ambitious program of Christian witness to the Cape Verde Islands. Reynolds was also a founder, influential in the naming, of The Eastern Nazarene College.