Coming to the United States when 11 years old, his early education was obtained in the public schools of Brooklyn, New York. For several years after graduation from the high school there he was engaged in mercantile life, but in 1887 resumed his studies at Colgate University in Hamilton, New York. In 1890, he entered the full Hebrew and Greek course in theology and was graduated from the Hamilton Theological Seminary in 1893.
He was for two years pastor at Binghamton, New York, afterward accepting a call to the pastorate of the First Baptist Church of Ridgewood, New Jersey, where he remained for three years. He resigned from the ministry in 1899 to devote himself to literary work, and has been a frequent contributor to religious and other periodicals. In 1903, he was appointed managing editor of the Encyclopedia Americana, to which he contributed many articles. Subsequently Rines was managing editor of the United Editors Encyclopedia, and in 1910-11 was general editor of The Foundation Library for Young People. From 1913 to 1915, Rines was managing editor of The German Classics, and after 1916 was editor-in-chief of the second edition of Encyclopedia Americana.