The son of businessman-politician Henry B. Payne and Mary Perry, he graduated from Phillips AcademyAndover in 1859, then studied at Yale University. At the outbreak of the U.S. Civil War in 1861 he enlisted in the Union Army, though he could have afforded to pay someone to go in his place. In 1863, he became colonel of the 124th Ohio Infantry. He was BrevettedBrigadier General March 13, 1865. Upon the end of the war, he began his career, investing in iron and then oil refining. His oil interests were the first acquired by Standard Oil, and he became a trustee of that firm and acted as a lobbyist. He was charged with bribing members of the Ohio Legislature to attain a Senate seat for his father (before the U.S. Senate was directly elected), and with bribing the Democratic Party to name his brother-in-law United States Secretary of the Navy, though the charges were dropped.