During the Civil War, Sprague served in the 44th New York Infantry, seeing action at the Battle of Gettysburg, where his unit was instrumental in helping repulse attacks on Little Round Top. The New York State Archives stores a lengthy article Sprague wrote on his military service.
He was president of both the New York Institute of Accounts and the Union Dime Savings Bank (which later became the Dime Savings Bank). Later in life, he was involved in the movement for reform of English spelling as part of the Simplified Spelling Board, of which he was the first treasurer.
He was heavily involved in the development of the first state certification of accountants in the United States.