He was the son of Charles S. Kelsey (a member of the Wisconsin State Senate 1861-64, and the Wisconsin State Assembly in 1867, 1873 and 1880) and Lucretia Parson (Bacon) Kelsey (d. 1868). He became a printer, then studied law, was admitted to the bar in 1875, and practiced law in Geneseo, Livingston County, New York.
He was a Republican member of the New York State Assembly (Livingston Co.) in 1894, 1895, 1896, 1897, 1898, 1899, 1900, 1901 and 1902. In November 1902, he ran for County Judge of Livingston County, but was unexpectedly defeated. Subsequently his party friends forced Theodore P. Gilman to resign the office of First Deputy Comptroller, and had Kelsey appointed to the post. When Comptroller Nathan Lewis Miller was appointed to the New York Supreme Court, Kelsey was appointed New York State Comptroller to fill the vacancy, and was elected at the New York state election, 1904, to succeed himself.
On May 2, 1906, Kelsey was appointed by Governor Frank W. Higgins to a three-year term as Superintendent of Insurance, and resigned the comptrollership. Early in 1907, Governor Charles Evans Hughes asked Kelsey to resign, but he refused. The governor then asked the New York State Senate to remove Kelsey on the ground that "while honest he utterly lacks in force and initiative", but after a lengthy hearing in the Judicial Committee, Kelsey was upheld by a vote of 27 to 24 on May 3, 1907. Then Governor Hughes appointed Matthew C. Fleming a Special Commissioner to examine the Insurance Department, and on February 2, 1908, Fleming declared Kelsey "unfit for the office" in his report to the State Senate, but Kelsey was maintained in office by an even larger majority.
Eventually, Kelsey resigned from the Insurance Department to be re-appointed First Deputy Comptroller by Charles H. Gaus on January 1, 1909, and acted as Comptroller after Gaus's death until the appointment, on November 11, of Clark Williams to fill the vacancy. A week later, Kelsey was forced to resign as First Deputy Comptroller.
He died after complications from a fall, and was buried in Geneseo.
-  The Rapid Transit Bill, mentioning Chairman Kelsey with wrong middle initial "C.", in NYT on April 12, 1901
-  Speculation about Gilman's imminent resignation, in NYT on December 29, 1902
-  Denial of Gilman's resignation in NYT on December 30, 1902
-  The speculation about Gilman's resignation continues, in NYT on January 13, 1903
-  Gilman resigned, in NYT on January 16, 1903
-  Appointed Supt. of Insurance, in NYT on May 3, 1906
-  Kelsey at work, denying resignation, in NYT on January 20, 1907 (giving wrong middle initial "T.")
-  His fight to stay in office, in NYT on February 13, 1907 (giving wrong middle initial "T.")
-  Vote in the state senate 27 to 24 for Kelsey, in NYT on May 3, 1907
-  The report on Kelsey's receivership at the Republic Savings and Loan Association, in NYT on January 3, 1908 (giving wrong middle initial "C.")
-  The Fleming Report, in NYT on February 3, 1908
-  His re-appointment, in NYT on December 15, 1908
-  Kelsey with middle initial H., in NYT on December 25, 1908
-  Acting Comptroller between death of Gaus and appointment of successor, in NYT on November 11, 1909
-  His resignation, in NYT on November 20, 1909
-  Obit in NYT on August 21, 1934 (subscription required)
- "Gossip About People of Note" in Old newspaper, of 1906 with short bio of Kelsey
-  His mother's burial record, from West Perry Cemetery, at RootsWeb
-  Members of the Wisconsin Legislature
Nathan Lewis Miller
|New York State Comptroller
William C. Wilson
|Superintendent of Insurance
Charles H. Gaus
|New York State Comptroller