Gilmore was born in New Orleans, Louisiana. He was instructed by private tutors, graduating from the Central High School of New Orleans in 1874, from Seton Hall College, South Orange, New Jersey, in 1877, and from the law department of the University of Louisiana (now Tulane University) at New Orleans in 1879. He was admitted to the bar in 1880 and commenced practice in New Orleans, Louisiana. From 1888 to 1896, Gilmore served as assistant city attorney. He was city attorney from 1896 until March 15, 1909, when he resigned. He served as delegate to the Democratic National Convention in 1908.