He was born in Tompkinsville, Kentucky, moving to Indianapolis with his family as a child. He first played in a band with Erroll "Groundhog" Grandy who mentored J. J. Johnson and Wes Montgomery. From 1938 to 1940, Coe was with Buddy Bryant's band and by the age of 20, was already touring with the Jay McShann band, which included Charlie Parker, Al Hibbler, Walter Brown, Bernard Anderson, Gene Ramey and Harold "Doc" West.
In the 1950s, Coe recorded for King as a member of Tiny Bradshaw's band, then made a session with his own combo (though the company insisted on billing him as Jimmy "Cole.") In 1953, States recorded his Gay Cats of Rhythm. In the late 1950s, Coe led the house band for the small Indianapolis-based label Note Records; some of the material was licensed to Checker, which had better distribution.
With his mid-60s big band he backed performers including Aretha Franklin, Roy Hamilton, and Gladys Knight & the Pips. Other musicians he worked with included Montgomery, Slide Hampton, David Baker, Freddie Hubbard, pianist Carl Perkins, Larry Ridley, Leroy Vinnegar, and doo-wop sensations The Students.
He died in Indianapolis in 2004.
- Eagle, Bob; LeBlanc, Eric S. (2013). Blues - A Regional Experience. Santa Barbara: Praeger Publishers. p. 147. ISBN 978-0313344237.
- Campbell, Robert L.; Kochakian, Dan and Büttner, Armin "The Jimmy Coe Discography"
- Chadbourne, Eugene at allmusic