He first played in a band with Erroll "Groundhog" Grandy who mentored J. J. Johnson and Wes Montgomery. From 1938 to 1940, Coe was with Buddy Bryant's band and by the age of 20, was already touring with the Jay McShann band, which included Charlie Parker, Al Hibbler, Walter Brown, Bernard Anderson, Gene Ramey and Harold "Doc" West.
In the 1950s, Coe recorded for King as a member of Tiny Bradshaw's band, then made a session with his own combo (though the company insisted on billing him as Jimmy "Cole.") In 1953, States recorded his Gay Cats of Rhythm. In the late 1950s, Coe led the house band for the small Indianapolis-based label Note Records; some of the material was licensed to Checker, which had better distribution.