James primarily played tuba early in his career, including with Gene Rodgers, June Clark, and Chick Webb in the late 1920s. He worked with Webb until 1933, including on recordings with Louis Armstrong. Around 1933 he switched to bass and continued working with Webb, as well as with Benny Carter, Bob Howard, Buster Bailey, Fletcher Henderson, and Red Allen. From 1934 to 1936 he played with the Mills Blue Rhythm Band, and later in the decade with Edgar Hayes, Mezz Mezzrow, Jabbo Smith, and Tommy Ladnier. He worked with Claude Hopkins and Zutty Singleton in the early 1940s, but stopped performing and recording after about 1943.