Lytell (born James Sarrapede) had his first professional experience at age twelve, and by the beginning of the 1920s he was recording with early jazz ensembles. He played in the Original Indiana Five in 1921 and the Original Memphis Five in 1922-25, and also played in the Original Dixieland Jazz Band in 1922-24. After the 1920s he rarely performed in jazz settings, spending more time as a studio and orchestra musician. He worked as a staff musician for NBC during this time, and in Johnny Green's orchestra in 1934-'35. From 1949 into the late 1950s he appeared in the New Original Memphis Five revival band, and recorded with Connee Boswell late in the 1950s. He continued to perform until a year before his death. As a leader, he recorded eighteen titles in 1926-28 and six more for London Records in 1950.