Allen began professionally playing music in 1927 after he joined Jimmie Lunceford's band, where he played the tuba. He switched to bass in 1932, remaining with Lunceford's orchestra until 1942. Allen's bass playing was a key element in the highly influential Lunceford ensemble. Allen was also an early experimenter with the electric bass. Among his best-known recordings with Lunceford is the tune "In Dat Mornin'".
After leaving Lunceford's orchestra, Allen opened a music store in New York City, playing occasional gigs until the 1960s. Allen died in New York City in 1983.