Born in Chicago on May 1, 1907, Alvis began on drums but switched to tuba and bass after playing with Jelly Roll Morton in 1927-28. He played tuba with Earl Hines from 1928 to 1930, and did arrangements for Hines as well.
He moved to New York City in 1931 and played with Jimmie Noone in the Mills Blue Rhythm Band from 1931 to 1934 and 1936. A very early double-bass solo can be heard on the latter group's "Rhythm Spasm" (1932). Alvis also occasionally played baritone saxophone in this ensemble as well, and was the group's tour manager. From 1935 to 1938 he played with Duke Ellington, working with fellow bassist/tubist Billy Taylor.