Shaw learned to play tuba in high school, but switched to bass soon after. In 1942 he worked with Fate Marable on riverboats traveling on the Mississippi River, then served in the Navy from 1942 to 1945. After his discharge he played with Armstrong in his last big band, from 1945 to 1947. Shaw and Sid Catlett then joined the Louis Armstrong All-Stars until 1950, when Shaw broke off to study music. He returned to play with Armstrong from 1952 to 1956, and performed in the 1956 musical High Society.
Following this he worked at CBS with Russ Case, did time in Teddy Wilson's trio, and played with Benny Goodman at the 1958 Brussels World's Fair. After a few years in Europe, he played again with Goodman on a tour of Central America in 1962. From 1962–64 Shaw played again with Armstrong, and occasionally accompanied him through the end of the 1960s. After the 1960s Shaw mostly freelanced in New York and kept playing until his death. He recorded only once as a leader, a live concert from 1991 of his Satchmo Legacy Band.