Harris was born and raised in Baltimore, Maryland. He played professionally in college and worked at the Royal Theater in Baltimore after graduating. He joined Lionel Hampton in 1941 and played with him for several years; he was one of three bassists in Hampton's ensemble, one of the others being Charles Mingus. Harris did some recording with Dizzy Gillespie in the 1940s as well. Fatigued from touring, Harris returned to Baltimore in 1949.
Soon after he worked in a band called Three Strikes and a Miss, again at the Royal Theater. While working here he was heard by Nat King Cole, who asked him to join his trio. During his tenure with Cole, Harris performed on some of Cole's best-known tunes, such as "Unforgettable" and "Mona Lisa". After his work with Cole, Harris returned to Baltimore and remained there, playing, teaching, and working as a furniture salesman until he died of cancer.