Harrison began on trombone at age 15, playing locally in the Toledo, Ohio area. He played semi-pro baseball, but chose music over a career in sports when he joined a traveling minstrel show in the late 1910s.
He led his own jazz ensemble in Atlantic City by 1919, and played in the bands of Charlie Johnson and Sam Wooding. He then moved to Detroit and played with Hank Duncan and Roland Smith. After returning to Toledo, he played gigs with June Clark and James P. Johnson, and followed this with a stint in New York City with Fess Williams.
In 1924, June Clark took over leadership of Harrison's ensemble, though he continued to perform in it. In 1925 he began working with Billy Fowler, where he remained for several years. He also played with Duke Ellington in the mid-1920s. Later in the decade Harrison played with Elmer Snowden and Fletcher Henderson. While on tour with Henderson in 1930, he took ill with a stomach ailment, and though he continued to play for several months with Chick Webb, he died of this condition in 1931.