Early in his career, Harris performed with Benny Goodman, Charlie Barnet, and Eddie Condon. He is renowned for his broad, thick tone and quick vibrato that remained for the duration of each tone. He went on to join Woody Herman's First Herd in 1944. He was also in the Four Brothers Second Herd during the late 1940s, and he worked with Herman again in the 1950s. He then teamed up with Charlie Ventura and later with Chubby Jackson. Together with Flip Phillips, he became a stalwart of Benny Goodman's group in 1959, although it has been said that Goodman was frequently irritated at Harris because of Harris' indifferent approach to "sight-reading," the skill of playing off of written music with fluency, an ability notable in Goodman and trumpeter Harry James. As an improviser, Harris seemed comfortable playing among divergent stylists, as shown on "Jazz at the Philharmonic" recordings, as his "one-off" style seemed to work in any context, from "Dixieland" to swing to "bebop" .
His solo on "Bijou" with Herman remains a classic, while his idiosyncratic treatment of the ballad "Everything Happens to Me" is noted for its vocality, and his treatment of the ballad "Everywhere" was inspiration for Roswell Rudd's free-contrapuntal "Everywhere." Later, Harris worked in Las Vegas, finally retiring to Florida.