Shertzer was born in New York City, began playing violin when he was nine years old and picked up saxophone when he was a teenager. He worked as a sideman for Gene Kardos at the club Birdland, then joined Benny Goodman's band, where he was the lead saxophonist until 1938, though he recorded with Goodman intermittently until the mid-1940s. He was in Tommy Dorsey's band from 1938-1940, and also recorded in the late 1930s with Bunny Berigan and Lionel Hampton. He worked with Billie Holiday in 1941 and again in 1944, then became a house musician for NBC radio and television. He worked as a session player for studio recordings of Ella Fitzgerald, Frank Sinatra, Sarah Vaughan, Sy Oliver, Louis Armstrong, and Artie Shaw in the period 1947-1953, and continued working with Goodman live, on television, and on record from 1951 until 1969. He was a member of the Tonight Show Band during its Johnny Carson era and was active in recording sessions until the mid-1970s. He died in New York City aged 67.