Born in Rochester, New York, he studied violin with Carl Markees, Henry Holst, and Willy Hess at the Hochschule für Musik in Berlin (now the Hochschule für Musik Hanns Eisler). Sumberg taught for many years at the Royal Conservatory of Music (Toronto) and founded the Conservatory String Quartet in 1929.
He performed with the Toronto Symphony Orchestra as principal second violin (1931-1957) and then associate concertmaster (1961-1975). He performed as a soloist and collaborated with many important musicians, such as Glenn Gould. In 1967 he recorded with Duke Ellington on Ellington’s "North of the Border" album. He adjudicated at festivals including "Canadian Open Old Time Fiddlers' Contest" 1955-84.
Sumberg was also known for his collection of fine violin bows. He often played on a Collin-Mezin violin, though it was not the finest instrument in his collection, which may have been a superb violin bearing the label Lorenzo Guadagnini, on which he played for many years.
He died in Toronto, Ontario in 1994. He was buried in the Mount Sinai Memorial Park.