Balaban resided as an adult in the Florida panhandle, where he worked as a farmer and played in regional ensembles from the 1950s. He held a regular gig from 1966 at the Dixieland jazz club Your Father's Mustache in New York City. He is the son of Barney Balaban, former president of Paramount Pictures.
He worked extensively as a sideman, for musicians such as Wild Bill Davison, Eddie Condon, Gene Krupa, Dick Wellstood, and Kenny Davern. He co-led Eddie Condon's house band with Ed Polcer from 1975; other noted musicians in this outfit included Vic Dickenson, Warren Vache, and Connie Kay. The club closed in the mid-1980s.
Balaban died at the age of 84 on December 29, 2013, after a brief illness. He was living with his wife of 62 years, Maxine, or Micki as she is called, at their lakefront home in West Haven, Connecticut.