Berberian was born in New York City as the son of Armenian immigrants. His father, Yervant Berberian, was an accomplished oud player and an instrument maker. Oud masters of Armenian, Turkish, and Greek heritage frequented his family's home. John Berberian started his musical education learning the violin, but by the age of 10 began imitating his father on the oud. At the age of 16, he had his first job as a musician, playing the oud in a band in Whitinsville, Massachusetts.
He graduated from Columbia University with a B.A. in economics, and received his MBA from Harvard Business School. While a student, he performed regularly in Manhattan nightclubs. From the early 1960s, he recorded for a variety of labels including MGM, RCA, Roulette, Verve and Mainstream Records.
He expanded beyond the ethnic music market in the mid to late 1960s, decades before "world music" became fashionable, with a series of LPs which explored fusions between traditional Middle Eastern music, psychedelic rock and jazz. These included Expressions East (1964), Music of the Middle East (1966), and Middle Eastern Rock (1969) with guitarist Joe Beck. He also set up his own recording studio and record label, Olympia. Simultaneously with his career as a musician, he worked as a purchasing agent in New York.
Berberian has commanded the respect of musicians worldwide. He has been featured in numerous concerts and dances throughout the USA, Canada and South America, and is one of only a handful of musicians worldwide given the title of Udi (oud master). He now lives in Massachusetts.