Berkowitz was born in Brooklyn, New York to a Romanian Jewish couple, Matilda and William Berkowitz who had emigrated from Roman and Bucharest. His father was instrumental in shaping young Ralph's musical culture and experience. In 1927, he enrolled at the Curtis Institute in Philadelphia where he later became a member of the teaching staff. In 1940, he became accompanist for Gregor Piatigorsky, with whom he appeared until the cellist's death in 1976. Other musical partners included the tenor Jan Peerce, cellist Felix Salmond, and violinist and composer George Enescu. He recorded extensively with Piatigorsky and others, including the violinist Eudice Shapiro.
From 1946 to 1951, Berkowitz served as an executive assistant to Serge Koussevitzky at Tanglewood and later became Dean of the Berkshire Music Center at Tanglewood in 1951, a position he held until 1961. As Dean, he presided over a faculty that included Aaron Copland, Leonard Bernstein, and many others. Illustrious students in those years included Zubin Mehta, Lorin Maazel and Claudio Abbado. He, along with Boston Symphony manager Todd Perry, was largely instrumental in keeping the Tanglewood Festival alive following Koussevitsky's death.
Berkowitz has been widely published as an arranger and composer. His A Telephone Call, for singer and orchestra is his most extended work.
In 1961 Berkowitz moved to Albuquerque, New Mexico, where he lived until his death in 2011. He first came to Albuquerque in 1940 to serve as a guest artist with a chamber music series called The June Music Festival. He remained active as an artist for the Festival into the 1980s. Berkowitz became manager of the then Albuquerque Civic Orchestra (now New Mexico Symphony Orchestra) when he moved to New Mexico and served until 1968, seeing the Orchestra through its move to its current home—Popejoy Hall at the University of New Mexico.
Berkowitz commissioned Daron Hagen to compose one of his most intellectually rigorous works, a set of Piano Variations based on a theme made up of pitches derived from Berkowitz's and Hagen's names, in 2002. The work is available from Carl Fischer. Berkowitz reached his centenary in September 2010 and died in August of the following year.
- "RCA Red Seal Century - Soloists & Conductors", 2 CD / RCA Records / 2001-10-23
- "Stravinsky: Petroushka Suite; Toch: Violin Sonata", 1 CD / Crystal Records / 1998-01-02 
- Ralph Berkowitz's obituary
- "Obituary of Ralph Berkowitz". The Albuquerque Journal. 2011-09-02. Retrieved 2011-09-05.
- Steinberg, David (2010-09-27). "Join party with Chamber Soloists". Albuquerque Journal. Retrieved 2010-10-02.