After a recital in Warsaw in 1921, and a debut with the Berlin Philharmonic in 1924 in which he played three concertos, he was engaged as concert-master of the Dresden Philharmonic from 1925 to 1929. In 1929 he was appointed concertmaster of the Berlin Philharmonic, but was forced by the rise of the Third Reich to leave the orchestra in 1934, despite Wilhelm Furtwängler's attempts to safeguard the Jewish members of the orchestra. He was also a member of a trio with Paul Hindemith and Emanuel Feuermann from 1930 to 1934.  After 1934, he toured Europe with the pianist Lili Kraus. He made his American debut in New York in 1938 at Carnegie Hall. While Goldberg and Kraus were on a tour of Asia, they and their families were interned in Java by the Japanese from 1942 to 1945.
He toured Australia for three months in 1946. Eventually he went to the United States and became a naturalised American citizen in 1953. From 1951 to 1965 he taught at the Aspen Music School. Concurrently he was active as a conductor. In 1955 he founded the Netherlands Chamber Orchestra in Amsterdam, which he led until 1979. He also took the ensemble on many tours. From the years 1977 to 1979 he was the conductor of the Manchester Camerata.
He taught at Yale University from 1978 to 1982, the Juilliard School in New York City from 1978 to 1980, the Curtis Institute of Music in Philadelphia from 1980 to 1981, and the Manhattan School of Music in New York starting in 1981. From 1990 until his death, he conducted the New Japan Philharmonic in Tokyo.
|Principal Conductor, Netherlands Chamber Orchestra