Rosbaud's first professional post was in Mainz, starting in 1921, as the music director of the city's new School of Music, which included conducting the municipal symphony concerts. He became the first chief conductor of the Hessicher Rundfunk Symphony Orchestra (later the Hr-Sinfonieorchester) of Frankfurt in 1928. During the 1920s and 1930s, he presented premieres of works by Arnold Schoenberg and Béla Bartók. During the Nazi era, his freedom to present new music was restricted. In 1937, he became the general music director of the city of Münster. In 1941, Rosbaud took the same position in Strasbourg, heading the Orchestre philharmonique.
In 1945 he was named music director of the Munich Philharmonic by United States occupation authorities. In 1948, Rosbaud's contract with the Munich orchestra was allowed to lapse because the city authorities wanted to move the orchestra's repertoire in a conservative direction. That year Rosbaud became the first chief conductor of the South West German Radio Orchestra in Baden-Baden, where he remained for the rest of his life. In 1954, he conducted the first performance of Schoenberg's opera Moses und Aron at 8 days' notice; this performance was issued on a 1957 commercial recording for Philips. He regularly took the SWR Symphony Orchestra to festivals of contemporary music, such as at Donaueschingen. He died in Lugano, Switzerland.
Gramophone recently remarked that Rosbaud "was one of the unsung heroes of mid-20th-century music, who ... gave thoroughly rehearsed and assimilated performances and premieres of the widest possible range of music".  In Fanfare, Peter J. Rabinowitz pointed to range of his sympathies, claiming it was "greater than that of just about any of his contemporaries except perhaps Bernstein, Scherchen, and Stokowski. Rosbaud is best remembered, probably, for his Mahler, his Bruckner, his work with the Second Vienna School ... and especially his commitment to the post-war avant-garde. But he was a world-class Mozartian, too (his Aix-en-Provence Mozart operas from the 1950’s hold up far better today than the better-known Busch recordings from Glyndebourne)—and he championed earlier music as well (he recorded Gluck’s Orphée and Rameau’s Platée). What’s more striking is that he was able to give his performances of each of these composers an entirely different signature."  Rosbaud was a highly cultured man, widely read and varied in his intellectual interests. Putting himself at the service of music he chose to perform, he commanded the respect of numerous notable composers of the 20th century. Prominent in his legacy are recordings of the music of Bruckner, Mahler, Stravinsky and Boulez. A tireless advocate of new music, he was closely associated with Karl Amadeus Hartmann, conducting premiere performances of Hartmann's opera Simplicius Simplicissimus and his Second and Fourth Symphonies, amongst others.