It was first established in Baden bei Wien near Vienna by Hungarian musicians who had fled their homeland after it was invaded by Soviet troops. This refugee ensemble gathered together some of Hungary's finest musical talent and was directed by none other than Zoltán Rozsnyai, former conductor of the Hungarian National Philharmonic. Through the ardent efforts of Rozsnyai and honorary president Antal Doráti, the Philharmonia Hungarica quickly matured into one of Europe's most distinguished orchestras. During the 1970s, Doráti and the orchestra, under contract with Decca Records, made a recording of the complete cycle of Joseph Haydn's symphonies; only one other ensemble, the Austro-Hungarian Haydn Orchestra, conducted by Ádám Fischer, has since repeated this feat. Doráti's recording has been widely reported as a world-first. However, this was not the case. Ernst Märzendorfer earlier recorded the complete Haydn symphonies with the Vienna Chamber Orchestra, although it was given a very limited release.
In 1976 the Philharmonia Hungarica recorded Jon Lord's critically acclaimed 'Sarabande' album, an original collection of movements inspired by the baroque dance suites.
From the orchestra's inception, the West German government sought to harness its anti-Soviet propaganda potential. As a result, the government generously funded the orchestra throughout the Cold War and continued extending subsidies even after the Iron Curtain fell in 1990. The full withdrawal of state subsidies at the start of 2001, combined with the long-term decline in concert attendances, aggravated the financial problems that threatened the orchestra's survival.
The beleaguered Philharmonia Hungarica finally disbanded after giving a farewell concert in Düsseldorf on 22 April 2001, featuring a performance of Anton Bruckner's Symphony No. 9 including the Finale (Ed. Nicola Samale, John A. Phillips, Benjamin-Gunnar Cohrs and Giuseppe Mazzuca) conducted by Robert Bachmann, as well as the German first performance of the Documentation of the Finale Fragment, edited by John A. Phillips, conducted and presented by Benjamin-Gunnar Cohrs. The concert was attended by an estimated audience of 150 in a concert hall meant to hold 2000 (Tonhalle Düsseldorf), after members of the German Orchestral Union had falsely announced in public that the concert would be cancelled. Plans to rescue the orchestra failed, because there was no political interest any longer to keep it alive.