Burgmann was elected as Bishop of Goulburn on 2 February, consecrated to the episcopate on 1 May and installed on 4 May 1934. He served until his retirement on 31 December 1960, although his diocese and position changed to Canberra and Goulburn in 1950.
Although he never joined a political party, Burgmann was active in Australian politics and maintained a strong interest in working-class issues. H. V. Evatt appointed him to the Australian delegation at the 1948 United Nations Assembly in Paris. Burgmann opposed the attempted banning of the Communist Party of Australia in 1951 and was described by the Prime Minister of Australia, Robert Menzies, in Parliament as a "most meddlesome priest".
Burgmann wrote prolifically throughout his life, mainly essays and booklets on social justice and reform, together with interpretations of biblical scripture. Burgmann College, established in 1971 and affiliated with the Australian National University, and Burgmann Anglican School in Canberra are named after him. He was the grandfather of Australian Labor Party politician Meredith Burgmann and academic Verity Burgmann and great grandfather of comedian Charles Firth and politician Verity Firth.
- "Ernest Henry Burgmann". Australian Dictionary of Biography Online.