Burgmann was elected as bishop of Goulburn on 2 February, consecrated on 1 May and installed on 4 May 1934. He served until his retirement on 31 December 1960, although his diocese and title changed to Canberra and Goulburn in 1950.
Although he never joined a political party, Burgmann was active in Australian politics and maintained a strong interest in working-class issues. H. V. Evatt appointed him to the Australian delegation at the 1948 United Nations Assembly in Paris. Burgmann opposed the attempted banning of the Communist Party of Australia in 1951 and was described by Prime Minister Robert Menzies in Parliament as a "most meddlesome priest".