In 1825 the Rev. John Dunmore Lang appealed for a minister for the church at Portland Head on the Hawkesbury River, New South Wales; McGarvie was chosen and arrived in Sydney in the Greenock on 22 May 1826 after a six-month voyage from Leith. Met on arrival by Reverend Lang, McGarvie went straight to Portland Head as the first Presbyterian minister of the Ebenezer Church, remaining until 1830.
In 1829 McGarvie contributed a series of biographical articles to the Sydney Gazette with a number of his poems under such pseudonyms as 'M., Ananbaba', 'A.B., Marramatta' and 'C.D., Warrambamba'. When the Sydney Herald commenced in 1831, with his brother William as one of its three original proprietors, he contributed the editorials, and continued as leader writer for a significant period. McGarvie was one of the founders in 1826 of the Sydney Dispensary (afterwards the Sydney Infirmary and Dispensary, and later Sydney Hospital) and was its honorary secretary from 1836 until his death. He was also a founder of the Sydney School of Arts. On the establishment of the Australian College in 1831 he conducted the mathematics department and also lectured in chemistry and natural philosophy.
In 1832 McGarvie accepted an invitation to start a second Presbyterian congregation in Sydney and held services in the courthouse until St Andrew's Scots Church was built at Kent Street near Bathurst Street. The church opened for divine service on 13 September 1835 and he remained as its minister until his death in 1853. Among the children he christened was John McGarvie Smith.
In 1840 the University of Glasgow awarded McGarvie an honorary Doctor of Divinity for his lifetime achievement. A volume of his sermons was published in Sydney in 1842. He died on 12 April 1853 and was buried at Gore Hill cemetery, he was unmarried.
- J. Maclehose, The Picture of Sydney (Syd, 1838); J. Cameron, Centenary History of the Presbyterian Church in New South Wales, vol 1 (Syd, 1905); A Century of Journalism: The Sydney Morning Herald, 1831–1931 (Syd, 1931); G. R. S. Reid, The History of Ebenezer, Australia's Oldest Church (Petersham, 1951); C. A. White, The Challenge of the Years: A History of the Presbyterian Church of Australia in the State of New South Wales (Syd, 1951); Truth (Sydney), 26 Mar 1911; McGarvie papers (Presbyterian Library, Assembly Hall, Sydney)