David Charteris McArthur, Melbourne's first banker (with the Bank of Australasia), sportsman (player in first recorded cricket match in Victoria and later captain of the Melbourne Cricket Club) and prominent public figure (the McArthur Gallery in the National Gallery of Victoria is named for him), purchased 84 acres (34 hectares) for £350 in 1838 from one Thomas Walker. He moved there (while keeping a "cottage" in Little Collins Street, Melbourne) in 1840 giving it the name Charterisville. It eventually consisted of a single-storey mansion, with coachhouse, cottages, stables and winery. In 1853 he acquired an adjacent 153 acres (62 hectares) "Waverley" for £850 from his brother-in-law William Darkes. The house was extended substantially around 1868 when McArthur retired. After his death in 1887, the property (by then 108 acres) was sold at auction to John Fergusson and John Roberts, who let the south half of the house to the painter Walter Withers, initiating a 40-year association with the arts.
"Charterisville" was owned by François de Castella, government viticulture expert, in the 1920s. It passed to Hubert de Castella, owner of the Yeringberg Winery, and remained in his family until around 1960.
In its most developed form, it was built on a U-shaped plan, for the most part of local sandstone, with a long east-facing front wing and north and south wings extending to the rear forming a courtyard. An extensive cellar was built under the drawing room. The north wing was demolished in 1962 and rear verandahs enclosed.
With the depression of the late 1880s, grand rural properties became practically valueless, and it was let to a dairy farmer, who (from September 1890 to 1904) let the south wing to painter Walter Withers and his family. They lived there from 1890 to 1894 before moving to nearby Heidelberg and sublet rooms to artists Hal Waugh, Arthur Bassett, Fred Monteath, Thomas Humphrey and Leon Pole.
The second wave of tenants included Harry Recknall, cartoonist Heiner 'Ernest' Egersdorfer (Heinrich Egersdörfer) and Jack Gordon. They were followed by Will Dyson, James Peter Quinn and Max Meldrum. In the summer of 1897-98 Norman Lindsay, Lionel Lindsay and Ernest Moffitt spent some months living in the gardener's cottage. It was here that Lionel introduced Norman to the techniques of etching (and was possibly introduced to German black and white art by Egersdorfer). It is believed Arthur Streeton painted his famous Still Glides the Stream here.
Australia's first recognised summer school of art was run by E. Phillips Fox and Tudor St George Tucker from 1893 to 1901 under the banner Melbourne Art School (Fox took over the lease from Withers in 1894). Students included Hugh Ramsay. This period was important in the history of Australian women's art. Violet Teague, Mary Meyer and Ina Gregory were among those who studied under Fox and painted in Charterisville's then vast grounds (the property was subdivided in 1916, 1920, 1927 and 1939). Students Asquith Baker and Ursula Foster were notable subjects of Fox's own paintings. Marshal Hall, director of the Conservatorium occupied one cottage, later occupied by Ernest Moffat.
Charterisville had a later role in the art history of Australia: outdoor scenes for the very successful 1905 movie The Story of the Kelly Gang were largely filmed there. The work of the Tait brothers (produced by John Tait and Nevin Tait, directed by Charles Tait to a script by Frank and John), it was billed as "the longest film ever made" and made a fortune for its backers. Charterisville was at the time being leased as a dairy farm by Lizzie Tait's family.
- "Bank Centenary". The Mercury. CXLIX (21,142). Tasmania. 30 August 1938. p. 6. Retrieved 1 March 2016 – via National Library of Australia.
- "Our First Artists' Colony". The Argus (31,064). Melbourne. 23 March 1946. p. 8 (The Argus Week-end Magazine). Retrieved 1 March 2016 – via National Library of Australia.
- "The First Girls' High School". The Argus (27,369). Melbourne. 8 May 1934. p. 5. Retrieved 1 March 2016 – via National Library of Australia.
- "Advertising". The Argus (13,030). Melbourne. 26 March 1888. p. 3. Retrieved 1 March 2016 – via National Library of Australia.
- "Floods at Ivanhoe and Heidelberg". The Argus (24,226). Melbourne. 29 March 1924. p. 33. Retrieved 1 March 2016 – via National Library of Australia.
- "Archived copy". Archived from the original on 2010-05-09. Retrieved 2010-02-15.
- "Archived copy". Archived from the original on 2010-10-10. Retrieved 2010-09-03.