Romney at the age of 63, who had previously lived at 32 Cavendish Square for 21 years, bought the house in 1796. A memoir of Romney was written by his brother, John, and published in 1830. In the memoir, John describes his dissuading George from buying four acres of land on Edgware Road on which to build a house, which John feared it would be "attended with more expense than his limited circumstance would allow." He was persuaded by his brother to buy the house on Holly Bush Hill and possibly add a "gallery and painting-room at small expense." The house at Holly Bush Hill was bought for roughly £700 (£63.3 thousand as of 2014) and his brother believed that another £500 would have added a gallery to the house which "would have answered all his purposes."
Despite the purchase the house John recalled that Romney "was under the influence of some worthless people, who profited by his impudence." These people encouraged Romney "in the scheme of building." The original house had at the time of Romney's purchase "a very large garden pleasantly situated, and an excellent stable, coach house...situated above the garden on top of the hill." Romney subsequently demolished the stable and built a new stable on adjoining ground and on the site of the old stable built a "whimsical structure, consisting chiefly of a picture and statue gallery" and a "wooden arcade for a riding house." In the course of the building works Romney removed some paintings and put them outside, where some were damaged by frost, and others were stolen. The building works cost Romney £2733 (£234 thousand) as of 2014). The arcade extension later sold for £357 (£30.6 thousand) as of 2014). and the original house for close to the £700 purchase price. One of the pictures damaged in Hampstead was Nature Unveiling Herself to Shakespeare, which had been alluded to in poetry by Helen Maria Williams.
Romney sold the house in 1799 three years before his death. The house was further enlarged in 1807 as the Hampstead Assembly Rooms with a tea room, ballroom and new card room. Bridget Cherry in Nikolaus Pevsner's Buildings of England describes Romney's House as "picturesque and weatherboarded."
In 1929 the house was bought by Clough Williams-Ellis. The English section of an international writers organisation called "For Intellectual Liberty" was hosted by the Williams-Ellises' at the house. Such notable writers as Cecil Day-Lewis, Goronwy Rees and Rose Macaulay met at Romney's House as part of "For Intellectual Liberty".
In July 2012 Romney's House was stormed by armed police after reports that guns had been fired in the area. A member of the public had seen an air rifle inside the house. Builders working at the house were questioned by police and several imitation firearms were recovered by police from the house and destroyed. No one was arrested in the incident.
Ethelbert White painted a portrait of Romney's House in 1940. The interior of Romney's House was redesigned by 6a architects in 2012. The Assembly Room was divided into two bedrooms, divided by a curving full height partition, and a staircase tower was added to unite the different levels of the house.
- "Romneys House". English Heritage list. English Heritage. Retrieved 9 December 2013.
- William Howitt (1869). The Northern Heights of London; Or, Historical Associations of Hampstead, Highgate, Muswell Hill, Hornsey, and Islington. pp. 254–.
- John Romney (1830). Memoirs of the Life and Works of George Romney. Baldwin and Cradock. p. 253.
- UK CPI inflation numbers based on data available from Gregory Clark (2014), "What Were the British Earnings and Prices Then? (New Series)" MeasuringWorth.
- Cherry and Pevsner 1999, p. 222.
- Cherry and Pevsner 1999, p. 222.
- "ROMNEY, GEORGE (1734-1802)". English Heritage. Retrieved 9 December 2013.
- Yvonne Kapp (2003). Time Will Tell: Memoirs. Verso. pp. 181–. ISBN 978-1-85984-510-3.
- "'False alarm' sighting of workmen with air gun caused Hampstead police siege". The Ham and High. Retrieved 9 December 2013.
- Julie Chamberlain (25 May 2007). "Collection is to everyone's taste". Coventry Evening Telegraph. Retrieved 14 December 2013. (HighBeam subscription required)
- "Projects - Romney's House". 6a architects. Retrieved 14 December 2013.
- Cherry, Bridget and Pevsner, Nikolaus. (1999) The Buildings of England: London 4: North Yale University Press ISBN 978-0-30009-653-8