The present Pitfour Castle was built for John Richardson (1760-1821), a wealthy local man involved with the salmon fisheries of the Tay, around 1784. The design of the new house is attributed to the architect Robert Adam (1728-1792): although there is no documentary evidence for his involvement, there are stylistic similarities with Adam's later "castle-style" homes, such as Dalquharran Castle in Ayrshire (1786) and Seton Castle in Lothian (1789). In 1825, the house was enlarged to designs by William Burn, with a library to the north-west and a clock tower to the south-west.
Having fallen into disrepair, restoration work on Pitfour Castle began in 1964. In 1966, Burn's entrance hall was removed, revealing Adam's original entrance front. The house and 12 acres (4.9 ha) was advertised for sale in 1967 for £25,000. In July 1969, as restoration was nearing completion, the east wing was badly damaged by fire. In 1974, it was rescued from demolition and converted into apartments by Lord Leslie Charles Field. The castle has since been subdivided into individual, privately owned apartments. The stable block has also been converted into a series of two storey dwelling houses, which along with the lodge house, are also privately owned. Some of the apartments are available to rent on short-assured tenancy agreements.
The curved staircase forms the centrepiece of the house. In the 1967 sale particulars, this was described as a "fine circular wrought iron staircase rising through the house to a cupola with painted cartoons by Ritchie". Historic Scotland refers to these cartoons as "figure panels by Zucchi", an Italian who worked with Robert Adam. Prior to its subdivision, the house had 12 principal bedrooms, four bathrooms, five reception rooms and staff accommodation.
- Gifford, John (2007), Perth and Kinross, Yale University Press, ISBN 978-0-300-10922-1
- "Pitfour Castle, Ref: 17628". Historic Scotland. Retrieved 4 March 2014.
- RCAHMS. "Pitfour Castle (28241)". Canmore. Retrieved 4 March 2014.
- Haldane, A. R. B. (1981), The great fishmonger of the Tay: John Richardson of Perth & Pitfour (1760-1821), Abertay Historical Society
- RCAHMS. "St Madoes, Pitfour Castle (28220)". Canmore. Retrieved 4 March 2014.
- "Pitfour Castle". Dictionary of Scottish Architects. Archived from the original on 4 March 2014. Retrieved 4 March 2014.
- Country Living Property Supplement, 13 July 1967, p.12