The manor of Clovelly was for over 600 years owned by the Cary family but in 1738 the estate was sold to locally born Lincoln's Inn lawyer Zachary Hamlyn (1677-1759). He rebuilt the old manor house in 1740, but this was burnt down in 1789 and rebuilt in the Gothic style by his successor. On his death he left the property to his great-nephew James Hammet (1735–1811) who changed his name to Hamlyn. He married Arabella Williams, a Carmarthenshire heiress, and was created a baronet in 1795 (see Hamlyn-Williams baronets). In that year he carried out substantial improvements to the house.
The 4,600-acre  estate passed on the death of the 3rd and last baronet in 1861 to his son Neville and then following his early death to the baronet's eldest daughter Susan Hester Hamlyn-Williams who married Henry Fane, creating the Hamlyn-Fane family.
The house and estate remain in the family and are managed by the Hon. John Rous (born 1950), great-great-grandson of Susan Hester Hamlyn-Fane, great-grandson of Prime Minister H. H. Asquith and son of the 5th Earl of Stradbroke.
All Saints Church
The parish church of Clovelly, dedicated to All Saints, is located immediately to the west of the manor house, and thus at a considerable distance from the village of Clovelly. It contains many mural monuments and monumental brasses to the Cary family, longtime lords of the manor, and to their successors the Hamlyns and Hamlyn-Fanes. Most of the stained glass windows are 19th century, but some small fragments of ancient stained glass are set into the north chancel window, displaying the arms of Berkeley and Newburgh Earl of Warwick (without the ermine cheveron).
The gardens and parts of the estate, but not the house, are open to the public.
- Heritage Gateway: architectural description of listed building Archived 2012-03-23 at the Wayback Machine
- The Acre-ocracy of England 1876 John Bateman
- Victoria County History re Sopley Church, Hampshire, in which parish was the estate of Avon Tyrrell, a possession of the Fane family which married the heiress of Clovelly: "The east window contains some old glass mixed with modern; among it is a shield of Berkeley impaled with a quarterly coat of Nevill of Raby (Nevill Earl of Salisbury would be the more natural coat), Newburgh Earl of Warwick (without the ermine cheveron), Clare and Despenser quarterly, and Beauchamp differenced with a crescent sable". Victoria County History, Hampshire, Volume 5, London, 1912, pp.127-132, Parishes: Sopley