As a result of that deed, Fitz Urse fled to Ireland and the manor passed into the ownership of his kinsman, Robert de Berham. The de Berham family (now called Barhams) became one of the great families in Kent.
But at the end of Elizabeth I's reign, the property passed to Sir Oliver Boteler and his wife, Anne. The Botelers (later they changed their name to Butler) were Royalists and Barham Court was sacked by Cromwell's New Army during the Civil War. William Butler, their son, was imprisoned in London for his support of the Kentish Royalist Petition of 1642, which indirectly led to the Battle of Maidstone 1648.
The last of the Butlers, Sir Philip, was responsible for changing the course of the old Tonbridge-Maidstone road, which used to run north of the church and then south of the house on its way to Barming and Maidstone. He had the road moved 'some hundred rods' (say five hundred and fifty yards) to the south.
When Edward Hasted visited the house in the 18th century, then owned by the Bouverie family, he described it as the greatest ornament of this part of the county. After that it passed to the Charles Middleton, 1st Baron Barham.
William Wilberforce was a frequent house guest of the first Lady Barham, who is said to have inspired and supported him in his fight against slavery. He loved the place and once wrote that 'for the charm of softness and elegance I never beheld a superior to Barham Court'.
The house was very badly damaged in a fire in the 1930s. Today the house itself has been converted into offices, with apartments attached.