The original structure of two stories and three bays was built by Matthew Robinson or Robertson, probably before the American Revolution. He died in 1792 and left the house to his son David. Around 1800, it was sold to the Lewis brothers, John and Samuel. They were probably responsible for an addition of two bays on the west side of the house at this time. Local tradition associates the house with a tavern, possibly at this time; however, if so, it was never licensed to dispense liquor. In addition to their reputed tavern-keeping, the Lewises also opened an iron mine on the farm. They lost the property in 1824, when it was sold by the sheriff to Isaac Wayne Van Leer.
Van Leer, a grandnephew of Anthony Wayne, was the son of William R. Van Leer, a local ironmaster. Isaac Wayne was a progressive agriculturalist and horticulturalist. He enlarged an existing structure into a barn soon after purchase, and replaced the old log kitchen with a new addition to the house in 1843. Van Leer also added gilt paint and other fashionable touches to the house and planted rarities in his gardens. His grandnephew, George Howard Earle III (later Governor of Pennsylvania) was born at the house in 1890. After Van Leer's death in 1896, the condition of Pleasant Hill deteriorated, particularly when it was rented out during the 1930s. It was restored in 1948 and remains a private residence.