The barony of Grove, with the manor of West Retford, was part of the large property granted by William the Conqueror to Roger de Busli. It was noted in Doomsday survey as "Grave". From Roger de Busli it came to Gerbert (or Gilbert) de Arches, Baron de Grove, in the early part of the reign of Henry II. Gilbert's great grand-daughter, Theophania, being a co-heiress, carried it to Malvesinus de Hercy in the reign of Henry III.
The Hercy family built the original wing of Grove Hall. The estate continued in the Hercy family till Sir John de Hercy died in 1570 with no children but with eight sisters. Grove Hall was bequeathed to one of the sisters, Barbara, who had married George Nevile of Ragnall.
It descended in the Nevile family until the latter end of the seventeenth century, when Sir Edward Nevile sold it to Sir Creswell Levinz, one of the Judges of the Common Pleas.
Sir Creswell Levinz was succeeded by his son, William Levinz, who resided at Grove and was High Sheriff of Nottinghamshire for 1707–08 and sometime MP for East Retford and afterwards for Nottinghamshire. This William Levinz left a son, William, who alienated the greatest part of his inheritance and in 1762 sold the manor and estate of Grove, with its appurtenances, to Anthony Eyre of Rampton and Adwick-le-Street.
Anthony Eyre's son, Anthony Hardolph Eyre, died in 1836 leaving two daughters, one of which, Frances, inherited Grove. She had married Granville Harcourt Vernon, son of the Archbishop of York. The property passed down in the Harcourt-Vernon family to Granville Charles FitzHerbert Harcourt-Vernon, who sold the house in 1946. 
A large brick house in the Old English style, with gable ends and mullion windows, had been erected at Grove at a period which is not known, and had since undergone considerable alterations. During the wars between the Houses of York and Lancaster, the Hercy family, with their neighbours the Stanhopes, of Rampton, were active supporters of the House of Lancaster, and during the arduous struggle for superiority were frequently surrounded by dangers of no common kind; till at length Victory
"To Tudor's brow transfer'd the gem, The long disputed diadem."
Afterwards, as a mark of their zeal, and as a remembrance of their past services, they each of them inserted in the walls of their respective mansions, a sculptured rose and crown, the device assumed by Henry VII and by many of his adherents. At Grove this device was placed in the house over a large Gothic window which lights the principal staircase.
Sir Creswell Levinz and his son made some alterations in the house, as did Anthony Eyre after he purchased it, entirely altering the character of it. He removed the whole of the ancient roof and pulled down a considerable part of the south-west front, in the place of which, under the direction of architect John Carr, he built a suite of rooms of more convenient dimensions. In making this alteration, he took down a stone tower, which must have been built in the time of Queen Elizabeth I, under which were found a considerable number of the coins of that Queen’s reign.
The lordship of Grove was extensive, containing about 1,500 acres (6.1 km2), part of which is covered with wood, and the rest is occupied, either in grazing, or to agricultural purposes.
The situation of Grove Hall, is said to be the most elevated and picturesque in the Nottinghamshire; on all sides, the views are pleasing and extensive: to the east the levels of Lincolnshire appear "beautifully tinted with variety, the view of which, is backed with the noble promontory on which part of the city of Lincoln stands, whilst the minster rears its venerable head, and overlooks the vast Plains which extend themselves until the ocean terminates their bounds". To the west the view is equally extensive, the ancient forest of Sherwood, "from the numerous woods and plantations which rear their heads in every direction, reminds the beholder of ancient days, when the famous oaks displayed their towering boughs; this very interesting view is only terminated by the hills of Kinderskout in Derbyshire".
- "Biography of Granville Charles FitzHerbert Harcourt-Vernon". University of Nottingham. Retrieved 23 March 2013.