Oyster Harbors was first inhabited by the Wampanoag tribe when they first settled on Cape Cod. A legend states Captain Kidd presumably had buried treasure at Noisy Point during this time, guarded by the witch Hannah Screecham. Around 1658, the island was reserved for Native Americans. However, after an expensive lawsuit in 1737, they sold the island to the Lovell family. For the next two centuries, the island would remain uninhabited, and was used only for salt works and pastureland. Because of this, the island's oldest house, the Nymphas Marston House which was built in 1680, was actually moved there from the main part of town.
In 1904, Harvard University professor Edward Channing, who also was the recipient of the Pulitzer Prize, built the first permanent house on the island, in the form of a hunting lodge. Eventually, Boston families began to build summer homes on the island as well. In 1925, Forris W. Norris, an entrepreneur real estate developer from Boston visited the Grand Island and saw its potential. The summer of 1925 he had purchased the holdings, previously owned by Richard and Helen Winfiled. That same year, the Olmsted Brothers laid out a golf course, and the island changed its name from Grand Island to Oyster Harbors. Along with about 30 other people, Norris formed Oyster Harbors, Inc. in 1929. In only 3 years this development had turned into a charming summer resort, with a wonderful golf course, harbors, roads, Cape Cod style homes, and a clubhouse. The landscape had been done by the Olmsted Brothers, top landscape architects of their time, who had also designed Central Park in New York, and the golf course was designed by prominent golf course architect, Donald Ross. Between 1925 and 1967 Oyster Harbors Club was owned by investors and stockholders, which for many years members did not know. At that time there was no initiation fee to be a member (members were invite only) and annual dues were paid on April first. If one had not received a bill for their annual dues, one would understand that he was no longer a member and could no longer use the facilities. These conditions existed until the establishment of the new club in 1968.
Upon the death of T. Kenneth Boyd in 1960, the majority stockholder and devoted supporter of Oyster Harbors, it was a financially hard time for the Club and there had been rumors that the Boyd family would sell the estate to the highest bidder. There had been a few who wished to purchase from the Boyd family including Howard Johnson, who planned to turn Oyster Harbors into a commercial resort with profit as their only motive. Two residents of the island, Paul Mellon and Harry Hoyt, Sr. became aware of the issue and became the primary stockholders and owned all the stock with the exception of one share owned by the late Alexis F. du Pont and his sister, and three shares owned by Gladys B. Thayer. Eventually in 1980 Mellon became the only stockholder of the corporation. Without Paul Mellon's determination to have Oyster Harbors remain a private island, the club, the course and the entire island would have been much different today.