The arboretum was planted in the 1920s under the direction of Arthur Fairchild who in 1947 deeded Magens Bay to the people of the Virgin Islands. Established as a private arboretum on his estate, it was said to contain 200 species representing 71 plant families, including 20 species local to the Virgin Islands and Puerto Rico. The arboretum was neglected after his death, but in 1974 the University of the Virgin Islands began to provide technical assistance, starting with the labeling of rare trees. The Charlotte Amalie Rotary Club aided in its restoration.
The arboretum reopened in 1995 only to be destroyed by Hurricane Marilyn three months later. However in 1997, another 160 trees were purchased and planted. Several are extremely rare native species, such as the small Solanum conocarpum tree, known from only one location in the dry forest of St. John. Other acquisitions included five plumeria varieties from the Honolulu Botanical Gardens.