In the early days of British rule in Hong Kong, the site was part of the Murray Parade Ground. Then in 1851, it was opened as a sports and recreation area and became home to the Hong Kong Cricket Club and its ground. In 1975 Chater Garden took up the space left by the club when it moved to Wong Nai Chung Gap. The garden was developed in the 1970s, and formally opened on 10 October 1978.
Due to its proximity to the seat of government, the garden has been used as a location for political rallies and demonstrations by groups in Hong Kong such as Falun Gong.