The collection was bequeathed to the people of Southwark by Henry Syer Cuming in 1902 and the museum opened in 1906. As described in Cuming's will it comprised, "My Museum illustrative of Natural History, Archaeology and Ethnology with my coins and medals and… other curios".
The building adjoins a local library. On this site, Charles Babbage, the Victorian mechanical computer pioneer, was born in 1791, although the original house has been demolished. A blue plaque records his birth.
On 25 March 2013 a fire seriously damaged the museum and library but the collections only suffered a very small loss. Around 98% of objects on display at the time of the fire were recovered and are now safely stored awaiting the new galleries. The museum remains closed while restoration works commence, which the council says 'will take several years'. The new architects for the scheme are looking at a completion date of late 2019/2020.