The building, which is within the confines of the County Ground, is the Old Priory Barn, a Grade II* listed building. It is the only surviving building of the AugustinianTaunton Priory which was founded about 1115, although there is some evidence that its early use was not as a barn but as a guesthouse or quarters for a Priory official. The local stone rubble has been repaired with red brick and has a tie-beam roof covered with tiles. The door and window openings are made of Bishops Lydeard stone. The exact dates of the building are disputed, but is argued to be from the late 15th or early 16th century, and replacing an earlier 13th or 14th-century building on the site, but incorporating some of the earlier building. It was a Scheduled Ancient Monument until 1999 when it was descheduled. The building was once used as a chapel by Frenchmen, possibly prisoners during the French Revolutionary Wars or Napoleonic Wars.
The Old Priory Barn was purchased by the Somerset CCC Supporters Club when it became available in 1979, as the cricket club was unable to afford it at the time. Over the next ten years the building was restored, and in April 1989 the museum was officially opened to the public.
The exhibits and displays in the museum primarily cover the cricket club's history, including Test match players such as Ian Botham and Marcus Trescothick. It also has a section devoted to the England women's cricket team, as the County Ground is their headquarters. The museum also hosts a collection of I Zingari memorabilia, a club with which current chairman Charles Clive Ponsonby-Fane has strong family links.