Herschel was founded by German Canadians and settled mainly by Mennonites. The village was named in honour of 19th Century English physicist and astronomer Sir John Fredrick William Herschel.
In 1935, a major fire destroyed 18 homes and businesses in Herschel. The blaze destroyed three general stores, the post office, a harness shop, the drug store, a pool hall, the Royal Hotel, a Chinese laundry, a hardware store, the municipal office, a restaurant, homes and other structures. Poor local roads were blamed for the extent of the blaze, with fire engines from Rosetown unable to reach the community for more than four hours.
In 2005, fossils of a plesiosaur were discovered in a coal mine ravine southwest of Herschel, and now, along with native petroglyphs attracts visitors from around the world. The fossils were described as a new species, Dolichorhynchops herschelensis.