A short-lived gold placer mining camp existed here in the 1860s, but the area was first settled by a prospector named Doby Brown in the late 1870s or early 1880s. By 1882 enough settlers had gathered to establish a post office. In 1888 when a local gold rush began at nearby Miners Basin, Castleton became important as a supply town. It had a general store, hotel, two saloons, and several other businesses. At its peak in 1895, the population exceeded that of Moab. In fact, when Grand County was organized in 1890, Castleton vied with Moab for the chance to be county seat.
The Panic of 1907 closed down the area's mines, and soon ranchers were Castleton's only residents. By 1910 the businesses were gone, leaving only the post office, and the population had dropped to 50. In 1930 there were six residents. A few people stayed in the dead town for decades, but in 1967 the county commission officially vacated Castleton as an occupied town.