Hortonville was first settled in the 18th century by the Hale, Martin, and Eddy families. These families built homes on farmland they owned in the western end of the village. The eastern end of the village, along the Cole River, became the site of several gristmills, lending the village the name "Swansea Factory." In the 1830s, a school, church, and Universalist hall were built in the village. Nathaniel Horton settled in the area in 1856 and helped the village gain its own post office, which was named Hortonville in his honor. After 1900, industrial development in the area stalled, and the village became a residential and agricultural area.