The village was once part of the Burton and Potter farms. Around 1813 Caleb Atwood built a textile mill in the village, known as the Dumplin Mould, and later the building became the Lamphear Machine Shop in the 1840s. The village was named for Elisha Harris who arrived in 1822 and formed the Harris Cotton Manufacturing Company, which eventually constructed several mill buildings in the area. The company continued under Harris' son-in-law after his death. In 1900 the Arkwright-Interlaken Manufacturing Company purchased the Harris Mill and kept it operational until 1954. Many of the mill buildings and worker housing survive today.