Elmo's history begins in 1870 when the Texas and Pacific Railway laid track through the area. A community sprang up at the railhead, and it was decided that the new town be named to honor Elmo Scott, a T&P Railroad surveyor. Elmo received a post office in 1873 and by the mid-1880s possessed several mills, five churches, its own schools and approximately 900 residents. Through the remainder of the 19th century, however, the population fell and by 1945 only 150 people called Elmo home. By 1990 this figure had fallen to 90; it remained at this level through to the 2000 Census.
Henry Qualls (July 8, 1934 – December 7, 2003) was an American Texas and country blues guitarist and singer. He found success late in his life after being "discovered" in 1993 by the Dallas Blues Society. He released his only album in 1994 but toured globally playing at a number of festivals. Qualls was born in Elmo and lived all his life in the settlement. 
^Robert Richard Butler, History of Kaufman County, Texas (M.A. thesis, University of Texas, 1940). Mabel Covington Keller, History of Kaufman County, Texas (M.A. thesis, North Texas State College, 1950).