The 3,000-square-foot (280 m2) house was built by John Wesley McElroy as a mansion for his wife, Catherine. McElroy was a local businessman and lawyer, and a brigadier general in the Confederate Army. During the war, the house was used as a hospital and the headquarters for the home guard. In 1889 the house was purchased by William Moore, a state senator and former captain in the Union Army. Moore's family lived in the house until 1917 and it became the first Post Office in Burnsville. During the 1970s the house was abandoned and fell into disrepair.
The house was purchased in 1987 by the Yancey History Association. The Association restored the building, which was opened as the Rush Wray Museum of Yancey County History in 2003. The museum houses period furniture and exhibits on local history.