Armstrong was born in 1858 in Frederick County, Maryland. As a child, he received piano lessons in Stuttgart, Germany. Later he began playing professionally and teaching lessons on the instrument. Armstrong published musical reviews in major newspapers such as the Chicago Tribune, where he was the music editor (1893 - 1898), and the New York Times. He also published reviews and essays in periodicals such as the Saturday Evening Post and the Saturday Review. In addition to his reviews and essays, Armstrong wrote several books. These include the novels Thekla (1887) and An American Nobleman (1892), and the essay series The Romantic World of Music.