Born in Sinking Spring, Pennsylvania, Specht was a violinist, having been taught by his father Charles G. Specht, a violinist, organist, and bandleader in his own right. He attended Combs Conservatory in Philadelphia, and led his first band in 1916, which toured the Western United States during World War I. He signed with Columbia Records in 1922, playing both with a larger dance ensemble and with a smaller, more jazz-oriented unit called The Georgians. One example of a Columbia recording is on Columbia # 27-D. Titled "Dear Old Lady", with the Hotel Alamac Orchestra and "Take,Oh Take Those Lips Away". This is on a Columbia record commonly known to record collectors as a "flag label".
He toured England several times, beginning in 1922, and set up a "School for Jazz Musicians" there in 1924. Specht encountered some difficulty with his English performances due to political and union woes, which were documented regularly in the popular music press of the day. He did not return to England after 1926, having become thoroughly dissatisfied with the treatment he received.
Specht's ensemble was the first orchestra to broadcast for the RCA company, and was the first ensemble to film after the end of the silent era. In 1929, Specht's orchestra was asked to play at the inauguration of Herbert Hoover, chosen over Paul Whiteman. As a radio bandleader in 1932, his band and the Three X Sisters harmony trio collaborated on ABC radio airwaves for several different musical formats. He continued to be popular into the 1930s, and led bands into the 1940s, during which time he developed arthritis which hampered his musical abilities. He lived in Greenwich Village late in his life and did arranging work for radio and television. He died in 1954 at the age of 59 in New York City.
A number of noted jazz and popular musicians played in Specht's ensembles, including Hank D'Amico, Russ Morgan, Sylvester Ahola, Arthur Schutt, Charlie Spivak, Joe Tarto, Art Christmas, Chauncey Morehouse, Clarence Zylman, and Lou Calabrese (Lou Breese).