Crowe began playing the banjo early on and was offered a job with Jimmy Martin's Sunny Mountain Boys in mid-1950s. He cut his first recordings with Jimmy Martin on December 1, 1956, and his last on August 17, 1960, recording a total of 33 songs on Decca Records. In the late 1960s, Crowe formed the Kentucky Mountain Boys, principally performing in the Lexington region. By the early 1970s, Crowe changed the band's name to The New South and included material from rock and country music sources. Crowe's New South band is widely considered one of the most influential bluegrass groups since the 1970s. Many influential musicians have been a part of the band over the years, including Keith Whitley, Ricky Skaggs, Tony Rice, Jerry Douglas, Doyle Lawson, and Bobby Slone.
Kentucky Educational Television in 2008 aired a biography of J.D. Crowe "A Kentucky Treasure: The J.D. Crowe Story" produced by H. Russell Farmer.
Crowe received the Bluegrass Star Award, presented by the Bluegrass Heritage Foundation of Dallas, Texas, on October 15, 2011. The award is bestowed upon bluegrass artists who do an exemplary job of advancing traditional bluegrass music and bringing it to new audiences while preserving its character and heritage.