Born in Alpine, Tennessee, he started building instruments at an early age. When he was 18, Ledford was given a scholarship to attend the John C. Campbell Folk School in Brasstown, North Carolina. He later attended Berea College, where he met his wife Colista. Ledford eventually transferred and graduated from what is now the Eastern Kentucky University in 1954. Ledford worked as a high school industrial arts teacher at George Rogers Clark High School in Winchester, Kentucky before becoming a full-time instrument maker.
He was also an original inductee in the Kentucky Stars, alongside Loretta Lynn, Rosemary Clooney, Bobbie Ann Mason, and Patricia Neal. A sidewalk plaque honoring him is in front of the Kentucky Theatre on Main Street in Lexington, Kentucky.
He devoted his career to instrument-making and to playing with the Cabin Creek Band. He also published a book of autobiographical stories and poems, entitled See Ya Further Up the Creek.
Ledford died from a stroke at the age of 79 in Winchester, Kentucky. He was survived by his widow, Colista; they had four children. His great nephews Jason Eubanks, Phillip Eubanks, and Jonathan Armak are currently in the Experimental Group Unstable, and use many of the instruments he made on their records. Their mother, Melissa Armak (born Melissa Fraley) was the bass player in the Cabin Creek Band during the late 1970s.
He was posthumously given an honorary Doctorate of Humanities at the Fall 2006 commencement ceremonies at Eastern Kentucky University, December 16, 2006, and deemed one of Kentucky's "Heroes, Saints and Legends" by Wesley Retirement Community in recognition of his contributions to music.