Jones was born in Harlan, Kentucky. He holds the unique distinction of being an All-American under both legends, Adolph Rupp (basketball) and Bear Bryant (football) when both coached at Kentucky. He is the only player to have his number retired in both football and basketball at Kentucky. At the University of Kentucky, Jones was a member of the Phi Delta Theta Fraternity. He won a gold medal in the 1948 Summer Olympics with Adolph Rupp's "Fabulous Five" and the Phillips 66ers.