McFadden is widely considered to be the greatest athlete in Clemson University history, lettering in three sports (football, basketball and track). In 1939, McFadden was voted the Associated Press' "Athlete of the Year". McFadden was also a two-time All-American in basketball (1938 and 1939) and lead the Tigers basketball team to a Southern Conference championship in 1939. McFadden also played halfback and punter on the football team and was named Clemson's first Associated Press All-American in football in 1939, which saw the Tigers play and win in their 1st bowl game (1940 Cotton Bowl Classic).
Upon graduating, McFadden played football for the National Football League's Brooklyn Dodgers. He was the #4 overall NFL draft pick in 1940. In his first, and only, year as a professional he played in 11 games. He had the longest rush in the NFL that year - 75 yards. He was tied for second for most yards per attempt with a 4.8 yards per carry average. He was also fifth in the league for most rushing yards per game. Defensively he had two interceptions. Despite his success, McFadden preferred the small town life and the family atmosphere of Clemson. He returned to the state of South Carolina to coach at his alma mater.
McFadden fought in World War II and upon returning to the United States returned to coaching. McFadden served as Clemson's head basketball coach from 1946-1956 in addition to stints as head track and assistant football coach. He retired from coaching in 1969 and took over the university's intramural department, which he directed for 15 years.
On September 19, 1987, Clemson University retired his basketball No. 23 and football No. 66.