Wilson attended Aldine Carver High School where he played football and was also the state champ in the shot put. While at Carver, he was a Jones scholar who was offered an academic scholarship to attend the University of Cincinnati, but he wanted to play football. Although he was an exceptional football player, he did not receive a scholarship offer from any white college in Texas because of segregation.
The superintendent of Wilson's high school had contacts at USC and Wilson received a scholarship to attend USC. While at USC, Wilson became the starting fullback and team captain of USC's 1962 national championship team.