Kelly was born in Jackson, Mississippi, U.S. He enrolled at Texas Southern University in 1979, and played basketball for the team for four seasons. A counselor gave him his nickname "Machine Gun Kelly" in his freshman year due to his prolific scoring; in his college career he scored 3,066 points — the fifth most in NCAA Division I history — and averaged 27.9 points per game. In addition, he achieved a career total of 1,085 rebounds, averaging 9.9 per game, making him the first player to score over 3,000 points and grab 1,000 rebounds in NCAA history. He was named Southwestern Athletic Conference Player of the Year two times and received the John McLendon Award, which honors the best player at a historically black college, in 1982 and 1983.