A 6'10" forward/center born in Nashville, Tennessee, Lee starred at Vanderbilt University in the mid-1960s. Lee was known for his rebounding skills and inside scoring prowess. In his junior season (1964–65), he led the Commodores to their first SEC championship (Overall: 24-4, SEC: 15-1). Vanderbilt reached the NCAA Mideast Regional Finals, where they lost to Michigan, 87-85. During his senior season (1965–66), he earned All-American honors and the SEC Player of the Year Award. Sportswriter Howell Pesier described him as "the greatest player in Vanderbilt history".