Walker was born in Roberta, Georgia. After being named Mr. Basketball in his home state of Georgia in 1982, Walker chose to play collegiately at the University of Kentucky. Walker had a very successful college career, being named to an All-SEC team four times and the All-American team twice. Walker's 1984 Kentucky team made it to the Final Four of the NCAA Tournament before losing to the Georgetown Hoyas.
In 1986, Walker set a record by scoring 11 times on 11 field goal attempts in the NCAA playoffs.
Walker was selected with the 5th pick of the 1986 NBA Draft by the New York Knicks. Walker played for five coaches in five years with the Knicks. On February 11, 1989, Walker won the NBA Slam Dunk Contest, competing three days after the death of his father. He also secured 3rd place in the 1990 contest. However, his success on the court was dwindling, and knee injuries forced him to leave the NBA and join the ACB league in Spain.
Walker returned to the NBA in 1993 to play two seasons with the Washington Bullets as a role player. He played a season (c.1998) in Japan before retiring from professional basketball.
As a publicity stunt, Walker once raced against a horse, and won. On August 14, 1986, at New York's Monticello Raceway, Walker and harness racehorse "Pugwash" ran against each other over one-sixteenth of a mile in 10.4 seconds. The New York Times wrote, "It is a well-established fact that a fast human can outrace a slow horse, at least if the human is unfettered and the horse is forced to pull a sulky and driver, and the distance isn't much more than a 100-yard dash."