Williamson was a wide receiver under legendary Alabama coach Bear Bryant for the 1961 and 1962 seasons. He caught Joe Namath's first touchdown pass at Alabama. He was one of five players who testified to an Alabama Legislative Committee after The Saturday Evening Post ran an article claiming that Georgia head coach Wally Butts had conspired with Bear Bryant to throw a football game. Both Bryant and Butts later were awarded money in libel suits against the paper. Williamson was the 55th pick in the 1963 American Football League Draft, drafted by the Boston Patriots, but he chose to stay at Alabama as a coach, helping the Tide win National Championships for the 1964 and 1965 seasons.
After a two-year coaching stay at the University of Arkansas, Williamson returned to Alabama for 1970-1971 before leaving for Arkansas again (from 1972 to 1974). Williamson then left for Memphis State University, this time becoming head coach (1975–1980). Williamson's teams finished 7-4 (1975, 1976), 6-5 (1977), 4-7 (1978), 5-6 (1979), and 2-9 (1980). Williamson was honored with the Southern Independent Conference Coach of the Year award twice. After being fired from Memphis, Williamson spent several years as the executive director of the Bluebonnet Bowl.
Williamson returned to coaching in 1983, when he was hired as an assistant for the Kansas City Chiefs under new coach John Mackovic. After Mackovic was fired following the 1986 season, Williamson was told by the new head coach that he would not be retained. He moved to the Tampa Bay Buccaneers, working under fellow former Alabama coach Ray Perkins as offensive coordinator. When Perkins was fired late in the 1990 season, Williamson was named interim head coach, leading the team to a 1-2 record. Named head coach in 1991, Williamson went 3-13 before being fired at the end of the season.
In 2000, Williamson was named assistant head coach under George Seifert; he was later named offensive coordinator as well, after Bill Musgrave resigned four games into the season. After the 2001 season, Williamson returned to coachng the receivers under new head coach, John Fox. Williamson was well known as one of the top receivers coaches in the league. He cemented this reputation during his time in Carolina; he developed Muhsin Muhammad and Steve Smith into Pro Bowl mainstays.
Williamson announced his retirement on January 18, 2010, after 15 seasons with the Panthers. He was the last member of the original 1995 staff still with the team.