McGee was born in Overton, Texas. He played high school football in White Oak, Texas. McGee was the first player in High School football history ever to rush for over 3,000 yards in a single season. He rushed for 3,048 his senior year as a White Oak Roughneck in 1949.
McGee was drafted by the Green Bay Packers in the 5th round (51st overall) of the 1954 NFL Draft. He served as the Green Bay Packers punter during a few years of his career. In his rookie season, McGee led the NFL in punting yards while catching 36 passes for 614 yards and nine touchdowns. He missed the next two seasons to serve as a pilot in the U.S. Air Force, but returned to become the Packers' leading receiver from 1958 to 1962. McGee was one of the few bright spots on the 1958 Packers team, which finished the season with a league-low 1-10-1 record, worst in Packers history. During that season, he led the NFL in yards per catch average (23.2), punting yards (2,716) and net yards average (36.0).
After Vince Lombardi took over as the team's head coach in 1959, McGee may be best known for his performance during the first Super Bowl game. He helped the team to six NFL championship appearances, five NFL championship wins, and two Super Bowl titles during the remaining years of his career. He was a Pro Bowl selection during the 1961 season.
Despite reductions in playing time due to injuries and age, McGee's final two seasons would be the ones for which his career is best remembered. In the 1966 season, McGee caught only four passes for 91 yards and a touchdown as the Packers recorded a 12-2 record and advanced to Super Bowl I against the Kansas City Chiefs. McGee did not expect to play in the game and he violated his team's curfew policy and spent the night before the Super Bowl out on the town. The next morning he told starting receiver Boyd Dowler, "I hope you don't get hurt. I'm not in very good shape," alluding to his hangover. Dowler went down with a separated shoulder on the Packers' second drive of the game, and McGee, who had to borrow a teammate's helmet because he had not brought his own out of the locker room, was put into the game. A few plays later, McGee made a one-handed reception of a pass from Bart Starr, took off past Chiefs defender Fred Williamson and ran 37 yards to score the first touchdown in Super Bowl history. This was a repeat of his performance in the NFL championship game two weeks earlier, when he had also caught a touchdown pass after relieving an injured Boyd Dowler. By the end of the game, McGee had recorded seven receptions for 138 yards and two touchdowns, assisting Green Bay to a 35-10 victory.
The following year, he recorded a 35-yard reception in the third quarter of Super Bowl II that set up a touchdown in the Packers 33-14 win over the Oakland Raiders. McGee retired shortly after the game and finished his 12-season career with 345 receptions for 6,346 yards and 12 carries for 121 yards. He scored 51 touchdowns (50 receiving and 1 fumble recovery). On special teams, he punted 256 times for 10,647 yards, an average of 41.6 yards per punt, and returned 4 kickoffs for 69 yards.
After retiring from football, McGee became a major partner in developing the popular Chi-Chi's chain of Mexican restaurants with restaurateur Marno McDermott.
McGee was inducted into the Packers Hall of Fame in 1975. His ties to the Packers continued from 1979 to 1998 when he served as the color commentator for radio broadcasts of Packers football games. With droll wit and keen insights, McGee was extremely popular as a color commentator and was named Wisconsin "Sportscaster of the Year" an unprecedented ten times over his career — in 1975, 1976, 1978, 1980, 1983, 1984, 1985, 1986, 1988 and 1989.