Adderley attended Michigan State University in East Lansing and played football under head coach Duffy Daugherty, primarily as a halfback. He led the Spartans in rushing yards as a junior in 1959 and pass receptions in both 1959 and 1960. Adderley was the co-captain of the team as a senior, and made the All-Big Ten Conference team and played in the East-West Shrine Game, the Coaches' All-American, and the College All-Star games. He was picked for the All-Michigan State University team in 1970.
1958: 9 Games - 37 carries for 143 yards and 2 TD. 6 catches for 100 yards.
1959: 9 Games - 93 carries for 413 yards and 2 TD. 13 catches for 265 yards and 2 TD.
1960: 9 Games - 68 carries for 251 yards. 9 catches for 154 yards and 2 TD.
In 1962, the move became permanent and Adderley went on to become an all-NFL selection five times in the 1960s. Packers coach Vince Lombardi remarked, "I was too stubborn to switch him to defense until I had to. Now when I think of what Adderley means to our defense, it scares me to think of how I almost mishandled him."
Adderley recorded 39 interceptions in his nine seasons with the Packers. He holds the Green Bay records for interceptions returned for touchdowns in a career (seven, record tied with Darren Sharper), and interceptions returned for touchdowns in one season (three, in 1965).
Adderley started for the Packers from 1961–69, then played three seasons (1970–72) with the Dallas Cowboys. While with the Packers, he won rings for five NFL championships and wins in the first two Super Bowls. Adderley was a factor in the Super Bowl II win over the Oakland Raiders, intercepting a pass by Raiders quarterback Daryle Lamonica in the fourth quarter and returning it 60 yards for a touchdown to put the game away. It was the first Super Bowl touchdown scored on an intercepted pass. After being traded to the Cowboys in 1970, Adderley became a vital cog in its "Doomsday Defense," assisting the Cowboys to a Super Bowl appearance in V and a win in VI.
Benched during the middle of the 1972 season, Adderley was traded to the Los Angeles Rams in the summer of 1973. He opted not to report and retired on August 7, after a dozen seasons in the NFL.
Along with two Packer teammates, offensive linemen Fuzzy Thurston (Colts) and Forrest Gregg (Cowboys), Adderley is one of only three players in pro football history to play on six world championship teams. However, in a revised edition of Instant Replay, a memoir by Packer teammate Jerry Kramer, Adderley is quoted as saying, "I'm the only man with a Dallas Cowboys Super Bowl ring who doesn't wear it. I'm a Green Bay Packer."
Adderley admired Packer head coach Vince Lombardi, but not Tom Landry of the Cowboys. His trade to Dallas in 1970, after a holdout and two weeks before the start of regular season, was due to a strained relationship with Lombardi's successor, Phil Bengtson, in his third and final year as Packer head coach. Adderley stated the Bengtson kept him off the Pro Bowl team in 1969 and requested to be traded. A year after his induction in Canton, Adderley became a member of the Packer Hall of Fame in 1981.
In his 12 NFL seasons, Adderley recorded 48 interceptions, which he returned for 1,046 yards and seven touchdowns, an average of 21.8 yards per return. He also recovered 14 fumbles (returning them for 65 yards) and returned 120 kickoffs for 3,080 yards and two scores.