Parker played his first three years of high school football in Macon, Georgia, but then moved north to play his senior season for Scott High School in Toledo, Ohio, graduating in 1953. He was recruited to play as a guard for the Ohio State Buckeyes from 1954 to 1956, playing on both the offensive and defensive lines. Parker was known for his size, strength and quickness, and these talents, used for clearing a path for running backs, helped demonstrate the feasibility of head coach Woody Hayes' three-yards-and-a-cloud-of-dust offense—an offensive philosophy that had been questioned as suitable for big-time college football. Due in part to Parker, Hayes won his first national championship in 1954.
Parker was selected by the Baltimore Colts in the first round of the 1957 NFL Draft as the eighth player selected overall. The Colts, with quarterbackJohnny Unitas, relied on a passing offense very different from the running offense of Ohio State. Nevertheless, Parker soon came to be known as the premier pass blocker in the game.
From 1957 until 1962, Parker played as an offensive tackle. He was selected to five Pro Bowl teams in those six years. In 1963 Parker moved to the offensive guard position, as a favor to his college coach Woody Hayes, to make room for another former Buckeye, Bob Vogel. Parker was selected to three more Pro Bowls from the guard position.
Many consider Parker to be the greatest lineman to ever play pro football. Parker was inducted into the Pro Football Hall of Fame in 1973, his first year of eligibility. He was the first full-time offensive lineman so inducted. In 1994, Parker was selected to the NFL 75th Anniversary All-Time Team. In 1999, he was ranked number 24 on The Sporting News' list of the 100 Greatest Football Players, second among guards behind John Hannah, and third among offensive linemen behind Hannah and Anthony Munoz, both of whom began their careers well after Parker retired.
For his 1984 book, The New Thinking Man's Guide to Pro Football,Paul Zimmerman "polled 25 old-timers and asked them to name their candidates for 'best ever' offensive linemen," and Parker was among those listed, stating that Parker was "the best pure pass-blocker who ever lived. Knew all the tricks — the quick push-off, the short jab — that are legal now."