Whitfield started for the New York Giants at left tackle during the 2006 season filling in after starter Luke Petitgout went down with injury. His time as a starter was marked by two episodes of him losing his cool and drawing costly personal fouls, twice on headbutts in key spots against the Dallas Cowboys and later in a week 16 loss against the New Orleans Saints. These actions lead to him receiving the nickname "Head-butt Bob" from many Giant fans as well as local columnists. Due to these actions and general inconsistent play, Whitfield was benched for the final game of the season against the Washington Redskins and the subsequent playoff game against the Philadelphia Eagles.
On February 12, 2007, he announced his retirement on Sirius NFL radio.
Bob also serves as a guest analyst for the UK's NFL Coverage on Sky Sports as well as contributing on local Atlanta television and radio.
On November 11, 2011, Whitfield was inducted into Stanford University's Hall of Fame and in an interview, Whitfield admitted he was considering a return to his alma mater to complete his degree. After 16 years playing professional football and spending 4 years enjoying retired life, Bob returned to Stanford in 2012 and received a degree in economics from Stanford University. Whitfield received his diploma during a commencement ceremony held at Stanford University on June 16. 2013 (Father's Day) with his kids by his side cheering him on.