Anderson's 2005 season ended prematurely when he tore an elbow ligament, necessitating Tommy John surgery. He attempted a comeback in 2006 with the Texas Rangers, but re-injured it during his rehab program. He had to undergo a second Tommy John surgery. During his convalescence in 2007, Anderson was a fill-in broadcaster for the Cleveland Indians.
On February 1, 2008, the Tampa Bay Rays signed Anderson to a minor league contract with an invitation to spring training. During spring training however, Anderson left the mound in the middle of a game, and followed that with an MRI. The MRI revealed he had a torn ulnar collateral ligament (for the third time), as well as a torn flexor mass muscle, both in his left elbow. Rays manager Joe Maddon commented by saying, "It can't be repaired; he's done. It's really a big disappointment."
Following the second Tommy John surgery, Anderson was out of baseball for the 2007 season, during which he occasionally filled in as a broadcaster for the Cleveland Indians on SportsTime Ohio, as well as doing several spring training games and a weekly highlight show.
In 2009 and 2010, Anderson again worked as a part-time TV analyst for the Rays, calling about 50 games for which Magrane's successor, Kevin Kennedy, was unavailable. In October 2010, the Rays announced that Anderson would become the team's full-time TV analyst beginning in 2011.