Washington was drafted fourth overall in the first round of the 1976 NFL Draft by the Chargers. A month prior to the start of the 1978 season, he was traded to Baltimore for Lydell Mitchell. Washington was with the Colts when he made his lone Pro Bowl appearance in 1979. He led the NFL that season with 82 receptions for 750 yards, and had 242 carries for 884 yards and seven touchdowns.
His most memorable performance was on September 18, 1978 in Baltimore's 34–27 victory over the New England Patriots on Monday Night Football, when he had a hand in three of the four touchdowns scored by the Colts in a fourth quarter in which both teams combined for 41 points. He helped lead off the scoring by throwing a 54-yard option pass to Roger Carr. Washington followed that up with a 23-yard touchdown catch from Bill Troup. Washington broke a 27–27 deadlock by scoring the gamewinner on a 90-yard kickoff return in a driving rainstorm. It was scored in the final seconds of the game after the Patriots came back from 27-13.