Downing's stay in Oakland was short lived, as he was dealt to the Milwaukee Brewers on June 11, 1970 with Tito Francona for Steve Hovley. Despite a respectable 3.34 ERA, Downing's record was 2-10 for a Brewers team that narrowly escaped losing 100 games (97). For the season, Downing went 5-13 with a 3.52 ERA and 79 strikeouts in 27 games and 22 starts between his two teams.
After giving up Hank Aaron's famous 715th home run on April 8, 1974, Downing made his third, and final post-season appearance that season. His Dodgers lost four games to one to the Oakland A's. Downing played two more full seasons with the Dodgers, and was released during the 1977 season with a 0-1 record and 6.75 ERA.
Downing served as a color analyst on Dodgers cable-TV broadcasts from 1980–87 and on Dodgers radio in 2005. He also broadcast for CBS Radio in the 1990s, and the Atlanta Braves in 2000. As of 2006, he remains on the Dodgers Speaker's Bureau.