Caldwell had his best season in 1978 when he went 22–9 with a 2.36 ERA and led the AL in Complete Games with 23. Caldwell was named the AL Comeback Player of the Year by The Sporting News and finished second in the Cy Young Award balloting to Ron Guidry. Caldwell finished in double figures in victories for 6 consecutive seasons for the Brewers (1978–1983) and won 2 games in the 1982 World Series against the St. Louis Cardinals in a losing effort. Caldwell was given his unconditional release by the Brewers organization in 1985.
In 1978, he was one of the three left-handed pitchers named "Mike" (the others being Mike Flanagan and Mike Willis) to hand the New York Yankees' Ron Guidry a loss in his 25–3 season. He and the Brewers shut out the Yankees and Guidry 6–0 on July 7, 1978.