Davis began his career in 1980 with the Philadelphia Phillies, and spent parts of five seasons with the San Francisco Giants. He started a career-high 27 games in 1984 for a 5–17 win–loss record. He became a primary reliever the following season, but he did not establish himself as a top reliever until being traded to San Diego during the 1987 midseason.
In 1988, Davis became the Padres' closer, earning 28 saves and 44 in 1989, appearing in the All-Star Game in both seasons. He was a Cy Young Award winner in 1989 after a 1.85 ERA and 65 games finished in addition to his league-leading save total. In the last month of the season, he pitched 25 innings and did not allow any of the 19 runners he inherited to score. He was the fourth closer so honored in Award history. No closer would win the NL Cy Young again until Éric Gagné in 2003.
Davis signed as a free agent for $10 million with the Kansas City Royals before the 1990 season. He pitched ineffectively early in the season and subsequently lost the closer role to Jeff Montgomery. After a brief stint for the Atlanta Braves in 1992, again he pitched with the Phillies and Padres from 1993–1994. After going out for two years, he returned to pitch until retiring with the Milwaukee Brewers in 1997. He never came close to matching his accomplishments for San Diego, earning only eleven saves over the last eight seasons of his career.
In a 15-season career, Davis posted a 51–84 record with a 4.17 ERA and 96 saves in 624 games pitched.