Kremer had a remarkable beginning to his big-league career. He posted an 18-10 record as a rookie in 1924, then followed that with seasons of 17-8, 20-6 and 19-8. He was third in the vote for the National League's most valuable player in 1926.
The Pirates won a pair of pennants during that stretch. Kremer threw two complete games in the 1925 World Series against the Washington Senators, and after winning Game 6 with a six-hitter, Kremer was brought back for four innings of relief in Game 7 and ended up the winning pitcher in that game as well. He also made one start in the 1927 World Series against a New York Yankees team thought by many to be the greatest baseball team of all time.
Kremer put up some of the most impressive numbers of his career in 1930, leading the league in wins (20), games started (38) and innings pitched (276).
For his career, he compiled an 143–85 record in 308 appearances, with a 3.76 ERA and 516 strikeouts. Kremer's 143 wins with Pittsburgh rank him eighth in franchise history, his .627 winning percentage ranks seventh, and his 1,9542⁄3innings pitched rank tenth.