Duffy, born in Cranston, Rhode Island, was a textile mill worker who had taken up baseball as a semipro for weekend diversion. He played a couple years of minor league ball in the New England League before jumping to the majors, starting up in the league's initial season of 1886, and playing on clubs in Hartford, Springfield and Salem, as well as the Lowell Massachusetts team in 1887.
Playing in Boston from 1891 through 1900, Duffy knocked in 100 runs or more eight times. In 1894 Duffy had one of the greatest seasons in baseball history, leading the league with 18 home runs, with 145 RBI and a .440 batting average (see Major League Baseball Triple Crown). Duffy's .440 average is the Major League single-season batting average record. He played with two other Hall of Fame outfielders during his career, Tommy McCarthy (as half of the "Heavenly Twins") and Billy Hamilton. Duffy finished his career in 1906 with 106 home runs which was, at the time, one of the highest career totals.
During the 1902 and 1903 seasons, Duffy was player-manager for the Western League's Milwaukee franchise and the following season was hired on as player-manager for the Phillies. Duffy went on to coach the Harvard varsity and freshman baseball squads from 1917 through 1919. He also managed the 1920 Toronto Maple Leafs to a .701 winning percentage — the best in the team's 83-year history, but only good enough for second place in the International League. In 1921, Duffy was hired as full-time manager of the Red Sox, guiding them for two seasons. He had previously been player-manager for the Milwaukee Brewers in 1901, the Phillies from 1904 to 1906 and the White Sox from 1910 to 1911.