In a 22-year career, Jones compiled a 229–217 record with 1223 strikeouts and a 3.84 ERA in 3,883 innings pitched. Jones signed his first professional contract in 1913, with a minor league team in Zanesville, Ohio. He made his major league debut with the Indians in 1914. Before the 1916 season, he was sent to Boston in the same trade that brought Tris Speaker to Cleveland.
In 1918, Jones joined the Red Sox starting rotation, ending with a 16–5 mark, a career-best 2.25 ERA, and a league-best .762 winning percentage. His most productive season came in 1921, when he posted career-highs in wins (23), strikeouts (98) and innings (298.2), and led the league in shutouts (5). But his most remembered season may have been 1923 as the ace of the Yankees' staff; he posted a 21–8 record with a 3.63 ERA and led his team to their first World Series title. Jones also no-hit the Philadelphia Athletics 2-0 on September 4 at Shibe Park, in a game in which he did not record a strikeout the entire game. As of 2014, Ken Holtzman is the only other no-hit pitcher not to strike out a batter the entire game, doing so for the Chicago Cubs in 1969. Jones was 2–1 against the New York Giants in that World Series, and his crucial relief work in the final game of the Series clinched the championship for the Yankees. Like most pitchers of his time, Jones relieved as well as started, and his eight saves in 1922 led the league's relief pitchers.
Jones lost a league-high 21 games in 1925. He pitched for the Browns a year later, and was waived to Washington in 1927. With the Senators, Jones regained his form, leading his team's staff with a 17–7 record. He enjoyed his last good season in 1930, ending with a 15–7 mark. After four years of service for the White Sox, Jones retired in 1935 as the oldest active player at the time (42). His 22 consecutive seasons pitching in one league is a major league record shared with Herb Pennock, Early Wynn, Red Ruffing and Steve Carlton.
Bill McGeehan of the New York Herald-Tribune dubbed him Sad Sam because, to him, Jones looked downcast on the field. Jones told Lawrence Ritter that the reason he looked downcast was because, "I would always wear my cap down real low over my eyes. And the sportswriters were more used to fellows like Waite Hoyt, who'd always wear their caps way up so they wouldn't miss any pretty girls". – Ed Walton, at Baseball Library