Willis was born on April 12, 1876 in Cecil County, Maryland. He moved to Newark, Delaware as a young boy, where he attended school. He attended high school at Newark Academy, and played both on the high school baseball team and in semi-pro baseball leagues throughout Delaware. Prior to joining the major leagues, Willis played the 1897 season at the University of Delaware (then known as Delaware College), and later coached the 1907 team and parts of the 1908 team.
Over a 13-year career, Willis played for three teams, the Boston Beaneaters (1898–1905), Pittsburgh Pirates (1906–1909) and St. Louis Cardinals (1910), compiling a 249–205 record with a 2.63 ERA. He was known as a workhorse and completed 388 of his 471 starts. Willis also holds the post-1900 record for complete games (45, in 1902) in a single season.
Willis wearing a catcher's mask, circa 1900.
Despite being a Hall of Fame pitcher, Willis holds the post-1900 record for most losses in a single season (29, in 1905). For the three seasons from 1903 to 1905, Willis compiled a dismal record with the Boston Beaneaters of 42 wins against 72 losses. However, his ERA during those three years averaged 3.02 and in two of those years his ERA was under 3.00. Despite Willis' performance on the mound during those three seasons, the Boston offense could only muster a combined .238 batting average over those seasons. When he changed teams to the Pittsburgh Pirates for 1906, whose offense had a combined batting average of .256 over the four years Willis was with the team, Willis compiled a record of 88–46. His ERA for those four years was 2.08.