O'Rourke was born in East Bridgeport, Connecticut, and worked on his family's farm while playing youth league and semi-pro baseball. He began his professional career as a member of the Middletown Mansfields in 1872, joining the one-year-old National Association team as a catcher. The Mansfields were not a top-tier team, and folded in August, but O'Rourke had impressed other teams sufficiently enough to be offered a contract with the Boston Red Stockings, with whom he played until 1878. On April 22, 1876, O'Rourke had the first base hit in National League history.
He graduated from Yale Law School in 1887 with an LL.B., practicing law in Bridgeport between early playing stints, and earning the nickname "Orator Jim" because of his verbosity on the field, his intellect, and his law degree—uncommon in a game regarded as a rough immigrant sport at the time.
After leaving the major leagues following the 1893 season he continued to play in the minor leagues until he was over 50 years old. As an executive of the Bridgeport team in the Connecticut League, in 1895 O'Rourke hired the first African American minor league baseball player in history.
In 1904 he made a final appearance with the New York Giants under manager and friend John McGraw, becoming at age 54 the oldest player ever to appear in the National League, and the oldest player to hit safely in a major league game. O'Rourke is one of only 29 players in baseball history to appear in Major League games in four decades.
In 1912 returned to the field to catch a complete minor league game at the age of 60.
One legend concerning O'Rourke is that he was asked to drop the "O'" from his last name when he signed a contract with Boston and its Protestant backers. The son of irish immigrants and the husband of a woman born in Ireland, O'Rourke refused, saying "I would rather die than give up my father's name. A million dollars would not tempt me."
Another legend about O'Rourke is that his signing by the Mansfields in 1872 was conditioned on the team finding someone to take over O'Rourke's chores on his parents' farm.
"O'Rourke has made a brilliant record for himself as an outfielder, being an excellent judge of a ball, a swift runner, and making the most difficult running catches with the utmost ease and certainty. As a thrower, too, he stands pre-eminent, being credited with a throw of 365 feet, the next to the longest yet accomplished by any player."