Murphy was born in Southborough, Massachusetts. He attended Yale University, which is how he got his nickname, and played for the National League's New York Giants in 1894, 1895, and 1897. Also called "Tot" or "Midget", Murphy was small even for his era. He was 5 ft 3 in (1.60 m) tall and weighed 125 lbs. He and Dickey Pearce are the two shortest players ever to have a career in Major League Baseball. In 1894, there was a book written about him that was entitled Yale Murphy, the Great Short-Stop, or, The Little Midget of the Giants.
Murphy started his professional baseball career in 1894. That season, he was a backup shortstop and outfielder, playing in a career-high 75 games, batting .272, and stealing 28 bases. In 1895, he played mostly in the outfield. He hit just .201 and did not play for the Giants in 1896. He returned for a few games in 1897 and then played one season (1900) in the New York State League.
Murphy batted .240 in 131 career games. After his baseball days were over, he became a physician.