Born in Lenoir, North Carolina, Allen spent part of his youth in the Baptist orphanage in Thomasville, North Carolina, and he attended Thomasville High School. Allen reached the Yankees in an unusual way. While working as a bellhop in a hotel, he was told to take some fans to the room of Yankee scout Paul Krichell. Allen told Krichell that he was a pitcher, and the scout arranged a tryout. Allen was an immediate success for the Yankees, debuting in 1932 with a 17–4 record and a 3.70 ERA for the world champions. He was less stellar in that year's World Series, starting Game 4 and leaving after giving up three runs off five hits in just two-thirds of an inning. He continued to post decent records for the Yankees, but a sore arm and his constant demands for more money threatened his career. For these reasons, he was dealt to the Indians before the 1936 season.
Allen turned things around in Cleveland, going 20–10 with a 3.44 ERA in 1936 and following that up by winning his first fifteen decisions of 1937, one short of the record held by Walter Johnson. He lost his next start 1–0 on an unearned run, but his 15–1 mark that year set a winning percentage record that lasted until Roy Face bettered it with an 18–1 record in 1959. In 1938, he won his first twelve decisions and made his only All-Star team. During the All-Star break, he suffered an unknown injury, some claim he slipped on a bar of soap in the shower, and never did approach his earlier success again, finally retiring in 1944 after six mediocre campaigns. He became a minor league umpire after retiring, eventually becoming the umpire-in-chief of the Carolina League. He died at age 54 in St. Petersburg, Florida.