Graham, a native of El Dorado, Kansas, played football at Kansas State University under head coach Bo McMillin. Known as "Rammin' Ralph", he was a first team All-Big Six Conference fullback each year from 1931 to 1933. He nearly led K-State to the Rose Bowl in 1931 before getting injured. Following his senior season, Graham was a starter in the East–West Shrine Game, becoming the first Kansas State football player to be named to an all-star game. Graham graduated as the all-time leading scorer at Kansas State, and held that record for 64 years until it was surpassed by kicker Martín Gramática in 1997.
Upon graduation in 1934, Graham entered the football coaching profession as an assistant to McMillin, who moved from Kansas State to Indiana University that season. Graham remained an assistant football and basketball coach at Indiana until 1942, where he also earned a master's degree and was head coach of the tennis team. During this time, Graham was the top assistant for Branch McCracken's 1940 national championship basketball team.
Graham returned to Kansas in 1942 to serve as athletics director and head football coach at the University of Wichita, now Wichita State University. During his first season as head coach, Graham led Wichita to a 5–4 record, including a win over his alma mater. After taking the next three seasons off to serve in World War II, Graham returned to Wichita to coach two more successful seasons in 1946 and 1947. Following the 1947 season he led Wichita to the Raisin Bowl, played on January 1, 1948.
Graham was the 18th head coach for the Shockers. His overall coaching record at Wichita was 17 wins and 13 losses. This ranks him fifth at Wichita in terms of total wins and 11th at the school in terms of winning percentage.
In the wake of his success at Wichita, Graham was hired as the 22nd head football coach at Kansas State in 1948. At the time he took the job, K-State was mired in a 26-game losing streak, the longest in NCAA history to that point. Graham managed to break the streak with a win in his third game. (The full 28-game streak is still the second longest in NCAA FBS history.) In his second season, Graham started 2–0 and Kansas State beat Colorado in the second to snap a 22-game conference losing streak. However, the season fell apart after that point and the team finished 2–8. After coaching one more season, and posting a 1–9 record, Graham resigned from Kansas State following the 1950 season with a 4–26–1 mark during his tenure. Notably, the situation Graham inherited at Kansas State was so bad that this record represented an improvement over the prior three seasons. During his years at Kansas State, Graham also coached the school's tennis team.
Following his years at Kansas State, Graham left the field of athletics.
Graham was a pioneer in integrating collegiate athletics in the region. While at Kansas State he supported African AmericanHarold Robinson in his football career and awarded him an athletic scholarship. Robinson broke the decades-long "color barrier" in the Big Seven Conference playing for Graham in 1949. Robinson later compared Graham to Branch Rickey, the general manager of the Brooklyn Dodgers who hired Jackie Robinson to integrate baseball in 1947. Integration was also a priority at Wichita, where one of Graham's best players was also African American—running back Linwood Sexton.