Rumph attended Atlantic Community High School in Delray Beach, Florida. He was among the SuperPrep National Top 50 players and was a SuperPrep All-American. As a senior, he gained 519 yards rushing and receiving with five touchdowns and totaled 108 tackles and snagged six interceptions for 172 yards.
Rumph was a quiet but productive player during his career at the University of Miami which culminated with a National Championship in his senior year. Rumph compiled 117 tackles (117 solo), 2 forced fumbles and 6 interceptions (returning one for a touchdown) during his time at Miami. He was a second team Big East selection his sophomore and senior years and a first team selection his junior year.
Rumph was drafted by the San Francisco 49ers in the 1st round of the 2002 NFL Draft, with the 27th overall selection. He was originally a cornerback, but the 49ers switched him to safety due to a combination of his strong tackling ability and poor man-to-man coverage skills. He missed large parts of the 2004 and 2005 seasons due to injuries.