Caroline played college football at the Illinois, lettering in 1953 and 1954. A halfback, Caroline led the nation in rushing with 1256 yards in 1953 as a sophomore, and was named All-American. The next year he was ruled ineligible to play due to academic reasons, so instead of returning to college his senior year, Caroline chose to play Canadian football. He initially signed with the Toronto Argonauts for $15,000 and played with them through September. They then stunned the league when he was placed on waivers; head coach Bill Swiacki, claimed he wasn’t good enough as a defensive back. The only team that could afford him was the division champion Montreal Alouettes. Caroline he rushed for 575 yards on the season. He played in the 1955 Grey Cup game. Following that year, Caroline earned a physical education degree from Florida A&M University and was drafted by the Chicago Bears with the 82nd overall pick in the 1956 NFL Draft .
Beginning in 1956, Caroline had a ten-year career with the Bears. Playing primarily defensive back, Caroline finished his career with 24 interceptions and six total touchdowns—two rushing, one receiving, one fumble recovery, and two on interceptions. Caroline intercepted Johnny Unitas's first NFL career pass attempt and returned it for a touchdown.