Scully attended the University of Notre Dame, where he played for the Notre Dame Fighting Irish football team from 1977 to 1980. As a senior in 1980, he was a team captain and was recognized as a consensus first-team All-American as the team's center. He is the writer of the popular Notre Dame anthem "Here Come the Irish."
The Atlanta Falcons chose Scully in the fourth round (109th pick overall) of the 1981 NFL Draft, and he played for the Falcons from 1981 to 1990. In his ten-year NFL career, he played in 112 games and started eighty-two of them.