In 1996, Heacock joined Ohio State University football coaching staff as the defensive line coach. When head coach John Cooper was fired in 2001, Heacock was one of only three assistants retained by the new head coach, Jim Tressel. As of 2008, Heacock was the most senior member of the Ohio State coaching staff. Heacock was promoted to defensive coordinator in 2005. In his first year in that position, the Ohio State defense was ranked first in the nation in rush defense. The same year, the defense ranked fifth in the nation for least number of points allowed and for total defense. According to sportswriter Dennis Dodd, "Statistically, the 2007 unit was among the best finishing first nationally in scoring defense, total defense and pass defense".
Heacock was awarded the Frank Broyles National Assistant Coach of the Year award in 2007. Tressel remarked that ""Jim Heacock's defense has allowed this young Ohio State team to become a national contender". Heacock is more modest, claiming that "We're all just in this for the same reason. ... There are other assistants who do every bit as much as I do. I kind of get in the way."