Houston played for the Ohio State Buckeyes as an end. He was a three-year starter under head coach Woody Hayes and twice the team MVP. He was elected the team captain as a senior.
Houston contributed on both offense and defense. Although known primarily for his excellent blocking and tackling, he was also the leading receiver on the 1959 team, including a 100-yard game that year against Michigan State. He was an All-America selection in 1958 and 1959.
Houston was chosen in the first round of both the AFL draft by the Buffalo Bills and the NFL draft by the Cleveland Browns. Like his older brother Lin Houston, Jim chose the Browns. He originally played as a defensive end under head coach Paul Brown, but later moved to linebacker under Blanton Collier. At linebacker he became a four-time Pro Bowl selection. In the 1969 NFL Championship Game against the Minnesota Vikings, Jim went to tackle Viking quarterback Joe Kapp low and Kapp tried to hurdle him. Kapp's knee hit Jim in the side of the head, knocking him out and breaking his jaw.