Beginning in 1960, he joined the Boston Patriots as a coach of the team's defensive linemen and linebackers. Two years later, he took a similar role with the Buffalo Bills; in his six years as a Bills coach, the team won two American Football League championships. In 1968, the Cleveland Browns hired Smith as an assistant personnel director. From 1969 to 1970, Smith coached in the New Orleans Saints organization. The following year, he became the Denver Broncos' offensive line coach. On November 17, 1971, Broncos head coach Lou Saban, who had also been Smith's boss in Boston and Buffalo, resigned and Smith was named his replacement for the season's last five games. The Broncos posted a 2–3 record under Smith. Following the 1971 season, he became the Houston Oilers' defensive line coach for 1972; after one season, he returned to the Browns and served multiple roles.