Ayers began his coaching career at Miami University (Ohio) in 1978, before moving on to West Point and Ohio State. His first head coaching position came at Ohio State in 1989, taking them to the number one regional seed in the NCAA men's tournament in both 1991 and 1992, but never reaching the Final Four. He was named National Coach of the Year in 1991, and Big Ten Coach of the Year in 1991 and 1992.
Ayers was named head coach of the Philadelphia 76ers on June 20, 2003, after serving as an assistant coach with the team since 1997. He compiled a 21–31 record, but was fired in the middle of the season. He then served as an assistant coach for the Orlando Magic. On July 24, 2007, Randy Ayers was hired as an assistant coach for the Washington Wizards. On August 5, 2009, Ayers returned to the Philadelphia 76ers to serve as an assistant coach under Eddie Jordan. Ayers then joined the coaching staff of the New Orleans Hornets (now Pelicans) in 2010.
Ayers married Carol Denise Peery of Columbus, Ohio on June 18, 1983. She is a high school Spanish teacher. They have two sons. Ryan Vincent (born July 16, 1986) was a student at the University of Notre Dame, a member of the Fighting Irish basketball team), and now plays professional basketball in France. Cameron Alexander (born September 18, 1991) is currently a basketball player at Bucknell University. Ayers's youngest brother, Tim Ayers, served as Mayor and City Commissioner of Springfield, Ohio from 1984 to 1990.