Ivey was reared in Calumet, a small mining town in Michigan's Upper Peninsula. He graduated from the University of Michiganin 1966, received a master's degree in folklore and ethnomusicology from Indiana University in 1970, and became a PhD candidate in folklore and history in 1971. He was the first full-time director of the Country Music Foundation and the related Country Music Hall of Fame and Museum, serving from 1971 to 1998..
Following government service Ivey founded the Curb Center for Art, Enterprise and Public Policy, at Vanderbilt University, serving as director from 2002 to 2012. He returned to Washington in 2007 as Team Leader in Arts and Humanities for the Barack Obama presidential transition.
Ivey has written and lectured extensively about the importance of cultural policy and the value of cultural engagement in the pursuit of a high quality of life. He coined the phrase "Expressive Life" to define the part of the human experience shaped by cultural heritage and creative practice.