Simon liked reggae, he listened to Jimmy Cliff, Desmond Dekker, and Byron Lee; he wanted to go to Kingston, Jamaica to record the song, as that was where Cliff had recorded his antiwar song "Vietnam" in 1970. The title has its origin in a chicken-and-egg dish called "Mother and Child Reunion" that Simon saw on a Chinese restaurant's menu. The song's lyrics were inspired by a pet dog that was run over and killed. It was the first death Simon personally experienced, and he began to wonder how he would react if the same happened to his wife, Peggy Harper. "Somehow there was a connection between this death and Peggy and it was like Heaven, I don't know what the connection was," Simon told Rolling Stone in 1972.
The song was recorded at Dynamic Sounds Studios at Torrington Bridge in Kingston, Jamaica, with Jimmy Cliff's backing group. Guitarist Huks ("Hux") Brown and bass guitarist Jackie Jackson were also long-time members of Toots & the Maytals. Cissy Houston sang background vocals on the recording. The song was recorded before writing lyrics, which was unusual for Simon. He had previously hoped to make "Why Don't You Write Me"— a song recorded with Art Garfunkel on Bridge over Troubled Water—to sound like a Jamaican song, but felt it ended up sounding like a "bad imitation." Simon was instructed by the musicians on the differences among reggae, ska, and bluebeat. He felt awkward at first because he was "the only white guy there and I was American." Simon later overdubbed piano and vocals to the track back in New York.