The original version, recorded by Dion, featured a gentle folk rock production from Phil Gernhard and arrangement from John Abbott. The feeling of the song is set with a gentle oboe and violin opening then featuring harp flourishes at multiple points, including the instrumental conclusion. The song also features a flugelhorn, an electric organ, bass, and drums. Dion felt during post production that the song needed more depth and added a track featuring him playing classical guitar notably at the bridge, lead ins and the close.
Quite unlike the ethnic rock sound that Dion had become famous for in the early 1960s, and even more unlike Holler and Gernhard's previous collaboration the 1966 novelty smash "Snoopy Vs. The Red Baron", "Abraham, Martin and John" nonetheless was a major American hit single in late 1968, reaching #4 on the U.S. pop singles chart, being awarded an RIAAgold record for selling a million copies. In Canada, it topped the charts, reaching #1 in the RPM 100 on November 25, 1968. In 2001 this recording would be ranked number 248 on the RIAA's Songs of the Century list. The record was also popular with adult listeners, reaching #8 on Billboard's Easy Listening survey. the personnel on the original recording included Vinnie Bell and Ralph Casale on guitar, Nick DeCaro on organ and David Robinson on drums
In April 1969 Andy Williams, a close friend of Robert Kennedy, recorded a version on his album Happy Heart, and sang the song on his show over a year after Robert Kennedy's death. Other famous late-1960s versions were recorded by Motown's Smokey Robinson & the Miracles (whose cover also became an American Top 40 single in 1969, reaching #33) and Marvin Gaye (whose cover became a top-ten hit (#9) in the United Kingdom in 1970). Gaye's version was never released in the U.S. as a single but was featured on his 1970 album, That's the Way Love Is, and was one of his first experiments with social messages in his music which would culminate in his legendary 1971 album, What's Going On.
During a 1981 tour, Bob Dylan sang the song in concert.
Subsequently, various artists have performed or recorded their renditions of the song, including the likes of Bon Jovi and Emmylou Harris, who performed it as part of a medley with the Nanci Griffith song "It's a Hard Life Wherever You Go" on her 1992 At the Ryman concert recording. Marillion have played the song in acoustic and electric versions; one such hybrid performance can be heard on their 1999 Unplugged at the Walls album.
Tori Amos performed the song at four of her concerts during the On Scarlet's Walk Tour in 2003, including Hamburg, Germany on 23 January 2003.
Paul Weller recorded an acoustic version of the song during the sessions for his 1992 record Paul Weller. It went unreleased until a deluxe edition of the record was issued in 2009.
The song is also featured on Tom Clay's 1971 "What the World Needs Now Is Love/Abraham, Martin, and John", a medley combining Dion's recording with Jackie DeShannon's recording of Burt Bacharach's "What the World Needs Now Is Love", along with vocals by The Blackberries. Clay's recording features narration (an adult asking a child to define several words associated with social unrest), sound bites from speeches given by President John F. Kennedy, Robert Kennedy, and Martin Luther King Jr., along with sound bites from the live press coverage of Robert Kennedy's assassination, and his eulogy by his brother Edward M. Kennedy. It reached No. 8 on the Billboard Hot 100 chart on August 14, 1971. In 1997, Whitney Houston sang a rendition of "Abraham, Martin and John" that aired on VH1 and HBO: Whitney Houston: Live Washington DC.