Scott and Russell had been introduced to each other by Johnny Mercer, at a California nightclub. Although Russell was dying of lymphoma and the pair met in person only three times, they managed to collaborate on the song.
In 1884, James Wells, Moderator of the United Free Church of Scotland, in his book The Parables of Jesus tells the story of a little girl carrying a big baby boy. Seeing her struggling, someone asked if she wasn't tired. With surprise she replied, "No, he's not heavy; he's my brother."
In a 1918 publication by Ralph Waldo Trine titled The Higher Powers of Mind and Spirit, he relates the following anecdote: "Do you know that incident in connection with the little Scottish girl? She was trudging along, carrying as best she could a boy younger, but it seemed almost as big as she herself, when one remarked to her how heavy he must be for her to carry, when instantly came the reply: 'He's na heavy. He's mi brither.'"
The first editor of Kiwanis magazine, Roe Fulkerson, published a column in September 1924 carrying the title "He Ain't Heavy, He's My Brother", the first use of the phrase exactly as it is rendered in the song title.
The Hollies' recording, which featured Elton John on piano, was released in the UK on 1 September 1969 and on 1 December 1969 in the US. "He Ain't Heavy" reached No. 3 in the UK and No. 7 in the US. The song, paired with "Carrie", was re-released in late 1988 in the UK following its use in a television advertisement for Miller Lite beer. It reached the No. 1 spot in the UK chart for two weeks in September 1988.
The Neil Diamond version entered at #68 on the Hot 100 on 7 November 1970  (UNI Records, 55264, length 4:09). The flip side was "Free Life". The song appears on the Neil Diamond album Tap Root Manuscript, which was released November 1970. The song was played by KGB-AM radio, San Diego, California, in late 1970, prior to the then-new Walk for Mankind, in dedication to those who would be walking for donations that day.
Initially, the idea was to re-release the 2009 single "The Fields of Anfield Road" by The Liverpool Collective featuring The Kop Choir; however, this idea was rejected by Peter Hooton as only a relatively small number of people would buy it. Inspired by Everton's Hillsborough tribute on 17 September 2012, the song was played at Goodison Park prior to their match against Newcastle United. It was then decided that a re-recording of this song by various artists including The Justice Tonight Band would be released as the charity single.