In an interview for VH1's "Heavy: The Story of Metal," Dio stated that the song is about the ability of each human being to choose between doing good and doing evil, essentially that each person has "heaven and hell" inside themselves. When the song has been performed live by Black Sabbath, Dio or Heaven & Hell, it's usually stretched out with an extended guitar solo, audience participation, ad-libbed lyrics, or additional lyrics regarding angelic and demonic apparitions and personal judgment.
Many later Black Sabbath lineups included this song in live sets, during which it was sung by future Sabbath frontmen Ian Gillan, Glenn Hughes, Ray Gillen, and Tony Martin. An official live recording of the song featuring Tony Martin on vocals was released on the 1995 live album Cross Purposes Live. Judas Priest frontman Rob Halford also sang the song with Black Sabbath on 14 & 15 November 1992, when he filled in on vocals for two concerts.
Keyboardist Geoff Nicholls, who was a session keyboard player during the recording of this album, is said to have been responsible for this song's famous bass line. The bassline closely resembles the bassline of "Mainline Riders" by Quartz, of which Nicholls was a former member. Geezer Butler was not available during initial recording of the song in late 1979. 
"Heaven and Hell" was ranked No. 11 in Martin Popoff's book The Top 500 Heavy Metal Songs of All Time. Popoff put together this book by requesting thousands of metal fans, musicians, and journalists to send in their favorite heavy metal songs. Almost 18,000 individual votes were tallied and entered into a database from which the final rankings were derived.
The song was named the 81st best hard rock song of all time by VH1.
The song was also briefly covered by Tenacious D for Dio's music video for the song "Push". At the beginning of the music video, Jack Black and Kyle Gass are shown standing on a sidewalk busking and singing "Heaven and Hell" with alternate lyrics. They stop when Dio walks up to them and tells them that he will pay them to play some Tenacious D songs.