The song was little-known outside the indie-pop scene until Nirvana recorded the song in November 1993. The song released in the album and the DVD was titled "Jesus Doesn't Want Me for a Sunbeam" on their live acoustic MTV Unplugged in New York. The Vaselines re-released the song in 1992 on the compilation albums The Way of The Vaselines: A Complete History and All the Stuff and More. On All the Stuff and More, the title was changed slightly to "Jesus Doesn't Want Me for a Sunbeam".
Two more versions of this song were released on the 2004 box setWith the Lights Out. This included an acoustic version recorded in Europe in 1994, along with a live electric performance on the DVD section of the box-set. Another version was released on 2011 as a bonus track on the 20th anniversary edition of Nevermind.
On the version contained on the MTV Unplugged in New York album, Kurt Cobain refers to the song as: "a rendition of an old Christian song, I think. But we do it the Vaselines' way."