In an early interview with Billy Corgan, he admitted that he had thought of the name for the song before he had written it, had labeled dozens of tapes with its name, and even considered naming the band "Siva" instead of "Smashing Pumpkins". Corgan has stated the riff was conceived on an acoustic guitar while working at a record store in Chicago. The song was one of three played at the band's 1991 session with John Peel, released on Peel Sessions.
Corgan originally titled the song "Shiva", referring to the Tantric concepts of Shiva and Shakti as opposing masculine and feminine forces, ignorant of any further implications of the name. Upon realizing that the name was more readily connected with the Hindu god Shiva, he removed the letter "h" from the title to lessen this association.
The song was only released as a single in the UK and Australia, as well as appearing on the Peel Sessions EP. On 29 January 2008, it was also made available as a downloadable track for the video game Rock Band.