Silver, who is Jewish, was also a founding member, along with Steele, of the short-lived rock group Fallout, the band that gave birth to Carnivore, which ultimately led to Type O Negative. After the breakup of Fallout, Silver formed hard rock band Original Sin.
Silver was first introduced to professional recording and production upon meeting and working with producer and Soundscape Recording Studio owner Richard Termini. Fallout’s earliest recordings trace back to the Termini Soundscape sessions from the late 1970s.
After the death of "TON" frontman Peter Steele, Silver became a certified EMT in New York, as stated by Silver, for over a year this had been his "plan B" to support his wife and kids. Assuming at the time that the band was finished, Silver had began EMT schooling during Steele's 2008 hiatus. Steele was even quoted as saying, "I wouldn't wait on me either- I don't blame Josh one bit!" After the death of Steele, Silver went right to work as an EMT, a position that he still holds.