Originally designed as a studio unit to showcase Hawkinson's songwriting, the band released Pleasures of Schizophrenia in October 2005. Steven Hyden of The Post Crescent wrote of the eccentric debut "the music straddles the line between melody and abrasiveness". The core group on Pleasures consisted of Marc Golde on Hammond B3, Greg Thiel on drums and percussion, and Hawkinson playing guitar and bass on many of the tracks as well as providing all the vocals. Several Wisconsin musicians made guest appearances on the recordings, including bassist Jason Brown (formerly of Hank Williams III), saxophonist Steve Cooper, drummer Eric Harkoff, and Danny Jerabek on accordion. Pleasures of Schizoprenia combined many different genres of music, from rock ("Divided by Zero") to funk ("Celebrate Yourself") to Tom Waits-style croonings ("No One Will Find Us", "Valentine Jesus") to gothic stylings that would be further shaped into a more consistent sound in the band's later recordings ("Vardoulacha", "Tangier").
The second CD release, Memoirs of Don Juan Gacy, was released in January 2008. The album again included the collaboration of Hawkinson, Golde, and Thiel, with Ken Rodgers joining the fold on bass guitar. Memoirs is notable for the inclusion of a cover of Peter Gabriel's "Games Without Frontiers", which managed to catch the attention of Italian author and Dusk (Genesis fanzine) contributor Mario Giametti. Also notable is "Tangier, Part II", a continuation (or perhaps reinvention) of the William S. Burroughs-influenced track on the previous album, and "Charenton", with a guest appearance on piano by jazz vocalist and pianist Helen Exner. Memoirs has received positive reviews, including a 9 out of 10 rating from Madison-based Maximum Ink Magazine. A video was released for the album's third track "Heaven Without You".
On June 17, 2017 it was announced on the band's Facebook page that Vulgar Pang will reform for a live television event at Rock Garden Studios in Appleton, WI for the Rock Garden Live series produced by Marc Golde. Joining Craig Hawkinson will be Chris Larson, Brad Warning, Kevin Junemann, and Jonathan Burke.
- Craig Hawkinson - vocals, synthesizers
- Chris Larson - guitar
- Brad Warning - guitar
- Kevin Junemann - bass guitar
- Jonathan Burke - drums
- Pleasures of Schizophrenia (2005)
- Memoirs of Don Juan Gacy (2008)
Both albums were recorded at Rock Garden Studios in Appleton, WI and engineered by Marc Golde.