Jourgensen has maintained that he was pressured by Arista management into producing the album in the then-popular "synthpop" style, which is in contrast to the harder industrial and heavy metal sounds he developed afterwards. However, there are reports of Jourgensen saying in the 1980s that when he discovered hardcore music, his musical direction changed. Additionally, video of local concerts Ministry performed in Chicago 1–2 years previous to their signing with Arista show the band dressed in "new wave" styles and playing new wave and synthpop music. Jourgensen assumes a false English accent for all of the songs, for which he also later expressed great disliking, though his ex-wife stated in a 2013 interview:
...the English accent thing was more an homage to the bands he loved than anything else. He was not trying to come off as British. The Stones used a southern accent and no one crawled up their ass for it.
With Sympathy was also released as Work for Love under BMG in Europe (with the same cover). There is a video available for "Revenge," which was one of three singles pulled from the album, the others being "Work for Love" and "I Wanted to Tell Her."
The album was long out of print and Jourgensen claimed for years that he had destroyed the master tapes. However, it was reissued in 2012 on Eastworld records with three bonus tracks. In 2014, Jourgensen himself posted a previously unreleased song recorded during the With Sympathy sessions to his website, allowing it to be downloaded for free.