Perverted by Language is the first Fall album to feature Brix Smith, then-wife of Mark E. Smith, who performs lead vocals on "Hotel Blöedel". However, the bulk of the album was recorded before she joined the band.
The album contains the track, "Eat Y'self Fitter", which John Peel picked as one of his Desert Island Discs; when Peel had first heard the track – in a session the band recorded in March 1983 – he stated on air that he had fainted and his producer, John Walters, had had to resuscitate him.
The album was the only full-length product of the band's renewed relationship with Rough Trade, whom they had previously left in 1981. This time, they fell out over the full-length video the band wished to make for the album. The video Perverted by Language Bis went ahead funded by the group themselves and producers Ikon, who released the finished product on VHS in 1984 (a DVD edition was released by Cherry Red in 2003). By that time, the group had signed to Beggars Banquet.
There have been five CD editions of the album. The first three (Line's 1984, Castle's 1993 and Cog Sinister's 1998) duplicated the original vinyl. The album was reissued through Castle Music in 1998 in a slightly different mix (only "Garden" was noticeably altered) and adding five bonus tracks.
The 2005 edition, again on Castle Music, carried the original mix of the album, all the bonus tracks from the 1998 version and came with an additional disc carrying live recordings from the era and a Peel Session, as well as the remixed "Garden" from the previous edition.