The album consists of songs (or in some cases, different versions of songs) that were not released on the band's studio albums. The original vinyl version had tracks 16 and 17 on an extra flexi disc. The album was certified gold by both BPI and the RIAA in December 2007. The title is a play on the famous ultimatum by Patrick Henry, "Give me Liberty, or give me Death!", and is intended as a commentary on American consumerism. Give Me Convenience or Give Me Death was the last Dead Kennedys album that Biafra approved the production of, which also led to it being the last album released through Alternative Tentacles.
The album includes the controversial "Pull My Strings" which was played only once on March 25, 1980, when Dead Kennedys were invited to perform at the Bay Area Music Awards in front of music industry bigwigs to give the event some "New Wave credibility". The band spent the day of the show practicing "California Über Alles", the song they were asked to play. In typically subversive, perverse style, the band became the talking point of the ceremony when after about 15 seconds into the song Jello Biafra said, "Hold it! We've gotta prove that we're adults now. We're not a punk rock band, we're a new wave band."
The band, who all wore white shirts with a big, black S painted on the front, pulled black ties from around the backs of their necks to form a dollar sign, then started playing "Pull My Strings", a satirical attack on the ethics of the mainstream music industry. The song also referenced The Knack's biggest New Wave hit, "My Sharona". The song was never recorded in the studio but this performance, the first and only time the song was ever performed, was included on the album.
Also included is "Night of the Living Rednecks", a recording of a segment of a concert in Portland, Oregon in 1979. When East Bay Ray broke a guitar string, Jello proceeded to tell a story over a bebop improvisation by the rest of the band about an encounter with a group of "dumb rich kids". The encounter supposedly took place on a previous trip to Portland with the band.