The title of the album is an allusion to the band's persistent difficulties with their record label, Capitol Records. Megadeth ultimately left Capitol and signed with Sanctuary Records, but were contractually obliged to release one further album with Capitol, hence the greatest hits compilation. With the exception of material from the band's debut album, Killing Is My Business... and Business Is Good!, which was released on Combat Records rather than Capitol, the compilation contains songs from all Megadeth's previously released albums at the time, in reverse-chronological order. In addition to the new track "Kill the King", Capitol insisted on the inclusion of "Dread and the Fugitive Mind", a track recorded for the album The World Needs a Hero, then in the production stage. The band agreed simply to release themselves from their obligations to Capitol, but the enmity between band and label was memorialized in the title of the album, intended to demonstrate that working with Capitol was a form of punishment. It is out of print. As of December 2005, Capitol Punishment had sold 200,000 copies in the U.S.
Nick Lancaster of Drowned in Sound criticized the compilation for including too many tracks from albums like Cryptic Writings and Risk instead of earlier, heavier material. Lancaster took particular aim at the inclusion of the "ludicrously moronic" "Crush 'Em". Despite that, he did give praise to the two then-new songs "Dread and the Fugitive Mind" and "Kill the King", citing them as a "return to form."
Similarly, Steve Huey of AllMusic also criticized the album for focusing more on commercial material at the expense of the band's earlier thrash material, as we as the exclusion of soundtrack songs like "Go To Hell", "Angry Again" and "99 Ways to Die". Huey summed up the compilation as "more of a sampler for casual fans", finding the release "too incomplete" and "too scattershot" otherwise.