The title of the album is an allusion to the band's persistent difficulties with their record label, Capitol Records. Megadeth ultimately left Capitol and signed with Sanctuary Records, but were contractually obliged to release one further album with Capitol, hence the greatest hits compilation. With the exception of material from the band's debut album, Killing Is My Business... and Business Is Good!, which was released on Combat Records rather than Capitol, the compilation contains songs from all Megadeth's previously released albums at the time, in reverse-chronological order. In addition to the new track "Kill the King", Capitol insisted on the inclusion of "Dread and the Fugitive Mind", a track recorded for the album The World Needs a Hero, then in the production stage. The band agreed simply to release themselves from their obligations to Capitol, but the enmity between band and label was memorialized in the title of the album, intended to demonstrate that working with Capitol was a form of punishment. It is out of print. As of December 2005, Capitol Punishment had sold 200,000 copies in the U.S.