"Sacrilege" was named "Best New Track" by Pitchfork Media February 26, 2013. Dan Martinson stated in his positive review, "Karen O falls for an angel who falls from the sky, the constant clank of Brian Chase's cymbals and Nick Zinner's guitar curlicues evoke "Gimme Shelter" with an exaggerated gospel swing."
The music video was directed by French collective Megaforce and premiered on March 26, 2013. The video stars English model and actress Lily Cole. The video consists of a chronologically reversed sequence of events outlining the rationale and events leading to a group of people in a small town shooting a man and burning a promiscuous woman (Cole) alive. The band does not appear in the video.