In order to create the layered guitar introduction and fade-out, Page used a Delta T digital delay unit. The squeak of John Bonham's drum pedal can be heard throughout the song.
Although the name of the song is "Houses of the Holy," the track does not appear on the earlier-released album Houses of the Holy. It was originally set to be the title track of that album but was removed when the band decided it did not fit. The song did not require any further re-mixing for its inclusion on Physical Graffiti, having already been fully mixed by audio engineerEddie Kramer at the Electric Lady sessions in June 1972.
Record producer Rick Rubin has remarked, "This is a funk jam with really interesting, jazzy chords. It's one of their more compact feeling songs. And it's the only Zep song to use what sounds like a cowbell."