The title song was set to be released on John's previous album Tumbleweed Connection. However, it was set aside and would eventually be re-recorded and serve as the title track of this album. Previous versions of the song (from the Tumbleweed sessions with Mick Ronson on guitar) can still be found, specifically on the remastered Tumbleweed Connection CD.
When it was released in 'The Classic Years' collection, it was the first album not to feature any bonus tracks. One known track recorded at the time, "Rock Me when He's Gone," was released on Rare Masters a few years before. The song was written for and recorded by one of John's long-time friends, Long John Baldry. This was John's first album in which he plays his piano and no other keyboards. This was the first album in which Davey Johnstone played guitar for John, a role that would continue for decades.
John, Johnstone, Dee Murray and Nigel Olsson, who would be the main musicians on the next album (Honky Chateau), would soon join with percussionist Ray Cooper and form the best-known line-up of his mid-1970s band. As with all John songs during this period, the lyrics were penned by his writing partner, Bernie Taupin. This was the last album to be recorded at London's Trident Studios. They relocated to Château d'Hérouville for the next three albums. Caleb Quaye and Roger Pope wouldn't play with John again until Rock of the Westies in 1975, following Murray and Olsson's departure from the band.
Upon its release, Madman Across the Water was almost ignored in John's homeland, barely reaching #41 on the UK Albums Chart and spending only two weeks there. It has been the lowest-charting album of his career to date. The album fared better in North America, peaking at #8 on the U.S. Billboard Top Pop Albums and placing at #10 on the year-end list of 1972. It received Gold by the RIAA in February 1972, achieving $1 million in sales at wholesale value just in the United States. In 1998, the album was certified Multi-Platinum, representing shipments of over 2 million units in the U.S.
The album cover for Madman Across the Water was embroidered over two weeks by Janis Larkham, wife of the album art director David Larkham. She used the back of an old Levi's jacket, and the original was gifted to John. There was no initial run of expensive album covers with similar embroidery, nor embossed versions; it was flat printed, though it did include a lavish booklet. Janis chose to credit herself as "Yanis" for this artwork.
When this album was issued on cassette MCA swapped the places of "Razor Face" and "Rotten Peaches" on side one and two of the tape. The SACD version of the album contained a longer version of "Razor Face" which extended the song-ending jam to 6:42 instead of the early fade on the original album.
The song "Holiday Inn" was written for Adam Diaz. An additional verse in the recorded version that was originally part of the song was either omitted during recording or edited out of the final album version. It can, however, be heard on "unofficial" live recordings of the song from some of his concerts during this period, which have circulated among collectors.