The song's lyrics were heavily influenced by The Lord of the Rings by J. R. R. Tolkien. The opening line ("Leaves are falling all around") is apparently a paraphrase of the opening line of Tolkien's poem "Namárië". The poem may also be the inspiration for the entire first verse.
The Tolkien references later in the song refer to abstract themes — Mordor, "girl so fair", the ring — without invoking any of Tolkien's stories in detail:
Mine's a tale that can't be told,
My freedom I hold dear;
How years ago in days of old
When magic filled the air,
'Twas in the darkest depths of Mordor
I met a girl so fair.
But Gollum, and the evil one crept up
And slipped away with her.
The guitar's jangly introduction employs a classic Jimmy Page technique: using regular open chords superimposed higher on the fretboard.
There has been much doubt around Bonham's percussions in the background throughout the song, having regard to the particular resonance that it captures. There are no versions consistent in this respect: often mistaken for bongos, some said of a small plastic bin for waste played with the hands from the side of the bottom, others about the sticks on his drum kit vinyl seat pad, or even the soles of the shoes he wore during the recording session and so rhythmically beaten with his drum sticks; the book John Bonham: A Thunder of Drums reports instead of a hard case for guitar on which Bonham improvised percussion background with bare hands.
In June 2008 Jimmy Page and John Paul Jones joined the Foo Fighters on stage at Wembley Stadium and performed "Ramble On" with vocals performed by Dave Grohl and Taylor Hawkins on drums. Interestingly, the two Foo Fighters reversed performing roles for "Rock and Roll" the other song performed with the Led Zeppelin pair.