Withers' maternal grandfather, Grackus Monroe Galloway, had been born into slavery. As a boy Withers would attend church with his grandmother, Lula, where she would sing and clap along with the hymns. He later said: "It was spontaneous singing, there was nothing programmed. People got up and sang and everybody would join in. It was my favourite kind of singing."
A central theme of the song is the protective and nurturing force of the hands, as expressed in the last verse:
The song was used as inspiration for the name of Oxford 6 piece sonic-folk band Grandma's Hands. For a while the band covered the song live until they realized they were unable to match the power of the original artist's performance.