The Canadian singer, who had achieved several hits from 1968 to 1971, had not had a Top 100 single since September 1971, and had been without a record label since early 1973. Nevertheless, he said in a 1974 interview, "I never mentally admitted defeat in spite of three years off the charts." He formed his own label, Ice Records, and personally financed the recording session that produced "Rock Me Gently". He could afford to record only two sides, and deciding the second side was good enough to be an A-Side, he put an instrumental of Rock Me Gently on its B-Side.
The single impressed Capitol Records executives, who signed Kim to a deal. "Rock Me Gently" debuted on the Hot 100 on June 22, 1974, and took 14 weeks to reach #1 on September 28. It also rose to #2 on the UK Singles Chart, and #10 in Ireland, and remains his only charting song in either the UK or Ireland. Even the instrumental B-Side received substantial airplay on R&B stations. It would be Kim's last top 10 hit in either country.
Part of the song was used in a 1970s UK TV commercial for Lever Brothers' Jif cleaning cream, using the lyric "When Jif’s your cleaner / Tough dirt goes / Away so gently / And it shows / Your home has never been loved like this before".