Withers was inspired to write this song after watching the 1962 movie Days of Wine and Roses. He explained, in reference to the characters played by Lee Remick and Jack Lemmon, "They were both alcoholics who were alternately weak and strong. It's like going back for seconds on rat poison. Sometimes you miss things that weren't particularly good for you. It's just something that crossed my mind from watching that movie, and probably something else that happened in my life that I'm not aware of."
For the song's third verse, Withers had intended to write more lyrics instead of repeating the phrase "I know" 26 times, but then followed the advice of the other musicians to leave it that way: "I was this factory worker puttering around," Withers said. "So when they said to leave it like that, I left it." (Withers, then thirty-one years old, was working at a factory making toilet seats for 747s at the time.)
The song was originally released as the B-side to another song called "Harlem". Disc jockeys played "Ain't No Sunshine" as the single instead, and it became a huge hit, the first for Withers.