The song is about purchasing $26 worth of heroin in a Harlembrownstone near the intersection of Lexington Avenue and 125th Street in New York City. The song is sung from the point of view of the purchaser who is presumably traveling to Harlem from another part of the city; the "man" in the song's title is a drug dealer. Along with traditional guitar, bass, and drums, the song also features pounding, percussive rock-and-roll barrelhouse-style piano. It is one of the band's more popular songs, and one of their many compositions featuring drugs as subject matter. After leaving the band in 1970, Lou Reed continued to incorporate the song into his solo live performances.
It was released as a single in October 1971 (with "There She Goes Again" on the B-side) as Andy Warhol's Velvet Underground on MGM Records.
Before the final album version of the song was re-recorded in May 1966, at T.T.G. Studios in Hollywood, a different take of the song was originally recorded in April 1966 at Scepter Studios in Manhattan. The original Manhattan take of the song is slightly shorter, the piano is less audible and instead of drums, a tambourine is employed. Also of note is that Reed sings "I'm waiting for the man" at the beginning of the song; on the album version, Reed sings "I'm waiting for my man."