The song is about purchasing $26 worth of heroin in a Harlembrownstone near the intersection of Lexington Avenue and 125th Street in New York City. The song is sung from the point of view of the purchaser who is presumably traveling to Harlem from another part of the city; the "man" in the song's title is a drug dealer. Along with traditional guitar, bass, and drums, the song also features pounding, percussive rock-and-roll barrelhouse-style piano. It is one of the band's more popular songs, and one of their many compositions featuring drugs as subject matter. After leaving the band in 1970, Lou Reed continued to incorporate the song into his solo live performances.
It was released as a single in October 1971 (with "There She Goes Again" on the B-side) as Andy Warhol's Velvet Underground on MGM Records.
The song was among a set of early songs to be recorded by Lou Reed, John Cale and Sterling Morrison in the band's Ludlow Street loft in Manhattan. This version of the song, free of percussion, has a considerably more folk and even blues influence in style than the album version. It is available on the first disc of the Peel Slowly and See box set.
Before the final album version of the song was re-recorded at T.T.G. Studios, in Hollywood, California, a different take of the song was originally recorded at Scepter Studios in New York City. This take of the song is slightly shorter, the piano is less audible and instead of drums, a tambourine is employed. Also of note is that Reed sings "I'm waiting for the man" at the beginning of the song. Though the album version, Reed sings "I'm waiting for my man."