The song is about purchasing $26 worth of heroin in a Harlembrownstone near the intersection of Lexington Avenue and 125th Street in New York City, sung from the point of view of the purchaser who is presumably traveling to Harlem from another part of the city; the "man" in the title is a drug dealer. Along with traditional guitars, bass, and drums, the song also features pounding, percussive rock-and-roll barrelhouse-style piano. It is one of the band's more popular songs, and one of their many compositions featuring drugs as subject matter. After leaving the band in 1970, Lou Reed continued to incorporate it into his solo live performances.
It was released as a single in October 1971 (with "There She Goes Again" on the B-side) as by Andy Warhol's Velvet Underground on MGM Records.
Before the final album version of the song was recorded in May 1966, at T.T.G. Studios in Hollywood, a different take was recorded in April 1966 at Scepter Studios in Manhattan. The original Manhattan take is slightly shorter, the piano is less audible and instead of drums, a tambourine is employed. Also of note is that Reed sings "I'm waiting for the man" at the beginning of the song; on the album version, Reed sings "I'm waiting for my man."