The song is about prostitution in the Neepsend district of Sheffield. Originally known as "Scummy", early reports had also suggested its name would be simply "Sun Goes Down", but the full name was confirmed on Arctic Monkeys' website. The line "and he told Roxanne to put on her red light" is a reference to The Police song "Roxanne".
The song's lyrics tell a story narrated from the point of view of a concerned individual who is approached by a scantily-clad girl, heavily implied to be a prostitute; he then observes a "scummy man" who has been hanging around the neighbourhood; the man is implied to be either the prostitute's pimp or a 'client' who is picking her up for sex. This section of the song musically consists of just vocalist Alex Turner singing accompanied by a pattern of electric guitar chords with a conspicuously clean tone. After the line "I said he's a scumbag don't you know" the song then changes drastically into a heavy rock style with a fast beat and driving guitar riff which is also played identically on the bass guitar. In the song's lyrics, now delivered a lot more venomously, the prostitute propositions the song's narrator and he turns her down politely; he then observes the "scummy man" arriving to pick her up in a Ford Mondeo. The girl is "delighted when she sees him" because "she must be fucking freezing, scantily clad beneath the clear night sky".
The song's chorus consists of the repeated line "they said it changes when the sun goes down around here", noting the stark difference between the appearance of the city at daytime and the dark prostitution trade the narrator observes at night. After the second refrain of the chorus, the song reverts to the style of the introduction. The last line, "I hope you're not involved at all", is either the narrator expressing his disappointment that the girl has turned to prostitution or a warning to the person to whom the narrator is telling the story, who also may be involved with the scummy man – or the listener themselves.
The song's music video was directed by Paul Fraser and premiered on MTV2 on 21 December 2005. It starred Lauren Socha and Stephen Graham. The video used footage from a longer film, Scummy Man, which used the same actors who appeared in the music video to tell the story of 'Nina', the nameless "that girl there" from the song.