The song focusses on a protagonist called Sam who is evidently being held, along with the track's narrator, in a political prison. Though set in the future, the story is related in the past tense, in a fashion Bowie has described as "future nostalgia... A past look at something that hasn't happened yet". Musically the song is noted for its "ultra-modern new wave guitar/synth sound", as well as for Bowie's use of varispeed vocals to illustrate Sam's downward spiral in the prison hospital – according to NME critics Roy Carr and Charles Shaar Murray, the effect is "as if the narrator of 'All the Madmen' inhabited the world of '1984'".
"Scream Like a Baby" was one of a number of tracks on Scary Monsters that evolved from pieces Bowie had written years before. It was originally composed in 1973, with different lyrics, as "I Am a Laser" for The Astronettes (Ava Cherry, Geoffrey MacCormack and Jason Guess). Bowie worked on an album for the group but it was eventually dropped, finally surfacing in 1995 as the Ava Cherry album People from Bad Homes; "I Am a Laser" was one of the tracks.