The album contains the track "I Wrote This Song for the Girl Paris Hilton" which got its name from a friend of Lukas Haas. Gallo had yet to meet Paris Hilton but she would later appear in his short film "Honey Bunny".
The song "Yes I Am Lonely" was used in the video gamePhantom Dust and is played during the ending credits of the game. It is also available on the original soundtrack.
The song "So Sad" recorded during the "When" sessions but not present on the album is notable for it was released as a vinyl 12" single cut at 78 RPM (and 45 RPM on the other side). It was limited to 300 copies.
The whole album was recorded in Vincent's home and made use of numerous vintage guitars and other equipment. A mellotron is used on some tracks, notable because of the scarcity of those instruments. Gallo worked daily for two and a half years arranging the studio set up for the recording of 'When', and yet spent only three weeks recording the album.