Condon said about the name and inspiration for the album: "Back in the early 1900s...there used to be this hot air balloon festival in Paris—[the album's] titled after that and after this very bizarre 1910 photo I found [by Léon Gimpel]. It's one of the first color photos ever made, at the World's Fair, and it...shows all these ancient hot air balloons about to take off in the middle of Paris. I just thought it was the most surreal image I'd seen in a long time." He also said about the album sound: "I was listening to a lot of Jacques Brel and French chanson music—pop songs shrouded in big, glorious, over-the-top arrangements and all this drama—and that was in some sense unfamiliar territory to me. So I started buying new instruments and relying on things I wasn't necessarily comfortable with, like French horns and euphoniums, carrying these big, epic big brass parts that I used to do all on trumpets, and working with accordion and organ instead of all ukulele—very much throwing myself in the world of classical pop music, I guess you could say."
According to the program notes given out during Beirut's performance at The Society for Ethical Culture in New York on September 24, 2007, each song on the album is intended to evoke a different French city (although Guaymas, Sonora is a Mexican city).
The album's second track, Nantes, was sampled on the 2012 10 Day mixtape from American rapper Chance the Rapper.