The band formed after Badwan introduced Zeffira to 1960s girl group music. Zeffira became intrigued by the likes of The Ronettes and acts produced by Joe Meek and Phil Spector, and soon began work on a demo version of a track (which eventually became "The Lull") in a similar style, which she sent to Badwan upon completion.
Using Zeffira's classical music contacts, the band were able to perform a short performance using a church organ and choir at the Vatican, in front of several important cardinals. Following positive press on the performance, they offered free downloads of two songs via their website, "Not a Friend" and a cover/remix of Grinderman's "When My Baby Comes".
On 28 February 2011, the duo released the Broken GlassEP on the Polydorlabel. It was recorded at Peter Gabriel's RealWorld Studios with record producer Steve Osborne. The Broken Glass EP featured three original compositions and a cover of a previously unreleased demo song by The Horrors, "Sunshine Girls". It met with positive press and was followed by the release of full-length album Cat's Eyes on 11 April 2011. Also recorded at RealWorld Studios, Cat's Eyes received positive reviews, with a four-star review from The Guardian and a score of 7.9 from Pitchfork Media, and yielded two singles, "Face in the Crowd" (on 2 May 2011) and "Over You" (on 25 July 2011).