Following the release of their debut album, Everything All the Time, founding members Mat Brooke, Chris Early and Tim Meinig all left the band. Band leader Ben Bridwell recruited Joe Arnone (guitar & keyboards), Creighton Barrett (drums) and Rob Hampton (bass & guitar) as replacements, although Joe Arnone left the band prior to their 2007 tour. Robin Peringer (Modest Mouse - guitar), Matt Gentling (Archers of Loaf - bass) and Ryan Monroe (keyboards) joined the band for the tour, although only Monroe stayed for the recording of Cease to Begin. After the album's release he became a full band member.
The album debuted at number 35 on the U.S. Billboard 200 chart and was #47 on Rolling Stone's list of the Top 50 Albums of 2007. It was also a minor hit in Scandinavia, charting in Norway, Sweden and Denmark.Cease to Begin was released in Japan on July 23, 2008 by Tear Bridge Records. The Japanese version featured an enhanced video for "Is There a Ghost" plus three live bonus tracks, including a cover of Ronnie Wood's "Act Together", written by Mick Jagger and Keith Richards, which he recorded for his album, I've Got My Own Album to Do.
Ben Bridwell described the sound of Cease to Begin, "I guess the first record had some kind of country-ish leanings...I think there's maybe a little bit more of [that] feeling on it, a little more down-home...not so much indie rock."
The first track, "Is There a Ghost", was made available for download on the band's MySpace page on August 28, 2007 and made number 93 on Rolling Stone's list of the 100 Best Songs of 2007. The song became the band's first single to chart in the United States, peaking at #34 on the Billboard Modern Rock Tracks chart. The album's second single "No One's Gonna Love You" was featured as a free iTunes download, distributed at Starbucks locations.