According to Adams, the album's title originates from a poster of Mariah Carey: "My manager called and said, 'You have 15 seconds to name this record,' "My eyes focused on this poster of Mariah wearing a T-shirt that said HEARTBREAKER. I just shouted, 'Heartbreaker!'"
English DJ Mark Ronson remixed the song "Amy" for his 2007 album Version, of which singer Kenna provides vocals. The song "To Be Young (Is To Be Sad, Is To Be High)" is featured in the 2006 film Accepted, the 2002 film The Slaughter Rule, and the 2003 film Old School. A version of this song was also released in 2009 by David Rawlings on the Dave Rawlings Machine album "A Friend of a Friend." "Come Pick Me Up" is featured in the film Elizabethtown (which also featured two other Ryan Adams songs) and in a Series 2 episode of Skins. It was also named #285 on Pitchfork Media's "Top 500 songs of the 2000s." Joan Baez would cover "In My Time of Need" in 2003.
The album was considered to be a fresh start for Ryan Adams after the demise of his previous band Whiskeytown. Allmusic's Mark Derning wrote that the album "is loose, open, and heartfelt in a way Whiskeytown's admittedly fine albums never were, and makes as strong a case for Adams' gifts as anything his band ever released", concluding that "the strength of the material and the performances suggest Adams is finally gaining some much-needed maturity, and his music is all the better for it."The A.V. Club 's Keith Phipps wrote: "Adams has recorded an intimate, largely quiet record that indisputably establishes his identity as an independent singer-songwriter".Pitchfork Media's Steven Byrd called it "an album of astonishing musical proficiency, complete honesty and severe beauty."Rolling Stone 's Anthony Decurtis was less enthusiastic, stating that "Adams' songs too often fail to rise above their plain-spoken details to take on the symbolic power he yearns for".