Memories of Us was Jones's first release after his divorce from Tammy Wynette. The years ahead would be an unpleasant time for Jones, to say the least, and the singer would soon begin his long, slow descent to the bottom. Jones scored two number-one hits in 1974 ("The Grand Tour" and "The Door"), but Memories of Us failed to produce any hits and the album stalled on the Billboard charts at number 43. In the liner notes to the 1983 Jones compilation Anniversary - 10 Years of Hits, producer Billy Sherrill writes that he was surprised that the song "Memories of Us", a personal favorite, failed as a single, and in his 1996 autobiography I Lived to Tell It All, Jones admitted that at the time he "constantly feared that my career, like my three marriages, was over. I had nothing left but my name, and my name was associated with missing personal appearances and not paying my debts."
Although Jones's life was in disarray, his singing remained as strong as ever, and the acrimony between him and Wynette in the aftermath of their bitter divorce made the songs he sang sound more authentic to his fans. Memories of Us actually features two songs written by the former couple: the up-tempo drinking song "Bring On the Clowns" and the eerily titled "She Should Belong to Me". Jones also had a hand in writing "A Touch of Wilderness" with Earl Montgomery, but it was his performance on the song "I Just Don't Give a Damn", which he wrote with Jimmy Peppers, that remains - at least in hindsight - one of the most riveting performances of his career. Thom Jurek of AllMusic calls the song "the bitterest cut Jones ever recorded. He claims he wrote it at 3 a.m. in the aftermath of the divorce, and it comes right from the Hank Williams tradition of catharsis songs. Jones condemns everyone and everything including himself. As he denies his shortcomings, he fires back simultaneously - with razor-sharp fineness - his anger. That fiddle floating in the background offers a portrait of loneliness and rage that is unbridled and self-destructive in the classic honky tonk style." In the liner notes to the 1999 Sony reissue of the album (which paired it with 1976's The Battle), Daniel Cooper writes that "I Just Don't Give a Damn" "hearkens back to the sort of material Jones's idol Hank Williams used to write when Hank was feeling his most embittered. In his prolific career, Jones has made dozens of bone-chilling recordings that languish as forgotten, seemingly throwaway album tracks and B-sides...Yet all these years later 'I Just Don't Give a Damn' rings harrowingly true and honest, like a sketched, 3 a.m. self-portrait by one of America's greatest artists." Remarkably, the song was ignored for years, with the singer commenting in his memoir, "'I Just Don't Give a Damn' was my eighty-sixth single record. It was on the Billboard survey for only two weeks and peaked at number ninety-two on the top 100. I had never released a record on a major label that did so poorly." The song was included on the 2006 Legacy compilation The Essential George Jones and a rare clip of the singer performing the song in 1975 at his Possum Holler club in Nashville is available on YouTube.
Roy Kasten of Amazon.com states "...Memories of Us is relentlessly dark, distinguished by the unbearably sad 'A Goodbye Joke' and 'What I Do Best,' in which pathos becomes great art. Surrounded by Sherrill's often-breathtaking production, Jones's voice never sounded so plangent, so aching." AllMusic describes the album as "one of those records that cannot be played all the time, but when in the proper space for a heartbreak record, none will fit the bill better."