Jones and Haggard were largely influenced by the Hank Williams and Lefty Frizzell tradition. They had also made no secret about how much they admired each other's work. In a Rolling Stone tribute to Jones after his death in 2013, Haggard recalled their first meeting: "I met him at the Blackboard Café in Bakersfield, California, which was the place to go in '61. He was already famous for not showing up or showing up drunk, and he showed up drunk. I was onstage—I think I was singing Marty Robbins' 'Devil Woman'—and he kicked the doors of the office open and said 'Who the fuck is that?'" Haggard added that Jones's voice was "like a Stradivarius violin: one of the greatest instruments ever made." Jones had said repeatedly over the years that, next to Hank Williams, Haggard was his favorite singer. A Taste of Yesterday's Wine features tributes to both of them: "Silver Eagle", written by Freddy Powers and Gary Church about Haggard, and "No Show Jones", written by Jones and Glenn Martin about the wayward singer's notorious inability to arrive at concert dates. The album's first single was the Willie Nelson-penned title track, which became a number one hit. A second single, "C.C. Waterback", reached number 10. The LP was produced by Billy Sherrill and featured backing vocals by Haggard's wife Leona Williams.
In the UK Hallmark Records issued an LP with the same title, artwork and tracks, but with a different track order.
Thom Jurek of AllMusic praises the album, marveling that the pair's voices "blend seamlessly and compliment each other in almost symbiotic fashion... Billy Sherrill in the producer's chair was swinging for the radio fences, and he got close, but even he stayed the hell out of the way most of the time here and let the music take its course, and this pair just treated each other deferentially."