The album was the second LP issued by Shelby Singleton in 1970 containing sides that Lewis had cut at Sun Records in the late 1950s and early 1960s. The first, The Golden Cream of the Country, had been extremely successful; in the wake of Jerry Lee's comeback, the previously unreleased track "One Minute Past Eternity" was released as a single and went to number 2 on the country charts. Like its predecessor, A Taste of Country featured a recent photo of Lewis, a shrewd move that created the impression that these were recent recordings.
The album features a mix of Lewis's early hits, such as his breakout version of "Crazy Arms" and the Hank Williams classic "You Win Again," with obscure B-sides and unreleased recordings. "It Hurt Me So" had been the B-side to the 1959 single "I'll Sail My Ship Alone" while "As Long As I Live" had been the flipside of the 1961 release "Save The Last Dance For Me" (it had also been featured on Lewis's second and final 1961 Sun LP Jerry Lee's Greatest). "Am I To Be The One" features Charlie Rich on background vocals. The album did nearly as well as Lewis's 1970 Mercury release She Even Woke Me Up To Say Goodbye, rising to number 15 on the Billboard country album charts (She Even Woke Me Up peaked at number 9).