Heart to Heart was released just weeks after Haggard and Williams had divorced after five years of marriage. Williams, who had replaced Bonnie Owens in Haggard's life both professionally and personally in 1974, grew to become increasingly frustrated with her supporting role in the Strangers, having harbored musical aspirations of her own; the Missouri native had been a well respected musician and singer in her own right (she had played bass in Loretta Lynn's first touring band) and wrote two #1 hits for Haggard: the telling "You Take Me For Granted" in 1982 and "Someday When Things Are Good" in 1983. According to the liner notes for the 1994 retrospective Down Every Road written by music journalist Daniel Cooper, she wrote the former while sitting on the bus in Ohio, then played it for Merle in front of several of his friends after Merle had reduced her to tears during a duet session they were recording. "He got big old tears in his eyes," Cooper quotes Leona, "and he said, 'Is that how you feel? And I said, 'Yes, it is.'" As Haggard wrote in his 1981 autobiography Sing Me Back Home, "I'd reached the point in my career where I felt in charge of my music...When Leona tried to make a suggestion, I resented it. She resented my resentment. So it went. She kept saying she felt like an outsider...I couldn't understand why she got so upset by the press leaning toward good ol' Bonnie and the snide remarks about Leona coming in and breaking up my 'happy home.'" In the documentary Learning to Live With Myself, fellow country star Tanya Tucker speculates that a professional competition came between the couple, remembering that Merle promised "after Leona he would never, ever marry another woman who wanted to be in show business." In his 2013 Haggard book The Running Kind, David Cantwell observes that Haggard "seems alternately to have supported and undermined her efforts" at establishing a solo career, citing their finally recording together on Mercury causing the label to lose interest in her forthcoming solo LP altogether. Their divorce served as a license to party for Haggard, who spent much of the next decade becoming mired in alcohol and drug problems.
Haggard had to obtain permission from Epic Records to record with Williams, who was signed to Mercury. Heart to Heart was Haggard's third duet album in two years, having recorded A Taste of Yesterday's Wine with George Jones and Pancho and Lefty with Willie Nelson. Heart to Heart was not the success those LPs had been, however, peaking at number 44 on the Billboard country albums chart. A single, the co-written "We're Stranger's Again," did not crack the country Top 40. Haggard contributed four songs to the set, including two written with Williams. Highlights include the Freddy Powers song "It's Cold in California" and "Don't Ever Let Love Sleep," as well as the title track, but on a couple of songs, "Sally Let Your Bangs Hang Down" and "I'll Never Be Free", Williams contributions are reduced significantly, reinforcing the stifling image of her as merely Haggard's backup singer. In the late nineties, Williams initially agreed—and then declined—to be interviewed by Tom Carter for Haggard's second autobiography My House of Memories, which would be published in 1999. Haggard alluded to this in the book and, apparently affronted, never mentions Williams by her name, simply referring to her as "wife number three." He also stated that Williams leaving him was "one of the best things to have ever happened to me." Williams did later participate in the American Masters documentary dedicated to Haggard.