Produced by Brian Holland and Lamont Dozier, Marvin Gaye released the song as a single in September 1964. It peaked at No. 6 on the Billboard Hot 100 pop singles chart in January 1965 and No. 4 on Billboard'sR&B Singles chart. Up to that point, it was Gaye's most successful single with record sales exceeding 900,000 copies. The song's personnel includes Marvin Gaye on lead vocals,The Andantes on background vocals, and The Funk Brothers on various instruments, including piano and percussion. Gaye also released a German-language version of the song entitled "Wie Schön Das Ist".
Allmusic critic Jason Ankeny described the song as a "radiant pop confection," noting that it was unusual for Gaye in being a "straightforward love song" that doesn't reflect Gaye's usual demons. Anekny commented on the soulfulness of the song, and particularly noted the piano riff.
Gaye's recording has been subsequently released on many greatest hits albums.
In 1966, Junior Walker & the All Stars released the song as a single, which reached No. 3 on the R&B Singles chart and No. 18 on the Billboard Hot 100 chart. According to Ankeny, this version makes up in grit what it lacks in Gaye's grace, increasing the "celebratory" feel of the song. Writing for AllMusic, Jason Ankeny described the version as "a rollicking floor-shaker fueled by his smoldering saxophone."
James Taylor released a cover version of "How Sweet It Is (To Be Loved by You)" on his 1975 album Gorilla and also as the lead single from the album. Taylor's 1975 single has been the most successful cover version to date, hitting No. 1 on the Easy Listening chart and No. 5 on the Billboard Hot 100 chart. In addition to James Taylor on lead vocals and guitar, other personnel include Carly Simon on harmony vocals, David Sanborn on saxophone, Clarence McDonald on piano, Fender Rhodes electric piano and possibly ARP String Ensemble, Lee Sklar on bass and both Jim Keltner and Russ Kunkel on drums with Kunkel doubling on tambourine. Author Ian Halperin believes that the song was included on Gorilla as a tribute to Simon, who was then his wife. It was produced by Lenny Waronker and Russ Titelman. After his success with "How Sweet It Is (To Be Loved By You)," Taylor continued to cover R&B hits applying his soft rock approach.
Rolling Stone Magazine critic Bud Scoppa described Taylor's version as "a relaxed rendition" and considered the cover to be Taylor's way of acknowledging Gaye as a source of inspiration for Taylor's romantic point of view at the time. Music critic Robert Christgau regarded Taylor's version as a "desecration of Marvin Gaye." Taylor biographer Timothy White described it as "music for the park on Sunday."