Stills wrote the song after being inspired by the tag line -- "If you can't be with the one you love, love the one you're with" which was a frequent remark by musician Billy Preston. Stills asked him for permission to use the line in a song and Preston agreed immediately. Of course Preston, himself, may have taken the idea from Harburg and Lane’s “When I’m Not Near the Girl I Love, I Love the Girl I’m Near”, from the 1947 Broadway show Finian’s Rainbow; it was sung by Tommy Steele in the 1968 film version.
Several acts have since covered the song. The most notable cover came in 1971 from The Isley Brothers, whose unique gospel-driven cover of the song sent it to the charts again reaching number three R&B and number eighteen pop, giving the group their fifth US top 40 pop single.
UK pop group Bucks Fizz covered the song as their eighteenth single in 1986. The single, released in August, was the follow-up to the group's comeback top ten hit "New Beginning (Mamba Seyra)" and was seen as a make or break release. Ultimately the song peaked at a low No.47 in the UK Singles Chart during a three-week run. In a review, music magazine Number One said that the song lacked bass and sounded rather "tinny" but predicted that it would be a hit. Member Mike Nolan puts the song's failure down to the decision to showcase other male member Bobby G as the lead singer. Their previous single had featured all the group equally and was a hit, while earlier flops had featured G on lead and this was a return to that format. A promotional video for the song was filmed featuring the group performing the song in a blue-toned studio accompanied by backing musicians. The single was released on 7" and 12" on Polydor Records with an extended mix on the latter, the B-side was a Bobby G composition, "Too Hard". Also included on the 12" was an extended mix of earlier single "I Hear Talk". A second 12" single was released featuring a dance edit of "Love the One You're With", backed by another alternate mix of "I Hear Talk". "Love the One You're With" was included on the group's fifth studio album Writing on the Wall released at the end of the year. It has subsequently been included on re-issues of the album in alternate mixes.
In 2012, the song was featured in Ridley Scott's sci-fi movie Prometheus. Idris Elba sings a short phrase from the song after describing a possibly fictional account of how his squeezebox used to belong to Stills.