After gaining a reputation in Miami clubs as a talented musician, Caldwell was signed to an exclusive contract with TK Records in 1978 by TK Records president Henry Stone. Heading to the studio, Caldwell recorded his first album, which was given a redo after Stone felt the album was good but "didn't have a hit". Caldwell returned to the studio and came up with the final product, which included "What You Won't Do for Love". The song's horn arrangement was written and recorded by Miami arranger Mike Lewis.
Caldwell wanted the song to be the sixth track on the album since he figured the song's second track, "My Flame", which featured him playing guitar, would be the hit. However, TK Records felt confident that "What You Won't Do for Love" would be the breakout hit. When it was released to R&B radio, TK Records did its best to shield Caldwell's identity (Caucasian) hoping not to alienate its predominantly black audience. However, when Caldwell began making performances live on stage, it only increased demand. The song would become Caldwell's most successful single and also his signature song, reaching number 9 on the Billboard Hot 100, number 6 on the Hot Selling Soul Singles chart, and number 10 on the Easy Listening chart.
According to the broadcast of American Top 40 for the week ending February 3, 1979, the week in which the song debuted at #38 on the Top 40, a heart-shaped pressing of the single was the most expensive single up to that point. The heart-shaped single was originally released as a promotional item only, but public demand led to 50,000 copies being pressed in time for Valentine's Day 1979 with a retail price of $7.98—about the price of a full LP album at the time. The single was also available on standard round vinyl as well.