The song interpolates the 1968 song "You Got What I Need" recorded by Freddie Scott, whose basic chord and melody provided the base for the song's chorus and made it famous. Due to the widespread popularity of the song and its influence on pop culture (and his failure to have another charting Hot 100 song), Biz was classified by VH1 as a one-hit wonder, and "Just a Friend" was ranked 81st on VH1's 100 Greatest One-Hit Wonders in 2000, and later as number 100 on VH1's 100 Greatest Songs of Hip Hop in 2008. Karma, a staff record producer for Cold Chillin' Records, told Vibe magazine in 2005 that he produced the single, but never received credit.
The music video, directed by Lionel C. Martin, chronicles the rapper/singer's woman problems. At the climax of the video, Biz Markie's character stumbles upon a girl he was trying to date – not his girlfriend – kissing another man she had previously referred to as "just a friend". It also includes a scene of Biz Markie singing the chorus dressed as Mozart in 18th-century clothing with a powdered wig in a candlelit room while playing the piano.
Appears as one of the featured music videos in the television show Beavis and Butt-Head, in the 2nd season episode entitled, "Home Improvement".
Appeared on the radio station The Mix 107.77 in the video game Saints Row IV and in one sequence was sung by the game's villain, Zinyak. It is later played at the end credits, sung by the cast of the game. It soon fades out, leaving only Michael Clarke Duncan, the original voice of Ben King, to sing the remainder of it. Michael died before his recording of Ben could be completed and his final song was Volition's own tribute to him.
Appears in a Heineken "Let a Stranger Drive You Home" commercial from 2009 with people after a night out singing along to the song in the back of a cab.