The song interpolates the 1968 song "You Got What I Need" recorded by Freddie Scott, whose basic chord and melody provided the base for the song's chorus and made it famous. Due to the widespread popularity of the song and its influence on pop culture (and his failure to have another charting Hot 100 song), Biz was classified by VH1 as a one-hit wonder, and "Just a Friend" was ranked 81st on VH1's 100 Greatest One-Hit Wonders in 2000, and later as number 100 on VH1's 100 Greatest Songs of Hip Hop in 2008. Karma, a staff record producer for Cold Chillin' Records, told Vibe magazine in 2005 that he produced the single, but never received credit.
The music video, directed by Lionel C. Martin, chronicles the rapper/singer's woman problems. At the climax of the video, Biz Markie's character stumbles upon a girl he was trying to date - not his girlfriend - kissing another man she had previously referred to as "just a friend". It also includes a scene of Biz Markie singing the chorus dressed as Mozart in 18th century clothing with a powdered wig in a candlelit room while playing the piano.