Internationally, "Only Wanna Be with You" topped the Canadian RPM Top Singles chart for three weeks, becoming the second-most successful song of 1995. It also found major popularity in Iceland, rising to number four on the Icelandic Singles Chart, and reached the top 40 in Australia and New Zealand. It is arguably the band's signature song. Darius Rucker still performs the song solo.
One verse of the song describes "[putting] on a little Dylan". The verse then references songs on Bob Dylan's Blood on the Tracks, quoting extensively from "Idiot Wind". The song then mentions "Tangled Up in Blue". According to Rucker, Dylan’s management were aware of the lyrics and had no problem with them; however, when the song became a hit, they objected. He told Rolling Stone that "it never got to the point where we were sued... When we first did that song we sent it to the publishing company and everything was fine. We played it for years and had a really big hit with it. Then they wanted some money, and they got it." VH1 reported that Dylan received a large, out-of-court settlement in 1995.
Also mentioned in the bridge of the song is a reference "I'm such a baby cause the Dolphins make me cry", alluding to Darius Rucker's favorite football team and their championship drought.