In 1970 he took on the persona of "Swamp Dogg" and released his first LP record, Total Destruction to Your Mind, which was reissued on March 5, 2013 by Alive Naturalsound Records. The cover of his 1971 album, Rat On!, featuring Williams on the back of an oversized white rat, was ranked as one of the worst album covers of all time.
Williams is also a noted songwriter and record producer. In an interview on NPR's Studio 360, Williams stated he was raised on country music: "Black music didn't start 'til 10 at night until 4 in the morning and I was in bed by then.. . . If you strip my tracks, take away all the horns and guitar licks, what you have is a country song."