According to lead singer Chuck Negron's book Three Dog Nightmare, the album's working title was The Wizards of Orange, with a cover featuring the band's members wearing orange make-up and posing in the nude. The band's record company, ABC/Dunhill, rejected the original album title and cover art, although some configurations of their first "greatest hits" album, 1971's Golden Bisquits, would later be packaged using It Ain't Easy's original cover photo.
Reviewing in Christgau's Record Guide: Rock Albums of the Seventies (1981), Robert Christgau wrote: "Admitting it won't gain me any of the hip cachet I crave, but I admired and enjoyed this group's first LP. I found the second mediocre and the live job that followed it wretchedly excessive, but this one—their fourth in just fourteen months—gets back: exemplary song-finding and not too much plastic-soul melon-mouthing or preening vocal pyrotechnique. Highlights: the hit version of Randy Newman's 'Mama Told Me Not to Come,' with just the right admixture of high-spirited schlock to turn it into the AM giant it deserves to be, and a departure from pre-Beatles times called "Good Feeling (1957)."