"Living in the Heart of the Beast" began as an unfinished instrumental that Hodgkinson presented to the group, which was cut up and performed live in 1974 with improvisational sections added. One such performance, Halsteren was recorded in Halsteren in September 1974, and appears in Volume 2: 1974–5 of The 40th Anniversary Henry Cow Box Set (2009). After the merger with Slapp Happy, Hodgkinson commissioned Blegvad to write lyrics for the piece for Krause to sing. But after several attempts, Blegvad admitted that he was "out of [his] depth", and Hodgkinson wrote the lyrics himself.
Chris Cutler's lyrics on "Beautiful as the Moon – Terrible as an Army with Banners" were the first song texts he had written, and the song was the first writing collaboration between Cutler and Fred Frith that later grew into Art Bears. The song also became the longest lasting "building block" the band used in subsequent live performances.
The album cover art work was by artist Ray Smith and was the last of the three "paint socks" to feature on Henry Cow's albums. Smith was an old friend of the band from Cambridge who had worked with them on two dance projects and had often supported them in performance art at concerts. Smith came up with the idea of the woven sock and insisted that the band's name should not appear on the front cover. As Cutler later explained, in a 2011 interview, the idea was extended through the whole album series, with the sock changing "to suit the temper of the music".
In 1991 East Side Digital issued a remixed version (by Tim Hodgkinson) of In Praise of Learning on CD with one extra track, "Lovers of Gold" (an alternate version of "Beginning: The Long March" created by Chris Cutler).
In 2000 Recommended Records and East Side Digital issued a remastered version of In Praise of Learning on CD with the original mixing restored and the bonus track omitted. The "Lovers of Gold" bonus track would later reappear on the Cow Cabinet of Curiosities disc in the The 40th Anniversary Henry Cow Box Set (2009).