Hess was the third son of Karl Ernst Christoph Hess (1755–1828), an engraver. The elder Hess had already acquired a name when in 1806 the elector of Bavaria, having been raised to a kingship by Napoleon, transferred the Düsseldorf academy and gallery to Munich. The elder Karl Hess accompanied the academy to its new home, and there continued the education of his children, who, along with the younger Karl, included Heinrich Maria von Hess and Peter von Hess.
The elder Karl Hess hoped that his son Karl would obtain distinction as an engraver. The younger Karl, however, after engraving one plate after Adrian Ostade, turned to painting under the guidance of Max Joseph Wagenbauer of Munich, and then studied under his elder brother Peter. But historical composition proved to be as contrary to his taste as engraving, and he gave, himself exclusively at last to illustrations of peasant life in the hill country of Bavaria. He became clever alike in representing the people, the animals and the landscape of the Alps, and with constant means of reference to nature in the neighbourhood of Bad Reichenhall, where he at last resided, he never produced anything that was not impressed with the true stamp of a kindly realism.