He studied medicine at the universities of Freiburg, Strasbourg and Würzburg, receiving his doctorate in 1885 with a thesis on cerebral syphilis. Following graduation he worked as an assistant to Karl Weigert at the Institute of Pathology in Frankfurt am Main, and to Albert Neisser at the university skin clinic in Breslau.
He later worked with his older brother, Salomon Herxheimer (1841–1899) in Frankfurt, where in 1894, he became director of the municipal dermatology clinic. Along with Paul Ehrlich, he was instrumental in founding the University of Frankfurt. In 1914 he became a professor for skin and venereal diseases at the new university. In August 1942, at the age of 81, he was taken to the Theresienstadt concentration camp, where he died a few months later.
Herxheimer is credited with providing an early description of acrodermatitis chronica atrophicans (Taylor's disease), which is a dermatological disorder associated with the later stages of Lyme disease. This condition is sometimes referred to as "Pick-Herxheimer disease", named along with co-discoverer Philipp Josef Pick (1834–1910). With Austrian dermatologist Adolf Jarisch (1850–1902), the Jarisch-Herxheimer reaction ("herxing") is named, which is an inflammatory reaction to Salvarsan, antibiotics or mercury, when using these agents to treat syphilis.
- Über acrodermatitis chronica atrophicans. Archiv für Dermatologie und Syphilis, Berlin, 1902, 61: 57-76.
- Ueber eine bei Syphilitischen vorkommende Quecksilberreaktion. Deutsche medizinische Wochenschrift, 1902, Berlin, 1902, 28: 895-897.
- Herxheimer, Karl In: Neue Deutsche Biographie (NDB). Band 8, Duncker & Humblot, Berlin 1969, ISBN 3-428-00189-3, S. 727 f.
- Karl Herxheimer @ Who Named It
- Wash Your Hands: Dirty Truth About Germs, Viruses, and Epidemics by Frédéric Saldmann
- Taylor's disease at Who Named It
- Herxheimer Reaction Chronic Illness Recovery
- The Treatment of syphilis with Salvarsan by Wilhelm Wechselmann