He studied medicine at Göttingen under Georg Meissner (1829–1905), and in 1867 received his doctorate at Munich. In 1868 he became assistant to Bernhard von Gudden (1824–1886) and Hubert von Grashey (1839–1914) at the mental institution in Werneck, and in 1870 was an assistant to Franz von Rinecker (1811–1883) at the Juliusspital in Würzburg.
In 1873 Jolly became director of the psychiatric clinic in Strassburg, where he became successor to Richard von Krafft-Ebing (1840–1902). In 1890 he succeeded Karl Friedrich Otto Westphal (1833–1890) as director of the neuropsychiatric clinic at the Berlin Charité.
Jolly is remembered for his pioneer research of myasthenia gravis, including the electrophysiological aspects involving abnormal fatigue associated with the disease which forms the basis of Jolly's test. He is credited with coining the term myasthenia gravis pseudoparalytica for the disorder.
He was the author of an influential treatise on hypochondria that was published in Hugo Wilhelm von Ziemssen's (1829–1902) Handbuch der speciellen Pathologie und Therapie. His Untersuchungen über den elektrischen Leitungswiderstand des menschlichen Körpers (1884) was fundamental to the study of electrical diagnostics.
His grave is preserved in the Protestant Friedhof III der Jerusalems- und Neuen Kirchengemeinde (Cemetery No. III of the congregations of Jerusalem's Church and New Church) in Berlin-Kreuzberg, south of Hallesches Tor.