Jolly was born in Heidelberg, the son of physicist Philipp Johann Gustav von Jolly (1809-1884), and studied comparative linguistics, Sanskrit, and Iranian languages in Berlin and Leipzig. His doctoral thesis was Die Moduslehre in den alt-iranischen Dialekten ("Moods in Ancient Iranian Dialects"). Jolly became a Professor in the University of Würzburg in 1877, in the fields of comparative linguistics and Sanskrit. In 1882-1883 he visited India as Tagore professor of law, Calcutta, where he gave twelve lectures later published as Outlines of an History of the Hindu Law of Partition, Inheritance and Adoption (1885).
In 1896, Jolly contributed to Grundriss der Indo-arischen Philologie und Altertumskunde ("Encyclopedia of Indo-Aryan Research"), later revised by Jolly and in 1928 translated by Balakrishna Ghosh under the title Hindu Law and Custom. In this volume Jolly discussed family law and heirship, law of things and obligations, offences and penalties, court procedure and customs and traditions. In 1901 he also contributed a study of Indian medicine, still considered one of the most complete and reliable studies of the history of Indian medical literature.
Jolly edited the law books of Vishnu, Narada, and Manu, and translated the first two for the Sacred Books of the East (Vol. 7, The Institutes of Visnu, 1880; and Vol. 33, The Minor Law-Books: Brihaspati, 1889). He retired in 1922, when he became co-editor of the Journal of Indian History, but continued to give lectures in Würzburg till 1928. He published a new critical edition of Kautilya's Arthashastra in collaboration with R. Schmidt in the "Panjab Sanskrit Series", 1923-1924.