Karolidis was born in 1849 in the village of Androniki (Turkish: Endürlük, now a suburb of Kayseri) in Cappadocia. His father Konstantinos Karolidis or Karloglou was a wealthy landowner and wheat merchant. Like most Cappadocian Greeks (see Karamanlides), Karolidis' mother tongue was Turkish, but he was educated at Greek schools, including two of the premier Greek-language institutions of the Ottoman Empire, the Great School of the Nation in Constantinople and the Evangelical School of Smyrna. In 1867 he enrolled in the School of Philosophy of the University of Athens, and in 1870 he went to Germany on a scholarship. He studied at the universities of Munich, Strasbourg and Tübingen and was awarded his doctorate in 1872.
On his return from Germany, he initially taught in the Greek high schools of Pera and Chalcedon. In 1876 he went to Smyrna to teach at the Evangelical School. There he remained until 1886, when he moved permanently to Athens in the independent Greek kingdom. After teaching in a high school for a few months, he was elected assistant professor of General History at the University of Athens. In 1893, he succeeded the dean of modern Greek historians, Constantine Paparrigopoulos, at the chair of Greek History. Initially, Karolidis pursued the idea of occupying a new seat for Oriental Studies, where he was more qualified, but his rivalry with Spyridon Lambros negated this prospect.
As he was still an Ottoman citizen, in 1908 Karolidis was elected to the Ottoman Parliament. His independent-mindedness during his tenure, especially with regards to his ardent anti-Slavic feelings and his hopes for a Greco-Turkish rapprochement, alienated him from the Greek authorities and those Ottoman Greeks who aligned themselves with the policies of the Greek kingdom. Disappointed, Karolidis initially thought to return to Athens and resume his university post, but in the event he was convinced to run as a candidate for the Committee of Union and Progress, the party of the Young Turks. This was seen as tantamount to treason by the nationalist Greek press, with rumours even spreading that he had converted to Islam. Karolidis was elected to Parliament and remained at Constantinople until September 1912. As war between the Ottoman Empire and the Balkan League, to which Greece had acceded in May, became inevitable, he left for Germany. He returned to Greece only towards the end of the First Balkan War in May 1913.
Karolidis resumed his teaching at the University of Athens only in September 1915. A convinced royalist, he supported King Constantine I during the National Schism, a fact which cost him his post following the victory of Eleftherios Venizelos and the king's abdication in June 1917. He was reinstated in early 1921, after Venizelos' electoral defeat, and kept his post until he was pensioned off in 1923. His political leanings changed abruptly in this period, following the Asia Minor Disaster, and he became fiercely critical towards the Greek monarchy. He died in Athens on 26 July 1930.
Karolidis's initial research, during the 1870s and 1880s, was focused on his home region of Cappadocia, with the publication of Kappadokika, a historical and archaeological dissertation on Cappadocia in 1874 and his studies on the city of Comana and the Cappadocian Greek dialect published in 1882 and 1885 respectively.
Although an eminent Orientalist, after his appointment to the University of Athens Karolidis largely neglected the field and instead turned to Greek and general history, in conformity with the chairs he held. He published 18 books and 38 articles in the period 1893–1908, including his three volume History of the 19th Century, a three volume-work (plus the introductory Introduction) which focuses on Greece, and his unfinished Universal or World History, only four of whose projected ten volumes were completed. The latter work is of particular importance as it was one of the rare studies on the historical method written in Greek until then. He is also notable as the editor of the revised edition of Paparrigopoulos' History of the Greek Nation, published in 1902–1903.
The period after the Balkan Wars was not very productive for Karolidis, but after 1922 he produced some of his most famous works, dealing with the post-Byzantine period of Greek history (Ottoman Greece and Modern Greece): the seven-volume Contemporary History (1922–1929) and the History of Greece (1925). The eighth volume of the Contemporary History was eventually published in 1932, incorporated in the sixth edition of Paparrigopoulos' History of the Greek Nation.
- Kechriotis, Vangelis (2016). "A Cappadocian in Athens, an Athenian in Smyrna, and a Parliamentarian in Istanbul: The Multiple Personae and Loyalties of Pavlos Carolidis". In Isom-Verhaaren, Christine; Schull, Kent F. Schull. Living in the Ottoman Realm: Empire and Identity, 13th to 20th Centuries. Indiana University Press. pp. 297–309. ISBN 978-0-253-01943-1.
- Touloumakos, Pantelis (2006). "Karolidis, Pavlos". Encyclopaedia of the Hellenic World, Asia Minor. Retrieved 27 May 2011.