Born in Ivangorod, Shakhmatov was brought up by his uncle near Saratov. He went to a public school in Moscow and developed interest for Old Russian language and literature at an early age. At the age of 16, his articles started to appear in the most authoritative journal of Slavic studies of that time - Archiv für slavische Philologie.
Shakhmatov furthered his education at the Moscow University (1883–1887), later delivering lectures in the same institution. His first monograph, published in 1886, examined the language of ancient Novgorod charters. In 1891 he became so enthusiastic about zemstvo that he gave up his scholarly pursuits for three years and held a minor administrative office in his native village.
In 1894 Shakhmatov returned to Moscow and won great acclaim for his Ph.D. dissertation, entitled Studies in the Sphere of Russian Phonetics. Five years later, he was admitted to the Russian Academy of Sciences, and over the following years became one of the most reputable academicians. He revived the Academy's linguistic periodicals, edited the academic dictionary of Russian language and was elected to represent the Academy at the State Council of Imperial Russia and Imperial State Duma.
In 1909, Shakhmatov moved to work in the St Petersburg University as a professor. By that time, he had been elected doctor honoris causa by the Charles University, Berlin University, Polish Academy of Sciences, and many other scholarly societies.
Shakhmatov participated in the Commission for the Study of the Tribal Composition of the Population of the Borderlands of Russia set up in February 1917. He also helped prepare sweeping reforms of Russian orthography, which would be implemented by the Bolsheviks in 1918. Shakhmatov refused to leave Petrograd for the West, a fatal decision that led to his premature death from malnutrition in 1920. The Academy subsequently cherished his memory and instituted a special Shakhmatov Prize, to be awarded "for the best works in source science, textology and linguistics".
Shakhmatov is best remembered for having pioneered textological research of early Russian chronicles, notably the Primary Chronicle. He established with a great degree of precision the stages of evolution of that key document, even attempting to reconstruct the postulated proto-version of Nestor's chronicle. His research proved seminal for subsequent generations of historians.
Shakhmatov was also responsible for publication and pioneering studies of minor or derelict Slavic languages. His studies of Slavic etymology revolved around the idea of close contacts and influences between the ancient Slavs and Celts, a hypothesis that was subsequently discarded. In particular, Shakhmatov was convinced that the Prekmurian language, spoken in Prekmurje and Vendvidék, contains Celtic extract, due to the existence of the ü and ö characters. In fact, Prekmurian is a variant of Slovene language, while the ü, ö characters are common in other dialects of Slovene, such as the Prlekijan, and partly the Carinthian. The Hungarian nationalists employed this theory of Shakhmatov against the Slovenes as part of magyarization of the Slovene March.
- Quelques remarques sur la brochure intitulée: La question du Prekmurje étudiée et présentée par M. Slavič. Par I. Melich et S. Mikola. Budapest – 1919.