Gershevitch was born to Russian parents fleeing from Germany to Switzerland at the outbreak of World War I. He enrolled in the University of Rome in 1933, and moved to England in 1938. In 1948, he became the first holder of a new Lectureship in Iranian Studies at Cambridge University. He became a Fellow of the British Academy in 1967 and later a corresponding member of both the Accademia dei Lincei and the Russian Academy. In 1971 he received an honorary doctorate from the University of Berne.
His work includes pioneering studies of the Bashkardi dialect, the decipherment of Bactrian, besides contributions to Sogdian and Avestan philology, Ossetic, Elamite and Zoroastrian studies and Achaemenid history.