Scheiber was born in Budapest into a rabbinical family from both paternal and maternal sides. He was ordained at the Seminary in 1938 as a student of Bernát Heller. After studies in London, Oxford and Cambridge, where he discovered many genizah fragments while analyzing medieval Hebrew manuscripts, he served as rabbi in Dunaföldvár from 1941 to 1944. In 1945, he became a professor at the rabbinical seminary and was its director from 1950 until his death. This institution retained its international fame throughout the Communist era, when it was the only place in the Eastern bloc where rabbis would be graduated for serving in Hungary and abroad. Furthermore, Scheiber joined the faculty of the University of Szeged in 1949, teaching oriental folklore.
He considered it his mission to explore the Hungarian Jewish past and perpetuate its memory, as well as to publish the contributions of great Hungarian-Jewish scholars, including the works of Wilhelm Bacher, Fauna und Mineralien der Juden by Immanuel Löw (1969) and the diary (Tagebuch) of Ignác Goldziher (1978). In 1957, he published a facsimile of the so-called Kaufmann Haggadah, called after David Kaufmann, MS 422 of the Kaufmann Collection in the Hungarian Academy of Sciences
He died on 3 March 1985, in Budapest. Each year, the Sándor Scheiber Prize is awarded by the Hungarian Ministry of Culture on 3 March, the anniversary of his death. Among the laureates is the biochemist Máté Hidvégi.