The library developed from the personal collection of Johannes Poliander, who donated it to the town council of Altstadt in 1541. J. Lomoller donated 300 predominantly legal works in 1594. It was housed in Altstadt's pauper house after its construction in 1628. To it was added the collection of vice-mayor Heinrich Bartsch (1627–1702). His son, city secretary Heinrich Bartsch Jr., created a librarian position in 1714, made the collection public in 1718, and added his own Bible collection. The first librarians were Johann Jakob Quandt and Gottlieb Siegfried Bayer.
By the second half of the 19th century, the public library was disorganized and in financial difficulties. It moved to the old campus of the University of Königsberg in Kneiphof in 1875 and was administered by August Wittich from 1875 to 1897. The councilor August Wilhelm Hensche donated his private collection in 1889. When the magistrate considered closing the library, Wittich focused on reorganizing the library and hiring a chief librarian, Ernst Seraphim. Seraphim energetically developed the library after Wittich's 1897 death and had the library rebuilt. Rudolf Reicke donated 2,150 volumes to the library in 1907. Two years later Seraphim and the library began publishing seven volumes of scientific publications with the title Mitteilungen.