Born to an English Roman Catholic family, Joseph Gillow was educated at St Cuthbert's College, Ushaw. In 1878 he married Ella McKenna, a brewer's daughter, securing him a private income allowing him to pursue his antiquarian interests.
He published various researches into the history of Catholicism in Lancashire, but his greatest achievement was the Biographical and Bibliographical Dictionary of the English Catholics (5 vols, 1885-1902). To fit his material into the five volumes allotted him by his publishers, Gillow needed to abbreviate the later volumes. No index was available until 1986.
Gillow was appointed honorary recorder of the Catholic Record Society at its foundation in 1904, and was a frequent contributor to its transactions.