He studied theology at the seminary in Saint-Dié-des-Vosges, later being named professor of botany at the Catholic University of Lille (1875). The Jardin botanique Nicolas Boulay at the university is named in his honor.
He was the author of numerous books and articles in the fields of bryology and paleobotany. The fossil pteridosperm genus Boulayatheca bears his name, as does the bryophyte genus Boulaya, being circumscribed by Jules Cardot in 1912. In addition to bryological/paleobotanioal research, he performed taxonomic work involving the genus Rubus.
He made contributions to the multi-volume "Flore de France; ou, Description des plantes qui croissent spontanément en France, en Corse et en Alsace-Lorraine" (Flora of France, description of plants native to France, Corsica and Alsace-Lorraine, 1893-1913). Some of his other principal works are as follows:
- Notice sur la géographie botanique des environs de Saint-Dié (Vosges), 1866 – Phytogeography associated with the environs of Saint-Dié.
- Muscinées (mousses, sphaignes, hépatiques), 1872 – Mosses, sphagnum and hepatics.
- Le terrain houiller du Nord de la France et ses végétaux fossiles, 1876 – The coal terrain of Nord and its fossil plants.
- Muscinées de la France, 1884 – Mosses native to France.
- Flore pliocéne du Mont-Dore, Puy de Dôme, 1892 – Pliocene flora of Mont-Dore, Puy de Dôme.