The St. Francis Xavier Church was purchased on October 10, 1863 and dedicated on February 21, 1864. The founders of the church were Black Catholic refugees from San Domingo, along with the Sulpician Fathers. The Sulpician Fathers had fled France in 1790 as refugees of the French Revolution and were affiliated with St. Mary's Seminary. On July 11, 1791, six ships from the French fleet arrived at Fell's Point, Baltimore, bringing a large number of Black Roman Catholic refugees from Cape Francois in the French colony of San Domingo. There were between 500 and 1,000 Black refugees, both slaves and free. The majority of the free Black refugees were educated and wealthy. When the Black refugees first settled in Baltimore they attended Mass in the basement of St. Mary's Seminary. Both the Sulpicians and the Haitian refugees spoke French. In 1828, Haitian women refugees founded the Oblate Sisters of Providence, a Catholic religious institute for African-American women. The Oblate Sisters were the first all-Black order of Roman Catholic nuns.
St. Francis Xavier's remains an African American parish and is affiliated with both the Jesuits and the Josephites.