The church was built in 1867 and the parish house completed in 1896. The church was built as a memorial to Samuel Colt who died in 1862 and three of his children who died in infancy. The parish house was built in 1895 as a memorial to Colt's son Caldwell Hart Colt who died in 1894. Potter had come out of retirement to design this building. The church and parish house were added to the National Register of Historic Places in 1975, and became a contributing property to the Coltsville Historic District in 2008.