It is a large three-story stone structure with modest Greek Revival styling. It has a central octagonal administrative section from which four wings of varying sizes radiate. The building was built out of slate quarried on site, with dressed dark granite corner quoining. The building was designed by architect Gridley J.F. Bryant and prison reform expert Rev. Louis Dwight. A similar design was used by Bryant for Boston's Charles Street Jail (built in 1848). The building was the fifth and final instance of an almshouse built by the city.