The Scribner House was built by Joel Scribner, one of the three brothers who founded New Albany. He and his brothers, Nathaniel and Abner, came from New York State and named their new town "New Albany" after the capital of their home state. It is located in downtown New Albany, on the southeast corner of State and Main Streets near the Sherman Minton Bridge. It is the oldest building in New Albany.
Joel built the house in 1814. It was made from ash, oak, and poplar trees were chopped down when clearing the property, built in a New England Federal style. In total, it is 31⁄2 stories. The first floor features two parlors and a hall. The second floor has 3 bedrooms. Both these floors have a rear porch that allowed a view of the Ohio River. The ground floor is the large kitchen with a wide fireplace used for cooking.
Twice a year open house events are held, the third Saturday in May and the first Sunday in December. The December open house includes a Victorian tea. From May through September, the house is open for tours on the second and fourth Saturday of the month from 1-3 pm. Other tours can be made by appointment. All third grade children in Floyd County also visit Scribner House during the school year.