Louis Wolff married Alice Muscanto, a flute player born in Vilnius who played with her sisters and brothers in a touring musical ensemble. Louis was a concert-quality violinist who enjoyed accompanying his wife and her siblings in their apartment in Brookline, Massachusetts. Louis and Alice had two children, Lea (b. July 1, 1921; d. December 1, 2007), a French teacher for many years in Boston public schools, and Richard (born August 20, 1923; d. February 14, 2009), also a cardiologist. Louis remarried after Alice's death, to Phyllis Raftell-Wolff, and together they had two more children, Sarah (b. 1954), an elementary school teacher, and Charles (b. 1959), a physician.