Her mother, Clara Guinzburg Pollitzer (born Clara Guinzburg), was the daughter of an immigrant rabbi from Prague. Her father, Gustave Pollitzer, ran a cotton company at Charleston, South Carolina. Anita Pollitzer had two sisters, Carrie (born 1881) and Mabel (born 1885) and a brother, Richard. In December 1928, she married Elie Edson, Pete Seeger's uncle. The couple moved to New York City and lived in an apartment on West 115th St.
Pollitzer may be best known for her friendship with Georgia O'Keeffe whom she met at Columbia University. Pollitzer introduced O'Keeffe to Alfred Stieglitz helping to forge one of the most significant artist relationships in the 20th century. In addition to her influence in art, Pollitzer was instrumental in the passage of the 19th Amendment and held positions of leadership in the National Woman's Party serving as National Chairman from 1945 until 1949.