Bayer was born and raised in Orange, Ohio, a suburb of Cleveland. She is the daughter of Carolyn (née Polster) and Todd Bayer. Her family is Jewish. Bayer graduated from Orange High School in 2000 and attended the University of Pennsylvania where she majored in communications and French, graduating in 2004. Her older brother Jonah is a music journalist and musician who has written for various music and guitar magazines, and has appeared as a music correspondent on Fuse TV, as well as being the guitarist for punk band United Nations. Bayer had leukemia, fighting it throughout her childhood after being diagnosed with the disease at a young age.
While attending college, Bayer interned on the television shows Sesame Street and Late Night with Conan O'Brien and participated in the cast of Bloomers, an all-female musical and sketch comedy troupe. Bayer was part of the all-Jewish cast of Second City's stage show Jewsical: The Musical, a musical that presented a comedic take on Jewish life and culture.
Bayer played the role of Kate Clark in the Chicago-based feature comedy Off the Cuff.
Bayer joined the cast of Saturday Night Live as a featured player on September 25, 2010. Beginning with the 2012–2013 season, Bayer was promoted to a repertory status. She has been praised for her performance on the show, with HitFix calling her "one of the strongest members of the ensemble", and Rolling Stone calling her "Saturday Night Live's secret weapon". In 2014, she was nominated for an American Comedy Award for her work on the show.