Bayley was born in 1745 in Fairfield, then in the Colony of Connecticut, to a family of French Hugenot descent, based in New Rochelle, New York. In 1766 he was apprenticed to New York City physician John Charlton. Bayley married John's sister, Catherine Charlton, at St. John's Episcopal Church (Elizabeth, New Jersey); her father was the rector of St. Andrew's Church, Staten Island. The couple had three children, including Elizabeth Ann Bayley. Later, known as Mother Seton, she became the foundress of the first group of Catholic Religious Sisters in the nation, and was the first native-born citizen of the United States to be declared a saint.
After three years of apprenticeship, Bayley went to London to continue his study of medicine. There he studied with an anatomist named William Hunter. Bayley studied there for three years, returning to America in 1772, where he opened a practice with Charlton, his brother-in-law and former instructor. He returned to England in 1775 to continue his work with Hunter.
A Loyalist, Bayley returned to America and enlisted in the British army as a surgeon at the start of the American Revolution and was stationed in Newport, Rhode Island. In 1777, however, the illness of his wife caused him to return to New York, where she soon died. He took no further part in the conflict. The following year he married Charlotte Barclay, with whom he had four children. They later separated.
Bayley's chief focus in his medical practice was the poor of the city. He helped to found the New York Dispensary, which operated in the Greenwich Village neighborhood well into the 20th century. He was the first American surgeon to amputate an arm at the shoulder successfully. In 1788 a mob attacked his laboratory during the "Doctors Mob Riots", after a rumor spread through the city about his alleged use of stolen human cadavers in his surgical classes. His anatomical collection was destroyed, but he escaped without injury. In 1792 he began to teach anatomy and surgery at Kings College of New York.
Due to an outbreak of yellow fever in the city, Bayley became interested in the disease, and began research on it, which helped to discover how it spread. As a result, around 1796, he was appointed as the first health officer of the Port of New York, in charge of a quarantine station at Tompkinsville, Staten Island (now St. George). He himself later contracted yellow fever from checking a ship just arrived from Ireland that was contaminated with it. He died from the disease on August 17, 1801, and was buried in the cemetery of the church served by his father-in-law.
Bayley Seton Hospital, once located in the Clifton section of Staten Island, was named for both Bayley and his daughter, Elizabeth. It was founded and run by the Sisters of Charity of New York, founded by her, until 2006.