He was the seventh son of Thomas Butterworth Bayley, of Hope Hall, near Manchester, where he was born 6 December 1777. His mother was Mary, only child of Vincent Leggatt. Bayley was educated at the grammar school of Winwick in Lancashire, and at Eton College, which he entered in May 1789, and left 9 December 1795. At Eton he was the associate of Sir William Pepys, Henry Hallam, William Frere, William Herbert, and others, who were known as the "literati"; and he contributed to the Musæ Etonenses. He came into residence at Trinity College, Cambridge, in April 1796. In February 1798 he obtained a university scholarship. In April he was elected a scholar of Trinity. He took his B.A. degree in 1800, and won the bachelor's prizes in 1801 and 1802. Richard Porson pronounced him the first Greek scholar of his standing in England, and in 1802 he was elected a Fellow of his college.
In 1803 he was ordained by Henry Majendie, Bishop of Chester, who appointed him his chaplain. Then he accepted the tutorship of Bishop George Pretyman Tomline's eldest son William Edward; and was appointed examining chaplain to the bishop. Tomline preferred him to the rectory of Stilton, Huntingdonshire, and to the sub-deanery of Lincoln, vacant by the death of William Paley in May 1805. He made improvements in the minster, worked to open the minster library to the public, and took part in the establishment of a public library in Lincoln. In 1810 he was presented to the united vicarages of Messingham and Bottesford, where he renovated the parish church, mostly at his own expense; and in 1812 to the vicarage of Great Carlton, near Louth, which he rarely visited, although he retained the benefice till his death. Later he was preferred to the archdeaconry of Stow with the prebend of Liddington (29 September) 1823); to the rectory of Westmeon with Privet, in Hampshire (1826); and to the twelfth stall in Westminster Abbey (1828), when he resigned his subdeanery and canonry at Lincoln. In 1824 Bayley proceeded to his degree of D.D. at Cambridge.
In 1827, in bad health, he declined to stand for the Regius Professorship of Divinity at Cambridge. His last days were passed chiefly at Westmeon, his Hampshire rectory. He repaired the church of the hamlet of Privet, and the rebuilding of the church of Westmeon was started 9 August 1843. When blind he recited the prayers from memory. He died 12 August 1844. He was buried in the same vault with his wife, who had died at Westmeon 17 June 1839, and the new church was consecrated by the Bishop of Winchester on 5 May 1846.
On 25 September 1803 he published ‘A Sermon preached at an Ordination held in the Cathedral Church of Chester,’ Manchester, 1803. This is the only printed sermon of the author in existence. In May 1826 he delivered a charge to the clergy of the archdeaconry of Stow, which was printed at Gainsborough in 1826 for private circulation, was reprinted in the following year, and is attached to the Memoir of Henry Vincent Bayley, D.D., which was printed for private circulation in 1846. In 1843 he became unable to write or read, and abandoned schemes for a new edition of Thomas Secker's ‘Eight Charges,’ and for a selection from the old and new versions of the Psalms of David.