Historically Cadmore End was in two parishes and two counties. Part was a detached part of the parish of Lewknor in Oxfordshire, part of a division of the parish known as Lewknor Uphill consisting of three detached parts. Part of Cadmore End was a manor in the parish of Fingest in Buckinghamshire. In 1852 it became the separate ecclesiastical parish of Cadmore End, and in 1896 the Oxfordshire part of the parish was transferred to Buckinghamshire.
The parish church of St Mary le Moor was built in 1851, and is a Grade II listed building. It replaced the mediaeval chapel with the same dedication at Ackhampstead.