The name Clewer comes from the word Clifwara meaning "cliff-dwellers", and is named after those who lived below the hill on which Windsor Castle was built. Historically, Clewer pre-dates New Windsor and still exists as a separate ecclesiastical parish. A Saxon settlement existed there, and it is thought that the settlement of Clewer may have grown up at a place where the river Thames could be forded. A wood-and-thatch Saxon church is believed to have existed on the site of the present church. The surviving font is thought to be Saxon, and is presumed to have belonged to the earlier church. Until the 1850s this font was in an improbable position at the west end of the north aisle and it is likely that it had never been moved from its position in the earlier Saxon church.
By the time of the Norman Conquest, there was a Manor of Clewer, mentioned in the Domesday Book as Clivore and recorded as having a church and mill. It was from here that William I took the lands on which he built his fort, which became Windsor Castle. The Manor of Clewer continued to receive a rent of 12 shillings per annum from the Crown for this land until the 16th century. The present St Andrew's Church is of Norman construction and it is traditionally believed that William I habitually attended mass there, as there was no chapel within the original castle. It has a 14th-century chantry chapel to the memory of the second wife of the hero of the Hundred Years' War, Sir Bernard Brocas. The family lived in the sub-manor of Clewer Brocas until rebellious activities obliged them to retreat to obscurity at Beaurepaire in Sherborne St John.