Duffy joined the indie band Felt as keyboard player in 1985. Felt frontman Lawrence later recalled "I put up notices in Virgin (Records) in Birmingham advertising for a guitarist saying 'Do You Want To Be A Rock 'N' Roll Star?' I'd put two up when this guy came up to me and said, 'I know this keyboard player. He's 16. He's just left school. He's a genius."
He played piano on Primal Scream's first two albums. He joined the band permanently after Felt disbanded at the end of 1989.
Duffy also played with The Charlatans following the death of founder-member Rob Collins, appearing with the group at Knebworth supporting Oasis in 1996 and contributing keyboard parts to the bands fifth album Tellin' Stories along with Collins. He has also played on recordings by Heidi Berry when she recorded for Creation Records and he played on several Airstream singles.