Duffy joined the indie band Felt as keyboard player in 1985. Felt frontman Lawrence later recalled "I put up notices in Virgin (Records) in Birmingham advertising for a guitarist saying 'Do You Want To Be A Rock 'N' Roll Star?' I'd put two up when this guy came up to me and said, 'I know this keyboard player. He's 16. He's just left school. He's a genius."
In 1989, Duffy played piano on Primal Scream's second album. He joined the band permanently after Felt disbanded at the end of the year.
Duffy also played with The Charlatans following the death of founder-member Rob Collins, appearing with the group at Knebworth supporting Oasis in 1996 and contributing keyboard parts to the bands fifth album Tellin' Stories along with Collins. He has also played on recordings by Heidi Berry when she recorded for Creation Records and he played on several Airstream singles.