Kaye joined West Bromwich Albion from Scunthorpe in May 1963 for £44,750, a club record for Albion at the time. He made his debut for the Baggies against Leicester City in August 1963. He managed Hull City for a time period of September 1974 – October 1977. He had a win percentage of 31.74%. He scored for West Brom in the second leg of their 1966 League Cup Final triumph over West Ham.