Harvey signed for his home town club Grimsby Town as a player before World War II, but never made the first team. After the end of hostilities, he decided to concentrate on coaching. His first managing appointment came in 1962, when he was made manager of Bedfordshire outfit Luton Town. In his first season Luton were relegated from Division Two, and Harvey resigned in November 1964. He returned to coaching, working at Swindon Town and Bristol City. Harvey returned to Grimsby in 1968, but with Harvey in charge the team were first relegated and then finished in the re-election spots of Division Four. Grimsby failed to be re-elected to the Football League. Harvey resigned after a year in charge, and never managed again.