His first management job was at Fourth DivisionMansfield Town in 1974 where re-election was just avoided. The following season was a triumph, Mansfield taking the title by six points without losing a game at home.
The next job was at Southend United where in his second season promotion from the Fourth Division was achieved once again, this time as runners up to Watford in 1978. Two seasons of struggle followed resulting in relegation. Once again promotion was achieved at the first attempt, this time as champions, with nine club records broken in the process. Third Division safety was then achieved and Smith's job was assured until a takeover left him out.
Sixteen months followed out of football selling insurance and applying for every vacant manager's position. Smith then took over at Third Division Plymouth Argyle in December 1984 from Johnny Hore and fifteenth position was achieved in his first part-season. The following year promotion was achieved to the Second Division as runners up to Reading. In 1987 Smith took Plymouth to 7th position in the second tier of English football, narrowly missing out on the end of season play-offs. This was one of the club's highest ever finishing positions and was achieved on a modest playing budget.
Smith's legendary status amongst Argyle fans was helped by his colourful personality, which included using poetry to describe his team's performance and wearing his tartan cap for good luck.
The summer of 1988 resulted in another move, back to Scotland and his home town club Dundee.
In 1989, Smith returned to the West Country enjoying a successful spell as manager of Torquay United before resigning on 2 April 1991 and retiring from management. Smith was reportedly approached by Plymouth Argyle in 1997 to return to the club as manager, but could not be tempted out of retirement.
However, he remained involved in football and ran highly popular soccer schools in the Plymouth area. Smith is featured on one of the many murals within Plymouth Argyle's Home Park stadium, reflecting his status as one of the club's most popular and successful managers.