Murphy had a brief career in semi-professional football for several non-league clubs in the early 1970s. These included Gravesend & Northfleet, Folkestone Town and Hastings United. However he failed to break through into professional football. Turning to coaching, Murphy began coaching at Nottingham Forest and Notts. County. He was then quickly catapulted into the spotlight when Derby County surprisingly appointed him as their manager in November 1976, having been reserve team coach under Championship winning Dave MacKay. However, he lasted only ten months in the role before being sacked in September 1977 having led the Rams to just seven wins in 35 games. He was replaced by Tommy Docherty.
After a brief break from football, Murphy took charge of a struggling Lincoln City side that was beset by financial problems and had just completed a lacklustre campaign in Division Four. In arguably his most successful period in management, Murphy secured Lincoln’s promotion to Division Three in the 1980–81 season with a second-place finish behind Southend United, thanks in large measure to Gordon Hobson's 21 goals. The following season was almost as successful, with the Imps knocking out First Division side Leicester City in the League Cup, eventually losing to Spurs, and leading the Division Three table for four months. A loss of form, possibly in part caused by a fierce boardroom row over transfer budgets, resulted in Lincoln throwing away their advantage and finishing in a disappointing sixth place.
The next two seasons were less impressive and Murphy eventually resigned in May 1985, after seven years at Sincil Bank. His departure came just 10 days before Lincoln played Bradford on the final day of the season in the game where 56 spectators died in the Bradford City disaster. The Murphy era is remembered fondly by the Imps faithful, with players such as Steve Thompson, Glenn Cockerill, Trevor Peake, Mick Harford, Tony Cunningham, Dave Felgate and John Fashanu (signed by Murphy) contributing to one of the finest teams in Lincoln's history.
While manager at Lincoln Murphy gained cult status for his notes in the club's match day programme. His column demonstrated a level of inventiveness with the English language and surrealist imagery that marked it out from any other offering in this medium. Lincoln programmes from this era are now collector's items far beyond what would be considered the norm for the publications of lower division and Conference clubs from this era.
Murphy quickly returned to management on 8 August 1985 as manager of Stockport County, but left the club on 24 October after a disappointing start to their Division Four campaign. He joined Al Ittihad in Jeddah as part of the coaching staff under Bob Houghton, manager of Malmo in the 1979 European Cup Final. However, in a bizarre U-turn, Murphy took charge of Stockport again in November 1986. In another ironic twist, Murphy masterminded the club’s dramatic escape from relegation to the Conference at the expense of Lincoln City, who were relegated in their place in the very last minutes of the season.
Just a few weeks after the end of the 1986-87 season, Murphy resigned as Stockport manager and began a second spell at Lincoln City on 26 May 1987. Over the next three years he was able to both guide Lincoln City back to league football in his first season and secure them two tenth-place finishes in Division Four. Despite this renewed success, Murphy left the club by mutual consent on 20 May 1990 and spent the next two years coaching at Leicester City FC with their youth team.
Murphy returned to football as manager of Southend United on 8 May 1992, but left just short of the end of a disappointing season on 1 April 1993, taking the post as Director of Football and being replaced by Barry Fry. Notably, he signed Stan Collymore from Crystal Palace FC for £150,000, and sold him for £2.65 million to Nottingham Forest. His next post in management was at the League of Ireland side Shelbourne from December 1994 to May 1995. He took the club from the edge of the relegation zone to a final day finish that left Shelbourne just a few points short of the title. The club were also the beaten finalists in that year's FAI Cup.
He then left to manage Notts County on 5 June 1995. Although the Magpies narrowly missed out on promotion to Division One (Championship) in his first season in charge after a 2–0 defeat to Bradford City in the playoff final at Wembley, the 1996-97 season was a disaster for Murphy and the club, with a series of heavy defeats that eventually led to his sacking on 23 December 1996.
Murphy joined the Tigers in 2002 as Assistant Manager to Peter Taylor.
With the arrival of Phil Parkinson as Manager with Frank Barlow as his Assistant at the start of the 2006-07 season, Murphy took up the new role of Director of Development at the club - responsible for all scouting and youth development. He reverted to Assistant Manager, however, when Barlow left the club towards the end of October. After Phil Parkinson also left the club, Murphy and First-team coach Phil Brown were appointed as joint caretaker managers. Phil Brown was later appointed as permanent manager, with Murphy reverting to his assistant role for the remainder of the season. With the appointment of Brian Horton, he resumed the role of Director of Development.
Murphy suffered a stroke in November 2007, and retired from his role at Hull City during the 2008-09 season.