During his time at Ferguson Park, McGlynn started coaching youth sides and gradually coaching rather than playing became his primary function. He was subsequently appointed co-manager at Easthouses Lily before returning to Musselburgh as manager in his own right.
In 1996, McGlynn was invited to assist with coaching the youth sides at Heart of Midlothian by newly appointed manager Jim Jefferies, initially on a part-time basis. Within two years he had earned a permanent position, eventually working his way up to the position of first-team coach.
McGlynn won his first manager of the month award as a Division One manager in March 2010. McGlynn led Raith to a comfortable 7th place and took the club into their first Scottish Cup semi-final for 47 years, beating Peterhead, Airdrie, Aberdeen and Dundee. Raith were eliminated by eventual Cup winners Dundee United.
McGlynn was appointed manager of Hearts on 26 June 2012. With the restricted budget imposed by the club's financial situation he was only able to bring in two new signings and he put his faith in many young players coming through the youth ranks.
Hearts started the SPL season badly under McGlynn, with only one win in the first six games, and lost in the Europa League play-off against Liverpool 2–1 on aggregate, drawing at Anfield. In the League Cup they defeated Livingston and Dundee United to reach the semi-final, but they lost 1–0 to city rivals Hibernian in the 4th round of the Scottish Cup. Hearts progressed to the 2013 Scottish League Cup Final, but poor league results led to McGlynn departing by mutual consent on 28 February 2013.
On 12 September 2013, McGlynn was appointed manager of Livingston, replacing the departing Richie Burke. He was initially given a contract until the end of the 2013–14 season. After overseeing good results that improved Livingston's league position, McGlynn agreed an extended rolling contract with the club.