McGhee began his career in his native Ayrshire with Cronberry in 1878 and then Boswell Thistle two years later before moving to Edinburgh side Hibernian in 1883. Hibernian had been founded for the benefit of Edinburgh's immigrant Irish Catholic population but the club quickly became a beacon for the Irish-Catholic minority throughout Scotland. They were able to attract players from Irish communities across the country and with this abundant supply of talent they rapidly became one of the dominant sides in Scotland. McGhee was one such player and, as a free-scoring inside forward, he developed into a key part of the side. He is still Hibs record goalscorer against Hearts with at least 24 goals. He scored a remarkable 5 goals in the first 25 minutes in the Rosebery Charity Cup final of 1887, Hibs winning 7–1.
In 1887, McGhee captained Hibs to victory in the Scottish Cup, the first time a side from the east of Scotland had won the tournament. A year earlier, alongside teammate, James Lundie, he had become the first Hibs player to be capped by the Scottish national side. Hibs suffered a rapid decline when the foundation of Celtic in 1888 usurped their position as the standard-bearers of the Irish community. After a brief period with St Bernards, McGhee followed the example of many of his former Hibs team-mates by moving to Celtic. He later ended his playing career with short spells with Dykebar and Abercorn.
In 1908, McGhee was hired as manager of Hibernian's city rivals Hearts, the third person to hold the role at the Tynecastle club. His appointment was not popular with the Hearts' support because he had been a prolific scorer against their side in his playing days with Hibs. He had also been at the centre of an incident that resulted in crowd disturbance in a Rosebery Cup match between the sides in 1886, when his tackle broke the leg of Hearts opponent Peter Bell. McGhee left Hearts in December 1909 after he had suspended the club's star player Bobby Walker due to a breach in team discipline (missing a game). When Walker was reinstated by the board, McGhee and his assistant David Smart resigned.
McGhee emigrated to the United States of America in 1910, where he settled in Philadelphia, sending for his family two years later. His son Bart would later play football for the United States as an Outside Left, earning selection in their squad for the 1930 World Cup and scoring their first goal at the finals.
- "Bart McGhee spotlight". US Soccer Hall of Fame. Retrieved 2007-04-20.
- "Hibs 7-1 Hearts 1". londonhearts.com. Retrieved 2007-06-23.
- Memorabilia mixes with poignant tales in new Easter Road exhibit, The Scotsman, May 31, 2008
- "Hearts 0-2 Hibs". londonhearts.com. Retrieved 2007-04-20.
- Speed, David (1984). The Heart of Midlothian Football Club: A Pictorial History, 1874-1984. Heart of Midlothian. p. 44. ISBN 0-9510124-1-X.