Originally form the central west town of Narromine, Gillespie joined Canterbury in their rebuilding after the 1980 premiership and was soon a success with his powerful tackling which earned him the well-known nickname "Cement". His defence was critical to Canterbury's retention of the title against a heavily favoured St. George side in 1985 after having scored a rare try in their powerful 26-nil win over Parramatta in the preliminary final.
The following year, despite not having played for New South Wales, Gillespie was favoured to tour with the Kangaroos before a horrible work injury which almost saw his finger severed ended his season. Then, 1987 saw two suspensions (in one of which he was, unusually, sent off on the first tackle of a match against Balmain) rob Gillespie of over a third of his season. He played for English side Hunslet during the 1987–88 Rugby Football League season. Returning to Australia "Cement" came back stronger in 1988 and played so well in the grand final that he was chosen for the World Cup final against New Zealand, but in 1989 he was so disappointing as to be briefly relegated to reserve grade before a four-week suspension for a dangerous tackle effectively ended his season.
In 1990 Gillespie finally established himself as a representative player with powerful displays for New South Wales and in the Test against France in Parkes. At the end of the 1990 NSWRL season, he went on the 1990 Kangaroo tour of Great Britain and France and played in all the Tests. He placed himself in the controversial draft and was chosen by Wests, for whom he was to play while an Australian regular. His high workrate was a vital part of two big wins against New Zealand in 1991 and against Great Britain during the 1992 Great Britain Lions tour of Australia and New Zealand, helping Australia retain The Ashes – though he was criticised at times for not being so committed at club level. 1993 saw Gillespie fall out with Warren Ryan and signed with Manly for 1994. However, the quicker 10-metre rule made his play – based on workrate – less effective and he lost in place in the NSW team after one match.
In 1995, despite being an integral part of a Manly team that threatened for a long time to march through the season undefeated – Gillespie was not considered too old and slow for the representative games until late in the year when he played for Australia against New Zealand. He played in Manly's 1996 premiership side but retired after their loss to Newcastle in the isolated ARL grand final the following year.