Jones joined Wrexham in 1971. He made his debut at the age of 17 in a Welsh Cup tie against local rivals Chester City; Wrexham lost 1–0. He did, however, win the Welsh Cup with the club in 1975, when they beat Cardiff City in the final. Jones established himself as a right-back and helped Wrexham to the quarter-finals of the FA Cup in 1974, the first time the club had reached that stage.
In 1977 left back Jones was part of the treble-chasing Liverpool team which won the League championship and reached the finals of the FA Cup and European Cup. Jones scored his first goal for the club on 9 November 1976 in the 5–1 league thrashing of Leicester City at Anfield. The treble, unprecedented in English football, was not forthcoming. Liverpool lost 2–1 in the FA Cup final at Wembley to Manchester United, though Jones supplied the accurate long pass for Jimmy Case to score Liverpool's goal. However, Jones became the first Welshman to receive a European Cup winners medal when Liverpool won their first European Cup in Rome four days later, defeating Borussia Mönchengladbach 3–1. A memorable banner was unfurled by Liverpool supporters at the European Cup final in Rome which said "Joey Ate The Frogs Legs, Made The Swiss Roll, Now He's Munching Gladbach".
Jones was in and out of the side the following year, with the renaissance of Tommy Smith and the emergence of young Scottish defender Alan Hansen severely reducing his first team opportunities to the extent that he left in the summer of 1978 after exactly 100 appearances and 3 goals for the Reds.
In 1982, Jones joined Chelsea for £34,000, having been signed by John Neal. Jones' was sent-off on his debut against Carlisle United at Brunton Park. However, Jones' committed attitude and pre-match fist-clenching ritual eventually made him a cult hero among the fans. He also proved instrumental in Chelsea's successful battle to avoid relegation to the Third Division. He was a part of the side which romped to promotion as Second Division champions in 1983–84. He remained with the club in the top flight for one more season, before surprisingly being sold to Huddersfield Town for £35,000 in August 1985. He finished his Chelsea career with 78 league appearances and 2 goals.
He joined Huddersfield in the summer of 1985 from Chelsea, and was named Town's player of the year in his first season. After two seasons he left to re-join Wrexham,  where he retired at the end of the 1991-92 season.
Jones underwent heart surgery in 2002 and has since scaled down his commitments with Wrexham, he works as the Under 18's and Reserve team coach. In 2001 he had a brief spell as caretaker manager between the departure of Brian Flynn and arrival of Denis Smith.
In 2005, Jones completed his autobiography entitled "Oh Joey, Joey!" about his life in football. This was a book of the week on Sky Sports News in February 2006.