Ball attended the Duncan of Jordanstone College of Art in Dundee from 1962, studying architecture. While there he met Malcolm "Molly" Duncan and Alan Gorrie. The three of them relocated to London separately, but joined forces again in 1971 to form the nucleus of the Average White Band (AWB).
They were later joined by Onnie McIntyre, Hamish Stuart and Robbie McIntosh, completing the original line-up. These six Scots were an unlikely group to be successful playing American styled funk, but went on to be nominated for three Grammy Awards in the Rhythm & Blues category. Ball was the principal composer of "Pick Up the Pieces" which topped the US Billboard chart on 22 February 1975. It was written from a rehearsal "jam" over which he superimposed the melody line and the bridge. Since then it has been covered by many musicians and used in television programmes, films and commercials. He co-wrote a total of forty three songs for the Average White Band.
Before forming AWB in 1971, Ball was a session musician in London, arranging and playing for Vinegar Joe, Badfinger, Kiki Dee and Elton John, Mama Cass, Bryan Ferry, Roxy Music and others. He has played on stage with Chaka Khan and Marvin Gaye. As an arranger with AWB he wrote and played with Michael and Randy Brecker.
In 1995 he released his first solo album, Street Struttin' and, in 2005, his second CD, Childsplay. He cites his influences on saxophone as Johnny Hodges, John Coltrane and Cannonball Adderley and, as an arranger, Oliver Nelson.