Cooley was born in Peterswell, County Galway in 1924. Both his parents were melodeon players, and Joe began playing accordion at age 10. As a teen, Joe played in the Midlands area and eventually found himself in Dublin in 1945, where he joined the Galway Rovers Band. There in Dublin, he met musicians Sonny Brogan and Johnny Doan, both of whom were to influence Cooley's musical style.
He was one of the earliest members of the Tulla Céilí Band when, as the St Patrick's Amateur Band, Tulla, they won the ceili band competition at Féile Luimní in 1946. He played with the Tulla on their first broadcast for Radio Eireann in 1948. At the end of that year he left the band to work in London. He rejoined the band when he returned from England towards the end of 1950. He also often played with Galway fiddler Joe Leary.
In 1954 Joe Cooley left for the US. Joe's brother, Seamus, who played banjo with the Tulla, went on a US trip and made a recording with them. He left the band in 1958 while on tour and stayed in the US. While in New York Joe was involved in the Joe Cooley Ceili Band and the Joe Cooley Instrumental Group. He moved from New York to Chicago and finally to San Francisco. In America he married Nancy McMahon from Killenana, Co Clare.
He is regarded as one of the most accomplished and influential Irish button accordionists. On one occasion, in Miltown Malbay, Co. Clare, Joe was presented with a beautiful Paolo Soprani C#/D accordion he dubbed "The Box" which would accompany him on his tour of America in the 1970s.
Joe returned to Ireland in the spring of 1973 after becoming ill with cancer. He toured the pubs of Clare and Galway with his friend banjo player Des Mulkere until shortly before his death in December 1973. His only commercial recordings were not released in Ireland until 1975.
His accordion is now believed to be in the hands of the great Irish accordionist Tony MacMahon of Clare, IE. One of Cooley's enduring legacies is a reel named "Joe Cooley's". Originally named "The Tulla Reel," the tune has since become associated with Cooley and is now a standard in the Irish traditional music repertoire.
- Gearóid Ó hAllmhuráin, "Tribute to Joe Cooley", Clare Champion, January, 1974
- Gearóid Ó hAllmhuráin (1998). A pocket history of Irish traditional music. O'Brien Press. p. 136.
- "Joe Cooley's"