Diallo was born in Guinea-Conakry and when young moved about the country following the job moves of his father, a surgeon. A portion of his mother's family lived in Senegal, which gave him a rich musical education. In Africa he taught himself to play the guitar, and worked with several Guinean groups such as The Sons of Ra'is and Syli Authentique.
Diallo moved to Europe in the mid-1980s, where he worked particularly with the Fatala group associated with Peter Gabriel's music label.
In 1991 Diallo moved to Vancouver, and in 1993 released his solo first album, Nene, which was nominated for a Juno award, as was his 1996 album Futur. He won his first Juno, for The Message in 1999 in the "Best World Music Album" category. He received further Junos in 2002 (for The Journey) and in 2004 with the African Guitar Summit compilation. He released Djama in 2005.