Levanon was born in Korets, Ukraine, the son of a haskalah scholar from Novohrad-Volynskyy (Samuel Bilansky). He was trained at the conservatory in Mykolaiv, Ukraine. After serving in the Red Army he emigrated in 1919 to Palestine, where he established himself as a composer and music teacher. His opus includes music for operettas, films (including the first Hebrew-language talking film, This is the Land [zot hi ha'aretz], 1935), instrumental pieces, and children's songs.
Yaacov Levanon, who taught in Arab schools in Jerusalem, was commissioned by the Grand Mufti of Jerusalem, Amin al-Husayni, to establish a children's orchestra to entertain, among other things, for the holiday Eid ul-Fitr. The relationship between them remained cordial until the 1929 Hebron massacre, in which the Mufti played a major role.