Sherrill began her career with Duke Ellington in 1942, aged 17, later becoming a member of his orchestra from 1944 to 1946. She had a hit with Ellington's tune "I'm Beginning to See the Light". Subsequently, she worked as a soloist, performing with Rex Stewart, Ray Nance, and others into the 1960s. She returned to Ellington for 1959's A Drum Is a Woman. She toured the U.S. in 1959 and then took a role in the Broadway show The Long Dream. She toured with Benny Goodman in the USSR in 1962 and then returned to sing with Ellington in 1963.
From 1970 to 1982 she had a children's television show, Time for Joya, later called Joya's Fun School. Later in the 1980s she hosted a children's show in the Middle East.
She died of complications from leukemia in 2010, aged 85. She was survived by a son, a daughter, a sister and two grandchildren.