Davis joined the 5th Dimension, then called the Versatiles, in 1966. The group's first big hit was with 1967's "Up, Up and Away", written by Jimmy Webb. The song won four 1968 Grammy Awards and was the title track to the 5th Dimension's first hit LP. A year later the group recorded Laura Nyro's "Stoned Soul Picnic". A medley of "Aquarius/Let the Sunshine In" (from the musical Hair) reached No. 1 on the Billboard Hot 100 chart in April to May 1969 and won the Grammy for Record of the Year. The group's recording of Nyro's "Wedding Bell Blues" topped the Hot 100 in November 1969. Davis sang the male lead on the group's singles, "Worst That Could Happen", "A Change Is Gonna Come/People Got To Be Free", and "I’ll Be Lovin' You Forever".
In 1975, Davis and McCoo left the 5th Dimension and began performing as a duo. Landing a contract with ABC Records, they recorded their 1976 debut album, I Hope We Get to Love in Time. The first single was the title track, which was a mid-chart hit. Their follow up, "You Don't Have to Be a Star (To Be in My Show)", was an even bigger hit, reaching No. 1 on the Billboard Hot 100 in January 1977. Davis and McCoo were awarded a gold single and a gold album as well as a Grammy Award for Best R&B Performance by a Duo or Group with Vocals. They became the first African American married couple to host a network television program, The Marilyn McCoo & Billy Davis Jr. Show, on CBS in Summer 1977. They released one more album on ABC in 1978, produced by Frank Wilson and containing the popular ballad, "My Reason To Be" by songwriters Judy Wieder and John Footman. The pair signed with CBS Records the following year and released their last album as a duo until October 2008 when the pair released The Many Faces of Love, a collection of hit songs from the 1960s and 1970s.
The album Marilyn and Billy featured the track "Saving All My Love for You", later sung by Whitney Houston, as well as a disco hit, "Shine On Silver Moon". The pair decided to go solo professionally in the early 1980s.
In 1982 Davis recorded a gospel album, Let Me Have A Dream, with Rev. James Cleveland. Davis followed up that project with a guest appearance on a jazz/pop album by Scott Scheer.
Davis starred in the musical "Blues in the Night" at the Old Globe Theatre. In the role of James "Thunder" Early, he also starred in the North Carolina Theatre production of "Dreamgirls". Kay McLain, of the Durham Herald-Sun, wrote, "Davis made an endearing character of Early…give him a microphone and the spotlight and he’s a pro." In his later guest appearances on the WB’s Jamie Foxx Show, he and McCoo played the affluent parents of Foxx's girlfriend, Fancy. Over the years, Davis has made frequent appearances on most of the national morning and night time talk and variety shows.