Born John Lewis Carrell (Carrell was printed incorrectly as Carroll on the record label), Carroll began recording for Decca Records in the middle of the 1950s. He released several singles, but none of them saw significant success, though they are now critically acclaimed. His records were eclipsed by the success of other rockabilly and early rock & roll musicians such as Elvis Presley, Jerry Lee Lewis, and Johnny Cash.
His career ended toward the end of the 1950s, but he made a comeback in 1974 with a Gene Vincent tribute song. He continued to record well into the 1980s. For many years he was connected with the Cellar Club in Fort Worth, Texas and other Cellar Clubs around the state. He died of liver failure on January 13, 1995, and is buried in his hometown of Godley, Texas. In 1996 a 33-track reissue of his early recordings was released as Rock Baby Rock It: 1955-1960.