Taylor was born in Livingstone Hospital, Dartford, Kent, England and he attended Dartford Grammar School. In July 1962, while he was at Sidcup Art College, the Rolling Stones was formed when Taylor, Mick Jagger and Keith Richards' three piece, Little Boy Blue and the Blue Boys joined Brian Jones and Ian Stewart's Rollin' Stones. Initially, Taylor played lead guitar in the band, but found himself pushed into bassist position to accommodate Jones, who quickly went on to become an accomplished slide guitarist. In November 1962, Taylor left to return to art college, and the band-mates posted an advertisement seeking another bassist. Bill Wyman responded, and succeeded him as The Rolling Stones' bassist in December 1962.
Later in 1963, Taylor formed The Pretty Things with vocalist Phil May and once again played his preferred guitar position. During a break from The Pretty Things after the release of their album, S.F. Sorrow (1968), Taylor produced Hawkwind's debut album, on which he also played guitar. His contribution to punk rock was a recording by Auntie Pus, and during the second half of the 1980s Taylor played guitar with the English post-punk band the Mekons. He recorded with Andre Williams in Chicago, Illinois for George Paulus' St. George Records. The core members of The Pretty Things remain in the band to this day, and released the Balboa Island album in 2007 on the Cote Basque record label.