Taylor was born in Grand Prairie, Texas was the second of seven children—four girls and three boys. He was raised by his mother Myrtle and step father James Stevenson. Myrtle was a domestic, a chef, a butcher, and an owner of a restaurant and Stevenson constructed parts for airplanes.
Taylor started playing sports while in junior high school and by the eighth grade he was playing football, baseball, basketball and competing in track.
Taylor attended and played high school football at Dalworth High School (the former segregated black high school, now an elementary school by the name of David Daniels. Dalworth High students moved to Grand Prairie High School). He also ran the high hurdles, threw the discus and shot put, and competed in the long jump for the track team. The school did not have a baseball team, but Taylor played baseball in a summer league. He was named All-State in track and football.
Although known as a successful running back, Taylor was switched to wide receiver in 1966 and led the NFL in receiving in both 1966 and 1967. He would play that position for the rest of his career and had a record-tying seven seasons with 50 or more receptions. On December 21, 1975, Taylor became the NFL's all-time receptions leader with his 634th career catch in the season finale against the Philadelphia Eagles.
Taylor retired after the 1977 season as the NFL's all time leading receiver with 649 receptions for 9,110 yards and 79 touchdowns. With 1,488 yards rushing and some kick return yardage, he totaled 10,803 combined net yards. With 11 rushing touchdowns and 79 on receptions, Taylor scored 540 points in his career. He earned first- or second-team All-NFL honors six times and was selected to play in eight Pro Bowls.
After he left coaching, Taylor worked for Jerry’s Ford in the Washington Metropolitan Area and sold boats for Fountain Boats in Annapolis, Maryland. He now does speaking engagements and serves as a consultant to the Redskins. In addition, the Charley Taylor Recreation Center in his native Grand Prairie, Texas has his namesake.
Taylor and his wife Patricia have been married since 1965. They have three children, Elizabeth, Erica, and Charles, Jr., and three grandchildren, Nathan, Jordyn, and Robyn. Taylor and his wife live in Reston, Virginia.