He was born in Memphis, Tennessee, the son of Everett and Estelle Axton. Estelle Axton and her brother, Jim Stewart, were the founders of Stax Records. By 1959, Packy Axton had become a member of the Royal Spades, a group formed by Steve Cropper, Donald "Duck" Dunn, Charlie Freeman and Terry Johnson, which expanded to include a horn section composed of Axton, Don Nix and Wayne Jackson. In 1961, they renamed themselves the Mar-Keys and had a major hit with "Last Night" (number 3 on the Billboard Hot 100, number 2 on the R&B chart). Axton and Jackson were the only members of the band to perform on the record; the other parts were played by session musicians. After Cropper left the band in 1961, soon followed by Dunn, Axton became the effective leader of the Mar-Keys and also worked as a session musician at Stax. He left in 1965 to live in Los Angeles, reportedly after a series of disagreements with Jim Stewart.
Later in 1965, the Stax Revue performed in Los Angeles, and radio disc jockey Magnificent Montague persuaded Axton to record there with Cropper, Booker T. Jones and Al Jackson, of Booker T. & the M.G.'s. They recorded an instrumental track together, "Hole in the Wall", which Montague then released as by the Packers; it reached number 43 on the pop chart and number 5 on the R&B chart. Axton formed a pickup group to promote the record. He later returned to Memphis, where he recorded several singles credited to the Pac-Keys, mostly featuring members of the Bar-Kays. After 1967, he ran the Satellite Record Shop in Memphis, occasionally performing with musicians such as Charlie Rich.