He was born in Memphis, Tennessee, the son of Everett and Estelle Axton. Estelle Axton and her brother Jim Stewart were the founders of Stax Records. By 1959, Packy Axton had become a member of The Royal Spades, a group formed by Steve Cropper, Donald "Duck" Dunn, Charlie Freeman and Terry Johnson, which expanded to include a horn section comprising Axton, Don Nix and Wayne Jackson. In 1961, they renamed themselves The Mar-Keys, and had a major hit (# 3 on the Billboard Hot 100, # 2 on the R&B chart) with "Last Night". In fact, only Axton and Jackson from the band appeared on the record which was mostly performed by session musicians. After Cropper left the band in 1961, soon followed by Dunn, Axton became the effective leader of the Mar-Keys, and also worked as a session musician at Stax. He left in 1965 to live in Los Angeles, reportedly after a series of disagreements with Jim Stewart.
Later in 1965, the Stax Revue performed in Los Angeles, and radio disc jockey Magnificent Montague persuaded Axton to record there with Cropper, Booker T. Jones and Al Jackson, of Booker T. & the M.G.'s. They recorded an instrumental track together, "Hole In The Wall", which Montague then released as by The Packers; it reached # 43 on the pop chart and # 5 on the R&B chart. Axton then formed a pick-up group to promote the record, later returning to Memphis. There, he recorded several singles credited to The Pac-Keys, which mostly featured members of The Bar-Kays. After 1967, he ran the Satellite Record Shop in Memphis, occasionally performing with musicians such as Charlie Rich.