Instead of using the original recordings, which were made for RCA, Columbia and Capitol Records, and therefore not eligible for use by his then-current label, Reprise, Sinatra used re-recorded versions for the majority of the album's songs, culling tracks from his prior Reprise albums. Three songs were specifically recorded for the project: "I'll Never Smile Again", "Come Fly With Me" and "Love and Marriage". There is also a narration from Sinatra that runs throughout the album.
Approximately 2,000 copies of this album were originally released in a special wooden slipcase containing 3D artwork on the cover in the form of a metal plaque. Each copy was numbered and contained a card signed by Sinatra himself. The packaging also included a booklet highlighting Sinatra's career. A majority of these special "Man and His Music" LP's were given away as door prizes by Sinatra at a party in Palm Springs. The party celebrated the singer's 50th birthday, as well as the airing of Sinatra's 1965 NBC television special of the same name (Frank Sinatra: A Man and His Music).