Garland studied music at Shaw University and the Aeolian Conservatory. He started by playing classical music but joined a jazz band, Graham Jackson's Seminole Syncopators, in 1924, where he first recorded. He had a long run of associations as a sideman on saxophone and clarinet, with Elmer Snowden (1925), Joe Steele, Henri Saparo, Leon Abbey (including a tour of South America), Charlie Skeete and Jelly Roll Morton in the 1920s. The 1930s saw him playing with Bobby Neal (1931) and the Mills Blue Rhythm Band; he was both a performer and an arranger for the Blue Rhythm Band from 1932 to 1936, when Lucky Millinder replaced him. Following this he played with Edgar Hayes (1937), Don Redman (1938), and Louis Armstrong (1939–42). In the 1940s he played with Claude Hopkins and others, and then returned to Armstrong's band from 1945-47. Following this he played with Herbie Fields, Hopkins again, and Earl Hines (1948). In the 1950s, he went into semi-retirement.
Joe had a Daughter named Joanne Garland-Tyler born October 27, 1943 she moved to Kalamazoo, Michigan in the late 60's.
Garland is credited as the composer (with Andy Razaf as lyricist) of the Glenn Miller hit "In the Mood", but "In The Mood"'s main theme, featuring repeated arpeggios rhythmically displaced, had previously appeared under the title of "Tar Paper Stomp", credited to jazz trumpeter/bandleader Wingy Manone. Manone recorded "Tar Paper Stomp" which did not become popular until the middle of 1930, just months before Horace Henderson used the same tune in "Hot and Anxious," recorded by his brother's band, The Fletcher Henderson Orchestra, on March 19, 1931.
This song was first performed by bandleaders Charlie Barnet and Artie Shaw, but fell out of favor because Garland's original arrangement was too long to fit on one side of a 78rpm record. Garland then brought "In the Mood" to Glenn Miller, who created a shorter arrangement.
- Flanagan, David; Kernfeld, Barry (2002). "Garland, Joe". In Barry Kernfeld (ed.). The new Grove dictionary of jazz, vol. 2 (2nd ed.). New York: Grove's Dictionaries Inc. pp. 13–14. ISBN 1561592846.
- Simon, George T. (1967). The Big Bands. The Macmillan Company. pp. 357–358. LCCN 67-26643.
Glenn, with his savvy as an arranger, made appropriate cuts, whittling it down to a length that would fit on one side of a record.