Marable was born in Paducah, Kentucky, and learned piano from his mother. At the age of 17, he began playing on the steam boats plying the Mississippi River. He soon became bandleader for boats on the Streckfus Line, which ran several paddlewheelers which held dances and excursions along the river from New Orleans, Louisiana to Minneapolis, Minnesota. Marable appreciated the new "jazz" sound being played by the New Orleans musicians, and the bulk of his band members were recruited from that city.
Members of Marable's bands were expected to be able to play a wide variety of music, from hot numbers to light classics, both play by head and from sheet music, and above all to keep the dancers happy. Marable was a strict bandleader, demanding musical proficiency and rigid discipline from all his bandmembers, yet allowing them to develop their individual strong points. For instance, Louis Armstrong's gift for improvisation was recognised as such by Marable, and he allowed him to improvise his breaks rather than play them note for note. Marable's band served as an early musical education for many other players who would later become prominent in jazz, including Red Allen, Baby Dodds, Johnny Dodds, Pops Foster, Narvin Kimball, Al Morgan, Jimmy Blanton, Elbert Pee Wee Claybrook, and Zutty Singleton.
In addition to piano and bandleading, Marable played the boats' steam calliope, a contraption that could be heard for miles up and down the river and poured down so much water from condensing steam that Marable performed wearing a raincoat and hood.