He was born in Sumner, Illinois and was the younger brother of Walter and Lester Melrose, who had set up the Melrose Brothers Music Company in Chicago, in 1918. He became one of the leading figures in the Chicago blues and jazz scene of the 1920s and 1930s.
Frank’s first instrument was violin, but he later took up piano. He was strongly influenced by his brothers’ business partner, Jelly Roll Morton. In 1924, he left home and began drifting around, playing and settling for short periods in St. Louis, Kansas City and Detroit. He also played occasionally in Chicago clubs with Morton.
In 1929, his brother Lester, recorded him performing piano solos, which were released under the pseudonym of "Broadway Rastus", and in 1930. Following another trip to Kansas City, he recorded "Jelly Roll Blues" and other tunes. These were issued in Brunswick Records’ “race” series under the pseudonym of "Kansas City Frank", and for some years were wrongly assumed to be the work of Morton.
In the 1930s, Melrose continued to play piano in small clubs and bars, either solo or as part of a band, while occasionally working in a factory to support his family. He also recorded sporadically with Johnny Dodds, George Barnes and others.
He played on his last recording session in 1941 with Bud Jacobson's Jungle Kings. On Labor Day of 1941, he was found dead in the street after being killed in a fracas in a club in Hammond, Indiana.