Middelschulte initially studied organ with August Knabe. He later attended the Royal Academic Institute for Church Music, where he studied organ and theory with August Haupt. After briefly holding a position at the Royal Institute and acquiring a post at the Lukaskirche in Berlin, he moved to Chicago in 1891. In 1893, he gave three performances entirely from memory at the Columbian Exposition. From 1896 - 1918 he was organist for what would later become the Chicago Symphony Orchestra. During the 1920s, he regularly returned to his native Germany to give performances. He is regarded as one of the most significant organists of his time, and was critically acclaimed for his performances of Johann Sebastian Bach. In 1939, after nearly fifty years in America, Middelschulte returned to Germany, where he died only four years later.
His students included Virgil Fox and Cecilia Clare Bocard. Fox frequently used as an encore to his performances Middelschulte's "Perpetuum Mobile", an elaborate piece that builds from a subdued sound to, by the end, fortissimo and played almost entirely on the pedals; the penultimate measure contains an ascending scalar flourish and the last measure a single chord both played on full organ.