At Jena, he befriended Gottlob Frege and collaborated with the Neo-Kantian philosopher of language Richard Hönigswald. Bauch was an influential figure in the Kant-Gesellschaft (Kant Society) and helped publish the Prussian Academy's edition of Kant's collected works. Until 1916, he was editor of the Kant Society's journal, Kant-Studien (Kant Studies). He was forced to resign after publishing an anti-semitic article in a right-wing nationalist tabloid, which caused a storm of controversy in the Kant Society. (Many Neo-Kantians, including Bauch's subsequent colleague Hönigswald, were Jewish, and quite a few were social democrats.) In 1917, Bauch founded a philosophical society of his own, the German Philosophical Society, which issued the journal Beiträge zur Philosophie des Deutschen Idealismus (Contributions to the Philosophy of German Idealism). Frege was among its contributors. When the Nazis came to power, Bauch's political views stood him in good stead. While many Neo-Kantians had to emigrate and some ended up in concentration camps (including, for a year, Hönigswald), Bauch became head of the German Philosophical Society in 1934.
Heinrich Rickert, whom Bauch studied under, was the most important leader of the so-called Badensian, or south-west German school of Neo-Kantianism after Wilhelm Windelband. Unlike its main rival, the Marburg School, the Badensian Neo-Kantians were more interested in practical philosophy than in the philosophy of science. They emphasized the distinction between fact and value and sought to use the concept of "value" for epistemological and ontological purposes. For example, to say that a sentence is "true" is sometimes equated with saying that it 'commands assent' (i.e., that it ought to be believed). Bauch, however, was a rather unorthodox scion of the Badensian school, so much so that some commentators regard him as representing a distinct variety of Neo-Kantianism. While Bauch shared an interest in the philosophy of value, he also had a much more lively interest in the philosophy of mathematics and logic than was common among the Badensian Neo-Kantians. Unlike Rickert, he was sympathetic to Gottlob Frege's logicism (which Rickert had rejected on the old-fashioned Kantian grounds that logic was analytic, mathematics synthetic), and was conciliatory toward the Marburg Neo-Kantians' belief in the unity of logic and mathematics.