Nerz was born in Hechingen, Province of Hohenzollern. A trained medical doctor, he played as an amateur for VfR Mannheim and Tennis Borussia Berlin before being appointed as Germany's first national manager and selector in 1923. At the time, football was not a major sport in Germany, and German football was seen as considerably inferior to that played by other central European countries such as Austria, Hungary, Czechoslovakia and Italy. Nonetheless, under Nerz the team — initially considered one of the weakest in Europe — gradually developed some consistency towards the end of the 1920s and early 1930s. Nerz studied a large number of league and cup games in England, as well as in Austria and Italy, and sought advice on coaching and tactics from internationally-respected coaches such as Jimmy Hogan, Hugo Meisl and Vittorio Pozzo in a quest to improve the standards of the German national team.
Germany did not enter the inaugural World Cup in 1930 in Uruguay, but by the time of the next tournament in 1934, held in Italy, Germany had become a strong side by European standards. In the event, Nerz guided Germany to victories over Belgium and Sweden; a semi-final defeat to Czechoslovakia was followed by a win over the hitherto heavily-fancied Austrians to secure a third-place finish. This would be Germany's best international performance until the 1954 World Cup, and it provided a huge boost to the popularity of the sport in Germany.
Dr. Otto Nerz had joined the Nazi Party relatively early, prior to Adolf Hitlergaining power in 1933; this, combined with his success in the World Cup, meant that the German government placed a high level of expectation on Nerz's team for the 1936 Summer Olympics in Berlin. However, Germany was eliminated early in the tournament after a shock defeat to rank outsiders Norway. Shortly thereafter, Nerz was relieved of his duties as coach and replaced by Sepp Herberger.
Following his dismissal from the German national job, Nerz took an administrative and coaching position with the football association of Berlin. Due to his Nazi Party membership, he was arrested as a prisoner of war after the Battle of Berlin and later interned in a camp in Sachsenhausen; after four years of imprisonment, he died on or around 18 April 1949 and was buried in a mass grave on the site of the camp.