Dörmer was born in Berlin, Germany. Deeply involved with church youth groups as a child, by age fifteen, he was frequenting Berlin's gay bars. Dörmer was 10 years old when he joined the German Youth Movement in 1922. In 1929, he founded his own youth group, called the "Wolfsring" (lit. "ring of wolves"), which combined sexual affairs, amateur theater performances, and travel. In 1932, he was promoted to youth leader and worked in the scout movement at a national level. He and his group tried to stay independent, but in October 1933 they were forced to join the Hitler Youth.
In April 1935, Dörmer was accused of homosexual activities with members of his troop, and from 1941 to 1944 he was imprisoned for corrupting the youth at Neuengamme concentration camp, a "holding tank for homosexuals, politicals, and non-German aliens."
After the war, Dörmer spent another eight years in prison on various charges. After his final release in 1963, he returned to Berlin to live with his father, who died in 1970. His 1982 application for reparations from the German government was rejected. He made an appearance in the 2000 documentary film Paragraph 175, which portrays survivors of persecution then-authorized under the German anti-male homosexuality law of the same name.