The district was created in 1815, when the Prussian province of Saxony was created. The area of Lohra-Clettenberg (the former dukedom County Hohnstein) and the previously free imperial city of Nordhausen were thereafter administrated together. In 1882-3 Nordhausen left the district, which led to its renaming as the Kreis Grafschaft Hohenstein (district county of Hohenstein) in 1888. After World War II the name reverted to Landkreis Nordhausen, and in 1950 the city of Nordhausen was reincorporated into the district. In the administrative reform of 1952 several municipalities changed districts: a number were transferred from the district of Sangerhausen to that of Nordhausen, while others were transferred from Nordhausen to the districts of Worbis, Wernigerode, Sondershausen, and Sangerhausen.
The district is located in the southern foothills of the Harz mountains. The highest point is the Große Ehrenberg, with an elevation of 635.3 m above sea level. The main river in the district is the Zorge. To the south lie the Dün, Hainleite, and Windleite ranges of hills.
The checkered red-silver part of the coat of arms is derived from the coat of arms of the County of Hohenstein, the historical territory to which the district belonged. The broken tree trunk symbolizes the destruction of the city of Nordhausen during World War II, while the new branch growing into the golden area stands for the city's reconstruction and its bright new future.