Like most of the medieval cities of trade, Haderslev had a royal castle, which was called Haderslevhus. The suffix "hus" (meaning house) was commonly used for castles in medieval Denmark (Koldinghus, Tønderhus, Ålborghus, Riberhus etc.). The castle was first mentioned in sources dating back to 1326, but it is it was most likely built in the second half of the 13th century, like most Danish city castles. These were home of the governor of the borough (and later the county), who took care of the king's (or in Southern Jutland, the duke's) possessions in and around the city. In the city castle, the taxes, duties, and fines were paid. Haderslevhus was located in the eastern part of the city, which was surrounded by a moat at the time. In this fortification lived the Danish king Christian the 3rd when he imposed the Reformation in Haderslev, in 1526. When Hans the Elder was proclaimed duke of Slesvig and Holstein (today Southern Jutland and Northern Germany), he took up residence in Haderslevhus, which now, in 1544, was an old, and worn-out building.
After a few years, the duke decided to build a new castle, called Hansburger. It was supposed to be a new and modern renaissance palace that suited him much better. As chief architect, he appointed Hercules von Oberberg - one of the most influential architects of the time. The western wing of the building was ready for use in 1562 and the duke moved in. Four years later, in 1566, the southern wing with the chappelle was completed. After the completion of Hansborg, the old building was demolished, and the area on which it stood for centuries, was sold in bits to the staff of the new palace. A small part of the castle is preserved to this day. In the 1570s, Duke Hans started to build the eastern and northern wings of Hansborg, but he died in 1580, and therefore did not see the completed castle. After the duke died, the Danish King Frederik the 2nd acquired parts of his lands, including Haderslev, and in 1580-1585 he finished the construction of the last two wings. The palace now stood as a whole - complete with four wings and two towers at the corners. In the area between the castle and the city, one could find stables, economic buildings, and residences for parts of the palace staff. The palace itself spanned over a 80x75m piece of ground - about the size of Kronborg, and a bit bigger than Sønderborg Castle.
Shortly after the completion of Hansborg, it was decided that the castle should be called by the original name, Haderslevhus—presumably to follow the old systematic naming of castles in trade cities at the time. So, actually, the castle was only called Hansborg in the short period of time between 1562 to about 1590. Otherwise, the castles has always been spoken of by the name Haderslevhus. The German imperial troops used the castle as headquarters in 1627-1629, and the building was severely damaged. In 1629-1642 a lot of reconstruction took place, but the castle was invaded by Torstenson's troops in 1643-1645, and in 1644, the castle caught fire, and burned to the ground. After this catastrophe, many of the bricks was used for the construction of the Danish Royal Library, and therefore a castle was never rebuilt. Haderslev had lost both of its castles.