The first information about the construction of the castle in Brody town refers to the 1580s. Crown Hetman (Polish: Hetman polny koronny) Stanisław Koniecpolski acquired Brody town in 1629 and built the castle bastion of a new type between 1630 and 1635 under the supervision of Italian architect Andrea del Aqua and based on a design by Guillaume Le Vasseur de Beauplan.
The castle was pentagonal in plan and included five bastions and curtain walls, that had octagon-formed dungeons in the middle. It was surrounded by the deep moat. The total number of casemats in the castle was 75, they were used as barracks and warehouses. The governor lived in a wooden house in the grounds of Brody castle. This house, as well as wooden chapel remained till the middle of 18th century.
The castle and the fortress withstood a siege by Cossack troops, that lasted several weeks during the Khmelnytsky Uprising in 1648. During the Battle of Berestechko (1651) the fortress was the Polish military base. Here the captive Cossaks were detained.
In the late 1660s the castle was renovated and thank to this the locals could hide there during Turkish campaigns in 1672 and in 1676.
The castle was acquired in 1704 by the Potocki family. Count Potocki reconstructed the castle for his own residence in Rococo style.
In 1772, after the First Partition of Poland, Brody town was withdrawn by Austria. Russian forces ravaged Brody during the War of the Polish Succession. At the order of the Austrian administration, most of the remaining fortifications were demolished by Wincenty Potocki in 1812: (the ravelin, the clock tower, two bastions from the city side, the moat was filled).
In the postwar period, the castle housed the military unit, and at that time it was badly damaged due to reconstructions. The military unit left the fortress only in the 1990s. After that the palace facilities were transferred to the local school, there is also the regional archive and exhibition halls of Brody Museum of Local History. Castle dungeons are still in ruins and are not used, that leads to considerable damages. The only surviving structure now is the much rebuilt two-storey palace of Stanislaw Potocki (1753–1805).