Rajgrod has a long and rich history, with first human settlements dating back to c. 9000 BC. In the Middle Ages, the Yotvingians founded here a settlement called Raj, which was located on a hill. Raj became main town of a Yotvingian tribe. According to chronicler Wigand of Marburg, in 1360 Polish King Kazimierz Wielki ordered the castellan of Wizna to build a defensive castle near Rajgrod. The fate of the castle has not been established, but probably it was destroyed by the Teutonic Knights.
The town of Rajgrod was first mentioned in 1429, when a man named Mikolaj of Rajgrod sold real estate to his brother Jan. Some time in the early 1440s, a gord was established here and Rajgrod emerged as a local wood trade center. After the Treaty of Melno, the gord was transferred from Polish Mazovia to the Grand Duchy of Lithuania. In c. 1505, King Aleksander Jagiellonczyk gave Rajgrod to a nobleman named Michal Glinski, and in 1509, King Zygmunt Stary handed the area of Rajgrod and Goniadz to Mikolaj Radziwill.
In 1568, Rajgrod was granted town charter, and next year, following the Union of Lublin, together with the region of Podlasie it was transferred back to the Kingdom of Poland. In 1570, Rajgrod became a royal town and a seat of a starosta (first starosta was Marcin Dulski). In c. 1602, a large manor house was constructed for the local administration. In 1679, King Jan III Sobieski confirmed Rajgrod’s privileges and charter, and in 1764, a new church was built.