The name Gulsvik is connected to the original farm of Gulsvik. The Gulsvik farm was known from the middle ages, when the owners were among the most prominent people in lower Hallingdal. Gulsvik is reckoned to be an old Norwegian folkland[disambiguation needed], which never came directly under the crown. The name Gulsvik is believed to derive from Gudulv (Godwulf), shortened to Gul, the name of the bay of Godwulf, a natural bay in the area.
In the late middle ages, the owners of Gulsvik farm were noblemen in allegiance to the king of Norway. The old Gulsvik clan is said to be one of the most widespread families in lower Buskerud, descending from a Birkebeiner soldier in the service of King Sverre. Dating from 1430, the farm became divided, as time passed into upper, middle and lower Gulsvik (Gulsvik nordre, Gulsvik mellom and Gulsvik søndre).
This area is called Gulsvik Farthing (Gulsvikfjerdingen) and contains a number of farms. The wider area south of Gulsvik was settled just as early, and the proper name of the area is Southern Flå Municipality, reckoned from the border of Krødsherad up to the end of Lake Krøderen (Innsjøen Krøderen). The Gulsvik Farthing is to the north, close up to Hallingdal gate, an entrance into the valley of Hallingdal. Bergensbanen railway terminated at Gulsvik Station from 1908 until the line was completed in 1909. Located at the top of Lake Krøderen, Gulsvik was a key port on the lake. When the ferry over Lake Krøderen was established, Gulsvik became the natural port on the northern side, with accommodations for tourism. 
Hallingporten is a 65 m long tunnel on Norwegian National Road 7 (Riksvei 7) just north of Gulsvik. The tunnel has a distinctive shape. The reason for this is that the current road tunnel has been added where there was previously a tog tunnel. The Hallingporten upper level is thus the old tog tunnel. There were skirmishes here in April 1940 during the Fighting in Hallingdal.