The district was created in 1821, when the duchy of Fulda became a province of Hesse, and was split into four districts. In 1866 the north of Hesse became a part of Prussia, including the Gersfeld area which previously belonged to Bavaria. In 1927 the city of Fulda left the district to become a district-free city, and in 1932 the remaining district was merged with the district of Gersfeld. 1972 the previously small municipalities were merged into 23 bigger ones, and in 1974 the city of Fulda lost its status as district-free city and joined the district again. In 1972 the major part of the Hünfeld district was added to the district.
The left side of the coat of arms shows a black cross, the sign of the monastery of Fulda. The lion on the right side is taken from the Hesse coat of arms, symbolizing that the district was part of Hesse 1821-1866, and again after 1945. The coat of arms was granted in 1936.