Gramsh has been inhabited since ancient times, as is confirmed by archaeological findings in the Tumulus of Cëruja.
In the Middle Ages, the area was under the possession of Prince George Arianiti. During Ottoman rule, it was a kaza administrative division within the Sanjak of Elbasan of the Monastir Vilayet.
From 1912 until 1947, this was the center of Gramsh area. With the new administrative division, it became the center of Gramsh district, in which were placed various state institutions. Gramsh was declared a city on July 10, 1960. Since 1965, it has been a municipality.
The city was home of one of the main military weaponry factories during the communist era. Devolli Hydro Power started in 1980, but was left between work due to the death of communist leader Enver Hoxha, and the beginning of a Transitional period until the 1990s that culminated in the overthrow of that system.
On October 2013, it was reported that the municipality debt was high and the municipality was close to bankruptcy.