The Old Norse form of the name was Hvammr, identical with the word hvammr which means "(small) valley", possibly referring the Steinsdalen valley west of Norheimsund. Before 1912, the municipality was named Vikør, which comes from the Old Norse word Vikøyar. It was the name of the farm on which the old Vikøy Church was located.
The coat-of-arms is from modern times; they were granted on 27 November 1981. The blue and silver arms are designed to look like the Fyksesund, a narrow fjord in the municipality. The shape of the fjord is that it is narrower in the middle and this is symbolized in the design of the coat-of-arms.
The old parish of Vikør was established as a municipality on 1 January 1838 (see formannskapsdistrikt). According to the 1835 census, the parish had a population of 2,321. On 1 January 1882, a small area of Ullensvang municipality (population: 22) was transferred to Vikør. In 1912, the municipality changed its name from "Vikør" to "Kvam".
On 1 January 1965, there were many major municipal changes in Norway due to the decisions of the Schei Committee. The Åsgrenda area on the Folgefonna peninsula (population: 61) was moved from Kvam to neighboring Ullensvang municipality. On the same date, Kvam also gained a considerable amount of territory, making it a much larger municipality. The following places were merged, giving the new Kvam municipality a population of 9,119.
Most of Kvam municipality (population: 6,759) except for Åsegrenda which was moved to Ullensvang
The part of Jondal municipality that was located on the northwest side of the Hardangerfjord, around the village of Tørvikbygd (population: 515)