Construction took place between 2900 and 2600 BC, though there were possibly buildings before 3000 BC. A tomb was later built into the site. Debris from the destruction of the tomb suggests the site was out of use between 2000 BC and 1700 BC.
The 13 primary stones form a circle about 13 m in diameter, with a long approach avenue of stones to the north, and shorter stone rows to the east, south, and west. The layout recalls a distorted Celtic cross. The individual stones vary from around 1 m to 5 m in height, with an average of 4 m, and are of the local Lewisian gneiss.
The first written reference to the stones was by Lewis native John Morisone, who in c. 1680 wrote that "great stones standing up in ranks [...] were sett up in place for devotione".
The tallest of the stones marks the entrance to a burial cairn where human remains have been discovered. Excavation in 1980 and 1981 showed that the burial chamber was a late addition to the site, and that it had been modified a number of times. Pottery finds suggested a date of 2200 BC for the erection of the circle. Alexander Thom and Gerald Hawkins suggested that the stones were a prehistoric lunar observatory. Others have proposed a relationship between the stones, the moon and the Clisham range on Harris. Critics of these theories argue that several alignments are likely to exist purely by chance in any such structure, and many factors such as the weathering and displacement of the stones over the millennia mean there can be no certainty of any alignments, original or otherwise. 
Local tradition says that giants who lived on the island refused to be converted to Christianity by Saint Kieran and were turned into stone as a punishment. Another belief says that at sunrise on midsummer morning, the "shining one" walked along the stone avenue, "his arrival heralded by the cuckoo's call." This legend could be a folk memory recalling the astronomical significance of the stones.
In 1974, the sculptor Gerald Laing created a work known as Callanish for Strathclyde University's campus in the centre of Glasgow. He planted 16 abstract steel girders in the ground, intended to relate to the configuration of the stones. The sculpture is popularly referred to as "Steelhenge".
In 1984, the new romantic band Ultravox used an image of the stones on the cover of their album Lament. They filmed their video of "One Small Day" at the site; it is the first single taken from that album.
In 1988 Jon Mark released a CD, The Standing Stones of Callanish, intended to evoke Britain's Celtic legacy.