The north of the island is hilly and dominated by Mullach Mòr. The modern buildings are on the west coast beneath its summit. The north is largely separated from the flatter southern half by Loch Mòr (big loch) and a strip of sand dunes.
The island was occupied from prehistoric times. From the fifteenth century, it was owned by the MacLeans. The thirteenth MacLean of Boreray left the island in around 1810, and the island was divided into twenty crofts. The population grew quickly, and 181 inhabitants were recorded in the 1841 census. Through the 19th century there were over 100 people living there.
Over-cultivation and the collapse of the kelp trade brought a gradual decline in the population. In 1923, the island was evacuated at the request of the islanders. One family stayed on until the 1960s, when the island was abandoned. In 1999, the present crofter started rebuilding work and remains the island’s sole regular inhabitant.