It was constructed around 747 CE, and is said to have been founded by Rōben. The temple contains a number of cultural assets. The temple possesses two fragments of manuscripts of the Records of the Grand Historian (Shiji 史記), the first of China's 24 dynastic histories, which are the only known extant fragments that pre-date the Tang dynasty (618–907). According to literature available at the temple complex, the guardian carvings at Sanmon/Todaimon are by Tankei and Unkei. Allegedly, Murasaki Shikibu began writing The Tale of Genji at Ishiyama-dera during a full moon night in August 1004. In commemoration, the temple maintains a Genji room featuring a life-size figure of Lady Murasaki and displays a statue in her honor.
The temple features in one of the Eight Views of Omi, originally drawn by Hiroshige. The print featuring the temple is called "The autumn moon at Ishiyama" (石山の秋月).