It was constructed around 762 CE, and is said to have been founded by Rōben. The temple contains a number of cultural assets. According to literature available at the temple complex, the guardian carvings at Sanmon/Todaimon are by Tankei and Unkei. Allegedly, Murasaki Shikibu began writing The Tale of Genji at Ishiyama-dera during a full moon night in August 1004. In commemoration, the temple maintains a Genji room featuring a life-size figure of Lady Murasaki and displays a statue in her honor.
The temple features in one of the Eight Views of Omi, originally drawn by Hiroshige. The print featuring the temple is called "The autumn moon at Ishiyama" (石山の秋月).