An in-memory database (IMDB; also main memory database system or MMDB or memory resident database) is a database management system that primarily relies on main memory for computer data storage. It is contrasted with database management systems that employ a disk storage mechanism. Main memory databases are faster than disk-optimized databases since the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory eliminates seek time when querying the data, which provides faster and more predictable performance than disk.
In applications where response time is critical, such as telecommunications network equipment and mobile advertising networks, main memory databases are often used. IMDBs have gained a lot of traction, especially in the data analytics space, starting mid-2000s mainly due to cheaper RAM.
In their simplest form, main memory databases store data on volatile memory devices. These devices lose all stored information when the device loses power or is reset. In this case, MMDBs can be said to lack support for the durability portion of the ACID (atomicity, consistency, isolation, durability) properties. Volatile memory-based MMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation. However, since Non-Volatile DIMMs (NVDIMM) have become available, most future IMDB or MMDB installations will indeed have the durability to meet full ACID support.
Many MMDBs have added durability via the following mechanisms:
- Snapshot files, or, checkpoint images, which record the state of the database at a given moment in time. These are typically generated periodically, or, at least when the MMDB does a controlled shut-down. While they give a measure of persistence to the data (in that not everything is lost in the case of a system crash) they only offer partial durability (as 'recent' changes will be lost). For full durability, they will need to be supplemented by one of the following:
- Transaction logging, which records changes to the database in a journal file and facilitates automatic recovery of an in-memory database.
- Non-Volatile DIMM (NVDIMM), a memory module that has a DRAM interface, often combined with NAND flash for the Non-Volatile data security. The first NVDIMM solutions were designed with supercapacitors instead of batteries for the backup power source. With this storage, MMDB or IMDB can resume securely from its state upon reboot.
- Non-volatile random access memory (NVRAM), usually in the form of static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM (EEPROM). With this storage, the MMDB system can recover the data store from its last consistent state upon reboot.
- High availability implementations that rely on database replication, with automatic failover to an identical standby database in the event of primary database failure. To protect against loss of data in the case of a complete system crash, replication of a MMDB is normally used in conjunction with one or more of the mechanisms listed above.
Some MMDBs allow the database schema to specify different durability requirements for selected areas of the database - thus, faster-changing data that can easily be regenerated or that has no meaning after a system shut-down would not need to be journaled for durability (though it would have to be replicated for high availability), whereas configuration information would be flagged as needing preservation.
Hybrids with on-disk databases
The first database engine to support both in-memory and on-disk tables in a single database was WebDNA: it was released in 1995. The advantage to this approach is flexibility: the developer can strike a balance between performance (which is enhanced by sorting, storing and retrieving specified data entirely in memory, rather than going to disk); cost, because a less costly hard disk can be substituted for more memory; persistence; and form factor, because RAM chips cannot approach the density of a small hard drive.
Manufacturing efficiency is another reason a combined in-memory/on-disk database system may be chosen. Some device product lines, especially in consumer electronics, include some units with permanent storage, and others that rely on memory for storage (set-top boxes, for example). If such devices require a database system, a manufacturer can adopt a hybrid database system at lower and upper cost, and with less code customization, than using separate in-memory and on-disk databases, respectively, for its disk-less and disk-based products.
Another variation is to have large amounts of nonvolatile memory in the server. For example Flash memory chips as addressable memory rather than structured as disk arrays. A database in this form of memory combines very fast access speed with persistence over reboots and power losses.
|ActiveSpaces||TIBCO Software||Proprietary with developer download||For Java/.Net./C, distributed, hybrid, event enabled, NewSQL|
|ALTIBASE HDB||Altibase Corporation||Proprietary||"Hybrid DBMS" that combines an in-memory database with a conventional disk-resident database|
|BigMemory||Terracotta, Inc. (Software AG)||Proprietary (free editions)|
|Datablitz (formerly Dali)||Bell Labs (Alcatel-Lucent)||Proprietary||Dali prototype was developed for in-house Bell Labs needs|
|DB2 BLU||IBM||Proprietary||IBM DB2 for Linux, UNIX and Windows supports Dynamic In-memory (in-memory columnar processing) Parallel Vector Processing, Actionable Compression, and Data Skipping technologies, collectively called ‘BLU Acceleration’ by IBM. Released in June 2013.|
|Ehcache||Terracotta, Inc. (Software AG)||Open source (Apache License)||For Java, distributed|
|EXASolution||EXASOL||Proprietary||Relational (SQL over ODBC, JDBC, or ADO.NET), multi-dimensional (MDX over ODBO or XMLA); EXASOL leads the well-respected international TPC-H benchmark, since 2008, in the datavolume-based categories 100GB, 300GB, 1TB, 3TB, and 10TB; EXASolution architecture: shared-nothing, column-store, in-memory database|
|H2 (DBMS)||H2||Mozilla Public License or Eclipse Public License||For Java|
|Hazelcast||Hazelcast Team||Open source (Apache License 2.0)||For Java, NoSQL, distributed|
|Kognitio Analytical Platform||Kognitio, Limited||Proprietary||Development of an in-memory database, specialized for analytical workloads was started at White Cross Systems, Limited in 1988. The first beta release of that system was in 1989. It was based on the INMOS Transputer. The first full production release was offered in 1992. White Cross merged in 2005 with Kognitio, Limited in the United Kingdom and is currently marketing version 8 of the same code base as the "Kognitio Analytical Platform".|
|MemSQL||MemSQL, Inc.||Proprietary||SQL relational|
|Microsoft COM+ IMDB||Microsoft Corporation||Proprietary||Defunct|
|Microsoft SQL Server||Microsoft||Proprietary||SQL Server 2012 contains an in-memory technology called xVelocity column store indexes targeted for data warehouse workload. The recently announced SQL Server 2014 will contain a in-memory technology with the code name Hekaton targeted for OLTP type workloads.|
|Oracle Coherence||Oracle Corporation||Proprietary with developer download||For Java, relational, distributed|
|Oracle Exalytics||Oracle Corporation||Proprietary||Appliance|
|Polyhedra||ENEA AB (previously Perihelion Software)||Proprietary, with a free to use edition (Polyhedra Lite)||Relational (SQL, ODBC, JDBC) in-memory database system originally developed for use in SCADA and embedded systems, but used in a variety of other applications including financial systems. Supports data durability via snapshots and journal logging, and high availability via a hot-standby. First release was in 1993; 8.7 released in March 2013. Polyhedra Lite was released under a free-to-use license in 2012.|
|SAP HANA||SAP AG||Proprietary|
|SQLFire||VMware||Proprietary||Relational, distributed, NewSQL|
|TimesTen||Oracle Corporation||Proprietary||Standalone database or in-memory cache for Oracle Database|
|UnQLite Embedded Database||Symisc Systems||BSD, SPL||UnQLite has support for in-memory databases as well on-disk databases using the same API with pluggable run-time interchangeable storage engines (B+tree, Hash, etc.)|
|VoltDB||VoltDB Inc.||Open source (GPL) / Proprietary||Relational; implements H-Store design|
|WebDNA||WebDNA Software Corporation||Freeware||Robust hybrid in-memory database system and scripting language designed for the World Wide Web. Resilience is provided by both in-memory and on-disk tables in a single database.|
|Xeround||Xeround Inc.||Proprietary/Not for sale, service only||Cloud database|
Many DBMS support in-memory-only storage engines, including:
- Distributed data store
- Embedded databases
- Comparison of relational database management systems
- "Definition: in-memory database". WhatIs.com. Retrieved 19 January 2013.
- Michael Vizard. "The Rise of In-Memory Databases". Slashdot. Retrieved 19 January 2013.
- "TeleCommunication Systems Signs up as a Reseller of TimesTen; Mobile Operators and Carriers Gain Real-Time Platform for Location-Based Services". Business Wire. 2002-06-24.
- "Falling RAM Prices Drive In-Memory Database Surge". SAP. Retrieved 19 January 2013.
- "Rise of In-Memory Databases Impacts Wide Range of Jobs". Dice.com. May 4, 2012. Retrieved 19 January 2013.
- Whole-system Persistence with Non-volatile Memories http://research.microsoft.com/apps/pubs/default.aspx?id=160853
- The Bleak Future of NAND Flash Memory http://research.microsoft.com/apps/pubs/default.aspx?id=162804
- Using NVDIMM as Storage, In-Memory Database Gains Durability & Keeps High Performance http://low-latency.com/article/using-nvdimm-storage-memory-database-gains-durability-keeps-high-performance
- "Truly these are the GOLDEN YEARS of Storage."
- TPC-H - Homepage. Tpc.org (2010-04-21). Retrieved on 2013-09-18.
- Enea Adds Fine-Grain Locking and Cascaded Deletes to the World's Best Relational In-Memory Database Server, Press Release (8.7), Enea AB
- Polyhedra Lite In-Memory Relational Database System Freeware Available Now from Enea, Press Release, Reuters
- Jack Belzer. Encyclopedia of Computer Science and Technology - Volume 14: Very Large Data Base Systems to Zero-Memory and Markov Information Source. Marcel Dekker Inc. ISBN 0-8247-2214-0.