In-memory database

An in-memory database (IMDB; also main memory database system or MMDB) is a database management system that primarily relies on main memory for computer data storage. It is contrasted with database management systems which employ a disk storage mechanism. Main memory databases are faster than disk-optimized databases since the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory reduces the I/O reading activity when querying the data which provides faster and more predictable performance than disk. In applications where response time is critical, such as telecommunications network equipment and mobile ads networks, main memory databases are often used.

ACID support

In their simplest form, main memory databases store data on volatile memory devices. These devices lose all stored information when the device loses power or is reset. In this case, MMDBs can be said to lack support for the durability portion of the ACID (atomicity, consistency, isolation, durability) properties. Volatile memory-based MMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation.

Many MMDBs add durability via the following mechanisms:

• Snapshot files, or, checkpoint images, which record the state of the database at a given moment in time. These are typically generated periodically, or, at least when the MMDB does a controlled shut-down. While they give a measure of persistence to the data (in that not everything is lost in the case of a system crash) they only offer partial durability (as 'recent' changes will be lost). For full durability, they will need to be supplemented by one of the following:

• Transaction logging, which records changes to the database in a journal file and facilitates automatic recovery of an in-memory database.

• Non-volatile random access memory (NVRAM), usually in the form of static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM (EEPROM). With this storage, the MMDB system can recover the data store from its last consistent state upon reboot.

• High availability implementations that rely on database replication, with automatic failover to an identical standby database in the event of primary database failure. To protect against loss of data in the case of a complete system crash, replication of a MMDB is normally used in conjunction with one or more of the mechanisms listed above.

Some MMDBs allow the database schema to specify different durability requirements for selected areas of the database - thus, faster-changing data that can easily be regenerated or that has no meaning after a system shut-down would not need to be journaled for durability (though it would have to be replicated for high availability), whereas configuration information would be flagged as needing preservation.

"Hybrid" in-memory/on-disk databases

The first database engine to support both in-memory and on-disk tables in a single database was released in 2003. The advantage to this approach is flexibility: the developer can strike a balance between performance (which is enhanced by sorting, storing and retrieving specified data entirely in memory, rather than going to disk); cost, because a less expensive hard disk can be substituted for more memory; persistence; and form factor, because RAM chips cannot approach the density of a small hard drive.

Manufacturing efficiency is another reason a combined in-memory/on-disk database system may be chosen. Some device product lines, especially in consumer electronics, include some units with permanent storage, and others that rely on memory for storage (set-top boxes, for example). If such devices require a database system, a manufacturer can adopt a hybrid database system at lower cost, and with less code customization, than using separate in-memory and on-disk databases, respectively, for its disk-less and disk-based products.

Commercial products

Research in main-memory database systems started around 1993 at Bell Labs. It was prototyped as the Dali Main-Memory Storage Manager. This research lead to first commercial main-memory database, Datablitz. In recent years, main memory databases have attracted the interest of larger database vendors. TimesTen, a start-up company founded by Marie-Anne Neimat in 1996 as a spin-off from Hewlett-Packard, was acquired by Oracle Corporation in 2005. Oracle now markets this product as both a standalone database and an in-memory database cache to the Oracle database. IBM acquired SolidDB in 2008, and Microsoft is widely rumored to be launching an in-memory solution in 2009.. SAP announced general availability of their own in-memory product, SAP HANA, in June 2011. VoltDB, founded by DBMS visionary Michael Stonebraker announced the general availability of its namesake in-memory database in May 2010, and offers versions of the product under open source (GPLv3) and commercial licenses.

Products

Product name	License	Description
Adaptive Server Enterprise (ASE) 15.5	Proprietary	enterprise database from Sybase)
Apache Derby	Apache License 2.0	Java RDBMS
Altibase	Proprietary	has in-memory and disk table; HYBRID DBMS
BlackRay	GNU General Public Licence (GPLv2) and BSD License
CSQL	GNU General Public Licence or proprietary
Datablitz	Proprietary	DBMS
Eloquera	Proprietary	In-memory, In-memory:persist modes
EXASolution	Proprietary	relational database management system from EXASOL), in-memory query processing, massively parallel data processing, column by column storage
eXtremeDB	commercial product	DBMS, also check out its open source PERST dbms.
Finances Without Problems	commercial product	primarily in-memory database with hybrid features
FleetDB	MIT	NOSQL db with Writing to an append-only log to provide durability.
H2	Mozilla Public License or Eclipse Public License	has a memory-only mode
HSQLDB	BSD license	has a memory-only mode
IBM TM1	Proprietary	in-memory BI and data analysis
InfoZoom	Proprietary	in-memory BI and data analysis
KDB	Proprietary	DBMS, also supports disk based data
#liveDB	Open Source	Prevalence engine
membase	Apache License	NoSQL, hybrid
Mercury	Proprietary	object oriented, in-memory data management with persistency support, in-built compiler, hosts application server on same machine; developed by staila technologies, an ETHZ spinoff
MicroStrategy		in-memory BI for MicroStrategy 9
MonetDB	MonetDB License
MySQL	GNU General Public License or proprietary	has a cluster server which uses a main-memory storage engine
Oracle Berkeley DB	Sleepycat License	can be configured to run in memory only
Panorama		for Windows and Macintosh, both single user and server versions
ParAccel	Proprietary	in-memory, columnar, relational, ACID-compliant; disk-based mode as well
Polyhedra IMDB	Proprietary	relational, supports High-Availability; acquired in 2001 by ENEA
QlikView		BI-tool developed by QlikTech
RDM Embedded	Proprietary	including hybrid
RDM Server	Proprietary	including hybrid
Redis	BSD	NoSQL
solidDB by IBM		including hybrid, HSB-based HA, Shared memory, embedded, XA, etc.
SAP HANA database	Proprietary	Database engine of the SAP In-Memory Appliance (SAP HANA) produced by SAP AG
SQLite	Public domain	hybrid, RAM and disk dbs can be used together
Starcounter		in-memory object relational dbms
Tarantool	BSD	NoSQL, extendable with Lua stored procedures
TimesTen by Oracle		in memory only or as a cache for Oracle Database
Vertipaq	Proprietary	Microsoft PowerPivot and Microsoft Analysis Services in-memory BI engine
VoltDB	GNU General Public License v3	in-memory
TREX		search engine in the SAP NetWeaver integrated technology platform produced by SAP AG
Xcelerix by Frontex		commercial product
WX2 by Kognitio		commercial product
Xeround		commercial product, database as a service, in-memory infrastructure with MySQL front-end

References

• Jack Belzer. Encyclopedia of Computer Science and Technology - Volume 14: Very Large Data Base Systems to Zero-Memory and Markov Information Source. Marcel Dekker Inc. ISBN 0-8247-2214-0. 

See also

• In Memory Data Grid - In Memory Distributed Data management
• Embedded Databases
• NoSQL - Alternative Scale-out Databases

External links

• SAP HANA Now Generally Available to Customers Worldwide
• In-Memory Database Systems Questions and Answers
• IBM Systems and Services for SAP HANA
• SolidDB and the secrets of speed - How the IBM in-memory database redefines high performance
• Oracle TimesTen in-memory database
• The Rebirth of the In-Memory Database