This article focuses too much on specific examples without explaining their importance to its main subject. (March 2016)
An embedded database system is a database management system (DBMS) which is tightly integrated with an application software that requires access to stored data, such that the database system is "hidden" from the application’s end-user and requires little or no ongoing maintenance. It is actually a broad technology category that includes
- database systems with differing application programming interfaces (SQL as well as proprietary, native APIs),
- database architectures (client-server and in-process),
- storage modes (on-disk, in-memory, and combined),
- database models (relational, object-oriented, entity–attribute–value model, network/CODASYL), and
- target markets.
The term embedded database can be confusing because only a small subset of embedded database products are used in real-time embedded systems such as telecommunications switches and consumer electronics devices. (See mobile database for small-footprint databases that could be used on embedded devices.)
- 1 Implementations
- 2 Comparisons of database storage engines
- 2.1 Advantage Database Server
- 2.2 Apache Derby
- 2.3 Empress Embedded Database
- 2.4 Extensible Storage Engine
- 2.5 eXtremeDB
- 2.6 Firebird Embedded
- 2.7 HarperDB
- 2.8 H2
- 2.9 HailDB, formerly Embedded InnoDB
- 2.10 HSQLDB
- 2.11 InfinityDB
- 2.12 Informix Dynamic Server
- 2.13 InterBase
- 2.14 LevelDB
- 2.15 LightningDB
- 2.16 Mimer SQL
- 2.17 MongoDB
- 2.18 MySQL Embedded Server Library
- 2.19 NexusDB
- 2.20 Oracle Berkeley DB
- 2.21 Raima Database Manager
- 2.22 RocksDB
- 2.23 solidDB
- 2.24 SQLite
- 2.25 SQL Server Compact
- 3 See also
- 4 References
Major embedded database products include, in alphabetical order:
- Advantage Database Server from Sybase Inc.
- Berkeley DB from Oracle Corporation
- CSQL from csqlcache.com
- Extensible Storage Engine from Microsoft
- eXtremeDB from McObject
- Firebird Embedded
- HSQLDB from HSQLDB.ORG,
- Informix Dynamic Server (IDS) from IBM
- InfinityDB from Boiler Bay Inc.
- InnoDB from Oracle Corporation
- InterBase (Both server and mobile friendly deeply embedded version) from Embarcadero Technologies
- MongoDB from MongoDB, Inc.
- RDM Database Manager from Raima
- SQL Server Compact from Microsoft Corporation
- Sophia Embeddable key-value storage
Comparisons of database storage engines
Advantage Database Server
Sybase's Advantage Database Server (ADS) is a full-featured embedded database management system. It provides both ISAM and relational data access and is compatible with multiple platforms including Windows, Linux, and Netware. It is available as a royalty-free local file-server database or a full client-server version. ADS has been around for many years and is highly scalable, with no administration, and has support for a variety of IDEs including .NET Framework (.NET), Object Pascal (Delphi), Visual FoxPro (FoxPro), PHP, Visual Basic (VB), Visual Objects (VO), Vulcan, Clipper, Perl, Java, xHarbour, etc.
Derby is an embeddable SQL engine written entirely in Java. Fully transactional, multi-user with a decent SQL subset, Derby is a mature engine and freely available under the Apache license and is actively maintained. Derby project page. It is also distributed as part of Oracle's Java SE Development Kit (JDK) under the name of Java DB.
Empress Embedded Database
Empress Software, Inc., developer of the Empress Embedded Database, is a privately held company founded in 1979. Empress Embedded Database is a full-function, relational database that has been embedded into applications by organizations small to large, with deployment environments including medical systems, network routers, nuclear power plant monitors, satellite management systems, and other embedded system applications that require reliability and power. Empress is an ACID compliant, SQL database engine with C, C++, Java, JDBC, ODBC, SQL, ADO.NET and kernel level APIs. Applications developed using these APIs may be run in standalone and/or server modes. Empress Embedded Database runs on Linux, Unix, Microsoft Windows and real-time operating systems.
Extensible Storage Engine
ESE is an Indexed Sequential Access Method (ISAM) data storage technology from Microsoft. ESE is notably a core of Microsoft Exchange Server and Active Directory. Its purpose is to allow applications to store and retrieve data via indexed and sequential access. Windows Mail and Desktop Search in the Windows Vista operating system also make use of ESE to store indexes and property information respectively.
McObject launched eXtremeDB as the first in-memory embedded database designed from scratch for real-time embedded systems. The initial product was soon joined by eXtremeDB High Availability (HA) for fault tolerant applications. The product family now includes 64-bit and transaction logging editions, and the hybrid eXtremeDB Fusion, which combines in-memory and on-disk data storage. In 2008, McObject introduced eXtremeDB Kernel Mode, the first embedded DBMS designed to run in an operating system kernel. Today, eXtremeDB is used in millions of real-time and embedded systems worldwide. McObject also offers Perst, an open source, object-oriented embedded database for Java, Java ME, .NET, .NET Compact Framework and Silverlight.
Firebird Embedded is a relational database engine. It's an open source fork of InterBase, is ACID compliant, supports triggers and stored procedures, and is available on Linux, OSX and Windows systems. It has the same features as the classic and superserver version of Firebird, two or more threads (and applications) can access the same database at the same time starting with Firebird 2.5. So Firebird embedded acts as a local server for one threaded client accessing its databases (that means it works properly for ASP.NET web applications, because there, each user has its own thread, which means two users could access the same database at the same time, but they would not be in the same thread, because ASP.NET opens a new thread for each user). It exports the standard Firebird API entrypoints. The main advantage of Firebird embedded databases is, that unlike SQlite or Access databases, they can be plugged into a full Firebird server without any modifications at all also is multiplatform (runs on Linux, OS X with full ASP.NET Mono support)
HarperDB is a cross-platform HTAP database with a 65mb install footprint written in Node.js. It has interfaces for both SQL and NoSQL with a Dynamic Schema. It has clustering and replication and can be installed on ARM6, ARM7, Linux, and Mac OSX.
Written in Java Open source very fast database engine. Embedded and Server mode, Clustering support, can run inside the Google App Engine. Supports encrypted database files (AES or XTEA). The development of H2 was started in May 2004, but it was first published on December 14, 2005. H2 is dual licensed and available under a modified version of the MPL 1.1 (Mozilla Public License) or under the (unmodified) EPL 1.0 (Eclipse Public License).
HailDB, formerly Embedded InnoDB
HailDB is a standalone, embeddable form of the InnoDB Storage Engine. Given that HailDB is based on the same code base as the InnoDB Storage Engine, it contains many of the same features: high-performance and scalability, multiversion concurrency control (MVCC), row-level locking, deadlock detection, fault tolerance, automatic crash recovery, etc. However, because the embedded engine is completely independent from MySQL, it lacks server components such as networking, object-level permissions, etc. By eliminating the MySQL server overhead, InnoDB has a small footprint and is well-suited for embedding in applications which require high-performance and concurrency. As with most embedded database systems, HailDB is designed to be accessed primarily with an ISAM-like C API rather than SQL (though an extremely rudimentary SQL variant is supported).
The project is no longer maintained.
HSQLDB is an opensource relational database management system with a BSD-like license that runs in the same Java Virtual Machine as the embedded application. HSQLDB supports a variety of in-memory and disk-based table modes, Unicode and SQL:2008.
Map-modeled and high-performance, InfinityDB Java embedded DBMS supports enhanced data structures that are flexible, including relational, EAV triples, ER, multimap, text index, trees, DAGs, taxonomies, graphs, ordered sets, large sparse arrays, BLOB's/CLOB's and mixed or custom. The multi-core speed is patent-applied-for. InfinityDB is transactional, compressing, fast, safe, strongly typed, and self-maintaining in a single file. The simple API avoids slow JSON and XML parsing and chunking as well as dangerous inflexible binary encodings like Java Object Serialization or POJOs. In use for over a decade with thousands of deployments.
Informix Dynamic Server
Informix Dynamic Server (IDS) is characterized as an enterprise class embeddable database server, combining embeddable features such as low footprint, programmable and autonomic capabilities with enterprise class database features such as high availability and flexible replication features. IDS is used in deeply embedded scenarios such as IP telephony call-processing systems, point of sale applications and financial transaction processing systems.
InterBase is an IoT Award-winning cross-platform, Unicode enabled SQL database platform able to be embedded within turn-key applications. Out of the box SMP support and on disk AES strength 256bit encryption, SQL 92 & ACID compliance and support for Windows, Macintosh, Linux, Solaris, iOS and Android platforms. Ideal for both small-to-medium and large enterprises supporting hundreds of users and mobile application development. InterBase Light is a free version that can be used on any mobile device and is ideal for mobile applications. Enterprises can switch to a paid version as requirements for change management and security increase. InterBase has high adoption in defense, airspace, oil and gas, and manufacturing industries.
LevelDB is an ordered key/value store created by Google as a lightweight implementation of the Bigtable storage design. As a library (which is the only way to use LevelDB), its native API is C++. It also includes official C wrappers for most functionality. Third-party API wrappers exist for Python, PHP, Go (pure Go LevelDB implementation exists but is in progress still), Node.js and Objective C. Google distributes LevelDB under the New BSD License.
LightningDB is a memory-mapped database for the OpenLDAP Project. It is written in C and the API is modeled after the Berkeley DB API, though much simplified. The library is extremely compact, compiling down to under 40KB of x86 object code, corruption proof, and orders of magnitude faster, more robust, more scalable, and more efficient than similar libraries like Berkeley DB, LevelDB, etc. The library implements B+trees with multiversion concurrency control (MVCC), Single level store, Copy on write and provides full ACID transactions with no deadlocks. The library is optimized for high read concurrency; readers need no locks at all. Readers don't block writers and writers don't block readers, so read performance scales perfectly linearly across arbitrarily many threads and CPUs. Third-party wrappers exist for C++, Erlang and Python. Lightning DB is distributed by the OpenLDAP Project under the OpenLDAP Public License. As of 2013 the OpenLDAP Project is deprecating the use of Berkeley DB, in favor of LightningDB.
An embedded zero maintenance version of the proprietary Mimer SQL database is available.
MongoDB is a cross-platform document-oriented database program. MongoDB stores data in flexible, JSON-like documents, meaning fields can vary from document to document and data structure can be changed over time. The document model maps to the objects in your application code, making data easy to work with. Ad hoc queries, indexing, and real time aggregation provide powerful ways to access and analyze your data. MongoDB is a distributed database at its core, so high availability, horizontal scaling, and geographic distribution are built in and easy to use. MongoDB is open-source, published under the GNU Affero General Public License. MongoDB is embedded across many industries (High Tech, Finserv, Retail, Healthcare) and use cases (Security / Fraud, IoT, CMS) by the most prestigious companies in the world. More on embedded here.
MySQL Embedded Server Library
The libmysqld, MySQL Embedded Server Library provides most of the features of regular MySQL as a linkable library that can be run in the context of a client process. After initialization clients can use the same C API calls as when talking to a separate MySQL server but with less communication overhead and with no need for a separate database process.
NexusDB is the commercial successor to the FlashFiler database which is now open source. They can both be embedded in Delphi applications to create stand-alone executables with full database functionality.
Oracle Berkeley DB
As the name implies, Oracle’s embedded database is actually Berkeley DB, which Oracle acquired from Sleepycat Software. It was originally developed at the University of California. Berkeley DB is a fast, open-source embedded database and is used in several well-known open-source products, including the Linux and BSD Unix operating systems, Apache Web server, OpenLDAP directory, OpenOffice productivity suite.
Raima Database Manager
Raima Database Manager, produced by Raima, was one of the first database management systems to be categorized as an embedded database. According to Raima's definition, the product is embedded in two senses: first, it is embedded within an application, becoming an extension to the application, and second, it is possible to use it in embedded computer/OS or real-time environments because of its small footprint and efficient operation. Its APIs (for C/C++ and SQL) have been designed to support the limited resources of embedded environments. Since its initial release, RDM Embedded has been continually evolving and is currently released as version 14.1.
RocksDB, created at Facebook, began as a fork of LevelDB. It focuses on performance, especially on SSDs. It adds many features, including transactions, backups, snapshots, bloom filters, column families, expiry, custom merge operators, more tunable compaction, statistics collection, and geospatial indexing. It is used as a storage engine inside of several other databases, including ArangoDB, Ceph, CockroachDB, MongoRocks, MyRocks, Rocksandra, and TiVK.
solidDB is a hybrid on-disk/in-memory, relational database and is often used as an embedded system database in telecommunications equipment, network software, and similar systems. In-memory database technology is used to achieve throughput of tens of thousands of transactions per second with response times measured in microseconds. High availability option maintains two copies of the data synchronized at all times. In case of system failure, applications can recover access to solidDB in less than a second without loss of data.
SQLite is a software library that implements a self-contained, server-less, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code, chiefly C, for SQLite is in the public domain. It includes both a native C library and a simple command line client for its database. It’s included in several operating systems; among them are Android, FreeBSD, iOS, OS X and Windows 10.
SQL Server Compact
Microsoft's SQL Server Compact is an embedded database with wide variety of features like multi-process connections, T-SQL, ADO.NET Sync Services to sync with any back end database, Merge Replication with SQL Server, Programming API: LINQ to SQL, LINQ to Entities, ADO.NET. The product runs on both Desktop and Mobile Windows platforms. It has been in the market for long time, used by many enterprises in production software (Case Studies). The product went through multiple re-brandings and was known with multiple names like: SQL CE, SQL Server CE, SQL Server Mobile, SQL Mobile.
- Graves, Steve. "COTS Databases For Embedded Systems", Embedded Computing Design magazine, January, 2007. Retrieved on August 13, 2008.
- Mullins, Craig. "Empress Offers an Effective Embedded Database Solution", 2005. Retrieved on 2008-12-09
- Gorine, Andrei and Krivolapov, Alexander. "Kernel Mode Databases: A DBMS Technology For High-Performance Applications", Dr. Dobb's Journal, April, 2008. Retrieved on August 13, 2008.
- HailDB homepage
- Shutting down HailDB, By Stewart Smith on August 19th, 2015, HailDB
- "Embedding Informix Dynamic Server", Retrieved on August 30, 2009.
- See Berkeley DB
- Niccolai, James. "Update: Oracle Buys Sleepycat open-source database vendor", "InfoWorld", 2006-02-14. Retrieved on June 12, 2008.
- "RocksDB Basics". Retrieved 2018-07-19.
- "RocksDB transactions". GitHub. Retrieved 2016-04-04.
- "How to backup RocksDB?". Retrieved 2017-07-19.
- "Checkpoints". Retrieved 2017-07-19.
- "RocksDB bloom filters". GitHub. Retrieved 2016-04-04.
- "Column families in RocksDB". GitHub. Retrieved 2016-04-04.
- "RocksDB TTL support". GitHub. Retrieved 2016-04-04.
- "RocksDB merge operator". GitHub. Retrieved 2016-04-04.
- "Universal compaction". GitHub. Retrieved 2016-04-04.
- "RocksDB perf context and IO stats context". GitHub. Retrieved 2016-04-04.
- "Spatial indexing in RocksDB". rocksdb.org. Retrieved 2018-07-19.
- "Comparing new RocksDB and MMFiles storage engines". Retrieved 2018-07-19.
- "Storage Devices — Ceph Documentation". Retrieved 2018-07-19.
- "Storage Layer - CockroachDB". Retrieved 2018-07-19.
- "mongodb-partners/mongo-rocks: MongoDB storage integration layer for the Rocks storage engine". Retrieved 2018-07-19.
- "MyRocks - A RocksDB storage engine with MySQL". Retrieved 2018-07-19.
- "Open-sourcing a 10x reduction in Apache Cassandra tail latency". Retrieved 2018-07-19.
- "RocksDB in TiKV - PingCAP". Retrieved 2018-07-19.
- Answer, Usman (29 October 2015). "Shipping a New Mindset with SQLite in Windows 10". Microsoft. Archived from the original on 2016-01-31. Retrieved 6 March 2016.