dbm

This article is about the family of database engines. For other uses, see DBM.

The dbm library was a simple database engine, originally written by Ken Thompson and released by AT&T in 1979. The name is a three letter acronym for database manager, and can also refer to the family of database engines with APIs and features derived from the original dbm.

The dbm library stores arbitrary data by use of a single key (a primary key) in fixed-size buckets and uses hashing techniques to enable fast retrieval of the data by key.

The hashing scheme used is a form of extendible hashing, so that the hashing scheme expands as new buckets are added to the database, meaning that, when nearly empty, the database starts with one bucket, which is then split when it becomes full. The two resulting child buckets will themselves split when they become full, so the database grows as keys are added.

The dbm library and its derivatives are pre-relational databases — effectively a hash fixed to disk. In practice, they can offer a more practical solution for high-speed storage accessed by key, as they do not require the overhead of connecting and preparing queries. This is balanced by the fact that they can generally only be opened for writing by a single process at a time. An agent daemon can handle requests from multiple processes, but introduces IPC overhead.

Successors

The dbm library has had many successors:

• Ndbm: In 1986 Berkeley produced ndbm (standing for New Database Manager). This added support for having multiple databases open concurrently.
• Sdbm: Some versions of Unix were excluding ndbm due to licensing issues, so in 1987 Ozan Yigit produced this public domain clone.
• GDBM (GNU dbm): A Free/Libre version written by Philip A. Nelson for the GNU project. It added support for arbitrary-length data in the database: previously all data had a fixed maximum length. The last version was released on November 13, 2011.
• Tdbm: Provided support for atomic transactions.
• TDB: Released by the Samba team, under the LGPL. From the homepage: TDB is a Trivial Database. In concept, it is very much like GDBM, and BSD's DB except that it allows multiple simultaneous writers and uses locking internally to keep writers from trampling on each other. TDB is also extremely small.
• QDBM: 'Quicker Database Manager'. Claims to be quicker, and was released under the LGPL by Mikio Hirabayashi in 2003.
• Berkeley DB: A version available under dual license, both strong copyleft and commercial. Oracle Corporation has supported and maintained Berkeley DB since February 2006.
• JDBM: JDBM is a transactional persistence engine for Java. It aims to be for Java what GDBM is for other languages (C/C++, Python, Perl, etc.)
• JDBM2: JDBM2 is a fork of JDBM 1.0 which adds various features (such as Map views) and performance enhancements.
• MapDB: MapDB (previously known as JDBM4) is a complete rewrite of JDBM which adds concurrency and fine grained locking, encryption, and compression as well as the features of JDBM2
• Tokyo Cabinet: A modern reimplementation of QDBM, also released under the LGPL by Mikio Hirabayashi in 2007.
• VSDB: An implementation of a dbm-like database written by John Meacham that supports full ACID semantics that places data safety above all. It includes transactions and rollbacks with no locking whatsoever, rather relying on atomic filesystem operations.
• Kyoto Cabinet: A straightforward implementation of DBM, released under the GPL also by Mikio Hirabayashi in 2009.
• aodbm: An append-only DBM-style database written by Daniel Waterworth.
• RaptorDB: A NoSQL engine written in C#.
• LevelDB: An open source on-disk key-value store written by Google Fellows Jeffrey Dean and Sanjay Ghemawat.
• Lightning Memory-Mapped Database: An open source on-disk key-value store without caches written by Howard Chu, initially for OpenLDAP.

References

• Matthew, Neil; Stones, Richard (2008), "Databases", Beginning Linux Programming, Wiley 
• Brachman & Neufeld. "TDBM: A DBM Library With Atomic Transactions". 
• Olson, Bostic & Seltzer. "Berkeley DB".