|Original author(s)||Margo Seltzer and Keith Bostic of Sleepycat Software|
|Developer(s)||Sleepycat Software, later Oracle Corporation|
6.2 / JE (7.0) / April 18, 2016
|Operating system||Windows, Unix-like|
|Size||~1244 kB compiled on Windows x86|
|License||dual licensed (GNU Affero General Public License and commercial permissive license) (version 6.x and upwards)
Sleepycat license (versions 2.0-5.x)
4-clause BSD license (versions 1.x)
Berkeley DB (BDB) is a software library intended to provide a high-performance embedded database for key/value data. Berkeley DB is written in C with API bindings for C++, C#, Java, Perl, PHP, Python, Ruby, Smalltalk, Tcl, and many other programming languages. BDB stores arbitrary key/data pairs as byte arrays, and supports multiple data items for a single key. Berkeley DB is not a relational database.
BDB can support thousands of simultaneous threads of control or concurrent processes manipulating databases as large as 256 terabytes, on a wide variety of operating systems including most Unix-like and Windows systems, and real-time operating systems. "Berkeley DB" is also used as the common brand name for three distinct products: Oracle Berkeley DB, Berkeley DB Java Edition, and Berkeley DB XML. These three products all share a common ancestry and are currently under active development at Oracle Corporation.
Berkeley DB originated at the University of California, Berkeley as part of BSD, Berkeley's version of the Unix operating system. After 4.3BSD (1986), the BSD developers attempted to remove or replace all code originating in the original AT&T Unix from which BSD was derived. In doing so, they needed to rewrite the Unix database package. Seltzer and Yigit created a new database, unencumbered by any AT&T patents: an on-disk hash table that outperformed the existing Dbm libraries. Berkeley DB itself was first released in 1991 and later included with 4.4BSD. In 1996 Netscape requested that the authors of Berkeley DB improve and extend the library, then at version 1.86, to suit Netscape's requirements for an LDAP server and for use in the Netscape browser. That request led to the creation of Sleepycat Software. This company was acquired by Oracle Corporation in February 2006, which continues to develop and sell Berkeley DB.
Since its initial release, Berkeley DB has gone through various versions. Each major release cycle has introduced a single new major feature generally layering on top of the earlier features to add functionality to the product. The 1.x releases focused on managing key/value data storage and are referred to as "Data Store" (DS). The 2.x releases added a locking system enabling concurrent access to data. This is what is known as "Concurrent Data Store" (CDS). The 3.x releases added a logging system for transactions and recovery, called "Transactional Data Store" (TDS). The 4.x releases added the ability to replicate log records and create a distributed highly available single-master multi-replica database. This is called the "High Availability" (HA) feature set. Berkeley DB's evolution has sometimes led to minor API changes or log format changes, but very rarely have database formats changed. Berkeley DB HA supports online upgrades from one version to the next by maintaining the ability to read and apply the prior release's log records.
The FreeBSD and OpenBSD operating systems continue to use Berkeley DB 1.8x for compatibility reasons;[dubious ] Linux-based operating systems commonly include several versions to accommodate for applications still using older interfaces/files.
Starting with the 6.0/12c releases, all Berkeley DB products are licensed under the GNU AGPL. Up until then Berkeley DB was redistributed under the 4-clause BSD license (before version 2.0), and the Sleepycat Public License, which is an OSI-approved open-source license as well as an FSF-approved free software license. The product ships with complete source code, build script, test suite, and documentation. The code quality and general utility along with the licensing terms have led to its use in a multitude of free and open-source software. Those who do not wish to abide by the terms of the GNU AGPL, or use an older version with the Sleepycat Public License, have the option of purchasing another proprietary license for redistribution from Oracle Corporation. This technique is called dual licensing.
Berkeley DB has an architecture notably simpler than that of other database systems like relational database management systems. For example, like SQLite, it does not provide support for network access – programs access the database using in-process API calls. Oracle added support for SQL in 11g R2 release based on the popular SQLite API by including a version of SQLite in Berkeley DB. There is third party support for PL/SQL in Berkeley DB via a commercial product named Metatranz StepSqlite.
A program accessing the database is free to decide how the data is to be stored in a record. Berkeley DB puts no constraints on the record's data. The record and its key can both be up to four gigabytes long.
The name "Berkeley DB" is given to three different products:
- Berkeley DB – the current version as of April 18, 2016 is 6.2.
- Berkeley DB Java Edition – the current version as of June 27, 2016 is 7.0.
- Berkeley DB XML – the current version as of March 14, 2016 is 6.0.18.
Each edition has separate database libraries, despite the common branding. The first is the traditional Berkeley DB, written in C. It contains several database implementations, including a B-tree and one built around extensible hashing. It supports multiple language bindings, including C/C++, Java (via JNI), C# .NET, Perl and Python.
Berkeley DB Java Edition (JE) is a pure Java database management library. Its design resembles that of Berkeley DB without replicating it exactly, and has a feature set that includes many of those found in the traditional Berkeley DB and others that are specific to the Java Edition. It has a log structured storage architecture, which gives it different performance and concurrency characteristics. Three APIs are available – a Direct Persistence Layer which is "Plain Old Java Objects" (POJO); one which is based on the Java Collections Framework (an object persistence approach); and one based on the traditional Berkeley DB API. The Berkeley DB Java Edition High Availability option (Replication) is available. Note that traditional Berkeley DB also supports a Java API, but it does so via JNI and thus requires an installed native library.
The Berkeley DB XML database specializes in the storage of XML documents, supporting XQuery via XQilla. It is implemented as an additional layer on top of (a legacy version of) Berkeley DB and the Xerces library. DB XML is written in C++ and supports multiple language bindings, including C++, Java (via JNI), Perl and Python.
Programs that use Berkeley DB
Berkeley DB provides the underlying storage and retrieval system of several LDAP servers, database systems, and many other proprietary and free/open source applications. Notable software that use Berkeley DB for data storage include:
- 389 Directory Server – An open-source LDAP server from the Fedora Project
- Bitcoin Core – The first implementation of the Bitcoin cryptocurrency
- Bogofilter – A free/open source spam filter that saves its wordlists using Berkeley DB
- Carbonado – An open source relational database access layer
- Citadel – A free/open source groupware platform that keeps all of its data stores, including the message base, in Berkeley DB
- Cyrus IMAP Server – A free/open source IMAP and POP3 server, developed by Carnegie Mellon University
- Evolution – A free/open source mail client; contacts are stored in addressbook.db using Berkeley DB
- Exim – An MTA developed at the University of Cambridge for use on Unix systems connected to the internet
- GlusterFS – Prior to v3.4, GlusterFS included a BDB storage translator
- HyperGraphDb – A general purpose, open-source data storage mechanism based on a powerful knowledge management
- Jabberd2 – An Extensible Messaging and Presence Protocol server
- KDevelop – An IDE for Unix-like operating systems
- Movable Type (until version 4.0) – A proprietary weblog publishing system developed by California-based Six Apart
- memcachedb – A persistence-enabled variant of memcached
- MySQL database system – Prior to v5.1, MySQL included a BDB data storage backend.
- OpenLDAP – A free/open source implementation of the Lightweight Directory Access Protocol (LDAP). Deprecated in favor of LMDB
- Oracle NoSQL – A NoSQL distributed key-value database
- Oracle Retail Predictive Application Server (RPAS) – RPAS (since 12.x?) uses Berkeley DB as the underlying persistence layer for the MOLAP engine used in several Oracle Retail Planning and Supply Chain products. (Berkeley DB replaced the previous Accumate/Acumen persistence layer used since the original development of the RPAS product by Neil Thall Associates, which was no longer supported by its final owner, Lucent and no longer sufficiently scalable).
- Postfix – A fast, secure, easy-to-administer MTA for Linux/Unix systems. Deprecated in favor of LMDB
- Parallel Virtual File System (PVFS) – A parallel file system for HPC clusters.
- Red Dwarf – A server framework originally developed by Sun, now open sourced, commonly used for game development.
- RPM – The RPM Package Manager uses Berkeley DB to retain its internal database of packages installed on a system
- Sendmail – A popular MTA for Linux/Unix systems
- Spamassassin – An anti-spam application
- Subversion – A version control system designed specifically to replace CVS
- Sun Grid Engine – A free and open source distributed resource management system
Oracle Corporation makes versions 2.0 and higher of Berkeley DB available under a dual license. The Sleepycat license is a 2-clause BSD license with an additional copyleft clause similar to the GNU GPL version 2's Section 3, requiring source code of an application using Berkeley DB to be made available for a nominal fee.
As of July 2011[update], Oracle's list price for non-copyleft Berkeley DB licenses varies between US$900 and 13,800 per processor. Embedded usage within the Oracle Retail Predictive Application Server (RPAS) does not require an additional license payment.
Sleepycat License (sometimes referred to as Berkeley Database License or the Sleepycat Public License) is an OSI-approved open source license used by Oracle Corporation for the open-source editions of Berkeley DB, Berkeley DB Java Edition and Berkeley DB XML embedded database products older than version 6.0.20. (Starting with version 6.0.20, the open-source editions are instead licensed under the GNU AGPL v3.) The name of this license is derived from the name of the company which commercially sold the Berkeley DB products, Sleepycat Software, which was acquired by Oracle in 2006. Oracle continued to use the name "Sleepycat License" for Berkeley DB, despite not using the term "Sleepycat" in any other documentation until it changed to GNU AGPL with version 6.0.20.
According to the Free Software Foundation, it qualifies as a free software license, and is compatible with the GPL. The Apache Foundation does not consider it an acceptable license, for their libraries (or own code), along with other copyleft ones; however considered indirect use ok, as with Apache Subversion that use Berkeley DB.
The license is a strong form of copyleft because it mandates that redistributions in any form not only include the source code of Berkeley DB, but also "any accompanying software that uses the DB software". It is possible to circumvent this strict licensing policy through the purchase of a commercial software license from Oracle Corporation consisting of terms and conditions which are negotiated at the time of sale. This is an example of dual licensing.
The effect of the dual license creates financial exposure for commercial organizations, since there is considerable risk of becoming liable for payment of license fees to Oracle. Some people consider it to be a "sneaky" license. Mike Olson, co-founder and CEO of Sleepycat Software and Cloudera, said that "This is good business if you can get it, but your relationship with your customer begins based on a threat and that's not a really healthy place to start out."
- Berkeley DB Reference Guide: What is Berkeley DB not?. Doc.gnu-darwin.org (2001-05-31). Retrieved on 2013-09-18.
- http://doc.gnu-darwin.org/am_misc/dbsizes.html Berkeley DB Reference Guide: Database limits Retrieved on 2013-09-19
- Olson, Michael A.; Bostic, Keith; Seltzer, Margo (1999). "Berkeley DB" (PDF). Proc. FREENIX Track, USENIX Annual Tech. Conf. Retrieved October 20, 2009.
- Seltzer, Margo; Yigit, Ozan (1991). "A New Hashing Package for UNIX". Proc. USENIX Winter Tech. Conf. Retrieved October 20, 2009.
- Brunelli, Mark (March 28, 2005). "A Berkeley DB primer". Enterprise Linux News. Retrieved December 28, 2008.
- "db(3)". Retrieved April 12, 2009.
- [Berkeley DB Announce] Major Release: Berkeley DB 12gR1 (22.214.171.124). Retrieved July 5, 2013.
- "The Sleepycat License". Open Source Initiative. October 31, 2006. Retrieved December 28, 2008.
- "Licenses". Free Software Foundation. December 10, 2008. Retrieved December 28, 2008.
- "Twitter / Gregory Burd: @humanications We didn't r ...".
- "Official Berkeley DB FAQ". Oracle Corporation. Retrieved March 30, 2010.
Does Berkeley DB support PL/SQL?
- RCE 35: PVFS Parallel Virtual FileSystem
- "Open Source License for Berkeley DB". Oracle Corporation.
For a license to use the Berkeley DB software under conditions other than those described here, or to purchase support for this software, please contact firstname.lastname@example.org.[dead link]
- "Major Release: Berkeley DB 12gR1 (126.96.36.199)". June 10, 2013. Retrieved July 15, 2013.
- "Various Licenses and Comments about Them". Free Software Foundation.
- "Which licenses may NOT be included within Apache products?".
- Mike Olson (co-founder and CEO of Sleepycat Software and Cloudera), lecture to Stanford University entrepreneurship students, 2013-11-13
- Oracle Berkeley DB Site
- Berkeley DB Programmer's Reference Guide
- Licensing pitfalls for Oracle Technology Products
- The Berkeley DB Book by Himanshu Yadava
- Launchpad.net – Berkeley DB at Launchpad
- Oracle Licensing Knowledge Net
- Oracle Berkeley DB Licensing Information
- Text of the Sleepycat License (old)