RocksDB

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
RocksDB
Rocksdb-icon.svg
Developer(s) Facebook
Initial release May 2012; 6 years ago (2012-05)
Stable release
5.14.2 / July 3, 2018; 2 months ago (2018-07-03)
Written in C++
Operating system Windows, macOS, Linux, FreeBSD
Type Embedded database
License Apache 2.0 or GPL 2
Website rocksdb.org

RocksDB is a high performance[1][2][3][4][5] embedded database for key-value data. It is a fork of LevelDB which was then optimized to exploit many central processing unit (CPU) cores, and make efficient use of fast storage, such as solid-state drives (SSD), for input/output (I/O) bound workloads. It is based on a log-structured merge-tree (LSM tree) data structure. It is written in C++ and provides official application programming interface (API) language bindings for C++, C, and Java; alongside many third-party language bindings. RocksDB is open-source software, and was originally released under a BSD 3-clause license[6][7][8]. However, in July 2017 the project was migrated to a dual license of both Apache 2.0 and GPLv2 license[9], possibly in response to the Apache Software Foundation's blacklist of the previous BSD+Patents license clause.[10][11]

RocksDB is used in production systems at various web-scale enterprises[12] including Facebook, Yahoo!,[13] and LinkedIn.[14]

Features[edit]

RocksDB, like LevelDB, stores keys and values in arbitrary byte arrays, and data is sorted byte-wise by key or by providing a custom comparator.

RocksDB provides all of the features of LevelDB, plus:

and others: List of RocksDB features that are not in LevelDB.

RocksDB is not an SQL database (although MyRocks combines RocksDB with MySQL). Like other NoSQL and Dbm stores, it has no relational data model, and it does not support SQL queries. Also, it has no direct support for secondary indexes, however a user may build their own internally using Column Families or externally. Applications use RocksDB as a library, as it does not provide a server or command-line interface.

History[edit]

RocksDB was created at Facebook by Dhruba Borthakur[25][26] in April 2012, as a fork of LevelDB with the initial stated goal of improving performance for server workloads.[27][28]

Integration[edit]

As an embeddable database, RocksDB can be used as a storage engine within a larger database management system (DBMS). For example, CockroachDB uses RocksDB as its storage engine.[29]

Alternative backend[edit]

The following projects have been started to replace or offer an alternative storage engines for already-established database systems with RocksDB:

MongoDB[edit]

The MongoRocks project provides a storage module for MongoDB where the storage engine is RocksDB.[30][31][32]

A related program is Rocks Strata, a tool written in Go, which allows managing incremental backups of MongoDB when RocksDB is used as the storage engine.[33]

MySQL[edit]

The MyRocks project creates a new RocksDB based storage engine for MySQL.[34][35] In-depth details about MyRocks were presented at Percona Live 2016.[36]

Cassandra[edit]

Cassandra on RocksDB can improve the performance of Apache Cassandra significantly (3-4 times faster in general, 100 times faster in some use-cases."Cassandra on RocksDB at Percona Live 2018". ) The Instagram team at Facebook developed and open-sourced their code, along with benchmarks of their performance results."Open-sourcing a 10x reduction in Apache Cassandra tail latency". 

Embedded[edit]

The following database systems and applications have chosen to use RocksDB as their embedded storage engine:

ArangoDB[edit]

ArangoDB has replaced its previous storage engine ("mmfiles") with RocksDB[37].

Ceph's BlueStore[edit]

The Ceph's BlueStore storage layer uses RocksDB for metadata management in OSD devices[38].

LogDevice LogsDB[edit]

LogDevice's LogsDB is built atop RocksDB[39].

Faktory[edit]

Facktory[40] the background job system uses RocksDB for storage[41].

SSDB[edit]

The ssdb-rocks[42] project uses RocksDB as the storage engine for the SSDB[43] NoSQL Database.

TiDB[edit]

The TiDB[44] project uses RocksDB as its storage engine[45].

Third-party language bindings[edit]

Third-party programming language bindings available for RocksDB, include:

References[edit]

  1. ^ "Performance Benchmarks". Retrieved November 29, 2015. 
  2. ^ "Benchmarking the leveldb family". Retrieved March 10, 2016. 
  3. ^ "Comparing LevelDB and RocksDB, take 2". Retrieved March 10, 2016. 
  4. ^ "Benchmarking LevelDB vs. RocksDB vs. HyperLevelDB vs. LMDB Performance for InfluxDB". Retrieved March 10, 2016. 
  5. ^ Golan-Gueta, Guy; Bortnikov, Edward; Hillel, Eschar; Keidar, Idit (April 21, 2015). "Scaling Concurrent Log-Structured Data Stores". EuroSys '15 Proceedings of the Tenth European Conference on Computer Systems. doi:10.1145/2741948.2741973. 
  6. ^ "Facebook's latest open source effort: a flash-powered database called RocksDB". Retrieved March 10, 2016. 
  7. ^ "Under the Hood: Building and open-sourcing RocksDB". Retrieved March 10, 2016. 
  8. ^ "RocksDB - Facebook's Database Now Open Source". Retrieved March 10, 2016. 
  9. ^ "GitHub pull request". Retrieved July 20, 2017. 
  10. ^ "Apache says 'no' to Facebook code libraries". Retrieved July 20, 2017. 
  11. ^ "GitHub issue". Retrieved July 20, 2017. 
  12. ^ "Users.md". Retrieved December 1, 2015. 
  13. ^ "RocksDB on Steroids". Retrieved March 10, 2016. 
  14. ^ "Benchmarking Apache Samza: 1.2 million messages per second on a single node". Retrieved March 10, 2016. 
  15. ^ "RocksDB transactions". GitHub. Retrieved 2016-04-04. 
  16. ^ "How to backup RocksDB?". Retrieved 2017-07-19. 
  17. ^ "Checkpoints". Retrieved 2017-07-19. 
  18. ^ "Column families in RocksDB". GitHub. Retrieved 2016-04-04. 
  19. ^ "RocksDB bloom filters". GitHub. Retrieved 2016-04-04. 
  20. ^ "RocksDB TTL support". GitHub. Retrieved 2016-04-04. 
  21. ^ "Universal compaction". GitHub. Retrieved 2016-04-04. 
  22. ^ "RocksDB merge operator". GitHub. Retrieved 2016-04-04. 
  23. ^ "RocksDB perf context and IO stats context". GitHub. Retrieved 2016-04-04. 
  24. ^ "Spatial indexing in RocksDB". rocksdb.org. Retrieved 2018-07-19. 
  25. ^ "First commit where RocksDB diverges from LevelDB". May 10, 2012. Retrieved March 15, 2016. 
  26. ^ "rocksdb README file". Nov 30, 2012. Retrieved March 15, 2016. 
  27. ^ "The History of RocksDB". November 24, 2013. Retrieved March 10, 2016. 
  28. ^ Borthakur, Dhruba (November 22, 2013). "RocksDB: A High Performance Embedded Key-Value Store for Flash Storage - Data@Scale". Retrieved March 10, 2016. ... The story of why we decided to do RocksDB ... 
  29. ^ Edwards, Jessica (2015-10-29). "Hello World: Meet CockroachDB, the Resilient SQL Database". The New Stack. Retrieved 2016-07-08. 
  30. ^ "mongodb-partners/mongo-rocks". 
  31. ^ "Integrating RocksDB with MongoDB". Retrieved July 19, 2018. 
  32. ^ "MongoDB + RocksDB at Parse". Retrieved December 1, 2015. 
  33. ^ "facebookgo/rocks-strata". 
  34. ^ "facebook/mysql-5.6". 
  35. ^ "MyRocks: MySQL on RocksDB" (PDF). Retrieved November 29, 2015. 
  36. ^ "MyRocks Deep Dive". Retrieved May 9, 2016. 
  37. ^ "Comparing new RocksDB and MMFiles storage engines". 
  38. ^ "Storage Devices -- Ceph Documentation". 
  39. ^ "LogDevice: a distributed data store for logs". Mark Marchukov, Facebook. 
  40. ^ "contribsys/faktory". 
  41. ^ "Introducing Faktory". Mike Perham. 
  42. ^ "ideawu/ssdb-rocks". 
  43. ^ https://ssdb.io
  44. ^ "ideawu/ssdb-rocks". 
  45. ^ https://pingcap.github.io/blog/2017/07/11/tidbinternal1/#rocksdb
  46. ^ "warrenfalk/rocksdb-sharp". 
  47. ^ "b1naryth1ef/rocksdb". 
  48. ^ "urbint/rox". 
  49. ^ "leo-project/erocksdb". 
  50. ^ "barrel-db/erlang-rocksdb". 
  51. ^ "tecbot/gorocksdb". 
  52. ^ "rocksdb-haskell: Haskell bindings to RocksDB". 
  53. ^ "RocksJava". 
  54. ^ "rocksdb". 
  55. ^ "iabudiab/ObjectiveRocks". 
  56. ^ "RocksDB - Perl extension for RocksDB - metacpan.org". 
  57. ^ "Photonios/rocksdb-php". 
  58. ^ "stephan-hof/pyrocksdb". 
  59. ^ "rocksdb-ruby - RubyGems.org - your community gem host". 
  60. ^ "spacejam/rust-rocksdb". 

External links[edit]