Apache Drill

From Wikipedia, the free encyclopedia
Jump to: navigation, search
Apache Drill
Developer(s) Apache Software Foundation
Stable release
1.12 / December 15, 2017; 33 days ago (2017-12-15)
Development status Active
Operating system Cross-platform
Licence Apache License, Version 2.0.
Website drill.apache.org

Apache Drill is an open-source software framework that supports data-intensive distributed applications for interactive analysis of large-scale datasets. Drill is the open source version of Google's Dremel system which is available as an infrastructure service called Google BigQuery. One explicitly stated design goal is that Drill is able to scale to 10,000 servers or more and to be able to process petabytes of data and trillions of records in seconds. Drill is an Apache top-level project.[1]

Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. For example, you can join a user profile collection in MongoDB with a directory of event logs in Hadoop.

Drill's datastore-aware optimizer automatically restructures a query plan to leverage the datastore's internal processing capabilities. In addition, Drill supports data locality, if Drill and the datastore are on the same nodes.[2]

Apache Drill 1.9 added dynamic user defined functions.

Apache Drill 1.11 added cryptographic-related functions and PCAP file format support.

Features[edit]

  • Schema-free JSON document model similar to MongoDB and Elasticsearch, without requiring a formal schema to be declared
  • Industry-standard APIs: ANSI SQL, ODBC/JDBC, RESTful APIs
  • Extremely user and developer friendly
  • Pluggable architecture enables connectivity to multiple datastores

Support[edit]

Drill is primarily focused on non-relational datastores, including Apache Hadoop text files, NoSQL and cloud storage. The following datastores are supported:

A new datastore can be added by developing a storage plugin. Drill's "schema-free" JSON data model enables it to query non-relational datastores in-situ .[3]

See also[edit]

References[edit]

Papers[edit]

Some papers influenced the birth and design. Here is a partial list:

External links[edit]