Jump to content

Graph database

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by 2604:2000:400d:9a00:8111:c893:f78c:576c (talk) at 20:23, 17 December 2017 (Updated JanusGraph release to 0.2.0 (Oct 2017), per https://github.com/JanusGraph/janusgraph/releases/tag/v0.2.0). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

In computing, a graph database is a database that uses graph structures for semantic queries with nodes, edges and properties to represent and store data. A key concept of the system is the graph (or edge or relationship), which directly relates data items in the store. The relationships allow data in the store to be linked together directly, and in many cases retrieved with one operation.

This contrasts with relational databases that, with the aid of relational database management systems, permit managing the data without imposing implementation aspects like physical record chains; for example, links between data are stored in the database itself at the logical level, and relational algebra operations (e.g., join) can be used to manipulate and return related data in the relevant logical format. The execution of relational queries is possible with the aid of the database management systems at the physical level (e.g., using indexes), which permits boosting performance without modifying the logical structure of the database.

Graph databases, by design, allow simple and fast retrieval[citation needed] of complex hierarchical structures that are difficult to model[according to whom?] in relational systems. Graph databases are similar to 1970s network model databases in that both represent general graphs, but network-model databases operate at a lower level of abstraction[1] and lack easy traversal over a chain of edges.[2]

The underlying storage mechanism of graph databases can vary. Some depend on a relational engine and “store” the graph data in a table (although a table is a logical element, therefore this approach imposes another level of abstraction between the graph database, the graph database management system and the physical devices where the data is actually stored). Others use a key-value store or document-oriented database for storage, making them inherently NoSQL structures. Most[according to whom?] graph databases based on non-relational storage engines also add the concept of tags or properties, which are essentially relationships having a pointer to another document. This allows data elements to be categorized for easy retrieval en masse.

Retrieving data from a graph database requires a query language other than SQL, which was designed for the manipulation of data in a relational system and therefore cannot “elegantly” handle traversing a graph. As of 2017, no single graph query language has been universally adopted in the same way as SQL was for relational databases, and there are a wide variety of systems, most often tightly tied to one product. Some standardization efforts have occurred, leading to multi-vendor query languages like Gremlin, SPARQL, and Cypher. In addition to having query language interfaces, some graph databases are accessed through application programming interfaces (APIs).

Description

Graph databases employ nodes, properties, and edges.

Graph databases are based on graph theory, and employ nodes, edges, and properties.

  • Nodes represent entities such as people, businesses, accounts, or any other item to be tracked. They are roughly the equivalent of the record, relation, or row in a relational database, or the document in a document database.
  • Edges, also termed graphs or relationships, are the lines that connect nodes to other nodes; they represent the relationship between them. Meaningful patterns emerge when examining the connections and interconnections of nodes, properties, and edges. Edges are the key concept in graph databases, representing an abstraction that is not directly implemented in other systems.
  • Properties are germane information that relate to nodes. For example, if Wikipedia were one of the nodes, it might be tied to properties such as website, reference material, or word that starts with the letter w, depending on which aspects of Wikipedia are germane to a given database.

The relational model gathers data together using information in the data. For example, one might look for all the "users" whose phone number contains the area code "311". This would be done by searching selected datastores, or tables, looking in the selected phone number fields for the string "311". This can be a time consuming process in large tables, so relational databases offer the concept of a database index, which allows data like this to be stored in a smaller subtable, containing only the selected data and a unique key (or primary key) of the record it is part of. If the phone numbers are indexed, the same search would occur in the smaller index table, gathering the keys of matching records, and then looking in the main data table for the records with those keys. Generally, the tables are physically stored so that lookups on these keys are fast.[3]

Relational databases do not inherently contain the idea of fixed relationships between records. Instead, related data is linked to each other by storing one record's unique key in another record's data. For example, a table containing email addresses for users might hold a data item called userpk, which contains the primary key of the user record it is associated with. In order to link users and their email addresses, the system first looks up the selected user records primary keys, looks for those keys in the userpk column in the email table (or more likely, an index of them), extracts the email data, and then links the user and email records to make composite records containing all the selected data. This operation, termed a join, can be computationally costly. Depending on the complexity of the query, the number of joins, and the indexing of the various keys, the system may have to search through multiple tables and indexes, gather lots of information, and then sort it all to match it together.[3]

In contrast, graph databases directly store the relationships between records. Instead of an email address being found by looking up its user's key in the userpk column, the user record has a pointer directly to the email address record. That is, having selected a user, the pointer can be followed directly to the email records, there is no need to search the email table to find the matching records. This can eliminate the costly join operations. For example, if one searches for all of the email addresses for users in area code "311", the engine would first perform a conventional search to find the users in "311", but then retrieve the email addresses by following the links found in those records. A relational database would first find all the users in "311", extract a list of the pk's, perform another search for any records in the email table with those pk's, and link the matching records together. For these types of common operations, a graph database (in theory at least) is significantly faster.[3]

The true value of the graph approach becomes evident when one performs searches that are more than one level deep. For example, consider a search for users who have "subscribers" (a table linking users to other users) in the "311" area code. In this case a relational database has to first look for all the users with an area code in "311", then look in the subscribers table for any of those users, and then finally look in the users table to retrieve the matching users. In contrast, a graph database would look for all the users in "311", then follow the back-links through the subscriber relationship to find the subscriber users. This avoids several searches, lookups and the memory involved in holding all of the temporary data from multiple records needed to construct the output. Technically, this sort of lookup is completed in O(log(n)) + O(1) time, that is, roughly relative to the logarithm of the size of the data. In contrast, the relational version would be multiple O(log(n)) lookups, plus more time to join all the data.[3]

The relative advantage of graph retrieval grows with the complexity of a query. For example, one might want to know "that movie about submarines with the actor who was in that movie with that other actor that played the lead in Gone With the Wind". This first requires the system to find the actors in Gone With the Wind, find all the movies they were in, find all the actors in all of those movies who were not the lead in Gone With the Wind, and then find all of the movies they were in, finally filtering that list to those with descriptions containing "submarine". In a relational database this will require several separate searches through the movies and actors tables, doing another search on submarine movies, finding all the actors in those movies, and the comparing the (large) collected results. In contrast, the graph database would simply walk from Gone With the Wind to Clark Gable, gather the links to the movies he has been in, gather the links out of those movies to other actors, and then follow the links out of those actors back to the list of movies. The resulting list of movies can then be searched for "submarine". All of this can be done via one search.[4]

Properties add another layer of abstraction to this structure that also improves many common queries. Properties are essentially labels that can be applied to any record, or in some cases, edges also. For example, one might label Clark Gable as "actor", which would then allow the system to quickly find all the records that are actors, as opposed to director or camera operator. If labels on edges are allowed, one could also label the relationship between Gone With the Wind and Clark Gable as "lead", and by performing a search on people that are "lead" "actor" in the movie Gone With the Wind, the database would produce Vivien Leigh, Olivia de Havilland and Clark Gable. The equivalent SQL query would have to rely on added data in the table linking people and movies, adding more complexity to the query syntax. These sorts of labels may improve search performance under certain circumstances, but are generally more useful in providing added semantic data for end users.[4]

Relational databases are very well suited to flat data layouts, where relationships between data is one or two levels deep. For example, an accounting database might need to look up all the line items for all the invoices for a given customer, a three-join query. Graph databases are aimed at datasets that contain many more links. They are especially well suited to social networking systems, where the "friends" relationship is essentially unbounded. These properties make graph databases naturally suited to types of searches that are increasingly common in online systems, and in big data environments. For this reason, graph databases are becoming very popular for large online systems like Facebook, Google, Twitter, and similar systems with deep links between records.

Properties

Compared with relational databases, graph databases are often faster for associative data sets[citation needed] and map more directly to the structure of object-oriented applications. They can scale more naturally[citation needed] to large data sets as they do not typically need costly join operations (here costly means when executed on databases with non-optimal designs at the logical and physical levels). As they depend less on a rigid schema, they are marketed as more suitable to manage ad hoc and changing data with evolving schemas. Conversely, relational database management systems are typically faster at performing the same operation on large numbers of data elements, permitting the manipulation of the data in its natural structure.

Graph databases are a powerful tool for graph-like queries. For example, computing the shortest path between two nodes in the graph. Other graph-like queries can be performed over a graph database in a natural way (for example graph's diameter computations or community detection).

History

In the pre-history of graph databases, in the mid-1960s Navigational databases such as IBM's IMS supported tree-like structures in its hierarchical model, but the strict tree structure could be circumvented with virtual records.[5][6]

Graph structures could be represented in network model databases from the late 1960s. CODASYL, which had defined COBOL in 1959, defined the Network Database Language in 1969.

Labeled graphs could be represented in graph databases from the mid-1980s, such as the Logical Data Model.[1][7]

Several improvements to graph databases appeared in the early 1990s, accelerating in the late 1990s with endeavors to index web pages.

In the mid-late 2000s, commercial atomicity, consistency, isolation, durability (ACID) graph databases such as Neo4j and Oracle Spatial and Graph became available.

In the 2010s, commercial ACID graph databases that could be scaled horizontally became available. Further, SAP HANA brought in-memory and columnar technologies to graph databases.[8] Also in the 2010s, multi-model databases that supported graph models (and other models such as relational database or document-oriented database) became available, such as OrientDB, ArangoDB, and MarkLogic (starting with its 7.0 version). During this time, graph databases of various types have become especially popular with social network analysis with the advent of social media companies.

List of graph databases

The following is a list of notable graph databases:

Name Version License Language Description
AllegroGraph 5.1 (May 2015) Proprietary, clients: Eclipse Public License v1 C#, C, Common Lisp, Java, Python Resource Description Framework (RDF) and graph database
ArangoDB 3.2.0 (July 2017) Free, Apache 2 C++, JavaScript The most popular (as of 2015) NoSQL database available under an open source license and that provides both document store and triple store abilities[9]
Blazegraph 2.1 (April 2016) commercial, or GPLv2 for evaluation Java RDF-graph database capable of clustered deployment and graphics processing unit (GPU), in commercial version; supports high availability (HA) mode, embedded mode, single server mode. Supports the Blueprints and SPARQL.[10][11]
Cayley 0.6.1 (April 2017) Free, Apache 2 Go Graph database[12]
Dgraph 0.9.4 (December 3, 2017) Free, AGPLv3 for server, Apache 2 for client Go Open source, scalable, distributed, highly available and fast graph database, designed from ground up to be run at web scale.[13][14]
DataStax Enterprise Graph v5.0.2 (August 2016) Proprietary Java Distributed, real-time, scalable database inspired by Titan; supports Tinkerpop and integrates with Cassandra[15]
Sparksee[16] 5.2.0 (2015) Proprietary, commercial, freeware for evaluation, research, development C++ High-performance scalable database management system from Sparsity Technologies; main trait is its query performance for retrieving & exploring large networks; has bindings for Java, C++, C#, Python, and Objective-C; version 5 is the first graph mobile database
GraphBase[17] 1.0.03b Proprietary, commercial Java A customizable, distributed, small size graph store with a rich tool set from FactNexus.
gStore[18] 0.4.1 (March 2017) BSD-3 C++ An engine to manage large graph-structured data; open-source for Linux operating systems; written fully in C++, with some libraries such as readline, antlr, etc.; use modes: native, server-client, or distributed.[19]
InfiniteGraph 3.0 (January 2013) Proprietary, commercial Java Distributed and cloud-enabled
JanusGraph 0.2.0 (October 2017) Free, Apache 2 Java Distributed graph database forked from Titan[20][21]
MarkLogic 8.0.4 (2015) Proprietary, freeware developer version Java Multi-model NoSQL database that stores documents (JSON and XML) and semantic graph data (RDF triples); also has a built-in search engine and a full-list of enterprise features such as ACID transactions, high availability and disaster recovery, certified security, scalability, and elasticity
Neo4j 3.1.1 (January 2017)[22] GPLv3 Community Edition, commercial & AGPLv3 options for enterprise and advanced editions Java, .NET, JavaScript, Python, Ruby Highly scalable open source, supports ACID, has high-availability clustering for enterprise deployments, and comes with a web-based administration tool that includes full transaction support and visual node-link graph explorer; accessible from most programming languages using its built-in REST web API interface, and a proprietary Bolt protocol with official drivers; most popular graph database in use as of January 2017[23]
OpenLink Virtuoso 8.0 (September 2017) Open Source Edition is GPLv2, Enterprise Edition is proprietary C, C++ Hybrid database server handling RDF and other graph data, RDB-SQL data, XML data, filesystem documents-objects, and free text; may be deployed as a local embedded instance (as used in the NEPOMUK Semantic Desktop), a one-instance network server, or a shared-nothing elastic-cluster multiple-instance networked server[24]
Oracle Spatial and Graph; part of Oracle Database 12.1.0.2 (2014) Proprietary Java, PL/SQL 1) RDF Semantic Graph: comprehensive W3C RDF graph management in Oracle Database with native reasoning and triple-level label security. 2) Network Data Model property graph: for physical/logical networks with persistent storage and a Java API for in-memory graph analytics
OrientDB 2.2.24 (July 2017) Community Edition is Apache 2, Enterprise Edition is commercial Java Second generation distributed graph database with the flexibility of documents in one product (i.e., it is both a graph database and a document NoSQL database at the same time); it has an open source commercial friendly (Apache 2) license; and is a highly scalable with full ACID support; it has a multi-master replication and sharding; supports schema-less, -full, and -mixed modes; has a strong security profiling system based on user and roles; supports a query language that is so similar to SQL which is friendly to those coming from a SQL and relational database background decreasing the learning curve needed. It has HTTP REST + JSON API.
SAP HANA SPS12 Revision 120 Proprietary C, C++, Java, JavaScript & SQL-like language In-memory ACID transaction supported property graph[25]
Sqrrl Enterprise 2.0 (February 2015) Proprietary Java Distributed, real-time graph database featuring cell-level security and mass-scalability[26]
Teradata Aster 7 (2016) Proprietary Java, SQL, Python, C++, R High performance, multi-purpose, highly scalable, and extensible MPP database incorporating patented engines supporting native SQL, MapReduce and Graph data storage and manipulation; provides an extensive set of analytic function libraries and data visualization abilities [27]
TigerGraph[28] 1.0 (2017) Proprietary C++ High performance, high scalability, both scale-up and scale-out native parallel graph database, supporting high expressive GSQL, RESTful API, and visualization GraphStudio SDK.
Microsoft SQL Server 2017[29] RC1 Proprietary SQL/T-SQL, R, Python Offers graph database abilities to model many-to-many relationships. The graph relationships are integrated into Transact-SQL and receive the benefits of using SQL Server as the foundational database management system.

APIs and graph query-programming languages

See also

References

  1. ^ a b Angles, Renzo; Gutierrez, Claudio (1 Feb 2008). "Survey of graph database models" (PDF). ACM Computing Surveys. 40 (1). Association for Computing Machinery. doi:10.1145/1322432.1322433. Retrieved 28 May 2016. network models [...] lack a good abstraction level: it is difficult to separate the db-model from the actual implementation
  2. ^ Silberschatz, Avi (28 January 2010). Database System Concepts, Sixth Edition (PDF). McGraw-Hill. p. D-29. ISBN 0-07-352332-1.
  3. ^ a b c d "From Relational to Graph Databases". Neo4j.
  4. ^ a b "Examples where Graph databases shine: Neo4j edition", ZeroTurnaround
  5. ^ Silberschatz, Avi (28 January 2010). Database System Concepts, Sixth Edition (PDF). McGraw-Hill. p. E-20. ISBN 0-07-352332-1.
  6. ^ Parker, Lorraine. "IMS Notes". vcu.edu. Retrieved 31 May 2016.
  7. ^ Kuper, Gabriel M (1985). The Logical Data Model: A New Approach to Database Logic (PDF) (Ph.D.). Docket STAN-CS-85-1069. Retrieved 31 May 2016.
  8. ^ "SAP Announces New Capabilities in the Cloud with HANA". 2014-10-22. Retrieved 2016-07-07.
  9. ^ Fowler, Adam (February 24, 2015). NoSQL for Dummies. John Wiley & Sons. pp. 298–. ISBN 978-1-118-90574-6.
  10. ^ Vaughan, Jack (January 25, 2016). "Beyond gaming, GPU technology takes on graphs, machine learning". TechTarget. Retrieved May 9, 2017.
  11. ^ Yegulalp, Serdar (September 26, 2016). "Faster with GPUs: 5 turbocharged databases". InfoWorld. Retrieved May 9, 2017.
  12. ^ "Google Releases Cayley Open-Source Graph Database". eWeek. November 13, 2014. Retrieved May 9, 2017.
  13. ^ "Ex-Googler startup DGraph Labs raises US$1.1 million in seed funding round to build industry's first open source, native and distributed graph database". Globenewswire. May 17, 2016. Retrieved July 31, 2017.
  14. ^ Bailey, Michael (May 18, 2016). "Cannon-Brookes, Blackbird, Bain back new migrant's graph start-up". afr.com. The Australian Financial Review. Retrieved July 31, 2017.
  15. ^ Woodie, Alex (June 21, 2016). "Beyond Titan: The Evolution of DataStax's New Graph Database". Datanami. Retrieved May 9, 2017.
  16. ^ "Sparksee high-performance graph database". Sparsity-technologies. Retrieved May 9, 2017.
  17. ^ Longbottom, Clive (May 1, 2016). "Graph databases: What are the benefits for CIOs?". Computer Weekly. Retrieved May 9, 2017.
  18. ^ "gStore Graph Database Engine".
  19. ^ Zou, Lei; Özsu, M. Tamer; Chen, Lei; Shen, Xuchuan; Huang, Ruizhe; Zhao, Dongyan (August 2014). "gStore: a graph-based SPARQL query engine".
  20. ^ He, Jing Chen (January 16, 2017). "JanusGraph – A Graph DB that carries forward the legacy of Titan". IBM. Retrieved May 9, 2017.
  21. ^ Woodie, Alex (January 13, 2017). "JanusGraph Picks Up Where TitanDB Left Off". Datanami. Retrieved May 9, 2017.
  22. ^ "Release Notes: Neo4j 3.1.1". Neo4j. Retrieved May 9, 2017.
  23. ^ "Ranking of Graph DBMS". DB-Engines. Retrieved May 9, 2017.
  24. ^ "Clustering Deployment Architecture Diagrams for Virtuoso". Virtuoso Open-Source Wiki. OpenLink Software. Retrieved May 9, 2017.
  25. ^ Rudolf, Michael; Paradies, Marcus; Bornhövd, Christof; Lehner, Wolfgang. The Graph Story of the SAP HANA Database (PDF). Lecture Notes in Informatics. {{cite conference}}: External link in |conferenceurl= (help); Unknown parameter |conferenceurl= ignored (|conference-url= suggested) (help)
  26. ^ Vanian, Jonathan (18 February 2015). "NSA-linked Sqrrl eyes cyber security and lands $7M in funding". Gigaom. Retrieved May 9, 2017.
  27. ^ Woodie, Alex (October 23, 2015). "The Art of Analytics, Or What the Green-Haired People Can Teach Us". Datanami. Retrieved May 9, 2017.
  28. ^ "Introducing TigerGraph, a Native Parallel Graph Database". 2017-09-19. Retrieved 2017-09-19.
  29. ^ "What's New in SQL Server 2017". Microsoft Docs. April 19, 2017. Retrieved May 9, 2017.
  30. ^ Svensson, Johan (5 July 2016). "Guest View: Relational vs. graph databases: Which to use and when?". sdtimes.com. BZ Media. Retrieved 30 August 2016.
  31. ^ TinkerPop, Apache. "Apache TinkerPop". tinkerpop.apache.org. Retrieved 2016-11-02.
  32. ^ "GSQL". doc.tigergraph.com/dev/. Retrieved 2017-10-01.
  33. ^ Kogan, Lior (2017). "V1: A Visual Query Language for Property Graphs". arXiv:1710.04470.
  34. ^ "V1: A Visual Query Language for Property Graphs". github.com. Retrieved 2017-11-01.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Graph_database&oldid=815881316"