This article needs additional citations for verification. (August 2017) (Learn how and when to remove this template message)
The Oracle Exadata Database Machine (Exadata) integrates Oracle Corporation hardware and software to create a computing platform that is specialized and optimized for running the Oracle Database. The goal of Exadata is to achieve higher performance and availability at lower cost by moving database algorithms and intelligence into storage and networking, bypassing the traditional processing layers.
Exadata combines scale-out compute servers, scale-out storage servers, InfiniBand networking and specialized Exadata Software, packaged in one or more hardware racks, in a wide range of sizes. Exadata compute servers use Intel Xeon processors and the Oracle Linux operating system to run Oracle Database software. Exadata storage servers use Intel Xeon processors to perform block storage functions and run Exadata Software that offloads data-intensive database processing.
Exadata debuted in 2008 as the first in Oracle Corporation's family of Engineered Systems for use in corporate data centers. In October 2015, Exadata became available in the Oracle Cloud as a subscription service, known as the Exadata Cloud Service. Databases deployed in this service include all the features and options of Oracle Database Enterprise Edition. Similarly, all Exadata features and capabilities are included in this service. Oracle databases in the Exadata Cloud Service are 100% compatible with databases deployed on-premises, which enables customers to transition to the Oracle Cloud with no application changes. Oracle Corporation manages this service – including hardware, network, Linux software and Exadata software, while customers have complete ownership of their databases.
In early 2017, a third Exadata deployment choice became available. Exadata Cloud at Customer is Exadata Cloud Service technology deployed on-premises and managed by Oracle Cloud experts. The Oracle Cloud at Customer program is intended to bring all the benefits of the Oracle public cloud while still satisfying security and regulatory constraints.
The Evolution of Exadata
Oracle Corporation releases a new generation of Exadata every twelve to eighteen months on average. At each release, Oracle refreshes most hardware components to the latest Intel Xeon processors, memory, disk, flash and networks. The hardware refreshes in themselves result in significant performance increases with every release. In addition, each generation of Exadata adds software features that improve some combination of performance, availability, security, management and workload consolidation. Beginning in 2015, features to support the Oracle Cloud were introduced.
The emphasis of each Exadata generation is described below.
Exadata V1, released in 2008, focused on accelerating Data Warehousing by delivering the full throughput of storage to the database. Per Oracle, Exadata achieves this by moving database filtering operations into storage, instead of sending all data to the compute servers and filtering it there. By filtering directly in storage, Exadata sends significantly less data from storage servers to compute servers, thus bypassing bottlenecks that exist in the network and inside storage arrays. Oracle refers to this capability as Exadata Smart Scan. Exadata V1 also supported a consolidation feature for allocating IO bandwidth between databases or workloads, called IORM (IO Resource Manager).
Exadata V1 was available in Full Rack or Half Rack sizes, and the choice of High Performance or High Capacity storage servers.
Exadata V2, released in 2009, added a Quarter Rack configuration and support for OLTP workloads via Flash storage and database-aware Flash Caching. Unlike standard caching algorithms, Oracle claims Exadata can improve the effectiveness of caching by understanding the database data structures and algorithms. For example, Exadata Flash Caching knows when a table is too big to fit entirely in cache and therefore will not cache table scans for that table.
Exadata V2 also introduced Hybrid Columnar Compression to reduce the amount of storage consumed by large Data Warehousing tables. Oracle claimed average compression ratios of 10x to 15x. Compression also has the potential to improve performance by reducing the number of data blocks that need to be read.
Storage Indexes in Exadata V2 increased performance by eliminating the need to read entire regions of storage, based on knowledge of the data contained in that region.
Exadata X2-2, the third generation, was released in 2010 and a second model of Exadata, Exadata X2-8, was introduced. The X2-8 and subsequent “8 socket” Exadata models feature Intel processors targeted at large memory, scale-up workloads. The use of Flash storage beyond caching began in this release with a Smart Flash Logging feature. Support for 10 Gigabit Ethernet connectivity was also added.
Data security through encryption was encouraged with the incorporation of hardware decryption in Exadata X2-2, largely eliminating the performance overhead compared to software decryption.
A Storage Expansion Rack based on Exadata X2-2 was added in 2011 to accommodate large, fast-growing Data Warehouses and archival databases.
Exadata X3-2 and X3-8 were released in 2012, including a new Eighth Rack X3-2 entry-level configuration. Flash storage capacity quadrupled and OLTP write throughput increased by 20x via the Write-Back Flash Cache feature.
A number of availability enhancements were added, bypassing slow or failed storage media, reducing the duration of storage server brownouts and simplifying replacement of failed disks.
Exadata X4-2 was released in 2013. Flash capacity doubled and Flash compression was added, effectively doubling capacity again. Network Resource Management was introduced, automatically prioritizing critical messages. InfiniBand bandwidth doubled with support for active/active connections.
Exadata X4-8 released in 2014, plus Capacity on Demand licensing, IO latency capping and timeout thresholds.
Exadata X5-2 and X5-8 were released in 2015 with a major set of enhancements. Flash and disk capacity doubled. Elastic configurations were introduced to enable expansion one server at a time. Virtualization was added as an option to Exadata along with Trusted Partitions for flexible licensing within a virtual machine. Database snapshots on Exadata storage enabled efficient development and testing. Oracle Database In-memory on Exadata included Fault Tolerant redundancy. The High Performance Exadata storage servers were replaced with all-flash (Extreme Performance) storage servers and Exadata became the first major vendor to adopt the NVMe Flash interface. Columnar Flash cache was introduced to automatically reformat analytics data into row format in Flash. Exafusion Direct-to-Wire protocol reduced messaging overhead in a cluster. IPv6 support was completed. Exadata Cloud Service was launched on the Oracle Cloud.
Exadata X6-2 and X6-8 were released in 2016. Flash capacity doubled. Exadata Cloud at Customer debuted, enabling Oracle Cloud benefits within corporate data centers.
Exadata X7-2 and X7-8 were released in 2017. Flash capacity doubled. Flash cards became Hot pluggable for online replacement. 10 TB drives debuted along with 25 Gigabit Ethernet connectivity. Oracle Database In-Memory processing was extended into Flash storage, and storage server memory was utilized for faster OLTP.
Exadata is designed to run any Oracle Database workload or mix of workloads, such as combining OLTP and Analytics processing.
Long running requests, characterized by data warehouse queries or other queries, reports, batch jobs and analytics, are reputed to run many times faster compared to a conventional, non-Exadata database server. Customer references often cite performance gains of 10x or greater. Analytics workloads can also use the Oracle Database In-Memory option on Exadata for additional acceleration. Exadata’s "Hybrid Columnar Compression" feature is intended to reduce the storage consumption of data warehouses and archive data as well as increase performance by reducing the amount of I/O.
Transactional (OLTP) workloads on Exadata benefit from the incorporation of flash memory into Exadata’s storage hierarchy, and the automatic "tiering" of data into memory, flash or disk storage. Special flash algorithms optimize flash for response time sensitive database workloads such as log writes. For high-end OLTP, all-flash storage eliminates the latency of disk media completely.
All Exadata workloads benefit from an internal InfiniBand fabric running a specialized database network protocol called iDB.
Exadata is available in one of two variants: one based on two-socket database servers and the other based on eight-socket database servers. The two models differ only in the hardware used for the compute servers. The networking, storage servers and software are the same in both models.
The most recent Oracle Exadata Database Machine is the X7 generation, introduced in October 2017.
The X7-2 compute servers feature a small form factor, 1 RU (Rack Unit) in height. They employ 2-socket Intel Xeon processors; each socket with 24 compute cores for 48 total cores per compute server. Memory starts at 384 GB and can be expanded to 1.5 TB.
The Exadata Database Machine base configuration has 2 compute servers and 3 storage servers, referred to as a "Quarter Rack". The same hardware is also available in an "Eighth Rack" configuration with half of the processing turned off and half of the storage capacity either turned off or removed until needed. As the database workload and/or data size increases, additional compute and storage servers may be added to increase the volume of work performed in parallel. This is commonly referred to as "scale-out".
The X7-8 compute server uses eight-socket servers that consume 5 RU in height and have greater memory capacity than the X7-2. Whereas each X7-2 compute server contains 48 compute cores, the X7-8 server contains 192 compute cores. This allows large database workloads to easily "scale-up" within a compute server while still supporting Exadata’s "scale-out" expandability across multiple servers. The larger memory capacity of the X7-8 also favors in-memory databases and very large OLTP, consolidation, and DW workloads. Like the X7-2, the Exadata X7-8 base configuration has 2 compute servers and 3 storage servers, but consumes a "Half Rack" of space. Additional compute and storage servers may be added until the rack is full.
There are two choices for Exadata storage servers; Extreme Flash and High Capacity. The Extreme Flash is an all-flash storage server containing 8 PCIe flash drives for a total of 51.2 terabytes of raw storage capacity in its current X7-2 configuration. The High Capacity storage server in its current X7-2 configuration contains 12 disks, 10TB each, for a total of 120TB of raw storage capacity. To improve I/O response times, the High-Capacity storage server also employ 25.6TB of PCIe flash to cache active data blocks. Exadata’s Smart Flash Cache, Smart Flash Log, Columnar Flash Cache and Write Back Flash Cache features determine how and when to use flash (see Storage Server Software).
In addition to adding storage servers into an Exadata Database Machine base configuration, storage servers may also be acquired with or added to Exadata Storage Expansion racks.
Storage performance specifications for a full rack Exadata configuration are as follows:
|Exadata Storage Server||Scan Rate||Read IOPS||Write IOPS|
|X7-2 Extreme Flash||350 GB/sec||5,970,000||5,400,000|
|X7-2 High Capacity||350 GB/sec||4,776,000||4,352,000|
Note: Based on an elastic configuration of 12 storage servers, 10 compute servers.
IOPS = 8K I/O Operations per Second from SQL
The Exadata Database Machine provides high-speed networks for internal and external connectivity. A 40 Gb/Sec InfiniBand network is used for internal connectivity between compute and storage servers and 10Gb/Sec and 1Gb/Sec Ethernet ports are included for data center connectivity. The InfiniBand network is also used as the cluster interconnect between compute servers. Exadata has a "direct-to-wire" protocol that allows the database to talk directly to the InfiniBand hardware, minimizing operating system overhead.
Prior to the X5-2 generation, Exadata systems were only available in fixed-size configurations of Eighth, Quarter, Half and Full Rack sizes. With the X5-2 Exadata release in January, 2015, "elastic configurations" were introduced. An elastic configuration has a customer-specified combination of database servers and storage servers. Elastic configurations allow individual storage or compute servers to be added to a base configuration until the physical rack is full. For example, an Exadata system optimized for in-memory database processing could be created by adding many compute servers, each with maximum memory. Conversely, an Exadata system optimized for a large data warehouse could be configured by adding many High-Capacity storage servers. The ratio of compute to storage servers can vary, depending on the characteristics of the intended workload. Elastic configurations may also be used to scale out earlier generation Exadata systems using X6-2 servers. In addition, Exadata Database Machines have always been able to span multiple racks using the built-in InfiniBand network connections. Thus, Exadata’s scale-out extends beyond a single physical rack.
Storage server software
Much of the technical differentiation in Exadata is contained in the software that runs in the Exadata storage servers. Unlike conventional storage arrays, Exadata storage servers employ Intel Xeon processors to execute database operations such as scans, joins and filtering; a feature known as Smart Scans. Running query filters directly in storage servers as data streams from disk and flash bypasses network limitations and offloads processing that would otherwise be performed in the database servers.
Storage server software also implements Smart Flash Cache, Columnar Flash Cache, Flash Logging and Write Back Flash Cache routines that use flash storage to improve I/O response times. I/O Resource Management allocates I/O bandwidth to databases or workloads according to specified priorities. Lastly, decompression of Hybrid Columnar Compressed data may be performed in Exadata storage servers.
Compute server software
Exadata compute servers run the Oracle Linux 6 operating system and Oracle Database 11g Release 2 Enterprise Edition or Oracle Database 12c Enterprise Edition. Exadata system resources can be optionally virtualized using the Xen-based Oracle VM. All Oracle Database options, such as Real Application Clusters, Multitenant, Database In-Memory, Advanced Compression, Advanced Security, Partitioning, Active Data Guard and others are optionally available with Exadata. Applications that are certified to a supported version of Oracle Database are automatically compatible with Exadata. No additional modifications or certifications are required.
Oracle Enterprise Manager 12c (EM) manages Oracle software and hardware, including the Exadata Database Machine. EM integrates with the built-in Exadata management tooling, as well as with customer’s existing systems management and helpdesk tools. Exadata plug-in for EM provides an integrated view of compute servers, storage servers, switches, and topology. In addition, it also provides discovery, monitoring and alerting capability for Exadata systems management.
There have been several generations of Exadata products.
The hardware components are not unique to Exadata. It is the integration of hardware and software components that distinguish Exadata from other database systems.
|Exadata generation (2-socket)||v1||v2||X2-2||X3-2||X4-2||X5-2||X6-2||X7-2|
|Disk storage (raw TB)||168||336||504||504||672||1344||1344||1680|
|Flash cache (raw TB)||N/A||5.3||5.3||22.4||44.8||89.6||179.2||358.4|
|Extreme flash (raw TB)||N/A||N/A||N/A||N/A||N/A||179.2||358.4||716.8|
|Max memory (Gio)||256||576||1,152||2,048||4,096||6,144||6,144|
|Exadata generation (8-socket)||N/A||N/A||X2-8||X3-8||X4-8||X5-8||X6-8||X7-8|
|Disk storage (raw TB)||N/A||N/A||504||504||672||1344||1344||1680|
|Flash cache (raw TB)||N/A||N/A||5.3||22.4||89.6||89.6||179.2||358.4|
|Extreme flash (raw TB)||N/A||N/A||N/A||N/A||179.2||179.2||358.4||716.8|
|Max memory (TB)||N/A||N/A||4||4||12||12||12||12|
- Millsap, Cary (Aug 24, 2012). "My Perspective on Exadata: E42012 Conference". Retrieved July 15, 2018.
- "Exadata Customer Success Videos". Retrieved July 16, 2018.
- "Oracle Exadata Configuration Assistant (OECA)". Oracle.
- "What's New in Oracle Exadata Database Machine". Oracle. Retrieved 2015-11-24.
- "New Oracle Exadata X7 Delivers In-Memory Performance from Shared Storage". Oracle. Retrieved 2017-10-25.
|Wikimedia Commons has media related to Oracle Exadata.|
- Oracle Steps With Moore’s Law To Rev Exadata Database Machines
- Oracle Exadata Database Machine
- Exadata X5-2 Data Sheet
- Benefits of running E-Business Suite on Exadata and Exalogic
- "History of Exadata", Flash DBA.
- Tanel Poder’s Exadata blog, from an expert in performance tuning on Exadata
- Arup Nanda’s blog, where you can search for Exadata-related posts
- The Enkitec blog, from a company specializing in Exadata and sponsoring yearly Exadata-centric conferences