oracle exadata storage technical overview · 2010-03-26 · • no exadata certification required...

<Insert Picture Here>

Oracle Exadata Technical Overview

Copyright © 2010, Oracle Corporation and/or its affiliates

Copyright © 2010, Oracle Corporation and/or its affiliates – 2 –

• Best Machine for Data Warehousing

• Best Machine for OLTP

• Best Machine for Database Consolidation

• Unique Architecture Makes it

• Fastest, Lowest Cost

Exadata Database MachineBest Platform to Run the Oracle Database

Exadata in the MarketplaceRapid adoption in all geographies and industries


http://www.huntermanufacturing.org.au/drupal/?q=node/11

Exadata Hardware Architecture

Database Grid Intelligent Storage Grid

InfiniBand Network

• Redundant 40Gb/s switches

• Unified server & storage network

• 14 High-performance low-cost

storage servers• 8 Dual-processor x64 database servers

OR

• 2 Eight-processor x64 database servers

Scaleable Grid of industry standard servers for Compute and Storage

• Eliminates long-standing tradeoff between Scalability, Availability, Cost

• 100 TB High Performance disk, or336 TB High Capacity disk

•5.3 TB PCI Flash

•Data mirrored across storage servers


Standardized and Simple to Deploy

• All Database Machines are the same

• Delivered ready-to-run

• Tested

• Highly supportable

• No unique configuration issues

• Identical to config used by Oracle Engineering

• Runs existing OLTP and DW applications

• Full 30 years of Oracle DB capabilities

• No Exadata certification required

• Leverages Oracle ecosystem

• Skills, knowledge base, people, partners

Deploy in Days,

Not Months


Exadata Storage Server Building Block

• High-performance storage server built from

industry standard components

• 12 disks - 600 GB 15000 RPM High

Performance SAS or 2TB 7200 RPM High

Capacity SAS

• 2 Six-Core Intel Xeon Processors (L5640)

• Dual ported 40 Gb/sec InfiniBand

• 4 x 96 GB Flash Cards

• Intelligent Exadata Storage Server Software

• Hardware by Sun

• Software by Oracle


Copyright © 2010 Oracle Corporation and/or its affiliates – 7 –

New - Exadata Database Machine X2-8 Full Rack Extreme Performance for Consolidation, Large OLTP and DW

• 2 x64 Eight-processor Database servers (Sun Fire 4800)

• High Core, High Memory Database Servers

• 128 CPU cores (64 per server)

• 2 TB (1 TB per server)

• 10 GigE connectivity to Data Center

• 16 x 10GbE ports (8 per server)

• 14 Exadata Storage Servers X2-2

• All with High Performance 600GB SAS disks

OR

• All with High Capacity 2 TB SAS disks

• 3 Sun Datacenter InfiniBand Switch 36

• 36-port Managed QDR (40Gb/s) switch

• 1 “Admin” Cisco Ethernet switch

• Redundant Power Distributions Units (PDUs)

Add more racks for additional scalability


Exadata Database Machine X2-2 Full Rack Pre-Configured for Extreme Performance

• 8 x64 Dual-procesor Database Servers (Sun Fire X4170 M2)

• 96 cores (12 per server)

• 768 GB memory (96GB per server)





OR





• Keyboard, Video, Mouse (KVM) hardware


Add more racks for additional scalability


• 4 x64 Dual-procesor Database Servers (Sun Fire X4170

M2)







OR







Can Upgrade to a Full Rack

Exadata Database Machine X2-2 Half Rack Pre-Configured for Extreme Performance


Can Upgrade to an Half Rack

Exadata Database Machine X2-2 Quarter Rack Pre-Configured for Extreme Performance

• 2 x64 Dual-procesor Database Servers (Sun Fire X4170 M2)







OR







Start Small and GrowField Upgradeable

Full Rack

Half Rack

Balanced Incremental Scaling for OLTP and DW

Quarter Rack


Scale to 8 Racks by Just Adding CablesFull Bandwidth and Redundancy


Complete Family Of Database MachinesFor OLTP, Data Warehousing & Consolidated Workloads

• Quarter, Half, Full and Multi-Racks • Full and Multi-Racks

Oracle Exadata X2-8Oracle Exadata X2-2


Database Server Operating System Choices

• Two Operating System Choices on the database servers

• Oracle Linux

• Solaris 11 Express (x86) – Coming Soon

• Customers will choose their preferred Database Server

OS at installation time

• Exadata Storage Servers will continue to be Oracle Linux



Exadata Product Capacity (Uncompressed)

X2-8

Full Rack

X2-2

Full Rack

X2-2

Half Rack

X2-2

Quarter Rack

Raw Disk1High Perf Disk 100 TB 100 TB 50 TB 21 TB

High Cap Disk 336 TB 336 TB 168 TB 72 TB

Raw Flash1 5.3 TB 5.3 TB 2.6 TB 1.1 TB

Usable Capacity with ASM

normal redundancy2

High Perf Disk 45 TB 45 TB 22.5 TB 9.25 TB

High Cap Disk 150 TB 150TB 75 TB 31.5 TB

Usable Capacity with ASM

high redundancy3

High Perf Disk 30 TB 30 TB 15 TB 6.25 TB

High Cap Disk 100 TB 100TB 50 TB 21.5 TB

1 - Raw capacity calculated using standard disk drive raw space terminology of 1 GB = 1000 x 1000 x 1000 bytes and 1 TB = 1000 x 1000 x 1000 x 1000 bytes.

2 - Actual space available for a database after mirroring (ASM normal redundancy) and leaving one empty disk to handle disk failures. Capacity calculated using normal space terminology of 1 TB = 1024 * 1024 * 1024 * 1024 bytes.

3 - Actual space available for the database computed after triple mirroring (ASM high redundancy). Capacity calculated using normal space terminology of 1 TB = 1024 * 1024 * 1024 * 1024 bytes.


Exadata Product Performance

X2-8

Full Rack

X2-2

Full Rack

X2-2

Half Rack

X2-2

Quarter

Rack

Raw Disk Data

Bandwidth1,3

High Perf Disk 25 GB/s 25 GB/s 12.5 GB/s 5.4 GB/s

High Cap Disk 14 GB/s 14 GB/s 7 GB/s 3 GB/s

Raw Flash Data

Bandwidth1,3

High Perf Disk 75 GB/s 75 GB/s 37.5 GB/s 16 GB/s

High Cap Disk 64 GB/s 64 GB/s 32 GB/s 13.5 GB/s

Disk IOPS2,3High Perf Disk 50,000 50,000 25,000 10,800

High Cap Disk 25,000 25,000 12,500 5,400

Flash IOPS2,3 1,500,000 1,500,000 750,000 375,000

Data Load Rate4 12 TB/hr 12 TB/hr 6 TB/hr 3 TB/hr

1 - Bandwidth is peak physical disk scan bandwidth achieved running SQL, assuming no compression.

2 - IOPs – Based on peak IO requests of size 8K running SQL. Note that other products quote IOPs based on 2K, 4K or smaller IO sizes that are not relevant for databases.

3 - Actual performance will vary by application.

4 - Load rates are typically limited by CPU, not IO. Rates vary based on load method, indexes, data types, compression, and partitioning

Exadata Storage Software Unique Features

• Exadata Smart Scans• 10X or greater reduction in data sent

to database servers

• Exadata Storage Indexes• Eliminate unnecessary I/Os

• Hybrid Columnar Compression• Efficient compression increases

effective storage capacity and increases user data scan bandwidths by a factor of up to 10X

• Exadata Smart Flash Cache• Breaks random I/O bottleneck by

increasing IOPs by up to 20X

• Doubles user data scan bandwidths

• I/O Resource Manager (IORM) • Enables storage grid by prioritizing

I/Os to ensure predictable performance

• Quality of Service (QoS)

• Actively meet and maintain SLAs

• Memory Guard to protect existing

current transactions from memory-

based failures


• Data Intensive processing runs in Exadata Storage Grid

• Filter rows and columns as data streams from disks (168 Intel Cores)

• Example: How much product X sold last quarter

• Exadata Storage Reads 10TB from disk

• Exadata Storage Filters rows by Product & Date

• Sends 100GB of matching data to DB Servers

• Scale-out storage parallelizes execution and removes bottlenecks

Exadata Intelligent Storage Grid

Most Scalable Data Processing


Simple Query Example

Exadata

Storage Grid

SUM

Optimizer Chooses

Partitions and Indexes to

Access

10 TB scanned

1 GB returned to servers

Oracle

Database Grid

What were my sales yesterday?

Select sum(sales)

whereDate=’24-Sept’

Scan compressed blocks in

partitions/indexes

Retrieve sales amounts for

Sept 24


Exadata Intelligent Storage

• Exadata storage servers also run more complex operations in storage

• Join filtering

• Incremental backup filtering

• I/O prioritization

• Storage Indexing

• Database level security

• Offloaded scans on encrypted data

• Data Mining Model Scoring

• 10x reduction in data sent to DB servers is common

Exadata Intelligent Storage

Grid


Exadata is Smart Storage

• Storage Server is smart storage, not a DB node

• Storage remains an independent tier

• Database Servers

• Perform complex database processing such as

joins, aggregation, etc.

• Exadata Storage Servers

• Search tables and indexes filtering out data that is

not relevant to a query

• Cells serve data to multiple databases enabling

OLTP and consolidation

• Simplicity, and robustness of storage appliance

Compute and Memory Intensive Processing

Data IntensiveProcessing



Exadata Storage IndexTransparent I/O Elimination with No Overhead

• Exadata Storage Indexes maintain summary

information about table data in memory• Store MIN and MAX values of columns

• Typically one index entry for every MB of disk

• Eliminates disk I/Os if MIN and MAX can never

match “where” clause of a query

• Completely automatic and transparent

A B C D

1

3

5

5

8

3

Min B = 1

Max B =5

Table Index

Min B = 3

Max B =8

Select * from Table where B<2 - Only first set of rows can match

Storage Index with Partitions Example

• Queries on Ship_Date do not benefit from Order_Date partitioning

• However Ship_date and Order# are highly correlated with Order_Date

• e.g. Ship dates are usually near Order_Dates and are never less

• Storage index provides partition pruning like performance for queries on

Ship_Date and Order#

• Takes advantage of ordering created by partitioning or sorted loading

Order# Order_Date

Partitioning Column

Ship_Date Item

1 2007 2007

2 2008 2008

3 2009 2009

Orders Table


Storage Index with Joins Example

A M C D

1

3

5

5

5

5

FactDimension

Name M

Accord 1

Camry 3

Civic 5

Prius 8

Bloom filter constructed with

min/max for M

Skip IO

Due to Storage Index

Perform IO and

apply bloom filter

Select count(*) from fact, dim

where fact.m=dim.m and dim.name=‘Camry’


Exadata Smart Flash Cache

Extreme Performance OLTP & DW

5X More I/Os than

1000 Disk Enterprise

Storage Array

• Exadata has 5 TB of flash

• 56 Flash PCI cards avoid disk

controller bottlenecks

• Intelligently manages flash

• Smart Flash Cache holds hot data

• Avoids large scan wipe-outs of cache

• Gives speed of flash, cost of disk

• Exadata flash cache achieves:

• Over 1.5 million IO/sec from SQL (8K)

• Sub-millisecond response times


Exadata Hybrid Columnar Compression

Highest Capacity, Lowest Cost

• Data is organized and compressed by column

• Dramatically better compression

• Speed Optimized Query Mode for Data Warehousing

• 10X compression typical

• Runs faster because of Exadata offload!

• Space Optimized Archival Mode for infrequently accessed data

• 15X to 50X compression typical

Qu

ery

Faster and Simpler

Backup, DR, Caching, Reorg, Clone

Benefits Multiply


Exadata Hybrid Columnar Compression

How it works

• Tables are organized into sets of a few thousand rows

• Compression Units (CUs)

• Within CU, data is organized by column, then compressed

• Column organization brings similar values close together, enhancing compression

• Useful for data that is bulk loaded and queried

• Update activity is light

• Compared to best conventional algorithms – Gzip, Bzip2

• Typically 2X the compression, 10X the performance

• Exadata servers offload filtering, projection, etc. for scans on compressed data

• Indexed accesses return compressed blocks to database so buffer cache benefits from compression

Reduces

Table Size

4x to 40x

4x to 50x Reduction

Compression Unit


Compression Ratio of Real-World Data

• Compression Ratio varies by

customer and table

• Trials were run on largest table

at 10 ultra large companies

• Average revenue > $60 BB

• Average Query Compression

ratio was 13x

• On top of Oracle’s already

highly efficient format


Exadata I/O Resource ManagementMixed Workloads and Multi-Database Environment

• Ensure different databases are

allocated the correct relative amount of

I/O bandwidth

• Database A: 33% I/O resources

• Database B: 67% I/O resources

• Ensure different users and tasks within

a database are allocated the correct

relative amount of I/O bandwidth

• Database A:

• Reporting: 60% of I/O resources

• ETL: 40% of I/O resources

• Database B:

• Interactive: 30% of I/O resources

• Batch: 70% of I/O resources

Exadata Cell

InfiniBand Switch/Network

Database A Database B

Exadata Cell Exadata Cell


Quality of Service ManagementThe Oracle Approach in Action

Policy-DrivenArchitecture

Implement and Control

Define and Enable

Evaluate and Report

Analyze and Recommend

Classify and Measure

QoS Policy SetMaintenance Policy

Weekend Policy

After Hours Policy

Business Hours Policy

Server Pool Allocations

Performance Objectives

Server

Pools

Performance

ClassesBusiness Rankings


Integrated, Task-Based ManagementNew EM Quality of Service Management Tool


DBFS - Scalable Shared File System

• Database Machine comes with DBFS shared Linux file system

• Shared storage for ETL staging, scripts, reports and other application files

• Files stored as SecureFile LOBs in database tables stored in Exadata

• Protected like any DB data – mirroring, DataGuard, Flashback, etc.

• 5 to 7 GB/sec file system I/O throughput

ETL Files in DBFS

Load into database using External Tables

ETL

More File Throughput than High-End NAS Filer



Exadata Storage Management & Administration

• Enterprise Manager

• Manage & administer Database and ASM

• Monitor the Exadata Database Machine Hardware

• Plug-ins to monitor the Exadata Storage Cells and other

components in the Database Machine

• Auto Service Request (ASR)

• File SRs automatically for common hardware faults

• Comprehensive CLI

• Local Exadata Storage cell management

• Distributed shell utility to execute CLI across multiple cells

• Embedded Integrated Lights Out Manager (ILOM)

• Remote management and administration of hardware

ASR for Exadata

Customer Datacenter

Customer

Oracle FieldEngineer

FRU replaced byField Engineer

Fault occurs

Oracle Support Services

Oracle Support Engineer

FRU dispatched by

Support Engineer

Service Request routed to Support Engineer

Oracle Case Management

System

Service Request (SR) created

ASR Service

Product's auto-diagnosis facility sends

SNMP trap to ASR Manager

SR creationemail notification

to customer

Fault telemetry securely

transmitted to Oracle

ASRManager


First Secure Database Machine

• Moves decryption from software

to hardware

• Over 5x faster

• Near zero overhead for fully

encrypted database

• Queries decrypt data at hundreds

of Gigabytes/second


Best Machine for Database Consolidation

• Exadata serves as farm/cloud for

databases

• Large memory enables many

databases to be consolidated

• Extreme performance for complex

workloads that mix OLTP, DW, batch,

reporting

• I/O and CPU resource management

isolates workloads

ERP

CRM

Warehouse

Data Mart

HR


37

• Fastest Disk Throughput

• Much Faster with Flash

• 50 TB of data fits in Flash

• Using 10x Query Compression

• Effective Query Throughput on

compressed data is even higher

• Hundreds of GB/sec

© 2010 Oracle Corporation

10

20

25

TeradataNetezzaTwinFin 12

Exadata

Query ThroughputGB/sec Uncompressed Data

Single Rack

Fla

sh

Disk

Faster than DW Appliances75 GB/sec!

Fla

sh

2650 4600

< 38

Disk

Disk

2.5

9

< 6

11???

25

IBM XIV

NetApp 6080

IBM DS8700

Hitachi USP V

EMC VMAX

Exadata V2

• Exadata is fastest

• and scales with more racks

• Storage Arrays already can’t

deliver disk bandwidth

• No extra bandwidth from Flash

• No CPU offload

• No Columnar Compression

• No InfiniBand

Storage Data Bandwidth(Uncompressed GB/sec)

Fla

sh

Faster than High End ArraysD

isk

ExadataV2

75 GB/sec!

1 RackMultiple Racks


More Data Capacity

© 2010 Oracle Corporation 39

Exadata10x Compression

Teradata 26501.4x Compression

(block compression is archival)

Netezza TwinFin2x to 4x Compression

EMC VMAX3x Oracle Compression

Equivalent Capacity Systems

All with Largest Disks, Best Compression

4X

3X

7X

41X for 4600 flash appliance

Exadata is Faster and 6X Lower Cost than

IBM’s Best - Power 795

IBM P795 + 4 DS8700s with Flash

$18,860,000

2 Exadata X2-8

$3,000,000

Comparison of Hardware Prices

• More CPU Cores

• More I/O Performance

• Same Storage Capacity• Excluding Exadata compression!

• HA Configuration


Exadata Database MachineIdeal Database Platform

• Best for Data Warehousing

• Best for OLTP

• Best for Database Consolidation

Hybrid Columnar Compression

Intelligent Storage Grid

Fastest, Lowest Cost

Smart Flash Cache

Business answers in seconds, not hours

Hardware

Decryption


• Mission-Critical Platforms

• Xeon® 7500 and Xeon® 5600 Processors – Combining to enable

Exadata V3 advances

• Appliance Ease Of Deployment, No Limits To Function

• Accelerated Encrypt/Decrypt w/Xeon 5600 processor for storage

cells

• High-end reliability and scalability for can’t-fail mission critical OLTP

with Xeon 7500 processor-based database machines

• Volume-Driven Economics, Competitive Innovation

• 100’s of millions of processors = lower cost/processor

• Compatible competition drives competitive innovation

Oracle and Intel

1.Average of 3x performance claim based on geometric mean of four industry-standard, common enterprise benchmarks (SPECjbb*2005, SPECint*_rate_base2006, SPECfp*_rate_base2006, and TPC Benchmark* E) comparing best published / submitted results on 4-socket (4S) Intel Xeon processor X7560 –based server platform to best published 4S Intel Xeon processor X7460 –based server platform as of March 26, 2010.

2. 8 socket system with 128 DIMM slots populated with 16GB DDR3 DIMMs3. 8X per Intel internal memory BW measurement 3.2.104. Estimate of Xeon 7500 vs older single core 4socket servers. See 20:1 Refresh Foil for details5. Per published history of Intel Xeon product performance6.See RAS list for new features

Average 3X performance

boost over Xeon® 74001

2 terabyte of memory (8 sockets)2

2-256 sockets

8X memory bandwidth3

Over 20x performance

vs. older, single-core servers4

Biggest Performance Leap Ever for Xeon5

12 Month Refresh ROI4

Higher VM Density and Investment

Protection

Flexible

Virtualization

I/O Virtualization

Intel VT Flex-Migration Assist

for virtualization pool investment protection

Over 20 new RAS

features7

Machine Check Architecture-

recovery Recover from fatal

errors

Mission Critical Reliability

Scalable

Performance

Advanced

Reliability

Intel®Xeon™Processor 7500 Series Benefits


Resources

• Oracle.com:http://www.oracle.com/exadata

• Oracle Exadata Technology Portal on OTN: http://www.oracle.com/technology/products/bi/db/exadata

• Oracle Exadata white papers: http://www.oracle.com/technetwork/middleware/bi-foundation/exadata-smart-flash-cache-twp-v5-1-128560.pdf

http://www.oracle.com/technetwork/database/features/availability/xmigration-11-133466.pdf

oracle exadata storage technical overview · 2010-03-26 · • no exadata certification required...

Documents