update exadata x6 database machine - nloug · pdf fileexadata x6 pricing unchanged to greatly...

Copyright © 2016 Oracle and/or its affiliates. All rights reserved.

Update Exadata X6 Database Machine

Rob Kolb

Sales Consultant, Oracle Nederland

May, 2016

[email protected]


Exadata Vision

Dramatically Better Platform for All Database Workloads

• Ideal Database Hardware - Scale-out, database optimized compute, networking, and storage for fastest performance and lowest costs

• Smart System Software – specialized algorithms vastly improve all aspects of database processing: OLTP, Analytics, Consolidation

• Full-Stack Integration – Database-to-disk optimization, automation, testing, patching, and support to reduce operational costs

2

Identical On-Premises and Oracle Public Cloud

Copyright © 2016 Oracle and/or its affiliates. All rights reserved. | 4

• Ideal Database Hardware

• Smart System Software

• Shared Flash Performance

• SPARC M7 DAX

Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

Ideal Hardware Architecture for Database Most Advanced - Highest Performance - Always Available - Starts Small, Scales Huge

5

Ideal Server Architecture Scale-Out with Fastest CPUs

Ideal Storage Architecture Scale-Out Servers, Database Offload

Ideal Capacity Architecture Tier PCIe Flash & Huge 8TB SAS3 Disks

Ideal Network Architecture Unified Ultra-fast InfiniBand

Ideal Flash Architecture Ultra-fast NVMe PCIe Flash

PCI Flash


• Scale-Out 2-Socket Database Servers

– Fastest Intel CPUs - 22 core Broadwell E5-2699 v4 (25% faster) – DDR4 DRAM frequency increased 13%

• Ultra-Fast Unified InfiniBand Internal Fabric

• Scale-Out Intelligent 2-Socket Storage Servers

– Intel 10 core Broadwell CPUs offload database processing (25% faster) – 8TB Helium Disk Drives (2X Larger since Oct ‘15) – 2X larger & 2X faster 3D V-NAND NVMe Flash cards

• High Capacity Rack - 1.3 PB Disk, 180 TB PCIe Flash • Extreme Flash Rack - 360 TB PCIe Flash

Exadata X6-2 Hardware Details (changes in red)

6

High-Capacity (HC) Storage

Extreme Flash (EF) Storage

Database Server


Elastic Configurations Incrementally Scale Compute & Storage Achieve any Level of Performance with Minimum Hardware

7

Multi-Rack

Database Server

Full Rack

• Enable Database CPU cores as needed with Capacity on Demand • Expand older Exadata machines with new servers

Extreme Flash Storage

High-Capacity Storage

Start Small 2 Database Servers 3 Storage Servers

Incrementally add DB or

Storage Servers

Add Racks to Continue

Scaling

25.6 TB PCI Flash 20 CPU Cores

44 CPU Cores

12.8 TB PCI Flash 96 TB Disk

20 CPU Cores


Exadata X6 Delivers Breakthrough DB IO Performance

8

2.5X 350 GB/sec Analytic Throughput

5.6 Million 8K OLTP Read IOPS 5.2 Million 8K OLTP Write IOPS

250 us IO latency at 2.4 Million IOPS

Scales higher as racks are added

Performance of 1 Exadata Rack with 10 DB servers and 12 Extreme Flash storage servers


Exadata X6 Pricing Unchanged to Greatly Improve Value

9

• Huge benefits over X5 for same price

– 2X disk capacity, 2X flash capacity, 2X faster flash

– 25% faster X6-2 Broadwell CPUs

– Faster DRAM

• Minimum DB licenses unchanged (COD)

– 16 cores for Eighth Rack, 28 cores for Quarter Rack

• Oracle SuperCluster updated to X6 storage

• New X6-8 8-socket Exadata updated to X6 storage





• SPARC M7 DAX


Smart System Software Introduced in 2015

12

Smart OLTP

• 3X faster OLTP messaging using direct DB to InfiniBand access

• Instant detection of Node Failure

• Sub-second Capping of I/O Latency by rerouting IOs to faster storage

Smart Licensing • Capacity-on-Demand reduces

DB license cost by disabling unneeded cores

• Trusted Partitions limit license scope of specialized options

Smart Analytics • 5X faster scans by converting data

to Columnar format in Flash Cache

• 3X faster JSON/XML by offloading to storage servers

Smart Consolidation • Zero Overhead VMs

• Snapshots for Test/Dev

• Set flash cache minimum size per DB to ensure QOS

• InfiniBand Partitioning

• IPv6 for Ethernet

FIN VM

SALES VM

DB Vault Option

Spatial & Graph Option

VM

VM


Exadata Software 12.1.2.3.0 Super Fast Software Updates

• 2.5X speed up in Storage Server Software Update

– Parallel firmware upgrades across components such as hard disks, flash, ILOM/BIOS, InfiniBand card

– Reduced reboots for Software updates

• Imaging speedup

– X5 High Capacity cell reimaged in less than 20 minutes

15


VLAN Support in OEDA • VLAN tagging using Oracle Exadata Deployment Assistant for Ethernet

networks

– Isolate Backup or Data Guard traffic from Client traffic

– Isolate traffic between virtual machines

– Provide QoS for Client traffic when configured on the switch

– Reduce switch port utilization in customer data centers

– Backup and Client networks can share the same network cables

• Best Practices

– VLAN networks must be bonded

– If backup network is tagged then the client network be tagged as well

• Restrictions

– IPv6 VLANs are not supported on Admin network and VMs

16


High Redundancy on Quarter and Eighth Racks

• Problem: On an eighth or a quarter rack the Voting Disks were on a Normal Redundancy disk group and thus susceptible to concurrent failures

• Solution: Create Voting Disks on database servers in addition to those on storage servers

• Best Practice: Use HIGH redundancy for DATA diskgroup and place voting disk in HIGH redundancy diskgroup

• Oracle Exadata Deployment Assistant automatically creates quorum disks

• Quorum Disk Manager Utility creates and manage quorum disks

• Minimum Grid Infrastructure Software version required:

– Oracle Database 12c Release 1 (12.1) release 12.1.0.2.160119 with these patches: 22722476 and 22682752

| Oracle Confidential – Highly Restricted 17


Storage Index Preservation across Rebalance • In event of a disk failure data needs to be rebalanced

out to disks on other cells

• Previously, storage indexes created for the regions on the failed disk were lost and recreated on the next scan

• Storage index entries will be moved along with data to the new disk during cell to cell offloaded rebalance

• Maintains application performance during rebalance

• Minimum Grid Infrastructure software version: – Oracle Database 12c Release 1 (12.1) release 12.1.0.2.160119 with patch

22682752


A B C

9 3

10 1

8 2

11 9

10 8

10 8

Min B = 5 Max B = 7

Region Index A B C D

3

1

2

9

8

8

6

7

5

Min B = 8 Max B = 9

Min B = 1 Max B = 3


Increased Maximum Number of Database Processes

• Significant increase in maximum number of database processes


Machine Type InfiniBand Bonding Type

Maximum Number of Processes with No Parallel Queries

Maximum Number of Processes with All Running Parallel Queries

8-socket (X2-8, X3-8) Active passive 28,500 25,000

8-socket (X4-8, X5-8) Active bonding 100,000 44,000

2-socket (X2-2, X3-2) Active passive 12,500 10,000

2-socket (X4-2, X5-2) Active bonding 25,000 14,000

If a subset of the workload is running parallel queries, the maximum database process count will be between the values in the “Maximum Number of Processes with No Parallel Queries" column and the “Maximum Number of Processes with All Running Parallel Queries" column

Minimum Database and Grid Infrastructure software version: Oracle Database 12c Release 1 (12.1) release 12.1.0.2.160119 with patches 22711561, 220609314 and 220609314


Miscellaneous Enhancements • Adaptive Scrubbing Schedule

– Repeat scrubbing in a week for disk drives with bad sectors

• IPv6 Support in ASR Manager

– Systems using IPv6 can now connect to Auto Service Request (ASR) using ASR Manager 5.4

• CREATE DIAGPACK to package trace files for alerts – CREATE DIAGPACK command now supports creating diagnostic packages for a

specified alert using the alertName parameter

• ASM Disk Size Checked When Reducing Grid Disk Size

– Prevents users from accidentally decreasing the size of a grid disk before decreasing the size of an ASM disk



Firmware Updates

• Bug fixes for 600GB and 8TB disk drives

– 600GB firmware update addresses issues with high rates of sector failures in some older 600GB disk drive as described in MOS note 2073916.1

– 8TB disk update improves recovery from media errors

• Updated firmware for hard disk controller in Exadata X2-2, X3-2, X4-2, X2-8, X3-8, and X4-8 to improve the robustness of handling rare critical errors impacting HBA functionality

• New InfiniBand switch firmware version 2.1.8-1 to support an updated Service Processor Module and bug fixes

• ILOM/BIOS firmware to improve robustness and security


https://mosemp.us.oracle.com/epmos/faces/DocumentDisplay?id=2073916.1





• SPARC M7 DAX


Shared Storage Has Many Advantages over Local Storage

• Much better space utilization

• Much better security, management, reliability

• Enables DB consolidation, DB high availability, RAC scale-out

• Shares storage performance

– Aggregate performance of shared storage can be dynamically used by any server that needs it

24

Servers

SAN/LAN

Shared Storage


Flash Performance is Wasted by Shared Storage Arrays

25

Array Heads

CPU

All-Flash Storage Array IO Path: many steps, each adds latency and creates bottlenecks

SAS/SATA PCIe Flash

Chips

Switches SAN/LAN

SSD Ctrl

Host HBA

• Recent improvements flash performance are causing 100X bottlenecks across shared storage stack

– Speed of one flash card is now similar to fastest SAN or LAN link

– Throughput of a few flash cards is too fast to transfer to servers

Latest PCIe Flash 5.4 GB/sec

SAN/LAN

SAN Link = 40Gb 5 GB/sec


Storage Arrays Fundamentally Bottleneck Flash Tiered, Hybrid or All Flash

• Flash Drives are much faster than SANs and LANs

– One PCI flash drive is 4X faster than a 10Gb Ethernet link

– 50 shared flash drives require 200x 10Gb Ethernets between host and array!

• Shared Arrays throttle modern flash by an order of magnitude

26

Storage Array

Servers

250 GB/s Flash Bandwidth

25 GB/s

10X Bottleneck! Even with 20x 10Gb/s Links


Exadata X6 IO is Much Faster than All-Flash EMC

• One High Capacity Exadata beats the fastest EMC XtremIO all-flash array in every performance metric

–2X lower latency (0,50 vs 0,25ms)

27

24

301

0

50

100

150

200

250

300

350

8 X-Brick EMCXtremIO

1 Rack HCExadata

GB

/se

c

12X

Analytic Scans

2 M

0

1

2

3

4

5

8 X-Brick EMCXtremIO

1 Rack HCExadata

OLTP Write IOPS

2.2X

EMC Performance does not scale higher, Exadata scales by adding racks

4.5 M


Exadata X6 IO is Much Faster than All Flash Pure Storage

• One High Capacity Exadata beats the fastest Pure Storage all-flash array in every performance metric

–4X lower latency (1.00 vs 0,25ms)

28

9

301

0

50

100

150

200

250

300

350

Pure Storage//M70

1 Rack HCExadata

GB

/se

c

33X

Analytic Scans

0

1

2

3

4

5

Pure Storage//M70

1 Rack HCExadata

OLTP Write IOPS

3.7X

Pure Storage Performance does not scale higher, Exadata scales by adding racks

4.5 M

1.2 M


0

100

200

300

400

500

Exadata SingleRack

Pure StorageLargest

EMC XtremeIO4-brick

Actual Throughput Potential Throughput*

Only Exadata Achieves Full Performance of Shared Flash

29

• Leading All-Flash Storage Arrays achieve under 3% of potential flash throughput

• Pure Storage – 132 MB/sec per flash drive

• EMC XtremIO – 120 MB/sec per flash drive

• Spinning disk level throughput!

• AND can’t scale-out for higher performance

• AND can’t share even this slow performance due to bottleneck at server inputs

• Exadata X6 achieves full flash throughput • 5400 MB/sec per drive

• Exadata also achieves much faster OLTP IOs

• 5.6 Million IOPs, 250us latency even at 2.4M IOs

Waste

d Flash

Po

ten

tial

Waste

d Flash

Po

ten

tial

Actu

al Thro

ugh

pu

t

*Potential Throughput based on number of flash devices


Exadata Storage Eliminates the Flash Bottleneck Full Scale-Out, PCIe NVMe Flash, Application Offload, Database Aware

• Exadata Moves Queries to Storage, not Storage to Queries

– ONLY platform with shared storage benefits and NO FLASH BOTTLENECK

• State-of-the-art PCIe Flash with NVMe

• Scale out architecture avoids bottlenecks

– Adds flash capacity and performance by adding servers

• Shared storage with memory level bandwidth is a paradigm change in the industry

– Get near DRAM throughput (800GB/s) , with the capacity of shared flash

30

PCIe Smart Scan

40 Gb/s 40 Gb/s

40 Gb/s 40 Gb/s

InfiniBand Switch

300 GB/s Scan Bandwidth

SQL


Preview: Redesigning Scan Offload for Memory Throughput

• With Exadata Flash throughput approaching memory throughput, SQL bottleneck moves from IO to CPU

• Exadata will automatically transform table data into In-memory DB columnar formats in Exadata flash cache

– Dual format architecture extended from DRAM to flash

• Enables fast vector processing for storage server queries

– Smart Scan results sent to DB using In-Memory Columnar format to reduce DB CPU usage

• Uniquely optimizes next generation flash as memory

32

In-Memory Columnar scans

In-Flash Columnar scans

Production in 2016





• SPARC M7 DAX


Smart Platform Extended into CPU Chip SPARC M7 Software in Silicon

34

• Traditional DB algorithms too complex for chips

• Big Change: In-memory algorithms are much simpler

• 5 years ago Oracle initiated a revolutionary project

–Build fastest ever microprocessor • Most processing cores (32)

• Most concurrent threads (256)

• Fastest Memory Bandwidth (160 GB/sec)

–Add In-Memory DB operations directly on chip

Software in Silicon


Real-Time Enterprise

TB’s ETL (GB/s)

Data Warehouse Server

Real-Time Enterprise – Up-to-the-minute Analytics Analytics at memory speed – 1000’s GB/s is much greater old 1-40 GB/s bottlenecks

• Key business processes radically transformed by orders magnitude

– Costs also dramatically reduced

on-disk

Row format

Millions IOPs

on-disk

Data Warehouse

on-disk

Typical fast IO 40GB/s

Column format

In-memory


Real-Time Enterprise – Big Data Analytics Accelerating Big Data analytics

Big Data Info

Digital

Universe Zetta bytes

Big Data:

Oracle NoSQL

Endeca Hadoop (“distills”)

Statistics

(“analyze”)

Real-Time Enterprise

TB’s

on-disk

Row format

Millions IOPs

Column format

In-memory


But: Cores & Caches Completely Saturated During Scans Scan via software consumes all of the cores

37

L3$ CACHE

L3$ CACHE mc

L3$ CACHE

L3$ CACHE

mc

mc

On-chip Crossbar mc

8 Cores 8 Cores

8 Cores 8 Cores

Notice that if cores perform in-memory scans, that caches are polluted (data missed into cache)

Data (columnar)

> 160 GB/s

Result


SPARC M7: Software In Silicon Features

Security in Silicon: Silicon Secured Memory

Cryptography Acceleration

Query Acceleration: Database Analytics Accelerator Engines

Capacity in Silicon: Decompression Engines

38


Equivalent of 32 extra cores plus 64 extra decompress cores

Query Acceleration: Behind The Scenes

OZIP decompress

DAX L3 Cache DRAM

Unpack Input

Predicate Evaluation

Bloom Filter Filter Rows

by Bit Vector

Run Length Expand

Pack Output

DAX L3 Cache DRAM

CORE CLUSTER CORE CLUSTER CORE CLUSTER CORE CLUSTER

CORE CLUSTER CORE CLUSTER CORE CLUSTER CORE CLUSTER

AC

CEL

ERA

TOR

S CO

HER

ENC

E, S

MP

& I/

O IN

TER

CO

NN

ECT

CO

HER

ENC

E, S

MP

& I/

O IN

TER

CO

NN

ECT

ME

MO

RY

CO

NTR

OL M

EM

OR

Y CO

NTR

OL

L3$ &

ON-CHIP NETWORK

AC

CELER

ATO

RS

SRAM Buffer

Dictionary & Lookup

Tables

39


SPARC M7 In-memory Advantages SPARC M7 fastest for analytics • Industry-leading SPARC M7 memory bandwidth for In-memory

• DAX offloads the cores & avoids cache pollution by not scanning with cores

• DAX decompresses OZIP data at same rate as scan-only

• DAX performs one-step range scans

40

SQL: select sum(lo_extendedprice*lo_discount) as revenue from lineorder, date_dim where lo_orderdate = d_datekey and d_year = 2012 and lo_quantity between 6 and 25 and lo_discount between 1 and 3

Processes: Decode values (DAX) & Sum aggregation (cores)

Hash Joins (cores) Bloom Filter Joins (DAX & cores)

Scans (DAX) Range Scans (DAX)

Analytics

M7 cores freed for OLTP

DAX DAX

DAX: Database Accelerator


Data (columnar)

Result

Software in Silicon: SPARC DAX DAX does scans, filtering, decompression directly DAX frees cores for other computations

41

L3$ CACHE

L3$ CACHE mc

L3$ CACHE

L3$ CACHE

mc

mc

On-chip Crossbar mc

DAX DAX

8 Cores 8 Cores

8 Cores 8 Cores

> 160 GB/s

Cores initiate DAX operations, but are mostly freed for other computational work


In-Memory Query Acceleration

SPARC M7

Cache

SQL Accel

SQL Accel

SQL Accel

SQL Accel

Core Core Core Core

• Dedicated Analytics accelerators built on chip

– Independently process streams of compressed data placed in system memory

– Like adding 32 additional specialized cores to chip

– Up to 220 Billion rows per second!

• Frees processor cores to run other applications, such as OLTP

• Decompresses data simultaneously to processing SQL functions

– Like adding 64 additional specialized cores

42


Decompress at memory speeds

Software in Silicon: Accelerating Oracle Database 12c

10X Faster

Software Decompress More than Doubles data size

Read Software

Scan Rea

d

Write

Wri

te

Rea

d

DA

X

Wri

te

Multiple steps

SQL: SELECT count(*) …WHERE lo_orderdate = d_datekey …AND lo_partkey = 1059538 AND d_year_monthnum BETWEEN 201311 AND 201312;

time

Oracle Confidential – NDA required. 5/11/2016


General-purpose NAS Storage

• Zero Overhead Virtual Machines

Unified Ultra-fast InfiniBand

Database & Application Servers

• Up to 8 SPARC M7 processors and 4 TB of RAM

• Software-in-Silicon

• Ultra-fast memory interconnect

• Secure application and Oracle Database virtualization

Exadata X6 Storage Servers

• All Exadata Storage Server features

• Elastic Storage Configuration

Oracle M7 SuperCluster with Exadata Storage & Software Consolidate Applications and Database, Large OLTP, Large In-Memory DB

SuperCluster M7-8

SQL in Silicon

update exadata x6 database machine - nloug · pdf fileexadata x6 pricing unchanged to greatly...

Documents