#oowbr - exadata database machine x6 what’s new and what’s coming, paulo portugal

Copyright © 2016 Oracle and/or its affiliates. All rights reserved.

Exadata Database Machine X6What’s New and What’s Coming

Paulo PortugalOracle Systems - BrazilJun, 2016

Copyright © 2016 Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted 2

Program Agenda

Exadata X6 Hardware Updates

Exadata Software New Features 12.1.x

Exadata Software – What’s Coming

1

2

3

Copyright © 2016 Oracle and/or its affiliates. All rights reserved. 3

Exadata X6 Hardware Updates

Continue Tradition of State Of the Art Hardware


• Scale-Out 2-Socket Database Servers– Fastest Intel 22 core Broadwell CPUs (25% faster)– DDR4 DRAM frequency increased 13%

• High-Capacity Unified InfiniBand Internal Fabric

• Scale-Out Intelligent 2-Socket Storage Servers– Intel 10 core Broadwell CPUs offload database processing (25% faster) – 8TB Helium Disk Drives (2X Larger since Oct)– 2X larger & 2X faster 3D V-NAND NVMe Flash cards– DDR4 DRAM 128 GB per Storage Server (increase vs X5)

Exadata X6-2 Hardware Details (changes in red)

4

High-Capacity (HC) Storage

Extreme Flash (EF) Storage

Database Server

• High Capacity Rack - 1.3 PB Disk, 180 TB PCIe Flash • Extreme Flash Rack - 360 TB PCIe Flash


3x Throughput, 35% More IOPS Compared to X5Exadata X6 Delivers Breakthrough DB I/O Performance

2.5X301 GB/sec Analytic Throughput5.6 Million 8K OLTP Read IOPS5.2 Million 8K OLTP Write IOPS

250 us I/O latency at 2.4 Million IOPS

(Scales higher as racks are added)Performance of 1 Exadata Rack with 10 DB servers and 12 Extreme Flash storage servers


Exadata X6 IO is Much Faster than All-Flash EMC

• One High Capacity Exadata beats the fastest EMC XtremIO all-flash array in every performance metric–2X better latency

8 X-Brick EMC XtremIO

1 Rack HC Exadata

0

50

100

150

200

250

300

350

24

301

GB/s

ec

12X

Analytic Scans

8 X-Brick EMC XtremIO

1 Rack HC Exadata

0

1

2

3

4

5

2 M

4.14M

OLTP Write IOPS

2.5X

EMC Performance does not scale higher, Exadata scales by adding racks

5.2 M


Exadata X6 IO is Much Faster than All Flash Pure Storage

• One High Capacity Exadata beats the fastest Pure Storage all-flash array in every performance metric–4X better latency

Pure Storage //M70

1 Rack HC Exadata

0

50

100

150

200

250

300

350

9

301

GB/s

ec

33X

Analytic Scans

Pure Storage //M70

1 Rack HC Exadata

0

1

2

3

4

5

OLTP Write IOPS

4X

Pure Storage Performance does not scale higher, Exadata scales by adding racks

5.2 M

1.2 M


Exadata X6-8 Database MachineLarge Scale OLTP, Consolidation, Database In-Memory

• No X6-8 Server planned– Broadwell E7 chip doesn’t provide substantial

benefit over Haswell

• Elastic Compute X5-8 Database Servers– 2 to 4 8-Socket DB servers per rack (X5-8 Servers)– Latest “Haswell” E7 18-core chip– Each with 144 Cores, up to 6TB RAM

• Elastic Storage X6-2 Storage Servers– 3 to 14 X6-2 storage servers per rack

10

Elastic


Database Server Storage Expansion Kit• 2 socket database servers have 8 disk bays, only 4 are populated out of the

factory• Virtual Machines need more storage on the database servers• X6-2 and X5-2 database servers now support 8 x 600 GB HDDs

– Only two supported configurations 4 drives or 8 drives– Servers will ship with only 4 drives out of the factory, customers can add 4 more hard

drives in the field

• Minimum software version – Exadata Storage Software 12.1.2.3.0• Disks will be available as an X-option

– 4 x 7111103 600 GB 10000 rpm 2.5-inch SAS-3 HDD with marlin bracket– The server will not recognize any configuration between 4 and 8 drives


Exa Software New Features 12.1.x

Some NF added after 12.1.X Exadata Software that should be know

Gurmeet Goindi

Please be explicit with release names


I/O Resource Management Profiles– Simplifies management of interdatabase plans for many databases– Associates databases with a manageable number of profiles

• Rather than defining plan directives for each database

– Databases map to profiles using the DB_PERFORMANCE_PROFILE instance parameter

– Example:

– Change parameter in Database:


Snapshot Databases for Test and Development– Exadata snapshot databases for test and development:

• Are based on a read-only copy of an existing database• Changes are written to a sparse disk group

– Creating sparse grid disks and a sparse disk group:

– Creating a snapshot database based on a pluggable database:


Cell-to-Cell Data Transfer• Cell-to-Cell transfer in 11.2.X:

1. The database server sends a read request to Cell A.2. Cell A sends data to the database server.3. Data is stored in the database

server memory.4. The database server sends data

to Cell B.

• Cell-to-Cell transfer in 12.1.X:1. The database server sends a transfer

request to Cell B.2. Cell B reads data from Cell A.

– Lower network bandwidth consumption

– Lower database server resource usage

– Used by ASM resynchronization, resilver, and rebalance operations

DatabaseServer

ExadataStorage Server A

ExadataStorage Server B

1

2

3

4

1

2


Exadata Storage Server Users, Roles, and Privileges

• In addition to the default administrative user accounts, you can:– Create users: CellCLI> create user <username> password = *– Create roles : CellCLI> create role <rolename>– Grant privileges on objects to roles:

CellCLI> grant privilege <action> on <object> ... to role <rolename>

• Actions include: alter, create, describe, drop, export, import, list, all actions

• Objects include: cell, celldisk, flashcache, flashlog, griddisk, role, all objects

– Grant roles to users: CellCLI> grant role { all | <rolename> } to user { all | <username> }


Exadata Storage Server Users, Roles, and Privileges: ExamplesCellCLI> create role administratorCellCLI> grant privilege all actions on all objects to role

administratorCellCLI> create user scott password=*CellCLI> grant role administrator to user scott

CellCLI> create role monitorCellCLI> grant privilege list on all objects to role monitorCellCLI> create user cellmon password=*CellCLI> grant role monitor to user cellmonCellCLI> create role gdskCellCLI> grant privilege create on griddisk to role gdskCellCLI> grant privilege alter on griddisk to role gdskCellCLI> grant privilege list on griddisk to role gdskCellCLI> create user dskman password=*CellCLI> grant role gdsk to user dskman


Running CellCLI Commands from Database Servers

• ExaCLI enables cell management from remote hosts:– Enables tighter control to cell access:

• No SSH connection to the cell is required• Communication uses https

– Requires access through a user with role-based privileges– Supports CellCLI command syntax, with some restrictions– Command syntax:

$ exacli -c [username@]host [-l username] [--xml] [--cookie-jar [filename]] [-e CellCLI_command]


ExaCLI: Examples$ exacli -l scott -c exa1cel01 Password=************ExaCLI>

$ exacli -c scott@exa1cel01Password=************ExaCLI>

$ exacli -c scott@exa1cel01 --cookie-jar -e list cellPassword=************

exa1cel01 online$

$ exacli -l scott -c exa1cel01 -e list celldisk CD_00_exa1cel01 normal ...


Executing Commands Across Multiple Servers Using exadcli• The exadcli utility allows you to simultaneously execute a

command on multiple Exadata Storage Servers:– Command types:

• ExaCLI commands• ExaCLI scripts

– Commands are executed in separate parallel threads using ExaCLI:• Requires access through a user with role-based privileges• Uses the same cookie jar mechanism as ExaCLI

– Interactive sessions are not supported.– Command output is collected and displayed in the terminal session executing the exadcli utility.


exadcli: Examples

$ exadcli –c exa1cel01,exa1cel02 -l celladministrator --cookie-jar –e list cell

Password=************exa1cel01: exa1cel01 onlineexa1cel02: exa1cel02 online

$ cat mycellsexacel01exacel02

$ cat cmd.txtlist cell

$ exadcli -g mycells -l celladministrator -x cmd.txtexa1cel01: exa1cel01 onlineexa1cel02: exa1cel02 online


I/O Resource Management for Flash• Commencing with Exadata software release 12.1.2.1.0, IORM manages flash

I/Os as well as disk I/Os:– Prioritizes flash OLTP I/O requests over flash scans– Uses existing IORM plan definitions– No additional user or administrator controls


CellCLI> ALTER IORMPLAN dbPlan=( (name=oltp, level=1, allocation=80, flashCache=on, flashLog=on), (name=dss, level=1, allocation=20, limit=50, flashCache=off, flashLog=on), (name=other, level=2, allocation=100, flashCache=off, flashLog=off))

Flash Cache and Flash Log Resource Control• IORM can control whether a database can use Exadata Smart Flash Cache or

Smart Flash Log:


PROD

TEST

DEV

Flash Cache Space Resource Management

• Flash cache usage limits can be set for each database:

CellCLI> ALTER IORMPLAN dbPlan=( (name=prod, share=8, flashCacheMin=4096G), (name=dev, share=2, flashCacheMin=1G, flashCacheLimit=4G) (name=test, share=1, limit=40, flashCacheSize=2048))


Setting the Exadata Cell I/O Timeout Threshold

– Use the ALTER CELL command to set the iotimeoutthreshold attribute:

CellCLI> ALTER CELL iotimeoutthreshold = '5s'

• Read I/O that takes longer than the threshold is cancelled and redirected to a mirror copy of data

– Designed to minimize abnormal read latencies• Cannot boost normal system performance

– Setting the threshold too low can negatively impact system performance• Review AWR reports to determine a reasonable setting


Automatic Hard Disk Scrubbing and Repair– Disks are periodically scanned to detect latent corruptions

• Scrubbing I/Os are issued when the disk is idle

– If a corruption is detected, and repair is required:• Data in the local Smart Flash Cache is used if possible• Otherwise, ASM is requested to read an available mirror

– Configuration options:• Setting the next start time:

CellCLI> ALTER CELL hardDiskScrubStartTime="<Timestamp>"

• Setting the scrubbing interval:

CellCLI> ALTER CELL hardDiskScrubInterval = [ daily | weekly | biweekly | none ]


Adaptive Hard Disk Scrubbing

– Disks with bad sectors are more likely to develop more bad sectors.• More frequent scrubbing makes sense.

– Starting with Exadata release 12.1.2.3.0, if a scrub detects bad sectors, the scrubbing interval may be automatically adjusted:• If the scrubbing interval is set to greater than one week, an additional scrub is scheduled in one

week.


Cell Diagnostic PackagesA cell diagnostic package contains related logs and traces:

• A diagnostic package is generated for each cell alert.–LIST DIAGPACK lists previously generated packages.

• Use CREATE DIAGPACK to manually create a package:

CellCLI> CREATE DIAGPACK packStartTime="<Timestamp>" [, durationInHrs=<duration> ]

CellCLI> CREATE DIAGPACK alertName=<alertName>

• Use the following methods to access diagnostic packages:– File location: Look under $LOG_HOME on each cell.– Email alerts: See attachment for each email alert.– Web interface: https://<cellHostname>/diagpack– REST APIs: https://<cellHostname>/diagpack/download?name=<packName> https://<cellHostname>/diagpack/download?alert=<alertName>– ExaCLI: exacli> DOWNLOAD DIAGPACK <packName> <destinationFolder>


Monitoring Database Servers with MS and DBMCLI: Overview

– Exadata database servers now have a management server (MS) similar to Exadata Storage Servers.– Monitoring and administration is performed using DBMCLI:

• DBMCLI works in conjunction with MS to perform monitoring and administration tasks.• DBMCLI session example:

[dbmadmin@exa1db01 ~]$ dbmcliDBMCLI: Release 12.1.2.1.0 - Production ...

DBMCLI> list dbserverexa1db01 online

DBMCLI> exitquitting

[dbmadmin@exa1db01 ~]$


Monitoring the InfiniBand Ports on Exadata Servers– InfiniBand port monitoring is automatically performed on Exadata servers.– Manually monitor server ports with the LIST IBPORT command:

CellCLI> LIST IBPORT DETAIL name: HCA-1:1 dataRate: "40 Gbps" hcaFWVersion: 2.11.1280 id: 0x0010e000014908b9 lid: 18 linkDowned: 0 linkIntegrityErrs: 0 linkRecovers: 0 physLinkState: LinkUp portNumber:

PS: ALTER IBPORT … RESET COUNTERS

1

rcvConstraintErrs: 0rcvData: 118626786672rcvErrs: 0rcvRemotePhysErrs: 0status: ActivesymbolErrs: 0vl15Dropped: 2xmtConstraintErrs: 0xmtData: 101435188674xmtDiscards: 0


• Starting with updates to Exadata release 12.1.2.2.0, Exadata database servers can be updated using patchmgr:– Target servers must run Exadata release 11.2.2.4.2, or later– patchmgr uses dbnodeupdate.sh on each node– Multiple servers can be patched in one operation

• But patchmgr cannot patch the server that it is running on– Usage:

• Download and stage patchmgr on one of the database servers– Check My Oracle Support note 1553103.1 for details

• Check prerequisites using: # ./patchmgr -dbnode <dbnode_list> -dbnode_precheck -dbnode_loc <patch_loc> -dbnode_version <version>

• Apply patch using:# ./patchmgr -dbnode <dbnode_list> -dbnode_upgrade -dbnode_loc <patch_loc> -dbnode_version <version>

Using patchmgr to Orchestrate Database Server Patching


Exadata Software 12.1.2.3.0

X6 Software Release - Shipping Since Feb 2016


Super Fast Software UpdatesAddresses #1 Customer Pain Point

• 2.5X speed up in Storage Server Software Update– Parallel firmware upgrades across components such as hard disks,

flash, ILOM/BIOS, InfiniBand card– Reduced reboots for Software updates

• Imaging speedup– X5 High Capacity cell reimaged in less than 20 minutes


High Redundancy on Quarter and Eighth RacksEnables Recommended MAA Configuration on Quarter and Eighth Racks

• Problem: On an eighth or a quarter rack the Voting Disks were on a Normal Redundancy disk group and thus susceptible to concurrent failures

• Solution: Create quorum disks on database servers in addition to those on storage servers

• Best Practice: Use HIGH redundancy for DATA diskgroup and place voting disk in HIGH redundancy diskgroup

• Oracle Exadata Deployment Assistant automatically creates quorum disks

• Minimum Grid Infrastructure Software version required:– Oracle Database 12c Release 1 (12.1) release 12.1.0.2.160119 with these

patches: 22722476 and 22682752


VLAN Support in OEDA• VLAN tagging using Oracle Exadata Deployment Assistant for Ethernet

networks– Isolate Backup or Data Guard traffic from Client traffic– Isolate traffic between virtual machines– Provide QoS for Client traffic when configured on the switch – Reduce switch port utilization in customer data centers – Backup and Client networks can share the same network cables

• Best Practices– VLAN networks must be bonded– If backup network is tagged then the client network be tagged as well

• Restrictions– IPv6 VLANs are not supported on Admin network and VMs


Storage Index Preservation across Rebalance• In event of a disk failure data needs to be rebalanced

out to disks on other cells• Previously, storage indexes created for the regions on

the failed disk were lost and recreated on the next scan• Storage index entries will be moved along with data to

the new disk during cell to cell offloaded rebalance• Maintains application performance during rebalance• Minimum Grid Infrastructure software version:

– Oracle Database 12c Release 1 (12.1) release 12.1.0.2.160119 with patch 22682752

36

A B C9 310 18 211 910 810 8

Min B = 5Max B = 7

Region Index A B C D312988675

Min B = 8Max B = 9

Min B = 1Max B = 3


Exadata Software What's Coming

Some of the future features


Exadata – Unique Smart Platform Software

Smart Storage• Hybrid Columnar Compressed Data

reduces space usage by 10x• Database aware Flash Caching gives

speed of flash with capacity of disk• DB backups optimized in storage

Smart OLTP• Special InfiniBand protocol for best

OLTP latency : 250us IOs @ 2MM IOPs• Ultra-fast transactions by using DB

optimized flash logging algorithms• Highly Available In-Memory DB by

mirroring memory across servers

Smart Consolidation• Critical DB messages always jump to

head of queue for ultra-fast latency (QoS)

• I/O issued by interactive users and important workloads is prioritized

• 4X more Databases in same hardware

Smart Analytics• Move queries to storage, not storage

to queries• Automatically parallelizes queries

across all storage servers• 100x Faster Analytics


Smart Platform Enhancements Introduced This Year

Smart OLTP• 3X faster OLTP messaging - DB talks

directly to InfiniBand, bypassing OS

• Instant detection of Node Failure

• Sub-second Capping of I/O Latency with failover to alternate storage

Smart Licensing • Capacity-on-Demand reduces

license cost by disabling cores• Trusted Partitions limit license

scope of specialized options

Smart Analytics• 5X faster scans by converting data

to Columnar format in Flash Cache• 3X faster JSON/XML with offload to

storage servers

Smart Consolidation• Zero Overhead Xen VMs• Snapshots for Test/Dev• Set flash cache minimum

size per DB to ensure QOS• InfiniBand Partitioning

FIN VM

SALES VM

DB Vault Option

Spatial & Graph Option

VM

VM


Smart Consolidation• Hierarchical Snapshots• 2X application connections*• Automated VLAN creation*

Smart OLTP• Smart Fusion Block Transfer

eliminates log writes when moving blocks between nodes*

• Automated Rolling Upgrade across full-stack

• 2X faster Disk Recovery

Smart Availability• Extended Distance Clusters• 2.5X faster software updates*• High Redundancy Quorum Disks

on quarter and eighth Racks*• Preserve Storage Index on rebalance*

Smart Analytics• In-Memory Columnar formats

in storage server• Aggregation in storage• Set-membership using new

kind of Storage Index

Preview: New Smart System Software

*Already Released


Dozens of Additional Smart Platform Capabilities

Smart Analytics• Storage Index data skipping• Storage offload for min/max

operations• Data Mining Storage Offload• Storage offload for LOBs and

CLOBs• Auto Flash Caching for Table

Scans• Reverse Offload to DB servers• Offload Index Fast Full Scans• Offloads Scans on Encrypted

Data, with FIPS compliance• Active Bonding of InfiniBand• Instant data file creation

Smart OLTP• Smart network packet

prioritization• I/O Prioritization by DB, User, or

workload to ensure QOS• Active AWR includes storage

stats for end to end monitoring• Write-back Flash Cache• Cell-to-Cell Rebalance preserving

Flash Cache• Secure disk and flash erase• Database scoped security• Full-stack security scanning• Exachk full-stack validation• NVMe flash interface for lowest

latency IO

Smart Availability• In-Memory Fault Tolerance• Offload backups to storage servers• Prioritize rebalance of critical files• Elimination of false drive failures• Flash and disk life cycle

management alert• Avoid reading Predictive failed

disks• Cell software transparent restart• I/O hang hardening• Prevent shutdown if mirror server

is down• Confinement of temporarily poor

performing drives

#oowbr - exadata database machine x6 what’s new and what’s coming, paulo portugal

Technology