Getting Started with

Exadata & Smart Scan

Aman Sharma

Aman Sharma About 12+ years using Oracle Database Oracle ACE Frequent Contributor to OTN Database forum(Aman….) Oracle Certified Sun Certified Member - AIOUG,IOUG Member - AIOUG North India Chapter team Twitter : @amansharma81 * Oracle Blog : http://blog.aristadba.com * Email: amansharma@aristadba.com

Who Am I?

What We Shall Talk About

• Exadata Introduction

• Exadata Hardware & Network Overview

• Exadata Software Overview

• Exadata Key Features

• Smart Scan

A Typical Database Server Setup

RAC Database Server Setup

So What Can Go Wrong?

DBA’s Life Can Be Hard!

Users/

Developer

DBA

Manager

Say Hello To Exadata

• Engineered machine catering both OLTP and DSS workloads

• Platform for Database Cloud • Extreme Performance &

High Availability • Well balanced platform

eliminating bottlenecks • Robust yet secure & easy to

setup and implement

Evolution of Exadata Technology

Exadata Model Year Storage(TB) Flash(TB) CPU(Cores) Memory(TB)

V1 2008 168 NIL 64 256

V2 2009 336 5.3 64 576

X2 2010 504 5.3 96 1152

X3 2012 504 22.4 128 2048

X4 2013 672 44.8 192 4096

Exadata-A Sneak Peak Inside

Image courtesy-Google Images

Exadata(X4-2) Available Configurations

Exadata-Key Features

Hybrid

Columnar

Compression

(HCC)

Smart

Scan

Infiniband

Network

Smart Flash

Cache

IORM Storage

Indexes

Exadata- Hardware& Network introduction using Quarter RAC

Database Server

02

Database Server

01

36 Port IB1

36 Port IB2

Database

Server 01 IB1(Leaf)

IB3(Spine)

IB2(Leaf)

Ethernet Switch

Client Access

10GbE-Client Access

1GbE-Client Access

Mgmt Access

ILOM

Mgmt Access

ILOM

Exadata-Software Overview

ORCL1 +ASM1

LIBCELL

ORCL2 +ASM2

LIBCELL

DB01 DB02

MS RS MS RS MS RS

iDB

CELLSRV CELLSRV CELLSRV

Smart(Storage) Scan

Hybrid

Columnar

Compression

(HCC)

Smart

Scan

Infiniband

Network

Smart Flash

Cache

IORM Storage

Indexes

Smart Scan/Data Offloading

• Data Offloading makes Exadata “magical”

• Offloading is – Work is offloaded to the Exadata storage(Cells)

– Not related to just SQL statement execution

– Going to optimize others operations i.e. Backups also

• Smart SCAN – Is related specifically to SQL statement processing

– Works for Table/Index Scans, Joins

– Uses the iDB protocol

– Does NOT work all the time

– Is NOT an optimizer feature

Smart Scan-From A Little Closer

• Smart Scan works for – Column Offloading

– Predicate Offloading

– Joins(using Bloom Filter)

• Data access must use Direct-Path Read

• To have Smart Scan enabled – Database must have set CELL_OFFLOAD_PROCESSING=TRUE

– Disk Group must have set CELL.SMART_SCAN_CAPABLE=TRUE(FALSE)

• To display Smart Scan in the execution plan, set – CELL_OFFLOAD_PLAN_DISPLAY=AUTO/ALWAYS/NEVER

Smart Scan-When It Doesn’t Work

• On a Clustered table

• On an Index Organized Table

• LOB/LONG Column is selected

• More than 255 columns are

referenced in the query

• No predicate clause or column filtering

is used <<partial list>>

Smart Scan- Example 1 • SQL> select count(*) from employees where deptartment = ‘SALES'; COUNT(*) ---------- 12849 Elapsed: 00:00:02.14 Execution Plan ---------------------------------------------------------- Plan hash value: 1273666552 ------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 1 | 7 | 3136 (1)| | 1 | SORT AGGREGATE | | 1 | 7 | | |* 2 | TABLE ACCESS STORAGE FULL| EMPLOYEES| 255K| 998K| 3136 (1)| ------------------------------------------------------------------------------ Predicate Information (identified by operation id): --------------------------------------------------- 2 - storage(“DEPARTMENT"=‘SALES') filter(“DEPARTMENT"=‘SALES')

Smart Scan-Going Beyond Execution Plans

• SQL> SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m 2 WHERE s.statistic# = m.statistic# AND 3 (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%' 4 OR s.name LIKE 'cell IO%'); NAME MB

– ---------------------------------------------------------------- ----------

physical read total bytes 131.371 physical write total bytes 0 cell physical IO interconnect bytes .453101219 cell physical IO bytes saved during optimized file creation 0 cell physical IO bytes saved during optimized RMAN file restore 0 cell physical IO bytes eligible for predicate offload 131.367438 cell physical IO bytes saved by storage index 0 cell physical IO bytes sent directly to DB node to balance CPU 0 cell physical IO interconnect bytes returned by smart scan .1512428219 cell IO uncompressed bytes 131.367438 10 rows selected. Elapsed: 00:00:00.17

Smart Scan-Example 2

• SQL> select count(*) from employees where deptartment = ‘SALES'; COUNT(*) ---------- 12849 Elapsed: 00:00:06.04 Execution Plan ---------------------------------------------------------- Plan hash value: 1273666552 ------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 1 | 7 | 3136 (1)| | 1 | SORT AGGREGATE | | 1 | 7 | | |* 2 | TABLE ACCESS STORAGE FULL| EMPLOYEES| 255K| 998K| 3136 (1)| ------------------------------------------------------------------------------ Predicate Information (identified by operation id): --------------------------------------------------- 2 - storage(“DEPARTMENT"=‘SALES') filter(“DEPARTMENT"=‘SALES')

But Is Smart Scan Really Helping?

• SQL> SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m 2 WHERE s.statistic# = m.statistic# AND 3 (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%' 4 OR s.name LIKE 'cell IO%'); NAME MB

– ---------------------------------------------------------------- ----------

physical read total bytes 131.371 physical write total bytes 0 cell physical IO interconnect bytes 89.352121219 cell physical IO bytes saved during optimized file creation 0 cell physical IO bytes saved during optimized RMAN file restore 0 cell physical IO bytes eligible for predicate offload 131.367438 cell physical IO bytes saved by storage index 0 cell physical IO bytes sent directly to DB node to balance CPU 0 cell physical IO interconnect bytes returned by smart scan 89.892121219 cell IO uncompressed bytes 131.367438 10 rows selected. Elapsed: 00:00:00.17

Smart Scan- When Execution Plan Lies

• SQL> SELECT DISTINCT event, total_waits, 2 time_waited/100 wait_secs, average_wait/100 avg_wait_secs 3 FROM V$SESSION_EVENT e, V$MYSTAT s 4 WHERE event LIKE 'cell%' AND e.sid = s.sid; EVENT TOTAL_WAITS ----------- ----------- WAIT_SECS AVG_WAIT_SECS ---------- ------------- cell smart table scan 65 1.54 .0236 cell single block physical read 305 4.34 .0142

When using Smart Scan, Look For

• Transactions happening over the buffers thus resulting in dirty buffers

• Dynamic Sampling

• Migration of rows or Chained rows

• Storage Cell Servers CPU’s are more busy than DB Servers

• If the required data is already in the buffer cache

Smart Scan Related Metrics & Wait Events

• For Wait Events – SELECT DISTINCT event, total_waits, time_waited/100 wait_secs,

average_wait/100 avg_wait_secs FROM V$SESSION_EVENT e, V$MYSTAT s WHERE event LIKE 'cell%' AND e.sid = s.sid;

• For Statistics – SELECT s.name, m.value/1024/1024 MB FROM V$SYSSTAT s, V$MYSTAT m

WHERE s.statistic# = m.statistic# AND (s.name LIKE 'physical%total bytes' OR s.name LIKE 'cell phys%' OR s.name LIKE 'cell IO%');

• Four Step Phase • Pre-Installation

– Uses Oracle Exadata Deployment Assistant(OEDA) – Customer side task

• Installation, Configuration & Deployment – Uses OneCommand

• Oracle Advance Customer Support(ACS) handles it

• Additional(Misc.) Configuration – For example, deploying EM Cloud Control

• Customer/ACS

• Post-Installation – Monitoring & maintaining of the Exadata machine

• Customer/ACS

Exadata-Deployment & Installation

• Latest version available at MOS# 888828.1

• Used by the Advanced Customer Support engineers

• Uses a series of the steps to configure the Exadata machine

• Executes required configuration steps automatically

• Customizable

OneCommand

• CELLADMIN user – Uses CELLCLI utility to do the administration tasks – Can manage storage components of the storage server – Default password- welcome

• CELLMONITOR user – For monitoring only – Doesn’t have any administrative privileges – Default password- welcome

• ROOT user – Has all the rights as of CELLADMIN user – Default password-welcome1

Oracle Exadata Machine Storage Server- System Accounts

Oracle Exadata Machine Storage Server- Storage Layout

Cell Server

System LUN’s Cell Disks

Oracle Exadata Machine Storage Server- Storage Layout

Cell Disk

Grid Disk(s)

Grid Disk(s) ASM Disk(s)

Oracle Exadata Machine Storage Server- Storage Layout

That’s All For Exadata Today!

• Exadata is an awesome combination of software & hardware

• Exadata does’nt mean that a DBA is not needed

• VM images of Exadata are NOT available

• Exadata is the future!

Exadata-Useful Books

• http://www.oracle.com/goto/exadata

• http://www.oracle.com/goto/oll

• https://community.oracle.com/community/database/high_availability/exadata

• Information Center: Oracle Exadata Database Machine (1306791.2)

• Information Center: Installing and Configuring Oracle Exadata Database Machine (1347995.2)

• Exadata Database Machine and Exadata Storage Server Supported Versions (888828.1)

• Oracle Exadata Best Practices (757552.1)

• Oracle Database Machine Monitoring Best Practices (1110675.1)

Exadata-Useful MOS#’s & URL’s

Thank You