demystifying sql tuning - oracle · demystifying sql tuning: tips and techniques for sql experts...

Demystifying SQL Tuning: Tips and Techniques for SQL Experts

Mughees A. MinhasDirector of Product Management, Database and System s ManagementSergey KoltakovProduct Manager, Database Manageability

Outline

• Identifying Problem SQL• Identifying Problem with the SQL

• Manual• Automatic

• Preventing SQL Regressions

Top System SQL: ADDM

• Automatic Database Diagnostic Monitor (ADDM)

• Provides database-wide performance diagnostic

• Self-diagnostic engine in the database

• Identifies top SQL for the entire system in the last hour

• Shows SQL impact

• Frequency of occurrenceSQLAdvisor

High-load SQL

IO / CPU issues RAC issues

Automatic Diagnostic Engine

Snapshots inAutomatic Workload

Repository

Self-Diagnostic Engine inside DB

System Resource

Advice

Network + DB config

Advice

Identify Top SQL using Top Activity

Performance Page

Top Activity

• Identify Top SQL by• Time

• CPU• I/O

• Wait class

• Real-time analysis: ASH data

• Historical analysis: AWR data

• More granular analysis

• Enables identification of transient problem SQL

Identify Problem SQL using AWR Compare Period Report

• Enables comparison of Top SQL from one period to another• Makes analysis of SQL variations diagnosis easier• AWR SQL capture configurable• Important to establish baseline period for comparison

• AWR Baselines

• Out-of-box Moving Window Baseline

Outline




Causes of Poor SQL PerformanceOperational Issues

• Stale optimizer statistics• CBO requires up-to-date statistics to optimize SQL execution• Not easy to determine when stale statistics are impacting plan

selection

• Poor plan selection due to incorrect optimizer estimates• Manually hinting SQL a solution:

• Requires significant expertise• Is time consuming and error-prone – trial and error method• Does not work for packaged applications

Causes of Poor SQL Performance Application Issues

• Missing access structures• Designing correct partitioning

strategy not easy

• Identifying useful indexes and MVs non-trivial

• Difficult to assess performance impact of new structures on DMLs

• Bad SQL design• Sometimes the only real

remedy is to restructure SQL

• Requires expertise, time, application knowledge

Identifying Problem with the SQL

• Manual• Real-time SQL Monitoring• ASH Report

• Automatic• SQL Tuning Advisor• SQL Access Advisor

• Shows what’s happening inside SQL execution

• Automatically monitors long running SQL• > 5 secs of CPU or I/O• All parallel statements

• Enabled out-of-the-box with no performance impact

• Monitors each SQL execution • Exposes monitoring statistics

• Global execution level• Plan operation level• Parallel Execution level

• Guides tuning efforts

Real-time SQL Monitoring

Real-time SQL Monitoring

Demo

Identifying SQL Problem: Example 1

Identifying SQL ProblemBig Plans

Identifying SQL ProblemExample 2

Identifying SQL ProblemPartially Parallelized

Identifying SQL ProblemExample 3

Identifying SQL ProblemPQ Skews

ASH Report

• ASH enables targeted performance analysis• Facilitates transient problem diagnosis

• Helps answers questions like “has this SQL gotten slower over time?”• ASH Report can be filtered by time, SID, SQL ID, Wait Class, Service,

Module, Action, Client IDPerformance

Time

*

Range

ASH Report: SQLMain Sections

ASH Report:Top Events

ASH Report:Activity Over Time

Identifying Problem with the SQL

• Manual• Real-time SQL Monitoring• ASH Report

• Automatic• SQL Tuning Advisor• SQL Access Advisor

Statistics Analysis

SQL Tuning Advisor Analysis

Access Structure Analysis (Single SQL)

SQL Design Analysis

Plan Tuning (SQL Profiling)

Comprehensive Analysis

Gather Missing or Stale Statistics

Create a SQL Profile

Add Missing Indexes (B* Tree)

Modify SQL Constructs

RecommendationSQL Tuning Advisor

SQL

SQL Profiles

Optimizer(Tuning Mode)

createsubmit

SQL Profiling

Optimizer(Normal Mode)

outputsubmit

SQL Profile

SQL TuningAdvisor

DatabaseUsers

Well-TunedPlan

After …

use

� Contains auxiliary information for a SQL statement– Customized optimizer settings: Based on past execution history (e.g., first_rows

vs. all_rows)

– Compensates for missing or stale statistics

– Compensates for errors in optimizer estimates– Doesn’t require any change to the SQL text: Ideal for Packaged Apps

Automatic SQL Tuning

• Automatically captures high-load SQL

• Automatically tunes SQL without changing application by creating SQL Profiles

• Automatically validates SQL Profiles by test executing them

• Automatically implements (optional) greatly improved SQL plans

• Automatically reports analysis • Automatically runs during

maintenance window

PackagedApps

Custom Apps

SQL Tuning Advisor

Auto Capture High-Load SQL

SQL Profiles

Nightly

Well-tuned SQL

Automatic implement

Manually implement

SQL Analysis

Report

SQL Access Advisor Analysis

Access Structure Analysis (Workload)

Comprehensive Analysis

Add New Indexes (B* Tree, Bitmap)

RecommendationSQL Access Advisor

SQL

Add New Partitions (Hash, Range)

Add New MV’s, Logs

Resource Consumption

• SQL Tuning Advisor• Limited mode: Resource consumption minimal

• Stats, index and SQL restructure analysis is cheap• Average is less than 1 second per SQL statement

• Comprehensive mode: Resource consumption may be significant• SQL Profiling can potentially consume non-trivial resources• Roughly comparable to amount of resources/time consumed

when executing SQL statement(s)

• SQL Access Advisor• Resource consumption depends on size of SQL workload• For small number of SQL, resource consumption not very high

Live vs. Remote Tuning

• Live system tuning• Run SQL Tuning Advisor in Comprehensive mode • Run SQL Tuning Advisor in Limited mode only if …

• System does not have spare resources to tune SQL• Run SQL Access Advisor for few SQL at a time

• Remote tuning• Tuning remotely if

• Cumulative resources/time consumed by all SQL statements being tuned significant

• System cannot spare resources• Use SQL Profile and SQL Tuning Set export/import capabilities

Remote SQL Tuning

Production System Test System

Tune SQL

Move SQL Profile

• Performed to shield production system from performance impact of SQL advisors

Move SQL

Transportable STS

Transportable SQL Profile

Storage

Remote SQL Tuning

• Use Automatic SQL Capture feature of SQL Tuning Set (STS) to capture SQL Workload

• Always validate SQL Profiles before enabling them

• For remote tuning, ensure test system is similar to production system

• Schema• Data distribution• Volume

• If test system smaller than production, set optimizer stats manually

DBMS_SQLTUNE.ACCEPT_SQL_PROFILE (task_name => ‘<tun ing task name>’, category => ‘MY_CATEGORY’);

ALTER SESSION SET SQLTUNE_CATEGORY=‘MY_CATEGORY’ ;

Outline




Preventing SQL Regressions

• Gather statistics for all objects (dictionary and user objects)• Volatile objects

• Gathers statistics when object at max size and then lock table• Delete all statistics and lock table – dynamic sampling will be used

• Restoring old optimizer statistics• Used when new stats result in poor execution plan selection • Scope: Table, schema and database • History maintained for 30 days • API: DBMS_STATS.RESTORE_TABLE_STATS

• Don't use the ANALYZE command• Officially obsolete for optimizer statistics• Cannot gather GLOBAL statistics for partitioned objects • Cannot gather statistics for external tables, fixed tables, etc.• Invalidates/recompiles all dependent cursors at once• DBMS_STATSmarks cursors as unusable and recompiles gradually


• Automatic statistics collection job• Out-of-the box, runs in maintenance

window• Configuration can be changed (at table

level)• Gathers statistics on user and dictionary

objects• Parameters chosen automatically based

on• DML monitoring

• Column usage monitoring

• Iterative sampling

• Uses new collection algorithm with accuracy of compute and speed faster than sampling of 1%

• Incrementally maintains statistics for partitioned tables – very efficient


• Extended Stats• Extended Optimizer Statistics provides a mechanism to collect statistics on

a group of related columns• Full integration into existing statistics framework

• Automatically maintained with column statisticsDBMS_STATS.CREATE_EXTENDED_STATS

• Pending Stats• Allows validation of statistics before publishing• Disabled by default• To enable, set table/schema PUBLISH setting to FALSE

DBMS_STATS.SET_TABLE_PREFS('SH','CUSTOMERS','PUBLIS H','false')

• To use for validationALTER SESSION SET optimizer_pending_statistics = TR UE;

• Monitor using view: USER_(TAB/COL/IND)_PENDING_STATS

Validating SQL Performance

StorageStorage

10.2Send SQL to execute11.1

Send execution stats

• Validate all changes that impact SQL plans, e.g., statistics refresh

• Use SQL Performance Analyzer (SPA) to validate statistics refresh

1. Capture SQL workload in STS using automatic cursor cache capture capability

2. Execute SPA pre-change trial3. Refresh statistics using PENDING

option4. Execute SPA post-change trial5. Run SPA report comparing SQL

execution statistics

• Before PUBLISHing stats• Remediate individual SQL for plan

few regressions• Revert to old statistics if too many

regressions observed

SQL Performance Analyzer (SPA)

Analysis Report

Compare SQL Performance

demystifying sql tuning - oracle · demystifying sql tuning: tips and techniques for sql experts...

Documents