oracle database 11g - self management features ... · pdf fileoracle database 11g - self...

Oracle Database 11g - Self Management Features, Diagnostics & Tuning

1

Zohar Elkayam, Service Director: Database Solutions

Agenda

• Automatic Workload Repository – AWR • Active Session History - ASH • Automatic Database Diagnostic Monitor – ADDM • Automatic Storage Management - ASM • Automatic Memory Management - AMM • Automatic SQL Tuning: Tuning Advisor, Automatic Tuning

Optimizer, Baselines, SQL Tuning Sets and SQL Profiles • Automatic Maintenance Tasks and Tools • Automatic DOP and Parallel Statement Queuing • Backup, Recovery and Flashback

2

?מי אני

ראש תחום בסיסי נתונים ויועץ , זהר אלקיים•

GlassHouse Technologies-אורקל בכיר ב

•DBA (תשתיתי ואפליקטיבי)1998אורקל מאז

יכולות Oracle SQL"הספר העורך הטכני של •

"המדריך לשולף המהיר, מתקדמות

zoharelkayam.wordpress.com :בלוגר•

3

http://www.hod-ami.co.il/index.php?module=shop;submodule=products;task=view;id=119






zoharelkayam.wordpress.com

4

About GlassHouse Technologies

• We provides IT Infrastructure services & solutions in most fields: storage, OS, virtualization, backup and databases

• Worldwide operations: US, UK, Swiss, Israel, Turkey and Australia

• Over 200 employees in Israel alone and ~1000 worldwide.

• Formally known as Integrity systems, in the databases business since 1995

Who Tunes?

The people who are involved with tuning: • Database administrators

• Application architects

• Application designers

• Application developers

• System administrators

• Storage administrators

5

What Does the DBA Tune?

Performance tuning areas: • Application:

• SQL statement performance • Change management

• Instance tuning: • Memory • Database structure • Instance configuration

• Operating system interactions: • I/O • Swap • Parameters

Shared with developers

Shared with SA

6

General Tuning Session

All tuning sessions have the same procedure: 1. Define the problem and state the goal. 2. Collect current performance statistics. 3. Consider some common performance errors. 4. Build a trial solution. 5. Implement and measure the change. 6. Decide: “Did the solution meet the goal?”

• No? Then go to step 3 and repeat. • Yes? Then create a new baseline.

7

Effective Tuning Goals

Effective tuning goals are:

– Specific

– Measurable

– Achievable

8

How to Tune

The procedures used to tune depend on the tool. – Basic tools:

• Dynamic performance views • Statistics • Metrics • Enterprise Manager pages

– AWR or Statspack – Automatic Database Diagnostic Monitor (ADDM) – Various custom DBA scripts

9

Using Features of the Packs

10

AWR – Automatic Workload Repository

11

Automatic Workload Repository

• Performance Data Warehouse for 10g/11g • AWR collects and stores performance data

– In-memory component (V$/Metric views) – “Persisted” in WR tables (SYSAUX) – 162 tables – WRI$, WRH$, WRM$

• Self managing “out of the box” • Set retention, frequency, baseline

12

AWR – “Statspack on Steroids”

• Similar to STATSPACK “snapshots”

• AWR snapshot automatically analyzed

• Accessible via GUI and API/SQL

• Reportable – AWRRPT.SQL

• High-impact SQL captured differently

• Stores session level info as well

13

Automatic Workload Repository: Overview

SGA

V$ DBA_*

ADDM Self-tuning component

Self-tuning component

… Internal clients

External clients EM SQL*Plus …

Efficient in-memory

statistics collection

AWR

snapshots MMON

14

What does the AWR collect?

Advisor

results

AWR

SQL

statistics

Base

statistics

Metrics ASH

16

Database Control and AWR

Edit snapshot parameters

Run AWR report

Manage snapshots

Manage baselines

17

Generating AWR Reports in EM

18

Generating AWR Reports in EM

19

API - DBMS_WORKLOAD_REPOSITORY

• Scripts in $OH/rdbms/admin – awrrpt.sql AWR report (STATSPACK)

– awrddrpt.sql AWR Diff-Diff report

– awrextr.sql frontends a DataPump dump

– awrinfo.sql Space usage by AWR/ASH

– awrsqrpt.sql Execution statistics for specific SQL statement

20

Generating AWR Reports in SQL*Plus

• We can use AWRRPT.SQL if we have access to it (usually server side script)

• We can use the actual procedures for HTML or TEXT output:

DBMS_WORKLOAD_REPOSITORY.AWRRPT_REPORT_TEXT (

L_DBID NUMBER IN,

L_INST_NUM NUMBER IN,

L_BID NUMBER IN,

L_EID NUMBER IN,

L_OPTIONS NUMBER IN DEFAULT NULL);

21

Manual AWRRPT command

Set heading off

Set trimspool off

Set linesize 80

Set feedback off

Set termout on

Spool awr_from_console.txt

select output from

table(dbms_workload_repository.awr_report_text(

&dbid, &inst_num, &bid, &eid));

spool off;

22

SQL> Spool awr_from_console.txt

SQL> select output from table(dbms_workload_repository.awr_report_text(&dbid,

&inst_num, &bid, &eid));

Enter value for dbid: 2492639615

Enter value for inst_num: 1

Enter value for bid: 953

Enter value for eid: 954

old 1: select output from

table(dbms_workload_repository.awr_report_text(&dbid, &inst_num, &bid, &eid))

new 1: select output from

table(dbms_workload_repository.awr_report_text(2492639615, 1, 953, 954))

WORKLOAD REPOSITORY report for

DB Name DB Id Instance Inst Num Startup Time Release RAC

------------ ----------- ------------ -------- --------------- ----------- ---

GHTEST 2492639615 ghtest 1 15-Aug-11 09:18 11.2.0.1.0 NO

[…awr report…]

SQL> spool off

23

Reading AWR report

24

25

How to read AWR?

• Depends on the type of application – OLTP – OLAP/DWH

• Depends on the problem – I/O – is it reading? Is it writing? – Long queries? – Is it a memory problem? – Is it high CPU load?

• Everyone has their own reading style

26

Where to start? AWR Header

27

The Main Report Menu

Top 5 Timed Events

Time Model Statistics

30

Foreground Wait Class

31

Foreground and Background Wait Events

• Breakdown of wait class

• Shows where the waits are

• Usually TMI but sometimes important

32

SQL Statistics Menu

Top SQL Reports SQL by Elapsed Time

SQL by CPU Time

SQL by Executions

SQL by Buffer Gets

34

IO Stats

• Tablespace IO Stats

• File IO Stats

35

Segment Stats - Reads

• Top Logical reads

• Top Physical reads

36

Segment Stats - Writes

• Segments by Physical Writes

Common Tuning Problems

• The most common tuning problems: – Inefficient or high-load SQL statements – Suboptimal use of Oracle Database by the application – Undersized memory structures – Concurrency issues – I/O issues – Database configuration issues – Short-lived performance problems – Degradation of database performance over time – Unexpected performance regression after environment changes – Locking issues

Tip: Using AWR Formatter add-on

• AWR Formatter 1.6 by Tyler Muth is a great add-on for Chrome browser that helps reading AWR html reports

• It allows "Smart Text" conversion of gets / reads / bytes to KB / MB / GB / TB. You can click the orange text to cycle through these units

• Combined view of key Top SQL sections • Link to MOS note for "PX" events • I/O Graphs of tablespace activity • … and more

38

http://tylermuth.wordpress.com/2011/07/12/awr-formatter-1-6-released/

http://tylermuth.wordpress.com/2011/07/12/awr-formatter-1-6-released/

Short Demo for AWR and AWR formatter

39

Workload Repository Snapshot

SYSAUX

SGA

In-memory statistics

6:00 a.m. 7:00 a.m.

8:00 a.m.

Snapshot 1 Snapshot 2 Snapshot 3

Snapshot 4 9:00 a.m.

9:30 a.m.

ADDM finds

top problems. MMON

40

AWR Snapshot Settings DBMS_WORKLOAD_REPOSITORY.MODIFY_SNAPSHOT_SETTINGS (

retention IN NUMBER DEFAULT NULL,

interval IN NUMBER DEFAULT NULL,

topnsql IN NUMBER DEFAULT NULL);

DBMS_WORKLOAD_REPOSITORY.MODIFY_SNAPSHOT_SETTINGS (

retention => 30 * 24 * 60,

interval => 15);

AWR Snapshot Settings best practice:

41

ASH - Active Session History

42

43

ASH – What’s up with sessions

• Historical view of active sessions

• V$ACTIVE_SESSION_HISTORY

• Active sessions sampled every second

• Stored in circular memory buffer

• Every 10th sample persisted in AWR

• Enables “after-the-fact” analysis!!!

ASH – Session states exposed!

• “On-the-spot” analysis • Retroactive analysis

– From memory buffer (V$ACTIVE_SESSION_HISTORY) – From persisted AWR data

(WRH$_ACTIVE_SESSION_HISTORY connected via SNAP_ID) • High load SQL execution behavior • Determine Blocking sessions and “hot”

segments • SESSION_STATE : “ON CPU” or “WAITING”

44

Active Session History: Mechanics

SGA

Statistics

V$SESSION

ASH

Recent history

Rolling

buffer

Workload repository

1sec

1sec

1sec

Every

60 minutes

MMON

MMNL

When 66% full

V$ACTIVE_SESSION_HISTORY

WRH$_ACTIVE_SESSION_HISTORY (partitioned)

No use

of SQL

Direct path

inserts

DBA_HIST_ACTIVE_SESSION_HISTORY

1 out

of 10

Viewers go

unlatched.

45

ASH – What is happening to me?!

SQL> select event, seq#, p1, p2, p3, blocking_session

2 from v$active_session_history

3 where session_id = 113 and session_serial# = 333;

EVENT Seq# P1 P2 P3 BlkSess

---------------------------- ------- ------------ ------------ ------------ ------

db file sequential read 34786 29 182279 1 0

db file scattered read 34870 19 103899 8 0

db file sequential read 34954 29 183370 1 0


enq: TX - row lock contention 35119 1415053318 524322 11255 142







46

ASH Report

• Summary of all user activity over the selected period

• Drill down to a more granular period • List details of only a Session, SQL ID, Wait

Class, Service, Module or Client ID over a particular period

• “Top” Background events, P1-P3 values, etc. • $OH/rdbms/admin/ashrpt.sql + GUI

47

Generating ASH Reports

48

ASH in Wait Drilldown

49

ASH Report: General Section

V$ACTIVE_SESSION_HISTORY

DBA_HIST_ACTIVE_SESSION_HISTORY

50

ASH Report Structure

51

ASH Report: Activity Over Time

52

ASH Text report Activity Over Time -> Analysis period is divided into smaller time slots -> Top 3 events are reported in each of those slots -> 'Slot Count' shows the number of ASH samples in that slot -> 'Event Count' shows the number of ASH samples waiting for that event in that slot -> '% Event' is 'Event Count' over all ASH samples in the analysis period Slot Event Slot Time (Duration) Count Event Count % Event -------------------- -------- ------------------------------ -------- ------- 19:50:18 (42 secs) 86 enq: TX - row lock contention 43 8.10 db file scattered read 39 7.34 CPU + Wait for CPU 4 0.75 19:51:00 (1.0 min) 119 enq: TX - row lock contention 58 10.92 db file scattered read 50 9.42 CPU + Wait for CPU 11 2.07 19:52:00 (1.0 min) 126 enq: TX - row lock contention 60 11.30 db file scattered read 50 9.42 CPU + Wait for CPU 13 2.45 19:53:00 (1.0 min) 123 enq: TX - row lock contention 59 11.11 db file scattered read 49 9.23 CPU + Wait for CPU 15 2.82 19:54:00 (38 secs) 77 enq: TX - row lock contention 37 6.97 db file scattered read 36 6.78 CPU + Wait for CPU 3 0.56

53

ADDM - Automatic Database Diagnostic Monitor

54

ADDM – Your unpaid Tuning Expert!

• Should be the starting point for most investigations

• Runs after every AWR snapshot

• Determines and records performance issue

• Recommends corrective action

• Generates probable benefit

• Suggest use of other advisors

• Common currency - “DB Time” (qualitative!)

55

ADDM Performance Monitoring layout

Snapshots

ADDM

In-memory

statistics

AWR

SGA

60 minutes

ADDM results

MMON

56

Database Control and ADDM Findings

57

ADDM Analysis Results

1

2

3

58

ADDM Recommendations

59

Database Control and ADDM Task

60

Top Performance Issues Detected

62

ADDM – Findings/Recommendations

• Hardware changes

• Database-configuration changes

• Schema-level changes

• Application changes

• Using other advisors (for example) – SQL Tuning / Access Advisor

– Segment Advisor

63

ADDM – Accessing “ADDuM”

• GUI! (easiest because of linkage) • $OH/rdbms/admin/addmrpt.sql • API – DBMS_ADVISOR In-built PL/SQL • Look at following views

– DBA_ADVISOR_LOG – DBA_ADVISOR_FINDINGS – DBA_ADVISOR_RECOMMENDATIONS – DBA_ADVISOR_ACTIONS – DBA_ADVISOR_RATIONALE

64

SELECT dbms_advisor.GET_TASK_REPORT(task_name)

FROM dba_advisor_tasks

WHERE task_id = (

SELECT max(t.task_id)

FROM dba_advisor_tasks t,

dba_advisor_log l

WHERE t.task_id = l.task_id AND

t.advisor_name = 'ADDM' AND

l.status = 'COMPLETED');

SQL> @?/rdbms/admin/addmrpt

…

Enter value for begin_snap: 8

Enter value for end_snap: 10

…

Enter value for report_name:

Generating the ADDM report for this analysis ...

Retrieving ADDM Reports by Using SQL

65

ADDM – Don’t stare at the screen!

SQL> select type, count(*) from dba_advisor_findings

where task_id in

(select task_id from dba_advisor_log

where execution_start > sysdate - 1)

group by type;

TYPE COUNT(*)

----------- --------

INFORMATION 46

WARNING 1

SYMPTOM 49

PROBLEM 79

66

ADDM – Don’t stare at the screen! SQL> select count(*) count, message from dba_advisor_findings

where task_id in

(select task_id from dba_advisor_log

where execution_start > sysdate - 1)

and type = 'PROBLEM‘ group by message order by 1 desc;

COUNT MESSAGE

----- -----------------------------------------------------------------

24 SQL statements consuming significant database time were found.

24 SQL statements were found waiting for row lock waits.

24 Individual database segments responsible for significant user I/O wait were found.

4 The execution plan of this statement can be improved by creating one or more indices

1 PL/SQL execution consumed significant database time.

1 Significant virtual memory paging was detected on the host operating system.

1 The throughput of the I/O subsystem was significantly lower than expected

67

ASM – Automatic Storage Management

71

Predictably Delivers

on Performance &

Availability

SLA’s

Increases Storage

Utilization and Agility

Simplifies and Automates

Database Storage

Management

Automatic Storage Management

• A storage manager designed to manage Oracle 10g/11g database files – offered at no additional cost – Volume Manager

– File System

– Clustering capabilities

Cost and Complexity Without Compromising Performance or Availability

72

Why use ASM?

• Fully Oracle integrated storage management system

• Provide an easy flexible layer to control your files & storage

• Provide more redundancy layer over your storage system

• Provide an integrated cluster file system for HA use

• Automatic tasks – Auto rebalancing (RBAL), auto striping - SAME

73

ASM Enhancements in 11gR2

• Improved Management – ASM Install & Configuration

Assistant (ASMCA)

– Full Featured ASMCMD

– ASM File Access Control

– ASM Disk Group Rename

– Datafile to Disk Mapping

• Tunable Performance – Intelligent Data Placement

Infrequently Accessed

Data

Frequently Accessed

Data

Automatic rebalancing

• ASM automatically rebalances when the configuration of a disk group changes. By default, the ALTER DISKGROUP statement does not wait until the operation is complete before returning. Query the V$ASM_OPERATION view to monitor the status of this operation.

• ASM automatically redistributes the file contents and eliminates the need for downtime when redistributing the content

• Control Rebalancing using the REBALANCE POWER clause in statements that add, drop, or resize disks.

75

Checking Rebalancing Operations

• ASM_POWER_LIMIT -The maximum power for a rebalancing operation on an ASM instance. The valid values range from 0 to 11, with 1 being the default.

• The higher the limit the more resources are allocated resulting in faster rebalancing operations

76

select GROUP_NUMBER, OPERATION, STATE, POWER, ACTUAL, SOFAR, ST_WORK,

EST_RATE, EST_MINUTES from v$asm_operation;

GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK EST_RATE

EST_MINUTES

------------ ----- ---- ---------- ---------- ---------- ---------- ----------

2 REBAL RUN 1 1 2364 15437 300 43

Oracle Managed Files

• OMF automatically creates files in designated locations. OMF also names files and removes them while releasing space when tablespaces or files are deleted

• Alter tablespace my_tbs add datafile size 5G;

77

AMM - Automatic Memory Management

78

Oracle Database Architecture

Spfile

Temp

Data file

Undo

PMON SMON RECO MMON

MMNL

PSP0

MMAN DBWn LGWR CKPT

CJQ0

S000

D000 QMNC

Qnnn FMON

ARCn CTWR

RVWR

Fixed

size

Large

pool

Java

pool

Streams

pool

Default

buffer

cache

Keep

buffer

cache

Recycle

buffer

cache

nK

buffer

caches

Redo

log

buffer

ASH

buffer

Sort

extent

pool

Global

context

pool

SGA

Flash

back

buffer

Instance

Flashback

logs

Redo log

files

Archive

log files

Control

files

SYSTEM

SYSAUX Change

tracking

file

Password

file

Shared

pool

79

Dynamic SGA • Implements an infrastructure to allow the server to

change its SGA configuration without shutting down the instance

• SGA size is limited by SGA_MAX_SIZE, which:

– Reserves virtual memory address space at instance startup

– Cannot be changed dynamically

• Allows for certain SGA components to be dynamically resized

80

Memory Advisories • Buffer Cache Advice (introduced in 9i R1):

– V$DB_CACHE_ADVICE

– Predicts physical reads for different cache sizes

• Shared Pool Advice (introduced in 9i R2): – V$SHARED_POOL_ADVICE

– Predicts parse time savings from having different sizes of the shared pool

• Java Pool Advice (introduced in 9i R2): – V$JAVA_POOL_ADVICE

– Predicts Java class load time savings from having different sizes of Java pool

• Streams Pool Advice (introduced in 10g R2) – V$STREAMS_POOL_ADVICE

– Predicts spill and unspill activity for various sizes

81

Automatic Shared Memory Management: Overview

• Uses dynamic SGA and memory advisors to automatically adapt to workload changes

• Maximizes memory utilization • Helps eliminate

out-of-memory errors

• Avoids relearning when using SPFILE

Online users Batch jobs

Buffer cache

Large pool

Shared pool

Java pool

Buffer cache

Large pool

Shared pool

Java pool

Online users Batch jobs

Streams pool Streams pool

82

SGA Sizing Parameters: Overview • With ASMM, five important SGA components can be

automatically sized.

• Nondefault buffer pools are not auto-tuned

• Log buffer is not a dynamic component but has a good default.

DB_KEEP_CACHE_SIZE

DB_RECYCLE_CACHE_SIZE

DB_nK_CACHE_SIZE

RESULT_CACHE_MAX_SIZE

SGA_TARGET

LOG_BUFFER SHARED_POOL_SIZE

DB_CACHE_SIZE

LARGE_POOL_SIZE

JAVA_POOL_SIZE

STREAMS_POOL_SIZE

Auto-tuned

parameters

Manual

dynamic parameters

Manual

static parameters

SGA_MAX_SIZE

83

Dynamic SGA Transfer Modes

• ASMM IMMEDIATE transfer mode: – Out-of-memory (ORA-04031) errors – Partial granules can be used.

• ASMM DEFERRED transfer mode: – Transparently executed in the background – Partial granules can be used.

• MANUAL transfer mode: – Used with ALTER SYSTEM commands – Resize must use full granules.

84

Memory Broker Architecture Statistic

deltas across

different time

periods Circular SGA

buffer of stats

captured by

MMON

Memory Broker Policy Module

Add two granules to shared pool.

Output: resize

requests Trade-off

different

components

benefit/lost

MMAN transfers

the memory.

resize

queue

MMAN

MMON

Behavior of Auto-Tuned SGA Parameters

• When SGA_TARGET is not set or is set to zero: – Auto-tuned parameters are explicitly set – Note: SHARED_POOL_SIZE includes internal startup

overhead. Value may need to be increased from previous releases.

• When SGA_TARGET is set to a nonzero value and auto-tuned parameters are: – Not set, the auto-tuned parameters default to zero – Set, a nonzero value is a lower bound

• When automatically adjusted. Current values in megabytes are shown by:

SELECT component, current_size/1024/1024 FROM V$SGA_DYNAMIC_COMPONENTS;

Behavior of Manually Tuned SGA Parameters

• Manually tuned components are: – KEEP and RECYCLE buffer caches

– Nondefault block size caches

– LOG_BUFFER

• Manually tuned components are user specified.

• Manually tuned components are included in SGA_TARGET to precisely control the SGA size.

Resizing SGA_TARGET

• The SGA_TARGET initialization parameter: – Is dynamic – Can be increased up to SGA_MAX_SIZE – Can be reduced until all components reach minimum size – Changes affect only automatically sized components

• SGA_TARGET includes everything in the SGA: – Fixed SGA and other internal allocations – Automatically sized SGA components – Manual SGA components

• SGA_TARGET allows precise sizing of the total shared memory allocation by the Oracle server

Disabling Automatic Shared Memory Management

• Setting SGA_TARGET to zero disables auto-tuning.

• Auto-tuned parameters are set to their current sizes.

• SGA size as a whole is unaffected.

SGA size = 8 GB

Parameters: SGA_TARGET = 8G

SHARED_POOL_SIZE=1G

SGA size = 8 GB

Original values

SGA_TARGET=0

Parameters: SGA_TARGET = 0

DB_CACHE_SIZE = 4G

SHARED_POOL_SIZE = 1.5G

LARGE_POOL_SIZE = 512M

JAVA_POOL_SIZE = 512M

STREAMS_POOL_SIZE = 512M

Automatic Memory Management: Overview AMM Disabled AMM Enabled

Untunable

PGA

Free

Buffer cache

Large pool

Shared pool

Java pool

Streams pool

SQL areas

Other SGA

SGA target

PGA target

OLTP BATCH

Buffer cache

Large pool

Shared pool

Java pool

Streams pool

SQL areas

Other SGA

Untunable

PGA

Free

BATCH

Buffer cache

Large pool

Shared pool

Java pool

Streams pool

Other SGA

SQL areas

Untunable

PGA

Memory target

90

Automatic Memory Management: Overview AMM

Memory target

Memory

max target

250 MB

350 MB

ALTER SYSTEM SET

MEMORY_TARGET=300M;

AMM

Memory target

Memory

max target

300 MB

350 MB

91

Oracle Database Memory Parameters

DB_KEEP_CACHE_SIZE

DB_RECYCLE_CACHE_SIZE

DB_nK_CACHE_SIZE

LOG_BUFFER

RESULT_CACHE_SIZE

SHARED_POOL_SIZE

DB_CACHE_SIZE

LARGE_POOL_SIZE

JAVA_POOL_SIZE

STREAMS_POOL_SIZE

SGA_TARGET

SGA_MAX_SIZE MEMORY_MAX_TARGET

MEMORY_TARGET

Others

PGA_AGGREGATE_TARGET

92

Automatic Memory Parameter Dependency

ST+PAT<=MT<=MMT

MT>0

ST>0 & PAT>0

ST>0 & PAT=0 PAT=MT-ST

ST=0 & PAT>0 ST=min(MT-PAT,SMS)

ST=60%MT

PAT=40%MT

MMT>0 MT=0

MMT=MT MMT=0

MT=0

Y Y N

N

Y

Both SGA and PGA can grow and shrink automatically.

ST>0

SGA and PGA are separately

auto-tuned.

Y

Only PGA is auto-tuned.

N

MT can be

dynamically

changed later.

SGA and PGA cannot

grow and shrink automatically.

Minimum possible values N

N

N

Y

Y

Y

N

93

Enabling Automatic Memory Management

94

Monitoring Automatic Memory Management

95

Monitoring Automatic Memory Management

• To monitor the decisions made by Automatic Memory Management, use the following views: – V$MEMORY_DYNAMIC_COMPONENTS (displays the

current status of all memory components) – V$MEMORY_RESIZE_OPS (displays a circular history

buffer of the last 800 completed memory resize requests) – V$MEMORY_CURRENT_RESIZE_OPS (displays current

memory resize operations)

• All SGA and PGA equivalents are still in place for backward compatibility.

96

SQL Tuning: SQL Tuning Advisor, Automatic SQL Tuning and

SQL Plan Management

97

SQL Tuning Loop: SQL Tuning in Oracle 10g Some meaningful automation

but the DBA is still required

Workload

SQL Tuning Candidates SQL Tuning

Advisor

ADDM

AWR

one hour

Generate

Recommendations

DBA

Invoke Advisor

Implement

DBA

Evaluate

Recommendations

DBA

98

SQL Tuning Advisor: Overview

• Comprehensive SQL tuning

– Detect stale or missing statistics

– Tune the SQL plan (SQL profile)

– Add missing indexes

– Restructure SQL statements SQL Tuning Advisor

99

Using the SQL Tuning Advisor

• Use the SQL Tuning Advisor to analyze SQL statements and obtain performance recommendations.

• Sources for SQL Tuning Advisor to analyze: – Top Activity: Analyzes the top SQL statements currently

active – SQL tuning sets: Analyzes a set of SQL statements you

provide – Historical SQL (AWR): Analyzes SQL statements from

statements collected by AWR snapshots

100

SQL Tuning Advisor Options

101

SQL Tuning Advisor Recommendations

102

Using the SQL Tuning Advisor: Example

103

Using the SQL Access Advisor Specify the scope

Review the selected options

Specify the workload

104

View Recommendations

106

View Recommendation Details

107

SQL Tuning Loop: SQL Tuning in Oracle 11g

It’s Automatic!

Choose

Candidate

SQL

one

week

Workload

SQL Tuning

Candidates

Test SQL Profiles Implement

SQL Profiles

Generate

Recommendations

or SQL Profiles

AWR DBA

View Reports /

Control

Process

108

Automatic SQL Tuning: Overview

• Automatic SQL Tuning automates the entire SQL tuning process and replaces manual SQL tuning.

• Optimizer modes: – Normal mode

– Tuning mode or Automatic Tuning Optimizer (ATO)

• SQL Tuning Advisor is used to access tuning mode.

• You should use tuning mode only for high-load SQL statements.

109

Automatic SQL Tuning Advisor

• Runs nightly – Scheduling handled by Automated Maintenance Task (AUTOTASK)

framework • Uses scheduler’s Maintenance Window • Uses DEFAULT_MAINTENANCE_PLAN (25% CPU) • DBA_AUTOTASK_* views and DBMS_AUTO_TASK_ADMIN package

– By default job is enabled on new installs, disabled on upgrades • Configurable

– DBA_ADVISOR_* views and DBMS_SQLTUNE package – ACCEPT_SQL_PROFILE, MAX_SQL_PROFILES_PER_EXEC,

MAX_AUTO_SQL_PROFILES, EXECUTION_DAYS_TO_EXPIRE, TIME_LIMIT, LOCAL_TIME_LIMIT, TEST_EXECUTE, etc.

• Can automatically implement plans with 3x improvement

110

SQL Statement Profiling • Statement statistics are key inputs to the optimizer.

• ATO verifies statement statistics such as:

– Predicate selectivity

– Optimizer settings (FIRST_ROWS versus ALL_ROWS)

• Automatic Tuning Optimizer (ATO) uses:

– Dynamic sampling

– Partial execution of the statement

– Past execution history statistics of the statement

• ATO builds a profile if statistics were generated:

exec :profile_name := -

dbms_sqltune.accept_sql_profile( -

task_name =>'my_sql_tuning_task');

111

Plan Tuning Flow and SQL Profile Creation

Optimizer (Tuning mode)

Create Submit

Output

SQL

profile

SQL Tuning

Advisor

Database

users

Well-tuned

plan

Optimizer (Normal mode)

Use

No application

code change

112


1. Identify candidates for SQL Tuning

2. Tune each statement individually by calling the SQL Tuning Advisor

3. Test SQL Profiles by executing the SQL statement

4. Optionally, automatically implement SQL Profiles with 3x improvement

114

Picking candidate SQL

AWR

Average Exec Hourly

1. Pull the top queries from the past week into four buckets:

Top for the past week

Top for any day in the past week

Top in any hour (single snapshot)

Top by average single execution

2. Combine four buckets into one, assigning weights

3. Cap at 150 queries per bucket

Candidate List

Daily Weekly

115


• Some SQL is ineligible for automatic tuning (they can still be manually submitted to the SQL Tuning Advisor)

– Parallel queries – Ad-hoc/rarely repeated queries (not repeated within a

week) – Long-running queries – Recursive SQL – DML (insert/update) or DDL (create table as select) – Statements that were recently processed (within the

past month)

116

SQL Plan Management: Overview

• SQL Plan Management is automatically controlled SQL plan evolution.

• Optimizer automatically manages SQL plan baselines. – Only known and verified plans are used. – Automatic plan verification requires SQL Tuning Pack License!

• Plan changes are automatically verified by default (can be changed manual). – Only comparable or better plans are subsequently used.

• The plan baseline can be seeded for critical SQL with SQL tuning set (STS) from SQL Performance Analyzer.

117

SQL Plan Management

• Next generation of Stored Outlines (Outlines are officially deprecated in 11g but can be converted to baselines): – CONTROLS plan evolution – GURANTEES plan stability

• Works hand-in-hand with Automatic SQL Tuning Advisor • Optimizer remembers SQL Plans

– Only known and verified plans are used – Plan changes can be tested and verified automatically or manually – Actually runs the statement to verify execution – evaluates real-

world performance – Plans can be transported between databases (e.g. QA -> Prod)

118

SQL Plan Management: When to use?

Protects against execution plan changes in many situations: • Database Upgrades

– Use OPTIMIZER_FEATURES_ENABLE

• System and Data Changes – Object or System Statistics – Session or System Parameters – Schema Changes (e.g. add index)

• Deployment of new application module – Can import plans that were pre-verified on a test system

119

SQL Plan Baseline: Tuning in 10g

1. First SQL Execution: Hard Parse

2. Environmental Change: stats job, smaller UGA, etc. 1. Plan Invalidated: Hard Parse results in new plan

HJ

HJ

GB

Parse Execute Good Plan

NL

NL

GB

Parse Execute Bad Plan!

120

SQL Plan Baseline: when using 11g 1. First SQL Execution: Hard Parse, STORE “BASELINE”

Parse

HJ

HJ

GB

Statement log

Plan history

HJ

HJ

GB

Plan baseline

Execute Good Plan

121

SQL Plan Baseline: when using 11g 2. Environmental Change, Plan Invalidated, Hard Parse – NEW PLAN IS NOT

EXECUTED BUT MARKED FOR VERIFICATION

Statement log

Plan history

HJ

HJ

GB

Plan baseline

NL

NL

GB

Parse

GB

NL

NL

122

SQL Plan Baseline: when using 11g 3. BASELINE (ORIGINAL) PLAN IS EXECUTED

Execute Good Plan!

Parse

HJ

HJ

GB

Statement log

Plan history

HJ

HJ

GB

Plan baseline GB

NL

NL

123

SQL Plan Baseline: when using 11g 4. AFTER LATER VERIFICATION, PLANS THAT IMPROVE PERFORMANCE

AUTOMATICALLY ADDED TO BASELINE

Statement log

Plan history

HJ

HJ

GB

Plan baseline GB

NL

NL

DBA Invoke or schedule

verification

Optimizer

checks if new

plan is as well

as or better

than old plan

Statement log

Plan history

Plan baselines

GB

NL

NL

HJ

HJ

GB

Plans which perform as well as or

better than original plan are added to

the plan baseline

GB

NL

NL

Plans which don’t

perform as well as the

original plan stay in the

plan history and are

marked unaccepted

124

SQL Plan Baseline: Architecture

SQL management base

Statement log

SYSAUX

Plan verification before integration to baseline

Repeatable

SQL

statement

SQL

profile

Automatic

SQL tuning

task

Plan history

HJ

GB

HJ

HJ

GB

HJ

HJ

GB

HJ

…

Plan

baseline

125

Loading SQL Plan Baselines

DBA

OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE

Plan history

1

2

3

4

Plan history dbms_spm

Staging

table

HJ

GB

HJ

Cursor

cache

Plan history

A

B

HJ

GB

HJ

HJ

GB

HJ

HJ

GB

HJ

HJ

GB

HJ

127

Evolving SQL Plan Baselines

Plan history

Automatic

SQL Tuning

DBA

SQL

Tuning

Advisor

>? HJ

GB

HJ

HJ

GB

HJ

128

--------------------------------------------------------------------------------

Evolve SQL Plan Baseline Report

--------------------------------------------------------------------------------

Inputs:

-------

SQL_HANDLE = SYS_SQL_593bc74fca8e6738

PLAN_NAME =

TIME_LIMIT = DBMS_SPM.AUTO_LIMIT

VERIFY = YES

COMMIT = YES

Plan: SYS_SQL_PLAN_ca8e6738a57b5fc2

-----------------------------------

Plan was verified: Time used .07 seconds.

Passed performance criterion: Compound improvement ratio >= 7.32.

Plan was changed to an accepted plan.

Baseline Plan Test Plan Improv. Ratio

------------- --------- -------------

Execution Status: COMPLETE COMPLETE

Rows Processed: 40 40

Elapsed Time(ms): 23 8 2.88

CPU Time(ms): 23 8 2.88

Buffer Gets: 450 61 7.38

Disk Reads: 0 0

Direct Writes: 0 0

Fetches: 0 0

Executions: 1 1

-------------------------------------------------------------------------------

Report Summary

-------------------------------------------------------------------------------

Number of SQL plan baselines verified: 1.

Number of SQL plan baselines evolved: 1.

SQL Plan Management

129

Important Baseline SQL Plan Attributes

select signature, sql_handle, sql_text, plan_name, origin, enabled,

accepted, fixed, autopurge

from dba_sql_plan_baselines;

SIGNATURE SQL_HANDLE SQL_TEXT PLAN_NAME ORIGIN ENA ACC FIX AUT

--------- ------------ -------- ---------------- ------------ --- --- --- ---

8.062E+18 SYS_SQL_6fe2 select.. SYS_SQL_PLAN_1ea AUTO-CAPTURE YES NO NO YES

8.062E+18 SYS_SQL_6fe2 select.. SYS_SQL_PLAN_4be AUTO-CAPTURE YES YES NO YES

…

exec :cnt := dbms_spm.alter_sql_plan_baseline(-

sql_handle => 'SYS_SQL_37e0168b0…3efe', -

plan_name => 'SYS_SQL_PLAN_8dfc352f359901ea',-

attribute_name => 'ENABLED', attribute_value => 'NO');

Plan history

Enabled but

not accepted Enabled and

accepted HJ

GB

HJ

HJ

GB

HJ

130

SQL Plan Selection

Plan part

of history? No HJ

GB

HJ

HJ

GB

HJ

HJ

GB

HJ > HJ

GB

HJ

No HJ

GB

HJ

Yes

Plan history

Plan

baseline

HJ

GB

HJ

HJ

GB

HJ

… HJ

GB

HJ

Yes

Plan part

of baseline? Yes

optimizer_use_sql_

plan_baselines=true? Yes

HJ

GB

HJ

No

No

Select baseline plan

with lowest best-cost

132

Possible SQL Plan Manageability Scenarios

DBA

Plan history

HJ

GB

HJ

Database Upgrade

Oracle Database 10g

Oracle Database 11g

HJ

GB

HJ

Well-

tuned

plan

No plan

regressions

HJ

GB

HJ

Plan history

HJ

GB

HJ

New Application Deployment Production database

No plan

regressions

HJ

GB

HJ

HJ

GB

HJ

Development database

Well-tuned

plan

HJ

GB

HJ

Baseline

plans

staging table

DBA

Plan history

134

SQL Performance Analyzer and SQL Plan Baseline Scenario

Plan history

HJ

GB

HJ

Oracle Database 10g

Oracle Database 11g

HJ

GB

HJ Well-tuned

plans

HJ

GB

HJ

HJ

GB

HJ

O_F_E=10

O_F_E=11

Regressing

statements

Before

change

After

change

HJ

GB

HJ

No plan

regressions

optimizer_features_enable

135

Loading a SQL Plan Baseline Automatically

Oracle Database 10g

Well-

tuned

plans

Oracle Database 11g

optimizer_features_enable=10.2.0.2 optimizer_capture_sql_plan_baselines=true

Plan history

Plan

baseline

HJ

GB

HJ

HJ

GB

HJ

No plan

regressions Oracle Database 11g


Plan history

Plan

baseline

HJ

GB

HJ

HJ

GB

HJ

No plan

regressions

HJ

GB

HJ

New plan

waiting

verification

HJ

GB

HJ

HJ

GB

HJ

Oracle Database 11g


Plan history

Better

plans

HJ

GB

HJ

HJ

GB

HJ

HJ

GB

HJ

Plan baseline

136

Purging SQL Management Base Policy

SQL> exec :cnt := dbms_spm.drop_sql_plan_baseline('SYS_SQL_37e0168b04e73efe');

SYSAUX

20% 1% 50% space

time

105

53

SQL Management

Base

Alert log

10%

SQL> exec dbms_spm.configure('SPACE_BUDGET_PERCENT',20);

SQL> exec dbms_spm.configure('PLAN_RETENTION_WEEKS',105);

DBA_SQL_MANAGEMENT_CONFIG

137

Automatic Maintenance Tasks

138

Automated Maintenance Tasks

• Default Autotask maintenance jobs: – Gathering optimizer statistics – Automatic segment advisor – Automatic SQL advisor

• Autotask Maintenance process: 1. The maintenance window opens. 2. The Autotask background process schedules jobs. 3. The Scheduler initiates jobs. 4. Resource Manager limits the impact of Autotask jobs.

139

140

Automatic Optimizer Statistics Collection

• Collects optimizer statistics for all schema objects in the database for which there are no statistics or only stale statistics.

• The statistics gathered by this task are used by the SQL query optimizer to improve the performance of SQL execution.

Statistic Gathering Options

Setting Statistic Preferences

ESTIMATE_PERCENT

NO_INVALIDATE

METHOD_OPT

GRANULARITY

INCREMENTAL

PUBLISH

STALE_PERCENT

DATABASE LEVEL

SCHEMA LEVEL

TABLE LEVEL

STATEMENT LEVEL

GLOBAL LEVEL

PREFERENCES

SCOPE

exec dbms_stats.set_table_prefs('SH','SALES','STALE_PERCENT','13');

DBA DBMS_STATS

set | get | delete | export | import

Optimizer

statistics

gathering

task

142

Restore Statistics • The DBMS_STATS package allows statistics to be restored as of a

timestamp using the RESTORE_*_STATS procedures.

– RESTORE_FIXED_OBJECTS_STATS

– RESTORE_SCHEMA_STATS

– RESTORE_SYSTEM_STATS

– RESTORE_TABLE_STATS

143

TEST

PROD

PUBLISH=FALSE

+

GATHER_*_STATS

EXPORT_PENDING_STATS

IMPORT_TABLE_STATS

expdp/impdp

ALTER SESSION SET OPTIMIZER_USE_PENDING_STATISTICS=TRUE

PUBLISH_PENDING_STATS

Dictionary

statistics Pending

statistics

DBA_TAB_PENDING_STATS

Deferred Statistics Publishing: Overview

144

145

Automatic Segment Advisor

• Identifies segments that have space available for reclamation, and makes recommendations on how to defragment those segments.

Segment Advisor

Automatic DOP and Parallel Statement Queuing

147

148

Automated Degree of Parallelism

• Oracle determine the degree of parallelism on each table/query based on a set of criteria and some initialization parameter settings

• Features of Oracle 11g Release 2

149

Automated Degree of Parallelism Settings

• To enable use the parallel_degree_policy parameter (by default this stuff is off - parameter is set to manual)

• For Auto DOP, setting this to LIMITED is sufficient

• For more functionality (in-memory parallel processing and parallel statement queuing) parallel_degree_policy should be set to AUTO

150

Automated Degree of Parallelism Settings

• The parameter for the threshold of parallel query evaluation is parallel_min_time_threshold. The default of this parameter is 10 seconds (set lower to allow more parallel queries evaluations)

• The maximum DOP that can be used is set by Parallel_degree_limit. By default this is set to the value of “CPU” multiplied with the value of the parameter parallel_threads_per_cpu

Automated Degree of Parallelism How it works

SQL

statement

Statement is hard parsed

And optimizer determines

the execution plan

Statement

executes serially

Statement

executes in parallel

Optimizer determines

ideal DOP

If estimated time

greater than threshold

Actual DOP = MIN(default DOP, ideal DOP) If estimated time less

than threshold PARALLEL_MIN_TIME_THRESHOLD

152

Parallel Statement Queuing

• Because of the expected behavior of more statements running in parallel it becomes more important to manage the usage of the parallel resources.

• Queries that does not have enough resources can put on “hold” until enough resources are available

• Oracle maintains FIFO queue for the waiting statements

• Use NO_STMT_QUEUING hint to avoid the queue

153

Parallel Statement Queuing Settings

• parallel_degree_policy must be set to AUTO

• The parameter for enabling the parallel statement query evaluation is parallel_server_target The default value is 4 times the default DOP.

• To avoid an arbitrary number of parallel processes to be running on a system, which may overload that system, the parameter parallel_max_servers provides a hard upper boundary

When the required

number of parallel servers

become available the first

stmt on the queue is

dequeued and executed

128

16 32 64

Parallel Statement Queuing How it works

SQL

statements

Statement is parsed

and Oracle automatically

determines DOP

If enough parallel

servers available

execute immediately

If not enough parallel

servers available queue

128 16 32 64

8

FIFO Queue

155

Parallel Statement Queuing Monitoring

• Monitoring can be done using Enterprise Manager SQL Monitor.

• Monitoring can also be done by using: SELECT s.sql_id, s.sql_text

FROM v$SQL_MONITOR m, v$SQL s

WHERE m.status='QUEUED'

AND m.sql_id = s.sql_id;

Backup, Recovery and Flashback

156

Flashback Technologies Error Detection & Correction

• Flashback revolutionizes error recovery – View ‘good’ data as of a past point-in-time – Simply rewind data changes – Time to correct error equals time to make error

• Low impact • Excellent tool for configuring QA, Dev and Training databases • Flashback is easy – simple commands, no complex procedure

Correction Time = Error Time + f(DB_SIZE)

0

20

40

60

80

Rec

ove

ry T

ime

Traditional Recovery

Flashback

Flashback Technologies

• Flashback Query

• Flashback Table

• Flashback Drop

• Flashback Transaction Query

• Flashback Database

• Flashback Data Archive

Flashback Query or Table

• Flashback Query: – SQL> SELECT .. FROM.. AS OF…. WHERE

• Flashback Table – SQL> flashback table … to timestamp … – SQL> flashback table … to SCN …

• Flashback Drop – Use of Recycle Bin – flashback table … to before drop;

Error Correction with Flashback • Flashback Database – restore

database to any point in time

• Flashback Table – restore contents of tables to any point in time (undo-based)

• Flashback Drop – restore accidentally dropped tables (based on free space in tablespace)

• Flashback Transaction – back out transaction and all subsequent conflicting transactions (redo-based)

Order

Database

Customer

Data Files Flashback Log

New Block Version

Disk Write

Old Block Version

• Fast point-in-time recovery strategy

• Eliminate the need to restore a whole database backup

• Continuous data protection for database

– Optimized, before-change block logging

– Restores just changed blocks

– Replay log to restore DB to desired time

• It’s fast - recover in minutes, not hours

• It’s easy - single command restore

Flashback Database

Flashback Database

• Flashback Database – can even go through RESETLOGS

• Can be done to Restore Points – create restore point PREQA_TESTING;

– create restore point PREQA_TESTING guarantee flashback database;

– drop restore point PREQA_TESTING;

163

Flash Recovery Area Prerequisites

• Archiving must be enabled • Flash recovery area must be configured using

– DB_RECOVERY_FILE_DEST_SIZE - size of flashback recovery area in bytes

– DB_RECOVERY_FILE_DEST - location of flashback recovery area

• For example:

SQL> ALTER SYSTEM SET db_recovery_file_dest_size = 10G;

SQL> ALTER SYSTEM SET db_recovery_file_dest = '/oradata/recovery';

164

Flashback Database Configuration

• To enable flashback logging database must be mounted but not open

• To disable flashback logging use:

• To check if flashback is currently enabled:

SQL> STARTUP MOUNT

SQL> ALTER DATABASE FLASHBACK ON;

SQL> ALTER DATABASE OPEN;

SQL> SELECT flashback_on FROM v$database;

FLASHBACK_ON

------------

YES

SQL> ALTER DATABASE FLASHBACK OFF;

165

Flashback Database Parameters

• One supported parameter: – DB_FLASHBACK_RETENTION_TARGET

• Specifies upper limit on how far back in time database may be flashed back

• Specified time in minutes - default value is 1440 minutes (24 hours)

• Affects number of flashback logs retained in flash recovery area

Flashback Data Archive Improved Time Travel in 11g

• Flashback Data Archive: query data as of 5 days, 5 weeks, 5 months, 5 years – whatever – in the past

• How does it work…

How Does Flashback Data Archive Work?

• Primary source for history is the undo data

• History is stored in automatically created history tables inside the archive

• Transactions and its undo records on tracked tables marked for archival

– Undo records not recycled until history is archived

• History is captured asynchronously by new background process (fbda)

– Default capture interval is 5 minutes

– Capture interval is self-tuned based on system activities

– Process tries to maximize undo data reads from buffer cache for better performance

– INSERTs do not generate history records

Total Recall • Alter base table – history table automatically adjusts

– Drop, Rename, Modify Column – Drop, Truncate Partition – Rename, Truncate Table

• Flashback query supported across DDL changes

• Complex DDL changes (e.g. table split) accommodated – Associate/Diassociate history table via DBMS_FLASHBACK_ARCHIVE package

Dro

p

Co

lum

n

Ad

d

Co

lum

n

time Flashback Version Query

Ad

d

Co

lum

n

Flashback Data Archive Setup

• Create a Flashback Archive tablespace:

• Set the table to use the flashback archive:

ALTER TABLE hr.employees FLASHBACK ARCHIVE fla1;

Table altered.

CREATE FLASHBACK ARCHIVE DEFAULT fla1 TABLESPACE

flasharch1

QUOTA 20G

RETENTION 1 YEAR;

Flashback archive created.

169

Best Practices – Undo-based Flashback (Table/Query)

• Use Undo Advisor (available through Enterprise Manager) to get recommendations on available undo retention for various sizes.

• Use fixed size undo – Undo retention automatically tuned for best possible

retention based on tablespace size and current system load.

• Be aware of DDL restrictions – not possible to query in the past if table structure is modified (e.g. drop/modify column, move table, etc.)

Undo Advisor

Best Practices – Flashback Database

• Tune FRA storage – Use ASM, configure enough disk spindles, etc.

• Use physical standby database to test Flashback logging • Use V$FLASHBACK_DATABASE_LOG to size log

space, after running workload > duration of Flashback retention period.

• Create Guaranteed Restore Point (GRP) without enabling Flashback logging – Saves disk space for workloads where same blocks are

repeatedly updated – Drop GRP to immediately reclaim space

Summary

• We talked about various automatic tuning and maintenance tools in Oracle 10g and 11g

• We talked about tuning of the database

• We talked about Flashback feature

• There are MORE automatic features and advisories we didn’t talk about (for example: ASSM)

• More info will be available on my blog:

ZoharElkayam.wordpress.com

173

ZoharElkayam.wordpress.com

Questions and Answers

174

Thank You! Zohar Elkayam

054-5996413

[email protected]

175

oracle database 11g - self management features ... · pdf fileoracle database 11g - self...

Documents