华为 exadata sql performance tuning detail methodology...•the purpose of this sharing session is...
TRANSCRIPT
<Insert Picture Here>
华为 Exadata SQL Performance Tuning
Detail Methodology
Agenda
1. Introduction
2. Oracle SQL Tuning Skill Set
3. Basic Skill
4. Recommended Methodology
Case Study 1: ……
Case Study 2: ……
……
5. Additional SQL Tuning Idea
6. Q&A
Introduction
• The purpose of this sharing session is to provide Huawei a
generic & systematic method that will help Huawei‟s developers
to find and fix a SQL performance problem in Exadata
environment.
• The recommended methodology will attempt to guide developer
along a process that will diagnose most common Oracle SQL
performance problems.
Oracle SQL Tuning Skillset
1. Basic Skill 基础技能
How to pick problematic SQL
How to get the execution plan
2. Advanced skill 推进技能
基础技能 - How to pick problematic SQL
• Identify the problematic SQL by:
AWR report – top elapsed time, top CPU time, top
buffer get etc
GV$SESSION &G V$SQL
Enable 10046 trace at session level
Report by user
基础技能 - Automatic Workload Repository (AWR) in
Oracle Database 11g
• Under SQL Statistics section, they are many useful
information SQL ordered by Elapsed Time
SQL ordered by CPU Time
SQL ordered by User I/O Wait Time
SQL ordered by Gets
SQL ordered by Reads
SQL ordered by Physical Reads (UnOptimized)
SQL ordered by Executions
SQL ordered by Parse Calls
SQL ordered by Sharable Memory
SQL ordered by Version Count
SQL ordered by Cluster Wait Time
Complete List of SQL Text
基础技能 - Automatic Workload Repository (AWR) in
Oracle Database 11g
• By example, under SQL ordered by Elapsed Time
• Easy to identify SQL that run very long time
基础技能 - Automatic Workload Repository (AWR) in
Oracle Database 11g
In general,
• Tune SQL that Elapsed Time per execution is high
• Tune SQL that Buffer Get per execution is high
• Tune SQL that CPU Time per execution is high
• Ask DBA for the AWR report
基础技能 - How to pick problematic SQL
• If the SQL is still running, query GV$SESSION and GV$SQL
• Example :select g.inst_id, g.sid, g.serial#, g.sql_id, g.event,
g.machine, g.sql_exec_start, l.sql_text,
l.PLAN_HASH_VALUE, g.blocking_session, g.SERVICE_NAME,
g.status, g.LOGON_TIME
from gv$session g , (select distinct sql_id, CHILD_NUMBER,
PLAN_HASH_VALUE, SQL_text from gv$sql) l
where g.username is not null and status = 'ACTIVE'
and g.sql_id = l.SQL_ID and g.SQL_CHILD_NUMBER =
l.CHILD_NUMBER
order by g.sql_exec_start ;
基础技能 - How to pick problematic SQL
• Identify problematic SQL that is running for a long time – refer to
SQL_EXEC_START
Oracle SQL Tuning Skillset
1. Basic Skill 基础技能
How to pick problematic SQL
How to get the execution plan
2. Advanced skill 推进技能
基础技能 – Why we need execution plan?
• For SQL tuning, we need to review and understand
the execution plan.
• Understand the execution plan is more important than
understand the program logic in SQL tuning
基础技能 - How to get SQL execution plan
If you get the problematic SQL from AWR, you can get the
execution plan by running
select * from
table(Dbms_Xplan.display_awr(‘<SQL_ID>', null,
null, 'ALL'))
基础技能 - How to get SQL execution plan
If you get the problematic SQL from AWR, you can get the
execution plan by running
SQL> select * from table(Dbms_Xplan.display_awr('0rbu5r3j5f7fg', null, null, 'ALL'))
2 /
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
SQL_ID 0rbu5r3j5f7fg
--------------------
SELECT DISTINCT T.COMPANY_ID, T.CONTRACT_NUMBER, T.PRODUCT_ID FROM
BL_RCR_RATE_SEPCIAL_TMP1 T WHERE T.PERIOD_ID = :B2 AND T.SOURCE_TYPE_ID
= :B1 AND T.COMPANY_ID IN (SELECT G.COMPANY_ID FROM
APD_RCR_SCH_BATCH_COMPANY G WHERE G.BATCH_ID = :B3 ) AND
T.PROGRAM_SOURCE IN ('REV_SPECIAL_PRE1', 'REV_SPECIAL_PRE2')
Plan hash value: 2906480639
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempS
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | |
| 1 | HASH UNIQUE | | 112K| 6819K| 886
| 2 | NESTED LOOPS | | 112K| 6819K|
| 3 | INDEX RANGE SCAN| APD_RCR_SCH_BATCH_COMPANY_U1 | 31 | 310 |
| 4 | INDEX RANGE SCAN| BL_RCR_RATE_SEPCIAL_TMP1_N1 | 3633 | 184K|
--------------------------------------------------------------------------------
Query Block Name / Object Alias (identified by operation id):
-------------------------------------------------------------
1 - SEL$5DA710D3
3 - SEL$5DA710D3 / G@SEL$2
4 - SEL$5DA710D3 / T@SEL$1
基础技能 - How to get SQL execution plan
• You can get SQL_ID - unique for each SQL statement and Plan
hash value – unique for each execution plan for this SQL for
further tuning use.SQL> select * from table(Dbms_Xplan.display_awr('0rbu5r3j5f7fg', null, null, 'ALL'))
2 /
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
SQL_ID 0rbu5r3j5f7fg
--------------------
SELECT DISTINCT T.COMPANY_ID, T.CONTRACT_NUMBER, T.PRODUCT_ID FROM
BL_RCR_RATE_SEPCIAL_TMP1 T WHERE T.PERIOD_ID = :B2 AND T.SOURCE_TYPE_ID
= :B1 AND T.COMPANY_ID IN (SELECT G.COMPANY_ID FROM
APD_RCR_SCH_BATCH_COMPANY G WHERE G.BATCH_ID = :B3 ) AND
T.PROGRAM_SOURCE IN ('REV_SPECIAL_PRE1', 'REV_SPECIAL_PRE2')
Plan hash value: 2906480639
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempS
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | |
| 1 | HASH UNIQUE | | 112K| 6819K| 886
| 2 | NESTED LOOPS | | 112K| 6819K|
| 3 | INDEX RANGE SCAN| APD_RCR_SCH_BATCH_COMPANY_U1 | 31 | 310 |
| 4 | INDEX RANGE SCAN| BL_RCR_RATE_SEPCIAL_TMP1_N1 | 3633 | 184K|
--------------------------------------------------------------------------------
Query Block Name / Object Alias (identified by operation id):
-------------------------------------------------------------
1 - SEL$5DA710D3
3 - SEL$5DA710D3 / G@SEL$2
4 - SEL$5DA710D3 / T@SEL$1
基础技能 - How to get SQL execution plan
If you get the problematic SQL from running session, you can view
the execution plan by running DBMS_XPLAN in particular
instance
Set long 20000000
Set pagesize 0
Set linesize 200
Select * from
table(dbms_xplan.display_cursor(‘<SQL_ID>’,null
, ‘ALLSTATS LAST’))
基础技能 - How to get SQL execution plan
If you get the problematic SQL from runnnig session, you can view
the execution plan by using SQL Monitor Report
Set long 20000000
Set pagesize 0
Set linesize 200
Select dbms_sqltune.report_sql_monitor(sql_id =>
‘SQL_ID’) from dual
<- Click me for sample output
<- Click me for sample output
基础技能 - How to get SQL execution plan
• If you only know the package, enable trace 10046
before execute the package
1. Example:
2. SQLPLUS>alter session set events „10046 trace name
context forever, level 12‟;
3. SQLPLUS>exec
PKG_BL_RCR_REV_FACT.SP_BL_RCR_REV
4. After done, go the “user_dump_dest” and get the trace file.
5. OS> cd /u01/app/oracle/diag/rdbms/dwdb/DWDB1/trace
6. OS> tkprof tracefile_name outputfile_name
explain=hwdw/password waits=y
<- Click me for sample output
Oracle SQL Tuning Skillset
1. Basic Skill 基础技能
How to pick problematic SQL
How to get the execution plan
2. Advanced skill 推进技能
Recommended Methodology
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
6. Do you compare 9i, 10g and 11g execution plan?
A. Compare each plans and bind the best to SPM
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
Why accurate statistic is so important?
• The cost based optimizer has the challenging job of
evaluating any SQL statement and generating the
"best" execution plan for the statement.
Object metadata – The DBA controls the quality of the
metadata via the dbms_stats package. This data includes the
number of rows in a table, the distribution of values within a
column and other critical information about the state of the
tables and indexes.
Disk I/O speed – The cost of disk I/O is the single most
important factor in SQL optimization. Disk I/O is measured in
thousandths of a second, an eternity for a database, and
something that needs to be avoided whenever possible.
Why accurate statistic is so important?
• Case study – S1:
SQL> SELECT SUM(MTA.BASE_TRANSACTION_VALUE) ABSORPTION_AMOUNT FROM
ODS_GL_CODE_COMBINATIONS GCC, ODS_MTL_TRANSACTION_ACCOUNTS MTA WHERE
MTA.REFERENCE_ACCOUNT = GCC.CODE_COMBINATION_ID(+) AND GCC.SEGMENT3
LIKE '128%' AND MTA.ACCOUNTING_LINE_TYPE <> 15 AND MTA.COST_ELEMENT_ID
<> 1 AND MTA.TRANSACTION_ID = 2455022551;
Why accurate statistic is so important?
• Case study – S1:
SQL> SELECT SUM(MTA.BASE_TRANSACTION_VALUE) ABSORPTION_AMOUNT FROM
ODS_GL_CODE_COMBINATIONS GCC, ODS_MTL_TRANSACTION_ACCOUNTS MTA WHERE
MTA.REFERENCE_ACCOUNT = GCC.CODE_COMBINATION_ID(+) AND GCC.SEGMENT3
LIKE '128%' AND MTA.ACCOUNTING_LINE_TYPE <> 15 AND MTA.COST_ELEMENT_ID
<> 1 AND MTA.TRANSACTION_ID = 2455022551;
• If No / Inaccurate table statistic:
– Delete statistic in tablesexec dbms_stats.delete_table_stats('ODSPUB', 'ODS_GL_CODE_COMBINATIONS')
exec dbms_stats.delete_table_stats('ODSCST', 'ODS_MTL_TRANSACTION_ACCOUNTS')
Why accurate statistic is so important?
• Case study – S1:
Execution plan with no table statistic
---------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 117 | 484 (1)| 00:00:07 | | |
| 1 | SORT AGGREGATE | | 1 | 117 | | | | |
|* 2 | HASH JOIN | | 22690 | 2592K| 484 (1)| 00:00:07 | | |
|* 3 | TABLE ACCESS BY GLOBAL INDEX ROWID| ODS_MTL_TRANSACTION_ACCOUNTS | 22690 | 1440K| 2 (0)| 00:00:01 | ROWID | ROWID |
|* 4 | INDEX RANGE SCAN | ODS_MTL_TRANSACTION_ACCOUNT_N2 | 262K| | 1 (0)| 00:00:01 | | |
| 5 | TABLE ACCESS BY INDEX ROWID | ODS_GL_CODE_COMBINATIONS | 210K| 10M| 481 (0)| 00:00:07 | | |
|* 6 | INDEX RANGE SCAN | ODS_GL_CODE_COMBINATIONS_N3 | 210K| | 448 (0)| 00:00:07 | | |
---------------------------------------------------------------------------------------------------------------------------------------
1 rows selected.
Elapsed: 00:00:08.00
• Why slow ? Because incorrect index was chosen
Why accurate statistic is so important?
• Case study – S1:
SQL> SELECT SUM(MTA.BASE_TRANSACTION_VALUE) ABSORPTION_AMOUNT FROM
ODS_GL_CODE_COMBINATIONS GCC, ODS_MTL_TRANSACTION_ACCOUNTS MTA WHERE
MTA.REFERENCE_ACCOUNT = GCC.CODE_COMBINATION_ID(+) AND GCC.SEGMENT3
LIKE '128%' AND MTA.ACCOUNTING_LINE_TYPE <> 15 AND MTA.COST_ELEMENT_ID
<> 1 AND MTA.TRANSACTION_ID = 2455022551;
• With accurate table statistic:
– Gather statistic for tables using auto sample size, for all columns size
autoexec
dbms_stats.gather_table_stats(ownname=>'ODSCST',tabname=>'ODS_MTL_TRANSACTION_ACCO
UNTS',estimate_percent=>dbms_stats.auto_sample_size, degree=>128, cascade=>True,
method_opt=>'FOR ALL COLUMNS SIZE AUTO');
exec
dbms_stats.gather_table_stats(ownname=>'ODSPUB',tabname=>'ODS_GL_CODE_COMBINATIONS'
,estimate_percent=>dbms_stats.auto_sample_size, degree=>32, cascade=>True, method_opt=>'FOR
ALL COLUMNS SIZE AUTO');
Why accurate statistic is so important?
• Case study – S1:
Execution plan with accurate table statistic
----------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
----------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 37 | 9 (0)| 00:00:01 | | |
| 1 | SORT AGGREGATE | | 1 | 37 | | | | |
| 2 | NESTED LOOPS | | | | | | | |
| 3 | NESTED LOOPS | | 2 | 74 | 9 (0)| 00:00:01 | | |
|* 4 | TABLE ACCESS BY GLOBAL INDEX ROWID| ODS_MTL_TRANSACTION_ACCOUNTS | 2 | 46 | 5 (0)| 00:00:01 | ROWID | ROWID |
|* 5 | INDEX RANGE SCAN | ODS_MTL_TRANSACTION_ACCOUNT_N2 | 4 | | 4 (0)| 00:00:01 | | |
|* 6 | INDEX UNIQUE SCAN | ODS_GL_CODE_COMBINATIONS_U1 | 1 | | 1 (0)| 00:00:01 | | |
|* 7 | TABLE ACCESS BY INDEX ROWID | ODS_GL_CODE_COMBINATIONS | 1 | 14 | 2 (0)| 00:00:01 | | |
----------------------------------------------------------------------------------------------------------------------------------------
1 rows selected.
Elapsed: 00:00:01.00
• Correct index was chosen with accurate statistic
Why accurate statistic is so important?
• Case study – S2:
SQL> SELECT * FROM table (DBMS_XPLAN.DISPLAY_CURSOR('&SQL_ID', NULL, 'ALLSTATS LAST'));
Enter value for sql_id: 0wbgug6xbr7gg
old 1: SELECT * FROM table (DBMS_XPLAN.DISPLAY_CURSOR('&SQL_ID', NULL, 'ALLSTATS LAST'))
new 1: SELECT * FROM table (DBMS_XPLAN.DISPLAY_CURSOR('0wbgug6xbr7gg', NULL, 'ALLSTATS LAST'))
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------
SQL_ID 0wbgug6xbr7gg, child number 0
-------------------------------------
SELECT 830 RECORD_TYPE_ID, TMP.PERIOD_ID, TMP.COMPANY_ID,
TMP.COMPANY_CODE, TMP.FIN_REGION_ID, TMP.REGION_CODE, TMP.ACCOUNT_ID,
TMP.ACCOUNT_CODE, TMP.PHYSICAL_PRODUCT_ID, TMP.PHYSICAL_PRODUCT_CODE,
TMP.SERVICE_PRODUCT_ID, TMP.SERVICE_PRODUCT_CODE, PCD.CUSTOMER_ID
CUSTOMER_ID, TMP.CUSTOMER_CODE, TMP.PROJECT_ID, TMP.PROJECT_CODE,
TMP.COST_CENTER_ID, TMP.CURRENCY_ID, TMP.FIN_CONTRACT_ID,
TMP.CONTRACT_NUMBER, TMP.SUB_PROJECT_ID, TMP.SUB_PROJECT_CODE,
TMP.EXPENSE_TYPE, TMP.SOURCE_TYPE, TMP.ATTRIBUTE1, TMP.ATTRIBUTE2,
TMP.ATTRIBUTE3, TMP.ATTRIBUTE4, TMP.ATTRIBUTE5, TMP.ATTRIBUTE6,
TMP.ATTRIBUTE7, TMP.SER_ASSIGN_STEP, TMP.PHY_ASSIGN_STEP,
TMP.REPORT_TYPE_FLAG, TMP.INDUSTRY_CLASS_ID, NVL(TMP.AMOUNT, 0) AMOUNT,
NVL(TMP.FUNC_AMOUNT, 0) FUNC_AMOUNT, NVL(TMP.RMB_AMOUNT, 0) RMB_AMOUNT
FROM BL_GTS_CST_CLASSIFY_TMP TMP, BL_PROJECT_CUST_DIM PCD WHERE
EXPENSE_TYPE IN ('C610', 'C630') AND TMP.PERIOD_ID = :B1 AND
NVL(TMP.RMB_AMOUNT,0) <> 0 AND TMP.SUB_PROJECT_ID = PCD.PROJECT_ID(+)
Why accurate statistic is so important?
• Case study – S2:
-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | NESTED LOOPS OUTER | | 1 |
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 1 |
|* 3 | INDEX RANGE SCAN | BL_PROJECT_CUST_DIM_N1 | 1 |
-----------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
filter(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
3 - access("TMP"."SUB_PROJECT_ID"="PCD"."PROJECT_ID")
• Optimizer uses nested loop to join those tables
because estimated row is small for both tables
Why accurate statistic is so important?
• Case study – S2:
SELECT COUNT(1) from BL_GTS_CST_CLASSIFY_TMP WHERE PERIOD_ID = 201103
-----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads
-----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.07 | 4294 | 4286
| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:00.07 | 4294 | 4286
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 1 | 1 | 293K|00:00:00.07 | 4294 | 4286
-----------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage("TMP"."PERIOD_ID"=201103)
filter("TMP"."PERIOD_ID"=201103)
• Based on table statistic, E-rows (estimate row: 1) VS A-
rows (actual rows: 293K) is huge different
• Why?
Why accurate statistic is so important?
Table Actual number of
record during execution
BL_GTS_CST_CLASSIFY_TMP 30,395
BL_PROJECT_CUST_DIM 196,557
Rows (1st) Rows (avg) Rows (max) Row Source Operation
---------- ---------- ---------- ---------------------------------------------------
30395 30395 30395 HASH JOIN OUTER (cr=3690 pr=3399 pw=0 time=1306343 us cost=647 size=6076431 card=30231)
30395 30395 30395 TABLE ACCESS STORAGE FULL BL_GTS_CST_CLASSIFY_TMP (cr=2965 pr=2959 pw=0 time=44754 us
cost=451 size=5743890 card=30231)
196557 196557 196557 INDEX STORAGE FAST FULL SCAN BL_PROJECT_CUST_DIM_N1 (cr=725 pr=440 pw=0 time=56939 us
cost=65 size=2162127 card=196557)
•Based on 10046 trace file, number of row processed is as above
Why accurate statistic is so important?
• Case study – S2:-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | NESTED LOOPS OUTER | | 1 |
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 1 |
|* 3 | INDEX RANGE SCAN | BL_PROJECT_CUST_DIM_N1 | 1 |
-----------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
filter(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
3 - access("TMP"."SUB_PROJECT_ID"="PCD"."PROJECT_ID")
• Optimizer uses nested loop to join those tables but
actual rows count for both tables are huge
Why is wrong with using Nested Loop for joining?
• Indexed Nested Loops is used primarily in low volume
joins; it is efficient over small volumes and versatile
enough to be used in a variety of situations.
• Although it is fully scalable, Indexed Nested Loops is
inefficient over large data volumes.
Why accurate statistic is so important?
What is wrong???
• Incorrect table statistic & histogram
Why accurate statistic is so important?
• Case study – S2:
SQL> select TABLE_NAME, COLUMN_NAME, LAST_ANALYZED, NUM_DISTINCT, LOW_VALUE, HIGH_VALUE, HISTOGRAM from dba_tab_columns
where TABLE_NAME = 'BL_GTS_CST_CLASSIFY_TMP'
TABLE_NAME COLUMN_NAME LAST_ANALYZED NUM_DISTINCT LOW_VALUE HIGH_VALUE HISTOG
------------------------------ ------------------------------ --------------- ------------ ---------- ---------- ------
BL_GTS_CST_CLASSIFY_TMP PERIOD_ID 11-JUN-11 1 C3150C05 C3150C05 FREQUE
SQL> select utl_raw.cast_to_number(high_value) from dba_tab_columns where table_name = 'BL_GTS_CST_CLASSIFY_TMP
UTL_RAW.CAST_TO_NUMBER(HIGH_VALUE)
---------------------------------
201104
SQL> select utl_raw.cast_to_number(low_value) from dba_tab_columns where table_name = 'BL_GTS_CST_CLASSIFY_TMP
UTL_RAW.CAST_TO_NUMBER(LOW_VALUE)
---------------------------------
201104
• Checked the table histogram, the highest / lowest value in period_id column
is 201104. Should have no 201103 data exist in the table.
Why accurate statistic is so important?
• Case study – S2:
-----------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads
-----------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.07 | 4294 | 4286
| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:00.07 | 4294 | 4286
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 1 | 1 | 293K|00:00:00.07 | 4294 | 4286
-----------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage("TMP"."PERIOD_ID"=201103)
filter("TMP"."PERIOD_ID"=201103)
• If select 201103 data, Optimizer estimates maxium 1 row need to
process because statistic indicates no 201103 data
• Optimizer, which uses nested loop to join those tables, is correct
according the table statistic & histogram
• Only table statistic & histogram is incorrect
Why accurate statistic is so important?
What is wrong??? How it happens???
Step 1 : Gather table statistic on today 1:00pm
• Only have 201104 data in table
B_GTS_CST_CLASSIFY_TMP
Step 2 : Execute ONE PASS at night 11:00pm
• Delete all data in BL_GTS_CST_CLASSIFY_TMP and then
insert 201103 data
Step 3 : Execute the same SQL
• Bad performance due to incorrect statistic – bad joining
method
Why accurate statistic is so important?
• Case study – S2:
• Let‟s see what happen after gather statistic
• Gather statistic for tables using auto sample size, for all columns size autoexec dbms_stats.gather_table_stats(ownname=>BLREP',tabname=>'
BL_GTS_CST_CLASSIFY_TMP',estimate_percent=>dbms_stats.auto_sample_size, degree=>32,
cascade=>True, method_opt=>'FOR ALL COLUMNS SIZE AUTO');
exec
• Check the DBA_TABLES & DBA_TAB_COLUMNS statistic information If data_type = „NUMBER‟, SELECT UTL_RAW.CAST_TO_NUMBER(low_value),
UTL_RAW.CAST_TO_NUMBER(HIGH_VALUE)
FROM DBA_TAB_COLUMNS WHERE TABLE_NAME = 'ODS_PROJ_CON_REG_RELATION' AND
COLUMN_NAME = 'PERIOD_ID'
If data_type = „VARCHAR2‟, SELECT UTL_RAW.CAST_TO_VARCHAR2(low_value),
UTL_RAW.CAST_TO_VARCHAR2 (HIGH_VALUE)
FROM DBA_TAB_COLUMNS WHERE TABLE_NAME = 'ODS_PROJ_CON_REG_RELATION' AND
COLUMN_NAME = 'PERIOD_ID'
Why accurate statistic is so important?
• Case study – S2:
-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | HASH JOIN OUTER | | 30K |
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 30K |
|* 3 | INDEX RANGE SCAN | BL_PROJECT_CUST_DIM_N1 | 196K |
-----------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
filter(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
3 - access("TMP"."SUB_PROJECT_ID"="PCD"."PROJECT_ID")
• Estimate rows become 30K
• Optimizer uses HASH JOIN, execution plan changed
Nested Loops VS Hash Join
• When to use Hash joins
At least one side of the join is returning many rows
Low cardinality indexes are not available on the join keys.
The join predicates use only equals (=) conditions.
Nested Loops VS Hash Join
• Nested Loops join is acceptable in a high volume SQL
are:
When the driving (inner) table will return 0 or 1 row. If you
have a join query where one of the tables is supplied with the
whole of a primary or unique key, Oracle can retrieve the row
(if there is one) and then perform a full table scan on the
second table. This is more efficient than either a sort-merge
or a hash join.
When the outer (second) table is very small (ie. fewer than
100 rows) and can fit into a single block. Since a single block
is the smallest amount of data Oracle can read, a Table that
fits into a single block can be accessed very fast with a Full
Table Scan.
Bad SQL execution plan caused by ……incorrect
statistic
• Elapsed time with correct statistic : 6min (201104)
• Elapsed time with incorrect statistic : 129min (201103)
• Elapsed time after gather statistic: 6min (201103)
推进技能 - Recommended Methodology
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
Incorrect statistic cause Optimizer :
Choose wrong INDEX
Choose wrong JOIN method
推进技能 - Recommended Methodology
For this case,
Step 1 : Gather table statistic on today 1:00pm
• Only have 201104 data in table
B_GTS_CST_CLASSIFY_TMP
Step 2 : Execute ONE PASS at night 11:00pm
• Delete all data in BL_GTS_CST_CLASSIFY_TMP and then
insert 201103 data
Step 3 : Execute the same SQL
• Bad performance due to incorrect statistic –> bad joining
method
推进技能 - Recommended Methodology
Solution 1:
• Gather statistic on dynamic working table after bulk
insert / delete
• In the package logic / MOIA job:
1. Delete table BL_GTS_CST_CLASSIFY_TMP
2. Insert record to BL_GTS_CST_CLASSIFY_TMP
3. Add Gather table statistic -BL_GTS_CST_CLASSIFY_TMP
4. Run other jobs
推进技能 - Recommended Methodology
Solution 2 :
-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | HASH JOIN OUTER | | 30K |
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 30K |
|* 3 | INDEX RANGE SCAN | BL_PROJECT_CUST_DIM_N1 | 196K |
-----------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
filter(("TMP"."PERIOD_ID"=:B1 AND
INTERNAL_FUNCTION("EXPENSE_TYPE") AND NVL("TMP"."RMB_AMOUNT",0)<>0))
3 - access("TMP"."SUB_PROJECT_ID"="PCD"."PROJECT_ID")
• If I know this should be the correct execution plan
推进技能 - Recommended Methodology
Solution 2 :
• Use SPM (SQL PLAN MANAGEMENT) for a statement,
subsequent executions of that statement will use the
SQL plan baseline.
• Example
1. Create a SPM for a correct execution plan
2. Next month when select data 201105, even high /
low value statistic only have 201104, as long as
the SQL is the SAME, the SQL will use the correct
SQL execution plan
推进技能 - Recommended Methodology
Solution 2 :
1. Create a SPM for a correct execution plan-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | HASH JOIN OUTER | | 30K |
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 30K |
|* 3 | INDEX RANGE SCAN | BL_PROJECT_CUST_DIM_N1 | 196K |
-----------------------------------------------------------------------
SQL_ID=0wbgug6xbr7gg PLAN_HASH_VALUE=656489123
SQLPLUS> VARIABLE CNT NUMBER ;
SQLPLUS> execute :CNT
:=DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE(SQL_ID =>
„0wbgug6xbr7gg‟, PLAN_HASH_VALUE => 656489123) ;
Check the added SPM
SQLPLUS> SELECT * FROM DBA_SQL_PLAN_BASELINES ORDER BY
LAST_MODIFIED
推进技能 - Recommended Methodology
Solution 2 :
1. Create a SPM for a correct execution plan
Check the added SPM:
SQLPLUS> SELECT SQL_HANDLE, PLAN_NAME, LAST_MODIFIED,
ENABLED, ACCEPTED, SQL_TEX FROM DBA_SQL_PLAN_BASELINES
ORDER BY LAST_MODIFIED
Check the SPM execution plan:
SQLPLUS> SELECT * FROM
TABLE(DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE(„SQL_HANDLE‟)
推进技能 - Recommended Methodology
Solution 2 :
2. Next month when select data 201105, even high /
low value statistic only have 201104, as long as
the SQL is the SAME, the SQL will use the correct
SQL execution planPLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------
SQL_ID 0wbgug6xbr7gg, child number 0
-------------------------------------
SELECT 830 RECORD_TYPE_ID, TMP.PERIOD_ID, TMP.COMPANY_ID,
TMP.COMPANY_CODE, TMP.FIN_REGION_ID, TMP.REGION_CODE, TMP.ACCOUNT_ID,
TMP.ACCOUNT_CODE, TMP.PHYSICAL_PRODUCT_ID, TMP.PHYSICAL_PRODUCT_CODE,
TMP.SERVICE_PRODUCT_ID, TMP.SERVICE_PRODUCT_CODE, PCD.CUSTOMER_ID
CUSTOMER_ID, TMP.CUSTOMER_CODE, TMP.PROJECT_ID, TMP.PROJECT_CODE,
TMP.COST_CENTER_ID, TMP.CURRENCY_ID, TMP.FIN_CONTRACT_ID,
TMP.CONTRACT_NUMBER, TMP.SUB_PROJECT_ID, TMP.SUB_PROJECT_CODE,
TMP.EXPENSE_TYPE, TMP.SOURCE_TYPE, TMP.ATTRIBUTE1, TMP.ATTRIBUTE2,
TMP.ATTRIBUTE3, TMP.ATTRIBUTE4, TMP.ATTRIBUTE5, TMP.ATTRIBUTE6,
TMP.ATTRIBUTE7, TMP.SER_ASSIGN_STEP, TMP.PHY_ASSIGN_STEP,
TMP.REPORT_TYPE_FLAG, TMP.INDUSTRY_CLASS_ID, NVL(TMP.AMOUNT, 0) AMOUNT,
NVL(TMP.FUNC_AMOUNT, 0) FUNC_AMOUNT, NVL(TMP.RMB_AMOUNT, 0) RMB_AMOUNT
FROM BL_GTS_CST_CLASSIFY_TMP TMP, BL_PROJECT_CUST_DIM PCD WHERE
EXPENSE_TYPE IN ('C610', 'C630') AND TMP.PERIOD_ID = :B1 AND
NVL(TMP.RMB_AMOUNT,0) <> 0 AND TMP.SUB_PROJECT_ID = PCD.PROJECT_ID(+)
推进技能 - Recommended Methodology
Solution 2 :
2. Next month when select data 201105, even high /
low value statistic only have 201104, as long as
the SQL is the SAME, the SQL will use the correct
SQL execution plan-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | HASH JOIN OUTER | | 30K |
|* 2 | TABLE ACCESS STORAGE FULL| BL_GTS_CST_CLASSIFY_TMP | 30K |
|* 3 | INDEX RANGE SCAN | BL_PROJECT_CUST_DIM_N1 | 196K |
-----------------------------------------------------------------------
Note
-----
- SQL plan baseline SYS_SQL_PLAN_fcc170b0a62d0f4d used for this statement\
• The same execution plan will use.
• If SPM is used, a note will indicate it when you
view the execution plan
推进技能 - Recommended Methodology
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
YES, go to next step
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
2. Does the SQL already have optimized joining
method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove /
add it and re-test
Case study J1
Problematic SQL:• SELECT COT.CONTRACT_NUMBER ,COT.ITEM_CODE ,COT.QUANTITY ,
Q_L.QUOTATION_ITEM_CODE ,Q_L.QUOTATION_QUANTITY FROM (SELECT /*+INDEX(DT
ODS_CP_ORDER_OM_DAILY_TEMP_N1)*/ DT.CONTRACT_NUMBER, DT.ITEM_CODE,
SUM(DT.QUANTITY) QUANTITY FROM ODS_CP_ORDER_OM_DAILY_TEMP DT GROUP BY
DT.CONTRACT_NUMBER, DT.ITEM_CODE) COT, (SELECT /*+USE_NL(QH,QL)
LEADING(QH) INDEX(QH ODS_CP_QUOTATION_HEADERS_U1) INDEX(QL
ODS_CP_QUOTATION_LINES_N1)*/ QH.CONTRACT_NUMBER,
QL.QUOTATION_ITEM_CODE, SUM(QL.QUOTATION_QUANTITY) QUOTATION_QUANTITY
FROM ODS_CP_QUOTATION_HEADERS QH, ODS_CP_QUOTATION_LINES QL, (SELECT
DT.CONTRACT_NUMBER, DT.ITEM_CODE, SUM(DT.QUANTITY) QUANTITY FROM
ODS_CP_ORDER_OM_DAILY_TEMP DT GROUP BY DT.CONTRACT_NUMBER,
DT.ITEM_CODE) ODT WHERE QH.QUOTATION_HEADER_ID =
QL.QUOTATION_HEADER_ID AND ODT.CONTRACT_NUMBER = QH.CONTRACT_NUMBER
AND ODT.ITEM_CODE = QL.QUOTATION_ITEM_CODE GROUP BY
QH.CONTRACT_NUMBER, QL.QUOTATION_ITEM_CODE) Q_L WHERE
COT.CONTRACT_NUMBER = Q_L.CONTRACT_NUMBER(+) AND COT.ITEM_CODE =
Q_L.QUOTATION_ITEM_CODE(+)
Case study J1
Problematic SQL:• SELECT COT.CONTRACT_NUMBER ,COT.ITEM_CODE ,COT.QUANTITY ,
Q_L.QUOTATION_ITEM_CODE ,Q_L.QUOTATION_QUANTITY FROM (SELECT /*+INDEX(DT
ODS_CP_ORDER_OM_DAILY_TEMP_N1)*/ DT.CONTRACT_NUMBER, DT.ITEM_CODE,
SUM(DT.QUANTITY) QUANTITY FROM ODS_CP_ORDER_OM_DAILY_TEMP DT GROUP BY
DT.CONTRACT_NUMBER, DT.ITEM_CODE) COT, (SELECT /*+USE_NL(QH,QL)
LEADING(QH) INDEX(QH ODS_CP_QUOTATION_HEADERS_U1) INDEX(QL
ODS_CP_QUOTATION_LINES_N1)*/ QH.CONTRACT_NUMBER,
QL.QUOTATION_ITEM_CODE, SUM(QL.QUOTATION_QUANTITY) QUOTATION_QUANTITY
FROM ODS_CP_QUOTATION_HEADERS QH, ODS_CP_QUOTATION_LINES QL, (SELECT
DT.CONTRACT_NUMBER, DT.ITEM_CODE, SUM(DT.QUANTITY) QUANTITY FROM
ODS_CP_ORDER_OM_DAILY_TEMP DT GROUP BY DT.CONTRACT_NUMBER,
DT.ITEM_CODE) ODT WHERE QH.QUOTATION_HEADER_ID =
QL.QUOTATION_HEADER_ID AND ODT.CONTRACT_NUMBER = QH.CONTRACT_NUMBER
AND ODT.ITEM_CODE = QL.QUOTATION_ITEM_CODE GROUP BY
QH.CONTRACT_NUMBER, QL.QUOTATION_ITEM_CODE) Q_L WHERE
COT.CONTRACT_NUMBER = Q_L.CONTRACT_NUMBER(+) AND COT.ITEM_CODE =
Q_L.QUOTATION_ITEM_CODE(+)
If Nested Loop is used in this case
-------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
-------------------------------------------------------------------------------------------------------------------------------------
......
| 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
| 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
.....
| 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• If nested loop is used for ODS_CP_QUOTATION_HEADERS
(rows 10M) and ODS_CP_QUOTATION_LINES (rows 180M)
……..
• 14604 rows selected.
• Elapsed: more than 2hr
Case study J1
If removed all HINTs and let Optimizer chooses:• SELECT COT.CONTRACT_NUMBER ,COT.ITEM_CODE ,COT.QUANTITY ,
Q_L.QUOTATION_ITEM_CODE ,Q_L.QUOTATION_QUANTITY FROM (SELECT /*+INDEX(DT
ODS_CP_ORDER_OM_DAILY_TEMP_N1)*/ DT.CONTRACT_NUMBER, DT.ITEM_CODE,
SUM(DT.QUANTITY) QUANTITY FROM ODS_CP_ORDER_OM_DAILY_TEMP DT GROUP BY
DT.CONTRACT_NUMBER, DT.ITEM_CODE) COT, (SELECT /*+USE_NL(QH,QL)
LEADING(QH) INDEX(QH ODS_CP_QUOTATION_HEADERS_U1) INDEX(QL
ODS_CP_QUOTATION_LINES_N1)*/ QH.CONTRACT_NUMBER,
QL.QUOTATION_ITEM_CODE, SUM(QL.QUOTATION_QUANTITY) QUOTATION_QUANTITY
FROM ODS_CP_QUOTATION_HEADERS QH, ODS_CP_QUOTATION_LINES QL, (SELECT
DT.CONTRACT_NUMBER, DT.ITEM_CODE, SUM(DT.QUANTITY) QUANTITY FROM
ODS_CP_ORDER_OM_DAILY_TEMP DT GROUP BY DT.CONTRACT_NUMBER,
DT.ITEM_CODE) ODT WHERE QH.QUOTATION_HEADER_ID =
QL.QUOTATION_HEADER_ID AND ODT.CONTRACT_NUMBER = QH.CONTRACT_NUMBER
AND ODT.ITEM_CODE = QL.QUOTATION_ITEM_CODE GROUP BY
QH.CONTRACT_NUMBER, QL.QUOTATION_ITEM_CODE) Q_L WHERE
COT.CONTRACT_NUMBER = Q_L.CONTRACT_NUMBER(+) AND COT.ITEM_CODE =
Q_L.QUOTATION_ITEM_CODE(+)
Case study J1 : Checking ……• -------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• -------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:20:21.00 | 957K| 2383K|
• |* 1 | HASH JOIN OUTER | | 1 | 99751 | 14604 |00:20:21.00 | 957K| 2383K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• | 6 | HASH GROUP BY | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• |* 7 | HASH JOIN | | 1 | 46M| 541K|00:09:55.83 | 945K| 2383K|
• | 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
• | 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
• | 10 | VIEW | VW_GBC_5 | 1 | 46M| 559M|00:20:02.72 | 628K| 2368K|
• | 11 | HASH GROUP BY | | 1 | 46M| 559M|00:18:31.21 | 628K| 2368K|
• |* 12 | HASH JOIN | | 1 | 46M| 627M|00:02:08.95 | 628K| 615K|
• | 13 | VIEW | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• -------------------------------------------------------------------------------------------------------------------------------------
Check if “E-Rows” close to “A-Rows”
If yes, go to next step
If no, gather statistics and re-test
Are the underlying tables and indexes analyzed? YES
Case study J1 : Checking ……• -------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• -------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:20:21.00 | 957K| 2383K|
• |* 1 | HASH JOIN OUTER | | 1 | 99751 | 14604 |00:20:21.00 | 957K| 2383K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• | 6 | HASH GROUP BY | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• |* 7 | HASH JOIN | | 1 | 46M| 541K|00:09:55.83 | 945K| 2383K|
• | 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
• | 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
• | 10 | VIEW | VW_GBC_5 | 1 | 46M| 559M|00:20:02.72 | 628K| 2368K|
• | 11 | HASH GROUP BY | | 1 | 46M| 559M|00:18:31.21 | 628K| 2368K|
• |* 12 | HASH JOIN | | 1 | 46M| 627M|00:02:08.95 | 628K| 615K|
• | 13 | VIEW | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• -------------------------------------------------------------------------------------------------------------------------------------
Does the SQL already have optimized joining method?
Case study J1 : Checking ……• -------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• -------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:20:21.00 | 957K| 2383K|
• |* 1 | HASH JOIN OUTER | | 1 | 99751 | 14604 |00:20:21.00 | 957K| 2383K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• | 6 | HASH GROUP BY | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• |* 7 | HASH JOIN | | 1 | 46M| 541K|00:09:55.83 | 945K| 2383K|
• | 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
• | 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
• | 10 | VIEW | VW_GBC_5 | 1 | 46M| 559M|00:20:02.72 | 628K| 2368K|
• | 11 | HASH GROUP BY | | 1 | 46M| 559M|00:18:31.21 | 628K| 2368K|
• |* 12 | HASH JOIN | | 1 | 46M| 627M|00:02:08.95 | 628K| 615K|
• | 13 | VIEW | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• -------------------------------------------------------------------------------------------------------------------------------------
Because the “A-Rows” (390K, 10M, 180M) are big numbers, hash join is themost appropriate join
Does the SQL already have optimized joining method? YES
Case study J1
If using the correct joining method
• 14604 rows selected.
• Elapsed: 00:20:21.85
Nested Loops VS Hash Join
• Is Hash Join better than Nested Loop?
Example:SELECT SD.SALESREP_ID,SD.SALESREP_CODE,SD.SALESREP_DESCRIPTION,
OE.CUST_PO_NUMBER
FROM
ODS_OE_ORDER_HEADERS_ALL OE, BL_FIN_SALESREP_DIM SD
WHERE OE.SALESREP_ID = SD.SALESREP_ID
AND SD.SOURCE_CODE_ID = 1
AND OE.FLOW_STATUS_CODE <> 'CANCELLED'
AND NVL(OE.PURGE_DELETE_FLAG,'N') = 'N' AND OE.SALESREP_ID IS NOT NULL
AND OE.CUST_PO_NUMBER = '0006821002170H' AND ROWNUM = 1
Case J2 – Nested LoopsElapsed: 00:00:00.00
---------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
---------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.01 | 7 | 4 |
|* 1 | COUNT STOPKEY | | 1 | | 1 |00:00:00.01 | 7 | 4 |
| 2 | NESTED LOOPS | | 1 | | 1 |00:00:00.01 | 7 | 4 |
| 3 | NESTED LOOPS | | 1 | 1 | 1 |00:00:00.01 | 6 | 4 |
|* 4 | TABLE ACCESS BY INDEX ROWID| ODS_OE_ORDER_HEADERS_ALL | 1 | 2 | 1 |00:00:00.01 | 4 | 4 |
|* 5 | INDEX RANGE SCAN | ODS_OE_ORDER_HEADERS_ALL_N4 | 1 | 2 | 1 |00:00:00.01 | 3 | 3 |
|* 6 | INDEX UNIQUE SCAN | BL_FIN_SALESREP_DIM_U1 | 1 | 1 | 1 |00:00:00.01 | 2 | 0 |
| 7 | TABLE ACCESS BY INDEX ROWID | BL_FIN_SALESREP_DIM | 1 | 1 | 1 |00:00:00.01 | 1 | 0 |
---------------------------------------------------------------------------------------------------------------------------------
Are the underlying tables and indexes analyzed? YES
Does the SQL already have optimized joining method? YES, those result set are small, 1row for ODS_OE_ORDER_HEADERS_ALL & 1 row for BL_FIN_SALESREP_DIM
Nested Loops VS Hash Join
• What if using Hash join?
Example :SELECT /*+ USE_HASH(OE SD) */
SD.SALESREP_ID,SD.SALESREP_CODE,SD.SALESREP_DESCRIPTION,
OE.CUST_PO_NUMBER
FROM
ODS_OE_ORDER_HEADERS_ALL OE, BL_FIN_SALESREP_DIM SD
WHERE OE.SALESREP_ID = SD.SALESREP_ID
AND SD.SOURCE_CODE_ID = 1
AND OE.FLOW_STATUS_CODE <> 'CANCELLED'
AND NVL(OE.PURGE_DELETE_FLAG,'N') = 'N' AND OE.SALESREP_ID IS NOT NULL
AND OE.CUST_PO_NUMBER = '0006821002170H' AND ROWNUM = 1
Case J2 – Hash JoinElapsed: 00:00:00.00
----------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | OMem |
----------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.01 | 68 | |
|* 1 | COUNT STOPKEY | | 1 | | 1 |00:00:00.01 | 68 | |
|* 2 | HASH JOIN | | 1 | 2 | 1 |00:00:00.01 | 68 | 841K|
|* 3 | TABLE ACCESS BY INDEX ROWID | ODS_OE_ORDER_HEADERS_ALL | 1 | 5 | 5 |00:00:00.01 | 8 | |
|* 4 | INDEX RANGE SCAN | ODS_OE_ORDER_HEADERS_ALL_N4 | 1 | 5 | 5 |00:00:00.01 | 3 | |
|* 5 | TABLE ACCESS STORAGE FULL FIRST ROWS| BL_FIN_SALESREP_DIM | 1 | 1591 | 3314 |00:00:00.01 | 60 | |
----------------------------------------------------------------------------------------------------------------------------------------
Estimate rows and actual rows almost the same
Same Elapsed time
Nested Loops VS Hash Join
NESTED LOOPS HASH JOIN
Elapsed Time: 00:00:00.01 00:00:00.01
Buffers Get: 7 68
•NESTED LOOPS has lower Buffers Get in small result set joining
•During SQL tuning, objective :
Reduce logical reads / buffer gets
Most reliable metric
Reduce CPU time
•Optimizer generates execution plans based on the 2 metrics
•Given a specific plan, CPU time and Buffer Gets won‟t change. But other metrics such as Elapsed time, Physical reads could change
推进技能 - Recommended Methodology
2. Does the SQL already have optimized joining
method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove /
add it and re-test
YES, go to next step
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or
remove INDEX() or similar hints
Exadata Overview - Hardware Architecture
Database Grid Storage Server
InfiniBand Network
• Redundant 40Gb/s switches
• Unified server & storage network
• 14 High-performance low-cost
storage servers• 8 Dual-processor x64 database servers
OR
• 2 Eight-processor x64 database servers
• 100 TB High Performance disk, or336 TB High Capacity disk
•5.3 TB PCI Flash
•Data mirrored across storage servers
Exadata Features
• Exadata Smart Scans
– 10X or greater reduction in data sent to database servers
• Exadata Storage Indexes
– Eliminate unnecessary I/Os
• Hybrid Columnar Compression
– Efficient compression increases effective storage capacity and increases user data scan bandwidths by a factor of up to 10X
• Exadata Smart Flash Cache
– Breaks random I/O bottleneck by increasing IOPs by up to 20X
– Doubles user data scan bandwidths
• I/O Resource Manager (IORM)
– Enables storage grid by prioritizing I/Os to ensure predictable performance
Smart IO – what?
– Smart IO is not Block IO
– Block IO - data is shipped to the location where it can be
processed - RDBMS
– Smart IO –
• Some of the processing is shipped to where data resides –
Exadata Storage Server
• Results from the storage layer may be further processed in the
RDBMS
Smart IO – why? ( for Performance)
– Reduced network IO
• Data get filtered due to smart IO operations offloaded to the
storage layer
– Reduces the processing burden on the host
– Horizontal parallelism –
• Concurrent processing of the smart IO requests by many
exadata storage servers
• Concurrent processing of smart IO requests, from a single
database process, by many threads within a single exadata
storage server
– Vertical (pipeline) parallelism –
• Exadata storage servers processing more results while
database is consuming results already returned
Smart IO – How?
– Smart IO implementation is distributed across both RDBMS and
Exadata storage server(s)
– RDBMS implements smart IO applications and may choose to use
smart IO as opposed to block IO
– RDBMS drives smart IO
– Exadata storage server serves smart IO
Smart Scan Pre-requisite
• There must be a full scan on an object• FTS (TABLE ACCESS STORAGE FULL)
• INDEX_FFS (INDEX STORAGE FAST FULL SCAN)
• BITMAP INDEX SCAN (BITMAP INDEX STORAGE FAST FULL SCAN)
• The scan must use Oracle‟s Direct Path Read mechanism• Mechanism changed in 11g favoring Exadata
• If a table smaller than _small_table_threshold, the table will still be cached in SGA though PARALLEL is used with PARALLEL_DEGREE_POLICY=MANUAL
• _small_table_threshold = 400M default
• The object must be stored on Oracle‟s Exadata Storage
• Smart Scan showed in execution plan doesn‟t mean it‟s really using Smart Scan
Explain plan – table scan – no exadata
-----------------------------------------------------------------------------------
| Id | Operation | Name |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| * 1 | HASH JOIN | |
| * 2 | HASH JOIN | |
| * 3 | TABLE ACCESS FULL | SALES |
| * 4 | TABLE ACCESS FULL | SALES |
| * 5 | TABLE ACCESS FULL | SALES |
------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------------------------------------------------
1 - access("T"."CUST_ID"="T2"."CUST_ID" AND "T1"."PROD_ID"="T2"."PROD_ID" AND "T1"."CUST_ID"="T2"."CUST_ID")
2 - access("T"."PROD_ID"="T1"."PROD_ID")
3 - filter("T1"."PROD_ID"<200 AND "T1"."AMOUNT_SOLD"*"T1"."QUANTITY_SOLD">10000 AND "T1"."PROD_ID"<>45)
4 - filter("T"."PROD_ID"<200 AND "T"."PROD_ID"<>45)
5 - filter("T2"."PROD_ID"<200 AND "T2"."PROD_ID"<>45)
Explain plan– table scan - exadata
-----------------------------------------------------------------------------------
| Id | Operation | Name |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| * 1 | HASH JOIN | |
| * 2 | HASH JOIN | |
| * 3 | TABLE ACCESS STORAGE FULL | SALES |
| * 4 | TABLE ACCESS STORAGE FULL | SALES |
| * 5 | TABLE ACCESS STORAGE FULL | SALES |
------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------------------------------------------------
1 - access("T"."CUST_ID"="T2"."CUST_ID" AND "T1"."PROD_ID"="T2"."PROD_ID" AND "T1"."CUST_ID"="T2"."CUST_ID")
2 - access("T"."PROD_ID"="T1"."PROD_ID")
3 - storage("T1"."PROD_ID"<200 AND "T1"."AMOUNT_SOLD"*"T1"."QUANTITY_SOLD">10000 AND
"T1"."PROD_ID"<>45)
filter("T1"."PROD_ID"<200 AND "T1"."AMOUNT_SOLD"*"T1"."QUANTITY_SOLD">10000 AND "T1"."PROD_ID"<>45)
4 - storage("T"."PROD_ID"<200 AND "T"."PROD_ID"<>45)
filter("T"."PROD_ID"<200 AND "T"."PROD_ID"<>45)
5 - storage("T2"."PROD_ID"<200 AND "T2"."PROD_ID"<>45)
filter("T2"."PROD_ID"<200 AND "T2"."PROD_ID"<>45)
Case study J1• -------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• -------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:20:21.00 | 957K| 2383K|
• |* 1 | HASH JOIN OUTER | | 1 | 99751 | 14604 |00:20:21.00 | 957K| 2383K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• | 6 | HASH GROUP BY | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• |* 7 | HASH JOIN | | 1 | 46M| 541K|00:09:55.83 | 945K| 2383K|
• | 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
• | 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
• | 10 | VIEW | VW_GBC_5 | 1 | 46M| 559M|00:20:02.72 | 628K| 2368K|
• | 11 | HASH GROUP BY | | 1 | 46M| 559M|00:18:31.21 | 628K| 2368K|
• |* 12 | HASH JOIN | | 1 | 46M| 627M|00:02:08.95 | 628K| 615K|
• | 13 | VIEW | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• -------------------------------------------------------------------------------------------------------------------------------------
The “INDEX FULL SCAN” prevents a Smart Scan on the table …
Smart Scan Pre-requisite
• There must be a full scan on an object• FTS (TABLE ACCESS STORAGE FULL)
• INDEX_FFS (INDEX STORAGE FAST FULL SCAN)
• BITMAP INDEX SCAN (BITMAP INDEX STORAGE FAST
FULL SCAN)
• The scan must use Oracle‟s Direct Path Read
mechanism
• Mechanism changed in 11g favoring Exadata
• If a table smaller than _small_table_threshold, the table
will still be cached in SGA
• _small_table_threshold = 400M default
Case study J1
• Try performing a full table scan instead and compare
the performance.
If you have an INDEX() hint, remove it.
If you have an RULE hint, remove it.
Add a FULL hint / INVISIBLE index to force a full table scan.
Case study J1• -------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• -------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:20:21.00 | 957K| 2383K|
• |* 1 | HASH JOIN OUTER | | 1 | 99751 | 14604 |00:20:21.00 | 957K| 2383K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• | 6 | HASH GROUP BY | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• |* 7 | HASH JOIN | | 1 | 46M| 541K|00:09:55.83 | 945K| 2383K|
• | 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
• | 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
• | 10 | VIEW | VW_GBC_5 | 1 | 46M| 559M|00:20:02.72 | 628K| 2368K|
• | 11 | HASH GROUP BY | | 1 | 46M| 559M|00:18:31.21 | 628K| 2368K|
• |* 12 | HASH JOIN | | 1 | 46M| 627M|00:02:08.95 | 628K| 615K|
• | 13 | VIEW | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• -------------------------------------------------------------------------------------------------------------------------------------
The “INDEX FULL SCAN” prevents a Smart Scan on the table …
ALTER INDEX ODS_CP_QUOTATION_HEADERS_U1 INVISIABLE
Candidate Indexes to Invisible for Smart Scan
Candidate Indexes to Invisible:
• INDEX RANGE SCAN - Oracle is reading 0 or more contiguous rows
from the index.
• INDEX FULL SCAN - Oracle is reading all rows from the index, and may
be accessing these rows in the underlying table.
• INDEX SKIP SCAN - Oracle is reading 0 or more rows from different
parts of the index, and may be accessing these rows in the underlying
table.
Generally Don‟t Invisible:
• UNIQUE INDEX UNIQUE SCAN - Oracle is reading 0 or 1 rows from
the index.
• INDEX FAST FULL SCAN - Oracle is reading all rows from the index,
and is not accessing these rows in the underlying table. ie. The index
contains all columns required to resolve the query without having to
lookup the table.
Before and After removing the index() hint
• ------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• ------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:03:32.09 | 740K| 615K|
• |* 1 | HASH JOIN OUTER | | 1 | 14604 | 14604 |00:03:32.09 | 740K| 615K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 25723 | 12200 |00:03:31.94 | 728K| 615K|
• | 6 | HASH GROUP BY | | 1 | 25723 | 12200 |00:03:31.94 | 728K| 615K|
• |* 7 | HASH JOIN | | 1 | 25723 | 548K|00:03:32.07 | 728K| 615K|
• | 8 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:01.44 | 99833 | 0 |
• |* 9 | HASH JOIN | | 1 | 46M| 627M|00:02:03.78 | 628K| 615K|
• | 10 | VIEW | VW_GBF_14 | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 11 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 12 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 13 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:20.74 | 615K| 615K|
• ------------------------------------------------------------------------------------------------------------------------------------
• -------------------------------------------------------------------------------------------------------------------------------------
• | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
• -------------------------------------------------------------------------------------------------------------------------------------
• | 0 | SELECT STATEMENT | | 1 | | 14604 |00:20:21.00 | 957K| 2383K|
• |* 1 | HASH JOIN OUTER | | 1 | 99751 | 14604 |00:20:21.00 | 957K| 2383K|
• | 2 | VIEW | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 3 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.14 | 12708 | 0 |
• | 4 | TABLE ACCESS STORAGE FULL | ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.11 | 12708 | 0 |
• | 5 | VIEW | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• | 6 | HASH GROUP BY | | 1 | 2665K| 12200 |00:20:20.85 | 945K| 2383K|
• |* 7 | HASH JOIN | | 1 | 46M| 541K|00:09:55.83 | 945K| 2383K|
• | 8 | TABLE ACCESS BY INDEX ROWID | ODS_CP_QUOTATION_HEADERS | 1 | 10M| 10M|00:00:05.51 | 316K| 0 |
• | 9 | INDEX FULL SCAN | ODS_CP_QUOTATION_HEADERS_U1 | 1 | 10M| 10M|00:00:01.33 | 11771 | 0 |
• | 10 | VIEW | VW_GBC_5 | 1 | 46M| 559M|00:20:02.72 | 628K| 2368K|
• | 11 | HASH GROUP BY | | 1 | 46M| 559M|00:18:31.21 | 628K| 2368K|
• |* 12 | HASH JOIN | | 1 | 46M| 627M|00:02:08.95 | 628K| 615K|
• | 13 | VIEW | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 14 | HASH GROUP BY | | 1 | 14604 | 14604 |00:00:00.13 | 12708 | 0 |
• | 15 | TABLE ACCESS STORAGE FULL| ODS_CP_ORDER_OM_DAILY_TEMP | 1 | 390K| 390K|00:00:00.10 | 12708 | 0 |
• | 16 | TABLE ACCESS STORAGE FULL | ODS_CP_QUOTATION_LINES | 1 | 180M| 180M|00:00:22.25 | 615K| 615K|
• -------------------------------------------------------------------------------------------------------------------------------------
Exadata - SmartScan
Before with good joining method:
14604 rows selected.
Elapsed: 00:20:21.85
After using smart scan:
14604 rows selected.
Elapsed: 00:03:32.90
Verify Smart Scan
• 10046 event traceElapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 976 0.00 0.00
SQL*Net message from client 976 256.36 304.17
asynch descriptor resize 13 0.00 0.00
gc cr multi block request 51 0.00 0.00
cell multiblock physical read 51 0.00 0.17
gc cr grant 2-way 18 0.00 0.00
cell single block physical read 18 0.00 0.00
reliable message 1 0.00 0.00
enq: KO - fast object checkpoint 2 0.00 0.00
cell smart table scan 728 0.00 0.19
Verify Smart Scan
• EXPLAIN PLAN / DBMS_XPLAN package
• Doesn‟t tell if Smart Scan really happen or not
• 10046 Trace• cell smart table scan
• cell smart index scan
• V$SESSTAT / V$MYSTAT
• cell scans
• V$SQL
• Offload Eligible Bytes
• IO_CELL_OFFLOAD_ELIGIBLE_BYTES
• IO_INTERCONNECT_BYTES
• DBMS_SQLTUNE.REPORT_SQL_MONITOR
• MONITOR hint
推进技能 - Recommended Methodology
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or
remove INDEX() or similar hints
YES, go to next step
“Many Oracle developers - usually those working on OLTP
systems - are told early in their careers that Full Table
Scans are bad. Many will then hold on to this prejudice
and never learn the truth.”
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
Case Study H1
• SQL Text
------------------------------
select /*+ FULL(ODS_OM_MTL_TRANSACTIONS) FULL() SQLID=4rf90vv0dbf8r */
t.transaction_id, t.transaction_date, t.organization_id, p.segment1 company_code,
p.segment3 account_code, p.segment4 product_code, p.segment6 region_code,
p.segment7 to_ic, r.segment1 item_code, s.subinventory_code, s.trx_source_line_id,
tt.description, q.transaction_type_name transaction_type, s.currency_code,
t.primary_quantity, t.base_transaction_value from ods_mtl_transaction_accounts t,
ods_mtl_material_transactions s,
ods_mtl_system_items_b r, ods_mtl_transaction_types q, ods_gl_code_combinations p,
ods_oe_order_lines_all ol, ods_oe_order_headers_all oh, ods_oe_transaction_types_tl
tt where tt.description like '%EMS%' and t.transaction_id = s.transaction_id and
t.organization_id = s.organization_id and s.transaction_type_id =
q.transaction_type_id and t.inventory_item_id = r.inventory_item_id and
t.organization_id = r.organization_id and t.reference_account =
p.code_combination_id and s.trx_source_line_id =
ol.line_id and ol.header_id = oh.header_id and oh.order_type_id =
tt.transaction_type_id -- and p.segment3 not in ('1260100', '1260200') and
s.transaction_type_id in (33, 15) and t.transaction_date >= to_date('2011-01-01',
'YYYY-MM-DD') and t.transaction_date < to_date('2011-02-01', 'YYYY-MM-DD') and
t.organization_id = 17221
This SQL is using Smartscan but can I speed it up?
Case Study H1 – Execution plan•
SQL Plan Monitoring Details (Plan Hash Value=1317483436)
==========================================================================================================================================================================================================================
| Id | Operation | Name | Rows | Cost | Time | Start | Execs | Rows | Read | Read | Mem | Activity | Activity Detail |
| | | | (Estim) | | Active(s) | Active | | (Actual) | Reqs | Bytes | | (%) | (# samples) |
==========================================================================================================================================================================================================================
| 0 | SELECT STATEMENT | | | | | | 1 | | | | | | |
| 1 | NESTED LOOPS | | | | | | 1 | | | | | | |
| 2 | NESTED LOOPS | | 15 | 181K | | | 1 | | | | | | |
| 3 | HASH JOIN | | 15 | 181K | 17 | +6 | 1 | 0 | | | 4M | | |
| 4 | TABLE ACCESS BY INDEX ROWID | ODS_MTL_SYSTEM_ITEMS_B | 5940 | 3931 | 22 | +1 | 1 | 107K | 19758 | 309MB | | 9.09 | Cpu (3) |
| | | | | | | | | | | | | | cell single block physical read (17) |
| 5 | INDEX RANGE SCAN | ODS_MTL_SYSTEM_ITEMS_B_N3 | 5940 | 36 | 17 | +6 | 1 | 107K | 615 | 10MB | | 0.45 | cell single block physical read (1) |
| 6 | NESTED LOOPS | | | | | | 1 | | | | | | |
| 7 | NESTED LOOPS | | 7510 | 177K | | | 1 | | | | | | |
| 8 | HASH JOIN | | 7467 | 148K | 1 | +22 | 1 | 0 | | | 1M | | |
| 9 | INDEX STORAGE FAST FULL SCAN | ODS_OE_TRANSACTION_TYPES_TL_N1 | 117 | 4 | 1 | +22 | 1 | 15 | | | | | |
| 10 | HASH JOIN | | 56419 | 148K | 199 | +22 | 1 | 0 | | | 157M | 0.45 | Cpu (1) |
| -> 11 | NESTED LOOPS | | | | 199 | +22 | 1 | 2M | | | | | |
| 12 | NESTED LOOPS | | 56419 | 128K | 199 | +22 | 1 | 2M | | | | 0.45 | Cpu (1) |
| -> 13 | NESTED LOOPS | | 56419 | 15041 | 199 | +22 | 1 | 2M | | | | | |
| 14 | INLIST ITERATOR | | | | 1 | +22 | 1 | 1 | | | | | |
| 15 | TABLE ACCESS BY INDEX ROWID | ODS_MTL_TRANSACTION_TYPES | 2 | 2 | 25 | +22 | 2 | 1 | | | | | |
| -> 16 | INDEX UNIQUE SCAN | IDX_ODS_MTL_TRANS_TYPES_U1 | 2 | 1 | 199 | +22 | 2 | 2 | | | | | |
| -> 17 | TABLE ACCESS BY INDEX ROWID | ODS_OM_MTL_TRANSACTIONS | 28210 | 14660 | 200 | +22 | 2
| | | | | | | | | | | | | | | cell single block physical read (116) |
| 18 | INDEX RANGE SCAN | ODS_OM_MTL_TRANSACTIONS_N13 | 53721 | 379 | 199 | +22 | 2 | 2M | 4148 | 65MB | | 1.36 | Cpu (2) |
| | | | | | | | | | | | | | cell single block physical read (1) |
| 19 | INDEX UNIQUE SCAN | ODS_OE_ORDER_LINES_ALL_U1 | 1 | 1 | 199 | +22 | 2M | 2M | 2M | 218K | 3GB | | 61.82 | Cpu (20) |
85602 | 1GB | | 5.91 | Cpu (8) |
| | | | | | | | | | | | | | cell list of blocks physical read (5) |
| 20 | TABLE ACCESS BY INDEX ROWID | ODS_OE_ORDER_LINES_ALL | 1 | 2 | 199 | +22 | 3M | 2M | 414K | 6GB | | 20.45 | Cpu (12) |
| | | | | | | | | | | | | | cell list of blocks physical read (31) |
| | | | | | | | | | | | | | cell single block physical read (2) |
| 21 | VIEW | index$_join$_007 | 3M | 17539 | | | | | | | | | |
| 22 | HASH JOIN | | | | | | | | | | | | |
| 23 | INDEX STORAGE FAST FULL SCAN | ODS_OE_ORDER_HEADERS_ALL_N3 | 3M | 4123 | | | | | | | | | |
| 24 | INDEX STORAGE FAST FULL SCAN | ODS_OE_ORDER_HEADERS_ALL_N9 | 3M | 9185 | | | | | | | | | |
| 25 | INDEX RANGE SCAN | ODS_MTL_TRANSACTION_ACCOUNT_N2 | 4 | 3 | | | | | | | | | |
| 26 | TABLE ACCESS BY GLOBAL INDEX ROWID | ODS_MTL_TRANSACTION_ACCOUNTS | 1 | 4 | | | | | | | | | |
| 27 | INDEX UNIQUE SCAN | ODS_GL_CODE_COMBINATIONS_U1 | 1 | 1 | | | | | | | | | |
| 28 | TABLE ACCESS BY INDEX ROWID | ODS_GL_CODE_COMBINATIONS | 1 | 2 | | | | | | | | | |
==========================================================================================================================================================================================================================
Table ods_oe_order_headers_all was executed 3M times.
Why inappropriate execution plan in Case Study
H1?
• Oracle's Cost Based Optimizer works by analyzing
several of the possible execution paths for a SQL and
choosing the one that it considers best. For instance,
a two table join could drive off table A and lookup
table B for each row returned, or it could drive off
table B. By adding in the possibilities of join methods
and index selection, the number of possible execution
paths increases.
Why inappropriate execution plan in Case Study
H1?
2 tables : table A & table B
Select * from A, B where A.a=B.a
1. Driving table A -> table B
Or
2. Driving table B -> table A
Why inappropriate execution plan in Case Study
H1?
3 tables : table A & table B & table C
Select * from A, B, C where
A.a=B.a and B.b=C.b
1. Driving (table A -> table B result set) -> table C
2. Driving (table B -> table A result set) -> table C
3. Driving (table B -> table C result set) -> table A
4. Driving (table C -> table B result set) -> table A
5. Driving table A -> (table B -> table C result set)
6. Driving table C -> (table B -> table A result set)
Why inappropriate execution plan in Case Study
H1?
• A three table join has three times as many
alternatives, a four table join has four times the
alternatives of a three table join. In general, the
number of possible execution paths for a join
statement is proportional to n! (ie. n x n-1 x n-2 x ... x
2 x 1), where n is the number of tables in the join.
No. of tables
No. of possible
execution plan
Why inappropriate execution plan in Case Study
H1?
• The problem of choosing the absolute best execution
path becomes near impossible as n increases.
Mathematicians call this an np-hard - or non-
polynomial - problem.
Why inappropriate execution plan in Case Study
H1?
• If you have a table join (ie. a FROM clause) with five
or more tables, and you have not included a hint for
join order (eg. ORDERED or LEADING ), then Oracle
may be joining the tables in the wrong order.
How to fix it
• If the tables are being joined in the wrong order, you
can supply a hint to suggest a better order.
• If Oracle is just starting with the wrong table, try a
LEADING hint to suggest the best table to start with.
SQLs with equi-joins will often get the rest of the joins
right if only they know where to start.
• For ultimate control, update the FROM clause to list
the tables in the exact order that they should be
joined, and specify the ORDERED hint.
Solution for Case Study H1
• SQL Text
------------------------------
select /*+ FULL(S) LEADING(ol) full(ol) full(oh) LEADING(t) FULL(t) FULL(p)
SQLID=4rf90vv0dbf8r */ t.transaction_id, t.transaction_date, t.organization_id,
p.segment1 company_code, p.segment3 account_code, p.segment4 product_code,
p.segment6 region_code, p.segment7 to_ic, r.segment1 item_code,
s.subinventory_code, s.trx_source_line_id, tt.description, q.transaction_type_name
transaction_type, s.currency_code, t.primary_quantity, t.base_transaction_value
from ods_mtl_transaction_accounts t,
ods_mtl_material_transactions s, ods_mtl_system_items_b r,
ods_mtl_transaction_types q, ods_gl_code_combinations p, ods_oe_order_lines_all ol,
ods_oe_order_headers_all oh, ods_oe_transaction_types_tl tt where tt.description
like '%EMS%' and t.transaction_id = s.transaction_id and t.organization_id =
s.organization_id and s.transaction_type_id = q.transaction_type_id and
t.inventory_item_id = r.inventory_item_id and t.organization_id = r.organization_id
and t.reference_account =
p.code_combination_id and s.trx_source_line_id = ol.line_id and ol.header_id =
oh.header_id and oh.order_type_id = tt.transaction_type_id -- and p.segment3 not in
('1260100', '1260200') and s.transaction_type_id in (33, 15) and t.transaction_date
>= to_date('2011-01-01', 'YYYY-MM-DD') and t.transaction_date < to_date('2011-02-
01', 'YYYY-MM-DD') and t.organization_id = 17221
Added HINT execution plan
With Smart Scan
470068 rows selected.
Elapsed: ~30mins
With Smart Scan + appropriate join
470068 rows selected.
Elapsed: 498 sec
推进技能 - Recommended Methodology
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
YES, go to next step
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
5. For low volume SQLs, are there any Full
Table/Partition Scans?
A. If yes, check if appropriate to add / use index
B. If no, go to next step
High Level Guideline for using INDEX
• If you think Oracle should be using an index to
resolve your query and it is not doing so, then make
sure
the index exists.
the index status is USABLE
• Discuss with the DBA the prospect of adding a new
index. Providing the index is efficient
High Level Guideline for using INDEX
• Index is good when
Used on a medium-large table (> 500 rows) as the
outer table of a Nested Loop join.
Used on a medium-large table (> 500 rows) in a
Nested Sub-Query.
• Still remember the step by step tuning Methodology?
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
6. Do you compare 9i, 10g and 11g execution plan?
A. Compare each plans and bind the best to SPM
Case Study C1
• SQL is running good in 9i & 10g but not after upgrade
to 11g
Compare execution plan
• To get the same 9i execution plan in 11g, before run
the SQL,
SQL>alter session set
"optimizer_features_enable"= '9.2.0.8';
• To get the same 10g execution plan in 11g, before
run the SQL,
SQL>alter session set
"optimizer_features_enable"= ‘10.2.0.5';
Compare execution plan
• If the execution plan in 9i is better than 11g, how can
we ensure the SQL using 9i plan to have better
performance?
• Added SQL>alter session set
"optimizer_features_enable"= '9.2.0.8'; in program
code ???
• Smart developer learns and do thing in a smart way
Smart Way
• Use SPM (SQL PLAN MANAGEMENT) for a
statement, subsequent executions of that
statement will use the SQL plan
baseline.
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
1. Are the underlying tables and indexes analyzed?
A. If yes, go to next step
B. If no, gather stats and re-test
2. Does the SQL already have optimized joining method?
A. If yes, go to next step
B. If no, check if inappropriate hint, if yes, remove / add it and re-test
3. Is the SQL using smart scan?
A. If yes, go to next step
B. If no, invisible indexes or add FULL() hint or remove INDEX() or similar
hints
4. Does SQL has appropriate join predicates?
A. If yes, go to next step
B. If no, adding appropriate HINT
5. For low volume SQLs, are there any Full Table/Partition Scans?
A. If yes, check if appropriate to add index
B. If no, go to next step
推进技能 - Recommended Methodology
• Most common SQL problems that are easy to identify and easy
to fix by below steps.
6. Do you compare 9i, 10g and 11g execution plan?
A. Compare each plans and bind the best to SPM
Additional SQL tuning idea
Additional SQL tuning idea
Select * from BL_PROJECT_CUST_DIM where PRIMARY_KEY_COLUMN = “xxxxxxxx”
-----------------------------------------------------------------------
| Id | Operation | Name | E-Rows |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | | |
| 1 | TABLE ACCESS STORAGE FULL| BL_PROJECT_CUST_DIM | 1 |
|* 2 | UNIQUE INDEX SCAN | BL_PROJECT_CUST_DIM_U1 | 1 |
-----------------------------------------------------------------------
Elapsed time : 0.0004sec
A simple SQL using a primary key and only have 1 record
Statistic are correct
Using correct index
What if this SQL in a CURSOR LOOP???
Additional SQL tuning idea
Process time
per record (sec)
No .of record in
loop
Elapsed time
(sec)
0.0004 10 0.004
0.0004 100 0.04
0.0004 1,000 0.4
0.0004 10,000 4
0.0004 100,000 40
0.0004 1,000,000 400
0.0004 10,000,000 4000
10046 Trace file in a cursor statement
• call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Execute 4 0.00 0.00 0 0 0 0
Execute 4 0.00 0.00 0 0 0 0
Execute 4 0.00 0.00 0 0 0 0
Execute 4 0.00 0.00 0 0 0 0
Execute 80 0.00 0.00 0 0 0 0
Execute 101407 6.77 6.77 0 0 0 0
Execute 101392 9.04 9.13 0 0 0 0
Execute 101407 13.60 13.78 0 0 0 0
Execute 101407 4.36 4.36 0 0 0 0
Execute 101407 9.22 9.06 0 0 0 0
Execute 101407 6.83 6.81 0 0 0 0
Execute 101407 11.23 11.13 0 0 0 0
Execute 101407 102.52 103.07 13 1234 721786 101407
Execute 22204 2.42 2.48 0 0 0 0
Execute 22089 6.11 6.28 6 5 177139 0
Execute 161265 6.69 6.72 0 0 0 0
Execute 161265 6.33 6.44 0 0 0 0
Execute 1 0.06 0.06 0 0 0 0
Execute 2 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Execute 139176 8.49 8.56 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Execute 139176 302.03 306.74 4750 6 2091868 0
Execute 1 0.00 0.00 0 0 0 0
Execute 139176 272.18 280.70 6351 557250 151754 139176
Execute 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 62 0 0
Execute 1 0.00 0.00 0 1 14 1
Execute 1 0.00 0.00 0 0 0 0
Execute 1596000 772.70 793.85 49170 596532 3867306 341993
Long term tuning suggestions ……Rewrite cursor
statement
• Too many loop cause by cursor statement
Rewrite cursor loop logic to batch / single statement
Effort to rewrite: High for existing, low for new
POC on SP_BL_CC_INV_MTL_BK1
Elapsed time using cursor loop : 80min
Elapsed time after rewrite : 20min
Record processed : ~1,300,000
END