advanced sql application tuning: find the proverbial needle in the haystack bert scalzo, ph.d....

Advanced SQL Application Advanced SQL Application Tuning: Find the Proverbial Tuning: Find the Proverbial

Needle in the HaystackNeedle in the Haystack

Bert Scalzo, Ph.D.Bert Scalzo, Ph.D.

Bert.Scalzo@Quest.comBert.Scalzo@Quest.com

About the AuthorAbout the Author

• Oracle DBA from 4 through 10g

• Worked for Oracle Education

• Worked for Oracle Consulting

• Holds several Oracle Masters

• BS, MS and PhD in Computer Science

• MBA and insurance industry designations

• Articles in

• Oracle Magazine

• Oracle Informant

• PC Week (now E-Magazine)

About Quest SoftwareAbout Quest Software

Where Can We LookWhere Can We Look

What is Tunable:

•Hardware

•Operating System

•Database

•Network

•Application

Too often people attempt to improve database application performance by focusing on just the first three: hardware, OS and RDBMS …

Performance PyramidPerformance Pyramid

Application

DBMS

OS

Hardware

Network

Golden Rule #1: It’s the application stupid!

Most application performance problems result from an application’s poor general design or grossly inefficient SQL

Where Should We LookWhere Should We Look

Do your developers know:•Sub-queries (correlated and not)•Outer Joins•Minus•Union and Union-All

Performance PyramidPerformance Pyramid

DBMS

OS

Hardware

Network

Application

Hardware Won’t CompensateHardware Won’t Compensate

Often people have unrealistic expectation that using expensive hardware is only way to obtain optimal application performance

•CPU•SMP•MPP

•Disk IO•15,000 RPM•RAID (EMC)

•OS•UNIX•64-bit

•Oracle•OPS / PQO•64-bit

Hardware Tuning ExampleHardware Tuning Example

Problem: Data load runtime 4+ hours•8 400-MHz 64-bit CPU’s•4 Gigabytes UNIX RAM•2 Gigabytes EMC Cache•RAID 5 (slower on writes)

Attempt #1: Bought more hardware•16 400-MHz 64-bit CPU’s•8 Gigabytes UNIX RAM•4 Gigabytes EMC Cache•RAID 1 (faster on writes)•Runtime still 4+ hours !!!

Application Tuning ExampleApplication Tuning Example

Attempt #2: Redesigned application•Convert PL/SQL to Pro-C•Run 16 streams in parallel•Better utilize UNIX capabilities•Run time = 20 minutes !!!

Attempt #3: Tuned the SQL code•Tune individual SQL statements•Use Dynamic SQL method # 2

•Prepare SQL outside loop•Execute Prep SQL in loop

•Run time = 15 minutes !!!

Lesson LearnedLesson Learned

Hardware:•Cost approximately $1,000,000•System downtime for upgrades•Zero runtime improvement•Loss of credibility with customer

Redesign:• 4 hours DBA $150/hour = $600•20 hours Developer $100/hour = $2000•Total cost = $2600 or 385 times less

Golden Rule #2: Application redesign much cheaper than hardware!

What Exactly is TuningWhat Exactly is Tuning

How you define tuning implicates how you will approach it. And the approach directly effects the tuning ROI that you can expect

Tuning is an art requiring creativity

Tuning is a science requiring regimen

Thus, we must use both intuition and rules …

How to Approach TuningHow to Approach Tuning

Tuning Approaches:

•Reactively (after the fact) - intuition•Proactively (as you go) - rules•Hybrid (combination) - both

Tuning as you go may be impractical, and after you’re done it may be too late. Often it’s best to apply general tuning guidelines as you go, and then identify problem areas for improvement.

Golden Rule #3: Best to tune before, during, and after!

Who will find the needle first?Who will find the needle first?

Carpenter Needs ToolsCarpenter Needs Tools

Too many people try to tune applications the hard way, by visual inspection based upon their own limited SQL knowledge

But SQL as a language is far too complex, and Oracle’s optimizers too dynamic to rely solely on the intuition of developers and DBA’s …

Golden Rule #4: Utilize tools to tune SQL!

Commercial Tuning ToolsCommercial Tuning Tools

Reactive• Quest’s Spotlight on Oracle• Oracle’s Performance Overview & Manager (OEM)• Embarcadero’s DBArtisan & Performance Center• BMC’s SmartDBA Cockpit & DBXray

Proactive• Quest’s SQL Navigator & TOAD with SQLab Xpert option• Oracle’s UTLPLP.SQL script (parallel explain plan script)• Oracle’s SQL Analyze (OEM)• Embarcadero’s Rapid SQL• BMC’s SQL Programmer• CA’s SQL Station (Plan Analyzer)

Hybrid• Quest’s SQLab Xpert• Oracle’s trace files and TKPROF utility• Oracle’s UTLBSTAT and UTLESTAT scripts• Oracle’s Stats Pack (only for versions 8i and newer)• Quest’s SQL Expert (formerly LECCO Technology)

Reactive: SpotlightReactive: Spotlight

High Logical & Physical IO’sHigh Logical & Physical IO’s

High Read & Write WaitsHigh Read & Write Waits

Examine Top SQL ConsumersExamine Top SQL Consumers

Explain Plan for Top SQLExplain Plan for Top SQL

Reactive SummaryReactive Summary

After the fact tuning dumped on the DBA.Often too little, too late ...

Pros:•DBA’s tend to know SQL very well

Cons:•DBA’s often focus more on OS and RDBMS •DBA’s often don’t know the application code•DBA’s often don’t know the data’s nature•DBA’s often don’t know the business rules•DBA’s generally have way too much to do

Proactive: TOAD Proc EditorProactive: TOAD Proc Editor

Proactive: TOAD SQL EditorProactive: TOAD SQL Editor

Proactive: TOAD SQL TuningProactive: TOAD SQL Tuning

Proactive SummaryProactive Summary

Tuning performed during coding by Developer.Often too much, too soon ...

Pros:•Developers focus is application code & data•Developers know the relevant business rules

Cons:•Developers must take time to identify problem

code•Developers often don’t tune SQL as well as DBA•Developers often measured by lines of code•Developers often not part of production support

Hybrid: SQLabHybrid: SQLab

Explain Plan for Top SQLExplain Plan for Top SQL

SQLab can Advise & Fix!!!SQLab can Advise & Fix!!!

Hybrid SummaryHybrid Summary

Best of both worlds for DBA’s and Developers.

SQL GuidelinesSQL Guidelines

Rule #1: Watch Indexed WHERE Conditions

Assume index on address (city, state)

•non-leading index column references cannot use indexes•where state = 'TX' [Index Not used]•where city = 'DALLAS' [Index Used]•where state = 'TX' and city = 'DALLAS' [Index Used]

•NOT, != and <> disable index use•where state not in ('TX', 'FL','OH') [Index Not used]•where state != 'TX' [Index Not used]

•NULL value references can never use indexes•where state IS NULL [Index Not used]•where state IS NOT NULL [Index Not used]

•expression references can never use indexes•where substr(city,1,3) = 'DAL' [Index Not used]•where city like 'DAL%' [Index Used]•where city || state = 'DALLASTX' [Index Not used]•where city = 'DALLAS' and state = 'TX‘ [Index Used]•where salary * 12 >= 24000 [Index Not used]•where salary >= 2000 [Index Used]

SQL GuidelinesSQL Guidelines

Rule #2:Watch Non-Indexed WHERE Conditions

•Oracle evaluates Non-Indexed conditions linked by AND bottom up

•Bad: select * from address whereareacode = 972 andtype_nr = (select seq_nr from code_table where type = ‘HOME’)

•Good: select * from address wheretype_nr = (select seq_nr from code_table where type = ‘HOME’) andareacode = 972

•Oracle evaluates Non-Indexed conditions linked by OR top down

•Bad: select * from address wheretype_nr = (select seq_nr from code_table where type = ‘HOME’) orareacode = 972

•Good: select * from address whereareacode = 972 ortype_nr = (select seq_nr from code_table where type = ‘HOME’)

SQL GuidelinesSQL Guidelines

Rule #3:Order Table in the FROM Clause

•important under rule based optimizer, and won't hurt under cost based optimizer

•order FROM clauses in descending order of table sizes based upon row counts

•for example

•select * from larger table, smaller table

•select * from larger table, smaller table, smallest table

•select * from larger table, smaller table, associative table

Note – rule based optimizer only

SQL GuidelinesSQL Guidelines

Rule #4:Consider IN or UNION in place of OR •if columns are not indexed, stick with OR

•if columns are indexed, use IN or UNION in place of OR

•IN example•Bad: select * from address where

state = 'TX‘ orstate = 'FL‘ orstate = 'OH‘

•Good: select * from address wherestate in ('TX','FL','OH')

•UNION example•Bad: select * from address where

state = ‘TX’ orareacode = 972

•Good: select * from address wherestate = ‘TX’

union select * from address where

areacode = 972

SQL GuidelinesSQL Guidelines

Rule #5:Weigh JOIN versus EXISTS Sub-Query

•use table JOIN instead of EXISTS sub-query

•when the percentage of rows returned from the outer sub-query is high

select e.name, e.phone, e.mailstopfrom employee e, department dwhere e.deptno = d.deptno and d.status = ‘ACTIVE’

•use EXISTS sub-query instead of table JOIN

•when the percentage of rows returned from the outer sub-query is low

select e.name, e.phone, e.mailstopfrom employee ewhere e.deptno in (select d.deptno from department d where d.status != ‘ACTIVE’)

SQL GuidelinesSQL Guidelines

Rule #6:Consider EXISTS in place of DISTINCT

•avoid joins that use DISTINCT, use EXISTS sub-query instead

•Bad: select distinct deptno, deptname from emp, dept whereemp.deptno = dept.deptno

•Good: select deptno, deptname from dept whereexists (select ‘X’ from emp where

emp.deptno = dept.deptno)

Note – only has to find one match

SQL GuidelinesSQL Guidelines

Rule #7:Consider NOT EXISTS in place of NOT IN

•avoid sub-queries that use NOT IN, use NOT EXISTS instead

•Bad: select * from emp wheredeptno not in (select deptno from dept where

deptstatus = ‘A’)

•Good: select * from emp wherenot exists (select ‘X’ from dept where

deptstatus = ‘A’ anddept.deptno = emp.deptno)

Note – only has to find one non-match

SQL GuidelinesSQL Guidelines

Rule #8:COUNT Using Indexed Column or Asterisk

•when counting rows, use COUNT on indexed column or asterisk

•select count(indexed_column) from table [Most Efficient]

•select count(*) from table

•Select count(non_indexed_column) from table

•select count(1) from table

Note – rule based optimizer only

SQL GuidelinesSQL Guidelines

Rule #9:Ordering Via the WHERE Clause

•a dummy WHERE clause referencing an indexed column will

•retrieve all records in ascending order (descending for 8i descending index)

•not perform a costly sort operation

•Bad: select * from addressorder by city

•Good: select * from address wherecity > ‘’

SQL GuidelinesSQL Guidelines

Rule #10:Use PL/SQL to reduce network traffic

•Utilize PL/SQL to group related SQL commands and thereby reduce network traffic

•Bad:select city_name, state_code into :v_city, :v_sate from zip_codes where zip_code = ‘75022’;

insert into customer (‘Bert Scalzo’,’75022’, :v_city, v_state);

•Good:begin select city_name, state_code into :v_city, :v_sate from zip_codes where zip_code = ‘75022’; insert into customer (‘Bert Scalzo’,’75022’, :v_city, v_state);end;/