module1_oracle query tuning

Tech Mahindra Limited confidential© Tech Mahindra Limited 2008

Oracle Query Tuning

Module 1

Objectives

At the end of this session, you will be able to:

Understand the tuning methodology domains

Understand the basic architecture of Oracle Server

Learn the stages in processing of SQL statement

Understand and optimize indexes and row access methods

2CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited

Agenda – Module 1

Tuning Methodology Domains

Basic Architecture of Oracle Server

Stages in Processing of SQL Statement

Indexes and Row Access Methods



Tuning Phases

Tuning Methodology

SQL Statement Tuning


Managing Performance

Start early

Set objectives

Tune and monitor compliance

Work together

Handle exceptions and changes


Tuning Phases

Performance Management has the following areas:Schema tuning

Data designIndexes

Application tuningSQL statementsProcedural code

Instance tuningDeals with utilization of SGA memory by oracle server

Database tuningDeals with physical management of data on the disk

User expectations: Service Level Agreements (SLA) form the basis of these

Hardware / Network tuning


Performance Problems

Inadequate consumable resources:CPUI/OMemory (may be detected as an I/O problem)Data communications resources

Design limitations

Locking


Tuning MethodologyTune business function

Tune data design

Tune process design

Tune SQL statements

Tune physical structure

Tune memory allocation

Tune I/O

Tune memory contention

Tune operating system

Networking issues


Tuning Roles

Application developer

Database administrator

Operating system administrator

1. Tune business function

2. Tune data design3. Tune process design

4. Tune SQL statements5. Tune physical structure

6. Tune memory allocation7. Tune I/O8. Tune memory contention

9. Tune operating system

10. Networking issues

Designer

Network administrator

Business Analyst


Overview of SQL Statement Tuning

Identify problematic SQLIdentify slow programs and look at SQL in the programUse SQL_Trace and TKPROF

Verify optimizer statisticsGather statistics, Maintain Statistics

Review execution plansFewest number of rows are being returned Views are used efficientlyEach table is being accessed efficientlyExamine predicates in the SQL statement


Overview of SQL Statement Tuning (Contd.)

Restructure SQL statementsWrite separate SQL statements for specific tasksControl the access path and join order with hints

Restructure indexesRemove nonselective indexes to speed the DMLReorder columns in existing concatenated indexes Add columns to the index to improve selectivity

Maintain execution plans


Brain Teasers

Performance Management must be started at the design stage itself. True/False

Creating more indexes will result in improved performance. True/False

_____ & _____ can be used to identify problematic SQL statements.


http://www.istockphoto.com/stock-illustration-6923591-thinking-outside-the-box.php

Summary

In this session, we learned:

Tuning SQL statements

Manage user expectations

Analyze the results at each step

Tune the physical schema

Choose when to use SQL

Reuse SQL statements when possible

Design and tune the SQL statement

Get maximum performance with the optimizer


Agenda – Module 1







Oracle Architecture Components

Oracle Database

Oracle Instances

System Global Area

Program Global Area


Oracle Server Architecture

Oracle Server

Oracle Database Oracle Instance

Physical

• Data Files

• Control Files

• Redo Log Files

Logical

• Tablespaces

• Segments

• Extents

• Blocks

• SGA Memory Structure

• Background Processes


Oracle Server Components

Instance

SGA

UserProcess

Shared Pool

Database

ServerProcess

Db BufferCache

Redo LogBuffer

ProgramUnit

PGA

Control Files

Data Files

Redo LogFiles

Archived log files

Parameterfile

Passwordfile


Oracle Database

A collection of data that is treated as a unit, which stores & retrieves related information

Three file types:

Control Files:

Contain information required to maintain and verify database integrityA database needs at least one control file

Data Files:

At least one data file for each tablespaceCan belong to only one tablespaceContain the data in database, including tables, indexes, rollback segments & temp segments

Redo Logs Files:

Contain a record of changes made to the database to enable data recoveryAt least two redo log files for a database

Database

Control Files

Data Files

Redo LogFiles


Oracle Instance

A means to access an Oracle database

Can open & use only one database at a time

Consists of SGA memory structure & background processes

Instance

SGA

Redo LogBuffer

Data BufferCache

Shared Pool

PMONDBW0SMON LGWRCKPT OthersBackground Processes

Memory Structures


System Global Area (SGA)

Stores database information

Consists of several memory structures:

Database Buffer Cache

Redo Log Buffer

Shared Pool

SGA

Redo LogBuffer

Data BufferCache

Shared Pool

Data Dict.Cache

LibraryCache


Database Buffer Cache

Stores the most recently used data

The data is read from, and written to data files

Size of each buffer in the buffer cache is equal to the size of an Oracle block

Size of a buffer based on DB_BLOCK_SIZE stored in init.ora

SGA

Redo LogBuffer

Data BufferCache

Shared Pool

Data Dict.Cache

LibraryCache


Redo Log Buffer

Records changes made to a database using the instance

The server process records changes in the redo log buffer

It records the block that is changed, the location of the change and the new value

The buffer is reused after it is filled, once all the old redo entries are recorded in the redo log files

SGA

Redo LogBuffer

Data BufferCache

Shared Pool

Data Dict.Cache

LibraryCache


Shared Pool

Stores the most recently executed SQL statements & used data from the data dictionary

The shared pool is sized by SHARED_POOL_SIZE stored in init.ora


Shared Pool (Contd.)

Has two components:

Library Cache: Stores information about the most recently used SQL statements. The library cache contains

The text of SQL statement

The Parse Tree: Compiled version of a statement

The Execution Plan

Data Dictionary Cache (Dictionary / Row Cache):

A collection of the most recently used definitions in the database

Stores information about database files, tables, indexes, columns, users, privileges etc.


Program Global Area (PGA)

PGA is a memory area that contains:Session informationCursor informationSQL execution work areasSort areaHash join areaBitmap merge areaBitmap create area

Work area size influences SQL performance

Work area can be automatically or manually managed

ServerProcess

PGA


Brain Teasers

The combination of the background processes and SGA is called ________.

Which of the following contains the most recently executed SQL statements & used data from the data dictionaryA. Shared PoolB. Database Buffer Cache C. Redo Log Buffer



Summary


Oracle architecture components

Oracle database

Oracle instances

System Global Area

Program Global Area


Agenda – Module 1







Stages in Processing SQL Statements:ParseBindExecuteFetch

Cursor Sharing


Processing a SQL Statement

1. Server process receives the request received from client machineand checks if query is already present in the Shared Pool

2. If it is not present in the Shared Pool, then it will copy the corresponding metadata from data file to data dictionary cache

3. Query is Parsed and placed it in library cacheChecks syntax, object names, and privilegesLock objects used during parse

4. Optimizer will find out different execution plans and execution cost for each execution plan

5. As per the cheapest plan, oracle copies data from hard disk datafile to database buffer cache

6. Execute: Identify rows satisfying the query criteria

7. Fetch: Return rows to PGA for formatting as per query requirement


Processing DML Statement

For DML statements like update, insert or delete steps are same till step number 5 of previous slide, processing a SQL queries

DML queries are processed in database buffer cache and NOT in hard disk directly

Corresponding change information (old value, new value, table name, column name) are placed in redo log buffer

When user commits, these redo log records are copied to redo log file in the hard disk

Locks are released


Stages in Processing SQL Statements

A Query has to go through:

1. Parse:

An SQL statement is passed from user process to Oracle

A parsed representation of the statement is loaded into a shared SQL area, which:

Searches for identical statements, else translates the statement, verifying it to be a valid statement

Checks syntax, object names from data dictionary & privileges

Locks objects used during parse

Creates & stores the execution plan

For distributed statements, routs all or parts of the statement to remote nodes that contain referenced data


Stages in Processing SQL Statements (Contd.)

Bind:

Oracle needs values for the variables listed in any statement

Binding obtains values for the variables

Execute: Processes the statements

For INSERT, no rows are locked as no data is being changed

For UPDATE & DELETE, all rows that the statement affects are locked from use by other users

Released after the next COMMIT, ROLLBACK or SAVEPOINT for the transaction

This ensures data integrity


Stages in Processing SQL Statements (Contd.)

Fetch: Returns rows to the user process

Rows are selected & ordered (if requested by the query)

Each successive fetch retrieves another row of the result, until the last row has been fetched


SQL Statement Processing Phases

Execute

Close

BindParse

Open

Fetch


Shared SQL Areas

SGA

Shared SQL

Cursor forSELECT statement 2

Cursor forSELECT statement 1

User A User B User C

SELECTstatement 1

SELECTstatement 2

SELECTstatement 1


Context Area or Cursor

Inside the shared SQL area, each SQL statement is parsed in its own part, known as a context area or cursor

Each cursor holds the following information:The parsed statement (static, dynamic, and recursive SQL, plus program units such as procedures and database triggers)The execution planA list of referenced objects


Benefits of Sharing Of Cursors

Improves performance of repeated query

Reduces parsing

Dynamically adjusts memory

Improves memory usage


Shared Cursor Requirements

Cursors can be shared only by SQL statements that have the following identical elements:

TextUppercase and lowercaseWhite space (spaces, tabs, carriage returns)Comments

Referenced objectsBind-variable data types


Sharing Cursors

If the case or the amount of white space is different, then the statements are not identical

If the objects belong to different users, then the statements are not identical

select * from sh.customers where cust_id=180;

select * from customers where cust_last_name ='Taylor';

select * from customers where cust_last_name ='Taylor';

OE

SH

select * from sh.CUSTOMERS where CUST_ID=180;


Bind Variables and Shared Cursors

Suppose you enter the following two statements:

Both statements are translated into:

select * from sh.customers where cust_id = :c

select * from sh.customers where cust_id = :d

select * from sh.customers where cust_id = :b1


Controlling Shared Cursors

The CURSOR_SHARING initialization parameter can be set to:

EXACT (default)Allows SQL statements to share the SQL area only when their texts match exactlyThis is the default behavior

SIMILARForces similar statements to share the SQL area without deteriorating execution plans

FORCEForces similar statements to share the executable SQL area, potentially deteriorating execution plans


Monitoring Shared Cursors

V$LIBRARYCACHE provides general information about the library cache

Information about individual SQL statements is contained in the following views:

V$SQLTEXT

V$SQLTEXT_WITH_NEWLINES

V$SQL_BIND_DATA

V$SQL and V$SQLAREA


Monitoring Shared Cursor Use

SELECT sql_text, version_count, loads,invalidations, parse_calls, sorts

FROM v$sqlareaWHERE parsing_user_id > 0AND command_type = 3ORDER BY sql_text;

version invali parsesql_text count loads dations calls sorts--------------- -------- ----- ------- ----- -----select * from 2 1 0 3 0customers whereCUST_ID = 180

select * from 1 2 1 1 0customers wherecust_id = 180


Monitoring Shared Cursor Use (Contd.)

Command_type=3Is for select command

sql_text: Gives you actual query which was fired

Parse calls: How many time query is being fired so far

Sorts: if query processing involved any sorting operation

Loads: how many times query is being parsed

Invalidations: if there are any DDL statements fired on the tables accessed by the query, while the query is getting processed thenearlier parsed version of the query is invalidated and parsing is done again

Version count: How many query calls are currently getting processed


Brain Teasers

Execution plan is prepared again for a query that is already executed. True/False

Which of the following is a possible option for the CURSOR_SHARING initialization parameter?A. EXACTB. SIMILARC. FORCE

D. All of the above

The Execution plan is prepared during the ___ phase.A. ParseB. Bind

C. Execute

D. Fetch



Summary


Different stages in Processing SQL Statements like Parse, Bind, Execute, Fetch

Cursor Sharing


Agenda – Module 1







ROWIDs

B* - Tree Index Structure

Bitmap Indexes


Index Hints

Function-based Indexes


An Index

Is a performance-tuning method of allowing faster retrieval of records

Creates an entry for each value that appears in the indexed columns

By default, Oracle creates B-tree indexes

Syntax:

CREATE INDEX index_nameON table_name (column1, column2, ... column_n)


ROWIDs

ROWIDs indicate the address of a row

Each table has a ROWID pseudo column

SQL> select car_id, rowid from ACCIDENT;

CAR_ID ROWID---------- ----------------------

10 AAAJQ7AAJAAAFSQAAA10 AAAJQ7AAJAAAFSQAAB11 AAAJQ7AAJAAAFSQAAC13 AAAJQ7AAJAAAFSQAAD10 AAAJQ7AAJAAAFSQAAE

objectID(6) Block No.(6)Data File No.(3) Row No.(3)


Types of Indexes

Unique and non-unique indexes

Composite indexes

Index storage techniques:

B* - TreeNormalDescendingReversal KeyFunction based

Bitmap


Unique & Non-Unique Indexes

Unique indexes are indexes that help maintain data integrity by ensuring that no two rows of data in a table have identical key values

A unique index can also be used to improve data retrieval performance during query processing

Non-unique indexes, on the other hand, are not used to enforce constraints on the tables with which they are associated

Instead, non-unique indexes are used solely to improve query performance by maintaining a sorted order of data values that are used frequently


Composite Indexes

Composite indexes are indexes that are generating using combination of values of two or more columns

We can combine at the max 32 columns to create Index

SQL> CREATE INDEX depatnumber_job2 ON EMPOYEES (deptno, job);


B - Tree Indexes

Each B-tree index has a root block as a starting point. Depending on the number of entries, there are one or more branch blocks that can have one or more leaf blocks

The leaf blocks contain all values of the index plus ROWIDsthat point to the rows in the associated data segment

Previous and next block pointers connect the leaf blocks so that they can be traversed from left to right (and vice versa)


B*- Tree Indexes

Index entry header

Key column length

Key column value

ROWID

Root

Branch

Leaf

Index Entry


Types of B - Tree Indexes

Normal Default B-tree indexCreated in ascending orderFrom start value to large value

DescendingIf majority of time, query is searching for element, which is onhigher side of valueSo instead of small value first, we can have index starting fromhigher value first. So that you can search very fast

Reversal KeyThis is also function based index. Instead of finding value on normal search, we can search from reversal of value


Types of B - Tree Indexes (Contd.)

Function-based indexes provide an efficient mechanism for evaluating statements that contain expressions in their WHERE clauses

You can create a function-based index toMaterialize computational-intensive expressionsFacilitate case-insensitive searchesProvide a simple form of data compressionCan be used for an NLS sort index

SQL> SELECT *2 FROM customers3 WHERE UPPER(cust_last_name) = 'SMITH';

SQL> CREATE INDEX FBI_UPPER_LASTNAME2 ON CUSTOMERS(upper(cust_last_name));


Indexes and Constraints

The Oracle Server implicitly creates or uses B*-Tree indexes when you define the following:

Primary key constraints

Unique constraints

CREATE TABLE channels( channel_id CHAR(1)

CONSTRAINT chan_pk PRIMARY KEY, channel_desc VARCHAR2(20)

CONSTRAINT chan_desc_nn NOT NULL, channel_class VARCHAR2(20), channel_total VARCHAR2(13)

);


Bitmap Indexes

Compared with regular B*-tree indexes, bitmap indexes are faster and use less space for low-cardinality columns

Each bitmap index comprises storage pieces called bitmaps

Each bitmap contains information about a particular value for each of the indexed columns


Example - Bitmap Index

SELECT cust_id, cust_gender, cust_marital_status, cust_income_level FROM customers;

CUST_ID C CUST_MARITAL_STATUS CUST_INCOME_LEVEL ---------- - -------------------- ------------------------------------------70 F D: 70,000 - 89,999 80 F Married H: 150,000 - 169,999 90 M Single H: 150,000 - 169,999 100 F I: 170,000 - 189,999 110 F Married C: 50,000 - 69,999 120 M Single F: 110,000 - 129,999 130 M J: 190,000 - 249,999 140 M Married G: 130,000 - 149,999


Example - Bitmap Index (Contd.)There are only two possible values for gender, bitmap indexes are ideal for these column

Do not create a bitmap index on cust_id because this is a unique columnCREATE BITMAP INDEX cust_genderON customers (cust_gender);

Gender ‘M’ Gender ‘F’---------------------------------------------------------------------cust_id 70 0 1

cust_id 80 0 1

cust_id 90 1 0

cust_id 100 0 1

cust_id 110 0 1

cust_id 120 1 0

cust_id 130 1 0


When to Use Bitmap Indexes

Use bitmap indexes for:Columns with low cardinalityColumns that are frequently used in:

Complex WHERE clause conditionsGroup functions (such as COUNT and SUM)

Very large tablesComplicated composite join condition with ‘and’ clause

SELECT COUNT(*) FROM customers WHERE cust_marital_status = 'married' AND cust_income_level IN ('H: 150,000 - 169,999', 'G: 130,000 - 149,999');

Status=‘married’

Cust_income_level=‘H’

Cust_income_level=‘G’

0 0 0 0 0 0

1 1 0 1 1 1

1 0 1 1 1 1

0 0 1 0 1 0

0 1 0 0 1 0

ORAND = AND =


Bitmap Join Index

In addition to a bitmap index on a single table, you can create a bitmap join index (Oracle9i or higher version)

A bitmap join index is a bitmap index for the join of two or more tables

A bitmap join index is a space-efficient way of reducing the volume of data that must be joined by performing restrictions in advance


What Is a Bitmap Join Index?

No join with the CUSTOMERS table is needed

Only the index and the SALES table are used to evaluate the following query

SELECT SUM(s.amount_sold) FROM sales s, customers cWHERE s.cust_id =

c.cust_idAND c.cust_city = 'Sully';

Sales Customers

CREATE BITMAP INDEX cust_sales_bjiON sales(c.cust_city) FROM sales s, customers cWHERE c.cust_id = s.cust_id;

SA

LE

SA

MT

CU

STID

CU

STID

NA

ME

CIT

Y


How Bitmap Join Indexes Work?

We have Customer and Sales serving as the one-to-many relationship

We create an index on the Sales using columns contained in the Sales and Customer tables

The idea behind a bitmap join index is to pre-join the low cardinality columns, making the overall join faster

To create a bitmap join index, issue the following Oracle DDL

CREATE BITMAP INDEX cust_sales_bjiON sales(c.cust_city) FROM sales s, customers cWHERE c.cust_id = s.cust_id;


How Bitmap Join Indexes Work? (Contd.)

When the user tries to find what the total cost of all sales forthe Sully customers by firing the query given below, the Oracle Database can use the bitmap join index and the SALES table

In this case, there is no need to compute the join between the two tables explicitly

Using the bitmap join index here is much faster as it eliminates the actual execution of the join condition for execution of the following query

SELECT SUM(s.amount_sold) FROM sales s, customers cWHERE s.cust_id = c.cust_idAND c.cust_city = 'Sully';


Skip Scanning of Indexes

Index skip scanning enables access through a composite index when the prefix column is not part of the predicate

Skip scanning is supported by:Cluster indexesDescending scansCONNECT BY clauses

Skip scanning is not supported by reverse key or bitmap indexes


Skip Scanning Index at Work

SELECT * FROM employees WHERE age BETWEEN 20 AND 29

F10F11F12F13F14F15

F10 F16 F20 F26 F30 M10 M16 M20 M26 M30

F10 M10

F16F17F18F19

F20F21F22F23F24F25

F26F27F28F29

F30F31F32F33F34F35

M10M11M12M13M14M15

M16M17M18M19

M20M21M22M23M24M25

M26M27M28M29

M30M31M32M33M34M35

Index on (GENDER, AGE)


Indexing GuidelinesCost-based optimization may avoid the use of nonselective indexes in query execution

Oracle Server maintains all indexes defined against a table regardless of whether they are used

Index maintenance can present a significant CPU and I/O resource demand in any write-intensive application

Build indexes only if necessary

Drop indexes that an application is not using

Use the EXPLAIN PLAN statement to determine whether the optimizer will choose to use a index

If you create new indexes to tune a statement that is currently parsed, then the Oracle Server invalidates the statement


Indexing Guidelines (Contd.)

When the statement is next executed, the optimizer automaticallychooses a new execution plan that could potentially use the new index

If you create new indexes on a remote database to tune a distributed statement, then the optimizer considers these indexes when the statement is next parsed

The way you tune one statement can affect the optimizer’s choice of execution plans for other statements

For example, if you create an index to be used by one statement,then the optimizer can choose to use that index for other statements in the application as well

So reexamine the application’s performance and rerun the SQL trace facility after you have tuned the statements that you initially identified for tuning


When to Index?

A key is a column or expression on which you can build an index

Follow these guidelines for choosing keys to index:

Consider indexing keys that are used frequently in WHERE clauses

Consider indexing keys that are used frequently to join tables in SQL statements

Index keys that have high selectivity

The selectivity of an index is the percentage of rows in a tablehaving the same value for the indexed key


When to Index? (Contd.)

An index’s selectivity is optimal if few rows have the same value

Do not use standard B*- Tree indexes on keys or expressions with few distinct values

Such keys or expressions usually have poor selectivity and therefore do not optimize performance unless the frequently selected key values appear less frequently than the other key values

You can use bitmap indexes effectively in such cases, unless a high-concurrency OLTP application is involved in which the index is modified frequently

Do not index columns that are modified frequently


Indexes and Foreign Keys

The Oracle database does not automatically create an index for aforeign key constraint

If the foreign key columns are often used in join conditions, then you should create an index on them to enhance the join process

There are locking implications to DML activity on parent-child tables

Should consider creating an index if often used in join conditions

CUSTOMERS#cust_id

SALES PRODUCTS#prod_id

CHANNELS#channel_id


Data Dictionary Information

You can query the data dictionary to see specific information about function-based indexes

SQL> select table_name, index_name, index_type2 from user_indexes3 where index_name = 'FBI_UPPER_LASTNAME';

TABLE_NAME INDEX_NAME INDEX_TYPE------- ---------------- ------------------CUSTOMERS FBI_UPPER_LASTNAME FUNCTION-BASED NORMAL

SQL> SELECT i.index_name, i.index_type2 , ic.column_name, i.status3 FROM user_indexes i4 , user_ind_columns ic5 WHERE i.index_name = ic.index_name6 AND i.table_name='SALES';


Identifying Unused Indexes

Oracle provides the capability to gather statistics in the database about the usage of an index

Benefits

If you find an index that is never used, you can drop the index and free the space it used

Indexes must be maintained during most inserts and deletes and some updates

Eliminating an unused index eliminates this overhead

Parsing is simplified when the optimizer has fewer indexes to consider


Enabling & Disabling Monitoring of Index Usage

The V$OBJECT_USAGE view displays information about the usage of an index

Each time the index is altered with the MONITORING USAGE clause, V$OBJECT_USAGE is reset for the specified index

Previous information is lost, and a new start time is recorded

The new V$OBJECT_USAGE view supports the identification of unused indexes with the following columns:

INDEX_NAME: Index name

TABLE_NAME: Corresponding table

MONITORING: Indicates whether monitoring is on or off

USED: Indicates whether an index has been used during the monitoring time

START_MONITORING: Time that monitoring began on an index

STOP_MONITORING: Time that monitoring stopped on an index


Basic Row Access Methods

Full Table ScansMay involve reading many sequential data blocks from database files on disk into the buffer cacheHowever, the Oracle Server can read several sequential blocks in a single I/O (multiblock I/O)You can influence multiblock I/O by setting the DB_FILE_MULTIBLOCK_READ_COUNT parameterUsing the parallel query capability can help speed up full table scans by allocating several processes to the work

Index ScansIndexes improve the performance of many SQL statementsHowever, an indexed query may read from both index and data blocksThese blocks are usually not sequential, so the query cannot use multiblock readsUse indexes only when they improve performance


Basic Row Access Methods

Fast Full Index Scans

The fast full index scan is an alternative to a full table scan when there is an index that contains all the columns that are needed for a query

It is faster than a normal index scan because it can use multiblock I/O and can be parallelized in the same way as a table scan

Unlike regular index scans, however, the rows do not necessarily come back in sorted order

If there is a predicate that narrows the index search to a rangeof values, then the optimizer does not consider a fast full index scan


Which Algorithm to be Used?

Full table scanIf table is very smallMajority of records satisfy the query criteriaNo suitable index is available

Index scanSuitable index is availableTable size is largeOnly few records are expected to satisfy the query criteriaOrder fetch

Skip scanSpecial type of queryQuery having condition on only a part of key for a composite index

Fast full scanOnly in special case like aggregate functionNo table read is required


Brain Teasers

Indexes can be created across columns. True/False

_____ index may provide the best performance for queries based on columns with low cardinality

Which of the following can be used when no table read is required?

A. Full table scanB. Index scanC. Fast full scan



Summary


ROWIDs

B* - Tree Index Structure

Bitmap Indexes


Index Hints

Function-based Indexes


module1_oracle query tuning

Documents