oracle/aix performance and tuning...oracle server architecture dbwr system global area buffer cache...

4/22/2008 © 2008 IBM Corporation

IBM Systems & Technology Group

Technical Conference

14 – 18 April, 2008, Sevilla, Spain

Oracle/AIX Performance and Tuning

Session ID : pS014

Fabienne Lepetit – Oracle/IBM Joint Solutions CenterSébastien Chabrolles – IBM – Power Benchmark PSSC Montpellier

[email protected] / [email protected]

Version 2.0

21.04.2008

IBM Systems & Technology Group – Technical Conference 2008

© 2008 IBM Corporation2 4/22/2008

This presentation…

� … is a Oracle/IBM joint work : IBM PSSC Power benchmark center & IBM/Oracle Joint Solutions Center.

� … is based on our experience and feedback of benchmarks

� … is a checklist which can be used in most production environment

� … can be widely distributed to BPs and customers

� … has been built with many written informations in order to be easily read and presented – don’t hesitate to reuse it!



Learning objectives

� Configure AIX to support Oracle with the best performance

� Setup Oracle to take advantage of AIX features

� Understand the joint effort to tune both Oracle and AIX and how they interact



Agenda

�Oracle Server Architecture

�Memory

� AIX VMM tuning

� Oracle Memory Structure Allocation

� IO

� AIX LVM Striping

� AIX mount option

� Oracle ASM

� Asynchronous IO

�CPU

�Monitoring & Tuning

� Methodology & tools

� Focus on Two Oracle Tools: AWR & Grid Control

�Conclusion



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



Oracle Server Architecture

DBWR

System Global Area

Buffer Cache Log BufferSharedSQL

DB Files

Program Global AreaPrivate SQLArea

Oracle

database

Oracle

instance

LGWR SMON…

DIA0Background processes

Memory structures

Redologs Control files

Database files

how to manage Oracle memory structures

how to implement thedata on disk



The components of an Oracle server are : � database = set of datafiles (data + redologs + control files)� instance = memory structures allocated at startup + background processes (automatically forked at startup) associated to a database

� Database files

� datafiles – contains data (tables, indexes…)� redologs – contains redo entries� control files – record the physical structure of the database

� Oracle memory structures = System Global Area (SGA) + Process Global Area (PGA)

� SGA � shared memory region that contains data and control information for one Oracle instance� allocated at instance startup and deallocated at instance shutdown � each instance has its own SGA

� PGA � memory buffer that contains data and control information for a server process� PGA is created by Oracle when a server process is started� contains private SQL area

Oracle Server Architecture (cont’d)



� The three main components of the SGA are :

• buffer cache:- store the most recently used blocks of data- improves performance by reducing disk I/O activity

• shared pool contains : - shared SQL area (parse tree and execution plan for SQL statements) - dictionnary cache- library cache

• redolog bufferstores redo entries (log of changes made to the database) written to the online redo log (used during database recovery)

Oracle Server Architecture (cont’d) – Focus on the SGA

System Global AreaBuffer Cache Log BufferSharedSQL

Program Global AreaPrivate SQL Area



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



� Objective :

Tune the VMM to protect computational pages

(Programs, SGA, PGA) from being paged out and force the LRUD to steal pages from FS-CACHE only.

1 - AIX is started, applications load some computational pages into the memory.

As a UNIX system, AIX will try to take advantage of the free memory by using it as a cache file to reduce the IO on the physical drives.

2 - The activity is increasing, the DB needs more memory but there is no free pages available. LRUD (AIX page stealer) is starting to free some pages into the memory.

3 - With the default setting, LRUD will page out some computational pages instead of removing some pages from the File System Cache.

Virtual Memory Management



Memory : VMM Tuning for Filesystems (lru_file_repage)

This VMM tuning tip is applicable for AIX 5.2 ML4+ or AIX 5.3 ML1+

► lru_file_repage :

● This new parametter prevents the computational pages to be paged out.

● By setting lru_file_repage=0 (1 is the default value) you’re telling the VMM that your preference is to steal pages from FS Cache only.

► “New” vmo tuning approach : Recommendations based on VMM developer experience.

Use “vmo –p –o” to set the following parametters :

lru_file_repage = 0

minfree = 120 x # lcpus Increase if page stealing activity, vmstat “po” column

maxfree = minfree +(maxpaghead x # lcpus)

minperm% = from 3 to 10

maxperm% = maxclient% = from 70 to 90

strict_maxclient = 1



Memory : VMM Tuning for Filesystems (“legacy”)

Recommended VMM tuning for AIX version prior to 5.2 ML4 or AIX 5.3 ML1.

The following VMM tuning as the same objective : protect the computational pages to be page out.

► vmo “legacy” tuning approach :

Use “vmo –p –o” to set the following parametters :

minfree = 120 x # lcpus Increase if page stealing activity, vmstat “po” column

maxfree = minfree +(maxpaghead x # lcpus)

minperm% = from 3 to 10

maxperm% = maxclient% = from 5 to 15 but < minperm%

strict_maxclient = 0



Memory : AIX Memory Management for OracleSystem-wide contention scope (user thread/kernel thread one-to-one mapping) – reduces memory usage

of Oracle process & makes AIX post/wait mechanism most effective

► In Oracle9i, set AIX environment variable AIXTHREAD_SCOPE=S

► Since Oracle10g, AIXTHREAD_SCOPE=S is Internally set

Pinning shared memory – prevents paging out SGA (only if App + DB on same AIX and >80%computationnal pages)

► On AIX part

● vmo –p –o v_pinshm = 1● Leave maxpin% at the default of 80% unless the SGA exceeds 77% of real memory :

vmo –p –o maxpin%=[(total mem-SGA size)*100/total mem] + 3

► On Oracle part (9i/10g)

● LOCK_SGA = TRUE

Large Page Support – improves performance of prefetching

► On AIX part

● vmo –r –o lgpg_size = 16777216 –o lgpg_regions=(SGA size / 16 MB)● chuser capabilities=CAP_BYPASS_RAC_VMM,CAP_PROPAGATE oracle (allow Oracle user ID to use

Large Pages)● ldedit –b lpdata oracle (to allow oracle binary to use large page data)● export LDR_CNTRL=LARGE_PAGE_TEXT=Y@LARGE_PAGE_DATA=M (prior Oracle user ID to

starting Oracle instance and listener to allow both large page text and large page data)



Memory : Oracle Memory Structures Allocation

8i : Manual management

► All Oracle memory structures were manually managed and were mostly static (db_block_buffers…)

9i : Dynamic memory resizing

db_cache_size (dynamic parameter)

►sga_max_size (static parameter) – maximum size of the SGA for the lifetime of the instance.

►pga_aggregate_target (dynamic parameter) – specifies the target aggregate PGA memoryavailable to all server processes attached to the instance

►additional parameter : db_cache_advice (dynamic parameter) – enables or disables statistics gathering used for predicting behavior with different cache sizes.

10g : Automatic Shared Memory Management (ASMM)

►sga_target (dynamic) – if set the db_cache_size is automatically sized (shared_pool_size, large_pool_size, java_pool_size as well).

Can be increased up to sga_max_size.

To use Automatic Shared Memory Management, sga_target must be <>0.



Memory : Oracle Memory Structures Allocation NEW!11g

11g : Automatic Memory Management (AMM)

►memory_target (dynamic parameter) – specifies the total memory size to be used by the instance – SGA and PGA. Exchanges between SGA and PGA are done according to the needs.

If sga_target and pga_aggregate_target are not set the policy is to give 60% of memory_target to the SGA and 40% to the PGA.

►memory_max_target (static parameter) – specifies the maximum memory size for the instance.

To use Automatic Memory Management, memory_target must be <>0.

See Metalink notes 443746.1 and 452512.1 explaining AMM and these new parameters.



Memory : Use jointly AIX dynamic LPAR and Oracle dynamic allocation of memory + AMM

Scenario :Memory_max_size= 18 GB

Real memory= 12 GB

Memory_target= 8 GB

SGA

+

PGA

Oracle tuning advisor indicates that SGA+PGA need to be increased to 11GB:

memory_target can be increased dynamically to 11GB but real memory is only 12GB, so it needs to be increased as well.

► Step 1 - Increase physical memory allocated to the LPAR to 15 GB

► Step 2 - Increase SGA+PGA allocated to the instance of the database to 11GB

AIX

+ free

Memory_max_size= 18 GB

Initial configuration Final configuration

� Memory allocated to the system has been increased dynamically, using AIX DLPAR

� Memory allocated to Oracle (SGA and PGA) has been increased on the fly

ssh hscroot@hmc "chhwres -r mem

-m <system> -o a -p <LPAR name> -q 3072"

alter system set memory_target=11GB;

Real memory= 15 GB

AIX

+ free

Memory_target= 11 GB

SGA

+

PGA



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



IO : Database Layout

Stripe and mirror everything (S.A.M.E) approach:

�Goal is to balance I/O activity across all disks, loops, adapters, etc...

�Avoid/Eliminate I/O hotspots

�Manual file-by-file data placement is time consuming, resource intensive and iterative

�Additional advices to implement SAME :• apply the SAME strategy to data, indexes

• if possible separate redologs (+archivelogs)

�Having a good Storage configuration is a key point :

� Because disk is the slowest part of an infrastructure

� Reconfiguration can be difficult and time consuming

Oracle

recommendation



� RAID-5 vs. RAID-10 Performance Comparison

ExcellentExcellentRandom Read

ExcellentFairRandom Write

GoodExcellentSequential Write

ExcellentExcellentSequential Read

RAID-10RAID-5I/O Profile

IO : RAID Policy with ESS, DS6/8K

� With Enterprise class storage (with huge

cache), RAID-5 performances are comparable to

RAID-10 one (for most customer workloads)

� Consider RAID-10 for workloads with a high percentage of random write activity (> 25%) and high I/O access densities (peak > 50%)

Use RAID-5 or RAID-10 to create striped LUNs

If possible try to minimize the number of LUNs per RAID array to avoid contention on physical disk.

HW Striping

LUN 1

LUN 2

LUN 3

LUN 4

Storage



IO : 2nd Striping (LVM)

1. Luns are striped across physical disks (stripe-size : ~ 64k, 128k, 256k)

2. LUNs are seen as hdisk device on AIX server.

3. Create AIX Volume Group(s) (VG) with LUNs from multiple arrays

4. Logical Volume striped across hdisks (stripe-size : 8M, 16M, 32M, 64M)

=> each read/write access to the LV are well balanced accross LUNs and use the maximum number of physical disks for best performance.

HW Striping

LUN 1

LUN 2

LUN 3

LUN 4

StorageAIX

hdisk1

hdisk2

hdisk3

hdisk4

LV

M S

trip

ing

Volume Group

LV

(ra

wo

r w

ith

jfs2

)



IO : LVM Spreading / Striping

2 different ways to stripe on the LVM layer :

1. Stripe using Logical Volume (LV)

► Create Logical Volume with the striping option : mklv –S <stripe-size> ...

Valid LV Stripe sizes:

● AIX 5.2: 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1M

1. AIX 5.3+: AIX 5.2 Stripe sizes + 2M, 4M, 16M, 32M, 64M, 128M

2. PP striping (AKA spreading)

► Create a Volume Group with a 8M,16M or 32M PPsize. (PPsize will be the “Stripe size”)

● AIX 5.2 : Choose a Big VG and change the “t factor” : # mkvg –B –t <factor> –s <PPsize>...

● AIX 5.3 +: Choose a Scalable Volume Group : # mkvg –S –s <PPsize> ...

► Create LV with “Maximum range of physical volume” option to spread PP on different hdisk in a

Round Robin fashion : # mklv –e x ...

jfs2 filesystem creation advice :

If you create a jfs2 filesystem on a striped (or PP spreaded) LV, use the INLINE logging option. It willavoid « hot spot » by creating the log inside the filesystem (which is striped) instead of using a uniq PP stored on 1 hdisk.

# crfs –a logname=INLINE …



IO : Disk Subsystem Advices

� Check if the definition of your disk subsystem is present in the ODM.

� If the description shown in the output of “lsdev –Cc disk” the word “Other”, then it means that AIX doesn’t have a correct definition of your disk device in the ODM and use a generic device definition.

# lsdev –Cc disk

� In general, a generic device definition provides far from optimal performance since it doesn’t properly customize the hdisk device :

exemple : hdisk are created with a queue_depth=1

1. Contact your vendor or go to their web site to download the correct ODM definition for your storage subsystem. It will setup properly the “hdisk” accordingly to your hardware for optimal performance.

2. If AIX is connected to the storage subsystem with several Fiber Channel Cards for performance, don’t forget to install a multipath device driver or path control module.

- sdd or sddpcm for IBM DS6000/DS8000

- powerpath for EMC disk subsystem

- hdlm for Hitachi etc....

Generic device definitionbad performance



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



IO : Filesystems Mount Options (DIO, CIO)If Oracle data are stored in a Filesystem, some mount option can improve performance :

�Direct IO (DIO) – introduced in AIX 4.3.

• Data is transfered directly from the disk to the application buffer, bypassing the file buffer cache hence avoiding double caching (filesystem cache + Oracle SGA). • Emulates a raw-device implementation.

�To mount a filesystem in DIO$ mount –o dio /data

�Concurrent IO (CIO) – introduced with jfs2 in AIX 5.2 ML1

• Implicit use of DIO. • No Inode locking : Multiple threads can perform reads and writes on the same file at the same time. • Performance achieved using CIO is comparable to raw-devices.

�To mount a filesystem in CIO:$ mount –o cio /data

Bench throughput over run duration – higher

tps indicates better performance.



� Benefits :

1. Avoid double caching : Some data are already cache in the Application layer (SGA)

2. Give a faster access to the backend disk and reduce the CPU utilization

3. Disable the inode-lock to allow several threads to read and write the same file (CIO only)

IO : Benefits of CIO for Oracle

� Restrictions :

1. Because data transfer is bypassing AIX buffer cache, jfs2 prefetching and write-behind can’t be used. These functionnalities can be handled by Oracle.

⇒ (Oracle parameter) db_file_multiblock_read_count = 8, 16, 32, ... , 128 according to workload

2. When using DIO/CIO, IO requests made by Oracle must by aligned with the jfs2 blocksize to avoid a demoted IO (Return to normal IO after a Direct IO Failure)

=> When you create a JFS2, use the “mkfs –o agblksize=XXX” Option to adapt the FSblocksize with the application needs.

Rule : IO request = n x agblksize

Exemples: if DB blocksize > 4k ; then jfs2 agblksize=4096

Redolog are always written in 512B block; So jfs2 agblksize must be 512



IO : Direct IO demoted

� How to detect demoted IO :

Trace command to check demoted io :# trace –aj 59B,59C ; sleep 2 ; trcstop ; trcrpt –o directio.trcrpt# grep demoted directio.trcrpt

Waits on redolog (with demoted IO, FS blk=4k)

Waits% Time -outs Total Wait Time (s) Avg wait (ms) Waits /txn log file sync 2,229,324 0.00 62,628 28 1.53

Waits on redolog (without demoted IO, FS blk=512)

Waits% Time -outs Total Wait Time (s) Avg wait (ms) Waits /txnlog file sync 494,905 0.00 1,073 2 1.00

� Extract from Oracle AWR (test made in Montpellier with Oracle 10g)



CIO/DIO implementation Advices

mount -o rw mount -o rw

mount -o rw mount -o cio *(1)

Cached by Oracle

Cached by AIX

mount -o rw mount -o cio (jfs2 + agblksize=512)

Cached by Oracle

Cached by AIX

mount -o rw mount -o rbrw

Use JFS2 write-behind …

but are not kept in AIX Cache.

mount -o rw mount -o rw

Oracle bin and shared lib.

with Standard mount options with optimized mount options

Cached by AIX Cached by AIX

Oracle Datafiles

Oracle Redolog

Cached by Oracle

Cached by Oracle

Cached by AIXCached by AIX

Cached by AIXOracle Archivelog

Oracle Control files

*(1) : to avoid demoted IO : jfs2 agblksize = Oracle DB block size / n



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



IO : Oracle Automatic Storage Management (ASM)

• ASM is a database file system which provide cluster file system and volume manager capabilities. Alternative to conventional file-system and LVM functions. For Oracle datafiles.

• Integrated to Oracle DB with no additional cost, for single or RAC databases

• With ASM, the management of Oracle datafiles is the same for the DBA on all platforms (Unix, Linux, Windows)

• Datafiles are striped across all ASM disks - I/O is spread evenly to prevent hot spots and maximize performance

• Online add/drop of disk devices with automatic online redistribution of data

LUN 1

LUN 2

AIX

hdisk1

ASM DISKGROUP

StorageOracle

ASM striping

LUN 4

LUN 3

hdisk2

hdisk3

hdisk4



IO : Oracle Automatic Storage Management (ASM)

• ASM is implemented as a special kind of Oracle instance, with its own SGA and backgroundprocesses (by default, less than 100M of SGA are allocated for the ASM instance) and generate very little activity except during rebalance operations.

• A single ASM instance can service one or more instance databases.

• Can maintain redundant copies of data

• An ASM-managed database will have approximately the same performance as a database implemented in raw-devices.

ASM Instance

ASM DISKGROUP

Database on

ASM

create tablespace DATA

datafile ‘+DG/data01.dbf’ size 15G;



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



• Allows multiple requests to be sent without to have to wait until the disk subsystem has completed the physical IO.

• Utilization of asynchronous IO is strongly advised whatever the type of file-system and mount option implemented (JFS, JFS2, CIO, DIO).

IO : Asynchronous IO (AIO)

�Posix vs Legacy

Since AIX5L V5.3, two types of AIO are now available : Legacy and Posix. For the moment, theOracle code is using the Legacy AIO servers.

aioQ

Application aioserversDisk

1

2

3 4



IO : Asynchronous IO (AIO) tuning

� Rule of thumb :

maxservers should be = (10 * <# of disk accessed concurrently>) / # cpu

maxreqs (= a multiple of 4096) should be > 4 * #disks * queue_depth

� but only tests allow to set correctly minservers and maxservers

• important tuning parameters :

check AIO configuration with : AIX 5.X : lsattr –El aio0 AIX 6.1 : ioo –L | grep aio

• maxreqs : size of the Asynchronous queue.

• minserver : number of kernel proc. Aioservers to start (AIX 5L system wide).

• maxserver : maximum number of aioserver that can be running per logical CPU

� Monitoring :In Oracle’s alert.log file, if maxservers set to low : “Warning: lio_listio returned EAGAIN” “Performance degradation may be seen”

#aio servers used can be monitored via “ps –k | grep aio | wc –l” , “iostat –A” or nmon (option A)



IO : Asynchronous IO (AIO) fastpath

• FS with CIO/DIO and AIX 5.3 TL5+ :� Activate fsfast_path (comparable to fast_path but for FS + CIO/DIO)

AIX 5L : adding the following line in /etc/inittab: aioo:2:once:aioo –o fsfast_path=1

AIX 6.1 : ioo –p –o aio_fsfastpath=1 (default setting)

• Raw Devices / ASM :� check AIO configuration with : lsattr –El aio0

enable asynchronous IO fast_path. : AIX 5L : chdev -a fastpath=enable -l aio0 (default since AIX 5.3)AIX 6.1 : ioo –p –o aio_fastpath=1 (default setting)

Application Disk

1

2

3

With fast_path, IO are queued directly from the application into the LVM layer without any “aioservers kproc” operation.

� Better performance compare to non-fast_path� No need to tune the min and max aioservers� No ioservers proc. => “ps –k | grep aio | wc –l” is not relevent, use “iostat –A” instead

AIX Kernel



� How to set filesystemio_options parameter

Possible values

ASYNCH : enables asynchronous I/O on file system files (default)DIRECTIO : enables direct I/O on file system files (disables AIO)SETALL SETALL SETALL SETALL : enables both asynchronous and direct I/O on file system filesNONE : disables both asynchronous and direct I/O on file system files

Since the version 10g, Oracle will open data files located on the JFS2 file system

with the O_CIO option if the filesystemio_options initialization parameter is set to

either directIO or setall.

Advice : set this parameter to ‘setall’…

Note : set the disk_asynch_io parameter to ‘true’ as well

IO : AIO,DIO/CIO & Oracle Parameters



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



CPU : Simultaneous Multi-Threading (SMT)

P6 Core

Registerthread1

Registerthread2

• Feature available on POWER5 and POWER6 system running AIX

5.3 or 6.1

• allows 2 instruction path to share access to the POWER execution units on every clock cycle

• instruction path = “virtual processor” = logical POWER processor (i.e. a server with 4 physical processors will have 8 logical processors with SMT enabled)

• SMT allows to use resources that would otherwise usually be

unexploited and can achieve significant throughput gains – up to

40% greater performance

• No source code modification needed to use SMT

How to control SMT:

� smtcl [–m on | off]

� “-m on” enables the SMT mode, “-m off” disables it

� “-w boot” makes SMT change effective on reboot

� “-w now” makes SMT change effective immediately but not persistent across reboots

thread2thread1



CPU: Micro-Partitioning

An example given based on a PeopleSoft/Oracle10g RAC benchmark (2,400 Virtual Users) ran at

PSSC MOP (DLPAR performance vs. SPLPAR one)

SPLPAR ConfigurationTotal Throughput (bytes):2,715,373,065Action_Transaction: 53.256 secs

DLPAR ConfigurationTotal Throughput (bytes):2,037,677,193Action_Transaction: 59.07 secs

Micro-Partitioning give the possibility to share processors between several LPARs. It improve CPU utilisation by

reducing the number of idle cycle. (available on POWER5 and POWER6 system running AIX 5.3+)



CPU: Compatibility Matrix

DL

PA

R

SM

T

Mic

ro-

Pa

rtitio

n

AIX 5.2

AIX 5.3

AIX 6.1

Oracle 9i

Oracle 10g

Oracle 11g



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



Monitoring & Tuning : Methodology

� tuning is an iterative process

� a minimun of changes should be performed at each iteration

� tuning is a joint effort between the system administrator and the DBA

� don’t hesitate to keep a log to track changes and the impact of these changes

OS monitoring

Oracle monitoring

design/parameters changes

DBA System administrator

impact ?

log of changes



Monitoring & Tuning (cont’d) : System Monitoring Tools

� vmstat – useful for obtaining an overall picture of CPU, paging, and memory usage

� lvmstat – useful to get logical volume I/O statistics

� iostat – allows to get statistics on disk activity (and also on terminals, CPU, adapters)

� lparstat – reports logical partition related information and statistics

� nmon – complete tool which gives information on all the components of the system

� filemon – uses the trace facility to report on the I/O activity of physical volumes, logical volumes, individual files, and the Virtual Memory Manager

� xmperf – allows to define monitoring environments to supervise the performance of the local and remote systems

� netstat – allows to monitor network activity

....



Monitoring & Tuning (cont’d) : Oracle Monitoring Tools

� AWR is a standard toolwhich allow the collection, automation, storage andviewing of performance data. Replaces utlbstat/utlestat and Statspack.

� Grid Control (10g) provide a graphical, web-based, centralized management of databases.

�Tkprof allows to take and exploit SQL traces, gives explain plans etc…

�Toad, SpotLight …graphical tools (Quest)



IO

storage monitoring tool

vmstat (wa)

iostat, nmon,filemon

accesses well spread? hot spots?

jfs mount options?

AIO?

Monitoring & Tuning (cont’d) : Methodology SummaryPerformance issue?

Monitor the system, diagnose the problem.Tools to use, questions to ask…

Metalink note 282036.1

alert.log

bdump,cdump,udump…

init.ora/spfile

AWR

Grid Control

strategy of analyze?

memory allocated to instances (SGA, PGA)

tkprof (parsing, explain plans, bind variables…)

( and check ulimits + AIXTHREAD_SCOPE=S...)

CPU

topas, nmon

vmstat (us, sys, id, r)

sar –P ALL

CPU bound?

runqueue?

workload ?

parallelization of workload?

MEMORY

topas, nmon

vmstat (pi, po)

pagingout activity?

memory allocated to Oracle?

other applications running?

other Oracle

instances?

Network

Topas, nmon

netstat,netpmon



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



Automatic Workload Repository (AWR)

… is the tool to use to collect information about the behavior of a database!

• Collects, automates, stores performance statistics for problem determination.

• AWR is setuped during the creation of the database is automaticgenerates reports in html formatreports more information than Statspack

�prefer AWR to statspack reports!

• Snapshots (set of statistics gathered at a single time) are taken automatically each 60 minutes or can be run manually:

execute dbms_workload_repository.create_snapshot();

• AWR reports are generated by running $ORACLE_HOME/rdbms/admin/awrrpt.sql and providing 2 snapshot Ids. Can be also generated using the Grid Control.

• To enable most of the new statistical gathering and advisors, ensure that the parameterSTATISTICS_LEVEL is set to TYPICAL (recommended).



Automatic Workload Repository (cont’d)

• Compare reports to mesure the impact of tuning and to check the behavior of the database

•Use Metalink and/or the Oracle documentation to get the definition of events, statistics etc…

• Advised to run manually additional snapshots during performance issue phases / optimization campaign

• No shutdown between two snapshots!

• Choose the most representative snapshots IDs to generate the report.

snapshot

period analyzed in AWR report



Performance tuning advisors

Automatic Database Diagnostic Monitor (ADDM)

• Based on AWR, invoked automatically every time that a new AWR snapshot is generated

• Powerful self-diagnostic engine which :

� analyze the system

� indentify major problems

� recommend corrective actions!!!

• ADDM reports are generated by running $ORACLE_HOME/rdbms/admin/addmrpt.sql and providing 2 snapshot Ids – Can also be generated using the Grid Control.

• In 11g, for RAC clusters, ADDM analyze the system at the cluster level.

SQL Tuning Advisor (STA)

• Detects missing or stale statistics, missing indexes…

• Suggests new execution plan which can be applied to the system and used without any change in the application

• Advised to use STA through Grid Control – but also possible to invoke it using DBMS_SQLTUNE package.

NEW!11g



AWR Report : Header & Hit Ratios

…



AWR Report : Top 5 timed events

This “Top 5 Timed Events” table is very important – it lists the events to focus on and will lead the performance analysis.

According the event listed in the table, refer to the corresponding section in the report.



AWR Report : SQL ordered by…

Sections « SQL statement ordered by… »present SQL statements ordered by differentscriterias.

They are useful to detect a problem of parsing, a response time of a particular statement etc…

SQL is clickable and the whole statement ispresented later in the report.



AWR Report : Cache sizes and advisors

Buffer Pool Advisory help to tune the bufferpool, givingestimation of the impact of changes of the size of the buffer pool.

Advisors exist for:- PGA memory - Shared Pool - Sga_target - …



Oracle Grid Control Targets : databases



Oracle Grid Control Focus on one database



Oracle Grid ControlPerformance graphs



Oracle Grid Control Advisor Central



Oracle Grid Control ADDM (Automatic Database Diagnostic Monitor)

Focus needed on

these statements!



Oracle Grid Control ADDM (Automatic Database Diagnostic Monitor)

Statement with most impact on performances

Run the advisor to get a tuning hint!



Oracle Grid Control SQL Tuning Advisor

You can implement the advice (or not!)



Oracle Grid Control Focus on one query



Oracle Grid Control

• Grid is the most powerful tool to identify issues, get advices, tune your database

• It facilitates DB administration tasks and allows to perform the maintenance of the database (backups etc…)

• Targets monitored by the Grid are the hosts, databases, OracleApplications Servers, Oracle eBusiness Suite etc…

• Allows to have a central point for the management of the targets

• Helps the DBA, but does not replace him/her! ;-)



Agenda


�Memory

� AIX VMM tuning


� IO



� Oracle ASM

� Asynchronous IO

�CPU




�Conclusion



• Implement and get the best of Oracle on AIX is the result of a joint effort between the OS administrator and the Oracle DBA.

• The tuning process is made of a first guess parameter setting and then, iterations to change progressively the configuration to get the best results…

• Prefer JFS2 + Concurrent IO or ASM.

• Always use Asynchronous IO servers and, if available, use FastPath.

• Implement SMT if available on your system.

• Use AIX, Oracle and storage monitoring tools (AIX : Nmon, Topas – Oracle : AWR & Grid Control – Storage : TPC…)

• … and feel free to contact us for additional questions!!!

[email protected]

Conclusion



Addendum - Useful note for Oracle running on AIX platforms

•Before installing an Oracle product on AIX, it is strongly advised to have a look at the

Metalink note 282036.1 : Minimum Software Versions and Patches Required to Support Oracle Products on IBM pSeries

It contains all mandatory AIX PTFs & APARs and Oracle patches to install on a system is updated on a regular basis and



Addendum - Certification matrix : Oracle database Enterprise Edition on IBM AIX based Systems

When upgrading to Oracle 10GR2 on AIX 6.1, download and run an updated version of rootpre script before proceeding with the Oracle Database install (refer to bug 6613550 for more details).

10th of april 2008



Oracle/AIX Performance and Tuning

Session ID : pS014

[email protected]@fr.ibm.com



Reference Materials – DRAFT -

Thanks to the authors of the following documents :

• Improving Database Performance with AIX Concurrent I/O – S. Kashyap, B. Olszewski, R Hendrickson

• Performance Management Guide – AIX5L version 5.3 documentation

• Oracle 9i & 10g on IBM AIX5L : Tips & Considerations – IBM/Oracle ICC

• Oracle Architecture and Performance Tuning on AIX – R. Ballough

• Configuring IBM TotalStorage for Oracle OLTP Applications – D. Martin

• Tuning IBM AIX5L for an Oracle Database – D. Massanari, M. Riviere



EMEA Oracle/IBM Joint Solutions Center

C/O Compagnie IBM France

Rue de la Vieille Poste

34006 Montpellier Cedex - FRANCE

[email protected] : +33 4 67 34 67 49



TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

The following are trademarks or registered trademarks of other companies.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes:

Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.

All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.

This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA, WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not

actively marketed or is not significant within its relevant market.

Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.

oracle/aix performance and tuning...oracle server architecture dbwr system global area buffer cache...

Documents