working with oracle automatic pga

8/11/2019 Working With Oracle Automatic PGA

1/53

Distance checker

Make sure you see these

select a value which is visible from last rows at 18

This is a positive highlightedvalue

This is a negative highlightedvalue

select a value which is visible from last rows at 14

This is a positive highlightedvalue

This is a negative highlightedvalue


2/53

Working with Automatic PGA

All graphs are taken from the document

SQL Memory Management in Oracle9i from

Benot Dageville and Mohamed Zait from Oracle

Corporation

Christo Kutrovsky

The Pythian Group

2005 Feb


3/53

What is PGA ?

Program Global Area

i.e. Process Global Area

Why is PGA different from SGA

What goes into PGA

Sort

Hash

PL/SQL variables and cursor memory


4/53

SGA

Oracle memory model

P1

P2

P3

P4

PGASORT

HASH

PGABIG

SORT

cursors

pl/sql var.

cursors

pl/sql var.

PGA Sortcursors

pl/sql var.

PGA Hashcursorspl/sql var.


5/53

Virtual memory mapping

P1

P2

SGA

32/64 bit addressing space

PGASort

cursorspl/sql var. PGA Hashcursorspl/sql var.

0 gb 1 gb 2 gb 3 gb 4 gb


6/53

The need for more memory

To make things go faster !

The fast = true parameter

Not really

More memory has different effects on

sorts and hash joins


7/53

Effects on Sorting

Sorting has ONLY 3 modes

optimal or cache (all in memory)

1 pass (the entire result set is written once todisk)

multi-pass (the entire result set is written

multiple times to disk)


8/53

Sorting response time

Response

time

Memory used

Optimal

1 pass


9/53

Crossing optimal => 1 passSQL> select count(*) from (select * from TBLSESSION t where rownum select count(*) from (select * from TBLSESSION t where rownum select count(*) from (select * from TBLSESSION t where rownum select count(*) from (select * from TBLSESSION t where rownum


10/53

Crossing optimal => 1 passselect substr(sql_text,57,15) as sql, operation_type as op, operation_id as id, policy,

round(estimated_optimal_size/1024/1024,2) as e_opt, round(estimated_onepass_size/1024/1024,2) as e_one,

round(last_memory_used/1024/1024,2) as l_mem, last_execution as last,

total_executions as tot, optimal_executions as opt, onepass_executions as one, multipasses_executions as mult,

round(active_time/1000000,2) as sec, round(max_tempseg_size/1024/1024,2) as tmp_m, round(last_tempseg_size/1024/1024,2) as tmp_L

from v$sql_workarea swa, v$sql sq

where swa.address = sq.address and swa.hash_value = sq.hash_value

and sql_text like 'select count(*) from ( select * from TBLSESSION%order by sql;

SQL OP ID E_OPT E_ONE L_MEM LAST TOT OPT ONE MULT SEC TMP_M TMP_L--------- ---- -- ----- ----- ------ ------- --- --- --- ---- ------ ----- -----


11/53

Other nuances of SORTs

When sorting, you are sorting the ENTIRE

result set. Thus you need as much memory

as ALL your select columns So yes, by adding an extra field (even a

fixed string) you could just pass the limit

and switch to one-pass sort select * is even worse


12/53


13/53

Switching to multi-pass

Multipass is a long way down the road,

compared to optimal => 1 pass

TYPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP

---- ------ --- ------ ------ ----- ------ ------ ------- ----

SORT AUTO 299 813.77 77.9877.98 78.01 87.27 1 6750


14/53

Top N queries

Remember top N queries?

SQL> select count(*) from

(select * from TBLSESSION twhere rownum


15/53

Effect on HASH joins

Hash joins have the same 3 modes: optimal

(cache, in memory), 1 pass and multi-pass

Hash joins benefit from additional memorybetween optimal and one-pass execution

That is, according to Oracle, in my testing

there was no measurable benefit


16/53


17/53

Nuances HASH joins

If the optimizer estimates it will go above

the bound, it will use temp right away

The estimated value is re-learned from thelast execution

Thus a 2ndexecution, without any changes,

may not use temp

If the estimate is bypassed, then it will

write to temp as needed, same way as a sort


18/53

Automatic PGA

Now to the automatic word


19/53


20/53


21/53

Global memory manager

Updates the global memory bound every

3 seconds

Reacts to over allocation by lowering thebound further


22/53

Some facts

Oracles SQL operators can adjust the memory

consumption in mid-execution

Sorts that switch from optimal to 1-pass willrelease the extra memory immediately

(depending on disk speed)

Background jobs processes will not free-up the

virtual memory, if sufficient memory is available


23/53


24/53

Drift

A drift of 10% is allowed

When the limit is passed, a foreground

bound recompute occurs


25/53


26/53

Monitoring PGA activity

Overview

Details

Advisors


27/53

Monitoring - OverviewSQL> select * from v$pgastat;

NAME VALUE UNIT

-------------------------------------- ---------- --------

aggregate PGA target parameter 2097152000 bytes

aggregate PGA auto target 1866138624 bytes

global memory bound 104857600 bytestotal PGA inuse 23664640 bytes

total PGA allocated 132822016 bytes

maximum PGA allocated 564828160 bytes

total freeable PGA memory 4194304 bytes

PGA memory freed back to OS 2165370847 bytes

total PGA used for auto workareas 0 bytesmaximum PGA used for auto workareas 111509504 bytes

total PGA used for manual workareas 0 bytes

maximum PGA used for manual workareas 529408 bytes

over allocation count 0

bytes processed 5318923366 bytes

extra bytes read/written 2912501760 bytescache hit percentage 99.45 percent


28/53

Monitoring - Overviewno bytesselect name,round(value/1024/1024,1) as Mb from v$pgastat where unit = 'bytes';

NAME MB

-------------------------------------- ---------

aggregate PGA target parameter 2000

aggregate PGA auto target 1779.7

global memory bound 100

total PGA inuse 22.5

total PGA allocated 126.7

maximum PGA allocated 538.7

total freeable PGA memory 4

PGA memory freed back to OS 2065058.6total PGA used for auto workareas 0

maximum PGA used for auto workareas 106.3

total PGA used for manual workareas 0

maximum PGA used for manual workareas 0.5

bytes processed 507256.6extra bytes read/written 2777.6


29/53

Monitoringgetting the details 1

When the query is running

v$sql_workarea_activeselect operation_type as type, policy, sid, round(active_time/1000000,2) as a_sec,

round(work_area_size/1024/1024,2) as wsize, round(expected_size/1024/1024,2) as exp,

round(actual_mem_used/1024/1024,2) as act,round(max_mem_used/1024/1024,2) as max,

number_passes as passes, round(tempseg_size/1024/1024,2) as temp from v$sql_workarea_active;

TYPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP

---- ------ --- ----- ----- ----- ----- ----- ------ ----SORT AUTO 299 284.5 54.73 54.73 53.75 87.27 1 3015


30/53


When it has finished (or for another query)

v$sql_workareaselect operation_type as op, operation_id as id, policy,


round(last_memory_used/1024/1024,2) as l_mem, last_execution as last,total_executions as tot, optimal_executions as opt, onepass_executions as one, multipasses_executions as mult,


from v$sql_workarea where address ='0000000381E23CF0'and hash_value ='1505362365';

OP IDPOLICY E_OPT E_ONE L_MEM LAST TOT OPT ONE MULT SEC TMP_M TMP_L

------------------ ----- ----- ----- ------- ---- ---- --- ---- ---- ----- -----

SORT 1AUTO 0.02 0.02 0.02 OPTIMAL 227 227 0 0 0.02

HASH-JOIN 2AUTO 0.63 0.63 0.34 OPTIMAL 227 227 0 0 0.02HASH-JOIN 9AUTO 0.88 0.88 1.14 OPTIMAL 146 146 0 0 0.01


31/53


You can link workareas to steps from the EX.PLANv$sql_plan

select rpad(' ', depth*3)||operation||' '||options||nvl2(object_name, ' -> ','')||object_name||decode(search_columns,0,NULL,'('||search_columns||')') as OP,cost, cardinality as CARD, bytes, id as "id",access_predicates as "ACCESS", filter_predicates asfilter,round(temp_space/1024/1024) as TMP_MB,partition_start ||nvl2(partition_start, ' - ', '')||partition_stop as P, partition_id,other, other_tag, cpu_cost, io_cost, distribution, object_owner, parent_id,optimizer from ( select * from V$SQL_PLAN where address= hextoraw('0000000381E23CF0') and hash_value = '1505362365'and child_number = 0) t connect by prior id = parent_id start with id

= 0 order by id, position;OP id ACCESS TMP_MB

---------------------------------------------------- -- --------- ------

SELECT STATEMENT 0

SORT ORDER BY 1

HASH JOIN 2"VID"="VID"

NESTED LOOPS 3

TABLE ACCESS INDEX ROWID -> TAG_SYM 4INDEX RANGE SCAN -> TS_VSID (1) 5 "VSID"=:1

TABLE ACCESS BY INDEX ROWID -> VEN_TAG 6

INDEX UNIQUE SCAN -> VT_TAG_PK (1) 7 "TAG"="TAG"

VIEW 8

HASH JOIN 9ROWID=ROWID

INDEX FAST FULL SCAN -> VI_NAME_PK 10INDEX FAST FULL SCAN -> VI_VID_UK 11


32/53


Summariesoverall per workarea sizev$sql_workarea_histogram

SELECT LOW_OPTIMAL_SIZE/1024/1024 low_mb,(HIGH_OPTIMAL_SIZE+1)/1024/1024 high_mb,optimal_executions,onepass_executions,multipasses_executions

FROM v$sql_workarea_histogram

WHERE total_executions != 0

and (low_optimal_size/1024/1024 >= 8 or total_executions > optimal_executions);

LOW_MB HIGH_MB OPTIMAL_EXECUTIONS ONEPASS_EXECUTIONS MULTIPASSES_EXECUTIONS

------ ------- ------------------ ------------------ ----------------------

2 4 3073 0 0

4 8 2033 0 0

8 16 402 0 016 32 1501 55 0

32 64 147 2295 0

64 128 0 374 0

128 256 0 28 0

256 512 0 3 0

512 1024 0 16 0

1024 2048 0 1 0


33/53


Summariesfinding the bad onesselect operation_type as op, operation_id as id, policy,


round(last_memory_used/1024/1024,2) as l_mem, last_execution as last,

total_executions as tot, optimal_executions as opt, onepass_executions as one, multipasses_executions as mult,


from v sql_workarea where max_tempseg_size is not null;

OP ID POLICY E_OPT E_ONE L_MEM LAST TOT OPT ONE MULT SEC TMP_M TMP_L

--------- -- ------- ----- ----- ----- ------ --- --- --- ---- ----- ----- -----

HASH-JOIN 4 AUTO 47.91 4.09 36.031 PASS158 0158 0 90.99 45 40


34/53

The Advisorsbeen really lazy

Similar to v$db_cache_advice

v$pga_target_advice

Shows estimate temp IO usage v$pga_target_advice_histogram can be

used to estimate opt/1pass/multi

executions for different workarea sizes


35/53

v$pga_target_adviceselect round(pga_target_for_estimate/1024/1024) as est_mb,pga_target_factor as factor,

round(bytes_processed/1024/1024) as p_mb,round(estd_extra_bytes_rw/1024/1024) as extra_mb,

estd_pga_cache_hit_percentage as hit_ratio,estd_overalloc_count as est_over from v$pga_target_advice;

EST_MB FACTOR P_MB EXTRA_MB HIT_RATIO EST_OVER

------ ------ ------ ---------- ---------- ----------

125 0.125 446362 768675 37 9797250 0.25 446362 293602 60 2075

500 0.5 446362 181392 71 0

750 0.75 446362 176313 72 0

1000 1 446362 136979 77 0

1200 1.2 446362 136391 77 01400 1.4 446362 136391 77 0

1600 1.6 446362 136391 77 0

1800 1.8 446362 136391 77 0

2000 2 446362 136391 77 0


36/53


37/53

To make it work

And now what you need to make it work


38/53

Documented parameters

pga_aggregate_target

thats it


39/53


40/53

Patches/Bugs that are required

2805580 - corruption with auto PGA,

9.2.0.4

3130972 - allow more then 1gb perworkarea 9.2.0.5

3586008 - the above 2 (as they conflict)

Check with Oracle what is required foryour patch level


41/53

Maximum workarea size

The maximum PGA workarea is hard

limited to 5%of pga_aggregate_target or

a max of 100Mb Can be controlled with

_smm_max_size

Value is in KB !


42/53

Maximum PGA per process

One process can have manyworkareas

Max total size can be controlled with

_pga_max_size

Value is in bytes, default 200Mb

CAUTION: virtual address space

limitation on 32 bit platforms


43/53

So whats possible?

Put really BIG sorts in memory !

A singleworkarea is still limited at 4gb

In addition due to steps-allocations, its

spilling much earlier, around 3.5 gb

A process however is notlimited to 4gb,

at least on 64 bitsystem


44/53

The system

Solaris with 64 bit Oracle

9.2.0.5

32 Gb RAM

16 Gb SGA

pga_aggregate_target = 8000m

_pga_max_size = 8000000000

_smm_max_size = 5000000


45/53

MonitoringThe adjusted systemSQL> select name,round(value/1024/1024,1) as Mb from v$pgastat where unit = 'bytes';

NAME MB

--------------------------------------- ----------

aggregate PGA target parameter 8000

aggregate PGA auto target 7139.4

global memory bound 4882.8

total PGA inuse 67.4

total PGA allocated 2940.5

maximum PGA allocated 9344.4

total freeable PGA memory 2783.5

PGA memory freed back to OS 2590908.6

total PGA used for auto workareas 0

maximum PGA used for auto workareas 6903.9

total PGA used for manual workareas 0

maximum PGA used for manual workareas 0.5

bytes processed 13542884

extra bytes read/written 1034611.9


46/53

A big sort 1 - running

select operation_type, policy, sid, round(active_time/1000000,2) as a_sec,

round(work_area_size/1024/1024,2) as wsize, round(expected_size/1024/1024,2) as exp, round(actual_mem_used/1024/1024,2) as act,

round(max_mem_used/1024/1024,2) as max, number_passes as passes, round(tempseg_size/1024/1024,2) as temp

from v$sql_workarea_active;

OPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP--- ------ ---- ------- -------- -------- -------- -------- ------- -----

SOR AUTO 26 148.66 3589.75 3589.75 3108.88 3108.88 0


47/53

A big sort 2 - spilling





OPER POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP---- ------ --- ------ ------- ----- ------- ------- ------ ----

SORT AUTO 26 246.38 3589.7546.413589.75 3589.75 01100


48/53

A big sort 3 - spilled (first fetch)





OPER POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP---- ------ --- ------ ----- ---- ---- ------- ------ ----

SORT AUTO 26 378.28 3.283.28 3.52 3589.75 1 2800


49/53

Is it seriously using 3.5gb of ram?

Oh yes, even more !select round(pga_alloc_mem/1024/1024,1) as alloc_mb, round(pga_used_mem/1024/1024,1) as used_mb, round(pga_max_mem/1024/1024,1) as

max_mb, pid, spid, serial#, program, traceid, background, pga_freeable_mem

from v$process order by alloc_mb desc

ALLOC_MB USED_MB MAX_MB SPID PROGRAM PGA_FREEABLE_MEM-------- ------- ------ ------ ------------------------ ------

4787.8 4780.14787.8 16291 oracle@test (TNS V1-V3) 0

10.9 0.2 10.9 4320 oracle@test (DBW0) 0

5.3 5.1 5.3 19355 oracle@test (TNS V1-V3) 0

5.3 5.1 5.3 19318 oracle@test (TNS V1-V3) 04.2 4.1 4.2 19312 oracle@test (NSV0) 0

1.4 1.3 1.4 16251 oracle@test (TNS V1-V3) 65536

1 0.4 1 16241 oracle@test (TNS V1-V3) 524288


50/53

Checking doubts on the OS side

I dont believe that, thats just oracle telling us/home/oracle>pmap -xs 16291|head

16291: oracleprod (LOCAL=NO)

Address Kbytes RSS Anon Locked Pgsz Mode Mapped File

0000000100000000 2272 2272 - - 8K r-x-- oracle

0000000100238000 104 96 - - - r-x-- oracle

/home/oracle>pmap -xs 16291|tail

7FFFFFFF7FE00000 176 176 - - 8K r-x-- ld.so.1

7FFFFFFF7FF2C000 16 16 16 - 8K rwx-- ld.so.1

FFFFFFFF7FFE0000 128 128 128 - 8K rw--- [ stack ]

---------------- ---------- ---------- -------- --------

total Kb 21931752 21904656 490701616961536 The above values are in Kb. ~16 gb sga + ~5 gbanonymous

memory, which is PGA = total 21 gbresident in use


51/53

Not used doesnt mean returned

select round(pga_alloc_mem/1024/1024,1) as alloc_mb, round(pga_used_mem/1024/1024,1) as used_mb,round(pga_max_mem/1024/1024,1) as max_mb, pid, spid, username, serial#, program,round(pga_freeable_mem/1024/1024,1) as freeable_mb

from v$process order by alloc_mb desc;

ALLOC_MB USED_MB MAX_MB PID SPID USERNAME SERIAL# PROGRAM FREEABLE_MB-------- ------- ------ --- ----- -------- ------- ------------- -----------

4100 3.74787.8 25 16291 oracle 2 (TNS V1-V3) 4096

10.9 0.2 10.9 3 4320 oracle 1 (DBW0) 0

5.3 5.1 5.3 18 19355 oracle 10 (TNS V1-V3) 0


52/53

Wrap-upwhat can be achieved

Use morememory for sorting/hashing

Yet controlled automaticallyto

avoid swapping Resulting in improvedresponse time and

less disk IO

Resulting in more IOcapacityavailablefor other tasks

Basically fast = true


53/53

The End

Thank you,

Questions?

[email protected]

Christo Kutrovsky

The Pythian Group
mailto:[email protected]:[email protected]

working with oracle automatic pga

Documents