ugif 10 2012 ppt0000001

User Group Informix France

Partition Defragmenter

Yoram Benchetrityoram.benchetrit@fr.ibm.com

Mercredi 3 Octobre 2012Mercredi 3 Octobre 2012

User Group Informix France

AgendaAgenda

�A bit of partition background�The problem with fragmented data�How does the feature work�Functionality & considerations�Defragmenter in OAT�Example

User Group Informix France

A bit of partition background�What is an extent ?

– Collection of physically contiguous pages on disk

– Extent can hold data page, index page, bitmap page etc…

�What is a partition ?– Logical collection of all the extents allocated for a

given table or index.

– Space for partition is allocated in units of extents and is not necessarily contiguous.

�Each partition are unique within the system– oncheck –pt <partnum>

User Group Informix France

A bit of partition background

Partition logical to physical extent relationship

User Group Informix France

A bit of partition background

Extents for table A

Extents for

other tables

Chunk 14

Free pages

�Before Defragmenting

User Group Informix France

�After Defragmenting

Chunk 14

Extents for table A

A bit of partition background

User Group Informix France

A bit of partition background

7

�Previous Solutions– Table unload and reload

�Oncheck –me– Off-line table reorganization– Non-recoverable

– Unsupported

User Group Informix France

A bit of partition background

�How many extents exist ?– Run oncheck –pe.– Run oncheck –pt.

– SQLdatabase sysmaster; select unique te_partnum, count(*) from systabextents group by 1 having count(*) > NN order by 1 desc;

• Where NN is a value.

User Group Informix France

The problem with fragmented data

�What’s the problem with many extents? – Full partition page.

• Prohibits new inserts.– Might not be at maximum number of rows.

• Need to either delete older entries, if possible.– or

• Requires a manual table recreation.

– (FYI: Actually, this can no longer exist in 11.70)

User Group Informix France

�Many extents degrades performance.– Need to skip over non-contiguous disk areas.– Slower retrieval of data.

– Mapping logical page numbers to physical addresses is slower.

– Chunk allocations are slower with long free list pages

– Archive extent-sorting phase is slower

The problem with fragmented data

User Group Informix France

�What’s the problem with many extents? – Many tables with many extents feels

overbearing.• Outputs from oncheck –pe & -pt.

The problem with fragmented data

User Group Informix France

How does the feature work

�Run new SQL API functions.– sysadmin database.

– execute function [admin|task] ("<defragment","<database>:[<owner>.]<table>[#index]")

• or

– execute function [admin|task] ("<defragment partnum", "<partnum> [, <partnum>, ... ]”);

User Group Informix France

How does the feature work

– What goes on behind the scenes.• Check validity.

• Create new extent.• Might need just one adding

• Start a loop and process all old extents.– Read old extent.– Move all old extent pages to the new extent.– Delete old extent.

• Loop end.

User Group Informix France

How does the feature work

�Log the work using five new log types.– Start defragmentation.

– End defragmentation.– Add new defragmentation extent.

– Remove old defragmentation extent.– Move page.

�Will result in a surge in logical log usage.

User Group Informix France

How does the feature work

�Monitoring progress.– onstat -g defragment.

• Searching for extents.• Merging extents.• Defrag_completed.• Defrag_failed.

User Group Informix France

How does the feature work

�Recoverable if server shuts down or crash

– Defragmenter cleaner thread will remove extra extents after fast recovery

• Rolls back current unit of work (UOW) if needed.– 08:52:06 Defragmenter cleaner thread now running– 08:52:06 Defragmenter cleaner thread cleaned:1 partitions

– Will the object be consistent ?• Yes.

– Choose to run again to complete the job.

User Group Informix France

�Can defragment table online�No locks placed on object�Only allowed one defragmenter running per

dbspace�Replication is supported (HDR, MACH11, ER)�Run via the SQL API interface.

Functionality

User Group Informix France

�Functionality not allowed – Functional indexes.

– VTI & VII objects.– Temporary tables.

– Sort tables.– Optical blobs.

– System catalog tables

Considerations

User Group Informix France

�Cannot stop once job submitted�Cannot run simultaneously when table being

– Altered– Truncated

– Drop– Compress, Repack, Shrink

�Not the same as compress/repack/shrink

Considerations

�Enterprise Replication.– Table needs to be in CDR alter mode.

• Mode unset automatically at end of job

User Group Informix France

�Operation is logging-intensive– Each page move is logically logged (DFMVPG)

• 70-80 bytes per page moved

�Small units of work to avoid long transaction.– It may send other transactions over the threshold

�New extents created at end of dbspace.�Schedule during low activity.�Perform after a backup.

Considerations

User Group Informix France

Defragmenter in OAT

21

On the OAT menu, expand

Space Administration > Space Optimization

User Group Informix France

Defragmenter in OAT

22

Select the tables.

On the Actions menu, click Optimize Space.

User Group Informix France

Defragmenter in OAT

23

Select Defragment extents and click OK.

User Group Informix France

�Automatic Storage Optimization– Can automate defragmenting using OAT.

– Enabled via scheduler task “auto_crsd”• This can also do compressing, repacking, shrinking.• ‘crsd’ stand for compress, repack, shrink, defragment

– This can also be enabled via SQL

Defragmenter in OAT

User Group Informix France

�‘auto_crsd’ task details– In OAT -> Task Scheduler -> Task Details

• UPDATE ph_task SET tk_enable = 'T‘ WHERE tk_name = 'auto_crsd';

Defragmenter in OAT

User Group Informix France

�‘auto_crsd’ task parameters.

– UPDATE ph_threshold SET value = 'F WHERE name = 'AUTODEFRAG_ENABLED';

– UPDATE ph_threshold SET value = ‘100‘ WHERE name = 'AUTODEFRAG_EXTENTS';

Defragmenter in OAT

User Group Informix France

Example

� SQL to create three fragmented tables– create table tab1 (a serial, b int, c int);– create table tab2 (a serial, b int, c int);– create table tab3 (a serial, b int, c int);– create procedure ins (a int, b int) define c int;

• for c = a to b• insert into tab1 values (0, 88, 99);• insert into tab2 values (0, 88, 99);• insert into tab3 values (0, 88, 99);• end for;

– end procedure;– execute procedure ins (1, 1000000);

User Group Informix France

Example

�oncheck –pe before defragment.

User Group Informix France

Example

� oncheck –pt before defragment.– FYI : in 11.70 No longer limited to one partn page.

User Group Informix France

Example

�SQL API function.– execute function task ("defragment", "oltr:tab1");

(expression) OK

User Group Informix France

Example

�oncheck –pe after defragment.

User Group Informix France

Example

�oncheck –pt after defragment.

User Group Informix France

Example

�onlog output

User Group Informix France

SummarySummary

�Informix Defragmenter Capabilities

– Partition Extents Defragmentation– New System Admin SQL API

– OAT GUI Interface– Automatic Defragmentation

– Can be completed with compress/repack/shrink for optimal storage optimization

User Group Informix France

Questions ?

User Group Informix France

MerciMerci

Yoram Benchetrityoram.benchetrit@fr.ibm.com

Mercredi 3 Octobre 2012Mercredi 3 Octobre 2012