db2 11 utilities update - ibm · pdf filedb2 11 utilities update haakon roberts ... load from...
TRANSCRIPT
© 2013 IBM Corporation
Information Management for System z
1
DB2 11 Utilities Update
Haakon RobertsDE, DB2 for z/OS & Tools Development [email protected]
© 2013 IBM Corporation2
Disclaimer
– Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
© 2013 IBM Corporation3
Agenda
Strategy Overview REORG Statistics Backup & Recovery UNLOAD & LOAD Compression Dictionaries General Enhancements Deprecation Questions & Feedback
© 2013 IBM Corporation4
Strategy
Support core function
Reduce CPU, ET & resource consumption
Maximize availability
Remove constraints & limitations
Simplify data management
© 2013 IBM Corporation5
Overview Better availability
– Outage reduction for online REORG through improved drain processing & SWITCH phase elapsed time reduction
– Better control of SWITCH phase timing– Online rebalance of data across partitions– Faster LOAD processing
Better performance & reduced resource consumption– Reduce/eliminate need for RUNSTATS through inline stats– Faster LOAD processing– Faster recovery from part-level inline image copies– Faster REORG processing– More utility parallelism & greater parallelism control– More zIIP exploitation– Optimizer input to RUNSTATS
© 2013 IBM Corporation6
Overview
Simplified data management & improved usability– Automated REORG mapping table management– Intelligent default settings for REORG– Better PBG management through deletion of unused partitions – Improved dataset management with part-level inline image copies– Improved XML handling with Crossloader support– System cloning improvements– Lift point in time recovery restrictions– More information & greater transparency through –DISPLAY UTILITY
improvements– Enhanced statistics profile support
© 2013 IBM Corporation7
Key performance numbers
CPU improvement for utilities with EXCLUDE NULL KEYS– LOAD 12%– REORG 25%– REORG INDEX 88%– REBUILD INDEX 72%– CHECK INDEX 79%– RUNSTATS 90%
Increased parallelism elapsed time improvement– REORG 21%
Inline stats vs. separate RUNSTATS– 40% elapsed time reduction for inline histogram stats– 28% elapsed time & 19% CPU reduction with inline distribution stats
© 2013 IBM Corporation8
Key performance numbers
Up to 81% zIIP-eligible CPU with RUNSTATS COLGROUP
Up to 40% zIIP-eligible CPU in REORG & LOAD with inline distribution stats
REORG SWITCH phase reduced by 90%
Up to 71% elapsed time reduction for REORG of subset of partitions– SORTNPSI option retrofitted to V9 & V10
RECOVER from part-level image copies reduced CPU by up to 50%, elapsed by up to 40%
LOAD from single input dataset elapsed time reduced by up to 70%
Crossloader support for FETCH CONTINUE for LOB & XML data– 28% CPU reduction
© 2013 IBM Corporation9
Improve performance of part-level REORG with NPSIs
New option to defer shadow index build until all keys passed through sort
New parm & zparm to govern– AUTO/ENABLE/DISABLE options
Retrofit to DB2 9 & 10 in PM55051
Result:– Customer test of REORG of 40% of partitions showed
55% ET reduction & 22% CPU increase– DB2 Sort gives additional ET reduction & cuts CPU to
less than original starting point
© 2013 IBM Corporation10
SWITCH phase impact relief – reduced application impact
Easier drain acquisition Prevent new claims on all target partitions whilst
waiting for drains– Faster drain acquisition for part-level REORG
New DRAIN_ALLPARTS option to momentarily drain all data parts– Eliminates claim-drain “deadlocks” for part-level
REORG with NPSIs Restructure SWITCH phase processing for outage
reduction– SWITCH phase ET reduction of 91% measured
when reorging 20 parts
© 2013 IBM Corporation11
Timing of SWITCH phase with MAXRO DEFER
Govern timing of drain and switch for long-running REORGs without the need to schedule separate –ALTER UTILITY command
New SWITCHTIME parameter to determine earliest point at which drain processing will be attempted
|-SWITCHTIME-NONE-------------------------------------------------|>> -|-------------------------------------------------------------------------------|
| | |
|-NEWMAXRO-NONE----| |-SWITCHTIME-|-timestamp--------------------------|------|---------------------------------| |-labeled-duration-expression-| |-NEWMAXRO-integer---|
© 2013 IBM Corporation12
Physically delete empty PBG partitions
Ability for REORG to physically delete empty PBG partitions
New zparm REORG_DROP_PBG_PARTS– DISABLE – keep V10 behaviour (default)– ENABLE – Delete empty PBG partitions on table space-level
REORG
Considerations:– Cannot be specified on REORG statement– If PBG created using NUMPARTS or ALTER ADD partition
used, REORG may prune to a lesser number of partitions– No PIT recovery to prior to a pruning REORG
• No facility to resurrect deleted partitions
© 2013 IBM Corporation13
Automated mapping table handling Scalability constraint with existing mapping table index limits # rows that can be
reorged
Mapping table DDL must change in 11 due to RBA/LRSN change
Requirements to automate mapping tables
Solution: New automated mapping tables in REORG
– Support mapping tables in PBGs
• Increases mapping index max size from 64Gb to 16Tb– Retrofitted to V9 in PM58177
– Automatically create new format mapping table if required
1. If mapping table specified & correct format then honour specification2. Else if specified but incorrect format then create new in same db as original3. Else if not specified and zparm DB specified then create in zparm DB4. Else create in implicit DB5. DROP at end of REORG or end of last REORG if multiple REORGs in job step
– NFM requires new format mapping table
• CM, CM*, ENFM, ENFM* supports old & new format mapping table
© 2013 IBM Corporation14
REORG without sorting data Increasingly REORGs are performed for reasons other than to regain
clustering of data, yet no ability to avoid cost of reclustering
REORG SHRLEVEL CHANGE does not support SORTDATA NO
Support SORTDATA NO with SHRLEVEL CHANGE
New RECLUSTER YES/NO option on SORTDATA NO– RECLUSTER NO – Do not unload data through clustering index and
do not sort data records in clustering order
© 2013 IBM Corporation15
Partition-level inline image copy Faster partition-level recovery from inline image copy Create partition-level inline image copies if using TEMPLATE with &PA or
&PART– No new option or keyword on REORG– PM93611:
• Support subscripting of &PA as long as subscript ensures uniqueness• Support writing to tape as long as STACK YES not specified
0
2
4
6
8
10
12
14
Elapsed CPU
Tablespaceimagecopy
Partitionimagecopy
RECOVER of single partition of a 20 partition table space
– ET reduced by 28%– CPU reduced by 49%
© 2013 IBM Corporation16
Improved REORG LISTDEF processing PARALLEL YES/NO option introduced in APAR in V9
– NO – Prevent REORG from processing multiple partitions in single REORG when input is partlevel LISTDEF
– Zparm REORG_LIST_PARALLEL at zparm level
Need compromise option for customers who want to take advantage of REORG parallelism but cannot afford to shadow many partitions at a time
New option LISTPARTS n to limit # of partitions to be processed in a single REORG if input is a part-level LISTDEF
Considerations:– PARALLEL YES/NO will be deprecated but still supported in 11
– PARALLEL NO = LISTPARTS 1
– PARALLEL YES = No LISTPARTS specification
– No REORG_LIST_PARALLEL n zparm option
© 2013 IBM Corporation17
REBALANCE enhancements
Improved availability & failure prevention
Support REORG SHRLEVEL CHANGE REBALANCE– Complements online ALTER LIMITKEY
Improve resiliency with enhanced distribution algorithm & improved handling of empty partitions
Build compression dictionary for all partitions– Previously, partitions that were empty at the start of REORG would not have a
dictionary built, requiring a subsequent REORG to gain compression
New SORTCLUSTER option to sort data in clustering as well as partitioning order to avoid AREO*
– Occurred when partitioning key not a superset of clustering key
© 2013 IBM Corporation18
REORG of LOB data
Support REORG of LOB data even though aux index is unavailable
– Problem in V10 if LOB tablespace is REORP and index is RBDP• LOBs can’t be reorged and index can’t be rebuilt
REORG SHRLEVEL NONE for LOBs changed to RC8 from 11 CM onwards
– Not supported in 10 NFM, but returns RC0 with MSGDSNU126I
© 2013 IBM Corporation19
Improved REORG serviceability
Need ability to use online REORG even when SYSLGRNX cannot be relied upon
Support LOGRANGES NO option for REORG SHRLEVEL CHANGE
© 2013 IBM Corporation20
REORG change of defaults to match best practices
Change default options:– DRAIN WRITERS to DRAIN ALL– DISCARD to DISCARD NOPAD YES– UNLOAD EXTERNAL to UNLOAD EXTERNAL NOPAD YES
© 2013 IBM Corporation21
Statistics Enhancements More zIIP offload for RUNSTATS distribution statistics
– Up to 80% zIIP-eligible
zIIP offload for inline statistics– Additional 30% offload to zIIP
Enhance inline statistics for RUNSTATS avoidance– Inline statistics collection on NPSIs during REORG with SORTNPSI– Inline histogram statistics– Inline DSTATS
New RUNSTATS RESET option to reset existing statistics
Improved PROFILE usability for LISTDEF processing– Gather default statistics if no profile exists for table
© 2013 IBM Corporation22
Statistics Enhancements
Optimizer determination of missing statistics– Optimizer identifies missing statistics & writes information to new
catalog table DSN_STAT_FEEDBACK– OQWT modifies statistics profile– Automation Tool detects profile change & builds RUNSTATS job– DSNACCOX similarly enhanced to recommend RUNSTATS
-ACCESS DATABASE … MODE(STATS) option to externalize RTS statistics
RTS overhead reduction
© 2013 IBM Corporation23
Backup & Recovery Enhancements Faster catalog/directory recovery
– Enhanced SYSLGRNX recording New VCAT name translation for RESTORE SYSTEM for system cloning
– Support logapply when RESTORE SYSTEM used for cloning purposes Improved recoverability with COPY-REORG concurrency
– Permit COPY to run concurrent with long-running REORGs
Avoid allocating empty image copy datasets for incremental or CHANGELIMIT copies Lifted many restrictions on point-in-time recovery prior to materializing REORG
– PIT recovery restrictions lifted for• LOB table spaces• XML table spaces• PBR table spaces• Including when immediate alters have occurred since materializing REORG
– PIT recovery restrictions still in place• Table space conversion• PBG table spaces• PBG partition pruning• Online DROP COLUMN
© 2013 IBM Corporation24
LOAD & UNLOAD Enhancements Crossloader support for XML data
Exploit FETCH CONTINUE for processing large LOBs & XML data in Crossloader
– Reduce vstor requirement– Avoid DSNU1178i errors– 28% CPU reduction
• Load of 1Mb LOBs
zIIP offload for LOAD REPLACE PART clearing of NPSIs– 100% offload to zIIP for LOAD REPLACE with dummy input
© 2013 IBM Corporation25
LOAD & UNLOAD Enhancements LOAD SHRLEVEL NONE PARALLEL with single input dataset
– Parallel data conversion– Not supported for PBGs– 50% ET reduction possible on single SYSREC load
Input
SYSRECRead
Part
1
Part
2
Conversion 1
Conversion 4
Conversion 2
Conversion 5
Conversion 3 Load task
© 2013 IBM Corporation26
LOAD & UNLOAD Enhancements LOAD SHRLEVEL CHANGE PARALLEL
– Supports non-partitioned as well as partitioned– Single input dataset– Not supported for PBGs– >80% ET reduction
Input
SYSRECRead task
Part
1
Part
2
Conversion 1
Conversion 4
Conversion 2
Conversion 3
© 2013 IBM Corporation27
Compression Dictionaries Avoid decompression failures for IFI 306 readers when new
compression dictionary built by REORG/LOAD
Old compression dictionary stored on log
New SYSCOPY record written pointing to old compression dictionary for CDC tables
IFI 306 read automatically retrieves old compression dictionary if necessary
Avoid need for replication target refresh when dictionary changes
© 2013 IBM Corporation28
General Enhancements
Greater parallelism for faster utilities– 11% elapsed time reduction measured for REORG, LOAD, REBUILD INDEX
PARALLEL option for parallelism control for LOAD, REORG, REBUILD INDEX, UNLOAD, CHECK INDEX
-DISPLAY UTILITY enhancements– Remove serialization between –DIS UTIL and –TERM UTIL– Jobname, start timestamp– Late addition: SWITCHTIME and NEWMAXRO
Utility impact reduction on bufferpools– Extend MRU for UNLOAD, REORG TABLESPACE, RUNSTATS TABLESPACE,
RUNSTATS INDEX, REBUILD INDEX, CHECK INDEX, CHECK DATA
Improved dataset cleanup in utility stored procedures– Previously, datasets remained allocated on utility failure, preventing cleanup
© 2013 IBM Corporation29
General Enhancements Improved TEMPLATE support for large / EF datasets and local time
values– DSNTYPE LARGE, EXTREQ, EXTPREF– New EATTR option on TEMPLATE to request extended attributes– New TIME LOCAL|UTC option
Enforce NUMTCB=1 for stored procedures
DSNACCOX performance
0
50
100
150
200
250
300
350
400
DSNACCOX query where DB=x
V10 Elapsed
V11 Elapsed
© 2013 IBM Corporation30
Deprecation REORG SHRLEVEL NONE for LOBs changed to RC8 from
11 CM onwards– Not supported in 10 NFM, but returns RC0 with MSGDSNU126I
Still supported in 11, but no longer documented: – REORG
• PARALLEL YES|NO – Superseded by LISTPARTS
• INDREFLIMIT• OFFPOSLIMIT• LEAFDISTLIMIT• UNLOAD ONLY• UNLOAD PAUSE• UNLOAD EXTERNAL
– COPY• CHANGELIMIT
© 2013 IBM Corporation31