ims database performance

@Copyright IBM Corporation 2013 5.2

Welcome to:

IMS Database Performance

Rick Long, IMS Level 2 Database Support Team

Silicon Valley Lab [email protected]

Disclaimer

© Copyright IBM Corporation 2009. All rights reserved.U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.

IBM, the IBM logo, ibm.com, IMS, IBM High Performance Pointer Checker for z/OS and CICS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

Other company, product, or service names may be trademarks or service marks of others.

© Copyright IBM Corporation 2013

It is every DBA's nightmare – Does the database need to be reorganized or tuned?

This session describes the key performance indicators and how to interpret the large volume of information provided by the IBM HPPC utility.

Abstract


Example

HALDB stats from 2005

Partition name Roots Segments

ECDBP01 100,467 574,731

ECDBP02 81,687 581,307

ECDBP03 49,113 293,427

ECDBP04 249 2,140,708

ECDBP05 142 1,335,117

ECDBP06 180 1,300,517

ECDBP07 192 1,780,635

ECDBP08 123 1,479,159

ECDBP09 186 1,378,110

ECDBP10 257 1,875,931

ECDBP11 581 1,583,505

ECDBP12 415 382,060

ECDBP13 74 481,514

ECDBP14 1,476 978,823

ECDBP15 2,119 1,090,954

ECDBP16 1,992 331,247

ECDBP17 378 29,498

ECDBP18 149 2,767

TOTAL 239,780 17,620,010

HALDB stats from 2008

Roots Segments Additions

100,467 576,357 1,626

81,687 583,377 2,070

124,261 684,172 390,745

309 6,662,891 4,522,183

161 4,301,836 2,966,719

182 3,745,620 2,445,103

194 5,129,972 3,349,337

123 4,616,059 3,136,900

194 4,071,069 2,692,959

258 6,850,596 4,974,665

779 6,406,064 4,822,559

1,418 1,641,717 1,259,657

113 1,301,952 820,438

1,604 2,482,972 1,504,149

2,305 2,915,532 1,824,578

2,251 1,047,040 715,793

840 332,103 302,605

149 2,767 0

317,295 53,343,096 35,723,086© Copyright IBM Corporation 2013

• Review of Database types–Full Function Database

• HDAM,HIDAM–HALDB

• PHDAM, PHIDAM,PSINDEX

• Reorganization Needs–What factors cause a need for Reorganization

Agenda


PRIMARY

DATA

KSDS ESDS / OSAMKEY

INDEX

• Hierarchic Indexed Direct Access Method

• 2 databases• Index• Data

– 2 access method combinations• KSDS/ESDS (VSAM/VSAM)• KSDS/OSAM (VSAM/OSAM)

– 'Native-mode' index– Unique root keys required– Pointers– Space reusable

HIDAM


KEY

KEY

ESDS / OSAM

RANDOMIZER

• Hierarchic Direct Access Method

– 1 database• Data

– 2 access methods• OSAM• ESDS (VSAM)

– Randomizer– Non-unique root keys allowed– Synonyms – Pointers– Space reusable

HDAM


1 .. 100

101 .. 200

201 .. 300301 .. 400

• High Availability Large Data Base–Access methods

• PHDAM Partitioned HDAM• PHIDAM Partitioned HIDAM

–Secondary Index• PSINDEX Partitioned Secondary Index

–Partitions• Key Range• Partition Selection Exit

–Partition Selection beforerandomize – each partition can be different

HALDB


• Record Structure

–Prefix–Segment

• Variable length• Fixed length

Common Record Structure

ROOT

DEP DEP DEP DEP

DEP


Current Structure for R1CUST

ADJSMNTSPAYMENTSCUSTINVCUSTLOCNDISTRICT

CUSTOMER

ADDRLINE CUSTORDN


Data set group 2

Data set group 3

5.2

IMS Database Pointers


IMS Pointers

DELETE BYTE(FF), Prefix Descriptor(FP)

SEGMENT CODE

SC DB PTF DATA (BYTES=)

SEGMENT

PREFIX

LCFPTB PCF LCF

• HALDB


PHYSICAL TWIN FORWARD(4 BYTES) PHYSICAL TWIN BACKWARD(4 BYTES)

PHYSICAL CHILD FIRST(4 BYTES)

LOGICAL CHILD FIRST(28 BYTES)

IMS Pointers

DELETE BYTE(FF), Prefix Descriptor(FP)

SEGMENT CODE

SC DB PTF DATA (BYTES=)

SEGMENT

PREFIX

ILKLCFPTB PCF LCF

• HALDB


PHYSICAL TWIN FORWARD(4 BYTES) PHYSICAL TWIN BACKWARD(4 BYTES)

PHYSICAL CHILD FIRST(4 BYTES)

LOGICAL CHILD FIRST(28 BYTES)

INDIRECT LIST KEY

PTR=T

PTR=TB

PTR=NT

COURSE COURSECOURSE

CLASS

INSTRUCT STUDENT

EDUC EXPR

PTF PointersPTB Pointers


COURSE COURSECOURSE

CLASS

INSTRUCT STUDENT

EDUC EXPR

Physical child pointers are stored on the parent, but definedon the SEGM for the child:

Physical Child Pointer Specification

PCF ONLY: SEGM NAME=. . . ,PARENT=. . .

PCF ONLY: SEGM NAME=. . . ,PARENT=((. . . ,SNGL))

PCF, PCL: SEGM NAME=. . . ,PARENT=((. . . ,DBLE))

PCF PointersTwin Forward Pointers


5.2

Reorganization Objectives


• Access segments –Fewest I/Os–Segment placement

• Database records –optimal number of blocks

• Blocks/ Cis–Free space– In RAA (Root Addressable Area)–Room in the data set

Reorganization Objectives


• Information about the current state of the DB–Regular Statically Information gathering

• HPPC (or equalivant)• Unload/Reload stats• Historical information for Trend analysis

• Information on how the DB is used–Sequential processing vs. Random access–Mass insert or delete processing–Heavy sequential BMP processing –Which DBs are most frequently accessed–Which DBs are most frequently updated

• Splitting of segments –Variable length segments –Compressed segments

Reorganization Requirements


• Getting Segments into the Most Desirable block

• Re-sequence Segments – Root segments into Sequence– Child segment twin chains into a block

• HDAM Tuning– Getting Root segments into HOME block– Reducing Rap chain lengths

• Larger RAA (Root Addressable Area)• Rap chain out of the block

• Unspliting Split segments

• Refreshing Free space in the data set

• “Mean time” to reorganization

Reorganization Reasons


• How long do I want to go between reorganizations

• What are the reasons I need to reorg? –Re-sequencing segments–Split segments–Long twin chains–Rap chains in too many blocks

“Mean time” to Reorganization


• HDAM/PHDAM Root

–Block it Randomizes to

–Rap Chain stays within the block

– Ideal length of Rap chain is the number of DB records which will fit in the block

• All roots in RAA

Most Desirable Block


• HIDAM/PHIDAM Root

– PTR=TB - In the block with the next highest key value

– PTR=NT - In the block with the next lowest key value

– PTR=T - HIDAM Root (T) • SHOULD NEVER be used as RAPs are used and causes

DELETE performance issues

– Overflow not relevant



COURSE COURSECOURSE

HIDAM ROOT Physical Twin PTR

PTR=TB KEY POINTER100000 RBA

200000 RBA

200100 RBA

400000 RBA

999999 RBACALL :ISRT

MDB

COURSE

In the block with the next highest key value


COURSE COURSECOURSE

HIDAM ROOT Physical Twin PTR

PTR=NOTWIN KEY POINTER100000 RBA

200000 RBA

200020 RBA

400000 RBA

999999 RBACALL :ISRT

COURSE

MDB

In the block with the next lowest key value


• Child Segment – Sequential key – Lowest key value

• Block with its parent

– Sequential key - not lowest value • In the block with the next lowest key value

– NOTWIN - Block with parent

– Rules • FIRST – Block with parent• LAST - Block with previous LAST twin• HERE - Block positioned on



CLASS

• Sequential key defined on child segment– ISRT for child segment

• Inserting the lowest key value• Child PTF set to the next highest key value (taken from

PCF)• MDB – block with parent


INSTRUCT STUDENT


CLASS

• Sequential key defined on child segment– ISRT for child segment

• Inserting the key that is not the lowest key value• Inserted in sequence and PTF set to next highest key

(taken from PTF) • MDB – block with next lowest key value


INSTRUCT STUDENT


1. In the most desirable block 2. In the second-most desirable block or CI.3. In any block or CI in the buffer pool on the same cylinder.4. In any block or CI on the same track, as determined by consulting the bitmap.

(longest segment type defined. Uncompressed from DBD LENGTH=)5. In any block or CI on the same cylinder, as determined by consulting the bitmap.6. In any block or CI in the buffer pool within plus or minus n cylinders. Specify n in

the SCAN= keyword in the DATASET statement in the DBD. For HALDB databases, the value of the SCAN= keyword is always 0.

7. In any block or CI plus or minus n cylinders, as determined by consulting the bitmap.

8. In any block or CI in the buffer pool at the end of the data set.9. In any block or CI at the end of the data set, as determined by consulting the

bitmap. The data sets will be extended as far as possible before going to the next step.

HD space search algorithm


5.2

Reorganization “How Tos”


• Size of the RAA

• Packing Density – Packing Factor = bytes of data / bytes in RAA

• General usage guidelines was 70%• High Insert Less then 70%• Mostly read - 80%

• Put a “P” in front – Make it PHDAM– Reduce Packing density to suit how long between reorgs. – If data comes and goes, you can reorg very infrequently

How Do I Tune a: HDAM


• Keep Free space in it.–Don’t let heavy insert activity all end up at the end of

the data set.

• CI free space for dependent segments

• CA free space for Root and dependents

How Do I Tune a: HIDAM


• NEVER

• Tune those PARTITIONS which need to be.

• Change of mind set from a database to a PARTITION– Review partition information– Reorg partitions independent of the others

• Exceptions– Re-partitioning– STILL ONLY Deal with PARTITIONS and as few a possible

How Do I Tune a: HALDB


• Change the V to an O - Make it OSAM

• Faster access method

• OSAM Sequential buffering –BMP reading a HIDAM data base in root sequence

How Do I Tune : VSAM


• CI free space in the block

• Full length is used for comparison with bitmap data for insert

• HDAM – lower packing density

How Do I Tune : Split Segments


How Do I Tune : Split Segments

0VL SEGMENT SPLIT STATISTICS ---------------------------

SEGMENT <----------- PREFIX AND DATA -----------> SC NAME TYPE OCCURS NOT SPLIT SPLIT IN SAME BLOCK NOT IN SAME BLOCK -- -------- ---- ------ ------------ ---------------- --------------- --------------------03 CUSTORDN FC 400 0 0.0 % 200 050.0 % 100 50.0 % 100 50.0 % -- -------- ---- ------ ------------ ---------------- --------------- --------------------TOTALS 400 0 0.0 % 200 050.0 % 100 50.0 % 100 50.0 %

0VL SEGMENT SPLIT STATISTICS ---------------------------

SEGMENT <----------- PREFIX AND DATA -----------> SC NAME TYPE OCCURS NOT SPLIT SPLIT IN SAME BLOCK NOT IN SAME BLOCK -- -------- ---- ------ ------------ ---------------- --------------- --------------------03 CUSTORDN FC 202 0 0.0 % 202 100.0 % 6 3.0 % 196 97.0 % -- -------- ---- ------ ------------ ---------------- --------------- --------------------TOTALS 202 0 0.0 % 202 100.0 % 6 3.0 % 196 97.0 %


• CI free space in the block

• Data Set Group– Long segment– Left in Hierarchy– How many other segments does it displace for their MDB?– After reorg, Twins might be in a single block

• Large block size

How Do I Tune : Long twin chains


Current Structure for TxCUST

ADJSMNTSPAYMENTSCUSTINVCUSTLOCNDISTRICT

CUSTOMER

ADDRLINE CUSTORDN


How Do I Tune Data Set Groups

DBD S1CNTPY O1CNTPY U1CNTPY P1CNTPYPHIDAM 1DSG 5PARTS

PHIDAM 1DSG 1PARTS

PHIDAM 3DSG 5PARTS

PHIDAM 1DSG 1PART

Elapse time in Mins 2.24 2.25 0.37 2.24CPU SECS 15.19 14 7.49 14.03


How Do I Tune : Processing Sec Index (whole DB)

JOB name

Elaspe time in mins CPU SEC

No of Order segms No of GN

S1ORDLS1 33.19 484.15 5,668,280 5,338,196S1ORDLS2 4.23 83.04 5,668,280 5,338,196U1ORDLS1 37.13 575.09 5,668,060 5,668,281U1ORDLS2 4.23 97.27 5,668,060 5,668,281P1ORDLS1 33.43 497.83 5,668,060 5,668,061P1ORDLS2 4.48 87.46 5,668,060 5,668,061O1ORDLS1 35.49 523.12 5,668,145 5,668,146O1ORDLS2 3.48 94.66 5,668,145 5,668,146


ORDER

PROD

Segment BPROD

PROD

INVOICEPRODINVOICEPROD

INVOICEPRODINVOICE

INVOICE

ORDER

ORDER

CUST

PRODinvoice

PROD

ORDER

INVOICEPRODINVOICEPPOD

PRODINVOICEPROD

PROD

PRODINVOICEPROD

INVOICEPROD

Long Twin Chains


ORDER

PROD

Segment BPROD

PROD

INVOICEPROD

INVOICE

PROD INVOICEPROD

INVOICE

INVOICE

ORDER

ORDER

CUST

PROD

invoicePROD

ORDER

INVOICE

PROD

INVOICE

PPOD

PROD

INVOICE

PROD

PROD

PROD

INVOICE

PROD

INVOICE

PROD

PAYMENT

SHIP

PAYMENT PAYMENTPAYMENT

SHIPSHIPSHIP

ORDER PROD PROD

Long Twin Chains


ORDER

PROD

Segment BPROD

PROD

PRODPRODPROD

PROD

CUST

PROD

PROD

PROD PROD

PPOD

PRODPROD

PROD

PROD PRODPROD

PROD PROD PROD

PROD

PROD PROD

PPOD

PRODPROD

PROD PROD PROD

Long Twin Chains


ORDER Segment BPROD

PROD

PROD

INVOICE

PRODPROD

INVOICEORDERORDER

CUST

PROD

PROD

ORDER

INVOICE

PROD

PPOD

PROD

INVOICE PROD

PRODINVOICE PROD

INVOICE

PROD

PAYMENT

SHIP

PAYMENT PAYMENTPAYMENT

SHIPSHIPSHIP

ORDER

PROD PROD

Long Twin Chains


IMS Database Performance

QUESTIONS?

[email protected]


Text Slide0DBNAME: C1CUST DB#: 001 PARTNAME: C1CUST7 PART ID: 00007 REORG#:00002 DSG#:A DDNAME:C1CUST7ADSNAME: RLONG.IMS10.C1CUST.C1CUST7.A00007 DBORG: PHIDAM

INTERVAL: START BLOCK 1 END BLOCK 359

--------------- BASE -------------- ------------ OVERFLOW ------------- SC SEGNAME OCCURRENCES SPLIT-PRFX SPLIT-DATA OCCURRENCES SPLIT-PRFX SPLIT-DATA

01 CUSTOMER 2000 0 0 0 0 0 02 DISTRICT 3997 0 0 0 0 0 03 CUSTORDN 34 34 0 374 374 408 04 CUSTINVN 34 0 0 782 0 0 05 PAYMENTS 0 0 0 0 0 0 06 ADJUSTMT 0 0 0 0 0 0

---------- ---------- ---------- ---------- ---------- ----------TOTALS 6065 34 0 1156 374 408 -NOTE : - VIRTUAL SEGMENTS ARE NOT SHOWN


Text Slide

DSNAME: RLONG.IMS10.C1CUST.C1CUST7.A00007 BLK# 0........1.........2.........3.........4.........5.........6.........7.........8.........9.........0

1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890

1 0 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200 0000000000001111111111111111111111111111111111111111111111111111111111111111111111111111111111111111300 11111111111111111111111111111111111111111111111111111111111

-BLOCKS W/O SPACE = 212 BLOCKS WITH SPACE = 147 NO OF BIT MAPS = 1


Text Slide

0DISTRIBUTION OF DEPENDENT SEGMENTS IN ROOT BLOCK #ROOTS WITH 00 DEPS = 2 #ROOTS WITH 12 DEPS = 0 #ROOTS WITH 01 DEPS = 4 #ROOTS WITH 13 DEPS = 0 #ROOTS WITH 02 DEPS = 1960 #ROOTS WITH 14 DEPS = 0#ROOTS WITH 03 DEPS = 0 #ROOTS WITH 15 DEPS = 0 #ROOTS WITH 04 DEPS = 34 #ROOTS WITH 16 DEPS = 0 #ROOTS WITH 05 DEPS = 0 #ROOTS WITH 17 DEPS = 0 #ROOTS WITH 06 DEPS = 0 #ROOTS WITH 18 DEPS = 0 #ROOTS WITH 07 DEPS = 0 #ROOTS WITH 19 DEPS = 0 #ROOTS WITH 08 DEPS = 0 #ROOTS WITH 20 DEPS = 0#ROOTS WITH 09 DEPS = 0 #ROOTS WITH 21 DEPS = 0 #ROOTS WITH 10 DEPS = 0 #ROOTS WITH 22 DEPS = 0#ROOTS WITH 11 DEPS = 0 #ROOTS WITH 23 DEPS = 0

#ROOTS WITH 24+ DEPS = 0 ----------------------------------------------------------------------------------#ROOTS IN THE DATA SET = 2000


Text Slide

0DISTRIBUTION OF DEPENDENT SEGMENTS BY SEGMENT CODE --------------------------------------------------0SEGM CODE #DEPIN #DEPIN PERCENTAGE SEGMENT CODE #DEPS IN PERCENTAGE

ROOTBLK ALLBLK (ROOT/ALL) ROOT BLOCK ALL BLOCKS (ROOT/ALL)------- ----- ----- ----- ------------- ------ ----- -----(SEGCODE 02 ) 3992 3997 99.87 (SEGCODE 13 ) N / A N / A N / A (SEGCODE 03 ) 34 408 8.33 (SEGCODE 14 ) N / A N / A N / A (SEGCODE 04 ) 34 816 4.16 (SEGCODE 15 ) N / A N / A N / A (SEGCODE 05 ) N / A N / A N / A (SEGCODE 16 ) N / A N / A N / A (SEGCODE 06 ) N / A N / A N / A (SEGCODE 17 ) N / A N / A N / A(SEGCODE 07 ) N / A N / A N / A (SEGCODE 18 ) N / A N / A N / A (SEGCODE 08 ) N / A N / A N / A (SEGCODE 19 ) N / A N / A N / A(SEGCODE 09 ) N / A N / A N / A (SEGCODE 20 ) N / A N / A N / A(SEGCODE 10 ) N / A N / A N / A (SEGCODE 21 ) N / A N / A N / A(SEGCODE 11 ) N / A N / A N / A (SEGCODE 22 ) N / A N / A N / A(SEGCODE 12 ) N / A N / A N / A (SEGCODE 23 ) N / A N / A N / A

(SEGCODE 24+) N / A N / A N / A ---------- ---- ----- ------ ------------- ------ ------ ----

#DEPS IN THE DS 4060 5221 77.76


DATABASE RECORD STATISTICS

SEGMENT <---- SEGMENT LENGTH ---> <-AVERAGE OCCUR-> AVG LGTH CUMULATIVE SC LV DG NAME OCCUR PRFX + DATA = TOTAL P/ROOT PER/PRNT DB RECORD LENGTH-- -- -- -------- ----- --- ------- ------ ----- ----- ------- ----------01 01 A CUSTOMER 2,000 38 130.0 168.0 1.0 168.0 168.0 02 02 A DISTRICT 3,997 18 62.0 80.0 2.0 2.0 159.9 327.9 03 02 A CUSTORDN 408 18 220.2V 238.2 0.2 0.2 48.6 376.5 04 02 A CUSTINVN 816 18 55.0 73.0 0.4 0.4 29.8 406.3 05 02 A PAYMENTS 0 18 25.0 43.0 0.0 0.0 0.0 406.3 06 02 A ADJUSTMT 0 18 56.0 74.0 0.0 0.0 0.0 406.3 -- -- -- -------- ----- --- ------- ----- ----- ----- ------- ----------TOTALS 7,221 AVERAGE DB RECORD LENGTH = 406.3

AVERAGE DB RECORD PREFIX LENGTH = 85.0


VL SEGMENT SPLIT STATISTICS

SC NAME TYPE OCCUR NOT SPLIT SPLIT SAME BLOCK NOT IN BLOCK -- -------- ---- ---- ---- ---- ------------- ---------------- -------------03 CUSTORDN FC 408 0 0.0% 408 100.0 % 14 3.4 % 394 96.6 % -- -------- ---- --- ----- ---- ------------ ---------------- --------- -----TOTALS 408 0 0.0% 408 100.0 % 14 3.4 % 394 96.6 %


ims database performance - ims ug singapore 8-2013

Technology

ibm logo

ibm trademarked terms

current list of ibm

ibm hppc utility

database need

trademark information

database datarandomizer