ims12 ims performance tools

© 2012 IBM Corporation 1 © 2015 IBM Corporation

IMS Technical Conference Australia 2015

IMS12: Are you getting the most out of your IMS performance tools?

John Hancy, Fundi Software

© 2015 IBM Corporation


2

Agenda

Useful new information available in the IMS log

Techniques for gaining insight from this new

information

Recent reporting improvements that you might find

useful

The IMS transaction index – what is it and what

can I do with it?

Interactive problem determination using the IMS

log and other data sources – a live demo

Not just for IMS – perhaps your colleagues might

find these tools useful too?



3

What customers are asking for

1. Total Cost of Ownership – CPU measurement,

apportionment (per subsystem), reduction

Mobile Workload Pricing (MWP) - see IMS newsletter

page 5

2. Providing application developers with relevant information

to tune their applications, including:

3. Mobile and distributed – what’s happening on z/OS?

4. Batch window optimization

5. Bridging CICS, DB2, MQ and IMS performance and

problem determination – connecting SME’s

6. Impact of software upgrades, new functions (sync callout

etc.) and application changes



4

The IMS log: what’s new or been hidden from view? Information my manager might be interested in

– Transaction workload rates coupled with response time and service levels

Database contention

– Lock wait time per transaction

– Maximum locks held per transaction

Database activity

– DB I/O count and elapsed time

– VSAM and OSAM read and write counts

Syncpoint

– Phase 1 and 2 breakdown

– Fast Path OTHREAD – post-syncpoint AREA update activity

External subsystem usage (DB2, MQ etc.)

– Subsystems used and the number of calls made

Shared queues

– Compare local-first versus global input queue times

Program switch

– Track the flow and delay in a program switch sequence

Synchronous call-out

– Elapsed time per call

BMP

– Checkpoint intervals, databases updated and used



5

LOGINFO – what is in your IMS log? Code Count MCNT Recs/Sec Avg len Max Len Byte/Sec MB %

01 IN 20,778 25 1,236 9,608 31,144 25.6 2.1 IMS Message

CQS PUT 20,059 24 1,229 9,608 29,899 24.6 2.0 Put onto shared queue

CQS RD 636 0 1,072 1,155 826 0.6 0.1 Read off shared queue

01 OUT 40 0 751 1,040 36 0.0 0.0 IMS Message

MSG SWI 40 0 751 1,040 36 0.0 0.0 Message switch

03 IN 2,398 2 1,214 9,350 3,530 2.9 0.2 IMS Message


CQS RD 187 0 537 1,155 121 0.1 0.0 Read off shared queue

03 OUT 20,478 24 1,664 9,608 41,325 34.0 2.8 IMS Message


CQS RD 29 0 481 1,008 16 0.0 0.0 Read off shared queue

MSG SWI 826 1 510 9,589 510 0.4 0.0 Message switch

07 1,417 25,953 1 456 456 783 0.6 0.1 Program schedule end

MPP 1,309 1,645 1 456 456 723 0.5 0.0 MPP

QUICK 67 24,300 0 456 456 37 0.0 0.0 MPP quick reschedule

FALSE 23 0 0 456 456 12 0.0 0.0 MPP false schedule

BMP 16 0 0 456 456 8 0.0 0.0 BMP

ABEND 2 8 0 456 456 1 0.0 0.0 Abended transaction

5921 1 0 160 160 0 0.0 0.0 FP DEDB ADS open

5937 16,838 20 202 202 4,122 3.4 0.3 FP syncpoint

5938 2 0 202 202 0 0.0 0.0 FP syncpoint failure

5950 388,059 470 2,276 8,313 1,070,760 883.3 72.5 FP database update

5954 1 0 4,230 4,230 5 0.0 0.0 FP DB update - DEDB open

5955 1,087 1 49 49 64 0.0 0.0 FP SDEP CI buffer obtained

5957 387 0 157 338 73 0.0 0.0 FP DB update - DMAC

5958 1,087 1 49 49 64 0.0 0.0 FP current SDEP CI write



6

Ranking transaction performance

Avg Avg Max >2.0 Tot Avg Avg Tran InputQ Process Process Process Process CPU DB Call Trancode Count Time Time Time Time Time Time Count STOCK 275 0.595132 3.569668 69.50787 12.00% 89.24169 0.095634 1 BALANCE 152 0.138350 2.858584 5.738182 50.00% 11.43433 0.044236 3 WITHDRAW 339 1.264292 2.517168 72.50948 10.26% 98.16957 0.049611 190 ORDERS 101 0.118796 2.515761 4.832235 27.20% 10.06304 0.041714 3 BANKING 147 0.024561 1.913816 2.492869 66.30% 3.827631 0.056932 2

Avg Avg Max >2.0 Tot Avg Avg Tran InputQ Process Process Process Process CPU DB Call Trancode Count Time Time Time Time Time Time Count WITHDRAW 339 1.264292 2.517168 72.50948 10.26% 98.16957 0.049611 190 STOCK 275 0.595132 3.569668 69.50787 12.00% 89.24169 0.095634 1 BALANCE 152 0.138350 2.858584 5.738182 50.00% 11.43433 0.044236 3 BANKING 147 0.024561 1.913816 2.492869 66.30% 3.827631 0.056932 2 ORDERS 101 0.118796 2.515761 4.832235 22.70% 10.06304 0.041714 3

Questions your management might want answers to…

1. Which high volume transactions are taking longest to process?

2. What percentage of high volume transactions failed to meet their two

second service level?

Use the information to target specific transactions for attention



7

Avg Avg Avg Min >0.001 Max Tran CPU Process Schedule Schedule Schedule Schedule Schedule Tran Count Time Time Time Time Required Time Time CHECKING 674 0.002497 0.004434 0.000163 0.000065 99.11% 0.15% 0.165942 CREDIT 335 0.003022 0.064634 0.000159 0.000075 78.21% 0.38% 0.007718 FINANCE 1,330 0.002205 0.010187 0.000153 0.000069 78.95% 0.19% 0.004631 MOBILE 2,101 0.000747 0.012407 0.000221 0.000106 5.33% 1.79% 0.004609 ORDER 1,645 0.001053 0.007522 0.000162 0.000096 60.55% 0.10% 0.003647 STOCK 1,681 0.001364 0.002444 0.000073 0.000034 99.82% 0.24% 0.003446 WITHDRAW 1,524 0.001207 0.007537 0.000068 0.000030 100.00% 0.07% 0.007919

Scheduling and Pseudo WFI

Transaction perspective

Dependent region perspective

Subsystem perspective

Avg Avg Avg Min >0.001 Max Tran CPU Process Schedule Schedule Schedule Schedule Schedule Region Count Time Time Time Time Required Time Time IBANKING 2,797 0.002384 0.015831 0.000145 0.000030 99.93% 0.11% 0.165942 IMOBILE 5,082 0.001804 0.025657 0.000143 0.000040 46.79% 0.21% 0.004631 IORDERS 2,679 0.002510 0.019839 0.000137 0.000031 99.93% 0.19% 0.048253 IWEB 1,111 0.001423 0.021483 0.000172 0.000039 8.46% 1.06% 0.004609

Avg Avg Avg Min >0.001 Max Tran CPU Process Schedule Schedule Schedule Schedule Schedule SSID Count Time Time Time Time Required Time Time IMSP 182,440 0.004517 0.034590 0.000119 0.000027 51.54% 0.21% 0.165942

Percentage of

transactions that

required a full

program schedule



8

Syncpoint – affect on response time

Process CPU DB Syncpt Phase 1 Phase 2 Phase 2 OTHREAD Trancode Count Time Time Calls Time Time Time FP Time Time BANK1 18 0.041045 0.006635 8 0.014769 0.000030 0.014739 0.014739 0.000980 BANK2 26 0.051985 0.005922 8 0.000071 0.000032 0.000039 0.000039 0.001107 BANK3 280 0.048873 0.004889 23 0.000038 0.000018 0.000019 0.000019 0.001203 ORDERS 299 0.044485 0.004687 23 0.000034 0.000017 0.000017 0.000017 0.001200 MOBILE1 316 0.040288 0.004833 23 0.000789 0.000019 0.000771 0.000028 0.000789 MOBILE2 307 0.057567 0.004862 23 0.000037 0.000018 0.000019 0.000019 0.001195 MOBILE3 282 0.033802 0.004694 23 0.000036 0.000019 0.000016 0.000016 0.001191 WEBORDER 19 0.039227 0.007359 9 0.000076 0.000034 0.000043 0.000043 0.001057 STOCK1 21 0.036017 0.006329 9 0.000070 0.000031 0.000039 0.000039 0.001034 STOCK2 15 0.299056 0.020897 21 0.000053 0.000036 0.000016 0.000016 0.000983 STOCK3 16 0.372236 0.020957 21 0.000056 0.000039 0.000017 0.000017 0.001064

Are you using external subsystems and DASD mirroring? These might

affect the time it takes for your transactions to commit: • Total syncpoint time

• Phase 1 and Phase 2

• Phase 2 attributable to Fast Path database only

• OTHREAD time to write to ADS (asynchronous – not part of SYNCPT)

Avg Avg Avg Avg Avg Avg Tran CPU Process Syncpt Sync PH1 Sync PH2 Sync PH2 SSID Count Time Time Time Time Time FP Time DB2P 47,764 0.005166 0.034179 0.005902 0.006919 0.000319 0.000049 None 91,899 0.001711 0.021559 0.001961 0.002202 0.000221 0.000204

What is the impact

of DB2 syncpoint?



9

Avg Avg Max Max Tran Process DB Call DB Lock HWM Trancode Count Time Count Time Locks NIGHTRUN 2 20.21559 208,528 3.014112 38,275 CHECING 5,585 0.034276 53 0.697122 62 CREDIT 1,160 0.145987 115 0.030345 63 FINANCE 129 0.012458 7 0.015962 25 MOBILE 7 0.324563 989 0.014670 294 ORDER 397 0.186213 59 0.013845 62 STOCK 48 0.079283 213 0.009023 33

Locking and database I/O

Locks taken

Database I/O performance and activity

Avg Avg Tot Tot Tot Tot DB IO DB IO OSAMRead OSAMWrit VSAMRead VSAMWrit Count Time Count Count Count Count 17,282 7.972630 0 0 30,722 3,841 21 0.017631 1,980 2 108,248 8,921 242 0.119385 171 0 18,243 1,145 22 0.010127 0 0 2,654 0 758 0.189440 0 0 5,307 0 229 0.120370 24 0 2,950 226 76 0.016838 0 0 3,662 0



10

Start 01Jun2014 12:04:23:01 IMS Performance Analyzer Database Update Activity-IMSP _____________________________ 5050 Database Program Proc Total Updates ISRT DLET REPL ________ ________ ____ _________ ________ ________ ________ ________ ___ ORDERS ONLINE APPL 152 DLI 152 38 76 38 I/O 114 38 0 76 ONLINE B/O 10 DLI 10 4 4 2 I/O 8 0 0 8 MOBILE APPL 317 DLI 317 80 158 79 I/O 238 80 0 158 MOBILEM B/O 55 DLI 55 22 22 11 I/O 44 0 0 44 REQUEST APPL 85 DLI 85 55 30 0 I/O 53 11 0 42 STOCK APPL 21 DLI 21 6 10 5 I/O 16 6 0 10 *Total* APPL 575 DLI 575 179 274 122 I/O 421 135 0 286 *Total* B/O 65 DLI 65 26 26 13 I/O 52 0 0 52

Database update activity What databases were updated, and by which program

ISRT, DLET and REPL counts

Application update and back-out



11

Avg Avg Tot Tran CPU Process ESAFcall SSID Trancode Count Time Time Count DB2P CHECKING 74 0.038407 0.227590 961 DB2P CREDIT 181 0.016318 0.167907 729 DB2P FINANCE 22 0.027207 0.142876 308 MQS1 FINANCE 22 0.027207 0.142876 308 DB2P MOBILE 56 0.025832 0.192728 282 MQS1 MOBILE 56 0.025832 0.192728 282 DB2P WITHDRAW 21 0.036852 0.211123 273

External subsystem usage According to the IMS log

External subsystem usage by transaction

Only the call counts are available, DB2 and MQ call latencies are not available

DB2 Plan -------- Connection --------- Thread SSID Name Name Type Count DB2P BANKING IMSP IMS MPP PSB=WITHDRAW 21 Class1: Thread Time Avg: Elapsed= .042498 CPU= .035198 Max: Elapsed= .126945 CPU= .107331 Class2: In-DB2 Time Avg: Elapsed= .002525 CPU= .000701 Max: Elapsed= .006999 CPU= .000769 Class3: Suspend Time Avg: Total = .002623 I/O= .000000 Max: Total = .012437 I/O= .000000

According to DB2 accounting

Class 1 (thread) and Class 2 (in-DB2) shows elapsed and CPU times in both IMS and DB2

Suspend analysis can identify the cause



12

Shared queues Local-first and false scheduling

Shared queues transaction performance

Avg Avg Avg Avg >1.0 >1.0 Avg Max >0.1

Queue Tran InputQ Process OutputQ Total InputQ Process CPU CPU CPU

IMS ID Type Count Time Time Time Time Time Time Time Time Time

IMS1 GLOBAL 493 0.152755 0.353057 0.111123 0.613036 4.26% 11.76% 0.018737 0.199393 5.77%

IMS1 LOCAL 222 0.164432 0.567554 0.000162 0.732051 4.05% 21.62% 0.028475 0.199393 7.37%

IMS1 LOCALF 8701 0.003339 0.056002 0.208816 0.249786 0.10% 0.92% 0.007174 15.15835 0.32%

IMS2 GLOBAL 472 0.157556 0.432707 0.000675 0.590806 4.45% 15.89% 0.023619 0.236885 6.86%

IMS2 LOCAL 224 0.234427 0.457905 0.000130 0.692367 7.14% 16.52% 0.022103 0.208415 5.58%

IMS2 LOCALF 8131 0.002857 0.065651 0.190159 0.242889 0.00% 0.97% 0.012947 31.13607 0.47%

Total 18243 0.015950 0.089237 0.186943 0.276655 0.42% 2.07% 0.010805 31.13607 0.87%

Region overview MPP ******* DLI Calls ******* WFI **** Schedule Distribution **** Total CPU Sched Trans DB DC Other SUBQ6 Quick =0 =1 >1 ------- ------- ------- ------- ------- ------- ------- ------- ----- ------- ------- ------- Total 181858 4636 16842 178088 41165 24530 41556K 6279 Sched 1408 2131 1097 /Sched 39 1.0 3.6 38.4 8.9 5.3 8963 1.4 Tran 0 2365 14477 /Tran 10 0.3 1.0 10.6 2.4 1.5 2467 Tr/Sch 0 1 13 /Minute 30309 772.7 2807.0 29681.3 6860.8 4088.3 6926016 1046.5 Sc/Min 234.7 355.2 182.8 % Total 100.00 100.00 100.00 100.00 100.00 100.00 100.00 100.00 %Sched 30.37 45.97 23.66

1. IMS workload broken down by queue type: GLOBAL – shared queues where transactions came in on another system

LOCAL – shared queues where transactions came in on this system

LOCALF – local-first where shared queue was bypassed

2. False schedule and WFI analysis:



13

Program Switch: Follow the transaction trail

13

Part 1 of 2

Transaction BANK0001 originated from lterm NEWYORK at 14.58.02.023922

It switched to trancode BANK0010

Trancode BANK0010 then switched to BANK0011

Four (4) further switches to Trancode BANK0015 completed the sequence

The overall transaction response time is always reported against the

originating transaction – 4.467306 seconds – from when the originating

transaction arrived on the message queue to when a response was sent back

to the terminal

The last transaction in the sequence responded back to the terminal – you can

tell because it has an output queue time

Notice the long switch times for the last 2 transactions – this contributed to

almost 0.5 seconds of the response time



14

Program Switch: Looking for bottlenecks

14

Part 2 of 2



15

DB DB FP FP Program CHKP ID Start time CPU Elapsed Get Upd Get Upd Database AccM NITERUN NITECP1 21.30.23.310 0.2329 1.3150 1332 1 1473 3 CUSTOMER ESDS PAYMENTS DEDB ORDERS CHKP STOCK CHKP WIDGETS CHKP NITERUN NITECP2 21.30.24.657 0.3147 1.4038 1465 2 1521 4 CUSTOMER ESDS PAYMENTS DEDB ORDERS CHKP STOCK CHKP WIDGETS CHKP

BMP analysis For each checkpoint interval, see

Elapsed and CPU times

Database activity

Databases updated and checkpointed

BMP NITERUN issued user checkpoints with an id

Syncpoints were taken at regular intervals, every 1.5 seconds

CPU time was consistent across each of the intervals

Both full function and fast path database activity occurred – gets and updates

One FF and one DEDB database were updated

Three databases were checkpointed – used by the BMP for read-only purposes



16 16

Synchronous callout – Tracking Part 2 of 2

BROWSE IPI000.QADATA.JASDEEP2.VB10.IM01001 Record 00000057 More: < >

Command ===> Scroll ===> PAGE

Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJU

Code Description Date 2011-04-20 Wednesday Time (Relative)

/ ---- Search limit reached (+5000) ------------------------- ----------------

01 Input Message TranCode=SMQLG 01.19.00.003853

35 Input Message Enqueue TranCode=SMQLG +0.000001

31 DLI GU TranCode=SMQLG Region=0006 +0.000057

6701 YOUT Sync callout message sent +0.000125

6701 YACK Received ACK +0.000618

6701 YRSP Sync callout response received +0.201386

03 Output Message Response LTerm=OTM00001 +0.201420

35 Output Message Enqueue LTerm=OTM00001 Region=0006 +0.201422

37 Syncpoint Region=0006 +0.201424

37 Syncpoint message transfer Region=0006 +0.201425

33 Free Message +0.201428

5612 End of Phase 2 Syncpoint Program=PSBLG Region=0006 +0.201431

31 Communications GU LTerm=OTM00001 +0.201440

36 Output Message Dequeue LTerm=OTM00001 +0.203894

33 Free Message +0.203895

07 Application Terminate TranCode=SMQLG Region=0006 +0.220357

******************************** Bottom of Data *******************************

Delay due to

sync callout



17 17

The sync callout response time is a

large percentage of the overall

processing time: indicative of a delay

Transactions w/ sync-callout

SUMM0001 Data from 10.18.37 19Apr2011 to 10.21.06 19Apr2011

Avg Avg Avg Avg Avg Avg Avg

Tran CPU InputQ Process SyncCout Total SyncCout SyncCout

Trancode Count Time Time Time RespTime IMS Time ACK Cnt NAK Cnt

MYORDER 57760 0.0023 0.0052 0.5673 0.4723 0.5725 1 0

Synchronous callout – Reporting Part 1 of 2



19

BROWSE FUW000.QADATA.FBOSP006.SMF.D130530.CICS + Record 00000088 More: < > Command ===> Scroll ===> CSR Navigate < 00.00.01.000000 > Date/Time 2013-05-30 09.22.22.669470 / ———— Tracking ———————————————————————— Thursday 2013-05-30 Time (Relative) CICS Transaction 11.03.41.587515 TranCode=FB66 Program=FBOCCP66 Userid=TWM LTerm=SC0TCP06 Terminal=CP06 RecToken=FUWTCIC/CB6F7079F5587284 Resp=19.441616 CPU=0.016455 DB2=6 ACCT=FTS3.SC0TCP06.6F7079F55872 Task=679

19

Transaction indexes – what are they and is there one for IMS?

Part 1 of 3

IMS Transaction 11.34.22.536129 UTC=11.34.22.536121 TranCode=FBOIAT42 Program=FBOIAP42 Userid=FUNTRM14 LTerm=FUNTRM14 Terminal=SC0TCP14 Region=0002 OrgUOWID=IDDG/CC17F14163997884 SSID=IDDG IMSRel=131 RecToken=IDDG/0000000F00000000 CPU=0.027747 InputQ=0.000356 Process=0.032033 OutputQ=0.000027 TotalTm=0.032416 RegTyp=MPP

DB2 Accounting 11.03.59.237323 TranCode=FB66 Userid=TWM ClientID=FUWTCIC SSID=DB2P ET1=19.43099 ET2=9.486542 CPU1=2.750604 CPU2=2.748489 Suspend=5.217919 ACCT=FTS3.SC0TCP06.6F7079F55872 Source=CICS SEL=3 CAL=3 GetPage=43839 MaxLock=1 LUWID=FTS3/DBA6LU/CB6F7079F61B/0002

CICS has CMF written to SMF 110

DB2 has accounting written to SMF 101

You can create an IMS accounting record from the IMS log



20

Code... 56FA Transaction Statistics STCK... CBFC740BD90293A4 LSN.... 000000001C254F89 Date... 2014-06-23 Monday Time... 14.58.01.912361.227 JOBNAME.... 'IMSPMSG1' PROGRAM.... 'BANKING' TRAN....... 'BANK2' CPU........ 0.009593 DBIO....... 0.068882 LockWait... 0.000544 IOcalls.... +15 VSAMread... +7 VSAMwrite.. +2 OSAMread... +6 OSAMwrite.. +1 ESAFcalls.. +12

Benefits of Collecting 56FA – transaction accounting statistics

TRANSTAT=YES cuts type x'56FA' transaction accounting statistics

records at transaction-end and BMP-checkpoint.

Overcomes the limitations of type x'07' which is cut at program

schedule termination and often accumulates statistics for many

transactions.

Without 56FA you cannot get accurate transaction CPU time or other

useful transaction-level details.

You

need

the

56FA

Part 2 of 3



21

IMS transaction index

21

IMSPerformanceAnalyzer

IMSProblemInvestigator

IMS PA batch job condensesall IMS log records forone IMS transaction intoone Transaction Index record

Save processing time by usingTransaction Index as inputto IMS PA instead of IMS log

Transaction Index contains performanceinformation (such as response time) derivedfrom IMS log fields, enabling more intelligentfiltering and analysis in IMS PI

IMS log

TransactionIndex

Whether Full or Exception, all the accumulated information from the IMS log about each transaction is condensed into a single transaction index record.

Use the transaction index as input into:

IMS PA reporting - Instead of re-processing large SLDS log files

IMS PI analysis - To enable more intelligent problem detection

ETL into Big Data “Inflight” option allows

for continuous log

processing without

missing a beat.

Part 3 of 3



22

How to address IMS Log privacy issues using IMS Problem Investigator

IMS SLDS IMS PI SCRUB

(Batch) Scrubbed

extract

IMSPI only feature:

IMS PI ISPF dialog:

Interactive analysis

IMS PI Batch:

Reporting, extract and CSV

IMS PI SCRUB

(Exit)

1

2

1 SCRUB utility 1. Systems Programmer creates a copy of the original SLDS with confidential data scrubbed

2. The scrubbed extract data set is made available to all IMS PI users – data set to <<CLEARED>>

3. 100% secure – user has no access to confidential data

SCRUB exit (ALZUAUTH) – IMSPI only 1. All IMS PI users have access to the original (un-scrubbed) SLDS

2. All IMS PI ISPF dialog and batch services scrub the log records prior to display or reporting

3. Not secure – other utilities such as ISPF browse or DFSERA10 will display the confidential data

2

External

Problem

Analysis



23

IMS Problem Investigator batch reporting

Code Description Date 2011-06-17 Friday Time Elapsed LSN Rec#

---- ------------------------------------------------------ ------------------- --------------- ---------------- ---------

01 Input Message 21.57.00.950009.909 0.078041 0000000005AD82B0 9752

TranCode=CDB41PDS Userid=EVHWBUDR LTerm=4012 Terminal=4012

OrgUOWID=BUDR/C7EF90B3BC1BD08C Port=4012 LogToken=C7EE6AA5E342DC08

SSN=087124 Socket=PERS CM=1 SL=0 Source=Connect

35 Input Message Enqueue 21.57.00.950025.284 0.000015 0000000005AD82B1 9753

TranCode=CDB41PDS Userid=EVHWBUDR LTerm=4012 Terminal=4012

OrgUOWID=BUDR/C7EF90B3BC1BD08C Port=4012 LogToken=C7EE6AA5E342DC08

SSN=087124 Socket=PERS CM=1 SL=0

08 Application Start 21.57.00.950310.565 0.000285 0000000005AD82B2 9754

TranCode=CDB41PDS Region=000F RecToken=BUDR/0021DB9C00000000

RegTyp=MPP TClass=2D TPrty=01

5607 Start of UOR 21.57.00.950310.627 0.000000 0000000005AD82B3 9755

Program=CDB41PT Region=000F IMSID=BUDR RecToken=BUDR/0021DB9C00000000

31 DLI GU 21.57.00.950329.127 0.000018 0000000005AD82B4 9756

TranCode=CDB41PDS Region=000F OrgUOWID=BUDR/C7EF90B3BC1BD08C

RecToken=BUDR/0021DB9C00000000

5600 Sign-on to ESAF 21.57.01.048592.688 0.098263 0000000005AD82B5 9757

Region=000F IMSID=BUDR RecToken=BUDR/0021DB9C00000000 SSID=DSBR

SHORT batch report mode – similar to the ISPF dialog view



24

Want to rid your IMS GEN of no longer used resources?

Uses IMS checkpoint records:

4004=Transactions, 4006=Databases, 4007=Programs

For databases:

Start of checkpoint interval Date=2014-06-26 Thursday Time=17.51.31

DDIR DDIROPEN DDIRDA CHKWOPN DDIRGOPN Used? -------- -------- ------ ------- -------- ----- ACCOUNTS 40 20 00 00 YES BANKING 00 00 00 00 NO ORDERS 00 00 00 00 NO STOCK 40 20 00 00 YES

DDIR Database name

DDIROPEN At least one DCB is opened

DDIRDA DMB has been dynamically allocated

CHKWOPN DB was open

DDIRGOPN GBLSTATUS = OPEN

Used? YES – at least one of the above is set, DB is in use or has been used

NO – none of the above are set, DB has not been used



25

IMS Problem Investigator - IMS Trace Record Analysis /DLI Trace

DLI Trace events in IMS log:

67FF SNAP Trace when transaction abends

67FA /TRACE SET ON TABLE DL/I , LOCK etc

See every DLI call,

then associate with

the database updates

(50’s)

IMS Trace Table records are now interpreted, and can be tracked against the problem transaction:

Part 1 of 2

TX

Transaction Index



26

IMS Problem Investigator - IMS Trace Record Analysis /TRACE SET ON TABLE LOCK

Locking events, including

SUSPEND and RESUME

and any Associated Delays

• Dispatcher

• DLI and Lock

• Log Router

• Scheduler

• Queue Manager

• Latch

• LU 6.2 (APPC)

• Fast Path

• RRS

• DASD log

• External Subsystem

• OTMA

• Storage Manager

Part 2 of 2

{{{



27

CICS DBCTL

For all CICS transaction you can analyze CMF (SMF 110) records.

However CMF records do not contain all of the details of the IMS events

that are triggered by DL/I requests. Those details are in the IMS log.

Transaction Analysis Workbench enables you to analyze CICS DBCTL

transactions using:

–CMF records

– IMS log records

–A combined view of both

In particular Workbench enables you to analyze exceptions:

CICS DBCTL transactions that abended or had a long response time.



28

CICS-DBCTL summary – from both their perspectives

ss

CICS

ss

IMS

2014-03-26 Wednesday CICS-DBCTL Summary Page 1 Tran APPLID CMF Count Response CPU Time IMS Reqs IMS Wait ABEND Rate/Sec ---- -------- --------- -------- -------- -------- -------- -------- -------- BANK CICSP1 60 11.12982 0.008967 35 4.256977 10 0 08 Count Elapsed CPU Time StaDelay Schedule IC Wait PS Wait --------- -------- -------- -------- -------- -------- -------- 42 10.94999 0.004092 0.011668 0.000183 0 0 07 Count DB call DB Gets DB Upds IO Count IO Time LockWait --------- -------- -------- -------- -------- -------- -------- 41 33 13 19 4 0.003438 3.980170 FP Count FP call FP Gets FP Upds FP Wait FP Fail --------- -------- -------- -------- -------- -------- 41 19 7 11 0 7 Synctime Phase 1 Phase 2 FP PH2 OTHREAD -------- -------- -------- -------- -------- 0.011938 0.006555 0.005383 0.002232 0.017659



30

IMS-DB2 transaction – Tracking end-to-end Part 1 of 4

BROWSE FUW000.QADATA.FBOSPM3C.IMS.D131011.INDEX Record 00006876 More: < > Command ===> Scroll ===> CSR Navigate < 00.00.01.000000 > Date/Time 2013-10-11 11.29.17.925791 / ———— Tracking ———————————————————————— Friday 2013-10-11 Time (LOCAL) CA01 IMS Transaction 11.33.58.759345 TranCode=FBOIAT42 Program=FBOIAP42 Userid=FUNTRM14 LTerm=FUNTRM14 Terminal=SC0TCP14 SSID=IDDG Region=0001 RecToken=IDDG/0000000E00000000 OrgUOWID=IDDG/CC17F12AB6B85204 InputQ=0.000332 Process=1.094017 OutputQ=0.000028 TotalTm=0.134377 CPU=0.111167 ———————————————————————————————————————————————————————————————————————————— . . . ———————————————————————————————————————————————————————————————————————————— 003 DB2 Thread accounting DBA6 11.33.58.893342 TranCode=FBOIAP42 Program=FBOIAP42 Userid=FUNTRM14 RecToken=IDDG/0000000E00000000 ClientID=IDDG Region=0001 LUWID=FTS3/DBA6LU/CC17F12AB853/0002 RESP=1.086945 CPU1=0.107330 CPU2=0.000688 I/O3=0 Source=IMS_MPP

SEL=4 DEL=1 CAL=1 LogRecs=9777 GetPage=19777 UpdPage=9776 MaxLock=162 ———————————————————————————————————————————————————————————————————————————— ******************************* Bottom of Data ********************************

1. Both the IMS and DB2 index records show there is a problem

2. The DB2 accounting (IFCID 003) shows excessive logging, buffering and locking – where and why?



31


BROWSE FUW000.QADATA.FBOSPM4C.IMS.D130703.INDEX Record 00005529 More: < > Command ===> Scroll ===> CSR Navigate < 00.00.01.000000 > Date/Time 2013-07-03 16.33.53.849552 / ———— Tracking ———————————————————————— Wednesday 2013-07-03 Time (Relative) CA01 IMS Transaction TranCode=FBOICT42 Region=0002 16.40.55.075503 01 Input Message TranCode=FBOICT42 +0.000000 31 DLI GU TranCode=FBOICT42 Region=0002 +0.000266 5600 Sign-on to ESAF Region=0002 SSID=DBA6 +0.005517 112 Thread allocate FBOIAP42 DBA6 +0.005996 177 Package allocation FBOIAP42 DBA6 +0.006304 380 SP entry FBOSPM4C STMT=001044 DBA6 +0.006511 177 Package allocation FBOSPM4C DBA6 +0.006681 060 SQL SELECT STMT=000009 DBA6 +0.006796 058 SQL call completion SQLCODE=0 STMT=000009 DBA6 +0.016090 061 SQL DELETE STMT=000011 DBA6 +0.016387 0020 Begin UR +0.017584 325 Trigger entry USERDEL STMT=000011 DBA6 +0.017762 177 Package allocation USERDEL DBA6 +0.017836 061 SQL INSERT STMT=000002 DBA6 +0.017887 058 SQL call completion SQLCODE=0 STMT=000002 DBA6 +0.018139 Continued on next foil...

1. IMS transaction starts with an Index record (CA01)

2. IMS log (green), DB2 trace (yellow) and DB2 log (turquoise) events are shown

3. Transaction calls a DB2 stored procedure which issues SQL calls, call results are shown

IMS

DB2



32


... Continuation from previous foil 061 SQL DELETE STMT=000003 DBA6 +0.018179 0600 Delete from a data page . . . +0.019696 0600 Delete from a data page +1.082712 058 SQL call completion SQLCODE=0 STMT=000003 DBA6 +1.083240 325 Trigger exit SQLCODE=0 DBA6 +1.083254 058 SQL call completion SQLCODE=0 STMT=000011 DBA6 +1.083310 499 SP statement execution detail DBA6 +1.083370 380 SP exit FBOSPM4C SQLCODE=0 STMT=001044 DBA6 +1.083400 053 SQL request SQLCODE=0 STMT=001044 DBA6 +1.083436 5600 Commit Prepare starting Region=0002 SSID=DBA6 +1.084442 084 Prepare start DBA6 +1.084894 0020 End commit phase 1 +1.085008 085 Prepare end DBA6 +1.086509 03 Output Message Response LTerm=FUNTRM07 +1.086577 3730 Syncpoint End of Phase 1 Region=0002 +1.086598 074 Terminate thread start DBA6 +1.086969 0020 Begin commit phase 2 +1.087056 0020 End commit phase 2 +1.087728 003 Thread accounting DBA6 +1.089002 075 Terminate thread end DBA6 +1.089944 5600 Commit Continue completed Region=0002 SSID=DBA6 +1.089970 37 Syncpoint Message Transfer Region=0002 +1.089987 5612 Syncpoint End of Phase 2 Program=FBOIAP42 Region=0002 +1.090012 31 Communications GU LTerm=FUNTRM07 +1.090054 36 Output Message Dequeue LTerm=FUNTRM07 +1.093952 ******************************* Bottom of Data ********************************

IMS

SYNCPOINT

DB2

Delay

4. SQL DELETE causes trigger to delete many more table rows, causing the delay



33

+0120 QW0058ID... Scan information +0120 Scan type.... 'SEQD' Rows processed... +24069 +0130 Rows examined.... +24069 +0138 Rows qualified after stage 1... +24069 +0140 Rows qualified after stage 2... +1 +0148 Rows inserted.... +0 +0150 Rows updated... +0 +0158 Rows deleted... +0 +0160 Pages scanned.... +428 +0164 Pages scanned (RI)... +0 +0168 Rows deleted (RI).... +0 +0170 Pages scanned (LOB).... +0 +0174 Pages updated (LOB).... +0 +01A0 QW0058TY... Statement-level information +01A0 SQL statement type... 4000 +01A8 Statement ID... +28917 +01B0 Sync reads... +0 Getpages... +428 +01C0 Rows examined.... +24069 +01C8 Rows processed... +0 Sorts...... +0 +01D8 Index scans.... +0 +01E0 Table space scans.... +1 +01E8 Buffer writes.... +0 +01F0 Parallel groups.... +0 +01F8 In-DB2 elapsed... 0.008537

Enhanced support for DB2 trace (IFCID) records Part 4 of 4

Detailed formatting of IFCID-specific fields:

+0120 QW0058ID... 'SEQD' Scan type Off QW0058IX... 'INDX' Index On QW0058SD... 'SEQD' Sequential data Off QW0058SW... 'SEQW' Sequential data workfile

ims12 ims performance tools

Software

ims message cqs

ims transaction index

ims log techniques

ims performance tools

ims message msg swi

ims newsletter page

shared queue cqs rd

message switch