SOA Testing IMS applicationsRegression testing of Z/OS, IMS DB2 and MQ at the SCC by replaying production workload

Process Summary

Extract 01 and 03 logrecs for a time period

Create Input file from 01 and 03 data

Sort the input file

Transfer input file to the SCC

Run the test

Input file

Each input record and it’s responses are grouped together consecutively on the log

00 control record• contains information about the message, userid, lterm,etc. Also contains the

elapsed time since the last message

01 record, contains the input message. • May be segmented over multiple messages

03 record contains a ‘response snippet’• A bit of the response to be compared for success• An indication of the expected response time• Includes Length and offset• One per response• May be 0 or more than one per input message

Log extract Process

Extract program

ProductionIMS LOGS

Apply optional

filters Sort

ExtractFile

Take this file to the flash copy test systems

Extract Program – LIVSOAL

Read IMS SLDS directly

• Produces smallest amount of data

• Use for normal testing

• Each system extracted separately, then merged with a sort

Produce from IMSPI extract

• Extract and merge 01 03 recs into a DASD dataset

• Run the extract program against the DASD dataset

• Useful for rerunning, developing, etc

• Still requires sort step afterwards

Distributed Environment - Simplified

MQ

JVM

JVM

JVM

JVM

AIX

MQ

OTMA MPR

MPR

MPR

MPR

DB2

MPR

IMS

Z/OS

WAS

Regression testing SOA applications

Subtask

Subtask

Subtask

Subtask

Subtask

Driver

MQ

OTMA MPR

MPR

MPR

MPR

DB2

MPR

IMS

Extractfile

Z/OS

Address Space

Driver Program LIVSOAD

Uses subtasking to drive MQ at the required message rate

Mainline controls pacing and subtask scheduling

Subtasks issue MQPUT, MQGET with wait option.

Waits on message correlid, like our real SOA systems

Driver Program LIVSOAD

Finds available input queues dynamically

Accepts commands via sysin or modify

Coordinates startup and shutdown of subtasks

Reads the input file

Creates the input message from segments

Build the MQIIH in front

POSTs subtask to do the MQ calls

Subtask Program LIVSOAS1

•Connects to MQ, creates reply queue• First time only

•Sends Message to IMS using MQPUT1• Uses the original userid authority• Uses the same commit mode• Uses the same MQ input queue for a particular TPIPE• 3270 messages round robined

•Waits for response with MQGET• Wait time factored on original response time• Waits on a specific correlid response message

•Compares the response received to the expected messages•Accumulates response times•Waits for more work•Disconnects from MQ on termination

Error Processing

•Count of errors maintained• Terminate after a maximum numbers

•MQ errors• Completion and reason codes written to syslog

•Comparison Errors• Snaps output record and snippets.

•All subtasks busy• Wait 1 second and retry• Repeat for up to 30 seconds before termination• Number of subtasks is adjustable• But beware of MQ IDBACK=20 – can reassemble/zap table, or set with

online command =MVMA SET SYSTEM IDBACK(500)

Checkpoint Processing

Replay log data until stop is triggered• By end of log data• By MAXMSGS processed• By MAXERRS reached• By ELAPSED minutes• By STOP command

Write the next record STCK to checkpoint file

Reposition here on next restart

Aim is only to replay a piece of log data once

Execution Parameters

Command Values SYSIN Modify Notes

MAXTASKS 1-100 Yes No Number of subtasks

PCTRATE 1-999 Yes Yes Percentage of the original message rate to send

MINRATE 0-999 Yes Yes Minimum rate override

MAXRATE 0-999 Yes Yes Maximum rate override

ELAPSED 0-999 Yes Yes Elapsed time for test

MAXMSGS 0-99999 Yes Yes Max messages for test

MAXERRS 0-99999 Max errors before term

ABEND N/A No Yes Terminates with dump

QMGR 8char Yes No Queue manager name

STGCLASS 8char Yes No Storage class name of

IMS as defined to MQ

STRTTIME 6char Yes No Time to start

STRTDATE 8char Yes No Date to start

PLEXNAME 8char Yes No Stop Accidents!!

Log extract Parameters

Command Values Notes

STRTDATE yyyymmdd Start date to extract

STRTTIME hhmmss Start time to extract

ENDTDATE yyyymmdd END date to extract

ENDTIMETIME hhmmss END time to extract

If not specified, reads from start to end of supplied log

Paired date and time must be both specified, or omitted

Log extract Filtering

Parameters Values Notes

T= TRAN name Can be generic

U= userid Optional, can be generic

M= lterm Optional, can be generic

O= 1-500 Compare offset, defaults to 0

L= 0,255 Length of comparison, default 100,

C N/A Uppercase compare data

X N/A Exclude

Examples T=ABC,M=CSQ*,O=12,L=10T=ABC,O=12,L=10,U

Testing Results

About 95% of transactions worked OK ‘out of the box’, OTMA+3270

With filtering, now 99.8% work

Not all transactions replay OK

SOA/OTMA workload is generally OK

Some transactions ‘call out’ via MQ to off mainframe – not available at SCC

Some transactions had a date/time in first 100 bytes – change the compare offset/length for these trans

Not all 3270 works

EDIT=UC and no MFS – gets uppercased when it didn’t previously

DRD could easily change to EDIT=ULC3270 traffic was not the reason for writing – a bonus if it worked

Testing Approach

Get a clean run, excluding stuff we can’t get to workRC=0 means function worked without errorsTune the comparison offset for some transactionsIMSGEN to remove EDIT=UC, or online command with DRDExclude transactions we cannot get to workRun the test before a changeRun the test after a changeNeed to pull off and compare IMF data for performance profile comparisons

Testing Notes

Startup mainview on SYSA/B

Start up summary trace and MWAIT for all transactions

Start up MQ MVMA and MVM1

Nobble JCL for CHIN

Make sure DB2s are started and connected, MSC links started, regions up

Start with a small, slow sample.

Expect ‘first time’ timeouts

Can run overnight, set STRTDATE and STRTTIME in SYSIN. Job will wait until that time to begin test

Testing Notes

Jobs on IV.MASTER.JOB(SOATESTn)

1 – log extract IMSA2 – Log extract IMSC3 – SORT SOA file4 - DUMP/TERSE SOA FILE5 - FTP GET/UNTERSE SOA FILE at SCC6 - Execute driver program7 – Terse IMF logrecords to take back to real systems8 – Restore IMF logrecs9 – LOAD IMS logrecs into SAS

Enhancements?

Do a before/after comparison job for IMF log data

Report the offset of compare failures

Summary report – list by highest errors

Can we do anything for stored procs?

Extract program – Use DBRC API, Dynamic allocation of SLDS, SORT in one step

IMS Connect

OTMACI