depository ibm business process management … · 7. created cluster with bpm configure job –...

21
© 2012 IBM Corporation DFK Data Warehouse Ge ne ral Ledger AP Sales Corrections PO Receiving Return to Vendor Warehouse Management Credit App Employee Change Notice OTHER A P PS - PC ACCT S REC APPS - PC INVENT ORY CONTROL APPS - PC INVENT ORY CONTROL APPS - PC Journal Entry Tool Kit Scorecard Resource Scheduling P09 - P17 Cyb. Mill. Millennuim 3.0 Banks - ACH and Pos to Pay Cobra Stock Status Polling On-line New Hire Entry CTS Plan Administrators (401K, PCS, Life) Post Load Billing Home Deliveries - Transfers Planning Purchase Order Solution Software Inventory Info Interface Sales Posting Price Management System Cycle Physical Inventory SKU Information Customer Repair Tracking I35 Early Warning System Merchandise Analysis Auto Replenishment CTO Intercept Counts Employee Purchase Tex A ACH Stock Options Customer Perceived In-Stock Tx SS Capital Projects Fixed Assets Recon File Repair EDI Coordinator Mesa D ata NEW Soundscan Re sum ix Op. Store Budget Reporting Tally Sheet Cash Receipts/Credit House Charges Ad Ex pe nse -Promo Analysis Price Marketing Support BMP - Bus performance Mngt Store Scorecard Price Testing Media Bonus/HR Hand Scan Apps Shows POS Sales Tax A04 - Cust Refund Chks Equifax Credit Cellular Rollover Satellite System Scanning VAN SKU Rep Host to AS400 Communication Layaways Bus Systems V04-Sign System Count Corrections N. P01- Employee Masterfile Customer Order ABC Co Universal Account Reconcilliation Depository Banks Cell Phones - ISP Tracking AAS PO Cash Over/ Short Coop SKU Selection Tool SKU Performance Supplier Compliance 1 DRK ABBX Misc Accounting/Finance Apps - PC/NT AIMS Mngr Approval Batch Forcasting Ad Measurement AIMS Reporting Ad Launcher Mkt Reactions Spec Source website Rebate Transfer Sign System W rite r Workspace PowerSuite Store Monitor Calendar Stores & Mrkts Due Dates Smart Plus Insertions Orders Budget Analysis Tool Print Costing Invoice App Reports Broadcast Filter Smart Plus Launcher General Maintenance Printer PO Printer Maintenance Vendor Maintenance Vendor Setup Connect 3 Connect 3 Reports Connect 3 PDF Transfe Spec Source SKU Tracking S20-Sales Polling Prodigy PSP In-Home Repair Warranty Billing System Process Servers (Imaging) DFK Data Warehouse Ge ne ral Ledger AP Sales Corrections PO Receiving Return to Vendor Warehouse Management Credit App Employee Change Notice OTHER A P PS - PC ACCT S REC APPS - PC INVENT ORY CONTROL APPS - PC INVENT ORY CONTROL APPS - PC Journal Entry Tool Kit Scorecard Resource Scheduling P09 - P17 Cyb. Mill. Millennuim 3.0 Banks - ACH and Pos to Pay Cobra Stock Status Polling On-line New Hire Entry CTS Plan Administrators (401K, PCS, Life) Post Load Billing Home Deliveries - Transfers Planning Purchase Order Solution Software Inventory Info Interface Sales Posting Price Management System Cycle Physical Inventory SKU Information Customer Repair Tracking I35 Early Warning System Merchandise Analysis Auto Replenishment CTO Intercept Counts Employee Purchase Tex A ACH Stock Options Customer Perceived In-Stock Tx SS Capital Projects Fixed Assets Recon File Repair EDI Coordinator Mesa D ata NEW Soundscan Re sum ix Op. Store Budget Reporting Tally Sheet Cash Receipts/Credit House Charges Ad Ex pe nse -Promo Analysis Price Marketing Support BMP - Bus performance Mngt Store Scorecard Price Testing Media Bonus/HR Hand Scan Apps Shows POS Sales Tax A04 - Cust Refund Chks Equifax Credit Cellular Rollover Satellite System Scanning VAN Ad Ex pe nse -Promo Analysis Price Marketing Support BMP - Bus performance Mngt Store Scorecard Price Testing Media Bonus/HR Hand Scan Apps Shows POS Sales Tax A04 - Cust Refund Chks Equifax Credit Cellular Rollover Satellite System Scanning VAN SKU Rep Host to AS400 Communication Layaways Bus Systems V04-Sign System Count Corrections N. P01- Employee Masterfile Customer Order ABC Co Universal Account Reconcilliation Depository Banks Cell Phones - ISP Tracking AAS PO Cash Over/ Short Coop SKU Selection Tool SKU Performance Supplier Compliance 1 DRK ABBX Cash Over/ Short Coop SKU Selection Tool SKU Performance Supplier Compliance 1 DRK ABBX Misc Accounting/Finance Apps - PC/NT AIMS Mngr Approval Batch Forcasting Ad Measurement AIMS Reporting Ad Launcher Mkt Reactions Spec Source website Rebate Transfer Sign System W rite r Workspace PowerSuite Store Monitor Calendar Stores & Mrkts Due Dates Smart Plus Insertions Orders Budget Analysis Tool Print Costing Invoice App Reports Broadcast Filter Smart Plus Launcher General Maintenance Printer PO Printer Maintenance Vendor Maintenance Vendor Setup Connect 3 Connect 3 Reports Connect 3 PDF Transfe Spec Source SKU Tracking S20-Sales Polling Prodigy PSP In-Home Repair Warranty Billing System Process Servers (Imaging) IBM Business Process Management Advanced: Process Server V8.5 for z/OS Problem Determination, Advanced Topics and Wrap-up Problem Determination, Administration, Operations, Toplogies, Migration, Performance (Updated Nov. 22, 2013) © 2012 IBM Corporation Topics Problem Avoidance & Trouble-Shooting Prevention Diagnosis Tools & Techniques Administration, Operations & Recovery Performance Migration Information Resources Summary & Feedback

Upload: trinhnguyet

Post on 26-Aug-2018

215 views

Category:

Documents


0 download

TRANSCRIPT

© 2012 IBM Corporation

DFK

Data Warehouse

GeneralLedger

AP

SalesCorrections

POReceiving

Return toVendor

WarehouseManagement

Credit App

EmployeeChange Notice

OTHER APPS - PC

ACCTS REC APPS - PCINVENT ORY CONTROL APPS - PC INVENT ORY CONTROL APPS - PC

Journal Entry Tool Kit

Scorecard

ResourceScheduling

P09 - P17Cyb.

Mill.

Millennuim 3.0

Banks - ACH and Pos toPay

Cobra

StockStatus

Polling

On-line NewHire Entry

CTS

Plan Administrators(401K, PCS, Life)

Post LoadBilling

HomeDeliveries

-Transfers

Planning

PurchaseOrder

SolutionSoftware

Inventory Info

Interface

Sales Posting

Price ManagementSystem

Cycle PhysicalInventory

SKUInformation

Customer RepairTracking I35 Early Warning

System

MerchandiseAnalysis

AutoReplenishment

CTO

Intercept

Counts

EmployeePurchase

Tex A

ACH

Stock Options

Customer PerceivedIn-Stock

Tx

SS

CapitalProjects

FixedAssets

ReconFile

Repair

EDICoordinator

Mesa D ataNEW Soundscan

Resumix

Op.

Store BudgetReporting

Tally Sheet

Cash Receipts/Credit

HouseCharges

Ad Expense

-PromoAnalysis

PriceMarketingSupport

BMP - Busperformance Mngt

StoreScorecard

PriceTesting

Media

Bonus/HR

Hand ScanApps

Shows

POS

SalesTax

A04 - CustRefund Chks

Equifax

Credit

CellularRollover

SatelliteSystem

Scanning

VAN

SKU Rep

Host to AS400Communication

Layaways

Bus Systems

V04-SignSystem

Count CorrectionsN.

P01-EmployeeMasterfile

CustomerOrder

ABCCo

Universal AccountReconcilliation

DepositoryBanks

CellPhones

- ISPTracking

AAS

PO

Cash Over/Short

Coop SKU SelectionTool

SKUPerformance

SupplierCompliance

1

DRK

ABBX

Misc Accounting/Finance Apps - PC/NT

AIMSMngr Approval

Batch ForcastingAd Measurement

AIMSReportingAd

Launcher

MktReactions

SpecSource

website

RebateTransfer

SignSystem

WriterWorkspace

PowerSuite

StoreMonitor

Calendar

Stores & Mrkts

Due Dates

Smart Plus

InsertionsOrders

BudgetAnalysis Tool

Print CostingInvoice App

Reports

BroadcastFilter

Smart PlusLauncher

GeneralMaintenance

Printer PO

PrinterMaintenance

VendorMaintenance

Vendor Setup

Connect 3

Connect 3Reports

Connect 3PDF Transfe

Spec SourceSKU Tracking

S20-SalesPolling

Prodigy

PSP

In-HomeRepair

WarrantyBillingSystem

Process Servers(Imaging)

DFK

Data Warehouse

GeneralLedger

AP

SalesCorrections

POReceiving

Return toVendor

WarehouseManagement

Credit App

EmployeeChange Notice

OTHER APPS - PC

ACCTS REC APPS - PCINVENT ORY CONTROL APPS - PC INVENT ORY CONTROL APPS - PC

Journal Entry Tool Kit

Scorecard

ResourceScheduling

P09 - P17Cyb.

Mill.

Millennuim 3.0

Banks - ACH and Pos toPay

Cobra

StockStatus

Polling

On-line NewHire Entry

CTS

Plan Administrators(401K, PCS, Life)

Post LoadBilling

HomeDeliveries

-Transfers

Planning

PurchaseOrder

SolutionSoftware

Inventory Info

Interface

Sales Posting

Price ManagementSystem

Cycle PhysicalInventory

SKUInformation

Customer RepairTracking I35 Early Warning

System

MerchandiseAnalysis

AutoReplenishment

CTO

Intercept

Counts

EmployeePurchase

Tex A

ACH

Stock Options

Customer PerceivedIn-Stock

Tx

SS

CapitalProjects

FixedAssets

ReconFile

Repair

EDICoordinator

Mesa D ataNEW Soundscan

Resumix

Op.

Store BudgetReporting

Tally Sheet

Cash Receipts/Credit

HouseCharges

Ad Expense

-PromoAnalysis

PriceMarketingSupport

BMP - Busperformance Mngt

StoreScorecard

PriceTesting

Media

Bonus/HR

Hand ScanApps

Shows

POS

SalesTax

A04 - CustRefund Chks

Equifax

Credit

CellularRollover

SatelliteSystem

Scanning

VAN

Ad Expense

-PromoAnalysis

PriceMarketingSupport

BMP - Busperformance Mngt

StoreScorecard

PriceTesting

Media

Bonus/HR

Hand ScanApps

Shows

POS

SalesTax

A04 - CustRefund Chks

Equifax

Credit

CellularRollover

SatelliteSystem

Scanning

VAN

SKU Rep

Host to AS400Communication

Layaways

Bus Systems

V04-SignSystem

Count CorrectionsN.

P01-EmployeeMasterfile

CustomerOrder

ABCCo

Universal AccountReconcilliation

DepositoryBanks

CellPhones

- ISPTracking

AAS

PO

Cash Over/Short

Coop SKU SelectionTool

SKUPerformance

SupplierCompliance

1

DRK

ABBX

Cash Over/Short

Coop SKU SelectionTool

SKUPerformance

SupplierCompliance

1

DRK

ABBX

Misc Accounting/Finance Apps - PC/NT

AIMSMngr Approval

Batch ForcastingAd Measurement

AIMSReportingAd

Launcher

MktReactions

SpecSource

website

RebateTransfer

SignSystem

WriterWorkspace

PowerSuite

StoreMonitor

Calendar

Stores & Mrkts

Due Dates

Smart Plus

InsertionsOrders

BudgetAnalysis Tool

Print CostingInvoice App

Reports

BroadcastFilter

Smart PlusLauncher

GeneralMaintenance

Printer PO

PrinterMaintenance

VendorMaintenance

Vendor Setup

Connect 3

Connect 3Reports

Connect 3PDF Transfe

Spec SourceSKU Tracking

S20-SalesPolling

Prodigy

PSP

In-HomeRepair

WarrantyBillingSystem

Process Servers(Imaging)IBM Business Process Management Advanced: Process Server V8.5 for z/OS

Problem Determination, Advanced Topics and Wrap-up

Problem Determination, Administration, Operations, Toplogies, Migration, Performance

(Updated Nov. 22, 2013)

© 2012 IBM Corporation

Topics

� Problem Avoidance & Trouble-Shooting➔ Prevention

➔ Diagnosis➔ Tools & Techniques

� Administration, Operations & Recovery

� Performance

� Migration

� Information Resources

� Summary & Feedback

3© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

PD is not a Science or Structured Methodology � Learned through (unsuccessful) experiences

� Skills & Teamwork (Sysprog, DBA, Sec.Admin...)

� Get good tools & learn how to use them.➨ UNIX: vi, viascii, awk, grep, a2e,

➨ java: jar, jad, debug

➨ TSO: ISPF, SDSF, MXI

➨ MVS: Operator Commands

➨ DB2: SPUFI, DB2Admin (ADBL)

➨ MQ: ISPF Panels

➨ WAS: dumpNameSpace, wsadmin, adminconsole

➨ Monitors, Sniffers, Drivers, ...

➨ Traces: WAS errorlog, IPCS, TraceJava, JDBC, JVM, JRAS, log4j, wsadmin

➨ Java tools

� Your Brain is your best tool . . .

� Resources & References: InfoCenter, admin Help, WebClient Help, IBMLink,

e-Support, TechDocs, RedBooks, Google, . . .

think

outside

the box

think

outside

the box

Problem Determination

4© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Problem Prevention (Avoidance)

� Adequate Hardware resources (CPU, Real Storage, DASD)

� Good Naming Standards & Planning

– Use the Spreadsheet!

� Software pre-reqs

– See the Program Directory.

� Tools (z/OS, Workstation, Web, ...)

� Skills (z/OS, UNIX, DB2, Security, TCP/IP, Java, ...)

� Adequate Testing Environments & Procedures

6© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Configuration – Problem Avoidance:(Here's what we set up before the Lab to avoid problems)

� Hardware:

– z/OS: 4Gb Real Memory, Six 3390-9 Public/Storage or SMS DASD volumes

– Linuz (for ProcessCenter):

� Software: z/OS, WAS, BPM, DB2, ...

– Installation Manager & Plug-ins

� Tailored Spreadsheet to produce: (from Techdoc WP102261)

– Response Files for WCT DMgr & EmptyNode + Augment Jobs

– BPM Configuration properties file

– DB2 Database Definitions

� Sample Scripts provided by ATS & BPM:PS Development to …

– Create Security profiles in RACF – Techdoc WP101427

– Create DB2 Tables

– Set up BPM Security

– Fix WAS Variables (port #s, Short names - based on techdoc TD105447)

Not much can go wrong if you avoid typing & spelling errors & follow directions.

7© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

What can possibly go wrong? (Common Problems)

Configuration

> Deployment

> Runtime

� Reading, Typing, Spellling

� Authorization problems

� Following directions? Poor directions?

� Real men don't need directions! ;-)

� Scripts not run in the correct order

� Space exhausted in config HFS, Joblogs, or /tmp

� Files or directories owned by another user

� Scripts interrupted

� ...

Logs and Traces

➔ Java, DB2, TCP/IP, ...

Check for messages ...

... in a Browser window

... in the Joblog, SYSLOG, & SYSPRINT

... in SYSLOG

... in log files (in the HFS)

If the problem scenario is reproducible ...

... capture relevant info (time, system, input, etc.)

... turn on more/other traces

... write down the steps how to reproduce

Search for these messages/symptoms ...

... in the WebSphere and Compute Grid InfoCenters

... on the WebSphere eSupport site ibm.com/software/webservers/appserv/zos_os390/support.html

... in the IBM knowledge database (IBMLink)

... in Google

If you still can't figure out your problem ...

� ... contact local experts or IBM Support for asst.

When first encountering a problem...

8© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

BPM:PS z/OS Configuration – Reviewing the steps (1st Lab)

1. Used the Spreadsheet to create response, BPM property and Sql files

2. Used WCT/PMT to create “Stacked” Jobstreams

3. Configured Security – scripts supplied

4. Configured the Deployment Manager – run the 6 jobs in order

5. Created DB2 Tables – scripts supplied

6. Created an Empty Node and Federation – run the 6 jobs in order

7. Created Cluster with BPM Configure job – automated by spreadsheet

8. Updated Variables & Bootstrap PS Database - scripts supplied

9. Installed sample applications & Tested them

Pretty Simple compared to Earlier Versions!

9© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Setting up Security

● The standard BBO%BRAK jobs do not include the following:

● Administrator's userID password not set

● STARTED profiles for servers not set if you are creating Empty Nodes

● The DORAC700 Exec copied from Techdoc TD101427 does both.

● Do NOT run any of the BBO%BRA% jobs except for the BBODBRAM job which creates the Home directories.

● Run it right after the DORAC700 Exec.

● Review the jobs you submitted from USER1.WAS.CNTL:

– RUNRC700 – runs the DORAC700 exec in USER1.WAS.DATA

– B#RACUID – Defined user Ids for BPM

– B#EJBROL – Defined EJBROLE profiles for BPM

10© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Misc. Configuration Problems – Ownership Errors

Ownerships can be corrupted: wasv8config/b6cell/b6dmnode/DeploymentManager/profiles/default ->ls -alt

drwxrwx--- 5 OMVSKERN B6CFG 416 Dec 23 14:50 configuration

Fix: (See FIXPERMS in USER1.WAS.CNTL)

cd /shared/zWebSphere/V8R0/bin; +

./bbowhfsb.sh /wasv8config/b6cell/b6dmnode/DeploymentManager; +

./bbowhfsc.sh /wasv8config/b6cell/b6dmnode/DeploymentManager b6admin b6cfg; +

./bbowhfsb.sh /wasv8config/b6cell/b6nodea/AppServer; +

./bbowhfsc.sh /wasv8config/b6cell/b6nodea/AppServer b6admin b6cfg

Or run these commands from the BBODHFSA job:

chmod -Rh 775 /wasv8config/b6cell/b6dmnode/DeploymentManager

chown -Rh B6ADMIN /wasv8config/b6cell/b6dmnode/DeploymentManager

chgrp -Rh B6CFG /wasv8config/b6cell/b6dmnode/DeploymentManager

11© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Problems Avoided when Creating the DB2 Tables

The following tools will save you time

and eliminate problems by automating the process:

– Spreadsheet generates the DDL

statements to define the Storage Group and Databases, and properties file used

by the BPMConfig.sh script

– BPMConfig.sh script run by the

BPZCNFGD job generates the necessary .sql files.

– createDatabase.sh script reads the .sql

files and creates the DB2 tables and other artifacts.

PDSTargetJobs

DB2Tables

WASConfig

xml

ResponseFiles

Spreadsheet

createDatabase.sh

zPMT

BPZCNFGD

Job

BPMConfig.sh

SQL tabledefinitions

BPMproperty File

1 2

34

1

12© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Common SQL Error Codes

� -104 Illegal Symbol [caused by extra blanks after '(' or ')' in col. 1 w/DB2 V9.]

� -204 Usually Table not defined (or wrong Schema prefix)

� -443 DB2 maintenance applied (Pacakages need to be re-bound)

� -551, -552, -553 Authorization problems (e.g., Missing GRANTs)

� -601 Duplicate name (e.g., already defined)

� -805 Packages need to be re-bound

� -904 Resource unavailable (e.g., DASD space)

� -913 Deadlock or Timeout

� -922 J2C Authentication errors (RACF APPL classprofiles missing)

� -4475 Missing DB2JccProperties file

13© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Federation Problems

1. If you don't change the UserID on the BBOWMNAM job to the admin's

● Change the userID to the administrator's

● Remove any files in /tmp created by the BBOWMNAN job

● Resubmit the job

2. If you Federate the Empty Node before augmenting it, …

� Error federating the empty node.

– ADMU0027E: An error occurred during federation The dmgr is not WPS enabled

– while the node is, we doesn't support such federation scenario!;

– rolling back to original configuration.

– Turns out there were errors in the Augment job

� Go to the /wasv8config/b6cell/b6nodea/AppServer/bin:

>./removeNode.sh -user b6admin -password b6admin -force

� Then go to the ISC and do a 'Force Delete' of the node.

� Then run the Augment job and then the Federate job....

14© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

DB2 Tools & Techniques

DASD space used by your table spaces

DB2 Table spaces are backed by VSAM datasets whose names have the following format:

Vcatname.DSNDBD.dbname.tsname.y0001.znnn

In the data set:

● Vcatname: The integrated catalog name (high level index), or VCAT

● DSNDBD: The data component of the VSAM cluster

● dbname: The database name

● tsname: The table space name or index name

Here is an example, sorted with the largest ones on the top.

Command ===> sort tracks Tracks

---------------------------------------------------------

WSLDB2.DSNDBD.B6SR01.WLP1194.I0001.A001 1185

WSLDB2.DSNDBD.B6SR01.BSPRTP1.I0001.A001 1059

WSLDB2.DSNDBD.B6SR01.BSPLTP3.I0001.A001 555

WSLDB2.DSNDBD.GLDDB.DESCTS.I0001.A001 300

WSLDB2.DSNDBD.GLDDB.DIRRSEAR.I0001.A001 300

15© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

DB2 Administrator's Tool - ISPF: Option 6 (TSO Command line) – 'ADBL' > 1 > 'D'

DB2 Admin ------------------- DSN9 Databases ----

Storage Buffer Created

Select Name Owner Group Pool DBID By

B6* * * * * *

------ -------- -------- -------- -------- ------ --------

B6CELLDB SYSADM1 B6DBSTO BP1 308 SYSADM1

T B6SR01DB SYSADM1 B6DBSTO BP1 312 SYSADM1

=========================================================

DB2 Admin --------------- DSN9 Tables, Views, and Alias

Sel Name Schema T DB Name TS Name

----- -----------------> -------- - -------- -------- -

BR LSW_TIME_PERIOD B8C1PS T B8SR01DB WLPT113

==========================================================

Admin -- DSN9 BROWSE B8C1PS.LSW_TIME_PE > -------------

****************************************************** Top of Data

ID NAME START_TIME END_TIME EFFECTIVE_DAYS

-- ----------------- ---------- -------- ---------------------------

0 24X7 0000 2359 mon,tue,wed,thu,fri,sat,sun

1 7AM-7PM M-F 0700 1900 mon,tue,wed,thu,fri

2 7AM-7PM Every Day 0700 1900 mon,tue,wed,thu,fri,sat,sun

3 9AM-5PM M-F 0900 1700 mon,tue,wed,thu,fri

4 9AM-1PM Sat 0900 1300 sat

***************************************************** Bottom of Data

18© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

jdbc Tracing

Specify in the jdbc properties file, normally named db2.jcc.propertiesFile:

db2.jcc.ssid=DSN9

db2.jcc.override.traceFile=/tmp/B6SR_tracejdbc.txt

See "Problem diagnosis with the IBM Data Server Driver for

JDBC and SQLJ”

http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp

19© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Runtime Tracing

Java Traces

Use the MVS modify command to set java tracing dynamically:

➨➨➨➨ Turn on Java Tracing for specified components

F <server_name>,TRACEJAVA='com.ibm.bpe.*=all=enabled'

➨➨➨➨ Reset to trace settings in configuration (was.env)

F <server_name>,TRACEINIT

Logs

- Messages in servants' & adjuncts' SYSPRINT & SYSOUT

- ffdc - "first failure data capture"

➨➨➨➨ Messages in servers' SYSPRINT shows the name & location:

FFDC closed incident stream file

/wasv6config/b6cell/nodeb/AppServer/profiles/default/logs/ffdc/

b6cell_b6nodeb_b6sr01b_STC32711_B6SR01BS_12.09.30_20.07.03_0.txt

<cell> <node> <server> <STC#> <STCname> <DateYMD> <TimeHMS>.txt

Strip_trailing_spaces.sh

iconv -f ISO8859-1 -t IBM-1047 $files | sed -e 's/'"$(printf '\015')"'$//' | sed -e's/[ \ ]*$//' | iconv -f IBM-1047 -t ISO8859-1 >.work_file_clean && mv .work_file_clean $files

Or use Mike's script:

/u/user1/wpswork/strip_trailing_spaces.sh . '*.sql'

25© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

wsadmin.sh tracing

Edit {profile_root}/properties/wsadmin.properties file:

Uncomment the line: #com.ibm.ws.scripting.traceString=com.ibm.*=all=enabled

#-------------------------------------------------------------------------

# The traceFile property determines where trace and logging

# output are directed. If more than one user will be using

# wsadmin simultaneously, different traceFile properties should

# be set in user properties files.

# The default is that all tracing and logging go to the console;

# it is recommended that a value be specified here.

# If the file name contains DBCS characters, use unicode format such as \uxxxx, ..

#-------------------------------------------------------------------------

com.ibm.ws.scripting.traceFile=/wasv6config/b6cell/nodea/AppServer/profiles/default/logs/wsadmin.traceout

#-------------------------------------------------------------------------

# The traceString property governs the trace in effect for

# the scripting client process.

# The default is no tracing.

#-------------------------------------------------------------------------

com.ibm.ws.scripting.traceString=com.ibm.*=all=enabled...

Then run the script and go look in the

{profile_root}/logs/wsadmin.traceout file.

29© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

ProcessCenter - Administrator

Launch from

http://<host>:<port>/ProcessCenter

UserID = tw_admin

Process Designer – Accessing the Process Center

Launch Windows Appl. From:

C:\IBM\ProcessDesigner\V8.0\eclipse.exe

31© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Connecting to the Process Center from the P.D.

● In the PD directory (C:\IBM\ProcessDesigner\v8.0) there is a file eclipse.ini

● Change the line

-Dcom.ibm.bpm.processcenter.url=http://192.168.17.232:9080

● to point to your ProcessCenter ip address... the port should be defaulted to 9080.

See traces in the same directory (orbtrc.11112011.1247.20.txt) or in C:\IBM\ProcessDesigner\v8.0\workspace\metadata\.log

● 12:47:20.448 com.ibm.ws.orbimpl.transport.WSTransport getHostIPAddress:137 Thread-10 ORBRas[default] java.net.UnknownHostException: wpspctr

● Add wpspctr to your /etc/hosts file so the PD can find your Process Center.

34© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Connecting to the Process Center

� Servers with the same TCPIP HostName & ServerName appear as the same Server.

– even if they have unique dotted IP addresses, and Port #s.

� That's why we defined each AppServer Node with the unique TCPIP

Host IP Address:

37© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Other tools & Techniques

� Tracing the Connection to the PC Repository:

F b1sr01b,tracejava='com.ibm.ws.repository.*=all=enabled'

� Clearing the osgi cache:/wasv8config/b1cell/b1nodeb/AppServer/bin-> ./osgiCfgInit.sh

OSGi cache successfully cleaned for /wasv8config/b1cell/b1nodeb/AppServer/profiles/default.

/wasv8config/b1cell/b1dmnode/DeploymentManager/bin-> ./osgiCfgInit.sh

OSGi cache successfully cleaned for /wasv8config/b1cell/b1dmnode/DeploymentManager/profiles/default.

38© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Security Errors – LTPA Tokens

Don't worry about these messages:

SECJ0371W: Validation of the LTPA token failed because the token

expired with the following info:

Token expiration Date: Thu Jan 26 13:03:00 EST 2012,

current Date: Thu Jan 26 15:00:33 EST 2012.

● The LTPA tokens expire 2 hours after they are created.

● They are created for users when they login to an app or the admin console.

● Most of the LTPA token expired messages are due to the servers, which use

them as credentials when they talk JMX.

● Time stamps on the server userid related LTPA token expired messages are

120 minutes apart.

● They are not serious and there's nothing you can do about them.

43© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Tracing in the Process Center

� In the ProcessCenter ICS, go to: Troubleshooting > Logs and Traces > pcsr01a > Change log detail levels

– Set it to something like: *=info: WLE.*=all

� To view the Traces & Logs: In the ISC: Troubleshooting > Logs and Traces

● Diagnostic Trace – Change trace settings

● JVM Logs – SystemOut.log,

SystemErr.log

● Process Logs – native_stdout.log,

native_stderr.log

● IBM Service Logs - activity.log

● Change Log Level Details

Server logs:

AppServer: /opt/IBM/WebSphere/BPM/profiles/PCEMPTYA/logs/pcsr01a/SystemOut.log

Dmgr: /opt/IBM/WebSphere/AppServer/profiles/PCDmgr01/logs/dmgr/SystemOut.log

51© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

IBM Support Assistant (ISA)• Contains many Diagnostic tools for debugging (mostly Java)

– ibm.com/software/support/isa/

– ibm.com/developerworks/websphere/techjournal/0906_supauth/0906_supauth.html

� Health Center

� Dump Analyzer, Heap Analyzer, Log Analyzer

� Garbage Collection and Memory Visualizer (GCMV)

� Memory Dump Diagnostic for Java (MDD4J)

� IBM Pattern Modeling and Analysis Tool for Java Garbage Collector (PMAT)

� IBM Thread and Monitor Dump Analyzer for Java (TMDA)

� IBM Trace and Request Analyzer for WebSphere Application Server (TRA)

� Log Analyzer

� Visual Configuration Explorer

See these Techdocs:

� WP101575 - “Introducing the ISA on z/OS”

� WP101612 - “Getting started - analysis GC, Heapdumps & Javacores”

52© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Which Tools for which Artifacts?

Artifact Tool

GCverbose xml (SYSOUT) GCMV

PMAT

Javacore TMDA

Heapdump Memory Analyzer

MDD4J

HeapAnalyzer

System Dump (jextracted) Memory Analyzer

Dump Analyzer

Logs & Traces TRA

Log Analyzer

Real Time Running Server Health Center

Configuration XML files VCE

53© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

PD DocumentationInfoCenter: Troubleshooting and support� Overview and New features

� Troubleshooting checklist for Business Process Manager

� Debugging applications

� Failed to log in to Process Center

� Troubleshooting administration tasks and tools

� Tools for troubleshooting your applications

� Recovering from a failure

� Disaster recovery

� Getting fixes

� Contacting IBM Software Support

Redbooks & Redpapers:� Problem Determination for WebSphere for z/OS SG24-6880� Problem Determination Methodology for WebSphere REDP6001� Problem Symptoms in WebSphere for z/OS REDP6002 � Problem Avoidance for WebSphere App. Server REDP6003� WAS for z/OS Problem Determination Means & Tools REDP6880

55© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

ND Topologies(see IEA foils)

Clustering basics: ● High availability

– The ability to always have the applications available for client applications

� Scalability

– The ability to grow the capacity based on business needs

• Workload management

• Improved resource utilization

• Easier administration

Horizontal

Ve

rtic

al

58© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Administration and Operations

� Installing Applications

� Starting & Stopping Servers

� Backing up HFSes & DB2 tables

� Recovering from outages

� The administrative console

� Command-line tools, scripts, and programming interfaces

� BPC Explorer and Business Process Archive Explorer

� Administering processes applications

� Deploying modules directly to IBM Business Process Manager

� Administering the IT infrastructure

� Troubleshooting administration

62© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Backing up and Restoring zFS with DFDSS

Backup -//STEP1 EXEC PGM=ADRDSSU,REGION=4M

//SYSPRINT DD SYSOUT=*

//OUTDD1 DD DSN=USER1.Z8CELL.BKUP.CONFIG,DISP=(NEW,CATLG),

// SPACE=(CYL,(180,50),RLSE),UNIT=(3390)

//SYSIN DD *

DUMP DATASET(INCLUDE(OMVS.WAS700.Z8CELL.Z8NODEA.CONFIG.ZFS)) -

OUTDDNAME(OUTDD1) CANCELERROR TOL(ENQF) OPTIMIZE(1) WAIT(2,2)

Restoring - (First unmount your zFSes)

//UNMNT EXEC PGM=IKJEFT01,DYNAMNBR=20,REGION=2M

//SYSTSPRT DD SYSOUT=*

//SYSTSIN DD *

UNMOUNT FILESYSTEM('OMVS.WAS700.Z8CELL.Z8NODEA.CONFIG.ZFS') IMMEDIATE

//RESTORE EXEC PGM=ADRDSSU

//SYSPRINT DD SYSOUT=*

//INODEA DD DISP=SHR,DSN=USER1.Z8CELL.BKUP.CONFIG

//SYSIN DD *

RESTORE DATASET(INCLUDE( OMVS.WAS700.Z8CELL.Z8NODEA.CONFIG.ZFS )) -

INDD(INODEA) REPLACE TGTALLOC(SOURCE) TOL(ENQF)

//MOUNT EXEC PGM=IKJEFT01,DYNAMNBR=20,REGION=2M

//SYSTSPRT DD SYSOUT=*

//SYSTSIN DD *

MOUNT FILESYSTEM('OMVS.WAS700.Z8CELL.Z8NODEA.CONFIG.ZFS') TYPE(HFS) -

MOUNTPOINT('/wasv6config/Z8cell/z8nodea')

Don't forget RACF changes, Procs, DB2 Tables for WPS, etc.

63© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Performance

� “Prescriptive Use Cases” - Co-Location Update Comparison

� z/OS Measurement & Tuning Tools:

– RMF, SDSF

– SMF Records

– WLM Tools

� Java Diagnostics

� Recommendations specific to BPM

67© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Tuning Checklist� Tracing and logging overhead - Minimize� DB2 Database tuning� Java tuning (Heap Size & Garbage Collection Policy)� Workload Profile� Messaging and Message Bindings � Workload Manager Service Class� MDB ActivationSpec� Thread & JMS Connection Pool Sizes� DataSource Connection Pool Size & Prepared Statement Cache Size� Messaging Engine Properties� Minimize security overhead� Disable automatic synchronization for ND� Run production servers in production mode� Large Objects� Maximum Concurrency� Messaging Tuning� Web Services Tuning� WebSphere MQ Tuning� Business Process Choreographer Tuning� Macro vs. Micro-flows

73© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Migration Termiology

• Migration vs. Upgrade

– Migration is 'side by side' and refers to movement of a group of objects

from one installation environment to another

• Required when delivering major new features, and profile updates

• Applies to 1st and 2nd digit release number changes, e.g. 6.1->6.2

• The new version must be installed side-by-side the old version of the

product

– Upgrade is 'in-place' and refers to replacement of a product with a newer

release of that same product

• Delivers updates or fixes to the existing components

• Applies to 3rd and 4th digit release number changes (interim fixes )

• The new release is installed on top of the existing installation.

74© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Migration Patterns - Three approaches:

● Runtime migration

➔ Install new version

➔ Migrate runtime configuration, applications and data using migration tools

● Manual migration

➔ Install new version

➔ Manually configure new deployment infrastructure and databases

➔ Manually re-install applications on new infrastructure

● Artifact migration

➔ Migrate applications to new version using authoring tools (WID, etc.)

➔ Install new version

➔ Manually configure new deployment infrastructure and databases

➔ Manually re-install applications on new infrastructure

● IBM Education Assistant:

➔ http://publib.boulder.ibm.com/infocenter/ieduasst/v1r1m0/index.jsp?

topic=/com.ibm.iea.ibpmgr/ibpmgr/7.5/Migration/BPMv75_WPSMigration/player.html

75© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Migration Method Considerations

Several different issues to consider when determining the

right migration method:● Production data

● Downtime

● Long-running processes and human tasks

● Application enhancements

● Target environment configuration

● Risk mitigation

● Selective or phased application migration

76© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Migration Tools

BPM migration command-line utilities

– BPMConfigureProcessServer

– BPMCreateRemoteMigrationUtilities

– BPMCreateTargetProfile

– BPMGenerateUpgradeSchemaScripts

– BPMMigrateProfile

– BPMMigrationStatus

– BPMQueryDeploymentConfiguration

– BPMSnapshotSourceProfile

– upgradeSchema (Common Database)

– BackupConfig

– MigrationDisablementReversal . . . . .

78© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Tools & Other Resources

� Tools

� Resources

� Other . . .

79© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Running shell scripts

� Running scripts in batch jobs with BPXBATCH or JZOS

� Running scripts without interruption in telnet

– nohup sh -c './xxx.sh ' >/tmp.xxx.out' 2>&1

� Halting scripts started with 'nohup':

– kill -9 <pid>

� Tracing shell scripts

– Insert 'set -x” at the beginning, or use this:

– sh -x './xxxsh' >/tmp.xxx.out' 2>&1

� wsadmin Tracing

– Uncomment this line in the

{profile_root}/properties/wsadmin.properties file

#com.ibm.ws.scripting.traceString=com.ibm.*=all=enabled

80© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

Using Wide 3270 ScreensHelpful in looking at JES Logs, SDSF, RMF Reports, and anything wide:

� Fix PCOM profiles for good keyboard and char-set mapping to display brackets & braces:

� Wide WG31.WS - Large screen 50 x 132

– TCPIP profile for large screen – in DSN=SYS1.TCPPARMS(PROFILEx)

– TELNETDEVICE DYNAMIC ,D4C32XX3

– (See Techdoc TD102151 “Creating dynamic 3270 screen size definitions for increased productivity”)

81© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

BPM V8.5 Library and other Resourceshttp://www.ibm.com/software/integration/business-process-manager/library/documentation/

● InfoCenter: http://pic.dhe.ibm.com/infocenter/dmndhelp/v8r5m0/index.jsp

● Quick Start Guides

● Product Documentation (PDFs)

● Tutorials (Hiring Sample)

● Announcements

84© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

ISC, PC, PD, IID Help

� Click on the “Help” or “?” button at the top of the console:

This brings up Specific Help Panels for the tool you are working with.

85© 2012 IBM CorporationIBM Americas Advanced Technical SkillsWashington Systems Center, Gaithersburg, MD

BPM:PS V8 z/OS Workshop Review

You've had an opportunity to work with the following areas in BPM: Process Server on z/OS:

1. Intro to IBM BPM

2. Review WAS ND on z/OS environment

3. Implementing BPM:PS for z/OS in a WAS ND env. + Lab

4. Configuring Process Center

5. Using Process Center, Process Designer & IBM Integration Designer

6. Overview of New functions in BPM Version 8

7. Problem Determination, Administration, Performance

Which ones were worth-while for you?

Which ones were not?What else would you like added?