ims datapower - ims ug phoenix 12-2013
TRANSCRIPT
© 2012 IBM Corporation
IMS and DataPower
Shyh-Mei F. Ho IBM Distinguished Engineer
[email protected] IMS On Demand SOA Chief Architect
SVL, San Jose, CA. USA
© 2012 IBM Corporation
Public Cloud
IBM Cast Iron
Synching data with SaaS apps to leverage new cloud economy
Private Cloud
BPM
WAS DB
DBCICS
DB2
SAP
.JCAPs
.NETPattern
Pattern
IMS
ODM
DB
IBM Integration portfolio
Mobile
Internet of Things
Trading partner communities
DeveloperCommunities
IBM Message Broker
Integration Bus provides universal connectivity for heterogeneous environments across enterprise processes, applications, and data
IBM MQ messaging
Messaging backbone provides reliable transport and data delivery across data center
IBM PureApplication System
Enterprises looking to achieve “more with less” by better managing IT resources as collectives
IBM Worklight
Productive multi-device development and management
IBM Web API Management
Manage your APIs to open up access encouraging innovation from App Developers
IBM MQTT
Reliable, efficient, scalable messaging for mobiles and sensors
DMZ DMZ
IBM DataPower Gateway Appliance
B2B Integration Gateway for secure collaboration with communities of trading partners
IBM Caching Appliance
Cache grids improve scale and performance
IBM DataPower Gateway Appliance
Integration Gateway for secure & controlled access to enterprise resources, while optimizing workload delivery
© 2012 IBM Corporation
WebSphere DataPower Gateway Appliances
Security & Integration Gateway Appliances
Internet Trusted Domain
Consumer
Application or Service
DMZ
DataPower DataPower
© 2012 IBM Corporation
DataPower Appliances Benefits
� Reduce Complexity: Replace software servers functionality with DataPower
Appliances, reduce infrastructure footprint, and off-load systems intensive processes.
� Lower TCO: DataPower Appliances have demonstrated reducing operational costs by
as much as 50%
� Reduce Time to Market: DataPower Appliances dramatically decrease the testing time
and amount of development required to upgrade your environment, most policy are
configuration driven as opposed to development driven
� Reduce Risk: DataPower Appliances provide the communication layer without requiring
application modification, and deliver improved security and audit
� Flexibility & Security: DataPower Appliances shield business applications from security
requirements, protocol changes and service versioning - no application modifications
needed
© 2012 IBM Corporation
Configuration-driven approach speeds time to market
� Uses intuitive pipeline message processing to secure, control, optimize, & integrate
services and application traffic
� Enforce security standards with zero coding
� Import/export configurations between environments
� Transaction probe shows message content between actions for debugging
© 2012 IBM Corporation
DataPower: Mainframe integrationOffload processing for reduced MIPS
Web Services Enablement for IMS, DB2, CICS
© 2012 IBM Corporation
Premier System z Services enablement through DataPower as Ubiquitous Gateway
IBM cross-brand initiative
� Corporative synergy between DataPower, System z, including IMS, CICS and DB2, to support DataPower as the premier System z gateway
–Position DataPower for mainframe with a complete range of connectivity and integration to all databases, in addition to all transactions
DataPower Processing Flow
Protocol transformation
WS Monitoring
Native z transports
Transaction distribution
Network-level HA and distribution
RACF-centric security
IMS
DB/2
CICS
© 2012 IBM Corporation8 IBM Confidential5 December 2013
IMS Integration with WebSphere DataPower
ServiceOriginator
IMS
Connect
Client
IMS
OTMA
IMS Application
IMS
Connect
SOAP/HTTP
Z Service Provider
� MQBridge to drive IMS transactions
� MQ client is embedded in DataPower
� IMS Connect to drive IMS transactions (inbound requests)
� IMS Connect client in DataPower natively connects to IMS Connect
• Inbound support only
• Commit mode 1, Sync Level NONE or Confirm
TCP wire protocol
Cobol/MQMQ Client
MQ Server
MQ
Brdg
© 2012 IBM Corporation
IMS Integration with WebSphere DataPower +
WebSphere DataPower 6.0 (GAed on 6/28/2013)
IMS Database integration
IMS Transaction integration
IMS Callout to DataPower is a leadership project
Many WW customers express interest in IMS Callout
DataPower Processing Flow
Protocol transformation
WS Monitoring
Native z transports
Transaction distribution
Network-level HA and distribution
RACF-centric security
IMS
New
New
© 2012 IBM Corporation
Recent DataPower deliverables for System z
1. IMS Callout – Allows applications running in IMS to make service calls to external web service providers using the existing IMS ICAL interface [DP v6.0]
2. IMS DB – Provides a new information-as-a-service interface for service consumers to access IMS databases using standard SQL (requires DataPower’s Database Connectivity feature) [DP v6.0]
3. DB2 WLM – Provides support for real-time load distribution of SQL calls from DataPower to Sysplex-aware DB2 instances running on z/OS [DP v6.0]
4. XI50z firmware upgrade – Provide support for DataPower firmware level v5.0 [1Q13]
IMS
CICS
DB2
1
2
3
4
Business Integration
© 2012 IBM Corporation
IBM WebSphere DataPower Family
Integration Appliance XI52� High density 2U form
� “Any-to-Any” conversion at wire-speed
� Bridges multiple transport protocols
� Mainframe integration & enablement
Service Gateway XG45� Entry-level device, slim footprint (1U)
� Security gateway (AAA, XML threat, etc)
� Service level management and monitoring
� Intelligent load distribution & dynamic routing
� Lightweight integration functions (optional module)
B2B Appliance XB62� High density 2U form
� B2B Messaging (AS1/AS2/AS3/ebMS)
� Trading Partner Profile Management
� B2B Transaction Viewer
Integration Blade XI50B/XI50z� Functionally equivalent to XI52
� Form factor flexibility
� XI50B: BladeCenter form factor
� XI50z: zEnterprise BladeCenter Extension (zBX) form factor
© 2012 IBM Corporation
DataPower Models for IMS solutions
XI52, XI50B, XB62, XG45 IMS DB support
XI52, XI50B, XI50Z, XB62IMS TM provider support
XI52, XI50B, XB62IMS TM synchronous callout support
DataPower Models supporting 6.0 release
WebSphere DataPower V6.0:
More capabilities planned for future DataPower releases
New
New
New
© 2012 IBM Corporation
DataPower Processing Flow
Protocol transformation
WS Monitoring
Network-level HA and distribution
RACF-centric security
IMS
IMS
Connect
Direct Access IMS database via DataPower
ODBM
ODBM
IMSDB
� IMS Open Database offers direct access to IMS database resources anywhere in the
IMSplex from z/OS and distributed environments– Support different APIs to leverage Distributed Relational Database Architecture (DRDA)
• IMS universal DB resource adapter to support J2EE, e.g. WebSphere
• IMS universal JDBC driver to make SQL calls
• IMS universal DL/I driver
– Open Database Manger (ODBM) works together with IMS Connect as a DRDA server for IMS
data
� DataPower to access IMS database directly via the Open Database capability, i.e.
via IMS Connect and ODBM– An IMS database is defined to DataPower as an SQL data source. For each IMS database that
you will access, you need to configure a separate SQL data source
New
© 2012 IBM Corporation
IMS Synchronous Callout with DataPower 6.0
� IMS synchronously go outbound to external server via DataPower− Implement IMS synchronous callout protocol− Support the common container model architecture with IMS Callout Front-side Handlers that retrieve IMS callout messages and send response data
� Using existing WTX Design Studio tooling and/or XSLT stylesheet for data transformation
� The handler internally creates one or more IMS Connect dedicated persistent socket connections to the host system, using Enterprise Suite V2.2 IMS Connect API in Java.
� The handler communicates with IMS Connect via a new DataPower dedicated user message exit, HWSDPWR1. – For shared queue environment, user can choose to create multiple IMS Callout connections, one for each IMS datastore.
Core Engine Front-side
Handler
IMS IMS Application
IMS
Connect
Front-side
New
© 2012 IBM Corporation15
Prerequisites for IMS synchronous callout
� Software requirements
� IMS V12 (IMS V13 is recommended)
– IMS Connect
– OTMA
� IBM WebSphere DataPower Firmware 6.0.0.0 or higher
� Hardware requirements
- IBM WebSphere DataPower appliance XI52, XI50B, XB62
� Tooling
� WebSphere Transformation Extender (WTX)
Provides mapping between different data formats.
� WTX maps can be built as deployable artifact for DataPower, providing data
transformation between IMS callout bytes and XML data for web services.
� A WTX map can be set using a DataPower-specified variable, then called
within XSL code in a DataPower policy.
IBM Confidential
© 2012 IBM Corporation
IMS Synchronous Callout DataPower & IMS Versions
� IMS V12
� IMS Connect
� DataPower User Exit Installation - Object Code Only user exit HWSDPWR1(new)Specified in the EXIT= parameter of the TCP/IP statement in the IMS Connect configuration file (HWSCFGxx).
� V13 PTF UK97704 & PTF UK97704� V12 PTF UK91544
� OTMA
� IMS Synch Callout user can specify a 1-to-8 byte mapname as the first 8 bytes in AIBUTKN so that this ID can be included in the OTMA state data in the callout message. The ID can be used as a unique service identifier for data transformation mapping and service routing
� V13 (available as base code)
� V12 PTF UK82636 (PM73135) AIB MAP name field
� IMS V13
� IMS Connect: Socket listening redesign
© 2012 IBM Corporation17
Prerequisites for IMS DB
� Software requirements
� IMS V12, IMS Catalog, ODBM and SCI
– IMS Catalog to access to metadata of IMS programs and databases
resources .
– IMS Connect
� ODACCESS statement in HWSCFGxx member of a concatenated PROCLIB data set
� IBM WebSphere DataPower Firmware 6.0.0.0 or higher
� Hardware requirements
� IBM WebSphere DataPower appliance XI52, XI50B, XB62, XG45
IBM Confidential
© 2012 IBM Corporation18
External Details
IBM Confidential
© 2012 IBM Corporation19
Synchronous Callout Solution Highlights
• The IMS callout connection is a DataPower “Front SideHandler” that can retrieve IMS callout messages and sendresponse data.
• The handler internally creates one or more IMS Connect dedicatedpersistent socket connections to the host system, using Enterprise Suite V2.2 IMS Connect API in Java.
• The handler communicates with IMS Connect via a new DataPowerdedicated user message exit, HWSDPWR1.
• For shared queue environment, user can choose to create multipleIMS Callout connections, one for each IMS datastore.
IBM Confidential
© 2012 IBM Corporation20
IMS Synchrouns Callout - DataPower Solution Highlights
IMS ApplicationIMS Application
ICAL A (Xid*)
ICAL B (Yid*)
ICAL C (Zid*)
ICAL A (Xid*)
ICAL B (Yid*)
ICAL C (Zid*)
* 8 bytes unique service id* 8 bytes unique service id
Each descriptor:
- Member, TPIPE- Adapter, Converter
- Timeout
Each descriptor:
- Member, TPIPE- Adapter, Converter
- Timeout
Destination
Routing
Descriptor
Destination
Routing
Descriptor
OTMAOTMA
IMS
Connect
IMS
Connect
DataPowerDataPower
resume
TPIPE
bytes
IMS
External
Service
X
Y
Z
External
Service
X
Y
Z
request bytes
with service id
response
bytes
COBOL copybook/
PLI import
for
ICAL request/response
A, B, C
COBOL copybook/
PLI import
for
ICAL request/response
A, B, C
WebSphere
Transformation
Extender (WTX)
WebSphere
Transformation
Extender (WTX)
Schema
from
services
X, Y, Z
Schema
from
services
X, Y, Z
deploy
import /
generate
type
tree
request/
response
In XML
IMS Callout
front side
handler
IMS Callout
front side
handler
WTX generated
Maps for
A-X, B-Y, C-Z
WTX generated
Maps for
A-X, B-Y, C-Z
WTX generated
Maps for
A-X, B-Y, C-Z
WTX generated
Maps for
A-X, B-Y, C-Z
WTX generated
Maps for
A-X, B-Y, C-Z
WTX generated
Maps for
A-X, B-Y, C-Z
Bottom-up or meet-in-middle
mapping
Transform rule: (user
defined XSL can
select a map based
on service id)
Transform rule: (user
defined XSL can
select a map based
on service id)
© 2012 IBM Corporation
DataPower: most policy are configuration driven via browser-based Web GUI
© 2012 IBM Corporation
DataPower Browser-based Web GUI
DP Developer / Admin
Create Service Gateways
© 2012 IBM Corporation23
Configure DataPower for IMS Synchronous Callout Requests
1. Configure the Multi-Protocol Gateway (MPG)2. Configure IMS Callout Front Side Handler.3. Configure the connection between DataPower and the backend external service provider.
4. Define a MPG “Policy”. Create one or more “Rule”(s) for the Policy
– Define the MPG processing policies and rules that determine the actions that DataPower takes the callout requests and responses that it handles.
5. Apply the changes, and save the configuration
IBM Confidential
© 2012 IBM Corporation
Multi-Protocol Gateway
� A Multi-Protocol Gateway (MPG) connects client requests that are transported over one
or more protocols to a remote destination that uses the same or a different protocol.
– MPG supports the FTP, HTTP, HTTPS, IMS™, MQ, NFS, SFTP, TIBCO EMS, and
WebSphere® JMS protocols.
– However, MPG cannot use a WSDL to determine the configuration.
� A MPG includes following capabilities
– Implement Reliable Messaging policies
– Implement WS-Addressing protocol enforcement
– Accept and send SOAP, raw XML, or unprocessed (binary) documents
– Transform XML to binary format documents and binary format documents to ML
– Filter, validate, transform, encrypt, or decrypt XML documents
– Route XML documents
– Sign documents or verify signatures
– Process large documents in the streaming mode
– Implement document-level security or service-level security
– Communicate with clients, servers, and peers with SSL encryption
– Monitor and control data traffic based on request sources and requested resources
– Allow, reject, strip, or process attachments (MIME, DIME, MTOM)
© 2012 IBM Corporation25
DataPower Web GUI Control Panel
IBM Confidential
© 2012 IBM Corporation26
Configure Multi-Protocol Gateway
IBM Confidential
© 2012 IBM Corporation27
Configure IMS Callout Front Side Handler
� One or more IMS Callout Front Side Handlers can be specified in a single MPG.– For each handler, one or more TPIPEs can be specified on the same IMS Connect host and port, and same
IMS data store.
� Specify IMS system parameters to configure the handler
IBM Confidential
© 2012 IBM Corporation
Define a Processing Policy for MPG
� A processing policy defines many, if not all, of the actions that are taken against the messages that
pass through the Multi-Protocol Gateway service.
– A processing policy consists of one or more rules.
– A rule consists of a matching rule and a processing rule.
– A matching rule defines the criteria to determine whether incoming traffic is processed by its
processing rule.
– A processing rule identifies the actions to perform against the incoming traffic.
� To access the configuration panel for defining processing policies, from the Configure Multi-Protocol
Gateway panel, click on the “+” button under Multi-Protocol Gateway Policy.
© 2012 IBM Corporation
Configure a Matching Rule & Match Actions
� In the Configure Multi-Protocol Gateway Style Policy panel, a rule is depicted as a line with symbols
on it.
– Each rule consists of a Matching Rule, which determines whether or not to process the incoming
data, a Results Action, and one or more processing actions in between.
– Each rule can be configured to flow from client to server, vice-versa, or in both directions.
� A matching rule determines whether and how to process incoming data.
– A matching rule is represented by a Match Action icon, .
– A Match Action icon is automatically placed on the rule line when you create your policy rule
� Double click on the Match Action icon to define the match rule. Click on the “+” button to add a new
Matching Rule; click on “Y” to edit an existing one.
© 2012 IBM Corporation
Configure a Transform Action (a map or XSL Stylesheet-driven Action)
� A Transform Action transforms a message from one format to another format
– For example: from COBOL byte arrays of the copybook of an IMS application program to XML
schema used by external service provider
� The Transform Action requires either a WTX map artifact or a stylesheet that maps the data
between the two formats.
– A stylesheet can also be used to select between multiple WTX maps.
� To add a Transform Action to the processing rule in the MPG Style Policy panel, drag the
“Transform action” icon, , onto the rule line right after the matching action .
� To configure a Results Action, drag the Results action icon, , onto the end of the line that
represents your processing rule.
� Double click on the Action icon on the line to configure the Rule.
© 2012 IBM Corporation31
Configure Multi-Protocol Gateway Style Policy
IBM Confidential
© 2012 IBM Corporation
Data Power Configuration in supporting IMS Callout
© 2012 IBM Corporation33
Operational Considerations
� Operational Characteristics
– DataPower administrator can configure an IMS Callout front side handler with the
following properties: IMSHost, IMSPort, DataStoreName, TPipe(s), UserID, Password,
Group, RetryErrorLimit, RetryInterval, Connection Timeout
– DataPower administrator can enable/disable an IMS Callout front side handler
– IMS Callout Message Header
IMS Callout Front side handler sets the two headers in the request to DataPower:
ims-callout-service-id: IMS ICAL AIBMAPNM field
ims-callout-correlation-token: Hex representation of ICAL correlation token
DataPower administrator can define the XSL in the transform policy to access the header fields in the MPG policy, e.g.
� service ID as the request identifier to select input/output transformation map � correlation token as the message ID in the outbound HTTP/SOAP request.
IBM Confidential
© 2012 IBM Corporation
� Operational Considerations
–An IMS Callout front side handler has the following “opstates”:
• Up: resume tpipe processing is in operation
• Down: no active processing
• Pending: in recovery mode (detected IMS Connect and/or IMS outage)
� Operational Recommendations
–When not in use, the administrator should disable the IMS Callout front side handler
Operational Considerations +
© 2012 IBM Corporation
Error Handling
� Error can occur during different stages of DataPower processing:– DataPower MPG inbound/outbound policy (including WTX map transformation)– Backend service
� Error response sent to IMS application– Return code and Reason code: X’0100’– Extended reason code: 2000-3000 (set as 2000+HTTP error code from DataPower)– Error message: REQUEST PROCESSING FAILED, CHECK EXTENDED REASON CODE.“
� Error retries– When encounters error during resume tpipe, administrator can configure the following to retry before the front side handler stops processing:• RetryErrorLimit: Number of time to attempt to resume a TPIP. Default 5.• RetryInterval: Interval to wait before attempting to resume the TPIPE. Default 3 seconds
� Self-Recovery support:– Once the IMS Callout front side handler goes down after reaching the max retries, if the error is due to network problem, the front side handler attempts to self-recover. A /DISPLAY OTMA command is issued every min to verify network connectivity and IMS availability. Once the command is successful, DataPower brings up the front side handler to resume operation.
� Non-operative Auto Detection support– DataPower detects when an IMS front side handler becomes non-operative and restarts the processing automatically.
© 2012 IBM Corporation36
Performance Considerations
� Performance Characteristics
– New IMS Callout code has no impact on IMS or IMS Connect performance
– Initial studies in a simplified environment show that the solution is capable of
processing 8k bytes request/response messages, passing through 10 OTMA
TPIPEs, with ICAL Response Time around 30 msec.
The workload generated for this particular test case was 800-1200 TPS
� Performance Considerations
– Adding TPIPEs can improve throughputY up to a certain point.
IBM Confidential
© 2012 IBM Corporation37 IBM Confidential
High Level Diagnostics
� Documentation to collect to diagnose a problem
� IMS/ICON Side
– IMS Recorder trace
– /DIS TMEMBER hws1 TPIPE tpipe1 SYNC.
– F ICON1,QUERY PORT NAME(5555) SHOW(ALL)
� DataPower Side
– Object Status Page
– FSH and MPG System Log
– Create a Log Target “IMS” is recommended
– IMS Callout Trace messages
Note: If an error occurs, make sure the problem is recreated with the DP
trace/logs on and make them available to the tech support.
© 2012 IBM Corporation38 IBM Confidential
High Level Diagnostics: Documentation to collect to diagnose a problem
� Object Status ExampleI locating
© 2012 IBM Corporation39 IBM Confidential
High Level Diagnostics: Documentation to collect to diagnose a problem
� Object Status Example
© 2012 IBM Corporation40 IBM Confidential
High Level Diagnostics: Documentation to collect to diagnose a problem
� Example of IMS Front Side Handler log
© 2012 IBM Corporation41 IBM Confidential
High Level Diagnostics: Documentation to collect to diagnose a problem
� IMS Callout Trace in DataPower. Example:
© 2012 IBM Corporation
Troubleshooting
© 2012 IBM Corporation43 IBM Confidential
Configure a Log Target for IMS Callout
� Set up a Log Target for “IMS” – Add a Log Category
© 2012 IBM Corporation
Configure a Log Target for IMS Callout +
� Configure a Log Target for “IMS” callout
© 2012 IBM Corporation45 IBM Confidential
Configure a Log Target for IMS Callout +
� Configure a Log Target for “IMS” callout – Event subscription
© 2012 IBM Corporation
Thanks