unit 3 - zos connect · unit 3 – z/os connect unit 3 - 2 © 2014 ibm corporation ibm americas...

Unit 1a - Overview

Unit 1a - 1

© 2014 IBM CorporationIBM Advanced Technical Skills

ZCONN1WebSphere Application Server Liberty Profile z/OS

z/OS Connect

WebSphere Application Server Liberty Profile z/OS

Unit 3 – z/OS Connect


Unit 3 - 2

© 2014 IBM CorporationIBM Americas Advanced Technical Skills

Gaithersburg, MD2

This page intentionally left blank


Unit 3 - 3


Gaithersburg, MD3

z/OS Connect at a High LevelThis is the picure we showed back in Unit 1:

Extensible …* By “batch” we mean a long-running job that uses the WOLA “host a service” API to listen for calls coming over from z/OS Connect

Liberty Profile z/OS

z/OS Connect

CICSCICS

IMSIMS

Batch*Batch*

Anything that supports REST/JSON

Data Conversion

Audit(SMF)

DiscoveryAccessControl

This can be IBM MobileFirst Platform, some other mid-tier

device, or even other mainframe programs

Configuration XML file

This topic we'll leave for the unit on securityGreen starts indicate

topics covered in this unit

Now we come to a high-level description of what z/OS Connect is. We've hinted at it several times in the earlier

discussion, but here we'll start the more direct discussion of it.

At its heart z/OS Connect is an application that runs inside a Liberty Profile z/OS server instance. The z/OS

Connect application (it's a servlet) provides the framework to handle REST and JSON sent by clients, and then to

invoke the backend program or service associated with the REST request. The relationship between REST

request and backend service is defined in the Liberty Profile configuration XML file. That's where you indicate

what URI patterns are to be recognized, and then what should be done when that URI pattern is received.

The backend programs or services that z/OS Connect is capable of communicating with is CICS, IMS and Batch.

By “batch” we mean a long running COBOL (or C/C++, PLI, or High Level Assembler) program that is “listening”

for a call from z/OS Connect.

In addition, z/OS Connect provides four functions within its framework – the ability to perform data conversion from

JSON to whatever the backend data format is; auditing using SMF records; a set of REST APIs to provide

information and details on the configured services; and the ability to configure granular access control so users

have defined limited access.

There's quite a bit of detail we've left unsaid here, of course. We have a whole unit on this where we'll get into that

discussion.


Unit 3 - 4


Gaithersburg, MD4

z/OS Connect is ExtensibleInfoCenter search string: twlp_admin_extend

HA …

A “service provider” is the mechanism used to reach the

backend. The built-in sevice provider is WOLA. You may add your own.

An “interceptor” is function called for each request. The built-in

interceptors are audit (SMF) and authorization. You may add your

own to do custom function.

A “data transformer” is code that transforms JSON to the format required by the backend. The built-in transformer is based on the CICS utility to transform JSON to COBOL COPYBOOK or other

language data structure.

When looking at z/OS Connect it's important to realize that it is an extensible framework. As delivered by IBM it

has certain functions coded, but if you want to extend z/OS Connect to have custom functionality, you can do it.

This is built on the capability of Liberty Profile itself to be extended with user-functions.

For z/OS Connect there are three areas that can be extended:

● Service Provider – this is the function that is used by z/OS Connect and Liberty to talk to the backend

system. The built-in service provider that comes with z/OS Connect is WOLA. That can be used to

communicate with CICS and long-running COBOL (or C/C++, PL/I or High Level Assembler). The IMS

support for z/OS Connect doesn't use WOLA; it uses a custom service provider the IMS people wrote to talk

to IMS Connect. That's an example of someone (IBM in this case) extending z/OS Connect with an

additional service provider.

● Interceptor – this is function that is called (if configured; this is optional) every time a service is invoked. The

built-in interceptors are for audit (SMF 120.11) and authorization (more granular control over who can use the

REST APIs vs. simply invoke services). If you want to have you own custom inteceptor, you can code it and

configure it with a service. (Note: interceptors can get a copy of the request data, but they can't change the

request payload.)

● Data Transformer – this is function that transforms the request JSON to the data format required by the

backend, and transform back to JSON when a response is returned. The built-in data transformation function

is based on the utility CICS provided for its web services function. That utility takes as input COBOL

COPYBOOK (or other programming language data structure) and provides the “bind” files and JSON schema

needed to do the transformation. If you have a different data transformation requirement, you can extend

z/OS Connect to do your own custom data transformation.

Information on how to do this is provided in the Knowledge Center under the search string shown at the top of the

chart.


Unit 3 - 5


Gaithersburg, MD5

z/OS Connect Can Be Made Highly AvailableLiberty Profile can be made highly available through duplicated server instances. z/OS Connect makes this relatively simple because it's stateless by nature:

Setup …


z/OS Connect

CICSCICS

IMSIMS

BatchBatch


z/OS Connect

CICSCICS

IMSIMS

BatchBatch

HTTPDistribution server.xml

Instances of Liberty Profile on different LPARs can

share a server.xml and be mirror copies of each other

Or have separate copies that are maintained to be very consistent with one another

REST is a stateless mechanism, so no need to

affinity route

If backend systems are present on each LPAR then environment is duplicated

and HA possible

When thinking about z/OS Connect as a 'gateway' into the mainframe for services, it's natural to start thinking

about how it might be made highly available. The first step in high availability is duplication of resources. Liberty

Profile and z/OS Connect can be duplicated fairly easily, and the nature of REST keeps it easy.

Two (or more) instances of Liberty Profile z/OS can share a copy of the server.xml using the Parallel Sysplex

shared file system facility. If you do that, then the two instances of Liberty Profile (and z/OS Connect running

inside) can be identical. They're separate physical instances running on two separate logical partitions, but in their

configuration and behavior they are the same.

Note: it is not necessary to use Liberty “collectives” for this. You may if you wish, but it is not required.

The nature of REST helps with this because REST is stateless. That means there is no requirement a client go

back to a particular instance of Liberty Profile for its calls. The first call from a client can go to one instance of

Liberty Profile, the second call to the other, and so on. Each REST call stands on its own, without a requirement

to maintain any state information about the previous call.

It's important to think about the backend systems being called by z/OS Connect, because ultimately that's what the

client is going after. If you configure up a duplicated instance of Liberty Profile with z/OS Connect, you need to

make sure the backend systems connected to by z/OS Connect are also duplicated. z/OS Parallel Sysplex helps

with this since instances of CICS or IMS on each LPAR can be made Sysplex aware and use shared data.

The key message is this – if having a highly available REST/JSON interface to your mainframe environment is

important, it is possible to duplicate instances of Liberty Profile and z/OS Connect to achieve this.


Unit 3 - 6


Gaithersburg, MD6

SetupHow to do initial enablement of z/OS Connect


Unit 3 - 7


Gaithersburg, MD7

z/OS Connect and WOLA are Features of LibertyAnd as such, they need to be enabled. This is done with the the featureManager shell script, which is provided with Liberty Profile z/OS:

Configuring services …

Install File System

(Mounted WRITE)

./featureManager install zosConnect-1.0 --when-file-exists=ignore

./featureManager install zosLocalAdapters-1.0 --when-file-exists=ignore

./featureManager featureList /<path>/features.txt

/<mount>

/bin

featureManager

<feature name="zosConnect-1.0">

<feature name="zosLocalAdapters-1.0">

Installs the z/OS Connect feature into Liberty

Installs the WOLA feature into Liberty

Lists the features currently installed in

Liberty image

Note: you must have Internet access to run these commands

After installing Liberty Profile z/OS V8.5.5.2 or higher using IBM Installation Manager (IM), you will have a file

system containing the Liberty Profile product files. z/OS Connect is supplied as a “feature” to Liberty Profile z/OS,

but it's a feature you have to “install.” This is done using the featureManager shell script, which is found under

the /bin directory of the install location.

Note: to perform the install the file system must be mounted write, and the LPAR on which this is being done must

have internet connectivity. The install process will access files from the IBM repository to complete the install.

There are two features to install – the z/OS Connect feature and the WOLA feature. If you were accessing just

IMS using their supplied JCA resource adapter and IMS Connect the WOLA feature would not be necessary. But

for accessing CICS or a long-running COBOL or other batch program, then WOLA is needed.

Use of the featureManager shell script requires that your UNIX environment has JAVA_HOME resolving to a

valid 64-bit JVM on the system. Once that's in place, then the command to install the z/OS Connect feature is as

shown in the chart. You will be prompted to accept the license, and once that's done it takes a minute or so and

the feature will be installed. Repeat the same thing for the WOLA support.

You can verify the features are present by using the featureManager again, but this time with the

featureList action. This will produce an XML file with the installed features listed out. Find the two features

(as shown) and that verifies the features are present in the Liberty Profile z/OS install.


Unit 3 - 8


Gaithersburg, MD8

Configuring ServicesConfiguring the z/OS Connect to host services to backend programs


Unit 3 - 9


Gaithersburg, MD9

It's All About the server.xml FileAlways keep in mind that z/OS Connect is only what the contents of server.xml tell it to be. The key to understanding z/OS Connect is understanding the XML structure.

Abstract flow …

The server.xml configuration file

Key Points:● Liberty Profile is, fundamentally, defined by its configuration (server.xml)

● z/OS Connect relies on what's in server.xml to understand how it is configured and how it is to behave

● There's no magic to this … there is a lot of detail, but no magic

We will start with a reminder about how the server.xml file – the primary configuration file for a Liberty Profile

server – is the focus for most of the discussion of z/OS Connect. A given instance of Liberty Profile will be only

what's defined in the server.xml. Further, z/OS Connect is in large part an application that runs inside Liberty

Profile, and it too relies on the configuration XML found in server.xml.

We start with this because we're about to show a fair amount of detail related to XML that gets updated in

server.xml. Going through some of the detail is unavoidable in any discussion of z/OS Connect. It is

unavoidable because to Liberty Profile and z/OS Connect, it really is all about what's in the server.xml file.


Unit 3 - 10


Gaithersburg, MD10

Abstract Flow of Processing of XML DefinitionsThe following flowchart is intended to offer an abstract view of XML definition processing, which will help put the XML examples that follow in context:

Essential XML …

Service Definition

Service Provider

Client

Backend

Maps

Calls

Invokes Returns

Returns

Returns

1

2

3

Interceptor List

Interceptor

Interceptor

Data Transformation

Data Bind Files and

JSON Schema Files

Call/Return

4

5

This represents one service; you may have many services defined. Each would have similar structure.

Before we launch into examples of XML, we need to set the stage with a high-level picture of what's going on with

the configuration XML. The flow-chart shown here is intended to represent a kind of “flow” that takes place within

the server.xml file definitions for z/OS Connect. This is not meant to represent the exact flow processing within

the z/OS Connect function; rather, just how the XML definitions relate at an abstract level.

1. The “service definition” is what identifies the receives URI and starts the process of mapping it to other

elements in the XML. If the received URI does not map to any service definition, then an error is thrown. But if

it maps to one of the service definitions then z/OS Connect has something to work with.

2. The XML will point to a service provider, which is a definition of how to access the backend system and

program.

3. The backend is invoked and the response is returned.

4. As part of the service definition, it's possible to define a reference to an “interceptor list,” which will invoke the

named “interceptors” (think of these as call-out programs invoked for each request received). This is optional;

if configured, interceptors are invoked … if not configured, then no interceptors are involved. Two interceptors

are supplied – one for audit (SMF 120.11), and another for role-based access authorization. The SMF

interceptor we'll briefly cover at the end of this unit; the authorization interceptor will be covered in the security

unit.

5. Another optional function is data transformation, which will turn the received JSON into the data format

expected by the backend program. This involves generating “bind” files which define how JSON is to be

mapped to the data structure the program expects, and how the response is mapped back to JSON. We have

a few charts on this later in the unit.

That's the flow. At its simplest it flows from service definition to provider to backend, then back to the client. But

you may also have optional definitions in the configuration that makes the flow for sophisticated.


Unit 3 - 11


Gaithersburg, MD11

Example of XML RelationshipsHere's an example of what the previous flowchart was illustrating … the service definition maps to both a service provider and a data transformation definition:

Essential XML …

<zosConnectService id="CICS"

serviceName="CICS-backend"

dataXformRef="xformJSON2Byte"

serviceRef="wolaCICS" />

<localAdaptersConnectService id="wolaCICS"

registerName="CICSREG"

serviceName="OLACB01"

connectionFactoryRef="wolaCF" />

Service DefinitionService Definition

Service ProviderService Provider

<zosConnectDataXform id="xformJSON2Byte"

bindFileLoc="/u/user1/bindfiles"

bindFileSuffix=".wsbind"

requestSchemaLoc="/u/user1/json"

responseSchemaLoc="/u/user1/json"

requestSchemaSuffix=".json"

responseSchemaSuffix=".json" />

Data TransformationData Transformation

Backend SystemBackend SystemThat is the essential nature of understanding the server.xml for z/OS Connect – service definition to provider definition and including any optional

interceptor inforamation

URI

Here's an example of the relationship in the XML for a “service” definition. This is not the complete server.xml, but

rather some key snippets to illustrate the flow and relationship of one section to another.

The starting point is the service definition. This defines the URI pattern for this service. In this example you have

a URI pattern of “CICS-backend” (how exactly this maps to a real URI we'll see in a bit). The service definition

points to a service provider. That defines the basic WOLA definitions to the backend CICS region and the

program to be invoked. Because the service definition has a data transformation reference, that points to XML

that defines where the transform files are located.

And that's the flow – service definition maps to service provider, and references in the service definition point to

optional functions, such as a data transformation information. At first this will be new and you'll have to become

accustomed to it, but with practice it will become second nature and you'll be copying and pasting sections and

modifying the key elements of the XML.


Unit 3 - 12


Gaithersburg, MD12

Essential server.xml ConfigurationBefore we get into any service and provider definition details, let's take a look at a few essential things that must be in place first:

Structure of service definition …

<featureManager>

<feature>jsp-2.2</feature>

<feature>appSecurity-2.0</feature>

<feature>zosConnect-1.0</feature>

<feature>zosLocalAdapters-1.0</feature>

</featureManager>

<keyStore id="defaultKeyStore" password="Liberty"/>

<webAppSecurity allowFailOverToBasicAuth="true" />

<basicRegistry id="basic1" realm="zosConnect">

<user name="Fred" password="fredpwd" />

</basicRegistry>

<authorization-roles id="zos.connect.access.roles">

<security-role name="zosConnectAccess">

<user name="Fred"/>

</security-role>

</authorization-roles>

These two <feature> definitions enable z/OS Connect and WOLA in the instance of Liberty

Profile z/OS

These are required security definitions.

This shows the simplest way to achieve the minimum security requirements:

● Internal (non-SAF) key and trust store

● Basic auth rather than default client certificate authentication

● Internal (non-SAF) user registry

● Internal (non-SAF) mapping of user to role

The first bit of XML we'll explore is some essential housekeeping things:

● Update of <featureManager> section to include both z/OS Connect and WOLA – this tells Liberty Profile to

load the features installed earlier. This is a relatively easy two element update of server.xml.

● Some security updates to accommodate the requirements of the z/OS Connect application. Those are discussed below.

The z/OS Connect application is marked “protected,” which means any user accessing it must be authenticated, and the connection must be SSL. Our objective at this point in the workshop is to keep security things as simple

as possible. Later, in the security unit, we'll go into more detail on how to secure z/OS Connect more properly.

The definitions shown on the chart provide for the minimum security requirements defined and managed entirely

within Liberty Profile, and not requiring SAF. Those include:

● The keystore for the SSL certificates. That's the <keystore> element.

● XML to tell z/OS Connect to use basic authentication rather than the default client certificates. Basic

authentication is simpler than client certs. Later we'll show using client certificates, which is what we'd expect is much more common for z/OS Connect.

● The definition of a userid for authentication. In this case it's “Fred.”

● Finally, the definition of the access role that allows “Fred” to use the z/OS Connect application. Without this

Fred would be able to log on, but would then get an error from z/OS Connect. With this z/OS Connect knows Fred is authorized to use the application.

Again … since z/OS Connect is a protected application, we have to put some security definitions in place. This

allows us to provide the minimum required in a way that is entirely based on XML updates. This isn't production-level security, but it's “good enough” for early validation and testing.


Unit 3 - 13


Gaithersburg, MD13

Structure of a Service DefinitionThis may have more XML elements than shown here, but this gives a sense for the key elements and what the service definition provides:

The URIs used …

<zosConnectService id="CICS_create"

serviceName="CICS-create"


interceptorsRef="interceptorList_1"


1

2

3

4

The serviceName= value determines the URI that will invoke this service:http://<host>:<port>/zosConnect/services/CICS-create?action=invoke

The dataXformRef= value points to another section of server.xml where the data transform definition for this service can be found. (More on data transform coming up.)

The interceptorsRef= value points to another section of server.xml where the optional interceptor list for this service can be found. (More on interceptors coming up.)

The serviceRef= value points to another section of server.xml where the service provider definition can be found. That is what defines how to access the backend system for this service.

Here's our first detailed example, and this shows a service definition. This is for a CICS program that does some

form of “create” function, as the name of the service implies. Let's walk through the numbered blocks:

1. The serviceName= element is used to indicate the URI pattern that can be used for this service. In this

example we're showing an “invoke URI” pattern as well (numbered block 2), but you could have just the

serviceName= value, and if that were the case then the URI would be based on the context of

/zosConnect/services followed by the serverName= value, followed by ?action=invoke.

Note: some do not like exposing the /zosConnect/services part of that, which is why the next thing we

show – the “invoke URI” pattern – is supplied. It hides that level a detail. That's on the next chart.

2. The dataXformRef= element points to another portion of XML where the data transformation definition is

maintained. This is optional; if you don't wish to do data transformation, you can omit this. We'll show data

transformation a bit later in this unit.

3. The interceptorsRef= element points to another portion of the XML where “interceptors” are specified.

This too is optional; if you don't wish to use any interceptors, then this may be omitted.

4. Finally, the serviceRef= element points to another portion of XML where the service provider is defined.

This is what defines the connectivity to the backend system and program. We'll see that in an upcoming

chart.

At a high level, that's it – a service definition provides the URI pattern for the service, and then provides

information on how to connect to the backend and whether optional data transformation and inteceptors will be

called as part of this.


Unit 3 - 14


Gaithersburg, MD14

The URI Clients Would Use for the Defined ServiceUsing the XML example on the previous chart, let's reinforce the understanding of the URI clients would use to invoke the service:

Service provider …


serviceName="CICS-create" invokeURI="/myCICSCreate"




https://<host>:<port>/zosConnect/services/CICS-create

https://<host>:<port>/myCICSCreate

Client

z/OSConnect

z/OSConnectHTTP PUT or POST with JSON

This is required

This is optional

Both achieve same thing … the “invokeURI” is simply a way to make the URI simpler and shield clients from undertanding it's z/OS Connect

This chart is introducing an additional element and adding it to the information on the previous chart. It is the

invokeURI= pattern. There are two URI patterns that would work for this service. One is the “raw” format – that

is, with /zosConnect/services and ?action=invoke after the service name. The other is a kind of “alias” –

that is, a simpler pattern is used and the details and ?action=invoke are hidden.

Both end up calling the same service. Which you use is really up to you. The “invokeURI” pattern is a way to hide

any details of the actual backend service name behind an alias. We anticipate the invokeURI= pattern will be

commonly used for just that reason.


Unit 3 - 15


Gaithersburg, MD15

The Service Provider – Access to BackendThe next piece of the puzzle is the service provider, which defines how z/OS Connect is going to access the backend resource:

Multiple services …


serviceName="CICS-create"

invokeURI="/myCICSCreate"




<localAdaptersConnectService id="wolaCICS"

RegisterName="CICSREG"

ServiceName="OLACB01"

connectionFactoryRef="wolaCF" />

Service definition from the previous chart

The WOLA registration name

used by the external address space

The backend program name to be called over WOLA

The WOLA JCA resource adapter ID to be used for the WOLA connection

Two pieces in place – service used to map URI; then provider maps connection to the backend

The “service provider” definition indicates to z/OS Connect how to access the backend service. There are two

choices – WOLA for CICS and long-running programs such as COBOL, or the IMS JCA resource adapter if

connecting to IMS via IMS Connect.

Note: in this workshop our focus is on CICS and long-running COBOL. The principles defined here are identical

for batch, but the syntax might be slightly different.

For WOLA connectivity the key things that need to be define are the WOLA registration name to communicate

across, the “service name” (the target program name) to invoke on the other side of the WOLA registration, and

the WOLA JCA resource adapter connect factory to use. (We're not showing the connection factory definition

here … it's fairly generic and is shown in detail in the WP102439 Techdoc at ibm.com/support/techdocs.)

The service definition points to the provider definition with the serviceRef= element. That's how the connection

is made between the service definition and the provider definition.


Unit 3 - 16


Gaithersburg, MD16

Multiple Services, Multiple ProvidersYou are not limited to a single service, of course … you may configure many services with many backend provider connections:

IMS …

CICS Region 1

A

CICS Region 2

B C

Batch Service

D

<zosConnectService id="ProgramA"

serviceName="ProgramA"

serviceRef="CICS1A" />

<zosConnectService id="ProgramB"

serviceName="ProgramB"

serviceRef="CICS2B" />

<zosConnectService id="ProgramC"

serviceName="ProgramC"

serviceRef="CICS2C" />

<zosConnectService id="ProgramD"

serviceName="ProgramD"

serviceRef="Batch" />

<localAdaptersConnectService id="CICS1A"

RegisterName="CICS1REG"

ServiceName="A” />

<localAdaptersConnectService id="CICS2B"


ServiceName="B" />

<localAdaptersConnectService id="CICS2C"


ServiceName="C" />

<localAdaptersConnectService id="Batch"

RegisterName="BATCHREG"

ServiceName="D" />

CICS1REG

z/OSConnect

z/OSConnect

BATCHREG

CICS2REG

There's no practical limit to this … one,

several, hundreds … your choice

The previous charts illustrated a single service, with a single provider. You are not limited to that … you can

configure multiple services and providers in a single instance of z/OS Connect. This chart illustrates this in a

somewhat high-level way.

Consider the diagram in the upper-left of the chart. A single instance of Liberty Profile and z/OS Connect is

connecting to two CICS regions and a long-running program. Three WOLA registrations are in place with the

names shown on the chart. Target backend programs A, B, C and D are deployed in the regions as indicated.

The service and provider definitions are as shown – four service definition, four provider definitions, and the

specifics of the service names highlighted in blue.

There's not really much new news here. This is really the same story we showed in the previous few charts, but

replicated so four services and four providers are represented.

Why four providers when there are only three WOLA registrations? Because the provider definition carries the

specifics of the target program to be called. In the case of the CICS Region 2 WOLA registration (CICS2REG)

there are two unique target programs to call (programs B and C). Therefore, two provider definitions to call out

each program (B and C). The same WOLA registration is used for both.

Note: keep in mind these are just definitions in the XML. This does not mean the z/OS Connect code is having to

load up four separate sets of connector code. The code is fairly efficient about such things; no heap is wasted

duplicating things just because the definition has separate provider sections as shown.


Unit 3 - 17


Gaithersburg, MD17

IMS and z/OS ConnectIMS has implemented z/OS Connect in a slightly different way. The same principles apply, but the service provider is another feature to Liberty Profile, not WOLA:

Data conversion …https://developer.ibm.com/wasdev/downloads/#asset/features-com.ibm.ims.imsmobile-1.0

z/OS Connect


server.xml

IMS Connect

IMSClient

IMS Explorer

Administration and Configuration of the z/OS Connect and IMS Mobile Feature environment

http://www-01.ibm.com/support/knowledgecenter/SS9NWR_3.1.0/com.ibm.ims.es31.doc/ies_overview.htm

● Part of IMS Enterprise Suite

● IMS Mobile Feature is a z/OS Connect “Service Provider”

● Everything else the same – discovery, data conversion, authorization interceptors, audit interceptors

ims-services.xml

IMS Mobile Feature

Service Provider

z/OS Connect works with IMS as well, but the way it operates against IMS is different from what we've seen to this

point with CICS. With CICS the “service provider” was WOLA. For IMS, the IMS development lab extended the

z/OS Connect framework and provided their own “service provider.” This service provider then communicates

with an instance of IMS Connect, which in turn communicates with the IMS region.

Note: IMS can serve as both a transaction manager and a database manager. The z/OS Connect support for IMS

includes interacting with transactions only. z/OS Connect can not at present be used to access data through the

IMS database function.

This is an example of z/OS Connect being extensible. The developers implemented the service provider interface

(SPI) of z/OS Connect to create this IMS-specific “service provider.” Everything else about z/OS Connect we've

discussed up to this point (and what we'll discuss after) is the same. That's because his IMS service provider

operates within the z/OS Connect framework … and that illustrates the value of z/OS Connect being extensible.

Anyone can write service providers for z/OS Connect. In this case it turns out another development group in IBM

did it.

This custom service provider is supported by a second configuration XML file called ims-services.xml. This

provides definitions related specifically to the IMS Mobile Feature. Further, when an instance of Liberty Profile

z/OS is running this IMS Mobile Feature, it can be accessed by a copy of IMS Explorer (which runs in Eclipse and

is a workstation-based graphical interface tool) to configure and manage the Liberty Profile server in which all this

runs.

Note: You will hear some speak of this as the “IMS Gateway Server.” When they do, they are referring to the

combination of the Liberty Profile z/OS server, z/OS Connect, and the IMS Mobile feature.

Also, it is possible to have this IMS Mobile feature and WOLA at work simultaneously in the same Libety Profile

server instance. That would provide access to IMS, CICS and long-running programs.

This functionality is part of the IMS Enterprise Suite. The URL second from the bottom on the chart provides an

overview of the IMS Enterprise Suite offering. The URL at the bottom provides a location where the IMS feature

can be downloaded.


Unit 3 - 18


Gaithersburg, MD18

Data ConversionConverting JSON to the data format of the backend program


Unit 3 - 19


Gaithersburg, MD19

Data Conversion at a High LevelThis subject focuses on converting JSON to the data format expected by the backend system:

Passthrough or data convert …

{

"account": "00012345",

"firstName": "Smith",

"lastName": "John",

"action": "Balance",

}

JSON

01 account PIC 9(8).

01 lastname PIC X(16).

01 firstname PIC X(12).

01 action PIC X(8).

COBOL

Transform this into this … and back again

To do this, z/OS Connect needs to understand the relationship between the two formats so it can do the transformation. That's the role of the data

tranform utility, which produces “bind” files and JSON schema files

The basic concept of data conversion is fairly straight-forward – if the backend program has a data format it

expects, and z/OS Connect is receiving JSON from the client, then z/OS Connect needs to understand both JSON

and the backend so its conversion utility can transform between the two. The data conversion utility provided with

z/OS Connect is based on the CICS web services conversion utility, which transforms COBOL COPYBOOK into

bind files and JSON schema.

Notes:

● The data conversion function of z/OS Connect is one of the extensible interfaces, so if the supplied conversion

utility does not meet your needs, you can write your own data conversion utility and plug it into the z/OS

Connect framework.

● Your backend may not require data conversion – that is, it may accept JSON, in which case z/OS Connect can

be configured to simply pass-through the JSON to the backend.

● If z/OS Connect exposes an API interface for JSON conversion to a backend program, it would be helpful to

have a way to “discover” (or query) for the JSON schema the interface requires. z/OS Connect provides that

as one of its REST APIs for management. We'll see that later in the unit.

● The conversion utility for z/OS Connect is Java-based, which means it's zIIP-offloadable work.


Unit 3 - 20


Gaithersburg, MD20

With No Data Conversion, JSON in and JSON outData conversion is optional; but if you don't use it, then understand that z/OS Connect will pass JSON unchanged to backend and expect JSON on return:

High-level schematic …

Client

Request JSON

Request JSON

Response JSON

Other Format

If a service has no data conversion definition, then z/OS Connect will simply pass the received JSON to the backend program unchanged

This will work only if the backend program understands JSON and the JSON format passed back

If z/OS Connect passes JSON to the backend program, then it expects to

get JSON in return

If something other than JSON is returned, then an error is thrown by z/OS Connect

You as administrator of z/OS Connect must have some knowledge of the data requirements

z/OS Connect takes in JSON data from clients. It is not designed to handle anything but JSON. JSON is what the

clients must send.

But the backend program may not understand JSON. Or it may. That's something you as administrator of the

z/OS Connect configuration must understand.

z/OS Connect provides a data conversion function to map JSON to whatever data structure the backend program

expects, and map the response back to JSON for the return. It's an optional function … you don't have to to use

it. But if you don't use it, then you should understand what to expect.

If you choose to omit the data conversion function, z/OS Connect will pass the received JSON – unchanged – to

the backend program. What the backend program does with the JSON is up to the backend program. If it doesn't

understand JSON it'll throw some kind of error.

However, if the backend program understands JSON and no data conversion is done by z/OS Connect, then that

backend program must return JSON. z/OS Connect expects JSON to be returned when JSON is passed back. If

JSON isn't returned, then z/OS Connect will throw an error.

That's the key message of this chart … you may forego use of the data conversion function of z/OS Connect if you

know the backend program accepts JSON, but you must make sure the program returns JSON as well. If so,

then z/OS Connect will be happy and return the JSON to the original requesting client.


Unit 3 - 21


Gaithersburg, MD21

High-Level Schematic of Data Conversion Processz/OS Connect uses a utility based on the CICS data conversion utility. It involves creating “bind” files and making those files available to z/OS Connect:

More detailed …

Supplied Data Conversion Shell Script

Data structure(example: COPYBOOK)

Generated Bind and JSON Schema Files

Supplied JCL to run the shell scripts and define input and output locations

<zosConnectService id="CICS"

invokeURI="/myCICSBackend"

serviceName="CICS-backend"



<zosConnectDataXform

id="xformJSON2Byte"

bindFileLoc="/u/user1/bindfiles"

bindFileSuffix=".wsbind"

requestSchemaLoc="/u/user1/json"

responseSchemaLoc="/u/user1/json"

requestSchemaSuffix=".json"

responseSchemaSuffix=".json">

</zosConnectDataXform>

This tells z/OS Connect where files

can be found and the suffix values

This chart is showing the essence of this function. It comes in two parts – (1) the generation of “bind” and

“schema” files from the data structure definitions of your target program; and (2) definitions in the server.xml to

tell z/OS Connect where the files are located.

The generation of the bind and schema files is done using a supplied utility, which is based on a similar utility

supplied with CICS for its web service function. The utility is invoked with JCL and takes as input the data

structure of your program (for example, the COPYBOOK structure) and outputs bind files (binary files that map

data structure fields to JSON) and the associated JSON schema files. For each target program in the backend

you would generate the bind and schema files.

Then you tell z/OS Connect where the files are located. Define a zosConnectDataXform element as shown,

and you point to it from the service definition. That zosConnectDataXform element provides information about

the location of the bind and schema files.

You may have multiple locations for such files. If so, then provide different zosConnectDataXform sections in

the server.xml, and point to whatever one you wish to use for a given service. z/OS Connect will go looking in

that location for the requested files.


Unit 3 - 22


Gaithersburg, MD22

More Detailed Schematic With NotesThis is from the WP102439 Quick Start Guide. See speaker notes for explanation of numbered blocks:

REST APIs …

WP102439 at ibm.com/support/techdocs

This is a more detailed layout of the process of creating and using the data conversion utility to create the bind and

JSON schema files. This come out of the WP102439 “Quick Start Guide.” The numbered blocks relate to the

following notes, also from the Quick Start Guide:

1. The starting point is the request and response data structure for the program. These must provided in a PDS

member. Just the data structure elements are provided here, not any of the other program source code.

2. A JCL job is used to call the BBGLS2JS JCL procedure. It's role is to read the inline parameters you supply

(block 3) and call the BBGLS2JS procedure (block 4)

3. The inline parameters provide the utility knowledge of where the data structure PDS (1) is located and what

member names to read in, and various other things to control the generation of the data conversion bind files.

4. The BBGLS2JS (LS2JS = “language structure to JSON structure”) procedure is supplied with z/OS Connect.

This is called by the job (2). No changes are needed to this procedure.

5. The utility writes bind files and JSON schema files out to the UNIX Systems Services location you specify in

the inline parameters (3).

6. Updates to the server.xml of the Liberty Profile server provide z/OS Connect with knowledge of where the

bind and schema files are located.


Unit 3 - 23


Gaithersburg, MD23

The REST APIsUsing the supplied REST APIs to monitor and manage the environment


Unit 3 - 24


Gaithersburg, MD24

REST APIs in ContextThe REST APIs we now refer to are supplied with z/OS Connect. They are used to discover services and manage the z/OS Connect instance:

APIs …


z/OS Connect

CICSCICS

IMSIMS

BatchBatch

Client

Role ID

The service APIs you define in server.xml

for your backend programs

The supplied REST APIs … these will not

appear in server.xml … they are built-in

This “Role ID” is a very loose term at the moment … initially all IDs will have access to all REST APIs, but later we'll discuss “authorization interceptors”

which provide a more granular access model to the APIs

z/OS Connect supplies a series of REST URI patterns that can be used to discover configured services and

manage them. The REST URI pattersn – or “APIs” – are built into the z/OS Connect function. You will not see

reference to them in the server.xml file. On the next chart we will see a listing of them and what they provide.

On this chart we take a moment to keep clear in our minds the difference between the URI patterns for your

exposed services and the APIs exposed by z/OS Connect itself. For the purposes of keeping them straight, we'll

call the supplied interfaces “REST APIs,” and we'll call the interfaces you configure “configured services.”

In this chart we show the REST APIs being accessed by a “Role ID.” That's a very generic term at the moment …

right now … with the security model we currently have in place … that “Role ID” is the ID used to access z/OS

Connect. If an ID has that access, they have access to use any of the REST APIs. But in the security unit we'll

show how z/OS Connect also provides a way to make access more granular. There are three access roles –

administrator, operator and invoke – that you can assign IDs to. But we won't go into that right now because it

clouds the APIs story a bit. We'll save that for the security unit.


Unit 3 - 25


Gaithersburg, MD25

The Supplied REST APIs for Discovery / ManagementHere's a listing of the APIs and a brief description of what each provides:

Example …

https://host:port/zosConnect/servicesReturns a JSON object with the configured services the user is authorized to access

https://host:port/zosConnect/services/service_nameReturns a JSON object with the details for the named service

https://host:port/zosConnect/services/service_name?action=statusReturns a JSON object with the status of the service (started or stopped)

https://host:port/zosConnect/services/service_name?action=getRequestSchemaReturns a JSON object with the request schema for the named service

https://host:port/zosConnect/services/service_name?action=getResponseSchemaReturns a JSON object with the response schema for the named service

https://host:port/zosConnect/services/service_name?action=getStatisticsReturns a JSON object with request statistics for the named service

https://host:port/zosConnect/services/service_name?action=stopStops the named service and returns a JSON object with the status

https://host:port/zosConnect/services/service_name?action=start Starts the named service and returns a JSON object with the status

These can be used by developers to understand the APIs exposed by z/OS Connect … and by administrators to control services

These are the REST APIs exposed by z/OS Connect. The blue service_name in the URI patterns is where you'd

put the service name as configured on the serviceName= element in the service definition.

Note: if your service has the invokeURI= pattern to create an “alias” for invoking the service, the REST APIs will

still work, but you have to use the serviceName= value, not the invokeURI= value.

The first API on the chart simply lists back all the configured services as seen in the server.xml file. That's a

way to “discover” the configured services in a summary format.

The next API is a way to get details on a specific service.

The action=status API will return the status of the service – started or stopped. At the bottom of the chart you see APIs that can be used to start or stop a service. A stopped service will throw an HTTP 404 error, which is

effectively the same thing as the service not existing.

The schema APIs will return the request and response schemas if data conversion is in use.

The getStatistics API will return some basic numbers on usage. It's not as robust a source of usage statistics as

the audit interceptor (the SMF writer), which we'll see next.


Unit 3 - 26


Gaithersburg, MD26

REST API Example – Get List of Configured ServicesHere's an example of the REST APIs in action:

Example …

REST API to get a list of configured services

In this example two configured services are returned

Here's an example of a REST client issuing the API to get a list of the configured services for an instance of

Liberty and z/OS Connect. In this example, it returns a JSON object with two configured services. If you had 100

configured services, the returned JSON object would show all 100 services.


Unit 3 - 27


Gaithersburg, MD27

REST API Example – Get Request SchemaHere's an example of the action=getRequestSchema REST API:

Audit interceptor …

getResponseSchema is also available

This allows a developer to not only know what services are available, but what the invoke pattern is and what the JSON request/response patterns are. They may then write their REST/JSON client to the API.

API Management utilities can query this interaface and catalog the results, providing a repository of API information for developers to use.

Knowing that a service exists is not enough … a REST client also needs to know what JSON to send in and what

JSON to expect. The action=getRequestSchema and action=getResponseSchema APIs will return the

configured data transformation JSON schema patterns for the service. With that, a developer can create the

REST client to use against this service.


Unit 3 - 28


Gaithersburg, MD28

InterceptorsCallout points for each request received


Unit 3 - 29


Gaithersburg, MD29

High-Level of Interceptor FrameworkThink of “interceptors” as code that is called for each request for a service:

Notes about interceptors …

InterceptorConfigured?

InterceptorConfigured?

Request Received

Request Received

CallBackend

CallBackend

Interceptor“List”

Interceptor“List”

Interceptor 1

Interceptor 2

:

Interceptor n

If a service has a reference to an interceptor list, the interceptors defined in that list are invoked for each each request that is received by z/OS Connect

This chart shows a very high-level representation of the concept of interceptors. An “interceptor” is simply code

that is called for each request that is received by z/OS Connect for a service. Rather than restrict interceptors to a

single bit of code that is called, z/OS Connect invokes a list of interceptors. That list may contain from one to

many references to interceptor code to call for each request.

On the next page we'll provide some more detail on interceptors ...


Unit 3 - 30


Gaithersburg, MD30

Overview of Interceptor FrameworkInterceptors represent optional code that is called upon receive of each request that comes into z/OS Connect:

Audit Interceptor …

OptionalYou may configure them if you wish, or not, depending on your needs.

May be configured at global level (all) or service level (some)This allows you to apply interceptors to those services you wish, without necessarily having to crawl through every one updating XML. It is also possible to “opt out” of globally defined interceptors. Quite a bit of flexibility.

Extensiblez/OS Connect comes with a Service Provider Interface (SPI) for interceptors so you can write your own if you wish.

Uses interceptors “lists”A interceptor “list” is simply a bit of XML that specifies which interceptors to invoke and the order in which to invoke them. You can define multiple lists, with some services calling one, and some services calling another.

Two IBM-supplied interceptorsIBM supplies two: one writes SMF 120.11 records, the other is provides a way to limit what the authenticated user can do in z/OS Connect

The topic of “interceptors” can be a little slippery if you jump straight into the syntax of the server.xml. To start

with we'll approach it from a higher level first, then later get into how they are configured.

Interceptors are designed to invoke code on each request received. This provides a way for z/OS Connect to

temporarily pass control to other code for some processing for the request.

Note: it's important to understand that the interceptor framework does not allow interceptors to change the JSON

payload coming in or going out. Interceptors can get a copy of the JSON, but interceptors can not modify the

payload.

Interceptors are optional … it is not required you use them at all, or if you do use them, it is not required you use

them on all services. They are configurable either at the global level so they apply to all the services defined, or at

the service level so they apply only to specific services, or a mixture of both. This is where the XML syntax can

get confusing … the combination of global, and service-level, and how the two relate can be tricky to understand at

first.

Interceptors are configured as part of a list. A service is not limited to invoking just one interceptor. It can invoke

a chain of interceptors if you want. That chain is defined in the XML as a list of interceptors, and if a service calls

the list then z/OS Connect will work through the interceptors in the list and call them, one after another, in the

order you specify.

The z/OS Connect interceptor framework is designed to be extensible, meaning you may write your own

interceptor if you wish. The interceptor framework comes with a Service Provider Interface (SPI) and a class

library for development of interceptors. You develop your own, register it to Liberty Profile and call it through the

configuration in the XML.

Finally, IBM supplies two “out of the box” interceptors for you to use – one is called the “audit interceptor” and it

writes SMF 120.11 records; the other is called the “authorization interceptor” and it provides a way to limit the

scope of activities a given user of z/OS Connect may perform. We'll briefly touch on each next.


Unit 3 - 31


Gaithersburg, MD31

Brief Review of Audit InterceptorThe “audit interceptor” is a supplied piece of code that will write SMF 120.11 records. Those records contain information on z/OS Connect usage:

Multiple services same backend work …

System Name

Sysplex Name

Jobname

Job Prefix

Address Space Stoken

Arrival Time

Completion Time

Target URI

Input JSON Length

Response JSON Length

Method Name

Service Name

Userid

Server

Identification Section

z/OS Connect

User Data Section

Notes:

● Liberty Profle z/OS itself does not cut SMF records; this is a function of z/OS Connect

● SMF 120.11 is relatively new format, so your analysis tool may or may not understand that format yet

● No CPU numbers in 120.11

● Can capture CPU at started task level for Liberty

● z/OS Connect is a pass-through function; it will use some CPU (mostly zIIP).

The “audit interceptor” is really a function of z/OS Connect that writes SMF 120.11 records for those services

configured to have them written.

SMF is a facility of z/OS that provides a highly efficient means of writing records capturing information about

activity on the system. There are many different SMF record “types,” each specific to the program or subsystem

that's writing the record. For example, CICS writes 110 records. The full-function WAS z/OS product writes 120

records. The record we'll referring to for z/OS Connect is a subtype of 120 … the 120 subtype 11, or 120.11

records.

Note: it's worth nothing that Liberty Profile itself does not write SMF records. This SMF capability of z/OS Connect

is a function of the z/OS Connect feature of Liberty. If you want to capture statistics for the Liberty Profile server

itself, you'd use the SMF Type 30 record for the address space.

The chart shows the record layout for the SMF 120.11 record. A record is cut for each request received by z/OS

Connect for the service configured to write SMF records.

A common question is whether the SMF 120.11 record captures CPU statistics; specifically CPU on GP and

speciality engine usage. The answer is it does not. Those numbers can be captured with the address space SMF

Type 30 records. That would not provide CPU statistics at the individual service level, but it would provide

numbers for the Liberty Profile server acting as the z/OS Connect gateway. z/OS Connect uses some CPU, but

keep in mind that (a) z/OS Connect is a pass-through function for the most part, with the data conversion routines

being perhaps the user of CPU, and (b) it's written in Java, so it would be dispatched to the speciality engines of

the LPAR.


Unit 3 - 32


Gaithersburg, MD32

CICSREG

Two Services Invoking Same Backend WorkHaving two defined services invoke the same backend work is doable. But why?

Authorization interceptor …


z/OS Connect

<zosConnectService id="Mobile Traffic"

serviceName="Mobile"

invokeURI="/Mobile"

serviceRef="CICSWOLA" />

<zosConnectService id="Internal Traffic"

serviceName="Internal"

invokeURI="/Internal"

serviceRef="CICSWOLA" />

<localAdaptersConnectService id="CICSWOLA"

RegisterName="CICSREG"

ServiceName="TRANXYZ" />

TRANXYZTRANXYZ

CICS

SMF 120.11 Mobile <data>


SMF 120.11 Internal <data>



SMF 120.11 Internal <data>


:


A question that comes up is how to differentiate two (or more) “channels” of input to work on the LPAR. One

potential means of accomplishing this using z/OS Connect is to configure multiple services that map to the same

work in the back-end. With the SMF 120.11 capability of z/OS Connect, you can then capture the number of

requests that are coming in through each service.

Note: this does not capture the CPU for the work accomplished in the backend systems. But it does provide a

way to identify and quantify the requests. If those requests can be equated with work in the backend that you can

measure separately, then it gives you a way to arrive at an estimate of backend work attributable to the input

“channel.”


Unit 3 - 33


Gaithersburg, MD33

Brief Review of Authorization InterceptorThe “authorization interceptor” is a supplied piece of interceptor code that will check to see if the user has the authority to perform the action requested:

Security …

Allowed to Enter?

AdministratorFull authority

OperatorStart, Stop, etc.

InvokeInvoke service only

“Fred” Yes

No

Go Away

The mechanics of how this is done and how it is configured is a

topic for the unit on security

This is controlled by a defined “role”

This is what the interceptor provides

The other interceptor supplied with z/OS Connect is the authorization interceptor, which – if configured – provides

a way to differentiate users to certain levels of authority.

The authorization interceptor provides three levels of authority – administrator (full authority), operator (the ability

to use the REST APIs to control the configured services, such as start and stop, or get the JSON schema), and

invoke (the ability to invoke the configured services, but not use the REST APIs).

The authorization intereceptor, like any interceptor, is optional. If you do not configure it then a user who has

access to the z/OS Connect servlet (via the application role) will have administrator authority. When the

interceptor is configured, then you may control at the user level.

This interceptor can be configured at the global level (all configured services), at the service level, or a

combination of the two. This is where the behavior can become a bit perplexing at first. To help you understand,

a guide has been published at the WP102439 Techdoc (ibm.com/support/techdocs) called “Interceptor

Scenarios.” This provides a series of five different scenarios of the authorization interceptor showing

combinations of global and service-level and how things would work.

That's all we're going to say about this interceptor at this point in time. The security unit, which comes next, will go

into more detail on security overall for z/OS Connect, as well as the authorization interceptor itself.


Unit 3 - 34


Gaithersburg, MD34

Setup of Security TopicTo set the stage for the details on security that are to come next


Unit 3 - 35


Gaithersburg, MD35

Intentionally Avoided Security Topic So FarWe have avoided the topic of security for the most part. Some hints of it came through in parts, but in general we've downplayed it. Here's what we'll have after lab:


z/OS Connect

server.xml

Client

CICS Region

ProgramProgram

2

SAF(RACF in our case)

3

45

1. z/OS Connect Servlet ProtectedThis means we require authentication and SSL; we can't avoid it. But we can make it simpler at first by not using SAF.

2. Definitions in server.xmlThe SSL definitions, the user registry, the role definition, and the switch to basic auth from client cert default done here.

1

3. Definitions in SAFSome STARTED, SERVER and CBIND profiles are required to support Liberty as STC and WOLA

4. Basic Auth from Clientz/OS Connect default is client certificate, but basic auth is easier to use initially.

5. No ID assertion into CICSCall to CICS program done under ID assigned to WOLA BBO$ link server task

Up to this point in the workshop we have intentionally downplayed the topic of security. We've referenced it a few

times, but usually to defer discussion to the unit on security. We have not been able to ignore it completely

because z/OS Connect requires some security by its nature – the z/OS Connect application is marked protected

and that calls out a need for authentication and SSL.

To keep the early focus on z/OS Connect and away from security, we kept the security definitions as simple as

possible. That meant having them defined in the server.xml rather than in SAF. In the security unit we'll

discuss the use of SAF.

The chart shows the security in place for z/OS Connect at the end of the upcoming lab. This is provided as a way

to “set the stage” for the upcoming discussion on security. Our workshop design strategy was – “Keep security

simple at first, then open the topic up at the end and dive deeper into security specifics.”

1. As noted, the z/OS Connect servlet is marked protected. That requires authentication and SSL.

2. To keep things relatively simple we're going to have you add XML to the server.xml file to satisfy the

security requirements. That avoids the use of SAF (for now) for those security artifacts.

3. But we can't avoid SAF completely. There are some profiles we must put into SAF to support the operations

of Liberty Profile z/OS. For the upcoming lab we'll package those as a job you submit. In the security unit

we'll discuss what those profiles are and what they do.

4. Because the z/OS Connect servlet is marked protected, it requires authentication. The default for z/OS

Connect is client certificates, but that involves some work to get the certificate into the client. Again, to keep

things simple initially we'll use “basic authentication,” which is the id-and-password prompt many of us are

familiar with when browsing the web.

5. Finally, while WOLA supports the assertion of ID from Liberty into CICS, we're not going to do that in the

upcoming lab. The invoke of the target CICS program will be done under the ID of the WOLA Link Server

Task.

End of UnitEnd of Unit

unit 3 - zos connect · unit 3 – z/os connect unit 3 - 2 © 2014 ibm corporation ibm americas...

Documents