making the dinosaur dance - rdz, rtc and urbancode lunch and learn slides

© 2015 IBM Corporation

IBM z Systems


RDz Lunch and Learn Series - "Making the dinosaur dance" - Modernizing Development and Deployment using RDz, RTC and UrbanCode Deploy With Chris Trobridge (IBM), Sean Babineau (IBM)


IBM z Systems

Information is confidential and must not be shared or redistributed without permission

from IBM. Plans are based on best information available and may change in future.

DISCLAIMER

© Copyright IBM Corporation 2015. All rights reserved.

IBM’s statements regarding its plans, directions, and intent are subject to change or

withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product

direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment,

promise, or legal obligation to deliver any material, code or functionality. Information about

potential future products may not be incorporated into any contract. The development,

release, and timing of any future features or functionality described for our products

remains at our sole discretion.

Performance is based on measurements and projections using standard IBM

benchmarks in a controlled environment. The actual throughput or performance that any

user will experience will vary depending upon many factors, including considerations

such as the amount of multiprogramming in the user’s job stream, the I/O configuration,

the storage configuration, and the workload processed. Therefore, no assurance can be

given that an individual user will achieve results similar to those stated here.


IBM z Systems

Agenda

Overview of IBM DevOps Solution

RDz –> RTC -> UC

What’s New with IBM DevOps Solution

SAFe

Hierarchical Components

Build enhancements

UC Integration

DEMO

PLEASE ASK QUESTIONS ANY TIME


IBM z Systems DevOps for z Systems Customers

Traditional and

Agile Planning Traceability

Creation and Build

Project

Manager

LOB

Manager Analyst Tester

Simple Deployment

Developers Program

Manager Operations Team

• Enable and manage the transition of large scale traditional development to

modern DevOps practices

• Optimize and integrate mainframe development to control and eliminate

multi-speed IT

• Centralized source code, automated build integrated testing and

automated deployment

• Dashboards, reporting , traceability and compliance for large scale

regulated multi-platform organizations

• Integrating mainframe development into cloud and hybrid scenarios

Core Products supporting capabilities:

• Rational Developer for System z

• Rational Team Concert

• Urban Code Deploy/Release

DEV OPS STAKE- HOLDERS

Orchestrated

Deployment

Collaboration

MGT

Consumers


IBM z Systems

BUILD TYPES

User

RDz Build – Translates RTC build instructions into JCL

Use : Developer ‘s initial build

Personal

RTC Dependency build for the individual developer

Use : To propagate a change for testing

Team

RTC Dependency build in stream

Use : Integration and production builds

Simulation

Dependency analysis without compilation

Use : build dependencies without overhead of compilation

Simulated


IBM z Systems

Deployment – Which deployment?

RTC Integrated Deployment

Simple linear deployment process

Small number of targets

UrbanCode Deployment

UC Deploy offers

Complex deployment process (multi-path, conditional and gated processes)

On-demand, scheduled and triggered execution

UC Release

Orchestration across applications/platforms

Visibility

Complex


IBM z Systems

Agenda

Overview of IBM DevOps Solution

RDz –> RTC -> UC

What’s New with IBM DevOps Solution

SAFe

Hierarchical Components

Build enhancements

UC Integration

DEMO

PLEASE ASK QUESTIONS ANY TIME


IBM z Systems

9

RTC Product Roadmap

RTC 6.0 • UrbanCode Deploy package integration

• Strategic Reuse

• RTC SCM for the Enterprise

• Subset management

• ISPF SRCHFOR/Share Like

• ISPF repository compare

• Build/promotion usability

• Subset editor

• Tracking and Planning

• Quick planner “Found in”

• Performance

• UI usability

• Ranking in hierarchical plans

• Aggregated change notifications

• SAFe

RTC 5.0.2 • Quick Planner - for agile teams who work in

a fast and fluid pace

• Full link history for audits and compliance

• Graphical Reporting

• Quality improvements

• ISPF client preview build/check-in history

• Translator variables

• Sequential file support

• Build/promote/deploy housekeeping

• Planning performance improvements

RTC next • ISPF Client build subsets • Metadata collection • Usability • Security/Compliance • Customer RFEs


IBM z Systems

Scaled Agile Framework™ Big Picture


IBM z Systems

SAFe™ - The engine inside IBM’s Collaborative DevOps IBM’s support for the Scaled Agile Framework © (SAFe™)

Scale lean and agile principles to the enterprise by

establishing a SAFe-based environment with fit-for-purpose

dashboards and reports, supporting the team, program

and portfolio levels in heterogeneous environments.

Get up and running quickly with out-of-the-box

infrastructure to lead a SAFe project

Improve agility and predictability with role-

based dashboards for visibility to continuously

measure progress and adjust planning in real

time to meet business goals

Simplify change to culture and process with

quick and easy access to SAFe best-practices

Operate Develop/ Test

Deploy

Steer

DevOps

Continuous Feedback

IBM SAFe solution includes content donated by

Scan to learn

more about

IBM’s support

for SAFe.

11

http://bit.ly/ibmsafesupport






IBM z Systems Strategic Reuse : Component Hierarchies

Why do we need component hierarchies?

RTC SCM already has components

Components organize software artifacts into reusable units

files and change set history information

Reuse is at the level of components/streams

Common software in its own component can be reused in many streams

in a stream are modifiable

This is good!


IBM z Systems

What does it mean to have component hierarchies?

Components will be shown as a tree rather than a flat list

Users will need to look at the only subset of components they work on and use

Collapse/ignore subtrees that are not interesting

Users can easily operate on an entire subtree of components, for example:

Adding a component would also add its subcomponents

Creating a baseline would baseline the whole subtree

Comparing two components would compare their subcomponents as well


IBM z Systems

How will hierarchy relationships be managed?

Each component can have a list of subcomponents

In a stream or workspace, the list is used to organize components into tree structures

The subcomponent list is optional

If no subcomponent lists exist, components are shown as a flat list, as always

The list does not point to a specific version of a component

It points to the “version of” that component in the local stream or workspace

displayed as


IBM z Systems

About reuse in Team Concert

A component can be used in many streams

A component can be reused multiple places in a hierarchy

Circular dependencies will be detected to avoid issues

A component and its subcomponents are used as a unit

Adding a subcomponent will add its full subtree


IBM z Systems

Hierarchy operations

Some examples of operations available on a hierarchy:

Add subcomponent – would add a component and its subtree

Remove subcomponent – would remove a subcomponent and its subtree

Create baseline – would baseline the component and its subtree

Replace baseline – would replace the baseline for the component and its subtree

Compare – would compare the components and their subtrees

Load component – load a component and all its subcomponents…

This also enables quickly finding all the places a component is reused within a stream. Some use cases:

Before upgrading a shared component to a new version, find all the programs that depend on it, so you’d know what would need to be updated and retested

After finding a defect in a shared component, find all the programs that may be affected


IBM z Systems

RTC for the Enterprise : Build Map Editor

The Build Map editor is reorganized. The input and output sections

are displayed on separate tabs, so that you can find useful

information more easily.


IBM z Systems

RTC for the Enterprise : Build Result editor

The Build Result Editor has been completely reworked to improve usability

New front page

New preferences page

New format

New filters

New icons


IBM z Systems

RTC for the Enterprise : Dependency build and promotion

Personal builds are prevented from using the wrong build workspace

A new precondition prevents you from requesting a personal dependency build that uses a wrong build workspace.


IBM z Systems

UrbanCode Deploy: Press a button, a complex app is deployed to an environment

Inventory System


IBM z Systems

Fit with Rational Team Concert

Team Concert:

RTC provides a good source control system, an automated build tool,

and work item tracking. It does a great job of helping developers

collaborate and make the changes that will need deploying.

Key Integrations:

• RTC Can automatically push new builds to UrbanCode Deploy

• UrbanCode Release can aggregate work-item information from

many projects into summary views showing how complete the

larger release effort is, and which applications to worry about.

Story:

RTC is helping your developers with Agile and continuous integration

(build). That will mean more stuff flowing downstream to release

teams. They will need better tools to keep up.


IBM z Systems

But for Mainframe Applications….

Application size and complexity mean only changes are deployed to highly dependent operating environments

Unique deployment targets

High volume of small deployments

RTC understands what needs to be in a deployment and how it needs to be deployed

SO……….


IBM z Systems

RTC for the Enterprise : UrbanCode Deploy Integration

RTC can now be used to directly generate z/OS packages that are

deployable from IBM UrbanCode Deploy

The packages can be built from Work Items and will only contain changed items.

RTC ensures the completeness and integrity of the packages and registers them with UrbanCode

RTC passes deployment information for the packages to UrbanCode Deploy


IBM z Systems

RTC Packaging for UrbanCode Deploy RTC can now package build results (e.g. for work items) into Urban

Code Deploy component versions to be deployed.


IBM z Systems

Build outputs can be set to deploy types…

Your language definition’s translators can specify deployment types on the outputs they generate during a build…

RTC Packaging for UrbanCode Deploy


IBM z Systems

…and deployTypes can direct UCD actions

And UrbanCode Deploy processes can use these deploy types to determine and execute special actions at deploy time…

Filter

Execute

RTC Packaging for UrbanCode Deploy


IBM z Systems

UrbanCode Release: Manage Related Applications Quick Qualifying Question: When you release, do you do just one app or do you coordinate changes to several?

Customer Portal

Inventory System

Customer Accounts

Credit Services

Planning is complex Development work must be coordinated

across teams. Release schedules aligned.

Development is complex. Interfaces understood, developed &

configured appropriately. Environments

composed & connected.

Testing complex. Are the correct versions of all applications in

my environment? In which application did

the bug occur? Is the bug a code issue or

misconfiguration of the multi-application

environment?

Change Management is

complex. If we pull an application from the Release

what will what other applications or projects

will be affected? and dozens or hundreds more…


IBM z Systems

UCD Basics

Applications are made of Components

Applications and Components have Processes

A Component Process is a fine-grained series of user-defined steps that operate on the component’s artefacts

An Application Process orchestrates the order that component processes will be put onto the deployment target. Application processes can run manually, automatically on a trigger condition, or on a user-defined schedule.

Application Environments are typically for different uses of the application usually but not always different stages of the development lifecycle (Dev, System Testing, QA, Production.....). Environments have Resources defined to them

Resources represent deployment targets . Each resource tracks which versions of various components are currently deployed to it.

The Calendar is used to view deployment history and plan upcoming deployments. It can also schedule recurring deployments that will kick off automatically.

4/16/15


IBM z Systems

A further selected glossary of RTC and UCD terms

June, 2015

mailto:[email protected]


IBM z Systems

Glossary of general RTC build terms: (link) Build Definition: An object that defines a build, such as a weekly project-wide integration

build. This links to all necessary information to run a build – e.g. build workspace, build engine(s), the list of build results, etc.

Build Engine: The representation of a build system that runs on a dedicated server.

Stream: In source control management, a repository object that includes one or more components. Streams are typically used to integrate the work that is stored in repository workspaces. Team members deliver their changes to the stream and accept changes from other team members into their repository workspaces from the stream.

Repository Workspace: A repository object that includes one or more components. Repository workspaces are typically used by individual team members to contain their changes in progress. Team members deliver their changes from their repository workspace to the stream and accept changes from other team members into their repository workspace from the stream. Every repository workspace has an owner, and only the owner can make changes in the workspace. See also workspace.

Build workspace: A repository workspace which is dedicated to building; it contains the configuration (i.e. state) of the source being built.

Build request: An instance of a request to process a build, executed from a build definition through a build agent, producing built outputs reported in a build result.

Build result: A consolidated record of activities, logs and reports generated from a build request.

http://www-01.ibm.com/support/knowledgecenter/SSCP65_6.0.0/com.ibm.help.common.jazz.calm.doc/topics/glossary.html?cp=SSCP65_6.0.0

http://www-01.ibm.com/support/knowledgecenter/SSCP65_6.0.0/com.ibm.help.common.jazz.calm.doc/topics/glossary.html?lang=en-us&cp=SSCP65_6.0.0#gloss_W__x2096037


IBM z Systems

Glossary of RTC-EE Dependency Build terms

Data set Definition: A Jazz model object that describes a data set on z/OS and is stored in the Rational Team Concert Jazz repository. If the data set already exists, the data set definition must specify just the data set name. If the data set is new, the data set definition must specify both the name of the data set, and the characteristics of the data set, such as record format. Every data set that a build process references must correspond to a data set definition.

Language Definition: A Jazz model object that serially connects the translators used to build an artifact. The association of a language definition to an artifact provides instructions for how the artifact should be built.

Source Code Data: Metadata, dependency properties, and other user-defined data that are created and updated periodically by running scanners against the source code. The data can be queried, edited, and used to analyze the impact of potential changes. Source code data is used by dependency builds to determine which dependant artifacts have changed and therefore require that buildable files be rebuilt.

System Definitions: A collective term referring to a group of definitions in Rational Team Concert, including IBM i libraries, z/OS data set definitions, language definitions, and translators.

Translator: A Jazz model object that describes a single build step in which a translator executable program is invoked with the required inputs and outputs. Inputs and outputs are the same as z/OS data sets or IBM i libraries, so a translator must reference multiple data set or library definitions.


IBM z Systems

Glossary of UrbanCode Deploy terms: (link) Application: One or more computer programs or software components that provide a

function in direct support of a specific business process or processes.

Component*: A representation of deployable items and the user-defined processes that operate on them, usually by deploying them.

Environment: A user-defined collection of resources that hosts an application.

Process: Automated tasks that run on agents. See generic processes, component processes, and application processes. See also application process, component process, generic process.

Resource: A user-defined construct that is based on the architectural model of IBM UrbanCode Deploy. A resource represents a deployment target.

Resource inventory: A list 0f all of the components that are deployed to an agent resource. See alsocomponent inventory.

Snapshot*: A collection of specific versions of components. Typically, a snapshot represents a set of component versions that are known to work together.

Version: A group of resources that represent a particular version of a component. See also full version, incremental version.

*Note: RTC also has Components, which are an atomically versionable (baseline-able) set of source, as opposed to UCD’s atomically deployable Component. RTC also has Snapshots, which are a collection of source baselines, as opposed to a collection of versions. RTC’s terms : source configurations, UCD’s : deployable units

http://www-01.ibm.com/support/knowledgecenter/SS4GSP_6.1.0/com.ibm.udeploy.doc/topics/glossary.html?lang=en

http://www-01.ibm.com/support/knowledgecenter/SS4GSP_6.1.0/com.ibm.udeploy.doc/topics/glossary.html?lang=en-us#gloss_A__x2000181

http://www-01.ibm.com/support/knowledgecenter/SS4GSP_6.1.0/com.ibm.udeploy.doc/topics/glossary.html?lang=en-us#gloss_C__x7204618


http://www-01.ibm.com/support/knowledgecenter/SS4GSP_6.1.0/com.ibm.udeploy.doc/topics/glossary.html?lang=en-us#gloss_G__x7205165




http://www-01.ibm.com/support/knowledgecenter/SS4GSP_6.1.0/com.ibm.udeploy.doc/topics/glossary.html?lang=en-us#gloss_F__x7260270

http://www-01.ibm.com/support/knowledgecenter/SS4GSP_6.1.0/com.ibm.udeploy.doc/topics/glossary.html?lang=en-us#gloss_I__x7260280


IBM z Systems

www.ibm.com/software/rational


IBM z Systems


IBM z Systems

4/16/15


IBM z Systems

RTC Build (in general)

Build Engine Stream

RTC repository Build host

Build

workspace

Build

Definition

Build

Request

Build

result

Source

Source

Buildable

source

Source

Built

executables

Programs


IBM z Systems

RTC Build (in general)

Build Engine Stream

RTC repository Build host

Build

workspace

Build

Definition

Build

Request

Build

Result

Source

Source

Buildable

source

Source

Built

executables

Programs

(2) Accept

Changes

(1) Request

Build

(3) Load source changes

(4) Build

Report results

Incoming

changes


IBM z Systems

Glossary of general RTC build terms: (link) Build Definition: An object that defines a build, such as a weekly project-wide integration

build. This links to all necessary information to run a build – e.g. build workspace, build engine(s), the list of build results, etc.

Build Engine: The representation of a build system that runs on a dedicated server.

Stream: In source control management, a repository object that includes one or more components. Streams are typically used to integrate the work that is stored in repository workspaces. Team members deliver their changes to the stream and accept changes from other team members into their repository workspaces from the stream.

Repository Workspace: A repository object that includes one or more components. Repository workspaces are typically used by individual team members to contain their changes in progress. Team members deliver their changes from their repository workspace to the stream and accept changes from other team members into their repository workspace from the stream. Every repository workspace has an owner, and only the owner can make changes in the workspace. See also workspace.

Build workspace: A repository workspace which is dedicated to building; it contains the configuration (i.e. state) of the source being built.

Build request: An instance of a request to process a build, executed from a build definition through a build agent, producing built outputs reported in a build result.

Build result: A consolidated record of activities, logs and reports generated from a build request.

http://www-01.ibm.com/support/knowledgecenter/SSCP65_6.0.0/com.ibm.help.common.jazz.calm.doc/topics/glossary.html?cp=SSCP65_6.0.0

http://www-01.ibm.com/support/knowledgecenter/SSCP65_6.0.0/com.ibm.help.common.jazz.calm.doc/topics/glossary.html?lang=en-us&cp=SSCP65_6.0.0#gloss_W__x2096037


IBM z Systems

RTC repository

RTC-EE Dependency Build has additional System Definitions

Data set

Definition Defines physical

attributes of

resources used in

build: e.g. reclen,

compiler location,

zFolders

Language

Definition Classifies source

members to a

type of buildable

object, & lists the

translators to

build with.

Translator

A build command.

Defines inputs &

outputs from data

set definitions

These configure what buildable resources are, and how to build them


IBM z Systems

RTC repository

RTC-EE Dependency Build

Scanners Language-dependent

scanners find &

update dependency

relationships

Build Maps

Representation of

the outputs which

are already built

Incoming

Changesets New source

changes that

haven’t been built

System

Definitions How to build

outputs from

inputs

Source Code

Data Repository of

dependency

relationships What needs to be

loaded?

What needs to be

rebuilt?

How to build it?

Dep

Build

Build Engine

Buildable

source

Source

Built

executables

Programs

Load what’s

needed

Build what’s

needed


IBM z Systems

RTC-EE: UrbanCode packaging integration

June, 2015

Sean Babineau, Rational Team Concert – Enterprise Extensions

[email protected]




IBM z Systems

Overview

UCD z/OS packaging from RTC before RTC 6.0

Continuous Integration model – build and deploy in series

You can call a post-build process:

Which collects the list of new outputs generated from that build, and generate a UCD package manifest from this

Invokes the UCD command-line interface on the host to generate and register this package as a UCD component version

Or, you can script a manifest and invoke UCD’s command interface (BUZTOOL rexx or shell script)

This package can then be deployed to z/OS environments configured in UCD

With RTC 6.0 (and UCD 6.1.1.*)

RTC-EE packaging model – build changes, deploy on demand

package at work item scope

and/or filter by object name/timestamp


IBM z Systems

RTC-UCD integration before RTC 6.0

DRTCBLD.rex

buildReport.xml

Post-build

Command

Build Def Buildreportparser.js

Buztool.sh

Ucd_shiplist.xml

*.SBUZEXEC(BUZTOOL)

UCD package repo

(in USS)

UCD pkg

UCD client

UCD server UCD version

RTC post-build command executes a REXX exec to parse build outputs and create package from it


IBM z Systems

UCD shiplist

A UCD shiplist is required input to tell BUZTOOL what resources to package

This can be manually edited XML file on the host, passed to BUZTOOL

Or it can be generated (the DRTCBLD post-build process generates this from build output)


IBM z Systems

UCD shiplist example

<?xml version="1.0"?>

<manifest type="MANIFEST_SHIPLIST">

<container name="TONY.MORT.RTCDEV.LOAD" type="PDS"

deployType="CICS_LOAD">

<resource name="JKEMORT" type="PDSMember" />

<resource name="JKECMORT" type="PDSMember" />

</container>

<container name="TONY.MORT.RTCDEV.DBRM" type="PDS">

<resource name="JKECMORT" type="PDSMember"

deployType="DBRM"/>

</container>

</manifest>

• This is an example shiplist used for creating UCD packages:


IBM z Systems

RTC-EE 6.0 - Packaging for UrbanCode Deploy integration

RTC packaging knows where | when | what outputs were built from RTC’s dependency build; it can exploit this knowledge for determining packaging contents.

UCD has a more sophisticated UI and framework for designing deployment processes, executing deployment processes, and keeping track of deployment states.

47

Package with

RTC’s intelligence Packages are stored

in UrbanCode Deploy

Use UrbanCode Deploy

to execute and track

deployment of packages


IBM z Systems

UCD packaging options

Specify the necessary parameters to interface with UCD to create UCD deployable packages


IBM z Systems

Deployment type

RTC-EE translator definitions: Specify Deployment types for outputs, which can be used by UCD to filter deploy actions based on type.


IBM z Systems

UCD z/OS component


IBM z Systems

UCD component versions


IBM z Systems

UCD version artifacts – i.e. package contents


IBM z Systems

UCD z/OS deployment process example

A. Get package to destination

B. Deploy datasets from package

C. Filter members by deploy type

D. Iteratively bind the filtered list


IBM z Systems

UCD process step using deploy type

Filter each member of deploy type

DBRM, and iterate the next

process step on the resulting list


IBM z Systems Strategic Reuse : Component Hierarchies

Why do we need component hierarchies?

RTC SCM already has components

Components organize software artifacts into reusable units

files and change set history information

Reuse is at the level of components/streams

Common software in its own component can be reused in many streams

in a stream are modifiable

This is good!


IBM z Systems

What does it mean to have component hierarchies?

Components will be shown as a tree rather than a flat list

Users will need to look at the only subset of components they work on and use

Collapse/ignore subtrees that are not interesting

Users can easily operate on an entire subtree of components, for example:

Adding a component would also add its subcomponents

Creating a baseline would baseline the whole subtree

Comparing two components would compare their subcomponents as well


IBM z Systems

How will hierarchy relationships be managed?

Each component can have a list of subcomponents

In a stream or workspace, the list is used to organize components into tree structures

The subcomponent list is optional

If no subcomponent lists exist, components are shown as a flat list, as always

The list does not point to a specific version of a component

It points to the “version of” that component in the local stream or workspace

displayed as


IBM z Systems

About reuse in Team Concert

A component can be used in many streams

A component can be reused multiple places in a hierarchy

Circular dependencies will be detected to avoid issues

A component and its subcomponents are used as a unit

Adding a subcomponent will add its full subtree


IBM z Systems

Hierarchy operations

Some examples of operations available on a hierarchy:

Add subcomponent – would add a component and its subtree

Remove subcomponent – would remove a subcomponent and its subtree

Create baseline – would baseline the component and its subtree

Replace baseline – would replace the baseline for the component and its subtree

Compare – would compare the components and their subtrees

Load component – load a component and all its subcomponents…

This also enables quickly finding all the places a component is reused within a stream. Some use cases:

Before upgrading a shared component to a new version, find all the programs that depend on it, so you’d know what would need to be updated and retested

After finding a defect in a shared component, find all the programs that may be affected

making the dinosaur dance - rdz, rtc and urbancode lunch and learn slides

Technology