The Canonical Data ZoneIssues in data selection for Service Oriented Architectures

Gary Farrow

IT Architect

Director Triari Consulting

2

Objectives

By the end of this session you will understand:

• The advantages of adopting a canonical data model in a SOA

• The key issues in selecting an appropriate canonical data model

• The architectural impact of operating using a canonical data zone

3

Agenda

1. Context

2. Reference Models– SOA Reference Architecture

– Canonical Data Meta-Model

3. The Canonical Data Zone Defined

4. Issues in Canonical Data Selection– Payments Hub and Payments Engine Case Studies

– CDM Strategies & Evaluation

5. Conclusions

4

Context

In this section the context of the canonical data problem is now described

5

Value PropositionScenario

• Co-existence with multiple external partners / standards

• Multiple delivery channels

Design Goals

• Interoperability with multiple industry standards

• Channel harmonisation

– Common business process

– Common data necessary to support these

• To reduce integration complexity

Advantages

• Architectural simplicity

• Reduces integration complexity

– IxO problem reduced to I+O

• Common processes

– Single layer decoupled from channels

• Supports high coherence principle

– Channel components are responsible for presentation

External Partners / Industry Standards

Internal Systems / Components

CanonicalData

Delivery Channels

External

Internal

Inputs (I)

Outputs (O)

Reusable services shared across different channel contexts

6

Reference Models

In this section two reference models are described that are used in the analysis of

the canonical data problem

7

SOA Reference Architecture

Overview

• Hierarchical model for services within the enterprise

• Four layers inter-leaved with integration services

• Use as a ‘ideal world’ framework for SOA solutions

– Flexible

– Support bespoke design

– Supports package solution

– Supports domain model (domains sit in the business services layer. )

Uses

• Couple with architectural principles to provide SOA best practice

• Illustrate Enterprise Architecture patterns

• Defined anti-patterns relating to SOA misuse

Context

• Use this for analysis of the data requirements for SOA

– Which layers of the architecture use canonical data?

– Are there different selections of canonical data the different layers?

8

Canonical Data Meta-Model

Require agreement on how logical information is represented3 Layered Model 1. Business Concepts• Describes the core entities of the business

and their relationships2. Message Models• Constructed from the business entities• Messages constructed to fulfil the

requirements of steps in a processes3. Syntax• The implementation technology of the

message models• Format in which the message is structured

• Programming Objects – e.g Java Object

• XML, JSON• Process Engine specific – e.g. IBM Service Component Architecture

/ Business Objects

• Data Dictionary captures all the required meta data

Standardising across all data categories in all SOA architectural layers is neither practical or necessary

9

Types of Data Standards

• Standards paradox• Two dimensions to a standard1. Open / Closed• Refers to the extent to which

information about the standard is published

1. Sponsored / Unsponsored• Refers to the maintenance of the

standard by a single business organisation or by a standardisation body

CDM Vulnerabilities– Basing on a standard makes it

vulnerable to changes in the standard– Extent of vulnerability dependent on

the type of the standard– Design objective make architecture

robust in response to changes

SWIFT

PDF

MS Word

J2EE

ISO xxxx

The wonderful thing about standards is that there are so many of them to choose from

10

The Zone

In this section the Canonical Data Zone is defined in terms of the Reference Models

11

Canonical Data Zone Defined

• Given the objectives of common processes and reuse of Business Services canonical data is considered key:

• In the Process Services layer

• Within its orchestrations

In the Zone

• Business Concepts are reused to achieve common processes

• Common Messaging model critical for service call between Process and Business Service layer

• Common Syntax is desirable within the process layer

Outside the zone:

• Business Concepts may be reused within the Data Services

• Common Messaging Model is not essential for Data Services

• Strong encapsulation of data services is preferred when implementing custom Business Services

• Package solutions provide black box business services

• Syntax implementation differ to support different technology solutions for the components

• Legacy

• Package solutions

• Actual data storage within a database

12

Issues in Canonical Data Selection

In this section, issues in selecting a canonical data model for a given business

domain are described in the context of payments processing case studies

13

Case Study 1: Payment Hub

• Scenarios - merger / acquisition / modernisation

• Payments hub solution proposed

• Common payments processing decoupled from product systems

• Route to two or more product systems

• Framework technology solution for the Hub chosen

Design Issues

• Package integration

– Core banking system vendors offer modules that already use industry specific format

– Transform BACS-Canonical-BACS (-Internal)

• Package design principles

– Use ‘out of the box’

– Scheme specific interfaces already provided

• Hub design principles

– Integration ‘heavy lifting’

– Execute common processes using canonical data model

– Minimise transformations (BACS-Canonical)

• Architectural tension

– Hub vs. package principle

– What is the appropriate CDM?

Framew

ork

14

Industry Standard CDZ

Scenario

• Scenario assumes open standard and adoption of the same within the CDZ

• Industry standard will change

• Enrichment of the payments message required

Advantages

• Out of the box message format & syntax

Disadvantages

• Processes and the business services interfaces are impacted in the change case

• Ability to respond rapidly is affected

• Does not meet functional business requirements

• Process and Business Services are tightly coupled to an external standard

– Impact is worse if standard is unsponsored

• Processes services are not harmonised

• Business services are not reused.

15

Custom CDZ

Scenario• Scenario assumes an open standard but

adoption of a custom canonical data model in the CDZ

• Industry standard changes• Integration service transforms into the

CDM

Advantages• Buffered from immediacy of industry

standard enhancements• Meets ‘real world’ requirements• Processes harmonised• Business Services reused

Disadvantages• Custom CDM development required

16

Qualitative Evaluation

Custom Industry Industry +

Industry+/-

Effort to Develop

Requirements Fit

Performance

Robustness to Change

• Strategies for CDM selection areshown opposite

Design Tradeoffs

• Industry+ and +/- offer bestrobustness to change with leastdevelopment effort

• Limit of reduction is determined bythe scope of the business processesthat must be supported

• Industry standard may offer leastdevelopment effort and providebest performance for demandingNFRs

• e.g. Faster Payments / CardsISO 8583

• Commonality of processing sacrificed to achieve NFR's

Design Solution

• IS020022 supports datarequirements for a number ofpayments schemes

• Enhanced / Reduced variant ofISO20022 selected for the Hubsolution

17

Case Study 2: Payments Engine

Scenario

• Relates to commercial payments processing CHAPS/SWIFT

• Payments engine package selected for payments processing

• Fulfilling Process Services &

• Some Business Services

• Engine weak on integration

• Lightweight integration middleware selected

• Reusable Services implemented using a CDM

• XML syntax / XSLT based transformations

Problem

• Business Concepts are similar in the channel and CDZ

• Message models in the CDZ are unique to the IT landscape of the bank

• CDZ Syntax is different to Channel Syntax

• Integration technology selected not suitable for the channel integration solution

• Delay and rework

Key Points

• Integration space is segmented

• Functional and NFR’s may be different

• Different technology implementations are allowed / required to support the CDZ

Package

XML/XSLT

SWIFT/MQ

CDZISO20022 / Custom / XML

ChannelSWIFT/ Proprietary/ message

18

Conclusion

The final section summarises some generalised design principles

19

Conclusions

• Principle: Channel harmonisation and Business Service reuse are maximised through a CDZ comprising the Process Services architectural layer and its associated Business Services interfaces

• A CDM based on an extended open, sponsored standard is generally optimal based on several identified evaluation criteria• Provides buffering to the immediacy of standards changes• To further optimise the CDM, consider also a simultaneously extended & reduced form of a standard

• It may be required to optimise the CDM to meet specific NFR’s• In these circumstances design tradeoffs have been highlighted

• Implementation of a CDZ infers that the Integration space is segmented• Variety of integration requirements must be supported • ‘One size fits all’ solution for integrations to and from the CDZ does not work

• Principle: The theoretical minimal CDM comprises the Business Concepts, Messaging Model and Syntax necessary to fulfil the business processes of a given domain

CDM = S Domain Data Requirements

20

Recap on Objectives

Now you have completed this session you understand:

• The advantages of adopting a canonical data model in a SOA

• The key issues in selecting an optimal canonical data model

• The architectural impact of operating using a canonical data zone

21

Thank You

Merci

Grazie

Gracias

Obrigado

Danke

Japanese

English

French

Russian

GermanItalian

Spanish

Brazilian PortugueseArabic

Traditional Cinese

Simplified Chinese

Thai