The Canonical Data ZoneIssues in data selection for Service Oriented Architectures
Gary Farrow
IT Architect
Director Triari Consulting
2
Objectives
By the end of this session you will understand:
• The advantages of adopting a canonical data model in a SOA
• The key issues in selecting an appropriate canonical data model
• The architectural impact of operating using a canonical data zone
3
Agenda
1. Context
2. Reference Models– SOA Reference Architecture
– Canonical Data Meta-Model
3. The Canonical Data Zone Defined
4. Issues in Canonical Data Selection– Payments Hub and Payments Engine Case Studies
– CDM Strategies & Evaluation
5. Conclusions
4
Context
In this section the context of the canonical data problem is now described
5
Value PropositionScenario
• Co-existence with multiple external partners / standards
• Multiple delivery channels
Design Goals
• Interoperability with multiple industry standards
• Channel harmonisation
– Common business process
– Common data necessary to support these
• To reduce integration complexity
Advantages
• Architectural simplicity
• Reduces integration complexity
– IxO problem reduced to I+O
• Common processes
– Single layer decoupled from channels
• Supports high coherence principle
– Channel components are responsible for presentation
External Partners / Industry Standards
Internal Systems / Components
CanonicalData
Delivery Channels
External
Internal
Inputs (I)
Outputs (O)
Reusable services shared across different channel contexts
6
Reference Models
In this section two reference models are described that are used in the analysis of
the canonical data problem
7
SOA Reference Architecture
Overview
• Hierarchical model for services within the enterprise
• Four layers inter-leaved with integration services
• Use as a ‘ideal world’ framework for SOA solutions
– Flexible
– Support bespoke design
– Supports package solution
– Supports domain model (domains sit in the business services layer. )
Uses
• Couple with architectural principles to provide SOA best practice
• Illustrate Enterprise Architecture patterns
• Defined anti-patterns relating to SOA misuse
Context
• Use this for analysis of the data requirements for SOA
– Which layers of the architecture use canonical data?
– Are there different selections of canonical data the different layers?
8
Canonical Data Meta-Model
Require agreement on how logical information is represented3 Layered Model 1. Business Concepts• Describes the core entities of the business
and their relationships2. Message Models• Constructed from the business entities• Messages constructed to fulfil the
requirements of steps in a processes3. Syntax• The implementation technology of the
message models• Format in which the message is structured
• Programming Objects – e.g Java Object
• XML, JSON• Process Engine specific – e.g. IBM Service Component Architecture
/ Business Objects
• Data Dictionary captures all the required meta data
Standardising across all data categories in all SOA architectural layers is neither practical or necessary
9
Types of Data Standards
• Standards paradox• Two dimensions to a standard1. Open / Closed• Refers to the extent to which
information about the standard is published
1. Sponsored / Unsponsored• Refers to the maintenance of the
standard by a single business organisation or by a standardisation body
CDM Vulnerabilities– Basing on a standard makes it
vulnerable to changes in the standard– Extent of vulnerability dependent on
the type of the standard– Design objective make architecture
robust in response to changes
SWIFT
MS Word
J2EE
ISO xxxx
The wonderful thing about standards is that there are so many of them to choose from
10
The Zone
In this section the Canonical Data Zone is defined in terms of the Reference Models
11
Canonical Data Zone Defined
• Given the objectives of common processes and reuse of Business Services canonical data is considered key:
• In the Process Services layer
• Within its orchestrations
In the Zone
• Business Concepts are reused to achieve common processes
• Common Messaging model critical for service call between Process and Business Service layer
• Common Syntax is desirable within the process layer
Outside the zone:
• Business Concepts may be reused within the Data Services
• Common Messaging Model is not essential for Data Services
• Strong encapsulation of data services is preferred when implementing custom Business Services
• Package solutions provide black box business services
• Syntax implementation differ to support different technology solutions for the components
• Legacy
• Package solutions
• Actual data storage within a database
12
Issues in Canonical Data Selection
In this section, issues in selecting a canonical data model for a given business
domain are described in the context of payments processing case studies
13
Case Study 1: Payment Hub
• Scenarios - merger / acquisition / modernisation
• Payments hub solution proposed
• Common payments processing decoupled from product systems
• Route to two or more product systems
• Framework technology solution for the Hub chosen
Design Issues
• Package integration
– Core banking system vendors offer modules that already use industry specific format
– Transform BACS-Canonical-BACS (-Internal)
• Package design principles
– Use ‘out of the box’
– Scheme specific interfaces already provided
• Hub design principles
– Integration ‘heavy lifting’
– Execute common processes using canonical data model
– Minimise transformations (BACS-Canonical)
• Architectural tension
– Hub vs. package principle
– What is the appropriate CDM?
Framew
ork
14
Industry Standard CDZ
Scenario
• Scenario assumes open standard and adoption of the same within the CDZ
• Industry standard will change
• Enrichment of the payments message required
Advantages
• Out of the box message format & syntax
Disadvantages
• Processes and the business services interfaces are impacted in the change case
• Ability to respond rapidly is affected
• Does not meet functional business requirements
• Process and Business Services are tightly coupled to an external standard
– Impact is worse if standard is unsponsored
• Processes services are not harmonised
• Business services are not reused.
15
Custom CDZ
Scenario• Scenario assumes an open standard but
adoption of a custom canonical data model in the CDZ
• Industry standard changes• Integration service transforms into the
CDM
Advantages• Buffered from immediacy of industry
standard enhancements• Meets ‘real world’ requirements• Processes harmonised• Business Services reused
Disadvantages• Custom CDM development required
16
Qualitative Evaluation
Custom Industry Industry +
Industry+/-
Effort to Develop
Requirements Fit
Performance
Robustness to Change
• Strategies for CDM selection areshown opposite
Design Tradeoffs
• Industry+ and +/- offer bestrobustness to change with leastdevelopment effort
• Limit of reduction is determined bythe scope of the business processesthat must be supported
• Industry standard may offer leastdevelopment effort and providebest performance for demandingNFRs
• e.g. Faster Payments / CardsISO 8583
• Commonality of processing sacrificed to achieve NFR's
Design Solution
• IS020022 supports datarequirements for a number ofpayments schemes
• Enhanced / Reduced variant ofISO20022 selected for the Hubsolution
17
Case Study 2: Payments Engine
Scenario
• Relates to commercial payments processing CHAPS/SWIFT
• Payments engine package selected for payments processing
• Fulfilling Process Services &
• Some Business Services
• Engine weak on integration
• Lightweight integration middleware selected
• Reusable Services implemented using a CDM
• XML syntax / XSLT based transformations
Problem
• Business Concepts are similar in the channel and CDZ
• Message models in the CDZ are unique to the IT landscape of the bank
• CDZ Syntax is different to Channel Syntax
• Integration technology selected not suitable for the channel integration solution
• Delay and rework
Key Points
• Integration space is segmented
• Functional and NFR’s may be different
• Different technology implementations are allowed / required to support the CDZ
Package
XML/XSLT
SWIFT/MQ
CDZISO20022 / Custom / XML
ChannelSWIFT/ Proprietary/ message
18
Conclusion
The final section summarises some generalised design principles
19
Conclusions
• Principle: Channel harmonisation and Business Service reuse are maximised through a CDZ comprising the Process Services architectural layer and its associated Business Services interfaces
• A CDM based on an extended open, sponsored standard is generally optimal based on several identified evaluation criteria• Provides buffering to the immediacy of standards changes• To further optimise the CDM, consider also a simultaneously extended & reduced form of a standard
• It may be required to optimise the CDM to meet specific NFR’s• In these circumstances design tradeoffs have been highlighted
• Implementation of a CDZ infers that the Integration space is segmented• Variety of integration requirements must be supported • ‘One size fits all’ solution for integrations to and from the CDZ does not work
• Principle: The theoretical minimal CDM comprises the Business Concepts, Messaging Model and Syntax necessary to fulfil the business processes of a given domain
CDM = S Domain Data Requirements
20
Recap on Objectives
Now you have completed this session you understand:
• The advantages of adopting a canonical data model in a SOA
• The key issues in selecting an optimal canonical data model
• The architectural impact of operating using a canonical data zone
21
Thank You
Merci
Grazie
Gracias
Obrigado
Danke
Japanese
English
French
Russian
GermanItalian
Spanish
Brazilian PortugueseArabic
Traditional Cinese
Simplified Chinese
Thai