25 years interface management

25 Years Interface Management25 Years Interface Management

28.11.2016

Tarmo Ploom, Integrtion Architecture

Key Messages

� A good child (interface) has many names – SOA, API, Micro-

service …

� Technologies come and disappear

� Interface descriptions remain� Interface descriptions remain

�Management of interfaces is key in sustainable IT-Landscapes

28.11.2016Tarmo Ploom, Integrtion Architecture 2

Agenda

� SOA @ Credit Suisse

� Interface Taxonomy

� SOA and middleware footprint by Credit Suisse

� Interface Management history by Credit Suisse

�Classification of Interface Management Repositories

� Interface Management System – IFMS

�Quality Control (QC) Process

� Key Performance Indicators

�Q/A


� Key objectives – Reducing complexity and costs– Increase agility and flexibility

� Service-oriented Architecture (SOA) – Separation of concerns� Service interface (contract)

– Consumer-interface-provider

Service-oriented Architecture – SOA at Credit Suisse

Consumer

Co

rpo

rate

Se

rvic

es

Legal & Compliance

Operations

Distribution

Processing

Finance

Risk

3rd

Se

rvic

e P

rovid

ers

Operations

– Consumer-interface-provider coupling

– Inside vs. outside data� Service implementation

� Integration Building Blocks� Service Repository (IFMS)

� Service interface descriptions, versioned

� Implementation-independent� Governance (e.g. Golden Source)� Service Bus

Channels

Service Bus

Service Repository

Work

flow

s

Provider

Consumer

Interface Implementatio

n


Interface taxonomy

�CS language:

− Event: fire and forget style asynchronous communication

− Service: request and response style synchronous communication

− Bulk: Bulk data transfer (>100 MB)

− Interface: abstraction of communication (not grouping)


SOA footprint by Credit Suisse

� Synchronous services

− Managed, ca 5100

− Unmanaged, ca 3000

� Bulk Services

− Managed, ca 3200

− Unmanaged ca 25 000

� Event services

− Managed, ca 50

− Unmanaged, ca 40 000


Middleware footprint (measurements or minimum estimates)

AVG calls per

day

in % AVG message

KB

AVG load per

day MB

in %

Bulk 228 571 0.1% 15000 3 348 214 77.0%

B2B FT proxy 8 667 0.0% 3938 33 333 0.9%

CORBA 30 000 000 10.7% 6 175 781 4.0%

28.11.2016Tarmo Ploom, Integrtion Architecture

CORBA 30 000 000 10.7% 6 175 781 4.0%

WS 1 400 000 0.1% 20 2 930 0.1%

MQ 250 000 000 89.1% 3 789 063 18.0%

Total 280 387 238 100.0% 4 349 321 100.0%

7

Implementation history

� 1991 – 1999, Client/Server Banking System:

− Two tier approach

− Paper based interface management

� First interface repository (1997 – 2001):

− For service (CORBA) interfaces only

− PHP/MySQL based

� Second interface repository (2002 – 2006):

− For service (CORBA) interfaces only

− MOF 1.3 based, JAP platform

− Paper based SOA governance processes


Implementation, problems from the past

�How to manage SOA landscape consisting of thousands of

interfaces?

�Who are active consumers of interfaces?

�What interfaces are deployed and used in production?

�Decommissioning of interfaces?

�How to make SOA governance less bureaucratic?�How to make SOA governance less bureaucratic?

� Integration of interface repositories with other repositories.

� Integration of interface development with interface repository.

�How to bridge gap between interface design and implementation?


Framework for analysing interface repositories

SOA IDE

(Integrated Development

Environment)

Model

Driven SOA

Third generation

interface

repository

Fourth generation

interface

repository


Management of interface metadata

(classical interface repository)

SOA Governance

(engineering, decommissioning, etc.)

First generation

interface

repository

Second

generation

interface

repository

10

First generation interface repository

SOA IDE


Environment)

Model

Driven SOA

Third generation

interface

repository

Fourth generation

interface

repository




SOA Governance


First generation

interface

repository

Second

generation

interface

repository

11

First generation interface repository

� Passive management of interface metadata

� Interface catalog

− paper based

− Excel

− Confluence

− Etc.

28/11/2016Tarmo Ploom, Integrtion Architecture 12

First Generation Interface Repository, Architecture

� Two main components:

− Interface Dictionary

− Reporting


First Generation Interface Repository, Problems

� Problems

− What if there are more than 1000 or more interfaces?

− Who are active consumers of an interface?

− What interfaces are deployed and used in production?

− SOA standards enforcement?

− SOA long term direction?


Second generation interface repository

SOA IDE


Environment)

Model

Driven SOA

Third generation

interface

repository

Fourth generation

interface

repository




SOA Governance


First generation

interface

repository

Second

generation

interface

repository

15

Second Generation Interface Repository

� Elements:

− Management of interface metadata

− Active semi-automated SOA governance processes

� Interface portfolio management

� SOA governance processes:

− Interface engineering

− Interface decommissioning

− Interface migration

− Interface usage


Second Generation Interface Repository, Common Architecture


Second Generation Interface Repository, Problems

� Problems:

− SOA bureaucracy

− Resistance to SOA bureaucracy

− Integration with other repositories

− Gap between design and implementation of services


Third generation interface repository

SOA IDE


Environment)

Model

Driven SOA

Third generation

interface

repository

Fourth generation

interface

repository




SOA Governance


First generation

interface

repository

Second

generation

interface

repository

19

Third Generation Interface Repository

� Elements:


− Active semi-automated SOA governance processes and

− Integrated development environment

� Integration: � Integration:

− Graphical design environment

− Design repository

− Application portfolio

− Log manager

� Infotype reuse:

− Interface infotype dictionary


Third Generation Interface Repository,Common Architecture


Third Generation Interface Repository, Problems

� Problems:

− Integration

− Gap between interface design and implementation


Fourth generation interface repository

SOA IDE


Environment)

Model

Driven SOA

Third generation

interface

repository

Fourth generation

interface

repository




SOA Governance


First generation

interface

repository

Second

generation

interface

repository

23

Fourth Generation Interface Repository,Model Driven SOA Repository

� Elements:


− Active semi-automated SOA governance processes

− Integrated development environment

− Generation of interface stubs, skeletons and code

� Linking design to implementation:

− Software configuration management

− IDL/WSDL/XSD/PL1/Java generator


Fourth Generation Interface Repository, Common Architecture


Service Description - IFMS Example 1/2

General

DescriptionDescription

Name

Brief/Full Description

Usage Notes

Changes

NotesBusiness Objects

Focus Business Object

Attachments

Classification

InterfaceType

Scope

InterfaceGroup

Example createFxTrade

� Synchronous service for building channel user and machine interfaces

� Service requires offer

− streamed quote offers

− request for quote

� Service Data Model

− Order master data (generic)

− Product data

InterfaceGroup

Direction

ActionType/Sub-Category

Compliance


Service Description − IFMS Example 2/2

Criticality

Contract

Non-Functionals

Conditions &

Exceptions

Signature

Criticality

C3, I1

OrderMasterData

id : UUID

extRef : String (0..n)

state : LifeCycleStateCd

FXOrder

master : OrderMasterData

cpty : PartyId

traded : ::Amount

rate : ::FXTradedRate

value : ::Date


Service Governance – Quality Assurance ProcessCentrally defined process and federated execution (service

landscape)

� QC1– Integration Architecture

– Is the service understandable?

� Is the name well defined?

� Is the description self-

explanatory?

– Is the service necessary?

– Is the service positioned correctly?

� Domain

QC1

QC2

QC3

Accept

Rework

Accept

(Registration,

De

sig

nC

od

in

g

Te

stin

gA

na

lysi

s

� Domain

� Public vs. internal vs. partner

� QC2 – Cross-functional review team

– Is the service and its data types

well-designed and following the

principles?

– Is the service using the correct

types?

� QC3 – Tool & Process Support

– Are all QC Obligations fulfilled?

– Is it implemented as specified?

QC3(Registration,

industry

standards)Go-Live

“Normal” relation to

development phases

Governance Process Entry Points

QC1: – New service

QC2: – Extension of public service

QC3: – Extension of internal service

– Additional implementation (transport)


Service Design Principles – First Principles 1/2Service description defined according to principles, the assessment

criteria

� Design from a business perspective

– Service name captures business

intention

� Defined naming conventions

– Standardized verbs

– Objects refer to business objects

� Defined semantics

– Read returns objects with

attributes

<CRUD Verb><Qualified Object>

createFxOrderOffercreateFxOrder

<action><Qualified Object>On<Event>

createConfirmationOnNewFxOrder

createSettlementOrderOnNewFxOrder

attributes

– Search returns objects with their

identifying attributes

– Business constants defined for code

values (enumerations, code tables)

– Reuse

� Data types available in the domain

– Recurring structures and variants

thereof

� Information types (from landscape)

– Service-fulfillment exception standard

� System exceptions

� Business exceptions

� Business attentions (degraded

Enum OrderState

{active, canceled, matured}

Type FxOrder is ComplexType

traded : Amount

rate : ExchangeRate

state : OderState

…

Exception : OfferExpired

Attention : CreditApprovalPending


Service Design Principles – First Principles 2/2Service description defined according to principles, the assessment

criteria

� Design preferences

– Generic vs. use-case specific

– Aggregated vs. elementary

– Asynchronous vs. synchronous

integration

– Coarse grained – “right-grained”

� Wide and high, but limited depth

– Wide: many attributes

– High: generalized business

Union Level {withMarkup Boolean,

withLinks Boolean}

Type FxOrder is ComplexType

identifier : OrderId

...

details : Level

party : PartyId

markup : FxMarkups (0..1)

– High: generalized business

objects

– Depth: few related business

object

with attributes

� Grouping supporting level of detail

selection

– Industry-standards vs. enterprise-

specific

� FpML, ISO 20022, etc.

� Design non-functional aspects

– Risk-adjusted criticality

� Confidentiality, integrity, availability

– Operational independence

– Size, latency, throughput

CT::CreditTransferSEPAasEBA

{pacs.008,pacs.002,camt.056,

camt.029,pacs.004}

Interbank credit transfer service using SEPA (Single Euro Payment Area) with the central counterparty EBA Clearing (Euro Banking Association).

ISO 20022 based protocol: messages mean simply “payment”, “error”, “request-recall”, “refuse-recall” and “accepted-recall”.

links : FxLinks (0..1)

...


Service DB/IFMS Adoption: 2001 - 2015

C2WS migration launched - Q4

2012

DWH import - Q4

2014


2012

IFMS 3.0 - Q3 2011

31

IFMS Adoption Trend: Q2 2013 – Q2 2015

Ratio modeled to registered increased from 24% to 57% from Q2/2013 to Q2/2015


IFMS Adoption Trend by Quarter: Interface Type

Increased the # of active synchronous interfaces by 370

or +8% in 12 months.

Increased the # of active bulk interfaces by > 2300 or

+330% in 12 months.


+330% in 12 months.

Increased the # of active message interfaces by 1

or +7% in 12 months.

33

IFMS Interface Structure in Q2 2015


Generator Stats: Facts & Figures Q1/14 – Q1/15

� > 600 Users out of 53 SubDepts

� > 12’000 Generations Accomplished

− 8’293 CLI’s

− 7’244 JSB’s

− 7’389 Web Service Wrappers

− 138 XSD’s

− 117 CORBA Wrappers− 117 CORBA Wrappers

� Total > 5.9 Mio Generated Artifacts

� The average generation time per artifact was reduced dramatically in order the handle the elevated load, caused by the increased user community and the intense C2WS activities.


QC Stats: Facts & Figures 2009 - 2015


IFMS Adoption: Provided & Consumed Interfaces

� 703 out of 5600 applications are

currently using IFMS*.

� In total 323 Applications provide

8’003 services and 624 applications

consume 13’910 services.

� The IFMS adoption is with >95% of

provisions and consumptions more provisions and consumptions more

or less limited to the IT PWS

department.

� A providing application provides in

median 7 services and a consuming

applications consumes in median 5

services.

� The top application (DWH Staging)

provides and consumes >1’000

services.

* Report date: May 27 2015


Applications using IFMS by region

AmericasEMEA

Switzerland


Americas

Asia-Pacific

Overall Applications using IFMSApplication Ownership:

Source: IFMS and CATI, Report date: 27.05.2015

38

Questions?


25 years interface management

Engineering