soa for managers full day - dovel technologiesdoveltech.com/wp-content/uploads/2017/09/soa... · 6...

1

Copyright © 2008, ZapThink, LLC 1

SOA for Managers

Jason BloombergManaging Partner

ZapThink LLC

Take Credit Code: SOAMAN


Business Constant: Change

CHANGE

CompetitionCompetition

Changing Changing MarketplaceMarketplace Customer Customer

DemandsDemands

Mergers & Mergers & AcquisitionsAcquisitions

Optimizing Optimizing ProcessesProcesses

New New TechnologiesTechnologies

Business Business PartnersPartners

A Business is Never A Business is Never STATICSTATIC

2


We’ve had IT challenges for years …

Source: Microsoft


… but even after yesterday’s promises…

Source: Microsoft

3


… we still have the same IT mess, only worse.


The Problems of IT areThe Problems of Business

4


If you are in a Hole, Stop Digging!

• IT Decision Making’s Fatal Flaw:– Choice between “take some extra time & money and do

it right” vs. “give me what I want the cheapest & fastest way” – guess which wins?

• Repeat enough times, leads to the IT “Rat’s Nest”


The Business Inflexibility Trap

• Inflexibility is the Mother of All Business Problems– If you’re flexible enough, you can solve all the other

problems

• Information Technology (IT) is an impediment to business change– It wasn’t supposed to be that way!

5


• Companies require Business Agility…

»Responding quickly and efficiently to change, and

»Leveraging change for competitive advantage

J

Business Agility

Agility is the key to innovationAgility is the key to innovation


Service Orientation:A Business Approach

• It’s not about connecting things, it’s about enabling business processes & continual change

• The core business motivation is business agility

• Rather than “rip and replace”old systems – make them work better together

• It’s not about technology, integration, or middleware

6


What is SOA?

• SOA is architecture – a set of best practices for the organization and use of IT, and the discipline to follow them

• Abstracts software functionality as loosely-coupled, Business Services

• Services can be composed into applications which implement business processes in a flexible way, without programming


Is SOA New?

• Service-Oriented Architecture has been around a while

• CORBA (Common Object Request

Broker Architecture) and DCOM (Microsoft Distributed Component Object

Model) two familiar approaches to SOA

• What’s new this time?

7


One Difference is Web Services

Web Services Coordination

WS-SecureConversation

WS-BusinessActivity

Web Services Metadata Exchange

Web Services for Remote Portlets

WS-Transfer

WS-CoordinationWS-Trust

BPEL and BPELJ

Web Services Business Activity Framework

Web Services Dynamic Discovery

Web Services Transaction

WS-AtomicTransaction

Web Services Eventing

WSDL

WS-Federation

WS-Acknowledgement

WS-ReliableMessaging

WS-Addressing

WS-Policy

WS-MessageData

WS-CallBack

WS-Choreography Interface

SOAP

UDDI


SOA vs. Web Services

MainframeLogic

MainframeLogic

EISEIS CustomApp

CustomApp

Web Services Web Services Web Services

Security SecuritySecurity

Messaging MessagingMessaging

TransactionsTransactions

MainframeLogic

MainframeLogic

EISEIS CustomApp

CustomApp

Web Services Web ServicesWeb Services

Standard n-Tier Architecture with Web Services

SOA leveraging Web Services

Source: HP, ZapThink

SOA InfrastructureSOA InfrastructureTransactions Messaging Security DiscoveryM

anag

emen

t

Mon

itori

ng

Content-Based Routing Transformations

Business Services

8


Confusing SOA & Web Services

• SOA is an architectural approach, Web Services are standards-based interfaces

• Web Services serve a role in SOA implementations when open standards are required

• Neither is necessary for the other

• Vendors promote this confusion by calling Web Services integration “SOA”


If not Web Services, Then What?

• To be a Web Service, a Service interface must conform to WSDL

• WSDL is seriously lacking in many respects

• So, many Services follow contract templates that go beyond WSDL

• The architects’ dilemma: Web Service or Service with internal standard contract template?

9


Web Services are the Trees….

SOA is the Forest


The Distributed Computing Pendulum

Mainframe

SOA

Web

Client/Server

Centralized Decentralized

10


Service orientation…the next big thing?

Approach TimeframeProgramming

ModelBusiness

Motivations

Mainframe timesharing 1960s –1980sProcedural (COBOL) Automated business

Client/server 1980s-1990s

Database (SQL) and fat client (PowerBuilder, Visual Basic)

Computing power on the desktop

n-Tier/Web 1990s-2000sObject-oriented (Java, COM)

Internet/eBusiness

Service orientation 2000s Message-oriented (XML)

Business agility


SOA: Paradigm Shift?

• SOA is more evolutionary than revolutionary• Leverages many established best practices

But…

• As fundamental a change as client/server or the rise of the Internet

11


What is Architecture?

The fundamental organization of a system embodied by its components, their relationships to each other and to the environment and the principles guiding its design and evolution. (IEEE P1471/D5.3)


Architecture is not About the Technology

Just as a building architect is more concerned with the space, not the walls, the IT architect is concerned with how people use the technology, not the technology itself

12


SOA Views


Many Perspectives on SOA

• All views relevant & important• Service-Oriented Architects must have all

perspectives

Data

Services

TechnologyBusiness Processes

13


SOA: A Technology View

ServiceConsumers

BusinessServices SOBAs SOA

InfrastructureInfrastructure

ServicesData

Services Layer

Legacy Applications

and Middleware

Databases

Governance and Security InfrastructureGovernance and Security Infrastructure

SecuritySecurity

ManagementManagement

RoutingRouting

TransformTransform

AA

BBCC

DD

EE

AA

BBCC

DD

EE

MessagingMessaging

RIAsRIAs

Desktop Desktop AppsApps

DiscoveryDiscovery

OtherOtherServicesServices

MashupsMashups


SOA: Infrastructure Service View

Source: SAIC

14


SOA: A Business Service View

Source: IBM


SOA as Enterprise Architecture

• SOA best practices are IT and business best practices

• Service Orientation is a business concept

• Over time, practice of Enterprise Architecture becoming Service-oriented

15


Addressing SOAOrganizational Challenges


Challenge Area #1: Building the Business Case

• Never start with SOA… start with a Business Problem to solve

• Remember: the business continues to change, so, consider the business case iteratively

• This asset is a form of documentation, and is created for human consumption

Think big, start small, succeed Think big, start small, succeed oftenoften

16


Business Drivers for SOA

• Reduction in integration expense– Middleware replacement/Legacy rejuvenation

• Increase in reuse – Lower redundancy, better customer visibility

• Greater visibility– Enablement of governance & compliance, improved

efficiency

• Business empowerment– Business control over flexible processes

• Increase in business agility– Improved competitiveness, faster innovation/time to

market


Challenges in Calculating ROI

Lack of historical data from SOA projects to quantify reuse

Challenge of differentiating between a well architected, non-SOA application vs. SOA implementation

Difficulty measuring business agility benefit

17


Business Driver: Cost Savings

• Reduction in integration expense– EAI replacement/EAI maintenance reduction– Legacy enablement/migration/rejuvenation


Traditional EAI, B2Bi Pkg. ImplementationsWeb Services "Adapters"

Reducing Integration Cost

Initial Costs Customization ChangesMaintenance

Custom Integration

Copyright (C) 2002 ZapThink, LLC

Re

lativ

e C

osts

The Relative Costs of Different Integration Approaches

Service-Oriented Integration

S e rv ic e -O r ie n te d In te g ra t io n

18


Key Questions

• Is the flattened cost of change goal realistic?

• Will an iterative approach reduce the rearchitecture risks & costs sufficiently?

• Would a middleware/ESB-centric approach to SOA diminish its potential integration cost savings benefits?


Business Driver: Reuse

• Reuse the old way: code reuse– Reusable code libraries and

subroutines – the “Holy Grail” of programming

– Branching code base reduces reusability

– Hard to write reusable code, as requirements are never clear

• Reuse the new way: Service reuse– Reuse at runtime based upon contracted functionality– Loose coupling leads to flexible reuse– Appropriate governance and flexible metadata essential!

19


Benefit: Increasing Efficiency Thru Service Reuse

New Composite Apps or Service-Enabled Existing Apps

Service Catalog

Service Model

Data Access Services Legacy API Services

Coarse-Grained Business Services

Presentation Services

1 32 4 5 6 987

1

1

1

1

1

2

2

2

2

3

3

3

3

4

4

4

5

55

5

6

6

6

6

7

7

7

7

8

8

8

8

9

9

9

9

Source: BEA Systems


SOA Reuse Governance

• Reuse only important in practice• Planning for reuse pointless without actual reuse!• Developers would prefer to build

anew• Application assembly a new

pattern• Reuse governance covers:

– Finding Services– Understanding Services– Composing Services– Publishing the resulting compositions

20


Reuse Challenges

• Many Services not designed for reuse

• Simply exposing Web Services does not increase reusability substantially

• Services are often too course grained to be reusable

• Core business systems do very different things and often don’t need to share Services anyway!


Business Driver: Visibility

• The Challenge of Business Intelligence:– Visibility into data

– Visibility into business processes

– Visibility into levels of compliance

Does your executive management Does your executive management have the visibility they need today?have the visibility they need today?

21


Visibility & Control

• Effective managers must delegate responsibility, yet maintain control

• “Micromanagement” rarely effective, doesn’t scale

• Compliance & governance business drivers depend upon visibility & control

• Heterogeneity impacts both visibility as well as control of the business

• SOA greater visibility greater control


Visibility & Heterogeneity

• SOA useful in environments of heterogeneity• Traditional Business Intelligence (BI)

leverages homogeneous data and pre-packaged heterogeneous data

• SOA fills in the gaps: real-time, ad hoc data needs

22


Complex Event Processing & SOA

• Visibility benefit of SOA• Enables real-time business

PotentialFraud: Lock

out user

OperationalSlowdown: Bring new

server online

Process Out of Compliance: Alert

manager

Sub Margin Deals: Notify sales

management

Bad Marketing Conversion:

Dynamically switch promotion

Source: Service Integrity


Business Driver:Business Empowerment

• The business drives the composition of Services

• SOA enables Service consumption and composition in unpredicted ways

• Enterprise Mashups: the “SOA use case” that brings SOA benefits to the business user

23


Business Driver: Business Agility

• Responding quickly and efficiently to change & leveraging change for competitive advantage

• Important to identify and quantify agility requirements– Time to value?– Efficient responses to regulatory

change?– More successful mergers/acquisitions?

• Remember to measure resulting agility!


When Not to Apply SOA

• When business requirements are stable• When the IT environment is homogeneous• When the business has sufficient visibility

based on current tools• When a particular performance requirement

calls for efficiency over flexibility

SOA success means applying SOA where needed, SOA success means applying SOA where needed, but if it ainbut if it ain’’t broke, dont broke, don’’t fix it!t fix it!

24


Challenge Area #2: Funding & Budgeting


The Wrong Question!

instead of…

SOA is great. How do I sell it to SOA is great. How do I sell it to the business?the business?

Here are our problems. How best Here are our problems. How best to solve them?to solve them?

25


Traditional IT Funding:Project Based

Departments allocate funds for IT projects that cater to their specific requirements

Typical funding in an enterprise is usually project based, and are prone to budgetary constraints and

maintenance expenses

Source: Wipro


Initial SOA Funding

A pilot SOA venture could be funded by a mid-level manager’s discretionary budget

Source: Wipro

26


Funding SOA Rollouts

Higher levels of management need to be involved. Enterprise-level SOA requires funding to be

obtained from the highest level - CIO or CFO

The SOA initiatives should be regarded as a line of business, administered and funded separately through the

CIO’s office

Source: Wipro


Common SOA Pitfalls

• Unclear business drivers

• Allowing a vendor to drive the initiative

• Confusing SOA and Web Services

• Too few qualified architects

• Lack of proper, early governance

• Unqualified consultants

• “Good money after bad” fallacy

27


The Problems with “VDA”

• Vendor “SOA Certification”Programs– Always product-specific, not

SOA-specific

• Vendors who design & build your SOA– Always start with their stack

• “One stop shopping” for SOA– Doesn’t give you best practices

““VendorVendor--Driven ArchitectureDriven Architecture””


Good Money after Bad…

• I spent money on a proprietary vendor solution, so now I need to make it work!

• We built inflexible EJB Services or .NET Services, so how to I make them flexible?

• We spent big money with that big consulting firm on our SOA initiative, but we don’t have anything to show for it!

28


SOA Growing Pains

• Does the business see the value?

• Are the architects working on the right problems?

• Is IT management investing properly?

• Are you letting vendors drive?


Organizational Issues

• The greatest challenge of SOA• Makes the technology part look easy by

comparison!

29


Challenge: Inertia in the Organization

• Architecture doesn’t have features and business executives pay for features!

• Moving to SOA means breaking down silos and sharing resources

• The technology change is easy – it’s the human change that’s the hard part!


Challenge: Balancing IT Control & Business Empowerment

• Answer: Governance• Establishing, communicating, and

enforcing policies, providing visibility into the levels of compliance, and dealing with issues

• Not just governance of SOA…governance with SOA

• But…avoid “big brother” effect

Who likes to be governed?Who likes to be governed?

30


Challenge: Reuse = Sharing

We all learned to share in kindergartenWe all learned to share in kindergarten……

But by the time we get to the working world, we But by the time we get to the working world, we forget how!forget how!


SOA by Any Name

• “SOA” is too “techie” for the business

• SOA is a broad set of best practices

• Many SOA best practices build on existing practices

Doing it Right More Important Doing it Right More Important than Calling it SOAthan Calling it SOA

31


SOA = Best Practices

• You don’t have to follow them all

• There’s no rule how many you must follow before you can say you’re “doing SOA”

• Key best practice: take an iterative approach

The Right Tool for the JobThe Right Tool for the Job


Is there an Architect in the House?

• The new level of discipline required by architecture– A formal approach to organizing IT resources

is still a relatively new practice

• Just how big is the big picture?– Architects must have an enterprisewide view

• Where are the architects?– It’s hard to learn architecture at college –

most learn on the job

32


Another Look: SOA Challenges

Source: Wipro


Interaction Challenges

• Cultural Issues– Network Ops and Developers don’t talk to each other

• Budget issues– Who pays for Service Infrastructure / intermediaries?

• Responsibility issues– Who is in control of policy?

Developer/ArchitectDeveloper/Architect Network OperationsNetwork Operations

Services blur the Application / Network Boundary!Services blur the Application / Network Boundary!

33


More Interaction Challenges

• Management issues– People tend to avoid risk, stay within “comfort zone” - may appear stubborn

• Technical issues– Architecture is a difficult subject

• Cultural issues– The “Ivory Tower” problem…

ArchitectArchitect Development/TestingDevelopment/Testing

Architecture is difficult to mandateArchitecture is difficult to mandate


The “Ivory Tower” Problem

• Architects create design and other artifacts, but don’t have the authority or mandate to require their use

• Development team considers them optional

• Business likes idea of architecture in principle, but short-term needs trump best practices

• When architects are external consultants, the “not invented here” syndrome makes the Ivory Tower worse – or sometimes the opposite!

34


The Power of the SOA Center of Excellence

• SOA experts who maintain a knowledge base of best practices– General and company-specific– Design time and runtime

• Drives SOA policy (either explicitly or implicitly)

• Can unify approaches across a large organization


Convincing Technical Specialists

• Among the most risk-averse are technical specialists – mid to late-career experts in a (typically legacy) technology (e.g., “COBOL Jockeys”)

• Architectural change threatens their careers

• Solution: – Work with younger developers to build acceptance

for SOA (eventually the TS’s will come around)– Take a “leave and abstract” approach over “rip and

replace”

35


Working with IT Middle Management

• Middle managers threatened by SOA because of the Service domain reorganization

• Solution:– Technical specialties still required– New opportunities for Service domain

management


Enabling Service Domains

• A Service Domain is a logical grouping of shared Services with a common business context

• Examples: customer-facing Services,purchasing-related Services

• Manage Services by managingthe Domains

• Move away from traditional IT silos for the purposes of managingServices, but retain technical teamsas needed

36


Service Domains Example

Source: CTOgroup


Service Domain Roles

SOA governance introduces new roles that the company must provide for:

• The domain owner– Manages relationships– Tracks the usage of Services

• Domain SO business analyst– Translates business requirements into Service definitions– Directs Service implementations

• Line of business representative– Communicates business requirements

• Domain developer– Builds and maintains Services

• Service tester– Certifies that each Service conforms to the business requirements

37


EA Challenges:The Role of the EA

• Easier to stay high level than do actual work!

• Drawing diagrams, doing presentations, and writing reports is much easier than actually going out and making real changes with real benefits


Enterprise Architecture Challenges

• Many organizations have a chasm between the traditional EA crowd and the SOA team

• EA has morphed from an approach for the betterment of corporate IT to a management practice, hence resistant to change

• The person that must understand & implement SOA should be the EA in charge

38


EA Challenges:The Risk of SOA

• Issue: “add-not-change” approach to architecture

• Adding applications, directories, and databases to an existing architecture is easy and risk-adverse

• Changing architectures around systemic notions such as SOA is difficult and risky

• Cultures often have “you fail, you're fired”approach, vs. “let’s try new things and seek improvement”


The Real Challenge: People, Change and Fear

• People are inherently resistant to change

• People consider job security, authority and responsibility when asked to share

• Fear is the strongest emotion of all!

39


SOA Intermediaries & Integration


SOA Infrastructure Starting Point: The Intermediary

• Connecting consumers to providers directly results in tight coupling

• Loose coupling depends upon something transparent in between: the intermediary

• Intermediation is a pattern: we’re not specifying a particular piece of software

40


Some Intermediary Roles

• Translate between transport protocols (e.g., MQ to HTTP)

• Translate between message protocols (e.g., SOAP to “Plain old XML”)

• Handle transformations on the fly

• Negotiate between different security domains (e.g., PKI to Kerberos using WS-Security)


Messaging Infrastructure

• Services send and/or receive messages

• TCP/IP provides underlying wire protocol

• Intermediaries abstract messaging protocols

• Challenges are at the content level

CanCan’’t have trains without the tracks!t have trains without the tracks!

41


SOA, Integration & Legacy

• Key goal of SOA: Integration as a byproduct of Service composition

• Goal of legacy integration: building Services to support this goal, NOT connecting systems to address a particular business need

Move away from Move away from ““connecting systemsconnecting systems””and toward and toward ““composing Servicescomposing Services””


Exposing Existing Capabilities

• Service interfaces exposed from existing systems

• Often pre-defined by existing software

• Low-level representations of internal application functions or interfaces often exposed as Web Services

42


The Continued Value of Legacy

• Definition of legacy: “anything that works”– Sometimes old, but not

necessarily

• Expensive & risky to replace– Often contains mission

critical business logic

• Expensive & risky to keep around– Maintenance consumes major

portion of IT budget

““Rip & ReplaceRip & Replace”” rarely a viable optionrarely a viable option


SOA and Legacy

• Rethink the way existing IT assets can meet new requirements

• Leverage SOA to abstract legacy assets as Services

• Compose legacy-based Services with other Services

43


Legacy Migration

• Pros:– Reduces ongoing costs

of maintenance– SOA abstracts

interfaces, easing replacement

• Cons:– System still provides

value– High cost and risk

Retire legacy applications & systems;Retire legacy applications & systems;Application consolidationApplication consolidation


Legacy Enablement

• Pros:– Lowers risk and cost of migration– Maintains value of legacy– Does not require SOA

• Cons:– Inflexible– Can lead to “ABOS” problems

Simplistic exposure of Simplistic exposure of legacy capabilities/data as Serviceslegacy capabilities/data as Services

44


Legacy Rejuvenation

• Pros:– Squeeze more value out

of existing assets– Increases business

agility– Non-invasive

• Challenge:– Requires new

architectural approach to legacy

Leverage legacy (especially mainframe)Leverage legacy (especially mainframe)as active SOA participantas active SOA participant


The Great ESB/SOA Middleware Boondoggle

• Does SOA infrastructure have to be built on traditional middleware?....NO!

• Should you begin a SOA project by purchasing an ESB/SOA middleware product?....NO!

• Are there viable intermediary approaches that don’t require additional middleware?...YES!

• So…why are so many vendors pushing the “ESB/middleware first” approach to SOA?

Because middleware is whatthey have to sell!

45


Buy More Middleware for SOA?

• You may already have enough middleware

• Buying more middleware leads to the “middleware for your middleware” problem

• With SOA, integration should be a byproduct of composition, but middleware is for integrating systems, not building composable Services

Focus on your needs: only buy more middleware if you need more middleware


Compounding the Problem: No Clear ESB Definition

• Distributed Service containers (runtimes) on a messaging infrastructure?

• An architectural approach to implementing SOA based on middleware?

• All the infrastructure needed to implement SOA?• A message broker that supports Web Services

standards?• Whatever integration technology you got with Web

Service interfaces added?

Bottom line: SOA should drive product selection Bottom line: SOA should drive product selection –– dondon’’t let t let the ESB vendors tell you their product should drive SOA!the ESB vendors tell you their product should drive SOA!

46


Do You need an ESB for Service Mediation?

• Enterprise Service Bus (ESB) term not consistently used across products

• Many ESB products do provide Service mediation

• Many ESB products also provide message transport or proprietary integration infrastructure, which you may not need

• Some products provide Service mediation without being an ESB, and do it better


Intermediary-Based Service Abstraction

Source: SOA Software

47


Building Intermediary-Based SOA Infrastructure

• Leverage existing middleware as appropriate

• Plan for distributed intermediaries as part of the architecture

• Intermediaries can be hardware (e.g., XML Appliances) or software (sometimes confusingly called ESBs)

• Sometimes you do need more middlewareAlways remember:Always remember:

business needs drive the architecture,business needs drive the architecture,and architecture drives the technology.and architecture drives the technology.

DonDon’’t let technology drive the architecture!t let technology drive the architecture!


SOA Infrastructure

48


Purchasing SOA Infrastructure

• Don’t get lost in the terminology!• There are many styles for SOA

implementation• Focus on your goals: Reuse? Governance?

Reduced integration cost? Agility?

ESB?EAI?

SOA?EDA?EDA?

Fabric?

Abstraction?SOA Infrastructure?

Service Grid?Service Network?


Technology Selection: Approach

Select your technology set.

Technology

Requirements

Technology

Requirements

Define requirements.

Technology analysis.

Technology

solution

Technology

solution

Vendors

Define candidate technology.

Technology selection.

Technology validation.

49


Technology Selection: Choices

• Many technologies are available

• The choice of technology will be a mix of products and vendors that meet the needs of the SOA implementation

• Rare for a single vendor to be able to solve all problems


Technology Selection: Challenges

• Difficult process

• Marriage of criteria and products often requires a pilot project

• The time it takes to select the right technologies could be as long as the actual SOA implementation

• Remember, A bad choice could ensure the failure of your SOA

50


Levels of SOA Infrastructure

Metadata ManagementGQMSecurityMessagingData IntegrationLegacy Integration


Challenges at the Content Level

• Content Security, as well as– Semantics– Taxonomies– Policy– Schemas– …

7

6

5

4

3

1

2Traditional perimeterTraditional perimeter--

based approaches wholly based approaches wholly inadequateinadequate

51


Is XML Required for SOA?

• Technically no, but in practice XML is widely adopted

• SOA is technology-independent• Many organizations are still using FTP and CSV

for message transport & format!


What about REST?

• Representational State Transfer (REST) is a collection of network architecture principles for defining and addressing resources, typically on the Web

• Today’s REST implementations leverage HTTP

• Simpler but less powerful than Web Services-based integration

52


XML: Foundation for Web Services

• XML now the lingua franca of heterogeneous interoperability

• Web Services depend on XML, yet SOA doesn’t technically require it…

<?xml version="1.0" encoding="utf-8" ?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing"xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">

<soap:Header><wsa:Action wsu:Id=GUID>

http://www.bluestonepartners.com/schemas/StockTrader/RequestQuote</wsa:Action><wsa:From wsu:Id=GUID><wsa:Address>

http://schemas.xmlsoap.org/ws/2003/03/addressing/role/anonymous</wsa:Address></wsa:From><wsa:MessageID> Message ID and UUID </wsa:MessageID><wsa:To wsu:Id=GUID>

http://localhost:8080/StockTraderSecure/StockTrader.asmx</wsa:To><wsu:Timestamp>

Contains Message Creation/Expiration TimeStamps</wsu:Timestamp>

</soap:Header><soap:Body>

<Symbol xmlns="http://www.bluestonepartners.com/schemas/StockTrader/">MSFT

</Symbol></soap:Body></soap:Envelope>

Source: http://www.windowsitlibrary.com/Content/1219/06/1.html


The XML Processing Problem

– Receive– Canonicalize– Authenticate– Decrypt– Validate– Transform– Parse– Aggregate– Sign– Encrypt– Transmit

• Steps required to process XML for each message:

For every message, at wire speedFor every message, at wire speed

53


parse…send… receive…

understand… store…update… validate…

…parse

The XML Performance Crisis

XML is “Heavy”$FF becomes:

<SampleValue>255</SampleValue>

Large XML data sets are sent to conform to

bloated XML “industry standard” schema

definitions

Eventually this huge document needs to be fully processed –

which means turning it into usable “objects”

Traditional applications are adapted to services,

sending thousands of messages over the wire

Source: RogueWave


Hardware vs. Software Approaches to Improve XML Performance

• Pros:– Dedicated processing

power– Control of variability– Hardened OS– Simple to install &

maintain• Cons:

– Must go in data center– More difficult to upgrade– Limited customizability

• Pros:– Developers in control– Easy to customize &

upgrade• Cons:

– Harder to install & maintain

– Relies upon hardware for performance

– Too much variability in configuration

Hardware Software

54


Critical XML Processing Challenge: Security

• Identity & Access Management– Letting the good guys do what

they are supposed to

• Threat prevention– Keeping the bad guys from

doing what they’re notsupposed to

• XML, Web Services, & SOA each compound the IT security challenge


The Context of IT Security

• XML is text-based, human readable – give hackers the keys and the instructions

• Existing security inadequate to address content security issues

• Any XML/Web Services/SOA security effort must be a part of a comprehensive enterprise security initiative

If your company has 20 doors, and you lock 19, are you 95% securIf your company has 20 doors, and you lock 19, are you 95% secure?e?

55


XML Threat Prevention

• XML messaging exacerbates existing threats, including:– SQL injection– Denial of service– Buffer overflow– Trojan horse

• And adds new threat vectors:– XML injection– WSDL scanning– Malformed, recursive, or oversized payloads– Schema poisoning


The SOA Security Challenge

Service-orientedcomputing

Network (Packet) Level

TCP/IP packetinspection

Perimeter-based

Fragmented Control

Islands of securityMaintained as separate

units

Closed Systems

Proprietary, closedAPIs

Tight couplingSingle administrator

Localized users

Application Level

XML-basedContent aware

Application control

Open Systems

Open, accessible APIsLoose coupling

Multiple administratorsDistributed users

Managed Security

Whole network policiesTiered administration

Traditional DistributedComputing

56


The Security Context Challenge

rschmelzer

RonSchmelzer

rschm123

Selective

Read / Write

Read Only

Full Read/Write???

???


Security Context Kludges

• Headless “nobody”user?

• Root access?

• Shared logins?

• Crossing fingers & hoping hackers won’t notice?

57


Solving the Security Context Challenge

Source: CA


21st Century Network Security

• The Old Days:– Internal = Trusted– External = Untrusted

• Today:– Internal = Untrusted– External = Hostile!

You MUST address security You MUST address security issues throughout the issues throughout the

networknetwork

58


Management & Loose Coupling

• Loose coupling is a managementproblem: require Services to behave as expected

• Must handle infrastructure management issues “behind the scenes”

• If a business user knows how Services work, then something is wrong


SOA Management: Many Facets

• System management – identifying root causes of technical problems, mitigating or solving them

• Metadata management – storing, managing, and enabling the use of various metadata

• Business Service management – managing business Key Performance Indicators (KPIs) in the context of SOA

59


The Problem with SOA Management…

• Managing Service interfaces & dependencies

• Application management

• Systems management

• Network management

• Root cause analysis• Etc…


The First Rule of SOA Management

• You need management when you offer your first “mission critical” Web Service

• Even one Service requires management

• Don’t wait!

60


The SOA Management Conundrum

ApplicationInfrastructure

Database

Systems

Network

Business Process

ApplicationJ2EE

IIS WebSphere MQ Series

Oracle9i

Windows UNIX

TCP/IP

Hire New Employee

SAP

Exchange/Domino

zOS

WS & SOA

Source: CA


Exception Management & SOA

• “Passive” management– Problem Alert Human intervention– Breaks loose coupling!

• “Active” management– Crosses threshold automated response

problem prevented– Maintains loose coupling– Much more difficult! State of

the Art!

Loose coupling hangs in the balance!

61


SOA & ITIL

• The IT Infrastructure Library (ITIL) is a set of infrastructure & operations best practices

• Originally defined IT Service Management and Business Service Management witha different sense of the word “service” than SOA

• Current version (ITIL v3) brings notions closer together

Source: iETSolutions


Metadata Management

62


The Secret Sauce: Metadata

• Enables composite applications to be builtdeclaratively (through configuration), not programmatically (with code)

• Business logic in composite applications represented in metadata

• Puts business logic in the hands of business users!


What are Metadata?

• Literally, data about data• More broadly, data about the workings of a

system, as opposed to the data the system works with

This is a purchase orderIt has an address fieldIt has an item fieldIt has a dollar field…

ABC Co.32 Widgets$47.00

Metadata

Data

63


Metadata for SOA

• Contract metadata– What a Service should do (functional)– How a Service should behave (non-functional)

• Process metadata– How processes are configured– How processes are composed

• Policy metadata– What rules apply in various

situations

• Schemas• Others…


What’s a Service Contract?

• Service contract – a document external to each participant that provides the information each participant needs to interact with the other

• Web Services Description Language(WSDL) – standard Service contract language– Woefully inadequate!

64


What’s in a Contract?

Functional Requirements

Non-Functional Requirements

Service Description,Service Operations

Infrastructure(Port Type, Binding)

URIs(Schemas, Namespaces)

Security

Quality of ServiceLevels

Transactionality Commerce / Biz Rules

Semantics Process InstanceState

ConsumerRequirements

Message ExchangePatterns Testing Versioning Deployment

Details


What’s NOT in the Contract

A "Contract" is an expression of A "Contract" is an expression of visiblevisible aspects of Service behavior.aspects of Service behavior.

It does NOT specify:• Service Implementation Details

– Programming model – Object references– In-memory representations

• Examples– Exposing Java Classes– Serialization of Java Objects

65


WSDL: Service Contract Starting Point

• Sample WSDL file• Describes port type,

binding, input, output, Service name and URL

<?xml version="1.0" encoding="UTF-8"?>

<definitions name ="DayOfWeek"

targetNamespace="http://www.roguewave.com/soapworx/examples/DayOfWeek.wsdl"

xmlns:tns="http://www.roguewave.com/soapworx/examples/DayOfWeek.wsdl"

xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xsd="http://www.w3.org/2001/XMLSchema"xmlns="http://schemas.xmlsoap.org/wsdl/">

<message name="DayOfWeekInput"><part name="date" type="xsd:date"/>

</message><message name="DayOfWeekResponse">

<part name="dayOfWeek" type="xsd:string"/></message>

<portType name="DayOfWeekPortType"><operation name="GetDayOfWeek">

<input message="tns:DayOfWeekInput"/><output message="tns:DayOfWeekResponse"/>

</operation></portType>

<binding name="DayOfWeekBinding" type="tns:DayOfWeekPortType"><soap:binding style="document"

transport="http://schemas.xmlsoap.org/soap/http"/><operation name="GetDayOfWeek">

<soap:operation soapAction="getdayofweek"/><input>

<soap:body use="encoded" namespace="http://www.roguewave.com/soapworx/examples" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>

</input>

Source: Rogue Wave

<output><soap:body use="encoded"

namespace="http://www.roguewave.com/soapworx/examples"

encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/></output>

</operation></binding>

<service name="DayOfWeekService" ><documentation>

Returns the day-of-week name for a given date</documentation><port name="DayOfWeekPort"

binding="tns:DayOfWeekBinding"><soap:address

location="http://localhost:8090/dayofweek/DayOfWeek"/></port>

</service>

</definitions>


Contract Metadata Beyond WSDL

Source: W3 - http://www.w3.org/2004/08/ws-cc/jsklms-20040903

State ofthe Art!

66


SOA Registry: The Original SOA Intermediary

Source: W3C, 2002


Metadata Management & SOA

• Metadata at the core of any SOA implementation– Contracts, policies, SOBA configurations,

schemas…

• SOA Governance depends upon metadata as well

• “Registry/Repository” at the center of the SOA metadata management puzzle

67


Complexities of SOA Metadata Mgmt. Marketplace

Registry/Repository

SOA Quality

PolicyManagement

Design TimeSOA

Governance

Service Lifecycle Management/Change Time Governance

Runtime SOA Governance


What is a Registry?

•• Metadata System of RecordMetadata System of Record for SOA– Interfaces

• UDDI – Web Services• LDAP• Microsoft Word• Word of Mouth

• What do you want from your registry?– Design time capabilities to discover Services– Interoperability across runtime infrastructure– Ability to use at runtime for binding….

68


What is a Repository?

•• Asset StoreAsset Store for SOA– Artifacts of design

• Conformance Documentation• Contract versions• Sample Code• Profiles• Schema

– Runtime operational store• Messages• Policies• Logs, Security Certificates, Keys, SAML artifacts• Transformations• Contracts

• Interfaces:• ebXML• CVS / Source Code systems• WebDAV• File Systems• SQL• XQuery


Summary: Metadata Management

• Repository: Where assets are stored (metadata and other assets)

• Registry: How to find and work with the assets you need – wherever they are (metadata about those assets)

• Products have aspects of both

• People confuse the terms because both data and metadata must be managed in an SOA

69


SOA Project Management


How Do You Eat an Elephant?

• One bite at a time!

• Don’t expect to have all the answers on day one

• Take a step-by-step approach, but…– Top-down only: have the plan, may not

be able to execute

– Bottom-up only: build Services, may not be reusable

• SOA planning must be both– Develop the vision (but not the details)

ahead of time

– Service development should be iterative

• Show business value at each step

70


Iterative:More than Step-by-Step

• Each iteration bounded by time, money, or scope

• Each iteration is a full project

• Improvements made to any part of earlier iterations in future iterations


Iterating SOA Initiatives

• Define the scope of SOA initiative within the enterprise

• Implement SOA in small steps, such as Service-orienting a single division or a portion of a division, instead of an entire enterprise all at once

• Small successes lead to larger, more strategic successes over time

• Establish the demarcation lines at the beginning of the project to provide better focus and understanding

71


SOA Pilots

• A few high ROI Services• Build acceptance for SOA• Get team up to speed• Work out the kinks• Pilot the governance model• Part of an iterative approach to SOA

• Piloting only the Services instead of the architecture• Remember, the pilot is one step on the roadmap

DANGER! Avoid the SOA Pilot PitfallDANGER! Avoid the SOA Pilot Pitfall


The SOA Roadmap

• Outlines specific steps across multiple timelines on specific actions to take in SOA projects

• Can cover infrastructure, architecture, organization, funding, governance, Service scope, and other aspects

• More detail provided in this module

72


The ZapThink SOA Roadmap

• An SOA roadmap provides a customizable, visual guide to SOA implementation

• The ZapThink Roadmap represents one approach to SOA implementation

• The goal of roadmapping is to visually represent implementation; the ZapThink roadmap goal is to minimize risk

• The actual roadmap order is flexible and depends on company situation & needs


SOA Governance

73


Business Empowerment vs. IT Control

• IT management requires control– How to scale management control

& avoid micromanagement?

• IT charged with empowering users– How to avoid policy breaches?

• Centralization of IT capabilities running into roadblocks– How to decentralize IT

responsibility without leading to redundant or incompatible capabilities?


Corporate Governance

• Establishing and communicating the policiesthat employees must follow

• Giving employees the tools they need to be compliant with those policies

• Enforcing policies

• Providing visibility into the levels of compliance in the organization

• Mitigating any deviations from established policy

From project to program to sustainable processFrom project to program to sustainable process

74


Governance & Regulatory Compliance

• Regulatory compliance one of the primary drivers for governance initiatives

• Companies must spend to become compliant, but not a penny more!

• Compliance can be very complex, with multiple regulations/jurisdictions

• Regulations are essentially arbitrary

The unpredictable nature of regulations The unpredictable nature of regulations are a prime contributor to the business are a prime contributor to the business

agility requirementagility requirement


The Business Motivation for Governance

Return On Investment

Risk Of Incarceration

75


How to Tackle Governance?

• Communication• Training• Human management• Knowledge management• Automation

IT can at best only help with partIT can at best only help with partof the governance processof the governance process


Governance Relationships(Part 1)

CorporateGovernance

ITGovernance

Corporate policiesapplied to IT

IT providesGovernance Tools

• IT serves dual governance role– Corporate policies must apply to IT– IT provides policy automation and knowledge

management tooling to the corporation

76


The Cornerstone of IT Governance is Architecture

• Architecture provides the overall organizational guidelines for all of IT

• Architectural processes implement IT governance

• An architecture board should drive IT governance


Elements of IT Governance Strategy

• Architecture board– Establish a cross-organizational architecture board with

the backing of top management to oversee implementation of IT governance strategy

• Architectural principles– Delineate comprehensive set of architectural principles to

guide, inform and support organization

• Architecture compliance strategy– Adopt specific measures to ensure compliance with the

architecture, including project impact assessments, a formal architecture compliance review process, and the involvement of architecture team in product procurement

77


Architectural Governance Processes

• Architecture review and approval• Architecture exceptions and escalation• Architecture maintenance• Architecture communication• Architecture compliance review


IT Governance Feedback Loop

Source: LogicLibrary

> Delivery of architectural guidelines and supporting materials

> Governance over project-specific selection of architectural materials

> Traceability of architectural materials used within a project

> Feedback on effectiveness of architectural materials

> Delivery of architectural guidelines and supporting materials

> Governance over project-specific selection of architectural materials

> Traceability of architectural materials used within a project

> Feedback on effectiveness of architectural materials

78


Governance RelationshipsPart 2

CorporateGovernance

ITGovernance

ArchitectureGovernance


ArchitectureDrives IT

Governance

GovernArchitecture



SOA Governance “in the Narrow”

• Governance of SOA initiative

• Design time governance– Are developers and other personnel following

the policies that apply to Services?

• Runtime governance– Are running Services and SOBAs conforming to

runtime policies?– Automating policies specific to Services in the

context of SOA

79


SOA Governance

Source: Wipro


Governance Relationships

CorporateGovernance

ITGovernance

ArchitectureGovernance

SOAGovernance


ArchitectureGovernanceIn Context

of SOAGovern SOAInitiatives

IT Governance inContext of SOA

ArchitectureDrives IT

Governance

GovernArchitecture


SOA Governance “in the narrow”

SOA Governance “in the broad”

80


Governance Relationships

CorporateGovernance

ITGovernance

SOArchitectureGovernance


SOADrives IT

Governance

GovernArchitecture



What is a Policy?

• A high-level overall plan embracing the general goals and acceptable procedures – a broad business definition (from Merriam-Webster)– Impossible to automate

• A set of rules that apply to the performance or behavior of a system and its users – a more technical definition– Easier to automate

• How to reconcile the two definitions?

81


Policy: Business vs. Technical Examples

• Corporate privacy policy• Non-discrimination

policy• Regulatory policies, like

those required by Sarbanes-Oxley

• Firewall policies• Developer coding

policies• Service reuse policies• Role-based assertions


The Challenge of Policy Automation

• Remember, policies are business concepts

• Governance is the way the businesscommunicates & manages policies

• Challenge: What policies can and should be automated?

• SOA helps automate policy activities by treating policies as metadata

82


Creating the Governance Framework

• Which policies are within scope?• Who is responsible for creating policies?• Which policies are automatable?• How will policies be created and communicated?• How will policies be represented?• How will policies be discovered?• What tools will people use to follow policies?• How will management get visibility into policy

compliance?• How will mitigation issues be addressed?


Governance Pitfall: Versioning

Without versioning policies:• Service provider changes

require consumer changes!

With versioning policies:• Service providers & con-

sumers change as per policy• Maintains loose coupling

ByeBye--bye loose coupling!bye loose coupling!

Challenge: Which Versioning Policies are Challenge: Which Versioning Policies are Right for YOU?Right for YOU?

83


Handling Service Versioning

• New requirements may involve only process configuration changes

• Services may support multiple contracts

• New requirement may require new contract

• Policy drives version selection & deprecation

Service consumers must support Service consumers must support deprecation policiesdeprecation policies State of

the Art!


Governance: The Key to Business Empowerment

• Governance: creating, communicating, & enforcing policies

• SOA enables the formalization of policies as metadata

Practical SOA requires GovernancePractical SOA requires GovernanceAndAnd

Effective Governance Requires SOAEffective Governance Requires SOA

More about SOA Governance to comeMore about SOA Governance to come……

84


SOA Quality


How do you manage change?

• SOA is all about continuous and sometimes unpredictable change

• Development issues

– How to handle versioning?

– How to handle metadata management?

– How to develop changing policies?

• Runtime issues

– Service availability

– Policy enforcement

– Guarantee Service-level agreement

– Maintain low TCO

85


The GQM Loop


Quality & the Service Lifecycle

86


SOA Quality Assurance

• First level – Web Service testing– Test Service consumers and providers

to ensure proper exchange of messages– Performance test to ensure scalability

and reliability

• Second level – integration/dependency testing– Ensure SOBAs work properly– Test underlying infrastructure

• Third level – metadata testing– Test ability to change contracts, policies, SOBA logic


Service Lifecycle Governance and Quality

• Connect design time to runtime

• Quality a never-ending goal

• Quality – much more than being bug-free– Meets business requirements as those

requirements continue to change– Meets Service levels and other policies as those

policies change

The agility requirement for SOA vastly The agility requirement for SOA vastly complicates the quality challengecomplicates the quality challenge

87


Service Composition, Business Process, & SOBAs


What is a Business Process?

• What a business does

• Range from fully manual (no IT involvement) to fully automated (no human involvement)

• Most processes are partially automated

88


The Role of Business Process Reengineering

• Focused on breaking down silos and establishing end-to-end processes like “procure to pay”and “order to cash”

• The concept was valid, but…– IT wasn’t yet up to the task of

automating such processes– BPR wasn’t grounded in architecture– Businesses were often too inflexible to take full

advantage of BPR– BPR became a code-word for laying people off


The Automation Paradox

• The more we automate, the more our remaining problems are difficult to automate

• Is IT about getting the technology to work together or to help the business meet its goals?

• Achieving the right balance among automation, agility & user empowerment is critical

89


Problems with Traditional BPM Tooling

• Traditional BPM often:– Represents static, design

time view of business processes

– Represents processes as they should be, or as they were, but not as they actually are

– Dictates to the business how processes should work

– Is tightly coupled, single point of failure


Business Process the Old Way…

• People plugged into rigid processes• Inflexible & brittle

90


Business Process the Service-Oriented Way…

• IT resources (among other resources) available to the business as needed

• Business users create composite applications by composing Services on the fly


Service-Oriented Process

• Business processes in the context of SOA• SOBAs as business empowerment tool• Integration becomes a side effect of

implementing Service-Oriented Processes thru Service composition

91


Varieties of Business Processes

• Static business processes– Stable, well-understood– Would benefit little from SOA

• Ad hoc business processes– May be “one off” or impromptu team

processes– Rarely automated today– Automation of ad hoc processes an

advanced benefit of SOA

• Flexible business processes– Processes that must respond to changing business

requirements– Key SOA business motivation


Service Composition: Supporting Business Process with Services

• A Service-Oriented Business Application (SOBA) is a composite application composed of Services that implements a business process

• SOBA is a Gartner term, but this definition is a bit different from theirs

92


Business Logic at the Composition Level

• Create business logic via composition of existing Services and information flows rather than via creation of new Services, if possible

• Compositions are Service-Oriented Business Applications (SOBAs)– SOBA design similar to Service

design– SOBAs leverage Services,

organizing them into solutions


SOBAs: Rethinking the Application

• SOA abstracts existing capabilities– Breaks down existing application

silos & leverages legacy assets

• Architecture guides composition of Services

• SOA empowers the business– The business drives business

process

93


SOBA Example

• Merchandising process example

• The old way: establish the process for ordering, delivering, and displaying winter coats on store racks– An early cold snap leads to

unexpected demand, causing an exception to be handled manually

• The Service Oriented way: a business user responsible for the merchandising SOBA updates it to deal with unexpected events, reducing the need for manual exception handling


Enterprise Applications and Process

• The problem with enterprise applications– Process bound to Functionality– High customization and integration cost

• New approach– Atomic Enterprise App Functionality– Separate Process Layer

94


Example: SAP NetWeaver

• Moving from selling business applications to business Services

• Offering the platform as well

• “Selling the trains and the tracks”

Source: SAP


Service Consumers & Enterprise Mashups

95


The Services Tipping Point

• Up till now, focus on building Services– SOA infrastructure– Legacy enablement– Identifying reusable

Services– Building loosely coupled

Services

• Now, focus moving to consuming Services– Finding the right Service for

the task at hand– Composing Services into

Service-Oriented Business Applications (SOBAs)

– Supporting rich user interfaces for Services and SOBAs


The Rise of the Service Consumer

Data/BI Legacy LOB Packaged Apps Trading Partners

Packaged Apps

Devices & People

Web 2.0

Portal

Operating Systems

Source: Microsoft

96


The Rise of the Mashup

• Mashup = a flexible composition of Services within a rich user interface environment

• In essence, a Mashup is a SOBA interface

State ofthe Art!

Source: http://web2.wsj2.com


What’s New about Mashups?

• Using the application includes creating and configuring the application

97


Empower Business Users?

• The mashup: leveraging the Web to compose Services into ad hoc apps

• Without management and governance, will never be appropriate in an enterprise environment

• How to empower users in the spirit of the mashup, but maintain necessary control?

Governance the key to the Governance the key to the ““Enterprise MashupEnterprise Mashup””


Web 2.0 vs. SOA

Web 2.0 SOA

Mashups

EnterpriseMashups

Enterprise 2.0

98


Mashup Example

Source: IDV Solutions


Enterprise Mashup in Action

Interactsvia

Phone

Views KPIsMakes Policy

Decisions

Interactswith

Web Site

Interactswith Phone& Screen

ReconfiguresSOBAs

MakesChanges toProcesses

Customer

Call Center Rep

Call Center Manager

Customer

Business Analyst

Executive

CustomerServiceSOBA

99


“Use Case” for SOA

• Enterprise Mashups are driving SOA adoption in many organizations– Put a visual face on SOA– Emphasize the business

empowerment benefit– Driven by business process-

centric motivations


Without Governance,Mashups are Dangerous

• Mashups enable unpredictable SOBAs

• Risks: – Confidentiality breaches– Unauthorized capabilities– Fraud

100


Without SOA, Mashups are Toys

• Loose coupling of underlying Services essential for reliability & agility

• How can Google & Amazon update their Services?


Thank You!

soa for managers full day - dovel technologiesdoveltech.com/wp-content/uploads/2017/09/soa... · 6...

Documents