5a. the nuts and bolts of soa - dovel...

1

Copyright © 2006, ZapThink, LLC 1

The Nuts & Bolts of SOA

Jason BloombergZapThink, LLC

Take Credit Code: NONUTS


The Three Core SOA Infrastructure Challenges

Security

Management

Governance

2


The ZapThink SOA Roadmap:Infrastructure Focus


Juggling SOA Infrastructure

• Successful SOA requires several infrastructure elements– Challenges: – Priorities?– Budgets?– Identifying needs?

• Iterative architectural approach should drive infrastructure decision

Security

Inte

gra

tion

Proc

ess

Man

agem

ent

Gove

rnan

ce

3


The SOA Reliability Conundrum

• Three aspects of reliability– Reliability of runtime

• Will the system work as planned?– Reliability of design

• How can we make sure we have consistency?

– Reliability of people• What if people don’t behave as they

should?

• Traditional view of reliability now insufficient– Reliable components do not guarantee

reliable Services– Loosely coupled composite apps require

new approaches to reliability

• SOA ROI hangs in the balance


XML: The Foundation for SOA

• XML now the lingua franca of distributed messaging

• The XML foundation must perform or the SOA will crumble

http://www.bluestonepartners.com/schemas/StockTrader/RequestQuote

http://schemas.xmlsoap.org/ws/2003/03/addressing/role/anonymous

Message ID and UUID

http://localhost:8080/StockTraderSecure/StockTrader.asmx

Contains Message Creation/Expiration TimeStamps

MSFT

Source: http://www.windowsitlibrary.com/Content/1219/06/1.html

4


The SOA Performance Issue

• SOA requires more metadata than ever…– Web Services-based SOA is mostly

XML-based– XML is text heavy and verbose– More metadata… more XML… more

headaches• The XML Processing challenge

– Per-message parsing, processing– Security steps– Validity steps– Mapping XML to memory models– That darn text format!

• Security, Reliability, Process, and Transaction now on a per-message basis

• High volumes of messages and very largemessages two separate problems


The OSI Stack Revisited

• Content Security, as well as– Semantics– Taxonomies– Policy– Schemas– …

7

6

5

4

3

1

2Traditional perimeterTraditional perimeter--

based approaches wholly based approaches wholly inadequateinadequate

5


The XML Processing Problem

– Receive– Authenticate– Decrypt– Validate– Canonicalize– Transform– Parse– Aggregate– Sign– Encrypt– Transmit

• Steps required to process XML for each message:


parse…send… receive…

understand… store…update… validate…

…parse

The XML Performance Crisis

XML is “Heavy”$FF becomes:

255

Large XML data sets are sent to conform to

bloated XML “industry standard” schema

definitions

Eventually this huge document needs to be fully processed –

which means turning it into usable “objects”

Traditional applications are adapted to services,

sending thousands of messages over the wire

Source: RogueWave

6


What about Binary XML?

• Binary– Efficient, Compact, Proprietary

• “Dumb” approach: compress & decompress

• Adds to processing overhead– Smarter approach: binary

representation of preprocessed XML

• Incremental transmission andprocessing of XML documents

Binary XML of limited use until some standard Binary XML of limited use until some standard becomes widely adoptedbecomes widely adopted


XML traffic on the Network

Network Traffic Utilization

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008

Other

XMLMail*HTTP*

Network Traffic Utilization

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008

Other

XMLMail*HTTP*

Makeup of XML Traffic

0%10%20%30%40%50%60%70%80%90%

100%

2004 2005 2006 2007 2008

Device-to-device messages

REST-based messages

"Proprietary" XML formats

Web Services

Makeup of XML Traffic

0%10%20%30%40%50%60%70%80%90%

100%

2004 2005 2006 2007 2008

Device-to-device messages

REST-based messages

"Proprietary" XML formats

Web Services

7


XML in the Networked Enterprise

Source: Tarari

Storage

StorageSwitch

XMLIntegrationSwitch/LoadBalancer

Router

FirewallFirewall XML Security

FirewallVPN

Bandwidth Mgr.

VoIP Server

IDS

Cache

Switch/LoadBalancer

SSL

XML Mgmt.

Switch/LoadBalancer

XML Router

XML Gateway

XML Transformation

Front-endServer

Back-endServer

Storage

Internet

PSTN

Access Control Front-End ServersBack-End Servers & Storage

Purpose:• First line security• Authentication• Access Control• WAN Mgmt

Purpose:• Apply security policies • Control traffic via Services• LAN Management• Switching & Routing

Purpose:• Execute applications• Enable internal server to

server communications• B2B & EAI

Purpose:• Database Access• Transaction Control

AV GatewayPub/SubContent Based Switching/Routing

Application Firewalls

Security & Gateways

Publish & Subscribe

Web Services Management

Web Acceleration/Control

“Raw” XML Acceleration

XML Databases

Web Application Servers

Events Processing


Transformation: A processing challenge

SOA relies on loose coupling, loose coupling relies on transformation

Problem: not all Consumers are the same!

• While 80% of a document format or Service might be appropriate, it’s the 20% differences that cause all the problems– Data format differences– Standards versioning– Protocol differences– Loosely coupling presentation from data

8


Hardware vs. Software Approaches to Improve XML Performance

• Pros:– Dedicated processing

power– Control of variability– Hardened OS– Simple to install &

maintain• Cons:

– Must go in data center– More difficult to upgrade– Limited customizability

• Pros:– Developers in control– Easy to customize &

upgrade• Cons:

– Harder to install & maintain

– Relies upon hardware for performance

– Too much variability in configuration

Hardware Software


Is Moore’s Law the Solution?

• One idea – throw more iron at the problem– Moore’s Law and its corollaries – growing

processor power, storage space,bandwidth…

– Cost per computing is going down…– But cost of management and complexity,

flexibility remains high• Biggest problem – where’s the loose coupling?

– What’s the logic in using a general-purposeapp server for application-specificprocessing?

– Bottlenecks require intelligent solutions to be cost-effective

• SOA impacts ALL endpoints, not just Service Providers!

9


Challenges in Content Processing

• Transformation & content-based routing -two parts of same problem: getting systems to understand what they are transmitting.

• Significant operational problems:– Requires intensive processing, especially at wire speed– How involved is a human in the mapping steps?– Simplifying the management of transformations

You must address BOTH transformation and performance issues without trading off either!


Concept review: XML Performance Concerns

• XML is text heavy & verbose.

• XML processing/parsing requires many steps.

• Human-readable XML introduces security issues.

• The volume of XML network traffic is rapidly increasing – SOA is just one driver.

• It’s important to take a smart approach to dealing with XML performance.

10


First Roadblock: Security

Service-orientedcomputing

Network (Packet) Level

TCP/IP packetinspection

Perimeter-based

Fragmented Control

Islands of securityMaintained as separate

units

Closed SystemsProprietary, closed

APIsTight coupling

Single administratorLocalized users

Application Level

XML-basedContent aware

Application control

Open Systems

Open, accessible APIsLoose coupling

Multiple administratorsDistributed users

Managed Security

Whole network policiesTiered administration

Traditional DistributedComputing


The “20 doors” problem

• Any Web Services security approach must be a part of a comprehensive enterprise security initiative

If your company has 20 doors, If your company has 20 doors, and you lock 19, are you 95% secure?and you lock 19, are you 95% secure?

11


Addressing Security Issues

• Should have security assessment in place

• Identify scope of identity management need

• Determine/develop enterprise ID policies (may be a project in its own right)

• Identify appropriate solution/vendor– Single sign-on– Application security– Web Services security


Message-Level Security

• Digitally signed SOAP request message (simplified)

• Security parts highlighted

12


The Security Context Challenge

rschmelzer

RonSchmelzer

rschm123

Selective

Read / Write

Read Only

Full Read/Write???

???


Solving the Security Context Challenge

NetworkFirewall

SAML

SOAPHTTPSMTPFTP

JMS/MQ

SecurityAppliance(2)

SOAP ProxyWSM

Enforcement(1)

TxMinderXML Agent

NetegrityPolicy Server

Proprietary Legacy

.NET

J2EE

TxMAgentSOAP

User Stores(LDAP, RDBMS, etc.)

WSMPolicies

NetworkFirewall

TxMAgent

WSMEnf.

WSMEnf.

SAML

Source: CA Netegrity

13


Identity Management: Kill Two Birds…

• Many enterprises already dealing with “Single Sign-On”– “Sticky Note” problem: too many

passwords for too many systems– Problems administering users– Too many people with root access– Unknown security holes

• Now: need enterprise ID & access management to prepare for SOA


21st Century Network Security

• The Old Days:– Internal = Trusted– External = Untrusted

• Today:– Internal = Untrusted– External = Hostile!

You MUST address security You MUST address security issues throughout the issues throughout the

networknetwork

14


Federated Security

• Coordinate two separate security domains via a trust relationship

• Can be internal to an enterprise or B2B

• Enables scalability of enterprise security

• Based upon standards:– SAML – Security Assertions Markup Language– WS-Security – Web Services Security– WS-Trust – Web Services Trust

• Similar approach offered by Liberty Alliance (see http://www.projectliberty.org)


Federated Security Illustration

.NET Client

AuthoritativeIdentity

Foundation

Web Services Consumer

AuthoritativeIdentity

Foundation

Web Services Provider

SecurityTokenService

local ID Token

J2EE

Token local ID

SecureSpan Bridge

SecureSpanGateway

Policy Service

SOAP Request SecurityToken

SecurityTokenService

Policy Service

WS-Trust

WS-Trust

Source: Layer 7 Technologies

State ofthe Art!

15


SOA Management: Many Facets

• System management – identifying root causes of technical problems, mitigating or solving them

• Metadata management – storing, managing, and enabling the use of various metadata

• Business Service management – managing business Key Performance Indicators (KPIs) in the context of SOA


The Problem with SOA Management…

• Managing Service interfaces & dependencies

• Application management

• Systems management

• Network management

• Root cause analysis• Etc…

16


The First Rule of SOA Management

• You need management when you offer your first “mission critical” Web Service

• Even one Service requires management

• Don’t wait!


The SOA Management Conundrum

ApplicationInfrastructure

Database

Systems

Network

Business Process

ApplicationJ2EE

IIS WebSphere MQ Series

Oracle9i

Windows UNIX

TCP/IP

Hire New Employee

SAP

Exchange/Domino

zOS

WS & SOA

Source: CA

17


Exception Management & SOA

• “Passive” management– Problem Alert Human intervention– Breaks loose coupling!

• “Active” management– Crosses threshold automated response

problem prevented– Maintains loose coupling– Much more difficult!

State ofthe Art!


SOA Enablement…

• Provide and enforce the SOA layer of abstraction

• Combine fine-grained APIs into coarse-grained business Services

• Mask complexity of underlying technology: message protocols, adapters, APIs, etc.

• Handle quality of service, scalability, etc. “behind the scenes”

18


The State of the Market

• All balls must be in the air at once

• Web Services do not create a permanent, distinct market

• New entrants jockeying for position while incumbents wait/build/acquire

Security

Inte

gra

tion

Proc

ess

Man

agem

ent

Tool

s


The need for SOA Infrastructure

• Service-Oriented Architecture is a set of design principles, methodologies, and best practices for implementing distributed computing using loosely-coupled, composable Services on a heterogeneous network.

• However, SOA doesn’t define exactly how to implement those Services, or what infrastructure to use to make sure the Services can communicate with each other or be made available on the network

• In fact, SOA as a design is not supposed to mandate specific implementation approaches – that’s how we can guarantee loose coupling in the face of heterogeneity.

19


Invocation Mechanisms in SOA

• SOA is more flexible than client/server –supports multiple invocation mechanisms

– Request/Reply

– Publish/Subscribe

– Routed Events

– Reliable Messaging

Serviceconsumer

Serviceprovider

Request channel Request

Reply channelReply

Topic

Data store

Serviceprovider

Serviceconsumer

Serviceconsumer

Message server

Subscribe

Input channel Event

Output channel Event

Output channel Event

Input channelServiceprovider

Serviceconsumer

Serviceconsumer

Output channel

Output channel

Content-basedRouterMessage

Serviceprovider

Serviceconsumer

Message bus

Message Message

Data store Data store


What are Events?

• Ordinary event – something that happens in the real world

• Ordinary business event – a meaningful change in the state of the enterprise or of something relevant to the enterprise– Customer order, the arrival of a shipment,

etc.

• Software event – a record of an ordinary event in software. – Data that describe the ordinary event,

typically in the form of a message

20


Lightweight Events in SOA

• When WSDL is overkill– Only binding information must be

contracted– Ad hoc contract may be sufficient– Situations where some tight coupling is OK

• When HTTP and SSL are sufficient– Most Web Services over HTTP today– SSL provides confidentiality but not content-

based security


Messaging Models

• Store-and-forward– Messages delivered to a one or more coordinating

authorities– These forward to end destination– Post office model– SMTP

• Point-to-point– Real-time connection between systems

21


Message-based Approaches to SOA

• Why SOA Prefers Message-based approaches over RPC-style– Fundamental tenet of loose coupling: not being aware of end point

requirements– Composited (virtualized) Web Services may require greater time for

processing, requiring asynchrony– B2B processes are often asynchronous– Distributed systems can be more reliable when they are asynchronous– Heterogeneous systems, especially those with limited bandwidth

devices, function better asynchronously.– Support human involvement in processes

• Messages, Events, part of the same asynchronous requirements

• ENTER THE Enterprise Service Bus….


The “Enterprise Service Bus”

• What is an ESB? TRICK QUESTION!– MOM++

• Messaging infrastructure or message-based middleware that can natively deal with Service interfaces and composition

– Service Intermediary / Broker• A broker-based intermediary (hub-and-spoke) for intermediating

Service requests on behalf of Service consumers and producers– EAI++

• EAI technology “warmed over” with Service interfaces, but nothing in the infrastructure itself that is Service-Oriented

– Architectural Pattern, not Product• ESB is not a product at all, but an “architectural pattern” – that is, you

can implement an ESB using whatever technology you want, as long as it gives you the design goals of secure, reliable, robust communication using message-based or asynchronous styles as well as synchronous.

• The real issue: – HTTP / Web Services by itself is too immature to support secure, reliable, and

robust long-lived Service interaction, so a stronger infrastructure is needed, hence the need for “something more” than just Web Services…

– People acknowledge that an ESB is a sort of infrastructure for implementing SOA, but there’s NO AGREEMENT on what an ESB should do, nor what it should look like.

SOA is something you do -- not something you buy.

22


Lack of Convergence on ESB Meaning

• Things that ESBs should do:– Facilitate Service exposure– Enable Service composition– Abstract Service runtime heterogeneity– Enable event-driven and asynchronous modes of Service interaction

• Things that ESBs can do:– Business Process Management / Modeling (BPM)– Registry / Repository capability– Provide an actual message bus / message-oriented middleware (not necessary!)

• Things that ESBs shouldn’t do:– EAI with Service interfaces. SOA represents an approach to integration that is

fundamentally opposed to a hub-and-spoke, tightly-coupled approach to integration. Composition favored over explicit mapping

“At this point in time, no single vendor provides a complete SOA infrastructure, and certainly no single product can provide a complete infrastructure.”

• Anne Thomas Manes


Mapping Market Convergence…

SOAImplementation

Framework

SO Mgmt

WSManagement

SystemsManagement

SOAEnablement

SOAI

ESB

IntegrationBrokers

App Server"Platforms"

ApplicationServers

Message-Oriented

Middleware

TransactionMonitors

EAI

SOProcess

BPM

BPI

SOSecurity

WS Security

PKI

IAM

EII

BI Analytics

BAM

SOA Tools

ModelingTools

ApplicationFrameworks

RAD Arch.Tools SODevelopment

WS Tools

IDEs

XMLAppliances

NetworkAppliances

SemanticIntegration

DataIntegration

ETL

Portals

PresentationTools

CMS

OLAP

NXDs

Databases

OperationalData Stores

DataWarehouses

Copyright © 2003 ZapThink LLC

B2Bi

SOII

SO Content

Core SOMarkets

MarketsRemaining

Distinct

TransitionalWS Markets

EstablishedMarkets

23


Next Steps?

• Take iterative approach to reduce risk• Security & management usually come first• Build SOA top-down (architectural plan) and

bottom-up (build Services from existing resources)

Security

Inte

gra

tion

Proc

ess

Man

agem

ent

Tool

s


Thank You!

ZapThink is an advisory, analysis, & influence firm focused exclusively on Service-Oriented Architecture, Web Services, & Enterprise Web 2.0.

Read our new book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business.

Jason Bloomberg

[email protected] Schmelzer

[email protected]

5a. the nuts and bolts of soa - dovel...

Documents