copyright solidsoft 2006. © solidsoft limited 2005 copyright solidsoft 2006 building an enterprise...

Copyright Solidsoft 2006

© Solidsoft Limited 2005


Building an Enterprise Service Building an Enterprise Service Bus with BizTalk Server 2006 & Bus with BizTalk Server 2006 &

WCFWCF

Charles YoungCharles YoungPrincipal ConsultantPrincipal Consultant


AgendaAgenda

Enterprise Service BusesEnterprise Service Buses

BizTalk Server and WCFBizTalk Server and WCF

Rethinking BizTalk ServerRethinking BizTalk Server

Iterating the designIterating the design

ESB Guidance ToolkitESB Guidance Toolkit


Enterprise Service Bus Enterprise Service Bus Common ThemesCommon Themes Defining ‘ESB’Defining ‘ESB’

• Many variations, products and technologiesMany variations, products and technologies• Commonality and convergenceCommonality and convergence

Distributed, ubiquitous ‘fabric’ for message exchange and integrationDistributed, ubiquitous ‘fabric’ for message exchange and integration Central administration and deploymentCentral administration and deployment Standards-based and secure messagingStandards-based and secure messaging Lightweight containersLightweight containers Message exchange patterns & routingMessage exchange patterns & routing Adaptation and mediationAdaptation and mediation Persistence and recoverabilityPersistence and recoverability Orchestration and choreographyOrchestration and choreography


The Microsoft SOA platform todayThe Microsoft SOA platform today

BizTalk Server 2006 R2BizTalk Server 2006 R2• Message brokerage (pub/sub, message stream processing)Message brokerage (pub/sub, message stream processing)• Integration toolset (adapters, transformation, EDI support, etc.)Integration toolset (adapters, transformation, EDI support, etc.)• OrchestrationOrchestration• Business activity monitoringBusiness activity monitoring• Business rules, Single Sign-on, B2B support, development tooling etc.Business rules, Single Sign-on, B2B support, development tooling etc.

Windows Communication Foundation (WCF)Windows Communication Foundation (WCF)• Service model – loosely-coupled compositionService model – loosely-coupled composition

Asynchronous messaging primitivesAsynchronous messaging primitives MEP (message exchange patterns) and extensible behavioural supportMEP (message exchange patterns) and extensible behavioural support

• Channel architecture - transports and encodingsChannel architecture - transports and encodings Windows Workflow Foundation (WF)Windows Workflow Foundation (WF)

• Activity model – tightly-coupled compositionActivity model – tightly-coupled composition Workflow models – sequential, state transitionWorkflow models – sequential, state transition Base activity libraryBase activity library Extensible architectureExtensible architecture RulesRules


Who are you kidding?Who are you kidding?

““Service Buses? Microsoft doesn’t get it!”Service Buses? Microsoft doesn’t get it!”• ……but Microsoft says: but Microsoft says:

““The Service Bus is a set of design patterns”The Service Bus is a set of design patterns”

““The Service Bus is about the platform, not the product”The Service Bus is about the platform, not the product”

““ESB is a subset of the service bus approach. Other subsets exist”ESB is a subset of the service bus approach. Other subsets exist”

““BizTalk Server is a mere hub & spoke message broker”BizTalk Server is a mere hub & spoke message broker”• BizTalk Server is pre-ESB, but firmly rooted in the same conceptsBizTalk Server is pre-ESB, but firmly rooted in the same concepts

““WCF is interesting, but not ESB”WCF is interesting, but not ESB”• That’s the point! It’s a That’s the point! It’s a foundationfoundation for building service buses. for building service buses.

• ‘‘Oslo’ will build on WCF Oslo’ will build on WCF extending the service bus concept across applications, enterprises and ‘cloud’ computingextending the service bus concept across applications, enterprises and ‘cloud’ computing


BizTalk Server BizTalk Server Service Bus Service Bus

Robust, recoverable message exchange .Robust, recoverable message exchange . Message Exchange Patterns (MEPs)Message Exchange Patterns (MEPs)

• 1-way, 2 way, reliable, ordered, etc1-way, 2 way, reliable, ordered, etc• No first-class support for duplex, but possible through custom adaptersNo first-class support for duplex, but possible through custom adapters

Powerful pub/sub mechanismPowerful pub/sub mechanism Process AutomationProcess Automation

• Orchestration engineOrchestration engine Distribution and ScalabilityDistribution and Scalability

• Hosting model across multiple servers (Enterprise edition only)Hosting model across multiple servers (Enterprise edition only)• Load-sharingLoad-sharing

Single-point administration (Enterprise edition only)Single-point administration (Enterprise edition only) Central ‘resource’ repository to aid deploymentCentral ‘resource’ repository to aid deployment Adapter frameworkAdapter framework

• OOB adapters, third-party adapters and custom adaptersOOB adapters, third-party adapters and custom adapters


BizTalk Server BizTalk Server Service Bus Service Bus

Centralised modelCentralised model• LicensingLicensing• ‘‘Heavyweight’ modelHeavyweight’ model

All messages persisted to Message BoxAll messages persisted to Message Box• No lightweight, non-persisted, non-recoverable modelNo lightweight, non-persisted, non-recoverable model• Low latency and real-time challengesLow latency and real-time challenges

Supports built-in service types onlySupports built-in service types only• Orchestrations, Receive and Send portsOrchestrations, Receive and Send ports• Must use adaptation for other servicesMust use adaptation for other services• Leads to over-use of orchestrationLeads to over-use of orchestration

No built-in support for common ESB design patternsNo built-in support for common ESB design patterns• Addressed via the ESB Guidance ToolkitAddressed via the ESB Guidance Toolkit


WCF WCF Service Bus Service Bus

Part of .NET FrameworkPart of .NET Framework• Freely distributable – no licensing costsFreely distributable – no licensing costs

Provides lightweight container modelProvides lightweight container model• Service and channel modelService and channel model• Hosting modelHosting model• WASWAS

Wide range of MEPs, including duplexWide range of MEPs, including duplex Extensive support for SOAP and WS-*Extensive support for SOAP and WS-*

• New support for Web 2.0 and RESTNew support for Web 2.0 and REST WCF LoB Adapter FrameworkWCF LoB Adapter Framework

• Platform-level adaptersPlatform-level adapters• Exposed as WCF bindingsExposed as WCF bindings


WCF WCF Service Bus Service Bus

Foundational onlyFoundational only Administration, management and deploymentAdministration, management and deployment

• Does not provide OOB enterprise-level toolingDoes not provide OOB enterprise-level tooling No pre-built support for pub/subNo pre-built support for pub/sub

• May be addressed in ‘Oslo’May be addressed in ‘Oslo’NB. early previews of ISB contain pub/sub modelNB. early previews of ISB contain pub/sub model

• Can be implemented today using custom codeCan be implemented today using custom code No pre-built persistence featuresNo pre-built persistence features

• May be addressed in ‘Oslo’May be addressed in ‘Oslo’ No pre-built support for many common ESB design patternsNo pre-built support for many common ESB design patterns

• May be addressed in ‘Oslo’May be addressed in ‘Oslo’• Some support today via ESB Guidance ToolkitSome support today via ESB Guidance Toolkit

Limited support for orchestration and choreographyLimited support for orchestration and choreography• Can use WF with WCFCan use WF with WCF


End Point Management

Pipelines, Maps

XLANG/sEngine

Orchestration

BizTalk Server – The message box viewBizTalk Server – The message box view

MessageAgent

MessageBox

Messaging

External Services

Se

nd

Po

rt

PrimaryTransport

SecondaryTransport(optional)

External Systems & Applications

Transport Proxies

Receive Port

ReceiveLocations


BizTalk Server – The service bus view BizTalk Server – The service bus view

End Point Management

Pipelines, Maps

XLANG/sEngine

Orchestration

MessageAgent

MessageBox

Messaging

External Services

Se

nd

Po

rt

PrimaryTransport

SecondaryTransport(optional)

External Systems & Applications

Transport Proxies

Receive Port

ReceiveLocations

Local

Endpoints

Internal

Endpoint

External Endpoints

BizTalk Server Service Bus

MessageBox

MessageAgent

Orchestration

Orchestration

Internal Endpoints


BizTalk Server and WCFBizTalk Server and WCF

BizTalk Server 2006 R2 WCF adapterBizTalk Server 2006 R2 WCF adapter• Receive and SendReceive and Send• Bindings, including custom bindingsBindings, including custom bindings

Bus ‘segments’ – Technology-specificBus ‘segments’ – Technology-specific• Idiosyncratic, but realisticIdiosyncratic, but realistic

No (or limited) common administration storyNo (or limited) common administration story No central deployment story across both segmentsNo central deployment story across both segments Different host modelsDifferent host models

• But…But… Design patterns can span segmentsDesign patterns can span segments Common standards and message typesCommon standards and message types Can implement common service directoriesCan implement common service directories


BizTalk Server – Basic SO DesignBizTalk Server – Basic SO Design


BizTalk Server and WCF – Iteration 1BizTalk Server and WCF – Iteration 1

Refactor business servicesRefactor business services• WCF services WCF services • WCF bus segmentWCF bus segment• MotivationMotivation

Stronger emphasis on contractsStronger emphasis on contracts Exploit WS-* standards, e.g., for message level securityExploit WS-* standards, e.g., for message level security Enhanced performanceEnhanced performance Alignment with future platformAlignment with future platform

Introduce Service DirectoryIntroduce Service Directory• Endpoint resolution and dynamic routingEndpoint resolution and dynamic routing• Optional dynamic transformation, itineraries, etc.Optional dynamic transformation, itineraries, etc.• Implemented using UDDI, rules engine, custom database, configuration file, etc.Implemented using UDDI, rules engine, custom database, configuration file, etc.• MotivationMotivation

Stronger emphasis on service orientationStronger emphasis on service orientation Improved decoupling of servicesImproved decoupling of services Simpler deployment and versioningSimpler deployment and versioning


BizTalk Server and WCF – BizTalk Server and WCF – Iteration Iteration 11



Re-factor orchestrationsRe-factor orchestrations• Use WCF services and WF workflows Use WCF services and WF workflows • MotivationMotivation

Reduce message box hops (e.g., calls to business services) Reduce message box hops (e.g., calls to business services) Improve latencyImprove latency Exploit WF extensibilityExploit WF extensibility

IssuesIssues• What about scalability?What about scalability?

Load balancing in very simple scenariosLoad balancing in very simple scenarios How can context and security state flow?How can context and security state flow? How can long-lived activities and correlation be handled?How can long-lived activities and correlation be handled?

• What about resilience and recoverability?What about resilience and recoverability? Push up to master orchestrations in BizTalkPush up to master orchestrations in BizTalk Implement custom WF hosts for persistence if necessaryImplement custom WF hosts for persistence if necessary

Use BizTalk orchestrations where necessaryUse BizTalk orchestrations where necessary Use WCF and WF where possibleUse WCF and WF where possible



Identify common concernsIdentify common concerns• Endpoint resolutionEndpoint resolution• Dynamic transformationDynamic transformation• ItinerariesItineraries• On / off rampsOn / off ramps

Message standards and canonicalisation Message standards and canonicalisation Avoid multi-part messages (ASP.NET and BTS SOAP adapter)Avoid multi-part messages (ASP.NET and BTS SOAP adapter)

Introduce common service bus facilitiesIntroduce common service bus facilities• Share service directory across segmentsShare service directory across segments• Re-factor ESB functions into re-usable components Re-factor ESB functions into re-usable components

E.g., implement resolution components in .NET, not as BizTalk orchestrationE.g., implement resolution components in .NET, not as BizTalk orchestration Use in pipelines, WCF services, etc.Use in pipelines, WCF services, etc.

• MotivationMotivation Support common design patterns and implementation across technology segmentsSupport common design patterns and implementation across technology segments Fully centralise service directoryFully centralise service directory



Patterns and PracticesPatterns and Practices• Guidance & best practiceGuidance & best practice• Documented design patternsDocumented design patterns• Production-ready code (open source)Production-ready code (open source)

Targets BizTalk Server R2 and WCFTargets BizTalk Server R2 and WCF Not ‘Oslo’!Not ‘Oslo’! Use as a ‘starter’ packUse as a ‘starter’ pack

• Extend or create componentsExtend or create components• Adapt codeAdapt code• Add to functionalityAdd to functionality



ItinerariesItineraries• Defined in SOAP headersDefined in SOAP headers• Dependant on BizTalk subscriptionDependant on BizTalk subscription• Message-box boundMessage-box bound

ResolutionResolution• Framework and componentsFramework and components• Caching – essential for performanceCaching – essential for performance• Pre-built components for UDDI, rules engine, WS-MEX, etc.Pre-built components for UDDI, rules engine, WS-MEX, etc.• Not bound to BizTalk Server or message box.Not bound to BizTalk Server or message box.

Dynamic TransformationDynamic Transformation• Direct invocation of BizTalk Server mapsDirect invocation of BizTalk Server maps• Not bound to message boxNot bound to message box• May be limited to BizTalk Server machinesMay be limited to BizTalk Server machines



Exception ManagementException Management• Unifies error handling in BizTalk ServerUnifies error handling in BizTalk Server

Failed message routingFailed message routing Custom exceptionsCustom exceptions

• Fault ReportingFault Reporting ESB Guidance Toolkit portalESB Guidance Toolkit portal BAMBAM

Management and OperationsManagement and Operations• Management PortalManagement Portal

Fault reportingFault reporting Alerts and notificationsAlerts and notifications AuditingAuditing PerformancePerformance

GovernanceGovernance• Integration with Business Process Alliance Integration with Business Process Alliance

AmberPointAmberPoint SOA SoftwareSOA Software


Today’s Platform: Gap AnalysisToday’s Platform: Gap Analysis

Spectrum of architectural concernsSpectrum of architectural concerns• Latency & throughput Latency & throughput Recoverability & traceability Recoverability & traceability

• Stateless Stateless Stateful services Stateful services

Unified administrationUnified administration• ‘‘Centralised’ model partly supports thisCentralised’ model partly supports this

Unified deployment and configurationUnified deployment and configuration• BizTalk offers some limited supportBizTalk offers some limited support

ESB design patternsESB design patterns• Use ESB Guidance Toolkit, Use ESB Guidance Toolkit, with carewith care, as starting point, as starting point


Aligning with the futureAligning with the future

‘‘Oslo’Oslo’• Microsoft’s vNext SOA platformMicrosoft’s vNext SOA platform

• Builds on WCFBuilds on WCF

• Introduces services bus technologies at platform levelIntroduces services bus technologies at platform level Service BusService Bus Internet Service Bus (ISB – see http://biztalk.net)Internet Service Bus (ISB – see http://biztalk.net)

BizTalk Server and ‘Oslo’BizTalk Server and ‘Oslo’• ““Nothing goes away” in vNextNothing goes away” in vNext

• Commitment to smooth migration pathCommitment to smooth migration path

• Deeper integration of BizTalk Server and WCF expectedDeeper integration of BizTalk Server and WCF expected


Extending the BusExtending the Bus

Geographic distributionGeographic distribution• Secure channelsSecure channels

• Standards & canonicalisationStandards & canonicalisation

Common PatternsCommon Patterns• Local integration serversLocal integration servers

• Bridges and gatewaysBridges and gateways

Challenge of ‘cloud’ computingChallenge of ‘cloud’ computing• ContextContext

• SecuritySecurity

• RoutingRouting

• ‘‘Oslo’ Internet Service BusOslo’ Internet Service Bus

Local Branch

Integration Server

Bridging Components

Local Bus / Systems

Inte

gra

tio

n

Se

rvic

e

National / Global Systems

Gateway

Inte

gra

tio

n

Se

rvic

e

Service Bus


Service Bus – BizTalk Server segment

Service Bus – remote segmentSystems & Applications

Master

Service Bus – WCF segment

Secure Channels

AuthenticatedConnections

Secure Channel

AuthenticatedConnection

FederatedAuthentication

Secure Channel

AuthenticatedConnection

BizTalk ServerTrust Model

Trust Boundaries and FederationTrust Boundaries and Federation


Resilience and RecoverabilityResilience and Recoverability

Leverage BizTalk ServerLeverage BizTalk Server• Push recoverability model ‘up the stack’Push recoverability model ‘up the stack’

• Master orchestration patternsMaster orchestration patterns

• Custom retry logicCustom retry logic

• Stateless servicesStateless services

Load-balancingLoad-balancing• BizTalk modelBizTalk model

• Using BizTalk and WCF with load-balancingUsing BizTalk and WCF with load-balancing


Service Proxying for ResilienceService Proxying for Resilience

Introducing State managementIntroducing State management• Complex to implementComplex to implement

• Centralised across farmsCentralised across farms

• Use WF serialisation capabilitiesUse WF serialisation capabilities

Service proxyService proxy• Server-side instance mangerServer-side instance manger

• Access to state serverAccess to state server

• Flow security and message contextFlow security and message context

• Affinity to instancesAffinity to instances

• Re-start instances over persisted stateRe-start instances over persisted state


Monitoring, Management and GovernanceMonitoring, Management and Governance

Unified Business Activity MonitoringUnified Business Activity Monitoring• Extend BAM across segmentsExtend BAM across segments

WCF BAM InterceptorWCF BAM Interceptor

Health monitoringHealth monitoring• Unification is a challengeUnification is a challenge

• Enterprise-wide event monitoringEnterprise-wide event monitoring MOMMOM

PoliciesPolicies• Reusable frameworksReusable frameworks

Enterprise libraryEnterprise library

• RulesRules WF RulesWF Rules MS BREMS BRE


Monitoring, Management and GovernanceMonitoring, Management and Governance

Establishing Management PortalsEstablishing Management Portals• MOSSMOSS

Flexible & powerfulFlexible & powerful Implementation requiredImplementation required

• BizTalk BAM portalBizTalk BAM portal

• ESB Guidance Toolkit management portalESB Guidance Toolkit management portal

Integration with Governance ToolsIntegration with Governance Tools• AmberPointAmberPoint

• SOA SoftwareSOA Software


SummarySummary

The Enterprise Service BusThe Enterprise Service Bus• Patterns, not productsPatterns, not products

• Today: BizTalk Server and WCFToday: BizTalk Server and WCF

• Tomorrow: ‘Oslo’, BizTalk Server and WCFTomorrow: ‘Oslo’, BizTalk Server and WCF

• Today’s challengesToday’s challenges

• Tomorrow’s promiseTomorrow’s promise


Call to ActionCall to Action

Think ‘service bus’Think ‘service bus’• Investigate patterns and conceptsInvestigate patterns and concepts

Understand WCF and WFUnderstand WCF and WF Investigate BizTalk ServerInvestigate BizTalk Server Use technologies togetherUse technologies together

• Keep it simple - don’t invest in over-complexityKeep it simple - don’t invest in over-complexity

• Re-factor intelligentlyRe-factor intelligently

• Play to strengthsPlay to strengths

Thank you

copyright solidsoft 2006. © solidsoft limited 2005 copyright solidsoft 2006 building an enterprise...

Documents

wcf biztalk server

service bus approach

message broker biztalk

r2 biztalk server

copyright solidsoft

r2 message brokerage

edi support

set of design patterns