enterprise service bus at dol

DOL SOA IMPLEMENTATION

DOL SOA Implementation Overview

August 25, 2008


Infrastructure Overview

3


P570 Hardware Virtualization and LPAR’s

4


Location 1p570-SN10EC3CD

Power5+ Max CPUs: 16 Owned: 16 Memory: 48

Location 2p570-SN10EC3BD


IBM HTTP Server

WebSphere Process Server

WebSphere Application Server

WebSphere Server registry repository

WebSphere Message Broker

WebSphere Federation Server

LPAR

AIX 5.3 & IBM Http Server 6.1.0.13

AIX 5.3, WAS 6.1.0.13 & WPS 6.1.0.1

AIX 5.3 & WAS 6.1.0.13

AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2

AIX 5.3 & WMB 6.1.0.2, ,

AIX 5.3 & WFS 9.1


AIX 5.3 & WAS-ND 6.1.0.13

AIX 5.3 WAS 6.1.0.13 & WSRR 6.1.0.2

AIX 5.3 & WMB 6.1.0.2

AIX 5.3 & WFS 9.1

AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1

IBM HTTP Server






LPAR

Installed Software Installed SoftwareData Center 1 Data Center 2

Legend:Legend:

= Production= Production

= Staging= Staging

= System= System

AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4 AIX 5.3, WAS 6.1.0.13 & IICE 8.4

WebSphere IICE Server

IBM HTTP Server




VIOS 1

VIOS 2


AIX 5.3 & WAS-ND 6.1.0.13

AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2

AIX 5.3 & WFS 9.1

AIX 5.3, IBM Http Server 6.1.0.13

AIX 5.3 & WAS-ND 6.1.0.13

AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1

AIX 5.3 & WMB 6.1.0.2


IBM HTTP Server





VIOS 1

VIOS 2AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4

5


Global Load Balancing and Software Clustering

6


7


Location 1p570-SN10EC3CD


Location 2p570-SN10EC3BD


IBM HTTP Server






LPAR


AIX 5.3, WAS 6.1.0.13 & WPS 6.1.0.1

AIX 5.3 & WAS 6.1.0.13

AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2

AIX 5.3 & WMB 6.1.0.2, ,

AIX 5.3 & WFS 9.1


AIX 5.3 & WAS-ND 6.1.0.13

AIX 5.3 WAS 6.1.0.13 & WSRR 6.1.0.2

AIX 5.3 & WMB 6.1.0.2

AIX 5.3 & WFS 9.1

AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1

IBM HTTP Server






LPAR

Installed Software Installed SoftwareData Center 1 Data Center 2

Legend:Legend:

= Production= Production

= Staging= Staging

= System= System

AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4 AIX 5.3, WAS 6.1.0.13 & IICE 8.4


IBM HTTP Server




VIOS 1

VIOS 2


AIX 5.3 & WAS-ND 6.1.0.13

AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2

AIX 5.3 & WFS 9.1

AIX 5.3, IBM Http Server 6.1.0.13

AIX 5.3 & WAS-ND 6.1.0.13

AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1

AIX 5.3 & WMB 6.1.0.2


IBM HTTP Server





VIOS 1

VIOS 2AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4

8


9


10


11


12


13


14


15


16


17


18



Enterprise Architecture Overview

20


SOA, ESB and BPEL

NYSDOL enterprise architecture implements SOA on top of an ESB, so that we could have the virtues of SOA, with room to grow.

SOA : Service Oriented Architecture is a business-centric IT architectural approach that supports integrating business as linked, repeatable business tasks, or services.

ESB : Enterprise Service Bus is an architectural pattern to integrate and manage services, not a software product. We can form an ESB with different software products leveraging specific functionality of each product to meet unique requirements.

BPEL : Business Process Execution Language is a standards-based way of orchestrating a business process composed of services.

We have also added BPEL to ESB (Process Server) to easily compose new services out of existing services.

21


ESB or BPELESB : Message routing, Message transformation, Protocol mediation, stateless transactions, integration middleware for off-the shelf products, security, logging, auditing, excellent performance, data-centric requirements.

BPEL : Stateful long-running business processes or transactional micro flows, human tasks, business rules, complex logic, process centric requirements .

ESB Pattern : Our ESB pattern is Gateway ESB pattern and provides a controlled point of external access to services. The gateway/ router is currently implemented as a software program in message broker which is separated from the hub where services are exposed.

22


NYSDOL Enterprise Architecture for SOA based Applications

Message Broker ESB

Websphere Application Server

Websphere Process Server

Web Server

Internal / External Users Browser

External Agencies, Web service clients

Websphere Service Registry and Repository

HTTP(s)

Mainframe

WS

service requests (SOAP/HTTP)

Lookup endpoint / policy

WSDL-E-WSDL (exposed on WMB)

WSDL

Legacy & other services – Q name / node in WMB

WS-Policies

PeopleSoft Content manager

All External web service requests

WS

WSWS

WS

WS

PS1 PS2

OracleDatabase

MQ

Common Logging Database

Policy server LDAP

1

2

3

4

Common logging service

Port / Routing Gateway

HUBProcess services, Legacy & other services exposed

Rec Req Lookup Route

Generic mediation module to lookup services in a process

PS

WS

MQ MQ

WSWS

Xpressions

MQ

OS

OS

Asynchronous call

Lookup to find endpoints for services / workflow policies

1

2

MQ / http(s)

a

b

d

c

Note: Web services are not exposed with E-WSDL in MB, instead they could either be invoked by process server (by BPEL in choreography) or by Message broker (composite service / simple services).

5

6

23


Major Security Considerations in SOASOA introduces new challenges to security as it lowers the barriers between applications (composite services formed with existing / new applications), overcomes technology differences, as interoperability is the key goal of SOA. Some of the new requirements of SOA security are

The identity must be decoupled from the services. All entities in SOA have identities - users, services, and so on, that needs to be properly identified so that appropriate security controls can be applied.

The need to seamlessly connect to other organizations on a real-time, transactional basis.

Each new choreography might require examination of the security policy to ensure it remains valid for this new combination.

The need to manage identity and security across a range of systems and services that are implemented in a diverse mix of new and old technologies

Protection of business data in transit and at rest. Providing end-to-end message security is also a key requirement, because messages can be traversing different transport mechanisms and trust zones. In addition, access must be provided to information (and systems) based on business drivers.

24


Security Aspects for SOAFunctional Aspects:

Authentication - Verifying identity of users

Authorization - Deciding whether or not to permit action on a resource.

Data Confidentiality – Protecting secrecy of sensitive data.

Data Integrity – Detecting data tampering and making sure neither the sender nor the receiver can deny the message that they sent or received.

Protections against attacks – Making sure attackers don’t gain control over applications.

Privacy – Making sure the application does not violate the privacy of users.

Audit - Important events need to be logged and available for real-time or later forensic review

Non-Functional Aspects:

Interoperability - This concern is specific to SOA, where different security solutions must not break compatibility of services that are otherwise compatible.

Manageability - As many different services needs to be protected, the security solution must be easily manageable.

Ease of Development – The security solution must be easy enough to adopt and implement.

Availability – The security solution must not impact the availability of the services.

25


Layered SOA and Security

The layered SOA requires all of these security elements to be present in each layer across infrastructure, application, business services, and development services.

26


New Security Approaches for SOA

Message Level Security : Different parts of a message can be protected differently, to make them usable only by intended parties in the message path.

Security as a service: Security service is central and not part of any application and could evolve in-line with business needs. It offers applications the ability to authenticate, authorize, encrypt/decrypt messages, sign/verify signatures and log messages

Policy-driven Security : Security requirements must not be hard-wired into applications or services themselves. Instead security requirements should be separated from business logic and declared as policies. Policies could be business, architectural, operational.

27


Datapower for SOA Security

Provides detailed logging and audit trail.

Datapower SOA security appliances are purpose-built, easy-to-deploy network devices. It provides integrated message-level security (supports WS-Security, WS-Policy, WS- SecurityPolicy, WS-ReliableMessaging, WS-SecureConversation, WS-Trust, SAML, and LDAP) .

Helps in generating dynamic content, content based routing, enables higher performance at wire speed

Provides protection against XML vulnerabilities by acting as an XML proxy and performing XML well-formedness checks, buffer overrun checks, XML schema validation, XML filtering, and XDoS protection .

Provides centralized security functions and acts as an enterprise wide single security- enforcement point for XML and Web services transactions .

It integrates with WSRR and other policy decision points like LDAP and Siteminder Policy server.

Offers robust service level management, policy management, and Web services management support .

http://www-01.ibm.com/software/integration/datapower/xs40/features/index.html#accesscontrol

http://www-01.ibm.com/software/integration/datapower/xs40/features/index.html#accesscontrol

28


Internal / External Users Browser

NYSDOL Security Architecture for SOA based Applications

External Agencies, Web service clients

Web Services Gateway

Authentication/ authorization (could use X509/Kerberos / digital certificates/ SAML, siteminder etc) XML Threat detectionEncryption (outgoing) / decryption (incoming)Pass all authenticated request to XI50, log unauthenticated requests(optionally notify)

All External web service requests (SOAP/ HTTP req)

Policy server LDAP

Finer level AuthorizationMessage level security XML Validation /XML AccelerationAudits, exception Logging, notification, RoutingGenerate LTPA Tokens / SAML Assertions for communication with service providers / backends

Web Server

1

1a

2

2a

Message Broker ESB

Websphere Application Server

Websphere Process Server

Websphere Service Registry and Repository

(WSDL, XSD, WS-policies)

2b

PeopleSoft Content manager

OracleDatabase

MQ MQ MQ

Xpressions

MQ

Mainframe

3

LTPA

HTTP(s)

1

2

3

4service requests (SOAP/HTTP(s))

HTTP(s)

2a

MQ / http(s)

LTPA

HTTP / SOAP

HTTP(s)

Authenticate & authorize xml requests (http basic auth)

MQ / http(s)

1a

LTPA


Service Development Overview

30


Integration Layer

Thin Client(Web Browser)

Message Broker and Enterprise Service Bus

Employer Profile Service Address Service Employer Tax Rate Service

Servlet ContainerJSF Pages

Java/J2EEBusiness Object

HTTP Request

HTTP Response

JMS / SOAP

Service Mediation

Service Provider


Integration Overview

32


JNDI Name Propertiesjms/cf/ESB_QM Host: , Port: , Channel: …jms/queue/ESB.ROUTER.REQUEST Qname: ESB.ROUTER.REQUESTjms/queue/ESB.REPLY Qname: ESB.REPLY

Mes

sage

Bro

ker Error and Exception H

andling

WSR

R

ESB

Rou

ter F

low

33


34



Router Overview

36


WMB

Service ConsumerSOAP/HTTP

Service ConsumerXML/WMQ

Service ProviderSOAP/HTTP

Service ProviderXML/WMQ

WSRR

ESB Router

S

X

S

X

SOAP/HTTPPort

XML/WMQPort

SOAP/HTTPPort

XML/WMQPort

S

X

S-X

X-S

Decouple service consumers and service providers Provide a set of ports associated with specific protocolsRoute to any service providers using any protocol

ESB Router

37


WMB

Service ConsumerSOAP/HTTP

Service ProviderSOAP/HTTP

Service ProviderXML/WMQ

WSRR

ESB Router

2

1 SOAP/HTTPPort

SOAP/HTTPPort

XML/WMQPort

3.1

3.2

ESB Router HTTP

1. Service consumer sends a SOAP request message over HTTP to the ESB2. ESB Router looks up in WSRR for requested service provider3. ESB routes the request to the service provider:

1. Listening for SOAP requests over HTTP2. Listening for XML requests over WMQ/JMS

38


ESB Router WMQ

1. Service consumer sends a XML message over WMQ to the ESB2. ESB Router look up the WSRR for requested service provider3. ESB route the request to service provider:

1. Listening for SOAP requests over HTTP2. Listening for XML requests over WMQ/JMS


Process Server Overview

40


Business Process

A business process is a sequential flow of execution paths described in WS-BPEL (Web Services Business Process Execution Language), including:

Which services are invoked

In Which order the services are invoked

The transformation of data output from one service as input to another

41


JNDI Name Propertiesjms/cf/ESB_QM Host: , Port: , Channel: …jms/queue/ESB.ROUTER.REQUEST Qname: ESB.ROUTER.REQUESTjms/queue/ESB.REPLY Qname: ESB.REPLY

Mes

sage

Bro

ker Error and Exception H

andling

WSR

R

ESB

Rou

ter F

low

42


Benefit Charges Adjustment process Assembly

Benefit Charges Adjustment process (Orchestration of composite services) (Service Provider / Service Consumer)

ExperienceRatingAccount service (Service Provider)

UITaxRateCalcRunner service (Service Provider)

Service Provider

Service Provider

Service Consumer

43


Benefit Charges Adjustment Process

BPEL

44


Mediation flow Assembly for WSRR lookup

WPS_WSRR_TXRateMediation module looks for the services in WSRR using the lookup node.

Lookup node retrieve the service end points & send the request to appropriate service .


Error & Exception Handling Overview

46


ERROR AND EXCEPTION HANDLING FRAMEWORK

47


ERROR AND EXCEPTION HANDLING FRAMEWORK

Exceptiondatabase

Check RetryPolicy

Automatic Retry

Enterprise Service Bus

Output queues

Review exceptions

MB Exception Console

Notify - Email, Pager

Re-submit Process1

4

5

2.16

8

Ext. AppListener

Resolve/Compensate

7.1 If the data is correct but the exception was due to systematic causes/conditions (db,application, adapter) then re-submit after the condition was corrected using Error properties.

7.2 If the exception is data related then an application support team member needs toreview the exception and make a decision how will re-issue or compensate the transaction

Support Team

Error Handling Flows

Input queues

ExceptionDetected

Log

Exception

Check Error

RulesException Rules

& Action

7.1

7.2

Error Properties & Action: This file will be used as properties files. It contains retry logic andaction information against any errors.

Retry queue

Proposed forNYDOL

Error queue

2.2

Notific

ation

3


WSRR Overview

49


Publish: add new services that are available and can be managed

•WSDL

•XSD (business objects)

Find: search for services using any metadata associated with that service

•endpoint lookup

•version of the services

Enrich: has the ability to enhance services with useful artifacts

•service availability

•policy enforcement

•notify users of changes

WebSphere Service Registry and Repository

50


WebSphere Service Registry and Repository con’t

Manage: manage the lifecycle of services in the registry

•enabling access control

•promote/retire

•change analysis through impact analysis

Govern: provide a central point of overall governance

•WPS, ESB, developer tools (RAD, WID, RSA, WBM)

•Delete, Retrieve, Update, Manage/Govern, Create

51


WebSphere Service Registry and Repository (to be)

1. For Service Providers• Manage multiple life cycles in the various stages of development

• Development• Test• Staging• Production

• Register• Define the whole process (for external users of WSRR)

• Provide metadata• Endpoint• Service Name• Port Type• Cost• What???

• What does the service do?• What are the capabilities of the service?

2. Contracts• Private• Production• Public

enterprise service bus at dol

Documents