105/14/2007 BNL

BondFlow: A Platform for Distributed Workflows over Web

Services

Janaka BalasooriyaDepartment of Computer Science

University of Missouri – Rolla

The BondFlow System was Developed in the Department of Computer Science

Georgia State University

2

Agenda

Web Services Infrastructure and Research Issues.

Distributed Workflows Over Web Services. Web Coordination Bonds Web Service Coordination Management Middleware BondFlow System

Future Directions. The BondFlow system as a web based tool kit Incorporate WS-Security protocol in Web Coordination

Bonds Minimum set of coordination primitives for web service

coordination

Q&A.

3

Web Services – Hide Network and System Heterogeneity

Conventional Distributed Systems:

Conventional middleware and Enterprise Application Integration (EAI) technologies have limited success.

Lacks interoperability.

CORBA

RMI

COM

A C

B

Web Services:

Uniform interface (WSDL: Web Service Description Language)

Common communication platform (SOAP: Simple Object Access Protocol)

Hide the network and system heterogeneity.

Solution

SOAPSOAP A

B

CA

B

C

Inter-Operability Protocols

4

Web Services Infrastructure: Current State of the Art

Directory

Service Request

orWS

PublishLook up

Invoke/Response

(SOAP)

Web Service Infrastructure: Current State of the Art

Identified by a Uniform Resource Identifier (URI)

Described and discovered

Composed to create complex applications

Loosely coupled

Web Services Can be:

5

Proliferation of The Web Service Technology Applicable In Many Application Domains

E-Commerce (Supply Chain, Banking, Travel Application) Scientific applications (bio-medical, space exploration ) Open Grid Service Architecture (Open Grid Services Architecture).

Web services based projects will reach $11 billion by 2008, compared to $1.1 billion in 2003 [International Data Coorporation, http://www.idc.com].

A Gartner (http://www.gartner.com) survey of 110 companies also indicates that 54% are already working on or planning to work on Web services projects.

Statistics

6

Web Service Applications : Purchase Order Workflow

Receive PurchaseOrder

InitiatePrice Calculation

CompletePrice Calculation

Initiate Product Scheduling

Invoice Processing

Complete Product

Scheduling

Decide on Shipper

Arrange Logistics

PC DS PS

(IP)

(RPO)

7

Issues

Detailed level Programming: Confined only to expert users.

Centralized Coordination: Web services are not live participants.

Web Service Application Development: Current Practice and Research Issues

Web Service Based Workflow Applications

Web services are distributed across the network.

Developing Purchase Order Workflow: Current Practice

IPC

IP

DS

RPO

IPS

Composite Web Process

Web service composition.

8

Current State of the Art

Currently, too many standards for Web service composition and coordination

No fundamental Web service composition framework

Solutions are not elegant

Fundamental issues have not been filtered out

Many Workflow Composition Languages

Web services are not active participants

Composite applications require

Centralized coordination

Significant application development effort

Performance and Security issues

Centralized Coordination

9

Workflow Languages (http://www.ebpml.org/status.htm)

10

Bonding Artifacts

WS

WS

WS WS

Our Vision

BondFlow System

Core set of primitives for web service coordination - Web Coordination Bonds

Extend the Web service infrastructure beyond the basic service architecture (invoke and respond) – Web Coordination Management Middleware (WSCMM)

Workflow configuration and execution platform

11

C

Web Coordination Bonds: Analogy Between Web Bonds and Chemical Bonds

Chemical Bonds: Simple but powerful enough to produce complex compounds.H

H

H

H

C H

H

H

H

CH4

Chemical Bonding

Four H atoms and One C atom

Analogously, Web services having well-defined interfaces are “Web atoms” and “Web hooks” in the Web services interface provides required Web bonding.

Order Pickup

DeliveryBilling

Delivery

Pickup

Billing

Order

Package Pickup and Delivery Workflow Over Web Services

Web Bonds

12

Web Coordination Bonds

Web Bonds enables an application to create and enforce interdependencies, constraints and automatic updates among groups of Web entities [HICSS, 04].

Two types : Negotiation bonds and Subscription bonds Subscription Bonds

Hotel(B)

Pickup(C)

Flight(A)

Subscription Bond: Change A then Try: Change B, Change C.

Semantics

Automatic flow of information.

13

Negotiation Bond

Negotiation bonds enforce

Dependencies and constraints across entities Trigger changes based on constraint satisfaction.

Negotiation-and Bond: Change A only if B and C can be successfully changed.

Semantic:

B

C

A

Online purchase

Verify product availability

Verify payment Information

14

Modeling Dependencies Using Web Bonds

Producer-Consumer Relationship

Produce web process

Consumer web process

AcceptOrder() PlaceOrder()

Accept Delivery ( ) Dispatch Goods ( )

Negotiation bond

Negotiation bond

Subscription bond

How capable web bonds are?

15

Capabilities of Web Coordination Bonds

o Web bonds have the modeling power of extended Petri nets (Petri net with inhibitor arcs) [HICSS ‘05].

o Web bonds can express all the benchmark patterns for workflows and for inter-process communication [HICSS ‘05].

-

Expressiveness:

Modeling Power:

16

Expressiveness: Modeling Benchmark Workflow Control Flow Patterns

Sequence

AND split followed by AND join

Sequence Using Web Bonds

AND split followed by AND join using web bonds

A B C A B C

A

D

C

B

E A

D

C

B

E

17

Bonding Artifacts

WS

WS

WS WS

Our Vision and Contributions

BondFlow System

Core set of primitives for web service coordination - Web Coordination Bonds

Extend the Web service infrastructure beyond the basic service architecture (invoke and respond)– Web Coordination Management Middleware (WSCMM) Workflow configuration and execution platform

18

Analogy: Evolution of Database Applications vs Web Service Applications

(a): Web server applications

Web Servers

APPL

(b): 1-Tier Web service applications

APPL

WSDL

Web Service

WSCMS: Web Service Coordination Management SystemWSMS: Web Service Management System [ICWS ’05] (d): 3-Tier web

service applications

Web ProcessWSMS

APPL

Web Coordination

WSCMS

Web Service

Proposed development for Web service infrastructure

DBDBDB

(a): File systems (b): 1-Tier database applications

File systems

(c): 2-Tier database applications

(d): 3-Tier database applications

UIMSUIMS

APPLAPPL

(Application)

APPL

DBMS DBMS DBMS

APPL

WFMS

Evolution of database applications

19

Web Service Coordination Management Middleware: Requirements [ICWS ’06]

Functional decomposition of

composite web process [ICWS ’06]

Composite Web process

Code

High-level Description of the workflow

Enorce Workflow Dependencies

(Coodrination Context)

Communication handling(Message Correlation,

State)

Process Messages

Preserve State Information, WS Method invocation

Enforce Workflow Dependencies

WSCMS

WSMS

Stateless Web Service

IPC

IP

DS

RPO

IPS

Composite Web Process

Stateless Web Service

20

Web Service Coordination Management Middleware: An Overview

WSMS Preserve state Process messages Coordinator proxy

object

WSCMS Maintain

coordination Information

Enforce workflow dependencies

SOAP/Messaging System

WF j WF k

WSMS WSCMS

i k i kj

Coordinator object handlerCoordination

context

Message Handler

WF i

j

Web Service

21

Coordinator Proxy Object

The coordinator object provides the same interface as the web service provides to the outer world.

Web service method invocations of the workflow take place through the coordinator object.

The web bond coordination layer ensures that pre and post method invocation dependencies are satisfied.

Runtime data

CoordinationContext

WSCP object kernel

Method Calls to embedded web service.

Coordination Logic

The Coordinator Proxy Object

22

Flow Within a Coordinator Object

Each web service method call is encapsulated by pre and post dependency check.

If Pre Execution Dependencies

Enforce Pre-Execution

Dependencies

Invoke Web Service Method

If Post Execution Dependencies

Enforce Post Execution

Dependencies

No

Yes

Yes

No

23

Bonding Artifacts

WS

WS

WS WS

Our Vision and Contributions

BondFlow System

Core set of primitives for web service coordination - Web Coordination Bonds

Extend the Web service infrastructure beyond the basic service architecture (invoke and respond)– Web Coordination Management Middleware (WSCMM) Workflow configuration and execution platform

24

BondFlow: System Architecture [HCW ‘05]

Web service interface module

Proxy object generator module

Workflow configuration module

Execution module.

Handheld based Execution [MW ’04]

Web Service Interface Module

WSDL Parser

WS Locator

Proxy Object Generator

Module

Workflow Configuration

Module

Web Services Registry (UDDI)

SOAP

Lookup for Web services

WSDL

Parsed WSDL

Workflow Execution Module

SOAP/ SyD

Web Bond Runtime

JVM

25

BondFlow System : Developers Perspective

Search web service (WS)-UDDI directory

WSDL parser

Select suitable WSs for the workflow

Coordinator objects

Configure workflow

Select WS pairs for bonding

Create bonds between methods of selected WS pairs.

Execute workflow

Parsed WSDL

26

Laptop

Workflow coordination

constraints for Web service2

Workflow coordination

constraints for Web service1

Workflow coordination

constraints for Web servicen

Smart phone

Internet/Wireless networks

SOAP/PRC

SOAP/PRC

SOAP/PRC

Server

Distributed Workflow Execution of the BondFlow System

27

Book Price Workflow

BN CE

eB

Barnes and Noble Price Quote

(http://www.xmethods.net/sd/2001/BNQuoteService.wsdl )

eBay Price Watcher

(http://www.xmethods.net/sd/2001/EBayWatcherService.wsdl  )

Amazon Query Service

(http://majordojo.com/amazon_query/amazon_query.wsdl )

CurrencyExchange

(http://www.atlaz.net/webservices/GetCurrencyExchange.wsdl  )

AM

28

Purchase Order Workflow Using Web Bonds

Receive PurchaseOrder

InitiatePrice Calculation

CompletePrice Calculation

Initiate Product Scheduling

Invoice Processing

Complete Product

Scheduling

Decide on Shipper

Arrange Logistics

29

Performance Measurements

Size of WSDL (number of methods) vs. Coordinator Proxy Object Generation Time Number of

Methods WSDL Size (KB)

Proxy Creation Time(ms)

Proxy Object Footprint (KB)

1 2 20 2.2

2 2.3 23 2.3

4 8 32 2.6

5 12 40 2.5

8 16 46 3.5

17 32 76 5.7

Proxy object generation time is very small and wrapper size is less than 10 KB even for a Web service with 17 methods.

30

Workflow Execution Timings

SB= Subscription bond, NB= Negotiation bond

• Bond related execution time is only a small fraction the total workflow execution time .

Workflow Description

Check for

Bonds (ms)

Execute Negotiation bonds (ms)

Execute subscript

ion bonds (ms)

Total workflow execution time (ms)

Bond related

timings as a

percentage %

#of SB’s = 3, #of

NB’s= 2 (book price)

48 17 17 5577 1.40%

#of SB’s=4 46 0 21 6406 1.07%

31

Footprint of Workflows

Workflow Proxy Objects (KB)

Coordination Context (KB)

Footprint (KB)

Purchase order

7.10 25.40 32.50

Online book purchase

5.82 19.80 25.62

32

Broader View

Invoke/Response

Composite Web Process

WS

WS

WS

Coordination among self-coordinating Web services using Web bonds

Limitations of Web service composition (orchestration) and coordination (choreography)

For experts only

Web Services not live participants

Centralized

Subscription Bond

Negotiation Bonds WS

WS

WS

33

Future Directions

Web based toolkit for workflow configuration and deployment

Develop the BondFlow system as a web based tool. Handheld based execution of such workflows. Application of the BondFlow system in different

domains (Biological workflows, E-Commerce applications)

34

Future Directions

Incorporation WS-Security protocol Specify how to preserve the confidentiality of SOAP

messages by attaching signatures and encryption headers.

Specify how to attach security tokens (binary format) to messages.

Goal Use WS-Security specifications to incorporate

security policies to Web Coordination Bond related communication.

35

Future Directions

Extend the web coordination bond concept towards “distributed coordination theory.”

Minimum set of coordination primitives that is sufficient to model workflow dependencies.

Idea:

Group workflow dependencies into equivalent classes.

Identify coordination primitives required that covers all the equivalent classes.

36

Q&A?Q&A?

37

Related Publications

[ICWS ’06] Janaka Balasooriya, Jaimini Joshi, Sushil K. Prasad, and Shamkant Navathe, “Distributed Coordination of Workflows over Web Services and Their Handheld-based Execution,” Accepted in the 4th Intl. Conf. on Web Services (ICWS’06), Chicago, September, 2006.

[ICWS ‘05] Janaka Balasooriya and Sushil K. Prasad, “Toward Fundamental Primitives and Infrastructure Enhancements for Distributed Web Object Coordination and Workflows,” In Proc. of IEEE, 3rd Intl. Conf. on Web Services (ICWS’05), Orlando, July, 2005.

[ICWS ‘05] Sushil K.Prasad, Anu G. Bourgeois, Praveen Madiraju, Srilaxmi Malladi, and Janaka Balasooriya, “A Methodology for Engineering Collaborative Applications over Mobile Web Objects using SyD Middleware,” In Proc. of IEEE, 3rd Intl. Conf. on Web Services (ICWS’05), Orlando, July, 2005 (acceptance rate 20%)

38

Related Publications

[HCW ‘05] Janaka Balasooriya, Mohini Padye,  Sushil Prasad, and Shamkanth B. Navathe, “BondFlow: A System for Distributed Coordination of Workflows over Web Services,” In Proc. of IEEE, 14th Heterogeneous Computing Workshop in Conjunction with IPDPS 2005. Denver, Colorado, USA, April.

[HICSS’05] Sushil K. Prasad and Janaka Balasooriya, “Fundamental Capabilities of Web Coordination Bonds: Modeling Petri Nets and Expressing Workflow and Communication Patterns over Web Services,” In Proc. of IEEE, 38th Hawaii Intl, Conf. on System Sciences (HICSS-38), Big Island, Hawaii, January, 2005.

[HICSS’04] Sushil K. Prasad and Janaka Balasoorya, “Web Coordination Bonds: A Simple Enhancement to Web Services Infrastructure for Effective Collaboration,” In Proc. of IEEE, 37th Hawaii Intl. Conf. on System Sciences (HICSS-37), Big Island, Hawaii, January, 2004.

39

Related Publications

[MW ’04] S. K Prasad ... et al. 2004.  “System on Mobile Devices (SyD): A Middleware Testbed for Collaborative Applications over Small Heterogeneous Devices and Data Stores,” Procs. ACM/IFIP/USENIX 5th International Middleware Conference, Toronto, Ontario, Canada, October 18th - 22nd

40

Thank you !Thank you !