evolution of workflow technology: usages, architectures, languages

University of Stu/gart Universitätsstr. 38 70569 Stu/gart Germany

Phone +49-‐711-‐685 88470 Fax +49-‐711-‐685 88472

Prof. Dr. Frank Leymann InsJtute of Architecture of ApplicaJon Systems

[email protected]‐stu/gart.de

EvoluJon of Workflow Technology: Usages, Architectures, Languages

© Frank Leymann 2

A Caveat!

n  “Workflow”, “Business Process” or simply “Process” are used synonymous n  Although someJmes people make a subtle difference:

n  a business process is what happens in the real world, n  a workflow is the corresponding representaJon in an IT

environment

n  Same is true for “Workflow Model”, “Business Process Model”, “Process Model”

n  SomeJmes, it’s ok to speak about “workflow” if “workflow model” is meant n  …and “(business) process” vs “(business) process model” n  It most ocen clear from the context what is meant

© Frank Leymann 3

Brief History of Workflow Technology (no scien>fic rigor intended)

Document RouJng/Case Processing

People Facing

1985

1990

2000

1995

2005

2010

BPR

ProducJon WF

WF AutomaJon

TransacJonal WF

WF-‐based Apps

WF StandardizaJon Begins

Web Service ComposiJon

WF use in EAI

Graphical Modeling

WF in eScience

Office AutomaJon, Forms Processing,…

Mega Programming

Modeling/ExecuJon Convergence

Business Processes as a Service

4

Agenda Out of the Galaxy EvoluJon Architectures Languages BPM Lifecylce Summary

5


© Frank Leymann 6

Why Care About Workflow Technology?

n  Companies use computers to support their business

n  The way to do business is prescribed via a business process

n  ApplicaJons support business processes and have to ensure compliance with business processes

e Applica>on = Business Process + Business Func>ons

n  Changes in how to perform business must be reflected as soon

as possible in applicaJons

© Frank Leymann 7

But the Really Important Thing is…

From most companies’ perspecJve:

Product = Process

Thus: n  # of products produced, sold,… = # of processes executed n  Cost of producing, selling,… = Cost of process execuJon n  Time of producing, selling,… = Time of process execuJon n  …and so on

⇒ Importance of Business Process Engineering (BPR) ⇒ Importance of enforcing to follow the process model ⇒ Importance of monitoring, analyzing, mining processes

© Frank Leymann 8

Fundamental Metamodel

Collect Credit

InformaJon Assess Risk

Request Approval

Accept Credit

Reject Credit

risk = ‘low’

Layman’s View J

salary < 1000

decision = ‘no’

© Frank Leymann 9

Three Dimensions of Workflows

What?

Who?

With?

Get Order

Check Customer

Accept Order

Reject Order

Always! Bad Customer?

Good Customer?

Credit Card Information

Customer Address

Worklists

Workitem

© Frank Leymann 10

Workflow Execu>on

Workitem

Worklist

WFMS

© Frank Leymann 11

Structure of Workflow-‐Based Applica>on

Business Process Models

Business Functions

… .EXE CICS MQ EJB Web Service

+

© Frank Leymann 12

Key SWE Aspect of Workflow-‐Based Applica>ons

Application

DBMS DBMS

WFMS

Data Independence

Application Server

Flow Independence

13


© Frank Leymann 14

1st Genera>on

n  Electronic document and folder rouJng (late 80s) n  Document = image, folder,... n  RouJng through enterprise's organizaJonal structure n  User associated electronic basket is key n  PotenJal flow of documents prescribed in advance

n  In "paper factories" (administraJon, insurance, banking,...) work mainly equates to processing documents, thus the term

workflow

has been used for rouJng documents between people

The Document Routing Age

© Frank Leymann 15

2nd Genera>on

n  FuncJons performed by users in 1st generaJon WFMS are mainly retrieval, browsing, ediJng, archiving,...

n  But “cases” represented by documents were recognized to be only part of larger business processes n  Not only execuJon of document management funcJons

required but also usage of other funcJons provided by applicaJon systems supporJng the operaJon of an enterprise

n  WFMS extensions needed to invoke any kind of executable

The People-Facing Age

© Frank Leymann 16

2nd Genera>on (cont.)

n  Launching executables requires parameter passing n  Thus, data flow features complemented available control flows n  In turn, control flows can now be expressed in terms of these

new parameters ("business rules") n  Data flow is used for integraJng applicaJons with long

temporal delays between their iniJaJons n  Parameters managed by data flow must be persistent

© Frank Leymann 17

3rd Genera>on

n  Being able to support large spectrum of business processes in compuJng environments made WFMS of strong interest for Business Process Reengineering (BPR) projects -‐ early 90s

n  Goal of BPR is to speedup business processes and reduce their costs. ResulJng requirements: n  Parallelism within workflows (à speedup) n  Deadline processing (à speedup) n  Monitor actual workflow status (à speedup) n  AudiJng of significant events, i.e. processing history

(à cost reducJon, compliance) n  Maintain execuJon history for analysis

(à cost reducJon, compliance) n  Process acJviJes without human intervenJon

(à speedup + cost reducJon)

The Dawn Of BPR

© Frank Leymann 18

4th Genera>on

n  Workflow-‐based applicaJons become state-‐of-‐the-‐art (mid 90s) n  Strict separaJon of business process logic and business funcJons

n  Business processes implemented via workflow system n  Business funcJons implemented "tradiJonally" (TP-‐monitor, applicaJon

server,...) n  Enterprises become dependent on WFMS

n  Similar to TP-‐Monitors and DBMS before n  The term produc>on workflow has been coined to indicate that WFMS

is driving operaJonal aspects of an enterprise (i.e. the “producJon line” of an enterprise)

n  Consequences: n  WFMS had to provide quality of services known before from

"producJon systems" like DBMS and TPM n  High/conJnuous availability n  Scalability n  Robustness

The Production Age

© Frank Leymann 19

5th Genera>on

n  ApplicaJon integraJon becomes very important (end of 90s)

n  OrganizaJonal integraJon becomes more and more important n  Interoperability of WFMS (building blocks), as well as Web

access required n  Workflows understood as business oriented "logical units of work" n  Advanced transacJon management funcJons required n  Forward recovery of workflows as well as workflow-‐based

applicaJons n  Backward recovery (spheres of atomicity and

compensaJon)

The Automation Age

© Frank Leymann 20

6th Genera>on

n  Web Services appeared about 2002 n  Workflows needed to make use of Web Services

n  Interact with an ESB n  Exploit human beings as Web services

n  Workflows have been considered early on as technology to aggregate services into new services n  “Recursive aggregaJon model for Web services” n  “OrchestraJon”

n  BPEL has been created to support all of this

The Service Age

© Frank Leymann 21

7th Genera>on

n  Modeling of business processes gained a lot of a/enJon about 2005 n  Domain experts became the immediate source of models

n  BPMN became predominant modeling language n  Transforming models created by domain experts into something executable must not loose semanJcs of the original process

n  ExecuJng BPMN evolved as a requirement n  …and BPMN 2.0 had been developed to support this

The Model-Execution Age

22


© Frank Leymann 23

Major Building Blocks of a WFMS

Modeling Tool (BPR, Workflows)

Workflow Management System

Buildtime

Runtime

Workflow Database

Process Model

Applications, Tools

© Frank Leymann 24

...And Their Correspondence In DBMS

Modeling Tool (BPR, Workflows)

Workflow Management System

Buildtime

Runtime

Workflow Database

Process Model

Applications, Tools

Data Modeling Tool Relational Algebra (Metamodel)

SQL DDL

Catalog

User Data (tables, indexes,…)

DBMS

SQL DML

StP, UDF,…

© Frank Leymann 25

The WfMC Reference Model

Business Modeling

Workflow Enactment Engine

AdministraJon, AudiJng

Workflow Enactment Engine

Worklist ApplicaJon

Worklist Processor

ApplicaJon InvocaJon

ApplicaJon

© Frank Leymann 26

Modern Terminology

Process Modeling Tool

Process (ExecuJon) Engine

Management Tool

Process Engine 2

Worklist ApplicaJon

Workitem Management

ApplicaJon InvocaJon

ApplicaJon FuncJon

© Frank Leymann 27

Invoking Ac>vity Implementa>ons

Worklist

Workitem ...

...

Activity

Pgm

Input Container Output Container

2

3 5

4

1

© Frank Leymann 28

Internal Component Flow

n  WFMS Navigator determines program to be executed n  "ExecuJon Messages" send to launching component called

Program ExecuJon Server (PES) n  "CompleJon Message" send back to Navigator when

invoked program returns

Navigator

MQM PES

<OS Address Space>

Pgm

Invocation State & Metadata

© Frank Leymann 29

Program Execu>on Server

n  WFMS won't be able to support all mechanisms to launch executables n  Thus, users should be able to build their own PESes, e.g. WFMS…

n  …provides interfaces between PES building blocks n  ….defines required messages exchanged between navigator and PES

(User-‐provided PES (UPES)) n  Specific metadata are needed by PES, e.g. security informaJon, mapping

prescripJons etc.

PES Info

Extract Data Mapping

Service Locator

Program Invocation

Invocation State & Metadata

Program Data

Format Pgm Operational

Data

PES

J Yes, this is core ESB funcJonality!!!

© Frank Leymann 30

Architectural Impact


Orchestrator Management Tool Orchestrator 2

Worklist ApplicaJon

Workitem Management Service Bus

Service

© Frank Leymann 31

WFMS

Workitem Manager

Navigator

WFMS Task Manager (aka Task Processor

aka Human Task Infrastructure)

TradiJonal Architecture Modern Architecture

Navigator

Architectural Impact: High-‐Level View

n  Work request no longer only created by WFMS n  Thus, Workitem Manager is split from WFMS and becomes a separate component in SOA environment (“Task Manager”)

© Frank Leymann 32



Process Engine

Management Tool

Process Engine 2

Task List Client

Task Manager Service Bus

Service

33


© Frank Leymann 34

How People are Used to Model

Logical Process Model (BPEL)

Conceptual Process Model (BPMN)

Process Engine

Transform

Deploy

Logical Data Model (e.g. SQL)

Conceptual Data Model (e.g. Entity-Relationship)

DBMS

Transform

Deploy

Process Management Stack Data Management Stack

© Frank Leymann 35



Process Engine

Management Tool

ExecuJon Engine 2

Task List Client

Task Manager Service Bus

Service

Conceptual Process Model

Logical Process Model

© Frank Leymann 36

Workflow Language Genealogy

FDL

FDML XPDL

WSFL

BPEL

XLANG

BPML WSCI BPSS

WS-CDL

(IBM)

(IBM)

(IBM)

(Microsoft)

(WfMC) (bpmi.org) (SUN, SAP, Oracle)

(OASIS ebXML)

(W3C)

(IBM, Microsoft, BEA à OASIS)

BPELJ

YAWL GSFL …

(Research Groups)

?…? BPEL4People BPEL-SPE

GPEL

BPMN 1.x (OMG)

Pi-Calculus++ (Microsoft)

BPEL4Chor

✜-Calculus

PM-Graphs

Petri-Nets BPMN 2.0 (OMG)

© Frank Leymann 37

Other Languages

n  Data Flow oriented approaches n  Especially used in scienJfic workflow community n  Only data dependencies between acJviJes are modeled

n  As soon as all input data of an acJvity is available, start the acJvity n  Examples: DAGMan, Kepler, Taverna, Triana

n  Research prototypes

n  DeclaraJve approaches n  Define constraints of valid execuJon “sequences”

instead of specifying all poten/al execuJon “sequences” n  Based on some temporal logic (e.g. LTL) n  Example: DECLARE

n  Research prototype

n  Rule-‐based approaches n  Mainly based on ECA rules n  Example: Lotus Notes workflow

ObservaJon: Products focus on

control flow languages

© Frank Leymann 38

Flow Defini>on Language (FDL): Example

PROCESS 'Loan Process'PROGRAM_ACTIVITY 'Collect Credit Information'

PROGRAM 'Credit Information Program'DONE_BY 'Loan Officer'

END 'Collect Credit Information'

PROGRAM_ACTIVITY 'Assess Risk'PROGRAM 'Risk Program'DONE_BY 'Financial Officer'

END 'Assess Risk'

CONTROLFROM 'Collect Credit Information'TO 'Assess Risk'

END 'Loan Process'

PROGRAM 'Credit Information Program'WINNT

EXEPATH_AND_FILENAME 'CIP.EXE'

END 'Credit Information Program'

PROGRAM 'Risk Program'AIX

EXEPATH_AND_FILENAME 'RP.EXE'

END 'Risk Program'

ROLE 'Loan Officer'RELATED_PERSON 'Leymann'

END 'Loan Officer'

ROLE 'Financial Officer'RELATED_PERSON 'Roller'

END 'Financial Officer'

PERSON 'Leymann'FIRST_NAME 'Frank'LAST_NAME 'Leymann'

END 'Leymann

PERSON 'Roller'FIRST_NAME 'Dieter'LAST_NAME 'Roller'

END 'Roller'

Process Logic ("What")

Process Logistics ("Who")

IT Infractructure ("With")

The “mother” of all workflow languages

© Frank Leymann 39

BPEL: Business Processes Between Web Services

X

Y

Z

pt3

o1

o2

pt1

o1

o3

o4

pt2

o2

o4

© Frank Leymann 40

BPEL: Business Processes As Web Services

X

Y

Z

pt

o1

o2

pt* o7

o8

o9

© Frank Leymann 41

BPEL: Aggrega>ng Web Services

X

Y

Z pt

o1

o2

pt3

o1

o2

pt1

o1

o3

o4

pt2

o2

o4

P

Q

pt* o8

o9

Precisely: BPEL provides a recursive aggregation

model for Web services

© Frank Leymann 42

Separa>ng Processes And Port Types

A

C

B

D

E

Process

Deployment Descriptors

Activity: Partner Role Port Type Operation ...

Application Port Types

© Frank Leymann 43

Run>me: Use of Deployment Informa>on

A

C

B

D

E

Process

Service Bu

s

Application

Deployment Descriptors

Activity: Port Type Operation Qos ... Policy

Ports

EPR

© Frank Leymann 44

Programming Model

Flow/Process/…

Components/Services/…

Application

Workflow System

Application Server

Programming in the Large

Programming in the Small

Deployment Service

Bus

Deployment Descriptor pT pLink Locator

© Frank Leymann 45

BPEL: Example (The Obvious One J)

TravelService

orderTrip receiveItinarary

getHotel getFlight

chargeCC

HotelService

bookRoom

AirlineService

bookFlight

CCService

payBill

Hotel

Airline

CCCompany

TravelAgent overnight?

© Frank Leymann 46

The Process Header And Partners

<process name=“TripBookingProcess" targetNamespace="http://fl.com/Travel/Booking" xmlns="http://schemas.xmlsoap.org/ws/2003/05/business-process" xmlns:tag="http://travel.org/wsdl/travelAgent"> <partnerLinks> <partnerLink name="Customer" partnerLinkType="tag:Traveler" myRole="TravelAgent"/> <partnerLink name="Hotel" partnerLinkType="tag:Hotel" partnerRole="HotelChain"/> <partnerLink name="Airline" partnerLinkType="tag:Airline" partnerRole="AirlineCompany"/> <partnerLink name="Billing" partnerLinkType="tag:CreditCard" partnerRole="CreditCardCompany"/> </partnerLinks>

© Frank Leymann 47

The Variables

<variables> <variable name="Order" messageType=“tag:trip"/> <variable name="Hotel" messageType="tag:hotelBooking"/> <variable name="Flight" messageType="tag:flightBooking"/> <variable name="payment" messageType="tag:payments"/> </variables>

© Frank Leymann 48

The Ac>vi>es (1/5)

<flow> <links> <link name="Itinerary-to-Hotel"/> <link name="Itinerary-to-Flight"/> <link name="Hotel-to-Charge"/> <link name="Flight-to-Charge"/> </links> <receive name=“receiveItinerary” partnerLink="Customer" portType="tag:TravelService" operation="orderTrip" variable="Order"> <source linkName="Itinerary-to-Hotel“ transitionCondition=“overnight=‘true’"/> <source linkName="Itinerary-to-Flight"/> </receive> ....

receiveItinarary

getHotel getFlight

chargeCC

overnight?

© Frank Leymann 49

The Ac>vi>es (2/5)

.... <assign> <copy> <from variable="Order" part="itinerary" query="/rooms"/> <to variable="Hotel"/> </copy> </assign> <assign> <copy> <from variable="Order" part="itinerary" query="/flights"/> <to variable="Flight"/> </copy> </assign> ....

© Frank Leymann 50

The Ac>vi>es (3/5)

... <invoke name=“getHotel” partnerLink="Hotel" portType="tag:HotelService" operation="bookRoom" inputVariable="Hotel"> <target linkName="Itinerary-to-Hotel"/> <source linkName="Hotel-to-Charge"/> </invoke> <invoke name=“getFlight” partnerLink="Airline" portType="tag:AirlineService" operation="bookFlight" inputVariable="Flight"> <target linkName="Itinerary-to-Flight"/> <source linkName="Flight-to-Charge"/> </invoke> ...

receiveItinarary

getHotel getFlight

chargeCC

overnight?

© Frank Leymann 51

The Ac>vi>es (4/5)

.... <assign> <copy> <from variable="Order" part="customerInfo"/> <to variable="Payment" part="customerInfo"/> </copy> <copy> <from variable="Hotel"/> <to variable="Payment" part="orders" query="/rooms"/> </copy> <copy> <from variable="Flight"/> <to variable="Payment" part="orders" query="/flights"/> </copy> </assign> ....

© Frank Leymann 52

The Ac>vi>es (5/5)

... <invoke name=“chargeCC” partnerLink="Billing" portType="tag:CreditCardService" operation="payBill" inputVariable="Payment“ joinCondition= "vprop:getLinkStatus(‘Hotel-to-Charge') or vprop:getLinkStatus(‘Flight-to-Charge')"> <target linkName="Hotel-to-Charge"/> <target linkName="Flight-to-Charge"/> </invoke> </flow> </process>

receiveItinarary

getHotel getFlight

chargeCC

overnight?

61


© Frank Leymann 63

BPMN

BPM Lifecycle: Modeling

n  Today, typically based on BPMN (2.0)

n  Done by domain experts

n  StaJc analysis used to “quickly” check obvious modeling errors

n  AnalyJcal simulaJon used to check saJsfacJon of KPIs

Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

© Frank Leymann 64

BPEL

BPM Lifecycle: IT Refinement

n  Today, ocen generaJon of BPEL from BPMN

n  Web services have to be determined that perform the tasks of the processes

n  Done by IT experts

Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

© Frank Leymann 65

BPMN 2.0

BPM Lifecycle: IT Refinement 2.0

n  If a BPMN 2.0 engine is used, no transformaJon is needed

n  As before, Web services have to be determined that perform the tasks of the processes


Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

© Frank Leymann 66

BPEL BPMN 2.0

BPM Lifecycle: Deployment

n  The Web services of the refined process model must be bound to endpoints n  Well, task can have

different implementaJons (like scripts, EJBs, REST APIs,…)

n  They must be bound too


Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

© Frank Leymann 67

BPEL BPMN 2.0

BPM Lifecycle: Execu>on

n  Either a BPEL engine executes the transformed BPMN model,

or n  …a BPMN 2.0 engine

executes the BPMN 2.0 model

n  In any case the bound task implementaJons are invoked during navigaJng the process models

Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

© Frank Leymann 68

BPM Lifecycle: Monitoring

n  During execuJon the actual status of each running process is available

n  Individual processes can be inspected and depicted

n  CollecJons of processes can be defined, and their aggregated status can be depicted

n  The status of (aggregaJons of) processes can be enriched with other informaJon to enable deeper inside

Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

© Frank Leymann 69

BPM Lifecycle: Analysis

n  The history of already completed processes is maintained

n  This allows post mortem analysis of (collecJons of) processes

n  Used to improve models of processes n  …to be/er saJsfy KPIs

n  Can be combined with data analysis to derive be/er models

n  Mining algorithms can be used n  …to improve control flow n  …to improve use of staff n  …

Modeling

IT Refinement

Deployment

ExecuJon

Monitoring

Analysis

KPIs

70


© Frank Leymann 71

Conclusion

n  Process knowledge is a key resource of a company n  Workflow technology is managing this knowledge n  The technology is mature and proven (≈25 years of evoluJon in pracJce)

n  WFMS architectures are compliant with a service-‐oriented environment

n  Two languages have been established in pracJce n  BPMN (2.0) for modeling n  BPEL for execuJon

n  Processes are managed in a lifecycle that typically use specialized technologies in an integrated manner

72

The End