nominal calculi for transactions: zsn in join ii

78
1 Nominal Calculi for Transactions: ZSN in JOIN II Roberto Bruni Dipartimento di Informatica Università di Pisa Models and Languages for Coordination and Orchestration IMT- Institutions Markets Technologies - Alti Studi Lucca

Upload: metta

Post on 11-Jan-2016

29 views

Category:

Documents


0 download

DESCRIPTION

Models and Languages for Coordination and Orchestration IMT- Institutions Markets Technologies - Alti Studi Lucca. Nominal Calculi for Transactions: ZSN in JOIN II. Roberto Bruni Dipartimento di Informatica Università di Pisa. Distributed 2PC. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Nominal Calculi for Transactions:  ZSN in JOIN II

1

Nominal Calculi for Transactions: ZSN in JOIN II

Roberto BruniDipartimento di Informatica Università di Pisa

Models and Languages for Coordination and Orchestration

IMT- Institutions Markets Technologies - Alti Studi Lucca

Page 2: Nominal Calculi for Transactions:  ZSN in JOIN II

2

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Distributed 2PC The distributed 2PC is a variant of the

decentralized 2PC with a finite but unknown number of participants When a participant P is ready to commit it

has only a partial knowledge of the whole set of participants

Only those who directly cooperated with P To commit P must contact all its

neighbours and possibly learn the identity of other participants from them

Page 3: Nominal Calculi for Transactions:  ZSN in JOIN II

3

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Commit in Distributed DataBases

Data can be inherently distributed e.g. customers accounts in different branches of

the same bank Data are distributed to achieve failure

independence e.g. replicated file systems

Partial failures can lead to inconsistent results Commits have to be coordinated among

participants to preserve data consistency

Page 4: Nominal Calculi for Transactions:  ZSN in JOIN II

4

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Distributed DataBases

DBuser

user

useruser

user

Centralized Distributed

Page 5: Nominal Calculi for Transactions:  ZSN in JOIN II

5

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Atomic Commitment Problem

Reach a globally consistent state despite failures Each participant has two possible decision values

commit All participants will make the transaction’s updates permanent

abort All will roll-back

Individual decisions are irreversible A commit decision requires unanimity of YES votes

Page 6: Nominal Calculi for Transactions:  ZSN in JOIN II

6

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Atomic Commitment Properties

Consensus All participants that decide reach the same

decision If any participant decides commit, then

all participants must have voted YES If all participants have voted YES and no

failures occur, then commit is decided Irreversibility

Each participant decides at most once

Page 7: Nominal Calculi for Transactions:  ZSN in JOIN II

7

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Commitment Protocols Atomic commitment protocol

satisfies all atomic commitment properties ensures that transactions terminate consistently

at all participating sites of a distributed database, even in presence of failures

Non-blocking if it permits transaction termination to proceed at

correct participants despite failures of others is the activity of ensuring that Sw and Hw failures do

not corrupt persistent data can limit time intervals of resource locking

Page 8: Nominal Calculi for Transactions:  ZSN in JOIN II

8

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Some Assumptions One of the participants acts as unique coordinator

(centralized version) At most one (if no failures, then there is one coordinator) A participant assumes the role of coordinator within a

fixed time interval from the beginning of the transaction The transaction begins at a single participant

called the invoker (not necessarily the coordinator) sends start messages to other participants

Only undeliverable messages are dropped All participants can communicate (useful later)

Page 9: Nominal Calculi for Transactions:  ZSN in JOIN II

9

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Generic ACP: Coordinator send VOTE-REQ[Tid] to all participants set-timeout wait-for vote[Tid] from all participants

if (all votes are YES) then broadcast (commit[Tid], participants)

else // at least one vote is NO broadcast (abort[Tid], participants)

on-timeout: // escape blocking wait-for broadcast (abort[Tid], participants)

Phase 1

Phase 2

Page 10: Nominal Calculi for Transactions:  ZSN in JOIN II

10

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Generic ACP: Participants set-timeout wait-for VOTE-REQ[Tid] from coordinator // 1

send vote[Tid] to coordinator if (vote==NO) then // unilateral abort

decide abort else

set-timeout wait-for decision from coordinator // 2

if (decision==abort) then decide abort else decide commit

on-timeout: termination-protocol // escape 2 on-timeout: decide abort //escape 1

Page 11: Nominal Calculi for Transactions:  ZSN in JOIN II

11

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Simple Broadcast broadcast(m,S) // Broadcaster

send m to all processes in S deliver m

// other processes in S upon-receipt m // non-blocking

deliver m

This corresponds to the 2PC Protocol

Page 12: Nominal Calculi for Transactions:  ZSN in JOIN II

12

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Timeout Actions Participants must wait

VOTE_REQ from coordinator If this takes too long can just decide abort

Coordinator collects votes No global decision is yet made Coordinator can decide abort

commit / abort from coordinator The participants already took a decision (YES) It is now uncertain It must consult other participants according to the

termination protocol

Page 13: Nominal Calculi for Transactions:  ZSN in JOIN II

13

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Termination Protocol (TP) What if a participant that voted YES times out

waiting for the response from coordinator? It invokes a termination protocol to contact:

the coordinator other participants (cooperative TP)

can have already voted or not yet voted There are failure scenarios for which no termination

protocol can lead to a decision Blocking scenario: correct participants cannot decide e.g. coordinator crashes during broadcast

all faulty participants deliver and crash all correct participants do not deliver the decision if faulty participants do not recover any decision could

contradict the decision of a participant that crashed

Page 14: Nominal Calculi for Transactions:  ZSN in JOIN II

14

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Non-Blocking ACP I set-timeout wait-for VOTE-REQ[Tid] from coordinator // 1

send vote[Tid] to coordinator if (vote==NO) then // unilateral abort

decide abort else

set-timeout wait-for decision from coordinator // 2

if (decision==abort) then decide abort else decide commit

on-timeout: decide abort // escape 2 on-timeout: decide abort //escape 1

skipped for lack of time

Page 15: Nominal Calculi for Transactions:  ZSN in JOIN II

15

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Non-Blocking ACP II broadcast(m,S) // Broadcaster as before // other processes in S

upon-first-receipt m send m to all processes in S // S can be sent along

VOTE_REQ deliver m

any process receiving m relays m to all others (if any correct process receives m, all correct process receive m, even if broadcaster crashes)

m is delivered only after relaying

skipped for lack of time

Page 16: Nominal Calculi for Transactions:  ZSN in JOIN II

16

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Recovery Participant p is recovering from a failure

Must reach a consistent decision Suppose p remembers its state at the time it

failed Before voting

it can unilaterally abort After deciding abort

it can unilaterally abort After receiving commit / abort from coordinator

it had already decided and must behave accordingly During the uncertainty period (voted YES)

Independent recovery is not possible! Termination protocol is needed

skipped for lack of time

Page 17: Nominal Calculi for Transactions:  ZSN in JOIN II

17

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Distributed Transaction Log

DTL is kept in stable storage at each site Its content must survive failures

Coordinators and participants at that site can record information about transactions Before/after sending VOTE_REQ, the coordinator C writes

start2PC(S,Tid) Before voting YES, a participant writes yes(C,S,Tid) Before/after voting NO, a participant writes abort(Tid) Before C sends commit, it writes commit(Tid) Before/after C sends abort, it writes abort(Tid) After receiving the decision, participant writes

commit/abortskipped for lack of time

Page 18: Nominal Calculi for Transactions:  ZSN in JOIN II

18

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Recovery From DTL If DTL contains start2PC (the site hosted the

coordinator) If it also contains commit/abort

The coordinator decided before failure Otherwise

The coordinator can decide abort (and record it in DTL) Otherwise

It contains commit/abort The participant has reached decision before the failure

Does not contain yes Either failed before voting or voted no The participant can unilaterally abort

Otherwise (it contains yes but not commit/abort) The participant failed in its uncertainty period Must use the termination protocol

skipped for lack of time

Page 19: Nominal Calculi for Transactions:  ZSN in JOIN II

19

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Cooperative TP: Initiator send DECISION_REQ[Tid] to all processes in S wait-for decision[Tid] from any process

if (decision==commit) then write commit in DTL

else // decision==abort write abort in DTL

skipped for lack of time

Page 20: Nominal Calculi for Transactions:  ZSN in JOIN II

20

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Cooperative TP: Responder

wait-for decision[Tid] from any process p if (abort(Tid) in DTL) then

send abort to p else if (commit(Tid) in DTL) then

send commit to p

skipped for lack of time

Page 21: Nominal Calculi for Transactions:  ZSN in JOIN II

21

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Evaluation of 2PC Criteria: Reliability vs Efficiency

Resiliency What failures can be tolerated?

Blocking Can processes be blocked? Under which conditions?

Time Complexity How long does it take to reach a decision?

Message Complexity How many messages are exchanged to reach a

decision? What are their dimensions?

Page 22: Nominal Calculi for Transactions:  ZSN in JOIN II

22

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Balancing Reliability and Efficiency are conflicting goals

each can be achieved at the expenses of the other The choice of protocol depends on which goal

is more important for a specific application Whatever protocol is chosen, we should

optimize for the case with no failures Hopefully the normal operating state of the system

Page 23: Nominal Calculi for Transactions:  ZSN in JOIN II

23

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Measuring Time Complexity

A round is the max time for a message to reach its destination

Timeouts are based on the assumption that such a delay is known Note that many messages can be sent in a single round

Two messages must belong to different rounds iff one cannot be sent before the other is received

Rounds are taken as time units We count the number of rounds needed for unblocked sites to reach

a decision, in the worst case This neglects the time needed to process messages

Reasonable: messages delays usually exceed processing delays Other two factors can be relevant:

DTL management (on stable storage) Broadcasting preparation (to a large number of processes)

Page 24: Nominal Calculi for Transactions:  ZSN in JOIN II

24

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Measuring Message Complexity

Total number of messages sent during the whole protocol Reasonable measure if individual

messages are not very large Otherwise we should measure the length

of messages, not merely their number Here messages are short, so we abstract

away from their lengths

Page 25: Nominal Calculi for Transactions:  ZSN in JOIN II

25

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Reliability of 2PC Resiliency

2PC is resilient to site failures communication failures

In fact, the cause of timeouts is not important

Blocking 2PC is subject to blocking

Probabilistic analysis can be performed depending on the probabilistic distribution of failures

Page 26: Nominal Calculi for Transactions:  ZSN in JOIN II

26

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Time Complexity of 2PC In absence of failure, 2PC requires 3 rounds

Broadcast VOTE-REQ Collect votes Broadcast global decision

If failures happen, The TP may need 2 additional rounds Broadcast DECISION_REQ Reply from a process outside its uncertainty period

Note that several TPs can be initiated separately in the same round

Up to 5 rounds, independently from the number of failures! But processes may remain blocked for an unbounded period of time

Page 27: Nominal Calculi for Transactions:  ZSN in JOIN II

27

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Message Complexity of 2PC

Let N+1 be the number of participants, including the coordinator

In each round of 2PC, there are N messages sent Hence, in absence of failures 2PC uses 3N messages Cooperative TP is invoked by all participants that

voted YES but did not receive commit / abort Let there be M such participants M initiators, each sending N DECISION_REQ (MN messages) At most N-M+1 processes will respond to the first request

In the worst case only one process abandons its uncertainty and will respond to another initiator: (N-M+1)+(N-M+2)+…+N

Page 28: Nominal Calculi for Transactions:  ZSN in JOIN II

28

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Calculating the Message Complexity of 2PC

In the worst case the total number of TP messages will be:

NM + i=1 (N-M+i) = NM + NM – M2 + M(M+1)/2 = 2NM – M2/2 + M/2 messages This quantity is maximum when M=N

N(3N+1)/2 messages The 2PC together with worst-case TP amount to

3N + N(3N+1)/2 = N(3N+7)/2 messages

M

Page 29: Nominal Calculi for Transactions:  ZSN in JOIN II

29

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Communication Topology The communication topology of a protocol

is the specification of who sends messages to whom e.g. in 2PC without TP, the coordinator sends

messages to participants and vice versa Participants do not send messages directly to each

other The topology is described as a tree of height 1

Coordinator

Participant Participant Participant Participant…

Page 30: Nominal Calculi for Transactions:  ZSN in JOIN II

30

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Alternative 2PCs To reduce time and message complexity of

centralized 2PC, two variations have been proposed, based on different communication topologies Decentralized 2PC

Communication topology is a complete graph Improve time complexity

Linear 2PC (aka Nested 2PC) Linearly ordered processes Reduce the number of messages

Page 31: Nominal Calculi for Transactions:  ZSN in JOIN II

31

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Decentralized 2PC Depending on its own vote, the coordinator sends YES or

NO to all participants Informs that it is time to vote Tells the coordinator’s vote

If the message is NO Each participant decides abort and stops

Otherwise, each participant sends back its vote to ALL OTHER PARTICIPANTS

After receiving all votes each process can decide autonomously

If all are YES and its own vote is YES, decide commit Otherwise it decides abort

Timeouts can be employed as in the centralized 2PC

Page 32: Nominal Calculi for Transactions:  ZSN in JOIN II

32

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Evaluation of Decentralized 2PC

In the absence of failures, only 2 rounds are necessary Coordinator voting YES / NO Each participant voting YES / NO

More messages are needed: N2+N messages N messages in the first round N2 messages in the second round (and this is just in absence of failures)

Page 33: Nominal Calculi for Transactions:  ZSN in JOIN II

33

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Linear 2PC Each participant can communicate only with its left

/ right neighbors The coordinator is the leftmost process

It sends its vote YES / NO to its right neighbor This message has a dual meaning as in decentralized 2PC

Each participant p waits for the vote from its left neighbor If it is YES, and p votes YES, then p tells YES to its right

neighbor Otherwise, p tells NO to its right neighbor

When the rightmost participant receives the vote, it makes the final decision commit / abort

The decision is propagated from right to left When the coordinator receives it, the protocol ends

Timeout periods are influenced by positions

Page 34: Nominal Calculi for Transactions:  ZSN in JOIN II

34

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Evaluation of Linear 2PC

Only 2N messages needed N votes from left to right N decisions from right to left (and this is just in absence of failures)

Unfortunately the same amount of rounds is needed: 2N rounds No two messages are sent concurrently

Page 35: Nominal Calculi for Transactions:  ZSN in JOIN II

35

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Comparison of 2PC Variants

Rounds

Messages

Centralized 2PC 3 3N

Decentralized 2PC

2 N2+N

Linear 2PC 2N 2N Hybrid communication topologies are also possible e.g. Linear for voting, complete for conveying decision

2N messages, N+1 rounds

The choice of the protocol might be influenced by the available communication topology

Page 36: Nominal Calculi for Transactions:  ZSN in JOIN II

36

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

From 2PC to 3PC In 2PC, if all operational participants are

uncertain, they are blocked They cannot decide abort even if aware that

processes they cannot communicate with have failed, because some of them could have decided commit before failure

The 3CP is an ACP designed to rule out this situation It guarantees that if any operational process is uncertain, then

no (operational / failed) process can have decided commit Thus, if p realizes that any operational site is uncertain, then p

can decide abort Why does 2PC violate this property?

A participant p can receive commit while q is still uncertain

skipped for lack of time

Page 37: Nominal Calculi for Transactions:  ZSN in JOIN II

37

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Sketch of 3PC: The Idea After the coordinator has found that all votes were

YES, it sends pre-commit messages to all participants When a participant p receives pre-commit, it knows

that all participants voted YES p is no longer uncertain, but does not decide commit yet p knows that it will decide commit unless it fails p acknowledges the receipt of pre-commit

When the coordinator collects all acks it knows that no participant is uncertain

The coordinator sends commit to all participants When a participant receives commit, it decides commit If a participant voted NO, then 3PC behaves as 2PC

skipped for lack of time

Page 38: Nominal Calculi for Transactions:  ZSN in JOIN II

38

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Sketch of 3PC: Some Notes

In absence of failures, 3PC involves 5 rounds and up to 5N messages

Participants have four possible states Aborted, Uncertain, Committable, Committed For p and q any two participants, only certain combinations of

their states are possible Timeouts can occur in five situations

3 are trivially handled 2 require a complex termination protocol

Election protocol (for a new coordinator) based on a linear ordering of participants

The new coordinator checks the states of all operational participants

Timeouts are again necessary

skipped for lack of time

Page 39: Nominal Calculi for Transactions:  ZSN in JOIN II

39

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Some References Concurrency control and recovery in

database systems (Addison-Wesley 1987) P. Bernstein, N. Goodman, V. Hadzilacos

Transaction processing: concepts and techniques (Morgan Kaufmann 1993) J. Gray, A. Reuter

Sagas (Proc. SIGMod’87, ACM, pp. 249-259) H. Garcia-Molina, K. Salem

Non-blocking atomic commitment (Chapter 6 of Distributed Systems, Addison-Wesley 1995)

O. Babaoglu, S. Toueg

Page 40: Nominal Calculi for Transactions:  ZSN in JOIN II

40

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

D2PC Every participant P acts as coordinator

During the transaction P builds its own synchronization set LP of cooperating agents

When P is ready to commit, P asks readiness to processes in LP (if empty P was isolated and can commit)

In doing so, P sends them the set LP

Other participants will send to P either a successful reply with their own synchronization sets or a failure message

(in this case, failure is then propagated) Successful replies are added to LP

The protocol terminates when LP is transitively closed

Page 41: Nominal Calculi for Transactions:  ZSN in JOIN II

41

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P3}

{P2}

Example: D2PC

{P2}

Page 42: Nominal Calculi for Transactions:  ZSN in JOIN II

42

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P3}

Example: P3 Enters ACP Phase

{P2}

{P2} [] ()

known parties

contacted parties

parties who replied

Page 43: Nominal Calculi for Transactions:  ZSN in JOIN II

43

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P3}

{P2} [P2] ()

<P3,{P2}>

Example: P3 Contacts Known Parties

{P2}

Hi, I am P3.I am ready to commit.I know P2.

Page 44: Nominal Calculi for Transactions:  ZSN in JOIN II

44

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P3} [] ()

<P3,{P2}>

{P2} [P2] ()

Example: P2 Enters ACP Phase

{P2}

Page 45: Nominal Calculi for Transactions:  ZSN in JOIN II

45

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P3} [P1,P3] ()

<P3,{P2}>

{P2} [P2] ()

<P2,{P1,P3}><P2,{P1,P3}>

Example: P2 Contacts Known Parties

{P2}

Hi, I am P2.I am ready to commit.I know P1 and P3.

Page 46: Nominal Calculi for Transactions:  ZSN in JOIN II

46

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

<P3,{P2}>

{P2} [P2] ()

<P2,{P1,P3}>

<P2,{P1,P3}>

{P1,P3} [P1,P3] ()

Example: Some Pending Messages Around

{P2}

Page 47: Nominal Calculi for Transactions:  ZSN in JOIN II

47

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P2} [P2] ()

<P2,{P1,P3}>

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P3)

Example: P2 Reads a Pending Vote

{P2}

Page 48: Nominal Calculi for Transactions:  ZSN in JOIN II

48

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P2} [P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P3)

Example: P3 Reads a Pending Vote

{P2}

Page 49: Nominal Calculi for Transactions:  ZSN in JOIN II

49

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P2} [P1,P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P3)

<P3,{P1,P2}>

Example: P3 Contacts the Newly Known Party

{P2}

Page 50: Nominal Calculi for Transactions:  ZSN in JOIN II

50

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P1,P2} [P1,P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P3)

<P3,{P1,P2}>

Example: P1 Enters ACP Phase

{P2} [] ()

Page 51: Nominal Calculi for Transactions:  ZSN in JOIN II

51

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P2} [P2] ()

{P1,P2} [P1,P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P3)

<P3,{P1,P2}>

<P1,{P2}>

Example: P1 Contacts Known Parties

Page 52: Nominal Calculi for Transactions:  ZSN in JOIN II

52

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P2} [P2] ()

{P1,P2} [P1,P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P3)

<P3,{P1,P2}>

<P1,{P2}>

Example: Some Pending Messages Around

Page 53: Nominal Calculi for Transactions:  ZSN in JOIN II

53

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

P2

P3

{P2} [P2] ()

{P1,P2} [P1,P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P1,P3)

<P3,{P1,P2}>

Example: P2 Reads a Pending Vote

Page 54: Nominal Calculi for Transactions:  ZSN in JOIN II

54

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

Q2

P3

{P2} [P2] ()

{P1,P2} [P1,P2] (P2)

<P2,{P1,P3}>

{P1,P3} [P1,P3] (P1,P3)

<P3,{P1,P2}>

Example: P2 can Commit!

Page 55: Nominal Calculi for Transactions:  ZSN in JOIN II

55

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

Q2

P3

{P2,P3} [P2] (P2)

{P1,P2} [P1,P2] (P2)

{P1,P3} [P1,P3] (P1,P3)

<P3,{P1,P2}>

Example: P1 Reads a Pending Vote

Page 56: Nominal Calculi for Transactions:  ZSN in JOIN II

56

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

Q2

P3

{P2,P3} [P2] (P2)

{P1,P2} [P1,P2] (P2)

{P1,P3} [P1,P3] (P1,P3)

<P3,{P1,P2}>

<P1,{P2,P3}>

Example: P1 Contacts the Newly Known Party

Page 57: Nominal Calculi for Transactions:  ZSN in JOIN II

57

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

Q2

P3

{P2,P3} [P2] (P2)

{P1,P2} [P1,P2] (P2)

{P1,P3} [P1,P3] (P1,P3)

<P3,{P1,P2}>

<P1,{P2,P3}>

Example: Some Pending Messages Around

Page 58: Nominal Calculi for Transactions:  ZSN in JOIN II

58

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

Q2

P3

{P2,P3} [P2 ,P3] (P2 ,P3)

{P1,P2} [P1,P2] (P2)

{P1,P3} [P1,P3] (P1,P3)

<P1,{P2,P3}>

Example: P1 Reads a Pending Vote

Page 59: Nominal Calculi for Transactions:  ZSN in JOIN II

59

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P1

Q2

P3

{P2,P3} [P2 ,P3] (P2 ,P3)

{P1,P2} [P1,P2] (P1,P2)

{P1,P3} [P1,P3] (P1,P3)

Example: P3 Reads a Pending Vote

Page 60: Nominal Calculi for Transactions:  ZSN in JOIN II

60

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Q1

Q2

Q3

{P2,P3} [P2 ,P3] (P2 ,P3)

{P1,P2} [P1,P2] (P1,P2)

{P1,P3} [P1,P3] (P1,P3)

Example: P1 and P3 Commit!

Page 61: Nominal Calculi for Transactions:  ZSN in JOIN II

61

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

ZS nets in JoinWe encode basic nets, which are expressive enough:

fork

open join close calc

[[ E open e ]] = E def D in e(put,

{ lock }) | state({ E })

[[ e calc e’ ]] = e(p, L) e’(p, L)

[[ e fork e’, e’’ ]] = e(p, L) def D in e’(p, L

{ lock })

| e’’(put, L { lock }) |

state( )

[[ e’, e’’ join e ]] = e’(p’, L’) | e’’(p’’, L’’) e(p’, L’ L’’) | p’’(L’’

L’, )

[[ e close E ]] = e(p, L) p(L, { E })

given a net (T,S) we define an agent def [[ T ]] in [[S]] , where

default compensation

Page 62: Nominal Calculi for Transactions:  ZSN in JOIN II

62

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

DTC in JOINthe definition D is the following

state(H) | put(L, F) commit(L \ { lock }, L , { lock }, F, H )state(H) failed() | release(H)commit({ l } L, L’, L’’, F, H) commit(L, L’, L’’ , F, H)

| l(L’, lock, fail )commit(L, L’, L’’, F, H) | lock(L’’’, l, f )

commit(L (L’’’ \ L’), L’ L’’’, L’’ { l }, F, H )commit(, L, L, F, H) release(F)commit(, L’, L’’, F, H) | fail() failed() | release(H)failed() | put(L, F) failed()failed() | lock(L, l, f) failed() | f()failed() | fail() failed()

Page 63: Nominal Calculi for Transactions:  ZSN in JOIN II

63

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Opening

a new thread is createdand two tokens released

the name of the new thread

the return address of the new thread

the state of the new thread;it contains the consumed resourcesto be returned in case of failure

[[ E open e ]] = E def D in e(put, { lock }) |

state({ E })

open

Page 64: Nominal Calculi for Transactions:  ZSN in JOIN II

64

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Progress

[[ e calc e’ ]] = e(p, L) e’(p, L)

set of threads, including self, the token has been in contact with

return address where to send the set of threads and the generated stable resources, if any calc

Page 65: Nominal Calculi for Transactions:  ZSN in JOIN II

65

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Joining

[[ e’, e’’ join e ]] = e’(p’, L’) | e’’(p’’, L’’) e(p’, L’ L’’} | p’’(L’’

L’, )

the second thread terminates

the first thread continues

the contacted sets of threads are conjoined

join

Page 66: Nominal Calculi for Transactions:  ZSN in JOIN II

66

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Splitting

the new thread starts with the augmented contact set

a new thread is createdand three tokens released

the original thread continuesbut its contact set is augmentedwith the name of new thread

the state of the new threadcontains no consumed resource

[[ e fork e’, e’’ ]] = e(p, L) def D in e’(p, L

{ lock })

| e’’(put, L { lock }) | state(

)

fork

Page 67: Nominal Calculi for Transactions:  ZSN in JOIN II

67

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Pre-commit

state(H) | put(L, F) commit(L \ { lock }, L , { lock }, F, H )

generated stable resources to be unfrozen in case of success

list of known threads including self

List of contacted threads already synchronized with

the thread receives the signal that it can commit locallyand evolves to a committed state

consumed resources to be returned in case of failure

list of known threadsto be contacted yet

Page 68: Nominal Calculi for Transactions:  ZSN in JOIN II

68

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Commitment

commit({ l } L, L’, L’’, F, H) commit(L, L’, L’’ , F, H) | l(L’, lock, fail )

commit(L, L’, L’’, F, H) | lock(L’’’, l, f ) commit(L (L’’’ \ L’), L’ L’’’, L’’ { l },

F, H )

commit(, L, L, F, H) release(F)

sending a message to a known party

failure port

success port

synchronization sets updated for transitive closure

receiving a message from a party

all parties in the transitive closure have been contacted: commit

frozen resources are released

Page 69: Nominal Calculi for Transactions:  ZSN in JOIN II

69

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Failures

state(H) failed() | release(H)

commit(, L’, L’’, F, H) | fail() failed() | release(H)

failed() | put(L, F) failed()failed() | lock(L, l, f) failed() | f()failed() | fail() failed()

local failure consumed resources are given back

handling global failure

Page 70: Nominal Calculi for Transactions:  ZSN in JOIN II

70

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Correctness The join version of the D2PC protocol

comes with a correctness theorem for success for failure

The result is exploited to state the adequacy of the ZSN encoding complex statement because of garbage

terms

Page 71: Nominal Calculi for Transactions:  ZSN in JOIN II

71

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Playful Digression:5 Methods for Complex Proofs

Proof by calculus "This proof requires calculus, so we'll skip it."

Proof by postponement "The proof for this is long and arduous, so it will be

written in a technical annex." Proof by mumbo-jumbo

"(B P ) , $ (C W ) QED" Proof by illegibility

"scribble, scribble QED" Proof by poor analogy

"Well, it's just like the other day in the bus ..."

Page 72: Nominal Calculi for Transactions:  ZSN in JOIN II

72

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

D2PC in Jocaml JoCaml is an extension of the Objective Caml

Support Join calculus programming model Functional language Support of OO and imperative paradigms

A running implementation of the controllers in Jocaml has been developed by Hernán Melgratti Given a description of a ZS net, it automatically

generates a JoCaml program that simulates the net behavior

Page 73: Nominal Calculi for Transactions:  ZSN in JOIN II

73

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Thread Coordinator in Jocaml

let def new_coordinator () = let def state! h | timeout! () = failed() | release h | deact timeout or failed!() | fail! () = failed () or failed!() | lock! (ll, l , f) = failed () | f () or failed!() | put!(l,f) = failed () or commit!(l, l1, l2,f, h) | fail!() = failed() | release h | deact

timeout or commit0!(l, l1, l2,f, h) | fail!() = failed() |release h | deact

timeout or commit0!(l,l1,l2,f,h) = match l with [] -> if (equiv l1 l2) then release f else

commit(l,l1,l2,f,h) t::ts -> {t (l1,lock,fail) |commit0(ts,l1,l2,f,h)} or commit!(l,l1,l2,f,h)|lock!(l3,ll,f) = let lnew = union l (difference l3 l1) in

commit0 (lnew, union l1 l3,union l2 [ll],f,h)

or state! h | put! (l,f) = commit0 (del lock l, l, [lock], f, h) | deact timeoutin reply failed,fail,lock,put, commit, state, timeout ;;

local failure

linear patterns

Page 74: Nominal Calculi for Transactions:  ZSN in JOIN II

74

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

A ZSNet in Jocamllet def aZSNet () = let def placeA!() = let newthread = new_coordinator() in { (state newthread) [placeA] | placeB (put newthread,[lock newthread]) | act (timeout newthread)}

or placeB!(p,l) = placeC!(p,l)

or placeB!(p,l) = let newthread = create_thread() in { (state newthread) []

| placeC (p, union l [lock newthread]) | placeD (put newthread, union l [lock newthread]) | act (timeout newthread)} or ...

in reply placeA, place H .....;;

spawn {placeA()|placeA()|placeH ()} ;;

OPEN

CALC

FORK

Initial Marking

Page 75: Nominal Calculi for Transactions:  ZSN in JOIN II

75

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Example (Problem description)

Apartment Rentals There is an offer of apartments that can

be rented immediately There are requests to rent offered

apartments from persons Also, there are request to change

apartment, i.e., a person take a new apartment only when somebody else rents her apartment

Page 76: Nominal Calculi for Transactions:  ZSN in JOIN II

76

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

P Q R S

A

B

C

* *

*

*

Pref

Example (ZS Net)Pw

Ps

QinB

QchB

Qs Ba

RinC

RchC

Rs Ca

Af

Aa

QtkA

QinA

RtkB

RinB

PtkC

PinC

Sw

Ss

StkA

SinA

P Q R S

A

B

C

Free

Wants

State

* *

*

*

Page 77: Nominal Calculi for Transactions:  ZSN in JOIN II

77

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

Implementation

Reflexive encoding*

ZS Net Generator

Initial State Matrix

Preferences Matrix

ZsNet

Jocaml Source Code Jogc

Page 78: Nominal Calculi for Transactions:  ZSN in JOIN II

78

Roberto Bruni @ IMT Lucca 19 April 2005

Models and Languages for Coordination and Orchestration

InstitutionsMarketsTechnologies

IMT

References

Orchestrating transactions in join calculus (Proc. CONCUR’02, LNCS 2421, pp. 321-336) R. Bruni, C. Laneve, U. Montanari