16 - distributed transactions

DistributedTransactionsCS425/ECE428– DISTRIBUTEDSYSTEMS– SPRING2019

Material derived from slides by I. Gupta, M. Harandi, J. Hou, S. Mitra, K. Nahrstedt, N. Vaidya

Client-ServerTransactions

Atomicity:all-or-nothing◦ Makeupdatesonashadowcopy◦ Realupdateoncommit,discardonabort

Consistency:invariantssatisfied◦ Checkandabortonviolations

Isolation:concurrenttransactionsseriallyequivalent◦ Two-phaselocking(strictorotherwise)

Durability:resultspreservedaftercrashes◦ Savecommittedupdatestodisk,recoverstateaftercrash

NIKITA BORISOV - UIUC 2

Client Server

open tx[…]

commit/abort

2019-03-28

DistributedTransactionsAtransactionthatinvokesoperationsatseveralservers.


T

A

Y

Z

B

C

D

T

T1

T2

T11

T12

T21

T22

A

B

C

D

F

H

K

Flat Distributed Transaction Nested Distributed Transaction

X

2019-03-28

CoordinationinDistributedTransactions


T

A

Y

Z

B

C

D

X

join

join

join

Coordinator Participant

Participant

Participant

T

CoordinatorOpen Transacton

TIDClose Transaction

Abort Transaction

ParticipantA

a.method (TID )1

2

Join (TID, ref)3

Coordinator & Participants The Coordination Process

2019-03-28

Distributedbankingtransaction


..

BranchZ

BranchX

participant

participant

C

D

Client

BranchY

B

A

participantjoin

join

join

T

a.withdraw(4);

c.deposit(4);

b.withdraw(3);

d.deposit(3);

openTransaction

b.withdraw(T, 3);

closeTransaction

T = openTransactiona.withdraw(4);c.deposit(4);b.withdraw(3);d.deposit(3);

closeTransaction

Note: the coordinator is in one of the servers, e.g. BranchX

2019-03-28

DistributedTransactionChallenges


Atomicity:all-or-nothing◦ Mustensureatomicityacrossservers

Consistency:invariantssatisfied◦ Generally donelocally

Isolation:concurrenttransactionsseriallyequivalent◦ Locksateachserver.

Durability:resultspreservedaftercrashes◦ Eachserverkeepslocalrecoverylog

2019-03-28

I.LocksinDistributedTransactionsEachserverisresponsibleforapplyingconcurrencycontroltoobjectsitstores.

Serversarecollectivelyresponsibleforserialequivalenceofoperations.

Locksareheldlocally,andcannotbereleaseduntilallserversinvolvedinatransactionhavecommittedoraborted.

Locksareretainedduring2PCprotocol.

Sincelockmanagersworkindependently,deadlocksare(very?)likely.

NIKITA BORISOV - UIUC 72019-03-28

DistributedDeadlocksThewait-forgraphinadistributedsetoftransactionsisdistributed

Centralizeddetection◦ Eachserverreportswaits-forrelationshipstocoordinator◦ Coordinatorconstructsglobalgraph,checksforcycles

Decentralized— edgechasing◦ Forward“probe”messagestoserversintheedgesofwait-forgraph,pushingthegraphforward,untilcycleisfound.


ProbesTransmittedtoDetectDeadlock


V

Held byW

Waits forHeld by

Waitsfor

Waits forDeadlockdetected

U

C

A

B

Initiation

W®U ® V ®W

W®U

W®U ® V

Z

Y

X

2019-03-28

EdgeChasingInitiation:WhenaserverS1 noticesthatatransactionTstartswaitingforanothertransactionU,whereUiswaitingtoaccessanobjectatanotherserverS2,itinitiatesdetectionbysending<TàU>toS2.

Detection:Serversreceiveprobesanddecidewhetherdeadlockhasoccurredandwhethertoforwardtheprobes.

Resolution:Whenacycleisdetected,oneormoretransactionsinthecycleis/areabortedtobreakthedeadlock.

Phantomdeadlocks=falsedetectionofdeadlocksthatdon’tactuallyexist◦ Edgechasingmessagescontainstaledata(Edgesmayhavedisappearedinthemeantime).So,alledgesina“detected” cyclemaynothavebeenpresentinthesystemallatthesametime.


TransactionPriorityTransactionsaregivenpriorities◦ E.g.,inverseoftimestamp◦ Totalorder

Whendeadlockcycleisfound,abortlowestprioritytransaction◦ Onlyoneabortedevenifseveralsimultaneousprobesfindcycle


II.AtomicCommitProblemAtsomepoint,clientexecutescloseTransaction()◦ Result->commit,abort

Atomicityrequiresall-or-nothing◦ Alloperationsonallserversarecommitted,or◦ Alloperationsonallserversareaborted

Whatproblemstatementisthis?


AtomicCommitProtocolsConsensus!◦ Impossibletobetotallycorrect◦ Possibletoensuresafety,atthe(possible)expenseofliveness◦ Plus,wealreadyhavealeader(coordinator)

Firstattempt:Coordinatordecides◦ Pickcommitorabort◦ Sendmessagetoallparticipants◦ (Retransmituntilacknowledged)

Problems?◦ Participantcrashesbeforereceivingcommitmessage◦ Participantdecidestoabort(deadlock,otherproblems)


Two-phaseCommitPhase1:allparticipantsvotetocommit orabort◦ Ifyouvotetocommit,storepartialresultsinpermanentstorage◦ Ifcrashaftervotetocommit,canrestoretransactionlater

Phase2:◦ Saveresultofvoteinpermanentstorage◦ Ifall votecommit,multicastcommit message◦ Ifany voteabort,multicastabort message


RPCsforTwo-PhaseCommitProtocol


Coordinator->ParticipantcanCommit?(trans)->Yes /No

Askwhetherparticipantcancommitatransaction.Participantreplieswithitsvote.

doCommit(trans)Tellparticipanttocommititspartofatransaction.

doAbort(trans)Tellparticipanttoabortitspartofatransaction.

Participant->CoordinatorhaveCommitted(trans,participant)

Confirmthatparticipanthascommittedthetransaction.(MaynotberequiredifgetDecision()isused– seebelow)

getDecision(trans)->Yes /NoAskforthedecisiononatransactionafterparticipanthasvotedYesbuthasstillhadnoreplyaftersomedelay.Usedtorecoverfromservercrashordelayedmessages.

2019-03-28

2PC– CoordinatorPhase1:◦ SendcanCommit? toallparticipants,tabulatereplies

Phase2:◦ Ifallvotesareyes,senddoCommit toallparticipants◦ Ifanyvotesareno,oranyparticipantdoesn’treplyaftertimeout,senddoAbort toallparticipants[whosaidyes]

◦ Storecommitdecisiontostablestoragetosupportrecovery

Recoveryaftercrash◦ Ifcommitdecisioninstablestorage,confirmwithparticipants(push)orwaitforgetDecision (pull)

◦ IfgetDecision calledoncommitnotinlog,replyNo


errsonsideofsafety

2019-03-28

2PC- ParticipantPhase1:receivecanCommit?◦ IfOKtocommit,replyYes andstoretransactioninpermanentstorage◦ IfnotOK,replyNo andabortimmediately

Phase2◦ IfreceivedoCommit,committransaction◦ IfreceivedoAbort,aborttransaction◦ Iftimeout,callgetDecision

Recoveryaftercrash◦ IfcrashedafteraYesinPhase1,callgetDecision◦ Ifshouldcommit,recovertransactionfrompermanentstorageandcommit


Thetwo-phasecommitprotocolPhase 1 (voting phase):◦ 1. The coordinator sends a canCommit? request to each of the participants in the transaction.◦ 2. When a participant receives a canCommit? request it replies with its vote (Yes or No) to the

coordinator. Before voting Yes, it prepares to commit by saving objects in permanent storage. If its vote is No, the participant aborts immediately.

Phase 2 (completion according to outcome of vote):◦ 3. The coordinator collects the votes (including its own).

◦ (a) If there are no failures and all the votes are Yes, the coordinator decides to commit the transaction and sends a doCommit request to each of the participants.

◦ (b) Otherwise the coordinator decides to abort the transaction and sends doAbort requests to all participants that voted Yes.

◦ 4. Participants that voted Yes are waiting for a doCommit or doAbort request from the coordinator. When a participant receives one of these messages it acts accordingly and in the case of commit, makes a haveCommitted call as confirmation to the coordinator.


Recall that server maycrash

2019-03-28


canCommit?

Yes

doCommit

haveCommitted

Coordinator

1

3

(waiting for votes)

committed

done

prepared to commit

step

Participant

2

4

(uncertain)prepared to commit

committed

statusstepstatus

v To deal with server crashesv Each participant saves tentative updates into permanent storage, right before

replying yes/no in first phase. Retrievable after crash recovery.v To deal with canCommit? loss

v The participant may decide to abort unilaterally after a timeout (coordinator will eventually abort)

v To deal with Yes/No loss, the coordinator aborts the transaction after a timeout (pessimistic!). It must annouce doAbort to those who sent in their votes.

v To deal with doCommit lossv The participant may wait for a timeout, send a getDecision request (retries until

reply received) – cannot unilaterally abort after having voted Yes but before receiving doCommit/doAbort!

2019-03-28

Two Phase Commit (2PC) Protocol


Coordinator Participant

Execute• Precommit

Uncertain•Send request to each participant• Wait for replies (time out possible)

Commit•Send COMMIT to each participant

Abort•Send ABORT to each participant

Execute

• Precommit• send YES to coordinator• Wait for decision

Abort•Send NO to coordinator

NOYES

request

notready ready

All YES

Timeout or a NO

Commit• Make transaction visible

Abort

COMMIT decision

CloseTrans()

ABORT decision

2019-03-28

SummaryDistributedTransactions◦ Morethanoneserverprocess(eachmanagingdifferentsetofobjects)◦ Oneserverprocessmarkedoutascoordinator◦ AtomicCommit:2PC◦ Deadlockdetection:Edgechasing=


16 - distributed transactions

Documents