20 distributed reliability protocols.pdf

8/2/2019 20 Distributed Reliability Protocols.pdf

1/31

DistributedD

atabaseSystems

D

istributedReliabilityProto

cols

Ozsu-Chapter12

DavidSilberberg


2/31

D.Silberberg

Distributed

DatabaseSystems

ReliabilityProtocols

2

Introduction

Goal:maintain

atomicityanddurab

ilityofdistributedtransactionsthatexecu

te

overmultipledatabases

Relevantcomm

ands

Begintrans

action

Read

Write

Abort

Commit

Commandsthatarenotproblematic

Begintrans

actionexecutesjustlikethatofcentralizedd

atabases

ReadandW

riteexecutesusingR

eadOnceWriteAll(R

OWA)algorithm

Ateachsite

Commands

areexecutedasinace

ntralizeddatabase

Abortisexecutedbyundoingthedatabaseseffectsasinthecentralizedcase


3/31

D.Silberberg

Distributed

DatabaseSystems


3

Actorsand

Algorithms

Abstraction

ofactors

Coordina

torresponsibleforcoordinatingthe

statesoftheother

activeme

mbersofthedistributeddatabasefor

aparticular

transactio

n

Participantaprocessthatcarriesoutoneofthecomponentsofa

distribute

dtransactionononemachine

Terminationprotocols

CommitandRecovery

Needspe

cialexecutionina

distributedsystem

Ifonesitefailsinitspartof

atransaction,wewanttheothersites

toterminateaswell


4/31

D.Silberberg

Distributed

DatabaseSystems


4

T

ermination

andRecovery

TerminationandRecovery

Oppositesidesoftherecovery

problem

Terminationhowdositesdealwithfailure?

Recovery

howdositesrecoverstateonceitisrestarted?

Maintainatomicityoftransactio

n

Somesitesmayfail

However,

thetransactionmust

beall-or-nothing

Nonblocking

protocols

Transactio

nterminationatone

sitedoesnotneedto

waitforfailedsite(s

)

torecover

Independentrecoveryprotocols

Recoverw

ithouthavingtocon

sultothersitesinatransaction

Reducesm

essageexchangeduringrecovery

Independe

nceimpliesnonblocking,butnotvisaver

sa


5/31

D.Silberberg

Distributed

DatabaseSystems


5

Two-PhaseCommit(2PC)P

rotocol

Ensuresato

miccommitme

ntofdistributed

transactions

Allsitesmustagreetocom

mitbeforeany

permanent

effectstake

place

Synchroniz

ationisnecessa

ry

Dependin

gonthetypeofco

ncurrencyalgorith

mused,some

schedulersmaynotbewillingtoterminateatr

ansaction

Anotherreasonatransactionmaynotbeready

tocommitis

becaused

eadlockspreventitfromcommitting


6/31

D.Silberberg

Distributed

DatabaseSystems


6

2PCProto

col(thatdoesnotconsiderfailures)

Onecoordina

torcontrolstheprocess

Othersactors

areparticipants

Coordinatorstartsbyaskingparticipantstopreparefordistributed

transaction,thenenterswaitstate

Participantsd

eterminewhetherornottheycancom

mitornot

Participantsv

ote

Ifonevotestocommit,itenter

sreadystateandwaitsforcoordinators

response

Ifonevotesabort,itforgetsthetransaction

Coordinatordecideswhethertra

nsactionshouldco

ntinue

Ifallparticipatesvotecommit,

itsendsaglobal-com

mittoallparticipan

ts

Ifjustone

aborts,itsendsaglo

bal-aborttoallparticipants

Participantsa

ct

Ifglobal-c

ommit,theycommittheirtransactions

Ifglobal-a

bort,theyaborttheirtransactions


7/31

D.Silberberg

Distributed

DatabaseSystems


7

Tw

oPhaseCo

mmitAlgorithm

Coordinator

Participant

Initial

Initial

Wait

Commit

Abort

Abort

Commit

writebegincomm

it

inlog

writeabortinlog

writecommitinlog

writeendof

Xactioninlog

readyto

commit?

writeabortinlog

writeabortinlog

writecommitinlog

anyno?

typeof

message?

Ready

writereadyinlog

pre

pare

vote-abort

vote-commit

no

yes

yes

no

abort

commit

globalcommitgl

obalabort

AC

K

ACK


8/31

D.Silberberg

Distributed

DatabaseSystems


8

O

bservations

ofSimple2PC

Eachpartic

ipantcanunilaterallyabort

Onceaparticipantaborts,itcannotchangeitsvote

Ifaparticip

antisreadytocommit,itcans

tillabortor

commitdependingonthefinaldeterminationofthe

coordinator

Thecoordinatormakesthe

finaldecisionbasedonanall-

or-nothing

vote(thisisatomicity)

Boththeco

ordinatorandp

articipantsenterstateswhere

theywaitforeachother

Bothsettimers

Iftheydo

nothearfromeac

hotherwithinace

rtaintimelimit,they

endthetransactionwithoutcommitting


9/31

D.Silberberg

Distributed

DatabaseSystems


9

AlgorithmConceptCentralized

Coordinator

Participants

Coordinator

Participa

nts

Coordinator

phas

e1

phase2

prepare

voteabortor

commit

globalcommit

orabort

committedor

aborted


10/31

D.Silberberg

Distributed

DatabaseSystems


10

Linear2PCAlgorithm

Centralized2P

Chassomedrawbac

ks

Therearem

anymessagesthatare

sentbetweenthecoordinatorandparticipants

Allparticip

antsmustbesynchroni

zed

Linear2PCaddressessomeoftheseissues

Thereisan

orderingofsitesforco

mmunicationsforthoseparticipatinginthe

distributed

transaction(1,2,,N)

Thecoordinatorisfirst

Algorithm

Coordinator(#1)sendspreparetoparticipant(#2)

Participant

(#2)decidestoabortorcommit

Ifabort,(#2)sendsabortmessage

to(#3)

Ifcommit,(#3)sendscommitmessageto(#3)

Allsubsequ

entparticipants

Ifreceiveabort,theysendabortm

essagetonext

Ifreceivecommit,theydetermine

iftheycancommitandse

ndtheirvoteonward

Lastparticipantsendsglobal-commitorglobal-abortback

from(#N)to(#N-1)to

to(#1)

DrawbackisthattheLinear2PCis

slow


11/31

D.Silberberg

Distributed

DatabaseSystems


11

Linear2PhaseCommit

1

2

3

N

prepare

VC/VA

VC/VA

GC/GA

GC/GA

GC/GA

VC/VA

GC/GA


12/31

D.Silberberg

Distributed

DatabaseSystems


12

D

istributed2

PhaseCom

mit

AdvantagesofDistributed

2PC

Allpartic

ipantscommunicatewitheachother

Allpartic

ipantsindependentlyarriveatconclusion

Eliminatesthesecondphase

ofcommunication

Disadvanta

gesofDistributed2PC

Eachparticipantmustbeaw

areofeveryother

participant

N2messa

gesmustbesent


13/31

D.Silberberg

Distributed

DatabaseSystems


13

Distributed

2PCDiagra

m

Coordinator

Participants

Coordinator+Participants

C

P P P P

P P P P

prepare

voteabortor

commit

globalcommitorab

ortdecision

madeindependently

C


14/31

D.Silberberg

Distributed

DatabaseSystems


14

Variatio

nsforPerfo

rmanceImprovement

Presumed2PC

Protocols

Reducemessagesamongcoordinatorandparticipants

Reducelogfileswrittenthroughpresumptionofoperations

PresumedAbort2PC

UsefulforR

EADandpartialUPDA

TEtransactions

Algorithm

Ifcoordinatordecidestoabort,itfo

rgetsaboutthetransaction

Participantspollthecoordinatorfo

rtheitscommit/abortdecision

Ifthereisnoentry,theparticipants

presumethatthetransactionhasaborted

Iftheco

ordinatormeanttocommit,itwouldnothaveforgottenaboutthetransaction

Advantages

Coordinatorcanforgetabouttransactionafterabortnoneed

towritetothetransaction

log

Participantsdonotacknowledgeanabort

Thecoordinatorabortrecorddoesnotneedtobeforcedtope

rmanentstorage

If

aparticipantfailsandreco

vers,thelackofanabortr

ecordisenoughtotellthe

participanttoabort

Participantsdonotneedtowriteabortseither


15/31

D.Silberberg

Distributed

DatabaseSystems


15

Presumed2PCCommit

Sincemosttransactionsarepresum

edtocommit,wepresumealackof

informationim

pliesacommit

Coordinatorduringthepreparepha

se

Forcesaco

llectingwritetoitsstab

lestoragelistingalltheparticipants

Participantsen

tercollectingstate

Coordinatorse

ndspreparestatementandenterswaitstate

Participantsde

cidewhattodo

Writeabortorcommitrecords

Sendvotem

essagetocoordinator

Coordinatorm

akesdecision

Ifabort,itwritesaglobal-abortrecordandsendsittoparticipants

Ifcommit,itwritescommitrecord

andsendsglobal-committhenitforgetsabo

ut

thetransaction

Participants

Iftheyrece

iveglobal-commit,theywritetherecordandd

onotacknowledge

Iftheyrece

iveglobal-abort,theya

bortandacknowledge


16/31

D.Silberberg

Distributed

DatabaseSystems


16

SiteF

ailures

Needprotoco

lstorecoverwhen

asitefails

Desiredcharacteristicsofprotocol

Independe

nteachsiteperformsitsownrecovery

Non-block

ingeachsitecanp

roceedwithoutwaitingforothersites

operations

Independe

ntimpliesnon-block

ing

Itispossible

todesignsuchprotocolsforasinglesitefailure

Itisnotpossibletodesigninde

pendentprotocolsfo

rmultiplesitefailures

Nextslidesw

illdemonstrateterminationandrecoveryalgorithmsfor

2PC

However,

theyareinherentlyb

locking

ThreePha

seCommit(3PC)willmakeourterminationandrecovery

algorithmsnon-blocking


17/31

D.Silberberg

Distributed

DatabaseSystems


17

TerminationProtocols

Performedwhencoordinatorand/orparticipatestimeou

t

Coordinatorand/orparticipantsdo

notreceivemessagebeforethetimeout

Weassume

thatthereasonforthe

lackofmessageisdue

toasitefailure

Terminationactiondependentonstate

Differentactionsareperformedat

differentpointsinthe2

PCalgorithm

Statetransitiondiagramshelpusu

nderstandthestateand

correspondingactions

2PCstatetransitiondiagrams

INITIAL

WAIT

COMMIT

ABORT

commit/prepare

vote-commit/

global-commit

vote-abort/

global-abort

IN

ITIAL

READY

COMMIT

ABORT

prepare/vote-commit

global-commit/

ack

global-abort/

ack

prepare/vote-abort

Coordinator

Participant


18/31

D.Silberberg

Distributed

DatabaseSystems


18

Coordinat

orTimeouts

Thisisstraightforwardonlytw

ocasestoconside

r

IfthetimeoutoccursintheWAITstate

Coordinatoriswaitingforparticipantdecisions

Cannotun

ilaterallycommitwithoutunanimousvote

However,

itcanunilaterallyab

ort

Writes

abortmessageinlog

Sends

participantsabortmess

age

IfthetimeoutoccursintheCOM

MITorABORT

states

Itdoesnotknowifparticipantshavecompletedtasks

Coordinatorrepeatedlysendso

utglobal-commitor

global-abo

rtmessagesuntilitreceivesresponses

Itwilleventuallyhearfromall

(wehope)

INITIAL

WAIT

COMMIT

ABORT

commit/prepare

vote-commit/

global-commit

vote-abort/

global-abort

INITIAL

WAIT

COMMIT

ABORT

commit/prepare

vote-commit/

global-commit

vote-abort/

global-abort


19/31

D.Silberberg

Distributed

DatabaseSystems


19

ParticipantTimeouts

Participantscantimeoutinboththe

INITIALand

READYstates

INITIALstatetimeout

Waitingfor

preparemessage

Assumethatthecoordinatorfailed

intheINITIALstate

Participantunilaterallyabortsafter

thetimeout

Ifpreparearrivesafterthispoint

Participantvotesabort

Or,igno

respreparemessagecoo

rdinatorthenwould

timeout

READYstatetimeout

Participantvotedtocommit,butdo

esnotknowofthe

globalvote

Cannotmak

eunilateraldecisionor

changeitsvote

Participantmustremainblockeduntilitreceivesthe

globaldecis

ion

Ifcoord

inatorfailed,itwillremain

blocked(bad)

However,itcanaskotherparticipantsforinformation

INITIAL

READY

COMMIT

A

BORT

prepare/vote-commit

global-commit/

ack

global-abort/

ac

k

p

repare/vote-abort

INITIAL

READY

COMMIT

A

BORT

prepare/vote-commit

global-commit/

ack

global-abort/

ackp

repare/vote-abort


20/31

D.Silberberg

Distributed

DatabaseSystems


20

OtherP

articipantsAddressthe

Blocking

Ifparticipantscommunicatewitheachother,theycanhelp

determinationacourseofaction

Participantth

attimesout(PT)canaskotherparticipants(PO)fortheir

state

IfsomePOisintheINITIALstate

POhas

notvotedandmaynot

haveevenreceivedap

reparemessage

POunilaterallyabortsandsen

dsvote-aborttoPT

IfsomePOisintheREADYstate

POhas

votedtocommit,buth

asnotreceivewordon

theglobalvote

Itcann

othelpPT

IfsomePOisintheABORTorCOMMITstates

POeitherunilaterallyaborted

oritreceivedaglobalvotefromthecoordinator

Itsend

sPTouteithertheglob

al-abortorglobal-commitvote


21/31

D.Silberberg

Distributed

DatabaseSystems


21

HowtheP

articipantI

nterpretstheResponses

PTreceivesvote-abortfromallotherparticipantsPO

Theyhaveallaborted

Thus,PTca

nabortaswell

PTreceivesvote-abortfromsomep

articipantsPO,butothersareintheREAD

Y

state

Atleastoneparticipantaborted,so

coordinatorwouldhav

etoabort

PTcanabortaswell

PTfindsthatallPOareintheREAD

Ystate

Noparticip

antcanproceed

Stuckwaiti

ngforcoordinatorPT

isbacktowhereitstar

ted

PTgetsglobal-abortorglobal-commitfromallotherparticipants

PTcanproc

eedaccordingly

Allparticip

antsmustbeinagreem

entitisimpossibleotherwise

PTgetsglobal-abortorglobal-commitfromsomeparticipants

OtherPOareintheREADYstate

PTcanproc

eedwitheitheranABO

RTorCOMMITbecausethecoordinatormust

havemade

theabortorcommitdecision


22/31

D.Silberberg

Distributed

DatabaseSystems


22

How

toHandle

theBlockedState

Inthethird

condition,thereisstillblockin

g

Itmustbethatthecoordinatorhasfailed

Allpartic

ipantselectanew

coordinator

Newcoordinatorrestartsthecommitprocess

Ifthecoord

inatorandoneparticipantfails

,thisprocessis

morediffic

ult

Intheend,

2PCisablockingprotocolbecauseofthiscase


23/31

D.Silberberg

Distributed

DatabaseSystems


23

RecoveryProtocols

Algorithms

thathelprecov

erstatewhencoordinatoror

participantsfailandrestart

Wouldlikeprotocoltoensu

reindependence

However,itisnotpossibletodesignaprotoco

ltobeindependent

andmain

tainatomicityatth

esametime

Duetoth

efactthat2PCisa

blockingprotocol

Assumptions

Writinga

ctioninlogandse

ndingmessagesis

anatomicoperatio

n

Notfullyrealistic

Canb

ehandledinastraightforwardway

Statetran

sitionoccursafter

sendingamessage


24/31

D.Silberberg

Distributed

DatabaseSystems


24

Coordinato

rSiteFailure

Failsinthe

INITIALstate

Nothinghasbeensentyet

Justrestartstransactionwhe

nthesiterestarts

Failsinthe

WAITstate

Thepreparemessagehasalreadybeensent

Notallparticipantsmayhav

ereceivedthepreparemessage

Thus,wh

enthesiterestarts,itresendstheprep

aremessage

Failsinthe

COMMITorA

BORTstates

Ithasalreadyinformedparticipantsofitsdecision

Uponrestart

Ifallparticipantssentacknowledge,coordinato

rdoesnothing

Ifsom

eparticipantshavenotresponded,thenthecoordinatorstarts

theterminationprotocol


25/31

D.Silberberg

Distributed

DatabaseSystems


25

Participant

SiteFailures

FailsinINITIALstate

Uponreco

very,theparticipant

shouldabortthetran

sactionunilaterally

Coordinatorwilleitherbeinth

eINITIALorWAIT

states

Ifinth

eINITIALstate,itwillsendthepreparemess

ageandmovetothe

WAIT

state

Coordinatorwilleithertimeoutorreceiveabortmessagefromfailed

participant

Eitherway,thetransactionwillbeabortedbythecoordinator

FailsinREADYstate

Participantalreadyvotedtocommit

Uponreco

very,theparticipant

canhandlethisasatimeoutandtransfer

thistothe

terminationprotocol

Manycasescanbehandledwithoutblocking,butthereisthepotentialfor

blocking

FailsinABO

RTorCOMMITs

tates

Theseare

terminationstates

Uponreco

very,nospecialactionneedbetaken


26/31

D.Silberberg

Distributed

DatabaseSystems


26

Three

PhaseCom

mit(3PC)Protocol

Non-blockingprotocoltoaddressthelimitationso

f2PC

Conditionsfo

rnon-blocking

NostateisadjacenttobothaC

OMMITandanABORTstate

Nonon-co

mmittablestateisad

jacenttoaCOMMITstate

WAITandREADYstatesof2P

Careproblematic

Botharea

djacenttobothABO

RTandCOMMIT

Bothnon-committable,butnex

ttoCOMMIT

INITIAL

WAIT

COMMIT

ABORT

commit/prepare

vote-commit/

global-commit

vote-abort/

global-abort

IN

ITIAL

READY

COMMIT

ABORT

prepare/vote-commit

global-commit/

ack

global-abort/

ack

prepare/vote-abort

Coordinator

Participant


27/31

D.Silberberg

Distributed

DatabaseSystems


27

ModifiedStateTransition

Solvethep

roblembyaddinganew(3rd)statebefore

COMMIT

INITIAL

WAIT

COMMIT

ABORT

commit/pr

epare

vote-commit/

prepare-to-comm

it

vote-abort/

global-abort

INITIAL

R

EADY

COMMIT

ABORT

prepare/vote-comm

it

prepare-to-commit/

ready-to-commit

global-abort/

ack

prepare/vote-abort

Coordinator

Participant

PRE-

COMMIT

ready-to-commit/

global-commit

PRE-

COMMIT

global-commit/

ack


28/31

D.Silberberg

Distributed

DatabaseSystems


28

3PCP

rotocol

Initial

Initial

Wait

Commit

Abort

Abort

Commit

writebegincomm

it

inlog

writeabortinlog

writecommitinlog

writeendof

Xactioninlog

readyto

commit?

writeabortin

log

writeabort

inlog

writeprepareto

commitinlog

anyno?

typeof

message?

Ready

writereadyinlog

pre

pare

vote-abort

vote-commit

no

yes

yes

no

abort

prepare-to-commit

global-abort

AC

K

ACK

writecommitinl

og

Pre-Commit

pr

epare-to-

commit

Pre-Commit

writeprepareto

commitinlog

ready-to-commit

commit


29/31

D.Silberberg

Distributed

DatabaseSystems


29

3PC

CoordinatorTimeoutP

rotocol

TimeoutinWAITstate

SameasCo

ordinatortimeoutinW

AITstatefor2PC

Unilaterallydecidestoabortthetransaction

Sendsglobal-abortmessagetoallparticipants

TimeoutinPR

ECOMMITstate

Coordinatordoesnotknowifnon-respondentshavemovedtoaPRECOMMIT

state

Itknowsthattheyareatleastinth

eREADYstatetheyhaveallvotedto

COMMIT

Sendsprepare-to-commitmessage

Globallycommitsthetransaction

Update

slog

Sendsglobal-commitmessagetoallparticipants

Moves

allparticipantstoPRECOMMITstate

TimeoutinCO

MMITorABORTs

tates

Coordinatordoesnotknowifparticipantshaveperformed

theirCOMMITSor

ABORTS

Theyareat

leastinthePRECOMM

ITstate(orREADYs

tateforanABORT).

Theyneedtofollowterminationprotocol(describedsoon

)


30/31

D.Silberberg

Distributed

DatabaseSystems


30

3PC

Participant

TimeoutProtocol

TimeoutinINIT

IALstate

Sameastimeoutin2PCINITIALstate

Waitingforpreparemessage

Assumethat

thecoordinatorfailedinth

eINITIALstate

Participantunilaterallyabortsafterthetimeout

Ifpreparearrivesafterthispoint

Participantvotesabort

Or,igno

respreparemessagecoordinatorthenwouldtimeout

TimeoutinREA

DYstate

Alreadyvote

dtoCOMMIT

Doesnotkno

wglobaldecision

Electsnewcoordinator

Newcoordin

atorterminatestransaction

TimeoutinPRE

COMMITstate

Alreadyrece

ivedprepare-to-commitmessage

Waitingforfinalglobal-commit

Electsnewcoordinator

Newcoordin

atorterminatestransaction


31/31

D.Silberberg

Distributed

DatabaseSystems


31

3

PCTerminationProtoc

ols

Newcoordinatorisselected

Sendsitsownstatetoallparticipantsaskingthemtoassumethestate

Allthe

participantsthathavepass

edthatstateignorethemessage

Theres

tmakestatetransitionsandsendbackappropriatemessages

Non-blocking

Protocol

IfcoordinatorinWAITstate

Globallyabortsthetransaction

IfparticipantinthePRECOMMITstate

T

hereisnotransitiontotheAB

ORTstate

N

ewtransitionfromPRECOMMITtoABORTneedstobeadded

IfcoordinatorinPRECOMMITstate

NoparticipantcanbeintheABORTstate(sincetheyhavea

llvotedtocommit)

Sendsoutglobal-commit

IfcoordinatorinABORTstate

Moves

allparticipantstoABORT

state

3PCRecovery

Protocolverysimila

rto2PCwithsomevariations

20 distributed reliability protocols.pdf

Documents