recovery

03/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery

Recovery


Integrity or correctness of data

Would like data to be “accurate” or “correct” at all times

EMP

Name

WhiteGreenGray

Age

523421

1


Integrity or consistency constraints

Predicates data must satisfy Examples:

- x is key of relation R- x y holds in R- Domain(x) = {Red, Blue, Green}is valid index for attribute x of R- no employee should make more than twice the average

salary


Definition:

Consistent state: satisfies all constraints Consistent DB: DB in consistent state


Observation: DB cannot be consistent always!

Example: a1 + a2 +…. an = TOT (constraint)

Deposit $100 in a2: a2 a2 + 100

TOT TOT + 100


a2

TOT

.

.50

.

.1000

.

.150

.

.1000

.

.150

.

.1100

Example: a1 + a2 +…. an = TOT (constraint)Deposit $100 in a2: a2 a2 + 100

TOT TOT + 100


Transaction: collection of actions that preserve consistency

Consistent DB Consistent DB’T


How can we prevent/fix violations?

cc: due to data sharing only recovery: due to failures only


Will not consider:

How to write correct transactions How to write correct DBMS Constraint checking & repair

That is, solutions studied here do not need to know constraints


Storage hierarchy

Memory Disk

x x


Operations:

Input (x): block with x memory Output (x): block with x disk

Read (x,t): do input(x) if necessary t value of x in blockWrite (x,t): do input(x) if necessary value of x in block t


Key problem Unfinished transaction

Example Constraint: A=B

T1: A A 2

B B 2


T1: Read (A,t); t t2Write (A,t);Read (B,t); t t2Write (B,t);Output (A);Output (B);

A: 8B: 8

A: 8B: 8

memory disk

1616

16

failure!


T1: Read (A,t); t t2 A=BWrite (A,t);Read (B,t); t t2Write (B,t);Output (A);Output (B);

A:8B:8

A:8B:8

memory disk log

Undo logging

1616

<T1, start><T1, A, 8>

<T1, commit>16 <T1, B, 8>

16


One “complication” Log is first written in memory Not written to disk on every action

memory

DB

LogA: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8>

A: 8B: 8

16BAD STATE

# 1


One “complication” Log is first written in memory Not written to disk on every action

memory

DB

LogA: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8><T1, commit>

A: 8B: 8

16BAD STATE

# 2

<T1, B, 8><T1, commit>

...


Undo logging rules

(1) For every action generate undo log record (containing old value)

(2) Before x is modified on disk, log records pertaining to x must be on disk (write ahead logging: WAL)

(3) Before commit is flushed to log, all writes of transaction must be reflected on disk


Recovery rules: Undo logging

(1) Let S = set of transactions with <Ti, start> in log, but no

<Ti, commit> (or <Ti, abort>) record in log

(2) For each <Ti, X, v> in log,

in reverse order (latest earliest) do:

- if Ti S then - write (X, v)

- output (X)

(3) For each Ti S do

- write <Ti, abort> to log


What if failure during recovery?

No problem! Undo idempotent


To discuss: Redo logging Undo/redo logging, why both? Checkpoints


Redo logging (deferred modification)

T1: Read(A,t); t t2; write (A,t);

Read(B,t); t t2; write (B,t);

Output(A); Output(B)

A: 8B: 8

A: 8B: 8

memory DB LOG

1616

<T1, start><T1, A, 16><T1, B, 16>

<T1, commit>

output16


Redo logging rules

(1) For every action, generate redo log record (containing new value)

(2) Before anything is modified on disk (DB), all log records for transaction that modify things (including commit) must be on disk


(1) Let S = set of transactions with <Ti, commit> in log

(2) For each <Ti, X, v> in log, in forward

order (earliest latest) do:

- if Ti S then Write(X, v) Output(X)

Recovery rules: Redo logging


Recovery is very, very SLOW !

Redo log:

First T1 wrote A,B Last

Record Committed a year ago Record

(1 year ago) --> STILL, Need to redo after crash!!

... ... ...

Crash


Undo Recovery – Better?

The first record without commit or abort should not be too far away from crash

But immediate update itself is expensive

... ... ...

first record without commit or abort


Solution: Checkpoint (simple version)

Periodically:

(1) Do not accept new transactions

(2) Wait until all transactions finish

(3) Flush all log records to disk (log)(4) Flush all buffers to disk (DB) (do not discard buffers)

(5) Write “checkpoint” record on disk (log)

(6) Resume transaction processing


Example: what to do at recovery?

Redo log (disk):

<T1

,A,1

6>

<T1

,com

mit

>

Ch

eck

poin

t

<T2

,B,1

7>

<T2

,com

mit

>

<T3

,C,2

1>

Crash... ... ... ...

...

...



Undo log (disk):

<T1

,A,8

>

<T1

,com

mit

>

Ch

eck

poin

t

<T2

,B,1

7>

<T2

,com

mit

>

<T3

,C,2

1>

Crash... ... ... ...

...

...


Nonquiescent Checkingpointing

“Quiescent” checkpointing stalls DB “Nonquiescent” checkpointing admits new transactions

while checkpointing


Nonquiescent Checkpoint Rules - undo

Write and flush a log record <START CKPT(T1, …, Tk)> where T1,…,Tk are active transactions

When all T1, …, Tk have completed, write and flush a log record <END CKPT>



Undo log (disk):

<T1

,A,8

>

<C

KPT(T

1,T

2)>

<T3

,B,1

7>

<T1

,com

mit

>

<T3

,C,2

1>

Crash... ... ...

...

<E

ND C

KPT>

<T2

,B,1

2>

<T2

,com

mit

>

Crash after end of checkpoint

Only need to undo all the incomplete transactions started after <START CKPT>



Undo log (disk):

<T1

,A,8

>

<C

KPT(T

1,T

2)>

<T3

,B,1

7>

<T1

,com

mit

>

<T3

,C,2

1>

Crash... ... ...

... …

<T2

,B,1

2>

<T2

,com

mit

>

Crash before end of checkpoint

Only need to undo all incomplete transactions started after <START CKPT> and those in <CKPT (…)>


Nonquiescent Checkpoint Rules - redo

Write and flush a log record <START CKPT(T1, …, Tk) where T1,…,Tk are active transactions

Flush all the updates of the transactions committed before <START CKPT(T1,…,Tk)>

Write and flush a log record <END CKPT>



Redo log (disk):

<T1

,A,8

>

<C

KPT(T

1,T

2)>

<T3

,B,1

7>

<T1

,com

mit

>

<T3

,C,2

1>

Crash... ... ...

...

<E

ND C

KPT>

<T2

,B,1

2>

<T2

,com

mit

>

Crash after end of checkpoint

Only need to redo all the transactions committed after the latest <START CKPT>



Redo log (disk):

<T1

,A,8

>

<C

KPT(T

1,T

2)>

<T3

,B,1

7>

<T1

,com

mit

>

<T3

,C,2

1>

Crash... ... ...

... …

<T2

,B,1

2>

<T2

,com

mit

>

Crash before end of checkpoint

Only need to redo all the transactions committed after the latest successful <START CKPT>


Key drawbacks:

Undo logging: need to update immediately, increasing I/O Redo logging: need to keep all modified blocks in memory until

commit


Solution: undo/redo logging!

Update <Ti, Xid, New X val, Old X val>


Rules

Page X can be flushed before or after Ti commit Log record flushed before corresponding updated

page (WAL) Flush log at commit


Non-quiesce checkpoint

LOG

for undo

Start-ckptactive TR:

Ti,T2,...

endckpt

.........

... Flush

dirty buffers


Examples what to do at recovery time?

no T1 commit

LOG

T1,-a

...CkptT1

...Ckptend

...T1-b

...

Undo T1 (undo a,b)


Example

LOG

...T1

a... ...

T1

b... ...

T1

c...

T1

cmt...

Ckptend

ckpt-sT1

Redo T1: (redo b,c)


Examples what to do at recovery time?

no T1 commit

LOG

T1,-a

...CkptT1

... ...T1-b

...

Undo T1 (all the actions)


Example

LOG

...T1

a... ...

T1

b... ...

T1

c...

T1

cmt...

ckpt-sT1

Redo T1: (redo b,c, a and the actions after last successful ckp-s)


Recovery process:

Backwards pass (end of log -> latest checkpoint start) construct set S of committed transactions undo actions of transactions not in S

Undo pending transactions follow undo chains for transactions in

(checkpoint active list) - S Forward pass (latest successful checkpoint start -> end of log)

redo actions of S transactions

backward pass

forward passstart

check-point


Recovery with Checkpoint Summary

Logging still obey the logging rules undo, redo, undo-redo

Recovery still obey rules Undo: undo all incomplete transactions Redo: redo all completed transactions Undo-redo: combined

Checkpoint provides a way to limit the transactions needing undo or redo


Summary

undo redo Undo-redo

commit commit means all updates on disk

Commit means some updates may start to migrate to disk

Commit does not signal disk update

recovery Undo all incomplete transactions

Redo all committed transactions

Undo all incomplete transactions and redo all committed transactions

Nonquiescent ckpt

<end ckpt(T)> means every tran started before the matching <start ckpt (T)> is on disk

<end ckpt(T)> means every tran committed before the matching <start ckpt(T)> is on disk

<end ckpt(T)> means every update before the matching <start ckpt(T)> is on disk

WAL


Summary

Consistency of data One source of problems: failures

- WAL

recovery

Documents

t t t2write b

b b 2t1

tread b

x memoryoutput x

x diskread x

t readb

necessary t value of

a2 tot