scalable logging algorithm for in-memory database … logging algorithm for in-memory database...
TRANSCRIPT
Scalable Logging Algorithm for in-Memory Database Systems
Henry Liu, Justin Kaashoek, Siye Zhu
S F
S F
S F
S F
S F
S F
● With serial logging, each transaction needs an LSN from the Global LSN
● Quickly becomes bottleneck with large number of transactions
Global LSNTxn
Global LSN
Txn
Txn
Txn
Txn
Txn
LSN
Txn
Txn
Txn
Batch logging provides one solution to this problem by having multiple loggers with multiple local LSNs instead of one global LSN. This removes the bottleneck
Global LSN
Txn
Txn
Txn
Txn
Txn
LSN
LSN+1
LSN+2
LSN+3
LSN+4
Txn
Txn
Txn
Txn
Txn
Local LSN 3
Local LSN 4
Local LSN 5
Local LSN 2
Local LSN 1LSN
LSN
LSN
LSN
LSN
Serial Logging Batch Logging
● Assume dependencies between loggers
● Sync before returning to user● Our implementation: Flush all
loggers when one becomes full● Drawback: high latency
Batch Logging
time
Log 1 Log 2 Log 3
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3
Transaction TimeOperations
time
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3
Transaction TimeOperationsLog TimeWait TimeCommit
time
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3
time
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Read(A)
Logging
Txn 2
Read(A)
Write(B)
Logging
Independent Logging?
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Read(A)
Logging
Txn 2
Read(A)
Write(B)
Logging
YESIndependent Logging?
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Write(A)
Logging
Txn 2
Read(A)
Write(B)
Logging
YESIndependent Logging?
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Write(A)
Logging
Txn 2
Read(A)
Write(B)
Logging
YESIndependent Logging?
No
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Write(A)
Logging
Txn 2
Write(A)
Write(B)
Logging
YESIndependent Logging?
No
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Write(A)
Logging
Txn 2
Write(A)
Write(B)
Logging
YESIndependent Logging?
NoNo
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Read(A)
Logging
Txn 2
Write(A)
Write(B)
Logging
YESIndependent Logging?
NoNo
• No dependency
• RAW (Read After Write)
• WAW (Write After Write)
• WAR (Write After Read)
…
Txn 1
Read(A)
Logging
Txn 2
Write(A)
Write(B)
Logging
YESIndependent Logging?
NoNo
YES
LSN Txn ID Data Tuple Dependency info
Logger 1
Wait Buffer 1
Logger 2
Wait Buffer 2
Parallel Logger
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3
Transactionstim
e
LSN Txn ID Data Tuple Dependency info
Logger 1
1 1 C [0, 0]
Wait Buffer 1
11 3A
[1,1]B
Logger 2
1 2 B [0,0]
Wait Buffer 2
Parallel Logger
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3
Transactionstim
e
LSN Txn ID Data Tuple Dependency info
Logger 1
1 1 C [0, 0]
11 3A
[1,1]B
Wait Buffer 1
Logger 2
1 2 B [0,0]
Wait Buffer 2
Parallel Logger
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Txn 1Txn 2
Txn 3
Transactionstim
e
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)Write(A)
Write(B)
Read(C)
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)
Write(A)
Write(B)
Read(C)
Txn 1 Txn 2 Txn 3
Read(A)
Read(B)
Write(C)
Read(A)
Write(B)
Write(A)
Write(B)
Read(C)
Txn 1 Txn 2 Txn 3 Txn 1 Txn 2 Txn 3
time
○○○
○○○○
