hope for the best, expect the worst - cornell university · 35 conclusions concept of...
TRANSCRIPT
1
Hope for the Best, Expect the Worstor what happens when
E[ f(good event) ] > E[ f(bad event) ]
Lukas KrocOctober 12, 2006
2
Outline● Overview of file systems● The basic idea: speculation● Applying the idea to file systems:
– local file systems– distributed file systems
● Implementation issues– performance results
● Conclusion
3
File Systems: What They Are● allow and organize access to data
– operations: create, write, read, delete● physical scenarios:
– local file systems– distributed file systems
● goal:– provide durability and performance given physical
limitations (latency, bandwidth)● consistency added for distributed systems
4
File Systems: How They Work
stolen from Paul Francis' CS414 lecture notes
5
File Systems: How They Work
stolen from Paul Francis' CS414 lecture notes
6
Main Issues● a trade-off between durability and performance
– durability calls for immediate access to the medium● synchronous access
– performance calls for caching● asynchronous access
● file system speedups:– local: use memory cache and disk buffer to delay
access– distributed: cache fetched files on clients
7
Papers for Discussion● Nightingale et al: Speculative Execution in a
Distributed File System (SOSP'05)– new way of dealing with issues of distributed file
system● Nightingale et al: Rethink the Sync (OSDI'06)
– applies ideas from above to issues of local file systems
● same basic idea, different scenarios– will reverse the order of presentation, easier first
8
Basic Idea
“Expect the best, be prepared for the worst”
● best = no power failure, cached data is valid● worst = power fails, cached data is invalid● prepared = able to recover a consistent state
after a bad event happened● expect = speculate that it will happen
9
Conditions for the Basic Idea to Work● highly predictable results of speculations
– crash will most likely not occur in the next 5 seconds
– data in the cache is most likely valid● computers have spare CPU cycles
– to perform “free” speculative computation● local overhead is lower than remote I/O
10
Outline● Overview of file systems● The basic idea: speculation● Applying the idea to file systems:
– local file systems– distributed file systems
● Implementation issues– performance results
● Conclusion
11
Local File Systems:Traditional Approach (ext3)
● i-node based● added journaling for
increased durability– meta-data only for
performance reasons
● 2 modes of operation:– synchronous: system call return only after done– asynchronous: system call returns immediately
stolen from Paul Francis' CS414 lecture notes
12
Problems of Traditional Approach
● synchronous mode:– durable (but only if using write barriers, or with disk
buffer disabled), but very slow● asynchronous mode:
– not durable, but fast
13
Local File Systems:New Approach
● shift of paradigm: don't promise anything to the application, promise it to the user– the promise = synchronous guarantees– the user = any external entity observing the process
⇒ external synchrony– asynchronous internal workings, synchronous
external guarantees– combines performance and durability benefits of
both
14
External Synchrony● Idea:
– speculate that everything will be properly written to disk
● Overview:– immediately return from write call (asynchrony)– buffer all external output of the application until the
write successfully happens– if write fails, discard the buffers
● Result:– better guarantees AND performance than ext3
15
External Synchrony: Schema
16
External Synchrony: Performance
17
Distributed File Systems:Traditional Approach (NFS)
● client-server approach● synchronous I/O
operations required for coherence– using RPC
● offers close-to-open consistency– weaker than local file
systems
18
Problems of Traditional Approach
● at least 2 round-trip-times required per close– very slow
● close-to-open consistency isn't very good– for how slow it is
19
Distributed File Systems:New Approach
● Idea:– speculate that close is successful, that a cached
data is valid....● Overview:
– use asynchronous RPCs, immediately returning– checkpoint the application (store its state) and
buffer all subsequent output– on success: output buffers, on failure: roll-back
● Result:– better guarantee AND performance than NFS
20
Speculative NFS: Schema
21
Speculative NFS: Performance
22
Overview of the Technique
Speculate on...power failure not occurring, cache being valid
...by means of...buffering externalized output, checkpointing the
process...in order to...
improve performance, increase consistency
23
Outline● Overview of file systems● The basic idea: speculation● Applying the idea to file systems:
– local file systems– distributed file systems
● Implementation issues– performance results
● Conclusion
24
Implementation:Buffering Externalized Output
● any kernel object with commit dependencies is uncommitted– any process that accesses uncommitted object is
marked uncommitted, and vice versa– any external output of such process is buffered by
kernel– logs are used to track dependencies
● once commit dependencies are removed, the buffers are output to external devices– also allows to group commits
25
Buffering Externalized Output (1)
26
Buffering Externalized Output (2)
27
Result: xsyncfs● adapted ext3 file system to use external
synchrony– internally works asynchronously, but looks
synchronous● commits journal transaction when:
– journal space exhausted, journal old....– user calls fsync()– output-triggered by buffered output
● adapts for throughput/latency optimization
28
xsyncfs: PerformancePostMark benchmark Apache build
29
Implementation:Checkpointing a Process
● checkpoint: a state-image of a process– copy-on-write fork of the process– not placed on the run queue
● output of the running processed buffered while the process is speculative (with a checkpoint)
● depending on the result of the speculation:– success: the checkpoint is discarded– failed: process terminated and checkpoint assumes
its identity and placed on the run queue
30
Propagating Causal Dependencies
31
Result: SpecNFS● preserves existing NFS semantics
– including close-to-open consistency● offers much better performance than NFS
● implemented using the same RPCs– but in an asynchronous, speculative manner
● follows the external-synchrony paradigm– what is observed has been committed
32
Result: BlueFS● strong consistency and safety guarantees
– single-copy file semantics (shared local disk)● still good performance
– still outperforms NFS
● prior to read/write, cached versions are speculated to be valid– in case of access conflict, roll-back occurs
33
SpecNFS & BlueFS: PerformancePostMark benchmark Apache build
34
SpecNFS & BlueFS: PerformanceApache build
35
Conclusions● Concept of speculation/roll-back introduced
– known in fault tolerance research already– applicable to general I/O issues– “Expecting the best, being prepared for the worst”
● Might help resolve the tension between performance and durability in file systems– not “proven by time” yet, but looks good
● The idea is applicable in a broader context– distributed simulations, processor cache warm-up