![Page 1: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/1.jpg)
SILTA Memory-Efficient, High-Performance Key-
Value Store
Presenter:Andrzej Findeisen
Based on paper of H. Lim, B. Fan, D.G. Andersen, and M. Kaminsky:“SILT: A Memory-Efficient, High-Performance Key-Value Store”in Proceedings ACM SOSP 2011
![Page 2: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/2.jpg)
Introduction
![Page 3: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/3.jpg)
Abstract
● SILT = Small Index Large Table● System for storing key-value pairs● Extremely efficient in terms of memory and
flash disk usage– 0.7 bytes of DRAM per entry
– 1.01 flash reads per entry retrieval
– Writing optimized for flash specific characteristics
● Assumption: uniformly distributed 160-bit keys
![Page 4: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/4.jpg)
Rationale
Moreover DRAM is:● 8 times more expensive than flash disk● 25 times power consuming than flash disk
![Page 5: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/5.jpg)
Comparison
![Page 6: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/6.jpg)
Goals
● Low read amplification:Issue at most 1 + ε flash reads for a single GET where ε is configurable and small (e.g. 0.01).
● Controllable write amplification and favoring sequential writes:The system should issue flash-friendly, large writes.
● Memory-efficient indexing:SILT should use as little memory as possible.
● Computation-efficient indexing:SILT’s indexes should be fast enough to let the system saturate the flash I/O.
● Effective use of flash space:Some data layout options use the flash space more sparsely to improve lookup or insertion speed, but the total space overhead of any such choice should remain small – less than 20% or so.
![Page 7: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/7.jpg)
Design
![Page 8: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/8.jpg)
Architecture
![Page 9: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/9.jpg)
LogStore
![Page 10: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/10.jpg)
HashStore
![Page 11: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/11.jpg)
SortedStore I
![Page 12: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/12.jpg)
SortedStore IITrie representation
● Repr(T) := |L| Repr(L) Repr(R)● Entropy coding● Worst case: scanning whole
trie● Therefore 2k buckets are
introduced (ex. k = 10 for 216 items)
![Page 13: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/13.jpg)
Performance
![Page 14: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/14.jpg)
Overheads
![Page 15: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/15.jpg)
Flash disk usage
● Data written to flash in bigger chunks● Example
– 256 GB disk, 10k erase cycles, 100M items, 4 SILT instances, 7.5M entries to merge, 1KiB entries, 5k updates per second
– WA = 5.4, 31 of HashStores (RA = 1.008), 102.4GB of data, 73MB of DRAM
– Device will last 3 years
![Page 16: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/16.jpg)
Memory usage and throughput
![Page 17: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/17.jpg)
Latency
![Page 18: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/18.jpg)
Throughput
![Page 19: SILT - mimuw.edu.pliwanicki/courses/ds/2012/... · SILT A Memory-Efficient, High-Performance Key-Value Store Presenter: Andrzej Findeisen Based on paper of H. Lim, B. Fan, D.G. Andersen,](https://reader035.vdocuments.mx/reader035/viewer/2022071218/605291216789da21277adf90/html5/thumbnails/19.jpg)
Thank you
Q&A