powerpoint presentation · 0 1 2 3 4 5 6 7 8 9 10 1000 25000 write amplifcation for diffferent...
TRANSCRIPT
•
•
•
•
•
•
•
•
•
•
new
new
new
Block ID Reference
Block1
Block2
Block3
…
ApplicationBuffer
Buffered I/O Page Cache
Direct I/O
StorageDevice
•
•
•
•
•
•
•
•
•
•
•
Shard 1
Shard 5
Shard 10
Shard 12
Shard 2
Shard 5
Shard 8
Shard 9
MapReduce Job
•
•
•
Memtable(Skiplist)
File.sst
A-B
L-M
C-D E-F G-K L-M
A-B C-D E-F Y-Z
A-F B-K
C-E
X-Z
A-B
L-M
C-D E-F G-M
L-MA-B C-D E-F Y-Z
A-F B-K
C-E
X-Z
A-B
L-M
C-D E-F G-M
A-B C-D E-F Y-Z
A-F
M-Q
X-Z
B-K
A-B C-D E-F G-M
A-B C-D E-F Y-Z
A-F
M-Q
X-Z
B-KM-Q L-ML-M
MemTable(SkipList)
index
•
•
Operations:
Put A1, Put B1, Put B2, Put B3,
Put A2, Put C1, Put A3
A1
A2
A3
B1
B2
B3
C1
•
•
Operations:
Put A1, Put B1, Put B2, Put B3,
Put A2, Put C1, Put A3
A1
A2
A3
B1
B2
B3
C1
•
•
Operations:
Put A1, Put B1, Put B2, Put B3,
Put A2, Put C1, Put A3
A1
A2
A3
B1
B2
B3
C1
•
•
Operations:
Put A1, Put B1, Put B2, Put B3,
Put A2, Put C1, Put A3
A1
A2
A3
B1
B2
B3
C1
•
•
Operations:
Put A1, Put B1, Put B2, Put B3,
Put A2, Put C1, Put A3
A1
A2
A3
B1
B2
B3
C1
index index
index
Prefix Hint
A
B
C
……
•
•
•
•
•
•
..., Delete(1000), Delete(1001), …, Delete(1999), ...
…, Put(999), Put(1000), Put(1001), Put(1002) …, Put(1999), Put(2000), …
•
DB::DeleteRange()
a b c d e
e
db->DeleteRange(“b”, “e”);
a
•
new
new
new
new
new
new
Key Value
ABC
BBC
BEE
…
newnew
Key Value
ABC
BBC
BEE
…
newnew
Key Value
ABC
BBC
BEE
…
newnew
Key Value
ABC
BBC
BEE
…
0 1 2 3 4 5 6 7 8 9 10
1000
25000
Write Amplifcation for Diffferent Value Sizes using db_bench
BlobDB + Rocksdb (Key+Value) StockRocksDB
•
•
Key Value
A A1
B B1
D D1
Key Value
C C1
D D2
F F1
Key Value
A A1
B B1
D D1
C C1
F F1
Key Value
A A1
B B1
D D1
Key Value
C C1
D D2
F F1
Key Value
D D1
C C1
F F1
if (key < “C”) {return true; // filter out
}return false;
•
•
•
•
•
•
•
lua::RocksLuaCompactionFilterOptions lua_opt;
lua_opt.lua_script = your_lua_script;
rocksdb::Options options;
options.compaction_filter_factory =
std::make_shared<rocksdb::lua::RocksLuaCompactionFilterFactory>(lua_opt);
rocksdb::DB* db;
auto status = rocksdb::DB::Open(options, path_to_db, &db);
•
•
•
•
•
•
