fast: a framework for flash-aware spatial trees
DESCRIPTION
FAST: A Framework for Flash-Aware Spatial Trees. Mohamed Sarwat , Mohamed Mokbel, Xun Zhou Department of Computer Science and Engineering University of Minnesota Suman Nath Microsoft Research. Flash Memory: Already exists. Mobile Devices. Enterprise Servers. - PowerPoint PPT PresentationTRANSCRIPT
University of Minnesota
FAST: A Framework for Flash-Aware Spatial Trees
Mohamed Sarwat, Mohamed Mokbel, Xun ZhouDepartment of Computer Science and Engineering
University of Minnesota
Suman NathMicrosoft Research
2
Flash Memory: Already exists
• Mobile Devices
• Enterprise Servers
3
Flash Memory Storage: Pros
Flash Memory Hard Disk
Electronic Device Mechanical Device
Fast Read Performance Slow Read Performance
High Shock Resistance Low Shock Resistance
Low Power Consumption High Power Consumption
Small Size and Light Weight Relatively Large Size and Heavy Weight
Flash Memory is Topnotch !
4
Flash Memory Storage: Cons
Erase-Before-Update Property Erase latency 1500 microsecondsRead latency 25 microsecondsWrite latency 200 microseconds
Limited Number of Erases per Block100,000 to 1000,000
5
DBMS Tree Indexes & Storage
All Database indexing algorithms were built with the implicit assumption that the underlying secondary storage is the hard disk.
Making Tree Index Structures Flash-Aware !
6
Existing Flash-Aware Trees
■ Devesh Agrawal, Deepak Ganesan, Ramesh Sitaraman, Yanlei Diao, and Shashi Singh. Lazy-Adaptive Tree: An Optimized Index Structure for Flash Devices. In VLDB 2009 Need to build a new kind of tree indexing. Not very practical..!!
■ Yinan Li, Bingsheng He, Robin Jun Yang, Qiong Luo, Ke Yi. Tree Indexing on Flash Disks. In ICDE 2009 (short), VLDB 2010(full) Need to build a new kind of tree indexing. Not very practical..!!
7
Existing Flash-Aware Trees
■ Chin-Hsien Wu, Li-Pin Chang, Tei-Wei Kuo. An Efficient R-Tree Implementation over Flash-Memory Storage Systems. In GIS 2003 Designed only for R-tree. No support for recovery
■ Chin-Hsien Wu, Li-Pin Chang, and Tei-Wei Kuo. An Efficient B-Tree Layer for Flash-Memory Storage Systems. In TECS 2007 Designed only for B-tree. No support for recovery
8
We need a Framework
1) Generic: One Framework works for all
2) Efficient: No need to throw your already existing index structure away
3) Durable: Designed with database systems in mind (recovery)
9
FAST is a Generic and Efficient Framework for
Flash-Aware Search Trees in database systems
FAST
10
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
11
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
12
Update Operation (Insertion/Deletion)
Main Memory Flash Memory
Operation Log
…….
Tree Modifications Table
Tree Index Structure
Insert (O1, MBR ….) || Nodes 1,2 Step 1
Step 2NodeNode 1
Node 2
MBR1 - value
Obj1 - Inserted
Example: Insert (Obj1) in R-tree
Step 3
13
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
14
Search Operation (Find/Range)
Main Memory Flash Memory
Operation Log
…….
Tree Modifications Table
Tree Index Structure
Insert (O1, MBR ….) || Nodes 1,2
Step1
Step 2
Node
Node 1
Node 2
MBR1 - value
Obj1 - Inserted
Example: Range query in R-tree
Step 4
Query Result
Step 5
Step 3
15
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
16
FlushingMain Memory Flash Memory
FAST Log File
Tree Index Structure
Insert (O1, MBR) || Nodes 1,2
MBR1 - value
Node 1 2 3 4 5
Dirty 00 1
Flush 2,3,4,5
75 %
Nodes 2,3,4,5 flushed
Same Block
129
12144
17185
Node 1
2 3 4 5
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
1 10 0 0
Delete (O2, MBR) || Nodes 12
Insert (O4, MBR) || Nodes 1,4,14
Insert (O5, MBR) || Nodes 4,17
Insert (O4, MBR) || Nodes 1,5,18
Step 1
Step 2
Step 4
Step 3
Step 5
17
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
18
Crash RecoveryMain Memory Flash Memory
FAST Log File
.....
Tree Index Structure
Insert (O1, MBR) || Nodes 1,2
Nodes 1,2,3,4,5, 14,17 flushed
1
2 3 4 5
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Delete (O2, MBR) || Nodes 12
Insert (O4, MBR) || Nodes 1,4,14
Insert (O5, MBR) || Nodes 4,17
Insert (O4, MBR) || Nodes 1,5,18
Log Maintenance
Compact Log
Insert (O4, MBR) || Nodes 18
Crash Recovery
Restart after crash
Restart after crash
MBR1 - value
129
12144
17185
Node
19
Experimental Evaluation (1/4)
■ We implemented FAST in PostGreSQL
■ We instantiate B-tree and R-tree instances of FAST, termed FAST-Btree and FAST-Rtree
■ We use two synthetic workloads: Lookup intensive workload (WL): 80% search and
20% update Update intensive workload, (WU): 20% search and
80% update.
■ number of workload operations to 10 million■ operations, main memory size to 256 KB ■ Tree index size to 512 MB, and ■ log file size to10 MB (~ 2% of Index size)
20
Experimental Evaluation (2/4)
Memory Size Log File Size
348 lines of code3070 lines of code
308 lines of code
9230 lines of code8150 lines of code
21
Experimental Evaluation (3/4)
Memory Size Log File Size
22
Experimental Evaluation (4/4)
Log Compaction Recovery
23
Conclusion
■ FAST is a generic framework for flash-aware index structures.
■ FAST guarantees the durability of update transactions applied to the underlying index structure.
■ FAST has good performance compared to its counterparts (LA-tree, FD-tree, and RFTL)
24
Questions