hylog: a high performance approach to managing disk layout wenguang wang yanping zhao rick bunt...
Post on 30-Mar-2015
213 Views
Preview:
TRANSCRIPT
HyLog: A High Performance Approach to Managing Disk
Layout
Wenguang Wang Yanping Zhao Rick Bunt
Department of Computer ScienceUniversity of Saskatchewan
Saskatoon, Canada
USENIX FAST 2004April 1, 2004 Wenguang Wang
April 1, 2004 USENIX FAST 2004 Wenguang Wang 2
Background
• The write performance of a storage system is impacted by– the disk characteristics
• Disk positioning time• Transfer bandwidth
– the strategy for writing• Overwrite• LFS (Log-structured File System)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 3
Overwrite
• Idea: new data are overwritten on top of old data
• Problems: lots of time lost in disk arm positioning in workloads with small writes scattered over the disk
April 1, 2004 USENIX FAST 2004 Wenguang Wang 4
LFS
• Idea: new data are accumulated and written to new disk locations in large sequential transfers
• Assumptions of the disk characteristics: large sequential transfers are more efficient than small block transfers
April 1, 2004 USENIX FAST 2004 Wenguang Wang 5
LFS (cont.)
• Advantages– good write performance– no small write penalty on RAID-5– fast recovery– easy to support snapshot and versioning
(WAFL)
• Problems: segment cleaning is expensive– For a year 1991 disk, TPC-B workload, and 50%
disk space utilization, cleaning overhead reduces overall system throughput by 33% (Seltzer et al. USENIX’95)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 6
Motivation
• Observation: disk sequential transfer bandwidth has improved 10x more than positioning time
• Question: how are Overwrite and LFS affected by this trend?
DEC RZ26(year 1991)
Cheetah X15 36LP
(year 2003)Diff.
Positioning time
15ms 5.6ms 2.7x
Transfer B/w 2.3MB/s 61MB/s 27x
April 1, 2004 USENIX FAST 2004 Wenguang Wang 7
Objective
• Revisit the performance of LFS under modern and future disks
• Evaluate the performance of LFS under disk arrays and concurrent users
• Attempt to perform better than LFS and Overwrite
April 1, 2004 USENIX FAST 2004 Wenguang Wang 8
Outline
• Background, Motivation, and Objective
• The analysis of LFS and Overwrite• The design of HyLog• Experimental methodology and
results• Conclusions and future work
April 1, 2004 USENIX FAST 2004 Wenguang Wang 9
Experimental Parameters
• Three SCSI disks– DEC RZ26 (year 1991)– Quantum atlas10k (year 1999)– Cheetah X15 36LP (year 2003)
• Page size: 8KB• Workload: uniformly distributed
random update (TPC-B)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 10
• In Overwrite:
• In LFS:
• T1 > T2
• Segment I/O Efficiency = T1 / T2
Modeling Write Performance
Time to write N pages is T1
Time to write a segment containing N pages is T2
April 1, 2004 USENIX FAST 2004 Wenguang Wang 11
A Simple Scenario
• Assume the segments to be cleaned are always 80% utilized (cleaning space utilization = 80%)
• LFS requires 5 seg. reads and 4 seg. writes to reclaim a free segment
• LFS requires 10 seg. I/Os (9 seg. for cleaning, 1 seg. for new data) to write a segment
• If Segment I/O Efficiency > 10, LFS is still faster than Overwrite!
April 1, 2004 USENIX FAST 2004 Wenguang Wang 12
Segment I/O Efficiency
0
5
10
15
20
25
30
35
40
45
0 500 1000 1500 2000 2500 3000 3500 4000
Seg
men
t I/O
Eff
icie
ncy
Segment Size (KB)
Cheetah X15 36LP (year 2003)
Quantum atlas10k, (year 1999)
DEC RZ26, (year 1991)
33
April 1, 2004 USENIX FAST 2004 Wenguang Wang 13
0
5
10
15
20
25
30
35
0 0.2 0.4 0.6 0.8 1
Wri
te C
ost
Cleaning Space Utilization
Overwrite vs. LFS
Overwrite
LFScleaning
LFS hole-plugging
1999 disk
0.88
April 1, 2004 USENIX FAST 2004 Wenguang Wang 14
Overwrite vs. LFS
• The crossing point where LFS has the same performance as Overwrite
Year of DiskCleaning
Space Utilization
Disk Space Utilization
1991 0.52 0.74
1999 0.88 0.94
2003 0.94 0.97
April 1, 2004 USENIX FAST 2004 Wenguang Wang 15
Disk Access Characteristics
• In most workloads, most writes are to a small number of pages (hot pages)
• Impact of skewness on LFS performance– Most of the cleaning cost comes from
cold pages– Most of the good write performance
comes from accumulating the writes to hot pages
April 1, 2004 USENIX FAST 2004 Wenguang Wang 16
HyLog (Hybrid Log-structured
Approach)• Separates the disk into two
partitions: hot partition and cold partition
• Uses log-structured approach to manage the hot partition
• Uses overwrite to manage the cold partition
April 1, 2004 USENIX FAST 2004 Wenguang Wang 17
0
5
10
15
20
25
30
35
0 0.2 0.4 0.6 0.8 1
Proportion of Hot Pages
Disk Space Utilization 97%
0
5
10
15
20
25
30
35
0 0.2 0.4 0.6 0.8 1
Proportion of Hot Pages
Disk Space Utilization 90%
Performance Potential of HyLog
Wri
te
Co
st
Disk: year 2003, workload: 80% references are in 20% pages
Overwrite
LFS
HyLog
Overwrite
LFS
HyLog
April 1, 2004 USENIX FAST 2004 Wenguang Wang 18
Design of HyLog• Key design issue: page separating
algorithm– Collects page write frequencies – Finds the hot page proportion to minimize
expected write cost– Determines the threshold of write
frequency from the desired hot page proportion
– Uses the threshold to distinguish hot pages from cold pages
April 1, 2004 USENIX FAST 2004 Wenguang Wang 19
Evaluation Methodology
• Trace driven simulation– Overwrite, LFS, WOLF, and HyLog are
implemented– TPC-C, Email, and OLTP traces– Year 1999, 2003, and 2008 disk models – No think time between requests
• Metrics– Throughput: # I/O requests finished per
second
April 1, 2004 USENIX FAST 2004 Wenguang Wang 20
HyLog adjusts the hot page proportion between 35-45%
Results – HyLog Page Separating Algorithm
0
2
4
6
8
10
12
14
0 0.2 0.4 0.6 0.8 1
Thro
ughp
ut (x
1000
)
Hot Page Proportion
Fixed Hot Page ProportionHyLog
TPC-C trace with 20 users and 4 disks, 98% disk space utilization
April 1, 2004 USENIX FAST 2004 Wenguang Wang 21
Results – Disk Space Utilization
TPC-C trace with 20 users and 4 disks
0
10
20
30
40
50
60
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Thro
ughp
ut (x
1000
)
Disk Space Utilization
Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)
LFS (atlas10k - year'99 disk)
1999 disk
April 1, 2004 USENIX FAST 2004 Wenguang Wang 22
Results – Disk Type
TPC-C trace with 20 users and 4 disks
0
10
20
30
40
50
60
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Thro
ughp
ut (x
1000
)
Disk Space Utilization
Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)
LFS (atlas10k - year'99 disk)
Overwrite (year’03 disk)
LFS/HyLog (year’03 disk)
1999 disk
2003 disk
April 1, 2004 USENIX FAST 2004 Wenguang Wang 23
Results – Disk Type (cont.)
TPC-C trace with 20 users and 4 disks
0
10
20
30
40
50
60
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
Thro
ughp
ut (x
1000
)
Disk Space Utilization
Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)
LFS (atlas10k - year'99 disk)
Overwrite (year’03 disk)
LFS/HyLog (year’03 disk)
Overwrite (year’08 disk)
LFS/HyLog (year’08 disk)
1999 disk
2003 disk
2008 disk
April 1, 2004 USENIX FAST 2004 Wenguang Wang 24
Results – Number of Users
TPC-C trace, disk space utilization 98%, year 1999 disk
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
5 10 15 20 25 30
Nor
mal
ized
Thr
ough
put
Number of Users
HyLog (8 disks)LFS (8 disks)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 25
Results – Number of Disks
TPC-C trace, disk space utilization 98%, year 1999 disk
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2 4 6 8 10 12 14
Nor
mal
ized
Thr
ough
put
Number of Disks
HyLog (7 users)LFS (7 users)
April 1, 2004 USENIX FAST 2004 Wenguang Wang 26
Results – RAID-5
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
Overwrite LFS WOLF HyLog Overwrite LFS WOLF HyLog
Thr
oug
hp
ut
RAID-0RAID-5
Disk Space Utilization 98%Disk Space Utilization 80%
TPC-C trace, year 1999 disk, 8-disk RAID-0, 9-disk RAID-5
April 1, 2004 USENIX FAST 2004 Wenguang Wang 27
Results – Other Traces
0
1
2
3
4
5
6
Overwrite LFS WOLF HyLog Overwrite LFS WOLF HyLog
Nor
mal
ized
Thr
ough
put
Disk Space Utilization=90%
Disk Space Utilization=98%
OLTP Email
Year 1999 disk
April 1, 2004 USENIX FAST 2004 Wenguang Wang 28
Conclusions
• On modern and future disks, LFS significantly outperforms Overwrite unless the disk space utilization is very high
• HyLog performs comparably to the best of Overwrite, LFS, and WOLF
April 1, 2004 USENIX FAST 2004 Wenguang Wang 29
Future Work
• Add fast recovery support in HyLog– All meta-data are considered as hot
pages
• Stabilize NetBSD LFS implementation and measure its performance
• Implement and evaluate HyLog in NetBSD
April 1, 2004 USENIX FAST 2004 Wenguang Wang 30
April 1, 2004 USENIX FAST 2004 Wenguang Wang 31
Nor
mal
ized
Thr
ough
put
5 10 15 20 25 30
2 4
6 8
10 12
14
0.4 0.6 0.8
1 1.2 1.4 1.6 1.8
Results – # Users and Disks
TPC-C trace, disk space utilization 98%, year 1999 disk
HyLogLFS
Number of Users
Number of Disks
top related