Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 102/25/2020
Carver: Finding Important Parameters for Storage System Tuning
18th USENIX Conference on File and Storage Technologies (FAST’20)
Zhen Cao1, Geoff Kuenning2, and Erez Zadok1
1Stony Brook University; 2Harvey Mudd College
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 202/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 302/25/2020
Motivationl Tuning storage system parameters can provide significant
gainsuDefault settings are often sub-optimal
l Storage system parameter space is hugel Manual tuning vs. auto-tuning
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 402/25/2020
Motivation (cont.)l More challenges to solve
u “Curse of dimensionality”uTuning efficiency is critical for storage systems
l Storage parameters come with different “importance”l Parameter selection is a critical part of auto-tuning
systems
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 502/25/2020
Carver Overviewl Measurement for storage parameter importancel Sampling method
uPick a small subset of configurationsl Parameter selection algorithm
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 602/25/2020
Key Contributionsl Quantitative analysis of importance of storage parameters on
system performancel Carver can pick important parameters efficientlyl Evaluated Carver on various datasets
uCarver can find a near-optimal subset of important parameters in all cases
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 702/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 802/25/2020
l Storage systemu File system, underlying storage hardware and any layers between them
l Parametersu Configurable optionsu E.g., block size
l Parameter valuesu E.g., 1K, 2K, 4K (Ext4 block size)
l Configurationu Combination of parameter values u E.g., [Ext4, 4K, data=ordered]
l Parameter spaceu All possible configurations
Concepts
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 902/25/2020
Challengesl Large parameter space
u Ext4: 59 parameters, 1037 configsu Distributed, large-scale
l Evaluation is time-consumingl Nontransferable tuning results
u Hardware, workload, etc.l Discrete and non-numeric
u Linux I/O scheduler: noop, cfq, deadline
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1002/25/2020
Dimensionality Reductionl Feature extraction
uProject high-dimensional data into low-dimensional spacesuConstructed featuresuLose physical meaning of original features
l Feature selectionuDirectly selects a subset of features from the original onesuRelationship between parametersu Impact of parameters on target variable
§ Throughput
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1102/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1202/25/2020
Carver: Measuring Parameter Importancel Categorical parameters & continuous target variablel Parameter Importance
uReduction in variance
l Conditional parameter importanceuCPI(P2 | P1 = p)uE.g., given P1 is fixed to “p”, find importance of P2
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1302/25/2020
Carver: Samplingl Limit the number of evaluations
uEvaluation of storage configurations is costlyl Sampling
uLatin Hypercube Sampling§ Latin square: one sample in each row and each column§ Generalization to higher dimensions§ Proved useful in system analysis
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1402/25/2020
Carver: Parameter Selection (1)
PI(X1) PI(X2) … PI(Xn)
Most Important Parameter
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1502/25/2020
Carver: Parameter Selection (2)
2nd Most Important Parameter
(Assume X1 is as most important parameter.)
…CPI(X2|X1=y1) CPI(X2|X1=yk)…
CPI(X2|X1)
MaxCPI(Xn|X1=y1) CPI(Xn|X1=yk)…
CPI(Xn|X1)
Max
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1602/25/2020
Carver: Parameter Selection (3)l Repeat the process until stopping criterion is metl Example stopping criteria:
uCertain number of parameters have been selecteduVariance within subsets of configurations (fixing values of selected
parameters) falls below a certain threshold.
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1702/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1802/25/2020
Evaluation: Testbedl Hardware
u Intel Xeon quad-core 2.4GHz CPU, 24G RAM, 4 drives (SAS-HDD 500GB, SAS-HDD 146GB, 1 SATA-HDD, SSD)
l FilebenchuWorkloads: fileserver, mailserver, webserver, dbserver
l Two Settings:uS1: default working set size uS2: 4G RAM, 10G working set size
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1902/25/2020
Evaluation: Settingsl Parameters spaces
u7 file system types, 4 workloadsu inode size, block size, block group, journal options, mount options,
special options, I/O scheduler, etc.l Methodology
uPre-collect benchmark results for all configurations§ 3+ runs for each configuration§ 500,000 data points
uEvaluate Carver on collected datasets
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2002/25/2020
Evaluation: Parameter Importance
0
3*107
PI
PI
0
5*105
BlockSize
InodeSize
BlockGroup
AtimeOption
JournalOption
SpecialOption
I/OSchd.
Dev.
PI
CPI (X | journal)
CP
I
Ext4, S1, Fileserver-default
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2102/25/2020
Setting Workload F/S #1 #2 #3 #4
S2 fileserver-10gb Ext4 Journal Option I/O Scheduler Inode Size -
S2 dbserver-10gb Ext4 Block Size Inode Size I/O Scheduler Journal Option
S2 webserver-10gb Ext4 Inode Size Flex BG Block Size Journal Option
S1 fileserver-def Btrfs Special Option Node Size Device -
S1 webserver-def Btrfs - - - -
Parameter Importance Summary
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2202/25/2020
Evaluation: Carver
0
20
40
60
80
100
0.1 0.2 0.4 0.8 1.6 3.2 6.4
10 20 40 80 160 320 640
Per
centa
ge
of
Runs
(%)
Percentage of Dataset (%)
Running Time (min)
#1#2#3
Ext4, S1, Fileserver-default
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2302/25/2020
Evaluation: Carver (Cont.)
0
20
40
60
80
100
4 8 16 32
20 40 80P
erce
nta
ge
of
Runs
(%)
Percentage of Dataset (%)
Running Time (min)
#1#2#3
Btrfs, S1, Fileserver-default
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2402/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2502/25/2020
Related Workl Parameter selection for systems
uLasso [Aken et al.]uAdaptive Sampling [Duan et al.]uProbabilistic Reasoning [Sullivan et al.]
l Auto-tuning storage systemsuGenetic Algorithms, Deep Q-Networks, etc.uComparative Study [Cao et al.]
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2602/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2702/25/2020
Conclusionsl Proposed Carver for efficiently selecting storage parametersl Provided thorough study of storage-parameter importancel Demonstrated Carver’s efficiency by testing it on small
fractions of the configuration space
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2802/25/2020
Future Workl Extend Carver to support other parameter-selection
techniquesuGroup Lasso, ANOVA, etc.
l Evaluate Carver with more optimization objectives and larger storage-parameter spaces
l Extend Carver’s parameter selection algorithm into multi-objective optimization
l Integrate Carver with auto-tuning algorithms
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2902/25/2020
Zhen Cao, Geoff Kuenning, and Erez Zadok
Carver: Finding Important Parameters for Storage System Tuning