carver: finding important parameters for storage system tuning · evaluation: carver 0 20 40 60 80...
TRANSCRIPT
![Page 1: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/1.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 102/25/2020
Carver: Finding Important Parameters for Storage System Tuning
18th USENIX Conference on File and Storage Technologies (FAST’20)
Zhen Cao1, Geoff Kuenning2, and Erez Zadok1
1Stony Brook University; 2Harvey Mudd College
![Page 2: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/2.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 202/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
![Page 3: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/3.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 302/25/2020
Motivationl Tuning storage system parameters can provide significant
gainsuDefault settings are often sub-optimal
l Storage system parameter space is hugel Manual tuning vs. auto-tuning
![Page 4: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/4.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 402/25/2020
Motivation (cont.)l More challenges to solve
u “Curse of dimensionality”uTuning efficiency is critical for storage systems
l Storage parameters come with different “importance”l Parameter selection is a critical part of auto-tuning
systems
![Page 5: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/5.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 502/25/2020
Carver Overviewl Measurement for storage parameter importancel Sampling method
uPick a small subset of configurationsl Parameter selection algorithm
![Page 6: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/6.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 602/25/2020
Key Contributionsl Quantitative analysis of importance of storage parameters on
system performancel Carver can pick important parameters efficientlyl Evaluated Carver on various datasets
uCarver can find a near-optimal subset of important parameters in all cases
![Page 7: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/7.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 702/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
![Page 8: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/8.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 802/25/2020
l Storage systemu File system, underlying storage hardware and any layers between them
l Parametersu Configurable optionsu E.g., block size
l Parameter valuesu E.g., 1K, 2K, 4K (Ext4 block size)
l Configurationu Combination of parameter values u E.g., [Ext4, 4K, data=ordered]
l Parameter spaceu All possible configurations
Concepts
![Page 9: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/9.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 902/25/2020
Challengesl Large parameter space
u Ext4: 59 parameters, 1037 configsu Distributed, large-scale
l Evaluation is time-consumingl Nontransferable tuning results
u Hardware, workload, etc.l Discrete and non-numeric
u Linux I/O scheduler: noop, cfq, deadline
![Page 10: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/10.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1002/25/2020
Dimensionality Reductionl Feature extraction
uProject high-dimensional data into low-dimensional spacesuConstructed featuresuLose physical meaning of original features
l Feature selectionuDirectly selects a subset of features from the original onesuRelationship between parametersu Impact of parameters on target variable
§ Throughput
![Page 11: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/11.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1102/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
![Page 12: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/12.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1202/25/2020
Carver: Measuring Parameter Importancel Categorical parameters & continuous target variablel Parameter Importance
uReduction in variance
l Conditional parameter importanceuCPI(P2 | P1 = p)uE.g., given P1 is fixed to “p”, find importance of P2
![Page 13: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/13.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1302/25/2020
Carver: Samplingl Limit the number of evaluations
uEvaluation of storage configurations is costlyl Sampling
uLatin Hypercube Sampling§ Latin square: one sample in each row and each column§ Generalization to higher dimensions§ Proved useful in system analysis
![Page 14: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/14.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1402/25/2020
Carver: Parameter Selection (1)
PI(X1) PI(X2) … PI(Xn)
Most Important Parameter
![Page 15: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/15.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1502/25/2020
Carver: Parameter Selection (2)
2nd Most Important Parameter
(Assume X1 is as most important parameter.)
…CPI(X2|X1=y1) CPI(X2|X1=yk)…
CPI(X2|X1)
MaxCPI(Xn|X1=y1) CPI(Xn|X1=yk)…
CPI(Xn|X1)
Max
![Page 16: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/16.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1602/25/2020
Carver: Parameter Selection (3)l Repeat the process until stopping criterion is metl Example stopping criteria:
uCertain number of parameters have been selecteduVariance within subsets of configurations (fixing values of selected
parameters) falls below a certain threshold.
![Page 17: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/17.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1702/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
![Page 18: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/18.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1802/25/2020
Evaluation: Testbedl Hardware
u Intel Xeon quad-core 2.4GHz CPU, 24G RAM, 4 drives (SAS-HDD 500GB, SAS-HDD 146GB, 1 SATA-HDD, SSD)
l FilebenchuWorkloads: fileserver, mailserver, webserver, dbserver
l Two Settings:uS1: default working set size uS2: 4G RAM, 10G working set size
![Page 19: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/19.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 1902/25/2020
Evaluation: Settingsl Parameters spaces
u7 file system types, 4 workloadsu inode size, block size, block group, journal options, mount options,
special options, I/O scheduler, etc.l Methodology
uPre-collect benchmark results for all configurations§ 3+ runs for each configuration§ 500,000 data points
uEvaluate Carver on collected datasets
![Page 20: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/20.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2002/25/2020
Evaluation: Parameter Importance
0
3*107
PI
PI
0
5*105
BlockSize
InodeSize
BlockGroup
AtimeOption
JournalOption
SpecialOption
I/OSchd.
Dev.
PI
CPI (X | journal)
CP
I
Ext4, S1, Fileserver-default
![Page 21: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/21.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2102/25/2020
Setting Workload F/S #1 #2 #3 #4
S2 fileserver-10gb Ext4 Journal Option I/O Scheduler Inode Size -
S2 dbserver-10gb Ext4 Block Size Inode Size I/O Scheduler Journal Option
S2 webserver-10gb Ext4 Inode Size Flex BG Block Size Journal Option
S1 fileserver-def Btrfs Special Option Node Size Device -
S1 webserver-def Btrfs - - - -
Parameter Importance Summary
![Page 22: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/22.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2202/25/2020
Evaluation: Carver
0
20
40
60
80
100
0.1 0.2 0.4 0.8 1.6 3.2 6.4
10 20 40 80 160 320 640
Per
centa
ge
of
Runs
(%)
Percentage of Dataset (%)
Running Time (min)
#1#2#3
Ext4, S1, Fileserver-default
![Page 23: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/23.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2302/25/2020
Evaluation: Carver (Cont.)
0
20
40
60
80
100
4 8 16 32
20 40 80P
erce
nta
ge
of
Runs
(%)
Percentage of Dataset (%)
Running Time (min)
#1#2#3
Btrfs, S1, Fileserver-default
![Page 24: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/24.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2402/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
![Page 25: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/25.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2502/25/2020
Related Workl Parameter selection for systems
uLasso [Aken et al.]uAdaptive Sampling [Duan et al.]uProbabilistic Reasoning [Sullivan et al.]
l Auto-tuning storage systemsuGenetic Algorithms, Deep Q-Networks, etc.uComparative Study [Cao et al.]
![Page 26: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/26.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2602/25/2020
Outlinel Motivationl Backgroundl Design of Carverl Evaluationl Related Workl Conclusions & Future Work
![Page 27: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/27.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2702/25/2020
Conclusionsl Proposed Carver for efficiently selecting storage parametersl Provided thorough study of storage-parameter importancel Demonstrated Carver’s efficiency by testing it on small
fractions of the configuration space
![Page 28: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/28.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2802/25/2020
Future Workl Extend Carver to support other parameter-selection
techniquesuGroup Lasso, ANOVA, etc.
l Evaluate Carver with more optimization objectives and larger storage-parameter spaces
l Extend Carver’s parameter selection algorithm into multi-objective optimization
l Integrate Carver with auto-tuning algorithms
![Page 29: Carver: Finding Important Parameters for Storage System Tuning · Evaluation: Carver 0 20 40 60 80 100 0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 20 40 80 160 320 640 Percentage of Runs (%) Percentage](https://reader034.vdocuments.mx/reader034/viewer/2022051915/6006e7febe7df06ec7087268/html5/thumbnails/29.jpg)
Carver: Finding Important Parameters for Storage System Tuning (FAST’20) 2902/25/2020
Zhen Cao, Geoff Kuenning, and Erez Zadok
Carver: Finding Important Parameters for Storage System Tuning