towards automating the configuration of a distributed storage system lauro b. costa matei ripeanu...

Towards Automating the Configuration of a

Distributed Storage System

Lauro B. Costa Matei Ripeanu{lauroc, matei}@ece.ubc.ca

NetSysLabUniversity of British Columbia

2

Distributed File Systems

Different workloads:Read/write only, high data similarity

Different optimizations: Temp local storage, deduplication,

replication

One size fits all: each choice may be the optimal for a specific workload, not for all

3

Configurable OptimizationsMosaStore1 and UrsaMinor2 propose file systems with configurable optimizations

User must choose the optimizations

1The Case for a Versatile Storage System, S. Al-Kiswany, A. Gharaibeh, M. Ripeanu, SOSP Workshop on Hot Topics in Storage and File Systems (HotStorage’09)2 Ursa Minor: versatile cluster-based storage. M. Abd-El-Malek, W. V. Courtright et al. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies - FAST’05

4

Tuning File System

Indentify the system parametersDefine a target metricdo

Define a target value

Configure the parameters

Measure and analyze the performance

while not satisfied

5

Tuning is Hard

Defining metrics and target values can be complex

Lack of knowledge of distributed systems, application or application’s workloads

Workload or infrastructure can change

Tuning is time-consuming

6

Deduplication: Detecting Similarity

Only the firstblock is different

File AX

Y

Z

BlocksHashing

AAAA

BBBB

CCCC

File B

W

Y

Z

HashingBlocks

DDDD

BBBB

CCCC

7

Deduplication for Checkpointing?Checkpointing applications write multiple snapshotsSuccessive snapshots may have high data similarity

Similarity depends on number of factors, e.g.:

– Process or application level– Frequency of checkpointing

8

How can we configure the file system parameters (optimizations) with minimal human intervention?

9

Agenda

Motivation: Configurable file systemsArchitecture to automatically configure a

FSFirst Case: Checkpointing applicationsImplementationEvaluationSummary and Future Work

10

Requirements

Be easy to configureMinimal human intervention

Be able to choose a satisfactory performancePerformance close to administrator’s intention

Have a reasonable automated configuration costOverhead small enough to make sense to use

11

Loop for Automated ConfigurationApplication

Distributed Storage System

Monitor

Actuator

Controller

PredictorUtility

Function

Metrics Metrics

Best Configuration

Change Configuration

Data

12

Controller

Utility function captures the metrics’ utility

It is simple for one target metric, e.g. timeIt reduces several target metrics to just one

dimension

Predictor estimates how a change affects the target metrics

Controller

PredictorUtility

Function

13

Controller

PredictorUtility

Function

Controller decides the configuration by comparing the utility of

current and predicted metrics

14

Agenda

Motivation: Configurable file systemsArchitecture to automatically configure a

FSFirst Case: Checkpointing applicationsImplementationEvaluationConclusionsFuture Work

15

Data Deduplication

Can save storage space and network bandwidth

Has high computational cost to hash data

Mechanism to choose among two options:data deduplication on or off

16

Control Loop for DeduplicationMetrics: time spent and storage space

Keep history for writes:– total time– number of blocks received– number of blocks similar

17

Utility Function Controller

PredictorUtility

Function

Administrator gives weights to capture the relative importance, e.g.:

– 1 x time + 0 x storage– 0.5 x time + 0.5 x storage

18

Predictor Controller

PredictorUtility

Function

Space

Time

No deduplication

Deduplication

number of blocks

I/O operations

consider similarity

+ time for hashing

data

19

Evaluation

Three aspects:– Effort to configure– Performance– Overhead

Experimental setup10 storage nodes, 1Gbps NICs

20

Evaluation: Workload

Synthetic

Similarity variedFor each similarity level, write 100 snapshots

Similar results for several snapshot sizes32, 64, 128, 256, 512 MBPlots for 256MB

21

Effort to Configure

Small effortAdministrator specifies the weights for each

metric

No effort in the default caseSystem optimizes for time

22

Optimizing for Time

23

Optimizing for Time

Hashing cost paid off by savings with I/O operations

24

Optimizing for Time

25

Overhead

Memoryless than 1KB

ComputationalLow similarity - within 5% in evaluated

casesHigh similarity – negligible

26

Summary

Initial study on automatically configuring a file system

Data deduplication configured properly with low overheads

27

Future Work

More parameters for similarity detectionvariable block boundary, block sizes, offload to

GPU

Constraints for utility functionse.g., best time for a maximum storage space

More optimizations and metricsreplication, buffer size, caching policiesreliability, energy

28

29

Mixing time and storage space

30

Application

Distributed Storage System

Monitor

Actuator

time, # blocks, #similar blocks

Deduplication on or off

Set deduplication

on or off

Data

time, # blocks, #similar blocks

Controller

PredictorUtility

Function

weights for metrics

31

MosaStore Architecture

Metadata Manager

Benefactors(Storage nodes)

Client(FS interface)

...

32

Prototype in MosaStore

Deduplication can be turned on and off on the fly

Write flow collects the measurements

Monitor and Controller are co-located with the client

33

Utility

Utility is a measure of the relative satisfaction: How happy the administrator is

Money is a good proxy, but complicatedFocus on simple cases

- 100% time + 0% space- 50% time + 50% space- Constraint on space, optimize for time

Function cannot use different units: normalize

34

MosaStore Architecture

Storage space aggregated from nodes in a network

Naming scheme: BYHASH, BYSEQ

File creation/write Collects metricIt has the option to activate similarity

detection