paraid: a gear-shiftingawang/courses/cis6935_s2012/paraid-ppt.pdf · paraid: a gear-shifting...

1

PARAID: A Gear-Shifting

Power-Aware RAID

Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang – Florida St. University

Peter Reiher – University of California, Los Angeles

Geoff Kuenning – Harvey Mudd College

2

Motivation

Energy costs are rising An increasing concern for servers

No longer limited to laptops

Energy consumption of disk drives 24% of the power usage in web servers

27% of electricity cost for data centers

More energy more heat more cooling lower computational density more space more costs

Is it possible to reduce energy consumption without degrading performance while maintaining reliability?

PARAID: A Gear-Shifting Power-Aware RAID

3

Challenges

Energy

Not enough opportunities to spin down RAIDs

Performance

Essential for peak loads

Reliability

Server-class drives are not designed for

frequent power switching


4

Existing Work

Most trade performance for energy

savings directly

e.g. vary speed of disks

Most are simulated results


5

Observations

RAID is configured for peak performance

RAID keeps all drives spinning for light loads

Unused storage capacity

Over-provision of storage capacity

Unused storage can be traded for energy savings

Fluctuating load

Cyclic fluctuation of loads

Infrequent on-off power transitions can be effective


6

Performance vs. Energy

Optimizations

Performance benefits

Realized under heavy loads

Energy benefits

Realized instantaneously

7

Power-Aware RAID

Skewed striping for energy savings

Preserving peak performance

Maintaining reliability

Evaluation

Conclusion

Questions


8

Skewed Striping for Energy Saving

Use over-provisioned spare storage Organized into hierarchical overlapping subsets


RAID

1 2 3 4 5

9


Each set analogous to gears in automobiles


RAID

Gears

1

2

3

1 2 3 4 5

10


Soft states can be reclaimed for space Persist across reboots


RAID

Soft States

Gears

1

2

3

1 2 3 4 5

11


Operate in gear 1

Disks 4 and 5 are powered off


RAID

Gears

1

2

3

1 2 3 4 5

Soft States

12


Approximate the workload

Gear shift into most appropriate gear Minimize the opportunity lost to save power

Energy

( Powered

On Disks )

Workload

( Disk Parallelism )

Conventional RAID PARAID

workload


13


Adapt to cyclic fluctuating workload

Gear shift when gear utilization threshold is met

time

load

utilization threshold

gear shift


14

Preserving Peak Performance

Operate in the highest gear

When the system demands peak performance

Uses the same disk layout

Maximize parallelism within each gear

Load is balanced

Uniform striping pattern

Delay block replication until gear shifts

Capture block writes


15

Maintaining Reliability

Reuse existing RAID levels (RAID-5)

Also used in various gears

Drives have a limited number of power cycles

Ration number of power cycles


16

Maintaining Reliability

Busy disk stay powered on, idle disks stay powered off

Outside disks are role exchanged with middle disks

busy

disks

power

cycled

disks

idle

disks

role exchange

Disk 1

Gear 1

Gear 2

Gear 3

Disk 2 Disk 3 Disk 4 Disk 5 Disk 6


17

File system

RAID

PARAID block mapping

Disk device driver

User space

Linux kernel

Soft RAID

Reliability manager

Load monitor

Gear manager

Admin tool

Logical Component Design


18

Data Layout



Gear 1

RAID-5

(1-4) 8 12 ((1-4),8,12)

16 20 (16,20,_) _

Gear 2

RAID-5

1 2 3 4 (1-4)

5 6 7 (5-8) 8

9 10 (9-12) 11 12

13 (13-16) 14 15 16

(17-20) 17 18 19 20

Resembles the data flow of RAID 1+0

Parity for 5 disks does not work for 4 disks For example, replicated block 12 on disk 3

19

Data Layout



Gear 1

RAID-5

(1-4) 8 12 ((1-4),8,12)

16 20 (16,20,_) _

Gear 2

RAID-5

1 2 3 4 (1-4)

5 6 7 (5-8) 8

9 10 (9-12) 11 12

13 (13-16) 14 15 16

(17-20) 17 18 19 20

Cascading parity updates For example, updating block 8 on disk 5

20

Update Propagation

Up-shift propagation (e.g. shifting from 3 to

5 disks)

Full synchronization

On-demand synchronization

Need to respect block dependency

Downshift propagation

Full synchronization


21

Asymmetric Gear-Shifting Policies

Up-shift (aggressive)

Moving utilization average + moving standard

deviation > utilization threshold

Downshift (conservative)

Modified utilization moving average + moving

standard deviation < utilization threshold

Moving average modified to account for fewer drives

and extra parity updates


22

Implementation

Prototyped in Linux 2.6.5

Open source, software RAID

Implemented block I/O handler, monitor,

disk manager

Implemented user admin tool to configure

device

Updated Raid Tools to recognize PARAID

level


23

Evaluation

Challenges

Prototyping PARAID

Commercial machines

Conceptual barriers

Benchmarks designed to measure peak performance

Trace replay

Time consuming


24

Evaluation

multimeter

USB cable client

server

power

supply

12v & 5v

power lines

power

measurement

probes

SCSI

cable

crossover

cable

Xeon 2.8 Ghz, 512 MB RAM

36.7 GB 15k RPM SCSI

P4 2.8 Ghz, 1 GB RAM

160 GB 7200 RPM SATA

RAID

RAID

RAID

RAID

RAID

BOOT


Measurement framework

25

Evaluation

Three different workloads using two different RAID settings Web trace - RAID level 0 (2-disk gear 1, 5-disk gear 2)

Mostly read activity

Cello99 - RAID level 5 (3-disk gear 1, 5-disk gear 2) I/O-intensive workload with writes

PostMark - RAID level 5 Measure peak performance and gear shifting overhead

Speed up trace playback To match hardware

Explore range of speed up factors and power savings


26

Web Trace

UCLA CS Dept Web Servers (8/11/2006 – 8/14/2006)

File system: ~32 GB (~500k files)

Trace replay: ~95k requests with ~4 GB data (~260 MB unique)


0

0.1

0.2

0.3

0.4

0.5

0.6

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96

hours

GB/hour

27

Web Trace Power Savings

0

10

20

30

40

50

60

0 5 10 15 20 25 30

hours

wattsRAID-0

PARAID-0

0

10

20

30

40

50

60

0 5 10 15 20 25 30

hours

wattsRAID-0

PARAID-0


64x – 60 requests/sec

128x – 120 requests/sec 256x – 240 requests/sec

0

10

20

30

40

50

60

0 5 10 15 20 25 30

hours

wattsRAID-0

PARAID-0

64x - 34%

128x - 28%

256x - 10%

Energy Savings

28

Web Trace Latency

0

0.2

0.4

0.6

0.8

1

1 10 100 1000 10000 100000

msec

RAID-0

PARAID-0

0

0.2

0.4

0.6

0.8

1

1 10 100 1000 10000 100000

msec

RAID-0

PARAID-0


256x

128x 64x

0

0.2

0.4

0.6

0.8

1

1 10 100 1000 10000 100000

msec

RAID-0

PARAID-0

256x - within 2.7%

64x - 240%

80ms vs. 33ms

Overhead

29

Web Trace Bandwidth

-20

30

80

130

180

0 5 10 15 20 25 30

hours

MB/secRAID-0

PARAID-0

-20

30

80

130

180

0 5 10 15 20 25 30

hours

MB/secRAID-0

PARAID-0


256x

128x 64x

0

20

40

60

80

100

120

140

160

180

0 5 10 15 20 25 30

hours

MB/secRAID-0

PARAID-0

256x - within

1.3% in high

gear

Overhead

30

Cello99 Trace

Cello99 Workload

HP Storage Research Labs

50 hours beginning on 9/12/1999

1.5 million requests (12 GB) to 440MB of unique blocks

I/O-intensive with 42% writes


31

Cello99 Power Savings

0

10

20

30

40

50

0 10 20 30 40 50

hours

wattsRAID-5

PARAID-5

0

5

10

15

20

25

30

35

40

45

50

0 10 20 30 40 50

hours

wattsRAID-5

PARAID-5





0

10

20

30

40

50

0 10 20 30 40 50

hours

wattsRAID-5

PARAID-5

32x - 13%

64x - 8.2%

128x - 3.5%

Energy Savings

32

Cello99 Completion Time

0.9

0.92

0.94

0.96

0.98

1

1 10 100 1000 10000 100000

msec

RAID-5

PARAID-5

0.9

0.92

0.94

0.96

0.98

1

1 10 100 1000 10000 100000

msec

RAID-5

PARAID-5


128x

64x 32x

0.9

0.92

0.94

0.96

0.98

1

1 10 100 1000 10000 100000

msec

RAID-5

PARAID-5

32x - 1.8ms,

26% slower

due to time

spent in low

gear

Overhead

33

Cello99 Bandwidth

1

10

100

1000

0 500000 1000000 1500000

request number

MB/secRAID-5

PARAID-5

1

10

100

1000

0 500000 1000000 1500000

requests

MB/secRAID-5

PARAID-5


1

10

100

1000

0 500000 1000000 1500000

request number

MB/secRAID-5

PARAID

64x 32x

128x

Overhead

< 1% degra-

dation during

peak hours

34

PostMark Benchmark

Popular synthetic benchmark

Generates ISP-style workloads

Stresses peak read/write performance of storage

device


35

Postmark Performance


0

50

100

150

200

1K files, 50K trans 20K files, 50K trans 20K files, 100K

trans

seconds

RAID-5 PARAID-5 high gear PARAID-5 low-gear

36

Postmark Power Savings


0

10

20

30

40

50

60

70

80

1 11 21 31 41 51 61 71 81 91 10 111 12 13 14 151 16 171

seconds

watt

s

R A ID 5

PA R A ID

37

Related Work

Pergamum

EERAID

RIMAC

Hibernator

MAID

PDC

BlueFS


38

Future Work

Try more workloads

Optimize PARAID gear configuration

Explore asynchronous update propagation

Speed up recovery

Live testing


39

Lessons Learned

Third version of design, early design too

complicated

Data alignment problems

Difficult to measure system under normal load

Hard to predict workload transformations due to

complex system optimizations

Challenging to match trace environments


40

Conclusion

PARAID reuses standard RAID-levels without

special hardware while decreasing their energy

use by 34%.

Optimized version can save even more energy

Empirical evaluation important


41

Research Theme

Data flow management

Storage

MANETs

Current state

Reminiscent of plumbing industry 200 years

ago

Limited interchangeable parts

Poorly understood interactions

42

Research Areas

Power-Aware RAID

Electric-field-based routing for MANETs

Conquest disk-persistent-RAM hybrid file

system

Optimistic replication

Real-time systems

43

Questions

PARAID: A Gear-Shifting

Power-Aware RAID

Contact

Andy Wang – [email protected]

http://www.cs.fsu.edu/~awang/conquest-2

mailto:[email protected]

44

PARAID Recovery

2.7 times slower than conventional raid

For example, 2 gear PARAID device

First, the soft state must recover

Second, data must be propagated

Third, conventional raid must recover

Recovery not as bad for read intensive

workloads

45

PARAID Gear-Shifting

256x 128x 64x

Number of gear switches 15.2 8.0 2.0

% time spent in low gear 52% 88% 98%

% extra I/Os for update

propagations

0.63% 0.37% 0.21%

128x 64x 32x

Number of gear switches 6.0 5.6 5.4

% time spent in low gear 47% 74% 88%

% extra I/Os for update

propagations

8.0% 15% 21%

Web Trace Gear-Shifting Stats

Cello99 Gear-Shifting Stats

paraid: a gear-shiftingawang/courses/cis6935_s2012/paraid-ppt.pdf · paraid: a gear-shifting...

Documents