September 10, 2014 Sam Siewert

SE420 Software Quality Assurance

RAID Backgrounder

Scalable Enterprise File systems Three Types of Media Storage – Direct Attached Storage – e.g. SATA (Serial ATA) – Network Attached Storage – e.g. NFS – Storage Area Networks – e.g. SAS (Serial Attached SCSI), Fiber

Channel

Flash / RAM based SSD Still 10x++ More Costly than Spinning Media – Predictions for Demise of HDDs and RAID? – Cost is the Driver (E.g. < $0.01 / GB tape, < $0.10 / GB HDD,

$1.00 / GB SSD)

Fast Storage is Either SSD, RAID or Hybrid

Sam Siewert 2

Multiple Disk Drives Disk Drives Fail – Like a Light-bulb – MTBF of 100’s of Thousands of Hours [3 to 5 Years at Duty

Cycle] – Difficult to Determine When Failure Might Occur – The Larger the Population, the More Often Failures will be Seen

Disk Drives Have Low Random Access [100 to 200 I/Os per Second] Idea – Write to them in Parallel and Mirror Data to Protect Against HDD Failures (Erasures)

Sam Siewert 3

RAID-10

Sam Siewert 4

A1 A1 A2 A2 A3 A3 A4 A4 A5 A5 A6 A6

RAID-1 Mirror RAID-1 Mirror RAID-1 Mirror

RAID-0 Striping Over RAID-1 Mirrors

A7 A7 A8 A8 A9 A9 A10 A10 A11 A11 A12 A12

A1,A2,A3, … A12

RAID Operates on LBAs/Sectors (Sometimes Files)

SAN/DAS RAID NAS – Filesystem on top of RAID RAID-10, RAID-50, RAID-60 – Stripe Over Mirror Sets – Stripe Over RAID-5 XOR Parity Sets – Stripe Over RAID-6 Reed-Soloman or Double-Parity Encoded Sets

EVEN/ODD Row Diagonal Parity Minimum Density Codes (Liberation) Reed-Solomon Codes

– Generalized Erasure Codes Cauchy Reed-Solomon, LDPC (Low Density Parity Codes), Weaver/Hover MDS (Maximal Distance Separation) – For each Parity Device, Another Level of Fault Tolerance is Provided

– Larger Drives (Multi-terabyte), Larger arrays (100’s of drives), and Cost Reduction are Driving RAID6 and Higher Levels

Sam Siewert 5

RAID5,6 XOR Parity Encoding

MDS Encoding, Can Achieve High Storage Efficiency with N+1: N/(N+1) and N+2: N/(N+2)

Sam Siewert 6

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Stor

age

Effic

ienc

y

Number of Data Devices for 1 XOR or 2 P,Q Encoded Devices

RAID6

RAID5

RAID-50

Sam Siewert 7

A1

RAID-5 Set RAID-5 Set

B1 C1 D1 P(ABCD)

E1 F1 G1 H1 P(EFGH)

I1 J1 P(IJKL) K1 L1 M1 P(MNOP) N1 P1 O1

P(QRST) Q1 R1 S1 T1

A2 B2 C2 D2 P(ABCD)

E2 F2 G2 H2 P(EFGH)

I2 J2 P(IJKL) K2 L2 M2 P(MNOP) N2 P2 O2

P(QRST) Q2 R2 S2 T2

RAID-0 Striping Over RAID-5 Sets

A1,B1,C1,D1,A2,B2,C2,D2,E1,F1,G1,H1,…, Q2,R2,S2,T2

A1

RAID-6 Set RAID-6 Set

B1 C1 D1 P(ABCD)

E1 F1 G1 P(EFGH)

I1 J1 P(IJKL) K1 M1 P(MNOP) N1 O1 P(QRST) Q1 R1 S1

RAID-0 Striping Over RAID-6 Sets

A1,B1,C1,D1,A2,B2,C2,D2,E1,F1,G1,H1,…, Q2,R2,S2,T2

Disk5 Disk1 Disk2 Disk3 Disk4

Q(EFGH)

Disk6

H1 QABCD)

Q(IJKL)

Q(MNOP)

Q(QRST)

L1 P1 T1

A2 B2 C2 D2 P(ABCD)

E2 F2 G2 P(EFGH)

I2 J2 P(IJKL) K2 M2 P(MNOP) N2 O2 P(QRST) Q2 R2 S2

Disk5 Disk1 Disk2 Disk3 Disk4

Q(EFGH)

Disk6

H2 QABCD)

Q(IJKL)

Q(MNOP)

Q(QRST)

L2 P2 T2

RAID-60 (Reed-Solomon Encoding)

RAID is an Erasure Code

RAID-1 is an MDS EC (James Plank, U. of Tennessee)

Sam Siewert 9

Comparison of ECs

Data Devices = n Coding Devices = m Total = m+n Storage Efficiency: R=n/(n+m) – RAID1 2-Way, R=1/(1+1)=50%, MDS=1, Reads 2x Speed-up, 1x

Write – RAID1 3-Way, R=1/(1+2)=33%, MDS=2, 3x Read, 1x Write – RAID10 with 10 sets, R=10/(10+10)=50%, MDS=1, 20x Read, 10x

Write – RAID5 with 3+1 set, R=3/(3+1)=75%, MDS=1, 3x Read (Parity

Check?), RMW Penalty, Striding Issues – RAID6 with 7+2 set, R=5/(5+2)=71%, MDS=2, 5x Read, Reed-

Solomon Encode on Write and RMW Penalty – Beyond RAID6?

Cauchy Reed-Solomon Scales, but Encode, Decode Complexity High Low Density Parity Codes, Simpler, but not MDS

Sam Siewert 10

Read, Modify Write Penalty

Any Update that is Less than the Full RAID5 or RAID6 Set, Requires 1. Read Old Data and Parity – 2 Reads 2. Compute New Parity (From Old & New Data) 3. Write New Parity and New Data – 2 Writes Only Way to Remove Penalty is a Write-Back Cache to Coalesce Updates and Perform Full-Set Writes Always

Sam Siewert 11

A1

RAID-5 Set

B1 C1 D1 P(ABCD)

E1 F1 G1 H1 P(EFGH)

I1 J1 P(IJKL) K1 L1 M1 P(MNOP) N1 P1 O1

P(QRST) Q1 R1 S1 T1

Write A1 P(ABCD)new=A1new xor A1 xor P(ABCD) A1 B1 C1 D1 P(ABCD) 0 0 0 0 0 0 0 0 1 1 0 0 1 0 1 0 0 1 1 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 0 …

Hands-On Coding Exercise(s) Examples-RAID-Unit-Test, stripetest.c

Sam Siewert 12

A B C D XOR

XOR[A,B,C,D] A,B,C,D Strips

[siewerts@localhost Examples-RAID-Unit-Test]$ ./stripetest Baby-Musk-Ox.ppm Baby-Musk-Ox.ppm.replicated read full stripe … hit end of file FINISHED [siewerts@localhost Examples-RAID-Unit-Test]$ [siewerts@localhost Examples-RAID-Unit-Test]$ diff Baby-Musk-Ox.ppm Baby-Musk-Ox.ppm.replicated