296.3:algorithms in the real world
DESCRIPTION
296.3:Algorithms in the Real World. Error Correcting Codes II Cyclic Codes Reed-Solomon Codes. Viewing Messages as Polynomials. A (n, k, n-k+1) code: Consider the polynomial of degree k-1 p(x) = a k-1 x k-1 + L + a 1 x + a 0 Message : (a k-1 , …, a 1 , a 0 ) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/1.jpg)
296.3 Page1
296.3:Algorithms in the Real World
Error Correcting Codes II– Cyclic Codes– Reed-Solomon Codes
![Page 2: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/2.jpg)
296.3 Page2
Viewing Messages as Polynomials
A (n, k, n-k+1) code:Consider the polynomial of degree k-1
p(x) = ak-1 xk-1 + + a1 x + a0
Message: (ak-1, …, a1, a0)
Codeword: (p(y0),p(y1), …, p(yn-1)) for distinct y0,…,yn-1
To keep the p(yi) fixed size, we use yi, ai GF(pr)
To make the yi distinct, n < pr
Unisolvence Theorem: Any subset of size k of (p(y1), p(y2), …, p(yn)) is enough to (uniquely) reconstruct p(x) using polynomial interpolation, e.g., LaGrange’s Formula.
![Page 3: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/3.jpg)
296.3 Page3
Polynomial-Based Code
A (n, k, 2s +1) code:
k 2s
Can detect 2s errorsCan correct s errorsGenerally can correct erasures and errors if
+ 2 2s
n
![Page 4: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/4.jpg)
296.3 Page4
Correcting Errors
Correcting s errors:1. Find k + s symbols that agree on a polynomial
p(x).These must exist since originally k + 2s symbols agreed and only s are in error
2. There are no k + s symbols that agree on the wrong polynomial p’(x)- Any subset of k symbols will define p’(x)- Since at most s out of the k+s symbols are
in error, p’(x) = p(x)
![Page 5: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/5.jpg)
296.3 Page5
A Systematic Code Message: (m0, m1, …, mk-1)
Find polynomial p(x) = ak-1 xk-1 + + a1 x + a0 such that
p(y0)= m0, p(y2)= m2, …, p(yk-1) = m0
Codeword: (m0, m1, …, mk-1, p(yk), p(yk+1), …, p(yn-1))
This has the advantage that if we know there are no errors (e.g., all points lie on the same degree k-1 polynomial), it is trivial to decode.
The version of RS used in practice uses something slightly different.
This will allow us to use the “Parity Check” ideas from linear codes (i.e., HcT = 0?) to quickly test for errors.
![Page 6: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/6.jpg)
296.3 Page6
Reed-Solomon Codes in the Real World
(204,188,17)256 : ITU J.83(A)2
(128,122,7)256 : ITU J.83(B)
(255,223,33)256 : Common in Practice– Note that they are all byte based
(i.e., symbols are from GF(28)).Decoding rate on 1.8GHz Pentium 4:
– (255,251) = 89Mbps– (255,223) = 18Mbps
Dozens of companies sell hardware cores that operate 10x faster (or more)– (204,188,17) = 320Mbps (Altera decoder)
![Page 7: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/7.jpg)
296.3 Page7
Applications of Reed-Solomon Codes
• Storage: CDs, DVDs, “hard drives”,• Wireless: Cell phones, wireless links• Sateline and Space: TV, Mars rover, …• Digital Television: DVD, MPEG2 layover• High Speed Modems: ADSL, DSL, ..
Good at handling burst errors.Other codes are better for random errors.
– e.g., Gallager codes, Turbo codes
![Page 8: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/8.jpg)
296.3 Page8
RS and “burst” errors
They can both correct 1 error, but not 2 random errors.– The Hamming code does this with fewer check bits
However, RS can fix 8 contiguous bit errors in one byte– Much better than lower bound for 8 arbitrary errors
bitscheck 88)7log(881
1log
n
nn
code bits check bits
RS (255, 253, 3)256 2040 16
Hamming (211-1, 211-11-1, 3)2
2047 11
Let’s compare to Hamming Codes (which are “optimal”).
![Page 9: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/9.jpg)
296.3 Page9
Galois Field
GF(23) with irreducible polynomial: x3 + x + 1 = x is a generator
x 010 2
2 x2 100 3
3 x + 1 011 4
4 x2 + x 110 5
5 x2 + x + 1
111 6
6 x2 + 1 101 7
7 1 001 1Will use this as an example.
![Page 10: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/10.jpg)
296.3 Page10
Discrete Fourier Transform (DFT)Evaluating polynomial at n points via matrix multiply: is a primitive nth root of unity (n = 1) – a generator
)1)(1()1(21
)1(242
12
1
1
1
1111
nnnn
n
n
T
0
01
0
1
1
0
k
n
k
k m
m
T
c
c
c
c
cTm 1Inverse DFT:
Evaluate polynomial mk-1xk-1 + + m1x + m0
at n distinct roots of unity, 1, , 2, 3, , n-1
![Page 11: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/11.jpg)
296.3 Page11
DFT Example = x is 7th root of unity in GF(23)/x3 + x + 1(i.e., multiplicative group, which excludes additive inverse)Recall = “2”, 2 = “3”, … , 7 = 1 = “1”
6
2
32
65432
6
771
61
51
441
3331
2222221
1111111
6
5
4
63
42
65432
1
1
1
1
1
1
1111111
T
Should be clear that c = T (m0,m1,…,mk-1,0,…)T is the same as evaluating p(x) = m0 + m1x + … + mk-1xk-1 at n points.
![Page 12: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/12.jpg)
296.3 Page12
Decoding
Why is it hard?
Brute Force: try k+2s choose k + s possibilities and solve for each.
![Page 13: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/13.jpg)
296.3 Page13
Cyclic Codes
A linear code is cyclic if: (c0, c1, …, cn-1) C (cn-1, c0, …, cn-2) C
Both Hamming and Reed-Solomon codes are cyclic.
Note: we might have to reorder the columns to make the code “cyclic”.
Motivation: They are more efficient to decode than general codes.
![Page 14: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/14.jpg)
296.3 Page14
Linear Code Generator and Parity Check Matrices
View message, codeword as vectors (m0, m1, …,mk-1) and (c0, c1, …,cn-1)
Generator Matrix: A k x n matrix G such that:
C = {m G | m k} Made from stacking the basis vectors
Parity Check Matrix: A (n – k) x n matrix H such that:
C = {v n | H vT = 0} Codewords are the nullspace of H
These always exist for linear codesH GT = 0
![Page 15: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/15.jpg)
296.3 Page15
RS Generator and Parity Check Polynomials
View message (m0, m1, …,mk-1) as polynomial m0 + m1x + … + mk-
1xk-1,
codeword (c0, c1, …,cn-1) as polynomial c0 + c1x + … + cn-1xn-1
Generator Polynomial: A degree (n-k) polynomial g(x) = g0 + g1x + … + gn-kxn-k such that:
C = {m g | m m0 + m1x + … + mk-1xk-1} such that g | xn – 1
Parity Check Polynomial: A degree k polynomial h(x) = h0 + h1x + … + hkxk such that:
C = {v n [x] | h v = 0 (mod xn –1)} such that h | xn - 1
These always exist for linear cyclic codesh g = xn - 1
![Page 16: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/16.jpg)
296.3 Page16
Poly multiplication via matrix multiplication
If g(x) = g0 + g1x + … + gn-kxn-k
We can put this generator in matrix form (k x n):
Write m = m0 + m1x +…+ mk-1xk-1 as (m0, m1, …, mk-1)
Then c = mG
knkn
knkn
knk
ggg
ggg
gggg
G
120
10
110
00
00
000
k-1
k-1n-k+1
![Page 17: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/17.jpg)
296.3 Page17
g generates cyclic codes
Codes are linear combinations of the rows.All but last row is clearly cyclic (left shift of next row)Right shift of last row is xkg mod (xn –1) = gn-k,0,…,g0,g1,…,gn-k-1
Will show xkg mod (xn –1) is a linear combination of other rows.Consider h = h0 + h1x + … + hkxk (gh = xn –1)
h0g + (h1x)g + … + (hk-1xk-1)g + (hkxk)g = xn – 1
(hkxk)g = (xn – 1) – (h0g + (h1x)g + … + (hk-1xk-1)g )
(hkxk)g mod (xn –1) = -(h0g + h1(xg) + … + hk-1(xk-1g))
xkg mod (xn –1) = -hk-1(h0g + h1(xg) + … + hk-1(xk-1g))
gx
xg
g
ggg
ggg
gggg
G
kknkn
knkn
knk
1120
0
110
00
00
000
![Page 18: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/18.jpg)
296.3 Page18
Viewing h as a matrixIf h = h0 + h1x + … + hkxk
we can put this parity check poly. in matrix form ((n-k) x n) :
00
00
00
01
01
01
hhh
hhh
hhh
H
k
kk
k
HcT = 0 (syndrome gives coefficients of xn-1 through xk in h c, which are the same as in h c mod xn-1)
k+1n-k-1
n-k-1
![Page 19: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/19.jpg)
296.3 Page19
Hamming Codes Revisited
The Hamming (7,4,3)2 code.
1011000
0101100
0010110
0001011
G
0011101
0111010
1110100
H
g = 1 + x + x3 h = x4 + x2 + x + 1
The columns are not identical to the previous example Hamming code.
gh = x7 – 1, GHT = 0
![Page 20: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/20.jpg)
296.3 Page20
Factors of xn -1
Intentionally left blank
![Page 21: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/21.jpg)
296.3 Page21
Another way to write gLet be a generator of GF(pr).Let n = pr - 1 (the size of the multiplicative group)Then we can write a generator polynomial as g(x) = (x-)(x-2) … (x - n-k), h = (x- n-k+1)…(x-n)Lemma: g | xn – 1, h | xn – 1, gh | xn – 1 (a | b means a divides b)Proof:
– n = 1 (because of the size of the group) n – 1 = 0 root of xn – 1 (x - ) | xn -1
– similarly for 2, 3, …, n
– therefore xn - 1 is divisible by (x - )(x - 2) …
![Page 22: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/22.jpg)
296.3 Page22
Back to Reed-Solomon
Consider a generator polynomial g GF(pr)[x], s.t. g | (xn – 1) Recall that n – k = 2s (the degree of g is n-k, n-k+1 coefficients)Encode (trick to make code systematic):
– m’ = m x2s (basically shift by 2s)– b = m’ (mod g), m’ = qg + b for some q
– c = m’ – b = (mk-1, …, m0, -b2s-1, …, -b0)
– Note that c is a cyclic code based on g - c = m’ – b = qg(i.e., given m we found another message q such that qgis “systematic” for m)
Parity check:- h c = 0 ?
![Page 23: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/23.jpg)
296.3 Page23
Example
Lets consider the (7,3,5)8 Reed-Solomon code.We use GF(23)/x3 + x + 1
x 010 2
2 x2 100 3
3 x + 1 011 4
4 x2 + x 110 5
5 x2 + x + 1
111 6
6 x2 + 1 101 7
7 1 001 1
![Page 24: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/24.jpg)
296.3 Page24
Example RS (7,3,5)8
g = (x - )(x - 2)(x - 3)(x - 4) = x4 + 3x3 + x2 + x + 3
h = (x - 5)(x - 6)(x - 7)= x3 + 3x3 + 2x + 4
gh = x7 - 1Consider the message: 110 000 110
m = (4, 0, 4) = 4x2 + 4
m’ = x4m = 4x6 + 4x4
= (4 x2 + x + 3)g + (3x3 + 6x + 6)c = (4, 0, 4, 3, 0, 6, 6) = 110 000 110 011 000 101 101
010
2 100
3 011
4 110
5 111
6 101
7 001
ch = 0 (mod x7 –1)
n = 7, k = 3, n-k = 2s = 4, d = 2s+1 = 5
![Page 25: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/25.jpg)
296.3 Page25
A useful theorem
Theorem: For any , if g() = 0 then 2sm() = b()
Proof: x2sm(x) = m’(x) = g(x)q(x) + b(x)2sm() = g()q() + b() = b()
Corollary: 2sm() = b() for {, 2, 3,…, 2s=n-k}
Proof: {, 2, …, 2s} are the roots of g by definition.
![Page 26: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/26.jpg)
296.3 Page26
Fixing errors
Theorem: Any k symbols from c can reconstruct c and hence m
Proof:We can write 2s equations involving m (cn-1, …,
c2s) and b (c2s-1, …, c0). These are2s m() = b()4s m(2) = b(2)…2s(2s) m(2s) = b(2s)
We have at most 2s unknowns, so we can solve for them. (I’m skipping showing that the equations are linearly independent).
![Page 27: 296.3:Algorithms in the Real World](https://reader030.vdocuments.mx/reader030/viewer/2022032708/56812d14550346895d91f6c8/html5/thumbnails/27.jpg)
296.3 Page27
Efficient Decoding
I don’t plan to go into the Reed-Solomon decoding algorithm, other than to mention the steps.
SyndromeCalculator
ErrorPolynomial
BerlekampMassy
ErrorLocations
ChienSearch
Error Magnitudes
ForneyAlgorithm
ErrorCorrectorc m
This is the hard part. CD players use this algorithm.(Can also use Euclid’s algorithm.)