introduction to channel coding
DESCRIPTION
An introduction to modern Channel Coding.TRANSCRIPT
![Page 1: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/1.jpg)
DIGITAL COMMUNICATIONS
Block 3
Channel Coding
Francisco J. Escribano, 2014-15
![Page 2: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/2.jpg)
2
Fundamentals of error control
● Error control:
– error detection (ARQ schemes)
– correction (FEC schemes)
● Channel model:
– discrete inputs,
– discrete (hard, rn) or continuous (soft, λ
n) outputs,
– memoryless.
Channelencoder
ChannelChannel
errorcorrector
bn
bn
rn
cn
λn
Channelerror
detectorRTx
![Page 3: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/3.jpg)
3
Fundamentals of error control
● Enabling detection/correction:
– Adding redundancy to the information: for every k bits, transmit n, n>k.
● Shannon's theorem (1948):
1) If , for ε>0, there is n, R=k/n const., so that Pb<ε.
2) If Pb is acceptable, rates R<R(P
b)=C/(1-H
2(P
b)) are achievable.
3) For any Pb, rates greater than R(P
b) are not achievable.
● Problem: Shannon's theorem is not constructive.
R<C=maxpX
I ( X ;Y )
![Page 4: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/4.jpg)
4
Fundamentals of error control
● Added redundancy is structured redundancy.
● This relies on sound algebraic & geometrical basis.
● Our initial approach:
– Algebra over the Galois Field of order 2, GF(2)={0,1}.
– GF(2) is a proper field, GF(2)m is a vector field of dim. m.
– Dot product · :logical AND. Sum +(-) : logical XOR.
– Scalar product: b, d ∈ GF(2)m
b·dT=b1·d
1+...+b
m·d
m
– Product by scalars: a ∈ GF(2), b ∈ GF(2)m
a·b=(a·b1..a·b
m)
– It is also possible to define a matrix algebra over GF(2).
![Page 5: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/5.jpg)
5
Fundamentals of error control
● Given a vector b ∈ GF(2)m, its binary weight is w(b)=number of 1's in b.
● It is possible to define a distance over vector field GF(2)m, called Hamming distance:
dH(b,d)=w(b+d); b, d ∈ GF(2)m
● Hamming distance is a proper distance and accounts for the number of differing positions between vectors.
● Geometrical view:
(0110)
(1011)
(1110) (1010)
![Page 6: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/6.jpg)
6
Fundamentals of error control
● A given encoder produces n bit outputs for each k bit inputs:
– R=k/n<1 is the rate of the code.
● The information rate decreases by R by the use of a code.
R'b=R·R
b < R
b (bit/s)
● If used jointly with a modulation with spectral efficiency η=Rb/B
(bit/s/Hz), the efficiency decreases by R
η'=R·η < η (bit/s/Hz)● In terms of P
b, the achievable E
b/N
0 region in AWGN is lower
bounded by:
E b
N 0(dB)⩾10⋅log10 (
1η '⋅(2
η'−1) )
![Page 7: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/7.jpg)
7
Fundamentals of error control
Source: http://www.comtechefdata.com/technologies/fec/ldpc
![Page 8: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/8.jpg)
8
Fundamentals of error control
● How a channel code can improve Pb (BER in statistical terms).
● Cost: loss in resources (spectral efficiency, power).
![Page 9: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/9.jpg)
9
Binary Linear block codes
![Page 10: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/10.jpg)
10
Binary Linear block codes
● An (n,k) linear block code (LBC) is a subspace C(n,k) < GF(2)n
with dim(C(n,k))=k.
● C(n,k) contains 2k vectors c=(c1…c
n).
● R=k/n is the rate of the LBC.
● n-k is the redundancy of the LBC
– we would only need vectors with k components to specify the same amount of information.
![Page 11: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/11.jpg)
11
Binary Linear block codes
● Recall vector theory:
– A basis for C(n,k) has k vectors over GF(2)n
– C(n,k) is orthogonal to an n-k dimensional subspace over GF(2)n (its null subspace).
● c ∈ C(n,k) can be both specified as:
– c=b1·g
1+...+b
k·g
k, where {g
j}
j=1,...,k is the basis, and
(b1...b
k) are its coordinates over it.
– c such that the scalar products c·hiT are null, when
{hi}
i=1,...,n-k is a basis of the null subspace.
![Page 12: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/12.jpg)
12
Binary Linear block codes
● Arranging in matrix form, an LBC C(n,k) can be specified by
– G={gij}
i=1,...,k, j=1,...,n, c=b·G, b ∈ GF(2)k.
– H={hij}
i=1,...,n-k, j=1,...,n, c·HT=0.
● G is a k×n generator matrix of the LBC C(n,k).
● H is a (n-k)×n parity-check matrix of the LBC C(n,k).
– In other approach, it can be shown that the rows in H stand for linearly independent parity-check equations.
– The row rank of H for an LBC should be n-k.
● Note that gj ∈ C(n,k), and so G·HT=0.
![Page 13: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/13.jpg)
13
Binary Linear block codes
● The encoder is given by G
– Note that a number of different G generate the same LBC
● For any input information block with length k, it yields a codeword with length n.
● An encoder is systematic if b is contained in c=(b1...b
k | c
k+1...c
n),
so that ck+1
...cn are the n-k parity bits.
– Systematicity is a property of the encoder, not of the LBC C(n,k) itself.
– GS=[I
k | P] is a systematic generator matrix.
LBCencoder
G
b=(b1...b
k) c=(c
1...c
n)=b·G
![Page 14: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/14.jpg)
14
Binary Linear block codes
● How to obtain G from H or H from G.
– G rows are k vectors linearly independent over GF(2)n
– H rows are n-k vectors linearly independent over GF(2)n
– They are related through G·HT=0 (a)
● (a) does not yield a sufficient set of equations, given H or G.
– A number of vector sets comply with it (basis sets are not unique).
● Given G, put it in systematic form by combining rows (the code will be the same, but the encoding does change).
– If GS=[I
k | P], then H
S=[PT | I
n-k] complies with (a).
● Conversely, given H, put it in systematic form by combining rows.
– If HS=[I
n-k | P], then G
S=[PT | I
k] complies with (a).
● Parity check submatrix P can be on the left or on the right side (but on opposite sides of H and G simultaneously for a given LBC).
![Page 15: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/15.jpg)
15
Binary Linear block codes
● Note that, by taking 2k vectors out of 2n, we are getting apart the binary words.
● Minimum Hamming distance between input words is
● Recall that we have added n-k redundancy bits, so that
dmin (GF (2)k )=minbi≠b j
{d H (bi ,b j ) |bi ,b j∈GF (2)k }=1
d min (C (n , k ))=minc i≠c j
{d H (c i ,c j ) | ci ,c j∈C (n , k )}>1
d min (C (n , k ))⩽n−k+1 (Singleton bound)
![Page 16: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/16.jpg)
16
Binary Linear block codes
● The channel model corresponds to a BSC (binary symmetric channel)
● p=P(ci≠r
i ) is the bit error probability of the modulation in AWGN.
Modulator
ChannelBSC(p)
AWGNchannel
r=(r1...r
n)c=(c
1...c
n)
Harddemodulator
0
1
0
1p
1-p
1-p
p
![Page 17: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/17.jpg)
17
Binary Linear block codes
● The received word is r=c+e, where P(ei=1)=p.
– e is the error vector introduced by the noisy channel
– w(e) is the number of errors in r wrt original word c
– P(w(e)=t)=pt·(1-p)n-t, because the channel is memoryless.
● At the receiver side, we can compute the syndrome s=(s1...s
n-k) as
s=r·HT=(c+e)·HT=c·HT+e·HT=e·HT.
● r ∈ C(n,k) ⇔ s=0.
Channeldecoder
H
s=(s1...s
n-k)=r·HTr=(r
1...r
n)
![Page 18: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/18.jpg)
18
Binary Linear block codes
Two possibilities at the receiver side:
● a) Error detection (ARQ schemes):
– If s≠0, there are errors, so ask for retransmission.
● b) Error correction (FEC schemes):
– Decode an estimated ĉ ∈ C(n,k), so that dH(ĉ,r) is the minimum
over all codewords in C(n,k) (closest neighbor decoding).
– ĉ is the most probable word under the assumption that p is small (otherwise, the decoding fails).
(0110)
(1011)
(1110) (1010)
ĉ1 e
1
e2
r2
r1
c
ĉ2
OK
![Page 19: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/19.jpg)
19
Binary Linear block codes
● Detection and correction capabilities (worst case) of an LBC with d
min(C(n,k)).
– a) It can detect error events e with binary weight up to
w(e)|max,det
=d=dmin
(C(n,k))-1
– b) It can correct error events e with binary weight up to
w(e)|max,corr
=t=⎣(dmin
(C(n,k))-1)/2⎦
● It is possible to implement a joint strategy:
– A dmin
(C(n,k))=4 code can simultaneously correct all error
patterns with w(e)=1, and detect all error patterns with w(e)=2.
![Page 20: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/20.jpg)
20
Binary Linear block codes
● The minimum distance dmin
(C(n,k)) is a property of the set of
codewords in C(n,k), independent from the encoding (G).
● As the code is linear, dH(c
i,c
j)=d
H(c
i+c
j,c
j+c
j)=d
H(c
i+c
j,0).
– ci, c
j, c
i+c
j, 0 ∈ C(n,k)
● dmin
(C(n,k))=min{w(c) | c ∈ C(n,k), c≠0}
– i.e., corresponds to the minimum word weight over all codewords different from null.
● dmin
(C(n,k)) can be calculated from H:
– It is the minimum number of different columns of H adding to 0.
– It is the column rank of H + 1.
![Page 21: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/21.jpg)
21
Binary Linear block codes
● Detection limits: probability of undetected errors?
– Note that an LBC contains 2k codewords, and the received word corresponds to any of the 2n possibilities in GF(2)n.
– An LBC detects up to 2n-2k error patterns.
● An undetected error occurs if r=c+e with e≠0 ∈ C(n,k)
– In this case, r·HT=0.
– Ai is the number of codewords in C(n,k) with weight i: it is
called the weight spectrum of the LBC.
Pu (E)=∑i=dmin
n
Ai⋅p i⋅(1− p)n−i
![Page 22: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/22.jpg)
22
Binary Linear block codes
● On correction, an LBC considers syndrome s=r·HT.
– Assume correction capabilities up to w(e)=t, and EC to be
the set of correctable error patterns.
– A syndrome table associates a unique si over the 2n-k
possibilities to a unique error pattern ei ∈ E
C with w(e
i)≤t.
– If si=r·HT, decode ĉ=r+e
i.
– Given an encoder G, estimate information vector b such that b·G=ĉ.
● If the number of correctable errors #(EC)<2n-k, there are 2n-k-
#(EC) syndromes usable in detection, but not in correction.
– At most, an LBC can correct 2n-k error patterns.
^
^
![Page 23: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/23.jpg)
23
Binary Linear block codes
● A w(e)≤t error correcting LBC has a probability of correcting erroneously bounded by
– This is an upper bound, since not all the codewords are separated by the minimum distance of the code.
● Calculating the resulting P'b of an LBC is not an easy task,
and it depends heavily on how the encoding is made through G.
● LBC codes are mainly used in detection tasks (ARQ).
P (E )⩽∑i=t+1
n
(ni )⋅pi⋅(1− p)n−i
![Page 24: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/24.jpg)
24
Binary Linear block codes
● Observe that both coding & decoding can be performed with low complexity hardware (combinational logic: gates).
● Examples of LBC
– Repetition codes
– Single parity check codes
– Hamming codes
– Cyclic redundancy codes
– Reed-Muller codes
– Golay codes
– Product codes
– Interleaved codes
● Some of them will be examined in the lab.
![Page 25: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/25.jpg)
25
Binary Linear block codes
● An example of performance: RHam
=4/7, RGol
=1/2.
![Page 26: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/26.jpg)
26
Multiple-error correction block codes
![Page 27: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/27.jpg)
27
Multiple-error-correction block codes ● There are a number of more powerful block codes, based on higher
order Galois fields.
– They use symbols over GF(r), with r>2.
– Now the operations are defined as mod r.
– An (n,k) linear block code with symbols from GF(r) is again a k-dimensional subspace of the vector space GF(r)n.
● They are used mainly for correction, and have applications in channels or systems where error bursts are frequent, i.e.
– Storage systems (erasure channel).
– Communication channels with deep fades.
● They are frequently used in concatenation with other codes that fail in short bursts when correcting (i.e. convolutional codes).
● We are going to study two important instances of such broad class of codes: Reed-Solomon, and BCH codes.
![Page 28: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/28.jpg)
28
Multiple-error-correction block codes
● An (n,k,t)q Reed-Solomon (R-S) code is defined as a mapping
from GF(q)k↔GF(q)n, with the following parameters:
– Block length n=q-1 symbols, for input block length of k<n symbols.
– Alphabet size q=pr, where p is a prime number.
● Minimum distance of the code dmin
=n-k+1 (it achieves the
Singleton bound for linear block codes).
– It can correct up to t symbol errors, dmin
=2t+1.
● This is a whole class of codes.
– For any q=pr, n=q-1 and k=q-1-2t, there exists a Reed-Solomon code meeting all these criteria.
![Page 29: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/29.jpg)
29
Multiple-error-correction block codes ● The R-S code is built with the help of polynomial algebra. ● The message
is mapped to a polynomial with given coefficients
● To get the corresponding codeword, the encoding function works evaluating the polynomial at n distinct given points
Note that all the operations are performed over GF(q).
s=( s1 ... sk ) ∈ GF (q ) k
p (a )=∑i=1
k
zi⋅ai−1 ; a , z i ∈ GF ( q )
c=(p(a1) ...p(an)) ∈ GF(q)n
![Page 30: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/30.jpg)
30
Multiple-error-correction block codes ● We can rewrite the encoding in a more familiar form
where A is the transpose of a Vandermonde matrix with structure
c=z⋅A , z=( z1 ... zk )
A=(1 1 ⋯ 1a1 a2 ⋯ an
a12 a2
2 ⋯ an2
⋮ ⋮ ⋱ ⋮
a1(k−1) a2
(k−1) ⋯ an(k−1)
)
![Page 31: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/31.jpg)
31
Multiple-error-correction block codes
● The number of polynomials in GF(q) of degree less than k is clearly qk, exactly the possible number of messages.
– This guarantees that each information word can be mapped to a unique codeword by choosing a convenient mapping s↔z.
● It is only required that a1,..,a
n are distinct points in GF(q),
(these are the points where the polynomial p(a) is evaluated to build the codeword)
– The points can be chosen to meet certain properties.
– Either the polynomial can be chosen in a given way.
![Page 32: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/32.jpg)
32
Multiple-error-correction block codes
● One way to build the framework for R-S codes consists in choosing a
1,..,a
n as n distinct points in GF(q) and build p(a)
by forcing the condition
● The polynomial is characterized as the only polynomial of degree less than k that meets the above mentioned condition, and can be found by using known algebraic methods (Lagrange interpolation).
● The codeword is given as
This is an instance of R-S systematic encoding.
p (ai )=si ∀ i=1,. .. k
cs=(s1 ... sk , p (ak+1 ) ...p (an ))
![Page 33: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/33.jpg)
33
Multiple-error-correction block codes
● In other possible construction, the polynomial is given by the mapping z=s
● And the points in GF(q) are chosen to meet certain convenient properties.
– Let α be a primitive root of GF(q). This means that, for any b ∊ GF(q), except 0, there exists an integer u / αu=b (mod q).
– ai=αi, i=1,...,q-1 (this spans GF(q), except 0).
p (a )=∑i=1
k
si⋅ai−1 , a ∈ GF ( q )
![Page 34: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/34.jpg)
34
Multiple-error-correction block codes ● Now we can rewrite
where this time A is the transpose of a Vandermonde matrix with structure
c=s⋅A , s=( s1 ... sk )
A=(1 1 ⋯ 1α α2 ⋯ αn
(α)2
(α2)
2⋯ (α
n)
2
⋮ ⋮ ⋱ ⋮
(α)(k−1 ) (α2)(k−1) ⋯ (αn)(k−1))
![Page 35: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/35.jpg)
35
Multiple-error-correction block codes
● In this last case, it can be demonstrated that the parity-check matrix of the resulting R-S code is
– It can correct t or fewer random symbol errors over a span of n=q-1 symbols.
H=(1 α α
2⋯ α
(q−2)
1 α2 (α2)2 ⋯ (α2)(q−2)
1 α3 (α3)2 ⋯ (α3)(q−2)
⋮ ⋮ ⋮ ⋱ ⋮
1 α2 t
(α2 t)
2⋯ (α
2 t)(q−2)
)
![Page 36: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/36.jpg)
36
Multiple-error-correction block codes
● The weight distribution (spectrum) of R-S codes has closed form
● We can derive bounds for the probability of undetected errors for a symmetric DMC with q-ary input/output alphabets, and probability of correct reception 1-ε.
Ai=(q−1i )q−2t
{(q−1 )i+∑
j=0
2 t
(−1 )i+ j( ij)(q
2 t−q i )}
2 t+1⩽i⩽q−1
Pu ( E )<q−2 t before any correction step
Pu ( E ,λ )<q−2 t∑h=0
λ
(q−1h ) (q−1 )
h if used first to correct λ errors
0⩽ϵ⩽q
q−1 0<λ<t
![Page 37: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/37.jpg)
37
Multiple-error-correction block codes
● Other kind of linear block code defined over higher-order Galois fields is the class of BCH codes.
– Named after Raj Bose and D. K. Ray-Chaudhuri.
● An (n,k,t)q BCH code is again defined over GF(q), where
q=pr, and p is prime.
● We have m, n, q, d=2t+1, l, so that
– 2≤d≤n. l will be considered later.
– gcd(n,q)=1 (“gcd” → greatest common divisor)
– m is the multiplicative order of q modulo n; m is thus the smallest integer meeting qm=1 (mod n).
– t is the number of errors that may be corrected.
![Page 38: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/38.jpg)
38
Multiple-error-correction block codes
● Let α be a primitive n-th root of 1 in GF(qm). This means αn=1 (mod qm).
● Let mi(x) the minimal polynomial for GF(q) of αi, ∀ i.
– This is the monic polynomial of least degree having αi as a root.
– Monic → the coefficient of the highest power of x is 1.
● Then, a BCH code is defined by a so-called generator polynomial
where “lcm” stands for least common multiple. Its degree is at most (d-1)m=2mt.
g ( x )=lcm (ml ( x ) ...ml+d−2 ( x ) )
![Page 39: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/39.jpg)
39
Multiple-error-correction block codes
● The encoding with a generator polynomial is done by building a polynomial containing the information symbols
● Then
● We may do systematic encoding as
s (x )=∑i=1
k
si⋅x i−1s=( s1... sk ) ∈ GF (q ) k →
c ( x )=∑i=1
n
c i⋅xi−1=s ( x )⋅g ( x ) , c i ∈ GF (q) → c=(c1 ... cn )
cs (x )= xn−k⋅s ( x )⏟
systematic symbols
+xn−k⋅s (x ) mod g ( x )⏟
redundancy symbols
![Page 40: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/40.jpg)
40
Multiple-error-correction block codes
● The case with l=1, and n=qm-1 is called a primitive BCH code.
– The number of parity check symbols is n-k≤(d-1)m=2mt.
– The minimum distance is dmin
≥d=2t+1.
● If m=1, then we have a Reed-Solomon code (of the “primitive root” kind)!
– R-S codes can be seen as a subclass of BCH codes.
– In this case, it can be verified that the R-S code may be defined by means of a generator polynomial, in the form
g ( x )=( x−α ) (x−α2 ) ... (x−α2 t )==g1+g2 x+g3 x2
+...+g2 t−2 x2 t−1+x2 t
![Page 41: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/41.jpg)
41
Multiple-error-correction block codes
● The parity check matrix for a primitive BCH code over GF(qm) is
– This code can correct t or fewer random symbol errors, d=2t+1, over a span of n=qm-1 symbol positions.
H=(1 α α
2⋯ α
(n−1)
1 α2
(α2)
2⋯ (α
2)(n−1)
1 α3 (α3)2 ⋯ (α3)(n−1)
⋮ ⋮ ⋮ ⋱ ⋮
1 α(d−1)
(α(d−1)
)2⋯ (α
(d−1))(n−1)
)
![Page 42: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/42.jpg)
42
Multiple-error-correction block codes
● Why these codes can correct error bursts in the channel?
BCH/R-Sencoder
BSC(p)
BCH/R-Sdecoder
Mapbits
to q-arysymbols
Mapq-ary
symbolsto bits
Mapq-ary
symbolsto bits
Mapbits
to q-arysymbols
s=(s1...s
k)
r=(r1...r
n)
b=(b1...b
k·log2(q))
b'=(b'1...b'
k·log2(q))
s'=(s'1...s'
k)
c=(c1...c
n)
cb
rb=c
b+e
b
p=Pb
![Page 43: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/43.jpg)
43
Multiple-error-correction block codes
● Why these codes can correct error bursts in the channel?
BCH/R-Sencoder
BSC(p)
BCH/R-Sdecoder
Mapbits
to q-arysymbols
Mapq-ary
symbolsto bits
Mapq-ary
symbolsto bits
Mapbits
to q-arysymbols
s=(s1...s
k)
r=(r1...r
n)
b=(b1...b
k·log2(q))
b'=(b'1...b'
k·log2(q))
s'=(s'1...s'
k)
c=(c1...c
n)
Other channel encoder/decoder,modulator/demodulator,
medium access technique
cb
rb=c
b+e
b
p=Pb
![Page 44: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/44.jpg)
44
Multiple-error-correction block codes
● Why these codes can correct error bursts in the channel?
BCH/R-Sencoder
BSC(p)
BCH/R-Sdecoder
Mapbits
to q-arysymbols
Mapq-ary
symbolsto bits
Mapq-ary
symbolsto bits
Mapbits
to q-arysymbols e
b=(...1111...)
s=(s1...s
k)
r=(r1...r
n)
b=(b1...b
k·log2(q))
b'=(b'1...b'
k·log2(q))
s'=(s'1...s'
k)
c=(c1...c
n)
Other channel encoder/decoder,modulator/demodulator,
medium access technique
cb
rb=c
b+e
b
p=Pb
![Page 45: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/45.jpg)
45
Multiple-error-correction block codes
● Why these codes can correct error bursts in the channel?
BCH/R-Sencoder
BSC(p)
BCH/R-Sdecoder
Mapbits
to q-arysymbols
Mapq-ary
symbolsto bits
Mapq-ary
symbolsto bits
Mapbits
to q-arysymbols e
b=(...1111...)
s=(s1...s
k)
r=(r1...r
n)
b=(b1...b
k·log2(q))
b'=(b'1...b'
k·log2(q))
s'=(s'1...s'
k)
c=(c1...c
n)
Other channel encoder/decoder,modulator/demodulator,
medium access technique
If bit error burst falls within a single symbol ri in GF(q), or at most spans over t
symbols in GF(q) within word r, it can be corrected!
cb
rb=c
b+e
b
p=Pb
![Page 46: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/46.jpg)
46
Multiple-error-correction block codes
● Note that algebra is now far more involved and more complex than with binary LBC.
– This is part of the price to pay to get better data integrity protection.
– Other logical price to pay is the reduction in data rate, R. k and n are measured in symbols, but, as the mapping and demapping is performed from GF(2) to GF(q), and viceversa, the end-to-end effective data rate is again
● But there is still something to do to get the best from R-S and BCH codes: decoding is substantially more complex!
Rb'=Rb⋅
kn
![Page 47: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/47.jpg)
47
Multiple-error-correction block codes
● We are going to see a simple instance of decoding for these families of nonbinary LBC.
● We address the general BCH case, as R-S can be seen as an instance of the former.
● Correction can be performed by identifying the pairs
● For this, we can resort to the syndrome
r ( x )=c ( x )+e ( x ) is the received codeword
e ( x )=e j1x j1+...+e jν
x j ν 0⩽ j1⩽...⩽ j ν⩽n−1
(x ji , e ji) ∀ i=1,... ,ν
(S1... S2 t ) , where S i=r (αi )=e (α
i) ∈ GF (qm)
![Page 48: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/48.jpg)
48
Multiple-error-correction block codes
S l=δ1β1l+...δνβν
l , i=l , ..2t
δi=e ji, βi=α
ji
● We can build a set of equations
● Based on this, a BCH or an R-S may be decoded on 4 steps:
1. Compute the syndrome vector.
2. Determine the so-called error-location polynomial.
3. Determine the so-called error-value evaluator.
4. Evaluate error-location numbers ( ) and error values ( ), and perform correction.
● The error-location polynomial is defined as
ji e ji
σ ( x )=(1−β1 x ) ... (1−βν x )=∑l=0
ν
σ l xl , where σ0=1
![Page 49: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/49.jpg)
49
Multiple-error-correction block codes
● We can find the error-location polynomial with the help of the Berlekamp's algorithm, using the syndrome vector.
– It works iteratively, in 2t steps.
– Details can be found in the references.
● Once determined, its roots can be found by substituting the elements of GF(qm) cyclically in σ(x).
– If , is an error-location number.
– The errors are thus located at such positions.
● On the other hand, the error-value evaluator is defined as
σ (αi )=0 α−i=α
qm−i−1
qm−i−1
Z0 ( x )=∑l=1
ν
δlβl ∏i=1, i≠l
ν
(1−βi x )
![Page 50: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/50.jpg)
50
Multiple-error-correction block codes ● It can be shown that the error-value evaluator can be calculated
as a function of known quantities
● After some algebra, the error values are determined as
– Where the denominator is the derivative of σ(x).
● With the error values and the error locations, the error vector is estimated and correction may be performed
Z0 ( x )=S1+(S2+σ1 S1 )x+(S3+σ1 S2+σ2 S1 )x2+
+...+(Sν+σ1 Sν−1+...+σν−1 S1 )xν
δk=−Z0 (βk
−1)σ ' (βk
−1 )
e (x )=∑i=1
ν
δi xji → c (x )=r ( x )− e ( x )
![Page 51: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/51.jpg)
51
Multiple-error-correction block codes
● BCH and R-S codes may be very powerful, but the amount of algebra required is very high.
– The supporting theory is very complex, and designing and analyzing these codes require mastering algebra and geometry over finite-size fields.
– All the operations are to be understood in GF(q) or GF(qm), when corresponding.
● Nonbinary LBC of the kind described are usually employed in sophisticated FEC strategies for specific channels.
– Binary LBC are more usual in ARQ strategies.
![Page 52: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/52.jpg)
52
Multiple-error-correction block codes
● Example BPSK+AWGN: RHam
=246/255, RGol
=1/2, RR-S
=155/255.
![Page 53: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/53.jpg)
53
Convolutional codes
![Page 54: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/54.jpg)
54
Convolutional codes
● A binary convolutional code (CC) is another kind of linear channel code class.
● The encoding can be described in terms of a finite state machine (FSM).
– A CC can eventually produce sequences of infinite length.
– A CC encoder has memory. General structure:
MEMORY: ml bits for l-th input
Forward logic (coded bits)
Backward logic (feedback)
k input streams
n output streams
notmandatory
notmandatory
Systematic output
![Page 55: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/55.jpg)
55
Convolutional codes
● The memory is organized as a shift register.
– Number of positions for input l: memory ml.
– ml=ν
l is the constraint length of the l-th input/register.
– The register effects step by step delays on the input: recall discrete LTI systems theory.
● A CC encoder produces sequences, not just blocks of data.
– Sequence-based properties vs. block-based properties.
1 2 3 4 ml
l-th input streaminput at instant i
to backwardlogic
to forwardlogic
d i( l ) d i−1
( l ) d i−2(l ) d i−3
(l ) d i−4(l ) d i−ml
(l )
![Page 56: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/56.jpg)
56
Convolutional codes
● Both forward and backward logic is boolean logic.
– Very easy: each operation adds up (XOR) a number of memory positions, from each of the k inputs.
● , is 1 when the p-th register position for the l-th input is added to get the j-th output.
c i( j)=∑
l=1
k
∑q=i
i−ml
g l , q−i( j ) ⋅d q
(l ) j-th output at instant i
inputs from all the k registers at instant i
g l , p( j ) , p=0,. .. , ml
Same structure forbackward logic
![Page 57: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/57.jpg)
57
Convolutional codes
● Parameters of a CC so far:
– k input streams
– n output streams
– k shift registers with length ml each, l=1,...,k
– νl=m
l is the constraint length of the l-th register
– m=maxl{ν
l} is the memory order of the code
– ν=ν1+...+ν
k is the overall constraint length of the code
● A CC is denoted as (n,k,ν).
– As usual, its rate is R=k/n, where k and n take normally small values.
![Page 58: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/58.jpg)
58
Convolutional codes
● The backward / forward logic may be specified in the form of generator sequences.
– Theses sequences are the impulse responses of each output j wrt each input l.
● Observe that:
– connects the l-th input directly to the j-th output
– just delays the l-th input to the j-th output q time steps.
g l( j )=(g l , 0
( j ) , ... , g l ,ml
( j) )
g l( j )=(1,0, ... ,0)
g l( j )=(0,. .. ,1(q th) , ... ,0 )
![Page 59: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/59.jpg)
59
Convolutional codes
● Given the presence of the shift register, the generator sequences are better denoted as generator polynomials
● We can thus write, for example
g l( j)=(g l ,0
( j) ,... , g l , ml
( j ) )≡ g l( j)(D)=∑
q=0
ml
g l , q( j)⋅D q
gl( j )=(1,0, ... ,0) ≡ g l
( j )(D)=1
g l( j )=(0,. .. ,1(qth) , ... ,0 )≡ gl
( j)(D)=Dq
gl( j )=(1,1,0, ... ,0) ≡ gl
( j )(D)=1+D
![Page 60: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/60.jpg)
60
Convolutional codes
● As all operations involved are linear, a binary CC is linear and the sequences produced constitute CC codewords.
● A feedforward CC (without backward logic - feedback) can be denoted in matrix from as
G(D)=(g1(1)(D) g1
(2)(D) ⋯ g1
(n )(D)
g2(1)(D) g2
(2)(D) ⋯ g2
(n )(D)
⋮ ⋮ ⋱ ⋮
gk(1)(D) gk
(2)(D) ⋯ gk(n )(D)
)
![Page 61: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/61.jpg)
61
Convolutional codes
● If each input has a feedback logic given as
the code is denoted as
g l(0)(D)=∑q=0
ml g l , q(0)⋅Dq
G(D)=(g1(1 )(D)
g1(0)(D)
g1(2)(D)
g1(0)(D)
⋯g0(n)(D)
g1(0)(D)
g2(1 )(D)
g2(0)(D)
g2(2)(D)
g2(0)(D)
⋯g2(n)(D)
g2(0)(D)
⋮ ⋮ ⋱ ⋮
gk(1 )(D)
gk(0)(D)
gk(2)(D)
gk(0)(D)
⋯gk(n)(D)
gk(0)(D)
)
![Page 62: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/62.jpg)
62
Convolutional codes
● We can generalize the concept of parity-check matrix H(D).
– An (n,k,ν) CC is fully specified by G(D) or H(D).
● Based on the matrix description, there are a good deal linear tools for design, analysis and evaluation of a given CC.
● A regular CC can be described as a (canonical) all-feedforward CC and through an equivalent feedback (recursive) CC.
– Note that a recursive CC is related to an IIR filter.
● Even though k and n could be very small, a CC has a very rich algebraic structure.
– This is closely related to the constraint length of the CC.
– Each output bit is related to the present and past inputs via powerful algebraic methods.
![Page 63: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/63.jpg)
63
Convolutional codes
● Given G(D), a CC can be classified as:
– Systematic and feedforward (NSC).
– Systematic and recursive (RSC).
– Non-systematic and feedforward.
– Non-systematic and recursive.
● RSC is a popular class of CC, because it provides an infinite output for a finite-weight input (IIR behavior).
● Each NSC can be converted straightforwardly to a RSC with similar error correcting properties.
● CC encoders are easy to implement with standard hardware: shift registers + combinational logic.
![Page 64: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/64.jpg)
64
Convolutional codes● As with the case of nonbinary LBC, we may use the
polynomial representation to perform coding & decoding.
– But now we have encoding with memory, spanning over theoretically infinite length sequences → not practical.
c ( D )=b ( D )⋅G ( D )
b ( D )=(b(1) ( D ) ... b(k) ( D ) ) ; b(i)( D )=∑j=0
b j(i) D j
b j( i) is the i−th input bit stream
c ( D )=(c(1) ( D ) ...c(n) ( D ) ) ; c(l) ( D )=∑h=0
ch(l) Dh
ch(l) is the l−th output bit stream
![Page 65: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/65.jpg)
65
Convolutional codes
● We do not need to look into the algebraic details of G(D) and H(D) to study:
– Coding
– Decoding
– Error correcting capabilities
● A CC encoder is a FSM!
The ν memory positionsstore a content (among 2ν possible ones) at instant i-1
Coder is said to be atstate s(i-1)
The ν memory positionsstore a new content
at instant iCoder is said to be at
state s(i)
k input bits determinethe shifting ofthe registers
And we get n relatedoutput bits
![Page 66: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/66.jpg)
66
Convolutional codes
● The finite-state behavior of the CC can be captured by the concept of trellis.
– For any starting state, we have 2k possible edges leading to a corresponding set of ending states.
ss=s(i-1)
s=1,...,2ν
se=s(i)
e=1,...,2ν
input bi=(b
i,1...b
i,k)
output ci=(c
i,1...c
i,n)
![Page 67: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/67.jpg)
67
Convolutional codes
● The trellis illustrates the encoding process in 2 axis:
– X-axis: time / Y-axis: states
● Example for a (2,1,3) CC:
– For a finite-size input data sequence, a CC can be forced to finish at a known state (often 0) by adding terminating (dummy) bits.
– Note that one section (e.g. i-1 → i) fully specifies the CC.
output 00
output 01s2
s1
s3
s4
s6
s5
s7
s8
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i i+1
input 0 input 1
![Page 68: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/68.jpg)
68
Convolutional codes
● The trellis illustrates the encoding process in 2 axis:
– X-axis: time / Y-axis: states
● Example for a (2,1,3) CC:
– For a finite-size input data sequence, a CC can be forced to finish at a known state (often 0) by adding terminating (dummy) bits.
– Note that one section (e.g. i-1 → i) fully specifies the CC.
output 00
output 01s2
s1
s3
s4
s6
s5
s7
s8
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i i+1
input 0 input 1
Memory:Memory:same input,same input,
different outputsdifferent outputs
![Page 69: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/69.jpg)
69
Convolutional codes
● The trellis description allows us
– To build the encoder
– To build the decoder
– To get the properties of the code
● The encoder:
H(D)↔G(D)
RegistersCombinational
logick n
CLK
ss=s(i-1)
s=1,...,2νs
e=s(i)
e=1,...,2νinput bi=(b
i,1...b
i,k)
output ci=(c
i,1...c
i,n)
![Page 70: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/70.jpg)
70
Convolutional codes
● The decoder is far more complicated
– Long sequences
– Memory: dependence with past states
● In fact, CC were already well known before there existed a practical good method to decode them: the Viterbi algorithm.
– It is a Maximum Likelihood Sequence Estimation (MLSE) algorithm with many applications.
● Problem: for a length N>>n sequence at the receiver side
– There are 2ν·2N·k/n paths through the trellis to match with the received data.
– Even if the coder starting state is known (often 0), there are still 2N·k/n paths to walk through in a brute force approach.
![Page 71: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/71.jpg)
71
Convolutional codes
● Viterbi algorithm setup.
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i
input bi → output c
i(s(i-1),b
i)
start s(i-1) → end s(i)(s(i-1),bi)
received data ri
Key facts:● The encoding corresponds to a Markov
chain model: P(s(i))=P(s(i)|s(i-1))·P(s(i-1)).
● Total likelihood P(r|b) can be factorized as a product of probabilities.
● Given , P(ri|s(i),s(i-1)) depends
only on the channel kind (AWGN, BSC...).
● Transition from s(i-1) to s(i) (linked in the trellis) depends on the probability of b
i:
P(s(i)|s(i-1))=2-k if the source is iid.
● P(s(i)|s(i-1))=0 if they are not linked in the trellis (finite state machine: deterministic).
s(i−1)→bi
s(i )
![Page 72: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/72.jpg)
72
Convolutional codes
● The total likelihood can be recursively calculated as:
● In the BSC(p), the observation (branch) metric would be related to:
● Maximum likelihood (ML) criterion:
P (r i|s(i) ,s(i−1))=P (r i|c i )→w (r i+ci )=d H (ri ,ci )
P (r∣b)=∏i=1
N /n
P (r i∣s(i) ,s(i−1))⋅P (s(i)∣s(i−1))⋅P (s(i−1))
b=arg {maxb
[P (r∣b ) ]}
![Page 73: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/73.jpg)
73
Convolutional codes
● We know that the brute force approach to ML criterion is at least O(2N·k/n).
● The Viterbi algorithm works recursively from 1 to N/n on the basis that
– Many paths can be pruned out (transition probability=0).
– During forward recursion, we only keep the paths with highest probability: the path probability goes easily to 0 from the moment a term metric ⨯ transition probability is very small.
– When recursion reaches i=N/n, the surviving path guarantees the ML criterion (optimal for ML sequence estimation!).
● The Viterbi algorithm complexity goes down to O(N·22ν).
![Page 74: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/74.jpg)
74
Convolutional codes
● The algorithm recursive rule is
● {Vj
(i)} stores the most probable state sequence wrt observation r
V j(0)=P (s(0)=s j ) ;
V j(i)=P (r i∣s
(i)=s j ,s(i−1)
=smax )⋅ maxs(i−1)
= smax
{P (s(i)=s j∣s(i−1)
=sl )⋅V l( i−1)}
s2
s1
s3
s4
s6
s5
s7
s8
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i i+1
MAX
MAX
![Page 75: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/75.jpg)
75
Convolutional codes
● The algorithm recursive rule is
● {Vj
(i)} stores the most probable state sequence wrt observation r
V j(0)=P (s(0)=s j ) ;
V j(i)=P (r i∣s
(i)=s j ,s(i−1)
=smax )⋅ maxs(i−1)
= smax
{P (s(i)=s j∣s(i−1)
=sl )⋅V l( i−1)}
Probability of themost probable state sequence
corresponding to the i-1previous observations
s2
s1
s3
s4
s6
s5
s7
s8
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i i+1
MAX
MAX
![Page 76: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/76.jpg)
76
Convolutional codes
● The algorithm recursive rule is
● {Vj
(i)} stores the most probable state sequence wrt observation r
V j(0)=P (s(0)=s j ) ;
V j(i)=P (r i∣s
(i)=s j ,s(i−1)
=smax )⋅ maxs(i−1)
= smax
{P (s(i)=s j∣s(i−1)
=sl )⋅V l( i−1)}
Probability of themost probable state sequence
corresponding to the i-1previous observations
s2
s1
s3
s4
s6
s5
s7
s8
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i i+1
MAX
MAX
Note that we maybetter work with logs:products ↔ additions
Criterion remains the same
![Page 77: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/77.jpg)
77
Convolutional codes
● Note that we have considered the algorithm when the demodulator yields hard outputs
– ri is a vector of n estimated bits (BSC(p) equivalent channel).
● In AWGN, we can do better to decode a CC
– We can provide soft (probabilistic) estimations for the observation metric.
– For an iid source, we can easily get an observation transition metric based on the probability of each b
i,l=0,1, l=1,...,k,
associated to a possible transition.
– There is a gain of around 2 dB in Eb/N
0.
– LBC decoders can also accept soft inputs (non syndrome-based decoders).
– We will examine an example of soft decoding of CC in the lab.
![Page 78: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/78.jpg)
78
Convolutional codes
● We are now familiar with the encoder and the decoder
– Encoder: FSM (registers, combinational logic).
– Decoder: Viterbi algorithm (for practical reasons, suboptimal adaptations are usually employed).
● But what about performance?
● First...
– CC are mainly intended for FEC, not for ARQ schemes.
– In a long sequence (=CC codeword), the probability of having at least one error is very high...
– And... are we going to retransmit the whole sequence?
![Page 79: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/79.jpg)
79
Convolutional codes
● Given that we truncate the sequence to N bits and CC is linear
– We may analyze the system as an equivalent (N,N·k/n) LBC.
– But... equivalent matrices G and H would not be practical.
● Remember FSM: we can locate error loops in the trellis.
i i+1 i+2 i+3
b
b+e
![Page 80: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/80.jpg)
80
Convolutional codes
● The same error loop may occur irrespective of s(i-1) and b.
b
b+e
i i+1 i+2 i+3
b
b+e
![Page 81: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/81.jpg)
81
Convolutional codes
● Examining the minimal length loops and taking into account this uniform error property we can get d
min of a CC.
– For a CC forced to end at 0 state for a finite input data sequence, d
min is called d
free.
● We can draw a lot of information by building an encoder state diagram: error loops, codeword weight spectrum...
Diagram of a (2,1,3) CC, from Lin & Costello (2004).
![Page 82: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/82.jpg)
82
Convolutional codes
● With a fairly amount of algebra, related to FSM, modified encoder state diagrams and so on, it is possible to get an upper bound for optimal MLSE decoding.
● For the BSC(p), the bound can be calculated as
P b⩽∑d
B d⋅erfc(√d RE b
N 0)
Bd is the total number of nonzero information bits associated with
CC codewords of weight d, divided by the number of information bits k per unit time... A lot of algebra behind...
BPSK inAWGN
Pb⩽∑d
Bd⋅(2√ p⋅(1−p ))d
![Page 83: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/83.jpg)
83
Convolutional codes
● There are easier, suboptimal ways to decode a CC, and performance will vary accordingly.
● A CC may be punctured to match other rates lower than R=k/n, but the resulting equivalent CC is clearly weaker.
– Performance-rate trade-off.
– Puncturing is a very usual tool that provides flexibility to the usage of CC's in practice.
Nativeconvolutional
encoderk n
Puncturingalgorithm
(prune bits) n'<n
![Page 84: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/84.jpg)
84
Convolutional codes● Examples with BPSK+AWGN using ML bounds.
![Page 85: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/85.jpg)
85
Turbo codes
![Page 86: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/86.jpg)
86
Turbo codes
● Canonically, turbo codes (TC) are parallel concatenated convolutional codes (PCCC).
● Coding concatenation has been known and employed for decades, but TC added a joint efficient decoding.
– Example of concatenated coding with independent decoding is the use of ARQ + FEC hybrid strategies (CRC + CC).
CC1
CC2
?
k input streams n=n1+n
2 output streams
Rate R=k/(n1+n
2)
b c=c1∪c2
![Page 87: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/87.jpg)
87
Turbo codes
● Canonically, turbo codes (TC) are parallel concatenated convolutional codes (PCCC).
● Coding concatenation has been known and employed for decades, but TC added a joint efficient decoding.
– Example of concatenated coding with independent decoding is the use of ARQ + FEC hybrid strategies (CRC + CC).
CC1
CC2
?
k input streams n=n1+n
2 output streams
We will see this isa key element...
Rate R=k/(n1+n
2)
b c=c1∪c2
![Page 88: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/88.jpg)
88
Turbo codes
● We have seen that standard CC decoding with Viterbi algorithm relied on MLSE criterion.
– This is optimal when binary data at CC input is iid.
● For CC, we also have decoders that provide probabilistic (soft) outputs.
– They convert a priori soft values + channel output soft estimations into updated a posteriori soft values.
– They are optimal from the maximum a posteriori (MAP) criterion point of view.
– They are called soft input-soft output (SISO) decoders.
![Page 89: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/89.jpg)
89
Turbo codes
● What's in a SISO?
SISO(for a CC)
0 10 1
P (bi=b )=12
P (bi=b∣r )
r
● Note that the SISO works on a bit by bit basis, but produces a sequence of APP's.
Probability density function of bi
![Page 90: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/90.jpg)
90
Turbo codes
● What's in a SISO?
SISO(for a CC)
0 10 1
P (bi=b )=12
P (bi=b∣r )
r
Soft demodulated valuesfrom channel
● Note that the SISO works on a bit by bit basis, but produces a sequence of APP's.
Probability density function of bi
![Page 91: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/91.jpg)
91
Turbo codes
● What's in a SISO?
SISO(for a CC)
0 10 1
P (bi=b )=12
P (bi=b∣r )
r
Soft demodulated valuesfrom channel
A priori probabilities(APR)
● Note that the SISO works on a bit by bit basis, but produces a sequence of APP's.
Probability density function of bi
![Page 92: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/92.jpg)
92
Turbo codes
● What's in a SISO?
SISO(for a CC)
0 10 1
P (bi=b )=12
P (bi=b∣r )
r
Soft demodulated valuesfrom channel
A priori probabilities(APR)
A posteriori probabilities(APP) updated withchannel information
● Note that the SISO works on a bit by bit basis, but produces a sequence of APP's.
Probability density function of bi
![Page 93: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/93.jpg)
93
Turbo codes
● The algorithm inside the SISO is some suboptimal version of the MAP BCJR algorithm.
– BCJR computes the APP values through a forward-backward dynamics → it works over finite length data blocks, not over (potentially) infinite length sequences (like pure CCs).
– BCJR works on a trellis: recall transition metrics, transition probabilities and so on.
– Assume the block length is N: trellis starts at , ends at .
αi ( j )=P (s( i)=s j ,r1,⋯ ,r i )
βi ( j )=P (r i+1 ,⋯,rN∣s(i)=s j )
γi ( j , k )=P (r i , s(i)=s j∣s(i−1)=sk )
s(0) s(N )
![Page 94: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/94.jpg)
94
Turbo codes
● The algorithm inside the SISO is some suboptimal version of the MAP BCJR algorithm.
– BCJR computes the APP values through a forward-backward dynamics → it works over finite length data blocks, not over (potentially) infinite length sequences (like pure CCs).
– BCJR works on a trellis: recall transition metrics, transition probabilities and so on.
– Assume the block length is N: trellis starts at , ends at .
αi ( j )=P (s( i)=s j ,r1,⋯ ,r i )
βi ( j )=P (r i+1 ,⋯,rN∣s(i)=s j )
γi ( j , k )=P (r i , s(i)=s j∣s(i−1)=sk )
FORWARD term
s(0) s(N )
![Page 95: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/95.jpg)
95
Turbo codes
● The algorithm inside the SISO is some suboptimal version of the MAP BCJR algorithm.
– BCJR computes the APP values through a forward-backward dynamics → it works over finite length data blocks, not over (potentially) infinite length sequences (like pure CCs).
– BCJR works on a trellis: recall transition metrics, transition probabilities and so on.
– Assume the block length is N: trellis starts at , ends at .
αi ( j )=P (s( i)=s j ,r1,⋯ ,r i )
βi ( j )=P (r i+1 ,⋯,rN∣s(i)=s j )
γi ( j , k )=P (r i , s(i)=s j∣s(i−1)=sk )
FORWARD term
BACKWARD term
s(0) s(N )
![Page 96: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/96.jpg)
96
Turbo codes
● The algorithm inside the SISO is some suboptimal version of the MAP BCJR algorithm.
– BCJR computes the APP values through a forward-backward dynamics → it works over finite length data blocks, not over (potentially) infinite length sequences (like pure CCs).
– BCJR works on a trellis: recall transition metrics, transition probabilities and so on.
– Assume the block length is N: trellis starts at , ends at .
αi ( j )=P (s( i)=s j ,r1,⋯ ,r i )
βi ( j )=P (r i+1 ,⋯,rN∣s(i)=s j )
γi ( j , k )=P (r i , s(i)=s j∣s(i−1)=sk )
FORWARD term
BACKWARD term
TRANSITION
s(0) s(N )
![Page 97: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/97.jpg)
97
Turbo codes
● The algorithm inside the SISO is some suboptimal version of the MAP BCJR algorithm.
– BCJR computes the APP values through a forward-backward dynamics → it works over finite length data blocks, not over (potentially) infinite length sequences (like pure CCs).
– BCJR works on a trellis: recall transition metrics, transition probabilities and so on.
– Assume the block length is N: trellis starts at , ends at .
αi ( j )=P (s( i)=s j ,r1,⋯ ,r i )
βi ( j )=P (r i+1 ,⋯,rN∣s(i)=s j )
γi ( j , k )=P (r i , s(i)=s j∣s(i−1)=sk )
FORWARD term
BACKWARD term
TRANSITION
s(0) s(N )
Remember,n components
for an (n,k,ν) CC
![Page 98: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/98.jpg)
98
Turbo codes
● BCJR algorithm in action:
– Forward step i=1,...,N:
– Backward step i=N-1,...,0:
– Compute the joint probability sequence i=1,...,N:
α0 ( j )=P (s(0)=s j ) ; αi ( j )=∑k=1
2ν
αi−1 ( k )⋅γi ( k , j )
βN ( j )=P (s(N )=s j ) ; βi ( j )=∑
k=1
2ν
βi+1 (k )⋅γi+1 ( j ,k )
P (s(i−1)=s j ,s(i)=sk ,r )=βi (k )⋅γi ( j ,k )⋅αi−1 ( j )
![Page 99: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/99.jpg)
99
Turbo codes
● Finally, the APP's can be calculated as:
● Decision criterion based on these APP's:
P (bi=b∣r )=1
p (r )⋅ ∑
s( i−1)→
b i=bs(i )
P (s(i−1)=s j ,s(i )=sk ,r )
log( P (bi=1∣r )P (bi=0∣r ) )=log(
∑s(i−1) →
b i=1s(i )
P (s(i−1)=s j ,s(i)=sk ,r )
∑s(i−1)
→b i=0
s(i )P (s(i−1)
=s j ,s(i)=sk , r ) ) >bi=10
<bi=0
0
![Page 100: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/100.jpg)
100
Turbo codes
● Finally, the APP's can be calculated as:
● Decision criterion based on these APP's:
P (bi=b∣r )=1
p (r )⋅ ∑
s( i−1)→
b i=bs(i )
P (s(i−1)=s j ,s(i )=sk ,r )
log( P (bi=1∣r )P (bi=0∣r ) )=log(
∑s(i−1) →
b i=1s(i )
P (s(i−1)=s j ,s(i)=sk ,r )
∑s(i−1)
→b i=0
s(i )P (s(i−1)
=s j ,s(i)=sk , r ) ) >bi=10
<bi=0
0
Its modulus is thereliability of the
decision
![Page 101: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/101.jpg)
101
Turbo codes
● How do we get γi(j,l)?
● This probability takes into account
– The restrictions of the trellis (CC).
– The estimations from the channel.
γi ( j , l )=P (r i ,s(i )=s j∣s(i−1)
=sl )=
= p (ri∣s(i )=s j , s(i−1 )
=sl )⋅P (s(i)=s j∣s(i−1 )
=sl )
![Page 102: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/102.jpg)
102
Turbo codes
● How do we get γi(j,l)?
● This probability takes into account
– The restrictions of the trellis (CC).
– The estimations from the channel.
γi ( j , l )=P (r i ,s(i )=s j∣s(i−1)
=sl )=
= p (ri∣s(i )=s j , s(i−1 )
=sl )⋅P (s(i)=s j∣s(i−1 )
=sl )
=0 if transition is not possible=1/2k if transition is possible
(binary trellis, k inputs)
![Page 103: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/103.jpg)
103
Turbo codes
● How do we get γi(j,l)?
● This probability takes into account
– The restrictions of the trellis (CC).
– The estimations from the channel.
γi ( j , l )=P (r i ,s(i )=s j∣s(i−1)
=sl )=
= p (ri∣s(i )=s j , s(i−1 )
=sl )⋅P (s(i)=s j∣s(i−1 )
=sl )
=0 if transition is not possible=1/2k if transition is possible
(binary trellis, k inputs) in AWGN for unmodulated c
i,m
1
(2πσ2 )n /2⋅e
−∑m=1
n
( ri ,m−ci ,m )2
2σ 2
![Page 104: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/104.jpg)
104
Turbo codes
● Idea: what about feeding APP values as APR values for other decoder whose coder had the same inputs?
SISO(for CC
2)
0 10 1
P (bi=b∣r1 )P (bi=b∣r 2)
r2
From CC1 SISO
![Page 105: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/105.jpg)
105
Turbo codes
● Idea: what about feeding APP values as APR values for other decoder whose coder had the same inputs?
SISO(for CC
2)
0 10 1
P (bi=b∣r1 )P (bi=b∣r 2)
r2
From CC1 SISO
This will happenunder someconditions
![Page 106: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/106.jpg)
106
Turbo codes
● APP's from first SISO used as APR's for second SISO increase updated APP's reliability iff
– APR's are uncorrelated wrt channel estimations for second decoder.
– This is achieved by permuting input data for each encoder.
CC1
CC2
k input streams n=n1+n
2 output streams
Rate R=k/(n1+n
2)
b c=c1∪c2
Π
d
![Page 107: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/107.jpg)
107
Turbo codes
● APP's from first SISO used as APR's for second SISO increase updated APP's reliability iff
– APR's are uncorrelated wrt channel estimations for second decoder.
– This is achieved by permuting input data for each encoder.
CC1
CC2
k input streams n=n1+n
2 output streams
INTERLEAVER(permutor)
Rate R=k/(n1+n
2)
b c=c1∪c2
Π
d
![Page 108: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/108.jpg)
108
Turbo codes
● The interleaver preserves the data (b), but changes its position within the second stream (d).
– Note that this compels the TC to work with blocks of N=size(Π) bits.
– The decoder has to know the specific interleaver used at the encoder.
b1 b2 b3 b4 bN
d π (2) d π (N ) d π (3 ) d π (1 ) d π (4)
d π (i )=b i
![Page 109: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/109.jpg)
109
Turbo codes
● The mentioned process is applied iteratively (l=1,...).
– Iterative decoder → this may be a drawback, since it adds latency (delay).
– Note the feedback connection: it is the same principle as in the turbo engines (that's why they are called “turbo”!).
SISO 1 SISO 2
Π−1
Π
r2
r1
APP1(l) APR
2(l)
APP2(l)
APR1(l+1)
from channel
![Page 110: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/110.jpg)
110
Turbo codes
● The mentioned process is applied iteratively (l=1,...).
– Iterative decoder → this may be a drawback, since it adds latency (delay).
– Note the feedback connection: it is the same principle as in the turbo engines (that's why they are called “turbo”!).
SISO 1 SISO 2
Π−1
Π
r2
r1
APP1(l) APR
2(l)
APP2(l)
APR1(l+1)
from channel
![Page 111: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/111.jpg)
111
Turbo codes
● The mentioned process is applied iteratively (l=1,...).
– Iterative decoder → this may be a drawback, since it adds latency (delay).
– Note the feedback connection: it is the same principle as in the turbo engines (that's why they are called “turbo”!).
SISO 1 SISO 2
Π−1
Π
r2
r1
APP1(l) APR
2(l)
APP2(l)
APR1(l+1)
from channel
![Page 112: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/112.jpg)
112
Turbo codes
● The mentioned process is applied iteratively (l=1,...).
– Iterative decoder → this may be a drawback, since it adds latency (delay).
– Note the feedback connection: it is the same principle as in the turbo engines (that's why they are called “turbo”!).
SISO 1 SISO 2
Π−1
Π
r2
r1
APP1(l) APR
2(l)
APP2(l)
APR1(l+1)
from channel
![Page 113: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/113.jpg)
113
Turbo codes
● The mentioned process is applied iteratively (l=1,...).
– Iterative decoder → this may be a drawback, since it adds latency (delay).
– Note the feedback connection: it is the same principle as in the turbo engines (that's why they are called “turbo”!).
SISO 1 SISO 2
Π−1
Π
r2
r1
APP1(l) APR
2(l)
APP2(l)
APR1(l+1)
from channel
![Page 114: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/114.jpg)
114
Turbo codes
● The mentioned process is applied iteratively (l=1,...).
– Iterative decoder → this may be a drawback, since it adds latency (delay).
– Note the feedback connection: it is the same principle as in the turbo engines (that's why they are called “turbo”!).
SISO 1 SISO 2
Π−1
Π
r2
r1
APP1(l) APR
2(l)
APP2(l)
APR1(l+1)
from channel
Initial APR1(l=0)
is taken withP(b
i=b)=1/2
![Page 115: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/115.jpg)
115
Turbo codes
● When the interleaver is adequately chosen and the CC's employed are RSC, the typical BER behavior is
– Note the two distinct zones: waterfall region / error floor.
![Page 116: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/116.jpg)
116
Turbo codes
● The location of the waterfall region can be analyzed by the so-called density evolution method
– Based on the exchange of mutual information between SISO's.
● The error floor can be lower bounded by the minimum Hamming distance of the TC
– Contrary to CC's, TC relies on reducing multiplicities rather than just trying to increase minimum distance.
P bfloor>
wmin⋅M min
N⋅erfc(√d min R
E b
N 0)
![Page 117: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/117.jpg)
117
Turbo codes
● The location of the waterfall region can be analyzed by the so-called density evolution method
– Based on the exchange of mutual information between SISO's.
● The error floor can be lower bounded by the minimum Hamming distance of the TC
– Contrary to CC's, TC relies on reducing multiplicities rather than just trying to increase minimum distance.
P bfloor>
wmin⋅M min
N⋅erfc(√d min R
E b
N 0)
Hamming weightof the error
with minimumdistance
![Page 118: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/118.jpg)
118
Turbo codes
● The location of the waterfall region can be analyzed by the so-called density evolution method
– Based on the exchange of mutual information between SISO's.
● The error floor can be lower bounded by the minimum Hamming distance of the TC
– Contrary to CC's, TC relies on reducing multiplicities rather than just trying to increase minimum distance.
P bfloor>
wmin⋅M min
N⋅erfc(√d min R
E b
N 0)
Hamming weightof the error
with minimumdistance
Errormultiplicity(low value!!)
![Page 119: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/119.jpg)
119
Turbo codes
● The location of the waterfall region can be analyzed by the so-called density evolution method
– Based on the exchange of mutual information between SISO's.
● The error floor can be lower bounded by the minimum Hamming distance of the TC
– Contrary to CC's, TC relies on reducing multiplicities rather than just trying to increase minimum distance.
P bfloor>
wmin⋅M min
N⋅erfc(√d min R
E b
N 0)
Hamming weightof the error
with minimumdistance
Errormultiplicity(low value!!)
Interleaver gain(only if recursive
CC's!!)
![Page 120: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/120.jpg)
120
Turbo codes
● Examples of 3G TC. Note that TC's are intended for FEC...
![Page 121: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/121.jpg)
121
Low Density Parity Check codes
![Page 122: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/122.jpg)
122
Low Density Parity Check codes
● LDPC codes are just another kind of channel codes derived from less complex ones.
– While TC's were initially an extension of CC systems, LDPC codes are an extension of the concept of binary LBC, but they are not exactly our known LBC.
● Formally, an LDPC code is an LBC whose parity check matrix is large and sparse.
– Almost all matrix elements are 0!!!!!!!!!!
– Very often, the LDPC parity check matrices are randomly generated, subject to some constraints on sparsity...
– Recall that LBC relied on extreme powerful algebra related to carefully and well chosen matrix structures.
![Page 123: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/123.jpg)
123
Low Density Parity Check codes
● Formally, a (ρ,γ)-regular LDPC code is defined as the null space of a parity check matrix J⨯n H that meets these constraints:
a) Each row contains ρ 1's.
b) Each column contains γ 1's.
c) λ, the number of 1's in common between any two columns, is 0 or 1.
d) ρ and γ are small compared with n and J.
● These properties give name to this class of codes: their matrices have a low density of 1's.
● The density r of H is defined as r=ρ/n=γ/J.
![Page 124: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/124.jpg)
124
Low Density Parity Check codes
H=[11110 000 00 00 000 00 00 00 000 111100 00 000 00 00 00 000 00 001111 000 00 00 00 000 00 00 000 011110 00 00 000 00 00 000 00 00 0111110 0010 00 100 0100 00 00 001 000 100 010 000 00 100 00 0100 010 00 000 100 010 00 0010 00 00 0100 010 001 00 000 00 010 0010 00 100 0110 000 100 00 0100 00 010 001 000 010 001 000 010 00 00 0100 00 100 0010 00 001 00 0010 00 010 000 100 100 00 000 100 001 000 010 00 01
]● Example of a (4,3)-regular LPDC parity check matrix
![Page 125: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/125.jpg)
125
Low Density Parity Check codes
H=[11110 000 00 00 000 00 00 00 000 111100 00 000 00 00 00 000 00 001111 000 00 00 00 000 00 00 000 011110 00 00 000 00 00 000 00 00 0111110 0010 00 100 0100 00 00 001 000 100 010 000 00 100 00 0100 010 00 000 100 010 00 0010 00 00 0100 010 001 00 000 00 010 0010 00 100 0110 000 100 00 0100 00 010 001 000 010 001 000 010 00 00 0100 00 100 0010 00 001 00 0010 00 010 000 100 100 00 000 100 001 000 010 00 01
]● Example of a (4,3)-regular LPDC parity check matrix
15⨯20
![Page 126: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/126.jpg)
126
Low Density Parity Check codes
H=[11110 000 00 00 000 00 00 00 000 111100 00 000 00 00 00 000 00 001111 000 00 00 00 000 00 00 000 011110 00 00 000 00 00 000 00 00 0111110 0010 00 100 0100 00 00 001 000 100 010 000 00 100 00 0100 010 00 000 100 010 00 0010 00 00 0100 010 001 00 000 00 010 0010 00 100 0110 000 100 00 0100 00 010 001 000 010 001 000 010 00 00 0100 00 100 0010 00 001 00 0010 00 010 000 100 100 00 000 100 001 000 010 00 01
]● Example of a (4,3)-regular LPDC parity check matrix
This H defines a(20,7) LBC!!!15⨯20
![Page 127: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/127.jpg)
127
Low Density Parity Check codes
H=[11110 000 00 00 000 00 00 00 000 111100 00 000 00 00 00 000 00 001111 000 00 00 00 000 00 00 000 011110 00 00 000 00 00 000 00 00 0111110 0010 00 100 0100 00 00 001 000 100 010 000 00 100 00 0100 010 00 000 100 010 00 0010 00 00 0100 010 001 00 000 00 010 0010 00 100 0110 000 100 00 0100 00 010 001 000 010 001 000 010 00 00 0100 00 100 0010 00 001 00 0010 00 010 000 100 100 00 000 100 001 000 010 00 01
]● Example of a (4,3)-regular LPDC parity check matrix
This H defines a(20,7) LBC!!!
r=4/20=3/15=0.2
15⨯20
![Page 128: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/128.jpg)
128
Low Density Parity Check codes
H=[11110 000 00 00 000 00 00 00 000 111100 00 000 00 00 00 000 00 001111 000 00 00 00 000 00 00 000 011110 00 00 000 00 00 000 00 00 0111110 0010 00 100 0100 00 00 001 000 100 010 000 00 100 00 0100 010 00 000 100 010 00 0010 00 00 0100 010 001 00 000 00 010 0010 00 100 0110 000 100 00 0100 00 010 001 000 010 001 000 010 00 00 0100 00 100 0010 00 001 00 0010 00 010 000 100 100 00 000 100 001 000 010 00 01
]● Example of a (4,3)-regular LPDC parity check matrix
This H defines a(20,7) LBC!!!
r=4/20=3/15=0.2
Sparse!
15⨯20
![Page 129: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/129.jpg)
129
Low Density Parity Check codes
H=[11110 000 00 00 000 00 00 00 000 111100 00 000 00 00 00 000 00 001111 000 00 00 00 000 00 00 000 011110 00 00 000 00 00 000 00 00 0111110 0010 00 100 0100 00 00 001 000 100 010 000 00 100 00 0100 010 00 000 100 010 00 0010 00 00 0100 010 001 00 000 00 010 0010 00 100 0110 000 100 00 0100 00 010 001 000 010 001 000 010 00 00 0100 00 100 0010 00 001 00 0010 00 010 000 100 100 00 000 100 001 000 010 00 01
]● Example of a (4,3)-regular LPDC parity check matrix
This H defines a(20,7) LBC!!!
r=4/20=3/15=0.2
Sparse!
λ=0,1
15⨯20
![Page 130: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/130.jpg)
130
Low Density Parity Check codes
● Note that the J rows of H are not necessarily linearly independent over GF(2).
– To determine the dimension k of the code, it is mandatory to find the row rank of H = n-k < J.
– That's the reason why in the previous example H defined a (20,7) LBC instead of a (20,5) LBC as could be expected!
● The construction of large H for LDPC with high rates and good properties is a complex subject.
– Some methods relay on smaller Hi used as building blocks,
plus random permutations or combinatorial manipulations; resulting matrices with bad properties are discarded.
– Other methods relay on finite geometries and lot of algebra.
![Page 131: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/131.jpg)
131
Low Density Parity Check codes
● LDPC codes yield performances equal or even better than TC's, but without the problem of their relatively high error floor.
– Both LDPC codes and TC's are capacity approaching codes.
● As in the case of TC, their interest is in part related to the fact that
– The encoding can be easily done (even when H or G are large, the low density of 1's reduces complexity of the encoder).
– At the decoder side, there are powerful algorithms that can take full advantage of the properties of the LDPC code.
![Page 132: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/132.jpg)
132
Low Density Parity Check codes
● There are several algorithms to decode LDPC codes.
– Hard decoding.
– Soft decoding.
– Mixed approaches.
● We are going to examine two important instances thereof:
– Majority-logic (MLG) decoding; hard decoding, the simplest one (lowest complexity).
– Sum-product algorithm (SPA); soft decoding, best error performance (but high complexity!).
● Key concepts: Tanner graphs & belief propagation.
![Page 133: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/133.jpg)
133
Low Density Parity Check codes
● MLG decoding: hard decoding; r=c+e → received word.
– The simplest instance of MLG decoding is the decoding of a repetition code by the rule “choose 0 if 0's are dominant, 1 if otherwise”.
● Given a (ρ,γ)-regular LDPC code, for every bit position i=1,...,n, there is a set of γ rows
that have a 1 in position i, and do not have any other common 1 position among them...
Ai={h1(i) ,⋯ ,hγ
( i)}
![Page 134: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/134.jpg)
134
Low Density Parity Check codes
● We can form the set of syndrome equations
● Si gives a set of γ checksums orthogonal on e
i.
● ei is decoded as 1 if the majority of the checksums give 1;
0 in the opposite case.
● Repeating this for all i, we estimate ê, and ĉ=r+ê.
– Correct decoding of ei is guaranteed if there are less than
γ/2 errors in e.
S i={si=r⋅h j(i )T=e⋅h j
(i)T , h j( i)∈Ai , i=1,⋯, γ}
![Page 135: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/135.jpg)
135
Low Density Parity Check codes
● Tanner graphs. Example for a (7,3) LBC.
● It is a bipartite graph with interesting properties for decoding.
– A variable node is connected to a check node iff the corresponding code bit is checked by the corresponding parity sum equation.
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
![Page 136: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/136.jpg)
136
Low Density Parity Check codes
● Tanner graphs. Example for a (7,3) LBC.
● It is a bipartite graph with interesting properties for decoding.
– A variable node is connected to a check node iff the corresponding code bit is checked by the corresponding parity sum equation.
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
Variable nodes orcode-bit vertices
![Page 137: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/137.jpg)
137
Low Density Parity Check codes
● Tanner graphs. Example for a (7,3) LBC.
● It is a bipartite graph with interesting properties for decoding.
– A variable node is connected to a check node iff the corresponding code bit is checked by the corresponding parity sum equation.
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
Variable nodes orcode-bit vertices
Check nodes orcheck-sum vertices
![Page 138: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/138.jpg)
138
Low Density Parity Check codes
● Tanner graphs. Example for a (7,3) LBC.
● It is a bipartite graph with interesting properties for decoding.
– A variable node is connected to a check node iff the corresponding code bit is checked by the corresponding parity sum equation.
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
Variable nodes orcode-bit vertices
Check nodes orcheck-sum vertices
The absence of shortloops is necessary for
iterative decoding
![Page 139: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/139.jpg)
139
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
![Page 140: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/140.jpg)
140
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
![Page 141: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/141.jpg)
141
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
This process, appliediteratively and under
some rules, yields
P (ci∣λ )
![Page 142: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/142.jpg)
142
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
This process, appliediteratively and under
some rules, yields
P (ci∣λ )λ softvalues
![Page 143: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/143.jpg)
143
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
This process, appliediteratively and under
some rules, yields
P (ci∣λ )λ softvalues
![Page 144: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/144.jpg)
144
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
This process, appliediteratively and under
some rules, yields
P (ci∣λ )
N(c5): check nodes neighbors of variable node c
5
λ softvalues
![Page 145: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/145.jpg)
145
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
This process, appliediteratively and under
some rules, yields
P (ci∣λ )
N(c5): check nodes neighbors of variable node c
5
λ softvalues
![Page 146: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/146.jpg)
146
Low Density Parity Check codes
● Based on the Tanner graph of an LDPC code, it is possible to make iterative soft decoding (SPA).
● SPA is performed by belief propagation (which is an instance of a message passing algorithm).
+ + + ++++
c1 c2 c3 c4 c5 c6 c7
s1 s2 s3 s4 s5 s6 s7
“Messages” (soft values)are passed to and from
related variable andcheck nodes
This process, appliediteratively and under
some rules, yields
P (ci∣λ )
N(c5): check nodes neighbors of variable node c
5
N(s7)
λ softvalues
![Page 147: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/147.jpg)
147
Low Density Parity Check codes
● If we get P(ci | λ), we have an estimation of the codeword sent ĉ.
● The decoding aims at calculating this through the marginalization
● Brute-force approach for LDPC is impractical, hence the iterative solution through SPA. Messages interchanged at step l:
P (ci∣λ )= ∑c ' :c ' i=c i
P (c '∣λ )
μc i→ s j
(l )(ci=c )=αi , j
(l)⋅P (ci=c∣λ i )⋅ ∏
sk∈N (c i )sk≠s j
μ sk→c i
(l−1)(c i=c )
μ s j→c i
(l )(ci=c )= ∑
c ∖ ck∈N (s j )c k≠c i
P (s j=0∣ci=c ,c )⋅ ∏c ' k∈N ( s j )
μck→ s j
(l)(c ' k=ck )
![Page 148: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/148.jpg)
148
Low Density Parity Check codes
● If we get P(ci | λ), we have an estimation of the codeword sent ĉ.
● The decoding aims at calculating this through the marginalization
● Brute-force approach for LDPC is impractical, hence the iterative solution through SPA. Messages interchanged at step l:
P (ci∣λ )= ∑c ' :c ' i=c i
P (c '∣λ )
μc i→ s j
(l )(ci=c )=αi , j
(l)⋅P (ci=c∣λ i )⋅ ∏
sk∈N (c i )sk≠s j
μ sk→c i
(l−1)(c i=c )
μ s j→c i
(l )(ci=c )= ∑
c ∖ ck∈N (s j )c k≠c i
P (s j=0∣ci=c ,c )⋅ ∏c ' k∈N ( s j )
μck→ s j
(l)(c ' k=ck )
From variable nodeto check node
![Page 149: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/149.jpg)
149
Low Density Parity Check codes
● If we get P(ci | λ), we have an estimation of the codeword sent ĉ.
● The decoding aims at calculating this through the marginalization
● Brute-force approach for LDPC is impractical, hence the iterative solution through SPA. Messages interchanged at step l:
P (ci∣λ )= ∑c ' :c ' i=c i
P (c '∣λ )
μc i→ s j
(l )(ci=c )=αi , j
(l)⋅P (ci=c∣λ i )⋅ ∏
sk∈N (c i )sk≠s j
μ sk→c i
(l−1)(c i=c )
μ s j→c i
(l )(ci=c )= ∑
c ∖ ck∈N (s j )c k≠c i
P (s j=0∣ci=c ,c )⋅ ∏c ' k∈N ( s j )
μck→ s j
(l)(c ' k=ck )
From variable nodeto check node
From check nodeto variable node
![Page 150: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/150.jpg)
150
Low Density Parity Check codes
● Note that:
– is a normalization constant.
– plugs into the SPA the values from the
channel → it is the APR info.
● : APP value.
● Based on the final values of P(ci | λ), a candidate ĉ is chosen
and ĉ·HT is tested. If 0, the information word is decoded.
αi , j(l )
P (ci=c∣λ i )
P(l) (c i=c∣λ )=βi(l)⋅P (c i=c∣λi )⋅∏
s j∈N (c i)
μs j→ ci
(l) (c i=c )
![Page 151: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/151.jpg)
151
Low Density Parity Check codes
● Note that:
– is a normalization constant.
– plugs into the SPA the values from the
channel → it is the APR info.
● : APP value.
● Based on the final values of P(ci | λ), a candidate ĉ is chosen
and ĉ·HT is tested. If 0, the information word is decoded.
αi , j(l )
P (ci=c∣λ i )
P(l) (c i=c∣λ )=βi(l)⋅P (c i=c∣λi )⋅∏
s j∈N (c i)
μs j→ ci
(l) (c i=c )
Normalization
![Page 152: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/152.jpg)
152
Low Density Parity Check codes
● LDPC BER performance examples (DVBS2 standard).
![Page 153: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/153.jpg)
153
Low Density Parity Check codes
● LDPC BER performance examples (DVBS2 standard).
Short n=16200
![Page 154: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/154.jpg)
154
Low Density Parity Check codes
● LDPC BER performance examples (DVBS2 standard).
Short n=16200
Long n=64800
![Page 155: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/155.jpg)
155
Coded modulations
![Page 156: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/156.jpg)
156
Coded modulations
● We have considered up to this point channel coding and decoding isolated from the modulation process.
– Codewords feed any kind of modulator.
– Symbols go through a channel (medium).
– The info recovered from received modulated symbols is fed to the suitable channel decoder
● As hard decisions.● As soft values (probabilistic estimations).
– The abstractions of BSC(p) (hard demodulation) or soft values from AWGN ( ⋉ exp[-|r
i-s
j|2/(2σ2)] ) -and the like for
other cases- are enough for such an approach.● Note that there are other important channel kinds not
considered so far.
![Page 157: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/157.jpg)
157
Coded modulations
● Coded modulations are systems where channel coding and modulation are treated as a whole.
– Joint coding/modulation.
– Joint decoding/demodulation.
● This offers potential advantages (recall the improvements made when the demodulator outputs more elaborated information -soft values vs. hard decisions).
– We combine gains in BER with spectral efficiency!
● As a drawback, the systems become more complex.
– More difficult to design and analyze.
![Page 158: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/158.jpg)
158
Coded modulations
● TCM (trellis coded modulation).
– Normally, it combines a CC encoder and the modulation symbol mapper.
output mk
output mj
s2
s1
s3
s4
s6
s5
s7
s8
s2
s3
s4
s6
s5
s7
s8
s1
i-1 i i+1
![Page 159: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/159.jpg)
159
Coded modulations
● If the modulation symbol mapper is well matched to the CC trellis, and the decoder is accordingly designed to take advantage of it,
– TCM provides high spectral efficiency.
– TCM can be robust in AWGN channels, and against fading and multipath effects.
● In the 80's, TCM become the standard for telephone line data modems.
– No other system could provide better performance over the twisted pair cable before the introduction of DMT and ADSL.
● However, the flexibility of providing separated channel coding and modulation subsystems is still preferred nowadays.
– Under the concept of Adaptive Coding & Modulation (ACM).
![Page 160: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/160.jpg)
160
Coded modulations
● Other possibility of coded modulation, evolved from TCM and from the concatenated coding & iterative decoding framework is Bit-Interleaved Coded Modulation (BICM).
– What about if we use an interleaver between the channel coder (normally a CC) and the modulation symbol mapper?
– A soft demodulator can also accept APR values and update as APP's its soft outputs in an iterative process!
CC Π
Softdemapper
Channel corruptedoutputs
APR values(interleaved
from CC SISO)
APP values(to interleaverand CC SISO)
![Page 161: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/161.jpg)
161
Coded modulations
● As TCM, BICM has special good behavior (even better!)
– In channels where spectral efficiency is required.
– In dispersive channels (multipath, fading).
– Iterative decoding yields a steep waterfall region.
– Being a serial concatenated system, the error floor is very low (contrary to the parallel concatenated systems).
● BICM has already found applications in standards such as DVB-T2.
● The drawback is the higher latency and complexity of the decoding.
![Page 162: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/162.jpg)
162
Coded modulations
● Examples of BICM.
![Page 163: Introduction to Channel Coding](https://reader033.vdocuments.mx/reader033/viewer/2022060115/557ccce5d8b42a43438b4fdd/html5/thumbnails/163.jpg)
163
References
● S. Lin, D. Costello, ERROR CONTROL CODING, Prentice Hall, 2004.
● S. B. Wicker, ERROR CONTROL SYSTEMS FOR DIGITAL COMMUNICATION AND STORAGE, Prentice Hall, 1995.