Channelcoding
WatcharapanSuwansan5suk
#11EIE467DigitalCommunica5ons
KingMongkut’sUniversityofTechnologyThonburi
Chapter8:Detec5on,coding,anddecoding
Today’stopicsconcernwiththechannelencoderandchanneldecoder
Sampler
message(inputwaveform)
Modulator
Interpolator Demodulator
message(outputwaveform)
Channel
transmiNer
receiver
binaryinterface
analogsequence
symbolsequence
Sourceencoder
Sourcedecoder
Quan5zer
Tablelookup
2
Channeldecoder
Channelencoder
Topics• Blockcoding(§8.5)• Thenoisy-channelcodingtheorem(§8.6)
3
AZerthislecture,youwillbeableto• State the difference between a soZ decision and a harddecision
• Listallcodewordsofablockcodeforasmallgeneratormatrix• Decodeanyblockcodeusinganop5malbutnaiveapproach,theminimum-distancerule
• Understandanopera5onalmeaningofthechannelcapacity• ComputethecapacityoftheAWGNchannel• Giveanexampleofchannelcodesthatachievethecapacity
4
BACKGROUNDONCHANNELCODING
5
Channelencoderaddsredundancytocombatnoise
• Thisexampleshowsarepe55oncode,whereeachinputbitisrepeated35mes
Channelencoder
010... 000111000...
6
Channeldecodercorrectsanyerrorandremoveredundancy
• Weassumethroughoutthislecturethatthechanneldecoderusestheminimum-distancerule
• For the repe55on code, theminimum-distance rule outputsthemajoritybit(amongagroupof3bits)
Channeldecoder
010... 000111000...
Channeldecoder
010... 010111000...Anotherexample:
Anerroroccursattheinput
7
Inputtothechanneldecodercanbebitsorrealnumbers
• Forsimplicity,assumePAMthroughoutthislecture
8
Channeldecoder
010... 011..
bits
Iftheinputisasequenceofbits(0’s,1’s),wesaythatthedemodulatorhasperformedaharddecision
Iftheinputisasequenceofrealnumbers,thedemodulatorhasperformaso,decision
Channeldecoder
100... −0.31.83.1...
realnumbers
Toimplementaharddecision,usethesamestructureofthedemodulatorthatwecovered
• Thesignaldecoderwhichturnsthesequenceofrealnumbersintoasequenceofbits
9
bits
signalencoder{0, 1} ! ±a
U0, U1, U2, . . .
Gaussiannoise
Z0, Z1, Z2, . . .V0, V1, V2, . . .
channelencoder
Cm =
(1, if Vm � 0
0, if Vm < 0
signaldecoder
C0, C1, C2, . . .
bits
B0, B1, B2, . . .
binaryinterface
channeldecoder
±a
realnumber
Toimplementaso,decision,removethesignalencoder
10
bitssignal
encoder{0, 1} ! ±a
U0, U1, U2, . . .
Gaussiannoise
Z0, Z1, Z2, . . .V0, V1, V2, . . .
channelencoder
B0, B1, B2, . . .
binaryinterface
realnumbers
channeldecoder
±a
realnumber
CASESTUDY:REPETITIONCODETHATREPEATSEACHBIT3TIMES
11
Considerachannelencoderthatrepeatseachinputbit35mes
• Thischannelcodeiscalledarepe55oncode• Eachoutput“000”or“111”iscalledacodeword
12
Channelencoder(repe55oncode)
0 000
Channelencoder(repe55oncode)
1 111
acodeword
acodeword
Example1:Channeldecoder(harddecision)
• Thedemodulatorperformsaharddecision• Thechanneldecoderusestheminimum-distancerule
• Ques5on:whatistheoutputfromthechanneldecoder?
• Answer:0
13
Channeldecoder(repe55oncode)
? 001
(Solu5on)twohypotheses:000and111
• The channel decoder decideswhether the input is closer indistanceto[000]orto[111]
14
channelencoder
channeldecoder
000 signalencoder
�a,�a,�a0
signaldecoder
000
noiselesschannel
channeldecoder
channelencoder
1111
signaldecoder
111
noiselesschannel
signalencoder
a, a, a
Solu5on:thedistancebetweeneachhypothesisand[001]
15
inputtochannelencoder
hypothesis:outputfromchannelencoder(codeword)
distance
0
1
c
= 1MinimumdistanceSodecode[001]tobit0
|c� v|
���⇥0 0 0
⇤�⇥0 0 1
⇤���
=p
(0� 0)2 + (0� 0)2 + (0� 1)2
���⇥1 1 1
⇤�⇥0 0 1
⇤���
=p
(1� 0)2 + (1� 0)2 + (1� 1)2
=p2
⇥1 1 1
⇤
111orasavector
⇥0 0 0
⇤
000orasavector
v =⇥0 0 1
⇤
Example2:Channeldecoder(soZdecision)
• Themodulatorusestheamplifica5onfactor• ThedemodulatorperformasoNdecision• Thechanneldecoderusestheminimum-distancerule.
• Ques5on:whatistheoutputfromthechanneldecoder?
• Answer:1
16
Channeldecoder? 5.17.7−4.5
a = 4
(Sol.)2hypotheses:[−4−4−4],[444]
• The channel decoder decideswhether the input is closer indistanceto[−4−4−4]orto[444]
17
channelencoder
channeldecoder
0000
channeldecoder
channelencoder
1111
signalencoder
�4,�4,�4
{0, 1} ! ±4
signalencoder
{0, 1} ! ±4
4, 4, 4
noiselesschannel
444
noiselesschannel
−4−4−4
Solu5on:computethedistancebetweeneachhypothesisand[5.17.7−4.5]
18
inputtochannelencoder
codeword Hypothesis Distancebetweenhypothesisand[5.17.7−4.5]
0
1
c
⇥�4 �4 �4
⇤
⇥4 4 4
⇤ ���⇥4 4 4
⇤�
⇥5.1 7.7 �4.5
⇤���
⇡ 9.3
���⇥�4 �4 �4
⇤�
⇥5.1 7.7 �4.5
⇤���
⇡ 14.8
Minimumdistance
Sodecode[5.17.7−4.5]tobit1
⇥1 1 1
⇤
111orasavector
⇥0 0 0
⇤
000orasavector
Wecancomputeprobabili5es.oferrorforsoZandharddecisions
19
X 2 {0, 1}channelencoder
channeldecoder
Y 2 {0, 1}PAMdemod.
(repe55oncode)
P {X 6= Y }
AWGNChannel
noisePSD=�2
PAMmod.
amplifica5onfactor=a
Eachbitisrepeatedn5mes,(nisassumedtobeodd:n=1,or3,or5,...)
Incaseyouarecurious,theprobabili5esoferrorequals...
• Detailsofcalcula5onarenotcoveredinthiscourse
20
perr(hard decision) =
nX
m=dn/2e
✓n
m
◆hQ⇣ a
�
⌘im h1�Q
⇣ a
�
⌘in�m
ceiling(roundup),e.g.,d1.3e = 2d4e = 4
perr(soft decision) = Q⇣p
na
�
⌘
nisthenumberofrepeatedbits(assumetobeodd,)n = 1, 3, 5, . . .
−10 −5 0 5 10
10−4
10−3
10−2
10−1
100
Comparingtheirprobabilityoferrors,soZdecisionisbeNerthanharddecision
21
harddecision
soZdecision
ProbabilityoferrorP {B 6= C}
SNRperbit(dB)a2
�2
Thelargerthenumberofrepeatedbits,thelowertheprobabilityoferror
• Adrawbackofalarge#ofrepe55onsisaslowdatarate
22
−10 −5 0 5 10
10−4
10−3
10−2
10−1
100
SNRperbit(dB)a2
�2
ProbabilityoferrorP {B 6= C}
hardsoZ
hardsoZ
repeatn=35mes
n=195mes
INTRODUCTIONTOTHECAPACITYOFTHEAWGNCHANNEL
23
Asharptransi5onexistsbetweenthedatarateandtheprobabilityoferror
• The plot is for the best channel code, whichminimizes theprobabilityoferroratagivendatarate
24
Thistransi5onpointiscalledthechannelcapacity
probabilityoferror
0datarate(transmissionspeed)bit/sec
aposi5venumber
C
CapacityoftheAWGNchannelequals...
• ClaudeE.Shannondiscoveredthisexpressionin1949
25
bit/sec
Transmissionbandwidth(Hz)
0.5×thenoisePSD(waN/Hz)
Receivedpower(waN) Thisra5oisthereceivedsignal-to-noisera5o(SNR)
C = W log2
✓1 +
P
WN0
◆
Thechannelcapacityisthemaximumreliablyspeed
Theorem(Channelcodingtheorem):• IfthetransmiNertransmitsatthedatarate<C(bit/sec),then– theprobabilityoferrorcanbemadearbitrarilysmall– byusingasuitablechannelcoding
• IfthetransmiNertransmitsatthedatarate>C(bit/sec),then– theprobabilityoferrorbecomes1– regardlessofwhatchannelcodingbeingused
26
Thestatementisimprecisebutsufficientforthisintroductorycourse
Channelcodescanbeclassifiedinto2majorclasses:blockandconvolu5onal
27
CharacterisYc Linearblockcode ConvoluYonalcode
Atthechannelencoder,acurrentblockofoutputbitsdependson...
Thiscourseonlycoversthelinearblockcodes
onlythecurrentblockofinputbits
boththecurrentandthepastblocksofinputbits
Methodstoencodeincludes...
• finitestatemachine• shiZregister
• matrixmul5plica5on• polynomial
mul5plica5on
Methodstodecodeincludes...
• minimum-distancerule• sum-productalgorithm
• Viterbialgorithm
Examplesofcodesinthisclass
• Repe55oncodes• Reed-Solomoncodes• Low-density-parity
checkcodes(LDPCs)
• Turbocodes
TherearetwocodesthatachievethecapacityoftheAWGNchannel
• Low-densityparitycheckcode(LDPC)– isanexampleofalinearblockcode– wasproposedbyGallager in1960butwerenotwidelystudiedforthenext20years
– appears in many standards (hard drive, digital television, etc.)duetoitsexcellentperformance
• Turbocode– isanexampleofaconvolu4onalcode– wasproposedbyBerrou,Glavieux,andThi5majshimain1993
28
LINEARBLOCKCODES
29
Checkyourbackgroundonmatrixmul5plica5on
• QuesYon1:mul5plythesematrices
• Modulo-2means:oddnumbersare1,evennumbersare0• QuesYon2:mul5plythesematrices(modulo-2)
30
⇥1 1 0
⇤2
41 0 1 10 1 0 10 0 1 0
3
5 =
⇥1 1 0
⇤2
41 0 1 10 1 0 10 0 1 0
3
5 =
Fortherepe55oncode,theencodermul5pliesaninputbitwithamatrix
• Fortheinput,theoutputis
• Fortheinput,theoutputis
31
Channelencoder(repe55oncode)
u 2 {0, 1}
G =⇥1 1 1
⇤
u = 0
uG
uG = 1⇥1 1 1
⇤=
⇥1 1 1
⇤
uG = 0⇥1 1 1
⇤=
⇥0 0 0
⇤
u = 1
G =
2
66664
3
77775
Matrixiscalledthegeneratormatrixandcontroltheoutput
• #ofbitsin≤#bitsout,i.e.• Ifwechange,theoutputchanges
32
G
G
anymatrixof0’sand1’s
#ofrows=k
=#ofbitsthatarereadintotheencoder#ofcolumns= n
=#ofbitsthatareoutpuNed
k n
Linearblockcodeencodesbymatrixmul5plica5on(modulo-2)
• Theencoderreadsbitsandcombinesthemintoavector:
• Computesthemodulo-2mul5plica5on
• Outputsthecodeword,avectorofbits:
33
n
k
c =⇥c1 c2 . . . cn
⇤
u =⇥u1 u2 . . . uk
⇤
=⇥c1 c2 . . . cn
⇤
c = uG thegeneratormatrix
Mnemonic:codewordconsistsof0’sand1’ssoweneedmodulo-2
Inthiscourse,weonlyconsider.inthesystema4cform
34
G
G =
2
66664
1 . . . 0
1...
.... . .
0 . . . 1
����������
3
77775anymatrixof0’sand1’s
iden5tymatrixofsize
rowsk
columnsn
k ⇥ k
=⇥I | P
⇤
Forasystema5c,thecodeword.consistsoftwoparts
• Example:for,andforinput• Theoutputis
35
G uG
uG = u⇥I | P
⇤=
⇥u | uP
⇤
Thecopyofinputbits
paritycheck
G =
1 0 10 1 1
�u =
⇥u1 u2
⇤
=⇥u1 u2 u1 + u2
⇤
paritycheck
modulo-2addi5on
c =⇥u1 u2
⇤ 1 0 10 1 1
�
Thecopyofinputbits
Example2:Linearblockencoding• Supposethegeneratormatrixis• Then,thecodewordforeachinputis
36
input output(codeword)u c = uG
⇥0 0
⇤ ⇥0 0 0
⇤
⇥0 1 1
⇤
⇥1 0 1
⇤
⇥1 1 0
⇤
⇥0 1
⇤
⇥1 0
⇤
⇥1 1
⇤
G =
1 0 10 1 1
�
0 + 0
0 + 1
1 + 0
1 + 1
(con5nued)Example2
• Supposetheinputtochannelencoderis001011• Then,theoutputis
37
Channelencoder
G =
1 0 10 1 1
�
00 1011
Example3• Supposethegeneratormatrixis
• Iftheinputtochannelencoderis
• Then,thecorrespondingcodewordis
38
G =
2
41 0 0 1 0 1 10 1 0 1 1 1 00 0 1 0 1 1 1
3
5
u =⇥0 1 1
⇤u1 u2 u3
uG = 0 1 1
= u1 + u2 = 0 + 1
1
= u2 + u3 = 1 + 1
0
= u1 + u2 + u3 = 0 + 1 + 1
0 1
= u1 + u3 = 0 + 1
[ ]
Ques5on:RefertothegeneratormatrixinExample3
• Supposethecodewordis
• Whatistheinput?
• Answer:
• Thefirstbitsofcodewordareinputbits.
39
uG
uG =⇥1 1 0 0 1 0 1
⇤
u
u =⇥1 1 0
⇤
G =
2
41 0 0 1 0 1 10 1 0 1 1 1 00 0 1 0 1 1 1
3
5
k = 3
Todecodetheblockcode,usetheminimum-distancerule
• Theminimum-distanceruleissimplebutslow• In prac5ce, a decoder uses a sum-product algorithm todecode– whichiscomplicated– butdecodesfastandisnear-op5mal
40
Herearestepstodecodethehard-decisionedinput
• Supposethedemodulatorperformsaharddecision• Sotheinputtothechanneldecoderconsistsof0’sand1’s
• Supposetheinputis• Todecode,
1. Generateallcodewords2. Findacodewordthatisnearestto:
3. Decodetothefirstbitsof
41
v =⇥v1 v2 . . . vn
⇤
k
(Selectanycodewordincaseof5e)
vc
c =⇥c1 c2 . . . cn
⇤
c
Example4:decodingahard-decisionedinput
• Supposethegeneratormatrixis• Channelencoderusestheminimum-distancerule• Theinputtochanneldecoderis• Ques5on:whatistheoutputfromthechanneldecoder?• Answer:Either00,or01,or11
42
Channeldecoder? 010
G =
1 0 10 1 1
�
v =⇥0 1 0
⇤
(con5nued)Solu5on
43
input codeword squareofdistance
u c = uG
⇥0 0
⇤ ⇥0 0 0
⇤
⇥0 1 1
⇤
⇥1 0 1
⇤
⇥1 1 0
⇤
⇥0 1
⇤
⇥1 0
⇤
⇥1 1
⇤
1
1
3
1
Thereare5es.Decodetoanyofthethreepossibili5es:⇥0 0
⇤
⇥0 1
⇤
⇥1 1
⇤
|c� v|2
v =⇥0 1 0
⇤
|c� v|2 = (0� 0)2 + (0� 1)2 + (0� 0)2 = 1
(con5nued)Matlabclear all;!G = [1 0 1; ... % generator matrix! 0 1 1 ];!v = [0 1 0]; % hard-decisioned input to channel decoder! !%----------------------------%!% channel encoder %!%----------------------------%![k,n] = size(G); % dimension of G!p2k = 2^k; % number of codewords! !% a list of k-bits: !% ulist(1) = [ 0 0 0 ... 0 0 ]!% ulist(2) = [ 0 0 0 ... 0 1 ]!% ...!% ulist(p2k) = [ 1 1 1 ... 1 1 ]!ulist = (dec2bin( 0:(p2k-1) ) == '1' );!
44
(con5nued)% a list of codewords!% codebok(m,:) = a codeword for the input ulist(m)!codebook = rem( ulist * G, 2 ) % modulo-2 multiplication! !%----------------------------%!% channel decoder %!%----------------------------%!% compute the squared distance between 'y' !% and each row of ’codebook'!dist_sq = sum( ( kron( v, ones(p2k,1) ) - codebook ).^2, 2)! !% find the row with the least distance![~, I] = min( dist_sq );! !% decode v to these bits!codebook(I,1:k)
45
HerearestepstodecodethesoZ-decisionedinput
• SupposethedemodulatorperformsasoNdecision• Sotheinputtothechanneldecoderconsistsofrealnumbers• Supposetheinputis• Todecode,
1. Generateallcodewords2. Mapeachcodewordtoavectorof’s:and3. Findavector(hypothesis)thatisnearestto:
4. Maptothecodeword:and5. Decodetothefirstbitsof
46
v =⇥v1 v2 . . . vn
⇤
±a
w v
w =⇥w1 w2 w3 . . . wn
⇤
k
w
c
�a ! 0 a ! 1
0 ! �a 1 ! a
Amplifica5onfactorofPAM(aposi5veconstant)
(Selectanycodewordincaseof5e)
Example5:decodingasoZ-decisionedinput
• Supposethegeneratormatrixis• Theamplifica5onfactorofPAMis• Channelencoderusestheminimum-distancerule• Theinputtochanneldecoderis• Ques5on:whatistheoutputfromthechanneldecoder?• Answer:01
47
a = 2
Channeldecoder? −2.21.8−1.5
G =
1 0 10 1 1
�
v =⇥�2.2 1.8 �1.5
⇤
(con5nued)Solu5on
48
input codeword hypothesis squareofdistance
u c = uG
⇥0 0
⇤ ⇥0 0 0
⇤
⇥0 1 1
⇤
⇥1 0 1
⇤
⇥1 1 0
⇤
⇥0 1
⇤
⇥1 0
⇤
⇥1 1
⇤
⇥�2 �2 �2
⇤
w |w � v|2
⇥�2 2 2
⇤
⇥2 �2 2
⇤
⇥2 2 �2
⇤
minimumdistance
14.73
12.33
44.33
17.93
|w � v|2 = (�2 + 2.2)2 + (�2� 1.5)2 + (�2 + 1.9)2 = 14.73
v =⇥�2.2 1.8 �1.5
⇤
(con5nued)Matlabclear all;!G = [1 0 1; ... % generator matrix! 0 1 1 ];!a = 2; % amplification factor !% soft-decisioned input to channel decoder!v = [-2.2 1.8 -1.5];!%----------------------------%!% channel encoder %!%----------------------------%![k,n] = size(G); % dimension of G!p2k = 2^k; % number of codewords !% a list of k-bits: !% ulist(1) = [ 0 0 0 ... 0 0 ]!% ulist(2) = [ 0 0 0 ... 0 1 ]!% ...!% ulist(p2k) = [ 1 1 1 ... 1 1 ]!ulist = (dec2bin( 0:(p2k-1) ) == '1' );!!
49
% a list of codewords!% codebok(m,:) = a codeword for the input ulist(m)!codebook = rem( ulist * G, 2 ) % modulo-2 multiplication!% convert 0 to -a and 1 to a!hypbook = a*(2*codebook-1)!%----------------------------%!% channel decoder %!%----------------------------%!% compute the squared distance between 'y' !% and each row of 'hypbook'!dist_sq = sum( ( kron( v, ones(p2k,1) ) - hypbook ).^2, 2)! !% find the row with the least distance![~, I] = min( dist_sq );!!% decode v to these bits!codebook(I,1:k)!
50
HereareacronymsandsymbolsSymbol Meaning
AWGN addi5vewhiteGaussiannoise
PAM pulseamplitudemodula5on
PSD powerspectrumdensity
generatormatrix
thenumberofrowsof
thenumberofcolumnof
amplifica5onfactorusedbyPAM
51
G
k
n
a (a > 0)
G
G
Summary• Overviewofchannelencoderanddecoder– func5ons– harddecisionvssoZdecision
• Casestudy:repe55oncode– encoder– decoder– probabilityoferror:hardvssoZdecisions
• CapacityoftheAWGNchannel– formula– channelcodingtheorem– capacityasthemaximumreliabledatarate– blockcodesvsconvolu5onalcodes– examples of capacity-achieving codes: low-density parity checkcodes,Turbocode
52
Summary• Linearblockcodes– encoder(matrixmul5plica5on)– decoder(minimum-distancerule)– hard-decision– soZ-decision– examples
53