hierarchies of local-optimality characterizations in ... · tanner graphs and tanner codes x v1 x...
TRANSCRIPT
Hierarchies of Local-Optimality Characterizations
in Decoding Tanner Codes
Nissim Halabi Guy Even
School of Electrical Engineering, Tel-Aviv University
July 6, 2012
1/17
Tanner Graphs and Tanner Codes
v1x1
v2x2
v4x4
v3x3
v5x5
v6x6
v7x7
v8x8
v10x10
x9 v9
G = (V ∪ J , E)
Variable Nodes Local-Code NodesV J
C4 C4
C1 C1
C2 C2
C3 C3
C5 C5
Tanner code C(G, CJ ) representedby bipartite graph
x ∈ C(G, CJ ) iff x ∈ Cj for everyj ∈ {1, . . . , J}
degrees: can be regular, irregular,bounded, or arbitrary
can allow arbitrary linear localcodes
minimum local distanced∗ , minj dj
2/17
Decoding of Tanner Codes over MBIOS Channels
ChannelEncoder
ChannelDecodercodeword
Noisy Channelnoisy codeword
λ(y) ∈ RN
u ∈ {0, 1}k
c ∈ {0, 1}Nu ∈ {0, 1}k c ∈ C ⊂ {0, 1}N
Memoryless binary-input output-symmetric channel (MBIOS)channels characterized by a log-likelihood ratio (LLR)observations λ:
λi(yi) , ln
(
Pr(yi | ci = 0)
Pr(yi | ci = 1)
)
ML-decoding:ml(λ) , argmin
x∈conv(C)〈λ, x〉
LP-decoding [following Feldman-Wainwright-Karger’05]:
lp(λ) , argminx∈P(G,CJ )
〈λ, x〉
where P(G, CJ ) , generalized fundamental polytope of aTanner code C(G, CJ )
3/17
Local-Optimality: Sufficient Condition for Successful
Decoding of Finite-Length Codes
Set of deviations B(w)d ⊂ R
N : finite set of vectorscorresponding to projections of w-weighted d-trees incomputation trees with height 2h of the Tanner graph
Definition ([Even-H’11])
A codeword x ∈ C is (h,w, d)-locally optimal w.r.t. λ ∈ RN if for
all vectors β ∈ B(w)d ,
〈λ, x⊕ β〉 > 〈λ, x〉
Theorem ([Even-H’11])
Let 2 6 d 6 d∗. If x ∈ C is (h,w, d)-locally optimal w.r.t. λ, then:
1 x is the unique ML codeword w.r.t. λ
2 x is the unique optimal solution of the LP-decoder w.r.t. λ
4/17
Parameters (h, w, d) for Deviations of Local-Optimality
B(w)d ⊂ R
N : finite set of projections of w-weighted d-treeswith height 2h in computation trees of the Tanner graph
h ∈ N - tree height /2w ∈ R
h+ - level weights
d ∈ N - degree of local-codes nodes in the sub-tree, 2 6 d 6 d∗
h = 2
wT (p)w1
w2 3-tree (d = 3)
An hierarchy statement: lo with parameter h ⇒ lo withparameter h′ > h
5/17
Height Hierarchy of Local-Optimality: Motivation
An iterative decoding algorithm (nwms) is guaranteed todecode lo-certified codeword in h iterations [Even-H’11]
Questions: what is the effect of increasing the number ofiterations? even when number of iterations exceeds the girth?
Theorem (Hierarchy of local-optimality based on height)
An (h,w, d)-strongly locally optimal codeword x w.r.t. λ is also
(k · h, w, d)-strongly locally optimal w.r.t. λ for any k ∈ N
x is slo with height parameter h ⇒ Iterative message-passingdecoding by nwms is guaranteed to decode the ML-certified xafter k · h iterations ∀k ∈ N+
Insight on convergence: If a codeword x is slo-certified afterh iterations, then x is the outcome of nwms infinitely manytimes
6/17
Strong Local-Optimality
Definition ((h,w, d)-Strong Local Optimality)
A codeword x ∈ C is (h,w, d)-strongly locally optimal w.r.t.
λ ∈ RN if for all vectors β ∈ B
(w)d ,
〈λ, x⊕ β〉 > 〈λ, x〉
B(w)d ⊂ R
N : finite set of projections of w-weighted reducedd-trees in computation trees with height h of the Tanner graphReduced: degT (root) = degG(root)− 1 (as if the root itselfhangs from an edge)
3−treedefines deviations for
local−optimalitydefines deviations for
Reduced 3−tree
strong local−optimality7/17
Strong Local-Optimality vs. Local-Optimality
Set of pairs (x, λ) s.t. x is locally-optimal w.r.t. λ:
loC(h,w, d) ,{
(x, λ) ∈ C ×R | x is (h,w, d)−lo w.r.t. λ}
Set of pairs (x, λ) s.t. x is strongly locally-optimal w.r.t. λ:
sloC(h,w, d) ,{
(x, λ) ∈ C×R | x is (h,w, d)−slo w.r.t. λ}
Lemma
sloC(h,w, d) ⊆ loC(h,w, d)
Corollary
slo ⇒ unique ml
slo ⇒ unique lp opt.
Empirically: sloapproaches lo as hincreases (h >> girth(G))
1 10 100 3200
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
h
|LO0N ,Λp(h, 1h, 2)|, p = 0.04
|SLO0N ,Λp(h, 1h, 2)|, p = 0.04
|LO0N ,Λp(h, 1h, 2)|, p = 0.05
|SLO0N ,Λp(h, 1h, 2)|, p = 0.05
|LO0N ,Λp(h, 1h, 2)|, p = 0.06
|SLO0N ,Λp(h, 1h, 2)|, p = 0.06
girth=6h=3
8/17
Height Hierarchy of Local-Optimality
Theorem (h-hierarchy of local-optimality)
For every k ∈ N and geometric level-weights w,
sloC(h,w, d) ⊆ sloC(k · h,w, d)
SLO0N (h, w, d)
RN
ML(λ) = 0N
LP(λ) = 0N
SLO0N (k · h, w, d)
SLO0N (2 · h, w, d)
Remark: Assume x is transmittedover an MBIOS channel with somebounded noise level, then x is slow.r.t. the received LLR with highprobability
Interesting because an iterativemessage-passing decodingalgorithm is guaranteed to find slo
codewords
9/17
Proof Method for Height Hierarchy of Local-Optimality
Proof via contrapositive statement:
Lemma (Symmetry of slo)
x is slo w.r.t. λ iff 0N is slo w.r.t. λ0 , (−1)x ∗ λ
x not (k · h,w, d)-slo w.r.t. λ
⇐⇒ 0N is not (k · h,w, d)-slo w.r.t. λ0
[symmetry]
⇐⇒ ∃β based on reduced d-tree of height2 · k · h s.t. 〈λ0, β〉 < 0 [slo]
⇒ ∃β′ based on reduced d-tree of height2 · h s.t. 〈λ0, β′〉 < 0 [“averaging”]
⇐⇒ 0N is not (h,w, d)-slo w.r.t. λ0 [slo]
⇐⇒ x not (h,w, d)-slo w.r.t. λ[symmetry]
pi
T
T i
2h
2h
2h
2 · k · h
10/17
Implications of Height Hierarchy to Message-Passing
Decoding
Setting:
Irregular Tanner codes
Local-codes = single parity-check code
Local-optimality: d = 2, arbitrary height h (not limited bygirth)
Normalized Weighted Min-Sum (NWMS) Algorithm [Even-H’11]
Normalize: take care of irregular degrees
Weighted: allow level weights
Min-Sum: based on Max-Product/Min-Sum algorithm
nwms(λ, h,w):
Input: λ ∈ RN - LLRs from the channel
h ∈ N - number of iterationsw ∈ R
h+ - level weights
Output: x ∈ {0, 1}N
11/17
BP-Based Decoding - nwms(λ, h, w): Init
Initialize:
Init check-to-variable messages with 0:
∀C ∈ J , ∀v ∈ N (C) : µ(−1)C→v ← 0
12/17
BP-Based Decoding - nwms(λ, h, w): Iterations
Iterate: for ℓ = 0 to h− 1:
message: variable node → check node
C
µ v→C
v µ(ℓ)v→C ←
wh−ℓ
degG(v)λv+
1
degG(v)− 1
∑
C′∈N (v)\{C}
µ(ℓ−1)C′→v
[degree normalization, level weights]
message: check node → variable node
C
v µC→
v
µ(ℓ)C→v ←
(
∏
u∈N (C)\{v}
sign(
µ(ℓ)u→C
)
)
· minu∈N (C)\{v}
{
|µ(ℓ)u→C |
}
13/17
BP-Based Decoding - nwms(λ, h, w): Decision
Decision:
for all v ∈ V do
µv ←∑
C∈N (v) µ(h−1)C→v
xv ←
{
0 if µv > 0,
1 otherwise.end for
14/17
NWMS: Decoding Guarantee via Local-Optimality
Theorem (Even-H’11)
If x is (h,w, 2)-locally optimal w.r.t. λ then nwms(λ, h,w)returns x
Height Hierarchy of Local-Optimality Implies:
If nwms finds an slo certified codeword x after h iterations, then
1 nwms outputs x every k · h iterations (infinitely many times)
2 nwms never outputs a codeword y 6= x for any number ofiterations
Holds for all h, decoding guarantee not limited by the girth!
No issue of convergence/divergence (as in density evolution)
ML-certificate by local-optimality
15/17
Degree Hierarchy of Local-Optimality Characterization
What is the effect of increasing the minimum distance of thelocal codes in Tanner codes?
Theorem (Hierarchy of local-optimality based on degrees)
An (h,w, d)-locally optimal codeword x w.r.t. λ is also
(h,w, d′)-locally optimal w.r.t. λ for any degree parameter d′ > d
Insight on the improvement of suboptimal decodings ofexpander codes as the minimum distance of local codesincreases.
16/17
Summary
Conclusions
Hierarchies of Local Optimality:
1 Degree hierarchy ⇒ take d as large as possible
2 Strong local-optimality ⇒ local-optimality (⇒ lp-opt. ⇒ML-opt.)
3 Height hierarchy of local-optimality
A strongly locally optimal codeword is infinitely often stronglylocally optimal (w.r.t. height parameter)A BP-Based algorithm nwms decodes x with slo certificateafter h iterations ⇒ nwms decodes x with slo certificateevery k · h iterations, for every k ∈ N
Open Questions
Height hierarchies for belief-propagation (sum-product)algorithm and other BP-based algorithms [no probabilisticassumptions as in monotonicity of DE]
17/17