phase transitions in large graphical models: from physics...
TRANSCRIPT
![Page 1: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/1.jpg)
Phase transitions in large graphical models:from physics to information theory and computer
science
Andrea Montanari
Stanford University
March 15, 2007
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 2: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/2.jpg)
Outline
1 An instructive story and many questions
2 The general theme: Phase transitions and Graphical models
3 A couple of applications (for time limits)Modern coding theoryRandom constraint satisfaction problems
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 3: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/3.jpg)
Outline
1 An instructive story and many questions
2 The general theme: Phase transitions and Graphical models
3 A couple of applications (for time limits)Modern coding theoryRandom constraint satisfaction problems
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 4: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/4.jpg)
Outline
1 An instructive story and many questions
2 The general theme: Phase transitions and Graphical models
3 A couple of applications (for time limits)Modern coding theoryRandom constraint satisfaction problems
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 5: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/5.jpg)
Outline
1 An instructive story and many questions
2 The general theme: Phase transitions and Graphical models
3 A couple of applications (for time limits)Modern coding theoryRandom constraint satisfaction problems
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 6: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/6.jpg)
Outline
1 An instructive story and many questions
2 The general theme: Phase transitions and Graphical models
3 A couple of applications (for time limits)Modern coding theoryRandom constraint satisfaction problems
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 7: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/7.jpg)
An instructive story and many questions
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 8: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/8.jpg)
Given a graph. . .
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 9: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/9.jpg)
. . . we want to partition its vertices . . .
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 10: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/10.jpg)
. . . to maximize the number of edges across.
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 11: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/11.jpg)
The physics version
Localized magnetic moments (spins)Antiferromagnetic interaction (graph)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 12: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/12.jpg)
The physics version
Localized magnetic moments (spins)Antiferromagnetic interaction (graph)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 13: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/13.jpg)
CS LENS
MAXCUT
NP-hard to approximate
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 14: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/14.jpg)
A few questions I like
What is the structure of low energy configurations/optimalcuts?
How does Nature find the optimum? How would we find it?
Is there a ‘physics theory’ to describe low energyconfigurations?Is there an ‘efficient algorithm’ to find optimal cuts?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 15: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/15.jpg)
A few questions I like
What is the structure of low energy configurations/optimalcuts?
How does Nature find the optimum? How would we find it?
Is there a ‘physics theory’ to describe low energyconfigurations?Is there an ‘efficient algorithm’ to find optimal cuts?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 16: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/16.jpg)
A few questions I like
What is the structure of low energy configurations/optimalcuts?
How does Nature find the optimum? How would we find it?
Is there a ‘physics theory’ to describe low energyconfigurations?Is there an ‘efficient algorithm’ to find optimal cuts?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 17: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/17.jpg)
A few questions I like
What is the structure of low energy configurations/optimalcuts?
How does Nature find the optimum? How would we find it?
Is there a ‘physics theory’ to describe low energyconfigurations?Is there an ‘efficient algorithm’ to find optimal cuts?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 18: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/18.jpg)
Start with ‘simple’ model
Connect each pair of vertices with probability 0.5 (independently)
A random partition yields
|CUT| ≈ 1
2|EDGES| .
SK (1972): How better is the optimal partition?
|CUT| = 1
2|EDGES|+ 1
4∆ |NODES|3/2 + · · ·
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 19: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/19.jpg)
Start with ‘simple’ model
Connect each pair of vertices with probability 0.5 (independently)
A random partition yields
|CUT| ≈ 1
2|EDGES| .
SK (1972): How better is the optimal partition?
|CUT| = 1
2|EDGES|+ 1
4∆ |NODES|3/2 + · · ·
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 20: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/20.jpg)
Where
∆ =1
4infq
{∫ ∞
0(1− q2(x))− φq(0, 0)
}∂φ(y ; x)
∂x= −1
2q′(x)
[∂2φ(y ; x)
∂y2+ x
(∂φ(y ; x)
∂y
)2]
φ(y ;∞) = |y |
Conjecture : Parisi (1979)
Proof : Guerra, Talagrand (2004)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 21: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/21.jpg)
CS LENS
∆ = infq F [q]Is there any hidden duality in the problem?
Flipping (spins 1 and 2) ≈ Flipping (1)+ Flipping (2)Can this fact be exploited algorithmically?
Physical dynamics is ‘local’How do local optimization algorithms work?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 22: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/22.jpg)
CS LENS
∆ = infq F [q]Is there any hidden duality in the problem?
Flipping (spins 1 and 2) ≈ Flipping (1)+ Flipping (2)Can this fact be exploited algorithmically?
Physical dynamics is ‘local’How do local optimization algorithms work?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 23: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/23.jpg)
CS LENS
∆ = infq F [q]Is there any hidden duality in the problem?
Flipping (spins 1 and 2) ≈ Flipping (1)+ Flipping (2)Can this fact be exploited algorithmically?
Physical dynamics is ‘local’How do local optimization algorithms work?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 24: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/24.jpg)
Phase transitions and Graphical models
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 25: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/25.jpg)
What is a phase transition?
An abrupt change in the state of a ‘large’ system assome control parameter is varied.
Example: water is liquid at 0.01◦C and solid at −0.01◦C.
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 26: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/26.jpg)
What is a phase transition?
A phase transition is accompanied by the emergenge oflong range correlations.
Example: water is liquid at 0.01◦C and solid at −0.01◦C.
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 27: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/27.jpg)
Probability+Locality
x1
x2 x3 x4
x5
x6
x7x8x9
x10
x11
x12
µ(x) =1
Z
∏(ij)∈E
ψij(xi , xj) , x = (x1, . . . , xn).
(statistical physics, counting, inference, estimation, coding,. . . )
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 28: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/28.jpg)
MAXCUT – Antiferromagnet
x1
x2 x3 x4
x5
x6
x7x8x9
x10
x11
x12
µ(x) =1
Z
∏(ij)∈E
exp{−βxixj} , xi ∈ {+1,−1} .
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 29: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/29.jpg)
Generic computational tasks
Optimization
x∗ = arg max∏
(ij)∈E
ψij(xi , xj) .
Partition function
Z =∑x
∏(ij)∈E
ψij(xi , xj) .
Marginals
µ(xi ) =∑x∼i
µ(x) .
Sampling.Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 30: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/30.jpg)
Generic computational tasks
Optimization
x∗ = arg max∏
(ij)∈E
ψij(xi , xj) .
Partition function
Z =∑x
∏(ij)∈E
ψij(xi , xj) .
Marginals
µ(xi ) =∑x∼i
µ(x) .
Sampling.Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 31: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/31.jpg)
Generic computational tasks
Optimization
x∗ = arg max∏
(ij)∈E
ψij(xi , xj) .
Partition function
Z =∑x
∏(ij)∈E
ψij(xi , xj) .
Marginals
µ(xi ) =∑x∼i
µ(x) .
Sampling.Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 32: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/32.jpg)
Generic computational tasks
Optimization
x∗ = arg max∏
(ij)∈E
ψij(xi , xj) .
Partition function
Z =∑x
∏(ij)∈E
ψij(xi , xj) .
Marginals
µ(xi ) =∑x∼i
µ(x) .
Sampling.Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 33: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/33.jpg)
CS LENS
Are far apart variables/particles strongly correlated?Can we approximate marginals µ(xi ) using only local information?
Can the system be found in different phases?What is the ‘qualitative’ structure of µ( · )?(conductance/concentration)
Does it relax rapidly to equilibrium?Can we sample/optimize with local algorithms?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 34: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/34.jpg)
CS LENS
Are far apart variables/particles strongly correlated?Can we approximate marginals µ(xi ) using only local information?
Can the system be found in different phases?What is the ‘qualitative’ structure of µ( · )?(conductance/concentration)
Does it relax rapidly to equilibrium?Can we sample/optimize with local algorithms?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 35: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/35.jpg)
CS LENS
Are far apart variables/particles strongly correlated?Can we approximate marginals µ(xi ) using only local information?
Can the system be found in different phases?What is the ‘qualitative’ structure of µ( · )?(conductance/concentration)
Does it relax rapidly to equilibrium?Can we sample/optimize with local algorithms?
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 36: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/36.jpg)
A couple of applications
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 37: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/37.jpg)
Modern coding theory
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 38: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/38.jpg)
To be concrete:coding over binary memoryless symmetric channels.
ENCODER CHANNEL DECODERm x y m
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 39: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/39.jpg)
ENCODER 1000010010111010︸ ︷︷ ︸N
110111011︸ ︷︷ ︸N/2
encoder ⇔ constraints over message bits
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 40: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/40.jpg)
LDPC codes [Gallager 1963, MacKay 1995]
x1 ⊕ x2 ⊕ x3 ⊕ x4 = 0 · · · x5 ⊕ x6 ⊕ x8 = 0
x1 x2 x3 x4 x5 x6 x7 x8
constraints over message bits ⇔ graphical representation
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 41: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/41.jpg)
x1 ⊕ x2 ⊕ x3 ⊕ x4 = 0 · · · x5 ⊕ x6 ⊕ x8 = 0
x1 x2 x3 x4 x5 x6 x7 x8y y y y y y y yy1 y2 y3 y4 y5 y6 y7 y8
µ(x |y) =1
Z (y)I(x1 ⊕ x2 ⊕ x3 ⊕ x4 = 0) · · · I(x5 ⊕ x6 ⊕ x8 = 0) ·
· Q(y1|x1) · · ·Q(y8|x8)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 42: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/42.jpg)
From 102 to 105 message bits
Random graph
Iterative message passing decoding
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 43: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/43.jpg)
Message Passing + Density evolution analysis
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 44: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/44.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 45: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/45.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 46: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/46.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 47: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/47.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 48: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/48.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 49: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/49.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 50: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/50.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 51: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/51.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 52: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/52.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 53: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/53.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 54: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/54.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 55: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/55.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 56: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/56.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 57: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/57.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 58: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/58.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 59: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/59.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 60: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/60.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 61: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/61.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 62: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/62.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 63: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/63.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 64: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/64.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 65: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/65.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 66: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/66.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 67: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/67.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 68: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/68.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 69: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/69.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 70: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/70.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 71: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/71.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 72: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/72.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 73: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/73.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 74: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/74.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 75: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/75.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 76: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/76.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 77: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/77.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 78: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/78.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 79: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/79.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 80: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/80.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 81: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/81.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 82: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/82.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 83: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/83.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 84: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/84.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 85: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/85.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 86: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/86.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 87: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/87.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 88: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/88.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 89: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/89.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 90: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/90.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 91: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/91.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 92: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/92.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 93: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/93.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 94: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/94.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 95: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/95.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 96: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/96.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 97: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/97.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 98: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/98.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 99: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/99.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 100: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/100.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 101: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/101.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 102: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/102.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 103: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/103.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 104: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/104.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 105: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/105.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 106: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/106.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 107: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/107.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 108: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/108.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 109: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/109.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 110: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/110.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 111: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/111.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 112: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/112.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 113: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/113.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 114: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/114.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 115: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/115.jpg)
Message Passing + Density evolution analysis
�Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 116: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/116.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 117: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/117.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 118: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/118.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 119: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/119.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 120: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/120.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 121: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/121.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 122: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/122.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 123: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/123.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 124: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/124.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 125: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/125.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 126: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/126.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 127: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/127.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 128: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/128.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 129: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/129.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 130: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/130.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 131: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/131.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 132: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/132.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 133: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/133.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 134: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/134.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 135: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/135.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 136: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/136.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 137: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/137.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 138: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/138.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 139: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/139.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 140: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/140.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 141: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/141.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 142: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/142.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 143: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/143.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 144: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/144.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 145: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/145.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 146: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/146.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 147: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/147.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 148: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/148.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 149: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/149.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise0.0 0.2 0.4 0.6 0.8 1.0
10−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 150: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/150.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise
error floor
wate
rfall
regio
n
average curve
n = 2500
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 151: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/151.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise
error floor
wate
rfall
regio
n
average curve
n = 2500
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 152: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/152.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise
error floor
wate
rfall
regio
n
average curve
n = 2500
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 153: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/153.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise
error floor
wate
rfall
regio
n
average curve
n = 2500
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 154: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/154.jpg)
Average (Message Passing) Performance
decoding error probability
channel noise
error floor
wate
rfall
regio
n
average curve
n = 2500
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
Example:
LDPC codes
n = 2500
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 155: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/155.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 100)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 156: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/156.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 250)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 157: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/157.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 500)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 158: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/158.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 1000)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 159: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/159.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 2500)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 160: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/160.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 5000)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 161: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/161.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 10000)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 162: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/162.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 25000)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 163: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/163.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 50000)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 164: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/164.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = 100000)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 165: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/165.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 166: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/166.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 167: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/167.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 168: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/168.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 169: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/169.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 170: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/170.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 171: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/171.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 172: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/172.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 173: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/173.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 174: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/174.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 175: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/175.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 176: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/176.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 177: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/177.jpg)
Asymptotic Average (Message Passing) Performance
decoding error probability
channel noise
n = 2500
n = 100
n = 250
n = 500
n = 1000
n = 5000
n = 10000
n = 25000
n = 50000
n = 100000
n=
+∞
n=
+∞
n = +∞
0.0 0.2 0.4 0.6 0.8 1.010−5
10−4
10−3
10−2
10−1
100
LDPC(n = +∞)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 178: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/178.jpg)
Constraint satisfaction problems
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 179: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/179.jpg)
k-satisfiability
N variables: x = (x1, x2, . . . , xN), xi ∈ {0, 1}
M k-clauses
(x1 ∨ x5 ∨ x7) ∧ (x5 ∨ x8 ∨ x9) ∧ · · · ∧ (x66 ∨ x21 ∨ x32)
Hereafter k ≥ 4 (ask me why at the end)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 180: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/180.jpg)
Uniform measure over solutions
F = · · · ∧ (xi1(a) ∨ x i2(a) ∨ · · · ∨ xik (a))︸ ︷︷ ︸a-th clause
∧ · · ·
µ(x) =1
Z
M∏a=1
ψa(xi1(a), . . . , xik (a))
ψa(xi1(a), . . . , xik (a)) =
{1 clause a satisfied0 otherwise
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 181: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/181.jpg)
(Factor) graph representation
x3
x1
x6
x4
x2
x5
x7
x
x
x
8
9
10
← variables xi ∈ {0, 1}
← clauses, e.g. (x5 ∨ x7 ∨ x9 ∨ x10)
Here : N = 10, M = 4
Distance: i , j ∈ {1, . . . ,N} 7→ d(i , j)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 182: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/182.jpg)
Random k-satisfiability
Each clause is uniformly random among the 2k(N
k
)possible ones.
N,M →∞ with α = M/N fixed.
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 183: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/183.jpg)
Phase transition in the structure of µ( · ) ⇔
Set of solutions of F (cavity method):
αd(k) αc(k) αs(k)
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 184: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/184.jpg)
⇔ Qualitative changes in the correlation strength⇔
α < αd(k)⇒ Weak correlations.
αd(k) < α < αc(k)⇒ Strong point-to-set correlations.
αc(k) < α < αs(k)⇒ Strong point-to-point correlations.
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 185: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/185.jpg)
⇔ Performance of message passing algorithms
α < αc(k)⇒ Belief propagation is asymptotically correct.
αc(k) < α < αs(k)⇒ Survey propagation.
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 186: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/186.jpg)
Conclusion 1: Peoples I should have cited/I should thank
Physics : M. Mezard, G. Parisi, R. Monasson, R. Zecchina,F. Ricci-Tersenghi, M. Weigt, G. Biroli, G. Semerjian,N. Sourlas. . .
CS : D. Achlioptas, D. Gamarnik, E. Mossel, E. Maneva, C. Nair,M. Bayati, D. Weitz, N. Creignou, M. Luby, A. Shokrollahi,A. Sinclair. . .
Probability : D. Aldous, M. Talagrand, A. Dembo, P. Diaconis,F. Martinelli, Y. Peres, F. Guerra, F. Toninelli. . .
EE : R. Urbanke, T. Richardson, M. Wainwright, B. Prabhakar,D. Shah, S. Tatikonda, J. Yedidia, D. Forney. . .
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science
![Page 187: Phase transitions in large graphical models: from physics ...users.cms.caltech.edu/.../montanari-caltech07.pdf · Outline 1 An instructive story and many questions 2 The general theme:](https://reader036.vdocuments.mx/reader036/viewer/2022080720/5f7a03a869cf131d1060e11f/html5/thumbnails/187.jpg)
Conclusion 2: If you want to know more about this. . .
M. Mezard, A. M., Upcoming book
T. Richardson, R. Urbanke, Modern Coding Theory,A. M., R. Urbanke, Les Houches lecture notes
2007 IT Symposium → Statistical Physics tutorial
2007 StatPhys symposium → IT Plenary Talk
google ee374
Andrea Montanari Phase transitions in large graphical models: from physics to information theory and computer science