Physical Fluctuomatics (Tohoku University) 1
Physical Fluctuomatics11th Probabilistic image processing by means of
physical models
Kazuyuki TanakaGraduate School of Information Sciences, Tohoku University
[email protected]://www.smapip.is.tohoku.ac.jp/~kazu/
Physical Fluctuomatics (Tohoku University) 2
Textbooks
Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 7.Kazuyuki Tanaka: Statistical-mechanical approach to image processing (Topical Review), Journal of Physics A: Mathematical and General, vol.35, no.37, pp.R81-R150, 2002.Muneki Yasuda, Shun Kataoka and Kazuyuki Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp.137-179, Advanced Communication Media CO., Ltd., December 2010 (in Japanese).
Physical Fluctuomatics (Tohoku University) 3
Contents
1. Probabilistic Image Processing
2. Loopy Belief Propagation3. Statistical Learning
Algorithm4. Summary
Physical Fluctuomatics (Tohoku University) 4
Image Restoration by Probabilistic Model
Original Image
Degraded Image
Transmission
Noise
Likelihood Marginal
PriorLikelihood
Posterior
}ageDegradedImPr{}Image OriginalPr{}Image Original|Image DegradedPr{
}Image Degraded|Image OriginalPr{
Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability.
Bayes Formula
Physical Fluctuomatics (Tohoku University) 5
Image Restoration by Probabilistic Model
Degraded
Image
i
fi: Light Intensity of Pixel iin Original Image
),( iii yxr
Position Vector of Pixel i
gi: Light Intensity of Pixel iin Degraded Image
i
Original
Image
The original images and degraded images are represented by f = (f1,f2,…,f|V|) and g = (g1,g2,…,g|V|), respectively.
Physical Fluctuomatics (Tohoku University) 6
Probabilistic Modeling of Image Restoration
PriorLikelihood
Posterior
}Image OriginalPr{}Image Original|Image DegradedPr{
}Image Degraded|Image OriginalPr{
Eij
ji ff ),(}Pr{
}Image OriginalPr{
fF
f
Random Fields
Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels.
i j
Product over All the Nearest Neighbour Pairs of Pixels
),...,,( Image Original ofVector State
),...,,( Image Original of Field Random
||21
||21
V
V
fff
FFF
f
F
Physical Fluctuomatics (Tohoku University) 7
Probabilistic Modeling of Image Restoration
PriorLikelihood
Posterior
}Image OriginalPr{}Image Original|Image DegradedPr{
}Image Degraded|Image OriginalPr{
Vi
ii fg ),(}|Pr{
}Image Original|Image DegradedPr{
fFgG
fg
Random Fieldsfi
gi
fi
gi
or
Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels.
),...,,( Image Degraded of Field Random
),...,,( Image Original of Field Random
||21
||21
V
V
GGG
FFF
G
F
),...,,( Image Degraded ofVector State
),...,,( Image Original ofVector State
||21
||21
V
V
ggg
fff
g
f
Physical Fluctuomatics (Tohoku University) 8
Bayesian Image Analysis
Eji
jiVi
ii ffgf},{
),(),(
PrPrPr
PrgG
fFfFgGgGfF
fg
fF Pr fFgG Pr gOriginalImage
Degraded Image
Prior Probability
Posterior Probability
Degradation Process
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.
E : Set of all the nearest neighbour pairs of pixels
V : Set of All the pixels
Physical Fluctuomatics (Tohoku University) 9
Estimation of Original Image
We have some choices to estimate the restored image from posterior probability. In each choice, the computational time is generally exponential order of the number of pixels.
}|Pr{maxargˆ gG iiz
i zFfi
2}|Pr{minargˆ zgGzF dzf iiii
}|Pr{maxargˆ gGzFfz
Thresholded Posterior Mean (TPM) estimation
Maximum posterior marginal (MPM) estimation
Maximum A Posteriori (MAP) estimation
if
ii fF\
}|Pr{}|Pr{f
gGfFgG
(1)
(2)
(3)
Physical Fluctuomatics (Tohoku University) 10
Prior Probability of Image Processing
Ejiji ff
Z },{
2
PR 21exp
)(1Pr
fF
2 41
Paramagnetic Ferromagnetic
Digital Images (fi=0,1,…,q1)
Sampling of Markov Chain Monte Carlo Method
V: Set of all the pixels Fluctuations increase near α=1.76….
E: Set of all the nearest-neighbour pairs of pixels
1
0
1
0
1
0 },{
2PR
1 1 ||21exp)(
q
z
q
z
q
z Ejiji
V
zzZ
Physical Fluctuomatics (Tohoku University) 11
Bayesian Image Analysis by Gaussian Graphical Model
Ejiji ff
Z },{
2
PR 21exp
)(1|Pr
fF
0005.0 0030.00001.0
Patterns are generated by MCMC.
Markov Chain Monte Carlo Method
PriorProbability
,if
E:Set of all the nearest-neighbour pairs of pixels
V:Set of all the pixels
||21},{
2PR 2
1exp)( VEji
ji dzdzdzzzZ
Physical Fluctuomatics (Tohoku University) 12
Degradation Process for Gaussian Graphical Model
2,0~ Nfg ii
Viii gf 2
22 21exp
2
1,Pr
fFgG
,, ii gf
Degraded image is obtained by adding a white Gaussian noise to the original image.
Degradation Process is assumed to be the additive white Gaussian noise.
V: Set of all the pixels
Histogram of Gaussian Random Numbers
Original Image f Gaussian Noise n
Degraded Image g
Physical Fluctuomatics (Tohoku University) 13
Bayesian Image Analysis
Eji
jiVi
ii ffgf},{
),(),(
PrPrPr
PrgG
fFfFgGgGfF
fg
fF Pr fFgG Pr gOriginalImage
Degraded Image
Prior Probability
Posterior Probability
Degradation Process
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.
E : Set of all the nearest neighbour pairs of pixels
V : Set of All the pixels
Physical Fluctuomatics (Tohoku University) 14
Bayesian Image Analysis
gG
fFfFgGgGfF
Pr
PrPrPr
fg
fF Pr fFgG Pr gOriginal Image
Degraded Image
Prior Probability
Posterior Probability
Degradation Process
if
iii fFf\
PrPrˆf
gGfFgG
Computational Complexity
画素
Marginal Posterior Probability of Each Pixel
Physical Fluctuomatics (Tohoku University) 15
Posterior Probability for Image Processing by Bayesian Analysis
Ejiji
Viii ffgf
Z },{
22
Posterior 21
21exp
,,1,,Pr
ggGfF
222
},{ 21
41
41exp, jijjiijiji ffgfgfff
1
0
1
0
1
0 },{
22Posterior
1 2 ||21
21exp,,
q
z
q
z
q
z Ejiji
Viii
V
zzgzZ g
Eji
jiji ffZ },{
},{Posterior
,),,(
1Prg
gGfF
Additive White Gaussian Noise 2
1
Physical Fluctuomatics (Tohoku University) 16
Bayesian Network and Posterior Probability of Image Processing
Eji
jiji ffZ },{
},{Posterior
,1Pr gGfF
E: Set of all the nearest neighbour pairs of pixels
V: Set of all the pixels
Probabilistic Model expressed in terms of Graphical Representations with Cycles
gGfFfFgG
gGfF
PrPrPr
Pr
Posterior Probability Bayesian Network
Physical Fluctuomatics (Tohoku University) 17
,
,
PrPr
,,,,,,Pr
},{
},{
111111
ijxxPfz
ff
fFfFfFfFfF
ji
z ijjiji
ijjiji
z
LLiiiiii
ii
gGzFgGfF
gG
Markov Random Fields
Eji
jiji ffZ },{
},{Posterior
,1Pr gGfF
Markov Random Fields
∂i: Set of all the nearest neighbour pixels of the pixel i
),( EV ),( EVii
Physical Fluctuomatics (Tohoku University) 18
Contents
1. Probabilistic Image Processing
2. Loopy Belief Propagation3. Statistical Learning
Algorithm4. Summary
Physical Fluctuomatics (Tohoku University) 19
Loopy Belief Propagation
Eji
jiji ffZ },{
},{Posterior
,1Pr gGfF
1
1
115114113112
115114113112
\11 PrPr
z
f
zMzMzMzMfMfMfMfM
fFf
gGfFgG
21
3
4
5
Probabilistic Model expressed in terms of Graphical Representations with Cycles
E: Set of all the nearest neighbour pairs of pixels
V: Set of all the pixels
Physical Fluctuomatics (Tohoku University) 20
Loopy Belief Propagation
MM
Simultaneous fixed point equations to determine the messages
1 2
1
11511411321}2,1{
11511411321}2,1{
221 ,
,
z z
z
zMzMzMzz
zMzMzMfzfM
21
3
4
5
Physical Fluctuomatics (Tohoku University) 21
})1210{,,(
,
,
1
0
1
0},{
1
0},{
,q,,,fViij
zMzz
zMfzfM
i
q
z
q
z ikiikjiji
q
z ikiikjiji
jji
i j
i
Step 1: Solve the simultaneous fixed point equations for messages by using iterative method
Belief Propagation Algorithm for Image Processing
i
ji
Step 2: Substitute the messages to approximate expressions of marginal probabilities for each pixel and compute estimated image so as to maximize the approximate marginal probability at each pixel.
})1210{,( Pr 1
0
,q,,,fVifM
fMfF iq
z ikiik
ikiik
ii
i
gG
iiqzi zPfi 1,,1,0
maxargˆ
Belief Propagation in Probabilistic Image Processing
Physical Fluctuomatics (Tohoku University) 22
Physical Fluctuomatics (Tohoku University) 23
Contents
1. Probabilistic Image Processing
2. Loopy Belief Propagation3. Statistical Learning
Algorithm4. Summary
Physical Fluctuomatics (Tohoku University) 24
Statistical Estimation of Hyperparameters
z
z
zFzFgG
gGzFgG
}|Pr{},|Pr{
},|,Pr{},|Pr{
},|Pr{max arg)ˆ,ˆ(,
gG
f g
Marginalized with respect to F
}|Pr{ fF },|Pr{ fFgG gOriginal Image
Marginal Likelihood
Degraded ImageVy
x
},|Pr{ gG
Hyperparameters , are determined so as to maximize the marginal likelihood Pr{G=g|,} with respect to , .
Physical Fluctuomatics (Tohoku University) 25
||
2
1
Vf
ff
f
Maximum Likelihood in Signal Processing
,Prmaxargˆ,ˆ,
gG
Degraded Image=Data
0 },|Pr{,0 },|Pr{ˆ,ˆˆ,ˆ
gGgG
Extremum Condition
||
2
1
Vg
gg
g
Hyperparameter
Original Image=Parameter
Incomplete Data
Marginal Likelihood
z
zFzFgGgG }|Pr{},|Pr{ },|Pr{
Physical Fluctuomatics (Tohoku University) 26
||
2
1
Vf
ff
f
Maximum Likelihood and EM algorithmDegraded Image=Data
0
,Pr,0
,Pr
ˆ,ˆˆ,ˆ
gGgG
Extemum Condition
||
2
1
Vg
gg
g
Hyperparameter
Original Image=Parameter
z
PP
Q
,,ln,,
,,
gGfFgGfF
)(),(,maxarg
)1()1( Update:Step M
)(),(, Calculate :Step E
),(ttQ
t,tα
ttQ
0
,,
0,,
,
,
Q
Q
Expectation Maximization (EM) algorithm provide us one of Extremum Points of Marginal Likelihood.
Q -function
Marginal Likelihood
Incomplete Data
z
zFzFgGgG }|Pr{},|Pr{ },|Pr{
Physical Fluctuomatics (Tohoku University) 27
Maximum Likelihood and EM algorithm
Physical Fluctuomatics (Tohoku University) 28
gCI
I
zgGzFzf
2
},,|Pr{ˆ
d
Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model
,, ii gf
otherwise0},{1
4Eji
jiji C
Multi-Dimensional Gaussian Integral Formula
Ejiji
Viii ffgf
},{
222 2
12
1exp},,|Pr{
gGfF
Posterior Probability
Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula
|V|x|V| matrix
E:Set of all the nearest-neghbour pairs of pixels
V:Set of all the pixels
Physical Fluctuomatics (Tohoku University) 29
Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model
1T
22
2
11||1
11
1Tr||
1
g
CI
CgCI
C
ttVtt
tV
t
T22
242
2
2
11
11||
1
11
1Tr||
1 gCI
CgCI
I
tt
ttVtt
tV
t
0
0.0002
0.0004
0.0006
0.0008
0.001
0 20 40 60 80 100 t
t
g f̂
gCI
Im2)()(
)(tt
t
2)(minarg)(ˆ tmztf iiz
ii
Iteration Procedure in Gaussian Graphical Model
f̂
g
Physical Fluctuomatics (Tohoku University) 30
Image Restorations by Exact Solution and Loopy Belief Propagation of Gaussian Graphical Model
.,,,maxarg1,1,
gttQtt
y
ghf ,ˆ,ˆˆ
0
0.0002
0.0004
0.0006
0.0008
0.001
0 20 40 60 80 100
Loopy Belief Propagation
Exact
0006000ˆ335.36ˆ
LBP
LBP
.
0007130ˆ624.37ˆ
Exact
Exact
.
Image Restorations by Gaussian Graphical Model
Original Image
MSE:315
MSE: 545 MSE: 447MSE: 411
MSE: 1512
Degraded Image
Lowpass Filter Median Filter
Exact
Wiener Filter
2ˆ||
1MSE
Vi
ii ffV
Belief Propagation
MSE:325
Physical Fluctuomatics (Tohoku University) 31
Original Image
MSE236MSE: 260
MSE: 372 MSE: 244MSE: 224
MSE: 1529
Degraded Image Belief Propagation
Lowpass Filter Median Filter
Exact
Wiener Filter
Image Restoration by Gaussian Graphical Model
Physical Fluctuomatics (Tohoku University) 32
Original Image
MSE:0.98893
Degraded Image
Belief Propagation
Image Restoration by Discrete Gaussian Graphical Model
Physical Fluctuomatics (Tohoku University) 33
Q=4
=1
20519.1ˆ05224.1ˆ
LBP
LBP
(t)
(t)MSE: 0.36011
SNR:1.03496 (dB)
SNR: 5.42228 (dB)
Original Image
MSE: 0.98893
Degraded Image
Belief Propagation
Image Restoration by Discrete Gaussian Graphical Model
Physical Fluctuomatics (Tohoku University) 34
Q=4
=1
20294.1ˆ02044.1ˆ
LBP
LBP
(t)
(t)MSE: 0.28796
SNR: 1.07221 (dB)
SNR: 6.43047 (dB)
Original Image
MSE:0.98893
Degraded Image
Belief Propagation
Image Restoration by Discrete Gaussian Graphical Model
Physical Fluctuomatics (Tohoku University) 35
Q=8
=1
660330ˆ96380.0ˆ
LBP
LBP
.
(t)
(t)MSE: 0.37697
SNR:1.89127 (dB)
SNR: 6.07987 (dB)
Original Image
MSE: 0.32060
MSE:0.98893
Degraded Image
Belief Propagation
Image Restoration by Discrete Gaussian Graphical Model
Physical Fluctuomatics (Tohoku University) 36
Q=8
=1
SNR:0.93517 (dB)
SNR: 5.82715 (dB)
623240ˆ85195.0ˆ
LBP
LBP
.
(t)
(t)
Physical Fluctuomatics (Tohoku University) 37
Contents
1. Probabilistic Image Processing
2. Loopy Belief Propagation3. Statistical Learning
Algorithm4. Summary
Physical Fluctuomatics (Tohoku University) 38
Summary
Image ProcessingBayesian Network for Image ProcessingDesign of Belief Propagation Algorithm for Image Processing
Physical Fluctuomatics (Tohoku University) 39
iz
LLiiiiii fFfFfFfFfFgGzF
gGfFgG
PrPr
,,,,,,Pr 111111
Practice 10-1
ii z ijjiji
ijjiji
z
fz
ff
,
,
PrPr
},{
},{
gGzFgGfF
For random fields F=(F1,F2,…,F|V|)T and G=(G1,G2,…,G|V|)T and state vectors f=(f1,f2,…,f|V|)T and g=(g1,g2,…,g|V|)T, we consider the following posterior probability distribution:
where ∂i denotes the set of all the nearest neighbour pixels of the pixel i.
and
Eji
jiji ffZ },{
},{Posterior
,1Pr gGfF
Here ZPosterior is a normalization constant and E is the set of all the nearest-neighbour pairs of nodes. Prove that
Physical Fluctuomatics (Tohoku University) 40
Practice 10-2
Viii gf 2
22 21exp
2
1Pr
fFgG
Histogram of Gaussian Random Numbers
Original Image f Gaussian Noise n Degraded Image g
Make a program that generate a degraded image g=(g1,g2,…,g|V|) from a given image f =(f1,f2,…,f|V|) in which each state variable fi takes integers between 0 and q1 by according to the following conditional probability distribution of the additive white Gaussian noise:
Give some numerical experiments for =20 and 40.
Physical Fluctuomatics (Tohoku University) 41
Practice 10-3Make a program for estimating q-valued original images f =(f1,f2,…,f|V|) from a given degraded image g =(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution for q valued images:
Give some numerical experiments.
Bij
jiij ffZ
,1PrPosterior
gGfF
222
21
81
81exp, jijjiijiij ffgfgfff
1
0
1
0
1
0
222Posterior
1 2 ||21
81
81exp
q
z
q
z
q
zjijjii
V
zzgzgzZ
}1,,2,1,0{ qfi
K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 .M. Yasuda, S. Kataoka and K. Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp.137-179, Advanced Communication Media CO., Ltd., December 2010 (in Japanese).
Make a program for estimating original images f =(f1,f2,…,f|V|) from a given degraded image g =(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution based on the Gaussian graphical model:
Give some numerical experiments.
Physical Fluctuomatics (Tohoku University) 42
Practice 10-4
Eji
jiji ffZ },{
},{Posterior
,1Pr gGfF
222
},{ 21
81
81exp, jijjiijiji ffgfgfff
),( if
||21222
Posterior 21
81
81exp Vjijjii dzdzdzzzgzgzZ
Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 and Appendix G.