1 markov random fields with efficient approximations yuri boykov, olga veksler, ramin zabih computer...
TRANSCRIPT
1
Markov Random Fields with Efficient Approximations
Yuri Boykov, Olga Veksler, Ramin ZabihComputer Science DepartmentCORNELL UNIVERSITY
2
Introduction
MAP-MRF approach
(Maximum Aposteriori Probability estimation of MRF)
• Bayesian framework suitable for problems in Computer Vision (Geman and Geman, 1984)
• Problem: High computational cost. Standard methods (simulated annealing) are very slow.
3
Outline of the talk
Models where MAP-MRF estimation is equivalent to min-cut problem on a graph • generalized Potts model• linear clique potential model
Efficient methods for solving the corresponding graph problems
Experimental results • stereo, image restoration
4
MRF framework in the context of stereo
MRF defining property:
Hammersley-Clifford Theorem:
),|(Pr),|(Pr pqpqp Nqffpqff
),(
),( ),(exp~)(Prqp
qpqp ffVf
• neighborhood relationships (n-links)
• image pixels (vertices)
pf - disparity at pixel p
),...,( 1 mfff - configuration
5
MAP estimation of MRF configuration
)Pr()|Pr(maxargˆ ffOff
p qp
qpqppp
f
ffVfOgf),(
),( ),()|(lnexpmaxargˆ
)|(Prmaxargˆ Offf
Observed data
Likelihoodfunction
(sensor noise)
Prior (MRF model)
Bayes rule
6
Energy minimization
Find that minimizes the Posterior Energy Function :f
),(
),( ),()|(ln)(qp
qpqp
p
pp ffVfOgfE
Data term
(sensor noise)
Smoothness term
(MRF prior)
7
Generalized Potts model
Clique potential
)(),( },{),( qpqpqpqp ffuffV
Penalty for discontinuity at (p,q)
Energy function
p qp
qpqppp ffufOgfE},{
},{ )(2)|(ln)(
8
Static clues - selecting
Stereo Image: White Rectangle in front of the black background
},{ qpu
constu qp },{
Disparity configurations minimizing energy E( f ):
constu qp },{
9
Minimization of E(f) via graph cuts
p-vertices(pixels)
Cost of n-link
},{},{ 2 qpqp u
Cost of t-link
pplp KlOg )|(ln},{
0
Terminals (possible disparity labels)
10
Multiway cutvertices V = pixels + terminalsedges E = n-links + t-links
• A multiway cut C yields some disparity configuration Cf
Remove a subset of edges C
• C is a multiway cut if terminals are separated in G(C)
Graph G = <V,E> Graph G(C) = <V, E-C >
11
Main Result (generalized Potts model)
Under some technical conditions on the multiway min-cut C on G gives___ that minimizes E( f ) - the posterior energy function for the generalized Potts model.
pKCf
• Multiway cut Problem: find minimum cost multiway cut C graph G
12
Solving multiway cut problem
Case of two terminals: • max-flow algorithm (Ford, Fulkerson 1964)• polinomial time (almost linear in practice).
NP-complete if the number of labels >2• (Dahlhaus et al., 1992)
Efficient approximation algorithms that are optimal within a factor of 2
13
Our algorithm
Initialize at arbitrary multiway cut C
1. Choose a pair of terminals
2. Consider connected pixels
14
Our algorithm
Initialize at arbitrary multiway cut C
1. Choose a pair of terminals
2. Consider connected pixels
3. Reallocate pixels between two terminals by running max-flow algorithm
15
Our algorithm
Initialize at arbitrary multiway cut C
1. Choose a pair of terminals
2. Consider connected pixels
3. Reallocate pixels between two terminals by running max-flow algorithm
4. New multiway cut C’ is obtained
Iterate until no pair of terminals improves the cost of the cut
16
Experimental results (generalized Potts model)
Extensive benchmarking on synthetic images and on real imagery with dense ground truth• From University of Tsukuba• Comparisons with other algorithms
17
Synthetic example
Image Correlation Multiway cut
18
Real imagery with ground truth
Ground truth
Our results
19
Comparison with ground truth
20
Gross errors (> 1 disparity)
21
Comparative results: normalized correlation
DataGross errors
22
Statistics
0
5
10
15
20
25
30
35
40
Multiway cut LOG-filteredL1
MLMHV Census Normalizedcorrelation
Gross errors
Errors
23
Related work (generalized Potts model)
Greig et al., 1986 is a special case of our method (two labels)
Two solutions with sensor noise (function g) highly restricted• Ferrari et al., 1995, 1997
24
Linear clique potential model
Clique potential
||),( },{),( qpqpqpqp ffuffV
Penalty for discontinuity at (p,q)
Energy function
p qp
qpqppp ffufOgfE},{
},{ ||2)|(ln)(ˆ
25
Minimization of via graph cuts
Cost of n-link
},{},{ 2 qpqp u
Cost of t-link
pplp KlOg )|(ln},{
)(ˆ fE
{p,q} part of graph G
a cut C yields someconfiguration Cf
cut C
26
Main Result (linear clique potential model)
Under some technical conditions on the min-cut C on gives that minimizes - the posterior energy function for the linear clique potential model.
pKCfG
)(ˆ fE
27
Related work (linear clique potential model)
Ishikawa and Geiger, 1998• earlier independently obtained a very similar
result on a directed graph Roy and Cox, 1998
• undirected graph with the same structure• no optimality properties since edge weights are
not theoretically justified
28
Experimental results (linear clique potential model)
Benchmarking on real imagery with dense ground truth• From University of Tsukuba
Image restoration of synthetic data
29
Ground truth stereo image
ground truth Generalized Potts model
Linear clique potential model
30
Image restoration
Noisy diamond image
Generalized Potts model
Linear clique potential model