1 markov random fields with efficient approximations yuri boykov, olga veksler, ramin zabih computer...

1

Markov Random Fields with Efficient Approximations

Yuri Boykov, Olga Veksler, Ramin ZabihComputer Science DepartmentCORNELL UNIVERSITY

2

Introduction

MAP-MRF approach

(Maximum Aposteriori Probability estimation of MRF)

• Bayesian framework suitable for problems in Computer Vision (Geman and Geman, 1984)

• Problem: High computational cost. Standard methods (simulated annealing) are very slow.

3

Outline of the talk

Models where MAP-MRF estimation is equivalent to min-cut problem on a graph • generalized Potts model• linear clique potential model

Efficient methods for solving the corresponding graph problems

Experimental results • stereo, image restoration

4

MRF framework in the context of stereo

MRF defining property:

Hammersley-Clifford Theorem:

),|(Pr),|(Pr pqpqp Nqffpqff

),(

),( ),(exp~)(Prqp

qpqp ffVf

• neighborhood relationships (n-links)

• image pixels (vertices)

pf - disparity at pixel p

),...,( 1 mfff - configuration

5

MAP estimation of MRF configuration

)Pr()|Pr(maxargˆ ffOff

p qp

qpqppp

f

ffVfOgf),(

),( ),()|(lnexpmaxargˆ

)|(Prmaxargˆ Offf

Observed data

Likelihoodfunction

(sensor noise)

Prior (MRF model)

Bayes rule

6

Energy minimization

Find that minimizes the Posterior Energy Function :f

),(

),( ),()|(ln)(qp

qpqp

p

pp ffVfOgfE

Data term

(sensor noise)

Smoothness term

(MRF prior)

7

Generalized Potts model

Clique potential

)(),( },{),( qpqpqpqp ffuffV

Penalty for discontinuity at (p,q)

Energy function

p qp

qpqppp ffufOgfE},{

},{ )(2)|(ln)(

8

Static clues - selecting

Stereo Image: White Rectangle in front of the black background

},{ qpu

constu qp },{

Disparity configurations minimizing energy E( f ):

constu qp },{

9

Minimization of E(f) via graph cuts

p-vertices(pixels)

Cost of n-link

},{},{ 2 qpqp u

Cost of t-link

pplp KlOg )|(ln},{

0

Terminals (possible disparity labels)

10

Multiway cutvertices V = pixels + terminalsedges E = n-links + t-links

• A multiway cut C yields some disparity configuration Cf

Remove a subset of edges C

• C is a multiway cut if terminals are separated in G(C)

Graph G = <V,E> Graph G(C) = <V, E-C >

11

Main Result (generalized Potts model)

Under some technical conditions on the multiway min-cut C on G gives___ that minimizes E( f ) - the posterior energy function for the generalized Potts model.

pKCf

• Multiway cut Problem: find minimum cost multiway cut C graph G

12

Solving multiway cut problem

Case of two terminals: • max-flow algorithm (Ford, Fulkerson 1964)• polinomial time (almost linear in practice).

NP-complete if the number of labels >2• (Dahlhaus et al., 1992)

Efficient approximation algorithms that are optimal within a factor of 2

13

Our algorithm

Initialize at arbitrary multiway cut C

1. Choose a pair of terminals

2. Consider connected pixels

14

Our algorithm

Initialize at arbitrary multiway cut C

1. Choose a pair of terminals

2. Consider connected pixels

3. Reallocate pixels between two terminals by running max-flow algorithm

15

Our algorithm

Initialize at arbitrary multiway cut C

1. Choose a pair of terminals

2. Consider connected pixels

3. Reallocate pixels between two terminals by running max-flow algorithm

4. New multiway cut C’ is obtained

Iterate until no pair of terminals improves the cost of the cut

16

Experimental results (generalized Potts model)

Extensive benchmarking on synthetic images and on real imagery with dense ground truth• From University of Tsukuba• Comparisons with other algorithms

17

Synthetic example

Image Correlation Multiway cut

18

Real imagery with ground truth

Ground truth

Our results

19

Comparison with ground truth

20

Gross errors (> 1 disparity)

21

Comparative results: normalized correlation

DataGross errors

22

Statistics

0

5

10

15

20

25

30

35

40

Multiway cut LOG-filteredL1

MLMHV Census Normalizedcorrelation

Gross errors

Errors

23

Related work (generalized Potts model)

Greig et al., 1986 is a special case of our method (two labels)

Two solutions with sensor noise (function g) highly restricted• Ferrari et al., 1995, 1997

24

Linear clique potential model

Clique potential

||),( },{),( qpqpqpqp ffuffV

Penalty for discontinuity at (p,q)

Energy function

p qp

qpqppp ffufOgfE},{

},{ ||2)|(ln)(ˆ

25

Minimization of via graph cuts

Cost of n-link

},{},{ 2 qpqp u

Cost of t-link

pplp KlOg )|(ln},{

)(ˆ fE

{p,q} part of graph G

a cut C yields someconfiguration Cf

cut C

26

Main Result (linear clique potential model)

Under some technical conditions on the min-cut C on gives that minimizes - the posterior energy function for the linear clique potential model.

pKCfG

)(ˆ fE

27

Related work (linear clique potential model)

Ishikawa and Geiger, 1998• earlier independently obtained a very similar

result on a directed graph Roy and Cox, 1998

• undirected graph with the same structure• no optimality properties since edge weights are

not theoretically justified

28

Experimental results (linear clique potential model)

Benchmarking on real imagery with dense ground truth• From University of Tsukuba

Image restoration of synthetic data

29

Ground truth stereo image

ground truth Generalized Potts model

Linear clique potential model

30

Image restoration

Noisy diamond image

Generalized Potts model

Linear clique potential model