distributed cosegmentation via submodular optimization on anisotropic diffusion gunhee kim 1 eric p....
Post on 19-Dec-2015
219 views
TRANSCRIPT
1
Distributed Cosegmentation via Submodular Optimization on
Anisotropic Diffusion
Gunhee Kim1 Eric P. Xing1 Li Fei-Fei2 Takeo Kanade1
1: School of Computer Science, Carnegie Mellon University2: Computer Science Department, Stanford University
November 9, 2011
2
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
3
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
4
Image Cosegmentation
Remove the ambiguity of what should be segmented out?
Jointly segment M images into K regions !
• Rother et al. 2006• Hochbaum and Singh, 2009• Joulin et al, 2010• Batra et al, 2010• Mukherjee et al, 2011• Vincente et al, 2010, 2011
(M = 3, K = 2)
5
Why is Cosegmentation Interesting?
Wide potential in Web applications
Photo-taking patterns of general users
My son joined baseball club.
I saw dolphins in aquarium.
6
Our Approach
Major challenges for web photos
Jointly segment M images into K regions
Work Model / Algorithm M K
Ours[J10]
[M11][H09][R06][B10][V10]
Anisotropic Diffusion/ SubmodularityDiscriminative ClusteringMRF+ Rank-1 global / Iterative opt.MRF+Reward global / Graph CutsMRF+L1 global / Trust Region GCBoykov-Jolly / Graph CutsBoykov-Jolly / Dual Decomposition
103
30 3022
502
Any222222
• [R06] Rother et al. 2006• [H09] Hochbaum and Singh, 2009• [B10] Batra et al, 2010
• [J10] Joulin et al, 2010• [V10] Vincente et al, 2010• [M11] Mukherjee et al, 2011
(1) Large-scale (2) Highly-variable
7
Contributions
A New optimization framework
Work Model / Algorithm M K
Ours[J10]
[M11][H09][R06][B10][V10]
Anisotropic Diffusion/ SubmodularityDiscriminative ClusteringMRF+ Rank-1 global / Iterative opt.MRF+Reward global / Graph CutsMRF+L1 global / Trust Region GCBoykov-Jolly / Graph CutsBoykov-Jolly / Dual Decomposition
103
30 3022
502
Any222222
• Constant-factor approximation of optimal• Easily parallelizable
• Automatic selection of K • Robust against wrong K
• [R06] Rother et al. 2006• [H09] Hochbaum and Singh, 2009• [B10] Batra et al, 2010
• [J10] Joulin et al, 2010• [V10] Vincente et al, 2010• [M11] Mukherjee et al, 2011
8
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
9
Diffusion
Diffusion in physics
Spread of particles (or energy) through random motion
from high concentration to low concentration
Examples• Electric current
• Heat diffusion[Wikipedia]
Heat Equation (Partial Differential Equation)
Temperature Diffusivity (conductance) tensor
10
Optimization
Maximize the sum of temperature of the system
max
s.t
K heat sources
Environment temperature
Maximize the sum of temperature
11
Correspondences
Temperature maximization and Image Segmentation
max
s.t
Heat Diffusion Points Temperature Heat sources Conductance
Image Segmentation
Pixels Segmentation confidence
Segment centers Similarity btw features of pixels
Image Segmentation
Select K pixels as segment centers, to maximize sum of segmentation confidence of every pixel.
12
Optimization
How can we solve this?
max
s.t
[Theorem] (Neuhauser, Wolsey, Fisher 1978)Let u be , nondecreasing, and submodular.
Then, the greedy algorithm finds a set such that
0.632.
13
Submodularity on Anisotropic Diffusion
[Theorem] Suppose that system is under linear anisotropic diffusion
(T1)
Let be temperature at time t point x when heat sources are attached to Then, the following holds for
(T2) is nondecreasing
(T3) is submodular
(Proof)
14
Submodularity on Anisotropic Diffusion
[Theorem] Suppose that system is under linear anisotropic diffusion
(T1)
Let be temperature at time t point x when heat sources are attached to Then, the following holds for
(T2) is nondecreasing
(T3) is submodular
(Proof)
x x
(Diminishing Return)
Induction on distance
15
Greedy Algorithm
Sketch of the greedy algorithm
max
s.t
Find the point with maximum marginal gain in every round.
1.
2. Iterate until 2.1.
2.2. Marginal gain
16
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
17
Diversity Ranking
Ranking items according to both centrality and diversity
Items
Rankingvalues
A BC
Centrality only: A > B > C
Centrality + Diversity: A > C > B
18
Optimization for Diversity Ranking
Simplification
max
s.t
(2) Steady-state
(4) Every v is connected to the ground with z
(1) System is a graph
(3) Diffusivity is defined by Gaussian similarity
19
Optimization for Diversity Ranking
Simplification
max
s.t
(2) Steady-state
(4) Every v is connected to the ground with z
(1) System is a graph
max
s.t
(3) Diffusivity is defined by Gaussian similarity
20
Examples of Diversity Ranking
Input data
(1) vertices
(2) features
(3) Conductance
21
Examples of Diversity Ranking
Input data 2nd item
3rd item Clustering
Marginal gain
1st item
22
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
23
Segmenting a Single Image
Input image 1. Superpixels (SP)
G = (V, E, W) G = (V, E, W)
2. Connect adjacent SPs
g(v) = Color
Texture
Construct image graph G = (V, E, W)
G = (V, E, W)
Optimization formulation is similar to that of diversity ranking
3. Features on SP 4. Conductance
24
Basic Behavior of Our Segmentation
Greedily select the largest and most coherent regions !
Input image K=2: sky K=3: tree K=4: wall
K=5: roof K=6: window K=7: building K=8: trash can
Source code is available !
Automatic selection of K
25
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
26
Cosegmentation
Segment selection should be coupled!
Single image segmentation
Objective 1: Segment should be large and coherent.
Objective 2: Segment should be similar to its corresponding ones in other images
+
Cosegmentation
27
Cosegmentation
Control source temperatures
A
B
A
B
Cosegmentation
A is better than Bto maximize the temperature
28
An Toy Example of Cosegmentation
Input images Likelihood Cosegmentation Segments
MSRC cow images (M=3, K=4)
Source code is available !
29
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
30
Two Experiments
Figure-ground cosegmentation with a pair of images
Scalable cosegmentation
• Goal: Compare with other state-of-the-art techniques
• Dataset: MSRC
ex. cat
• Goal: Feasibility for Web photos
• Dataset: ImageNet
ex. green lizard
31
Exp1. Figure-Ground Cosegmentation
Segmentation accuracies for 100 random pairs of MSRC
[6, 7] Use their implementation without modification
[6] ICCV 2009[7] CVPR 2010
32
Cosegmentation on MSRC
Cosegmentation Examples (K=8)
Ours (K = 8)(1) Multiple instances
(2) Robust against wrong choice of K
Normalized cuts (K = 8)
33
• Problem Statement
• Submodular Optimization on Diffusion
• Applications Diversity Ranking
Single Image Segmentation
Cosegmentation
• Experiments
• Conclusion
Outline
34
Conclusion
Prove the temperature in anisotropic diffusion is submodular.
(1) A large-scale edge-preserving image smoothing(2) Layered motion segmentation
Diversity ranking Cosegmentation Single-image segmentation
Source code is available !
Next step
What’s done
smoothing Optical flow
35
Conclusion
What’s done
Cosegmentation for Web photos was proposed• Arbitrary K and a larger M by order of magnitude • Easily Parallelizable
• Automatic selection of K • Robust against wrong K
(Ours) (Ncuts)
36
Thank you !Stop by our poster at 80!