auxiliary cuts for general classes of higher-order functionals 1 ismail ben ayed, lena gorelick and...
TRANSCRIPT
1
Auxiliary Cuts for General Classes of Higher-Order Functionals
Ismail Ben Ayed, Lena Gorelick and Yuri Boykov
2
Sg,E(S) B(S)Sg,E(S)
Standard Segmentation Functionals
I
Fg)|Pr(I Bg)|Pr(I
Sp
g(p)E(S)
bg)|Pr(I(p)
fg)|Pr(I(p)lng(p)
S
Historic Data
Linear terms are not enough
3
Standard model
Learned distributions
Linear terms are not enough
3
Segmentation with log likelihoods
Learned distributions
B(S)S, g
Linear terms are not enough
3
Standard model
Target distributions
Segmentation with log likelihoods
B(S)S, g
Segmentation with log likelihoods
Linear terms are not enough
3
Standard model
Target distributions
B(S)S, g
Segmentation with log likelihoods
Linear terms are not enough
3
Standard model
Target distributions
B(S)S, g
Segmentation with log likelihoods
Linear terms are not enough
3
Standard model
Learned distributions
Obtained distributions
B(S)S, g
From log-likelihoods to higher-order terms
4
S, g
T 2L||-|| S
Rother et al. 06, Ben Ayed CVPR 10, Gorelick et al. ECCV 12, Jiang et al. CVPR 12
Standard vs. High-order
5
Input High-orderLikelihoods
Likelihoods High-order
Input
Regional Functional Examples
Volume Constraint
1g(p) ibin for (p)gi
S,gi
S1,|S|
2
0VS1,
6
Bin Count Constraint
k
1i
2
iiVS,g
Regional Functional Examples
1g(p)
S1,|S|
ibin for (p)gi
S,gi
Volume Constraint
2
0VS1,
6
7
Contribution: Bound Optimization of General Higher-Order Terms
Non-Linear Combination of Linear Terms
i
iiS,gFR(S)
Optimization
B(S)R(S)minS
Higher-order Pairwise
Sub-modular
8
9
Prior Art: General-Purpose Techniques Based on Functional Derivatives
S
F
t
S
Prior Art: General-Purpose Techniques Based on Functional Derivatives
9
-- Level Sets: Ben Ayed et al. CVPR 2008
-- Line search: Gorelick et al. ECCV 2012
S
F
t
S
Can be slow
9
-- Level Sets: Ben Ayed et al. CVPR 2008
-- Line search: Gorelick et al. ECCV 2012
S
F
t
S
F differentiable
Prior Art: General-Purpose Techniques Based on Functional Derivatives
9
-- Level Sets: Ben Ayed et al. CVPR 2008
-- Line search: Gorelick et al. ECCV 2012
S
F
t
S
Parameters?
Prior Art: General-Purpose Techniques Based on Functional Derivatives
Prior Art: Specialized Techniques
9
Volume constraint: Werner, CVPR 2008
Norms between bin counts: Mukherjee et al. CVPR 2009, Jiang et al. CVPR 2012
Bhattacharyya: Ben Ayed et al. CVPR 2010, Punithakumar et al. SIAM 2012
Only particularcases
Auxiliary Function Optimization
tS S
F
tA
10
)(SA)F(S ttt
F(S)(S)AminS t
S
1t
Auxiliary Function Optimization
S
F
10
F(S)(S)AminS t
S
1t
tA
)(SA)F(S ttt
tS 1tS
Auxiliary Function Optimization
S
F
10
F(S)(S)AminS t
S
1t
tA
)(SA)F(S ttt
)F(S 1t
tS 1tS
Standard Tricks for
Deriving Auxiliary Functions
12
• Cauchy-Schwarz inequality
• Quadratic bound principle
• First-order expansion
• Jensen’s inequality
E.g.: EM is based on this approach
Jensen’s Inequality bound
11
Ωppp
Ωppp
)F(yαyαF
1;αΩp
p
0α
p
11
Ωppp
Ωppp
)F(yαyαF
1;αΩp
p
0α
p
Unary Terms
Jensen’s Inequality bound
11
F
21
α)y(1αyF
1y
2y
)α)F(y(1)αF(y21
Jensen’s Inequality bound
Auxiliary Function Derivation
13
Sp
g(p)Sg,tS
S
Auxiliary Function Derivation
13
Sp
g(p)Sg,
(p)χg(p) SSp t
1χS
0χS
tS
Auxiliary Function Derivation
13
1χS
0χS
tS
(p)χg(p)FSg,F SSp t
Auxiliary Function Derivation
13
(p)χg(p)FSg,F SSp t
(p)χSg,Sg,
g(p)F S
t
Spt
t
Constant
1χS
0χS
tS
Auxiliary Function Derivation
13
Sum to 1
pα p
y
1χS
0χS
tS
(p)χg(p)FSg,F SSp t
(p)χSg,Sg,
g(p)F S
t
Spt
t
Auxiliary Function Derivation
13Jensen’s Linear auxiliary function
1χS
0χS
tS
pα p
y
(p)χg(p)FSg,F SSp t
(p)χSg,Sg,
g(p)F S
t
Spt
t
Difference with other methods: the volume constraint case
tS S1,
14
2
0V
0V
2
0VS1,
Difference with other methods: the volume constraint case
S1,
14
1tS
2
0V
0V
Gradient Descent
2
0VS1,
tS
Difference with other methods: the volume constraint case
S1,
14
2
0V
0V
Trust Region: Gorelick et al. CVPR 13
2
0VS1,
tS1tS
Difference with other methods: the volume constraint case
S1,
14
2
0V
0V
Auxiliary Cuts
2
0VS1,
tS1tS
General Form of the Functionals
15
B(S)R(S)minS
Higher-order Sub-modular
General Form of the Functionals
15
B(S)(S)AminS t
SS
1t
t
Linear bound Sub-modular
tS
1tS
B(S)R(S)minS
Higher-order Sub-modular
General Form of the Functionals
15Graph Cut
B(S)R(S)minS
Higher-order Sub-modular
B(S)(S)AminS t
SS
1t
t
Linear bound Sub-modular
tS
1tS
Experimental examples
L2 Bin Count (Aux. Cuts vs. Level Sets)
Level-Set, dt=1 Level-Set, dt=50 Level-Set, dt=1000Init Aux. Cuts
16
User input ResultUser
input
Iter 2
User inputResult B-J
Initial
segment
Iter. 3Iter. 217
L1 Bin Count
18
inputs
Input
L2 Volume Constraint
User input
B-J
B-J and Volume
Conclusions
19
Advantages:
• Derivative-free
• No optimization parameters, e.g., step size
• Easy to implement
• Never worsen the energy at each iteration
Conclusions
19
Limitations:
• The form of F should verify some conditions
• Limited to nested evolutions of segments
tS2tS 1tS
Conclusions
19
Extensions:
• More general forms of F
• Arbitrary evolutions of segmentstS
1tS
19
inputs
Input
Thanks