closing remarks: what can we do with multiple diverse...

CVPR 2013 Diversity Tutorial

Closing Remarks: What can we do with multiple

diverse solutions?

Dhruv Batra Virginia Tech


(C) Dhruv Batra 2

CRF

. . .

Diverse Segmentations

Example Result

Now what?


Your Options •  Nothing

–  User in the loop

•  (Approximate) Min Bayes Risk –  Use solutions to estimate the distribution and optimize Bayes

Risk

•  Re-ranking –  Pick a good solution from the list

(C) Dhruv Batra 3

Increasing Side Information


Interactive Segmentation •  Setup

–  Model: Color/Texture + Potts Grid CRF –  Inference: Graph-cuts –  Dataset: 50 train/val/test images

(C) Dhruv Batra 4

Image + Scribbles Diverse 2nd Best 2nd Best MAP MAP

1-2 Nodes Flipped 100-500 Nodes Flipped


Interactive Segmentation

(C) Dhruv Batra 5

89%

90%

91%

92%

93%

94%

95%

96%

MAP M-Best-MAP Confidence DivMBest

+0.05%

+1.61%

+3.62%

(Oracle) (Oracle) (Oracle)

M=6

Seg

men

tatio

n A

ccur

acy

Better





Risk


(C) Dhruv Batra 6


Statistics 101 •  Loss

–  PCP, Pascal Loss, etc

•  “True” Distribution

•  Expected Loss:

•  Min Bayes Risk

(C) Dhruv Batra 7

XMAP

X

P (X )P (ygt | x) =

L(ygt, y)

BR(y) = EP

�L(ygt, y)

�

�

ygt∈Y

L(ygt, y)P (ygt)miny∈Y


Structured Output Problems •  Min Bayes Risk

•  Two Problems

•  Approximate MBR:

(C) Dhruv Batra 8

Intractable Intractable

XMAP

X

P (X )

�

ygt∈Y

L(ygt, y)P (ygt)miny∈Y

miny∈DivMBest

�

ygt∈DivMBest

L(ygt, y)P (ygt)


Semantic Segmentation •  Setup

–  Models: •  Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]

•  Second-Order Pooling [Carreira ECCV ‘12]

–  Inference: •  Alpha-expansion •  Greedy

–  Dataset: Pascal Segmentation Challenge (VOC 2012) •  20 categories + background; ~1500 train/val/test images

(C) Dhruv Batra 9


(C) Dhruv Batra 10

CRFDiverse Segmentations

Large-Margin Re-ranking


Semantic Segmentation

(C) Dhruv Batra 11

Input MAP Best of 10-Div



(C) Dhruv Batra 12

PAC

AL

Acc

urac

y

Better

#Solutions / Image

44%

47%

50%

53%

56%

59%

1 2 3 4 5 6 7 8 9 10

MAP [State-of-art circa 2012]

15%-gain possible

Same Features Same Model

DivMBest (Oracle)

Rand (Re-rank)

MBR





Risk


(C) Dhruv Batra 13


(C) Dhruv Batra 14




(C) Dhruv Batra 15



ψ( , )


(C) Dhruv Batra 16



ψ( , )α� −α�ψ( , )


(C) Dhruv Batra 17



ψ( , )α� −α�ψ( , ) ≥ 1− ξilossi

minα,ξi

||α||2 + C�

i

ξiDiscriminative Re-ranking of Diverse Segmentation

[Yadollahpour et al., CVPR13, Wednesday Poster]



(C) Dhruv Batra 18

PAC

AL

Acc

urac

y

Better

#Solutions / Image

44%

47%

50%

53%

56%

59%

1 2 3 4 5 6 7 8 9 10

MAP [State-of-art circa 2012]

DivMBest (Oracle)

Rand (Re-rank)

DivMBest (Re-ranked) [Y.B.S., CVPR ‘13]

MBR


Qualitative Results: Success

(C) Dhruv Batra 19



(C) Dhruv Batra 20



(C) Dhruv Batra 21


Qualitative Results: Failures

(C) Dhruv Batra 22



(C) Dhruv Batra 23



(C) Dhruv Batra 24


Summary •  All models are wrong

•  Some beliefs are useful

•  Diverse Multiple Solutions –  A way to get useful beliefs out.

•  DivMBest + Reranking –  Big impact possible on many applications!

(C) Dhruv Batra 25


Summary

•  What does my model believe?

(C) Dhruv Batra 26

�

Posterior Summary


Thanks!

(C) Dhruv Batra 27

closing remarks: what can we do with multiple diverse...

Documents