![Page 1: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/1.jpg)
1
Learning to Detect Natural Image Boundaries Using Local Brightness,
Color and Texture CuesDavid Martin, Charless Fowlkes, Jitendra Malik
Computer Science DivisionUniversity of California at Berkeley
Presented by Jean-Francois LalondeDisclaimer: Most of the slides are from D. Martin
![Page 2: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/2.jpg)
2
Canny
What is a Boundary?
Original Human
![Page 3: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/3.jpg)
3
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
![Page 4: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/4.jpg)
How do humans do this?
• Low-level cues? – Brightness? Color? Texture?
• Mid-level cues? – Continuity? Closure? Symmetry?
• High-level cues? – Context? Object recognition?
• This paper: what is the optimal way to use LOCAL information?
4
![Page 5: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/5.jpg)
5
Non-Boundaries Boundaries
• Psychophysics of localization:– Multi-Attribute Boundaries [Rivest/Cavanagh 1996]
• luminance, color, motion, texture• Information pooled prior to localization
– Texture Boundaries [Landy/Kojima 2001]• frequency, orientation, contrast
• Their approach: Supervised learning to optimally combine boundary cues.
![Page 6: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/6.jpg)
6
DataflowImage
Optimized CuesPb
Brightness
Color
Texture
Benchmark
Human Segmentations
Cue Combination
Model
![Page 7: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/7.jpg)
7
Brightness and Color Features
• 1976 CIE L*a*b* color space• Brightness Gradient BG(x,y,r,θ)
– χ2 difference in L* distribution
• Color Gradient CG(x,y,r,θ)– χ2 difference in a* and b*
distributions
θ
r(x,y)
![Page 8: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/8.jpg)
8
Texture
• Texture Gradient TG(x,y,r,θ)– χ2 difference of texton histograms
– Textons are vector-quantized filter outputs
TextonMap
![Page 9: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/9.jpg)
9
Non-Boundaries Boundaries
I
T
B
C
![Page 10: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/10.jpg)
10
Cue Combination Models• Classification Trees
– Top-down splits to maximize entropy, error bounded
• Density Estimation– Adaptive bins using k-means
• Logistic Regression, 3 variants– Linear and quadratic terms– Confidence-rated generalization of AdaBoost (Schapire&Singer)
• Hierarchical Mixtures of Experts (Jordan&Jacobs)– Up to 8 experts, initialized top-down, fit with EM
• Support Vector Machines (libsvm, Chang&Lin)– Gaussian kernel, ν-parameterization
Range over bias, complexity, parametric/non-parametric
![Page 11: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/11.jpg)
11
BenchmarkGoal
FewerFalsePositives
Fewer Misses
![Page 12: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/12.jpg)
12
ClassifierComparison
![Page 13: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/13.jpg)
13
Various Cue Combinations
![Page 14: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/14.jpg)
14
Detector ComparisonCanny 2MM Us HumanImage
![Page 15: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/15.jpg)
15
Two Decades of Local
Boundary Detection
![Page 16: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/16.jpg)
Human vs machine on local patches
16
![Page 17: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/17.jpg)
17
Conclusion1. A simple linear model is sufficient for cue combination
– All cues weighted approximately equally in logistic– Linear model supported by psychophysics
2. Texture gradients are a powerful and necessary cue3. Significant improvement over state-of-the-art in local
boundary detection– Pb(x,y,θ) good for higher-level processing
4. Human performance on patches??– ECVP’03
![Page 18: Learning to Detect Natural Image Boundaries Using Local ...efros/courses/LBMV07/presentations/0201Proba… · Boundaries Using Local Brightness, Color and Texture Cues David Martin,](https://reader033.vdocuments.mx/reader033/viewer/2022060919/60ab5a28771fef5eb5070664/html5/thumbnails/18.jpg)
18
The End