Edge Detection Evaluation in Boundary Detection Framework

Feng Ge

Computer Science and Engineering, USC

• Edge detection Detect pixels with strong gradient of “gray-level”

• Error– False negative(Missing ): Not detected Edges

– False positive: detected false edges

– Orientation error: shift from real position

– Dislocation error: shift from real direction

• How to evaluate these errors?

Edge Detection Error

Evaluation Criteria

• Ground Truth– Human or predefined results?

• Quantificaition– Measuring and expressing in number means good.

• Generality– Real images in large number

Combined 3 criteria are good evaluation methods!

Overview

• Subjective vs Objective – Human vision checking – Quantitative measurement

• With ground truth vs Without– Standard for evaluation – Some characters,e.g, continuation,coherence.

• Synthetic vs Real images– Simple structure– Complicated structures

Motive—in boundary detection framework

• Problem: Boundary detection algorithms work well in synthetic data, while poorly in real images

• This gap,we believe, is largely introduced by edge detection

Experiment Settings: Image Database

• Large: 1030 images• Generality• Unambiguous• Manually extracted

ground truth

Experiment Settings: Evaluation Flowchart

Experiment Settings: Detectors

• Edge & Line Detector: Canny & Line Approximation

• Boundary detector: Ratio-Contour

Experiment Settings: performance measurement

Experiment

• Original imagesimage->edge->fragments->bounday->evaluation

• Synthetic imagestexture images->fragments

--->bounday->evaluation ground truth->adding noise

• Semi-synthetic images original images->background

-->bounday->evaluation ground truth->adding noise

Experiment --Synthetic images

• Result– Much better than

original images

• Problem– Background

correlation changed– Irregular

background in texture images

Experiment –Semi-synthetic images

mean Std median min maxDislocation(pixel) 1.06 0.83 0.85 0 4.48Orientation(degree) 20.4 24.9 11 0 135Length (pixel) 3.9 3.6 2.82 1 138Total missing rate: 0.35

•Model simulation

•Edge-map error analysis

Result-1• Simulate edge missing

Procedure:Sample ground truth, random delete some percentage of fragments

Result-2• Simulate edge detection error: missing & dislocation

– Fix dislocation error, vary missing rate (a)– Fix missing error, vary dislocation error (b)

(a) (b)

Conclusion• Our noise model is close to real edge error,

as regarding to the simulated result• Edge missing and dislocation are mainly

encountered errors in edge detection.• Edge dislocation is more crucial in edge

error compared with missing error

Discussion-1• Error introduced by line detection

Error based on edge-map is different from based on line approximation:

Missing rate Dislocation error mean variance mean variance Based on edge 0.4 0.12 0.12 0.07 Based on line 0.35 0 1.06 0.83

Discussion-2• Model error

– Gaussian distribution assumption

• Based on boundary detection– Globally, not locally– Introduce some error, but statistically, reasonable

• Image database– Low resolution– Ground truth error

Future work

• Distinguish errors introduced by line approximation from edge detection

• Noise model refinement• Substitute line with curve in edge-map

approximation• Data base improvement

Thank You !