1

Segmentation-summary and evaluation techniques

Section 6.5

and

Chalana and Kim (1997)“A Methodology for Evaluation of Boundary Detection Algorithms on Medical Images”

2

Segmentation - summary

ThresholdingEdge-basedRegion-basedModel-based

HoughTemplate matching (supervised)

3

Thresholding

Simplest segmentation processFocus is on automatic thresholding

Histogram-basedStatistical properties ( 1st order K means, 2nd-order Otsu)

Pros:FastWorks well for detecting non-touching objects of constant reflectivity or light absorption of their surfaces

Cons:segmentation results may contain holes inside regions corresponding to objects or several objects merged into one region

4

Edge-based

Work with results of edge detectors (discontinuities in gray-level, colour, texture etc)Main goal: grouping local edges into an image where only edge chains with a correspondence to existing objects or image parts are presentCommon problems:

Sensitivity to image noiseLow contrasted borders

5

Region-based

Image is partitioned into a minimal number of regions that satisfy an homogeneity criterionCommon problems

Choice of homogeneity criterion: if too low, undersegmentation; if too high, oversegmentationPixel aggregation: choice of seed pixel; choice of similarity measure; choice of similarity thresholdResults of region merging are typically undersegmentedResults of region splitting are typically oversegmented and have a blocky aspect Split-and-merge approaches

6

Model-based segmentation

Hough transformBased on a geometric model of the primitive; model is parametricDetection of the primitive is equivalent to detection of maxima in the accumulator space. Maxima detection in accumulator space is typically performed by thresholdingFast for lines, reasonably fast for circles, computational complexity increases exponentially with number of parameters in the geometric description of the primitiveLimited applicability

7

Model-based segmentation

Template matchingSupervised segmentation (we know what we are looking for)

For fixed size templates: Image is scanned and a similarity measure (correlation) is performed between the neighborhood centered at each image location and the templateIf target objects have variable size, image pyramids (gaussian smoothing of various levels of detail)Problems: does not work for deformable objects, is not view-invariant

8

More advanced segmentation techniques

Graph-cut segmentationDeformable models

SnakesLevel Sets

9

Region segmentation: minimum-cost graph cutA graph cut is a partitioning of a graph where the cost of the cut is thecumulative cost of cutting the respective arcs (links).

10

Deformable models

Powerful top-down techniqueUses a model (prior knowledge) and deforms it to fit the image data (current information)Main idea:

1. Establish the problem as the minimization of some cost function.2. Use established optimization techniques to find the optimal

11

Active contours (snakes)

Based on the minimization of an energy functionThe original formulation of this “energy” was

Esnake = wintEint + wimageEimage + wconEconwhere each term is as follows:

Eint Internal Energy (Keeps the snake from bending too much)Eimage Image Energy (Guides the snake along important image features)Econ Constraint Energy (Pushes or pulls the snake away from or towards user-defined positions)

The total energy for the snake is the integral of the energy at each point:

12

Level sets

Implicit formulation of the curvesSegmentation not affected by topological changesAn unknown number of objects can be detected simultaneously

13

Evaluating segmentation techniques

As in other areas of vision, evaluation is a problemWe need to know what the correct result isWe need some way to compare the result of each algorithm to the ideal situation

From Tony Pridmore’s Lecture Notes on Image Processing and Interpretation, University of Nottingham

14

Evaluating segmentation

Possible approachesGround truth – get a ‘correct’segmentation and compare the results of the algorithm to itEvaluations based on region properties – we want the regions to be uniform, and for adjacent regions to be differentEvaluating robustness

If we deliberately introduce noise or partially mask the object of interest, how will the segmentation result be affected?

Adapted from Tony Pridmore’s Lecture Notes on Image Processing and Interpretation, University of Nottingham

15

Ground truth segmentation

Typically used in medical imaging applicationsIssue: human segmentations can vary significantlyHow do we build a ground truth segmentation from several human segmentations?

16

Statistical ground truth

17

Additional reading (ELEC 536)

Chalana And Kim: Evaluation Of Boundary Detection Algorithms On Medical Images, IEEE Transactions on Medical Imaging, 1997

18

Ground truth in other applications

Experiment: segmenting an image by hand

Adapted from Tony Pridmore’s Lecture Notes on Image Processing and Interpretation, University of Nottingham

19

Ground truth in other applications

Experiment: segmenting an image by hand

Adapted from Tony Pridmore’s Lecture Notes on Image Processing and Interpretation, University of Nottingham

20

Ground truth in other applications

Human segmentation of complex scenes is subjective; it depends on visual representation among many other things Are human segmentations consistent?

Adapted from Tony Pridmore’s Lecture Notes on Image Processing and Interpretation, University of Nottingham

21

Comparing image segmentations

Suppose we have a agreed ground truthWe need to compare two sets of regionsWhat does it mean for two sets of regions to be similar?Is the number of regions important?Does it matter if two regions are merged or if one is split in two?

Ground truth partition Which result is better?

Adapted from Tony Pridmore’s Lecture Notes on Image Processing and Interpretation, University of Nottingham

22

Current measures of similarity

Applicable when only one region of interest in imageRegion-based: Mutual overlap (6.5.1.1)Limits

Does not give any information about boundariesConceals quality differences between segmentationsAssumes a closed contourLarge errors for small objects

23

Current measures of similarity

Border-based (6.5.1.2)Hausdorff distance

Idea: consider pixels in the two contours as sets of finite points

24

Hausdorff distance (cont’d)

http://en.wikipedia.org/wiki/File:Hausdorff_distance_sample.svg

25

Unsupervised evaluation

We can compute statistics about theregions

The averages of adjacent regions should be differentThe standard deviation within a region should be low

Segmentation-summary and evaluation techniquesSegmentation - summaryThresholdingEdge-basedRegion-basedModel-based segmentationModel-based segmentation More advanced segmentation techniquesSlide Number 9Deformable modelsActive contours (snakes)Level setsEvaluating segmentation techniquesEvaluating segmentationGround truth segmentationStatistical ground truthAdditional reading (ELEC 536)Ground truth in other applicationsGround truth in other applicationsGround truth in other applicationsComparing image segmentationsCurrent measures of similarityCurrent measures of similarityHausdorff distance (cont’d)Unsupervised evaluation