Active Appearance Models

master thesis presentation

Mikkel B. Stegmann

IMM – June 20th 2000

Presentation outline

Aim

Method

Metacarpals – a case study

Discussion

Conclusion

Aim

To locate non-rigid objects in digital images

The vision utopia

• Fully automated

• General

• Specific

• Robust

• Accurate

• Holistic

• Non-parametric

• Fast

Active Appearance Models

A model-based approach towards segmentation

A priori knowledge is not programmed into the model, but learned through observation

Relies on statistical analysis of shape and texture variation in a training set

Derives a compact object class description which can be used to rapidly search images for new object instances

Model building

1) Data capture

Shape: point annotationTexture: pixel sampling

3) Statistical analysis

Principal component analysis on shape and texture

3) Combining shape and appearance

Shape and texture PCA is combined into a 3rd PCA

4) Model truncation

Parameters are truncated to satisfy a variance constraint

2) Normalisation

Shape: pose alignment using the Procrustes shape metric

Texture: photometric normalisation

Shape analysis

Shape is represented by a linear spline of landmarks:

X = ( x1, … , xn, y1, … , yn)T

• Assumes point correlation

• Requires point correspondence

Alignment w.r.t. position, scale, orientation

Principalcomponentanalysis

Compact shape representation

10 20 30 40 50 60 70 80 90 100

10

20

30

40

50

60

70

80

90

100

Texture analysis

Texture – the intensities across the object – is sampled inside the shape using a suitable warp function

Warp function: A piece-wise affine warp using the Delaunay triangulation

g = ( x1, … , xn)TPrincipalcomponentanalysis

Compact texturerepresentation

Combined Model

Shape and texture is combined into a compact model representation

This representation is capable of derforming in a similar manner to what is observed in the training set

Thus making the model specific to the class of objects it represents

Generative (self-contained)

Model Optimisation

Deforms the AAM to fit the image being searched

Assumes a linear relationship between model parameters and the observed fit:

C = RX

Solved using multivariate linear regression on alarge set of experiments

Actual dy (pixels)

Pre

dic

ted d

y (

pix

els

)

Implementation

Open source C++ API based on the Windows platform[and partly on VisionSDK, LAPACK, Intel MKL, ImageMagick a.o.]

Well documented [cross-referenced HTML and PDF]

Fast [using Intel BLAS for matrix handling and widely use of dynamic programming]

Suitable for education & research[lots of visual and numerical documentation: *.m *.avi *.bmp]

Example usage included[in the form of a console interface]

Metacarpals – a case study

20 x-ray images of the human hand supplied by Pronosco

Metacarpal 2, 3, 4 annotated using 50 points on each

Difficult segmentation problem due to large shape variability and the ambiguous nature of radiographs

Building the model

• Annotation of set of training images

• Capture of shape & texture

• Statistical analysis on shape & texture

Modes of variation

0 5 10 15 20 250

5

10

15

20

25

0 5 10 15 20 250

5

10

15

20

25

30

35

40

45

0 5 10 15 20 250

2

4

6

8

10

12

14

16

18

Shape Texture Combined

Metacarpal AAM

Image modality: radiographs (x-rays)

20 images/shapes in training set

300 points in shape model

~10.000 pixels in texture model

95% variation explained using 16 model parameters

Search

Metacarpal results

Using automatic initialisation

• Good mean location accuracy 0.98 pixel (point to border)

• Acceptable mean texture fit 6.57 gray levels (byte range)

Difficult to locate the exact bone extents at the proximal and distal end

mean pt. errors

proximal

distal

Discussion

“Hidden” benefits

• Automatic registration

• Variance analysis (group/longitudinal studies)

• Discrimination/interpretationusing the model parameters

Weaknesses

• Requires landmarks (point correspondence)

• Can only deform texture by moving edge points

• Not robust to large-scale texture noise

Discussion - cont’d

Image modalities on which AAMs has been evaluated successfully:

• Radiographs - x-rays of human hands

• Normal gray scale images - hands, pork carcasses

• MRI - human hearts

Initialisation has been added, thus making AAM a fully automated segmentation method

The AAM approach extends to 3D and multivariate imaging

Conclusion

AAM has been implemented and extended as a fully automated and data-driven approach towards image segmentation

AAM performs well on very different segmentation problems and different image modalities

Properties

• General

• Specific

• Captures domain knowledge without the need for technical knowledge

• Robust

• Non-parametric

• Self-contained

• Fast

fin

http://www.imm.dtu.dk/~aam