detection on faces
DESCRIPTION
Detection on Faces. Levente Saj ó. Detecting Face. Knowledge-Based Methods A face often appears in an image with two eyes that are symmetric to each other, a nose and a mouth Problem: it is difficult to translate human knowledge into well defined rules Feature-Invariant Approaches - PowerPoint PPT PresentationTRANSCRIPT
Detection on FacesDetection on Faces
Levente Sajó
2008.02.28
2
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Detecting FaceDetecting Face
• Knowledge-Based Methods– A face often appears in an image with two
eyes that are symmetric to each other, a nose and a mouth
– Problem: it is difficult to translate human knowledge into well defined rules
• Feature-Invariant Approaches– Facial Features
• Using edge detectors, facial features such as eyes, nose, mouth, and hair line are extracted
• A statistical model is built to describe their relationships
• Problem: lighting, shadows– Skin Color
2008.02.28
3
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Detecting FaceDetecting Face
• Template Matching Methods– Predefined templates
• correlation values with the standard patterns are computed eg.: for the face contour, eyes, nose and mouth independently
• it can not effectively deal with variation in scale, pose and shape
– Deformable Templates – Shape Models
• How to represent the template?
• How to model deformations?• Efficient matching
algorithms
2008.02.28
4
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Detecting FaceDetecting Face
Appearance-Based Methods– Eigenfaces – PCA– Neural Networks– SVM– AdaBOOST (by Viola and Jones, 2001)
• Quite accurate• Fastest – 10-15-times faster then the
others• An extended version is implemented in
opencv (by Lienhart, 2002)• This method is used in Google Portrait,
too
2008.02.28
5
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Boosted Cascade DetectorBoosted Cascade Detector
• Consists 3 parts– Encoding the image data known as
“integral image”– Running the boosting algorithm
known as AdaBoost, to select the appropriate feature
– a cascade of templates that allows simple feature sets to quickly discard most of the uninteresting parts of the image
2008.02.28
6
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Integral ImageIntegral Image• An integral image is constructed by replacing
each image pixel value i(x,y) with a value that corresponds to the pixel sum, above and to the left of the pixel
2008.02.28
7
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Feature ClassifiersFeature Classifiers• Each classifier consists of a set of adjacent positive and
negative regions• Different feature types:
• If template is 24x24, then it produces 30000 different feature classifiers
• Each individual feature classifier attempts to discriminate between faces/non-faces by summing the difference between adjacent regions
2008.02.28
8
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Boosting AlgorithmBoosting Algorithm• Boosting algorithms aim to combine
many “weak classifiers” into one effective “strong classifier”.
• AdaBoost achieves this by selecting the more promising classifiers.
2008.02.28
9
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Cascaded DetectorsCascaded Detectors• A series of classifiers are applied to every sub-window. • The initial classifier eliminates a large number of
negative examples with very little processing. • Subsequent layers eliminate additional negatives
2008.02.28
10
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
BCD ExperimentsBCD Experiments
• Detecting whole face• Detecting facial features
• The whole face template is more robust than any of the facial feature templates
2008.02.28
11
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Facial Expression RecognitionFacial Expression Recognition
• Recognizing the basic emotional expressions from video– neutral, anger, joy, sadness, surprise,
disgust, fear• Steps:
– Localizing the face– Preprocessing !!– Classification (with SVM, AdaBoost)
2008.02.28
12
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Preprocessing 1Preprocessing 1
• Gábor Transformation (Bartlett, 2003)– 48x48 face images were used– Normalized so, that the approximate distance
between center of eyes to be 24 pixel– The images were converted into a Gabor
magnitude representation, using a bank of Gabor filters at 8 orientations and 5 spatial frequencies
2008.02.28
13
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Preprocessing 2Preprocessing 2• Feature Extraction (Michel, El Kaliouby, 2003)
– In earlier work, using an SVM classifier on displacements of manually defined facial landmarks in image sequences yielded high classification accuracy
– Uses a face template to initially locate the position of the 22 facial features
– Their position is tracked over subsequent frames– A vector of feature displacements is calculated by
taking the Euclidean distance between feature locations in a neutral and a “peak” frame
2008.02.28
14
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Shape ModelsShape Models
• Predefined Shape Models– Setting the position of the landmarks
manually• Active Shape Models (Cootes, 1995)
– Using a set of manually labeled face image as a training set
– PCA is used to reduce the dimensionality of the training set
– During the training stage, for each landmarks their mean position and variance are determined
2008.02.28
15
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Shape ModelsShape Models
2008.02.28
16
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Localizing Facial FeaturesLocalizing Facial Features• Using face detection
techniques to detect facial features – local feature detectors (Cristinacce, Cootes, 2004)– Normalised correlation– BCD
• Shape models can be used to reduce the number of false detections by only selecting plausible configurations of feature matches
• With shape models the approximate location and orientation of the face can be used to perform a local search of facial features.
2008.02.28
17
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Shape SearchShape Search
• Unconstrained search– A bounding box of every feature are
calculated– The feature detector is then applied
to each region– The best match of is detector is taken
• Constrained search – Combinatoric Shape Search (CSS)– Few of best responses are taken from
each local detector– All combinations of feature locations
are checked
2008.02.28
18
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Next Steps…Next Steps…• Previous method works fine on neutral
faces• The same method can be modified to
work on faces with emotions• Other approach for localizing facial
features?– Color based– Intensity based (edges, thresholds)
2008.02.28
19
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Next Steps…Next Steps…
2008.02.28
20
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Next Steps…Next Steps…
2008.02.28
21
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Next Steps…Next Steps…
2008.02.28
22
Detecting Faces
Boosted Cascade Detector
Facial Expression Recognition
Shape Models
Localizing Facial Features
Next Steps...
Next Steps…Next Steps…