Recognition and tracking of human body parts

AlgirdasBeinaravičiusGediminas Mazrimas

Salman Mosslem

Project introduction Background subtraction techniques Image segmentation

◦ Color spaces◦ Clustering

Blobs Body part recognition Problems and conclusion

Contents

Goal: recognition of human body parts for a subject from video sequence images

Background subtraction/Foreground extraction

K-Means clustering for color images Blob-level introduction Body part recognition

Introduction. Project tasks

What is background subtraction? Background subtraction models:

◦ Gaussian model◦ “Codebook” model

Background subtraction

Learning the model Gaussian parameters estimation

Thresholds - Foreground/Background determination

Background subtractionGaussian model

Non-parametric model

Background subtraction“Codebook” model

Background subtractionModel comparison

Original image

Background subtractionusing Gaussian model

Background subtractionusing Codebook model

How important is image segmentation?

Color spaces◦ RGB◦ HSI◦ I3 (Ohta), YCC (LumaChroma), HSV…

Clustering◦ K-Means◦ Markov Random Field

Image segmentation

RGB (Red Green Blue)◦ Classical color space◦ 3 color channels (0-255)

In this project:◦ Used in background subtraction

Image segmentationColor space: RGB

HSI (Hue Saturation Intensity/Lightness)◦ Similar to HSV (Hue Saturation Value)◦ 3 color channels:

Hue – color itself Saturation – color pureness Intensity – color brightness

◦ Converted from normalized RGB values◦ Intensity significance minimized

In this project:◦ Used in clustering◦ Blob formation◦ Body part recognition

Image segmentationColor space: HSI

Image data (pixels) classification to distinct partitions (labeling problem)

Color space importance in clustering

Image segmentationClustering

Clustering without any prior knowledge Working only with foreground image Totally Kclusters Classification based on cluster centroid and

pixel value comparison◦ Euclidean distance:

◦ Mahalanobis distance:

Image segmentationClustering: K-Means

Image segmentationClustering: K-Means example

Image segmentationClustering: K-Means Euclidean/Mahalanobis distance comparison

Euclidean distance Mahalanobis distance

Image segmentationClustering: K-Means color space comparison

RGB HSI

Probabilistic graphical model using prior knowledge

Usage:◦ Pixel-level◦ Blob-level

Concepts from MRF:◦ Neighborhood system◦ Cliques

Image segmentationClustering: MRF

Image segmentationClustering: MRFNeighborhood system

Cliques

Higher level of abstraction◦ Ability to identify body parts◦ Faster processing

Blobs

Label. Set of area pixels. Centroid. Mean color value. Set of pixels, forming convex hull. Set of neighboring blobs. Skin flag.

BlobsParameters

Input: K-means image/matrix. Output: Set of blobs

BlobsInitial creation

Particularly important in human body part recognition.

Can not be fused. Technique to identify skin blobs:

◦ Euclidean distance

BlobsSkin blobs

Conditions:◦ Blobs have to be neighbors◦ Blobs have to share a large border ratio◦ Blobs have to be of similar color

◦ Small blobs are fused to their largest neighbor Neither of these conditions apply to skin

blobs

BlobsFusion

Associate blobs to body parts

Body part recognition (I)

Skin blobs play the key role:◦ Head and Upper body:

Torso identification Face and hands identification

◦ Lower body: Legs and feet identification

Body part recognition (II)

Body part recognition (III)

Computational time Background subtraction sensitivity Subject clothing Subject position Number of clusters in K-Means algorithm Skin blobs

Problems (I)

Problems (II)

Problems (III)

Main tasks completed Improvements are required for better

results

Possible future work:◦ Multiple people tracking◦ Detailed body part recognition◦ Algorithm improvements with better computer

hardware usage for live video images

Conclusion and future work

?

Questions, comments