Robust TrackingRobust Tracking

by Ilya Levnerby Ilya Levnerilya@cs.ualberta.cailya@cs.ualberta.ca

OutlineOutline

• Introduction

• Survey of existing techniques– Anti-failure– Post-failure

• IFA

– Pros & Cons

• Research Direction

IntroductionIntroduction

x*(t) – the true state of target objectXin(t) – the input state setXout(t) – the output state set Accuracy – x*(t) є Xout(t) Precision - |Xout(t)|-1

Robustness - The ability to accurately and precisely track objects under less than ideal conditions.

RobustnessRobustness CategoriesCategories

• Anti-failure – prevention of errors (I.e. precision loss) and failure.

• Post-failure – recovery from failure

Anti-failureAnti-failure

• Seeded by attempts in 1953 to create robust statistical estimators.

• Has received the most attention from the vision community w.r.t post-failure.

Anti-failure ApproachesAnti-failure ApproachesWindow Processing and Foveation

– Ignore/blur the image around the target to avoid/remove background distractions.

Robust Matching Techniques– Handle occlusion by

• detection of outliers

Color cue concentration– Enables a tracker to handle changes in lighting.

CONDENSATION– Generalized Kalman filters– Handle agile motion

Post-failure RobustnessPost-failure Robustness

“The synthesis of reliable organisms from unreliable parts” von Neumann, 1952

• Implies non-catastrophic error/ failure recovery

Post-failure (cont)Post-failure (cont)

• Most active research in field of planning– Replanning paths

• Behaviour-Based Robotics– “emergent” intelligence known to display post-

failure robustness. (subsumption architecture)

• Limited work in the visual tracking domain

Incremental Focus of Attention Incremental Focus of Attention System (IFA)System (IFA)

• Biologically inspired methodology– Pre-attentive mechanisms select a target subregion– Post-attentive examine the subregion for relevance

• IFA uses a hierarchy of:– selectors that search for a subregion containing the

target – trackers which keep the focus on the target.

color thresholdingcolor thresholding

blob trackingblob tracking

template-based trackingtemplate-based tracking

targettarget statestate

full configuration spacefull configuration space

algorithmic layersalgorithmic layers

feature-based trackingfeature-based tracking

IFA (Face Tracking)IFA (Face Tracking)

Technique ComparisonTechnique Comparison

Robustness to Occlusion

• Anti-failure Method– robust statistics to filter out the non-signal data.

• SSD tracker with oulier detection

• Post-failure Method– IFA

• Same SSD tracker without outlier mechanism at the top layer.

ResultsResultsIdeal Conditions (no occlusion)

– Equal precision– AF is 15-20% slower due to overhead processing

Small Occlusions– AF tracks at full precision– PF drops to color blob tracking, resulting in a significant

loss of precision. (Recovery within 100msec)

Large-Full Occlusions (>40% of target)– AF looses target region and never recovers– PF takes between 150ms to several seconds for recovery to

full precision.

Anti-failureAnti-failureProsPros

• Handles specific perturbations well.

• Can avert catastrophic failure

ConsCons• Different modes of failure

require individual contingencies

• In ideal conditions slows down the system

• Difficult if not impossible to achieve in real-world systems (too many things can go wrong)

Post-failurePost-failure

ProsPros• A single procedure usually

enough to recover from any failure type.

• Dormant in ideal conditions

ConsCons• Meaningless in a catastrophic

failure case(s).

• Fixed (hand-coded) hierarchy

• Slow recovery at times

ExtensionsExtensions

Add Learning Module(s) to IFA

• Motion prediction (AF / PF – recovery optimization)

• Dynamic Tracker Selection ?

Motion PredictionMotion Prediction

• Past attempts used Kalman Filter

• Why not try HMM/CHMM to predict motion.– Let δμ* be the true change in motion

parameters (state)– ~δμ computed motion parameters

(observations)

Plan of Action

1) Construct a rudimentary IFA system, 3-4 layers consisting of:

• x,y translation trackers at 1,1/2,1/4 resolution• Color blob tracker

2) Construct an HMM for each level within the hierarchy

3) Couple the HMM’s together (CHMM)

Hidden Markov Model(s)

Coupled HMM, A matrix.

Issues

• Relearning– The A matrix will become uniform, i.e. all state

transitions have equal probabilities.– How can recent states transitions have more

weight in the Baum-Welch training phase ??

• Search depth

References