PowerPoint Presentation

Tracking with CACTuS on JetsonRunning a Bayesian multi object tracker on a low power, embedded systemSchool of Information Technology & Mathematical Sciences

September 2015

David Webb

Supervisor: David Kearney1Object Tracking OverviewWhat is object tracking?

Object tracking, also referred to as video tracking, is a sub domain of computer vision, concerned with detecting, correlating and tracing the path of an object(or objects) as it travels, relative to the background, over multiple frames of video (or a sequence of images).Object Tracking OverviewHow is object tracking used?

Real world applications of object tracking include

Security and SurveillanceLoitering TrafficSuspicious packages Theft

Business intelligencePeople counting

Object Tracking OverviewHow is object tracking used?

MilitarySituational awareness

RoboticsObject manipulation Environment awareness

Human Computer InteractionGesture Recognition Gaze Tracking

Object Tracking OverviewWhat is involved in object tracking?

DetectionDetermining the existence of an object at a given location (or pixel) in an image

TrackingAssociating the location of an object over multiple images

RecognitionClassification of the nature of the object being tracked

Object Tracking OverviewClassic detection tracking recognition

Classic algorithms suffer from ambiguity in measurements due to noise

In the presence of noise, classic trackers make hard and fast assumptions about the nature of the target object

This can lead to poor robustness, where the tracker is unable to correctly maintain the track of an object

Object Tracking OverviewCompetitive Attentional Correlation Tracker using Shape (CACTuS)

CACTuS is designed to avoid problems associated with measurements taken in the presence of noise by using a Bayesian approach to propagate uncertainty through the tracking chain.

CACTuS TrackerCACTuS Multiple Object Tracker

Shape Estimating Filter (SEF)Each SEF is capable of tracking and describing one objectDescribes an objects location, velocity and shapeUses a Bayesian approach to propagate uncertainty by representing each state as a 2D array of probabilities

The following operations are performed on each SEFPrediction ObservationUpdate

CACTuS TrackerCACTuS Multiple Object Tracker

CACTuS Combines Multiple SEFsSEFs are compelled by a competitive mechanism to focus on different objects

Multiple SEFs in ParallelThe computation of each SEF is independent of the other SEFs, allowing for all SEFs to be computed in parallel, excluding the competition phase

CACTuS TrackerCACTuS Algorithm

Computational ComplexitySEFs perform the prediction phase using 2D convolution operations on several 2D matricesThe complexity of the convolution of a 2D matrix A with 2D matrix B is approximately O(MNmn), where A is M x N and B is m x n

CACTuS TrackerDemonstration

CACTuS TrackerCACTuS Algorithm

Current Implementations and PerformanceIn the testing of the current CACTuS implementation, the following performance figures were observed on our test PC

Number of SEFsImplementationTime per frame (ms) (average)Prediction phase time per frame (ms)16MATLAB1506916C++ CPU67054716C++ OpenMP13910716C++ OpenMP + SSE129107 PC Specs:Intel Core i7 4790K 4.0GHz16GB DDR3-1600 RAMNVIDIA GeForce GTX960 2GB DDR512NVIDIA Jetson PlatformNVIDIA Jetson TK1 DevKit

Tegra K1 System on Chip (SoC)4 Plus 1 Quad Core ARM Cortex A15 2.3GHz CPUKepler GPU with 192 CUDA Cores

2 GB memory16GB eMMc storageUbuntu 14.04

NVIDIA Jetson PlatformWhy the Jetson?

As the majority of computational time is taken by the convolution operations, and the convolution operation is easily parallelised, it is expected that the GPU of the Jetson platform will provide increased performance due to its highly parallel nature

The Jetson is also a convenient form factor and conforms to a low power budget of only 10W

Tracking on Jetson with CACTuSProject Goals

Create an implementation of CACTuS that compiles and runs on the Jetson platform

Accelerate the implementation of CACTuS using CUDA

Achieve a minimum of 10 frames per second of processing with 2 or more SEFs on the Jetson platformPerformance EnhancementUsing CUDA to increase the performance of CACTuS

Performance profiling was used to focus efforts.Number of SEFsImplementationTime per frame (ms) (average)Prediction phase time per frame (ms)16MATLAB1506916C++ CPU67054716C++ OpenMP + SSE129107 PC Specs:Intel Core i7 4790K 4.0GHz16GB DDR3-1600 RAMNVIDIA GeForce GTX960 2GB DDR516C++ CUDA Conv652416C++ CUDA + OMP 291516C++ CUDA Predict271316Performance on JetsonBenchmarking the performance of CACTuS on Jetson

Number of SEFsImplementationTime per frame (ms) (average)Prediction phase time per frame (ms)16MATLAB Desktop1506916C++ CPU Desktop67054716C++ CPU Jetson80853016C++ OMP Jetson22712916C++ CUDA Conv38920816C++ CUDA Predict30215916C++ Hybrid19379A hybrid CPU and GPU approach performed best17Performance on JetsonBy reducing the number of SEFs, performance can be gained, this is a trade off that reduces the number of targets that can be tracked and the overall accuracy and track latency of the trackerNumber of SEFsImplementationTime per frame (ms) (average)Prediction phase time per frame (ms)4MATLAB Desktop40184C++ OMP Desktop35334C++ OMP Jetson77454C++ CUDA Predict89414C++ Hybrid603318ConclusionThe performance of the platform is acceptable for marginal usage as a tracking platform. Achieving 16 frames per second with 4 SEFs, the 10 fps goal has been exceeded.

As a side note, running CACTuS with CUDA acceleration on the desktop in under 30ms per frame for 16 SEFs, demonstrates Real Time processing of the synthetic video, which has not been previously achieved with this version of the CACTuS algorithm.