Object Detection,

Tracking, Classification,

and CountingSHOUNAK MITRA

ADVISOR: PROFESSOR TAT. S. FU, PHD, P.E.

CIVIL AND ENVIRONMENTAL ENGINEERING DEPARTMENT

UNIVERSITY OF NEW HAMPSHIRE

1

Overview

Significance of the project

Use of camera angles

Video Demonstration

Object Detection

Noise and Shadow issues

Tracking

Classification

Counting

2

Project Significance: Pedestrian Detection and Counting

Synchronization of the Objects passing

over the bridge and the readings of

strain gauges and accelerometers.

3

Clip Obtained from Prof. Bell’s lab

(Travis and Griggs)

Camera angles obtained from DOT

Algorithm 5

Read

Video File

Background

Separation/

Foreground

Detection

Foreground

Filtration

Blob

Analysis

Detect

Boxes

Noise and

Shadow

Issues

Classification

and

Counting

Detection Phase

Processing PhaseFinal Phase

Object Detection FlowVIDEO FRAME FOREGROUND DETECTION

FOREGROUND FILTRATION OBJECT DETECTION AND COUNTING

6

Noise Removal

Preprocessing and Thresholdng:

Deleting boxes formed at unexpected

locations

Kalman Filter

7

What is Kalman Filter?

A Kalman filter is an optimal recursive data processing algorithm

The Kalman filter incorporates all information that can be provided to it. It processes all available measurements, regardless of their precision, to estimate the current value of the variables of interest

Computationally efficient due to its recursive structure

Assumes that variables being estimated are time dependent

8

What does it do?

Predictor: predicts parameter values ahead of current

measurements

Noise Reduction: reduces noise introduced by inaccurate

detections

Tracking: Facilitates the process of association of multiple objects to

their tracks

9

Kalman Filter AKA Predictor - Corrector

(1) Project the state ahead

xˆ-k = Axˆk – 1 + Buk – 1

(2) Project the error covariance ahead

P-k = APk – 1 AT + Q

10

Measurement Update (“Correct”)

(1) Compute the Kalman gain

K. k = P-kH

T (HP-kH

T + R)–1

(2) Update estimate with measurement zk

xˆk = xˆ-k + Kk(zk – Hxˆ-

k )

(3) Update the error covariance

Pk = (I – KkH )P-k

Time Update State: Responsible for projecting forward in time the current state

and the error covariance estimates to obtain the a priori estimates for the next

time step.

Measure update state: Responsible for feedback, i.e. for incorporating a new

measurement into the a priori estimate to obtain an improved a posteriori

estimate.

Tracking using Kalman Filter 11

The Problem of Shadow 12

Object Misclassification

Overlapping of Objects

Shadow

Region

Shadow Detection Flow

YES NO

13

Specify

Threshold for

Shadow (Sth)

Get Current

Frame Fn

Store

Background

Frame B

Apply Gaussian

Smoothening

(GB & GFn)

Dn =

B/Fn < 1

Multiply by a

factor > 20

(RDn)

Shadow Detection FlowBackground in RGB Scale Background in Gray Scale Foreground in Gray Scale

Unfiltered Shadow detection Thresholding of Shadow Filtered Shadow Binary Scale

14

Classification

Color coded classification

Centroid lying in the color

15

Color Coded Classification and

counting

16

DELETED

SHADOW

REGION

Demonstration Video 17