Abnormal Object Detection byCanonical Scene-based Contextual Model

Sangdon Park2012.10.15.

2

Introduction

Problem Statement

Which objects are abnor-mal?

Input Output

Abnormal Object Detection (AOD)

3

Introduction

Problem Statement

Position-violating abnormal object

Co-occurrence-vio-lating

abnormal objectScale-violating abnormal object

Three types of Abnormal Objects

4

Introduction

Motivation

Photo-shop

Artist

Duck Climbing

Increasing number of Abnormal Images

Applicable to Visual Surveillance

5

Introduction

Motivation

NOT affluent object re-lations

quantitative object re-lations

affluent context typesprior-free object

search

(1) M. J. Choi, A. Torralba, and A. S. Willsky, Context Models and Out-of-context Ob-

jects, To appear in Pattern Recognition Let-ters, 2012.

Tree-relation among ob-jects

Limitation of the conventional method(1)

6

Introduction

Contributions

Abnormal Object Detection

object-level annotation

Generative model for AOD Satisfies four conditions for AOD

Especially, affluent object relationships to strictly handle geometric context

Solve new emerging problem

Novel latent Model

New abnormal dataset

7

Agenda

Conventional Method

Proposed Method

Evaluations

8

Conventional Method

Tree-based model

Tree-basedCo-occurrence

model

Tree-basedsupport model

Efficient, but lack of relationship among ob-ject

9

Proposed Method

Overall process

10

Proposed Method

Image representation

Represent image by a set of bounding boxes that are ex-tracted by object detectors

Each image consists of bounding boxes (=100, in this paper)

Transform “image coordinate” to “camera coordinate” by simple triangulation

Represent position and scale information altogether

Object-level image represen-tation “Undo” projectivity

11

Proposed Method

Main Idea

11

Which object is ab-normal?

Which object is less co-occur, floated/sunken, or big/small?

Define dist. of normal data & Com-pare?

Compare the input with the distribution of normal objects

Check likelihood of input given the dist.

Identify abnormal ones!

How to represent the distribution of normal scene? Construct the Canonical Scene (CS) model How to compare the input scene with the normal scene? Matching transformation T for CS Similarity measure to compare the input scene and transformed CS

12

Proposed Method

Model

Define “Canonical Scene”

Natural distributions of normal objects

Less co-occurring objects does not exist

“Objects” are on the ground plane

Follows leaned truncated Gauss-ian distribution

“Outdoor” CS

13

Proposed Method

ModelDefine matching transformation & similar-

ity measure

Matching transformation T: 2D isometric transformation

Similarity measure ),,|,(),( ,,,,,, ,

TlsxKpxLm nononononoTls no

14

Proposed Method

Model

Return to the goal

Appearance Model)|( cyp

Defined as conven-tional model

Model

Decom-pose

Location(Contextual) ModelKlxK d),,|,( Tsp

Defined by previous similarity measure

Prior model),,,( Tsp cl

Prior on la-tent variables

15

Proposed Method

Model

Parameters of Canonical

Scene

Isometry

Generative model

16

Proposed Method

Inference by Pop-MCMC

Advantages of Pop-MCMC Multiple Markov chains with genetic opera-

tions escape from local optimum

Efficient when the objective function is multi-modal and/or high dimensional

17

Proposed Method

Learning

Estimate T, thus making complete data Assumes all “objects” in normal images are on the

ground plane T is a transformation that transform ground plane in

world coord. to slanted plane in camera coord.1T

Learning strat-egy

Algorithm

18

Evaluation

New Abnormal Dataset

#images 149

#Co-occur-rence

38

#Position 53#Scale 44#mixed 14

Only abnormal objects are an-notated

Scene types are also anno-tated

19

Evaluation

Quantitative comparisons

CO+SUP: M. J. Choi, A. Torralba, and A. S. Willsky, Context Models and Out-of-context Objects, To appear in Pattern Recognition Letters, 2012.

Proposed method(“red”) outperforms the baseline(“green”)

20

Evaluation

Qualitative comparisons Because of

affluent ob-ject relation, floating per-son is de-tected as most abnor-mal objects

21

Evaluation

Qualitative results

Only top-5 most abnormal objects are represented

22

Conclusion

Learning Full parameter learning is required Annotation errors Cannot estimate ground

plane strictly poor performance on detecting scale-violating abnormal objects

New abnormal dataset Generative model Satisfies four conditions for AOD

Especially, affluent object relationships to strictly han-dle geometric context

State-of-the-art performance

Novel Model for Abnormal Object Detection

Limitations