classification of abnormal crowd behavior using...

CLASSIFICATION OF ABNORMAL CROWD BEHAVIOR USING IMAGE

PROCESSING AND STATE MACHINES

NG TZE JIA

A thesis submitted in fulfilment of the

requirements for the award of the degree of

Master of Engineering (Electrical)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

AUGUST 2015

iii

An appreciation to all who made this possible

especially my parents, supervisor and fellow friends.

iv

ACKNOWLEDGMENT

It is a valuable experience for me in working on this project and many people

have contributed towards my thoughts and understanding in making this a success. I

would like to express my gratitude to the people who have offered their precious help

and advices to me.

First of all, I would like to express my deepest gratitude to my project

supervisor, Dr. Usman Ullah Sheikh for guidance, motivations, advices and

friendship. This thesis would not be presented here without his continuous

assistances in overcoming all problems and difficulties.

Next, I wish to convey my sincere appreciation to my family for their love,

blessing and support both mentally and financially. Last but not least, I would like to

thank all my fellow friends who were involved directly or indirectly for their

knowledge, support, kindness and the time we spent together. Their views and tips

suggested were handy indeed.

v

ABSTRACT

The study of crowd behavior in public areas or during public events such as

subway station, airport and shopping mall had been started two decades ago. In this

thesis, an automated video surveillance to detect abnormal activities in a crowd using

the concept of state machine is proposed. This method is divided into three stages

which are pre-processing, feature extraction and behaviour classification. In pre-

processing, frame differencing is used for segmentation while optical flow is

performed to estimate the crowd motion. Extracted features consist of global and

local features. Global features will consider the features on the whole frame whereas

local features only consider the features on each detected object. Based on extracted

features, abnormal crowd behaviour can be classified using state machines. The

proposed state machine contains four states which will evaluate different features in

different states respectively. The frames that are able to reach the final state of the

behaviour in its state machine will be classified as the behaviour. The behaviours that

can be detected are walking, running, crowd formation, crowd splitting and panic

crowd. The method is validated using UMN data set and PETS 2009 data set. The

result of the classification has achieved an accuracy of 96.3%.

vi

ABSTRAK

Kajian tentang kelakuan orang ramai di kawasan awam atau dalam acara

awam seperti stesen kereta api, lapangan terbang dan pusat membeli-belah telah

dikaji sejak dua dekad yang lalu. Dalam tesis ini, sistem pengawasan video

automatik yang boleh mengesan aktiviti orang ramai yang tidak normal dengan

menggunakan konsep mesin keadaan telah dicadangkan. Kaedah ini dibahagikan

kepada tiga peringkat iaitu pra-pemprosesan, pengekstrakan ciri-ciri dan pengelasan

kelakuan. Dalam pra-pemprosesan, penolakan gambar digunakan untuk segmentasi

manakala aliran optik digunakan untuk menganggar gerakan orang ramai.

Pengekstrakan ciri- ciri meliputi ciri-ciri global and ciri-ciri tempatan. Ciri-ciri global

merupakan ciri-ciri yang diambil daripada seluruh gambar manakala ciri-ciri

tempatan merupakan ciri- ciri yang diekstrak daripada objek-objek yang dikesan.

Berdasarkan ciri-ciri yang diekstrak, kelakuan orang ramai yang tidak normal boleh

dikelaskan dengan menggunakan mesin keadaan. Mesin keadaan yang dicadangkan

mengandungi empat keadaan yang akan menilai ciri-ciri yang berbeza dalam

keadaan masing-masing. Gambar yang dapat mencapai keadaan yang terakhir bagi

mesin keadaan kelakuan akan dikelaskan dalam kelakuan tersebut. Kelakuan yang

dapat dikelas dalam sistem ini ialah berjalan, berlari, orang ramai berkumpul, orang

ramai berpecah dan orang ramai panik. Kaedah ini telah disahkan dengan

menggunakan set data UMN dan set data PETS 2009. Ketepatan hasil pengelasan

telah mencapai 96.3%.

vii

TABLE OF CONTENT

CHAPTER TITLE PAGE

DECLARATION ii

DEDICATION iii

ACKNOWLEDGEMENT iv

ABSTRACT v

ABSTRAK vi

TABLE OF CONTENTS vii

LIST OF TABLES x

LIST OF FIGURES xii

LIST OF SYMBOLS xiv

1 INTRODUCTION 1

1.1 Introduction 1

1.2 Background 2

1.3 Research Questions 4

1.4 Problem Statement 5

1.5 Objectives of Study 5

1.6 Scope of Work 6

1.7 Contributions 6

1.8 Thesis Organization 7

2 LITERATURE REVIEW 9

2.1 Introduction 9

2.2 Pre-processing 11

2.3 Feature extraction 12

viii

2.4 Object Tracking 19

2.5 Summary 28

3 METHODOLOGY 30

3.1 Introduction 30

3.2 Pre-processing 31

3.2.1 Image Resize 31

3.2.2 Convert RGB to Grayscale Image 32

3.3 Segmentation 33

3.3.1 Obtain Region of Interest(ROI) 33

3.3.2 Removing Small Components 34

3.3.3 Dilation 34

3.3 Feature Extraction 35

3.3.1 Optical Flow 35

3.3.2 Global Features 36

3.3.3 Local Features 39

3.4 Crowd Behavior Recognition using Simple State

Machine Concept

41

3.4.1 Classify Running and Walking Components 42

3.4.2 Classify Crowd Formation Event 43

3.4.3 Classify Crowd Splitting Event 48

3.4.4 Classify Crowd Evacuation/Panic Event 49

3.6 Summary 53

4 RESULT & DISCUSSION 55

4.1 Introduction 55

4.2 Dataset 55

4.2.1 UMN Dataset 56

4.2.2 PETS 2009 Dataset 57

4.3 Simple Graphical User Interface (GUI) 62

4.3.1 Load Image 62

4.3.2 Behavior Selections 62

4.4 Frame Distribution 64

ix

4.5 Calculation of Error Rate and Accuracy 64

4.6 Categories of Crowd Behaviors 66

4.7 Ground Truth 67

4.7.1 UMN Dataset Ground Truth 69

4.7.2 PETS 2009 Dataset Ground Truth 70

4.8 Results 70

4.8.1 Walking and Running 70

4.8.2 Crowd Formation 72

4.8.3 Crowd Splitting 73

4.8.4 Evacuation or Panic Crowd 74

4.8.5 Result Summary 77

4.9 Discussion 78

4.9.1 Result Comparisons 78

4.9.2 Reasons of Fail Detections 80

4.9.2.1 Illumination 80

4.9.2.2 Occlusion 82

4.9.2.3 Crowd Formation and Splitting Exist but not

Obvious

82

4.9.2.4 Uncertainties during Transition of Behaviors 82

4.10Summary 83

5 CONCLUSION 84

5.1 Conclusion 84

5.2 Recommendations and Future Improvements 85

REFERENCES 87

x

LIST OF TABLES

TABLE NO. TITLE PAGE

2.1 Different types of feature extraction analysis and related

works.

14

2.2 Shape feature descriptors and related works. 15

2.3 Appearance-based feature descriptors and related work. 16

2.4 Feature selection for object tracking. 17

2.5 Types of Optical Flow Method. 18

2.6 Categories of object tracking approach and related works. 20

2.7 Categories of event recognition approach and related

works.

22

2.8 Examples of learning models and its related work. 23

4.1 Shows Calibration data of setting up the cameras. 59

4.2 Cameras used to film the datasets. 61

4.3 UMN dataset frames distribution in each video. 64

4.4 PETS 2009 dataset frames distribution in each video. 65

4.5 The ground truth for UMN datasets. 69

4.6 The ground truth for PETS 2009 datasets. 70

4.7 Error rate for walking behavior detection in UMN dataset. 71

4.8 Error rate for walking behavior detection in PETS 2009

dataset.

72

4.9 Error rate for crowd formation detection in UMN dataset. 73

4.10 Error rate for crowd formation detection in PETS 2009 74

xi

dataset.

4.11 Error rate for crowd splitting detection in UMN dataset. 73

4.12 Error rate for crowd splitting detection in PETS 2009

dataset.

75

4.13 Error rate for panic event detection in PETS 2009 dataset. 76

4.14 Error rate for panic event detection in UMN dataset. 77

4.15 Summary of results. 77

4.16 PETS 2009 dataset experimental result compared with

other approaches.

79

4.17 UMN dataset experimental result compared with other

approaches.

79

xii

LIST OF FIGURES

FIGURE NO. TITLE PAGE

2.1 A bar chart which shows number of papers related to

human or crowd behavior analysis published each year.

10

2.2 Types of approach in crowd analysis. 11

3.1 General flow of the method proposed. 30

3.2 Groups of directions. 38

3.3 Example to find inward and outward vectors. 38

3.4 Two cropped objects in the scene with their respective

centroid.

46

3.5 A scene that is being divided into nine partitions. 46

3.6 The crowd formation state machine. 47

3.7 Three different situations when there are 2 objects

moving in the same region in different directions.

47

3.8 The crowd splitting state machine. 49

3.9 The evacuation state machine. 51

3.10 Flow chart of the overall proposed system. 53

4.1 Scenes in UMN datasets. 58

4.2 Eight different views from PETS 2009 dataset. 58

4.3 Location of the 8 cameras installed. 61

4.4 Simple graphical user interface. 63

4.5 A frame loaded into the interface. 63

4.6 The result of a selected frame. 63

xiii

4.7 The examples of results obtained. 68

4.8 Poor lighting in (a) indoor and (b) outdoor scene. 81

4.9 Strong lighting causing solid shadows and unusual

brightness.

81

4.10 An example of a scene with occlusion. 81

xiv

LIST OF SYMBOLS

𝑨 - Image

𝐴𝑎 - Average acceleration

Ar - Range of average acceleration

Atp - Total area of objects detected in the partition

𝑩 - Structuring element

𝑩𝒔 - Symmetric of 𝐵 defined by Eq. 3.2 below.

b - Number of magnitude of optical flow vectors found in an object

𝐶𝑎 - Acceleration constant for user to control its sensitivity

𝐶𝑐 - Crowd constant

𝐶𝑣 - Average velocity constant for user to control its sensitivity

𝐶𝑤𝑟 - Walk/run constant set by user to control its sensitivity

𝑬 - Euclidean space

ℎ - Height of the image

𝐿𝑎 - Local average vector length

Ldmv - Direction with majority vectors

𝐿𝑗 - Magnitude of the optical flow vectors found in an object

𝐿𝑚𝑎𝑥 - Longest vector in found in an object

Loutward - Ratio of the outward vectors in an object

m - Number of outward vectors

n - Number of optical flow vectors found in a frame

http://en.wikipedia.org/wiki/Rotational_symmetry

xv

Nfps - Number of frames per second

Nood2 - Number of objects moving towards each other

Np - Number of objects in a partition

Poutward - Ratio of outwards vectors

𝑃1 - Lowest point in y-axis of the first cropped image

𝑃2 - Lowest point in y-axis of the second cropped image

𝑝 - Number of parts that an image needs to be divided vertically

𝑞 - Image part where q starts with 1 until p

𝑅 - Ratio of size of an individual at a specific position

𝑆1 - Size of the first cropped image

𝑆2 - Size of the second cropped image

𝑉𝑎 - Average magnitude of the optical flow vectors,

𝑉𝑖 - Magnitude of the optical flow vectors.

𝑉𝑎 - Average vector magnitude of the current frame

𝑉𝑎−1 - Average vector magnitude of the previous frame

𝑉𝑚𝑎𝑥 - Longest vector magnitude in found in frame

𝑉𝑖 - Vectors magnitude found in a frame.

𝒗 - Optical flow vectors

𝒗𝑐 - Vector point from 𝒗 towards the center of the frame

𝒗𝑥 - x-component of an optical flow vector

𝒗𝑦 - y-component of an optical flow vector

𝑉𝑗 - Magnitude of the outward vectors

𝑉𝑖 - Magnitude of the optical flow vectors.

𝑉𝑟𝑚 - Range of vector to determine walking and running event

Vr - Range of average acceleration

𝒛 - Translation vector

𝜃 - Angle between the flow vector, 𝒗 and the vector point from 𝒗

towards the center of the frame, 𝒗𝑐.

87

REFERENCES

Abrishami, V., Rezaee, A., Baherzadeh, H., & Abrishami, H. (2009). Real-time

pedestrian detecting and tracking in crowded and complicated scenario. 3rd

International Conference on Imaging for Crime Detection and Prevention ICDP

2009, 39.

Abuarafah, A. G., Khozium, M. O., & AbdRabou, E. (2012). Real-time Crowd

Monitoring using Infrared Thermal Video Sequences. Journal of American Science,

8(3), 133–140.

Alahi, A., Jacques, L., Boursier, Y., & Vandergheynst, P. (2009). Sparsity-driven people

localization algorithm: Evaluation in crowded scenes environments. In

Performance Evaluation of Tracking and Surveillance (PETS-Winter), 2009

Twelfth IEEE International Workshop on (pp. 1–8). IEEE.

Aliakbarpour, H., & Khoshhal, K. (2011). HMM-Based Abnormal Behaviour Detection

Using Heterogeneous Sensor Network. Technological Innovation for Sustainability.

Springer Berlin Heidelberg.

Alper, Y., Omar, J., & Mubarak, S. (2006). Object tracking: A survey. ACM Computing

Surveys, 38(4), 13.

Andrade, E. L., Blunsden, S., & Fisher, R. B. (2006). Hidden Markov Models for

Optical Flow Analysis in Crowds. 18th International Conference on Pattern

Recognition (2006), 1(c), 460–463. doi:10.1109/ICPR.2006.621

Andrade, E. L., Blunsden, S., & Fisher, R. B. (2006). Modelling Crowd Scenes for

Event Detection. 18th International Conference on Pattern Recognition (ICPR’06),

175–178. doi:10.1109/ICPR.2006.806

Andrade, E. L., Fisher, R. B., & Blunsden, S. (2006). Detection of emergency events in

crowded scenes.

Antonakaki, P., Kosmopoulos, D., & Perantonis, S. J. (2009). Detecting abnormal

human behaviour using multiple cameras. Signal Processing, 89(9), 1723–1738.

doi:10.1016/j.sigpro.2009.03.016

88

Basharat, A., Gritai, A., & Shah, M. (2008). Learning object motion patterns for

anomaly detection and improved object detection. 2008 IEEE Conference on

Computer Vision and Pattern Recognition, 1–8. doi:10.1109/CVPR.2008.4587510

Beauchemin, S. S., & Barron, J. L. (1995). The computation of optical flow. ACM

Computing Surveys (CSUR), 27(3), 433-466.

Beleznai, C., & Bischof, H. (2009). Fast human detection in crowded scenes by contour

integration and local shape estimation. Computer Vision and Pattern Recognition,

2009. CVPR 2009. IEEE Conference.

Benabbas, Y., Ihaddadene, N., & Djeraba, C. (2011). Motion Pattern Extraction and

Event Detection for Automatic Visual Surveillance. EURASIP Journal on Image

and Video Processing, 2011, 1–15. doi:10.1155/2011/163682

Benezeth, Y., Jodoin, P. M., Saligrama, V., & Rosenberger, C. (2009). Abnormal events

detection based on spatio-temporal co-occurences. IEEE Conference on Computer

Vision and Pattern Recognition (2009), 2458–2465.

doi:10.1109/CVPR.2009.5206686

Bicego, M., Cristani, M., & Murino, V. (2006). Unsupervised scene analysis: A hidden

Markov model approach. Computer Vision and Image Understanding, 102(1), 22–

41. doi:10.1016/j.cviu.2005.09.001

Bouttefroy, P. L. M., Bouzerdoum, A., Phung, S. L., & Beghdadi, A. (2008). Local

estimation of displacement density for abnormal behavior detection. In Machine

Learning for Signal Processing, 2008. MLSP 2008. IEEE Workshop on (pp. 386–

391). IEEE.

Calderara, S., Cucchiara, R., & Prati, A. (2007). Detection of abnormal behaviors using

a mixture of von mises distributions. In Advanced Video and Signal Based

Surveillance, 2007. AVSS 2007. IEEE Conference on (pp. 141–146). IEEE.

Cao, T., Wu, X., Guo, J., Yu, S., & Xu, Y. (2009). Abnormal crowd motion analysis. In

Robotics and Biomimetics (ROBIO), 2009 IEEE International Conference on (pp.

1709–1714). IEEE.

Chan, A. B., Morrow, M., & Vasconcelos, N. (2009). Analysis of crowded scenes using

holistic properties. In Performance Evaluation of Tracking and Surveillance

workshop at CVPR (pp. 101–108).

Chen, D.-Y., & Huang, P.-C. (2011). Motion-based unusual event detection in human

crowds. Journal of Visual Communication and Image Representation, 22(2), 178–

186. doi:10.1016/j.jvcir.2010.12.004

89

Chen, D.-Y., & Huang, P.-C. (2013). Visual-Based Human Crowds Behavior Analysis

Based on Graph Modeling and Matching. Sensors Journal, IEEE, 13(6), 2129–2138.

Chen, Y., Liang, G., Lee, K. K., & Xu, Y. (2007). Abnormal behavior detection by

multi-SVM-based Bayesian network. In Information Acquisition, 2007. ICIA’07.

International Conference on (pp. 298–303). IEEE.

Cheriyadat, A. M., & Radke, R. J. (2008). Detecting dominant motions in dense crowds.

Selected Topics in Signal Processing, IEEE Journal of, 2(4), 568–581.

Cho, S.-B., & Park, H.-J. (2003). Efficient anomaly detection by modeling privilege

flows using hidden Markov model. Computers & Security, 22(1), 45–55.

Cristani, M., Raghavendra, R., Del Bue, A., & Murino, V. (2013). Human behavior

analysis in video surveillance: A Social Signal Processing perspective.

Neurocomputing, 100, 86–97. doi:10.1016/j.neucom.2011.12.038

Cui, P., Sun, L.-F., Liu, Z.-Q., & Yang, S.-Q. (2007). A sequential monte carlo approach

to anomaly detection in tracking visual events. In Computer Vision and Pattern

Recognition, 2007. CVPR’07. IEEE Conference on (pp. 1–8). IEEE.

Davies, A. C., Yin, J. H., & Velastin, S. A. (1995). Crowd monitoring using image

processing. Electronics & Communication Engineering Journal, 7(1), 37–47.

Dimitrijevic, M., Lepetit, V., & Fua, P. (2006). Human body pose detection using

Bayesian spatio-temporal templates. Computer Vision and Image Understanding,

104(2), 127–139.

Djeraba, C., Lablack, A., & Benabbas, Y. (2010). Multi-Modal User Interactions in

Controlled Environments (Vol. 34). springer.

Du, D., Qi, H., Huang, Q., Zeng, W., & Zhang, C. (2013). Abnormal event detection in

crowded scenes based on Structural Multi-scale Motion Interrelated Patterns. In

Multimedia and Expo (ICME), 2013 IEEE International Conference on (pp. 1–6).

IEEE.

Elgammal, A., Duraiswami, R., Harwood, D., & Davis, L. S. (2002). Background and

foreground modeling using nonparametric kernel density estimation for visual

surveillance. Proceedings of the IEEE, 90(7), 1151–1163.

Fagette, A., Jamet, P., Racoceanu, D., & Dufour, J. (2013). Particle Video for Crowd

Flow Tracking.

Fei, T., Sundong, L., & Sen, G. (2009). A novel method of crowd estimation in public

locations. 2009 International Conference on Future BioMedical Information

Engineering (FBIE), 339–342. doi:10.1109/FBIE.2009.5405848

90

Garate, C., Bilinsky, P., & Bremond, F. (2009). Crowd event recognition using hog

tracker. In Performance Evaluation of Tracking and Surveillance (PETS-Winter),

2009 Twelfth IEEE International Workshop on (pp. 1–6). IEEE.

Ge, W., Collins, R. T., & Ruback, B. (2009). Automatically detecting the small group

structure of a crowd. In Applications of computer vision (wacv), 2009 workshop on

(pp. 1–8). IEEE.

Ghidoni, S., Cielniak, G., & Menegatti, E. (2013). Texture-based crowd detection and

localisation. Intelligent Autonomous Systems 12, 725–736.

Gong, S., Ng, J., & Sherrah, J. (2002). On the semantics of visual behaviour, structured

events and trajectories of human action. Image and Vision Computing, 20(12), 873–

888.

Guo, J., Wu, X., Zhong, Z., & Yu, S. (2009). An intelligent surveillance system based on

RANSAC algorithm. 2009 International Conference on Mechatronics and

Automation, 2888–2893. doi:10.1109/ICMA.2009.5246385

Haidar Sharif, M., & Djeraba, C. (2012). An entropy approach for abnormal activities

detection in video streams. Pattern Recognition, 45(7), 2543–2561.

Han, J., Feng, M., & de With, P. H. N. (2008). A real-time video surveillance system

with human occlusion handling using nonlinear regression. In Multimedia and Expo,

2008 IEEE International Conference on (pp. 305–308). IEEE.

Haque, M., & Murshed, M. (2010). Panic-driven event detection from surveillance video

stream without track and motion features. Multimedia and Expo (ICME), 2010

IEEE International Conference On. IEEE, 173–178.

Hu, M., Ali, S., & Shah, M. (2008). Learning motion patterns in crowded scenes using

motion flow field. In ICPR (pp. 1–5).

Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., & Maybank, S. (2006). A system for learning

statistical motion patterns. IEEE Transactions on Pattern Analysis and Machine

Intelligence, 28(9), 1450–64. doi:10.1109/TPAMI.2006.176

Hu, W., Xie, D., & Tan, T. (2004). A hierarchical self-organizing approach for learning

the patterns of motion trajectories. Neural Networks, IEEE Transactions on, 15(1),

135–144.

Huang, D.-S., Wunsch, D. C., Levine, D. S., & Jo, K.-H. (2008). Advanced Intelligent

Computing Theories and Applications. With Aspects of Artificial Intelligence:

Fourth International Conference on Intelligent Computing (1st ed.). Springer

Publishing Company, Incorporated.

91

Husni, M., & Suryana, N. (2010). Crowd event detection in computer vision. In Signal

Processing Systems (ICSPS), 2010 2nd International Conference on (Vol. 1, pp.

V1–444). IEEE.

Ihaddadene, N., & Djeraba, C. (2008). Real-time crowd motion analysis. In Pattern

Recognition, 2008. ICPR 2008. 19th International Conference on (pp. 1–4). IEEE.

Ismail, A. (2009). Tracking of a person in semi-dense crowd, (November), 77.

Jacques Jr, J. C. S., Braun, A., Soldera, J., Musse, S. R., & Jung, C. R. (2007).

Understanding people motion in video sequences using Voronoi diagrams. Pattern

Analysis and Applications, 10(4), 321–332.

Jager, M., Knoll, C., & Hamprecht, F. A. (2008). Weakly supervised learning of a

classifier for unusual event detection. Image Processing, IEEE Transactions on,

17(9), 1700–1708.

Junior, S. J. (2010). Crowd analysis using computer vision techniques. IEEE Signal

Processing Magazine, 27(5), 66–77.

Kelly, P., O’Connor, N. E., & Smeaton, A. F. (2009). Robust pedestrian detection and

tracking in crowded scenes. Image and Vision Computing, 27(10), 1445–1458.

doi:10.1016/j.imavis.2008.04.006

Khalid, S., Sajjad, S., Jabbar, S., & Rho, S. (n.d.). Accurate and efficient shape matching

approach using vocabularies of multi-feature space representations.

Khan, S. M., & Shah, M. (2009). Tracking multiple occluding people by localizing on

multiple scene planes. Pattern Analysis and Machine Intelligence, IEEE

Transactions on, 31(3), 505–519.

Khansari, M., Rabiee, H. R., Asadi, M., & Ghanbari, M. (2007). Crowded scene object

tracking in presence of Gaussian White noise using undecimated wavelet features.

2007 9th International Symposium on Signal Processing and Its Applications, 1–4.

doi:10.1109/ISSPA.2007.4555609

Kilambi, P., Masoud, O., & Papanikolopoulos, N. (2006). Crowd analysis at mass transit

sites. In Intelligent Transportation Systems Conference, 2006. ITSC’06. IEEE (pp.

753–758). IEEE.

Kim, C., & Hwang, J.-N. (2002). Fast and automatic video object segmentation and

tracking for content-based applications. Circuits and Systems for Video Technology,

IEEE Transactions on, 12(2), 122–129.

Kim, G.-J., Eom, K.-Y., Kim, M.-H., Jung, J.-Y., & Ahn, T.-K. (2010). Automated

measurement of crowd density based on edge detection and optical flow. In

92

Industrial Mechatronics and Automation (ICIMA), 2010 2nd International

Conference on (Vol. 2, pp. 553–556). IEEE.

Kratz, L., & Nishino, K. (2009). Anomaly detection in extremely crowded scenes using

spatio-temporal motion pattern models. In Computer Vision and Pattern

Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 1446–1453). IEEE.

Krausz, B., & Bauckhage, C. (2012). Loveparade 2010: Automatic video analysis of a

crowd disaster. Computer Vision and Image Understanding, 116(3), 307–319.

doi:10.1016/j.cviu.2011.08.006

Lei, C., & Yang, Y. H. (2009, September). Optical flow estimation on coarse-to-fine

region-trees using discrete optimization. In Computer Vision, 2009 IEEE 12th

International Conference on (pp. 1562-1569). IEEE.

Lempitsky, V., Roth, S., & Rother, C. (2008, June). Fusionflow: Discrete-continuous

optimization for optical flow estimation. In Computer Vision and Pattern

Recognition, 2008. CVPR 2008. IEEE Conference on (pp. 1-8). IEEE.

Li, N., Xu, D., Wu, X., & Liang, G. (2013). Anomaly detection in crowds using a space

MRF with incremental updates. In Fifth International Conference on Digital Image

Processing (p. 88782J–88782J). International Society for Optics and Photonics.

Li, N., & Zhang, Z. (2011). Abnormal Crowd Behavior Detection Using Topological

Methods. In Software Engineering, Artificial Intelligence, Networking and

Parallel/Distributed Computing (SNPD), 2011 12th ACIS International Conference

on (pp. 13–18). IEEE.

Li, W., Wu, X., Matsumoto, K., & Zhao, H.-A. (2010). A new approach of crowd

density estimation. In TENCON 2010-2010 IEEE Region 10 Conference (pp. 200–

203). IEEE.

Li, X., & Qin, W. (2012). A crowd behavior model based on reciprocal velocity obstacle

algorithm. Procedia Engineering, 29, 2887–2893.

Li, Y., Huang, C., & Nevatia, R. (2009). Learning to associate: Hybridboosted multi-

target tracker for crowded scene. In Computer Vision and Pattern Recognition,

2009. CVPR 2009. IEEE Conference on (pp. 2953–2960). IEEE.

Liu, C. (2009). Beyond pixels: exploring new representations and applications for

motion analysis (Doctoral dissertation, Massachusetts Institute of Technology).

Loy, C., Xiang, T., & Gong, S. (2011). Detecting and discriminating behavioural

anomalies. Pattern Recognition, (0), 1–32.

93

Ma, R., Li, L., Huang, W., & Tian, Q. (2004). On pixel count based crowd density

estimation for visual surveillance. Cybernetics and Intelligent Systems, 2004 IEEE

Conference On. Vol. 1. IEEE, 1–3.

Mahadevan, V., Li, W., Bhalodia, V., & Vasconcelos, N. (2010). Anomaly detection in

crowded scenes. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE

Conference on (pp. 1975–1981). IEEE.

Marana, A. N., Costa, L. da F., Lotufo, R. A., & Velastin, S. A. (1998). On the efficacy

of texture analysis for crowd monitoring. In Computer Graphics, Image Processing,

and Vision, 1998. Proceedings. SIBGRAPI’98. International Symposium on (pp.

354–361). IEEE.

McHugh, J. M., Konrad, J., Saligrama, V., & Jodoin, P. (2009). Foreground-adaptive

background subtraction. Signal Processing Letters, IEEE, 16(5), 390–393.

Mehran, R., Oyama, A., & Shah, M. (2009). Abnormal crowd behavior detection using

social force model. IEEE Conference on Computer Vision and Pattern Recognition

(2009), 1(1), 935–942. doi:10.1109/CVPR.2009.5206641

Muñoz-Salinas, R., Medina-Carnicer, R., Madrid-Cuevas, F. J., & Carmona-Poyato, A.

(2009). People detection and tracking with multiple stereo cameras using particle

filters. Journal of Visual Communication and Image Representation, 20(5), 339–

350.

Oliver, N. (2000). A Bayesian computer vision system for modeling human interactions.

Pattern Analysis and Machine Intelligence, IEEE Transactions on 22.8, 22(8), 831–

843.

Parvathy, R., Thilakan, S., Joy, M., & Sameera, K. M. (2013). Anomaly Detection Using

Motion Patterns Computed from Optical Flow. In Advances in Computing and

Communications (ICACC), 2013 Third International Conference on (pp. 58–61).

IEEE.

Pathan, S. (2010). Crowd behavior detection by statistical modeling of motion patterns.

Soft Computing and Pattern Recognition (SoCPaR), 2010 International Conference

Of. IEEE, 81–86.

Pathan, S. S., Al-Hamadi, A., & Michaelis, B. (2011). Using conditional random field

for crowd behavior analysis. In Computer Vision–ACCV 2010 Workshops (pp. 370–

379). Springer.

Piciarelli, C. (2008). Trajectory-based anomalous event detection. Circuits and Systems

for Video Technology, IEEE Transactions, 18(11), 1544–1554.

94

Piriou, G., Bouthemy, P., & Yao, J.-F. (2004). Extraction of semantic dynamic content

from videos with probabilistic motion models. In Computer Vision-ECCV 2004 (pp.

145–157). Springer.

Qiao, W., Wang, H., Wu, X., & Liu, P. (2009). Crowd target extraction and density

analysis based on ftle and glcm. Image and Signal Processing, 2009. CISP’09. 2nd

International Congress on IEEE, (0).

Raghavendra, R., Cristani, M., Del Bue, A., Sangineto, E., & Murino, V. (2013).

Anomaly Detection in Crowded Scenes: A Novel Framework Based on Swarm

Optimization and Social Force Modeling. In Modeling, Simulation and Visual

Analysis of Crowds (pp. 383–411). Springer.

Raghavendra, R., Del Bue, A., Cristani, M., & Murino, V. (2011). Abnormal crowd

behavior detection by social force optimization. In Human Behavior Understanding

(pp. 134–145). Springer.

Reisman, P., Mano, O., Avidan, S., & Shashua, A. (2004). Crowd detection in video

sequences. In Intelligent Vehicles Symposium, 2004 IEEE (pp. 66–71). IEEE.

Robertson, N., & Reid, I. (2006). A general method for human activity recognition in

video. Computer Vision and Image Understanding, 104(2), 232–248.

Rodriguez, M., Ali, S., & Kanade, T. (2009). Tracking in unstructured crowded scenes.

In Computer Vision, 2009 IEEE 12th International Conference on (pp. 1389–1396).

IEEE.

Saxena, S., Brémond, F., Thonnat, M., & Ma, R. (2008). Crowd behavior recognition for

video surveillance. In Advanced Concepts for Intelligent Vision Systems (pp. 970–

981). Springer.

Shao, L., & Ji, L. (2009). Motion Histogram Analysis Based Key Frame Extraction for

Human Action/Activity Representation. 2009 Canadian Conference on Computer

and Robot Vision, 88–92. doi:10.1109/CRV.2009.36

Sharif, M. H., Uyaver, S., & Djeraba, C. (2010). Crowd behavior surveillance using

bhattacharyya distance metric. In Computational Modeling of Objects Represented

in Images (pp. 311–323). Springer.

Shum, H. Y., & Szeliski, R. (1998, January). Construction and refinement of panoramic

mosaics with global and local alignment. In Computer Vision, 1998. Sixth


Sjarif, N. N. A., Shamsuddin, S. M., Hashim, S. Z. M., & Yuhaniz, S. S. (2011). Crowd

Analysis and Its Applications. In Software Engineering and Computer Systems (pp.

687–697). Springer.

95

Srivastava, S., Ng, K. K., & Delp, E. J. (2011). Crowd flow estimation using multiple

visual features for scenes with changing crowd densities. In Advanced Video and

Signal-Based Surveillance (AVSS), 2011 8th IEEE International Conference on (pp.

60–65). IEEE.

Stauffer, C., & Grimson, W. (2000). Learning patterns of activity using real-time

tracking. Pattern Analysis and Machine Intelligence, IEEE Transactions, 22(8),

747–757.

Su, H., Yang, H., Zheng, S., Fan, Y., & Wei, S. (2013). The Large-Scale Crowd

Behavior Perception Based on Spatio-Temporal Viscous Fluid Field. IEEE

Transactions on Information Forensics and Security, 8(10), 1575–1589.

doi:10.1109/TIFS.2013.2277773

Su, Z., Wei, H., & Wei, S. (2014). Crowd Event Perception Based on Spatiotemporal

Weber Field. Journal of Electrical and Computer Engineering, 2014.

Talu, M. F., Türkoğlu, İ., & Cebeci, M. (2011). A hybrid tracking method for scaled and

oriented objects in crowded scenes. Expert Systems with Applications, 38, 13682–

13687. doi:10.1016/j.eswa.2011.04.153

Tehrani, M. A., Kleihorst, R., Meijer, P., & Spaanenburg, L. (2009). Abnormal motion

detection in a real-time smart camera system. In Distributed Smart Cameras, 2009.

ICDSC 2009. Third ACM/IEEE International Conference on (pp. 1–7). IEEE.

Thida, M., Eng, H.-L., & Remagnino, P. (2013). Laplacian Eigenmap With Temporal

Constraints for Local Abnormality Detection in Crowded Scenes. IEEE

Transactions on Cybernetics, 1–10. doi:10.1109/TCYB.2013.2242059

Trinh, H., Fan, Q., Pan, J., Gabbur, P., Miyazawa, S., & Pankanti, S. (2011, May).

Detecting human activities in retail surveillance using hierarchical finite state

machine. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE


Tsai, Y., Shih, H., & Huang, C. (2006). Multiple Human Objects Tracking in Crowded

Scenes. 18th International Conference on Pattern Recognition (ICPR’06), 51–54.

doi:10.1109/ICPR.2006.841

Tziakos, I., Cavallaro, A., & Xu, L.-Q. (2010). Event monitoring via local motion

abnormality detection in non-linear subspace. Neurocomputing, 73(10-12), 1881–

1891. doi:10.1016/j.neucom.2009.10.028

Ullah, H., & Conci, N. (2013, September). Structured learning for crowd motion

segmentation. In Image Processing (ICIP), 2013 20th IEEE International

Conference on (pp. 824-828). IEEE.

96

Ullah, H., Tenuti, L., & Conci, N. (2013). Gaussian mixtures for anomaly detection in

crowded scenes. In IS&T/SPIE Electronic Imaging (p. 866303). International

Society for Optics and Photonics.

Vaswani, N., Roy-Chowdhury, A. K., & Chellappa, R. (2005). “ Shape Activity”: a

continuous-state HMM for moving/deforming shapes with application to abnormal

activity detection. Image Processing, IEEE Transactions on, 14(10), 1603–1616.

Viertl, N., Beleznai, C., & Birchbauer, J. (2009). A pedestrian detection system

combining motion detection, spatial grouping and classification. In Image and

Signal Processing and Analysis, 2009. ISPA 2009. Proceedings of 6th International

Symposium on (pp. 194–199). IEEE.

Wang, B., Ye, M., Li, X., Zhao, F., & Ding, J. (2012). Abnormal crowd behavior

detection using high-frequency and spatio-temporal features. Machine Vision and

Applications, 23(3), 501–511.

Wang, S., & Miao, Z. (2010). Anomaly detection in crowd scene. IEEE 10th

INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS,

1220–1223. doi:10.1109/ICOSP.2010.5655356

Wang, T., Chen, J., & Snoussi, H. (2013). Online Detection of Abnormal Events in

Video Streams. Journal of Electrical and Computer Engineering, 2013, 1–12.

doi:10.1155/2013/837275

Wang, T., Chen, J., Zhou, Y., & Snoussi, H. (2013). Online least squares one-class

support vector machines-based abnormal visual event detection. Sensors (Basel,

Switzerland), 13(12), 17130–55. doi:10.3390/s131217130

Wang, T., & Snoussi, H. (2013). Histograms of optical flow orientation for abnormal

events detection. In Performance Evaluation of Tracking and Surveillance (PETS),

2013 IEEE International Workshop on (pp. 45–52). IEEE.

Wang, X., Ma, X., & Grimson, W. E. L. (2009). Unsupervised activity perception in

crowded and complicated scenes using hierarchical bayesian models. IEEE

Transactions on Pattern Analysis and Machine Intelligence, 31(3), 539–55.

doi:10.1109/TPAMI.2008.87

Wu, S., Wong, H.-S., & Yu, Z. (2014). A Bayesian Model for Crowd Escape Behavior

Detection.

Wu, X., Liang, G., Lee, K. K., & Xu, Y. (2006). Crowd density estimation using texture

analysis and learning. In Robotics and Biomimetics, 2006. ROBIO’06. IEEE


97

Wu, X., Ou, Y., Qian, H., & Xu, Y. (2005). A detection system for human abnormal

behavior. Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ

International.

Xu, L., & Anjulan, A. (2008). Crowd behaviours analysis in dynamic visual scenes of

complex environment. Image Processing, 2008. ICIP 2008. 15th IEEE

International, 9–12.

Yang, Z., Zhang, T., Yang, J., Wu, Q., Bai, L., & Yao, L. (2013). Violence detection

based on histogram of optical flow orientation, 9067(Icmv), 906718.

doi:10.1117/12.2051390

Yen, S.-H., & Wang, C.-H. (2013). Abnormal Event Detection Using HOSF. In IT

Convergence and Security (ICITCS), 2013 International Conference on (pp. 1–4).

IEEE.

Yu, Z., & Chen, Y. (2008). A Robust Motion Detection Algorithm for Complex

Background Using Statistical Models. Robotics, Automation and Mechatronics,

2008 IEEE Conference On. IEEE, 00.

Zhan, B., Monekosso, D. N., Remagnino, P., Velastin, S. a., & Xu, L.-Q. (2008). Crowd

analysis: a survey. Machine Vision and Applications, 19(5-6), 345–357.

doi:10.1007/s00138-008-0132-4

Zhan, B., Remagnino, P., Monekosso, D. N., & Velastin, S. (2009). The Analysis of

Crowd Dynamics: From Observations to Modelling. In Computational Intelligence

(pp. 441–472). Springer.

Zhang, D., Peng, H., Haibin, Y., & Lu, Y. (2013). Crowd Abnormal Behavior Detection

Based on Machine Learning. Information Technology Journal, 12(6).

Zhang, J., Zhang, Y., Wang, Y., Yan, G., & Ma, J. (2009). Analysis for Real-Time

Video Monitoring and Determine the Degree of Crowd. In Intelligent Networks and

Intelligent Systems, 2009. ICINIS’09. Second International Conference on (pp.

390–392). IEEE.

Zhang, T., Lu, H., & Li, S. Z. (2009). Learning semantic scene models by object

classification and trajectory clustering. In Computer Vision and Pattern

Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 1940–1947). IEEE.

Zhao, T., Nevatia, R., & Wu, B. (2008). Segmentation and tracking of multiple humans

in crowded environments. Pattern Analysis and Machine Intelligence, IEEE

Transactions on, 30(7), 1198–1211.

98

Zheng, X., Zhao, Y., Li, N., & Wu, H. (2009, May). An automatic moving object

detection algorithm for video surveillance applications. In Embedded Software and

Systems, 2009. ICESS'09. International Conference on (pp. 541-543). IEEE.

Zhu, J., Lao, Y., & Zheng, Y. (2010). Object tracking in structured environments for

video surveillance applications. Circuits and Systems for Video Technology, IEEE

Transactions, 20(2), 223–235.

Zhu, X., Liu, J., Wang, J., Fu, W., & Lu, H. (2013). Weighted interaction force

estimation for abnormality detection in crowd scenes. Computer Vision–ACCV

2012, 507–518.

Zweng, A., & Kampel, M. (2010). Unexpected human behavior recognition in image

sequences using multiple features. In Pattern Recognition (ICPR), 2010 20th


classification of abnormal crowd behavior using...

Documents