Hierarchical Method for Foreground DetectionUsing Codebook Model

Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng HsuIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR

VIDEO TECHNOLOGY, VOL. 21, NO. 6, JUNE 2011

Outline

• Background Model Construction– Block-Based Background Subtraction– Pixel-Based Background Subtraction

• Hierarchical Foreground Detection• Background Models Updating with the

Short-Term Information Models• Experimental Results

Background Model Construction

• This method involves two types of codebooks(CBs), block-based and pixel-based CBs.

• The modeling of two CBs is similar to the former CB[14]

[14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.

Background Model Construction

Background Model Construction

• There are two different time intervals for training (xt).

• (1 ≤ t ≤ T) and (t > T) for training the background models and foreground detection.

• The updating algorithms are separated into two parts for different time zones.

The Features Used in Block-Based Background Subtraction

• A frame xt of size P x Q is divied into multiple non-overlapped blocks of size M x N.

• The former block truncation coding(BTC) reduce the frame into two means,high-mean and low-mean.

• In this paper ,we have four means to represent a frame, high-top mean (μht ), high-bottom mean (μhb), low-top mean (μlt ), and low-bottom mean (μlb).

The Features Used in Block-Based Background Subtraction

The Features Used in Block-Based Background Subtraction

• Each means have three colors(RGB),so each codebook have 12 dimensions.

Updating Block-Based Background Models (CBs) in the Training Phase

• a specific block can be represented as a vector Vb = {vb

t|1 ≤ t ≤ T }.

• A CB for a block can be represented as C = {ci|1 ≤ i ≤ L}, consisting of L codewords

• An additional weight wi is geared for indicating the importance of the ith codeword.

• Codebook size is (P/M)x(Q/N)

Updating Block-Based Background Models (CBs) in the Training Phase

Updating Block-Based Background Models (CBs) in the Training Phase

Updating Pixel-Based Background Models (CBs) in theTraining Phase

• The same as block-based method.• Codebook size is P x Q.• Each codebook is 3 dimensions (RGB)

Hierarchical Foreground Detection

• After the background models training as indicated before the time point T, the two CBs are applied to the proposed hierarchical foreground detection.

• The foreground is obtained by background subtraction.

Foreground Detection with the Block-Based CB

• the input vector (vbt) extracted from a block is

compared with the ith block-based codeword (ci) to determine whether a match is found

• When a vbt is classified as background, the

corresponding block is also used to update the pixel-based CB.

Foreground Detection with the Pixel-Based CB

• This subsection introduces how to classify a pixel in a block to foreground or background.

• The foregrounds are classified into one true foreground and two fake foregrounds (shadow and highlight).

Foreground Detection with the Pixel-Based CB

Foreground Detection with the Pixel-Based CB

Background Models Updating with theShort-Term Information Models

• an additional variable timeics is involved to

store the updated time for estimating whether the corresponding ith codeword (ci

s ) has been updated for a specific period or not.

• If the duration is longer than a predefined parameter Ds

delete, the corresponding cis is

simply a temporary foreground.

Background Models Updating with theShort-Term Information Models

• When cis , is favor to strong stationary (

wics ≥ Dadd), the short-term information model

can be considered as a part of the true background model.

• This additional value is employed for filtering out ci which meets the states of eventually moving as foregrounds with the predefined parameter Ddelete.

Experimental Results

• λ = 5 for block-based , λ = 6 for pixel-based, η = 0.7, θcolor = 3, β = 1.15 ,γ = 0.72

,Dupdate = 3, and α = 0.05, Dadd = 100, Ds

delete = 200, and Ddelete = 200

Experimental Results

• [9]MOG• [5]color model• [11][25] hierarchical MOG• [14]CB

[9] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 1999, pp. 246–252.[5] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detection moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 10, pp. 1337–1342, Oct. 2003.[11] Y.-T. Chen, C.-S. Chen, C.-R. Huang, and Y.-P. Hung, “Efficient hierarchical method for background subtraction,” Pattern Recognit., vol. 40, no. 10, pp. 2706–2715, Oct. 2007.[25] C.-C. Chiu, M.-Y. Ku, and L.-W. Liang, “A robust object segmentation system using a probability-based background extraction algorithm,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518–528, Apr. 2010.[14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.

• C)MOGd)Color modele)CBf)g) hierarchical MOG

C)MOGd)Color modele)CBf)g) hierarchical MOG

C)MOGd)Color modele)CBf)g) hierarchical MOG

Experimental Results

Experimental Results

Experimental Results

Conclusion

• The block-based stage can enjoy high speed processing speed and detect most of the foreground without reducing TP rate.

• Pixel-based stage can further improve the precision of the detected foreground object with reducing FP rate.

• Short-term information is employed to improve background updating