Multimed Tools ApplDOI 10.1007/s11042-010-0722-9

Ice hockey shooting event modeling with mixturehidden Markov model

Xiaofeng Wang · Xiao-Ping Zhang

© Springer Science+Business Media, LLC 2011

Abstract In this paper we present a new event analysis framework based on mixturehidden Markov model (HMM) for ice hockey videos. Hockey is a competitive sportand hockey videos are hard to analyze because of the homogeneity of its framefeatures. However, the temporal dynamics of hockey videos is highly structured.Using the mixture representation of local observations and Markov chain property ofhockey event structure, we successfully model the hockey event as a mixture HMM.Based on the mixture HMM, the hockey event could be classified with high accu-racy. Two types of mixture HMMs, Gaussian mixture and independent componentanalysis (ICA) mixture, are compared for the hockey video event classification. Theresults confirm our analysis that the mixture HMM is a suitable model to deal withvideos with intensive activities. The new mixture HMM hockey event model couldbe a very useful tool for hockey game analysis.

Keywords Event analysis · Mixture hidden Markov model · Finite state machine ·Independent component analysis · Video content analysis · Hockey video analysis

1 Introduction

In the past several years, the field of multimedia content and computer vision analysisis dominated by content-based image and video indexing and retrieval systems.

Parts of this article also appeared in the Proc. of the 1st ACM International Workshopon Events in Multimedia (EiMM09) in conjunction with ACM Multimedia 2009,October 19–24, 2009, Beijing, China.

X. Wang · X.-P. Zhang (B)Department of Electrical and Computer Engineering, Ryerson University,Toronto, ON, Canadae-mail: xzhang@ee.ryerson.ca

Multimed Tools Appl

The goal is to replace the traditional keyword-based system, which has the drawbackof being subjective and inefficient. But most systems have limited performanceby using only low level features such as color, texture, shape and motion. Thefundamental reason is that there is a huge semantic gap between low level featuresand high level semantic meanings. An efficient and straightforward way to narrowthe semantic gap is to better understand the image and video content to constructan appropriate model. Videos have rich structure and information that could beexplored for the usage of indexing and retrieval. The task of video content analysisand event modeling is to find meaningful structure and patterns from visual data forthe purpose of efficient classification and mining of videos. Many algorithms havebeen developed to address the problem. Video event analysis task includes videoparsing, content indexing and content abstraction and representation. Video parsingis to segment a video to different level of segments. Early work focused on low levelparsing, i.e., the video shot boundary detection [8, 30]. After the shots are segmented,the next task is to classify these shots into different categories. It is to label the videoshots and give meaningful annotations to these shots. The video event classificationis to classify shots to different events. The event detection in sport video [7, 9] is apopular research topic in recent years and could be addressed by using non-structuralmodels such as support vector machine [21] and structural models such as the hiddenMarkov model (HMM) [7]. The structural model has the advantage of including thevideo temporal dynamics in the modeling process.

The HMM [14, 20] is a widely used structural model in many video analysisalgorithms [1, 2, 4–6, 10–13, 17–19, 22–29]. In [22], a 2D HMM framework isproposed to model the temporal patterns in video for sports classification problem.Keyword sequence based highlight detection using HMMs is discussed in [23]. In[25] unsupervised classification based on color ratio and motion in soccer domain isdiscussed and the observation model is Gaussian mixture. In [18] the audio featuressuch as applause and cheering are modeled using HMMs. In [2], baseball highlightsare modeled by HMMs using various kinds of features. It is extended to maximumentropy model in [6]. HMMs are also used in other sports domains such as tennis[12] and volleyball [28]. The hierarchical HMM presented in [19, 25] is complex be-cause of its tree-like structure, computation burden in implementation, and domaindependence on the feature space and pre-defined events. In [32], based on the non-Gaussian property of visual features, the independent component analysis (ICA)mixture [15] observation model is applied in HMMs for golf video event detection.Though HMMs are widely used in sports such as soccer and baseball, to our bestknowledge, the HMM has not been used in the ice hockey video applications.

In this paper, we present a mixture HMM framework for event modeling in icehockey domain. In [16], finite state machine (FSM) is used to represent the hockeyplay for game animation. However, FSM has not been used for hockey event analysisas far as we know. The HMM is one of the most common types of probabilistic FSMin practice. It is a transducer-style FSM in which the outputs and the transitionsare modeled by a random process. Encouraged by the animation of hockey usingFSM, we apply the HMM to the event analysis of hockey videos. Detecting hockeyshooting event is very useful because it is a rough indicator of a team’s performance.The hockey video is seldom used in event analysis. A possible reason is its intensiveactivity and homogenous frame color. The hockey is a competitive sport with players

Multimed Tools Appl

skating at a speed that could reach 32 km/h. The speed increases the importance ofthe individual frame for the video analysis, preventing key frame based algorithms tobe used. Frames in hockey video are very similar to each other. The backgroundis all white with players exchanging position frequently. We apply the mixtureHMM to model hockey shooting and non-shooting events. The mixture could catchthe distribution of local observations, and the state transition in the HMM couldcapture the view changing in hockey scene. The combination of the mixture andthe HMM increases the accuracy of hockey event modeling. We apply the mixtureHMM model to professional hockey game videos to classify hockey video shots tothree classes, hockey shooting, hockey non-shooting and other irrelevant events. Theresults confirm our analysis that the mixture HMM model is suitable for hockey eventmodeling, and that the ICA mixture HMM is better than the Gaussian mixture HMMfor hockey event modeling.

The main contributions of this paper are as follows.

– Although the HMM has been applied to sports video content analysis in manyliteratures, the mixture HMM has not been formulated in detail for supervisedsport video content analysis. This paper formulates the ICA mixture HMM andthe Gaussian mixture HMM in a single mixture HMM framework for hockeyvideo analysis.

– The comparison between the Gaussian mixture HMM and the ICA mixtureHMM is studied in this paper.

– Hockey videos are less studied in sports event analysis. We apply the mixtureHMM to this kind of high activity video.

The paper is organized as follows. First, the HMM formulation is given for videoevent analysis in Section 2. Second, we present the new mixture HMM frameworkfor hockey video analysis in Section 3. Third, the results are shown to provethe effectiveness of the new framework with two kinds of mixtures in Section 4.Finally, we conclude the paper with summaries and possible future research topicsin Section 5.

2 The mixture HMM formulation for video event analysis

The HMM is an extension of Markov chain which finds applications in video eventanalysis. For example in the soccer event detection, there are several camerasworking together, but at certain time period only one is used for broadcasting. Theselection of the cameras is hidden to event analysis. The cameras could be at hiddenstates in event modeling. It makes less sense to find out all hidden states of the videobecause it is troublesome to label all states and the problem could be solved withoutknowing the hidden states. The HMM is a sequential model, which has following fiveimportant components.

– The state space H is usually countable. There are N states in our formulation.The state at time t is ht ∈ H, which is a hidden state in video analysis. Hiddenstates are usually chosen to carry physical meanings. For example, in the soccerevent detection, the states could be different camera views.

Multimed Tools Appl

– The output observation set X. The element xt ∈ X is a vector which includesobservation features of frame at time t in video analysis.

– A state transition matrix P = {pij}, where pij = P( j|i) is the probability oftransition from the state i to the state j, i, j ∈ H.

– An observation distribution Q = {qhtxt }, where the probability of the observationgiven the state at time t is qhtxt = P(xt|ht), xt ∈ X, ht ∈ H.

– An initial state distribution π = (π1, π2, · · · , πN), where πi is the probability ofthe state i at the initial time t = 1, i = 1, 2, . . . , N.

Given the elements of the model as above, the probability of observing followingsequence x = [x1, x2, · · · , xt, · · · , xT ] of a HMM model is

P(x) = P(x1, x2, · · · , xt, · · · , xT)

=∑

h1,...,hT

πh1 P(x1|h1)

T∏

t=2

P(ht|ht−1)P(xt|ht)

=∑

h1,...,hT

πh1 qh1x1

T∏

t=2

pht−1ht qhtxt . (1)

Mixture model is often used in sport event analysis in observation model qhtxt . Theparameters of the model λ = (P, Q, π) could be learned from a training event usingthe expectation maximization (EM) algorithm. Given a new input video event framesequence, the probability of it belonging to a model with parameters learned intraining could be computed with forward and backward algorithms [20].

3 A new mixture HMM framework for hockey event analysis

3.1 Hockey shooting events

Ice hockey is a competitive sport usually played on a hockey rink in an arena. Thereare altogether six players including one goaltender per side in a normal play session.Most of time players are skating at a very high speed that could reach as high as32 km/h. The objective of the game is to score goals by shooting a hard rubber disccalled the puck into the opponent’s goal net. The shooting is made by a long stickwith a blade at one end. We could divide a hockey video into three kinds of events,the shooting event, the non-shooting hockey event and the commercials. A hockeyshooting is defined as a scoring attempt, which is directed on the opponent’s goalnet, resulting in a goal or requiring the goaltender to make a save. Because a countof how many shootings are taken by a team is often used as a rough guide to whichteam is being more aggressive and dominant, the automatic detection of hockeyshooting events is a very useful application for coaching and analyzing the game.Because of the similarity in background color, the traditional non-structural modelswithout temporal graph information could not have the same kind of performanceas structural models such as HMM [7]. We use the views as hidden states in HMM.The hockey video events usually contain two kinds of views, global view and close-up view, which could be used as states in our event modeling as shown in Fig. 1. Thetwo views are reflected in features as shown in an example features of one hockey

Multimed Tools Appl

Fig. 1 The state transition illustration of hockey event analysis and example state images

shooting event, Fig. 2. The global views show the global situation of the play. Theclose-up views are the zoom-in views of certain activity such as goal and players.Though the states are hidden, they carry clear physical meanings. The hidden statesdo not need to be found explicitly. They are used implicitly in HMM training andtesting process.

3.2 Hockey event modeling and detection with mixture HMM

As discussed in Section 2, the mixture HMM could be applied to sport event analysis.We develop a new framework of hockey shooting analysis as shown in Fig. 3. First,the video features are extracted for each frame of hockey video event. The featurescould be further processed by principal component analysis (PCA) or ICA to reducethe dimension. The difference between hockey and other sports event detection task

Fig. 2 The feature illustrationfeatures of two states of ahockey video shooting. Thetwo axes are two features. Onepoint denotes a frame in shot

1 1.5 2 2.5 3 3.5 4 4.5−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

IC1

IC2

Multimed Tools Appl

Fig. 3 The new mixture HMMbased hockey video eventanalysis procedure

is the features of each individual frame are used since the speed of players andcameras following them. Second, the parameters of the HMM and mixtures of eachkind of event are learned from training videos. Typical events in this stage shouldbe selected to have desire parameters. Third, the probabilities of all events in thevideo belonging to the three models are computed. The highest likelihood score ofthe three are the most likely class for the test event.

3.3 Mixture HMMs and parameter learning

A mixture HMM is an HMM with a mixture observation model. The observationdistribution Q = {qht xt } needs to be determined before the model could be used inice hockey. Two specific mixtures are tested and applied in our new framework, theGaussian mixture and the ICA mixture. The mixture model is

qhtxt = P(xt|ht = j)

=K∑

k=1

P(xt|C jk; θ jk)P(C jk), (2)

where C jk indicates the k-th mixture component of the hidden state j, P(C jk) is themixture coefficients and P(xt|C jk; θ jk) is the probability of xt for the hidden state ht =j and component C jk. The parameters of P(xt|C jk; θ jk) are denoted by θ jk. Note thatwe suppose the hidden state ht = j to be consistent with conventional formulationwhich will not affect the calculation.

Multimed Tools Appl

There are two kinds of mixture distributions P(xt|C jk; θ jk) as follows.

– For Gaussian mixture, the probability P(xt|C jk; θ jk) follows multivariateGaussian N (xt|μ jk,� jk) distribution,

P(xt|C jk; θ jk) = 1

(2π)D/2�1/2jk

· exp{−1

2(xt − μ jk)

′�−1jk (xt − μ jk)}. (3)

Here the parameters are mean and covariance θ jk = [μ jk,� jk] and D is thedimension of the observation xt.

– In ICA mixture the observation xt could be expressed as follows,

xt = M jks jk + μ jk xt ∈ Ck, (4)

where M jk is mixing matrix, μ jk is the bias and s jk is independent sources fork-th component of the mixture. The probability of observation xt could beexpressed as

P(xt|C jk; θ jk) = exp(log P(s jk) − log(det {M jk})). (5)

Here the parameters θ jk include the bias μ jk, the mixing matrix M jk and theparameters of independent component P(s jk). det{·} denotes the determinantof a matrix. The number of ICA mixture parameters depends on the choice ofindependent component distribution.

In the mixture HMM model, the parameters of both the mixture θ jk, P(C jk) andHMM λ could be learned using EM algorithm iteratively [20, 32]. The forwardvariable αt(i) and backward variable βt(i) are defined as follows,

αt(i) = P(x1, · · · , xt, ht = i|λ), (6)

βt(i) = P(xt+1, · · · , xT , |ht = i,λ). (7)

Based on the variable αt(i) and βt(i), three probabilities of joint probabilities, used inthe iterative calculation processes, are defined as

ξt(i, j) = αt(i)pijβt+1( j)q jxt+1∑Nj=1 αt( j)βt( j)

, (8)

γt(i) = αt(i)βt(i)∑Nj=1 αt( j)βt( j)

, (9)

γt( j, k) = γt( j) · P(xt|C jk; θ jk)P(C jk)∑Kk=1 P(xt|C jk; θ jk)P(C jk)

. (10)

The formula (8) defines the conditional probability of state transition fromstate i at time t to state j at time t + 1 given the observation sequences x =[x1, x2, . . . , xt, . . . , xT ] and the parameters of the model λ. γt(i) is defined as theconditional probability of being in state i at time t given observations x and para-meters λ [20]. γt( j, k) is the generation of γt( j) for non-Gaussian mixture observation

Multimed Tools Appl

model [32] and denotes the conditional probability of being in state j at time t giventhe k-th mixture component accounting for xt.

The re-estimation formulae for the parameters are shown in Table 1. Otherparameters of ICA mixture could be re-estimated using the standard ICA mixture al-gorithm presented in [15]. During the training process, the parameters λ, θ jk, P(C jk)

are learned for each event.In the hockey case, there are three sets of parameters corresponding to shooting,

non-shooting and other events. Based on these parameters the probability of eachinput sequence belonging to certain event could be calculated using standard forwardprocedures as follows,

– Initialization:

α1(i) = πiqix1; (11)

– Induction:

αt+1( j) =[

N∑

i=1

αt(i)pij

]q jxt+1; (12)

– The probability:

P(x) =N∑

i=1

αT(i). (13)

The sequence with the highest probability is the event that the input sequence mostlikely belongs to.

Table 1 The re-estimation of parameters of Gaussian and ICA mixture HMM

Gaussian mixture HMM ICA mixture HMM

πi∗ = γ1(i) πi

∗ = γ1(i)

pij∗ =

∑T−1t=1 ξt(i, j)

∑T−1t=1 γt(i)

pij∗ =

∑T−1t=1 ξt(i, j)

∑T−1t=1 γt(i)

P(C jk)∗ =∑T

t=1 γt( j, k)∑T

t=1∑K

k=1 γt( j, k)P(C jk)∗ =

∑Tt=1 γt( j, k)

∑Tt=1

∑Kk=1 γt( j, k)

μ∗jk =

∑Tt=1 γt( j, k)xt∑T

t=1 γt( j, k)μ∗

jk =∑T

t=1 γt( j, k)xt∑Tt=1 γt( j, k)

�∗jk =

∑Tt=1 γt( j, k)(xt − μ jk)(xt − μ jk)t

∑Tt=1 γt( j, k)

�M jk ∝ P(xt|C jk; θ jk)P(C jk)∑K

k=1 P(xt|C jk; θ jk)P(C jk)

∂

∂ M jk

log P(xt|C jk; θ jk)

The superscript ∗ denotes that the variable should be estimated iteratively

Multimed Tools Appl

4 Experimental results

Mixture HMMs are applied to ice hockey video content analysis tasks. Two kindsof mixture methods, Gaussian mixture and ICA mixture are compared by theclassification accuracy of ice hockey event. Note that choosing the number of mixturecomponents of mixture HMM is non-trivial. However it could be optimized usingtraining or validation sets.

4.1 Event detection in ice hockey video

A 30-min professional ice hockey TV program is used in this study. The hockey videois first segmented to shots using independent component analysis (ICA) as in [31,32]. The 256 bin illumination-invariant color histogram features [3] are extracted foreach frame. These features are projected into a compact two-dimensional featurevector using ICA. The cuts and gradual transitions detection is performed on thisICA subspace using an iterative clustering algorithm based on adaptive thresholdingas in [31]. Second, the compact two-dimensional feature vectors are used as originalfeatures in HMM. The ice hockey video is divided into 235 video shots. Three events:ice hockey shootings, ice hockey non-shootings and other irrelevant events. The icehockey shooting is usually a sequence of frames, which features following activities:the player catching the puck, the player making a shot by hitting the puck toward thenet, the goaltender trying to catch the puck, the camera focusing on the goaltender orgoal net and then the global situation of the arena. Irrelevant event sequences includecommercial advertisements, the scene of audience, etc. Example event sequencesare shown in Fig. 4. There are 28 hockey shooting events, 162 hockey non-shootingevents and 45 irrelevant events in the selected video.

(c) Event 3: other event

(b) Event 2: a hockey non shot event

(a) Event 1: a hockey shot event

Fig. 4 An example for three ice hockey events. a hockey shooting; b hockey non-shooting; c otherevent

Multimed Tools Appl

−0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.15−0.5

0

0.5

1

1.5

2

IC1

IC2

0 50 100 1501

1.5

2

2.5

−0.25 −0.2 −0.15 −0.1 −0.05−0.6

−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

IC1

IC2

0 50 100 150 200 2501

1.5

2

2.5

0.8 1 1.2 1.4 1.6 1.8 2−4

−3

−2

−1

0

1

2

IC1

IC2

0 20 40 60 80 100 120 1401

1.5

2

2.5

Fig. 5 Left column: the ICA mixture components for three events (upper: ice hockey shooting event;middle: ice hockey non-shooting event; lower: other event). The axes are two features of compactfeature space. Right column: the HMM state sequence for three events (upper: ice hockey shootingevent; middle: ice hockey non-shooting event; lower: other event). There are two hidden states: globalview and close-up view

Multimed Tools Appl

Two hidden states, global view and close-up view, are used in this study. Assumingthe independent sources of all components of the mixture are Laplacian distributedin the ICA mixture model, the mixture parameters and HMM parameters are learnedfrom one training shot of each event. The training data is one shot from each typeand the testing data are all the remaining shots. Then for each testing video shot, thelikelihood probabilities of three events are calculated using estimated ICA mixtureand HMM parameters, and used for the event classification.

The mixture components and their state transitions of three example events areshown in the left and right column of Fig. 5, respectively. The mixture componentsshown in Fig. 5 provide the possible feature distribution of the three categoriesof events: ice hockey shooting (upper), ice hockey non-shooting (middle), andirrelevant (lower), in the ICA subspace. The feature similarity of hockey shootingand non-shooting events can be observed from Fig. 5. Without graph structureincluded in the model, it is difficult to discriminate two events. One solution is toinclude temporal dynamic information, which can be captured by the HMM.

The state transitions of example event sequences shown in the right column ofFig. 5 represent the view changing properties of these three categories of events. Thehockey shooting event features following states, a global view of the activity that ahockey player shots the puck, a close-up view of the puck and the goal tender andthen a global view of the players. The non-shooting events, with cameras focusingon global situation of the game and specific events or players alternatively, havemore frequent view transitions than the shooting events. The state transitions of thecommercial events, as shown by the lower-right plot in Fig. 5, indicates a randomview changing property that is not found in hockey shooting and non-shootingevents.

The confusion matrices of these three events, using the ICA mixture HMM(shown in bold) and Gaussian mixture HMM (shown in parenthesis), are given inTable 2. The rows are the true classes and the columns are detected classes. It can beobserved that many non-shooting events are classified to shooting events mistakenly,and some shooting events are classified to non-shooting events mistakenly. This maybe caused by the feature similarity of these two events as observed from Fig. 5. Itmight be improved by applying more features, such as the motion vector and othermodalities (audio, caption, etc.).

The ice hockey event detection accuracy of the ICA mixture HMM and theGaussian mixture HMM are compared in Table 3. The accuracy is defined as theratio between the number of correctly labeled events and the total number of events.

Table 2 The confusion matrix for hockey event classification using the ICA mixture HMM (in boldfonts) and the Gaussian mixture HMM (in parenthesis)

Shooting events Non-shooting events Others

Shooting events 22 (20) 6 (8) 0 (0)Non-shooting events 45 (57) 117 (104) 0 (1)Others 5 (2) 8 (28) 32 (15)

The rows are the true classes and the columns are detected classes

Multimed Tools Appl

Table 3 The accuracy of icehockey event classification

Method Accuracy (%)

ICA mixture HMM 73.01Gaussian mixture HMM 59.15

The performance of the ICA mixture HMM is about 14% better than the Gaussianmixture HMM.

In both mixture HMM models, the hidden Markov modeling exhibits reasonableaccuracy for hockey shooting event modeling. There are several reasons. First, thelocal observations of the ice hockey frames could be described by mixture models.With fewer mixture components, the ICA mixture HMM is better than the Gaussianmixture for local observation modeling. Second, the temporal view changing processcan be modeled as a hidden Markov process. The two view types are the hiddenstates. The feature distribution is characterized by a mixture model and the chaintemporal information is captured by HMM. The new framework combines thegood properties of the two and shows good performance in ice hockey eventdetection.

5 Conclusion

An event analysis framework based on mixture HMMs is presented for ice hockeyshooting event detection. Hockey domain knowledge is applied to find a suitablehockey event model. According to the view changing properties of hockey shootingthat can be modeled by FSMs, we apply mixture HMMs with hidden view states tomodel hockey shooting events. We compare two kinds of mixture HMMs, Gaussianmixture HMMs and ICA mixture HMMs. The experimental results show that theICA mixture HMM performs better than the Gaussian mixture HMM for hockeyevent detection. The results confirm our analysis that mixture HMM is a suitablehockey shooting event detection model. The presented new mixture HMMs basedevent detection method is a generic framework, which can be applied to contentanalysis of other competitive sports. Future work includes investigation of otherrelevant features, e.g., audio features, multi-modality HMMs, and finer classificationfor hockey events and other types of sports. Since many types of video eventshave similar FSM structure, the new framework may also find applications in videocontent analysis tasks other than sports.

References

1. Baillie M, Jose J, van Rijsbergen C (2004) HMM model selection issues for soccer video. In: Proc.int. conf. content-based image and video retrieval, pp 70–78

2. Chang P, Han M, Gong Y (2002) Extract highlights from baseball game video with hiddenMarkov models. In: Proc. int. conf. image processing, vol 1, pp 609–612

3. Drew M, Wei J, Li Z (1998) Illumination-invariant color object recognition via compressedchromaticity histograms of color-channel-normalize images. In: Proc. int’l. conf. computer vision,pp 533–540

4. Ekin A, Tekalp AM, Mehrotra R (2003) Automatic soccer video analysis and summarization.IEEE Trans Image Process 12(7):796–807

Multimed Tools Appl

5. Gibert X, Li H, Doermann D (2003) Sports video classification using HMM. In: Proc. int. conf.on multimedia and expo, vol 2, pp 345–348

6. Gong Y, Han M, Hua W, Xu W (2004) Maximum entropy model-based baseball highlightdetection and classification. Comput Vis Image Underst 96(2):181–199

7. Gong Y, Xu W (2007) Machine learning for multimedia content analysis. Springer, Berlin8. Hanjalic A (2002) Shot-boundary detection: unraveled and resolved. IEEE Trans Circuit Syst

Video Technol 12(2):90–1049. Hanjalic A (2004) Content-based analysis of digital video. Kluwer, Norwell

10. Huang J, Liu Z, Wang Y (2000) Joint video scene segmentation and classification based onhidden Markov model. In: Proc. int. conf. on multimedia and expo. New York, pp 1551–1554

11. Jin G, Tao L, Xu G (2004) Hidden Markov model based events detection in soccer video. In:Proc. int. conf. on image analysis and recognition, pp I:605–I:612

12. Kijak E, Oisel L, Gros P (2003) Hierarchical structure analysis of sport videos using HMMs. In:Proc. int. conf. image processing, pp II:1025–II:1028

13. Kim H-G, Roeber S, Samour A, Sikora T (2004) Detection of goal events in soccer videos.In: Society of Photo-Optical Instrumentation Engineers (SPIE) conference series, vol 5682,pp 317–325

14. Lavee G, Rivlin E, Rudzsky M (2009) Understanding video events: A survey of methods forautomatic interpretation of semantic occurrences in video. IEEE Trans Syst Man Cybern C ApplRev 39(5):489–504

15. Lee T, Lewicki MS, Sejnowski TJ (2000) ICA mixture models for unsupervised classification andautomatic context switching. IEEE Trans Pattern Anal Mach Intell 22(10):1078–1089

16. Li F, Woodham R (2009) Video analysis of hockey play in selected game situations. J Image VisComput 27(1–2):45–58

17. Liu S, Xu M, Yi H, Chia L, Rajan D (2006) Multimodal semantic analysis and annotation forbasketball video. EURASIP J Appl Signal Process 2006:182–182

18. Liu Z, Huang J, Wang Y (1998) Classification of tv programs based on audio informationusing hidden Markov model. In: IEEE signal processing society workshop on multimedia signalprocessing, pp 27–32

19. Naphade M, Huang T (2002) Extracting semantics from audiovisual content: the final frontier inmultimedia retrieval. IEEE Trans Neural Netw 13(4):793–810

20. Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speechrecognition. Proc IEEE 77(2):257–286

21. Sadlier D, O’Connor N (2005) Event detection in field sports video using audio-visual fea-tures and a support vector machine. IEEE Trans Circuit Syst Video Technol 15(10):1225–1233

22. Wang J, Xu C, Chng E (2006) Automatic sports video genre classification using pseudo-2d-hmm.In: The 18th international conference on pattern recognition

23. Wang J, Xu C, Chng E, Tian Q (2004) Sports highlight detection from keyword sequences usinghmm. In: IEEE international conference on multimedia and expo

24. Wang Y, Liu Z, Huang J (2000) Multimedia content analysis using both audio and visual cues.IEEE Signal Process Mag 17(6):12–36

25. Xie L, Chang S (2004) Structure analysis of soccer video with hidden Markov models. PatternRecogn Lett 25(7):767–775

26. Xie L, Sundaram H, Campbell M (2008) Event mining in multimedia streams. Proc IEEE96(4):623–647

27. Xiong Z, Radhakrishnan R, Divakaran A, Huang TS (2003) Audio events detection basedhighlights extraction from baseball, golf and soccer games in a unified framework. In: Proc. int.conf. acoust., speech, signal processing, pp 401–404

28. Xu G, Ma Y, Zhang H, Yang S (2005) An HMM-based framework for video semantic analysis.IEEE Trans Circuit Syst Video Technol 15(11):1422–1433

29. Yang Y, Lin S, Zhang Y, Tang S (2007) Statistical framework for shot segmentation and clas-sification in sports video. In: Proc. Asian conference on computer vision, pp II:106–II:115

30. Zhang H, Kankanhalli A, Somaliar S (1993) Automatic partition of full-motion video. Multime-dia Syst 1(1):11–28

31. Zhou J, Zhang X (2005) Video shot boundary detection using independent component analysis.In: Proceedings. (ICASSP ’05). IEEE international conference on acoustics, speech, and signalprocessing, vol 2, pp 541–544

32. Zhou J, Zhang X (2008) An ICA mixture hidden Markov model for video content analysis. IEEETrans Circuit Syst Video Technol 18(11):1576–1586

Multimed Tools Appl

Xiaofeng Wang received the B.E. and M.E. degrees from the Faculty of Information Engineering,Beijing University of Posts and Telecommunications, Beijing, China, in 1998 and 2001, respectively.He is currently working toward the Ph.D. degree at Ryerson University, Toronto, Canada.

His research interests include multimedia data hiding, information forensic and multimedia datafusion and mining.

Xiao-Ping Zhang received the B.S. and Ph.D. degrees from Tsinghua University, in 1992 and 1996,respectively, all in electronic engineering. He holds an MBA in Finance and Economics with Honorsfrom the University of Chicago Graduate School of Business.

Since Fall 2000, he has been with the Department of Electrical and Computer Engineering,Ryerson University, where he is now Professor and Director of Communication and Signal Process-ing Applications Laboratory (CASPAL). Prior to joining Ryerson, from 1996 to 1998, he was apostdoctoral fellow at the University of Texas, San Antonio and then at the Beckman Institute,the University of Illinois at Urbana-Champaign. He held research and teaching positions at theCommunication Research Laboratory, McMaster University, in 1999. From 1999 to 2000, he was aSenior DSP Engineer at SAM Technology, Inc., at San Francisco, and a consultant at San FranciscoBrain Research Institute. His research interests include signal processing for communications,multimedia retrieval and video content analysis, computational intelligence, and applications inbioinformatics, finance and marketing. He is a frequent consultant for biotech companies andinvestment firms. He is the publicity co-chair for ICME’06 and program co-chair for ICIC’05.

Dr. Zhang received Science and Technology Progress Award by State Education Commissionof China, for his significant contribution in a National High-Tech Project, in 1994. He is a registeredProfessional Engineer in Ontario, Canada, a Senior Member of IEEE and a member of Beta GammaSigma Honor Society.