learning attentional policies for tracking and recognition

Learning Attentional Policies for Tracking and Recognition in Videowith Deep Networks

Loris Bazzani, Nando de Freitas [email protected], [email protected]

University of British Columbia, Canada

Hugo Larochelle [email protected]

University of Toronto, Canada

Vittorio Murino [email protected]

Istituto Italiano di Tecnologia, Italy

Jo-Anne Ting [email protected]

University of British Columbia, Canada

Abstract

We propose a novel attentional model for si-multaneous object tracking and recognitionthat is driven by gaze data. Motivated bytheories of the human perceptual system, themodel consists of two interacting pathways:ventral and dorsal. The ventral pathwaymodels object appearance and classificationusing deep (factored)-restricted Boltzmannmachines. At each point in time, the ob-servations consist of retinal images, with de-caying resolution toward the periphery of thegaze. The dorsal pathway models the loca-tion, orientation, scale and speed of the at-tended object. The posterior distribution ofthese states is estimated with particle filter-ing. Deeper in the dorsal pathway, we en-counter an attentional mechanism that learnsto control gazes so as to minimize trackinguncertainty. The approach is modular (witheach module easily replaceable with more so-phisticated algorithms), straightforward toimplement, practically efficient, and workswell in simple video sequences.

1. Introduction

Humans track and recognize objects effortlessly and ef-ficiently, exploiting attentional mechanisms (Rensink,

Appearing in Proceedings of the 28 th International Con-ference on Machine Learning, Bellevue, WA, USA, 2011.Copyright 2011 by the author(s)/owner(s).

2000; Colombo, 2001) to cope with a vast stream ofdata. In this paper, we use the human visual systemas inspiration to build a model for simultaneous ob-ject tracking and recognition from gaze data, as shownin Figure 1. The proposed model also addresses theproblem of gaze planning (i.e., where to look in orderto achieve some goal, such as minimizing position orspeed uncertainty).

The model consists of two interacting modules, ven-tral and dorsal, which are also known as the what andwhere modules respectively. The dorsal pathway is incharge of state estimation and control. At the lowestlevel of the dorsal pathway, a particle filter (Doucetet al., 2001) is used to estimate the states of the ob-ject under consideration, including location, orienta-tion, speed and scale. We make no attempt to im-plement such states with neural architectures, but itseems clear that they could be encoded with grid cells(McNaughton et al., 2006) and retinotopic maps as inV1 and the superior colliculus (Rosa, 2002; Girard &Berthoz, 2005). At the higher level of the dorsal path-way, a policy governing where to gaze next is learnedwith an online hedging algorithm (Auer et al., 1998).This policy learning step could be easily improved us-ing other bandit approaches and Bayesian optimiza-tion (Brochu et al., 2009; Cesa-Bianchi & Lugosi, 2006;Chaudhuri et al., 2009). The dorsal attentional mech-anism is responsible for controlling saccades and, to asignificant extent, smooth pursuit (Colombo, 2001).

The ventral pathway consists of a two hidden layerdeep network. The second layer corresponds to amulti-fixation RBM (Larochelle & Hinton, 2010), as

Learning Attentional Policies for Tracking and Recognition

z t+2

a

b

x

vt

vt+1

xt+1

bt+1

t+1

a t+1

t

t

t

t

v

x

b

t+2

t+2

t+2

t+2

t+2a

πPolicy

R R R

Belief state

Reward

Actions

Gaze observation

States

Video frames

ct+2

ht+2

2

z zh th

t+1h

t+2

Object class

First hidden layers

Second hidden layer

t t+1

Figure 1. From a sequence of gazes (vt,vt+1,vt+2), themodel infers the hidden features h for each gaze (that is,the activation intensity of each receptive field), the hiddenfeatures for the fusion of the sequence of gazes and the ob-ject class c. Only one time step of classification is kept inthe figure for clarity. zt indicates the relative position ofthe gaze in a template. The location, size, speed and ori-entation of the gaze patch are encoded in the state xt. Theactions at follow a randomized policy πt that depends onthe cumulative reward Rt−1. This particular reward is afunction of the belief state bt = p(xt|a1:t,h1:t), also knownas the filtering distribution. Unlike typical commonly usedpartially observed Markov decision models (POMDPs), thereward is a function of the beliefs. In this sense, the prob-lem is closer to one of sequential experimental design. Withmore layers in the ventral v − h − h2 − c pathway, otherrewards and policies could be designed to implement higher-level attentional strategies.

shown in Figure 1. It accumulates information fromthe first hidden layers at consecutive time steps. Forthe first layers, we use (factored)-restricted Boltzmannmachines (RBMs) (Hinton & Salakhutdinov, 2006;Ranzato & Hinton, 2010; Welling et al., 2005), butautoencoders (Vincent et al., 2008), sparse coding (Ol-shausen & Field, 1996; Kavukcuoglu et al., 2009), two-layer ICA (Koster & Hyvarinen, 2007) and convolu-tional architectures (Lee et al., 2009) could also beadopted in this module. At present, we pre-train theseappearance models.

The proposed system can be motivated from differ-ent perspectives. First, starting with (Isard & Blake,1996), many particle filters have been proposed for im-age tracking, but these typically use simple observa-tion models such as B-splines (Isard & Blake, 1996)and color templates (Okuma et al., 2004). RBMs aremore expressive models of shape, and hence, we con-jecture that they will play a useful role where simple

appearance models fail. Second, from a deep learn-ing computational perspective, this work allows us totackle large images and video. The use of fixationssynchronized with information about the state (e.g.location and scale) of such fixations, eliminates theneed to look at the entire image or video. Third,the system is invariant to image transformations en-coded in the state, such as location, scale and orien-tation. Fourth, from a dynamic sensor network per-spective, this paper presents a very simple, but effi-cient, novel way of deciding how to gather measure-ments dynamically. Lastly, in the context of psychol-ogy, the proposed model realizes to some extent thefunctional architecture for dynamic scene representa-tion of (Rensink, 2000). The rate at which differentattentional mechanisms develop in newborns (includ-ing alertness, saccades and smooth pursuit, attentionto object features and high-level task driven attention)guided the design of the proposed approach and was agreat source of inspiration (Colombo, 2001).

Recently, a dynamic RBM state-space model was pro-posed in (Taylor et al., 2010). Both the implementa-tion and intention behind that proposal are differentfrom the approach discussed here. To the best of ourknowledge, the approach presented here is the first suc-cessful attempt to combine dynamic state estimationfrom gazes with online policy learning for gaze adap-tation, using deep belief network models of appear-ance. Many other dual-pathway architectures havebeen proposed in computational neuroscience, includ-ing (Olshausen et al., 1993; Postma et al., 1997), butwe believe ours has the advantage that it is very sim-ple, modular (with each module easily replaceable),suitable for large datasets and easy to extend.

2. Model specification

We describe the dorsal pathway in Sections 2.1 (state-space representation), 2.2 (reward function and controlpolicy) and 2.3 (observation model for the state-spacemodel). The corresponding state estimation and con-trol algorithms are presented in Section 3. The ventralpathway is described briefly in Section 2.3.

2.1. State-space model

The standard approach to image tracking is basedon the formulation of Markovian, nonlinear, non-Gaussian state-space models, which are solved withapproximate Bayesian filtering techniques. In this set-ting, the unobserved signal (object’s position, velocity,scale, orientation or discrete set of operation) is de-noted {xt ∈ X ; t ∈ N}. This signal has initial distribu-tion p (x0) and transition equation p (xt|xt−1,at−1) .


Here at ∈ A denotes an action, at time t, defined ona discrete state space of size |A| = K. The observa-tions {ht ∈ H; t ∈ N∗}, are assumed to be condition-ally independent given the state process {xt; t ∈ N}.(Note that from the state space model perspective theobservations are the hidden units of the first layer ofthe ventral path model.) In summary, the state-spacemodel is described by the following distributions:

p (x0)

p (xt|xt−1,at−1) for t ≥ 1

p (ht|xt,at) for t ≥ 1,

where x0:t , {x0, ...,xt} and h1:t , {h1, ...,ht} rep-resent the states and the observations up to time t,respectively. For the transition model, we will adopta classical autoregressive process. The appearancemodel p (ht|xt,at) is slightly more involved and willbe discussed in Section 2.3.

Our aim is to estimate recursively in time the poste-rior distribution p (x0:t|h1:t,a1:t) and its associatedfeatures, including the marginal distribution bt ,p (xt|h1:t,a1:t) — known as the filtering distributionor belief state. This distribution satisfies the followingrecurrence:

bt∝ p(ht|xt,at)∫p(xt|xt−1,at−1)p(dxt−1|h1:t−1,a1:t−1).

Except for standard distributions (e.g. Gaussian ordiscrete), this recurrence is intractable.

2.2. Reward function and policy

To complete the specification of the model, we needto introduce the policy π(·) and an instantaneous re-ward function rt(·). The reward can be any desiredbehavior for the system, such as minimizing posterioruncertainty or achieving a more abstract goal. We fo-cus on gathering observations so as to minimize theuncertainty in the estimate of the filtering distribu-tion: rt(bt) , u[p(xt|h1:t,a1:t)]. More specifically, asdiscussed later, this reward will correspond to the vari-ance of the importance weights of the particle filter ap-proximation p(xt|h1:t,a1:t) of the belief state. In ourcurrent implementation, each action is a different gazelocation. The objective is to choose where to look soas to minimize the uncertainty about the belief state.

We also need to introduce the cumulative reward ofthe control algorithm for each action:

RT (aT = k) =

T∑t=1

rt(p(xt|h1:t,at = k,a1:t−1)).

The actions are distributed according to the followingstochastic policy:

πt(at = k|Rt−1) =exp(ηRt−1(at = k))∑Kj=1 exp(ηRt−1(at = j))

,

where η > 0 is a parameter. We have defined thepolicy in this way as it enables us to borrow decisionmaking algorithms from the online learning framework(Auer et al., 1998) with very little effort. Here, we willadopt the hedge algorithm for full information gamesdescribed in (Auer et al., 1998). Although this algo-rithm works well and has vanishing regret, as men-tioned in the introduction, one could adopt other ban-dit techniques (Cesa-Bianchi & Lugosi, 2006; Chaud-huri et al., 2009) or Bayesian optimization (Brochuet al., 2009) to extend the policy to continuous actionspaces and treat imperfect information games (onlyone gaze allowed at each time step).

2.3. Appearance model

We use (factored)-RBMs to model the appearance ofobjects and perform object classification using thegazes chosen by the control module. These undirectedprobabilistic graphical models are governed by a Boltz-mann distribution over the gaze data vt and the hid-den features ht ∈ {0, 1}nh . We assume that the recep-tive fields w, also known as RBM weights or filters,have been trained beforehand. We also assume thatreaders are familiar with these models and, if other-wise, refer them to (Ranzato & Hinton, 2010; Swerskyet al., 2010).

In image tracking, the observation model is often de-fined in terms of the distance of the observations withrespect to a template τ ,

p (ht|xt,at) ∝ e−d(h(xt,at),τ),

where d(·, ·) denotes a distance metric and τ an objecttemplate (for example, a color histogram or spline).In this model, the observation h(xt,at) is a functionof the current state hypothesis and the selected ac-tion. The problem with this approach is eliciting agood template. Often color histograms or splines areinsufficient. For this reason, we will construct the tem-plates with (factored)-RBMs as follows. First, opticalflow is used to detect new object candidates enteringthe visual scene. Second, we assign a template to thedetected object candidate, which consists of severalgazes covering the field of motion, as shown in Fig-ure 2 for K = 9 gazes. The same figure also showsthe typical foveated observations (higher resolution inthe center and lower in the periphery of the gaze) andthe receptive fields for these observations learned be-forehand with an RBM. The control algorithm will be


Figure 2. (a) Template with 9 gazes initialized automati-cally when motion is detected. (b) Foveal observation cor-responding to gaze G5 in the template. (c) The most activeRBM filters for this observation.

used to learn which of the gazes in the template aremore fruitful. That is, each action will correspond tothe selection of one of these gaze options. Finally, wedefine the likelihood of each observation directly interms of the distance of the hidden units of the RBMh(xt,at,vt) to the hidden units of each template re-gion h(x1,at = k,v1), k = 1 : K, initialized in thefirst frame. That is,

p (ht|xt,at = k) ∝ e−d(h(xt,at=k,vt),h(x1,at=k,v1)).

The above template is static, but conceivably onecould adapt it over time.

The appearance module also performs object recogni-tion, classifying a sequence of gaze instances selectedwith the gaze policy. We implement a multi-fixationmodel very similar to the one proposed in (Larochelle& Hinton, 2010), where the binary variables zt (seeFigure 1) are introduced to encode the relative gaze lo-cation, at in the present implementation, in a factored-RBM. We refer the reader to this citation for detailedinformation on this part of the model. We experi-mented with a single fixation module, but found themulti-fixation module of (Larochelle & Hinton, 2010)to increase classification accuracy. To improve the es-timate the class variable ct over time, we accumulatethe classification decisions at each time step.

Note that the process of pursuit (tracking) is essen-tial to classification. As the target is tracked, the al-gorithm fixates at random locations near the target’slocation estimate. The size and orientation of thesefixations also depends on the corresponding state esti-mates. Note that we don’t fixate exactly at the targetlocation estimate as this would provide only one dis-tinct fixation over several time steps if the trackingpolicy has converged to a specific gaze. It should alsobe pointed out that instead of using random fixations,one could again use the control strategy proposed inthis paper to decide where to look with respect tothe track estimate so as to reduce classification un-certainty. We leave the implementation of this extra

1. Initialization, t = 0.

• For i = 1, ..., N , x(i)0 ∼ p (x0) , set t = 1, and initialize the

template.

• Set R0(a0 = k) = 0 for k = 1, . . . , K , where K is thenumber of gazes in the template

2. Importance sampling step

• For i = 1, ..., N , x(i)t ∼ qt

(dx

(i)t

∣∣∣ x(i)0:t−1,h1:t, a1:t

)and

set x(i)0:t =

(x(i)0:t−1, x

(i)t

).

• For i = 1, ..., N , k = 1, . . . , K, , evaluate the importanceweights

w(i),kt ∝

p(ht|x

(i)t , at = k

)p(x(i)t |x

(i)0:t−1, at−1

)qt

(x(i)t

∣∣∣ x(i)0:t−1,h1:t, a1:t

) .

• Normalise the importance weights w(i),kt =

w(i),kt∑N

j=1w

(j),kt

3. Gaze control step

• Update the policy

πt(at = k|Rt−1) =exp(ηRt−1(at = k))∑K

j=1 exp(ηRt−1(at = j))

• Receive rewards rt,k =∑N

i=1

(w

(i),kt

)2for k = 1, . . . , K

• Set Rt(at = k) = Rt−1(at = k)+rt,k for k = 1, . . . , K

• Sample action k? according to the policy πt(·).

• Set w(i)t = w

(i),k?

t for i = 1, ..., N

4. Selection step

• Resample with replacement N particles(x(i)0:t; i = 1, . . . , N

)from the set

(x(i)0:t; i = 1, . . . , N

)according to the normalized

importance weights w(i)t .

• Set t← t + 1 and go to step 2.

Figure 3. Particle filtering algorithm with gaze control.

attentional mechanism for future work.

3. Algorithm

Since the belief state cannot be computed analyti-cally, we will adopt particle filtering to approximateit. The algorithm is shown in Figure 3. We referreaders to (Doucet et al., 2001) for a more in depthtreatment of these sequential Monte Carlo methods.Assume that at time t − 1 we have N � 1 par-

ticles (samples) {x(i)0:t−1}Ni=1 distributed according to

p (dx0:t−1|h1:t−1,a1:t−1). We can approximate thisbelief state with the following empirical distributionp (dx0:t−1|h1:t−1,a1:t−1) , 1

N

∑Ni=1 δx(i)

0:t−1(dx0:t−1).

Particle filters combine sequential importance sam-pling with a selection scheme designed to obtain N

new particles {x(i)0:t}Ni=1 distributed approximately ac-

cording to p (dx0:t|h1:t,a1:t).


3.1. Importance sampling step

The joint distributions p (dx0:t−1|h1:t−1,a1:t−1) andp (dx0:t|h1:t,a1:t) are of different dimension. We

first modify and extend the current paths x(i)0:t−1

to obtain new paths x(i)0:t using a proposal kernel

qt (dx0:t|x0:t−1,h1:t,a1:t) . As our goal is to design a se-quential procedure, we set qt (dx0:t|x0:t−1,h1:t,a1:t) =δx0:t−1

(dx0:t−1) qt (dxt| x0:t−1,h1:t,a1:t), that isx0:t = (x0:t−1, xt). The aim of this kernel is to obtainnew paths whose distribution qt (dx0:t|h1:t,a1:t) =p (dx0:t−1|h1:t−1,a1:t−1) qt (dxt| x0:t−1,h1:t,a1:t) isas “close” as possible to p (dx0:t|h1:t,a1:t). Since wecannot choose qt (dx0:t|h1:t,a1:t) = p (dx0:t|h1:t,a1:t)because this is the quantity we are trying to approx-imate in the first place, it is necessary to weight thenew particles so as to obtain consistent estimates. Weperform this “correction” with importance sampling,using the weights:

wt = wt−1p (ht|xt,at) p (dxt|x0:t−1,at−1)

qt (dxt| x0:t−1,h1:t,a1:t).

The choice of the transition prior as proposal distri-bution is by far the most common one. In this case,the importance weights reduce to the expression forthe likelihood. However, it is possible to constructbetter proposal distributions, which make use of morerecent observations, using object detectors (Okumaet al., 2004), saliency maps (Itti et al., 1998), opti-cal flow, and approximate filtering methods, as in theunscented particle filter. One could also easily incor-porate strategies to manage data association and othertracking related issues. After normalizing the weights,

w(i)t =

w(i)t∑N

j=1 w(j)t

, we obtain the following estimate of

the filtering distribution:

p (dx0:t|h1:t,a1:t) =

N∑i=1

w(i)t δ

x(i)0:t

(dx0:t) .

3.2. Gaze control step

We treat the problem of choosing a gaze (from thetemplate) with a portfolio allocation algorithm calledHedge (Freund & Schapire, 1997; Auer et al., 1998).Hedge is an algorithm that, at each time step t, up-dates the policy πt(at|Rt−1) for each allowed action(see (Auer et al., 1998)). It then selects an action k?

according to this policy as shown in Figure 3. Theimmediate reward is defined as the variance of thenormalized importance weights. This choice is moti-vated by the fact that the (factored)-RBM observationmodel for the state-space representation, p (h|x,a), isvery peaked.

Note that we assumed a perfect information game.However, it is possible to only observe one of thegazes at each time using the EXP3 algorithm from(Auer et al., 1998) or Bayesian optimization techniques(Brochu et al., 2009).

3.3. Selection step

The aim of the selection is to obtain an “unweighted”approximate empirical distribution p (dx0:t|h1:t,a1:t)of the weighted measure p (dx0:t|h1:t,a1:t). The basicidea is to discard the samples with small weights andmultiply those with large weights. The introduction ofthis key step led to the first operational SMC method;see (Doucet et al., 2001) for details of implementationof this black-box routine.

4. Experiments

In this section, three experiments are carried out toevaluate quantitatively and qualitatively the proposedapproach. The first experiment provides comparisonsto other control policies on a synthetic dataset. Thesecond experiment, on a similar synthetic dataset,demonstrates how the approach can handle large vari-ations in scale, occlusion and multiple targets. Thefinal experiment is a demonstration of tracking andclassification performance on several real videos. Forthe synthetic digit videos, we trained the first-layerRBMs on the foveated images, while for the real videoswe trained factored-RBMs on foveated natural imagepatches (Ranzato & Hinton, 2010).

The first experiment uses 10 video sequences (one foreach digit) built from the MNIST dataset. Each se-quence contains a moving digit and static digits in thebackground (to create distractions). The objective isto track and recognize the moving digit; see Figure 4.The gaze template had K = 9 gaze positions, chosenso that gaze G5 was at the center. The location of thetemplate was initialized with optical flow.

We compare the policy learning algorithm against al-gorithms with deterministic and random policies. Thedeterministic policy chooses each gaze in sequence andin a particular pre-specified order, whereas the randompolicy selects a gaze uniformly at random. We adoptedthe Bhattacharyya distance in the specification of theobservation model. A multi-fixation RBM was trainedto map the first layer hidden units of three time con-secutive time steps into a second hidden layer, andtrained a logistic regressor to further map to the 10digit classes. We used the transition prior as proposalfor the particle filter.

Tables 1 and 2 report the comparison results. Track-


Table 1. Tracking error (in pixels) on several video sequences using different policies for gaze selection.

0 1 2 3 4 5 6 7 8 9 Avg.Learnedpolicy

1.2(1.2)

3.0(2.0)

2.9(1.0)

2.2(0.7)

1.0(1.9)

1.8(1.9)

3.8(1.0)

3.8(1.5)

1.5(1.7)

3.8(2.8)

2.5(1.6)

Deterministicpolicy

18.2(29.6)

536.9(395.6)

104.4(69.7)

2.9(2.2)

201.3(113.4)

4.6(4.0)

5.6(3.1)

64.4(45.3)

142.0(198.8)

144.6(157.7)

122.5(101.9)

Randompolicy

41.5(54.0)

410.7(329.4)

3.2(2.0)

3.3(2.4)

42.8(60.9)

6.5(9.6)

5.7(3.2)

80.7(48.6)

38.9(50.6)

225.2(241.6)

85.9(80.2)

Table 2. Classification accuracy on several video sequences using different policies for gaze selection.

0 1 2 3 4 5 6 7 8 9 Avg.Learnedpolicy

95.62% 100.00% 99.66% 99.33% 99.66% 100.00% 100.00% 98.32% 97.98% 89.56% 98.01%

Deterministicpolicy

99.33% 100.00% 98.99% 94.95% 5.39% 98.32% 0.00% 29.63% 52.19% 0.00% 57.88%

Randompolicy

98.32% 100.00% 96.30% 99.66% 29.97% 96.30% 89.56% 22.90% 12.79% 13.80% 65.96%

ing accuracy was measured in terms of the mean andstandard deviation (in brackets) over time of the dis-tance between the target ground truth and the esti-mate; measured in pixels. The analysis highlights thatthe error of the learned policy is always below the er-ror of the other policies. In most of the experiments,the tracker fails when an occlusion occurs for the de-terministic and the random policies, while the learnedpolicy is successful. This is very clear in the videos at:http://www.youtube.com/user/anonymousTrack

The loss of track for the simple policies is mirrored bythe high variance results in Table 1 (experiments 0, 1,4, and so on). The average mean and standard devi-ations (last column of Table 1) make it clear that theproposed strategy for learning a gaze policy can beof enormous benefit. The improvements in trackingperformance are mirrored by improvements in classifi-cation performance (Table 2).

Figure 4 provides further anecdotal evidence for thepolicy learning algorithm. The top sequence showsthe target and the particle filter estimate of its loca-tion over time. The middle sequence illustrates howthe policy changes over time. In particular, it demon-strates that hedge can effectively learn where to lookin order to improve tracking performance (we chosethis simple example as in this case it is obvious thatthe center of the eight (G5) is the most reliable gazeaction). The classification results over time are shownin the third row.

The second experiment addresses a similar video se-quence, but tracking multiple targets. The image scaleof each target changes significantly over time, so thealgorithm has to be invariant with respect to thesescale transformations. In this case, we used a mix-ture proposal distribution consisting of motion detec-tors and the transition prior. We also tested a saliencyproposal but found it to be less effective than the mo-

tion detectors for this dataset. Figure 5 (top) showssome of the video frames and tracks. The videos allowone to better appreciate the performance of the multi-target tracking algorithm in the presence of occlusions.Tracking and classification results for the real videosare shown in Figure 5 and the accompanying videos.

5. Conclusions and future work

We have proposed a decision-theoretic probabilisticgraphical model for joint classification, tracking andplanning. The experiments demonstrate the signifi-cant potential of this approach. There are many routesfor further exploration. In this work we pre-trained the(factored)-RBMs. However, existing particle filteringand stochastic optimization algorithms could be usedto train the RBMs online. Following the same method-ology, we should also be able to adapt and improve thetarget templates and proposal distributions over time.This is essential to extend the results to long video se-quences where the object undergoes significant trans-formations.

Deployment to more complex video sequences willrequire more careful and thoughtful design of theproposal distributions, transition distributions, con-trol algorithms, continuous template models, data-association and motion analysis modules. Fortunately,many of the solutions to these problems have alreadybeen engineered in the computer vision, tracking andonline learning communities. Admittedly, much workremains to be done.

Saliency maps are ubiquitous in visual attention stud-ies. Here, we simply used standard saliency tools andmotion flow in the construction of the proposal distri-butions for particle filtering. There might be betterways to exploit the saliency maps, as neurophysiologi-cal experiments seem to suggest (Gottlieb et al., 1998).


T = 69/300T = 20/300 T = 142/300 T = 181/300 T = 215/300 T = 260/300

G1 G2 G3 G4 G5 G6 G7 G8 G90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

G1 G2 G3 G4 G5 G6 G7 G8 G90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

G1 G2 G3 G4 G5 G6 G7 G8 G90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

G1 G2 G3 G4 G5 G6 G7 G8 G90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

G1 G2 G3 G4 G5 G6 G7 G8 G90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

G1 G2 G3 G4 G5 G6 G7 G8 G90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1

0

1

0

1

0

1

0

1

0

1

01

0

1

0

1

0

1

0

1

0

1

0

G1 G2 G3 G4 G5 G6 G7 G8 G9 G1 G2 G3 G4 G5 G6 G7 G8 G9 G1 G2 G3 G4 G5 G6 G7 G8 G9 G1 G2 G3 G4 G5 G6 G7 G8 G9 G1 G2 G3 G4 G5 G6 G7 G8 G9 G1 G2 G3 G4 G5 G6 G7 G8 G9

1 2 3 4 5 6 7 8 90 1 2 3 4 5 6 7 8 90 1 2 3 4 5 6 7 8 90 1 2 3 4 5 6 7 8 90 1 2 3 4 5 6 7 8 90 1 2 3 4 5 6 7 8 90

Figure 4. Tracking and classification accuracy results with the learned policy. First row: position of the target and estimateover time. Second row: policy distribution over the 9 gazes; hedge clearly converges to the most reasonable policy. Thirdrow: cumulative class distribution for recognition.

jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

jennifer lopez

jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

jennifer lopez jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

jennifer lopez

jennifer lopez

jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

jennifer lopez

jennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopezjennifer lopez

Figure 5. (Top) Multi-target tracking with occlusions and changes in scale on a synthetic video. (Middle and bottom)Tracking in real video sequences.

One of the most interesting avenues for future work isthe construction of more abstract attentional strate-gies. In this work, we focused on attending to regionsof the visual field, but clearly one could attend to sub-

sets of receptive fields or objects in the deep appear-ance model.


Acknowledgments

We thank Ben Marlin, Kenji Okuma, Marc’AurelioRanzato and Kevin Swersky. This work was supportedby CIFAR’s NCAP program and NSERC.

References

Auer, Peter, Cesa-Bianchi, Nicolo, Freund, Yoav, andSchapire, Robert E. Gambling in a rigged casino: theadversarial multi-armed bandit problem. Technical Re-port NC2-TR-1998-025, 1998.

Brochu, Eric, Cora, Vlad M, and de Freitas, Nando. A tu-torial on Bayesian optimization of expensive cost func-tions, with application to active user modeling and hier-archical reinforcement learning. Technical Report TR-2009-023, CS dept, UBC, 2009.

Cesa-Bianchi, Nicolo and Lugosi, Gabor. Prediction,Learning, and Games. Cambridge University Press, NewYork, 2006.

Chaudhuri, Kamalika, Freund, Yoav, and Hsu, Daniel. Aparameter-free hedging algorithm. In NIPS, 2009.

Colombo, John. The development of visual attention in in-fancy. Annual Review of Psychology, pp. 337–367, 2001.

Doucet, A, de Freitas, N, and Gordon, N. Introduction tosequential Monte Carlo methods. In Doucet, A, de Fre-itas, N, and Gordon, N J (eds.), Sequential Monte CarloMethods in Practice. Springer-Verlag, 2001.

Freund, Yoav and Schapire, Robert E. A decision-theoreticgeneralization of on-line learning and an application toboosting. Journal of Computer and System Sciences, 55:119–139, 1997.

Girard, B. and Berthoz, A. From brainstem to cortex:Computational models of saccade generation circuitry.Progress in Neurobiology, 77(4):215 – 251, 2005.

Gottlieb, Jacqueline P., Kusunoki, Makoto, and Goldberg,Michael E. The representation of visual salience in mon-key parietal cortex. Nature, 391:481–484, 1998.

Hinton, GE and Salakhutdinov, RR. Reducing the dimen-sionality of data with neural networks. Science, 313(5786):504–507, 2006.

Isard, M and Blake, A. Contour tracking by stochasticpropagation of conditional density. In European Com-puter Vision Conference, pp. 343–356, 1996.

Itti, L., Koch, C., and Niebur, E. A model of saliency-basedvisual attention for rapid scene analysis. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 20(11):1254 –1259, 1998.

Kavukcuoglu, K., Ranzato, M.A., Fergus, R., and Le-Cun,Yann. Learning invariant features through topographicfilter maps. In Computer Vision and Pattern Recogni-tion, pp. 1605–1612, 2009.

Koster, Urs and Hyvarinen, Aapo. A two-layer ICA-likemodel estimated by score matching. In InternationalConference of Artificial Neural Networks, pp. 798–807,2007.

Larochelle, Hugo and Hinton, Geoffrey. Learning to com-bine foveal glimpses with a third-order Boltzmann ma-chine. In Neural Information Processing Systems, 2010.

Lee, H., Grosse, R., Ranganath, R., and Ng, A.Y. Convo-lutional deep belief networks for scalable unsupervisedlearning of hierarchical representations. In InternationalConference on Machine Learning, 2009.

McNaughton, Bruce L., Battaglia, Francesco P., Jensen,Ole, Moser, Edvard I., and Moser, May-Britt. Path in-tegration and the neural basis of the ’cognitive map’.Nature Reviews Neuroscience, 7(8):663–678, 2006.

Okuma, Kenji, Taleghani, Ali, de Freitas, Nando, andLowe, David G. A boosted particle filter: Multitargetdetection and tracking. In ECCV, 2004.

Olshausen, B. A. and Field, D. J. Emergence of simple-cellreceptive field properties by learning a sparse code fornatural images. Nature, 381:607–609, 1996.

Olshausen, Bruno A., Anderson, Charles H., and Essen,David C. Van. A neurobiological model of visual at-tention and invariant pattern recognition based on dy-namic routing of information. Journal of Neuroscience,13:4700–4719, 1993.

Postma, Eric O., van den Herik, H. Jaap, and Hudson,Patrick T. W. SCAN: A scalable model of attentionalselection. Neural Networks, 10(6):993 – 1015, 1997.

Ranzato, M.A. and Hinton, G.E. Modeling pixel meansand covariances using factorized third-order Boltzmannmachines. In Computer Vision and Pattern Recognition,pp. 2551–2558, 2010.

Rensink, Ronald A. The dynamic representation of scenes.Visual Cognition, pp. 17–42, 2000.

Rosa, M.G.P. Visual maps in the adult primate cerebralcortex: Some implications for brain development andevolution. Brazilian Journal of Medical and BiologicalResearch, 35:1485 – 1498, 2002.

Swersky, K., Chen, Bo, Marlin, B., and de Freitas, N. A tu-torial on stochastic approximation algorithms for train-ing restricted Boltzmann machines and deep belief nets.In ITA Workshop, pp. 1–10, 2010.

Taylor, G.W., Sigal, L., Fleet, D.J., and Hinton, G.E. Dy-namical binary latent variable models for 3D human posetracking. In Computer Vision and Pattern Recognition,pp. 631–638, 2010.

Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.A.Extracting and composing robust features with denois-ing autoencoders. In International Conference on Ma-chine Learning, pp. 1096–1103, 2008.

Welling, M., Rosen-Zvi, M., and Hinton, G. Exponentialfamily harmoniums with an application to informationretrieval. Neural Information Processing Systems, 17:1481–1488, 2005.

learning attentional policies for tracking and recognition

Documents