reading assignmentsilab.usc.edu/classes/2001cs599/notes/12-visual-attention.pdf · 2001. 4. 3. ·...
TRANSCRIPT
1Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Computational Architectures in Biological Vision, USC, Spring 2001
Lecture 12. Visual Attention
Reading Assignments:
None
2Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
3Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
4Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Several Forms of AttentionAttention and eye movements:
- overt attention (with eye movements)- covert attention (without eye movements)
Bottom-up and top-down control:- bottom-up control
based on image featuresvery fast (up to 20 shifts/s)involuntary / automatic
- top-down controlmay target inconspicuous locations in visual sceneslower (5 shifts/s or fewer; like eye movements)volitional
Control and modulation:- direct attention towards specific visual locations- attention modulates early visual processing at attended location
5Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
What is attention then?
Attention is often described as an information processing bottleneck.
Controls access to higher levels of processing, short-term memory and consciousness.
Hence, the strategy nature has developed to cope with information overload is to break down the problem of analyzing a visual scene:
from a massively parallel approach
to a rapid sequence of circumscribed recognitions.
6Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
7Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
8Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
9Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
10Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
11Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
12Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
13Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
14Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
15Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
16Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
First Computational Model
Koch & Ullman, Hum. Neurobiol., 1895
Introduce conceptof a single topo-graphic saliencymap.
Most salientlocation selectedby a winner-take-allnetwork.
17Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
18Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Shifter CircuitsAnderson & van Essen, PNAS, 1987
Information dynamically routed throughcortical hierarchy. Yields rotation- andscale-independent representation.
19Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Shifter Circuits (cont.)
Olshausen et al., J Neurosci, 1993
Implemented shifter circuits and demonstrated proof of concept.
Control neurons in the pulvinar send the (attention-based) control signals that will determine the �passing� region of the circuit, through a modulation of intracortical connection weights.
Perform recognitionusing associativememory at toplevel.
20Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
only attended item reaches output layer
21Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Selective Tuning ModelTsotsos Tsotsos et al., Artificial Intelligence, 1995et al., Artificial Intelligence, 1995
-- attention modulates neurons to earliest levels; wherever thereattention modulates neurons to earliest levels; wherever there is a is a manymany--toto--one mappingone mapping
-- signal interference controlled by surround inhibition signal interference controlled by surround inhibition throughout processing network throughout processing network
-- task knowledge biases computations throughout task knowledge biases computations throughout processing networkprocessing network
-- attentionalattentional control is local, distributed and internalcontrol is local, distributed and internal
-- competition is based on WTA competition is based on WTA (different form than previous models)(different form than previous models)
-- pyramid representation with reciprocal pyramid representation with reciprocal convergence and divergence convergence and divergence
neuron �sees� thisneuron �sees� thisreceptive fieldreceptive field
subject �attends�subject �attends�to single itemto single item
22Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
attentional spotlight
effective receptive field of selected unit in unattended case
layers of input abstraction hierarchy
inhibitory attentional beam
"pass" zone
"inhibit" zone
The basic idea (BBS 1990)
23Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Selective Tuning Model
processingpyramid
inhibited pathways
passpathways
unit of interestat top
input
Caputo & Guerra 1998Caputo & Guerra 1998Bahcall & Kowler 1999Bahcall & Kowler 1999Vanduffel, Tootell, Orban 2000Vanduffel, Tootell, Orban 2000Smith et al. 2000Smith et al. 2000
KastnerKastner, De Weerd, , De Weerd, Desimone, Ungerleider, Desimone, Ungerleider, 19981998
24Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
25Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Guided SearchWolfe, Psychonomic Bull. & Rev., 1994
How can we combine information from several modalities? Use top-down (task-dependent) weighting.
26Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
27Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
28Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
29Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
30Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
31Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
32Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
33Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
34Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
35Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
36Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
37Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
38Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
39Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
40Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
41Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
42Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
43Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Imag
e C
ompr
essi
on
44Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Evaluation of Advertising
45Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
46Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
47Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Brefczynski & DeYoe, N
ature Neuroscience 1999
48Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
49Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Treu
e &
Mar
tinez
-Tru
jillo,
Nat
ure
1999
50Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
51Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Attentional Modulation in Humans
Gandhi et al, 1999
52Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
53Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
54Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
55Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
56Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
57Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
58Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
59Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
60Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
61Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
62Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
Attentional ModulationHernandez et al.
Picture naming by bi-lingualpersons.
Increased attention/concentrationdue to increased difficultywhen non-native tongue?
Broca: speech generationSupramarginal: articulation &phonology processing
Cingulate: emotion, memory,vigilance, attention?
63Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
64Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention
65Laurent Itti: CS599 � Computational Architectures in Biological Vision, USC 2001. Lecture 12: Visual Attention