the retina overview of the visual system perception &...
TRANSCRIPT
-
1Percep
tion&
Atten
tion
Percep
tioniseffo
rtlessbutits
underly
ingmech
anism
sare
incred
ibly
sophisticated
.
•Biologyofthevisu
alsystem
•Rep
resentatio
nsin
prim
aryvisu
alcortex
andHeb
bian
learning
•Object
recognitio
n
•Atten
tion:Interactio
nsbetw
eensystem
sinvolved
inobject
recognitio
nan
dsp
atialprocessin
g
2Percep
tion&
Atten
tion
Somemotiv
atingquestio
ns:
1.Whydoes
prim
aryvisu
alcortex
encodeorien
tedbars
oflig
ht?
2.Whyisvisu
alsystem
split
into
what/
where
path
way
s?
3.Whydoes
parietal
dam
agecau
seatten
tionproblem
s(neg
lect)?
4.How
dowereco
gnize
objects
(across
locatio
ns,sizes,ro
tations
with
wild
lydifferen
tretin
alim
ages)?
3Overv
iewoftheVisu
alSystem
Hierarch
iesofsp
ecializedvisu
alpath
way
s,startingin
retina,to
LGN
(thalam
us),to
V1&
up:
opticchiasm
temporal
temporal
nasal
LGN
V1
V2,V
4...
V2,V
4...
right fieldleft field
4TwoStream
s:Ven
tral“w
hat”
vs.Dorsal
“where”
V1
V2V
3V
4T
EO
TF
TE
PO
V3A
MT
FS
T
MS
T
VIP
PGP
G
TE
V1
p
mm
d
5TheRetin
a
Retin
aisnotapassiv
e“cam
era”
Key
prin
ciple:
contrast
enhan
cemen
tthat
emphasizes
chan
ges
over
space
&tim
e.
−+−
−−
−+
+
+
+
a) On−
centerb) O
ff−center
+
−−
+−
+
− +
−
+
retinal
outputgan
glio
ncells
6LGN
oftheThalam
us
A“relay
station”,b
utso
much
more.
•Organ
izesdifferen
ttypes
ofinform
ationinto
differen
tlay
ers.
•Perfo
rmsdynam
icprocessin
g:mag
nocellu
larmotio
n
processin
gcells,atten
tional
processin
g.
•On-an
doff-cen
terinform
ationfro
mretin
aispreserv
edin
LGN
-
7Prim
aryVisu
alCortex
(V1):
EdgeDetecto
rs
V1combines
LGN
(thalam
us)
inputsinto
orien
teded
gedetecto
rs:
−+−
−−
−+
+
+
+
a) On−
centerb) O
ff−center
+
−−
+−
+
− +
−
+
−−
−−
+−−
−−
+−−
−−
+−−
−−
+
+−
•Edges
differ
inorien
tation,size
(spatial
frequen
cy),an
d
positio
n.
•Forcoheren
tvisio
n,need
todetect
vary
ingdeg
reesofall
these.
8Prim
aryVisu
alCortex
(V1):
EdgeDetecto
rs
V1combines
LGN
(thalam
us)
minputsinto
orien
teded
ge
detecto
rs :
−+−
−−
−+
+
+
+
a) On−
centerb) O
ff−center
+
−−
+−
+
− +
−
+
−−
−−
+−−
−−
+−−
−−
+−−
−−
+
+−
V1
edgedetector
•Edges
differ
inorien
tation,size
(spatial
frequen
cy),an
dpositio
n.
•Forcoheren
tvisio
n,need
todetect
vary
ingdeg
reesofall
these.
9Prim
aryVisu
alCortex
(V1):
Topograp
hy
LR
LR
4 2−3 blobs
orientations
occularity
hypercolumns
Pinwheel
Hyperco
lumn:Fullset
ofcodingforeach
positio
nPinwheel
can
arisefro
mlearn
ingan
dlateral
connectiv
ity:nothard
-wired
!
10Rero
utin
gofVisu
alInfo
toAudito
ryCortex
•Sharm
a,Angelu
cci&
Sur(2000),N
ature
Rero
uted
fibers
from
Retin
a→
audito
rythalam
us(M
GN)→
A1
•Ifvisu
alproperties
arelearn
ed,th
eysh
ould
dev
elopin
A1.
11Rero
utin
gofVisu
alOrien
tationModules
inA1
Ba-d
:Orien
tationmap
s,dark
-highact
forgiven
orien
tation
(botto
m
right).
C:co
mposite
map
oforien
tationpreferen
ces
D:red
dots=pinwheel
centers
12Visu
alBeh
aviorAfter
Rero
utin
gRightVisu
alField
vonMelch
ner,
Pallas
&Sur(2000)
-
13Visu
alAcu
ityAfter
Rero
utin
g
→Solearn
ingis
powerfu
l,butso
isev
olutio
n!
14A
Questio
n
What
mak
esvisu
alcortex
visu
alcortex
?Whydoes
itrep
resent
orien
tedbars
oflig
ht?
15Prim
aryVisu
alRep
resentatio
ns
Key
idea:
Orien
teded
gedetecto
rscan
dev
elopfro
mHeb
bian
correlatio
nal
learningbased
onnatu
ralvisu
alscen
es.
16TheModel:
Sim
ulatin
goneHyperco
lumn
•Natu
ralvisu
alscen
esare
prep
rocessed
bypassin
gthem
(separately
)throughlay
ersofon-cen
teran
doff-cen
terinputs
•Hidden
layer:
edgedetecto
rsseen
inlay
ers2/
3ofV1;
Lay
er4(in
put)just
represen
tsunorien
tedon/offinputs
likeLGN
(butcan
bemodulated
by
attentio
n)
17TheModel:
Sim
ulatin
goneHyperco
lumn
•Heb
bian
learningonly
•KWTA
inhib
competitio
nforsp
ecialization(see
Ch4)
18[v1rf.p
roj]
-
19TheRecep
tiveField
s0
12
34
56
78
910
1112
13
Red
=on-cen
ter>
off-cen
ter,Blue=off-cen
ter>
on-cen
ter
20
2122
23Somedifferen
ces,butpinwheels
stillem
erge
24Percep
tionan
dAtten
tion
1.Whydoes
prim
aryvisu
alcortex
encodeorien
tedbars
oflig
ht?
Correlatio
nal
learningbased
onnatu
ralvisu
alscen
es.
Refl
ectsreliab
lepresen
ceofed
ges
innatu
ralim
ages,w
hich
vary
in
size,positio
n,orien
tationan
dpolarity.
→model
showshow
docu
men
tedV1properties
canresu
ltfro
m
interactio
nsbetw
eenlearn
ing,arch
itecture
(connectiv
ity),an
d
structu
reofen
viro
nmen
t.
-
25Percep
tionan
dAtten
tion
1.Whydoes
prim
aryvisu
alcortex
encodeorien
tedbars
oflig
ht?
Correlatio
nal
learningbased
onnatu
ralvisu
alscen
es.
2.How
dowereco
gnize
objects
(across
locatio
ns,sizes,ro
tations
with
wild
lydifferen
tretin
alim
ages)?
3.Whyisvisu
alsystem
split
into
what/
where
path
way
s?
4.Whydoes
parietal
dam
agecau
seatten
tionproblem
s(neg
lect)?
26TheObject
Reco
gnitio
nProblem
Problem
:Reco
gnize
object
regard
lessof:locatio
n,size,ro
tation.
Sam
e
Diff
Thisishard
becau
sedifferen
tpattern
sin
samelocatio
ncan
overlap
alot,w
hile
thesam
epattern
sin
differen
tlocatio
ns/
sizes/rotatio
ns
cannotoverlap
atall!
2728 G
radual
Invarian
ceTran
sform
ations(Fukush
ima,’80)
Increasin
grecep
tivefield
sizeen
ables:
Conjunctio
noffeatu
res(to
form
more
complex
objects);
and
Collap
singover
locatio
ninform
ation(“sp
atialinvarian
ce”)
29 Grad
ual
Invarian
ceTran
sform
ations(Fukush
ima,’80)
ifdid
spatial
invarian
cein
onefell
swoop:bindingproblem
-can
’ttell
Tfro
mL
30 Grad
ual
Invarian
ceTran
sform
ations(Fukush
ima,’80)
Goal:
Units
atthetopofthehierarch
ysh
ould
represen
tcomplex
object
features
inalocatio
nan
dsize
invarian
tfash
ion
(alsowan
tben
efits
oftop-downam
plifi
cation,pattern
completio
n,d
istributed
repsetc)
-
31TheModel
LGN
_On
LGN
_Off
V1 V2
V4/IT
Output
V1=orien
tedlin
e(ed
ge)
detecto
rs,hard
-coded
V2units
encodeconjunctio
nsofV1ed
ges
across
asu
bset
ofsp
ace
Each
V4unitpay
satten
tionto
allofV2
32TheObjects
01
23
4
56
78
9
1011
1213
14
1516
1718
19
Each
object
ispresen
tedat
multip
lelocatio
ns,sizes
Netw
ork’sjobisto
activate
theap
propriate
Outputunit(0-19)
for
eachobject,reg
ardless
oflocatio
nan
dsize
33[objrec.p
roj]
34
3536
-
3738
3940
4142
-
4344
Gen
eralization
•Can
thenetw
ork
gen
eralizeto
unseen
view
sofstu
died
objects?
•In
other
words:Does
trainingthenet
toreco
gnize
aset
of
objects
inasize/
locatio
ninvarian
tfash
ionhelp
itreco
gnize
new
objects
inasize/
locatio
ninvarian
tfash
ion?
•Proced
ure:
–Tak
eanet
trained
on18
objects
–Train
with
2new
objects
inonly
somelocatio
ns/
sizes
–Test
thenet
with
nonstu
died
“view
s”(sizes/
locatio
ns)
of
new
objects
4546
Gen
eralization
•Can
thenetw
ork
gen
eralizeto
unseen
view
sofstu
died
objects?
yes
•Approx.75%
correct
onnovel
view
sfollo
wingtrain
ingon10%
ofpossib
lesizes/
locatio
nsExplan
ation:Distrib
uted
represen
tationsan
dHeb
blearn
ing!
•V4rep
resents
objectfeatures
inalocatio
n/size
invarian
tway
•Each
object
activates
adistrib
uted
pattern
ofthese
invarian
t
feature
detecto
rs
4748
-
4950
5152
5354
01
23
4
56
78
9
1011
1213
14
1516
1718
19
Yeah
,butthese
objects
arereg
ularly
shap
ed,straig
htlin
es...
what
aboutreal
objects?
-
55
56“E
mer”
therobotreco
gnizin
gobjects..
Video
Dem
o
57
O’Reilly,
CogSci2009
58why?
bidirconssu
pportattracto
rsacro
ssmultip
lelev
elsofnet
toam
plify
consisten
t
info
59A
Challen
ge
-
60State
oftheArt
61Still
missin
g...
Motio
n
•Neu
ronsin
areaMTvery
sensitiv
eto
motio
n
•Lotsofwork
onhow
downstream
areasinteg
ratemotio
nsig
nals
across
timeto
detect
coheren
ce(e.g
.Shad
len,
New
some,etc)
•Thomas
Serre
has
shownthat
motio
nsig
nals
very
reliable
for
discrim
inatin
gbetw
eenparticu
laractio
ns(eg
throwinga
baseb
all)
•Should
beab
leto
solveproblem
via
bidirectio
nal
influen
ceof
motio
ninteg
rationsig
nals,o
bject
recognitio
n,an
dsp
atialatten
tion(next)....
62Percep
tionan
dAtten
tion
1.Whydoes
prim
aryvisu
alcortex
encodeorien
tedbars
oflig
ht?
Correlatio
nal
learningbased
onnatu
ralvisu
alscen
es.
2.How
dowereco
gnize
objects
(across
locatio
ns,sizes,ro
tations
with
wild
lydifferen
tretin
alim
ages)?
Tran
sform
ations:
increasin
gly
complex
featural
encodings,in
creasinglev
elsof
spatial
invarian
ce;Distrib
uted
represen
tations.
3.Whyisvisu
alsystem
split
into
what/
where
path
way
s?
4.Whydoes
parietal
dam
agecau
seatten
tionproblem
s(neg
lect)?
63Spatial
Atten
tion:Unilateral
Neg
lect
Patien
tcopyinga
scene
Self
portrait,
copying,
linebisectio
ntask
s:In
allcases,p
atientswith
parietal/
temporal
lesionsseem
toforget
about1/
2ofsp
ace!butthey
stillsee
it!
6465
-
6667
6869
7071
-
7273
74Effects
ofParietal
Lesio
nsonPosn
erTask
40−
60−
80−
100−
120−01
2
Intact
Lesioned
Neutral
Valid
Invalid
•Patien
tsperfo
rmnorm
allyin
the“n
eutral”
(nocu
e)conditio
n,
regard
lessofwhere
thetarg
etispresen
ted
•Patien
tsben
efitjustas
much
ascontro
lsfro
mvalid
cues
•Patien
tsare
hurtmore
than
contro
lsbyinvalid
cues
75Possib
leModels
+
Alert
Interrupt
Localize
Disengage
Move
Engage
Inhibit
Object
V1
(features x location)
Spatial
Atten
tionem
erges
from
bidirectio
nal
constrain
tsatisfactio
n&
inhibito
rycompetitio
n.
76Sim
ple
Model
Input
V1
Spat1
Spat2
Obj1
Obj2
targ
cue
Output
Object 1 (C
ue)
Object 2 (T
arget)
77[attn
simple.p
roj]
-
78Posn
erTask
Data
Valid
Invalid
Diff
AdultNorm
al350
39040
Elderly
Norm
al540
60060
Patien
ts640
760120
Elderly
norm
alized(*.65)
350390
40Patien
tsnorm
alized(*.55)
350418
68
79Posn
erTask
Sim
s
•Themodel
explain
sthebasic
findingthat
valid
cues
speed
target
processin
g,while
invalid
cues
hurt
•Also
explain
sfindingthat
patien
tswith
small
unilateral
parietal
lesionsben
efitnorm
allyfro
mvalid
cues
inipsilateral
field
butare
disp
roportio
nately
hurtbyinvalid
cues.
•Noneed
toposit
“disen
gag
e”module!
•Also
explain
sfindingofneglect
ofcontralateral
visu
alfield
afterlarg
e,unilateral
parietal
lesionswhen
somestim
ulusis
presen
tin
ipsilateral
field
(“extin
ction”)
80More
Posn
erLesio
nFun
•Retu
rningto
patien
twith
leftparietal
lesion...
•What
hap
pen
sifcu
esare
presen
tedin
contralateral(affected
)
hem
ifield
?(“R
everse
Posn
er”)
81More
Posn
erLesio
nFun
Retu
rningto
patien
twith
leftparietal
lesion...
•What
hap
pen
sifcu
esare
presen
tedin
contralateral(affected
)hem
ifield
?
Pred
ictions:
•Smaller
ben
efitforvalid
cues
•Patien
tssh
ould
behurtless
than
contro
lsbyinvalid
cues.
82[attn
simple.p
roj]
83Inhibitio
nofRetu
rn
•Typically,targ
etdetectio
nisfaster
ontrials
with
valid
vs
invalid
cues
•H
owever
,ifthecu
eispresen
tedforalonger
time(eg
.500
ms),
perfo
rman
ceisfaster
on
invalidvsvalid
trials
•Can
explain
interm
sofaccom
modation
(neu
ralfatig
ue)
-
84[attn
simple.p
roj]
85Sim
ple
model:
toosim
ple?
•Has
uniqueone-to
-onemap
pingsbetw
eenlow-lev
elvisu
alfeatu
resan
dobject
represen
tations(notrealistic)
•Does
notad
dress
issueofsp
atialatten
tionwhen
tryingto
perceiv
emultip
leobjects
simultan
eously
•“C
omplex
”model
combines
more
realisticmodel
ofobject
recognitio
n(startin
gfro
mLGN)with
simple
attentio
nmodel
→Can
use
spatial
attentio
nto
restrictobject
processin
gpath
way
tooneobject
atatim
e,enab
lingitto
sequen
tiallyprocess
multip
leobjects.
•Lesio
nsofen
tiresp
atialpath
way
cause
simultan
agnosia:
inab
ilityto
concu
rrently
recognize
twoobjects
86Complex
Model
LGN
_On
LGN
_Off
V1
V2
V4
/IT Output
Spat1
Spat2
Target
Spat1
has
recurren
tprojnsto
encourag
efocu
sononereg
ionofsp
ace
Butonly
mech
anism
forsw
itchingisacco
mmodatio
n...
87Percep
tionan
dAtten
tion
1.Whydoes
prim
aryvisu
alcortex
encodeorien
tedbars
oflig
ht?
Correlatio
nal
learningbased
onnatu
ralvisu
alscen
es.
2.How
dowereco
gnize
objects
(across
locatio
ns,sizes,ro
tations
with
wild
lydifferen
tretin
alim
ages)?
Tran
sform
ations:
increasin
gly
complex
featural
encodings,in
creasinglev
elsof
spatial
invarian
ce;Distrib
uted
represen
tations.
3.Whyisvisu
alsystem
split
into
what/
where
path
way
s?
Tran
sform
ations:em
phasizin
gan
dcollap
singacro
ssdifferen
ttypes
ofrelev
antdistin
ctions
4.Whydoes
parietal
dam
agecau
seatten
tionproblem
s(neg
lect)?
Atten
tionas
anem
ergen
tproperty
ofcompetitio
n
88Gen
eralIssu
esin
Atten
tion
Atten
tion:
•Prio
ritizesprocessin
g.
•Coordinates
processin
gacro
ssdifferen
tareas.
•Solves
bindingproblem
svia
coordinatio
n.
Butatten
tionsh
ould
bemuch
more
flexible
than
justsp
atialbias!
Later:
how
toincorporate
goals,rein
forcem
entprobab
ility,into
attentio
nal
allocatio
n