vidna kognicija iv
DESCRIPTION
Vidna kognicija IV. Danko Nikoli ć. Sinestezija. S i n este z i j a. Gr čki: ( syn ) ujedinjenje + ( aesthesis ) osjeti. “ Ujedinjenje osjetila ”. Ramachandran & Hubbard, 2003. Osnovne karakteristike sinestezije. Osjetna iskustva izazvana neuobičajenim podražajem - PowerPoint PPT PresentationTRANSCRIPT
Vidna kognicija IV
Danko Nikolić
Sinestezija
Sinestezija
Grčki: (syn) ujedinjenje + (aesthesis) osjeti
“Ujedinjenje osjetila”
Ramachandran &Hubbard, 2003.
Graphemes -> colors = 517/778 = 66.5% Smells -> temperatures = 1/778 = 0.1%
Time units -> colors = 177/778 = 22.8% Smells -> touch = 5/778 = 0.6%
Musical sounds -> colors = 144/778 = 18.5% Sounds -> kinetics = 4/778 = 0.5%
General sounds -> colors = 113/778 = 14.5% Sounds -> smells = 14/778 = 1.8%
Phonemes -> colors = 77/778 = 9.9% Sound -> tastes = 48/778 = 6.2%
Musical notes -> colors = 75/778 = 9.6% Sound -> temperatures = 4/778 = 0.5%
Smells -> colors = 53/778 = 6.8% Sound -> touch = 31/778 = 4.0%
Tastes -> colors = 51/778 = 6.6% Tastes -> sounds = 1/778 = 0.1%
Pain -> colors = 45/778 = 5.8% Tastes -> temperatures = 1/778 = 0.1%
Personalities -> colors = 43/778 = 5.5% Tastes -> touch = 4/778 = 0.5%
Touch -> colors = 31/778 = 4.0% Temperatures -> sounds = 1/778 = 0.1%
Temperatures -> colors = 19/778 = 2.4% Touch -> smell = 2/778 = 0.3%
Orgasm -> colors = 8/778 = 1.0% Touch -> sounds = 3/778 = 0.4%
Emotions -> colors = 8/778 = 1.0% Touch -> tastes = 5/778 = 0.6%
Kinetics -> sounds = 3/778 = 0.4% Touch -> temperatures = 1/778 = 0.1%
Musical notes -> tastes = 1/778 = 0.1% Vision -> smells = 8/778 = 1.0%
Personalities -> smells = 3/778 = 0.4% Vision -> sounds = 12/778 = 1.5%
Personalities -> touch = 1/778 = 0.1% Vision -> tastes = 16/778 = 2.1%
Smells -> sounds = 4/778 = 0.5% Vision -> Temperatures = 2/778 = 0.3%
Smells -> tastes = 1/778 = 0.1% Vision -> touch = 8/778 = 1.0%
Osnovne karakteristike sinestezije
• Osjetna iskustva izazvana neuobičajenim podražajem
• Živo i detaljno perceptivno iskustvo• Automatska asociacija; Nema voljne kontrole• Od ranog djetinjstva (“otkad znaju za sebe”)• Asocijacije se (uglavnom) ne mijenjaju
vremenom
Poznate osobe sinestete
• Wassily Kandinsky
• Vladimir Nabokov
• Franz Liszt
• Nikolai Rimsky-Korsakov
• Nikola Tesla
• Richard Feynman
Sensorno iskustvo
Induktor Dodatno iskustvo
Sensorno iskustvo
Drugačiji model
Semantička reprezent.
Induktor
Dodatno iskustvo
Sensorno iskustvo
Sensorno iskustvo
Test for eccentricities:1 Report that Synaesthesia disappeared at higer eccentricities: An indication for a perceptual aspect of synaesthesia
We retested our synaesthetes: none showed a dissapearence of synaesthesia at higheccentricities
All our synaesthetes reported, that letters and digits do change color depending on theirmeaning, e.g, O and 0.So we tested our synaesthetes with
-3 Homographs: -Montage (Installation)/Montage (Wochentage)-Collagen (Plural von Collage)/Collagen (Bindemittel)-Heroin (Suchtmittel)/Heroin (Heldin)
Dixon et al 2000
-one synaesthete vs. control-arithmetic test, where the concept of a digit can inducesynaesthesia
Rich an Mattingley 2003 Ambiguous stimuli
-14 synaesthetes vs. 14 controls-first, a stroop test was performed-next, they asked the subjects to attend to either the globalor the local stimuli only-=> the interference effect was significantly reduced, comparedto the first experiment
Palmeri et al., PNAS, 2002
However, Edquist et al. (2006) could not replicate these results on a larger group of synaesthetes.
[Oben sehen Sie eine Studie aus dem Jahr 2002. Hier war die Aufgabe, in dem schwarz-weiss Bild das Ziel, in diesem Fall die zwei, zu finden. Da der Proband das Ziel farbig sah (rechtes Bild) war er schneller, als die Vergleichsgruppe (sog. “pop out” Effekt). Dieses Ergebnis konnte allerdings 2006 nicht wiederholt werden.]
Serial visual search
Set size
Tim
e
Pop out
Serial search
Shape search task
-same 2 synaesthestes-40 controls-Display: 5´s scattered randomly on a white screen, amongthese 2´s forming a global shape like a triangle (4 shapes)- each stimulus was presented for 1 s
Result: The synaesthetes were significantly (81.25%)better than the control (59,4%)
A crowding study
Stroop zadatak
Stroop, 1935
ZelenaZelenakongruentna nije kongruentna
CAR CAR
• Result: Naming the congruent colour is faster than naming the incongruent colour
• No effect with words without a colour meaning:
• Indicates automatic and involuntary processing
Stroop task:
Grün, Rot, Blau, Gelb
vs.
Grün, Rot, Blau, Gelb
Conclusion: reading is an automatic process
Stroop u sinesteziji
• Zadatak: “Imenuj stvarnu boju slova.”
• Resultat: Sinestete imenuju pravu boju slova brže ako ta boja odgovara sinestetskoj boji (kongruentne boje) nego ako se te dvije boje razlikuju (ne kongruentne) (e.g., Odgaard et al.,
1999).
• Result: Synaesthetes are faster in naming the true color if this color is the same as (congruent to) synaesthetic than if these colors are different (incongruent) (e.g., Odgaard et al.,
1999).
• Tipični Stroop zadatak ne dokazuje da synaesthesia proizvodi stvarna perceptualna iskustva.
• Npr. Stroop efekt se može očekivati čak i za obične asociacije, “Znam da je limun žute boje”.
Međutim:
Sensorno iskustvo
Induktor Dodatno iskustvo
Sensorno iskustvo
Studija I: Priroda dodatnih iskustava
Colour-opponency in synaesthetic experiences
Activation in colour-specific areas
• fMRI provided evidence for an activation of the colour specific areas V4/V8 of the visual cortex during synaesthetic color experience. (e.g., Hubbard et al., 2005; Sperling et al., 2006)
• This suggests the involvement of early stages of visual processing
Colour opponency Theory
• First stage: Receptor stage consists of the three photopigments (blue, green and red cones).
• Second stage: neural processing stage where the colour opponency occurs.
Krug boja:
Sinestetska boja
Oponentna boja
Nezavisna boja
Color-opponency in synesthesia• Test color opponency in synaestetic experiences by Stroop
task
+-
+-
or
• Stroop effect can occur for non-synaesthetic associations. For example, “I know that lemon has yellow color”.
• Typical Stroop task is not a direct proof that synaesthesia produces a real perceptual-like experience.
Hipoteze• Grafem-boja sinestezija
-
-Sem. Real
Oponentne boje
Sem. Real
-
-
Ne oponentne boje
+
+Syn. Real
Iste boje
• Semantičke asociacije (npr. žuti limun)
-
-Syn. Real
Oponentne boje
Syn. Real
Ne oponentne boje
Congruentwith synesthetic (C)
Incongruent opponent (O)
Dizajn:
C I O B
Design
• 6 synaesthetes: 2 projectors, 4 associators
• 5 graphemes per participant, presented in three colours plus baseline condition
• Each grapheme was presented 10 times; a total of 200 trials
• 2 control participants per synaesthete, matched by gender and age
C G :
IH :
N B :
8 A E O U
U I E A 3
3 5 9 M A
K L :
T Q :
K T :
4 A E 3 U
A E I O U
2 5 7 E O
Six synaesthetes:
Distribution of colors
ResultsR
esp
on
seT
ime
[ms]
Stimulation condition
650
700
750
800
850
900
950
1000
1050
1100
C I O B
Control subjects(n=12)
Res
po
nse
Tim
e[m
s]
Stimu lation condition
650
700
750
800
850
900
950
1000
1050
1100
C I O B
Synaesthetes(n=6)
p=0.89 p=0.005
C I O B C I O B C I O B C I O B C I O B C I O B
Stimu lation condition (individual)
6 00
7 00
8 00
9 00
1 00 0
110 0
1 20 0
(C G ) (T Q ) (IH ) (N B) (KL ) (KT )
Individual synaesthete subjects
Stimu lation cond ition (grapheme)
IH :
Individual graphemes for one subject
Conclusions
• Reaction times for incongruent colours were overall slower than the congruent condition
• Reaction times for the congruent colour showed a facilitation
• Strong indication that synaesthesia arises within colour-opponency
Congruent(C)
Incongruentopponen t (O)
Incongruentindependen t (I)
Kontrolni eksperiment
Semantičke veze:
Tri objekta:
Res
po
nse
Tim
e[m
s]
C CI IO O
ResultsControl subjects
(n=8)Synaesthetes
(n=4)
~50 ms
p=0.19
Res
po
nse
Tim
e[m
s]
Stimulation condition
650
700
750
800
850
900
950
1000
1050
1100
C I O B
Res
po
nse
Tim
e[m
s]Stimu lation condition
650
700
750
800
850
900
950
1000
1050
1100
C I O B
Res
po
nse
Tim
e[m
s]
C C CI I IO O O
IH :
Individual results for one subject
Conclusions• In grapheme color synaesthesia, the interference in
the Stroop task has two components: – i) semantic component and – ii) color-opponent component.
Nikolić, Lichti & Singer, Psychological Science (2007)
• This indicates early visual representation of synaesthetic color experiences.
• Colour-opponency Stroop task is an objective method to test whether someone is a synaesthete.
Sensorno iskustvo
Induktor Dodatno iskustvo
Sensorno iskustvo
Zaključak
Colour opponency
task
Second study
Serial visual search
[Zweite Studie. Ein Studie über serielle visuelle Suche in der Synästhesie.]
Sensorno iskustvo
Induktor Dodatno iskustvo
Sensorno iskustvo
Studija II: Sensorna priroda induktora
Serial visual search
Set size
Tim
e
Pop out
Serial search
Palmeri et al., PNAS, 2002
One synaesthete:
However, Edquist et al. (2006) could not replicate these results on a larger group of synaesthetes.
Larger sample than 1
• Six synaesthetes with grapheme-colour synaesthesia. • Synaesthesia proven by colour opponency task. • Also, highly consistent selection of colours over several
month period. • Four ‘associators’ and two ‘projectors’.• No “pre-selection” procedure. Everyone who responded to
our advertisement was taken into the study. The sample represents the population.
• Every synaesthete had different colours associated to digits 2 and 5.
• Two control subjects per synaesthete, matched by gender and age (±1 year).
Hypothesis 1: Only projectors can search fast
Hypothesis 2: Need to be aware that shapes represent digits
Two sessions:1st) Without instruction: No mention of digit shape; No instruction to use synaesthesia.
--half an hour break--
2nd) With instruction: subjects informed that shapes indicate digits and are asked to take advantage of synaesthetic colour associations.
Hypothesis 3: Fast processing of distractors
Familiar shapes increase the speed of normal visual search (i.e., without synaesthesia); Malinowski & Hübner (2001).
Res
pons
e ti
me
(ms)
Ciljni objekt Distraktor
Naša studija:
- Veći broj različitih podražaja:
Type of stimuli
×
Digit instruction
×
Synaesthesia
×
Target
5×2×2×2
Repeated measure design
Sample• The same six synaesthetes • No “pre-selection” procedure. Everyone who
responded to our advertisement was taken into the study. The sample represents the population.
• Highly consistent selection of colours over several month period.
• Four ‘associators’ and two ‘projectors’.• Two matched control subjects per synaesthete.
Results
Target-present synaesthetes
-500
500
1500
2500
3500
4500
0 10 20 30 40
Size
RT [m
s]
T D
0
20
40
60
80
100
120
1 2 3 4 5
Slo
pe
[ms/
elem
ent]
Synaesthetes
Controls
Target present
0
10
20
30
40
50
60
70
Synaesthetes Controls
Slo
pe
(ms/
elem
ent)
Overall average:
48
50
52
54
56
58
60
62
64
CG Controls
0
10
20
30
40
50
60
70
80
90
100
KT Controls
0
5
10
15
20
25
30
35
40
45
TQ Controls
0
5
10
15
20
25
30
35
40
45
50
KL Controls
0
10
20
30
40
50
60
70
IH Controls
54
56
58
60
62
64
66
NB Controls
Slo
pe (
ms/
elem
ent)
Slo
pe (
ms/
elem
ent)
Projectors Associators
0
20
40
60
80
100
120
140
2,5 rot M,2 2,5 4, inv4 inv4, 4
CGControl
0
20
40
60
80
100
120
140
2,5 rot M,2 2,5 4, inv4 inv4, 4
KTControl
0
10
20
30
40
50
60
70
80
90
2,5 rot M,2 2,5 4, inv4 inv4, 4
TQControl
0
20
40
60
80
100
120
2,5 rot M,2 2,5 4, inv4 inv4, 4
KLControl
0
20
40
60
80
100
120
2,5 rot M,2 2,5 4, inv4 inv4, 4
IHControl
0
20
40
60
80
100
120
140
2,5 rot M,2 2,5 4, inv4 inv4, 4
NBControl
Projectors Associators
Slo
pe (
ms/
elem
ent)
Slo
pe (
ms/
elem
ent)
0
5 0
1 0 0
1 5 0
2 0 0
2 5 0
3 0 0
C G
C o n tro ls
0
20
40
60
80
100
120
140
C G
C ontro ls
Target present Target absent
0
10
20
30
40
50
60
70
80
Session1 Session2
Slo
pes
(m
s/el
emen
t)
SynaesthetesControls
Instruction about shapes representing letters
Synaesthetes
0
20
40
60
80
100
120
2,5 rot M,2 2,5 4, inv4 inv4, 4
Session1Session2
Controls
0
20
40
60
80
100
120
2,5 rot M,2 2,5 4, inv4 inv4, 4
Session1
Session2
Slo
pe (
ms/
elem
ent)
Digit instruction, details:
Conclusions I
• Synaesthetes do not process distractors faster than do non-synaesthetes.
• An explicit instruction to use synaesthesia did not improve performance.
• There is nothing special about projectors regarding serial search.
We had to reject all three hypotheses:
Conclusions II
• Synaesthetes do not perceive perceptual ‘pop-out’ in serial search tasks.
• Synaesthetes are not, in any way, faster than non-synaesthetes in this task.
• Fast serial search is not typical of synaesthesia.
• … in agreement with Edquist et al. (2006).
• It is dangerous to use sample size = 1.
Implications
(…what does it mean)
Sensory Sensory
Models of synaesthesia
Inducer Concurrent
Definicije sinestezije
osjet koji se obično odvija u jednom osjetnom modalitetu, odvija se i kada se podražuje drugi modalitet
izmješanost osjetila
percipiranje sensornih podataka jednoga osjetila sa drugimjedno sensorno iskustvo opisano u terminima drugog
križanje osjetnih percepcija
Sensory Sensory
Implications
Models of synaesthesia
Inducer Concurrent
Serial search task
Colour opponency
task
Sensory Sensory
- Much evidence about sensory concurrent
Inducer Concurrent
- Evidence about sensory inducer?
?
Massive evidence for the role of semantic inducers
Dixon et al. (2000): 5 + 2 =
Simner and Ward (2006) Taste of words on the tip of the tongue.
Simner: crossover of senses
Berteletti et al.: dice
All our synaesthetes reported, that the shape “O” changes its colour depending on the context, i.e., whether it is perceived as a letter or as a digit zero. Therefore, the meaning determines the colour.
A E I O U O 1 2 3 4 5
Test for context
Three homographs (same spelling, different pronunciation, different meaning): - Montage (installation or Monday)- Collagen (colleges or binding agent)- Heroin (narcotic or female hero)
The colours depended on the pronunciation/meaning.
Studija III: Semantička reprezentacija
Semantička reprezent.
Induktor
Dodatno iskustvo
Sensorno iskustvo
Sensorno iskustvo
Test učenjem:
Novo slovo
Crvena bojaA
Glagoljica
Od 16 ispitanika, 14 je izjavilo da je nakon kraće ili duže vježbe novo glagoljičko slovo dobilo sinestetsku boju originalnog latiničkog slova.
Obično je 5 do 10 minuta dovoljno.
Semantička reprezent.
Induktor
Dodatno iskustvo
Sensorno iskustvo
Sensorno iskustvo
Zaključak:
Sensorno iskustvo
Induktor Dodatno iskustvo
Sensorno iskustvo
Klasični model:
Rezultati sugeriraju:
Semantička reprezent.
Induktor
Dodatno iskustvo
Sensorno iskustvo
Sensorno iskustvo
Definicije sinestezije
osjet koji se obično odvija u jednom osjetnom modalitetu, odvija se i kada se podražuje drugi modalitet
izmješanost osjetila
percipiranje sensornih podataka jednoga osjetila sa drugimjedno sensorno iskustvo opisano u terminima drugog
križanje osjetnih percepcija
SinestezijaPercepcija značenja
Osjećanje koncepata
Za ne-sinestete koncepti us abstraktni
Sinestete: koncepti postaju konkretna perceptivna iskustva
(zamenke, imena, melodije)
Sinestezija
Grčki: (syn) ujedinjenje + (aesthesis) osjeti
“Ujedinjenje osjetila”
“Osjećanje ideja”
Grčki (idea) ideja, koncept + (aesthesis) osjeti
Ideaestezija
Sensory Sensory
Model of synaesthesia
Semantic
Inducer
Concurrent
Sensory Sensory
Semantic !
Hvala na pažnji