The Effects of Phonological and Semantic Information on Color Perception in Grapheme-color SynesthesiaJuHyun LEE, Graduate School of Joshibi University of Art and DesignKatsuaki SAKATA, Joshibi University of Art and Design

ligtAn ideogram

Responses of three cones

LGN

Visual cortex 1

Visual cortex 4

Response to wavelength

Cardinal axis (bluelish-red-cyan, lavender-lime)

Color opponency(red-green, yellow-blue)

The mechanism of color vision

The levels of color processing

The levels of letter processing

Phonology

Semantics

Morphology

Color category

Unique Hue

interaction?

In summary, grapheme-color synesthesia of ideograms likely occurs due to an interaction between opponent processes in color perception and semantic and/or phonologic processing.

(1) Homophonic characters pronounced /ku/and /kou/ (2) Comparing identical phonology and identical meaning (3) Identical meaning with a phonogram and an ideogram

1. Introduction

2. Experiment

3. Results

Reference

These results suggest that CIE L*a*b* color space is good for systematically evaluating synesthetic colors. For instance, most synesthetic colors were plotted along the yellow-blue axis, although there were different hues for the same conditions. This means that there was a wide range of synesthetic color within the yellow-blue axis compared to the red-green axis. Systematic color change in color space presumably indicates the level of synesthesia, because CIE L*a*b* color space is based on the chromatic appearance of opponency proposed by E. Hering. These results imply that letter processing might interact with the opponent-process of color vision and/or later stages. The groups with the same hue but different saturation indicate the involvement of early processing stages related to categorical color perception.

1) Bridgeman, B., Wintero, D., Tseng,P.(2010) Dynamic phenomenology of grapheme-color synesthesia. Perception, vol 39, 671-676.2) David Branga, Romke Rouwb, V.S. Ramachandrana, Seana Coulsona. (2011) Similarly shaped letters evoke similar colors in grapheme-color synesthesia. Neuropsychologia, 49, 1355-1358.3) Ilaria Bertelettia, Edward M. Hubbardb, Marco Zorzi. (2010) Implicit versus explicit interference effects in a number-color synesthete, Cortex, 46(2), 170-177.4) Jamie Ward. (2004). Emotionally Mediated Synesthesia. Cognitive Neuropsychology, 21 (7), 761-772.

1.1. Morphological forms in grapheme-color synesthesia The letters with similar feature of morphological forms within a language (Brang et al., 2011, Watson et al., 2012), and between different languages (Witthoft & Winawer, 2006)

The identical synesthetic colors were elicited by several linguistic factors such as meaning, phonology, and orthography. They said synesthetic color effect environment of the language-learning (Asano and Yokozawa, 2012).

1.2. Phonological information in grapheme-color synesthesia The similar phonology of alphabets between English and Cyrillic (Witthoft & Winawer, 2006) : Synesthetic color were not elicited by non-learned language (Mroczko et al., 2009). The letters with identical consonants in Japanese(Asano&Yokosawa, 2011), and in Norwegian alphabet(Svartdal&Iversen, 1989)

Japanese = ideographic character (Kanji) and phonographic characters (Hiragana and Katakana).

1.3. Sementic information and/or emotional connotation in synesthesia The incongruent condition of digit stroop task and Arithmetic Stroop task in which color patches were presented instead of calculated values produced longer response times(Dixon et al., 2000). Dot patterns like dice also produced the longer response time even if synesthete did not experience synesthetic color (berteletti et al., 2010). Higher percents of word-colored synesthetic responses were elicited by familiar words with synesthete, and emotional valence(positive vs. negative) had an influence on positive synesthetic colors (e.g., pink, orange etc.) and negative synesthetic colors (e.g., brown, grey etc.) (Ward, 2004).

A phonogram

Anideogram

In condition 1, similar synesthetic colors were elicited by identical consonants. This result is consistent with previous findings that synesthetic color is strongly influenced by consonants (Asano &Yokozawa, 2011). Additionally, the fact that different synesthetic colors were elicited by the same consonants suggests that other factors besides consonants affect synesthetic color. In condition 2, synesthetic colors elicited by ideograms were equivalent to the meaning of the stimuli, even though they had the same phonological features. These data suggest that synesthetic color is influenced by meaning rather than phonology. The results of condition 3 also support the influence of an ideogram's semantic information. However, each phonogram character in a word evoked its own distinct color, which was elicited by individual letters. This suggests that phonographic character might not affect how word meaning is determined from its context. In general, the factors affecting synesthetic color differed depending on the characteristics of the letters. Synesthetic color is most likely to be initially and strongly influenced by semantic information that implies direct meaning, otherwise synesthetic color is influenced by phonological rather than semantic information, like in a phonogram.

Letter processing

Color processing

The similar synesthetic colors were elicited by

2+3 =

If this color is sythesthic color of '7' ?

4. Discussion

5) Mike J. Dixon, Daniel Smilek, and Philip M. Merikle. (2004) Not all synaesthetes are created equal:Projector versus associator synaesthetes. Cognitive, Affective, & Behavioral Neuroscience, 4 (3), 335-343.6) Mroczko, A., Metzinger, T., Singer, W., & Nikolic L , D. (2009). Immediate transfer of synesthesia to a novel inducer. Journal of Vision, 9(12), 1-8.7) Michiko Asano, Kazuhiko Yokosawa.(2011) Synesthetic colors are elicited by sound quality in Japanese synesthetes. Consciousness and Cognition, Volume 20, Issue 4, 1816-1823.8) Michiko Asano, Kazuhiko Yokosawa. (2012) Synesthetic colors for Japanese late acquired graphemes. Consciousness and Cognition, 21(2), 983-993.9) Nathan witthoft and Jonathan winawer. (2006) Synesthetic color determind by having colored refrigerator magnets in childhood. Cortex, Vol 42(2), 175-183.10) Svartdal F., Iversen T. (1989) Consistency in synesthetic experience to vowels and consonants: Five case studies. Scandinavian Journal of Psychology, 30, 220-227.

<Arithmetic Stroop task> <digit stroop task>

All subjects did not always experience same synesthetic color to same stimuli but there was little variation within subjects. Therefore each subject rating data for each letter averaged over all three testing session plotted on CIE 1976 L*a*b* Uniform Color Space presented in Figure 1-3. Similarity of synesthetic color was measured by Euclidean distance on a*b* plane surface (ab chroma Cab*) Two subjects experienced two or three synesthetic color on one stimuli（IT：「医」「校」、MT：「苦」

「句」「校」）and picked the strong one by doing their questionnaire. One subject did not experience the stimulus(IT:「工」). So we excluded the data from further analysis.

2.1. Participants : Three female grapheme color synesthetes, IT, MT, AH with normal color vision. All reported experiencing synesthetic colors when viewing Hiragana, Katakana and Kanji characters as well when viewing letters of the English alphabet and Arabic numerals.

2.2. Experimental apparatus : Stimuli were presented in Macbeth Judge II in dark room. illuminant was D65 and illumination was 1100 lx. Viewing distance was kept at 75cm.2.3. Stimuli : All letters were printed out in black on a white background and its typeface was Gothic. They were subtending a visual angle of 4.6 degree×6.5 degree. Each session comprised of 19 trials and was repeated three times and sessions were separated by intervals of at least 1 month.

phonology

meaning

orthography Morphological forms did not affect to synesthetic color (Lee and Sakata, 2012).

which stage of color perception is influenced by letter processing by comparing ideograms' phonologic and semantic information.

In summary, grapheme-color synesthesia of ideograms likely occurs due to an interaction between opponent processes in color perception and semantic and/or phonologic processing.

Ni

Q 9 九 ニ 二 Ⅱ2

phonology

meaning

stimuli

The type ofgrapheme

Kyu

KanjiKanjiEnglish alphabet

Arabic numeral

Roman Numeral

Roman Numeral

Kata-kana

twonine

あかhiragana (phonogram)

kanji (ideogram)

meaning

akaphonology

red

赤

blue

ao

あお

青

The letters with an identical consonants

句 九 工 高 校 光

phonology

stimuli

meaning distressed phrase nine engineers high lightcollege

苦

Kou 「こう」Ku 「く」

< synesthetic color of color name in all subjects >

<meaning> IT: [F(1,13)=91.25, p<.001]<phonology> IT: [F(1,37)=16.12, p<.001] < meaning> MT: [F(1,13)=21.15, p<.001]

< meaning> AH: [F(1,13)=12.05, p<.005]

b*

a*

-100

-50

0

50

100

-100 -50 0 50 100

句 九

苦

光

ニ (Katakana)

Ⅱ

2

校工,

高

二 (kanji)

9

Q

b*

a*

-100

-50

0

50

100

-100 -50 0 50 100

句

九, 9

苦 光

Ⅱ, 2

校

高

二 (kanji)

ニ (Katakana)

Q

b*

-100

-50

0

50

100

-100 -50 0 50 100

句

9九

苦

光

校

高

二 (kanji)

ニ (Katakana)

Ⅱ

2工

Q

100

a*

b*

-100

-50

0

50

100

-100 -50 0 50 100

MT 光 (light)

IT 赤 (red)

AH 赤 (red)

MT 赤 (red)

MT 青 (blue)

AH 青 (blue)

IT 青 (blue)

AH 光 (light)

IT 光 (light)

a*