an fmri study of scrambling effects on sentence comprehension

Journal of Neurolinguistics 22 (2009) 151e166www.elsevier.com/locate/jneuroling

Scrambling effects on the processing of Japanesesentences: An fMRI study

Jungho Kim a,b,*, Masatoshi Koizumi a,g, Naho Ikuta d, YuichiroFukumitsu a, Naoki Kimura a, Kazuki Iwata e, Jobu Watanabe f, Satoru

Yokoyama d, Shigeru Sato c, Kaoru Horie c, Ryuta Kawashima d,g

a Graduate School of Arts and Letters, Tohoku University, Sendai, Japanb Postdoctoral Fellowships for Foreign Researchers, Japan Society for the Promotion of Science (JSPS), Tokyo, Japan

c Graduate School of International Cultural Studies, Tohoku University, Sendai, Japand Department of Functional Brain Imaging, IDAC, Tohoku University, Sendai, Japan

e LBC Research Center, Tohoku University 21st Century Center of Excellence Program in Humanities, Sendai, Japanf Consolidated Research Institute for Advanced Science and Medical Care, Waseda University, Tokyo, Japan

g RISTEX, JST, Tokyo, Japan

Received 11 January 2008; received in revised form 8 June 2008; accepted 26 July 2008

Abstract

The present study aims to confirm the cortical correlates of scrambling effects, a free word orderphenomenon that has been observed in a variety of cross-linguistic investigations but whose mechanismstill remains unclarified. Many syntax-oriented hypotheses on scrambling have been provided to developthe structural basis of the free word order permutation in Japanese, leading to the most recent phrasalarchitecture, in which the object noun phrase of a transitive sentence ‘‘moves’’ to a higher position thanthe subject to form an asymmetric structure including antecedentegap relationships. Such a configura-tional structure formed by scrambling operation predicts that the scrambled sentences have a morecomplex structure than canonical sentences, and that the former requires a greater burden on cognitiveprocesses in related areas within the brain. Based on this general assumption, we employed an experi-mental method of whole-sentence presentation of Japanese transitive sentences, for both canonical tran-sitive sentences (SubjecteObjecteVerb) and their scrambled counterparts (ObjecteSubjecteVerb). Theresult showed more activation at the left inferior frontal gyrus (IFG) and the left dorsal prefrontal cortex(DPFC) during the comprehension of scrambled sentences than that of canonical sentences. This indicates,in accordance with previous findings on scrambling from neurolinguistic perspectives, that the scrambling

* Corresponding author. Department of Linguistics, Graduate School of Arts and Letters, Tohoku University,

Kawauchi, Aoba-ku, Sendai-shi, Miyagi-ken 980-8576, Japan. Tel./fax: þ81 22 795 5983.

E-mail address: [email protected] (J. Kim).

0911-6044/$ - see front matter � 2008 Elsevier Ltd. All rights reserved.

doi:10.1016/j.jneuroling.2008.07.005

mailto:[email protected]

http://www.elsevier.com/locate/jneuroling

152 J. Kim et al. / Journal of Neurolinguistics 22 (2009) 151e166

in Japanese is indeed one of the grammatical operations and that the parsing strategy for the asymmetricantecedentegap relationship demands an additional cognitive activation in the brain.� 2008 Elsevier Ltd. All rights reserved.

Keywords: Scrambling; Word order; Sentence comprehension; fMRI

1. Introduction

A rapidly increasing number of studies that focus on the neural basis of language cognitionhave recently shown that the processing of grammatical information induces cortical activationaround specific brain regions that show linguistic activities. Mounting evidence suggests thatthe left inferior frontal gyrus (left IFG) is responsible for syntactic processing across languages(Ben-Shachar, Palti, & Grodzinsky, 2004; Caplan, 2001; Dapretto & Bookheimer, 1999;Indefrey, Hagoort, Herzog, Seitz, & Brown, 2001; Newman, Just, Keller, Roth, & Carpenter,2003; Sakai, Hashimoto, & Homae, 2001) while more recent studies show that the corre-sponding region also engages in the processing of semantic information such as animacy effects(Grewe et al., 2006, 2007). Despite the findings indicating that linguistic complexity producesan increase of activity in Broca’s and Wernicke’s areas (Just, Carpenter, Keller, Eddy, &Thulborn, 1996), issues still seem unclarified as to what type of sentence construction in whichlanguages is realized as a cortical activation associated with specific areas in the left hemi-sphere. In languages such as English, formation of an alternative sentence construction is quiteprominent; in an interrogative sentence, for example, a Wh-phrase and an auxiliary move to theleft periphery of a sentence so that the movement yields more complex construction than thebasic declarative sentence. On the other hand, languages such as German and Japanese allow‘‘scrambling’’ which represents the free permutation of noun phrases. Although there have beena number of imaging studies that dealt with this particular grammatical operation in German,imaging experiment concerning Japanese is almost completely absent at this point of time. Thepresent study therefore aims to offer an insight into these issues, by investigating neuralcorrelates of the processing of scrambling in Japanese.

Japanese is a strictly verb-final language, and either the subject or object noun phrase intransitive construction can precede one another. More specifically, the order of SubjecteObjecteVerb (SOV) as in John-ga ringo-o tabe-ta (John-Nominative (Nom) apple-Accusative(Acc) eat-PAST) can be converted, or ‘‘scrambled’’ into the order of ObjecteSubjecteVerb(OSV) as in Ringo-o John-ga tabe-ta (apple-Acc John-Nom eat-PAST) without interfering withthe grammaticality or truth value of a sentence. In this sense, although sentence initialconstituents receive focus for pragmatic reasons, Japanese scrambling is completely semanti-cally vacuous. Due to this trait of free word order phenomenon in Japanese, it was onceconsidered that Japanese has a flat, nonconfigurational syntactic structure, which enables theunrestricted switchover of the word order as in [S John-ga ringo-o tabe-ta] and [S Ringo-o John-ga tabe-ta] (‘‘S’’ stands for the highest sentential node). It was first proposed by Hale (1983)that human languages can be typologically classified into either configurational or non-configurational e the former has a hierarchical structure and requires a strictly rigid word order(e.g. English), while the latter has a flat structure and allows a rather free permutation of nounphrases (e.g. Warlpiri). This proposal had once prevailed as a theory that can satisfactorilyexplain various linguistic phenomena. Indeed, Hale’s hypothesis had a strong influence on the

153J. Kim et al. / Journal of Neurolinguistics 22 (2009) 151e166

analyses for languages that permit scrambling, such as Japanese, Korean, Hindi, German, andTurkish. However, the most recent view in the field of theoretical linguistics regards Japanese asa configurational language, which has now become a standard view since the mid-1980s. It isargued that Japanese has a hierarchical, asymmetric structure in which the object in the OSVorder is associated with two positions: the one in front of the subject, which is pronounced, andthe one inside the verb phrase (VP), called a gap or trace (indicated by t in the followingschematic structure), which is not pronounced but has a grammatical function: [S Ringo-oi

[S John-ga [VP ti tabe-ta ]]]. Under the postulation of a configurational structure in Japanese,a scrambled OSV order is ‘‘derived’’ from a structure similar to its canonical SOV counterpart;a movement operation of the object noun phrase from its basic position in VP to a higherposition is postulated. It can be consequently assumed that this process makes the structure ofthe scrambled OSV order more complex than that of the canonical SOV order. Obviously, thediscussion on the configurationality of natural languages, including Japanese, is still in prog-ress, and the grammatical peculiarity of scrambling may provide important clues that could leadto the clarification of universal linguistic competence in humans.

Although the asymmetric configuration is undoubtedly one of the universal constraint, thereare some cross-linguistic difference with regard to grammatical characteristic of scrambling.Scrambling in East Asian languages such as Japanese is peculiar in the following two senses;(1) it lacks motivation of verbal agreement, and (2) it can cross the clausal boundaries. First,Japanese scrambling differs from that of agreement languages such as German with respect tothe non-existence of a requirement of agreement. While German has number and genderagreement on nouns and verbs, such grammatical clue is absent in Japanese, which leaves caseparticles and thematic-roles to be the crucial factors for the comprehension of a sentence.Secondly, Japanese allows scrambling of noun phrases out of multiple embedded clauses. Onthe other hand, German scrambling is domain-sensitive, which means that the range in whichscrambling can be applied is quite limited than Japanese. German ‘‘middle-field’’ andembedded clauses are the instances of such domains from which noun phrases cannot beextracted. Since Japanese scrambling seems to behave differently from other scramblinglanguages, it is at least worth pursuing whether scrambling in this particular language conformsto the results of the imaging experiments based on other scrambling languages.

In addition to the linguistic studies that show asymmetric structure in Japanese scrambledsentences, behavioral data from psycholinguistic experiments have proved that the clausalarchitecture of Japanese is no doubt configurational. For example, Aoshima, Phillips, andWeinberg (2004) demonstrated that the configuration between a scrambled Wh-phrase anda gap has an influence on reaction time in that the complex configuration involves slowerresponse by the parsers. Additionally, Koizumi and Tamaoka (2004) and Tamaoka et al. (2005)showed that the scrambling in a variety of sentence constructions in Japanese such as ditran-sitive construction, passive construction, and dative-subject construction induces longer reac-tion time. Because scrambling yields a more complex structure and an additional movementprocess, the comprehension of scrambled sentences presumably leads to a longer reading timethan that of canonical sentences, which have a simple structure and do not require movement.As is expected, previous research showed the effect of filleregap dependency: during anincremental parsing of experimental sentences: human parsers rapidly detect any types ofsyntactically replaced constituent, store them in their temporal memory, and retrieve theinformation of the constituents at the point of the encounter with the gap or trace, namely,the original position of the removed elements within sentences. Several psycholinguisticstudies have revealed this cognitive interaction between the structural complexity caused by


scrambling and the filleregap parsing strategy (Mazuka, Ito, & Kondo, 2002; Miyamoto &Takahashi, 2002; Nakano, Felser, & Clahsen, 2002). Incidentally, one may wonder whether thiscan be accounted for by the factor of frequency; less frequent the sentence form is, the morecognitive burden associated with the longer reading times. This, however, cannot be upheld.According to the finding by Kaiser and Trueswell (2004), factors such as frequency andcontextual support for the scrambled noun phrases do reduce the processing difficulty of thenon-canonical sentences, but they could not completely neutralize the difficulty with thescrambled sentences. This indicates that the complexity of sentence structure is the main factorfor the longer reading times in psycholinguistic studies.

When we further take neurolinguistic issues into account in accordance with the psycho-linguistic investigations on the human computational systems, two fundamental questions ariseat this point as to the comprehension of scrambled sentences in Japanese: What are our neuralcorrelates that regulate movement operation such as scrambling in a free word order language?Are the neural effects of movement evoked by the scrambling the same as those observed inother types of movement such as the WH movement, topicalization, and cleft construction? Toour knowledge there have been few discussions in the literature that intrinsically pursued thecorrelation between cortical activities and scrambling effects in Japanese. In addition to theresults provided by previous psycholinguistic studies, an independent support from the neu-rolinguistic perspective seems to be necessary for the clarification of the scramblingphenomenon and its effects on neural networks.

There is rich background for the issue on relation between alternative word order and neuralcorrelates in English and German. For example, Ben-Shachar et al. (2004) showed that thesyntactic transformation required for English object extraction demands higher activation inBroca’s area. This result conforms to the previous psycholinguistic studies in that the syntacticcomplexity involves increase in the working memory and, consequently, more activation in thebrain. For German, Bornkessel, Zysset, Friederici, von Cramon, and Schlesewsky (2005) andGrewe et al. (2006) claimed that the activation in Broca’s area cannot be attributed only to thepurely syntactic factors but other language-specific principles such as pronominal preferencefor subjects, thematic hierarchy, and animacy also involve the activation in the associated brainarea. Therefore they suggest that the definition of Broca’s area being purely syntactic processoris too narrow. It is therefore interesting to see how these discussions can be applied to Japanese,whose grammatical property is crucially different from English and German in many respects.Unfortunately, however, only few imaging studies on Japanese scrambling have been so farconducted. For example, Inui et al. (1998) suggested that activation in the left IFG involves thereanalysis of constituents from a matrix clause to a relative clause. However, the pure syntac-tically oriented scrambling effects were not considered in their fMRI study for their materials donot include simple transitive sentences. Ueno and Kluender (2003) detected localized potentialsfrom the filleregap dependencies between scrambled constituents and their traces, but theclarification of in-depth spatial resolution was arguably not their intention because theirobservation was based on an ERP experiment in which they focused more on temporal reso-lution. Our main interest is whether scrambling in simple transitive sentences at a spontaneouspace of comprehension coincides with the model of incremental filleregap parsing in theconfigurational structure. It should be also emphasized that the experimental methodologydifferentiates between ours and the previous imaging studies of the comprehension of Japanese.If scrambled sentences have a more complex structure than canonical sentences, and thescrambling operation involves an additional syntactic movement, then more activation in the leftIFG is reasonably expected during the comprehension of scrambled sentences than during that of


canonical sentences. This expectation was borne out in the current study, suggesting that the leftIFG is crucially involved in the processing of alternative word orders.

2. Materials and methods

2.1. Participants

Thirty-six undergraduate and graduate students from Tohoku University (28 males, 8females, mean age¼ 20.8 years old, SD¼�1.72) participated in the experiment. All wereright-handed, healthy native speakers of Japanese. The handedness survey was based onEdinburgh Handedness Inventory (Oldfield, 1971). All subjects were paid U4000 (equivalent toUS$38.10) for their participation. The whole session took approximately 2 h. Before theexperiment, we provided participants with a sufficient explanation of the entire experiment andits safeness, based on the guidelines of Tohoku University. Only participants from whom weobtained their written informed consent took part in the actual experiment.

2.2. Materials and design

A session consisted of four conditions: canonical sentence (CS) condition, scrambled sentence(SS) condition, word condition for word lists (WL), and rest (R) condition (see Table 1, Fig. 1). Inour experiment, stimuli were presented in whole-sentence-presentation method, and all stimuliequally contained kanji, katakana and hiragana, to make the stimuli look most natural. Thesubjects were asked to judge the semantic plausibility of the stimuli. The CS and SS conditionsincluded 42 sentences: 28 semantically correct sentences and 14 semantically unnatural oranomalous sentences. To avoid the possibility of the participants’ semantic judgment before theencounter of the verb, we prepared the stimuli in such a way that semantic anomalies can only bedetected at the point of the verb. Of the 14 semantically incorrect sentences, 7 were in violation ofthe subjecteverb congruity, and the other 7 were in violation of the objecteverb congruity. Thestimuli for the scrambled and canonical condition were distributed into several lists: while oneparticipant saw ‘‘Yoji-o obasan-ga tasuketa (baby-Acc woman-Nom saved: SS)’’, anotherparticipant saw its counterpart ‘‘Obasan-ga yoji-o tasuketa (woman-Nom baby-Acc saved: CS)’’.All incorrect sentences as well as correct sentences in the CS condition have counterparts in theSS condition, and the plausibility of the stimuli is controlled for the two conditions. In the CS andSS conditions, the participants were instructed to determine whether the sentences they just readmade sense, and to respond by using the index finger of their right hand to make a ‘yes’ responseon a keypad or the middle finger of their right hand to make a ‘no’ response. We measured thereaction time between the time of whole-sentence presentation and the time of presentation of thefollowing fixation cross (maximum¼ 4500 ms).

The WL condition contained 42 sequences of words. Each sequence in WL condition con-tained three noun phrases with unified case particles or three verbs in the past tense. A stimuluswith different three words was treated as a ‘‘correct’’ stimulus, whereas a stimulus including a pairof the same words was regarded as an ‘‘incorrect’’ stimulus. In the WL condition, subjects wereinstructed to determine whether the sequence just presented included three different phrases. Inthe R condition we presented a fixation cross at the center of the screen. The experiment wasbased on blocked design: each block included 6 sentences or 6 word lists and was repeated seventimes. Time-length of a block was 51 s, with 20 s of R condition between the task blocks. In the Rcondition block, subjects were asked to look at a fixation cross and to rest for 20 s. Taking into

Table 1

Sample of stimuli used in CS, SS, and WL conditions

(1) Canonical sentence (CS) condition

28 Correct SOV sentences

Woman-Nom Baby-Acc Saved

‘‘The woman saved the baby.’’

14 Incorrect SOV sentences

Nurse-Nom Pet-Acc Employed

‘‘The nurse employed the pet.’’

(2) Scrambled sentence (SS) condition

28 Correct OSV sentences

Baby-Acc Woman-Nom Saved

‘‘The woman saved the baby.’’

14 Incorrect OSV sentences

Pet-Acc Nurse-Nom Employed

‘‘The nurse employed the pet.’’

(3) Word list (WL) condition

28 Correct word lists

Mother-Nom Police-Nom Friend-Nom

14 Incorrect word lists

Arranged Searched Arranged

‘‘Nom’’ and ‘‘Acc’’ are abbreviations for nominative, and accusative, respectively. S stands for subject, O stands for

object, and V stands for verb. In Japanese, case particles indicate subject, object and dative when attached to a noun.

NP-ga and NP-o refers to a nominative case-marked noun phrase, an accusative case-marked noun phrase, respectively.


account the participants’ ease of adapting to the environment inside an MRI scanner, the firstblock was designed to consist of sentences that were not a part of the target; it was not included inthe final statistical analysis. Consequently, all stimuli were pseudorandomized for each partici-pant. At the beginning of each block, either (sentence task) or (word task) was presentedto let the participants know the type of task that follows in the blocks. All stimuli were controlledusing Director 8.5 (Macromedia Inc., San Francisco, CA).

2.3. fMRI data acquisition

Data recordings were obtained by fMRI using a 1.5 Tesla Siemens Symphony scanner(Siemens, Erlangen, Germany) at Tohoku University, under the preliminarily determinedcondition in Echo Planar Imaging (TR: 3800 ms; TE: 50 ms; FOV: 192 mm; slices: 42;thickness: 3 mm; data matrix: 64� 64 voxels; flip angle¼ 90�). After the attainment offunctional imaging, anatomical T1-weighted MDEFT images (thickness: 1 mm; FOV: 256 mm;data matrix: 192� 224; TR: 1900 ms; TE: 3.93 ms) were also acquired from all participants.

2.4. fMRI data analysis

To specify the cortical region exhibiting activation during the various tasks, statisticalanalyses of image processing were carried out using SPM2, operating on a MatLab platform,

Fig. 1. Design of three tasks for experiment. In the CS and SS conditions, the participants were instructed to determine

whether the sentences they just read made sense, and to respond by pressing one of two buttons (‘Yes’ and ‘No’). In the

WL condition, subjects were instructed to determine whether the sequence just presented included three different

phrases. Reaction time was measured from the time of whole-sentence presentation or three-word phrases to the time of

the participants’ responses, but no later than the next fixation cross (maximum¼ 4500 ms). In the R condition we

presented a fixation cross at the center of the screen. Note that the letters in the actual experiment were presented in

yellow colour. (For interpretation of colour, please refer to the web version.)


developed by the Wellcome Department of Cognitive Neurology. The origin of EPI andT1-weighted anatomical images were synchronized to realign the head motion. The realigneddatasets were spatially normalized to the Montreal Neurological Institute (MNI) standard braintemplate, and then we performed smoothing of images using a 9 mm Gaussian filter. Ananalysis of the tasks for each participant was conducted at the first statistical stage and groupstatistical analysis at the second stage. To exclude the factor of difficulty in CS, SS and WL taskaffecting the range of brain activation, we carried out a statistical processing by ANCOVA,which includes the confounding covariate of response time and error rate for each participant inall three tasks (CS, SS, WL). Contrasts between the SS vs. CS and CS vs. SS conditions werecalculated by one sample t tests (n¼ 36) and masked by the SS vs. WL and CS vs. WL(significance threshold for masking was p< .05 uncorrected). We set the statistical threshold atp< .05 (corrected for family wise error rate (FWE)).

CS minus R determines the specific cortical activation associated with the comprehension ofcanonical sentences, whereas SS minus R determines the activation inherent to the compre-hension of scrambled sentences. For CS minus WL and SS minus WL, we can exclude theeffects of the lexical factor on sentence processing, so that the activation obtained from theanalysis is interpreted with reference to word order processing only. The locations of corticalactivation specified by the interconditional subtraction were closely mapped out on the basis ofthe Co-planar stereotaxic atlas of the human brain (Talairach & Tournoux, 1998).

3. Results

3.1. Behavioral data

A series of the analysis of variance including all three conditions showed significantdifferences both in reaction time [F(2,70)¼ 72.5, p< .001] and error rate [F(2,70)¼ 6.0,


p< .005]. As shown in Fig. 2, the comparison of reaction time between all possible pairsshowed significant differences: [F(1,35)¼ 57.9, p< .001] for CS vs. SS, [F(1,35)¼ 55.2,p< .001] for CS vs. WL, and [F(1,35)¼ 85.8, p< .001] for SS vs. WL. The comparison oferror rate between CS vs. SS and SS vs. WL showed significant differences [F(1,35)¼ 5.2,p< .05] [F(1,35)¼ 8.7, p< .01], but not for CS vs. WL [F(1,35)¼ 2.9, p¼ .099].

3.2. Imaging data

Fig. 3 and Table 2 show the activated areas in each condition. From the direct comparisonbetween CS vs. R and SS vs. R, we identified the activated brain regions involved in theprocessing of canonical and scrambled sentences, respectively. The result showed activations inthe bilateral inferior frontal gyri and left middle temporal gyrus. The activations around the leftmotor area and left fusiform gyrus are possibly due to the actions of button pressing and theperception of word forms and characters, respectively. Furthermore we conducted an analysis ofWL vs. R, and the result is provided in Table 2 and Fig. 3. As shown in Table 2 and Fig. 3, wedid not observe activation in middle temporal gyrus (MTG), though the activation in MTG wasrealized in the comparison between CS vs. R or SS vs. R. It is known that MTG is involved inlexical-semantic processing and sentence processing (Demonet et al., 1992; Price, Moore,Humphreys, & Wise, 1997; Wise et al., 1991). The purpose of the WL condition in our study,however, was to have the participants judge whether the three words were completely differentagainst one another in terms of their surface forms. As a result, the participants were notactually engaged in sentential or semantic processing.

To identify the brain regions associated with the processing at the whole-sentence level, wedirectly compared CS vs. WL and SS vs. WL. The subtraction revealed activations in the leftIFG, left middle temporal gyrus (MTG), and left superior temporal gyrus (STG). In the CS andSS conditions (each compared with the R condition and WL condition), similar regions were

Fig. 2. Behavioral data. (a) Reaction time (ms); a series of the analysis of variance including all three conditions

showed significant difference [F(2,70)¼ 72.5, p< .001]. (b) Error rate (%): there was a significant difference between

CS and SS, SS and WL conditions [F(1,35)¼ 5.19, p< .05] [F(1,35)¼ 8.68, p< .01], but CS and WL conditions

showed no significant difference [F(1,35)¼ 2.87, p¼ .09]. Stars denote a significant difference between the conditions

(n.s.¼ non-significant, *p< .05; **p< .01; ****p< .001).

Fig. 3. Activated brain regions identified by comparisons between CS vs. R, SS vs. R, WL vs. R, CS vs. WL, and SS vs.

WL conditions. We set the statistical threshold at p< .05 (FWE).


activated including Broca’s and Wernicke’s areas. This suggests that most cognitive processesinvolved in the comprehension of scrambled sentences are also involved in the comprehensionof canonical sentences. The activation in MTG indicates that the area involves the perception of‘‘sentences’’ rather than non-sentential task-demands such as word list judgment. This alsoconforms to the previous finding that the area also plays a role in semantic processing(Humphries, Binder, Medler, & Liebenthal, 2007). Finally, and most importantly, we carriedout a direct comparison of CS vs. SS and SS vs. CS to determine the brain regions that areinvolved in the processing of scrambled sentences. The former comparison showed no acti-vation in any region, while the latter revealed activations in the left IFG and left dorsalprefrontal cortex (DPFC) (Fig. 4).

4. Discussion

Our experiment induced apparent activation in the left IFG and related areas. Because it isassumed in the linguistic and psycholinguistic literature that scrambling is a grammaticaloperation that creates an asymmetric complex structure, our fundamental premise was that thestructure formation requires additional cognitive processes such as a gap-filling parsing. Thecurrent results thus support the idea that scrambled sentences in Japanese have a more complexstructure due to the presence of a filleregap dependency. Moreover, the observed activation inthe left IFG is consistent with the proposal that the region plays a role in syntactic processing,as noted earlier (Embick, Marantz, Miyashita, O’Neil, & Sakai, 2000; Hagoort, 2005; Justet al., 1996; Musso et al., 2003; Sakai, Homae, & Hashimoto, 2003). However, there remainsome points that need to be clarified. Below we examine in detail the activated regionsdetermined in our studies.

4.1. Left inferior frontal gyrus

The left IFG, also called Broca’s area, is known as the region for linguistic comprehensionincluding the processing of syntactic information (Caplan, Alpert, & Waters, 1998, 1999;

Table 2

Activated brain regions

Brain regions Talairach coordinates t-Value BA Brain regions Talairach coordinates t-Value BA

x y z x y z

CS vs. R SS vs. R

L. medial frontal gyrus �4 4 60 17.12 6 L. middle frontal gyrus �40 0 46 14.35 6

L. middle frontal gyrus �40 �2 42 12.43 6 �52 20 28 12.36 9

R. middle frontal gyrus 54 36 26 7.93 46 R. middle frontal gyrus 58 30 24 11.04 46

58 28 24 7.64 46 52 36 22 10.78 46

56 32 16 6.58 46 36 �4 62 7.25 6

R. superior frontal gyrus 6 16 48 13.99 6/8 52 42 �10 6.69 10/47

L. inferior frontal gyrus �42 10 30 11.78 9/44 Superior frontal gyrus 0 8 60 16.69 6

�56 28 �2 10.85 47 L. superior frontal gyrus �6 0 68 13.4 6

R. inferior frontal gyrus 34 24 2 8.27 47 R. superior frontal gyrus 8 18 48 12.61 8

34 26 �8 7.53 47 L. inferior frontal gyrus �42 4 34 12.82 6/9

44 10 30 7.82 9/44 R. inferior frontal gyrus 46 10 32 9.53 9/44

L. superior parietal lobule �28 �62 46 12.01 7 L. superior parietal lobule �30 �62 46 11.68 7

R. superior parietal lobule 32 �62 48 9.03 7 R. superior parietal lobule 32 �64 52 9.94 7

L. middle temporal gyrus �58 �38 2 8.27 21 L. middle temporal gyrus �54 �38 0 7.06 21

L. inferior occipital gyrus �18 �92 �10 16.89 17 �62 �30 2 6.66 21

R. uvula 32 �62 �26 12.41 L. inferior occipital gyrus �18 �92 �10 15.95 17

L. lentiform nucleus �18 0 12 10.26 L. lentiform nucleus �18 6 4 8.15

L. thalamus �14 �18 10 6.73 L. thalamus �12 �16 8 7.05

R. cuneus 22 �92 0 12.05 R. thalamus 10 �10 12 6.25

L. extra-nuclear �20 �18 �10 7.05 L. extra-nuclear �18 �4 12 11.6

R. extra-nuclear 14 4 12 6.53 R. extra-nuclear 14 2 12 6.86

R. insula 32 24 0 9.94 47

L. declive �16 �78 �18 12.39

R. lingual gyrus 20 �92 �4 12.79 17

WL vs. R CS vs. WL

L. medial frontal gyrus �6 2 60 15.33 6 L. inferior frontal gyrus �56 28 4 8.28 45

R. middle frontal gyrus 34 0 64 9.04 6 �44 26 �12 7.91 47

50 36 22 6.6 46 �50 20 16 7.68 44/45

L. superior frontal gyrus �4 16 50 10.53 8 L. middle temporal gyrus �58 �40 4 7.48 21/22

R. superior frontal gyrus 10 0 70 6.34 6 �68 �46 2 5.98 21

44 36 30 6.93 9 �54 0 �20 6.25 21

L. superior parietal lobule �30 �60 45 11.72 7 �60 �2 �10 6.13 21

16

0J.

Kim

etal.

/Journal

ofN

eurolinguistics22

(2009)151e

166

R. superior parietal lobule 30 �62 52 10.29 7 L. superior temporal gyrus �60 �28 2 6.96 21/22

30 �64 42 9.25 7 L. parahippocampal gyrus �22 �14 �14 7.66 28

L. inferior parietal lobule �48 �34 46 12.77 40

R. inferior parietal lobule 54 �42 52 6.15 40 SS vs. WL

L. precentral gyrus �36 �20 66 11.63 4/6 L. inferior frontal gyrus �54 26 8 9.61 45/46

L. lingual gyrus �12 �86 �4 10.84 17/18 �52 38 0 8.21 45/46

R. lingual gyrus 18 �86 �2 8.8 17/18 L. middle temporal gyrus �56 �44 4 9.18 21/22

L. declive of vermis �2 �74 �18 9.7 �46 �62 22 7.14 39

L. extra-nuclear �18 �10 18 10.6 L. superior temporal gyrus �62 �28 2 7.5 21/22

R. inferior frontal gyrus 46 8 32 8.37 44 L. middle frontal gyrus �52 24 24 11.26 9/46

R. culmen 18 �48 �20 7.27 �40 0 46 8.86 6

12 �52 �16 7.02 R. middle frontal gyrus 56 28 22 7.46 45/46

L. superior frontal gyrus �4 12 64 6.74 6

R. pyramis 12 �80 �30 7.05

R. uvula 16 �72 �26 6.8

CS vs. SS SS vs. CS

No significant activation L. dorsal prefrontal cortex �42 0 50 7.79 6

L. inferior frontal gyrus �54 26 20 6.51 45/46

The comparisons between CS vs. R, SS vs. R, WL vs. R, CS vs. WL, SS vs. WL, CS vs. SS, and SS vs. CS. For each contrast, the activated anatomical region, the right or left

(R, L) hemisphere, approximate Brodmann areas, t values, and coordinates of the local maxima of the significance coordinate system are given (p < .05, corrected (FWE)).

x, y, and z represent Talairach and Tournoux coordinates (Talairach & Tournoux, 1998). The extend threshold was set at k ¼ 20.

16

1J.

Kim

etal.

/Journal

ofN

eurolinguistics22

(2009)151e

166

Fig. 4. Brain activation detected by SS vs. CS comparison. (a, c) maps in the Talairach space show activations

determined by the SS vs. CS comparison. (b) Stands for the result of ROI analysis for L. DPFC (�42, 0, 50). There were

significant differences in the comparison of the three conditions (F(2,70)¼ 40.3, p< .001). (d) Stands for the result of

ROI analysis for L. IFG (�54, 26, 20). There were significant differences in the comparison of the three conditions

(F(2,70)¼ 64.3, p< .001).


Caplan, Alpert, Waters, & Olivieri, 2000; Caplan et al., 2001; Dapretto & Bookheimer, 1999;Embick et al., 2000; Just et al., 1996; Stromswold, Caplan, Alpert, & Rauch, 1996). Mussoet al. (2003) extended this view and suggested that the left IFG is a specific region for theuniversal processing of natural languages. Given this, the significant left IFG activationobserved in the current study during the comprehension of scrambled sentences is consistentwith the linguistic hypothesis that scrambled sentences in Japanese have a more complexstructure due to the presence of a filleregap dependency. The uni-lateral activation found in ourstudy is consistent with the results in the literature which suggested that the syntacticprocessing involves uni-lateral activation, particularly a dominance in the left hemisphere(Dapretto & Bookheimer, 1999; Embick et al., 2000; Luke, Liu, Wai, Wan, & Tan, 2002; Roder,Stock, Neville, Bien, & Rosler, 2002).

It has been suggested that there may be other confounding factors, i.e., factors other thanfilleregap relation might be affecting online sentence processing. From a linguistic point ofview, there are at least three major factors to consider: (i) the thematic hierarchy (Agent-NonAgent (Age-NAge) order may be easier to process than NonAgent-Agent (NAge-Age)order), (ii) the case hierarchy (Nominative-NonNominative (Nom-NNom) order may be easierto process than NonNominative-Nominative (NNom-Nom) order), and (iii) the animacy hier-archy (Animate-NonAnimate (Ani-NAni) order may be easier to process than NonAnimate-Animate (NAni-Ani) order). Since it is well known that all of these factors play important rolesin grammars of various languages (Comrie, 1981), it is not surprising that they do so in online


sentence processing as well. In fact, an increasing number of behavioral and functional brainimaging studies, those on German sentence comprehension in particular, provide evidence forsuch possibility. For examples, Bornkessel et al. (2005) reported an influence of the thematichierarchy on the activation of the pars opercular part of the left IFG. Grewe et al. (2006) foundthe increase of the left IFG activation in the NAni-Ani order compared with the Ani-NAniorder. Frisch and Schlesewsky’s (2001) event-related potential study demonstrated that animacyinformation interacts with case marking in the online computation of the thematic hierarchy.

While factors such as those listed above affect the processing burden and the activation in theleft IFG beyond any doubt, we consider that the results of the studies cannot be straightforwardlyapplied to the current experiment. This is due to linguistic difference in the types of scramblingbetween Japanese and German. First, Bornkessel et al. (2005) deal with scrambling in theembedded clauses, such as (1) [SV [SOV]] order and (2) [SV [OSV]] order. Japanese also allowsscrambling in the embedded clause as in (1) [S [SOV] V] and (2) [S [OSV] V] orders, butadditionally allows [O S [S gap V] V] order (known as ‘‘long distance scrambling’’), which is notallowed in German. Considering the fact that our experiment used simple transitive sentence andnot the embedded clause scrambling, and that the domain of scrambling is different in Germanand Japanese, the role of animacy and word order in German cannot directly contribute to ourresult here. Similar remark applies to the results by Grewe et al. (2006). They employed passiveconstruction as their experimental stimuli, but the landing cites of the removed constituents fortransitive sentences and passive sentences are generally considered to be different; constituentsin passive construction occupy positions that stand for arguments, but the scrambled constituentsoccupy non-argument positions. These points therefore indicate that at least the linguisticproperties for German and Japanese scrambling in these studies do not completely correspondwith each other. This, of course, demands further investigations for the degree of differences orsimilarities between scrambling in the two languages.

At this point, we can only say that Japanese scrambled sentence has a non-canonicalstructure, which demands an additional cognitive load in a human parser. Along with the line ofour argument, Hagiwara and Caplan (1990) reported that the Japanese scrambled transitivesentences are harder to recognize for Broca’s aphasics. By conducting an experiment in whichaphasic participants were instructed to pick up an animal toy that was mentioned in a shortconversation, they found that the participants had more difficulty in comprehending scrambledsentences than canonical sentences. Intriguingly, the accuracy of responses to canonical sen-tences was approximately 90%, whereas that to scrambled sentences remained only at a chancelevel of approximately 64%. Their results indicate that, when there is some dysfunction inBroca’s area, it becomes harder to identify the scrambled word order in which the accusativenoun phrase leaves a trace and moves to the initial position in a sentence. That is, the structuralcomplexity due to a filleregap dependency may account for such a difference in accuracy inaphasics’ responses (cf. Grodzinsky, 2000).

4.2. Left dorsal prefrontal cortex (L. DPFC)

From the direct comparison between the SS vs. CS condition, we observed the distinctcortical activation in the left dorsal prefrontal cortex (left DPFC) together with the activation inthe left IFG. Presently, the role of this region is twofold. One is, as proposed in previous studies,that the left DPFC is a specific brain region involved in syntactic processing (Friederici, Meyer,& von Cramon, 2000; Hashimoto & Sakai, 2002; Indefrey et al., 2001; Kang, Constable, Gore,& Avrutin, 1999; Moro et al., 2001; Newman, Pancheva, Ozawa, Neville, & Ullman, 2001).


The other function of the region assumed in previous imaging studies is the manipulation oftask difficulty caused by verbal memory load. Braver et al. (1997), Cohen et al. (1997), andRypma, Prabhaken, Desmond, Glover, and Gabrieli (1999) reported activation in the left DPFCand its related areas such as the left dorsolateral prefrontal cortex as the difficulty of the task oras the volume of the working memory increased. In their paradigm, the difficulty of tasks wasregulated, for instance, by controlling the length of the visual stimuli (Cohen et al., 1997).These studies, however, have focused much more on the factors for cognitive perception ratherthan on those for sentence processing.

We consider that the activation observed in the left DPFC, as well as the left IFG, is alsoa reflection of the processing of grammatical complexity posed by the scrambling effect. Asshown in Fig. 3, the activation of the left DPFC is greater in the comparison of SS vs. WLcondition than the comparison between CS vs. WL condition. The subtraction of the WLcondition enables the detailed observation of cortical activation during grammatical pro-cessing; thus we could observe the greater activation in the scrambled condition withrespect to an increased processing load. The scrambling effect, under our hypothesis,provides a natural solution toward the incrementation of working memory. In our experi-ment, it can be posited that the memory load for the incremental parsing significantlyincreased owing to enhancement of instantaneous visual recognition of a whole sentence.Given the accession of visual perception, the participants might have an additional workingmemory storage besides the temporal retrieval of the scrambled object. It seems reasonableto assume that such processes resulted in the activation in the left DPFC. In brief, theactivation in the left DPFC supports the initial assumption that scrambled sentences inJapanese have a more complex structure than canonical sentences. However, the funda-mental role of DPFC still remains unclarified, and further study of this particular region isnecessary.

5. Summary

We obtained imaging data from our experiment on scrambling effects in Japanese. Ourresults showed an association between the activation in the left inferior frontal gyrus andthe online perception of structural complexity at least attributable to the asymmetricstructure building of scrambling. Our results agree with suggestions in previous works thatBroca’s area is independent of cross-linguistic variation and is specialized for universalcompetence. Our results are in accordance with previous studies such as that of Hagiwaraand Caplan (1990) who observed the cognitive complexity of scrambling effects in apha-sics. In future studies, further investigations on other factors that influence word orderalternations, such as the effects of specificity and/or animacy of noun phrases may benecessary.

Acknowledgments

This study was supported in part by the following grants from the Japanese Ministry ofEducation, Culture, Sports, Science and Technology: (i) a Grant-in-Aid for Scientific Research NoP06005 (M.K.) (ii) a Grant-in-Aid for Scientific Research (B) No 19320056 (M.K.), and (iii) the21st Century Center of Excellence (COE) Program entitled ‘‘A Strategic Research and EducationCenter for an Integrated Approach to Language, Brain and Cognition’’ (Tohoku University).


References

Aoshima, S., Phillips, C., & Weinberg, A. (2004). Processing filler gap dependencies in a head-final language. Journal

of Memory and Language, 51, 23e54.

Ben-Shachar, M., Palti, D., & Grodzinsky, Y. (2004). Neural correlates of syntactic movement: converging evidence

from two fMRI experiments. NeuroImage, 21, 1320e1336.

Bornkessel, I., Zysset, S., Friederici, A. D., von Cramon, D. Y., & Schlesewsky, M. (2005). Who did what to whom? The

neural basis of argument hierarchies during language comprehension. NeuroImage, 15, 221e233.

Braver, T. S., Cohen, J. D., Nystrom, L. E., Jonides, J., Smith, E. E., & Noll, D. C. (1997). A parametric study of

prefrontal cortex involvement in human working memory. NeuroImage, 5, 49e62.

Caplan, D. (2001). Functional neuroimaging studies of syntactic processing. Journal of Psycholinguistic Research, 30,

297e320.

Caplan, D., Alpert, N., & Waters, G. (1998). Effects of syntactic structure and propositional number on patterns of

cerebral blood flow. Journal of Cognitive Neuroscience, 10, 541e552.

Caplan, D., Alpert, N., & Waters, G. (1999). PET studies of syntactic processing with auditory sentence presentation.

NeuroImage, 9, 343e351.

Caplan, D., Alpert, N., Waters, G., & Olivieri, A. (2000). Activation of Broca’s area by syntactic processing under

conditions of concurrent articulation. Human Brain Mapping, 9, 65e71.

Caplan, D., Vijayan, S., Kuperberg, G., West, C., Waters, G., Greve, D., et al. (2001). Vascular responses to syntactic

processing: event-related fMRI study of relative clauses. Human Brain Mapping, 15, 26e38.

Cohen, J. D., Peristein, W. M., Braver, T. S., Nystrom, L. E., Noll, D. C., Jonides, J., et al. (1997). Temporal dynamics of

brain activation during a working memory task. Nature, 386, 604e608.

Comrie, B. (1981). Language universals and linguistic typology. Oxford: Basil Blackwell.

Dapretto, M., & Bookheimer, S. Y. (1999). Form and content: dissociating syntax and semantics in sentence

comprehension. Neuron, 24, 427e432.

Demonet, J. F., Chollet, F., Ramsay, S., Cardebat, D., Nespoulous, J. L., Wise, R., et al. (1992). The anatomy of

phonological and semantic processing in normal subjects. Brain, 115, 1753e1768.

Embick, D., Marantz, A., Miyashita, Y., O’Neil, W., & Sakai, K. L. (2000). A syntactic specialization for Broca’s area.

Proceedings of the National Academy of Sciences of the United States of America, 97, 6150e6154.

Friederici, A. D., Meyer, M., & von Cramon, D. Y. (2000). Auditory language comprehension: an event-related fMRI

study on the processing of syntactic and lexical information. Brain and Language, 74, 289e300.

Frisch, S., & Schlesewsky, M. (2001). The N400 reflects problems of thematic hierarchizing. NeuroReport, 12,

3391e3394.

Grewe, T., Bornkessel, I., Wiese, R., Friederici, A. D., von Cramon, D. Y., & Schlesewsky, M. (2006). Linguistic

prominence and Broca’s area: the influence of animacy as a linearization principle. NeuroImage, 32, 1395e1402.

Grewe, T., Bornkessel-Schlesewsky, I., Zysset, S., Wiese, R., von Cramon, D. Y., & Schlesewsky, M. (2007). The role of

the posterior superior temporal sulcus in the processing of unmarked transitivity. NeuroImage, 35, 343e352.

Grodzinsky, Y. (2000). The neurology of syntax: language use without Broca’s area. Behavioral and Brain Sciences, 23, 1e71.

Hagiwara, H., & Caplan, D. (1990). Syntactic comprehension in Japanese aphasics: effects of category and thematic

role order. Brain and Language, 38, 159e170.

Hagoort, P. (2005). On Broca, brain, and binding: a new framework. Trends in Cognitive Sciences, 9, 416e423.

Hale, K. (1983). Warlpiri and the grammar of non-configurational languages. Natural Language and Linguistic Theory,1, 5e47.

Hashimoto, R., & Sakai, K. L. (2002). Specialization in the left prefrontal cortex for sentence comprehension. Neuron,

35, 589e597.

Humphries, C., Binder, J. R., Medler, D. A., & Liebenthal, E. (2007). Time courses of semantic processes during

sentence comprehension: an fMRI study. NeuroImage, 36, 926e932.

Indefrey, P., Hagoort, P., Herzog, H., Seitz, R. J., & Brown, C. M. (2001). Syntactic processing in left prefrontal cortex is

independent of lexical meaning. NeuroImage, 14, 546e555.

Inui, T., Otsu, Y., Tanaka, S., Okada, T., Nishizawa, S., & Konishi, J. (1998). A functional MRI analysis of compre-

hension processes of Japanese sentences. NeuroReport, 9, 3325e3328.

Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1996). Brain activation modulated by

sentence comprehension. Science, 274, 114e116.

Kaiser, E., & Trueswell, J. C. (2004). The role of discourse-context in the processing of a flexible word order language.

Cognition, 94, 113e147.


Kang, A. M., Constable, R. T., Gore, J. C., & Avrutin, S. (1999). An event-related fMRI study of implicit phrase-level

syntactic and semantic processing. NeuroImage, 10, 555e561.

Koizumi, M., & Tamaoka, K. (2004). Cognitive processing of Japanese sentences with ditransitive verbs. Gengo Kenkyu(Journal of the Linguistic Society of Japan), 125, 173e190.

Luke, K.-K., Liu, H.-L., Wai, Y.-Y., Wan, Y.-L., & Tan, L. H. (2002). Functional anatomy of syntactic and semantic

processing in language comprehension. Human Brain Mapping, 16, 133e145.

Mazuka, R., Ito, K., & Kondo, T. (2002). Cost of scrambling in Japanese sentence processing. In M. Nakayama (Ed.),

Sentence processing in East Asian languages (pp. 131e166). Stanford: CSLI.

Miyamoto, E. T., & Takahashi, S. (2002). Antecedent reactivation in the processing of scrambling in Japanese. In

T. Ionin, H. Ko, & A. Nevins (Eds.), The proceedings of HU-MIT 2001-MIT working papers in linguistics 43

(pp. 123e138). Cambridge: MIT.

Moro, A., Tettamanti, M., Perani, D., Donani, C., Cappa, S. F., & Fazio, F. (2001). Syntax and the brain: disentangling

grammar by selective anomalies. NeuroImage, 13, 110e118.

Musso, M., Moro, A., Glauche, V., Rijntjes, M., Reichenbach, J., Buchel, C., et al. (2003). Broca’s area and the language

instinct. Nature Neuroscience, 6, 774e781.

Nakano, Y., Felser, C., & Clahsen, H. (2002). Antecedent priming at trace positions in Japanese long-distance

scrambling. Journal of Psycholinguistic Research, 31, 531e571.

Newman, S. D., Just, M. A., Keller, T. A., Roth, J., & Carpenter, P. (2003). Differential effects of syntactic and semantic

processing on the subregions of Broca’s area. Cognitive Brain Research, 16, 297e307.

Newman, S. D., Pancheva, R., Ozawa, K., Neville, H. J., & Ullman, M. T. (2001). An event-related fMRI study of

syntactic and semantic violations. Journal of Psycholinguistic Research, 30, 339e364.

Oldfield, R. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia, 9, 812e815.

Price, C. J., Moore, C. J., Humphreys, G. W., & Wise, R. J. S. (1997). Segregating semantic from phonological

processes during reading. Journal of Cognitive Neuroscience, 9, 727e733.

Roder, B., Stock, O., Neville, H., Bien, S., & Rosler, F. (2002). Brain activation modulated by the comprehension of

normal and pseudo-word sentences of different processing demands: a functional magnetic resonance imaging

study. NeuroImage, 15, 1003e1014.

Rypma, B., Prabhaken, V., Desmond, J. E., Glover, G. H., & Gabrieli, J. D. E. (1999). Load-dependent roles of frontal

brain regions in the maintenance of working memory. NeuroImage, 9, 216e226.

Sakai, K. L., Hashimoto, R., & Homae, F. (2001). Sentence processing in the cerebral cortex. Neuroscience Research,

39, 1e10.

Sakai, K. L., Homae, F., & Hashimoto, R. (2003). Sentence processing is uniquely human. Neuroscience Research, 46,

273e279.

Stromswold, K., Caplan, D., Alpert, N., & Rauch, S. (1996). Localization of syntactic comprehension by positron

emission tomography. Brain and Language, 52, 452e473.

Talairach, J., & Tournoux, P. (1998). Co-planar stereotaxic atlas of the human brain. New York: Thieme.

Tamaoka, K., Sakai, H., Kawahara, J. I., Miyaoka, Y., Lim, H., & Koizumi, M. (2005). Priority information used for the

processing of Japanese sentences: thematic roles, case particles or grammatical functions? Journal of Psycholin-

guistic Research, 34, 281e332.

Ueno, M., & Kluender, R. (2003). Event-related brain indices of Japanese scrambling. Brain and Language, 86,

243e271.

Wise, R. J. S., Chollet, F., Hadar, U., Friston, K., Hoffner, E., & Frackowiak, R. (1991). Distribution of cortical neural

networks involved in word comprehension and word retrieval. Brain, 114, 1803e1817.

an fmri study of scrambling effects on sentence comprehension

Documents