neurolinguistics must be computational

Arbib & Caplan (1979) 1

Neurolinguistics must be computational

THE BEHAVIORAL AND BRAIN SCIENCES (1979), 2, 449-483

Michael A. Arbib

Computer & Information Science Department. Center for Systems Neuroscience, University of Massachusetts,

Amherst, Mass. 01003

David Caplan*

Computer & Information Science Department, Center for Systems Neuroscience, University of Massachusetts,

Amherst. Maas 01003

•Current address: Division of Neurology, Ottawa Civic Hospital, Ottawa, Ontario, Canada KlY4E9.

[449]

Abstract: We provide a threefold taxonomy of models in neurolinguistics: faculty models, which embrace the

work of classical connectionists and holists and of such modern workers as Geschwind; process models, which are

exemplified by the work of Luria, and which fractionate psycholinguistic tasks and ascribe the components to

particular brain regions; and representational models, which use specific linguistic representations to build a

psycholinguistic analysis of aphasic performance. We argue that Further progress requires that neurolinguistics

become more computational, using techniques from Artificial Intelligence to model the cooperative computation

underlying language processing at a level of detail consonant with linguistic representations. Finally, we note that

current neurolinguistics makes virtually no contact with the synapse-cell-circuit level of analysis characteristic of

twentieth-century neuroscience. We suggest that the cooperative computation models we envisage provide the

necessary intermediary between current neurolinguistic analysis and the utilization of the fruits of modem

neuroanatomy, neurochemistry, and neurophysiology.

Keywords: aphasiology; artificial intelligence; brain theory; connectionism; language; neurolinguistics;

psycholinguistics

Introduction

in this paper we discuss and classify a number of models that relate language function to neural processes. We

suggest that there is a progression to be found in such models, whose culmination would naturally lead to what we

term "computational process models." The development of such models would allow the application of cybernetics

and brain theory to the study of neural mechanisms involved in language.

The paper is divided into five sections. The first three deal with existing neurolinguistic models, for which we

have provided a taxonomy based on the range of empirical data with which a model attempts to deal and the

linguistic and psycholinguistic analyses it adopts. In the fourth section we review work from Artificial Intelligence,

which suggests certain properties of a more adequate process model. In the fifth section we outline the relation of

such a model to neural mechanisms.

Our intention is to provide a framework for assessing the empirical adequacy of proposed neurolinguistic models

and to suggest several directions for their further development. We perceive a conceptual gap between many of the

existing, clinically oriented theories of neural mechancims pertaining to language and recent psycholinguistic

approaches to the field. In our view, it is inadequate to bridge this gap by utilizing the the neural aspects of one type

of model and the phenomenological analysis of another. Rather, we think this gap must be filled by deepening both

the phenomenological and neural analyses and we suggest ways in which this might be accomplished. This paper


does not report new observations, but attempts to clarify conceptual issues in neurolinguistics by confronting classic

neurologi. cal approaches with developments in psycholinguistics, Artificial Intelligence, neuroscience, and

cybernetic modelling.

I. Faculty models

"Connectionist" models of neurolinguistics (which have also been termed "localist" and "topist") date from the

earliest scientific work on the relation of language capacities to brain (Broca, 1861a,b, 1865; Wernicke, 1874). They

are centered on the identification of major functional psycholinguistic tasks, such as speaking and comprehending

speech, and tend to treat such tasks as unanalyzed wholes. The models are all derived from the study of aphasic

patients, and the data involve judgments by observers about the relative disruption of, and, to a lesser degree,

qualitative changes in, these major tasks. A connectionist model then specifies the cerebral locus of these task-

defined components of language function and advances limited hypotheses as to their interaction.

Details of connectionist models were widely deflated among French, German, and English neurologists in the

late nineteenth century (Charcot, 1863; Grashey, 1885; Lichtheim, 1885; Kussznaul, 1877; Broadbent, 1872;

Bastian, 1897). Other workers rejected the observational accuracy of the empirical data upon which this approach

was based (Marie, 1906; Moutier, 1908), offered alternative clinical observations and models (Jackson, 1874, 1878;

Goldstein. 1948), or criticized the logic by which the connectionists arrived at their models, and offered alternatives

(Freud, 1891). These critics have been referred to as "holists," but we suggest, contrary to common belief, that their

approach is not significantly different from that of the connectionists. Geschwind (1964) has also taken the position

that these models are similar, but our reasons for grouping them together differ from his.

From Wernicke to Lichtheim

Broca (1861a,b, 1865, 1869) argued that the faculty for articulate speech was located in the third frontal gyros,

adjacent to the Rolandic motor strip (on the left in right-handers). Wernicke (1874) published the first

comprehensive model of language representation in the brain, based on observed aphasic symptom-complexes in

addition to those described by Broca. The symptom-complex that bears Wernicke's name consists of both a -

receptive" and an "expressive" disorder of language.

[450]

Wernicke argued for interaction between components of a neural system subserving language performance. He

suggested that the first temporal gyros was the anatomical area responsible for comprehending spoken language and

storing the auditory impressions of words. He postulated (p. 93) that the co-occurrence he observed of a deficit in

speech with the comprehension disturbance in a patient without an apparent lesion in the region of Broca's area was

due to the interruption of information flow during speaking:

. A sound image of the word or syllable is transmitted to some sensory portion of the brain ... the sensation of

innervation of the movement performed is laid down in the frontal motor areas as a representation of speech

movement ... When, later, the

spontaneous movement, the consciously uttered word, takes place, the associated representation of movement is

innervated by the memory image of the sound.-


This interpretation led to the first and prototypical diagram of the location of language functions in the brain

(Figure I). The major source of data upon which ensuing connectionist theories were to build was the recognition

that a psycholinguistic task - speaking, comprehending spoken speech, reading, writing - was disturbed.

Constellations of task-defined functional deficits were grouped into "symptom-complexes," or syndromes. On

the basis of inferences from pathological material, Wernicke concluded that failure of one component would impair

a second in a manner behaviorally distinguishable from a primary failure of the second component itself.

Wernicke and his followers imposed a variety of conditions on the construction of these models. Wernicke

implicitly required that the model be consistent with theories in both neuroscience and psychology. It was on the

physiology of Meynert that he based the location of his functional components - that for comprehension of spoken

speech in association cortex adjacent to the post-thalamic auditory radiations and that for speech production in

association cortex adjacent to the motor area - as well as the sensory-to-motor direction of information flow and

component control.

Lichtheim (1885) attempted to classify all aphasic syndromes in terms of the connectionist model (Figure 2). He

postulated that three main cerebral "centers" are involved with language: the motor programming center (M)

described and located by Broca; the sensory center For auditory word memories (A) described and located by

Wernicke; and the center for concepts (B), which he thought of as carrying out crossmodal and intermodal

associations of the properties of objects (and possibly of more abstract entities) to produce concepts, and which he

suggested were a function of a large and unspecified region in the brain. He argued that these three centers were

mutually connected and that connections to the periphery were established for language by auditory input into A and

by commands for motor output from M.

Figure 1. Wernicke's (1874) original diagram, showing the principle pathways for language. a: peripheral

auditory pathway; a: sound center for words; b: motor center for words; $: peripheral motor pathways for speech.

Reproduced from Moutier (1908).


Figure 2. Lichtheim's (1885) diagrammatic representation of the centers an pathways involved in language use.

R. concept center; M: motor speech centc (Broca's area) ; A: auditory speech center (Wernicke's area); E: motor

center fc control of musculature involved in writing; 0: visual center. Reproduced fror Moutier (1908).

Lichtheim assumed that there is no possibility for the su titutior of alternate pathways or the development of new

components ter s lesion. He attempted to characterize syndromes in terms of les rn "of the diagram" and predicted

seven distinct types of aphasia: motor and sensory aphasia, transcortical motor and sensory aphasia; subcortical

motor and sensory aphasia, and conduction aphasia. He described cases of each syndrome. The clinical relevance Of

this classification is attested to by its survival in today's neurological literature (Benson & Geschwind, 1975).

It is worthwhile to examine some of the details of Lichtheim's model. In discussing Wernicke's aphasia,

Lichtheim argued that a lesion in center A of Figure 2 would result in the loss of five linguistic functions: the

understanding of spoken language, the understanding of written language, the faculty of repetition, the faculty of

writing to dictation, and the faculty of reading aloud. Three faculties would be spared: writing, copying words, and

volitional speech. It was Wernicke's argument that the disorder of volitional speech seen in this syndrome was due

to the necessary arousal of auditory memories for words in speaking. Lichtheim's model permits a direct innervation

of the motor center M from the concept center B; on superficial analysis, this aspect of Lichtheim's model seems to

make the wrong prediction about speech output with posterior lesions. Lichtheim turns this possible deficit into an

explanatory principle in its own right. He accepts the role of center A in spontaneous speech. The normal production

of speech thus involves a double activation of hi through two pathways: a direct one from A to M, and a second one

from A to B and thence to M. A lesion anywhere in the arc from A to B and from A to M will give rise to

paraphasia. The pathway from the concept center B to the motor speech center M is sufficient to ensure that speech

is produced, but it needs the additional input from the sensory center A to ensure that speech is correct.

The underlying principle is that where a performance is dependent upon two inputs to one center, disturbance of

one input will lead to a partial performance. This situation, in which the motor speech area is deprived of its input

from the auditory store for words, resulting in fluent but paraphasic speech, is thus contrasted with one in which the

speech area is deprived of its input from the concept center in transcortical motor aphasia, with loss of the capacity

for volitional speech.

Lichtheim's article is replete with detailed arguments of this type, and, aside from its influence on the clinical

classification of aphasiac, it is notable for its effort to base the analysis of aphasia and


[451]

neurolinguistics on as few components and as principled a set of interactions as possible.

Modern connectionism

It is now a commonplace observation, made originally and perhaps most forcefully by Head (1926), that the

development of models along the lines of these prototypes led to a "chaotic" theoretical development, characterized

by intricate diagrams, replete with centers and connecting tracts, whose components and connections proliferated

with the discovery of new constellations of symptoms. We shall later argue that neurolinguistic models must

perforce be complex. In any case, as Geschwind (1967) points out and as Marshall (1979) recognizes, not only has

the clinical classification of syndromes been useful to the neurologist, but the basic theoretical approach, involving

the delimitation of centers and connections, has proven capable of predicting new patterns of breakdown and has

been the basis of most neurolinguistic theories. Nor is it associated with workers of only the last century. The

theoretical work of Geschwind, to which we now turn, follows upon earlier connectionist approaches.

A large part of Geschwind's work can be seen as an effort to embed the theory of cerebral language

representation into a broader context that includes the relation of language representation as a whole to skilled motor

activity and complex perceptual performance (the analysis of apraxia and agnosia), but we shall not discuss this

important dimension here. Geschwind (1965) argues that, in addition to the centers postulated by nineteenth-century

theorists, there is an area in the inferior parietal lobe that is peculiar to human beings and consists of the angular and

supramarginal gyri. This is an "extensive, evolutionarily advanced parietal association area developing not in

apposition to the primary projection areas for vision, somesthetic sensibility, and hearing, but rather at the point of

junction of these , areas." tts role is posited to be the facilitation of nonlimbic (nonaffective) crossmodal

associations, in particular between the auditory stimuli of language (words) and the sensory properties of the items

to which the words refer. In this way, this structure assumes importance in human naming ability (Figure 3).

Geschwind (1965) is explicit in his emphasis that the emergence of naming in man, based on the anatomical

connections just described (which are absent or rudimentary in the other primates) is a "prerequisite" for speech.

Geschwind's second addition to the basic theory is the precise description of pathways to and between the speech

centers. The syndrome of alexia without agraphia (the inability to read though one can write) may serve as an

example (Dejerine, 1892). Patients with this syndrome tend to have vascular lesions in the left occipital areas as well

as the posterior callosurn. They are linable to read words, but they can characteristically name objects presented

visually, and their performance may also be less impaired with some classes of linguistic visual stimuli, such as

letters and numbers. Several patients have been observed to retain their ability to read (from a card) words, such as

"bank" or "post office," that frequently appear on public buildings. Geschwind (1965) suggests that objects and

pictures are named after visual presentation because associations to the visual form are aroused in other modalities

in right hemisphere areas with intact callosal connections to the speech areas of the left hemisphere. He notes (1979)

that the kinds of linguistic entities just described as being preserved in this syndrome, must belong to a class of

stimuli whose normal processing depends on connections other than the splenium. He speculates that such stimuli

may be perceived as ideographs or, at least, as items whose visuo-auditory connection does not depend upon

analyzing the visual stimulus into subportions that are separately associated with an auditory subcomponent of the

total auditory entity (as is the case with grapheme- and syllable-based orthographies). The relative resistance of

ideographic representations to disruption in Japanese patients whose syllabary orthography is affected in this

f


syndrome is consistent with this analysis (Sasanuma & Pujirnara, 1971). In an important paper, Geschwind (1969)

takes up in detail the question of the anatomical pathways underlying such connections.

Earlier theorists considered that the function of Broca's area was the production of speech, and that the linguistic

representations in this area were the motor programs for speech. Similar functional/representational dualities

characterized all centers. Geschwind's analysis of naming follows this pattern inasmuch as a function (object

naming) is associated with a representational system (associating modality-specific information about objects and

words) at an anatomical site (the inferior parietal lobe). It is reasonable to

Figure 3. Geschwind's (1972a) model of pathways involved in naming. Geschwind's captions: Upper —

Saying the name of a seen object, according to Wernicke's model, involves the transfer of the visual pattern

to the angular gyros, which contains the -rules- for arousing the auditory form of the pattern in Wernicke's

area. From here the auditory form is transmitted by way of the arcuate tasciculus to Broca's area. There the

articulatory form is aroused, is passed on to the face area of the motor cortex and the word then is spoken.

Lower — Understanding the spoken name of an object involves the transfer of the auditory stimuli from

fleschrs gyros (the primary auditory cortex) to Wernicke's area and then to the angular gyrus, which arouses

the comparable visual pattern in the visual association cortex. Here the Sylvian fissure has been spread

apart to show the pathway more clearly. (From Geschwind, 1972. Reprinted with permission from

Scientific American, Inc. and the author.)


[452]

consider that this center may function as part of these overt tasks, because it retrieves the full lexical form of

words from semantic, auditory, and other cues, as each task requires. (We believe this to be an unsatisfactory

approach to naming, word meaning, and sentential meanings; see Fodor et al., 1974, for a criticism of approaches

along these lines.)

Geschwind's model is thus made up of components fundamentally oriented to particular psycholinguistic tasks

taken as a whole, although, as in the earlier models, there is provision for some limited interaction of entire

components in regulating the performances of others; and, perhaps in the case of the naming function, there is

provision for embedding one component within another. This last possibility marks a feasible transition from these

"faculty" models to what we term "process" models. We discuss such models in Section 2. Note that our proposed

classification of models is, of course, not absolute, but is based on the dominant properties of each model, and that

most connectionist models incorporate some process features. and vice versa.

Geschwind's second point, his stress on multiple pathways, clearly enriches the theory of neurolinguistics in an

important way, with partial duplication of mechanisms, some of which are arguably more efficient and hence

normally utilized. Geschwind (1979) notes that these mechanisms do not always appear to operate on linguistic

elements that form a natural class in linguistic theory, and that their analysis can therefore enrich the theory of

linguistic and psycholinguistic taxonomies derived from work on the normal system. Such "submerged" aspects of

linguistic taxonomy and psycholinguistic function have also "surfaced" from other studies (for example, Bradley et

al., 1979) and appear to be a potentially significant by-product of work with abnormal populations.

"Holist" models

Perhaps the most serious challenges to the connectionist theorists came from those who believed that

neurolinguistics should rest on a totally different set of functions. As Geschwind (1964) points out, the latter

theorists invariably concurred with the connectionists concerning the characterization of aphasic syndromes and the

occurrence of particular syndromes after particular lesions. He thus suggests that nothing separates these theorists

from the connectionists and ignores the point these theorists tried to make regarding the inappropriateness of

connectionist data for the construction of neurolinguistic theory.

The best examples of such work are by Jackson (1879, 1878) and Goldstein (1948). To give an example of

Jackson's work, we consider his description of a subtype of what would be considered Brace's aphasia, in which

patients have the following marked disturbance of speech: In ordinary conversational contexts they are usually mute.

In special situations, they do speak. Typically, they can repeat at least some portions of a presented utterance; they

speak in emotional situations and those of great personal significance. Jackson observed that their most common

responses were oaths, uttered in emotionally distressing situations, and "fixed utterances," which Jackson took to

represent transformations of the words in the patient's mind at the actual moment of illness. What Jackson termed

utterances with "propositional content," that is, utterances in which predicative and other relations between words

(and the real-world entities they referred to) were expressed and that were appropriate to but not uniquely

determined by the conversational situation, were absent -with the exception of the single words "yes" and "no,"

which Jackson argued were the most elementary and general propositions.

Jackson argued that observing functional repertoires of this sort would lead to a theory of language function that

was not task-specific and to a theory of brain function that did not consist of centers and connections. However, he


offered no theory of the representation of propositions nor did he distinguish between the ideational and linguistic

form of propositions. In a similar vein, Goldstein argued that general functional principles, such as the ability to

assume what he termed an "abstract attitude" (an attitude, we remark, that is probably a prerequisite for the

production of a Jacksonian propoe_ tion), were lost in aphasia. (A modern development of this approach is found in

Locke et al., 1973, in which Jacksonian functional capacities are related to a hierarchical model of the neuraxis

based on the work of Yakovlev, 1998.)

The Jacksonian approach incorporates the claim that the functional losses underlying the deficit also underlie

deficits in other realms of behavior in the aphasic patient. It emphasizes the overlap of linguistic and nonlinguistic

functions, but uses only a rudimentary characterization of language itself. We believe this approach thus avoids the

central issue in neurolinguistic theory, which consists primarily of the study of the representation and utilization of

the linguistic code in and by neural tissue, and only secondarily of how this code is subject to functional factors that

also regulate other cognitive and perceptual-motor capacities (although we also believe that neurolinguistics will

remain shut off from recent dramatic progress in cellular neurobiology until fruitful comparisons of linguistic

behavior with other perceptual-motor behaviors are developed to the point where animal models can be introduced

for relevant aspects of neurolinguistic processing).

The main features of faculty models

We have suggested that the most important characteristic of the faculty models is their analysis of

psycholinguistic function into major overt language tasks, while treating each of these tasks as individual, essentially

unanalyzed components of a general hinguage faculty. This is an example of a purely "top-down" m elling approach

in which the designation of system components a not complemented by an account of the mechanisms (whether

comp tional or neural) through which component function is realized. Th\e„ process models to be discussed in the

next section are also -top-downin this sense. Both of these top-down classes of models stand in contrast to those

described in Sections 3 and 4, as do the faculty models to the process models described in Section 2.

The top-down models are not specific enough about the nature of the information represented in their

components to indicate either the exact processing of that information by each component or the utilization of the

output of one component by another. To a limited extent, the theories we have considered adopt terminology drawn

from a pretheoretic linguistic and psycholinguistic vocabulary, speaking of "small, grammatical words," "sound

patterns," and "grammatical phrases." But they do not specify the nature of these terms or give them a role in the

explanation of the syndromes. These theoretically naive elements, part of the description of aphasic syndromes, are

incidental to these models.

Another limitation of top-down theories is that they cannot provide an account of the representation and

processing of language which is adequate to approach the neural mechanisms underlying these processes. Marshall

(1979) notes that it is generally agreed that the corpus callosum transmits "high-level" information and he asks

whether anyone would care to speculate as to the "neural code" in which such information is carried. None of "high-

level information," "sound patterns of words," or "motor sequences of speech" gives us an adequate description of

what the neural code is coding. A more precise, computational vocabulary is a prerequisite for this type of

neurolinguistic theory construction.


2. Process models

The class of models included in the process category consists of those whose functional analysis yields

components responsible for portions of a psycholinguistic task. The functional analysis is more detailed than in the

faculty models. In every case, a component accomplishes only a part of a psycholinguistic task; the entirety of the

task requires the interaction of several components.

Interestingly, the historical origins of this type of model in pretheoretic neurolinguistic observations are prior to

1861 (the year

[453]

of Broca's landmark paper) and go back at least as far as Lordat's (1843) remarkably modern-sounding

description of stages in speech production, which he published as a result of introspections on his own aphasia. In

this century, we can look to the work of Head (1926) for a componential analysis of language along pretheoretic

linguistic lines intersecting all psycholinguistic tasks, with an effort to describe aphasic syndromes in terms of the

functional components. Pick (1913) provides a model for making use of subcomponents of tasks; Brown (1977,

1979) has revived and extended Pick's analysis. In this section we shall outline the model presented by A. R. Luria,

which we believe exemplifies this class of models.

Luria's approach is founded upon the idea of a functional system (based on the work of Anokhin, 1935, and

Bernstein, 1935) in which an invariant task can be performed by variable mechanisms to bring the process to an

invariant result, with the set of mechanisms being complex and, to an important degree, interchangeable. From such

a background, as well as from the developmental studies of Vygotsky (1934, trans. 1962) and his own wide

experience in neurology and psychology, Luria formulated (1973, pp. 33-34) the following program for

neuropsychology:

"It is accordingly our fundamental task not to localize higher human psychological processes in limited areas of

the cortex, but to ascertain by careful analysis which groups of concertedly working zones of the brain are

responsible for the performance of complex mental activity; what contribution is made by each of these zones to the

complex functional system; and how the relationship between these concertedly working parts of the brain in the

performance of complex mental activity changes in the various stages of its development."

We shall consider four diagrams based on Luria's (1973) analyses of object naming, speech production, speech

comprehension, and repetition. His analysis of the naming of objects can be seen in Figure 4. Each box corresponds

to a brain region and to functions suggested by clinical data. However, to save space we shall discuss only a

sampling of the boxes. (Boxes are labelled with the same capital letter if they correspond to the same brain region;

they differ in the number of primes if they are attributed to different hypothetical functions. In each figure, the

functional attribution of a region is taken from Luria, whereas the arrows are our own indication of a plausible

information flow.)

In the object-naming task, no acoustic model is given to the subject. Instead, he is to look at an object and code

his visual perception of it by an appropriate spoken word. Clearly, object naming requires reasonably precise visual

perception. Luria singles out the left temporo-occipital zone (box A), where lesions disturb both the ability to name

objects and the ability to evoke visual images


Figure 4. Block diagram of subsystems involved in Luria's analysis of naming objects.

Figure 5. Block diagram of subsystems involved In Luria's analysis of the verbal expression of motives.

in response to a given word, as the anatomical site of this component. A patient with such a lesion cannot draw a

named object, even though he can copy a drawing line-by-line. In short, lesions here seem to impair the

transformation between an array of isolated visual features and a perceptual unity into which the features are

integrated.

The next step (box B) is to discover the appropriate name and to inhibit irrelevant alternatives. Lesions of the left

tertiary parietooccipital zones yield verbal paraphasias - the appearances of irrelevant words that resemble the

required word in morphology, meaning, or phonetic composition. Irrelevant sensory features of the object or of the

articulatory or phonetic information associated with its name can evoke a response as easily as correct features. It is

as if the inhibitory constraints were removed in a competitive process. Such lesions do not disturb the phonological

representation of language: prompting with the first sound of a name does trigger its recall.


Luria also includes phonemic analysis (box E) in the naming of objects. Lesions of the left temporal region

disturb the phonemic organization of naming, yielding literal paraphasias, in which words of similar phonemic

organization are substituted. In strong contrast with the verbal paraphasias induced by box B lesions, prompting with

the initial sound of the name does not help the patient with a left temporal lesion.

This model exemplifies Luria's view of the brain as a functional system, and illustrates why we describe his work

as generating process models. It is clear that box E is not just for sensory phonemic analysis and that box D is not

purely for motor articulatory analysis. Rather, both systems participate in all brain functions that require exploitation

of the network of representations that define a word within the brain. Convergence of the proper word can be

accelerated by the cooperative exploitation of both phonemic and articulatory features, and of others as well.

Luria's description of the processes involved in speech production is brief (Figure 5). The frontal lobes are

essential for the creation of active intentions or the forming of plans. Frontal lesions (box F) do not disturb the

phonemic, lexical, or logico-grammatical functions of speech, but they do disturb its regulatory role. Lesions of the

left inferior fronto-temporal zone (box C) yield "dynamic aphasia" - the patient can repeat words or simple sentences

and can name objects, but is unable to formulate a sentence beyond "Well ... this ... but how? . . Luria thus views the

task of this region as being to reeode the plan (formulated by box F) into the "linear scheme of the sentence," which

makes clear its predicative structure.

Turning to the understanding of speech we can follow Luria's analysis of the process whereby the spoken

expression is converted, in the brain of the hearer, into its "linear scheme," from which the general idea and the

underlying conversational motive can be extracted.

As we see in Figure 6, box E performs its usual role of phonemic analysis, supplying input to box H. Lesions

here, in the posterior zones of the temporal or temporo-occipital region of the left hemisphere, leave phonemic

analysis unimpaired, but grossly disturb the recognition of meaning. Luria very tentatively suggests that this may be

due to the impairment of concerted working of the auditory and visual analyzers. The intriguing suggestion here

seems to be that phonological representations serve to evoke a modality-specific representation (akin to a visual

image; see Kosslyn et al.: "On the De-Mystification of Mental Imagery" BBS 2(4) 1979) rather than directly evoking

linguistic semantic representation. The former

[454]


Figure 6. Block diagram of subsystems involved in Luria's analysis of speech understanding.

representation aids the evocation of the appropriate semantic and syntactic representation for further processing.

Luria identifies three subsystems involved in syntactic-semantic analysis: speech memory, logical scheme, and

active analysis of most significant elements. Lesions of the parieto-temporo-occipital zones of the left hemisphere

(box J) impair perception of spatial relations, constructional activity, complex arithmetical operations, and the

understanding of logico-grammatical relations. A sentence with little reliance on subtle syntax - "Father and mother

went to the cinema but grandmother and the children stayed at home" - is still understood, wheres a sentence like "A

lady came from the factory to the school where Nina worked" cannot be understood Understanding a sentence

requires not only the retention of its elements, but their simultaneous synthesis into a single, logical scheme. Luria

argues that data on parieto-temporo-occipital lesions give neurological evidence of a system specifically adapted to

this synthesis for those constructions where identical words in different relationships receive different values. Box J

plays a role when the grammatical codes - case relations, prepositions, word order, and so forth - are decisive in

determining how the words of the sentence combine to give its overall meaning.


Figure 7. Block diagram of subsystems involved in Luria's analysis of repetitive speech.of "planning" to

"linear schemes," J - F in and F G out, seems simplistic).

Figure 7 schematizes Luria's views of the brain regions involved when a subject repeats sentences spoken to him.

In Figure 8, we have incorporated the four preceding diagrams into a single, partial specification of a neurally based

language device. The figure represents the components and connections in Luria's (1973) analysis, and is

supplemented by several additional lines representing information not specified directly but indirectly inferrable

from Luria's work.

Figure 8 makes visually apparent the features of Luria's approach that distinguish it from the models of Section 1.

Each psycholinguistie task is performed by several components acting in parallel and sequential fashion. Many

components are involved in several tasks. One might consider whether more shared components could be derivable

from the system (G and J, for instance, might interact through some shared process) and whether some components

could not be partitioned (I seems overburdened). It also seems clear that additional components might profitably be

considered (the relation of "planning" to "linear schemes," J - F in and F G out, seems simplistic).

The component interaction in this model is far more complex than that in the faculty models. Of course, such

features of neurolinguistic models must be empirically motivated and are not desirable in and of themselves. It

seems reasonable, on the basis of numerou\results in psycholinguistics (Fodor et al., 1979; Garrett, 1976 ; Fran r

1978; Frazier & Fodor, 1978), to consider feedback and feed ward mechanisms, parallel as well as sequential

component activation, nd other such interactions in neurolinguistic models. Such features of model seem to be the

natural consequence of extending the neurolinguistic theory to include a more detailed and adequate range of data.

Finally, we note that the expansion of task-related components into smaller divisions allows a wider range of

interaction between linguistic and nonlinguistic processes.

We close this discussion with the observation that this model still fails to be computational. There is insufficient

specification of the input-output codes of the components to allow a clear conception of exactly how the components


function individually or how they utilize information from each other. Several components, such as E, are relatively

well specified in this regard (though none are fully specified to the point, for instance, where they could be directly

transposed to a computer), but others, such as G and J, are grossly underspecified and seem to involve the

construction of a variety of different linguistic representations.

Figure 8. Synthesis of the block diagrams of Figures 4-7

3. Representational models

The impact of transformational-generative grammar upon the concerns of psycholinguists has been enormous

(Fodor et al., 1974),

[455]

and a number of applications of linguistic theory to aphasiological + descriptions have also been published. We

shall outline two areas, each representing the work of a number of, investigators, in which such applications have

been made: the study of phonemic paraphasias and the characterization of the functional deficit in Broca's aphasia.

The nature of phonemic paraphasias constitutes one of the better studied aspects of aphasic production. We shall

describe two studies oI particular significance to the present theme. Schnitzer (1972) analyzed the dyslexic errors of

a twenty-six-year-old woman patient who manifested a prominent dyslexia as part of an otherwise mild aphasic

disturbance following evacuation of a left-sided subdural hematoma. (The patient came to autopsy where a cortical

contusion of the left supramarginal gyros was the only relevant CNS abnormality.) Her errors were most prominent

in derivationally complex, multisyllabic words of Latinate etymology. Kehoe and Whitaker (1973) report that she

was unable to read words such as "degradation" but correctly pronounced the nonsense string "maygradation" and

words of Anglo-Saxon origin such as "yestermorn."

Sehnitzer's analysis of the patient's errors indicates that about 40 to so percent of them could be seen as the result

of a single error mechanism. He utilizes the analysis of English phonology developed by Chomsky and Halle (1968),

in which the phonological component of the grammar for English operates in a complex deterministic fashion on an

input termed the "systematic phonemic representation" and yields an output that adds a number of phonological

features, including stress contour, and affects other features such as vowel quality. The input to this level is well

1


defined in terms of the theory of which this is a component; it consists of the lexical phonological representation

plus lexical morphological and redundancy rule effects, plus a set of syntactic markings derived from the "surface

structure" created by the syntactic component of the grammar, operated on by readjustment rules. The output of the

phonology is equally well defined. Schnitzer argues that seemingly complex dyslexic errors can be viewed as the

result of extremely simple changes in the input to the phonological component, and of the subsequent operation of

phonological rules on the erroneous inputs to produce observed transformations of the target word.

A rather different approach as been undertaken by Lecours and his colleagues (Lecours & Lhermitte, 1969;

Lecours et al., 1973; Lecours & Caplan, 1975). He postulates a rich error mechanism to account for the phonemic

paraphasias observed in the spontaneous speech, reading, and repetition of a large number of French "fluent"

aphasics. In his model, paraphasias are due to the operation of this error mechanism on the output of the normal

French phonology. Lecours utilizes a vocabulary similar to Schnitzer's for the characterization of the linguistic

elements involved in errors; in both cases, these elements consist of phonemes made up from a distinctive feature

analysis.

Despite its positive features, Lecours's model has one grave disadvantage: It is a descriptive model that seeks to

use an economical set of error generators to describe compactly a varied, statistically characterized corpus of

paraphasias. It views brain damage as creating explicit "error generators." An alternate approach would seek to show

what discriminations must be made in transforming some internal representation into a sequence of phonemes, and

would then analyze conditions in which errors would be unavoidable with limited computing resources.

Another use of modern linguistics to represent precisely the relevant "data structures" is involved in the analysis

of the deficit seers in Broca's aphasia. Kean (1979) argues that there is a linguistic characterization of the range of

abnormal phenomena seen in this syndrome, namely, all errors are due to disturbances within the Phonological

component of a grammar. It is apparent that many aspects of the clinical syndrome traditionally recognized as

Broca's aphasia may be characterized at the phonological level of a grammar. Such aspects include the dysarthric

components of speech output, segmental paraphasias, prosodic abnormalities, and others. Kea n establishes that a

phonological` characterization can — and must — be given for the syndrome's elements, which have traditionally

been conceived in other terms. In particular, she argues that the "agrammatic" aspect of the speech of these aphasics

is formulable only as a phonological deficit, not a syntactic one.

The generalization Kean suggests is that the speech of Broca's aphasics demonstrates a tendency to reduce to

what is psychologically construed as the minimal string of "phonological words" in the language. Kean points out

that the linguistic definition and psychological construal of phonological words will differ from language to

language, with the result that, if this statement is a true description of the aphasic syndrome, the speech seen in this

type of aphasic may differ in different languages but may still conform to the generalization.

This is but one example of the increasingly active study of aphasic symptoms and syndromes in precise linguistic

terms. Other studies include those of Bradley et al., 1979, Caramazza and Zurif, 1978, Dennis, 1979; Goodenough et

al., 1972; Scholes, 1978. The virtue of the models used in this approach is that they explicitly designate theoretical

constructs that are derived from general theories of language structure. These are incorporated in a neurolinguistie

theory in which abnormal operations can be specified and applied to language disorder. The components involved

and their input/output relations can be specifically stated. This allows precise descriptions of the dynamics


underlying performance, and, equally importantly, may provide serious hypotheses about what the neural code

actually encodes.

The limitation currently evident in these models is that they are not integrated into general process models. We

have chosen the term "representational" to designate these models because many, though not all, focus on the

linguistic representations stored and utilized in aphasic syndromes, with less emphasis on the processing routines

that operate on these representations (see, however, Bradley et al., 1979 and Kean, 1979 for a discussion of

processing). This is a remediable situation, occasioned by the nature of the models themselves. The very detail and

precision that make these models appealing from the point of view of descriptive and explanatory adequacy

complicate their introduction into general process models (such as Luria's) that have empirical justification. To our

knowledge, no general computational neurolinguistic process models have yet been proposed. We therefore turn to

Artificial Intelligence to illustrate how such models of language might be developed, and to brain theory to illustrate

the representation of nonlinguistic phenomena in neural tissues in ways consonant with the demands of

computational neurolinguistics.

4. Artificial Intelligence and linguistics

This section reviews work in Artificial Intelligence (Al) providing computational concepts that complement the

representational advances in linguistics. We shall particularly stress these concepts that we believe relevant to the

development of neural models of language. One Al model that has already gained wide acceptance among

psycholinguists is the Augmented Transition Network (ATN) introduced by Thorne, Braley, and Dewar (1968) and

developed by Woods (1970). This model shows how a grammar may be represented not as a transformational

"competence" grammar but rather as an actual parsing system that can make initial parsing decisions with each

initial segment of a word-string. Wanner and lvtaratsos (1978) have offered an ATN model of relative clause

comprehension; Bresnan (1978) has suggested an ATN model at the heart of a radical recasting of transformational

grammar that transfers most power from transformations to the lexicon; while lvlarslen-Wilson (1976) reports on

shadowing experiments that argue for constraints on linguistic representations due to "on-line" processing by a

hearer, although he cautions against uncritical incorporation of the ATN "metaphor" into psycholinguistic theory.

Winograd (1972) developed an Al system that could analyze simple English sentences to obey commands,

answer questions, or update a data base. He used an ATN-type grammar that could call on semantic information to

cut down the number of possible parsings. The Winograd system was designed to analyze sentences

[455]about blocks on a table. Each time the syntactic routines had posited that certain words formed a noun group, for

example, the program would call a semantic routine to check whether this noun group could actually describe

something in the system's table-top "world." This computational device captures something of the way in which

people can cut down the number of interpretations of a sentence on the basis of pragmatic information.

In addition to providing the words of the input with syntactic labels in a manner consistent with the rules of the

grammar - with the string being accepted as a sentence only when such a labelling can be found - text understanding

involves a translation process that yields a representation of the intention behind an utterance. If the input is a

command, the system must recognize it as such and translate it into a program for carrying out the command. This

process may well involve first translating the command into an internal representation, and then calling upon

planning processes to translate this into a detailed program of action tailored to the current situation. If the input is a


question, the system must recognize it as such and translate it into a program for retrieving relevant information

from its data base. A second translation is then required to express this information as an answer in natural language.

Transferring this perspective from the computer to the human being, we can suggest that the task of speech

perception is to organize a string of words into pieces that map naturally into the internal processes that constitute

the person's response to the utterance - while production serves to recode a "brain representation" into a syntactically

correct string of words. We thus stress translation between "internal" and linguistic representations of meaning.

While the above material indicates the growing utility of AI studies for psycholinguistics, it does not address the

specifically neurolinguistic question of how a performance may be mediated by the interaction of a number of

concurrently active processes. We thus devote the rest of this section to an exposition of HEARSAY (Erman and

Lesser, 1975, 1980; Lesser et al., 1975), an Al speech-understanding system that is explicitly based on the

cooperation of multiple processes. It has not yet fed into psycholinguistic theory, let alone neurolinguistic theory,

but we shall indicate in Section 5 a number of ways the model could be modified to make it a more suitable test-bed

for simulating the neural basis of language.

The HEARSAY-11 system has a very explicit representational structure, based on a set of different levels of

representation. The raw data, whose interpretation is the task of the system, are represented at the "parameter" level

as a digitized acoustic signal. The system will, via intermediate levels, generate a representation at the "phrasal"

level of a description according to a grammar that contains both syntactic and semantic constraints. The combination

of phrasal and lexical information can then be used to generate the appropriate response to the verbal input.

HEARSAY uses a dynamic global data structure called the "blackboard," which is partitioned into the various

levels. At any time in the system's operation, there are a number of hypotheses active at these levels, and there are

links between hypotheses at one level and those they support at another level. For example, in Figure 9 we see a

situation in which there are two surface-phonemic hypotheses, "L . and 'D', which are consistent with the raw data at

the parameter level. The 'L' supports the lexical hypothesis "will," which in turn supports the phrasal hypothesis

"question," while the 'D' supports "would," which in turn supports the -modal question" hypothesis at the phrasal

level. Each hypothesis is indexed not only by its level but also by the time segment over which it is posited to occur,

though this is not explicitly shown in the figure. We also do not show the "credibility rating - that is assigned to each

hypothesis.


Figure 9. Multiple hypotheses at different levels of the HEARSAY blackboard (Lesser at al.. 1975.

Reprinted with permission from the Institute of Electrical and Electronics Engineers, Inc. and the authors.)

HEARSAY also embodies a strict notion of constituent processes and provides a scheduler to control the activity

of these processes and their interaction through the blackboard data base. Each process is called a knowledge source

(KS) and is viewed as an agent that embodies some area of knowledge and that can take action based on that

knowledge. Each KS can make errors and create ambiguities. Other KS's cooperate to limit the ramifications of

these mistakes. Some knowledge sources are grouped as computational entities, called modules, in the final version

of the HEARSAY-II system, The knowledge sources within a module share working storage and computational

routines that are common to the procedural 7mputations of the grouped KS's.

HEARSAY is based on the "hypothesize-and-test" paradigm that views solution-finding as an iterative process,

with each iter ion involving the creation of a hypothesis about some aspect of t problem and a test of the plausibility

of the hypothesis. Each step rests on a priori knowledge of the problem, as well as on previously • generated

hypotheses. The process terminates when the best consistent hypothesis is generated, thereby satisfying the

requirements of an overall solution.

Figure 10. The C2 configuration of HEARSAY-11. The levels are represented by the solid lines, labelled at

the left. The KS's are represented by the circle-tailed arrows and are linked to their names by the dashed

lines. Each KS uses hypotheses at the tail-end level to create or verify hypotheses at the head-end level.

(Erman and Lesser, 1980. Reprinted with permission from Prentice-Hall, Inc. and the authors.)

The choice of levels and KS's varies from implementation to implementation of HEARSAY, which is thus a class

of models or a modelling methodology rather than a single model, (In fact, the HEARSAY methodology has been

used in computer vision with picture-point/line-segment/region/object levels replacing the

acoustic/phonetic/lexical/phrasal levels of the speech domain; Hanson and Riseman, 1978.) The C2 configuration of

HEARSAY-II is shown in Figure 10. We see that each KS takes hypotheses at one level and uses them to create or

verify a hypothesis at another (or possibly the

[457]

same) level. In this particular configuration, processing is bottom-up from the acoustic signal to the level of word

hypotheses, but involves iterative refinement of hypotheses both bottom-up and top-down before reaching a phrasal

hypothesis that is given a high enough rating to be accepted as the interpretation of the given raw data

I


As we have seen, the 'ICS's cooperate via the blackboard in this iterative formation of hypotheses. In HEARSAY,

no KS "knows" what or how many other KS's exist. This ignorance is maintained to achieve a completely modular

KS structure that enhances the ability to test various representations of a KS as well as possible interactions of

different combinations of KS's.

The current state of the blackboard contains all current hypotheses. Subsets of hypotheses represent partial

solutions to the entire problem. A subset of hypotheses is defined relative to a contiguous time interval. A given

subset may compete with other partial solutions or with subsets having time intervals that overlap the given subset.

We thus regard the task of the system as a search problem. The search space is the set of all possible networks of

hypotheses that sufficiently span the time interval of the utterance connecting hypotheses directly derived from the

acoustic input to those that describe the semantic content of the utterance. The state of the blackboard at any time,

then, comprises a set of, and possibly overlapping, partial elements of the search space. No KS can single-handedly

generate an entire network to provide an element of the .earth space. Rather, we view HEARSAY as an example of

"cooperative computation": the KS's cooperate to provide hypotheses for the network that provides an acceptable

interpretation of the acoustic data. Each KS may read data, add, delete, or modify hypotheses, and attribute values of

hypotheses on the blackboard. It may also establish or modify explicit structural relations among hypotheses. The

generation and modification of hypotheses on the blackboard is the exclusive means of communication between

KS's.

Each KS includes both a precondition and a procedure. When the precondition detects a configuration of

hypotheses to which the KS's knowledge can be applied, it invokes the KS procedure, that is, it schedules a

blackboard-modifying operation by the KS. The scheduling does not imply that the KS will be activated at that time,

or that the KS will indeed be activated with this particular triggering precondition, because HEARSAY uses a "focus

of attention" mechanism to stop the KS's from forming an unworkably large number of hypotheses. The blackboard

modifications may trigger further KS activity - acting on hypotheses both at different levels and at different times.

Any newly generated hypothesis would be connected by links to the seminal hypothesis to indicate the implicative

or evidentiary relation between them.

Each hypothesis has an associated set of attributes, some optional, others required. Several of the required

attributes are: the name of the hypothesis and its level; an estimate of its time interval relative to the time span of the

entire utterance; information about its structural relationships with other hypotheses; and reliability ratings.

Changes in validity ratings reflecting creation and modification of hypotheses are propagated automatically

throughout the blackboard by a rating policy module called RPOL. The actual activation of the knowledge sources

occurs under control of an external scheduler: the -schedule KS" (Figure 11). The schedule KS constrains KS

activation by functionally assessing the current state of the blackboard with respect to the solution space and the set

of KS invocations that have been triggered by KS preconditions. The KS most highly rated by the scheduler is the

one that is activated by the scheduler (Hayes-Roth & Lesser, 1977).


Figure 11. HEARSAY-II architecture. The blackboard is divided into levels. Each KS interacts with just a

few levels. A KS becomes a candidate For application if its precondition is met. However, to avoid a

"combinatorial explosion- of hypotheses on the blackboard, a scheduler is used to restrict the number of

KS's that are allowed to modify the blackboard (Lesser and Erman, 1979. Reprinted with permission from

the authors.)

This account by no means exhausts the details of HEARSAY-II, but it does make explicit a number of properties

that suggest that it contains the seeds of the proper methodology for combining the best features of the faculty and

process models with those of the representational models. First, we see in it the explicit specification of the different

levels of representation and an interpretive strategy Wherein components interact via the generation and

modification of inultiple tentative hypotheses. This process yields a network of interconnected hypotheses that

supports a satisfactory interpretation of the original utterance. Second, HEARSAY exhibits a style of "cooperative

computation" (Arbib, 1975, sect. 5). Through data-directed activation. KS's can exhibit a high degree of

asynchronous activity and parallelism. HEARSAY explicitly excludes the direct calling of one KS by another, even

if both are grouped as a module. It also excludes an explicitly predefined centralized control scheme. The multilevel

representation attempts to provide for efficient sequencing of the activity of the KS's in a nondeterministic manner

that can make use of multiprocessing, with computation distributed across a number of concurrently active

processors. The decomposition of knowledge into sufficiently simple-acting KS's is intended to simplify and

localize relationships in the blackboard.

Two other observations come from the studies of Al models in general and from recent psycholinguistic

approaches. First, a grammar interacts with and is constrained by processes for understanding or production. Second,

linguistic representations and the processes whereby they are evoked interact in a "translation" process with

information about how an utterance is to be used. Al and psycholinguistics thus provide a framework for considering


an ever-widening domain of concern, beginning with the narrowly constrained mediation by the linguistic code

between sound and meaning, and extending to include processing and intentional concerns.

5. Computational neurolinguistics and neuroscience

We begin with a consideration of the implications of the computational process models we have outlined for the

sort of regional analysis of neural organization pertinent to language. We turn to HEARSAY as a well-defined

example of a cooperative computational model of language comprehension and observe that incorporating it into a

neural framework raises several important conceptual questions. We present four of these, along with a brief

discussion that sheds light on possible neural mechanisms at this regional level. We then turn to comments on the

status of neurolinguistics vis-i-vis modern cellular neurobiology.

First, in HEARSAY, changes in validity ratings reflecting creation

[457]

and modification of hypotheses are propagated throughout the blackboard by the rating policy module, RPOL.

These ratings are the basis for the determining by a single schedule KS, which blackboard-manipulating KS will

next be activated This use of a single scheduler seems -undistributed" and "nonneural" - in a brain region, one may

explore what conditions lead to different patterns of activity, but not to scheduling. However, the particular

scheduling strategy used in HEARSAY is a reflection of the exigencies of implementing the system on a serial

computer. Serial implementation requires us to place a tight upper limit on the number of activations of KS's, since

they must all be carried out on the same processor. The HEARSAY methodology can, however, be extended to a

parallel 'implementation" in the brain, in which we may view each KS as having its own "processor" in a different

portion of the brain. When viewing a HEARSAY system as a neural model, we can think of RPOL as a subsystem

within each schema or KS, so that the propagation of changes in validity ratings can be likened to the relaxation

procedures in neural nets that we shall discuss below,

Second, we have seen that the processes in HEARSAY are represented as KS's and that certain KS's may be

aggregated into modules. It would be tempting, then, to suggest that in HEARSAY-style implementations of process

models, such as that of Luria in Figure 8, each brain region would correspond to either a KS or a module Schemas

would correspond both functionally and structurally, to much smaller units - perhaps at the level of application of a

single production in a semantic template grammar (functionally) or of the activation of a few cortical columns

(neurally). However, a major conceptual problem arises because in a computer implementation a KS is a program,

and it may be called many times, As a result the circuitry allocated to working through each "instantiation" is

separate from the storage area where the "master copy" is stored. But a brain region cannot be copied ad libitum, and

if we identify a brain region with a KS we must ask "How can the region support multiple simultaneous activations

of its function?" We may hypothesize that this is handled by parallelism (which presumably limits the number of

simultaneous activations). Alternatively, we may actually posit that extra runnable copies of a program may be set

up in the cortex as needed.

Such a proposal can be found in Arbib's (1966, 1970) suggestion for a Distributed Information Processing

Machine (DIPM), which provided a new mode of computer organization, originally designed to offer a perspective

on certain patterns of recovery following brain lesions. In the original formulation. the computer was divided into a

central program area in which programs are stored and a surrounding computation area. To execute a program,

DIPM had to assign to it a region in computation space, so that data would be appropriately transformed as they


passed through this region of the network. If the program was used frequently, it would come to "own" a specific

computation region, thus reducing switching time. However, were this specific computation area to be damaged,

DIPM still had the mechanisms available for assigning a new area to the program. Whereas the DIPM model

segregated program space from computation space, the view of cooperative computation sketched above suggests

that we recast the model - forming DIPM Mark II, as it were - to no longer require such a strict separation Instead,

we balance the cooperation of the subsystems in achieving overall goals by the competition, via normal growth and

learning rules, of the subsystems for computing resources (cf. Norman & Bobrow, 1975).

The DIPM model posited that a program will, with repeated use, store in its computation area much useful

material that will ease later computation. For example, if a function is repeatedly evaluated for certain values of the

argument, the machine will store a table for these values. The use of this table will make later computations both

faster and more precise (Figure 12). If a lesion damages the computing area of a program but leaves the program

itself undamaged, the program has to compete for more computing space in order to resume its action. Then if only a

small part of the computing space is removed, this will cause little trouble, However, if the area removed is large -

even if the program remains in some sense intact - there may be major problems of competition for the remaining

resources,

Figure 12, The Distributed Information Processing Machine (DIPM) of cortical reallocation of computing

area for different programs

as well as routing problems caused by the severing of major anatomical pathways carrying messages in and out

of the relevant computation area. In short, such damage may be relatively nonspecific, but will increase as the area

removed increases. At the other extreme, if the sole "copy" of a program is destroyed, then that function, will be

lost, except to the extent that the system can approximate '1 with other functions. We mentioned that, with use, a


program might ild up a table of useful values. If we destroy this table, performance be impaired in both speed and

precision even if the program itself ifs, undamaged and gets access to sufficient new computing space. However, as

repeated use gradually reestablishes the lost table, performance will better and better approximate the level of

performance prior to damage. This approach accords well with those situations in which a series of small lesions

produce less functional impairment than does a single lesion of the same magnitude (Eidelberg & Stein, 1974).

The third "nonneural- feature of current HEARSAY implementations is the use of a centralized blackboard. This

is not, perhaps, such a serious problem, for examination shows that the levels of the blackboard are really quite

separate data structures and that they are linked only via KS's. For each level, we may list those KS's that write on

that level ("input" KS's) and those that read from that level (output" KS's). From this point of view, it is quite

reasonable to view the blackboard as a distributed structure made up of those pathways that link the different KS's.

One conceptual problem remains. If we think of a pathway carrying phonemic information, for example, than the

signals passing along it will encode just one phoneme at a time But our experience with HEARSAY suggests that a

memoryless pathway alone is not enough to fill the computational role of a level on the blackboard. Rather, the

pathway must be supplemented by neural structures that can support a short-term memory of multiple hypotheses

over a suitably extended interval of time.

Finally, we note that the HEARSAY methodology requires that "no KS know what or how many other KS's

exist" - a KS is simply activated whenever data meeting its precondition appear on the blackboard and it writes the

result of its procedures at some appropriate level. In particular, this approach is said to preclude one KS's directly

activating another. But this restriction would seem to vanish once we "neuralize" HEARSAY, for if we decentralize

scheduling and reconceptualize levels as pathways from "input" to "output" KS's, we may certainly view an "input"

KS as sending a signal along the pathway to activate directly an -output- KS.

An immediate research project for computational neurolinguistics, then, might be to approach the programming

of a truly distributed speech-understanding system (free of the centralized scheduling in the current implementation

of HEARSAY) with subsystems meeting

[459]

constraints such as those in our Figure 8 reanalysis of Luria's data. To date, there seems to be no detailed

simulation of this kind, though Hudson's (1977) Ph.D. thesis contains a (nonimplemented) analysis of

neurolinguistic/psycholinguistic data. The analysis is in the spirit of the present paper. Chapter XI of Hudson's thesis

offers a somewhat ad hoc flow diagram for normal performance without reference to neurological data, and then, in

Chapter XII, he defines the effects of neural damage as "buffer threshold alterations," relating these to brain regions

to see if his model predicts the clinical effects.

In the remainder of this section we shall address the possibility of broadening the range of neural mechanisms

involved in linguistics beyond this type of regional analysis. It is striking that the neural mechanisms specified by

neurolinguistic theories are all derived from pre-Sherringtonian neuroscience. None of the modern neuroscientific

analysis of synapses, cells, and circuits has come to play any serious role in neurolinguistics. Cross

neurophysiological observations involving event-related potentials (Desmedt, 1977) and subseizure stimulation of

exposed cortex (Whitaker and Ojemann, 1977) provide the first steps toward the observation of the physiological

events we believe responsible for language processing, but empirical investigations in this area are limited.


To see how cooperative computation models may provide the bridge from neurolinguistics to synapse-cell-circuit

neuroscience, we must first address the fact that modern neuroscience is based on animal experiments, while

language is in the main a peculiarly human attribute [see Cognition and Consciousness in Nonhuman Species, BBS

1(4) 1978] While it would require another lengthy paper to develop this theme, let us here simply outline the thesis

that there are important parallels between visual perception and speech understanding on the one hand, and between

speech production and motor control on the other. The basic notion is that visual perception, like speech

understanding, requires the segmentation of the input into regions, the recognition that certain regions may be

aggregated as portions of a single structure of known type, and the understanding of the whole in terms of the

relationship between these parts. We use the term "schema" to refer to the internal representation of some

meaningful structure and view the animal's internal model of its visually-defined environment - and the human

being's internal model of the state of a discourse - as an appropriate assemblage of schemas (Arbib, 1977; cf. the

slide-box metaphor of Arbib, 1972; for a similar linguistic perspective see Fillmore, 1976). The generation of

movement then requires the development of a plan on the basis of the internal model of goals and environment to

yield a temporally ordered, feedback-modulated pattern of overlapping activation of a variety of effectors (Arbib,

1980) so that the word-by-word generation of speech may be seen as a natural specialization of the general problem

of motor control. A key concept in the analysis of perception and movement is that of the action/perception cycle -

we perceive to get the information on which to base action, but each action extends our sampling of the

environment. A theory of schemas must include an account of the input-matching routines that determine schema

activation and of action routines that enable the system to use the knowledge made available by this activation. This

cycle corresponds to the role of one speaker in ongoing discourse. While there are no direct one-to-one

correspondences between sensory-motor and language computations, there are overall similarities that allow us to

investigate aspects of the neural mechanisms of language by examining the neural mechanisms relevant to

perceptual-motor activity.

Neuroscience has taught us how to trace the coding of information in the visual periphery (Hubel & Wiesel,

1977; Lettvin et al., 1959), how to view the cerebral cortex as being composed of columns or modules between

single cells and entire brain regions in complexity (Hubei & Wiesel, 1979; Mountcastle, 1978, Szentagothai &

Arbib, 1974); how to analyze spinal circuitry involved in motor outflow and the later stages of its cerebral and

cerebellar control (Phillips & Porter, 1977; Cranit, 1970; Eccles et al., 1967. Neuroscience has also taught us about

the role of synaptic change in behavioral plasticity (Kandel, 1978). To some extent these analyses may be integrated

into models of perceptual and motor processes (Figure 13).


Figure 13. Stages in visual perception and control of movement. Many aspects are omitted, as are all of the

important -return- pathways.

We have good neurophysiological data on retinal response to neuronal stimulation and on the different -feature

extraction" processes in a number of visual systems in different animals. At the motor periphery, we have good

neurophysiological data on basic motor patterns, their tuning by supraspinal mechanisms (the Russian school has

been particularly productive here), and the spinal cord rhythm generators and feedback circuitry that control the

musculature. This partial list could he extended and complemented by a list of major open problems at these levels.

The important point here is that near the visual and motor peripheries there is a satisfying correspondence between

cellular analysis and our processing concepts.

In between, the results are fewer and tend to be somewhat more speculative. By what process are the often

disparate activities of feature detectors wedded into a coherent "low-level" representation of the world? How is the

representation integrated into the ongoing internal model (the -schema-assemblage," as we have posited it to be)?

How are the internal models and goals of the organism combined in a planning process that yields the distributed

control programs that orchestrate the motor synergies? We have models for all these processes (Arbib, 1980), but

many are couched in a language closer to Al than to neurophysiology, and the body of available neural data with

which they can make contact is still relatively small. Nonetheless, it does seem to us that progress is well under way

in the neural analysis of "perceptual structures and distributed motor control." We briefly sketch one effort of this

kind.

The problem of visuo-motor coordination in the frog and the toad has yielded to a behavioral analysis coupled

with lesion and single-cell analysis (Ewert, 1976; Ingle, 1976). Didday (1970, 1976) offered a model of competitive

interaction in neural nets consistent with the data of 1970, and Amari and Arbib (1977) developed a general theory


of competition and cooperation in neural nets, which has proved to have much similarity with the relaxation and

constraint satisfaction techniques that have become popular in the AI literature

[460]

for the resolution of conflicting hypotheses (Rosenfeld et al. 1976; Shortliffe, 1976; Waltz, 1978). Experiments

have shown patterns of interaction between tectum and prethalamus, demonstrated the interaction between prey-

approach and predator-avoidance, isolated processes of facilitation and habituation, and demonstrated how obstacles

in the environment can modify appetitive and aversive behaviors. These observations have been integrated into new

models (Lara et al., 1979) in which visuo-motor coordination is achieved by competition and cooperation in neural

nets that meet some constraints of anatomy and physiology at the synapse-cell-circuit level.

Given the development of such models of visuo-motor coordination, one can suggest that the neural circuitry that

carries information of this sort in nonhuman species is homologous to that involved in information systems in the

human being. Microscopic and macroscopic differences between human and nonhuman brains will doubtless play a

role in providing the particular computational content of each species (cf. the discussion of Geschwind in Section 1),

but, in the absence of evidence to the contrary, we may adopt the view that the basic information-carrying elements

(membranes, synapses, cells, circuits, etc.) will be closely related. The task is to provide more empirical support for

particular neural models in animals and to relate the models developed in the animal to comparable perceptual-

motor systems in man, and thence to systems involved in language.

The last of these tasks requires determining the points of contact between the language system and perceptual-

motor systems. We have noted that many workers have been concerned with the relation of aspects of language to

perceptual, motor, and other cognitive systems In this connection, we refer to our comments on Jackson's views of

propositions, Geschwind's approach to the agnosias and apraxias, and Luria's concern with start/stop mechanisms

shared between linguistic and nonlinguistic motor activities- We also note the influence on Luria of Bernstein

(1967), whose work has had such a profound influence on the Moscow school, which combines neurophysiological

and mathematical analyses of motor control with the construction of actual robots (Celfand i Tsetlin, 1962;

Okhotimskii et al., 1979).

We believe that the pursuit of these connections will require a framework of cooperative computation in which,

in addition to interaction between components of a language processor alone, there are important interactions

between components of linguistic and nonlinguistic systems. Consider, for example, Luria's analysis of the naming

of visually presented objects (Figure 41. The link between a representation of an object derived from visual

information and a representation of the object in a linguistic curie must be made. We have seen that, unlike the

simple, unanalyzed analysis of this operation as a "nonlimbic cross-modal association" offered by Geschwind,

Luria's view is that the linguistic system utilizes multiple components to arrive at the appropriate item. Some of

these components (e.g., box D related to articulatory analysis) include representations at remote linguistic levels.

The retrieval of the appropriate word involves the exploitation of the entire network of representations that define a

word, and convergence on the proper word is accelerated by cooperation of the entirety of the components of the

language device. Similarly, one can imagine a comparable cooperative mechanism for the assignment of a

representation to an object from visual data. A fuller analysis of such a process would involve, for example,

distinguishing the 3-D shape of an object from its functional role and might utilize these two different sorts of


information in different routines (one of which, in this example, might plausibly involve information of a "motor"

sort).

It is interesting to consider to what extent our suggestions are similar to those recognized as constraining factors

in neurolinguistic theory by Wernicke, its first exponent. In Section 1, we pointed out that Wernicke insisted that

neurolinguistic theories be consonant with plausible accounts of psychology and physiology. We have, in essence,

reiterated his insistence on these constraints. What distinguishes his theory from the outline of theories we have

proposed here is the greater appreciation of just how rich the accounts of the psychology of language and the

neurophysiology of information- carrying are. Plausible neurolinguistic theories will have to integrate these

discoveries in a synthesis as rich as the concepts in the contributing fields.

ACKNOWLEDGMENTS

Preparation of this paper was supported in part by the Sloan Foundation grant For "A Program in Language and

Information Processing" at the University of Massachusetts at Amherst.

A first draft of this paper was prepared while M. A. Arbib was on sabbatical in 1976-77 at the School of

Epistemics at the University of Edinburgh. The hospitality of James Peter Thorne is gratefully acknowledged, as is

his stimulating discussion of many facets of linguistics.

Open Peer Commentary

Sheila E. Blumstein: Phrenology, "boxology,” and neurology

Department of Linguistics, Brown University, Providence. RI_ 02012 and A asia Research Center, V. A.

Hospital, Boston. MA 02130

It is an interesting phenomenon that in the history of science the theoretical nature of brain processing has

reflected the state of technological advances of the time. A & C's attempt fails into this category, and unfortunately,

will tail as did its predecessors'. To attempt to build a computational model for neurolinguistics by using artificial

intelligence is laudable. There are at least two ways of building such a model: (1) on the basis of known

physiological and neurological facts, to simulate a known process and its behavioral consequences by means of

computer implementation, or (2) on the basis of the state of the art in computer technology, to build a system that

simulates a particular behavioral outcome. In the latter case, the resultant system (usually a brute-force operation)

may or may not have any correspondence to the actual physiological, neurological, or psychological process

involved in a particular behavior. And it is the latter course that A & C so enthusiastically follow. For example, in

their discussion of HEARSAY, they say ". . we may view each KS (knowledge source) as having its own 'processor'

in a different portion of the brain.- Why? Perhaps because in that way the system works; there is certainly no

evidence from aphasia in particular, or neurology. in general, to support the notion of such an underlying system. In

short, computer implementation need not and probably does not (at this time) have anything to do with neurology,

physiology, or psychology. To believe that this is the case requires an existentialist leap from "boxology" to

computer technology to physiology and neurology.

Ao& C review in detail various approaches to aphasia research and are especially critical of the so-called

connectionist view. In particular, they criticize Geschwind's disconnection theory, primarily because it "tails" to

delineate a theory of linguistic processing and brain organization. However, Geschwind's model never attempts to


explicate linguistic processing. It merely tries to show the interrelations among a number of behavioral tasks

(syndromes) and the areas of the brain implicated in performing such tasks.

In fact, the elaboration of a process model of functioning in the manner of Luria does not seem to be as distinct

an approach from that of the connectionists as the authors imply. The process model represents a further analysis of

the tasks of the holists into their functional components, correlating these functions with particular brain structures.

it does not represent a totally new view or analysis of language and brain structure.

The discussion of models of language and the brain culminates for A

[461]

& C ir1 representational models. They specifically discuss the work of Schnitzer and Keaniq.v.1 as examples of

the insights such approaches may provide However, a careful analysis of these two approaches falls far short of what

A & C promise us. In particular, Kean's model fails to account for known and attested linguistic features of Broca's

aphasics that cannot be attributed to a deficit in the phonological component, one that is so morphologically rich that

it in tact encompasses many aspects of grammar traditionally considered morphological in nature. These features

include comprehension deficits based on notions of embedding, syntactic complexity, and semantic reversibility

(Zurif & Caramazza, 1976; Bfumstein, Statlender. Goodglass, & Biber, 1979; Goodglass, Blumstein, Gleason,

Hyde. Green, & Statlender, 1979; Kolk, 1979). As to Schnitzer's phonological analysis, in point of tact, it is not a

model of phonological processing Rather, it represents an ingenious and sophisticated analysis of a task dependent

phenomenon-reading (shades of the connectionists), in one aphasic patient. It neither describes all of the data

obtained (40 to 50 percent) nor does it consider that such errors might be accounted for in terms of surface structure

reading strategies rather than in terms of a complex theoretical framework.

Can Artificial Intelligence contribute to our understanding of language and the brain? Despite all of the negative

comments enumerated above, it probably can - but not at this time and not with the procedures elaborated by A & C

When the state of the art in aphasia is more sophisticated, when physiological, neurological, and psychological

processes can be tied directly to language and cognitive behavior, then we will be ready for computational or

representational modelling of language and the brain.

Hugh W. Buckingham, Jr. Must neurolinguistics be computational?

Department of Audiology and Speech Sciences. Purdue University. West Lafayette, Ind. 47907

There is no doubt that the field of neurolinguistics stands in need of theory. At a 1965 N. 1. Ft conference on

brain mechanisms underlying speech and language (Milliken & Darley, 1967), Noam Chomsky challenged the

largely neuropsychologically oriented group on the grounds that the field of study relating language and the brain

was heavy on the data side and light on the theory side. There, as in many other places (1975, 1976) Chomsky asked

(p. 75), "When a person has command of a language ... what kind of things does he know? What kind of information

is in some fashion represented in his nervous system?" If the notion of "representation" is added to "utilization," we

can approach what A & C are offering in their paper. For them, the central task of neurolinguislics is "the study of

the representation and utilization of the linguistic code in and by neural tissue."

The first question that comes to my mind is the relation between A & C's "linguistic code" and Chomsky's

"things" a person knows when he knows language [see Chomsky: "Rules and Representations 88,5 3(1) 19801. Here

we are immediately faced with a problem, since I see nothing in the linguistics of A & C that looks like what I feel

Chomsky has in mind as the things human beings know when they know a language. Rather, A & C have in mind a


model of the Competence of a performer in the form of programs for computation. They focus on language "tasks,"

if you will; the tasks are familiar to all aphasiologists: object naming, speech production, speech perception,

repetition, writing, reading, and so forth. Chomsky, on the other hand, is interested in the nature of the knowledge

speakers-hearers demonstrate when they judge sentences like "What violins are easy to play sonatas on?" as

grammatical, but "What sonatas are violins easy to Play on?" as ungrammatical_ The representation (or utilization)

of this kind of knowledge is nowhere to be found in A & C's model I do not even see a description of the task "judge

grammaticality," which may Prove more meaningful to model than the tasks considered in the Present paper. Models

such as those proposed by Woods (1970, 1973) and Marcus (1976) come closer to capturing more sophisticated tYPes

of linguistic knowledge, but then they are less clear as to how their models can be related to human brain function. It

almost appears Mai in order io make the model square with the nervous system, its linguistic sophistication is

reduced.

What A & C do, however, is switch psychological metaphors to computer metaphors and then correlate the latter

with the nervous system Presumably, A & C would accuse Chomsky, along with the "hoists" like Hughlings

Jackson, Henry Head, and Kurt Goldstein, of avoiding the central issue of actually locating and describing the

requisite biological structures and processes presumed to mediate mental function. Chomsky has repeatedly claimed

that his work deals only with the abstract conditions that neural structures must meet; he says nothing about what

these structures must or might be, or where they may be.

A & C's model (or methodology, as they say) is more in line with Cognitive Simulation (CS) than with Artificial

Intelligence (AI), although they do not make this clear in their paper [see Pylyshyn: "Computational Models and

Empirical Constraints" BB5 1(1) 1978). Where there is no concomitant claim concerning psychology, many

computer programs can be written to do things. These nonpsychological, nonmetaphorical programs from Al are

devised to solve problems, play games, prove theorems, recognize scenes, and so forth, but the programmers do not

necessarily imply (or care, for that matter) that their programs perform the tasks as human beings would. On the

other hand, Johnson-Laird and Wason (1977, p. 8) write that "Computer simulation takes computers rather literally

as devices for imitating human beings; Artificial Intelligence is not concerned with human behavior except insofar

as it throws light on how to solve such problems as the analysis of visual scenes, or the proving of theorems in an

intelligent way," CS models then clearly make psychological claims.

The parallel processing capabilities of HEARSAY are certainty superior to strictly serial processing; everything

we know leads us to believe that mental processes for language operate in parallel and not in series. On the other

hand, information must be digitized into discrete units to be manipulable by a machine at all levels. Nevertheless, as

Dreyfus (1979, p. 266) writes, "There is no reason to suppose that the human world can be analyzed into

independent elements, and even if it could, one would not know whether to consider these elements the input or the

output of the human mind," It is unlikely that human beings need such machine-ready discrete input, and it is even

more unlikely that all possible "preconditions" can ever be specified by any programmer claiming to simulate

human mental function. A & C write that "Each KS includes both a precondition and a procedure. When the

precondition detects a configuration of hypotheses to which the KS's knowledge can be applied, it invokes the KS

procedure . . ." However, human beings, for example, "perceive" phonological units where there is absolutely

nothing in the physical signal_ How can HEARSAY do this? 1 he same set of acoustic parameters may be perceived

as different phonological units, and different sets of parameters will at times be perceived as the same phonemic


unit. Expectancy sets can greatly influence how human beings perceive things in the world. That is, in different

situations, human beings will act differently on the same stimuli Dreyfus's comments are particularly relevant here.

He writes (1979, p. 258), "Since a computer is not in a situation ... it must treat all facts as possibly relevant at all

times." I interpret A & C's "preconditions" to be analogous to Dreyfus's "facts": according to A & C they must be

known and specified so that the KS's can recognize and act upon them.

Again, "A KS becomes a candidate for application if its precondition is met." Once more, I believe Dreyfus's

criticisms are compelling when he writes (p. 258), "This leaves Al [Dreyfus should have said CS here: HWB)

workers with a dilemma: they are faced either with storing and accessing an infinity of facts, or with having to

exclude some possibly relevant facts from the computer's range of calculations." The whole situation is even more

puzzling when we consider that Dreyfus, who more than anyone else has criticized CS, is a phenomenologist, and

yet A & C propose that we deepen our phenomenological and neural analyses in order to bridge the conceptual gap

between clinical neurological and psycholinguistic approaches to language. By deepening phenomenological

analyses, many have actually been led further away from the physical system.

I must admit that I am not sure what we learn from A & C's "Distributed Information Processing Model"

(actually developed by

[462]

certain patterns of recovery following brain lesions." There has been much recent neuropsychological and

neurophysiological work devoted to studying patterns of recovery of function, (Stein, Rosen, & Butters, 1974;

Finger, 1978). It is from these studies that we gain our understanding of vicarious takeover and "competition for

undamaged cortex," of development of alternative connective routing and the like. Constructing a computer program

to model these phenomena seems to me to be more of an exercise in engineering than anything else. In this vein it is

interesting to note that even Geschwinct, the most avid modern-day localizer and cerebral connectionist, has

reservations concerning too literal an interpretation of the computer metaphor. He writes (1972, p. 761), "I would

have to accept the view that what one might call the realization in hardware of an axiomatic system would not

necessarily be, so to speak, isomorphic with that system. One would hardly expect that damage to an ideal computer,

capable of deriving theorems of Euclidean geometry, would produce loss of individual axioms."

I have no doubt that physiological recovery of function can be understood in terms of (i.e., in the vocabulary of)

analyses of synapses, cells, circuits, neurochemistry, and so forth; I have serious reservations as to whether this level

of description will tell us anything at all about how human beings recover linguistic function qua linguistic function.

Imagine what we would have to do with what we know about language in order to analyze it (or its recovery) in the

terminology of synapses, cells, and circuits. I feel that whatever we might do. the result would be quite uninspiring

for the linguist, or perhaps even for the "experimental mentalist" (Fodor, Bever, & Garrett, 1974, p. xvii).

In addition, A & C seem to have a strange view of the physiological events for language processing_ Referring to

the electrophysiological studies of Whitaker and Ojemann (1977), A & C say that, "Gross neurophysiological

observations involving . . . subseizure stimulation of. exposed cortex provide the first steps towards observation of

the physiological events we believe responsible for language processing . . ." What Whitaker and Ojemann showed

was that (in the spirit of Penfield & Roberts, 1959) stimulating various relatively discrete regions within the general

(a bit larger than we once thought) left frontotemporoparietal language cortex could produce naming blocks, partial

naming blocks and no naming blocks at all. As a consequence, the notion of a "graded localization" for this task e.,


naming) was established. Nowhere do we observe the physiological events for language processing. At best, we see

the quantity and location of an outside electric current necessary to induce blocked access to the lexicon in a naming

task.

The neuropsychologicai model that A & C incorporate in their work is taken from A. R. Luria, the Russian

neuropsychologist. ft is a "process" model, which conceives of functional centers working synergistically as a

complex whole. It is aimed at describing the combined cerebral processes underlying the performance of such

language tasks as those mentioned above, among other neuropsychological phenomena. No task is located in any

one center; tasks are viewed as performed through the integrative processing of various centers working

concurrently. In this way, the tasks are considered analyzable into separate components. Nevertheless, unlike A & C,

I do not consider Luria's view of tasks as analyzable to be so different from the classical localizers, that is. when you

scrutinize what their models implied.

A & C feel that the classical localizers simply placed the various tasks in centers and left them as unanalyzable

wholes. Nothing could be further from the truth. For instance, Wernicke would never have localized reading in any

one area; nor would he have failed to appreciate that the process had several crucial subcomponents. In addition,

Lichtheim (1885) should certainly not be accused of considering tasks as unanalyzable and strictly located. Consider

the task. "understanding of spoken language," with reference to A & C's Figure 2. Lesions at M, between M and A,

between B and M, and between M and m did not result in the loss of this task, while lesions at A, between A and B,

and between A and a did so. Lichtheim was aware that there were several stages involved in the comprehension of

verbal acoustic input; the pathway up the acoustic tract from a to A had to be intact as proper understanding of

spoken language, the interaction between the well as the cortical area for auditory representations. Furthermore, for

auditory cortex A and the more conceptual cortical regions 8 had to be unimpeded. Clearly, then, this task could not

have been understood by Lichtheim to be an unanalyzable whole located in some single center. Rather, for the

understanding of linguistic auditory stimuli, Lichtheim analyzed the task into three minimal subcomponents:

subcortical auditory innervation, cortical auditory images, and conceptual associations. All interacted for the total

performance of understanding spoken language. The same reasoning applies to the other tasks.

One final point I would like to make is that not everyone is as fast as Luria to localize on the basis of lesion data.

To understand this, we must consider two versions of the localization hypothesis. A weak version can be considered

a "functional vulnerability" position, which simply states that function X is vulnerable to disruption secondary to

pathological involvement in a certain region of the nervous syStem and not in others. The overriding difficulty here

has usually been due to unclear, vague or incomplete descriptions of function X, as well as the frustrating lack of

discreteness at the boundaries of the regions involved. If function X is understood as an unanalyzable whole psycho-

linguistic task, then clearly problems accrue. If function X is considered to be some analyzed component part of a

task, then the whole enterprise is strengthened. Nevertheless, what we know from Luria is still based upon lesion

evidence. The strong version would go beyond the functional vulnerability statement to the claim that function X is

located in the specific region itself, Luria localizes, but he localizes components of functional tasks. The weak

version was a reaction to the strong version and was formulated at least by the time of Jackson (1874) whn wrote,

"To locate damage which destroys speech and lO locate speech are two different things." This has been echoed by

historians of neurology (Riese. 1959, p. 148) and more recently by Zangwili (1978, p. 126).


In conclusion, I think that the computerization of neurolinguistics has some definite obstacles—both practical

and conceptual—to overcome. As long as the programmers never lose sight of the fact that the whole enterprise is

metaphorical, and that the mind is not the brain nor is reducible to it (see Puccetti & Dykes: "Sensory Cortex and the

Mind-Brain Problem" 885 1(3) 19781, then work will no doubt continue and is likely to produce some new

questions and hypotheses. There will be many of us, however, who will continue primarily to study the linguistic

nature of aphasia and recovery from it. For this reason, my answer to the question "must neurolinguistics be

computational?" would be "No, but some may choose to do it that way.”

Gillian Cohen Are computational models like HEARSAY psychologically valid?

Department of Experimental Psychology, UnlVeranty of Oxford. Oxford OX r 3U0, England

A & C's proposal that the vague and poorly specified models of language processing employed by neurologists

should be replaced by the much more precise and detailed models developed by computer science has much to

recommend it. But they have overlooked some of the difficulties that must arise in marrying neurolinguistics to

Artificial Intelligence (Al); the suitability of HEARSAY as the chosen bride for this marriage is questionable. Al

offers us a number of different models, each designed to perform a limited subset of language processing tasks.

Neurafinguistics requires a general, comprehensive model incorporating the whole range of human language

abilities such as understanding speech, answering questions, producing stories, describing the perceptual world, and

holding conversations. A system that can perform only a few of these tasks may be qualitatively as well as

quantitatively different from the human language-user The problem is twofold: to select which of Al's task-specific

models could be generalized to the required extent; and to assess how tar such a model corresponds to human

performance. A computational model of language processing, however precise and however efficiently it operates,

will be no he to neurolinguistics if the procesSes and representations it utilizes differ from those employed by the

human language' user. A & C have attempted to confront this problem by assessing how far HEARSAY is

consistent with neurophysiology, but, although paying

[463]

hp service to Wernicke's insistence that models be consistent with Physiology and psychology, they have made no

effort to consider how well HEARSAY conforms to the findings of cognitive psychology and psycholinguistics.

In fact, comparison of the HEARSAY system with human speech recognition reveals a number of differences.

One of the most striking is the degree to which the two systems rely on top-down contributions from the knowledge

sources. The information HEARSAY derives from initial acoustic analysis el the input is crude, sketchy, and prone

to error. Recognition is heavily dependent on the refinement and amplification of this information by the subsequent

hypothesize-and-test procedures, which in turn depend on prior knowledge of syntax and semantics- In the human

speech recognizer, the lower level stage of phonemic analysis is considerably more efficient. We can identify

sentences spoken out of context and we can identify anomalous or ungrammatical sentences, random strings of

words, or even nonsense syllables- That is, human speech recognition can operate reasonably well without invoking

prior knowledge of syntax or semantics. Of course, recognition is facilitated, and is faster and more accurate, when

input sentences conform to syntactic and semantic constraints. Nevertheless, the ability to extract relatively rich and

unambiguous phonemic information from the signal renders the human system much less dependent on hypothesis

testing than on HEARSAY.


Furthermore, man/machine differences in speed of processing and in memory capacity render it unlikely that the

human speech recognizer could operate hypothesize-and-test procedures to the extent that HEARSAY does. The

capacity of HEARSAY to store and schedule multiple hypotheses exceeds the known capacity of human working

memory. In the human system the rapid decay of acoustic information means that it quickly becomes unavailable for

retesting against revised hypotheses. The human speech recognizer must handle a fast continuous input, and

recognition must not lag far behind the speaker. Although we may sometimes revise earlier decisions in the light of

later information, we cannot postpone decisions for long. Unidentified signals are bumped out of store by

subsequent input. Machine systems have a much greater capacity for backtracking.

Another problem in the application of Al models to neurolinguistics arises out of the difference in the size of the

data base. A! systems are restricted to small vocabularies and limited semantic domains. By contrast, the knowledge

structures, linguistic, semantic, and oragmatic, accumulated by the human language-user over a lifetime are

enormous_ The set of all possible hypotheses is so extensive that if an Al system were equipped with knowledge

sources on a comparable scale the problem of search would be much more acute, and more powerful heuristics

would be required for scheduling and rating hypotheses, In human speech recognition the search is quite tightly

constrained at the outset by the information derived from phonemic analysis. But in HEARSAY the relatively

impoverished phonemic information does not serve to constrain the search procedures to the same extent, and it is

doubtful whether the system could function if it were endowed with knowledge structures on the human scale.

ewe are to use computational models in neurolinguistics it would be as well to consider how far they have

psychological validity.

Lyn Frazier Constraining models in neurolinguistics

Department of Linguistics, South Collage, University of Massachusetts, Amheret, Mese 01003

A & C's classification of neurclinguistic models as faculty models, process models, and representational models

clearly illustrates the need for a neurolinguistic model based on both a rich theory of linguistic knowledge and a

detailed specification of the particular processing routines employed by seeakers and hearers. Though I agree with A

& C that computational process models can be used to fill this need, I think that they do not pay sufficient attention

to the constraints that must be placed on these models. As a result, the article invites the conclusion that Progress in

neurolinguistics is simply a matter of applying techniques from Artificial Intelligence (Al) to model the

computational processes used to manipulate linguistic representations. In its most extreme form, this view implies

that the task of developing an adequate model for language processing can be achieved simply by using any Al

techniques available to "implement" the grammar, that is, the view gives rise to the notion that we can construct a

model for language processing by merely finding some (arbitrary) way of interpreting the grammar as a process

model_

1 do not mean to imply that A & C either maintain or condone the extreme form of this position; in fact, they are

careful to note that the linguistic representations and the components of a computational process model must be

motivated by empirical evidence. However, the available evidence concerning language processing surely constrains

our models in many respects beyond those that A & C mention. For example, a large body of psycholinguistic

evidence concerning systematic errors in speech production and perception, uniform preferences for one

interpretation of an ambiguous construction, differences in the relative processing complexity of similar

constructions, and so forth (cf. Fodor, Bever & Garrett, 1974, Frazier, 1978 and references therein), provides


valuable information about the size of the units important in speech processing, the nature of the processing routines

employed, and so forth. Hence, this type of evidence places rather stringent constraints on models of language

processing. These constraints are got necessarily taken into account on the above characterization of the task of

neurolinguistics. In short, A & C's strong emphasis on the use of Al techniques to develop computational process

models, together with their lailure even to mention the variety of ways that empirical data constrain these models,

can all too easily lead to a situation where ease of computer implementation, rather than empirical considerations

(e.g., the psychological and neurological data), are used to motivate and evaluate proposals about human language

processing_ This is not to deny that Al can make important contributions to neurolinguistics but simply to stress that

these contributions derive from its capacity to suggest hypotheses, clarify issues, help render empirical models of

language processing more explicit, and test the predictions of such models once they have been developed

One major contribution of A & C's article is their identification of a serious problem it the field of

neurolinguistics, namely, the existence of a "conceptual gap between existing clinically oriented theories of the

neural mechanisms pertaining to language and recent psycholinguistic approaches to the field " They suggest that

"this gap must be filled by deepening both the phenomenological and neural analyses" of language processing.

However, it is not obvious how merely deepening our analyses will serve to bridge this gap_ It seems to me that this

gap arises because in practice psycholinguists and neurolinguists (or clinicians) typically rely on two different

sources of data when they are constructing models of language processing. This results in two different classes el

models: one primarily concerned with accounting for aphasic syndromes and the relation of language processes to

neural mechanisms, and one primarily concerned with the psychological mechanisms underlying normal

(undisrupted) language processing. If this is in fact the source of the problem, then bridging the gap between these

two classes of models entails constructing models for normal language processing with concern for the relation of

such models to neural mechanisms and their ability to account for observed aphasic syndromes. That is,

psycholinguists and neurolinguists must take seriously the fact that they are constructing just a single model of

human language processing. Since it is only in cases where the disruption or loss of mechanisms involved in normal

language processing will not account for the linguistic behavior of brain-damaged individuals that an independent

"neurolinguistic or clinical" theory must be invoked, surely Occam's razor dictates that at present the best theory of

disrupted language function is no theory at all. Of course, ultimately an account of disrupted language processing

may well have to include some statements about information or mechanisms acquired or adapted to cope with loss

or reorganization of normal language processes, but surety these statements should be added to our theory only once

it has been shown that an adequate theory of normal language processing will not automatically account for the

linguistic behavior of aphasics.

Bradley's (1978) investigation of lexical access procedures provides a rather striking example of how this

approach can be used

[464]

to eliminate the gap between clinical and psycholinguistic models of language processing. After specifying in

some detail the mechanisms that normals rely on to access closed class lexical items, Bradley was able to show how

the disruption of this normal system automatically accounted for the behavior of a group of aphasic patients. Her

concern for the actual mechanisms underlying normal processing relieved her analysis of the need to postulate

independent or additional theoretical entities to account for disrupted language functioning. And her concern for the


clinical data allows us to formulate at least a good working hypothesis concerning what region of the brain is

crucially involved in supporting one aspect or component of normal language functioning.

If this type of investigation is what A & C have in mind when they look to "deeper" phenomenological and

neural analyses to fill the gap between clinical and psycholinguistic models of language, then I agree with them

completely_ However, it seems to me that what differentiates an investigation like Bradley's from many previous

investigations is not simply a deeper analysis of the phenomena studied, but the constrained nature of her model.

Frank R. Freemen Computers are dumb.

Department of Neurology. Vanderbilt School of Medicine and Neurology Service, VA Medical Center,

Nashville, Tenn, 37203

The ideas outlined by A & C in their abstract certainly sound worthwhile. I am a member of what they call the

faculty school of language function (the term faculty has been pejorative since the last days of phrenology) and must

admit that lately we have not made much headway. Sometimes it seems that the study of human language has been

displaced by the study of ape communication; the latter even rated an entire issue of Eses_

But when A & C get down to the details, they come up blank. They hide this emptiness with a series of scientific

sounding jargon terms: "X otters an ATN model of relative clause comprehension; Y suggests an AIN model at the

heart of a radical recasting of transformational grammar which transfers most power from transformations to the

lexicon; while Z reports on shadowing experiments which argue for constraints on linguistic representations due to

on-line processing by a hearer, though he cautions against uncritical incorporation of the ATN metaphOr into

psycholinguistic theory." From my reading of this paper, the field of so-called artificial intelligence as presently

constituted has nothing whatsoever to do with brain function and even provides a poor analogy.

by Howard Gardner

Veterans Adminianedian Hospital, Beaton, Maas. 02130. Boston University School of Medicine, Boston,

Mass. 02215, and Project Zero, Harvard University, Cambridge, Mess. 02138

Computational neurolinguistics: promises, promises. In their interesting, it somewhat provocatively entitled,

essay. A & C have in fact written two loosely connected papers, plus a coda. The first paper treats the history of

aphasiology in three acts; the second describes some current efforts in artificial intelligence to model aspects of

language; the coda considers possible relations between knowledge of brain function (particularly at the cellular

level) and models of linguistic processing. Each of these portions merits separate comment, but before turning to

that task, some considerations of scientific method seem appropriate.

In my view (and A & C at least pay lip service to such a conception), scientific progress depends upon a constant

and productive interaction between phenomena in the world that demand to be understood and theoretical constructs

and models that serve as candidate descriptions and explanations of these phenomena. Neither phenomena alone, nor

constructs alone, will suffice. Excessive reporting of what has occurred, of the sort found in a detailed case study,

may provide a rich feeling for a phenomenon, but, like a literary work, yields little insight into the processes

undergirding the phenomenon. Unrelieved theorizing risks the construction of castles in the sand: the resulting

edifice may be lovely to behold, but a few hard or annoying facts can cause the whole structure to founder. The

scientific enter prise advances when intriguing phenomena and careful theorizing inform one another; from such

interplay come bette descriptions, more powerful predictions, and increasingly valid explanatory accounts. Only in

the long run can one know which direction of research will yield such a happy result.

4


In the first part of their paper, A & C describe three approaches to aphasiology that are said to differ in terms of

specificity, sophistication, and desirability. We all need strawmen and heroes, but A & C's sketch may have veered

too far in the direction of caricature_ At least it is not clear to me that the differences among the connectionist,

process, and representational models are principled ones, and if so what the differentiating principles are_

A & C describe the connectionist model as focusing on "whole" language functions, without either breaking

these functions down into component parts or considering the interactions among them. They also irdicate a lack of

concern on the part of the connectionist about the mechanisms that underlie and order linguistic processes. They

contrast the traditional connectionists with Luria. whom they view as recognizing the subcomponents of linguistic

functions. acknowledging the interactions among these components, and insisting that particular functional units can

be involved in a variety of linguistic and nonlinguistic activities.

Whle f would allow some differences in emphases between the "connectionist" and the "processor" approaches, I

think that in tact Wernicke and Geschwind on the one hand, and Luria on the other, are involved in quite similar

enterprises. One could make a case for either of them fitting either description, depending primarily on the level of

analyses (and the kinds of diagrams) with which each is concerned at a particular moment. When either is discussing

a specific lin istic function (cf. Geschwind [1962) or Wernicke (1885) on reading, Ge h-wind 11965] on varieties of

naming errors). there is considers appreciation of the subcomponents involved in the process and their\ potential for

involvement in various performances. Moreover, Geschwind's emphasis upon syndromes is no less than an insstence

that there may be certain intuitively unrelated behaviors that reflect the same underlying processes. Thus, in his

discussion of alexia without agraphia, Geschwind (1965) notes the relative sparing of number reading and object

naming in the presence of impairments of color naming, word reading, and the deciphering of musical notation.'

Such an analysis proceeds from a recognition of the subprocesses (e.g., tactile association) that figure in linguistic

behaviors and can hardly be considered a wholesale acceptance of the received categories of linguistic analysis On

the other hand, when either the connectionists or the p-ocessors are discussing language at a general level, there is an

(understandable) tendency to revert to undecomposed (or holistic) characterization of functions like reading,

naming, or even auditory comprehension. In tact, it seems to me that the major difference between these two schools

lies in an area ignored by A & C. the developmental bias of the Russian school and its concomitant endorsement of

the notion of hierarchical relations among functioning systems, relations that can be altered through development or

dissolution of neural structures.

A & C also overstate Wernicke's localizing tendencies. In paraphrasing Luria's contributions, they remark that

"Both systems participate in all brain functions which require exploitation of the networks of representation that

define a word within the brain." But compare this with two statements by Wernicke: "Any higher psychic processes

exceeding these primary assumptions cannot be localized but rest on the mutual interaction of these fundamental

psychic elements which mediate their manifold relations by means of the association fibers . (1885, 177-178) "The

cerebral hemispheres, as a whole, in tato, as an organ of consciousness, function as the executor of the motor speech

center in spontaneous speech production." (Ibid., 178-179) Again, it would be hard to know who said what.

Both connectionist and process models are compared, to their detriment, with the representational approach.

Instances of this later model, unlike the earlier "pretheoretical" or "theoretically naive views, are based on current

linguistic theorizing and are assumed. on that account, to be superior_ I want to stress that t endorse theoretical

[465]


efforts in contemporary linguistics; any neurolinguist should be apprised of them. But I refuse to go to the next

step and to assume that work based on those theories is inherently preferable to that based on more commonsense,

less abstract (and abstruse) notions. Certain Phenomena unlikely to have engaged the interest of those wedded to

current linguistic theory - for example, the selective ability of patients with comprehension deficits to follow whole

body commands, or the variety of possible impairments associated with repetition in aphasia -nave turned out to be

highly illuminating for neurolinguistics and even to to provide information relevant to current theoretical

distinctions (cf. Davis et al.. 1978). In turn, who is to say that the many subtle distinctions propounded in

contemporary generative grammar will Prove to have any greater relevance to neurolinguistics than they have had to

developmental psycholinguistics (cf_ Bates, 1976; Brown, 1973; Fodor & Garrett, 1966)? [See Chomsky: "Rules

and Representations" BBS 3f 1) 1980.]

In fact, t found the discussion of the representational theories very disappointing. A & C seem to have done little

more than to report some apparently inconclusive (if not conflicting) efforts to explain phonemic paraphasias and

paratexias. as well as to cite the controversial efforts by Kean to reduce Broca's aphasia to a phonological

disturbance (ct_ Kolk, 1978). The other work referred to, much of good quality, does not seem to depend upon, or to

be greatly illuminated by, advances in linguistics of the past decades_ 1 finished this section unconvinced that recent

psycholinguistic studies of aphasia have added much to those carried out by pretheoretical predecessors. (In fact, a

major achievement of current studies, whether theoretically motivated or not, has been to suggest which concepts of

linguistic theory seem tenable.) Nor do I feel that either the connectionists or the processors would and much to

disagree with in A & C's account of how neurolinguistics should proceed. It sounds just like common

"pretheoretical" scientific procedure to me_

In the second part of the paper, A & C introduce an elaborate model (HEARSAY) of how speech understanding

might be carried out by a machine. Though A & C are sensitive to the fact that language may be 'integrally tied to,

and make use of, other behavioral systems, they take it as a working premise that one should begin by modelling

language functions separately. I am sympathetic to this simplifying assumption, but feel that it places the ultimate

validity and relevance of this model very much in jeopardy. In particular, the role played by various pragmatic and

cognitive systems in language comprehension is largely ignored.

But there is a more serious worry surrounding such an approach Even if the computer, which carries out these

functions, can perform in a humanlike way, it remains an open question whether this is the way the brain (or the

mind) actually carried out such operations In other words. HEARSAY and its variants exhibit the limits of all efforts

of Al (see Pyhyshyre "Computational Models and Empirical Constraints" BBS 1(1) 1978].

In what strikes me as the most innovative part of the paper, A & C discuss some of the problems that must be

circumvented before HEARSAY can be consistent with what is known or suspected about the functioning of the

brain_ Here. I feel, lies a potentially productive interplay between an elegant, abstract model and some phenomena

worthy of elucidation. But as I suggest facetiously in the title of these remarks, the crackling of promissory notes

pervades this whole section of the paper. Neurolinguistics may someday be computational (whatever that may

mean), but I am hardly persuaded that it must be 80 - any more so than it must be transformational, or phys:olcgical,

or Phenomenological.

The coda touches on possible links between the microstructure of language functions and the microcellular

organization of the nervous System. Discussions on this issue probably excite even the most Jaundiced worker, and


A & C are right to single out this line of study as especially vital. But even as they recognize that facts about the

brain must constrain computational models of language, I would wish to add that knowledge about neuropsychology

should continue to constrain theories of brain function, and that acquaintance with intriguing Phenomena should

continue to guide neuropsychology

Merrill Garrett and Edgar Zurif Neurolinguistics must be more experimental before it can be

effectively computational

Department of Psychology, Massachusetts taatituta of Technology. Cambridge, Mass. 02139; and Aphasia

Research Center, Boston University. School of UMW sine and Boston V.A. Hospital, Boston, Mass. 02130

There are a number of virtues, and some difficulties as wel , in the views promoted by A & C in their interesting

review paper. We will dwell upon the latter_

It would be captious to criticize their goal of an account of both aphasia and normal language processing. an

account that might be called computational, as they employ the term; it would not be so to question their emphasis

on computer modeling or on implemented Al systems to achieve that goat, Our reasons are simple and twofold_ (1)

By and large the features of AI systems, which are actually related to currently investigable hypotheses about

language processes, are not those that stem directly from the successes or failures of implementation, that is, the

detailed procedures required for programming on a given computer. They are, where representational, largely

derivable from the distributional arguments of linguistic analysis, or, where process oriented, principally the fruit of

those intuitive or experimentally plausible decompositions of language function that (sometimes) yield decisions

about the architecture of the systems. (2) The neurological interpretation of even those features of computational

accounts that are appealed to in discussions of aphasia, for example, is quite unconstrained. The interpretive

conventions for neurological instantiations are even less compelling for the details of the procedures in implemented

systems.

We wilt speak briefly to each of these points with reference to remarks in A & C's paper.

1. The relevant features of computational models of language. A & C suggest that a computational model of

language function will be one that specifies the relevant vocabularies with sufficient precision to permit procedural

implementation (e.g., formal linguistic constructs will suffice), and that associates such representational claims with

general process models (systems of the ATN type, like those of Winograd, Woods, Kaplan. etc ) They further imply,

but wisely do not assert, that implementation is a condition of a model's being satistactorally computational. Indeed,

here and elsewhere much of the advertising for Al systems revolves around observations about their explicitness,

which arises from their having been programmed to some extent It's not that we think this is fa se advertising, nor

indeed that explicitness is not a great virtue. Rather, we want to caution against the possible conclusion that the

necessary or useful degree of explicitness arises only from the programming effort and especially against the likely

conclusion that the actual fruits of the programming effort are the most relevant features of computational models.

In fact, the actual details of computational procedures are rarely the basis of a test of language performance and are

seldom invoked in favor of the use of a given Al system as a model of human information processing. That this is so

does not mean it has necessarily been a good thing or that it will continue to be so. We think it is no accident,

however.

Illustrations of the reasonableness of our characterization may be taken from A & C's paper. In their initial

discussion of the Al approach, they recall Winograd's system as an example of an Al system that incorporates


procedures that "capture something of the way in which people can cut down the number of interpretations of a

sentence on the basis of pragmatic information_" The procedures so noted submit syntactically possible noun-phrase

groups to an interpreter in the special semantic domain for which Winograd's system was designed, and only those

noun-phrase groups that are possible claims about that domain are retained But note what is of interest, judging by

what our attention is called to, is not the detailed character of the procedures per se, but, rather, their place in the

organization of the analysis routine. Their (01E1 in the system reflects a feature of language processing that we take,

quite independently, to be a plausible and intuitively well-founded observation about human language use, that is,

we choose the usual before the bizarre. Whether the question of that

[466]

macroscopic truth's being properly represented as a feature of human parsing routines is not settled by its

implementation in a program, the attractiveness of the computational model as a representation of human mentation

is certainly enhanced precisely by the incorporation of some semantically based selection procedure for competing

sentence analyses.

The HEARSAY example that A & C deal with at length is of precisely the same sort. The detailed procedural

steps involved in some particular implementation of the model are simply not related, or are relatable in a motivated

way, to either normal or disturbed language function_ What one does relate to human language processes are the

Organizational characteristics - the architecture - common to the several implementations of HEARSAY: A & C are

clear about this. It is of interest that the system is modular in a disciplined way and that it takes hypotheses at one

"level" of description and submits them for test against other levels of analysis because we take it, again on

independent grounds, to be a plausible hypothesis about human language processing. The fact that there is a specific

schedule in terms of which the various subcomponents of the larger process interact with each other is not only not

discussed for the relevance of its detail, but its very plausibility is a feature of neurally instantiated (as opposed to

computer instantiated) models is questioned in a later section of the paper.

Finally, we note that the modular "construction" of language processors is precisely a hypothesis at this point - at

least in the sense that the linguisticalty defensible levels of representation for languages are regularly related to a

computationally defensible decomposition of the language faculty. It should be remembered that the experimental

resolution of this question will have a profound effect on our evaluation of the neural interpretations of the

HEARSAY type of system that A & C essay in the final sections of their paper.

Neural interpretations of computational models_ In their discussion of the "non-neural" features of HEARSAY

and ways to rationalize them neurally, A & C are at pains to connect their speculation with their opening analysis

and classification of early neurolinguistic models. The ease with which they agree to connectionist and /or holist

modes of interpretation illustrates our second reservation; we have no reasonable criteria by which to evaluate the

adequacy of connections drawn between features of HEARSAY computations, for example, and the -regional

analysis of neural organization" they essay. Accounts of the neurophysiology of structures in the brain that carry

information may be rich, as A & C claim, in relation to the operation of sensory and motor systems (although that,

too, is arguable), but they are certainly not well enough understood, in terms of the integrated functions of the

association areas of the brain, for us to discover the ways in which linguistic representations are neurally represented

and transmitted.

Er


Even at the more molar level of choice betweel the "holist" views of Jackson and connectionist views there is

frustratingly little basis for interpretive decisions. Jackson is certainly wrong in assuming that a non-task-specific

account of language function yields a theory of brain function that "does not consist of centers and connections."

There is no neurological imperative in Jackson's version of aphasic syndromes. A faculty psychology with

connectionist bent is not committed to a task-specific account of language performance in terms of his analysis. The

non-task-specific character of language function does not require a neurology different from centers and

connections; the activities of the centers may simply be redefined. The crux will be an independent defense of the

decomposition of the language function. That may arise either from the sort of distributional analysis typical of

linguistic enquiry or from experimental and observational study of language performance

Conclusion. It seems to us quite likely that decisions about which family of computational models to prefer and

which view of their neural instantiations to adopt will be more affected by the outcome of detailed experimental

observations of normal and disturbed language functions than will the experimental enquiries themselves be affected

by the details of procedures developed for computer implementations of computational models. That situation will

change, perhaps. One hopes so, for when we have so experimentally constrained our models that a single family, of

the HEARSAY sort, for example, is the preferred

theoretical type, we may be in a reasonable position lc exploit the procedural rigor of implemented systems for

the purpose of contrast in their sundry versions.

Harold Goodglass Is model building advancing neurolinguistics?

Boston Veterans Administration Medical Center and Department of Neurology, Boston University, Boston.

Mass. 02 130

To a clinical aphasiologist like the present commentator, descriptions of the capabilities of Artificial Intelligence

models suggest dazzling intellectual accomplishments, and their development should certainly be encouraged.

However, as one reads A & C's description of the power of the AIN augmented transition network) model on one

hanc and of the HEARSAY system on the other, one is struck by the apparent fact that these are only two of many

alternative ways of adapting man-made computers to simulate cognitive operations. While they represent advances

in computer technology, there is no evidence that they reach their ends in the same way as the brain. Even when one

feeds clinically derived rules to a computer to generate possibly deviant performances (cf. Lecours et al., 1973) the

result is sterile, insofar as shedding light on how the brain does these things is concerned [see Pytyshyn:

"Computational Models and Empirical Constraints" BBS 1(1) 1978)

The authors cite the progress made in tracing the coding of information at the visual and motor periphery and

imply that similar accomplishments are in the offing for language. However, the accomplishments in the visual and

motor realms have as their basis the actual documentation of the relatively straightforward neural connections in

these systems, in which there is a near one-to-one relation between stimu us and behavior. While, surely, the

interpretation of signaoding in a neural network calls on the same skills as does co Liter modeling, we are not told

that computer models led to the pre nt understanding of coding in the visual and motor systems.

In lad, there is nowhere near the invariance between signal and response in the language system that exists in the

peripheral visual and motor systems. Virtually all the notions that we have of the correspondence between lesion

locus and selective forms of language disturbance are based on the effects of ablative lesions due to strokes. None of

the studies of cortical brain stimulation (Penfield & Roberts, 1959 Ojemann & Whitaker. 1978) has succeeded in


duplicating these syndromes by electrical disruption of local brain activity. Moreover, even in the case of destructive

lesions, effects are often different from brain to brain and are influenced by age (Brown & Jaffe, 1975; Goocglass &

Geschwind, 1976) and handedness, as well as individual differences in brain organization. Even if there were greater

constancy in the relation between lesion site and language symptom, it is axiomatic that one cannot ascribe a

deficient function to the site of a lesion that produces a behavioral abnormality. A & C violate this axiom in their

flow chart model of the naming process, which is attributed to Luria 's analysis. Citing the statement that "lesions of

the left tertiary pane to-occipital zones yield verbal paraphasia," they assign this area to a box whose function is "to

discover the appropriate word and inhibit irrelevant alternatives" or "selective naming." This naive faculty invention

is faulty on two clinical grounds: first, that semantically and phonologically motivated paraphasias may be

associated with lesions elsewhere in the posterior language zone, particularly the superior temporal gyrus; second,

that the ability to discover the appropriate name may be severely damaged without paraphasia as a by product, as in

anemic aphasia.

I cannot argue with the ideal that ultimately we would tike to have a model that links normal and pathological

language with a computational view of the brain. However, we have a long way to go both in the analysis of the

psychological nature of disturbed language processes in aphasia and in the documentation of their anatomical

correlates. While process or computational models may be closer to the ultimate form needed for computer

simulation than what the authors term "faculty" models, their contribution to our present understanding Of

neuroiinguistics is not necessarily any greater.

ACKNOWLEDGEMENT

This work was supported in part by the Medical Research Service of the Veterans Administration and by USPHS

grant NS 06209 and 07615.

[467]

Samuel H. Greenblatt Is neurolinguistics ready for reductionism?

Section of Neurological Surgery. tepartmenl of Pleuroaciences, Medical College of Ohio, Toledo. Ohio 43899

The problem of reductionism has plagued the relationship between biology and physics since the 17th century.

Due to the rapidly expanding success of mathematical biology (including neurophysiology). the issue seems to have

been buried in recent decades. It still lies fallow, however, ready to sprout whenever the winds of change uncover it.

A & C have examined the perennial issue again, but having done so, they propose to bury it once more, because they

conclude that neurolinguistics can reach full bloom only if it becomes computational; that is, it must be able to be

conducted on a computer.

At the practical level, I can only agree with their reading of the omens Everything seems to point in the direction

of that computer. Whether or not "the mind" can ever be reduced to a mathematical expression is irrelevant. Our job,

really, is just to keep working and see where we come out. If this view is accepted, we are left with more mundane,

but nonetheless important, problems. Not the least among these is how to know when any particular biological

science is ready to be reduced to computational terms. On this point, A & C seem to be saying that neurolinguistics

ought to be ready. They base this conclusion on the tact that there are now computer programs that appear to be able

to perform the analysis. My own conclusions are somewhat less optimistic because I think that there are still some

glaring deficiencies in our information about the neurophysiology of behavior at the level of gross connectionism.


Among the most intriguing of these deficiencies is the extremely crude state of our knowledge about the "rules"

of connectionism It is sometimes said of the cerebral hemispheres that everything is connected to everything. But if

this is realty true, we are defeated at the outset. Therefore, there must be some "rules." or consistent patterns, that

describe how partially analyzed packages of behaviorally related data are transferred from one area of the brain to

another. One of the few available guides in this area is the incomplete set of principles sometimes known as

Flechsig's rules (Flechsig, 1901). which was discussed by Geschwind (1965) but not mentioned by A & C.

Flechsig based his principles of interhemispheric connections on his studies of human myelcgenesis. He

concluded that ipsilateral primary sensory and motor areas are poorly interconnected among themselves, and that

homologous primary areas are poorly connected between the hemispheres. but that there are strong transcallosal

connections of homologous association areas. However. Fleshsig did not say clearly whether there are any

differences between the connections of "intermediary zones" (primary association areas; e.g., Wernicke's area) and

"terminal zones" (secondary association areas, e.g. interior parietal lobule). This problem is extremely important for

modern connectionist theory. More recent work indicates that there probably are significant differences between the

transcallosal connections of primary and secondary association areas (Pandya & Vignolo, 1969).

Worrying about the rules of connectionism is not really as anachronistic as it might seem to A & C because the

same neurophysiological principles would apply to "process" and to "representational" models as well as to

connectionist models If I have understood A & C correctly, they are claiming that the application of artificial

intelligence methods to neurolinguistics may help us discover some of these rules by providing testable hypotheses.

Anyone interested in neurobehavioral studies can only second this motion and hope that it carries. But it is a long

way from single cell studies to integrative action in the cerebral hemispheres, and animal experiments have some

very severe limitations in the study of human cognitive behavior. Our colleagues in computer technology may find

that they have created sorr.e elegant hypotheses whose validation is frustratingly delayed by the inadequacies of

physiological technology.

Terry Halwes An embarrassment of riches in nascent neurolinguistics.

Haskins Laboratories, New Haven, Cora 06510

A & C have offered as a menu of ideas fcr advancing neurolinguistics These

are very good ideas indeed, to judge from the few with which I am familiar. For example, the work of Bernstein

and his students on the coordination of movements in skilled action deserves much wider attention than it has

received in this country, and it is encouraging to see the authors recommending that those ins:ghts be applied to the

study of language. Some of the other suggestions that are less famOar sound like interesting reading. Here, however,

I wish to focus on the suggestion that "neurolinguistics must be computational."

That suggestion has two sides Computers are used in neuropsychological theory both as the basis of a metaphor -

"the mind/ brain is like a computer" - and as embodiments of various theories - the theory in question is

implemented as a computer program and subjected to simulated interactions with an environment

Anything can form the basis of a scientific metaphor, and the brain-computer metaphor is a familiar one. How

useful it may be is still an open question, but A & C seem to be extending the metaphor somewhat, discussing not

computers in general, but some particular strategies such as HEARSAY, of using computers. These are certainly

interesting and plausible bases for neuropsychological metaphors, and although we have very little experience with

them, the situation will presumably improve


Here I wish to address the second side of the suggestion - the intention of embodying neurolinguistic theories in

computer programs. The authors do intend "computational" to be taken in that sense. Consider, for example, the

passage beginning "An immediate research project for computational neurolinguistics, then, might be to approach

the programming of a truly distributed speech-understanding system . ." in the section entitled: Computational

Neurolinguistics and Neuroscience.

1 agree wholeheartedly with the authors' goal. Implementing neuropsychological theories as computer programs

may be the only way to keep track of all the many facets of the topic. But that goal will not be easily attained. My

purpose here is to point out certain dangers on the route.

(1) When theories are embodied as computer programs, the theorist gains certain definite advantages, but at

considerable expense. Since computer orograms are always completely explicit, embodying a theory in a computer

program requ res the theorist to be explicit For a mature theory that may be an appropriate step. But most theories in

neuropsychology are not that mature. There lies the first danger.

"Freezing" an immature theory into an explicit form too early can seriously retard its development. In the process

of implementation, the theorist is forced to make arbitrary decisions about the theory - decisions that have empirical

implications but for which there is little or no relevant evidence - rather than concentrate on developing the more

central points. Scores of examples of this phenomenon may be found in the literature of "mathematical" psychology,

as well as in the literature on computer models in psychology To embody one good idea in a computer program may

require a hundred arbitrary choices, all of which become part of the "theory."

No area in the general domain of neurolinguistics, not even the most highly developec, seems ready for

thoroughgoing formal treatment. Even in linguistics, which is viewed by many as a paragon of theoretical

sophistication, the available theories are less explicit then they seem. No full analysis, mapping semantc

representations onto phonetic representations, has ever been carried out for any language. Rather, the literature of

theoretical linguistics offers suggestions as to how such analysis, or how some aspect of such analysis, might or

should be (or "must" be) carried out

Major obstacles to the formalist program in linguistics remain to be dealt with, espec ally in the areas of

semantics and phonetics. These are not simply details that need to be filled out, but major issues that have never

been resolved. As one example, consider Chomsky and Halle's (1968) claim that the phonetic distinctive features are

in one-to-one correspondence with simple perceptual, acoustic, and articulatory structures. Some experimental

phoneticists are pursuing that suggestion, others are quite convinced that no such correspondences exist.

We could, of course, find individual linguists or small groups of them who would willingly offer sets of

rep•esentations and lists of rules for

[468]

relating them. But across the field as a whole we find little commonality of assumptions, of "facts," or of cannons

of evidence, let alone of theoretical principles. 1 do not intend to imply that linguists are not progressing. My point

is that linguistics, which may indeed be the most highly developed of all the parent areas of neurotinguistics, is in no

position to provide an easy answer to a question like ''What kinds of representations and rule structures should we

include in our neurolinguistic theory?" For it is not primarily the facts that are at issue in linguistics, but exactly the

questions of what sorts of categories should be represented in a linguistic theory, and how.


Should we take a survey of representatives of various linguistic camps, build computational neurolinguistic

theories based on the suggestions of each faction, and then empirically evaluate the performance of the various

alternatives? How should the evaluation proceed? Linguists have no generally agreed-upon methods of testing

empirical claims of their theories. This is a lack that is evident also in the outlying areas like psycholinguistics.

Ganong, for example, has recently argued that none of the experiments that are believed to demonstrate distinctive-

feature representations in perception should be so interpreted. It is not that Ganong is necessarily right in his

arguments, but it is simply that the question of how to handle evidence in this relatively well-developed area is still

very much at issue.

Therefore, those who enter into the proposed project will have to develop their own criteria for designing and

testing most of the components of the theory. Perhaps these components could be written so as to be easily modified

as improved understanding develops

None of the problem areas surveyed above is likely to be improved by implementation of one or more of the

alternative theories as a computer program. A program written along the lines of the A & C proposal should not be

expected to be of much value except as an exploration of possible strategies for integrating information from diverse

sources of knowledge. That would be a valuable contribution, to be sure, but one with definite limits.

A program can be written with "dummy" entries for any components that are not well understood. A more or less

crude approximation of what a given component might look like may be installed as a temporary expedient, so that

the system configuration as a whole can be evaluated. So. for example, in the evaluation trials of the HEARSAY II

program at the end of the five-year ARPA Speech Understanding Research project, none of the component

knowledge sources were realistic simulations of their natural human counterparts. With the possible exception of the

KS that constrairs word candidates to English syllable structure, the modules installed in HEARSAY II were not

even expected to simulate natural input-output relations_ The syntax. for example, was an artificially constructed

finite-state grammar, test utterances were constrained to conform to that syntax (Lea and Shoup 1979). We can

likewise expect quite a lot of that type of fill-in in the early stages of the project proposed by A & C.

The second danger, then. arises from the fact that we have no real understanding of the potential degree of

generality of any conclusions we may draw from studies of the integration of diverse sources of knowledge, when

the knowledge sources used in the simulation differ in unknown ways from their natural counterparts. It is hoped

that some study of this question will be included in the early stages of the proposed project.

Even when the theory to be implemented as a computer program is well developed, the theorist must face the

problem of distinguishing the content of the theory from the details of the implementation. That distinction must be

made if the program/theory is to be tested the way scientific theories are tested.

A computer program as an implementation of a scientific theory of human language skill must meet much more

stringent criteria than those used to pursue the goal of Al_ Al programs have merely to get the job done - in this case

the job of hearing, reading, and understanding language, plus uttering and writing it appropriately. How an Al

program does its job may be evaluated in terms of criteria of speed, cost, efficiency, and reliability. However, a

program that implements a Scientific theory is intended not only to get the job done, but to do it just the way human

beings do it. It might be expected to make the same errors that human beings do, for example, and no others lot

Pylyshyn: "Computational Models and Empirical Constraints" BBS 1(1) 1978).


Those more stringent criteria for scientific theories place demands of a special sort on the theorist and the

programmer. In order to be able to test experimentally the empirical claims a theory makes regarding how human

beings execute various skills, the experimenter must be able to tell what those claims are. That is, he must be able to

distinguish which aspects of the program and its operation should be measured and compared to measures of human

functioning_ Therefore, the theorist and the programmer must take pains to make it obvious just which aspects of

the program represent scientific content of the theory, and which are to be taken as the background of the

implementation, necessary for the program to run on some particular computer.

That problem is minimal when the theory to be implemented is mathematically explicit, since the means by

which the computer arrives at the values specified by the mathematical equations are obviously not intended to be

taken as part of the empirical content of the theory. and are not likely to be confused with that content. An excellent

example of that approach in neuropsychology is reported by Pellionisz and Llinas (1979). They draw a conclusion

that might be taken as supoorting the position taken by A & C: "This combined approach of mathematical treatment

(which provides an abstract language) and computer simulation (which, by accommodating data into the model, exp

ores the significance of deviations from linear character) appears at present to be the most adequate technique for

dealing with central nervous system function" (from the Abstract, page 323).

But the requirement of explicit mathematical treatment in the approach used by Pellionisz and Ulnas seems to be

missing ffrpm the computational approach as suggested by A & C.

A & C use language (and flow charts) reminiscent of discussion of computer programming strategies rather than

the language of fun tonal analysis. That difference may be appropriate to the difference in topics Pellionisz and Una .

s are modeling a domain well suited to description by continuous functions of a few variables, while A & C are

concerned with modeling neurolinguistic processes that may involve categorical choices (among phonetic,

structural, semantic, and pragmat c variables) that possibly should be modeled by some sort of formal logic. But

given the appropriateness of the computational metaphor to the theoretical content of the domain, the use of

computational concepts as content in a theory implemented in a computer program raises the everpresent possibility

of confusion to new heights.

It thus seems especially appropriate that a computational neurolinguistics should include, in its early phase. an

effort to develop conventions of implementation aimed at establishing and preserving a clear separation of the

empirical content of the theory from the details of implementation.

Patrick T.W. Hudson What is computational neurolinguistics anyway?

Department of Exporimonml Psychology, Celiffereffy of Nijmegen Ememusleart re Nijmegen, The

Netherlands

A & C distinguish a number of different types of models for neurolinguistics and eventually propose that

neurolinguistics must be computational. I do not yet find that they have fully presented or won that case. Nor do I

wish to corrmit myself to any more than that there must be a computational neurolinguistics, alongside other

interesting forms of neurolinguistics. While remaining sceptical that the approaches proposed by A & C are always

what I mean by computational neurolinguistics, I fully support the dea that computational neurolinguistics should be

the most interesting and profitable concern in aphasiology. It seems trivially obvious that computational models, by

which one means models sufficiently articulated to be implemented, are the only models capable of spanning the

independent levels of linguistic and neurological description.


We may distinguish two extremes on a continuum of neurolinguistic problems and explanations: at the

neurological end, linguistic components play a minor role. However, even the most extreme neurological

[469]

descriptions have made some reference to a (psycho-) linguistic differentiation of functions, while concentrating

upon the localization of symptoms- If the same syndrome were to appear as a result of damage to radically different

parts of the brain this would be of interest_ This approach requires some linguistic analysis because the definition of

two symptom-complexes as the "same" depends upon how fine-grained is the analysis of the deficit.

The purely linguistic approach, in contrast, may be distinguished in that, given a linguistically interesting

syndrome, other neurological facts are of no relevance and cannot be entered into such models at all. It is of little

importance where in the brain the symptoms lend to occur. In the limit it would not make any difference .if both

Wernicke's and Broca's aphasia were found in the same region of the brain, because such facts are impossible to

work into a linguistic model. In extreme cases, the aphasia data are to be treated only as evidence for or against

particular linguistic theories. A classic example is Jakobson's monograph (1941), relating language acquisition and

dissolution at the phonological level. A modern example is Kean's (1978, 1979) detailed linguistic analysis of

agrammatism, in which the notion of mechanism lies very close to the components of linguistic theory

Computational neurolinguistics falls naturally between these two extremes and currently offers the best serious

chance of saying why a particular form or locus of damage should give rise to particular deficits. It is the

explanatory level that describes mechanisms and their operation so as to predict what will happen when a

mechanism is damaged or its mode of operation is distorted. Specifically it is the level at which one can first detail,

beyond mere handwaving, what one means by terms such as brain damage, dissolution, or deviation.

I define computational neurolinguistics in two stages Fest, as the formulation of models for linguistic processes

that accurately reflect the linguistic and psycholinguistic evidence of normal performance and that are couched in a

framework that defines the mechanisms whereby those processes are carried out. (This is a definition of an adequate

psycholinguistic process model, which subsumes competence models.) A neurolinguistic model must, however, be

constrained still further and is, therefore, more interesting. A fully adequate model of this class would enable one to

state clearly both (i) the normal operation of the system and (ii) the specific ways in which the processes can break

down or otherwise deviate from normal functioning. Because I have postulated a specific set of disorders ie the

proposed devices, this model should characterize those and only those syndromes that occur (or. most interestingly,

predict new syndromes). At a level of description adequate to cope with the full range of the phenomena, it seems

obvious that the model must define a workable system and be, in principle, implementable. Such models must be

almost trivially computational in the sense that they define a working system. This is not to say that the model must

be implemented, but only that it should stand open to that test. My own model (1977) has only begun, it is hoped, to

approach a richness of description in which such tests might eventually be possible.

I have specifically postulated a set of mechanisms and operations and then explored the effect of specific

disruptions on the behavior of the system as a whole. The model was first constructed for normal language

performance and measured against a range of psycholinguistic. phenomena for both production and perception of

speech. The elements, boxes, and processes postulated for this model were justified in terms of an analysis of

psycholinguistic functioning that specifically excluded any reference to abnormal behavior. An attempt was made to

specify elements in sufficient detail to elucidate the complex set of constraints based on the internal consistency of


elements and processes. (Many noncomputational process models do not and need not face all the problems of the

interrelations of representations, in lexical access and the subsequent use of lexical items in production and

perception, a computational model must.) This having been done, it became clear that a restricted number of

malfunctions, such as alterations to the thresholds of butters for accessing memory or various disruptions to structure

building mechanisms, could be entertained and their effects investigated. The small number of possible disorders

mapped quite successfully on Luria's (1973) characterizatioe of aphasic disturbances and onto the more ciassical

taxonomy.

Much of my model is certainty ad hoc as A & C point out, but the range of performance required introduces

severe restrictions on possible models The effect of exploring possible alterations to the system, in terms of

predicting their potential effects, led to a model close to the classical brain map of the connectionists in which the

entities that are labelled map onto devices found in the model derived from nonneurological data and are not a mere

retelling of symptoms. When we have computational models sufficiently well articulated we should be able to state

that certain activities are the function of specific tissue masses, that certain information must be transmitted along

specific lines, and that certain general computational operations must be possible. We are also able to state that a

number of boxes and systems proposed in the model do not have a physical representation as localized systems in

the brain diagram. We can state that certain functions, possibly incistinguishable at tne level of process models, must

be seen to be the result of more general properties of the brain and cannot, as such, be physically localized in any but

the grossest of terms. This begins to to what I call computational neurolinguistics.

We can now examine the two specific models proposed by A & C. The "Luria" model, admitted to be

noncomputational, also fails, as did Luria (1973) himself. Despite his own strictures about localization of symptom

versus function, this remains a detailed analysis of the symptoms For example, detects in selective naming (B) are

seen as the result of damage to the selective naming function; this is not a model that meets regLirement i above or a

model for normal functioning without reference to the neurological data. Secondly, the boxes are so loosely

described that the specific form and effects of damage cannot be adequately specified; they thus fail requirement ii.

Luria's analysis and the reworking here are clearly forms of neurolinguistics, but are noncomputational in the sense

of not being fully articulated. If one accepts that a greater degree of spec; licity would have been better, then one is

hallway to accepting the thesis that neurolinguistics ought to be computational.

The description of cne Al model, HEARSAY, begins to meet requirement i. It appears tc have been chosen for

exposition, however, because it exemplifies cooperative computation. The HEARSAY approach is interesting and

operationally successful (Klatt. 1977). but is it a model that adequately meets requirement i? Certainty not as regards

production. Unlike A & C, I find it unconvincing as a full model for psycholinguistic functions. Furthermore, their

treatment fails my own test of neurolingustic adequacy in that the "neuralized" version is not tested by comparing

the hypothesised effects of specific disruptions with the clinical data. But this is where things become interesting. 11

we have a model that is (a) linguistically appropriate, (b) details mechanisms for the linguistic processes, and (c)

goes on to define ways in which disruptions occur such that the distorted performance maps onto the clinical data,

we have something If we have a set of mechanisms that are at least descriptively adequate for both normal and

abnormal functioning, we have the start of a concrete specification to be related (subject to constraints on

reductionism) to the low-level neuroscience constructs A & C wish to introduce to neurolinguistics.


by Mary-Louise Kean and George E. Smith Issues in core linguistic processing.

School of Social Sciences. University of .C.'elifornia. Irvine, Calif. 82717 and Department of Philosophy,

Tulle University, Medford. Mass. 02155

The proposal of A&C for developing a computational process model in neurolinguistics is a natural evolution of

the ideas that have traditionally motivated the field. Even in research on specific problems, for example,

agrammatism in Brace's aphasia, a comparatively detailed model of the on-line linguistic processor would offer

obvious advantages. Furthermore, as the authors suggest, such a model would help bring to bear on one another

results from such diverse areas of research as neurophysiology, functional neuroanatomy, psycholingiestics, and

theoretical linguistics. We agree with the authors about the research advantages of the

[470]

can serve such research ends without being right or even a close approximation thereto. (In fact, we believe that

any model proposed at this time will surely bear title relation to the truth.) So the issue is not whether the specific

kind of model A&C propose is right or wrong. Rather, the issue is whether their proposed model will lead to

significant progress toward the indicated ends. We do not think it will.

First, we want to mention some technical difficulties we have with the HEARSAY II approach. It is unclear to us

exactly what sorts of hypotheses the proposed processor generates and tests. Nonetheless, we have doubts about the

role of hypothesis testing in linguistic processors. Our doubts center on the proposal for an iterative procedure that

continues until some thresoold is exceeded. We suspect that much linguistic processing involves a more direct

computational procedure, more like number crunching than like problem solving. If this is correct, a hypothesis

testing model, with its focus on problem solving, could be systematically misleading.

The distinctive thing about a HEARSAY II type model is that it provides an elegant realization of "cooperative

computation." We take it, from the emphasis that the authors place on it, that they view the problem of

implementing cooperative computation as the crucial one in constructing a linguistic processor. We can see why

someone might think this. Widely diverse information is most surely brought to bear in the normal use of everyday

language. It is natural to think, then, that the crucial computational problem is to avoid having the linguistic

processor swamped by this information. Even granting that diverse information is used in linguistic processing (a

thesis we reject), we nevertheless think that there is a more Critical and richer problem to be addressed in

constructing a linguistic processor, the problem of linguistic representations. Given an explicit theory of the various

levels of linguistic representation, many of the details of the computational procedure will be virtually transparent.

We believe that work on the processor should therefore concentrate on the problem of representation, and not the

problem of cooperative computation.

Our misgivings about the role of cooperative computation run deeper than this. We see the core function of the

linguistic processor to be the generation of phonological, morphological, syntactic, and logical representations, with

the diverse nonlinguistic information exploited in everyday language use being inaccessible to this core. HEARSAY

II, as a "modeling methodology," is not incompatible with this view. But the emphasis on cooperative computation

suggests that even in the realization of the representations of the core the main computational problem is seen as

involving the marshalling of diverse information. In fact, the discussion of Figure 9 indicates that the authors take

speech act considerations to be relevant to the normal determination of the phonological representation of a


sentence. It is not that we deny a role to speech act considerations in language comprehension, but we do reject the

idea that such considerations infuse the core linguistic processor.

HEARSAY II represents a plausible approach to a wide variety of cognitive processes such as problem solving_

Indeed, A&C point out that the model they are proposing is one of a number of implementations of HEARSAY 11

(the C2 configuration). Adopting HEARSAY II as a basis for a linguistic processor suggests that human linguistic

capacity is simply an instance of general human intelligence. Our concern above was that the HEARSAY Il

approach was, by implication, rejecting the hypothesis of the autonomy of linguistic capacity. Our concern here is

that it is rejecting the hypothesis of the modularity of cognitive systems (of which we take human linguistic capacity

to be just one). To reject these two hypotheses is to reject the cornerstone of the most fruitful research in the area of

human cognition. In this respect HEARSAY II does not, we fear, represent a productive evolutionary development_

There are strong methodological reasons for adopting the autonomy and modularity hypotheses, over and above

any empirical considerations supporting them. One role these hypotheses play in research is to limit the degrees of

freedom open to accommodating new data and discoveries. In particular, the hypotheses restrict the options

available for revising theories in the face of confuting evidence. We take it that it is A&C's intent that the

HEARSAY II model play just such a roe in neurolinguistic research. The trouble is that the HEARSAY II model

will tend to do just the opposite. It will consistently provide too many degrees of freedom where they are least

desired. HEARSAY II, for example, is as compatible with a theory of human cognitive capacity based on the

hypothesis of modularity as it is with one that is not. ft is also as compatible with an explicitly nonpsychological

theory of linguistic structure concerned with possible languages, such as Montague grammar, as it is with theories of

linguistic structure that are explicitly addressed to capturing the structure of humanly possible languages. A&C

remark that "the development of models along the lines of these (connectionistj prototypes led to a 'chaotic'

theoretical development characterized by intricate diagrams, replete with centers and connecting tracts, whose

components and connections proliferated with the discovery of new constellations of symptoms." Because the

HEARSAY II approach provides so many degrees of freedom, we tear just such an eventuality for research based on

it

A HEARSAY II model thus appears inappropriate to us as a research tool. The question is then raised as to what

kind of computational process model in neurolinguistics is appropriate. As we have suggested. the appropriate

model is one of the core linguistic processor, devoted to processing phonological, morphological, syntactic, and

logical representations. Thai is. it should be an autonomous cognitive module with no access to information of a

nonlinguistic nature. This approach makes the modeling problem more constrained (and hence in some ways more

difficult) since the class of processors is significantly reduced when access to information is restricted. New data and

discoveries will have to be accomodated primarily via revisions in the System of linguistic representations. At the

sme time, however, the modeling problem would be more tractable becat ge of its narrower, more specified scope.

There is no reason to belie that such a model is any less likely than the HEARSAY II approach I be fruitful in the

study of neural mechanisms and structures

ACKNOWLEDGMENTS

Preparation of this paper was supported in part by NIMH grant 1-.F32- MHC7189-01. This paper was prepared

while Kean was on leave at the Department of Psychology, Massachusetts Institute of Technology.

1


Terence Langendoen Linguistics must be computational too

Linguistics Program. CONY Graduate Canter, 33 West 42 Street, New York, N. Y. 70038 and Department or

English, Brooklyn College, Brooklyn, N.Y. 11210

. A generative grammar of a language is a device that computes the sentences of a language. However, the

computation is only partial: sentences are computed from words in the syntactic component (Chomsky, 1965). and

words are computed from morphemes in the morphological component (Arorott, 1976; Siegel, 1979). but

morphemes themselves are not computed. Rather, they are simply listed in a dictionary.

A morpheme is a rather complicated object, made up of an undeitying phonological representation, a semantic

representation. and a "morpholexical" representation that expresses its distribution in words. Thus the question

arises as to why morphemes, too, are not computed like words and sentences. The answer is that there is only a finite

number of morphemes, but from this it only follows that morphemes can be listed, not that they should be. Suppose

that the morphemes of a language are computed by a component that we dub the morphemic component. What

would such a component be like?

Ever since Saussure ( 1915) formulated the doctrine of the arbitrariness et the linguistic sign, it has been assumed

that the pairing of phonological and semantic structures in morphemes is arbitrary. This can to expressed by the

claim that the rules of the morphemic component are of the form (1), where A is a morphemic category, lis a

phonological structure, and s is a semantic structure (we ignore morpholexical structures here and throughout this

discussion).

But as Saussure also pointed out, the arbitrariness of the linguistic sign is not absolute. Even if we put sound

symbolism aside, we find many cases in which whole sets of morphemes are semantically and

[471]

phonologically related. For example, English has many pairs of morphemes, such as break (verb) and break

(noun), that are phonologically identical, and whose semantic difference is predictable by rule. It is often claimed

that the verb stem of such pairs is basic and that the noun stem is derived by a phonologically zero affix, but while

we may consider the verb stem to be basic, there is no evidence that the noun stem' is morphologically complex_

Thus, rather than describing the relation between such pairs in the morphological component, we do so in the

morphemic component of English as follows. Let E (for "empty"), N (for "noun stem"), and V (for "verb stem") be

morphemic categories; let N and V be axioms of the morphemic component; Let brak be the phonological structure

of break; and let s, and s„' be the semantic structure of Creak (verb) and break (noun) respectively. The rules of the

morphemic component include those given in (2).

Now consider the relation between the morphemes break (verb) and broke, past tense of break (verb). The latter

is an inflected form of the former and is therefore derived from it; nevertheless, broke is not morphemically complex

(many linguists have wasted a lot of time trying to figure out ways to analyze forms like broke into two

morphemes). Accordingly, the relation between these forms is also to be described in the morphemic component To


do so, let V (for "inflected verb stem") and P (for "past tense") be morphemic categories; let brak be the

phonological structure of broke; and let sb,c,,,„, be the semantic structure of broke, past tense of break (verb). The

rules of the morphemic component of English also includes those in (3), in which. (3b) replaces (2b).

The morphemic analyses just given obviously only begin to provide a glimpse of the kind of work that remains to

be done before we can say that a complete account of how morphemes are computed in English has been given.

Nevertheless, this glimpse is sufficient to show that even when such en account is provided, a generative grammar is

still not completely computational We still have to describe how phonological and semantic structures are

computed.

Only a finite number of phonological and semantic structures enter into the construction of morphemes. By

means of the recursive devices in the morphological anti syntactic components, however, an infinite number of

phonological and semantic structures enter into the construction of words, phrases, and sentences. The set of

phonological structures that enter into the construction of the morphemes, words, phrases, and sentences of a

language nevertheless does not exhaust the set of phonological structures of a language; very many such structures

for example. English brag. still remain unpaired with semantic structures at any level. Such "meaningless"

phonological structures of a language may be called " accidental gaps" (Halle, 1962); these contrast with

phonological structures called "systematic gaps," which are not part of the language at all (for example, bnak is not

part of English). To compute the phonological structures of a language, we propose a phonological component,

whose terminal vocabulary consists of the phonemes of that language, together with certain other elements. The

rules of this component compute the Phonological structures from phonemes much in the way that the syntactic

component computes the sentences from words. This component is, in other words, a genuine "phonological

grammar" (Householder, 1959) that is quite unlike the phonological component of a standard generative grammar

(Chomsky & Halle. 1968)_ It computes Phonological structures completely without regard for their roles in the

meaningful constructs of the language. Rather, the phonological structures of those constructs are constrained to be

members of the set computed by the phonological component.

Likewise, to compute the semantic structures of a language, we propose a semantic component whose terminal

vocabulary consists of the semantic primitives ("sememes") of the language. together with certain other elements.

The rules of the semantic component compute the semantic structures of the meaningful constructs of the language

from sememes much in the way that the phonological component computes the phonological structures from

phonemes. Again, there is nothing comparable to a semantic component of this sort in a standard generative

grammar; the "semantic component" of such a grammar (Katz & Postal, 1964; Katz, 1972) must be distributed

among the morphemic, morphological, and syntactic components proposed here, since its function is exclusively to

show how semantic structures combine in the formation of derived readings for morphemes, words, phrases, and

sentences. It says nothing about how such structures are computed in the first place.

The question -arises whether there are any accidental semantic gaps in a language, parallel to the accidental

phonological gaps discussed above_ Since we have no direct intuitions about them, it is certainly possible that they

do not exist (Katz. 1979). On the other hand, perhaps they do exist, but we simply do not have direct access to them.


Many linguists have been claiming of late that generative grammars should be "psychologically real" and

modifications in the theory of grammar have been proposed in an effort to achieve this (Bresnan, 1978). However

we do not have any clear idea of what psychological reality for grammars really is. This is why I think A & C's

paper is important. It focuses on the need for psycholinguistic models to be both neurologically based and fully

computational. Quite independently, linguistic models also need to be fully computational. ft remains to be seen

whether the optimal psycholinguistic computational model resembles the optimal ingustic computational model in

any way.

Simeon Locke Localization, representation, and re-representation in neurolinguistics.

Department of Neurology, Harvard University Medical School and Beth Israel Hospital, Boston, Masa_ 02215

Of the many valuable implications that emerge from the paper by A & C two in particular provoke comment.

Both have been addressed before, but restatement seems warranted.

The first of these has to do with the historical tradition in which we have been raised, and that continues to

contaminate our thinking despite the usual disclaimers. Recognizing that clinical data are based on the effects of

lesions, the authors emphasize that the connectionis1 uses this information and ". . specifies the location in the brain

of these task-defined components of language function . . " (italics added). The logical elision is never

acknowledged. Disruption of a function, particularly if we accept the notion of distributed function, can never

localize either the function or even the "representation" of the function_

This issue of representation makes the transition to the second point. Customarily (and understandably) we do

not define representation - the "how" of it or what the authors refer to as "synapsecell-circuit neuroscience." Rather,

we address the question of the "where" of it as if it were an entity. Jackson, it seems to me, was careful to avoid this

trap by insisting on the distinction among representation, rerepresentation, and re-rerepresentation. This hierarchical

formulation is evident in HEARSAY II, which operates on levels from parameter to phrase. This model (and models

are metaphor) treats neurolinguistics in the most literal sense, using language structure as the determinant of

computational representation. For me (and perhaps for the authors as well) the more interesting, though less

vulnerable, question is how "nonlinguistic" neurological functions - the cellular, the distributed, the systematized,

and the behavioral - are translated in hierarchical form to culminate in language. In computational models the

rerepresentation and translation from magnetic core to binary system to computer language to English is better

understood with the result that the "representation" from level to level is less intriguing perhaps than the end stage

program itself. In the nervous system it is the

[472]

"representation" and "translation" at each level that remain to be learned. Although the right moment for this has

not yet come, it is my hunch that a literal linguistic approach, computational or otherwise, will not be the access

route.

John C. Marshall The sense of computation

Interfekofteire Werkgroep Taal- an Spraakgedrag. University of Nijmegen. The Nailwriand&

When A & C write that neurolinguistics must be computational, there is certainly one sense of that overworked

metaphor for which their claim must be true: We would surety all agree that effectively computable theories are to

be prefered to vague speculation. But there are a number of issues that must be broached before one can even

contemplate the construction of such theories. There are, for example, questions concerning the domain of


neurolinguistic theory and the principles that might guide us to truly explanatory idealizations. More specifically,

there is the problem of how to relate the different levels (linguistic, psychological, and neurological) on which

descriptions of language representation in the brain can be formulated.

For obvious reasons, neurolinguistics developed in the context of clinical medicine. This origin accounts for the

fact that the discipline has always been dominated by two concepts the idea of symptom-complexes, and the notion

of unique leSiOn sites that are responsible for such syndromes. The only surprise in all of this is how well many of

the traditional nineteenth-century symptom-complexes (Broca's aphasia, Wernicke's aphasia, Anomic aphasia, for

example) have stood the test of time - as gross descriptions. Given this background in which the goal of the

enterprise is to predict lesion-sites from symptom-complexes (and vice versa), the notation devised by Baginsky,

Wernicke, Broadbent, and Lichtheim functioned as an effective device for teaching a typology of the aphasias that

indeed achieved a fair measure of observational adequacy. Bui there has always been a systematic ambiguity in our

interpretation of box-and-arrow notation.

The ambiguity concerns the way in which we label the components of conventional diagrams. If a tesioned

Wernicke's area were simply labelled "Wernicke's aphasia," or a lesion of the arcuate fasciculus were labelled

"Conduction aphasia," there would be no (theoretical) problem. The sole issue would be the truth of the claim that,

typically, a lesion of area X results in behavioral syndrome X'. The price, however, is that there is now no

explanation at all of why the observed correlation should obtain. Wernicke accordingly attempted to deepen his

analysis and edge toward explanatory adequacy by relabelling the primitives of the notation. Boxes (actually circles)

became "functional centers" and arrows become "transmission lines" that are construed anatomically as cell-

assemblies and fiber tracts. Thus the first temporal gyrus becomes the repository of "auditory images," the third

frontal convolution the storehouse of "articulatory images," and the arcuate fasciculus a telephone line that links

them. This move, combined with rigorous adoption of a simplicity criterion (minimize the number of boxes and

arrows) was obviously a step in the right direction, for it enabled previously unnoticed syndromes to be predicted

and then confirmed by observation. So far so good, but -computationally" the system just did not work. There is no

way in which the Lull symptom-complex of Wernicke's or Brace's aphasia can be understood solely in terms of the

selective unavailability of auditory and articulatory images (Geschwind, 1969).

It is unfortunate, then, that A & C should have chosen to pursue a strategy so closely related to Wernicke's Their

"diagramization" of Luria (Figure 8) once more relabels the old notation, leading one to ask for the evidence that

Lexical Analysis (Box H), say, is a theoretically meaningful component at any level of description; worse, the

notation fails to distinguish such presumably disparate "functions" as Phonemic Analysis (Box E) and Switching

Control (Box C). A & C concede, of course, that their block diagram "tails to be computational," but matters do not

improve much when they turn to a model (HEARSAY II) that certainly is computational by anyone's standards. A

new difficulty now arises: It is totally unclear what specific hypotheses are supposed to be embedded within the

blooming, buzzing contusion of the implementation. The system is "interactive," but, in the absence of empirical

constraints on the number and nature of allowable interactions this boils down to letting the thing run subject only to

an arbitrary brake on the exponential growth of the analysis scribbled on the blackboard. It follows from the failure

to distinguish theory and implementation that HEARSAY is unlikely to suggest a new account of, for example,

jargon aphasia or transcortical motor aphasia.


But perhaps we are equating neurolinguistics too directly with aphasiclogy. Let us, rather, ask what a general

theory of neurolinguistics might #ock like. As A & C imply, at least three levels of explanation would be involved.

First, a theory of linguistic representations; second, a theory of the psychological processes that compute tokens of

the types specified by linguistic theory; third, an account of the neuroanalomical and neurophysiological

mechanisms that instantiate the relevant psychological functions. The central problem of neurolinguistics has been

to make results obtained at one of these levels bear upon those obtained at other levels. And indeed some scholars

have argued that such levels of description are "nearly independent" (Marr & Poggio, 1976). Although there are

obvious curbs from top-to-bottom (a psychological theory that is, in principle, incapable of computing the structural

descriptions generated by an adequate grammar can be ruled out of consideration), it has proved particularly difficult

to discove- constraints that neurophysiological findings impose on higher levels.

A & themselves acknowledge that "synapse-cell-circuit neuroscience ' has had little influence on the

development of neurolinguistics. One could construct a priori an argument why this shoudl be so: The brain is

densely interconnected (Szentagothai, 1978); even "small" naturally occuring lesions will destroy vast numbers of

cells and perturb neuronal and biochemical pattern-formation over +Ade areas. t is therefore highly unlikely that

lesions will give rise constellations of behavior that have any internal coherence in terms of the representations and

algorithms needed to characterize normal function ng. Arguments of this type look quite reasonable, although the

conclusion is apparently false. For example, studies of primary memory for speech and natural sounds indicate that

separate input buffers are required to account for observations of double-dissociation of function between the two

stimulus classes (Shallice, 1979). Analyses of Broca's aphasia have shown that the pattern of toss and retention can

be characterized at an independently motivated levet of grammatical representation (Kean, 1979) that is distinct

from the level perturbed in jargon aphasia (Buckingham, 1977). For many years we have assumed that such double

dissociation places strong constraints on how one is allowed to interconnect functional boxes. Yet nerve net

simulation studies have demonstrated that double dissociation can be found alter lesion of a fully distributed system

in which each neural element participates in all behaviors (Wood, 1978), the exact form of the deficit depends

nonetheless on the number and location of the damaged elements.

I draw from Wood's study three conclusions that are, happily, in the spirit of A & C's paper: (1) Problems of

functional localization cannot be solved in their entirety within the domain - neurological differential diagnosis - that

gave rise to them (2) Neurophysiology must become computational; at the moment, half the world believes in

grandmother cells (Barlow, 1974) and the other hall believes that "attempts to ascribe perceptual or cognitive

function to the activity of single cells or localized anatomical regions are both experimentally and logically untenab

e" (John & Schwartz, 1978). A little computation might convince both sides that they are wrong. (3) We shall never

understand the form of language-representation in the brain unless we conduct intensive studies of the recovery

process; serious modelling at the network level should help us to see how qualitative changes in behavior may

follow from quantitative changes in neuronal hardware. Traditional diagram-makino has certainly never shed much

light on the behavioral effects of "late changes in the nervous system" that Geschwind (1974) so lucidly reviews.

I think I could agree, then, with A & C's philosophy if they would let me repharse it as "Neurolinguistics must be

theoretical," and it they could now move from metaphor to model. Barlow (1972) writes: ".

if the activity of a low-level neuron is like the occurrence of a letter, that of a high-level neuron is /Ike the

occurrence of a word—a meaningful


[473]

combination of letters." Fine, but I shall let George Eliot have the last word: We recoginze the alphabet; we are

not sure of the language."

Richard F. Reiss A neurolinguistic computation: how must -must- be understood?

Lower Heyford, Oxforderhite,'Englend

The most interesting part of this paper is its protean title. It is very apt Its creation and meaning should provide a

nice problem for any neurolinguistic theory, especially one animated by a computer.

Since no such theory exists above some vague rudiments and since it is doubtful that A & C's impatient ambition

will be satisfied soon, one can only analyze their chosen title in shifty light from the paper it heads.

Unfortunattey, some verb forms can speak in many modes As it stands, this paper's title has no clearly particular

meaning because that depends upon the mood of "must be." Most papers have titles in the indicative mood, but in

this case the "must" virtually rules that out. Futhermore, the paper is riddled (quite properly) with speculative words

such as "may," "might," "can imagine," and so forth. This quality suggests that the title be read in the subjunctive

mood

However, the subjunctive is too soft to fit the paper's air of thrusting confidence. Is it possible, then, that the

authors intend the imperative mood? No. They are surely too modest to presume to dictate the future course of all

neurolinguistics.

Still, there is an odor of command about the title. Perhaps it is in the obligative mood. That would be natural to

those who see a lawful necessity in their technical opinions. Contrarily - and ironically - one may faintly hear in

"must be" a tone of plaintive appeal (stemming from vested interest?) that casts the title into the precative mood.

Indeed one might even inter the invocative mood, although the text contains no references to gods.

If that be too extreme, certain features of the paper strongly imply at least an incantative mood in the title. It is a

small step from cant to incantation, and this work, like many in the allied literature, gives an impression that the

authors hope to breathe life into dells of clay (or silicon) by the relentless chanting of statements in the magic triple-

speak jargon. Mentalistic, physiological, and computer terms are simply mashed together by brute syntactic force;

sanitary canons of logic are blithely ignored.

Lest this criticsm give too much comfort to philistine conservatives, I hasten on to more generous views. One

might take the title to be in the chiliastic mood and see the turbid jargon as merely an unsightly byproduct of

reckless enthusiasm. Given the depressing review that consumes the first half of the paper, one wonders at the

jubilant spirit of these authors (and their ilk). Obviously the neural side of neurolirguistics is a mess of obscure

evidence and conjecture unlikely to be much improved in this century. To call this paper's schemes premature is to

indulge in silky understatement. At the minimum one should read "must be" in the anticipative mood of courageous

hopers.

Or it might be read in the hortative mood, for the text exudes an urgent optimism not unlike that of sincere

salesmen. Thus the hucksterish aspect of their cant sounds less alarming, almost endearing. Until, that is, one recalls

the past thirty years of inflated claims and disappointing performance in similar products. Eventually it becomes

tiresome to watch no end of fences rushed for showy jumps and Promising riders thrown, some crippled for life and

their sound horses Maligned to boot.


"Enough of this precious analysis," the authors may well object. "Any fool can see that we meant our title to be

read in the simple predictive mood.-

Well. that's as may be, but the paper's plethora of alternative tones does tend to drown the thin predictive theme.

However, I am willing to abort my moody line of interpretation on the grounds of small textual clues that point to

the title's true significance. For example, we find in section 5 the helpful phrase' ". . the particular computational

content of each species . ."

This and other slips reflect a prevailing usage of "computation" that has destroyed its power of discrimination.

Actually, there is no puzzle about the paper's title: it is a cryptic but simple tautology. by Barry Richards

School of Egistemica, University or Edinburgh, Edinburgh EH8 91-W, Scotland

Neurolinguistics: grammar and computation. On a computational model the grammatical competence of a

speaker is typically rendered as a processing capacity which interacts with various 'pragmatic' processing capacities

for the production and understanding of utterances. Although the boundary between the syntactic and semantic

features of language may not be entirely precise, we can assume that the distinction is reasonably clear and hence

allow, as do some programs, that there are different processing systems corresponding to the two types of features.

Bresnan (1978) and others suggest that these systems should be seen as involving only a very limited short-term

memory, in contrast to the memory used by the pragmatic processes o' comprehending language in context. The

extensive memory which speakers accumulate and carry forward from context to context is seen to imply that there

is relatively little pragmatic processing required to comprehend particular utterances: much of the contextual

information that would have to be processed is already there in store. It might be conjectured then that the bulk of

the processing underlying comprehension must be either syntactic or semantic. Even so, Bresnan concludes that this

too must be rather limited, for the usual rapidity with which language is understood in context is, she assumes,

ncompatible with extensive Processing, whatever its sort. Accordingly, she would recommend that an acceptable

model of comprehension should minimize the amount of processing and maximize the information in store.

It is interesting to speculate how such a model might be empirically justified. that is, how it might be realized

neurologically. Luria would suggest that the neural mechanisms of grammatical processing are located in some

particular region of the brain, perhaps in the parietotemporo-occipital zones of the left hemisphere. Lesions in these

zones have been shown to interfere with the interpretation of certain grammatical constructions, and hence it may

not seem unreasonable to conjecture that the general capacity for processing grammatical information is located

there. Regionalization, however, should not be taken to imply that this capacity works independently or in isolation

from the other languace processing capacities. As A & C point out, language comprehension appears to involve

"cooperative computation.- at least among the syntactic, semantic and pragmatic components. The form of

interaction envisaged is one in which the computation of one component depends upon the course of computation in

the others. On the assumption that these components are all regionally located, the model of the brain, so far as

language is concerned, is that of a network of regions which interact cooperatively in bringing about comprehension.

Now what, we might ask, would be the effect of a lesion in such a system? If we conceive of it as yielding a gap

in an integrated process, we might expect that the capacity to comprehend language in context will break down

entirely, such as would happen in a program containing an intermediate step for which its successor is not

determined. Such a simple view would not, however seem to be satisfactory: lesions do not typically cause a

complete collapse in the system, i.e. one where there is no output whatsoever. Deviance there surely is, but a


deviance measured by some unexpected output. For this reason we might be inclined to think of lesions as

generating nonstandard programs, perhaps by allowing certain processes to be bypassed or even by creating new

ntermediate components. Brain damage would accordingly be interoreted as yielding 'error generators' which derive

as deviations from standard processes.

This naturally leads to the question of how an aphasic syndrome might indicate or identify the sort of deviation

involved. In the case of phonemic paraphasias A & C sympathize with Schnitzer's hypothesis that the problem lies

with the input to the phonological component and hence suggest that we might "seek to show what discrimination

must be made in transforming some internal representation into a sequence of phonemes, and then analyze

conditions in which errors would be unavoidable with limited computing resources." When dealing with sensory

perception like hearing, there would seem a reasonable possibility of testing conjectures about the neural

mechanisms for converting internal representations to phonemes. But when we, so to

[474]

speak, move deeper and deeper into the processing model, say to the grammatical component, we might wonder

whether the empirical data can effectively constrain the choice of processing deviation.

Consider, for example, the phenomenon wherein a sentence like

1) Father and mother went to the cinema but grandmother and the children stayed home

can be understood, but not a sentence like

2) A lady came from the factory to the school where Nina worked

Luria would explain the difficulty as art inability of the hearer to synthesize the grammatical elements into a

single logical scheme. From a computational point of view this disability might be interpreted as consisting in a lack

of processing capacity. But where, we would ask, is the oeprivation to be located in the model? Is the loss one of

syntactic capacity, semantic capacity or some interactive process involving the two? A satisfactory answer to this

question will obviously have to specify how the effects of the lost capacity can be isolated from the other processing

capacities, otherwise there would be no insight into how the deviant comprehension system can understand (1) but

not (2). It might be useful, therefore, to speculate in detail as to how this isolation might be realized. if only to

indicate more definitely how a process model can mediate a neurological justification of a descriptive account of

language comprehension.

It would also be interesting to speculate about the general empirical constraints which affect the 'translation' of

computational processes into neural mechansims. In the case of phonological processes it might be assumed that

there are enough empirical data to generate a reasonably precise model, one yielding an illuminating simulation of

neural mechanisms. But for syntactic and semantic processes the data, such as they are, would seem to be neither as

stable nor as transparent. Hence, there might be reason to wonder whether there is an adequate empirical base to

generate a simulation that can be similarly illuminating. The known points of contact, as it were, between neurology

and processing are very limited and this might be seen to render simulation a premature enterprise.

For example, we might query the neural import of the shift from processing to memory in the model adumbrated

above. It might be argued, from a psychological standpoint, that the evidence does not justify the assumption. The

rate of comprehension in context, though typically very rapid, is not obviously to be measured in terms of the

amount of processing_ what takes a long time to comprehend may plausibly take little processing, and equally what

takes a short time to comprehend may take a lot of processing Surely it is not impossible that a great deal of


processing may occur in a very short time. We might note that the complexity of the HEARSAY program would

seem to assume just such a possibilitiy Without secure psychological evidence of the actual distribution between

processing and memory, we might become worried about the possibility of finding, in a model which slants the

matter, an accurate simulation of neural processes. It would be reassuring if we could envisage a neural input which

might constrain or even determine our view of how the facts might be.

To put it generally, it would be most interesting to see how, or perhaps whether, neurology might 'transcend' the

vagaries of psychology to constrain the building of processing models, lf we could appreciate this, we might

understand more clearly how neurolinguistics must be computational.

Roger C. Schank Process models and language

oep,rmene of Computer Science. Yale University. New Haven, Conn. 06520

A & C's contention that process models are the best way to view neurolinguiStics seems easy enough to grant,

given the perspective of someone immersed in Artificial Intelligence (Al), and thus in process models. The central

questions, it seems to me. are what exactly A & C mean by process models and what neuroscientists might gain

from describing the phenomena in which they are interested in such terms.

In the most general sense, a process model is no more than a point of view Simply stated, the point 01 view is

that to describe a phenomenon it is best to deal with the flow of control among processes. , A process, in this view, is

just something that happens, it is a test-action rule, perhaps. Thus a process model, in its general form, is just a set of

rules that simulate the overall input-output behavior of a system when executed in the proper specified order.

Now what I have stated above is not particularly radical, or even interesting, to anybody working in the area of

computation. And I would not bother to state it now - indeed it seems so obvious to most computer scientists that

they often marvel that it ever has to be stated - we-e it not for that fact that I am well acquainted with the utter lack of

comprehension of process models in linguistics.

Generative linguists have failed to understand the role of process models, relying instead on the ill-thought-out

competence/performance distinction This distinction had allowed transformational linguists to propose models that

do not correspond to the input-output behavior of any known system. The competence /periormance distinction -las

thus taken on the role of a kind of religion, explaining the unexplainable, in this case, that linguists' models have

tailed to be useful for any computational model or to correspond to any psychological evidence.

Some linguists and psychologists have, as A & C note, become enamored of late with augmented transition

networks. abandoning their lack of interest in process This is all well and good considering where these linguists

have been coming from but paradigm shifts ought not to be made too much of until they are finished What I mean

by this is the substance of my uneasiness with A & C's article.

Assuming that neuroscientists are among those who need to be informed of process models and their advantages

for science in general (1 have no evidence of their need to be so informed one way or the other; I will just take A &

C's word for it), what do they ned to be informed of?

A & C seem to imply that there is a body of knowledge o oth linguistics and Al that, if only neuroscientists paid

attention to it. could build upon all this work and create giant functioning models oS, the brain. I could not disagree

more strongly.

At present. linguists have only the barest understanding of what is going, on in language processing. issues such

as the interaction of knowledge in disambiguation. the nature of meaning, the role of inference in understanding, the


application of high level knowledge structures for predictive understanding, the construction of sentence fragments

prior to the completion of thought are only a very few of the language processing phenomena that are only

beginning to be studied and can hardly be claimed to have provided much in the way of results. The whole problem

of the interconnection of language and memory is not even remotely understood right now What is it that

neuroscientists are supposed to be learning from linguists? It cannot be much more than ill-formulated theoretical

assessments of very low-level syntactic phenomena, the only kinds of phenomena that linguists have systematically

studied. (And what has been studied was worked on outside the framework of a process model and thus often

outside any connection with meaning!)

Vio hat can neuroscientists learn from Al? A & C suggest that they buffo giant computer models on the order of

HEARSAY. Such a suggestion cannot possibly be taken seriously. The complexity of building a system such as

HEARSAY cannot be easily described Suffice it to say that it is necessary to have five to ten extremely bright

computer scientists, well trained for a number of years prior to beginning work on such a project, and then you will

get a HEARSAY in a few years.

And, while HEARSAY was a marvelous achievement and was very innovative in several areas, especially in its

overall system design. it really never worked all that well. Very few Al models ever do We in Al build them because

by doing so we learn a great deal about complex processes and the kinds of knowledge people have.

To suggest that neuroscientists build them is to suggest that neuroscientists stop being neuroscientists and

become Al investigators I do not see what use that would serve

From a more general perspective, however, 1 agree with what I construe to be A & C's real message: To

understand brain functions better, it would probably be a good idea if neuroscientists and Al people got together to

discuss issues concerning how neuroscientific

[475]

data might serve to validate or invalidate Al process models and how Al theories might suggest avenues of

research in neuroscience.

Marc L Schnitzer Computational neurolinguistics and the competence-performance distinction

avadr,,ent,o English. University of Puerto Rico, Rio Piedras, Puerto Pico 0093?

. A & C's article is a significant contribution to ne.00linguistic theory_ Neurolinguistics has been lacking

substantial theoretical grounding, when by "theoretical" one understands something beyond mere synthetic overview

of results of experimental and case studies A & C's contribution is an important step toward understanding what is

required of neurolinguistics if it is to become a scientific discipline. As LaMendella (1979) notes in his review of

neurolinguistics for the Annual Reviews of AnthropologY:"Theoretical linguistics could not be content to merely

identify a 'phonological' level of language structure and not go on to derive a theory of the organization of

phonological structures. Theoretical neurotinguists should not be content to identify a 'motor speech region' (Broca's

area) or a 'sensory speech region' (Wernicke's area) without having as a high priority the description of the

organization of speech functions and subfunctions which accounts for the capacity of these areas to process speech."

tt is just such a goal toward which A & C direct their article. In the first three sections, they classify approaches

to neurohnguistic theorizing into faculty, process, and representational models This taxonomy seems to be an

entirely appropriate approach to the problem. The "faculty model" is shown to be inadequate insofar as it ignores the

complexity of language. The critique of "process models," although virtually restricted to the work of Luria, is


important in that it makes clear what is required for a model to be computational. The main criticism of

"representational models" is that they fail to integrate linguistic findings into "general process models." The

approach presented in section 4 involving an Al program called HEARSAY-II, although suggestive in terms of its

computational possibilities as a model of language comprehension is rightfully found lacking in that, as an Al

program, it contains components that are not neurologically feasible many of its features are determined not on a

neurological basis but on the basis of implementational capabilities of a computer. The authors raise some of these

questions in the fifth section of their paper, where they rightly argue for the necessity to integrate computationally

oriented theorizing with micro level neurophysiological analysis

But the paper ignores the problem of the relation between knowledge and processing. Within linguistics and

psycholinguistics this distinction has been i efeirect to as the competence-performance distinction. I have argued

(1978) for a neurolinguistic model that can incorporate both knowlecge and behavior in a way that deals with these

notions as two aspects of ore system. Although more data are needed, preliminary data (e.g., ZJrif et al, 1972,

Schnitzer, 1974) indicate that at least some aphasics present similar deficits across a variety of input and output

tasks. In such tasks, computational procedures would necessarily be different from each other. This points to the

likelihood of the necessity of an underlying knowledge system that can be utilized in different kinds of behaviors.

Thus, what seems to be needed is a model of language, consistent with what is known about neurological

processing, that can represent language as a system of knowledge that can be brought to bear in all language tasks.

I have argued (1978) in favor of a stratificational network model for language. Such networks consist of relations

only - no items and no DrOcesses. Thus such models viewed statically can be considered as representing linguistic

knowledge per se. These same models may also be viewed dynamically, in which impulses traveling through the

relational network are part of the routines for performing various hnguistic tasks. Viewed as such, stratificational

grammar may in fact satisfy A & C's requirements for a computational model, while provid- mg a model of

underlying knowledge.

But as yet it is not totally clear that we need to incorporate a model of knowledge in neurolinguistic theory. More

evidence is needed. If it is true that some patir nts with language deficits manifest characteristic deficits regardless c

f the linguistic task they are performing, then we do in fact need tc come to grips with the problem of knowledge as

well as of computatio'i If this is not the case, that is, it linguistic deficiency caused by brain damage is task

dependent, then perhaps we can view knowledge as an epiphenomenon and concentrate our efforts on computational

models, as A & C have suggested.

One Author's Response

by Michael A. Arbib

Cooperative computation as a concept for brain theory

A rather dynamic• pattern of travel has precluded a cooperative reply to the commentaries from both co-authors

before the editorial deadline, and so I have prepared this response without benefit of Dr. Caplan's dual expertise in

linguistics and clinical neurology. I hope to see his response in a later issue. Meanwhile, I shall use the first person

plural when discussing our joint opinions as expressed in the paper, and the first person singular in presenting my

own response to the commentaries.


Our target article, "Neurolinguistics must be computational", has three themes: (1) a history of aphasiology that

offers a trichotomy of faculty models, process models, and representational models; (2) a brief account of Artificial

Intelligence (Al) models of language processing, with particular stress on HEARSAY H to advance the reader's

notion of cooperative computation, and on this basis, a sketch of how one might go about building a cooperative

computational model of neural mechanisms underlying language function; (3) the suggestion that studies of the

action/perception cycle might provide a bridge between neurolinguistics and synapse-cell-circuit neuroscience,

After analyzing general themes of computer modelling and neurolinguistic theory, we will take up the three

themes in turn and will then close with a review of the issues involved in developing a computational

neurolinguistics.

Are computer concepts relevant to the brain and behavioral sciences? In 1963, 1 toured many neurophysiology

laboratories. A few had recently received LINK laboratory computers, and there was much debate as to whether

such devices had a useful role to play or would simply encourage the wholesale collection of data unrestrained by

the discriminating judgment of the experimenter. Today, there is no debate - many of today's most exciting

neurophysiological experiments would be unthinkable without the use of a computer. But if the debate about the use

of computers to aid experiments in neuroscience has ended, it is clear from the commentators that the use of

computational concepts to aid theory in neuroscience or cognitive science still raises instinctive fears. FREEMON

finds it unnecessary to adorn with argument the simple assertion that "computers are dumb." BLUMSTEIN is

unequivocal in saying that our attempt at brain theory must fail as did those of our predecessors that reflected the

state of technological advances of the time. But some predecessors have succeeded. For example, Houk, Robinson,

Stark, and Young have built upon technological advances in control theory to give us new insights into the neural

processing involved in the muscular apparatus, eye movements, and so forth. [See Roland: "Sensory Feedback to the

Cerebral Cortex During Voluntary Movement in Man" BBS 1(1) 19781 But perhaps Blumstein's negativity comes

from misascribing to us a naïve crudity that we do not possess. We do not say that the brain is a computer in the

sense of an anatomical isomorphism with the latest DEC or IBM machine, or in the sense of serial execution of the

steps of an

[476]

externally provided program. Rather, we try to confront our knowledge of both Al and brain modelling witl an

appreciation of the findings of clinic and laboratory to develop a stock of concepts, especially those of cooperative

computation, that can contribute to the development of neurolinguistic theory. The theory of effective computation

of the 1930s gave us the rewriting systems that provided the framework for Chomsky's approach to the syntax of

natural languages (although, mysteriously, Chomsky gave no signs of being aware of this debt in his original

monograph on "Syntactic Structures" and so may have absorbed these ideas only indirectly). It may thus be debated

whether our particular choice of concepts is useful for neurolinguistics, but there are no general grounds of

technological fashion on which they can be rejected. [See also Chomsky. "Rules and Representations" BBS 3(1)

1980.1

In The Metaphorical Brain (1972, p. 11), I noted that -when we have refined them and discarded many of their

false importations, [our metaphors may eventually yield concepts that] can accommodate most of our questions

about a given subject matter, (at which stage] we can afford to ignore their metaphorical origins," We no longer

think a discussion of the thermodynamics of the cell is really about steam engines. Thermodynamics now applies to

1


steam engines but is not restricted to them. In the same vein, cooperative computation is a body of theory (still

poorly developed and mired in a restricted set of examples) not restricted to computers. 1 want to develop a general

theory that applies to the analysis of brains as much as to the design of networks of electronic computers

Given a theory of some aspect of cooperative computation in the brain (the computer as metaphor), we may still

wish to test it by computer simulation (the computer as tool). But BUCKINGHAM seems convinced by Dreyfus

(1979) that the task of computer modelling of mental processes is impossible. However, none of the objections he

cites are compelling. Does it really block linguistic theory that -information must be digitized into discrete units" to

be manipulable by a machine? I will rest content with a model of speech understanding that can handle tape-

recorded signals or with a theory of production whose output is the typing of flawless English prose? Top-down

processes in programs like HEARSAY account for the expectancy effects that disturb Buckingham; and the state of

the HEARSAY blackboard is an, admittedly crude, model of a "situation- that determines what is relevant at any

given time. What of Dreyfus's assertion that -Al workers are faced with storing and accessing an infinity of facts, or

with having to exclude some possibly relevant facts from the computer's range of calculations?" Well, the same is

true of NASA's calculations about the flight of spaceships -thus the uncertainty as to whether Pioneer II would

safely traverse the rings of Saturn. Note that I do not claim that cognitive science has come near the explanatory

adequacy of Newtonian mechanics. Nor would I deny that some of Dreyfus's arguments remind us that modelling is

hard. But I remain totally unconvinced that he has shown the limitations of computer modelling of mind and brain.

That some workers in early Al were filled with a euphona that led to excessive claims justifies a critical examination

of those claims; but I see no bar to centuries of progress that will call on the combined talents of experimentalists

and theorists, making use of the computer as a tool and of computation as a rich source of concepts for the analysis

of complex systems.

What's in a title, M'Lord? The title of our paper, with its possible misinterpretation that "Neurolinguistics must be

computational, to the exclusion of all else," has led to much discussion. HUDSON captures more of our intention

when he expresses the belief that "there must be a computational neurolinguistics, alongside other forms of

neurolinguistics.- We see the study of the neural underpinnings of language as a cooperative enterprise:

psycholinguists, anatomists, and clinicians will, we hope, be joined by computational neurolinguists. We did not

mean to imply, as scHaml suggests. that "neuroscientists stop being neuroscientists and become AI investigators."

Nor do believe that AI models can just "plug in- to neurolinguistics. A computational neurolinguist must (that word

again) combine knowledge of cooperative computation, linguistics, and neuroscience, but with the emphasis on

theoretical analysis and computer modelling, rather than on clinical observation or linguistic experiments.

rsnss gives a devastating critique of our title. Perhaps the kindest of his analyses (a set-up, I should note) is that

"at the minimum one should read 'must be' in the anticipative mood of courageous hopers." And GARDNER finds

the crackling of promissory notes pervading the whole last section of the paper_ These comments are to the point

The paper does not report on new data and it does not present a new model. Rather, it reviews two main styles of

neurolinguistic research - those of the neurologist (faculty and process models) and the linguist (representational

models) - and suggests that they are complementary. We examine the HEARSAY model from Al and some

modelling of neural mechanisms from Brain Theory (BT) and suggest that these are aspects of an evolving theory of

"cooperative computation." We outline ways in which these new (still half-formed) theoretical ideas might be used


to construct a more adequate neurolinguistic theory. Perhaps, then, a more adequate title would be "The Promise of

Concepts of Cooperative Computation in the Future Development of Theoretical Neurolinguistics."

Ir. admitting that the paper is in "the anticipative mood," I would not ...rant the reader to think that 1 take our

promises lightly. The last ten years have, to my mind, seen concepts of cooperative computation become firmly

entrenched in Al and BT. My study, such as it is, of neurolinguistics makes me confident (a personal, not a

scientific, judgment) that too few neurologists and linguists are aware of these issues, and it is for this reason that the

type of "debate" that BBS makes possible seems so appropriate I detect iri,„ some commentators a reactionary

attitude - it is hard enough to k up with clinical data and linguistic theory without being burdened .th computational

theory as well? But many of the commentaries ha served well to make explicit many issues raised most implicitly in

the paper, and to help define some of the steps that might be taken to turn our promises into solid scientific research.

The goals of neurolinguistics. As MARSHALL has stressed, neurolinguistics developed in the context of clinical

medicine, with the goal of predicting lesion-sites from symptom-complexes and vice-versa. But, as BUCKINGHAM

notes in quoting Jackson (1874), "to locate damage which destroys speech and to locate speech are two different

things." For this reason we cited with approval Luria's (19,3) assertion that our fundamental task is to ascertain

"which groups of concertedly working zones are responsible for the performance of complex mental activity [and)

what contribution is made by each of these zones to the complex functional system." (The remainder of the quote

discusses developmental studies, and GARDNER is quite right to stress that these studies, which we did not discuss,

provide a major feature of Luria's approach.) The citation f rora Luria transfers the emphasis from the brain-

damaged patient to the normal subject. We seek a theory of how n brain regions interact in some normal

performance. The performance of the patient with damage to one of these regions (but note the important caveats

about the scant respect that lesions and tumors show for the boundaries of such regions) is then to be explained in

terms of the interaction between the remaining n-I brain regions, rather than in terms of the properties of the region

removed. As RICHARDS notes, the system must still be able to produce a performance, despite the missing region -

quite unlike the total breakdown that would follow removal of a subroutine from a program for a conventional serial

computer (see Arbib (1975, section 5) for further discussion). In any case, FRAZIER is quite right to stress that one

should append additional mechanisms to the model to account for the effects of damage "only once it has been

shown that an adequate theory of normal language processing will not automatically account for the linguistic

behavior of aphasics" - the only delicacy being that it may require a rather subtle theory to determine the force of the

"automatically" in the above quote. It was for this reason that we expressed caution, in Section 3, about Lecours's

explicit postulation of "error generators- as underlying phonemic paraphasias. Note, however, that it is the data on

abnormal

[477]

behavior that provide some of our best clues about the neurological validity of processes postulated in a model of

the normal, which leads me to doubt HUDSON'S strategy of building a model for normal function without reference

to the neurological data.

BUCKINGHAM notes Chonisky's stress on grammaticality as a core question in linguistics and suggests that this

kind of knowledge is nowhere to be found in our model. We do not have a model, but in Figures 9 and 10

HEARSAY'S grammar was mentioned in passing. We deleted a fuller discussion of grammar simply because we


assumed that it was on matters of cooperative computation, rather than on the more familiar topic of grammar, that

we had something to add to current discussion of neurolinguistics.

One's perspective on neurolinguistics would seem to depend a great deal on one's starting point. We have seen

that the classical starting point is the neurological clinic. My own starting point is the study of neural mechanisms of

visuomotor coordination and an evolutionary perspective that leads me to search for common mechanisms for

"action-oriented perception" and "language use" to provide a base from which to explore their differences. We thus

asserted at the end of Section 4 that "a grammar interacts with and is constrained by processes for understanding or

production [; while] linguistic representations and the processes whereby they are evoked interact with information

regarding how an utterance is to be used, in a 'translation' process.- This hypothesis is based on a view of language as

evolving in a context of well-developed cognitive abilities (which are then modified in turn, just as visuomotor

processes in the superior colliculus are modified by descending pathways from the visual cortex). In total contrast

are those linguists and philosophers, exemplified by KEAN & SMITH, who look for an "autonomous" theory of

language processing. They reject the thesis that diverse information is used in language processing and see "the core

of the linguistic processor to be the generation of phonological, morphological, syntactic, and logical

representations, with the diverse nonlinguistic information exploited in everyday language use inaccessible to this

core.- This thesis seems to have evolved from Chomsky's original claim for autonomy of syntax. This claim was

based on a divorce of linguistics from processing, let alone neurological, issues. Kean & Smith are confident that

"given an explicit theory of the various levels of linguistic representation, many of the details of the computational

procedure will be transparent." A computer scientist is tempted to reply that "given an explicit specification of the

computational process, many of the appropriate data structures will become obvious." In short, I think it would be

silly to discourage intensive efforts in neurolinguistics that are primarily linguistic, but these can only be a part of

the overall emphasis when the goal is to understand the interaction of multiple brain regions. Here I believe that the

neuroanatomist and the brain theorist have much to offer that the linguist cannot.

scRANK rightly emphasizes that Al may help us address the role of knowledge, memory, and inference in

language processing. However, as FRAMER for one, would emphasize, an arbitrary process model need not have

psychological validity, and much remains to be done to determine what the "scripts" and "frames" of Al have to

offer neurolinguistics beyond the basic insights into schemas offered by Bartlett (1932) in his psychological studies

of "Remembering." By the same token, KEAN a SMITH should heed GrinDNER's question "Who is to say that the

many subtle distinctions propounded in contemporary generative grammar will prove to have any greater relevance

to neurolinguistics than they have had to developmental psycholinguistics?" Note, too, Gardner's assumption -

shared, Presumably, by any neurologist who studies agnosia and apraxia along with aphasia - that "language may be

integrally tied to, and make use of, other behavioral systems." He felt that we took it "as a working premise that one

should begin by modelling language functions separately" and urges attention to the role played by various

pragmatic and cognitive systems in language comprehension. We agree, and I apologize if our expository structure -

HEARSAY first, sensorimotor processes second - obscured this agreement Certainly, neither Geschwind's nor

Luria's analysis of naming can find their common refinement, save in the context of a thoroughgoing analysis of

visual perception. Of course, such a refinement must avoid the "naive faculty invention" criticized by OinonCLASs.

From trichotomy to dichotomy. We suggested that faculty models (e.g., Wernicke's, Lichtheim's and

Geschwind's) tend to treat major functional psycholinguistic tasks as unanalyzed wholes, while process models (e.g.


Luria's) provide more detailed analyses in which each component may contribute to several psycholinguistic tasks

with the entirety of any task requiring the interaction of several components. I am convinced by the commentaries of

BLUMSTEIN, BUCKINGHAM, GARDNER, and MARSHALL that there is no such sharp dividing line. However,

this distinction is not crucial to our argument. What is important is the contrast between the emphasis on localized

components of the connectionist and faculty models on the one hand, and the linguistic turn of the representational

models on the other. We do not, as GOODGLASS suggests, offer Luria's analysis as our own. We offer it as an

important historical achievement, not as the endpoint of analysis. I thus regard Goodglass's critique of our Figure 4

as in no way damaging to our argument, but simply as advancing the argument for a more delicate parcellation of

subfunctions.

I do not have the expertise to debate uLums-rEtiv's views on the limitations of the work of SCHNITZER and

KEAN, although my own prejudice is tc agree with the inadequacy of a purely phonological approach But

Blurnstein and GARDNER misrepresent us when they say that the "discussion of models of language and the brain

culminates for A Si C in representational models," or that we view such models as inherently superior to faculty and

process models. As stated at the end of Section 3, we value the "representational" models for their attempt to use

modern linguistic methods to try better to characterize different levels of linguistic code involved in performance,

but we are unhappy with these models to the extent that they do not address the data on subtask localization that we

reviewed in our discussion of the (no longer so neatly separated) faculty and process models. We believe that a

central task of neurolinguistics is to develop an integrated theory that draws on the insights of both approaches.

There seems little argument with this observation amongst the commentators, but one should stress that some

followers of Jackson would consider our faith in subtask localization unjustified. I must reemphasize that it is highly

unlikely that the codes posited by psycholinguistics will match with neural codes until after much "iterative

adjustment" as linguists and neurologists come to calibrate their theories and experiments better to the needs of

cooperative research.

To lay some of CREENBLATT's concerns to rest, let use stress that the integrated theories we envisage will

incorporate a great deal of neuroanatomy as we try better to characterize regions within the brain and the detailed

projections that link them. But what manner of "region" will enter meaningfully into neurolinguistics? Such gross

anatomical regions as the parietal lobe or the cerebellum are too large, while individual "columns," let alone

individual neurons, are too fine to make contact with conceptualizations at the level we have diagrammed in Figure

8. (Does a word in the lexicon have its own set of columns in the cortex? Would such a set contain one column or

many thousand? It is perhaps premature to speculate, but we find some clues in a recent study of changing

visuomotor representations in the cortex of the cat (Spinelli & Jensen [1979]). Classical cytuarchitectonics and

modern biochemistry provide different answers for differentiating the brain into a multiplicity of separable

populations. Figure 19 of Szentagothai and Arbil) (1975) shows the neuron network of the cerebellar cortex

schematically transformed into five two-dimensional matrices, reminding us of the insight to be gained by

subdividing a region into interacting layers; while Boylls's synergy controller model, as shown in chapter V,

suggests that the cerebellar cortex in isolation cannot be viewed as a meaningful unit in motor control, and that it is

only in relation to a number of adjacent nudei that is posited role in the adjustment of motor synergies can be

defined. What is interesting for our discussion of methodology is that Boylls's choice of units resulted from the

[478]


confluence of a theoretical analysis refining the Bernsteinian theory of synergies and a detailed review of the

constraints and mechanisms documented in the anatomical and physiological literature.

The status of the cooperative computation methodology. BUCKINGHAM took pains to lecture us on the

distinction between AI and CS (computer simulation of human cognitive processes), but in fact our distinction is

between Al and BT. In other words, we go beyond the general notion of a process model in SCHANK's sense of "a

set of rules that simulate the overall input-output behavior of a system when executed in the proper specified order"

to one in which the various processes are mapped onto anatomically characterizable portions of the brain. (The

problem of choosing the -grain- for these "portions" was briefly- discussed above.) The division between Al and BT

is made very clearly in our paper. Nonetheless, BLUMSTEIN accuses us of "enthusiastically" plugging in a model

based on state-of-the-art computer technology without regard for known physiological and neurological facts. She

misses the point that we did not offer the C2 configuration of HEARSAY II, 'Sketched in Figure 10, as a model of

language processing in the brain. Rather, we outlined in Section 5 how the methodology instantiated in Figure 10

might plausibly be adapted to contribute to neurolinguistic modelling. A valid criticism of our paper (and this

response) is that it contains no example of a model within the new methodology. This is at the heart of

MARSHALL's complaint about "the buzzing, blooming confusion" and his request that we move from metaphor to

model. But Blumstein's criticism is, to my mind, invalid. She states that -in their discussion of 'Heresay', [sic on

original manuscript, corrected in proof] they say '. . we may view each KS (knowledge source) as having its own

'processor' in a different portion of the brain'. Why? Perhaps because that way the system works; there is certainly

no evidence from aphasia in particular, or neurology, in general, to support such an underlying system. - Since we

have not stated what the KSs might be, or what the relevant anatomical regions would be, in a neurolinguistic

HEARSAY, her statement can only be true if she believes that no anatomical localization of linguistic processes is

possible. The discussion of Boylls's modelling of motor control above indicates the kind of interactive definition of

region and function that we envisage - and which will be necessary in neurolinguistic theory no matter what the fate

of our promises (which are, after all, based on HEARSAY) may prove to be. A genuine problem for a cooperative

computational theory is raised by RICHARDS when he asks "whether there is an adequate empirical basis to

generate a simulation that can be . . . illuminating-" My suspicion is that only detailed models can generate the

questions that can be used to generate an adequate empirical base- Thus, if our case seems persuasive for

cooperative computation as a dominant style for brain function, then this concern with the adequacy of the empirical

data only adds to the urgency of the case for detailed model construction in this style.

Before going further, I want to correct a misunderstanding based on our order of exposition. Since the focus of

our target article is linguistic, we chose to introduce cooperative computation via a linguistic model, namely

HEARSAY. However, the cooperative computational style entered brain theory long before HEARSAY was

envisioned. Warren McCulloch's concept of "redundancy of potential command" led to the Kilmer-McCulloch

model of mode selection in the reticular formation (1969) and the Didday model of frog visuomotor activity (1970).

These are both models of cooperative computation in the brain and are summarized in Arbib (1972, Chapter 7).

Thus it is historically more accurate to refer to the style of modelling espoused in Section 5 as the "C 2 (cooperative

computation)-style- rather than the "neurologized HEARSAY-stylesince the latter arose from an attempt to fit the

linguistic aspects of HEARSAY into the preexisting (albeit still rudimentary) framework of cooperative computation

in BT. We turn, then, to the basic implications of adopting a C2 approach to neurolinguistics


First, I would strongly maintain, contrary to the narrowly linguistic view of KEAN & SMITH, that

neurolinguistics should not be restricted to the construction of purely linguistic models. I think it is a virtue of the 0-

style that it provides a framework for the future marriage of a parsing system with a cognitive system and an

intentional evaluation system. COHEN laments that HEARSAY is not a "comprehensive model incorporating the

whole range of human language abilities such as understanding speech. answering questions, producing stories,

describing the perceptual world, and holding conversations.- It is true that the implementation described in Section 4

only addresses speech understanding, but the C2-methodology espoused in Section 5 is meant to accommodate such

a general model. As a matter of practicality, one must pick a subtask, whether for modelling or for experimentation,

and one cannot know a priori whether this subtask is -neurologically valid." One will model it as well as one can,

adjusting the subsystems as one seeks to accommodate psychological and neurological data. The real interest comes

when we look up from our own subtask to look at a cooperative computational model of another subtask. My

expectation is that certain "knowledge sources" (KSs) in the two models will have major similarities, and that it will

take relatively little work to adjust these to have a single definition of each KS compatible with each model. As a

result, by identifying the instances of each KS in the two models, we may obtain a single model applicable to both

subtasks. We ;saw a hint of this when we offered, in Figure 7, a block diagram of subsystems involved in Luria's

analysis of repetitive speech and fourd that all the blocks had occurred in earlier analyses of other language tasks.

The exciting question is this- Would these identifications extend to the level of computational specificity? My

suspicion is that the commonality would not be possible until after extensive redefinition. By the same token, 1

imagine that we would learn much from seeking a computational redefinition that stiksumes the analyses of naming

of objects presented in Figures 3a and

SCHNITLER would seem to argue for a similar perspective whe he argues for his stratifications] network model

for language, althou much remains to be done to see to what extent his specific model is compatible with our

cooperative computational framework. Schnitzer is right to emphasize that more evidence is needed to determine the

extent to which linguistic deficiency caused by brain damage is task-dependent. But I think he is wrong to view

"knowledge" as something we ignore when we turn to computational models, and I think he is also wrong to view

"knowledge" as necessarily task-independent. I shall make the point in the language of HEARSAY: to say that some

KSs may be invoked in a variety of tasks is not to deny that each task may invoke a specific set of KSs.

COHEN stresses that HEARSAY (the actual implementation of Section 4) would perform better were the initial

acoustic analysis not so crude, sketchy, and error-prone and notes that we can recognize strings of words and

nonsense syllables without syntactic or semantic clues (an argument against the core of KEAN S SMITH?). She thus

argues that the hypothesize-and-test style of HEARSAY (the general methodology, this time) is not appropriate to

psycholinguistic theories. Kean & Smith also reject the use of the C2 methodology because they believe it stresses

unduly the marshalling of diverse information. Yet in the same paragraph as their rejection of the thesis that diverse

information is used in linguistic processing they state that "Widely diverse information is most surely brought to

bear in normal everyday language use." They give the game away when they later state that —It is not that (they]

deny a role to speech act considerations in language comprehension, but [they] do reject the idea that such

considerations infuse the core linguistic processor." They are so convinced that there is a core of -phonological,

morphological, syntactic and logical representations [to which] the diverse nonlinguistic information exploited in

everyday language use [is] inaccessible" that they blithely exclude "normal everyday language use" from the domain


of neurolinguistic explanation. Our proposed cooperative computational methodology does not; and we explicitly

reject the hypothesis of the autonomy of linguistic capacity. Our approach does in fact rest on a modularity

hypothesis - but, in line with our critique of faculty models, we view the modules as being far more refined than the

overall "cognitive systems" of which Kean & Smith take human linguistic capacity to be just one.

Nonetheless, as cam= a zume also observe, we have not proven

[479]

the neurological validity of the hypothesize-and-test style of cooperative computation - that the brain is modular

in a disciplined way and that it takes hypotheses at one "level" of description and submits them for test against other

levels of analysis, with multiple hypotheses active at any time. COHEN argues against this view on the basis of

man/machine differences in speed and memory capacity, but I believe that her objections arise in part from

emphasizing the high-level conscious processes of “serial" cognitive psychology at the expense of neural analysis

that focuses on the concurrent activity of hundreds of regions, each containing many millions of neurons. Similarly,

RICHARDS notes that Bresnan's arguments for limited processing underlying comprehension may be based on an

insufficient appreciation of the processing capacity inherent in neural parallelism. Again, many of the problems of

search in a serial computer implementation are solved in the brain if it can make use of a highly parallel content-

addressable memory. Perhaps, as Cohen argues. the brain can simply switch in the right process at the right time in a

smooth progression from input to output without the "competition and cooperation" of multiple hypotheses.

However, such anatomical data as the fact that there are more fibers going "the wrong way" from visual cortex to the

lateral geniculate than those going "the right way" do seem to run counter to straight-through processing (and

compare the discussion in Arbib [1972, pp. 109112]). Despite the arguments on both sides, the status of the

hypothesize-and-test style is an open question for Future research.

From regions to cells and circuits. In the latter part of Section 5, we suggested the desirability of building a

bridge from the regionby-region analyses of neurolinguistics to the synapse-cell-circuit analyses of much of 20th-

century neuroscience. We posited that the analysis of the brain in terms of cooperative computation of interacting

regions may provide a high-level description of each region sufficiently precise to allow detailed experiments on the

implementation of the function of the region in actual neural circuitry. It is only at this lower level that we can then

explore those disease states that are better described in terms of biochemistry than in terms of localizable lesions.

Unfortunately, this general thesis was not discussed in the commentaries. Instead, BUCKINCHANI provided a

useful discussion of recovery of function and expanded our discussion of Whitaker and Ojernann (1977),

disagreeing with us on the value of the DIPM model and on the proper interpretation of "first steps toward

observation of physiological events"; while MARSHALL offered a useful introduction to the simulation of lesioned

networks by Wood and to the status of "grandmother cells" in neural theories of perception.

coonotass noted our citation of the progress made in tracing the coding of information at the visual and motor

periphery and suggested that we imply that similar accomplishments are in the offing for language. But our

emphasis was quite different. We suggested that the existence of this neurophysiological information could constrain

our models of visuomotor coordination, and that these might in turn constrain our neurolinguistic models as we

compared visual perception to speech understanding, speech production to motor control, and the role of internal

model updating in the action-perception cycle to that in conversation. Incidentally, I See no grounds whatsoever for

Goodglass's suggestion that there is an invariance between signal and response in the visual and motor systems, so


the lack of invariance in the language system does not militate against the use of analogies between visuomotor and

language processes.

The status of computational models. Marty Ringle (personal communication) has stressed a "grain" problem for

neurolinguistics that is quite apart from the problem of defining anatomical regions. He stresses that the overall

characteristics of computational models seem to be too general for the neurolinguist who is trying to develop a

detailed model, while the fine structure of the computational model may be quite irrelevant insofar as neural

processing is concerned HALWEs notes the dangers of implementation, warning that "to embody one good idea in a

computer program may require a hundred arbitrary choices." GARRET & ZURIF -caution against the possible

conclusion that the necessary or useful degree of explicitness [of computational models' arises only from the

programming effort [and comment that] the actual details of computational procedures are rarely the basis of a test

ot language performance and are seldom invoked in argument for the use of a given AI system as a model of human

information processing." HUDSON notes that his own computational neurolinguistic model was analyzed by pencil

and paper withoit computer implementation, but that it could still provide valuable insights as the subsystem

interactions were constrained to yield a model that is "(a) linguistically appropriate, (la) details mechanisms for the

linguistic processes, and (c) goes on to define ways in which disruptions occur such that the distorted performance

maps onto the clinical data.- And I earlier distinguished the use of the computer as a tool for data-processing and

simulation from the use of the concepts of computation for the analysis of complex systems. What all this says is

that when we advocate the cooperative computation style for neurolinguistic theory, we accept and even encourage

that much of that analysis will he of the pencil and paper kind, without recourse to computer simulation to explore

the implications of the model. Nonetheless, we do expect computer simulation to be a useful part of computational

neurolinguistics, and so I turn to a few observations on the style of such simulations.

SCHANK attempts to discourage the implementation of HEARSAY-style models in neurolinguistics since

HEARSAY-II took many man-years to program and does not perform very well. In response to the former

objection, 1 would suggest that development of programming methodology will make many aspects of programming

such systems relatively routine and efficient - this has certainly been the case with the writing of compilers and

operating systems. The latter objection seems more serious. HALWES has noted the "patches" involved in

implementing HEARSAY-II, and still the system achieves less than 90 percent recognition of sentences based on a

1,000-word vocabulary. What is to be gained by degrading a model like that to explain the data of aphasiology? My

initial, somewhat feeble, answer is that models are not to explain everything GOODGLASS is concerned that

individual variability may block systematic explanation, but I would hold that we should not aim to predict the exact

sentences produced or comprehended by each individual patient with brain damage. Rather, we should seek to

understand the pattern of abilities that survive certain repeated patterns of brain damage - where both kinds of

patterns are abstracted from a variety of individual cases. At this level of approximation, we might test whether or

not the pattern of performance of a model degrades in a fashion consistent with experiment whether or not the base

of "normal performance" is the same for model and human being.

Another clue comes from stananaLL's citation of Marx and Poggio's views on the "near independence" of

different levels of analysis, or the notion of top-down design in computer programming (Alagie & Arbib, 1978). It

seems to be fruitful to describe subsystems in a language that is relatively independent of the details of

implementation - whether the latter be in terms of neural circuitry in the brain or the machine language of some


computer. Thus a neurolinguistic analysis at the regional level may, to a first approximation, be formalized in an

abstract language of cooperative computation. The interactions and internal state transitions posited in this model

can then be implemented within some computer programming language. But if the program is properly designed,

runs of the program test the high-level model, not the details introduced in the implementation. In the same way, a

simulation of a dynamic system described by Newton's laws may involve the choice of a Runge-Kutta method for

numerical integration of the differential equations, but the resultant trajectories should not depend on the numerical

analysis chosen, save perhaps in fine detail. If distinctions of this kind are rigidly maintained, there seems no need to

share HaewEs's fear that "the use of computational concepts as content in a theory implemented in a computer

program — [will raise) confusion to new heights."

In this context, we would agree with LOCKE's emphasis on the

[480]

importance of representation, rerepresentation, and rererepresentation, but would disagree with his suggestion

that in computational models the rerepresentation and translation iis) from magnetic core to binary system to

computer language to English." This particular sequence of translations exists in computer science and provides an

analogy that may help us think about the brain, but it is no part of a computational model of neurolinguistics per se.

Such a model is expressed in an abstract language of cooperative computation to which the details of electronic

implementation are irrelevant; the translations explicitly represent those we posit to occur in the brain.

Having stressed that many implementation details are hidden in a high-level model, I must still admit that the

formulation of the model will force one to go beyond the experimental data. HALWES seems to feel that this is a

hurdle-for implementation. It is, but it is also a hurdle for experimentalists, since any experiment is designed within

a context of accepted data and theory. The experimentalist can help the modeller by being more confident of her or

his facts; the modeller may help the experimentalist by designing flexible models that can be run on different

occasions with different assumptions. If the system yields acceptable behavior under some hypotheses and not

others, this may provide a valuable stimulus to, and focussing of, experiment.

In summary, I would argue that our target article and the attendant commentaries justify the following

conclusions about neurolinguistics:

There is a body of moderately reliable information relating symptom-complexes to localized lesions, but much

needs to be done to relate symptom-complexes to the interaction of remaining brain regions rather than to properties

of the site of the lesion. It is an article of faith shared by most neurolinguists that such an analysis is in principle

possible.

There is a body of psycholinguistic research that seeks to refine linguistic categories to provide clues to the -

neural code- of language processing. The neural validity of many of the posited codes is still controversial.

A new framework is needed to develop, modify, and integrate the approaches outlined in 1. and 2. We have

expressed the hope that precise models using the language of cooperative computation (based on studies in both

brain theory and artificial intelligence) may provide such a framework Our proposal is viable to the extent that

further work is justified, but the hypothesize-and-test style is controversial and needs full experimental testing on the

basis of detailed modelling.


9. Computational models are abstract models and must not be confused with crude comparisons of brain and

computer. Much can be learned from computational models by pencil and paper theorizing, but computer

simulation should allow more detailed study of their properties.

5. Neurolinguistics has been too isolated from general issues of neural modelling, and from an appreciation of the

relevance of issues in visual perception and motor control. We argue that the cooperative computational style of

modelling can integrate neurolinguistics with studies of visuomotor coordination, and that the "modules" positedin a

Cr-model can provide a bridge to synapse-cell-circuit neuroscience.

REFERENCES

AdagiC, S.. and Arbib, M. A. (1978) The Design of Well-Structured and Correct Programs. Berlin: Springer-Verlag

[MAA)

Amari, S., and Arbib, M. A. (197'7) Competition and cooperation in neural nets.. In Systems neuroscience, ed., J.

Metzler. pp. 119-165_ New York: Academic Press. [MAA]

Anokhin. P. K. (1935) Problems of centre and periphery in the physiology of nervous activity. Gorki: cosizdat (in

Russian),

Arbib, M. A. (1966) Distributed information processor—a computer and brain model_ Proceedings of the 19th

Annual Conference on Engineering in Medicine & Biology, p. 31 (abstract only). [MAA]

(1970) On modelling the nervous system. Principles and practice of bionics, ed. H. E. von Gierke, W. D. Keidel,

and H. L Oestreicher, pp. 45-57.

Slough, England, Techuivision Book Services. [MAA]

(1972) The metaphorical brain. New York: Wiley Interscience.

[MAAJ

(1975) Artificial intelligence and brain theory: unities and diversities. Annals of Biomedical Engineering 3:238-274.

[MAA]

(1977) Parallelism, slides, schemas, and frames. In Systems: Approaches, theories, applications ed. W. E. Hartnett,

pp. 27-43. Dordrecht, Holland: D. Reidel Publishing Company. [MAA]

(1980) Perceptual structures and distributed motor control. In Motor Control, ed. V. B. Brooks. Vol. III, Section on

Neurophyslology, Handbook of Physiology. Bethesda, Md.: American Physiological Society. (MAA)

Aronoff, M. (1976) Word formation in generative grammar. Cambridge, Mass.: MIT Press. [DTL]

Barlow. H. B. (1972) Single units and sensation: A neuron doctrine for perceptual psychology? Perception 1:371-

394. LICK

(197,.,) Inductive inference, coding, perception, and language. Perception 3:123-134. [JCM)

Bartlett, F. C. (1932) Remembering; a study in experimental and social psychology. Cambridge, England:

Cambridge University Press. [MAA]

Bastian, H. C. (1869) On the various forms of loss of speech in cerebral disease. British and Foreign Medical-

Chirurgical Review 43:209-236, 470-492. [MAA]

(1897) On some problems in connexion with aphasia and other speech defects. (The Lurnleian Lectures.) Lancet

1:933-942, 1005-1017, 11311137. [MAA)

Bates, K (1976) Language in contest. New York: Academic Press. [HGa]


Bernstein, N. A. (1935) The relationship between coordination and localization. Arkhiv. Biol. Nauk 38 (in Russian).

[MA Al

(196'i The coordination and regulation of movements. Oxford: Pergamon Press. IMAM

Blurnstein, S., Statlender, S., Goodglass. M., and Biber, C. (1979) Comprehension strategies determining reference

in aphasia: a study of reOexivizat n. In preparation. [SEB]

Bradley, D. C. (1978) Computational distinctions of vocabulary type. Ph.D.

dissertation, Massachusetts institute of Technology, Cambridge, Mass. [MAA]

Bradley, D. C., Garrett, M. F., and Zurif, E. R. (1979) Syntactic deficits in Broca's aphasia. In Biological studies of

mental processes, ed. D. Caplan. Cambridge, Mass.: MIT Press. (MAA]

Bresnan, J. (1978) A realistic transformational grammar. In Linguistic theory and psychological reality, ed. M.

Halle, J. Bresnan, and G. A. Miller p 159. Cambridge, Mass.: MIT Press. [MAA, BR]

Broadbent, W. 1-T (1872) On the cerebral mechanism of speech and thought. Medical and Chirurgical Transactions

55:145-194. [MAP.]

Broca, P. (1861a) Rernarques sur le siege de la faculte du langage articule. suivies d'une observation craphemie.

Bulletin de Is. Societe A natomique 6:330-357. [MAA]

(1861b} Nouvelle observation d'aphemie produite par une lesion de Is amine posterieure des deuxiime et troisieme

circonvolutions frontales. Bulletin dr 1.4 Societe Amin:marque 6:398-407. [MAA]

(1865) Sur [a faculte du langage articule. Bulletin de la Societe d'Anihropologie 6:493-494. [MAA]

(1869) Sur le s4ge de is faculte du langage articule. Teti', Med. 74:2.54-256; 75:265-269. (MAA)

Brown, J. W. (1977) Mind, brain, and consciousness: The neuropsychology of cognition. New York: Academic

Press. [MAA[

(1979) Brain structure and language production: a dynamic view. In Biologi- cal studies of mental processes, ed. D.

Caplan, Cambridge, Mass .7 MIT

BrownPr,essJ., (MAA[

and Jaffe, J. (1975) Hypothesis on cerebral dominance. Neuropsy-

cl.alogia 13:107-110. [HGo)

Brown, R. (1973) A first language: The early stages. Cambridge: Harvard University Press. IFIG1

Buckingham, H. W. (1977) The conduction theory and neologistic jargon. Language and Speech 20:174-184. [JCM]

C,ararr_azza A and Berndt, R. S. (1978) Semantic and syntactic processes in aphasia a review of the literature.

Psychological Bulletin 85:898-918. [MAA]

Cararrazza, A., and Zurif, E. B. (1978) Comprehension of complex sentences in children and aphasics. In Language

acquisition and language breakdown ed. A. Caramazza and E. B. Zurif. Baltimore: Johns Hopkins University

Press. [MAA]

Chaim.* J. M. (1863) Sur uric nouvelle observation draphemie. Gazette ffebdomarlaire 10:473-474. (MAA]

Chorn.sky, N. (1965) Aspects of the theory of syntax. Cambridge, Mass.: MIT Press.

(1975) Reflection on language. New York: Pantheon. [HWB]

[481]

(1976) On the biological basis of language capacities. In The neuropsychology of language. ed. R. W. Rieber. New

York: Plenum Press.


rHWBI

Chomsky, N , and Halle, M. (1968) The sound pattern of English. New York: Harper and Row. [MAA]

Davie, L, Foldi, N. S., Gardner, H., and Zurif, E. 13. (1978) Repetition in the transcortical aphasias. Brain and

Language 6:226-238. [HGa]

Dejerine, J. (1892) Contribution a retude anatomo-pathologique et dinique des diff erentes varietes de cecite verbale.

Memoirs Societe Biologique 4:61-90. [MAA]

Dennis, M. (1979) Language acquisition in a single hemisphere: semantic organization. In Biological studies of

mental processes, ed. D. Caplan. Cambridge, Mass.: MIT Press, [MAA]

(Forthcoming) Language acquisition in a single hemisphere: syntactic strategies. Preprint. [MAA]

Dennis, M., and Kohn, B. (1975) Comprehension of syntax in infantile hemiplegics after cerebral hemidecortication:

left hemisphere superiority. Brain and Language 2:472-482. [MAA]

Desmedt, J. E., ed. (1977) Language and hemispheric specialization in Man: Cerebral event-related potentials.

Basel: S. Karger. [MAA]

Didday, R. L. (1970) The simulation and modelling of distributed informa-

tion processing in the frog visual system. Ph.D. dissertation, Stanford University, Stanford, Calif. [MAA]

(1976) A model of visuomotor mechanisms in the frog optic tectum. Mathematical Biosciences 30:169-180. [MAR]

Dreyfus, H. L. (1979) What computers can't do: The limits of Artificial Intelligence. New York: Harper and Row.

[1-1W1]

Eccles, J. C., Ito, M., and Szentagothai, J. (1967) The cerebellum as a neuronal machine. Berlin: Springer Verlag.

[MAA]

Eidelberg, E., and Stein, D. G. ,,I974) Functional recovery after lesions of the nervous system. Neuroscience:

Research Program Bulletin 12_

[MAR]

Erman, L., and Lesser, V. R. (1975) A multi-level organization for problem-solving using many, diverse,

cooperating sources of knowledge_ In Proceedings of the 4th International Joint Conference on Artificial

Intelligence (Tblisi, USSR), pp. 483-490. Cambridge, Mass.: The Al Lab, Massachusetts Institute of Technology.

[MAR)

(1980) The HEARSAY-II system: a tutorial. In Trends in speech recognition, ed. W. A. Lea. Englewood Cliffs, N.

J.: Prentice-Hall. [MAR]

Ewert, J. P. (1976) The visual system of the toad: behavioral and physiological studies on a pattern recognition

system. In The amphibian visual system, a multidisciplinary approach. ed. K. V. Fite, pp. 141-202. New York:

Academic Press. [MAA]

Fillmore. C. J. (1976) Frame semantics and the nature of language. In Origins and evolution of language. Annals of

the Neu York Academy of Science. [MAR)

Finger, S., (1978) Recovery from brain damage: Research and theory. New York: Plenum Press. [HWB]

Flechsig, P. (1901) Developmental (myelogenetic) localisation of the cerebral cortex in the human ,nbjeet Lancet

2:1027-1029. [SHC]

Fodor, J. A., Bever, T. G., and Garrett, M. F. (1974) The psychology of language: An introduction to

psycholinguistics and generative gramnwr. New York: McGraw-Hill. [MAA,HWB]

A]


Fodor, J. A. and Garrett, M. (1966) Some reflections on competence and performance. In Lyons, J. and Wales, R.

(eds.) Psycholinguistic papers. Edinburgh: Univ. of Edinburg Press. [HG]

Frazier, L (1978) On comprehending sentences: syntactic parsing strategies,

Ph.D. dissertation, University of Connecticut, Storrs, Corm. NAM Frazier, L., and Fodor, J. D. (1978) The sausage

machine: a new two-stage

parsing model. Cognition 6:291-325. [MAA]

Freud. S. (1891) On.aphasia. Reprinted in translation, 1953 New Southgate, England: Image. [MAA)

Garrett, M. G. (1976) Syntactic processes in sentence production. In New approaches to language mechanisms, ed.

R. J. Wales and E. Walker. Amsterdam: North Holland [MAA]

Canong, F. The internal structure of consonants in speech perception: Acoustic cues, not distinctive features,

Forthcoming. (THI

Gelfand, I. M., and Tsetlin, M. L. (1962) Some methods of control for complex systems. Russian Mathematical

Surveys 1715-117. [MAA]

Geschwind, N. (1962) The anatomy of acquired disorders of readmg In Reading disability ed. J. Money, Baltimore:

pp.115-128. The Johrs Hopkins University Press. (FIGa]

(1964) The paradoxical position of Kurt Goldstein in the history of aphasia. Cortex 1:214-224. [MAA]

(1965) Disconnexion syndromes in animal and man. Brain 88(23:237-294; (3):585-644. [MAR, HGa]

(1969) Problems in the anatomical understanding of aphasia. In Contribu How to clinical neurology, ed. A. L

Benton. Chicago: Aldine. [MAA. JCM]

(1967) Werticke's contribution to the study of aphasia. Cortex 3:449-463. [MAA]

(1970) The organization of language and the brain Science 170:940-944. [MAR]

(1972a) Language and the brain. Scientific American 22(4):76-83. [MAA]

(1972) A review of A. R. Luria's Traumatic Aphasia. Language 48:755763. [HWB]

(1974) Late changes in the nervous system: An overview. In Plasticity and recovery of function in the central

nervous system, ed. D. G. Stein, J. J. Rosen, and N. Butters. New York: Academic Press. [JCM]

(1979) Some comments on the neurology of language. In Biological studies of mental processes, ed. D. Caplan.

Cambridge, Mass.: MIT Press. [MAA]

Goldstein, K. (1948) Language and language disturbances. New York. Grupe and Stratton_ [MAA]

Goodenough, C., Zuni, E. B., and Weintraub, S. (1972) Aphasic's attention to grammatical morphemes. Language

and Speech 20:11-19. [MAA]

Goodglass, H., Blumstein, S. E., Gleason, J. B., Hyde, M. R., Green, E., and Statlender. S. (1979) The effect of

syntatic encoding on sentence comprehension in aphasia. Brain and Language. 7:201-209. (SERI

Goodglass, H., and Geschwind, N. (1976) Communication disorders (aphasia). In Handbook of perception, Vol. VII,

eel. E. C Carterette and M. Friedman. New York: Academic Press. [HGo]

Granit, R. (1970) The basis of motor control, New York: Academic Press. [MAA]

Grashey, H. (1885) Ueber aphasie und Ihre Bezeikungen zun Wohrnehmung. Archly. fur Psychiatrie 16:65.4-688.

[MAA]

Halle. M. (1962) Phonology in generative grammar. Word 18:54-72[DTL]


Hanson, A. R., and Riseman, E. M. (1978) Segmentation of natural scenes. In Computer Vision Systems, ed. A. R.

Hanson and E. M. Riseman, pp. 129163. New York: Academic Press. [MAR]

Hayes-Roth, F. and Lesser, V. R. (1977) Focus of attention in the HEARSAYiI Proceedings of OW 5th

International Conference on Artificial

Intelligence. pp. 27-35. Cambridge, Mass. (MAA)

Head. H. (1926) Aphasia and kindred disorders of speech. New York: Hafner. [MAA]

Householder, F W. (1959) On linguistic primes. Word 15:231-239 [DTL]

Hubei, D. H., and Wiesel, T. N. (1974) Sequence regularity and geometry of orientation columns in the monkey

striate cortex. Journal of Comparative Neurology 158:267-294. [MAR]

(1977) Functional architecture of macaque monkey cortex. Proceedings of the Royal Society of London 13198:1-59.

[MAR]

Hudson, P. H. (.I.976) Cerebral dominance and language: an experimen-

tal and theoretical investigation, Ph D. dissertation, University of St. Andrews, St. Andrews, Scotland. [MAA]

Ingle, D. (1976) Spatial vision in anurans. In The amphibian visual system, ed. K. V. Fite, pp. 119-141. New York:

Academic Press. (MAA]

Jackson, J. H. (1.374) On the nature of the duality of the brain. Reprinted in J. Taylor (ed.) 1931. Selected writings

of John Hughlings Jackson, I and II. London: Kidder and Stoughton. (MAA, HWB]

(1878, 1879) On affections of speech from disease of the brain. Brain L304330; 2,203-222, 323-356. [MAA]

Jakobson, R (1941) Kindersprache, Aphaste und Allgerneine Lautgesetze. Uppsala: Alatqvist and Wiksell. [PTWH]

John, E. R., and Schwartz, E. L (1978) The neurophysiology of information processing and cognition_ Annual

Review of Psychology 29-1-29 [JCM]

Johnson-Laird, F. N., and Wason, P. C. eds. (1977) Thinking: Readings in cognitive science. Cambridge:

Cambridge University Press. [HWB]

Kandel, E. R. (1978) A cell-biological approach to learning. Bethesda, Md.: The Society for Neuroscience. [MAA I

Katz, J. 1. (1972) Semantic theory. New York: Harper and Row. [DTL]

(1974) Effability and translation. In Meaning and translation, ed. F. Guenthner and M. Guenthner-Reutter. London:

Duckworth. [DTL]

Katz, J. J., and Postal, P. M. (1964) An integrated theory of linguistic descriptions. Cambridge, Mass.: MIT press_

[DTL]

Kean, M.-L. (1978) The linguistic interpretation of the aphasic syndromes. In Explorations in the biology of

language, ed. E. Walker, Montgomery, Vt.: Bradford Books. [MAA]

(1979b) Grammatical representations and the description of language processing. In biological studies of mental

processes, ed. D. Caplan. Cambridge, Mass: MIT Press. [MAA]

[482]

(1979a) Agrarnmatisin: A phonological deficit? Cognition 7:69-83. ]JCM)

Kehoe, W, J.. and Whitaker, ft A. (1973) Lexical structure disruption in aphasia: a case study. In Psycholinguistics

and aphasia, ed. H. Goodglass and S. Blutristein. Baltimore and London: Johns Hopkins University Press.

(MAAJ

Klatt, D. H. (1977) Review of the ARPA Speech Understanding Project. Jour-


nal of the Aron:Hoz/ Society of America 62:1345-1366. [PTWHJ Kolk, H. (1978) The linguistic interpretation of

Broca's aphasia. A reply to

M.-L. Kean. Cognition 6:353-361.

Kolk, H. (1974) Judgement of sentence structure in Broca's aphasia Neuropsychologia. In press. [SEBJ

Kussmaul. A. (1877) Die Storungen der Sprache. Zierrissera's Handbuch der speciellen Pathologie and Therapie

12:1-300. [MAAI

Lara, R., Arbib, M. A., and Cromarty, A. S. (1979) Neural modelling of prey-predator interactions in frog and toad

visuomotor coordination. Society for Neuroscience Abstracts IV. [MAA]

Lea. W. A.. and Shoup, J. E. (1979) Review of the ARPA SUR Project and Survey of Current Technology in Speech

Understanding. Los Angeles: Speech Communications Research Laboratory. [TI-1)

Lecours, A, R., and Caplan, D. (1975) Review of S. BlurrLstein. a phonological investigation of aphasic speech.

Brain and Language 2:237-

254. [MAA]

Lecours, A. R., and Deloche, C., and Lhermitte, F. (1973) Paraphasies phonemiques: description et simulation sur

ordinateur. Caltoques IR1A-Inf orrnatique Medicate 311-350. [MAA, HG)

Lecours, R., and Lhermitte, F. (1969) Phonemic paraphasias: linguistic

structures and tentative hypotheses. Cortex 5:193-228. IMAM

Leger, V. R., and Erman, L. D. (1979) An experiment in distributed interpretation. Report No. CMU-CS-79-120,

Computer Science Department, Carnegie-Mellon University, Pittsburgh, Penn. [MAA)

Lesser. V. R., Fennel, R. D., Erman, L. D., and Reddy, D. ft. (1975) Organization of the HEARSAY-II speech

understanding system. IEEE Transactions on Acoustics, Speech, and Signal Processing 23:11-23. (MAA]

Letivin, J. Y., Maturana, ET, McCulloch, W. S., and Pitts, W. H. (1959) What the frog's eye tells the frog's brain.

Proceedings of the Institute of Radio Engineers 1940-1951. [MAA]

Lichtheim, L (1885) On aphasia. Brain 7:431-484. (MAA, HWB]

Locke, S., Caplan, D., and Kellar, L. (1973) A study in neurofinguistics. Springfield, Ill.: Ch. Thomas Press. [MAP.)

Lordat, J. (1843) Analyse de la parole pour servir a la theorie de divers cas ahe et de paralalie. Reprinted in La

naissance de la neuropsychologie du langage, ed. H. Hecaen and J. Dubois. Paris: Flammarion, 1969. [MAA]

Luria, A. R. (1973) The working brain. Harmondsworth, England: Penguin. [MAA]

Marcus, M. (1976) A design for a parser for English. In Proceedings of the Association for Computing Machinery

Conference. Houston, Texas. [HWB]

Marie, P. (1906) La troisieme circonvolution frontale gauche ne joue aucun role special dans Is function du langage.

Semaine Mkdicale 26:241-247_ (MAAI

Marshall, J. C. (1979) On the biology of language acquisition. In Biological studies of mental processes, ed. D.

Caplan. Cambridge, Mass.: MIT Press. [MAA)

Marslen-Wilson, W. (1976) Linguistic descriptions and psychological assumptions in the study of sentence

perception. In New approaches to language mechanisms, ed. R. J. Wales and E. Walker, pp. 203-229.

Amsterdam: North-Holland. [MAA]

Mare D., and Poggio, T. (1976) From understanding computation to understanding neural circuitry. M.I.T. A.I.

Memo 357. [JCM]


Milliken, C. H., and Darley, F. eds. (1967) Brain mechanisms underlying

speech and language. New York: Gnsne and Stratton. [11W13]

Mounteastle, V. B. (1978) An organizing principle for cerebral function: the unit module and the distributed system.

In The mindful brain, ed. G. M. Edleman and V. B. Mountcastle. Cambridge, Mass.: MIT Press. [MAA]

Moutier, F. (1908) L'aphasie de Broca. (Travaile du laboratoire de M. le Professeur Pierre Marie.) Paris. [MAA]

Norman, D. A., and Bobrow, L. S. (1975) On data-limited and resource-limited processes. Cognitive Psychology 1.

[MA A]

Okhotirriskii. D. E., Gurfinkel V. S. Devyanin, E. A., and Platnnov, A. K. (1979) Integrated walking robot

development. In Machine Intelligence, Vol. 9, ed. D. Michie. Edinburgh: Edinburgh University Press. [MAA]

Ojemann, G., and Whitaker, H. A. (1978) Language localization and variability. Brain and Language 6:239-260.

[HGo)

parietal lobe in the rhesus monkey. Brain Research 15:49-65_ [SHG) Pellionisz, A., and Llinis, R.(1979) Brain

Modeling by Tensor Network The-

ory and Computer Simulation. The Cerebellum. Processor for

Predictive Coordination_ Neuroscience 4:323-348. EMI)

Penfield, W., and Roberts. L (1959) Speech and Brain mechanisms. Princeton. N. J., Princeton University Press.

[HWB, HGO)

Pick, A. (1913) Die Agrammatisclsen SprachstOrungen, Berlin: Springer Verlag. [MAA]

Phillips, C. C., and Porter, R. (1977) Corticospinal neurones: Their role in moemerit. New York: Academic Press,

[MA A]

Riese, W. (1959) A history of neurology. New York: MD Publications. [HWBJ

Rosenfeld, A., Hummel, R. A., and Zucker, S. W. (1976) Scene labelling by relaxation operations. IEEE

Transactions on Systems, Man and Cybernetics SNIC-6,420-433. [MAAJ

Sasanuma, S., and Fujimora, 0. (1971) Selective impairment of phonetic and nor.phonetic transcription of words

in Japanese aphasic patients: Kam versus Kanji in visual recognition and writing_ Cortes 7:1-18[MAA]

Saussure, F. de. (1915) Court' de linguistique genii.' ale. Paris: Payot. (DTL]

Schnitzer, M. L (1972) Generative phonology—evidence from aphasia. Pennsylvania State Monograph tt34.

Pennsylvania State University Press, University Park, Penn. (MAA)

Schnitzer, M. L. (1974) Aphasiological evidence for five linguistic hypotheses. Language 50:300-315. [M LS]

(19781 Toward a neurolinguistic theory of language. Brain and Language 6:342-361. [MLSI

Scholes, R. F. (1978) Syntactic and lexical components of sentence comprehension. In Language acquisition and

language breakdown, ed. A. Cara.:\ ma:_u_a and E. B. Zurif. Baltimore: Johns Hopkins University Press

[MAAI

Shortliffe, E. H. (1976) Computer-based medical consultations: MYCIN. Amsterdam and New York: Elsevier.

[MAAI

Siegel, I). (1979) Topics in English morphology. New York: Garland Press. [DTh]

Shallice, T. (1979) Neuropsychological research and the Fractionation of memory systems. In Perspectives in

memory research, ed. L.-G. Nilsson. Hillsdale: Erlbaum. [JCM]

Spinelli, D. N., and Jensen, F. E. (1979) Plasticity: the mirror of experience. Science 203,75-78_ [MAAJ


Stein, D. G., Rosen, J. J., and Butters, N., eds. (1974) Plasticity and recovery of function in the central nervous

system_ New York: Academic Press. [HWB)

Szentagothai, J. (1978) The neuron network of the cerebral cortex: A functional interpretation. Proceedings of the

Royal Society London B. 201219-248. [JCM]

Szentigathai, J., and Arbib, M. A., (1974) Conceptual models of neural organization. Neurosciences Research

Program Bulletin 12:307-510. (Also published as a book by MIT Press, 1975.) {MAAJ

Thorne, J. P., Bratley, R., and Dewar, H. (1968) The syntactic analysis of English by machine. In Machine

Intelligence, Vol. 3. ed. D. Michie, pp. 28i.-309. Edinburgh: Edinburgh University Press. [MAA]

Vygotsky, L S. (1962) Thought and language (translation). Cambridge, Mass.. MIT Press_ [MAA]

Waltz, D. L (1978) A parallel model for low-level vision. In Computer vision systems, ed. A. R. Hanson and E. M.

Riseman, pp. 175-186. New York: Academic Press. [MAA]

Wannee E., and Maratsos. M. (1978) An ATN approach to comprehension. In Linguistic theory and psychological

reality, ed, M. Halle, J. Bresnan, and G. A. Miller, pp. 119-161_ Cambridge, Mass: MIT Press. [MAA]

Wernicke, C. (1874) Die aphasische symptomen complex. Breslau_ [MAA]

(188,S,) Recent works on aphasia. Reprinted in Wernicke's works on aphasia, ed. G. Eggert. The Hague: Mouton.

1977 [HGR]

Whitaker, H. A., and Ojemann, G. A. (1977) Graded localization of naming from electrical stimulation mapping of

left cerebral cortex. Nature 270:50-51. [MAA1

Winograd, T (1972) Understanding natural language. New York: Academic Press. [MAA)

Wood, C. C. (1978) Variations on a theme of Lashley: Lesion experiments on the neural model of Anderson,

Silverstein, Ritz. and Jones. Psychological Review 85:582-591. NMI

Woods, W. (1970) Transition network grammars for natural language analysis. Conmunications of the Association

for Computing Machinery 13:59160i. [MAA, HWBJ

[483]

(1973) An experimental parsing system for transition network grammars. In Natural language processing, ed. R.

Bustin. Englewood Cliffs, N. J.: Prentice-Hall. [FMB]

yakovlev, P. I. (1948) Motility, behaviour and the brain. Journal of Neurological and Mental Disorders 107.313-

335. [MAA]

zangwill, 0. L. (1976) Aphasia and the concept of brain centers. In Psychology and biology of language and

thought: Essays in honor of Eric Lenneberg, ed. George A. Miller and Elizabeth Lenneberg. New York:

Academic Press. (1-1W13)

Zurif, E. B., and Caramazza, A. (1976) Psycholinguistic structures and aphasia: Studies in syntax and semantics. In'

Studies in neurolinguistics I, ed. H. Whitaker and H. Whitaker. New York: Academic Press. [MAA,

SEB]

Zurif, E. B., Caramazza, A., and Myerson, R. (1972) Grammatical judgements of aphasic patients.

Neuropsychologia 10405-417. [MAA, MU]

neurolinguistics must be computational

Documents