16

, t). "3 01 by drift (21.8% frequent drift, 1.3% rare drift.en ,- /0 .r rafting), and 4% by wmd. .His figures and mine are clo~e for overall dispersal. by

d b t they are reversed for wind dispersal and. dnft.r s, u .., th LandIthougb. floating vegeta,tion ongma'tmg on ~ mam _

'bee-n reported from the vicinity of the archIpelago'.' ",.IS 18" d'f h h d- even cast up on its beaches ' , n t as a a mm.or,Ie in plant co,lonisation of the Galapagos. Those specIeslat have a.rrived in t,his way are all members of the. man­-ove, beach or !>alt flat associations. I ca~ find no eVI.dence,r any vascular plant species having arnved by raftln?The small number of species that have been den~ed

trough oceanic drift is not surprisi.ng..Alth.ough dI'lf.t­spersed species have relat.ivdy hIgh. ImmIgratIOn .rates In

lmparison with those whloh are dlsper~ed by blrd~ andind, very few specoies are adapted .to t?lS n:ode of mtro­lotion. In addition, their h-igher ImmIgratIOn I'at~s ande small numbers of halbitats available to them ~n t~e

<lJapagos also reduce the opportuni.ties for ~ndemlsm l:nlis group". The role of man in th~ .mtroduc.to!on of exotic)ecies into the islands is surpr.lsmg (Table 1). Evenooker" rema.rked on the number of alien species thatan had added ,to the flora by the time of Darwin's visit1835. Man has introduced 124 weeds by the latest count,

1d 57 of his cultivated exotoics have escaped and now re­'oduce themselves in the wi'ld. Today Homo sapiens hasplaced birds ,as the most impo.rtarit factor in the .dis­mination of plants to the Gal·apagos. The deletenousfects on the native flora of some of the species introduoed

man are just beginning to be appreciated,,19·"-". .Recent potassium-argon studies" indicate that t~e archi­:Iago has ,a probable maximum age of 3 Myr. ThIS mea~s

at the arrival and establishment of one successful d-lS­minule about every 7,900 yr would account for the pre­nt indigenous flora. Introduction and subsequent extinc­)n cannot be estimated, but they certainly have occurredwell. This is a ,higher rate of introduction ~han Fosberg's'·

timate of one successful disseminule every 20-30,000 yraccount for the derivation of the -indigenous vascula'r

)ra of the Hawaiian Islands, but the latter are muchrther from their source areas than are the Galapagos.Such a .fe.la,tively high rate of introduc-tion is not surpris­g. These volcanic islands are a jumble of open pioneer.bitats inhibited by a weedy flora. As others have pointed.It, dispersal is only half the battle; establishment is theher half. In the Galapagos, those plants -that have becometablished are a'lmost all we1edy, a phenomenon first re­gnised by Darwin'. The chance of immigrants survivingany specific locality is greater initially than later when

ore cJo~ed communities have evolved. Weedy plants,'ing adapted to open habitats, have been at an advantagelen their disseminules reached the Galapagos Islands.'leir offspring have given us the present flora.e. S. Adkisson, S. Carlquist, P. R. Grant, F. James,

E. Linkins, R. Mayes, R. A. Paterson, S. H. Porter,A. West and 1. L Wiggins all made helpful comments

I an earlier version of this communication.

DUNCAN M. PORTER

:partment of Biology,rginia Polytechnic Institute and State University,'acksburg, Virginia 24061

eived September 20; accepted Novernber9, 1976.

)arwin, C., Narrative a/the Surveyin/? Voyages of His M.a/esty'.\· S.hips Adl:ent~reand Beagle, Between the Years 1826 and 1836. De.\Trtbl."g ~helr Exa.01/n.atIOnof the Southern Shores of South America. and the SeaRle s ClrcumnavI/?atlon 0/Ihe Globe, 3, Journal and Remurks, /832-1836 (Colburn, London, 1839).

venson', H. K., Am. J. Bal .• 22, 208-277 (1935).Viggins, r. L., and Porter, D. M., Flora of the Galapagos Islands (Stanford

University Press, Stanford, 197 I).liasson, U., But. NO/iser. 123,346-357 (1970).liasson, U., Bul. NOliser, 125, 320-322(1972).liasson, U., Op. bal. Soc. bol. Lund., 36, 1-117 (1974).chofield, E. K., Bioi. ConserI'., 5, 48-51 (1973).leber, D., Bufl. Soc. neuc/l6lel. Sci. not., 96,17-30 (1973).

-- ----,..----·------...r

Nature Vol. 264 December 23/30 1976

9 Hamann, 0., Bot. Notiser, 127, 245-251 (1974).10 Hamann, 0., Bol. Notiser., 127, 252-255 (I 974) .11 Hamann, 0., Bot. Notiser., 127,309-3 16 (1974).12 Wiggins, l. L., Madrono, 23, 62-64 (1975).13 Carlquist, S., Bull. Torrey bol. Club., 94, 129-162 (1967).14 Carlquist, S., Island Biology (Columbia University Press, New York, 1974).15 Hooker. J. D., Trans. Lilln. Soc., Land., 20, 235-262 (1847).16 Robinson, B. L., Proc. Am. Acad. Arts Sci., 38. 77-269 (1902).17 Stewart, A., Proc. Cali!. Acad. Sci., Ser. 4, 1,7-288 (191 I).18 Stewart. A., PI. Wid., 18, 192-200 (191 5) .19 Thornton, 1.. Darwin's Islands. A Natural History 0/ the Calapagos (Natural

History Press, New York, 1971).20 Harris, M., A Field to the Bird.! o!Galapagos (Collins, London. 1974).21 Agassiz, A., Bull. Mus. mmp. Zool. Harl'., 23, 1-89 (1892).22 Beebe, W., The Arcturus Advemure (Putnam. New York. 1926).23 Townsend, C. H., Bull. N. Y. zool. Soc., 31,148-168 (1928).24 Fitz-Roy, R., Narrative a/the Surveying Voyages of His Majesty's Ships Adl'etl1ure

Qnd Beagle, 2, Proceedings 0/ the Second Expedition, 1831-1836, Under theCommand o!Capiain Robert Fitz-Roy, R. N. (Colburn, London, 1839).

25 Johnson, M. P., and Raven, P. H., Sciellce, 179, 893-895 (1972).26 Eckhardt, R. C., BioScience, 22, 585-590(1972).27 Porter, D. M., BioSciellce, 23, 276 (I 973).28 Hamann, 0., Bioi. Conserv., 7,37-59 (1975).29 Bailey, K., Sciellce, 192,465-467 (1976).30 Fosberg, F. R., in Insects 0/ Hawaii, .1 (edit. by Zimmerman, E. C.) (University of

Hawaii Press, Honolulu, 1948).

Hearing lips and seeing voices

MOST verbal communication occurs in contexts where the listenercan see the speaker as well as hear him. However, speechperception is normally regarded as a purely auditory process. Thestudy reported here demonstrates a previously unrecognisedinfluence of vision upon speech perception. It stems from anobservation that, on being shown a film of a young woman'stalking head, in which repeated utterances of the syllable [ba] hadbeen dubbed on to lip ITIOVements for [gal, normal adults reportedhearing [da]. Wi th the reverse dubbing process, a majori ty reportedhearing [bagba] or [gaba]. When these subjects listened to thesoundtrack from the film, without visual input, or when theywatched untreated film, they reported the syllables accurately asrepetitions of [ba] or [gal. Subsequent replications confirm thereliability of these findings; they have important implications forthe understanding of speech perception.

To further confirm arid generalise the original observation, newmaterials were prepared. A woman was filmed while she fixated atelevision camera lens and repeated ba-ba, ga-ga, pa-pa or ka-ka.Each utterance was repeated once per second, with an interval ofapproximately 0.5 s between repetitions. From this master re­cording four dubbed video-records were prepared in which theoriginal vocalisations and lip movements were combined asfollows: (1) ba-voice/ga-lips; (2) ga-voicejba-lips; (3) pa-voice/ka­lips; (4) ka-voice/pa-lips. Dubbing was carried out so as to ensure,within ·the temporal constraints of telerecording equipment, thatthere was auditory-visual coincidence of the release of theconsonant in the first syllable of each utterance. Each recordingcomprised three replications of its auditory-visual composite.Four different counterbalanced sequences of recordings (1)-(4)were prepared, each with a ten-second gap of blank film betweensuccessive segments. The recordings were suitable for relay via a19-inch television monitor; audio-visual reproduction was ofgood quality.

Twenty-one pre-school children (3-4 y), 28 primary schoolchildren (7-8 yr) and 54 adults (18-40 yr) were tested. The adultsample was predominantly male; there were approximately equalnumbers of boys and girls in the younger samples. Subjects wereindividually tested under two conditions: (1) auditory-visual,where they were instructed to watch the film and repeat what theyheard the model saying, and (2) auditory only, where they facedaway from the screen and again had to repeat the model'sutterances. Every subject responded to all four recordings ((1)-(4)above) under each condition, each time in a different sequence;sequence of presentation was counterbalanced across subjects.

For the purpose of analysis, a correct response was defined as anaccurate repetition of the auditory component of each recording.Under the auditory-only condition accuracy was high, withaverages of 91, 97 and 99% for pre-school, ·school age andadult subjects respectively; errors were unsystematic. Under theauditory-visual condition, where subjects heard the originalsoundtrack, errors were substantial. For pre-school subjects

/-

Nature Vol. 264 December 23/30 1976 747

Table 1 Stimulus conditions and definition of response categories from auditory-visual condition

Stimuli Response CategoriesAuditory Visual

component component Auditory Visual Fused Combination Other

ba-ba ga-ga ba-ba ga-ga da-da

ga-ga ba-ba ga-ga ba-ba da-da gabgabagba dabdabaga gaglagaba etc.

pa-pa ka-ka pa-pa ka-ka ta-ta tapaptakaftaetc.

ka-ka pa-pa ka-ka pa-pa kapka katpakpa kafapaka kakpatkapa etc.

average error rate was 59%, for school children 52% and for adults92%.

Subsequent analysis was confined to detailed consideration ofresponses to the auditory-visual presentations. Responses werefirst categorised according to the operational definations illus­trated in Table I.

The meaning of'auditory' and 'visual' categories is self-evident.A 'fused' response is one where information from the twomodalities is transformed into something new with an element notpresented in either modality, whereas a 'combination' responserepresents a composite comprising relatively unmodified elementsfrom each modality. Responses which could not be un­ambiguously assigned to one of these four categories wereallocated to a small, heterogeneous 'other' category. Table 2presents the percentage of responses in each category.

Table 2 shows that the original observation of the effect of[ba]/[ga] presentations on adult responses is highly replicable; 98%of adult subjects gave fused responses to the ba-voice/ga-lipspresentation and 59% gave combination responses to its comple­ment. The effect is also generalisable, at least to other stopconsonants; 81 %of adults gave a fused response to pa-lips/ka­voice and 44% gave combination responses to its complement. Theeffects. however, are more pronounced with [ba]/[ga] than with[pa]/[ka] com binations; the latter comment applies to all ages.

The data in Table 2 also illustrate that the auditory perception ofadult subjects is more influenced by visual input than is that ofsubjects in the two younger groups; the latter do not differconsistently from each other. It is interesting to note that whereresponses are dominated by a single modality, this tends to be theauditory for children and the visual for adults. However, it shouldalso be noted that the frequency offused responses to ba-voice/ga­lips, and pa-voice/ka-lips presentations is at a substantial level for

pre-school and school children alike. These auditory-visualillusions, therefore, are observable across a wide age span,although there clearly are age-related changes in susceptibility tothem, particularly between middle childhood and adulthood.

Appropriate analyses confirm that the various effects reportedfor the auditory-visual condition are statistically significant.Alone, however, the data fail to testify to the powerful nature of theillusions. We ourselves have experienced these effects on manyhundreds of trials; they do not habituate over time, despiteobjective knowledge ofthe illusion involved. By merely closing theeyes, a previously heard [da] becomes [ba] only to revert to [da]when the eyes are open again.

Contemporary, auditory-based theories of speech perceptionare inadequate to accommodate these new observations; a role forvision (that is, perceived lip movements) in the perception ofspeech by normally hearing people is clearly iilustrated. Our ownobservations and those of others l indicate that, in the absence ofauditory input, lip movements for [gal are frequently misread as[da], while those for [ka] are sometimes misread as [tal; [pal and[ba] are often confused with each other but are never misread as[ga, da, ka or tal. It is also known that, in auditory terms, vowelscarry information for the consonants which immeduately preceedthem 2

. Ifwe speculate that the acoustic waveform for [ba] containsfeatures in common with that for [da] but not with [gal, then atentative explanation for one set of the above illusions is suggested.Thus, in a ba-voice/ga-lips presentation, there is visual infor­mation for [gal and [da] and auditory information with featurescommon to [da] and [bal. By responding to the common infor­mation in both modalities, a subject would arrive at the unifyingpercept [da]. Similar reasoning would account for the [tal responseunder pa-voice/ka-lips presentations.

By the same token, it could be argued that with ga-voice/ba-lips

Table 2 Percentage of responses in each category in the auditory visual condition

Stimuli ResponsesAuditory Visual Subjects Auditory Visual Fused Combination Other

ba-ba ga-ga 3-5 yr (/1=21) 19 0 81 0 07-8 yr (/1 = 28) 36 0 64 0 0

18-40 yr (/1 = 54) 2 0 98 0 0

ga-ga ba-ba 3-5 yr (/1=21) 57 10 0 19 147-8 yr (/1 = 28) 36 21 II 32 0

18-40 yr (/1 = 54) II 31 0 54 4

pa-pa ka-ka 3-5 yr (/1=21) 24 0 52 0 247-8 yr (/1=28) 50 0 50 0 0

18-40yr(/1=54) 6 7 81 0 6

ka-ka pa-pa 3-5 yr (/1=21) 62 9 0 5 247-8 yr (/1 = 28) 68 0 0 32 0

18-40 yr (n = 54) 13 37 0 44 6

748

and ka-\'oice,'pa-lips combinations the modalities are in conflict,having no shared features. In the absence of domination of onemodality by the other, the listener has no way of deciding betweenthe two sources of information and therefore oscillates betweenthem, variously hearing [babga], [papka], and so on.

The pOSI fi/c(() nature of this interpretation is acknowledged.More refined experimentation is required to clarify the nature andontogenetic development ofthe illusions and their generality needsfurther investigation. We are at present working on these issues butbelieve that the finding now reported are of some interest in theirown right.

Full details of the dubbing procedure are available from us. Wethank the staff of the University of Surrey AVA Unit for theirtechnical assistance in preparing materials and also Susan Ballan­tyne whose lip movements we filmed.

HARRY MCGURK

JOHN MACDoNALD

Deporlll1cl1loj'P.I'Jch%g)"Ul1ircrsit\· oj'Surre)"Gui/dford,Surrcy Gl..iJ 5XH, UK

Recei\"cd July 15: ~t(c('rlcd November II. 1976.

I Binnie. C. A.. \1olllgomery. A. A.. and Jackman, P. L. J. Speech H('(JI'illg Res.. 17,619-630( 1974).

2 Liberman. A. ~ .. Dcbtlre. P. C. ~md Cooper. F. S., Am. J. Psydl .. 65, 497-516 (1952).

Squeezing speech into the deaf earDEAFNESS caused by attenuation at the middle ear-conductiondeafness-can be ameliorated by hearing aids and surgicalprocedures, but damage to the cochlea presents more severeproblems'. We are altering the amplitudes of different partsof the speech wave relative to one another, so that the featuresnormally determining intelligibility are selectively amplified.Tests using speech modified in this way will show whethermajor aspects of speech processing are unchanged by sen­sorineural deafness, and, if so, whether the improvement islarge enough to justify the use of such processing in hearingaids.

The principal characteristics of sensorineural deafness due tocochlear damage are: (I) loss of sensit ivity to high frequencies;(2) tinnitus-whistling or hissing sounds; (3) recruitment­reduction of dynamic range for intensity, the threshold fordetection being raised while the maximum acceptable intensityis near normal. Both conduction and sensorineural deafnesscan be simulated in the normal ear 2 . 3 ; conduction deafnessby ear plugs and sensorineural deafness by adding mask ingwhite noise (that is, noise containing equal power in all audiblefrequencies). Linear amplification will compensa'e for thesignal attenuation of conduction deafness. It cannot, however,overcome loss of hearing associated with the limited dyna:llicrange of the damaged cochlea. Standard hearing aids appliedin the latter case can amplify the peak sound intensities aboveacceptable limits, overloading the middle ear to producedistortion, discomfort and sometimcs pain without givingadequate intelligibility to normal speech. They may alsoproduce further damage to the hair cells·· 5.

Fortunately, the energy peaks are not important as carriers ofspeech information". Most of the speech information is carriedby the zero-energy crossover points of the wa\'e. This impliesthat the form of the speech wave is not inherently important,and that liberties can te taken with the specch wave for ahearing aid. Gregory and Wallace 2 concluded that it should bepossible to prevent overloading while retaining the nccessaryinformation, by using suitable peak clipping in hearing aids.Amplilude clipping was, however, found (by ourselves andothers) to produce unpleasantly distorted sounds, and theFourier components generated by sharp clipping give maskingtones which reduced intelligibility. "Rounding" the flat tops

Na/lire Vol. ]64 December nj.w /976

of the clipped waves with a high cut filter gives some improvc­ment, but removes information since some of the generatedcomponents are in the speech frcquency band. Alternatives,such as frequency transposition, were also attempted but wereabandoned for various reasons". Amplitude compression (AGC)is standard practice for broadcasting and recording. It is onlysuitable for hearing aids if the "attack time" is made shortenough to limit individual energy peaks (of the order of hun­dreds of microseconds) and so minimise peak overloading.The "release time" must also be made short enough to preserveinformation of consonants following vowels without causingsound level nuctuations ("breathing")'.

Villchur used a 2-channel amplitude compressor with anattack time of "less than lms" and a release time of 2000 dB s -'.He found improvements of syllable recognition of from 22 ~:,

to 160/;, in quiet and 10":, to 230"',', in a "cocktail party"situat ion. "I mprovement was den ned as

100(score with processing-scOr~ Without)

score Without processlllg

An alternative and possibly complementary approach isinstantaneous amplitude clipping, but without the squaringand generation of cross modulation products in the speechband associated with peak clipping. This has for a long timeseemed impossible; but it can now be achieved.

The solution we have adopted is to generate the inevitablecross-modulation products outside the speech band. This isdone by amplitude modulating a high frequency carrier withthe speech, and clipping the carrier"". The cross-modulationproducts are now multiples of the chosen carrier frequency. Bychoosing a suitable carrier frequency (50 kHz-lOO kHz) theharmonics and cross-modulation products generated by theclipping are widely separated from the speech band, and so canbe removed. The result is instantaneous lim itat ion of theaudio peaks without flattening or appreciable intermodulationproduct distortion.

Considering first normal audio frequency peak clipping, anywave exceeding the clirping level will be flattened. For two

Fig. 1 Unprocessed and processed wave forms, from "beating"a pair of AF sine waves. a, The unprocessed wave. h, Processedby audio frequency peak clipping. c, Processed by the carrier

clipping system.