JOAN GAY SNODGRASS, ROCHELLE VOLVOVITZand ELEANOR RADIN WALFISH

New York University, University Heights, Bronx, N. Y. 10453

Recognition memory for words, pictures,and words + pictures

Using the method of signal detection theory (SDT), recognition memory wasmeasured in two groups of Ss, with one group seeing words alone and picturesalone and the other group seeing words alone and pictures + words. Both groupswere run for three sessions under each of three presentation probabilityconditions. Recognition memory was better for pictorial stimuli in each group,but there was no improvement in recognition memory for the combined cues ofword + picture over picture alone. Four measures of recognition memory,including two based on SDT and one based on high threshold theory, werehighly correlated.

The present study is concerned withmemory for two different kinds ofstimuli-words and pictures, and theircombination-in a recognition memoryparadigm. Two questions are posed: Ismemory for pictorial material betterthan for verbal material? How doesperformance for their combined cuescompare with performance for eachseparate cue?

There is a good deal of evidencethat, for a variety of tasks, pictorialimagery variables play an importantrole. For example, words rated higherin imagery are remembered better thanthose rated low in imagery (Paivio,1969), and recognition memory forpictures is better than that for words(Shepard, 1967).

One would also expect thatpresenting two cues in a recognitionmemory paradigm, such as thecombination of a word with itspictorial illustration, would improverecognition memory even more. Forexample, if the cues were storedindependently, one would expectpredictably better performance withtwo cues than with one. Furthermore,Tulving & Thomson (1971) haveshown that the presentation of twocues present during learning improvesrecognition memory over thepresentation of a single cue,supporting the view that recognitioninvolves retrieval operations.

SUBJECTSSixteen Ss ( eight males and eight

females), all undergraduates of NewYork University, served in theexperiment. Eight Ss served inGroup A, in which recognitionmemory for words was compared withrecognition memory for pictures, andeight Ss served in Group B, in whichrecognition memory for words wascompared with recognition memoryfor words + pictures. Half of the Ss ineach group were female.

APPARATUS AND PROCEDUREThe basic procedure was identical

for both groups of Ss. Using a

Psychon. Sci., 1972, Vol. 27 (6)

procedure first introduced by Egan(1958), a recognition memory taskwas carried out using the methodologyof signal detection theory (SDT). Sswere presented with an inspectionpack of 60 cards, exactly half of whichwere words alone and half eitherpictures alone (for Group A Ss) orpictures + words (for Group B Ss). Thepictures were drawings illustrating thewords, and they appeared with theirappropriate word in the word +picture condition. Ss in both groupsreceived the same words alone and thesame pictures. The only differencebetween stimuli experienced byGroup A and those experienced byGroup B was that the drawings wereaccompanied by the words theyillustrated for Group B and appearedalone for Group A. Of the words inthe word alone set, 91% were ratedAA or A in the Thorndike & Lorge(1944) counts, and 9% had countsranging from 48 to 8 per millionwords; words chosen to be illustratedfor the picture alone or the word +picture condition consisted of 86% AAor A words and 14% with frequenciesbetween 48 to 8 per million words. Sswere permitted to view the 60 itemsfor 2 min. After the inspection period,S was given a 2-min rest period, whichwas followed by the recognition testperiod, during which S was presentedwith a shuffled pack containing bothold and new items. The new itemsconsisted of words alone and picturesalone for Group A and of words aloneand words + pictures for Group B. S'stask was to separate the items intothose which were old and those whichwere new. He was permitted as muchtime as he wished during this part ofthe experiment.

Each S served in three sessions,during which he experienced adifferent set of words, pictures, orwords + pictures. Each S received eachof the three signal presentationprobabilities, 0.4, 0.5, and 0.6. Theorder of presentation probabilities and

the particular set associated with eachprobability were determined randomlyfor each S. The presentationprobability was varied by varying thenumber of new items introduced inthe test session with the 60 old items,Thus, 90, 60, or 40 new items wereadded during the recognition test forthe 0.4, 0.5, and 0.6 conditions,respectively. Each S was informed atthe beginning of the recognition testperiod of the presentation probabilitycondition in effect, and Ss weremotivated to earn a maximum numberof points by the offer of a prize of$3.00 to the S with the largest numberof points. A symmetric payoff matrixawarding 1 point for hits and correctrejections and imposing a penalty of 1point for false alarms and misses wasemployed throughout the experiment.The purpose of the presentationprobability manipulation was tomanipulate S's criterion so that anROC curve could be fit to the datapoints, and from it d' (the SDTmeasure of sensitivity, or, in thiscontext, memory strength) calculated.This is similar to the approach ofMurdock (1965), except that he usedrating scales rather than variation inpresentation probabilities.

RESULTSThe overall hit and false alarm rates

across Ss for both Group A andGroup B Ss are shown in Table 1. It isapparent that presentation probabilityhad a small and inconsistent effect onresponse bias; thus, it was not possibleto calculate SDT measures of d' fromROC curves. Instead, d' was calculatedfrom the combined hit and false alarmrates for each presentation probabilitycondition. In addition, three othermeasures of recognition memory werecalculated. These were the probabilityof a correct response, P(C); themeasure of true probability of hits,P*(H), derived from high thresholdtheory, given by:

P*(H) = P(H) - P(FA)1 - P(FA)

Table 1Hit and False Alarm Rates for Groups A

and B for Each PresentationProbability Condition

PresentationGroup A

Probability 0.4 0.5 0.6---Words H .688 .721 .771Alone FA .200 .225 .300

Pictures H .850 .812 .817Alone FA .069 .054 .050

Group B

Words H .692 .775 .708Alone FA .117 .188 .206

Words + H .800 .725 .833Pictures FA .042 .058 .031---._-----------

345

*Each measure is based on data combined across Ss.

Table 2Four Measures of Recognition Memory: d", Probability Correct, P*(H), and A' for

Each Group and Each Presentation Probability*

Group APresentationProbability 0.4 0.5 0.6

d' 1.33 1.34 1.27Words P(C) .744 .748 .736Alone P*(H) .610 .640 .673

A' .830 .832 .821

d' 2.52 2.49 2.55Pictures P(C) .890 .879 .884Alone P*(H) .839 .801 .807

A' .939 .934 .937

Group B

d' 1.69 1.64 1.37Words P(C) .788 .794 .751Alone P*(H) .651 .723 .632

A' .871 .870 .835

d' 2.57 2.17 2.83Words + P(C) .879 .834 .901Pictures P*(H) .791 .708 .828

A' .935 .907 .948

where P*(H) is the true probability ofcorrect detections, P(H) is theobserved hit probability, and P(FA) isthe observed false alarm probability.This measure is based on the highthreshold assumption that either anitem is learned or is not learned duringthe inspection phase; items which arelearned pass the recognition thresholdand are correctly recognized; for someproportion of trials on which thethreshold is not exceeded, S guessesand his guessing probability isderivable from his false alarm rate(Green & Swets, 1966). A thirdmeasure is a nonparametric measure ofrecognition memory, A' which is ameasure of the area of the ROC curveenclosed by a nonparametric ROCcurve passing through a single pointdefined by a pair of hit and false alarmrates (Pollack, 1970; Grier, 1971). A'is calculated according to thefollowing formula:

A' = 1/2

+ [P(H)-P(FA)][l +P(H)-P(FA)][4P(H)] [1- P(FA)]

These four measures are shown inTable 2, for each group and eachpresentation probability condition. Allfour measures are approximately thesame across the three presentationprobability conditions. Furthermore,there is good, although not perfect,agreement among the four measures.The Pearson product momentcorrelation coefficient across the 12conditions ranges from a low of +.934between P*(H) and A' to a high of+.997 between P(C) and A'. P(C)shows the highest average correlation(+.981) with the other three measures.

346

Table 3 presents averaged values ofthe four measures of recognitionmemory for each of the groups. It isapparent that pictorial stimuliproduced higher recognition memoryscores than words alone. A t testbetween the words alone and thepicture alone stimuli for Group A onthe measure P(C) was highlysignificant (t 7.31, df 23,p< .001), as was a ttest on P(C)between the words alone and thewords + picture stimuli for Group B (t- 4.18, df 23, p < .001).Furthermore, no improvement inperformance was obtained when boththe cues, picture and word, werepresented over that obtained with thepicture alone. There was no significantimprovement in recognition memoryin terms of differences in P(C) scoresbetween words alone and picturesalone stimuli for Group A andGroup B (t = 1.62, df = 46, n.s.). Thus,the presentation of two cues-theword and the picture-did not improverecognition performance over thepresentation of a picture alone.

DISCUSSIONThe finding that recognition

memory is better for pictorial than forverbal stimuli is not surprising. It hasbeen demonstrated repeatedly, bothbetween pictures and words andbetween high imagery and low imagerywords. Presumably, more associativecues may be attached to stimuli whichhave rich visual as well as verbalassociates, so that both recall andrecognition performance are improved.

What is surprising, however, is thatthe addition of verbal and visual cuesdid not result in superior recognitionperformance. Certainly, the results ofTulving & Thomson (1971) suggest

that two cues present both duringpresentation and test should enhancerecognition memory performance.

Even on statistical grounds, onewould expect an improvement ofperformance with two cues over one;the exact way in which thatimprovement is predicted dependsupon the particular theory ofrecognition memory one adopts. Usingthe concepts of SDT, one predicts thatif the two cues are independent, iftheir memory strengths are normallydistributed, and if the standarddeviations of the two distributions areassumed equal, the predicted d' fromthe addition of two signals (or thepresentation of two to-be-rememberedstimuli, in this case) is given by d' w + p

= oJ d' w 2 +d' p 2, where w = wordsalone, p = pictures alone, and w+p =words + pictures (Green & Swets,1966). This formula predicts, ofcourse, that the d' measure for bothshould be higher than for either alone;in particular, using the data fromGroup A, it predicts that d' for bothshould be 2.84; using the d' fromGroup B for words and that fromGroup A for pictures, it predicts a d'of 2.97.

A high threshold theory of memory,and one which predicts that the twocues are independent and that the Sresponds positively if either or bothcues exceed threshold, predicts thefollowing for P*(H) for w+p:

P*(Hw + p )

= P*(Hw ) + P*(H p ) - P*(Hw ) P*(H p ) .

This theory predicts that P*(H w + p)for Group B should be .934 whenextrapolated from Group A.

In fact, any model which predictsthat both cues are independent andthat the S responds to theircombination fails to fit the presentdata; one possible account of thepresent data is that S stored only thepictorial cue during the words +picture stimuli, and hence responded

Table 3Average Values for Each Measure of Recog­nition Memory Across Presentation Prob­

ability Conditions for Each Group

Words PicturesAlone Alone

d' 1.31 2.52

Group AP(C) .743 .884P*(H) .641 .816A' .828 .927

Words Words +Alone Pictures

d' 1.57 2.52

Group BP(C) .778 .871P*(H) .669 .776A' .859 .930

Psychon. Sci., 1972, Vol. 27 (6)

to only that cue during the recognitionphase.

REFERENCESEGAN. J. P. Recoenition memory and the

operating characteristic. IndianaUniversity. Hearing and CommunicationLaboratory, AFCRC TN 58 51,AD-152650, June 15, 1958.

GREEN. D. :vi., & SWETS. J. A. Signaldetection theory and psychophysics. NewYork: Wiley, 1966.

GRIER, ,J. B. Nonparametric indexes lor

Psychon. Sci., 1972, Vol. 27 (6)

sensitivity arid bias: Computing fo rm ulas.Psychological Bulletin. 1971, 75,424-429.

:\lURDOCK, B. B. Signal-detection theoryand short-term memory. Journal ofExperimental Psychology, 1965, 70.443-447.

PAIVIO, A. Mental imagery in associativelearning and memory. PsychologicalReview, 1969, 76, 241-261.

POLLACK, I. A nonparametric procedurefor evaluation of true and false positives.Behavior Research Methods &Instrumentation. 1970, 2, 155-156.

SHEP ARD. R. N. Recognition memory forwords. sentences and pictures. JournaJ ofVerbal Learning & Verbal Behavior,1967,6,156-163.

THORNDIKE, E. L., & LORGE. E. Theteacher's word booh of 30,000 words.New York: Bureau of Publications,Teachers College, Columbia University,1944.

TULVING, E., & THOMSON, D. M.li.etrieval processes in recognitionmemory: Effect of associative context.Journal of Experimental Psychology.1971,87,116-124.

347