9 0 047

AUDITORY PATTERN MEMORY

Mechanisms of Tonal Sequence Discrimination by Human Observers

0Ln

qto Robert D. SorkinDepartment of Psychological Sciences

Purdue UniversityWest Lafayette, Indiana 47907

Current address: Department of Psychology,University of Florida, Gainesville, FL. 32611

30 October 1988

Final Technical Report for Period1 September 1987 - 31 August 1988 D T IC

SELECTESFEB 0819 8

CkH

Prepared for

AIR FORCE OFFICE OF SCIENTIFIC RESEARCHBolling Air Force Base, DC 20332-6448

Approved f--89 2 8 028

Mr... . idn Uai t ed umm m m

~'~~~T' CLASSIFICATION OF THIS PAGE

REPORT DOCUMENTATION PAGE Aoved

I. REPORT SECURITY CLASSIFICATION lb. RESTRICTIVE MARKINGS

2a. SECURITY CLASSIFICATION AUTHORITY 3. DISTRIBUTION IAVAILABIUTY OF REPORT

2b. DEC.ASSIFICATION1OWNGRADING SO4EDULE '. -

4. PERFORMING ORGANIZATION REPORT NUMBER(S) S. MONITORING ORGANIZATION REPORT NUMBER(S)

_AVOSR-*TM- 8 9- 0 047Ga. NAME OF PERFORMING ORGANIZATION 6b. OFFICE SYMBOL 7a. NAME OF MONITORING ORGANIZATION

Purdue University AFOSR/NL

.ADDRESS (City, State, and ZIPCode) 7b. ADDRESS (hy, State. nd ZIP Code)Building 410

West Lafayette, IN 47907 Bolling AFB DC 20332-6448

Sa. NAME OF FUNDING/SPONSORING 8ib. OFFICE SYMBOL 9. PROCUREMENT INSTRUMENT IDENTIFICATION NUMBERORGANIZATION apphbk) AFOSR-87-0349

.ADDRESS (Chy, State, and ZIP Code) 10. SOURCE OF FUNDING NUMBERSPROGRAM PROJECT TASK WORK UNITELEMENT NO. NO. NO. ACCESSION NO.

11. TITLE (Andul bcrity Ctasaffcation)

Auditory Pattern Memory12. PERSONAL AUTHOR(S)

Sorkin, Robert D.134. TYPE OF REPORT 13b. TIME COVERED 14. DATE OF REPORT (Year, Mont Day) IS. PAGE COUNT

Final I FROM 87-9-1 TO 88-8-31 88-10-3016. SUPPLEMENTARY NOTATION

1T. COSATI CODES 18. SUBJECT TERMS (Continue ot reverns if necessm y and identtfy by block number)FIELD GROUP SUGROUP Auditory perception, auditory sequence discrimina-

tion auditory patterns, auditory memory, temporaluncertainty,models of auditory siqnal processinQ.

19. ABSTRACT (Continue on rfvers at neceu ideifity by block number) ,

A two-process model of pattrn discrimination was developed to describehow tonal sequences are processed, stored, and discriminated by humanobservers. The model was evaluated in tasks in which observers wererequired to discriminate between th spectral or temporal patterns encodedin two sequences of tcnes. The expekmental results supported theassumptions of a trace/context coding 'eory. The trace mechanism isrelatively insensitive to temporal tran ormations made to frequency-codedpatterns but relatively sensitive to temp ral transformations made totemporally coded patterns. The effects of intervening maskers on thetrace were also evaluated. t< L-. ordJ

20. DISTRIBUTION / AVAILABIUTY OF ABSTRACT 21. ABSTRACT SKURITY CLASSIFICATIOND' UNCI.ASSIFIEDUNUMITED 0 SAME AS RPT'. , O. TIC USERS

22. ?*ME OF WPISIBLE INDIVIDUAL 22b. TEE PHONE (hiclude Area Code) IZc. OFFICE SYMBOL

DO Form 1473, JUN 86- P'eviousetoare ob kt. , SE I Y GASS-ICATION 9 TKIS-VAGE

i

'

SUMMARY

A two-process model of pattern discrimination was developedto describe how tonal sequences are processed, stored, anddiscriminated by the human auditory system. In this model, thecomparison of auditory stimuli involves processsing in twoseparate concurrent modes, with the mode contributing the leastvariance having the dominant effect on performance. In the'sensory trace' mode, the subject compares a trace of theauditory sensation to the corresponding trace of a subsequentstimulus. The trace is assumed to decay with time, but beindependent of the number and range of possible stimuli. In the'context coding' mode, stimuli are discriminated by comparingencoded representations, each representation consisting of acomparison between the given stimulus and the overall context ofother stimuli heard. The quality of the encoded representation isassumed to depend on the size of the auditory context.

This model was applied to tasks in which a subject comparedtwo sequences of tones and judged whether the frequencypatterns were the same or different. Several aspects of thesequences were varied: (a) the temporal variability of thesequences, (b) the correlaton between the temporal envelopes ofeach pair of sequences presented on a trial, and (3) the timeinterval between the two sequences. In the 'correlated'condition, the two sequences on a trial had identical temporalpatterns, while in the 'uncorrelated' condition, all sequencetemporal patterns were generated independently. In order tostudy the properties of the trace mechanism, various temporal andspectral transformations were made to the second sequence of eachpair. In addition, maskers consisting of short tonal sequenceswere interposed between the first and second sequence on eachtrial.

Performance in the correlated conditions was independent oftemporal variability, but decreased with increases in the timeinterval between the sequences. Performance in the uncorrelatedcondition decreased with increased temporal variability, and wasindependent of the length of the intersequence interval. Theseresults support the assumptions of the trace/context theory; goodfits were obtained between data from human observers and thepredictions of the two-process model. Performance in thecorrelated condition was insensitive to temporal transformationsmade to the second sequence, suggesting either (a) that tracescan be temporally scaled for comparison with other stimuli, or(b) that sensory traces can be generated for separate aspects ofa stimulus. Performance under different frequencytransformations produced a pattern of results consistent with(b). The effects of interfering events occurring during the timeinterval between the sequences were not consistent withconventional assumptions about interruption or masking of thetrace mechanism.

A series of experiments on temporal pattern discriminationwas begun in which the observer had to discriminate between twotonal sequences having the same or different temporal patterns.

ii

These tasks were evaluated under different levels of patterndifference and average duration, and under different amounts oftemporal compression and expansion. Performance was found to bevery sensitive to the temporal transformation imposed on thestimuli. Models of temporal pattern discrimination are currentlybeing evaluated in a revised task paradigm.

Some general assumptions of the signal detection model wereevaluated in experiments on the detection and recognition ofmultiple element auditory and visual displays. A major interestin these experiments was to devicribe how the observer aggregatesinformation over the elements of the sequentially orsimultaneously presented elements of the display. The resultswere consistent with expectations: (a) in sequential auditorydisplays, first and last arriving elements are emphasized, (b)with visual displays, elements in the fixated region areemphasized (the effective size of the region depends on theelement coding and format), (c) with both auditory and visualdisplays, high information-carrying elements are emphasized, and(d) with the visual displays thus far evaluated, the basicrelationships predicted by the recognition-detection model aresupported.

Accession For

NS GRA&I

0EC.4o

iii i Avallability Codes

-and/ .. or

TABLE OF CONTENTSPage

1. INTRODUCTION AND PURPOSE 1

1.1 Basic Experimental Paradigm 1

2. TWO-PROCESS PATTERN DISCRIMINATION THEORY 3

2.1 Empirical Tests of the Two-process Model 5

2.2 Experiments on the Trace Mechanism 8

2.2.1 Duration Scaling 8

2.2.2 Frequency Transformation 12

2.2.3 Masking 12

2.3 Temporal Pattern Discrimination 15

3. DETECTION AND RECOGNITION OF MULTIPLE ELEMENT DISPLAYS 16

4. REFERENCES 20

5. PROJECT PERSONNEL 22

6. ADVANCED DEGREES AND DISSERTATIONS 22

7. PUBLICATIONS AND MANUSCRIPTS 22

8. INTERACTIONS 23

iv

j ,

LIST OF FIGURESFigure Page1 Examples of pairs of sequences present on trials

of the experimental task. 2

2 Average performance (plotted points) as a functionof the gap standard deviation and condition (C,S,U)(8 tones, mean gap - 100 as). Solid lines aretheoretical functions. 6

3 Average performance (plotted points) as a functionof the mean gap duration and condition (12 tones,gap standard deviation-40 mas, ISI-0.5 s). Solidlines are theoretical functions based on parameterfits to data obtained with 8 tone sequences at amean gap of 100 ms and a range of gap variabilities. 6

4 Average performance (plotted points) as a functionof intersequence interval (ISI); same subjects andconditions as figure 2 (8 tones, mean gap-50ms, gapstandard deviation-30 is). Solid lines aretheoretical functions based on parameter fits tothe data of figure 2. 7

5 Examples of pairs of tonal sequences in the durationtransformation experiment. 10

6 Average performance as a function of intersequenceinterval, gap standard deviation, and temporaltransformation, for the correlated condition. 11

7 Average performance as a function of intersequenceinterval, gap standard deviation, and temporaltransformation, for the uncorrelated condition. 11

8 Average performance as a function of intersequenceinterval with and without intervening masker (gapstandard deviation-40 us). 14

9 Average performance in the temporal discriminationtask as a function of the standard deviation of thedifference jitter. 17

10 Average performance in the temporal discriminationtask as a function of the transformationfactor. The top curve is the condition when bothsequences undergo the same transformation; theabsciasca is proportional to average duration.The lower curve is the condition when only thesecond sequence is transformed. 18

11. Average performance in the temporal discriminationtask as a function of the transformationfactor. The data in the lower curve of figure 10have been corrected for the effect of duration. 19

v

1.!

1. INTRODUCTION AND PURPOSE

The goal of this research is to understand how humanlisteners process and store brief sequences of sounds.Specifying how humans encode and remember such stimulus patternsis an important goal for a theory of auditory informationprocessing, and is a necessary requirement for understandingcomplex perceptual processes such as speech. We have developed ageneral paradigm for studying the memory processes involved inthe perception of auditory patterns, using tasks in which thetemporal and spectral properties of tonal sequences aresystematically manipulated.

1.1 Basic Experimental Paradigm

When a subject must judge the similarity of the frequencypatterns defined by two sequences of tones, variation in thetiming of the tones can have a deleterious effect on performance.This effect is most evident when the tonal sequences have uniquetemporal patterns. Under such conditions, it may be difficultfor the subject to tell whether or not the frequency patterns arethe same or different. Our experiments have demonstrated thatvariability in the temporal aspects of a tonal sequence has alarge effect on a subject's ability to make discriminations basedon the frquency pattern of the sequence. We have employed thissequence discrimination task to evaluate a model of human patterndiscrimination, and have studied some properties of the memoryprocess and candidate mechanisms of the sequence comparisonprocess.

Many investigators have noted that varying the parameters of.a tonal sequence can have a large effect on performance in adiscrimination or detection task. For example, Watson and hiscolleagues have reported a number of studies in which a subjectmust detect small differences between two tonal sequences whoseparameters vary across the sequence of experimental trials. Thistype of trial-by-trial variation can have an effect on thedetectability of a small change in the frequency, duration, orintensity of one component of the tonal sequence (Watson, Kelly& Wroton, 1976; Watson & Kelly, 1981; Spiegel, & Watson, 1981;and Leek & Watson, 1984). Watson (1987) has pointed out thatthese effects can be much larger than those obtained from singletone (2IFC) discrimination tasks in which the tone parametersvary over trials.

We reported similar effects in an experiment testing thediscrimination of temporal jitter in tonal sequences (Sorkin,Boggs & Brady, 1982), and in more recent frequency patterndiscrimination experiments in which the temporal pattern wasvaried (Sorkin, 1987b). The subject's task in the recentexperiments was a Same/Different task: subjects were presentedwith a pair of tonal sequences and had to determine if the twosequences had the same or different frequency patterns.Subjects were to ignore variations in the temporal structure ofthe sequences, such as jitter in the tone onsets, durations, orinter-tone gaps.

1 #

Figure 1 illustrates some of the tonal sequences possibleon trials in our sequence experiments. In condition, C, thecorrelated temporal condition, the pattern of tone onsets,durations, and inter-tone gaps was identical in the two sequencesof each trial. For this particular pair of sequences thefrequency pattern is different in the two sequences, so that thecorrect response is "different" on this trial. The second pairof sequences illustrates condition U, the uncorrelated condition.Here the tone onsets, durations, and gaps were uncorrelatedbetween the two sequences. The frequency pattern is identical onthe trial shown, hence the correct response is "same". Thetemporal variability of the sequences was the major experimentalvariable in this paradigm: in the correlated condition, thetemporal variation was only present across trials; in theuncorrelated condition, the variation was present both within andacross trials.

1 2 3 4 12 34

1 24 2 13 34

1 23 4 12 34

1 234 1 2 3 4

1 12 .4

S 1 23 4 12 3.4

Figure 1. Examples of pairs of sequences present on trialsof the experimental task.

2

m d Jdm m n mm~d~ m mmml nllBI,

There are other ways that the temporal properties of thesequences could be varied. For example, the last pair ofsequences illustrates the synchronized (S) condition. In thiscondition the tone onsets were identical in the two sequences,but the durations and gaps varied. In this particular pair ofsequences the frequency pattern is identical, so the correctresponse is usame". In another condition (figure 5) alldurations and gaps in the first sequence of each pair were scaledupward by a constant factor, so that the new sequence was anexpanded version of the first. In all tasks, the subjects weretold to base their response only on the tonal pattern of eachsequence.

2. TWO-PROCESS PATTERN DISCRIMINATION THEORY

In a standard type of signal detection task, detectability,d', is proportional to signal power, I, and to the square root ofthe signal duration, t. It is inversely proportional to thesquare root of the sum of two variances: The external noisepower density, Un2 , and any internal noise or sensory variancers2. Equation 1 illustrates this relationship.

d' - kI t/[e-n2+ s] (1)

The detectability is also a function of various memory parametersthat are specific to the particular task. For example, a single-interval yes-no detection task clearly has different memoryrequirements than does a two-interval intensity discriminationtask.

Tanner (1961) proposed a general framework for categorizing _the memory requirements of a variety of single and two-intervalpsychophysical tasks; this scheme has been extended andquantified by a number of workers including Sorkin (1962), andMacmillan, Kaplan, and Creelman (1977) in a study of categoricalperception. Tanner's model included a short-term decaying memoryfor the acoustic input to the system plus various interferencefactors and long-term memory factors. The trace decay componentof the Tanner model was extended in a paper by Kinchla & Smyzer(1967) and later incorporated as the trace component of thetrace-context model of Durlach and Braida (1969) and theircolleagues (Berliner and Durlach, 1973). In Sorkin (1984), weproposed that the Durlach and Braida dual mode model could beextended to the sequence discrimination task.

According to the Durlach-Braida model, a subject can employtwo different processing modes in a discrimination task: a tracemode and a context mode. In the trace mode, the comparisonoperation is performed on internal memory tracis of the inputsignals. The trace is a direct representation of the acousticinput, e.g. a precategorical replica, and it deteriorates orpicks up noise over time. The longer the time interval betweentwo inputs to be discriminated, the more use of the trace modewill lead to degraded performance. In the context mode, eachinput is first categorized into one of a defined set of codes.The comparison operation is then performed on these encodedrepresentations. Once encoded, the internal data are not subject

3

to degradation over time. However, there is a context variancepresent which is a function of the difficulty of the encodingprocess. Task variables such the stimulus range are assumed toaffect this context noise. Performance in a given task willusually involve trade-offs in the internal noise associated witheach mode of operation.

These considerations are incorporated into the Durlach andBraida models in the following manner:

2 -2 -2 -1 1/2d'- c /[ + ( I + f ) 3 (2)

1 s c t

where cI is a constant in our tasks incorporating factors such asthe detectability and discriminability of the elementary tones(in a minimum uncertainty task), and,

2 2 2,t , and I are variance components added to the

s c t

discrimination process as a consequence of the sensory, context,and trace aspects of the task, respectively. It can be seen froman examination of equation 2 that the trace and context varianceswill operate together in determining which process, or whethereither process, will have a large effect on perfomance. Forexample, if the variance associated with one process is muchsmaller than the variance associated with the other process, thesmaller process will have the dominant effect.

The sensory variance refers to the noise from internal andexternal sources other than those associated with the encodingand storing of the sequence information. In the sequencediscrimination paradigm the sensory variance is assumed to besmall relative to the context and trace variances. The contextvariance is assumed to be a function of the number of differentsequence patterns that may be present on a trial. The tracevariance is assumed to be a function of the time period overwhich the information from the first sequence, e.g. the tracedatum, must be held in order to make a comparison with theinformation from the second sequence.

Consider the discrimination model applied to the conditionsin the sequence comparison tasks of the present study. Supposethat the two sequences of a trial have identical temporalenvelopes, as in the correlated condition of figure 1. A directcomparison of the traces of the two sequences will be a goodstrategy for discriminating any difference in the frequencypatterns, provided that the time period between the sequences isnot excessive. In the uncorrelated condition, when the two inputsequences do not share identical temporal patterns, the use ofthe trace mode may not be effective. This is because there arenow differences between the traces of the two inputs that are notrelevant to the frequency discrimination task. These irrelevantdifferences are associated with the decorrelation of the temporalenvelopes of the two sequences. We would expect these

4

'

differences to result in added variance in the trace comparisonprocess. This reasoning led us to define the trace variance asfollows:

2 2- c [ISI + nd + (n-l)g] + c f( L , g, ... ) (3)

t 2 3 g

where c2 and c3 are constants, ISI is the inter-sequence-interval, n is the number of tones in each sequence, d is theaverage duration of each tone, g is the average duration of thegaps between the tonesf a is the gap standard deviation, andf( ) is a function describins the added variance due to thedecorrelation of the temporal envelopes of the sequences to bediscriminated. This function incorporates the effect ofirrelevant (i.e. not frequency-contour) differences between thesequences on a trial. The effects of such differences will beminimal when context mode processing is dominant.

2.1 Empirical Tests of the Two-process Model

Predictions of specific versions of the model were evaluatedin a series of sequence comparison experiments reported in Sorkin(1987b). The results of these experiments are summarized infigures 2 through 4. Figure 2 shows the average performance ofthree subjects in the conditions illustrated in figure 1. Theindependent variable is the standard deviation of the time gapbetween tones. The smooth curves are model fits to the data ofthe uncorrelated and correlated condition. Performance in thecorrelated condition in these axperiments is generallyindependent of gap variability; performance in the uncorrelatedconditions drops to an asymtotic value. These results have beenobtained in a number of different experiments using differentrules to generate the different frequency patterns over trialsand different frequency-difference manipulations. Performance inthe synchronized condition increases slightly as a function ofgap variability.

Figure 3 shows performance as a function of the average gapduration, for a fixed level of gap variability. Performance inthe uncorrelated condition increases as a function of duration,while performance in the correlated condition drops to the samevalue. The smooth curves shown in the figure are modelpredictions based on parameter fits to data in a separate set ofconditions in which the mean gap duration was fixed and the gapvariability was varied. Two points about figure 3 should benoted: First, the effect of the relative gap variability, r/g,is mainly on performance in the uncorrelated condition (fromfigure 2 we know that gap variability does not affect performancein the correlated condition). Second, performance in thecorrelated condition drops at high gap durations, because of theeffect of the total sequence duration on the trace decay time.

5

3 o

SU00

Co

SaUa

0 oL 0O 20" 3'0 40Gap Standard Deviation (Ms)

Figure 2. Average performance (plotted points) as a functionof the gap standard deviation and condition (C,S,U)(8 tones, mean gap - 100 ms). Solid lines aretheoretical functions.

3

2

C 0

U

0 50 100 i0 200Mean Gap Duration (rns)

Figure 3. Average performance (plotted points) as a functionof the mean gap duration and condition (12 tones,gap standard deviation=40 ms, ISI-0.5 s). Solidlines are theoretical functions based on parameterfits to data obtained with 8 tone sequences at amean gap of 100 ms and a range of gap variabilities.

6

3.00 0

2.00o

0

2.004-

C 00. 1.00

Co

Uo

0.00-0.00 0.50 1.00 1.50 2.00 2.50 3.00

Inter-sequence-nterval (s)

Figure 4. Average performance (plotted points) as a functionof intersequence interval (ISI); same subjects andconditions as figure 2 (8 tones, mean gap=50ms, gapstandard deviation-30 ms). Solid lines aretheoretical functions based on parameter fits tothe data of figure 2.

7

The difference between performance in the correlated anduncorrelated conditions is also apparent when examined as afunction of the intersequence interval shown in figure 4;performance in the correlated condition drops over intersequenceintervals of from 500 to 3000 ms. Performance in theuncorrelated condition is essentially independent of ISI (ordrops much more slowly). It is evident that the two-componentdiscrimination model expressed in equations 2 and 3, captures theessential aspects of the results obtained to date. The effectsof gap duration and variability, intersequence interval, andcorrelation, on sequence discrimination performance are allconsistent with the model.

The analysis of sequence discrimination behavior into twoparticular areas: the sensory trace process, and the sequencecomparison process. In addition, we have broadened the paradigmto include the discrimination of temporal jitter as a function ofthe spectral aspects of the patterns. The objective was tobetter describe the trace process and to assess its sensitivityto certain types of transformations (such as linear time scaling)and to potentially interfering inputs (such as irrelevant soundsoccurring during the inter-sequence-interval).

2.2 Experiments on the Trace Mechanism

An important question in sequence comparison is thesensitivity of the trace and context mechanisms to particulartypes of transformations in the sequences to be compared. Thecontext mechanism is assumed to be insensitive to the length ofthe ISI and to the effects of certain events occurring during theISI period. The trace mechanism, on the other hand, is assumedto be highly sensitive to the duration of the ISI and to certainother potentially interfering events, such as the occurrence ofsimilar stimuli during the ISI period. The specific effects ofother types of transformations of the sequences are ofconsiderable theoretical interest. Linearly scaling the durationof the initial or final sequence, for example, should have adifferent effect on performance while in the two modes. Such ascaling may or may not impose additional processing demands oneither mode.

2.2.1 Duration Scaling

One interesting question concerns whether durationcompression or expansion will produce decrements in the tracemode which are a function of the ISI. That is, will durationscaling interact with the increase in trace variance over time?The answer depends on one's concept of the trace process. If thetrace is a true raw representaion of the input, then compressingor expanding the sequence should degrade performance in a fashionsimilar to the addition of temporal jitter to the sequenceintervals, and an interaction will result. If, however, thetrace is (effectively) only a partially encoded representation ofthe tone bursts and the temporal (rhythmic, etc.) relationshipsamong the tone bursts, then such transformations may have a quitedifferent effect. In that case, the effect of such atransformation would not produce interactions with ISI, but

8

rather would resemble the effects of adding temporal jitter inthe context mode: e.g. produce a fixed decrement, depending onthe magnitude of the transformation.

We performed some experiments testing these predictions(Sorkin and Snow, 1987). In one experiment, we modified themanner in which the durations of the tones and gaps of the secondsequence in a trial were generated. In the 'normal' durationtransformation condition, the second sequence in a trial wasgenerated without modification. Trials in this condition were ofthe same type as those shown earlier. In the 'expanded' durationtransformation condition, the durations of all tones and gaps inthe second sequence of the trial were multiplied by a factor of1.4, producing a sequence 40% longer than it would have beenwithout the transformation.

Duration transformation was manipulated factorially withtemporal correlation, producing the 'correlated-expanded'condition, in which the two sequences in a trial had the sametemporal pattern prior to expansion of the second sequence, andthe 'uncorrelated-expanded' condition, in which the temporalpatterns of the two sequences in a trial were different prior toexpansion of the second sequence. Typical trials from each ofthese conditions are shown in figure 5.

We were interested in whether operation in the tracemode requires that the two sequences have correlated temporalenvelopes. Another way to put the question is to ask whethersubjects would treat expansion of the second sequence of a trialas a difference in the sequence temporal patterns- similar to thatfound in the uncorrelated condition- or if, instead, they would -

be able to scale the duration of the sensory-trace of onesequence to match that of the other. If such a scaling processwere possible, then one would expect to see the same generalpattern of results in the correlated-expanded condition as in thecorrelated-normal condition. If such a scaling process were notpossible, then the pattern of performance in the correlated-expanded condition should closely resemble that found in theuncorrela ;ed-normal condition.

The results can be seen in figure 6 and 7. Figure 6 showsperformance averaged across subjects for the correlatedconditions. Performance in both the normal and expanded caseswas independent of gap standard deviation, but decreased withincreasing ISI. Figure 7 shows performance in the uncorrelatedconditions. In this condition, performance was independent ofISI in the conditions involving large differences in sequencetemporal patterns within a trial (i.e. those in which the gapstandard deviation was 40), but decreased with ISI in thoseconditions which did not involve large temporal patterndiffernces (i.e. those in which the gap standard deviation was10). Note that performance in both the uncorrelated-normal anduncorrelated-expanded conditions decreased with increasing gapstandard deviation. The results from the 'normal' durationtransformation conditions replicate the results of ourearlier experiments while the results from the 'expanded'conditions support the proposed duration-scaling hypothesis.

9

I ... . . . ..

Corre lot ed Uncor related

*Sone* "Soe"

411 23 4 2 3 2211 3 4 3 4

JrLrUL-uL RY-j-L J~JFfF4II 23 4 23 22 1 3 43 4

" Ifferent D I f f eren t

1 24 123 4 3 2 4 3 1 3 2 1

_I-LLJ LPJ'Ln__l-jFLLI 2 2 32 1 4 3 2 1 32 3 3 1 4..~~J-ULu'F L__FLLLJLh

Time

Figure S. Examples of pairs of tonal sequences in the duration

transformation experiment.

10

• a.

Correlated3.5i&

3.0 IL

2.0a Normal a 1O0 Expanded. a - 10& Normal, a. 40o Expanded. a,=40

.56 56o0 1000 lib 2000 2500 30b0 3to 4000

Is'

Figure 6. Average performance as a function of inter-sequence-interval, gap standard deviation, and temporal tranzo-formation, for the correlated conditions.

3.5 Uncorreloted

3.0

:b2.5

2-0-.2.0 q'- -=- -° '

0 Normal, or, M 10o Expanded, a,- 10A Normal. a =400 Exponded. 0,=40

3 5&~) 1000 isbo 2000 250 O ~.~ ~~Is'

Figure 7. Average performance as a function of inter-sequence-Interval, gap standard deviation, and temporal trans-formation, for the uncorrelated conditions.

~11

mI

Although performance in the expanded conditions is poorer thanthat in the normal conditions, perhaps due to trace-decay duringduration scaling, the general pattern of results is the same,leading one to believe that S's were able to compensate forsequence expansion in these conditions, perhaps by "scaling" thesensory traces.

2.2.2 Frequency Transformation

A second question related to trace transformations concernsthe degree to which the trace mechanism is sensitive to frequencytransformations. Because of the octave generalizationphenomenon, we would expect that the trace would be insensitiveto octave frequency transformations, but not to othertransformations. An experiment was run in which the observershad to make a same/different judgment on the frequency contour ofthe sequence. In this task the definition of a correct responseis very sensitive to the observer's understanding of theinstructions; therefore, one should be cautious in interpretatingthe results. Four intersequence interval conditions were run(500, 1000, 2000, and 4000 ms.) with four frequencytransformations (0%, 10%, 50%, and 100%). Performance droppedfrom a d' of about 2.5 with 0% transformation to a d' ofapproximately 1.0 with a 100% transformation; performance droppedabout 0.8 d' units between the 500 and 4000 Ms. interseuenceintervals. There was no significant interaction betweenfrequency transformation and intersequence interval, indicatingthat observers can not scale frequency in the same way asduration.

2.2.3 Masking

Another experimental manipulation studied in the context oftrace vs. context mode processing, was the effect of interferingsound inputs which occur during the intersequence interval.Other investigators, such as Masarro and his colleagues (Kallmanand Massaro, 1983) have argued for the occurrence of aninterruption in forming and consolidating information from thepre-contextual trace. The correlated-uncorrelated jitterparadigm allows evaluation of the differential effects of suchpotentially interfering signals. That is, the interfering signalshould not only produce effects on performance dependent on thetime interval between the offset of the sequence and the onset ofthe signal, but should also produce effects which demonstrate ashift from trace-mode like processing to context-mode likeprocessing. These effects should be observable as a differentialdependence of performance on the gap 7ariability and the ISI.

An experiment to evaluate this hypothesis was performedusing 8-tone tonal sequences similar to those describedpreviously (at frequencies of 500, 909, 1667, and 2857 Hz.)equated for perceived loudness (at SPLs of 70, 70, 68, and 64dBA, respectively). Tone bursts were 50 ma. long with linearonset and offset envelopes of 2 ms. The gap durations were drawnfrom a normal distribution (approximate) with a mean of 50 ms anda standard deviation of either 10 ms or 40 ns. The intersequenceintervals were 1000, 2000, and 4000 s. The frequency pattern of

12

I

the first sequence contained two of each of the four possibleW, frequencies in a random order; if the trial was a 'different'

trial, the frequencies of the second or third, fourth or fifth,and sixth or seventh tones in the second sequence (selectedrandomly) were changed to one of the three remaining frequencies.The correlated and uncorrelated manipulations were the same asdescribed in the previous experiments.

In the masked conditions, a three-tone sequence was playedduring the ISI, starting 350 ms after the end of the last tone inthe first sequence. The duration of each tone in the mask was 50ms. Mask gap durations were drawn from an approximation to anormal distribution with a mean of 50 as. In the 'fixed' maskcondition, the standard deviation of this distribution was 0 ms.In the 'random' mask condition, the standard deviation of thisdistribution was 25 as. Tones in the 'fixed' mask were 1231 Hz,at a level of 70 dBA. Tones in the 'random' mask were selectedat random from the set of four which comprised the eight-tonecomparison sequences, subject to the constraint that no frequencycould occur more than once in the same mask. Trials in thenormal phase consisted of a 500 ms visual ready signal, a 500 mswait, the first sequence, the variable ISI, the second sequence,and observer response followed by trial feedback. Trials in themask phase were the same except for the occurrence of the maskduring the ISI (the total ISI was the same).

Observers had extensive experience in the non-maskconditions, from having participated in previous experiments onsequence discrimination. In the mask phase of the experiment,temporal correlation and mask type were held constant during eachsessions (6 blocks of 100 trials), while gap standard deviationand ISI were manipulated factorially across blocks within eachtest session. The order of ISI and gap deviation was random.There were 8 testing sessions, each observer participating in twoblocks from each of the 24 experimental conditions produced bythe combination of levels of temporal correlation, mask type, gapdeviation, and ISI.

As in our previous experiments, only performance in theuncorrelated conditions decreased with increasing gap standarddeviation. Performance in the random mask condition was lower inboth the correlated and uncorrelated conditions than performancein the normal or fixed-mask conditions. However, performance inthe masked-correlated conditions was superior to that in any ofthe uncorrelated conditions. Figure 8 shows the averagedresults as a function of ISI for the (- 40 ms condition. (Thepattern of performance obtained in the correlated anduncorrelated conditions for I - 10 ms was essentially the sameas that found for &'- 40 us.) The standard error of each pointwas estimated by taking the square root of the mean of thevariances (Gourevitch and Galanter, 1967) of the individualsubject d's. The effects of the masker appear to be independentof the correlated/uncorrelated manipulation. One possibleconclusion is that the 'trace' process is not more susceptible tothe effects of an intervening masker than is the context process.

13

3.5C:None

3.0 C:Fixed

C:Random' 2.5 U:None Q

U:Fixed a- - - -... -... - ..., - )

2.0

U:Random -

1.5 I I0 1000 2000 3000 4000 5000

ISI

Figure 8. Average performance as a function of intersequenceinterval with and without intervening masker (gapstandard deviation=40 ms).

14

L

3.3 Temporal Pattern Discrimination

In these experiments, the observer was asked to disciminatebetween two tonal sequences which had the same or differenttemigzi]. or rhthmic, rather than frequency patterns. Theinitial objectives were to model the discrimination process usingthe types of sequence comparison algorithms developed forfrequency pattern discrimination (e.g. Sorkin, 1987a). Inaddition, we were interested in testing for the trace/contextdistinctions observed in the frequency discrimination task and,in addition, in evaluating the effects of temporal transformationor scaling on rhythmic discrimination. The expectation was thatdiscrimination performance would be highly resistant to thetemporal scaling manipulation, over a range of timetransformations.

The stimuli and conditions were similar to those previouslydescribed. Tone sequences consisted of 8, 1000 Hz tone bursts of50 ms duration at 75 dBA. The intersequence interval was 1000 or3000 ms. The gaps were approximately Normal with a mean of 50 msand a minimum gap of 5 ms. On 'same' trials, the durations ofthe gaps in the second sequence were identical to those of thefirst. On 'different' trials, jitter was added to the durationof each gap in the first sequence to obtain the durations of gapsin the second sequence. The amount of jitter was determined by auniform distribution with a mean of 0 ms and a variable standarddeviation, &diff" During the first phase of the experiment, apsychometric function of 0diff was obtained, shown in figure 9.The value of 0"diff was varied from 11 to 40 ms (varied acrossblocks of trials and sessions). Performance increased with 0-diffin the expected fashion.

ISI and temporal transformation were varied during the nextphase of the experiment; temporal scaling was accomplished bymultiplying each tone and gap duration in the second sequence bya duration transformation parameter following normal sequencegeneration. The jitter was added to sequences in 'different'trials prior to the duration transformation. There were sevenduration transformation conditions (varied across blocks) rangingfrom 0.4 (maximum compression) to 1.6 (maximum expansion).Observers were informed prior to each session, whichtransformation condition they were running. rdiff was heldconstant at 30 is; the order of presentation of the durationtransformation condition was random (but with no repeats). Anadditional set of conditions were run in which the first sequencein each trial were also duration-transformed; in other words,additional control conditions were run in which the average onsetto onset intervals were held constant within a trial and block.These effectively non-transformed conditions enable us to factorout the effect of duration on the duration transformation task.

The results of the temporal transformation manipulation areshown in figures 10 and 11. The top curve of figure 10 indicateshow temporal discrimination improves with net total averageduration of the sequence, for a fixed ratio of diff to tone+gapduration. The lower curve shows the effect of transformation inthe condition when the second sequence is transformed relative to

15

the first. Figure 11 shows the effect of transforming the secondsequence when the effect of duration is factored out. It can beseen that the effect of temporal transformation is relativelysymmetric about zero and that the resulting function is highlypeaked. Little effect of ISI can be observed on performance.This sensitivity to expansion or contraction was unexpected andis not consistent with our current models of the discriminationprocess (especially in light of the lack of an effect of ISI).Because of the presence of across-trial variation in the gaps,and because of the manner in which the difference jitter wasadded, this discrimination task is relatively complicated todefine from an ideal observer's point of view. Accordingly, wehave modified the task somewhat in our current experiments.

3. DETECTION AND RECOGNITION OF MULTIPLE ELEMENT SIGNALS

In addition to these experiments on sequential auditoryinputs, we have continued to gather data on detection with multi-element auditory and visual signals. These experiments aredesigned to evaluate how information relevant to a detectiondecision is aggregated over multiple display elements (either ina sequence of display elements, or in a simultaneous presentationof display elements). We reported some of these results inSorkin et al. (1987) in Sorkin, Mabry, and Weldon (1988), and inMabry (1987). Experiments have continued on two questions: (a)the effects of varying the statistics of the displayed elementsover the spatial position of the display, and (b) whether therecognition-detection model (Green and Birdsall, 1978; Green,Weber, and Duncan, 1977) is valid for the multiple-elementsituation. The data acquisition phase of this project wascompleted; initial analysis indicates support for the extensionof the recognition-detection model.

16

a

0 0 -0 - N) In(O/ 0 ( ~ 0 Lnl 0 ( 0

0

0C~c

0

(D<

C)

00 o)

--h,- 0

C,)

r- .0-

S 0

Figure 9. Average performance in the temporal discriminationtask as a function of the standard deviation of thedifference jitter.

17

II

3.0

2.5 Constant

2.0

1.5

1.0 - Transformed

0.5

0.0 - ISI =1000

-0.5 I 1 i i I i0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8

Transformation Factor

Figure 10. Average performance in the temporal discriminationtask as a function of the transformationfactor. The top curve is the condition when bothsequences undergo the same transformation; theabscissca is proportional to average duration.The lower curve is the condition when only thesecond sequence is transformed.

18

3.0

2.5

2.0--

1.5 //

1.0 II 1

0.5"0.5 ISI = 1000ISI = 3000

-0.5 I I I I I I I I

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8Transformation Factor

Figure 11. Average performance in the temporal discriminationtask as a function of the transformationfactor. The data in the lower curve of figure 10have been corrected for the effect of duration.

19

4. REFERENCES

Berliner, J. E., & Durlach, N. I. (1973). Intensityperception. IV. Resolution in roving-level discrimination. JAcoust. Soc. AM.,, U, 1270-1287.

Durlach, N. I., & Braida, L. D. (1969). IntensityPerception. I. Preliminary theory of intensity resolution. J.Acoust. Soc. Aa., _U, 372-383.

Green, D. M. and Birdsall, T. G. Detection and recognition.Psvcholoaical Review, 1978, U, 192-206.

Green, D. M., Weber, D. L., and Duncan, J. E. Detection andrecognition of pure tones in noise. Journal o1 ActiSociety al America, 1977, 62, 948-954.

Howard, J. H., Jr., O'Toole, A. J., Parasuraman, R., &Bennett, K. B. (1984). Pattern-directed attention in uncertain-frequency detection. Prcpt in I PsychoDhysics, U5, 256-264.

Kallman, H. J., & Massaro, D. W. (1983). Backward masking,the suffix effect, and preperceptual storage. Journal 21Experimental P.L. rningl . Memor. and Conition, 9(2),312-327.

Kinchla, R. A., & Smyzer, F. (1967). A diffusion model ofperceptual memory. Prceptin I PsychoDhysics, 2, 219-229.

Kruskal, J. B. (1983). An Overview of Sequence Comparison,in: TjM Warjs. String Edits. =d Macromolecules: The Theory andPractice 21 Seauenne Compaisoan, D., Sankoff and J. B. Kruskal,eds., Addison-Wesley, Reading.

Leek, M. R. (1987). The role of attention in complex soundresolution, in: Auditor y ProcessQing2 Complex Sounds, W. A. Yostand C. S. Watson, eds., Erlbaum, Hillsdale.

Leek, M. R. and Watson, C. S. (1984). Learning to detectauditory pattern components, J. &Coust. Soc. 91 Am., 76, 1037-1044.

Macmillan, N. A., Kaplan H. L., & Creelman, C. D. (1977).The psychophysics of categorical perception. PsvcholoaicalReview, BA, 452-471.

Sorkin, R. D. (1962). Extension of the theory of signaldetectability to matching procedures in psychoacoustics. Journal21 t Acoustical Society 21 Aerica, .3A, 1745-1751.

Sorkin, R. D. (1984). Discrimination of binary tonesequences. J. Acoust. S, An.,, 75(Suppl. 1), S21.

Sorkin, R. D. (1987a). Computational models of tonalsequence discrimination. In W. A. Yost & C. S. Watson (Eds.),Auditory Jing 21Comle Soud. Hillsdale, NJ: Erlbaum.

20

?A

i wk

Sorkin, R. D. (1987b) Tonal Sequence Discrimination.Temporal factors in the discrimination of tonal sequences. J.kggust. Soc, &m., la, 1218-1226.

Sorkin, R. D., Boggs, G. J., and Brady, S. L. (1982).Discrimination of temporal jitter in patterned sequences oftones. J. E=r. Pych. Han Performance nd Pergep in, 8, 46-57.

Sorkin, R. D. and Snow, M. T. (1987). Discrimination oftime-expanded tonal sequences. J. A 2og. AM_,, I, 40.

Spiegel, M. F., & Watson, C. S. (1981). Factors in thediscrimination of tonal patterns. III. Frequency discriminationwith components of well-learned patterns. J. cus, Soc. AM.,19(l), 223-230.

Tanner, W. P., Jr. (1961). Physiological implications ofpsychophysical data. Ann, .. . . ., 1 82, 752-765.

Watson, C. S. (1976). Factors in the discrimination of word-length auditory patterns. In S. K. Hirsh, D. H. Eldredge, I. J.Hirsh, & S. R. Silverman (Eds.), Hearingnd Davis: Essayshonoring Hallowel Davis. St. Louis, MO: Washington UniversityPress, 175-189.

Watson, C. S. (1987). Memory capacity and selectiveattention: Central vs peripheral limits on the discrimination oftonal patterns., in: AhitoyPce nqof Complex Sounds, W. A.Yost and C. S. Watson, eds., Erlbaum, Hillsdale.

Watson, C. S., and Foyle, D. C. (1983). Temporal andcapacity limitations in auditory memory. Jouralo2fheAcoutal~~ Societ~y 21 Am!Rric, 73, S 44.

Watson, C. S. & Kelly, W. J. (1981). The role of stimulusuncertainty in the discrimination of auditory patterns. In D. J.Getty & J. H. Howard (Eds.), Auitor And visual patternr n (pp. 27-35). Hillsdale, NJ: Erlbaum.

Watson, C. S., Kelly, W. J., & Wroton, H. W. (1976). Factorsin the discrimination of tonal patterns. II. Selective attentionand learning under various levels of stimulus uncertainty. J.Acoust. Aoc,. L, IQ, 1176-1186.

Watson, C. S., Wroton, H. W., Kelly, W. J., & Benbassat, C.A. (1975). Factors in the discrimination of tonal patterns. I.Component frequency, temporal position, and silent intervals. J.Acoust.Q. A., 57, 1175-1185.

21

5. PROJECT PERSONNEL

Elvers, G. C. Graduate Student, Department of PsychologicalSciences, Purdue University.

Mabry, T. R. Graduate Student, Department of PsychologicalSciences, Purdue University.

Pezzo, M. Graduate Student, Department of PsychologicalSciences, Purdue University.

Sorkin, R. D. Professor of PsychologicalSciences, Purdue University.

Snow, M. P. Graduate Student, Department of PsychologicalSciences, Purdue University.

6. ADVANCED DEGREES/DISSERTATIONS

Master of Science to T. R. Mabry. & Detecion Theory Anlysis 2Visual Displays, Purdue University, December, 1987.

Doctor of Philosophy to G. C. Elvers. (expected), Purdue

University, May, 1989.

7. PUBLICATIONS and MANUSCRIPTS

Sorkin, R. D. Temporal factors in the discrimination of tonalsequences. Journalo 2 =Ac_ ic Society 2f America, 1987,82, 1218-1226.

Sorkin, R. D., Robinson, D. E., and Berg, B. G. A detectiontheory method for evaluating visual and auditory displays.r.edi L 1 = .th HU .n iFactors Societ, 1987, 2, 1184-1188.

Sorkin, R. D., Kantowitz, B. H., and Kantowitz, S. C. LikelihoodAlarm Displays, Human Factors, 1988, 2Q, 445-459.

Sorkin, R. 0. Review of M. Loeb, "Noise and Human Efficiency",AmerLcan JoUrnalo 2 Pvchogg, 1988, i, 290-293.

Sorkin, R. D. Why are people turning off our alarms? Journal 9&th& Aostical S ey2 America, 1988, §A, 1107-1108.

Sorkin, R. D., Wightman, F. L., Kistler, D. J., and Elvers, G. C.An exploratory study of the use of movement correlated cues in anauditory head-up display. Human Factors, (in press).

Sorkin, R. D., Mabry, T. R., & Weldon, M. Acquisition ofinformation from visual displays; a signal detection theoryanalysis. (submitted).

Sorkin, R. D. and Robinson, D. E. Performance of a System with a HumanOperator and an Autoclassifier (in preparation).

Snow, M. P. and Sorkin, R. D. Scaling and Interference Processesin Auditory Memory (in preparation).

22

tI

m m m m m m-b-- m m m m m --- -U

8. INTERACTIONS

Meeting of the Acoustical Society of America, Miami, Fl.,November 1987. Sorkin, R.D. and Snow, H.P. Discrimination oftime-expanded tonal sequences. J. Acoust. Soc. Amer., 1987, 82,S40.

Meeting of the Air Force Office of Scientific Research.Conference on Auditory Pattern Perception, Evanston, Ii.,December, 1987. Sorkin, R. D., "Two-process model of tonalsequence discrimination".

Conference on Directional Hearing sponsored by the NationalResearch Council/Committee on Hearing, Bioacoustics andBiomechanics, Washington, D.C., October, 1988. Sorkin, R. D."Auditory head-up display: Observer use of cues correlated withhead movement".

Member, National Research Council/Committee on Hearing,Bioacoustics and Biomechanics.

Panelist, National Research Council/Committee on Hearing,Bioacoustics and Biomechanics, Panels on the classification ofComplex, Non-speech Signals, and the Removal of Noise from aSpeech/Noise Signal.

23A.

p L. . .. .. ..

u "mm" • m• mm~nnm N• Sm lHIHNnm~m ' f