DOT/FAA/RD-92/3

DOT-VNTSC-FAA-92-3Office of Flight StandardsResearch and Development ServiceWashington. DC 20591

Design of instrument ApproachProcedure Charts: ComprehensionSpeed of Missed Approach InstructionsCoded in Text or Icons

RSPA/VNTSC

isHsimM^'

COCKPIT HF PROGRAM

David W. Osborne

EG&G Dynatrend, Inc.

M. Stephen Huntley, Jr.U.S. Department of TransportationResearch and Special Programs AdministrationJohn A. VolpeNational Transportation Systems CenterCambridge, MA 02142

Final ReportFebruary 1992

This document is available to the publicthrough the National Technical Information ServiceSpringfield, Virginia 22161

©U.S. Department of TransportationFederal Aviation Administration

NOTICE

This document is disseminated under the sponsorshipof the Department of Transportation in the interest

of information exchange. The United States Governmentassumes no liability for its contents or use thereof.

NOTICE

The United States Government does not endorseproducts or manufacturers. Trade or manufacturers'

names appear herein solely because they are consideredessential to the object of this report.

1. Report No.

DOT/FAA/RD-92/3

2. Government Accession No.

4. Title and Subtitle

Design of Instrument Approach Procedure Charts:Comprehension Speed of Missed Approach InstructionsCoded in Text or Icons

7. Author(s)David W. Osborne* and M. Stephen Huntley, Jr.

9. Performing Organization Name and Address

U.S. Department of TransportationResearch and Special Programs AdministrationJohn A. Volpe National Transportation Systems CenterCambridge, MA 02142

12. Sponsoring Agency Name and Address

U.S. Department of TransportationFederal Aviation Administration

Research and Development ServiceWashington, DC 20591

15. Supplementary Notes

*EG&G Dynatrend

16. Abstract

Technical Report Documentation Page

3. Recipient's Catalog No.

5. Report Date

February 1992

6. Performing Organization Code

DTS-45

8. Performing Organization Report No.DOT-VNTSC-FAA-92-3

10. Work Unit No. (TRAIS)

FA2E2/A2155

11. Contract or Grant No.

13. Type of Report and PeriodCoveredFinal ReportJuly 1991-December 1991

14. Sponsoring Agency Code

ARD-210

Instrument approach procedure (IAP) charts are often cluttered and confusing. The quantified effects of chart designchanges on information transfer are needed by chart manufacturers to make changes uhich will enhance information transferand human performance. The present study was conducted as part of a continuing effort at the Volpe NationalTransportation Systems Center Human Performance Laboratory to develop human performance-based design guidelines for IAPcharts.

The objectives of this experiment were to determine whether encoding missed approach instructions in text or icons wouldresult in more efficient information transfer, and if the information transfer efficiency for either coding technique wasdependent upon the level of information content. Twelve pilots currently licensed for instrument (IFR) flightparticipated as subjects. Text instructions were either taken directly or developed from instructions found on NationalOcean Service (N0S) IAP charts. Because of formatting inconsistencies in current NOS missed approach instructions, astandard format was developed. In order to approximate the range of information content found in current NOS missedapproach instructions, these instructions possessed one of three levels of information content: low, medium, and high.Comprehension speed was measured by counting the number of one second presentations (glances) subjects required to viewthe instructions in order to verbally report them. Report accuracy was also measured. Subjects completed questionnairesconcerning their flight experience, preferences for IAP chart manufacturers, and preference for text or iconic coding ofthe instructions.

Across the range of information content levels, iconic missed approach instructions were comprehended more quickly and asaccurately as instructions coded in text of the font style and size used by NOS. Regardless of coding technique, reportaccuracy was significantly worse for instructions with a high information content level. Subjects indicated a strongpreference for using iconic missed approach instructions in single pilot IFR conditions.

17. Keywords

Instrument approach procedure charts,aeronautical charts, missed approach,icons, human factors, information transfer

18. Distribution Statement

DOCUMENT IS AVAILABLE TO THE PUBLIC THROUGH

THE NATIONALTECHNICAL INFORMATION SERVICE.SPRINGFIELD, VA 22161

19. Security Classification (of this report) 20. SecurityClassification (ofthis page) 21. No. of Pages 22. Price

UNCLASSIFIED UNCLASSIFIED 68

Form DOT F1700.7 (8/72) Reproduction ofthis completed page authorized

Preface

This report describes an experiment which examined the effects of encoding missedapproach instructions in text or icons on pilots' comprehension speed and accuracy.Twelve pilots currently licensed for instrument (IFR) flight participated as subjects. Iconicmissed approach instructions were comprehended more quickly and as accurately asinstructions coded intext. Subjects indicated a strong preferencefor using iconic missedapproach instructions in single pilot IFR conditions.

The study was conducted as part of a continuing effort at the Volpe NationalTransportation Systems Center to develop human performance-based design guidelinesfor instrument approach procedure charts. This report was prepared for the AviationResearch and Development Office of the Federal Aviation Administration.

The report was prepared by the Operator Performance and Safety Analysis Division ofthe Office of Research and Analysis at the Volpe Center, and was completed under thedirection of M. Stephen Huntley, Jr., Volpe Center Cockpit Human Factors ProgramManager. The research and report preparation were the responsibility of David W.Osborne, EG&G Dynatrend.

in

METRIC/ENGLISH CONVERSION FACTORS

ENGLISH TO METRIC METRIC TO ENGLISH

LENGTH (APPROXIMATE)I inch (in.) a 2.5 centimeters (cm)

1 foot (ft) a 30 centimeters (cm)

1 yard (yd) = 0.9 meter (m)1 mile (mi) = 1.6 kilometers (km)

AREA (APPROXIMATE)1 square inch (sq in, in2) = 6.5 square centimeters (cm2)

0.09 square meter (m2)

0.8 square meter (m2)2.6 square kilometers (km2)4,000 square meters (m2)

1 square foot (sq ft, ft2)1 square yard (sq yd, yd2)1 square mile (sq mi, mi2)1 acre = 0.4 hectares (he)

MASS - WEIGHT (APPROXIMATE)1ounce (oz) = 28 grams (gr)1 pound (lb) = .45 kilogram (kg)

1 short ton = 2,000 pounds (lb) = 0.9 tonne (t)

VOLUME (APPROXIMATE)1 teaspoon (tsp)

1 tablespoon (tbsp)

1 fluid ounce (floz)

1 cup(c)

' pint (pt)

1 quart (qt)

1 gallon (gal)1 cubic foot (cu ft, ft3)

1 cubic yard (cu yd, yd3)

5 milliliters (ml)

15 milliliters (ml)

30 milliliters (ml)

0.24 liter (I)

0.47 liter (I)

0.96 liter (I)

3.8 liters (I)

0.03 cubic meter (m3)

0.76 cubic meter (m3)

TEMPERATURE (EXACT)[{x-32)(5/9)]'F = y'C

LENGTH (APPROXIMATE)1 millimeter (mm)

1 centimeter (cm)

1 meter (m)

1 meter (m)

1 kilometer (km)

0.04 inch (in)

0.4 inch (in)

3.3 feet (ft)

1.1 yards (yd)0.6 mile (mi)

AREA (APPROXIMATE)1 square centimeter (cm2) a 0.16 square inch (sq in, in2)

1square meter (m2) = 1.2square yards (sqyd, yd2)1 square kilometer (kn2) = 0.4 square mile (sq mi, mi2)1 hectare (he) = 10,000 square meters (m2) a 2.S acres

MASS - WEIGHT (APPROXIMATE)1 gram (gr) a 0.036 ounce (oz)

1 kilogram (kg) • 2.2 pounds (lb)1 tonne (t) a 1,000 kilograms(kg) = 1.1short tons

VOLUME (APPROXIMATE)1 milliliter (ml)

lliter (I)

lliter (I)

lliter (I)

1 cubic meter (m3)

1 cubic meter (m3)

0.03 fluid ounce (fl oz)

2.1 pints (pt)

1.06 quarts (qt)

0.26 gallon (gal)36 cubic feet (cu ft, ft3)

1.3 cubic yards (cu yd. yd3)

TEMPERATURE (EXACT)l(9/5)y + 32]°C = x°F

QUICK INCH-CENTIMETER LENGTH CONVERSION

INCHES

1 I10

CENTIMETERS 0 12 3 4 5 6 7 8 9 10 II 12 13 14 IS 16 17 13 19 20 21 22 23 24 2S

25.40

QUICK FAHRENHEIT-CELSIUS TEMPERATURE CONVERSION

•40° -22° -4° 14' 32° 50° 68° 86° 104° 122° 140* 158* 176° 194° 212°

I I I I I I I I I I I•40' -30' -20" -10° 0° 10° 20' 30' 40° 50° 60° 70° 80° 90° 100'

For more exact and or other conversion factors, see NBS Miscellaneous Publication 286, Units of Weights andMeasures. Price 52.50. SOCatalog No. C13 10286.

IV

TABLE OF CONTENTS

Section Eige

1. INTRODUCTION 1-1

1.1 PRESENT RESEARCH ISSUES 1-31.2 OBJECTIVES AND HYPOTHESES 1-4

2. METHOD 2-1

2.1 SUBJECTS 2-12.2 APPARATUS 2-12.3 PROCEDURE 2-42.4 PERFORMANCE MEASURES 2-62.5 EXPERIMENTAL DESIGN 2-7

3. RESULTS 3-1

3.1 PILOT FLIGHT EXPERIENCE AND IAP CHART USAGE 3-13.2 MEAN GLANCES FOR CORRECT TRIALS 3-13.3 MEAN ERRORS 3-13.4 QUESTIONNAIRE RESPONSES 3-6

4. DISCUSSION 4-1

4.1 MEAN NUMBER OF GLANCES AND A PARADIGM BIAS 4-24.2 MEAN ERRORS 4-34.3 REDESIGNING ICONS IN RESPONSE TO SUBJECT MATTER

EXPERT REVIEWS 4-34.4 FOLLOW-ON RESEARCH WILL BE CONDUCTED 4-44.5 DESIGN RECOMMENDATIONS 4-4

APPENDIX A - ASSESSMENT OF INFORMATION CONTENT LEVEL A-1

APPENDIX B - INFORMED CONSENT FORM B-1

APPENDIX C - FLIGHT EXPERIENCE AND IAP CHART PREFERENCEQUESTIONNAIRE C-1

APPENDIX D - TRAINING MATERIAL FOR TEXT INSTRUCTIONS D-1

APPENDIX E - TRAINING MATERIAL FOR ICONIC INSTRUCTIONS E-1

APPENDIX F - SELF-TEST FOR ICONIC INSTRUCTIONS F-1

v

TABLE OF CONTENTS (Cont.)

Section Page

APPENDIX G - SUBJECTIVE RATINGS QUESTIONNAIRE G-1

APPENDIX H - MOST FREQUENT RESPONSES TO QUESTIONNAIRE ITEMS . . H-1

APPENDIX I - VNTSC PROTOTYPE IAP CHART FOR ILS RWY 3CASPER/NATRONA COUNTY INTL 1-1

REFERENCES R-1

VI

LIST OF FIGURES

Figure Page

1. NATIONAL OCEAN SERVICE IAP CHART FOR THE ILS APPROACH TORUNWAY 4 LEFT AT NEWARK INTERNATIONAL AIRPORT 1-2

2. SCHEMATIC DIAGRAM OF THE EXPERIMENTAL APPARATUS 2-5

3. MEAN GLANCES FOR EACH CODING CONDITION IN EACHINFORMATION CONTENT LEVEL 3-3

4. MEAN ERRORS FOR CODING CONDITION IN EACH CONTENT

LEVEL 3-5

5. IDENTICAL MISSED APPROACH INSTRUCTIONS DEPICTED INTWO VERSIONS OF ICONS 4-2

LIST OF TABLES

Table Page

1. EXAMPLES OF INCONSISTENT FORMATTING 2-2

2. STANDARD FORMAT FOR CONSTRUCTING MISSED APPROACHINSTRUCTIONS (STIMULI) 2-3

3. EXPERIMENTAL DESIGN 2-7

4. MEAN GLANCES FOR CORRECT TRIALS 3-2

5. ANALYSES OF VARIANCE FOR MEAN GLANCES 3-2

6. MEAN ERRORS 3-4

7. ANALYSIS OF VARIANCE FOR MEAN ERROR 3-4

vii/viii

EXECUTIVE SUMMARY

Instrument approach procedure (IAP) charts are often cluttered and confusing.Additionally, the workload imposed by terminal operations can reduce the informationprocessing resources available to the pilot for interpreting IAP charts, increasing thedifficulty of chart interpretation. Specific deficiencies in the design of IAP charts have beenidentified, however, the quantified effects of chart design changes on information transferare needed by chart manufacturers to make changes which will enhance informationtransfer and human performance.

In response to this requirement, the Federal Aviation Administration (1990) incorporatedthe investigation of human performance as a function of IAP chart design into theirNational [Research] Plan for Aviation Human Factors. The present study was conductedas part of a continuing effortat the Volpe National Transportation Systems Center HumanPerformance Laboratory to develop human performance-based design guidelines for IAPcharts.

The objectives of this experiment were to determine whether encoding missed approachinstructions in text or icons would result in more efficient information transfer, and if theinformation transfer efficiency for either coding technique was dependent upon the levelof information content.

Twelve pilots currently licensed for instrument (IFR) flight participated as subjects. Textinstructions were either taken directly or developed from instructions found on NationalOcean Service (NOS) IAP charts. Because of formatting inconsistencies in current NOSmissed approach instructions, a standard format was developed. In order to approximatethe range of information content found in current NOS missed approach instructions,these instructions possessed one of three levels of information content: low, medium,and high.

Comprehension speed was measured by counting the number of one secondpresentations (glances) subjects required to view the instructions in order to verballyreport them. Report accuracy was also measured. Subjects completed questionnairesconcerning their flight experience, preferences for IAP chart manufacturers, andpreference for text or iconic coding of the instructions.

Across the range of information content levels, iconic missed approach instructions werecomprehended more quickly and as accurately as instructions coded in text of the fontstyle and size used by NOS. Regardless of coding technique, report accuracy wassignificantly worse for instructions with a high information content level.

Subjects indicated a strong preference for using iconic missed approach instructions insingle pilot IFR conditions. When subjects were asked which icons they would like toseechanged to make them more clear, the most commonly chosen icons were those whichincorporated radial information. The recommendations of these pilots were echoed by

ix

members of the aeronautical charting committees of the Society of Automotive Engineersand the Air Line Transport Association after reviewing the design changes illustrated bythe prototype IAP chart included as an appendix in this report.

Further research must be conducted before implementation of iconic missed approachinstructions on NOS IAP charts can be recommended as a full replacement for thecurrent text. In the interim, it is recommended that both text and icons be included onNOS IAP charts. The application of both coding techniques is illustrated in the prototypeIAP chart.

1. INTRODUCTION

The International Civil Aviation Organization (ICAO) (1985) defines the purpose ofinstrument approach procedure (IAP) charts as providing "flight crews with informationwhich will enable them to perform an approved instrument approach procedure to therunway of intended landing including the missed approach procedure and whereapplicable, associated holding patterns" (p. 24). In the United States, the largestmanufacturers of IAP charts are the U.S. Department of Commerce National OceanService (NOS) and Jeppesen Sanderson. The instrument landing system (ILS) approachto runway 4 left at Newark International Airport is charted by NOS in Figure 1.

NOS organizes its IAP charts into six main sections (for an exhaustive listing andexplanation of information in each chart section, refer to Federal Aviation Administration,1968). While the content and general formatting standards and recommended practicesfor IAP charts are determined by organizations such as the ICAO and the Inter-AgencyCartographic Committee (IACC), specific formatting conventions and symbol sets aredetermined by the chart manufacturer.

Hansman and Mykityshyn (1990) have explained the determination of chart design as theresult of responding to evolutionary pressures. Forces such as changes instandards andrecommended practices originatingfromthe ICAO. IACC, Federal Aviation Administration,and the National Transportation Safety Board, liability concerns of the chart manufacturer,and individual pilot feedback all combine to affect design changes. The input fromprofessionalorganizationssuch as the SocietyofAutomotive Engineers G-10 AeronauticalCharting Subcommittee, Air Line Pilots's Association, Aircraft Owners and PilotsAssociation, and the inertia imposed by initial design format could be added to this list.

Interpreting the complex and dense information on these charts can be challenging,particularly under poor cockpit lighting. In certain circumstances, this task must beaccomplished while operating in the terminal area during periods of high pilot workload(Kantowitz &Casper, 1988). The workload imposed by terminal operations can reducethe information processing resources available for interpreting IAP charts, increasing thedifficulty of chart interpretation. Relatively more complex approach procedures areaccompanied by correspondingly more complex IAP charts, compoundingthe workloadrequirements.

Pilot feedback, as previously mentioned, and structured surveys such as that conductedby Cox and Connor (1987) have identified specific deficiencies in the design of IAPcharts. However, the identification of design deficiencies is insufficientto improve systemperformance at the pilot-lAP chart interface. The quantified effects of chart designchanges on information transfer are needed by chart manufacturers to makechanges tothe charts which will enhance human performance.

1-1

Top Margin

Plan View

Profile View

Minimums

iottom Margin

Amdl 10 91038

LS RWY 4L

STIILWATER

109.6 SIW

Chan 33

ROBBINSVUIE

113.8 R»v\Chan 85

Not for Navigation

Al-285 (FAA)

CCHTS NECK• "54 COl

Chan IQ1

NEWARK INTL (EWR)NEWARK. NEW JERSEY

AXIS ARR 115 7 134 825

OEP 132 45NEW YORK APR CON

128.55 379.9NEWARK TOWER

118 3 257 6

GRI1Y

I-EWR|I2 I) IOM

I

1955

MISSED APPROACHClimb to 2000. then climbingUrfi rumto 3000 via STW'r.121lo MORNS lit and hold.

ELEV 18 J R~r 29 Idg 6502'

Rwy 4R Idg 8110'

130A

A120

S-IIS41

S-10C41

sidestep

RWY 4R

CIRC1ING

211/18 200(20O-W)

580/24 S69(600-W) 580/50569 1600-11

580/50 568(600 I)

660-1 642(700-1) 660-1%642 (700-1V.)

211/20200 (200-'41

580/60569 1600-1". |

580-1'/i568 (600-1 VI)

920-3902(10000)

Inopcroft** tabic doc* not apply lo AtS Rwy 40 lor Cat A and B; Cat C and 0•ncreoM: *iwb.lily \*» m.le S?&

ILS RWY4L 40'42'N-74'10'W

181

"M ±' :©94Aa '?'

*/ * TDZ'Cl R~yi 41.039' 5 8 NM / 4R and 221

Irom fAF /lOIN R-, 221*»/ "EllR-,i 22Rond 29

H1RI R-ri 41-22R. 4R22lond 11-29

FAf lo MAP 5.8 NM

60 I 90Knott 120 130

M.n:See, 5:48 | 3:52 254 1:56

NEWARK. NEW JERSEY

NEWARK INTL (EWR)

Airport Sketch

FIGURE 1. NATIONAL OCEAN SERVICE IAP CHART FOR THE ILS APPROACH TORUNWAY 4 LEFT AT NEWARK INTERNATIONAL AIRPORT

1-2

Human performance-based design guidelines, empirically derived as a function of theinformation transfer characteristics of design alternatives, were not available to chartmanufacturers prior to the study conducted by Multer, Warner, DiSario, and Huntley(1990). Recognizing this, the Federal Aviation Administration (1990) incorporated theinvestigation of human performance as a function of IAP chart design into their National[Research] Plan for Aviation Human Factors. The present study was conducted as partof a continuing effort at the Volpe National Transportation Systems Center HumanPerformance Laboratory to develop human performance-based design guidelines for IAPcharts.

1.1 PRESENT RESEARCH ISSUES

The missed approach procedure is one of the most hazardous aspects of an instrumentapproach, and typically one in which pilots have the least experience. Pilots will executea missed approach when, upon reaching the missed approach point, prevailing factors(e.g., visibility conditions, obstructions on the runway) preclude landing on the intendedrunway.

ICAO (1985) emphasizes the importance of clearly conveying the instructions forexecuting the missed approach procedure in their definition of the purpose of IAP charts.On NOS IAP charts, these instructions are displayed in the one of the upper corners ofthe profile view (see Figure 1). Although pilots should have memorized the missedapproach procedures, it is not uncommon for them to beforgotten by thetime they mustbe executed. Additionally, communicating with air traffic controllers may interfere withremembering the missed approach.

The missed approach instructions must be coded to allow the most efficient transfer ofinformation possible becausethe number and complexity oftasks in terminal operationsreduce pilots' spareattention. Stokes andWickens (1988), andStokes, Wickens, and Kite(1990) have compiled results ofaviation human factors studies which have investigatedthe efficacy of depicting information pictorially. Almost exclusively, the application ofpictures andicons has been restricted toconveying system status, rather than proceduraldirections. Research in displaying highway traffic sign messages provided the mostrelevant assessments of coding efficacy for procedural directions.

Dewar, Ells, and Mundy (1976) conducted a series of experiments comparing responsetimes for subjects' verbal reports of traffic sign messages encoded in symbols or text.When the stimuli were large, text messages were identified faster than symbolicmessages. However, this difference did not hold for small stimuli. No differences werefound in the number of errors elicited by the coding techniques. Dewar, et al. postulatedthatalthough the response procedure was identical for both types ofcoding, it could beargued that the information processing requirements for vocalizing a clearly legible textmessage were substantially less than those for vocalizing symbols.

1-3

Therefore, in their re-examination of response time to traffic sign messages as a functionof text or symbolic coding, Ells and Dewar (1979) employed a same - different judgementparadigm. Subjects responded to symbolic messages faster than text messages. Thenumber of errors generated by both coding techniques did not differ. Whitaker andStacey (1981) replicated the findings of Ells and Dewar (1979) in two experiments.Symbolic coding elicited faster response times, and coding techniques producedequivocal numbers of errors.

Based on the findings of Ells and Dewar (1979) and Whitaker and Stacey (1981), iconicencoding of missed approach instructions would be expected to result in fastercomprehension without degrading accuracy. However, using icons to convey missedapproach instructions would differ from their applications in these studies inseveral ways:the information content of a set of missed approach instructions is higher than that of asingle highway trafficmessage; a sequence of activities need to be conveyed, rather thanone activity; the instructions are given for movement in three dimensions rather than two;and, alphanumerics must be included with the icons (e.g., altitudes, headings, radials).

Given these differences, the consistent findings of Ells and Dewar (1979) and Whitakerand Stacey (1981) remain highlycompelling and merit investigating the feasibility of iconicencoding of missed approach instructions.

1.2 OBJECTIVES AND HYPOTHESES

The objectives of this research were to determine whether encoding missed approachinstructions in text or icons would result in more efficient information transfer, and if theinformation transfer efficiency for either coding technique was dependent upon the levelof information content.

This experiment tested the following hypotheses:

1) Iconic encoding of missed approach instructions will elicit significantly fasterinformation transfer across all information content levels.

2) Increasing the information content level of the instructions will degrade informationtransfer. This effect will be greater for instructions encoded in text than for thoseencoded in icons.

3) Coding techniques will not generate significantly different amounts of error.

1-4

2. METHOD

2.1 SUBJECTS

Twelve pilots (eleven males) currently licensed for instrument (IFR) flight and having atleast 20/20 visual acuity (normal or corrected) participated as subjects (Ss) in thisexperiment. Subjects who were employees of the Volpe National Transportation SystemsCenter (VNTSC) were given an account number to which they charged theirtime, and Ssunaffiliated with VNTSC were paid $50.00.

2.2 APPARATUS

Text instructions were either taken directly or developed from instructions found on NOSinstrument approach procedure (IAP) charts. Because of formatting inconsistencies incurrent NOS missed approach instructions such as those shown in Table 1, a standardformat was developed. All stimuli adhered to the standard formatting depicted in Table2. Each set of instructions began with one of the entries in the box at the upper leftcorner, and ended with "and hold". Entries from the other boxes were selected tocomplete the instructions.

Text stimuli were created by using WordPerfect (WordPerfect Corporation) and VenturaPublisher (Xerox Corporation) installed on an IBM compatible 386 computer. All textstimuli (iconic stimuli also incorporated text) were printed in a 14 point Helvetica fontclosely matched to the font used by NOS for its IAP charts.

Iconic stimuli were created by using DrawPerfect (WordPerfect Corporation) installed onthe same computer. Current NOS symbols were recreated, and new icons weredeveloped, such as climbing right turn to intercept "radiaP' and cross "radial" as shownin Appendix D. All stimuli were printed using Freedom ofthe Press (Custom ApplicationsIncorporated) post-script language interpreter installed on a Hewlett-Packard Laser Jet3 with 300 dots per inch resolution. Printouts were reduced photographically andproduced as pin-registered slides.

All slides were displayed at a contrast ratio of 11.88:1.91 candelas/meter2 by adjustingthe transmissivity of polarizing filters mounted on the tachistoscopic slide projectors'lenses. The contrast ratios were measured using a Soligor (Spot Sensor II) spotphotometer and a calibration slide with the same darkness as the experimental slides.

Missed approach instructions possessed oneofthree levels of information content (low:6 items; medium: 9 items, and high: 17-19 items), as determined by the procedureshown in Appendix A. These levels were chosen to approximate the range of informationcontent found in current NOS missed approach instructions.

2-1

TABLE 1. EXAMPLES OF INCONSISTENT FORMATTING*

Page 14: "Climbing left turn to 2500 via 002 radial..."

Climbing left turn to 2500 via OSH R-002...

Page 184: "Climb to 2500 on BUU VOR R-221..."

Climb to 2500 via BUU R-221...

Page 222: "Climb to 2300 via R-225..."

Climb to 2300 via MTO R-225...

Page 226: "Climb to 2400 then right turn to 3200..."

Climb to 2400 then climbing right turn to 3200...

♦Formatting inconsistencies in missed approach instructions found in U.S. TerminalProcedures - East Central (EC) Volume 3 of 3, Effective 19 September 1991. Actualentries are given above entries derived from the standard format depicted in Table 2.

2-2

TABLE2.STANDARDFORMATFORCONSTRUCTINGMISSEDAPPROACHINSTRUCTIONS(STIMULI)

Climbto10000

Climbingleftturnto10000

Climbingrightturnto10000

Climbonheading100*to10000Climbingleftturnonheading100*to10000Climbingrightturnonheading100*to10000ClimbviaABCR-100to10000

ClimbingleftturntointerceptABCR-100to10000

ClimbingrightturntointerceptABCR-100to10000

.thenleftturn

,thenrightturn

,thenleftturnonheading100*,thenrightturnonheading100*,thenleftturntointerceptABCR-100

.thenrightturntointerceptABCR-100.crossABCR-100

.crossABC10DME

,thenclimbonheading100*to10000tointerceptABCR-100,thenclimbingleftturnonheading100'to10000tointerceptABCR-100

,thenclimbingrightturnonheading100*to10000tointerceptABCR-100

,thenclimbingleftturnto10000

.thenclimbingrightturnto10000

,thenclimbonheading100*to10000,thenclimbingleftturnonheading100*to10000,thenclimbingrightturnonheading100*to10000,thenclimbviaABCR-100to10000

,thenclimbingleftturntointerceptABCR-100to10000

,thenclimbingrightturntointerceptABCR-100to10000

,thenreversecoursetotheleft

.thenreversecoursetotheright

,thenreversecoursetotheleftclimbingto10000

,thenreversecoursetotherightclimbingto10000

,thenreversecoursetotheleftclimbingonheading100*to10000,thenreversecoursetotherightclimbingonheading100*to10000

directABCVOR

directABCVOR/DME

directABCVORTAC

directABCNDB

directABCNDB/DME

directAbcdeLOM

toABCVOR

toABCVOR/DME

toABCVORTAC

toABCNOB

toABCNDB/DME

toAbcdeLOM

toAbcdeWPT

toAbcdeINT

andhold

AGerbrandstachistoscopic projector system presented all stimuli and was controlled bya Solutions (IBM compatible 286) computerand bySs using two pushbuttons (see Figure2). Stimuli were projected at a size which allowed all alphanumerics to subtend the samevisual angle as the alphanumerics in missed approach instructions on actual NOS IAPcharts at the same viewing distance. To maintain a constant viewing distance of 20 inchesfrom the Da-Lite 16.5 x 18 inch back projection screen, Ss placed their chins in a chinrest. Subjects' verbal reports were given to the experimenter (E) using a Realistic (model33984 B) microphone and recorded on TDK (model D120 lECI/Type I) cassette tapesusing a Marantz (model PMD420) cassette recorder.

Ambient room illumination was 4 footcandles at the subjects' eye point as measured bya Sylvania light meter (model DS-2050). Visual acuity was measured using a Graham-Field eye test chart (model 2867-1264).

2.3 PROCEDURE

After successfully completingvisual acuitytesting, each S was seated at a table and readan informed consent form summarizing the purpose and general procedures of theexperiment (see Appendix B). Ss then completed a questionnaire concerning their flightexperience and preferences for IAP chart manufacturers (see Appendix C). The order inwhich Ss were exposed to both coding techniques (text and iconic) waspseudorandomlycounterbalanced. Subjects completed training and both practice and experimental trialsfor one coding technique before exposure to the other.

Training for the text and iconic instructions began with Ss studying the examples shownin Appendices D and E, respectively. Unlike the text instructions, Ss were required tolearn a new "vocabulary" of icons. Subsequently, a self-test was provided to Ss whichallowed them to assess their competency (see Appendix F). Examples depictedinstructions of all information content conditions. Ss had as much time as they wished tostudy the instructions, and the E was available to answer questions. After completing theself-test on their own, the E reviewed the items in a pseudorandom order, and the Sswere required to demonstrate 100% accuracy in order to proceed.

Upon completion of training, Ss were seated in front of the projection screen. The E thenadjusted the chin rest and introduced the response procedure. The procedure began withthe display of a fixation point with the same contrast ratio as the instructions. Subjectspressed the display button to remove the fixation point and display the instruction slidefor one second. The fixation point then returned. Subjects read the instructions andverbally reported them through their microphone to the E.

Subjects repeated this procedure until they believed that either they had reported thecomplete instruction correctly, or they had reported as well as they could and wished toproceed to the next instruction. Subjects then pressed the carousel button to prepare the

2-4

ro

Com

bini

ngM

irro

rs 3 Sp

eak

er

Pro

ject

or

Co

ntr

oll

er

Mic

rop

ho

ne

Pro

ject

ion

Scr

een

Ch

inR

est

•

Pil

ot

Sh

utt

er

Co

ntr

oll

er

oo

AA

Caro

use

lB

utt

on

Dis

play

Bu

tto

n

Cassett

eR

eco

rder

QO

Mic

rop

ho

ne

FIG

UR

E2.

SCH

EM

AT

ICD

IAG

RA

MO

FT

HE

EXPE

RIM

ENTA

LA

PPA

RA

TU

S

Ex

per

imen

ter

s

Com

pute

r

system for displaying the next instruction. The computer recorded the number of buttonpresses for each instruction, and the E scored the Ss' reporting accuracy.

Subjects who performed the procedure correctly for the final 10 trials (trials 18-27) werejudged as understanding the response procedure and were allowed to proceed to theexperimental trials. Both during the practice trials and before the experimental trialsbegan, the E verbally instructed Ss to minimize the number of stimulus presentationswhile reporting the instructions as accurately as possible.

In the practice trials, within each coding technique, the 27 instructions (three instructioninformation content sets - 9 instructions per set) were randomized in 3 blocks of 9instructions, with each information content level appearing 3 times per block. Ss wereexposed to the same 27 instructions in two forms, text and iconic, for a total of 54practice trials.

During and after the practice trials, the E answered Ss' questions. AfterSs had completedthe practice trials, they indicated when they were ready to begin experimental trials. Intheexperimental trials, within each coding technique, the 90 instructions (three sets - 30instructions per set) were randomized in 15 blocks of six instructions, with eachinformation content level appearing twice per block. Ss were exposed to the same 90instructions in two forms, text and iconic, for a total of 180 experimental trials.

After Ss had completed the second session of experimental trials, they completed thequestionnaire shown in Appendix G. This questionnaire asked Ss for their subjectiveratings of the coding techniques. They were also asked to recall difficulties they haveencountered with IAP charts while flying to help identify other problem areas in chartdesign.

2.4 PERFORMANCE MEASURES

Dewar, et al. (1976) argued that using mean time to verbal report as a measure ofcomprehension speed for text and symbolic messages may have artificially inflated theresponse times for symbolic stimuli. In their experiments, accurate verbal reporting ofsymbolic stimuli required subjects to first comprehend a pictorial message and thenconvert the information into a verbal format. This conversion step was not necessary forverbally reporting text stimuli and may have inflated the mean time to verbally reportsymbolic stimuli. Therefore, the present study employed mean number of glances as ameasure of comprehension speed. Aglance was measured as one display button press,a measure which was independent of mean time to verbal report.

Report accuracy was measured as either correct (1) or incorrect (0). Because the correctexecution of a missed approach instruction depends upon performing all steps in theirproper order, errors resulting from misidentification or omission of information, or

2-6

transpositions in the perceived order of execution necessitated a binary report accuracyscoring system.

Both the mean number of glances and the mean report accuracy were computed for bothcoding techniques (text and iconic), and each information content level (low, medium, andhigh).

2.5 EXPERIMENTAL DESIGN

The experimental design is presented in Table 3. The within-subjects independentvariables were coding technique (text and iconic), and information content level (low,medium, and high). Analyses of variance and the necessary post hoc tests wereconducted on the data.

TABLE 3. EXPERIMENTAL DESIGN

CodingTechnique

Text Icon

Information

Content

Low Medium High Low Medium High

Subjects S1

S12

2-7/2-8

3. RESULTS

3.1 PILOT FLIGHT EXPERIENCE AND IAP CHART USAGE

Subjects' ages ranged from 24 to 61 years, with a median age of 45.5 years. Total flighthours ranged from 330 to 21000 hours, with a median of 948.5 hours. The range of actualIFR flight hours was from 3to 1750 hours, with a median of 62 hours. Simulated IFR flighthours (accomplished with toggles, a hood, or a flight simulator) ranged from 0 to 200hours, with a median of95.5 hours. Twelve, three, and one ofthe Ss had general aviation,Part 121, and Part 135 experience, respectively. Two Ss, including a helicopter pilot, hadmilitary flight experience.

All 12 Ss stated that they used NOS IAP charts, eight of which used them always orfrequently. The Jeppesen-Sanderson IAP charts were used always or frequently by sixSs, while four other Ss never used them. Denoting their preference for eithermanufacturers* IAP charts by assigning each a rank of 1 (most preferred) or 2 (leastpreferred), the eight Ss who used both the Jeppesen-Sanderson and NOS IAP chartsranked them at 1.25and 1.75, respectively. This difference was not significant (p_ > .28)as shown by the Binomial Test (Siegel, 1956).

3.2 MEAN GLANCES FOR CORRECT TRIALS

Table 4 presents the mean glances for correct trials (MG). The results of an analysis ofvariance (ANOVA) conducted on MG are summarized in Table 5. Both main effects,coding technique and information content level, were significant (p. < .05).

The MG for iconic coding was significantly lower than that for text. All pairwise contrastsbetween MG for each information content level were then conducted with the Tukey-Kramer procedure (Kirk, 1982), and all were significant. From lowest to highest MG, theinformation content levels were: low, medium, and high. The MG data are depicted inFigure 3.

3.3 MEAN ERRORS

Table 6 presents the mean error (ME) data. Table 7 summarizes the results of an ANOVAconducted on ME. The main effect for information content levelwas significant (p_ < .05).

The Tukey-Kramer procedure (Kirk, 1982) was used to conduct all pairwise contrastsbetween ME for each information content level. The ME for the high information contentlevel was significantly higher than those for both the low and medium levels. Figure 4presents the ME data.

3-1

TABLE 4. MEAN GLANCES FOR CORRECT TRIALS

Information Content Level

Low Medium Highcoding Mean

Text 2.18 3.58 6.55 4.09

(0.50) (0.88) (1.43) (2.07)

Icon 1.62 2.81 5.63 3.32

(0.56) (0.85) (1.43) (1.95)

Mean 1.90 3.20 6.09

(0.60) (0.95) (1.50)

Note: Standard deviations are given in parentheses.

TABLE 5. Analysis of Variance for Mean Glances

Source

CodingSub * Coding

Info Level

Sub * Info Level

Coding * Info LevelSub * Coding * Info Level 22

*E < .05

df SS MS

1 294.24 294.24

11 85.78 7.80

2 6366.44 3183.22

22 125.26 5.69

2 11.31 5.66

22 38.08 1.73

3-2

37.73*

559.05*

3.27

7 T

6 -

5 --

3

S 3 4-

Low Medium

Information Content Level

High

FIGURE 3. MEAN GLANCES FOR EACH CODING CONDITION IN EACHINFORMATION CONTENT LEVEL (CORRECT TRIALS ONLY)

3-3

TABLE 6. MEAN ERRORS

Information Content Level

Low Medium HighCoding Mean

Text 0.02

(0.15)0.01

(0.10)0.03

(0.16)0.02

(0.14)

Icon 0.01

(0.11)0.02

(0.15)0.05

(0.22)0.03

(0.16)

Mean 0.02

(0.13)0.02

(0.13)0.04

(0.19)

Note: Standard deviations are given in parentheses.

TABLE 7. ANALYSIS OF VARIANCE FOR MEAN ERRORS

Source df

Coding 1Sub * Coding 11

Info Level 2

Sub * Info Level 22

Coding * Info Level 2Sub * Coding * Info Level 22

*p. < .05

3-4

SS MS

0.04 0.04

0.40 0.03

0.21 0.10

0.54 0.02

0.12 0.06

0.49 0.02

1.01

4.31'

2.62

0.05 -r

0.04 --

0.03 --

0.02 --

0.01 --

Low Medium

Information Content Level

High

FIGURE 4. MEAN ERRORS FOR EACH CODING CONDITION IN EACH INFORMATION

CONTENT LEVEL

3-5

3.4 QUESTIONNAIRE RESPONSES

A summary of the most frequent responses to the questionnaire items is presented inAppendix H. The highest concurrence of answers occurred for the first item. Eleven of the12 pilots indicated that in single pilot IFR conditions, they would rather have the iconicthan the text version of the missed approach instructions. The Binomial Test (Siegel,1956) of this difference was significant (e < .01).

The most common reason given for this preference rating was that icons were perceivedas being easier to comprehend, although they were more difficult to verbalize than text.Four subjects reported icons which incorporated radial information as being very difficultto read. Whether "read" in this case refers to "comprehend" and/or "verbalize" is unclear.Perhaps related to these comments were the statements of two Ss who reported thatthey would prefer the text version when flying in a two-person crew. Instructions conveyedin text may be easier to verbally brief to another crewmember in preparation of theapproach.

Four items concerned problems the pilots had experienced with current missed approachinstructions, and the IAP charts in general. The two most frequent responses were thatthe missed approach instructions were difficult to visualize and/or remember, and therelatively low contrast, small print was difficult to "find" (search for) and read while flying.

3-6

4. DISCUSSION

4.1 MEAN NUMBER OF GLANCES AND A PARADIGM BIAS

Subjects required significantly fewer glances to read andverbally report iconically codedinstructions across all information content levels (p. < .05). Mean number ofglances (MG)for icons was expected to vary less than MG for text as a function of information contentlevel, however, the coding technique * information content level interaction was notsignificant (p. > .05).

This differential effect was expected for two reasons. First, although a set of instructionscoded in icons conveys the same information as when coded in text, iconic coding usesfewer figures andtakesadvantage of redundancies in relationships between instructions.For instance, the phrase "Climb to 4500', then climbing left turn to 4900' direct CELVORTAC and hold."consists of 14 words, numerals, or alphanumeric identifiers, while theiconic version contains 7 figures, numerals, or alphanumeric identifiers (see Figure 5).Note that the absence of a radial between the second and third icons necessarily implies"direct", an example of how icons capitalized on redundant relationships betweeninstructions.

Referring to the Examples of Missed Approach Instructions shown in Appendix E, themean number of words, numerals, or alphanumeric identifiers in text instructions for low,medium, and high information content levels are 10, 12.8, and 27.2, respectively. Themean number of figures, numerals, or alphanumeric identifiers for the correspondingiconic instructions are 5, 6.6, and 11.8. Therefore, more information was expected to beconveyed per glance for icons than for text.

Second, icons integrated the instructions such that each step was more holisticallyconveyed. Each maneuver was boxed, and all information relevant to that maneuverwasprovided. The breaking out of each maneuver had already been accomplished. Whenreading text, pilots had to break out each discrete step themselves.

These two factors were expected to partially compensate for increases in informationcontent for iconic instructions, producing correspondingly more divergent MG valuesbetween coding techniques. The data did not support this expectation. The absence ofa differential effect for coding technique as a function of information content level mayhave been due to a bias in the experimental paradigm.

Inorder to verbally report iconic instructions, subjects were required to interpret and thentranslate the pictorial/alphanumeric information into words. Accurate interpretation of theicons was essential to subsequent translation into verbal reporting, whereas verbalreporting of text instructions did not require the translation step, nor did it evennecessitate understanding the instructions to accurately verbally report them. Thisdifference in information processing requirements was the rationale for using MG ratherthan mean time to verbal response (Dewar, et al., 1976) as the measure ofcomprehension speed. Inferring comprehension speed from measuring the number of

4-1

MISSED APPROACH

4000

t4900 «^*p CD

4000

MISSED APPROACH

6400

OBLR-320CD

Icons of the design used in this experiment

MISSED APPROACH

4800

<& C3R-091

4000

MISSED APPROACH

6400

GBLR-320 ^ CDR-320

Redesigned icons proposed by subjects and aeronautical charting committees

FIGURE 5. IDENTICAL MISSED APPROACH INSTRUCTIONS DEPICTED IN TWOVERSIONS OF ICONS

4-2

one second presentations subjects requiredto view the stimulus in order to report it wasexpected to circumvent the potential bias noted by Dewar, et al. However, the iconic -verbal translation step may have attenuated the amount of information conveyed duringverbal reporting.

4.2 MEAN ERRORS

Of the 2160 verbal reports, 52 (2%) were inaccurate. When the errors were groupedaccording to type (e.g., left/right confusion, wrong altitude, omitted altitude) nosystematicdifferences were found. Coding techniques did not elicit significantly (p > .05) differentnumbers of errors. Note that after a relatively brief period of training, subjects reportediconic instructions as accurately as instructions coded in text.

Regardless of coding technique, report accuracy was significantly worse for instructionswith a high information content level (p < .05), as illustrated in Figure 4. All instructionswere either taken directly or developed from instructions found on NOS IAP charts. Inactual IAP charts, as the information content level of the instructions increases, thecomplexity of the information typically increases concurrently. The stimuli reflected thisrelationship. Whether the significant increase in errors is due to the greater amount ofinformation and/or the accompanying increase in complexity is unknown.

4.3 REDESIGNING ICONS IN RESPONSE TO SUBJECT MATTER EXPERTREVIEWS

When Ss were asked which icons they would like to see changed to make them moreclear, the most commonlychosen icons were those which incorporated radial information.Four Ss indicated that these icons were very difficult to read. They were not asked toarticulate whether "read" in this case referred to "comprehend" and/or "verbalize". Therecommendations of these pilots were echoed by members of the aeronautical chartingcommittees of the Society of Automotive Engineers and the Air LineTransport Associationafter reviewing the design changes illustrated by the prototype IAP chart shown inAppendix I. This prototype also incorporates redesigns proposed by Multer, et al. (1990).

Recommended changes to particular missed approach instruction icons are depicted inboth Appendix I and Figure 5. Arrows within icons instructing straight out climbs havebeen shortened. Previous icon combinations describing turns following climbs have beenconsolidated into a single icon. Radial icons have been broken out into two separateicons, conveying the maneuver first and then the radial. The missed approach holdingpattern icon now uses an arrow to indicate the direction of turns, and provides the radialof the inbound leg. The location of the navigation aid relative to the pattern is indicatedby a filled circle.

4-3

4.4 FOLLOW-ON RESEARCH WILL BE CONDUCTED

This experiment determined that iconic missed approach instructions werecomprehended more quickly and as accurately as instructions coded in text, across awide range of information content levels. However, further work must be accomplishedbefore we fully understand the means by which icons facilitate information transfer.

Another experiment is currently being designed to answer the following questions:

1) Are the redesigned icons easier to interpret and/or verbalize than both their currentcohorts and text?

2) Iconic instructions require a larger area for depiction than the text instructions. Ifthe text was enlarged to occupy the same area as the icons, wouldcomprehension speed for text improve?

3) Are instructions coded in text or icons more susceptible to interference fromconcurrent piloting tasks?

4.5 DESIGN RECOMMENDATIONS

The iconic missed approach instructions evaluated in this experiment werecomprehended more quickly and as accurately as instructions coded in text of the fontstyle and size used by NOS. Pilots indicated a strong preference for using icons in singlepilot IFR conditions. However, further research must be conducted before implementationof iconic missed approach instructions on NOS IAP charts can be recommended as afull replacement for the current text.

In the interim, it is recommended that both text and icons be included on NOS IAPcharts.The application of both coding techniques is illustrated in the prototype IAP chart (seeAppendix I).

4-4

APPENDIX A

ASSESSMENT OF INFORMATION CONTENT LEVEL

A-1

Assessment of Information Content Level

Items which are underlined are considered to be discrete information elements.

Instructions Information Content

MISSED APPROACH

Climbing right turn to 7000 6 itemsdirect CTH VOR and hold.

MISSED APPROACH

Climbing right turn to 2000 6 itemsdirect NTL VORTAC and hold.

MISSED APPROACH

Climb to 5000. then climbing right turn 9 itemsto 6000 direct NLV NDB/DME and hold.

MISSED APPROACH

Climb via PEB R-347 to 9100 9 items

to PEB VOR/DME and hold.

MISSED APPROACH

Climb to 4000. then climbing left turn 18 items

via ESI R-001 to 6400. then

climbing left turn to intercept LNA

R-320 to 8000 to LNA VORTAC

and hold.

MISSED APPROACH

Climb to 6100 direct ASN VORTAC, 19 itemsthen climbing left turn to intercept

ASN R-297 to 7100. then

reverse course to the right

climbing to 8900 direct ASN VORTAC

and hold.

A-2

APPENDIX B

INFORMED CONSENT FORM

B-1

Informed Consent Form

You have been asked to participate in the Missed Approach Instructions Study conducted by theOperator Performance Division of the Volpe NationalTransportation Systems Center (VNTSC).

The study's purpose is to assess the speed and accuracy with which pilots read missed approachinstructions. If you agree to participate, you will be asked to press buttons which control a slideprojector system which will briefly display missed approach instructions. You will then read andreport those instructions over a microphone. You will also be asked to complete shortquestionnaires regarding your flight experience, instrument approach procedure chart use, andyour impressions of the missed approach instructions used in this experiment.

The experiment will take approximately 2.5 hours to complete. There are no risks involved inparticipating in this study. Please do not hesitate to ask questions about the study at any time.Your data will be kept strictly confidential and your name will not be associated with your data.

Your participation in this study is strictly voluntary. If you agree to participate, you will eitherbe given a VNTSC account number to which you may charge your time, or you will be paid$50.00. You are free to withdraw at any time without penalty. Your cooperation is sincerelyappreciated.

David W. Osborne, Ph.D.Engineering PsychologistEG&G DynatrendVolpe National Transportation Systems Center(617) 494-3409

Signature and Age of Participant Date

Name (please print)

Address and Phone Number

Signature of researcher Date

B-2

APPENDIX C

FLIGHT EXPERIENCE AND IAP CHART PREFERENCE QUESTIONNAIRE

C-1

Pilot Experience

1. Age: Gender: Male Female

2. Approximately, what is your:

total flight time hours

total IFR flight time hours

3. Please indicate the type of civil aviation experience you have:

Part 121 Part 135 General Aviation Corporate

4. Do you have any military flight experience? Yes No

If yes: approximately what is your total military flight time? hours

Pleaselist the type(s) of aircraft flown:

5. Please rate the frequency with which you use the following instrument approach procedurecharts:

Always Frequently Sometimes Rarely Never

Jeppesen 12 3 4 5

NOAA 12 3 4 5

Other 12 3 4 5

Please specify other.

6. Please rank your preference for these manufacturers' instrument approach procedure charts(1 = most preferred, 3 = least preferred - if you do not use one of these types of charts, write"NA"):

Jeppesen

NOAA

Other

C-2

APPENDIX D

TRAINING MATERIAL FOR TEXT INSTRUCTIONS

D-1

MISSED APPROACH

Climbingrighlturnto 7000diraci CTH VOR and hold.

MISSED APPROACH

Climbing let)turnto 3000direct Grupp LOM am) Kid.

MISSED APPROACH

Climbingrighlturnto 2000direct NTLVORTAC and hold.

MISSED APPROACH

Climbing leu turnto 5090gates Vikm WPT and hold.

MISSED APPROACH

Ormbing ngnt turn to 11200direct ORN NDB and hold.

Examples of Missed Approach Instructions

MISSED APPROACH

CBmo lo5000. thenclimbingrighl turn to 6000 diroct LNVNDB/DME and hold.

MISSED APPROACH

Climb to 4000,thanclimbinglott lum to 4900 direct AHEVORTAC and hold.

MISSED APPROACH

Ctmb via PEB R-347 to»100 to PEB VOR/DMEand hold.

MISSED APPROACH

Climb to 7900.thanclimbinglalt lum to 8500 direct PNA

VOR/DME and hold.

MISSED APPROACH

Climb via DNCR-024 toB300 to 0NC VOR and hold.

D-2

MISSED APPROACH

C8mb to4000. then cUrobing lefthen viaESI R-001 to6400, thenclimbing toftturntoMereepiLNAfi-320to BOOO to LNAVORTACand hold.

MISSED APPROACH

Climb to8100dlract ASNVORTAC.thenclimbing latl turnto InterceptASN R-297to 7100, than ravaraocourse to tha rightclimbingto 6900direct ASN VORTAC and hold.

MISSED APPROACH

Climbing right turnonholding227*to 1000. crott NSR R-270, thanebmblngrightturnto Intareapt PMBR-100 to 2800 to PMBVOR/DMEand hold.

MISSED APPROACH

Qimb to 5200. Ihen etlmbjngrtghtturn*>tnorcapt FPC R-314 to 8200, crossFPC 0DME. then ngm turn direa GTSVORTAC and hold.

MISSED APPROACH

Oimb to 11000direct LCIVORTAC,then c&mbing IanturnlobttereeptLO R-073 to 12000. than revartecourse to the Wt diraci LCIVORTACand hold.

APPENDIX E

TRAINING MATERIAL FOR ICONIC INSTRUCTIONS

E-1

Explanation of Iconic Missed Approach Instructions

Please do not turn to pages 8-10 until you have completed studying all previous pages.

These pages describe the pictorial depiction of missed approach instructions.

Page 2 is a review of standard NOAA chart symbols. The identifiers (e.g., PAR, PKN, Birdd)will always appear in the locations shown.

Pages 3 and 4 show symbols that we have developed. Their meanings are given to the right ofeach symbol.

In the climb and/or turn symbols, the direction of flight will always be presented on the bottom,any heading or radial information will be presented above the direction (arrow), and above that,the altitude to which you are climbing is shown. If you are already at the altitude shown, thenmaintain that altitude while performing the maneuver.

On page 4, please note the difference between the turn to intercept (radial) symbols and thecross (radial) symbols.

Pages 5-7 show examples of how these symbols are combined to form a set of missed approachinstructions. The text version of these instructions is shown to the left of each set of instructions.

Please study pages 2-7 carefully until you feel comfortable with the symbol meanings and the"grammar" of the instruction sets. If you have any questions, please feel free to ask theresearcher.

When you feel comfortable with the material, proceed to the self-test on pages 8-10. Afterinterpreting the symbols, check your answer with the text under cover sheet. If you haveforgotten or misinterpreted anything, please refer back to the previous material or ask theresearcher for clarification. It is very important that you understand the symbols and caninterpret the instructions accurately.

E-2

<S>

VOR

BMd

A

Intersection

Lewis

DME

Navigational Aid Symbols

H

VOR/DME

MCH

NDB

1$

VORTAC

DKS

IS

NDB/DME

cz>

Brown

Waypoint

PSL

LOM

Missed Approach Holding Pattern

E-3

Climb and Reverse Course Symbols

10000

Climb to 10000

4000

**>| Climbing left turn to 4000

8000

(** Climbing right turn to 8000

12000

r\ Reverse course to the left climbing to 12000

(or, if you are already at 12000)

Reverse course to the left

3000

r^ Reverse course to the right climbing to 3000

(or, if you are already at 3000)

Reverse course to the right

5000

270* Climb on heading 270° to 3000

11000

060- Climbing left turn on heading 060° to 11000

™ Climbing right turn on heading 178° to 7000

E-4

Climb via Radial, Cross Radial, and Cross DME Symbols

10SOO

ETLR-137

3000GLR R-344

12000

FTI R-109

r*

SBAR-121

Chiel

Climb via ETL R-137 to 10500

Climbing left turn to intercept GLR R-344 to 3000

(or, if you are already at 3000)

Left turn to intercept GLR R-344

Climbing right turn to intercept FTI R-109 to 12000

(or, if you are already at 12000)

Right turn to intercept FTI R-109

Cross SBA R-121 (you are already at 8000)

Cross Chief 8 DME

E-5

Examples of Missed Approach Instructions

MISSED APPROACH

Climbingrightturn to 7000direct CTH VOR and hold.

MISSED APPROACH

Ctimbtng toll turnto 3000direct Grupp LOM and hold.

MISSED APPROACH

Climbing nght turn to 2000direct NTL VORTAC and field.

MISSED APPROACH

Climbing lah turn to 5000dtrec! Vdun WPT and hold.

MISSED APPROACH

Climbing rigrti turn to 11200direct ORN NOB and hold.

E-6

MISSED APPROACH

7000

r <3> CD

MISSED APPROACH

3000 Orupp

c >

MISSED APPROACH

2000

r4£> CD

MISSED APPROACH

5000 VBun

c >

MISSEO APPROACH

11200

rORN

< )

Examples of Missed Approach Instructions

MISSED APPROACH

Ctmb to SOW.then cSmbingrightturn to 6000 direct LNVNDB/DME and hold.

MISSED APPROACH

Climbto 4000. than dancingaatt turn to 4800 direct AHEVORTAC and hold.

MISSED APPROACH

CbnbvtaPEBR-347to9100 to PEB VOR/DMEand hold.

MISSEDAPPROACH

Climbto 7900. then climbington turn to 8500 direct PNAVOR/DME and hold.

MISSEDAPPROACH

Climb via DNCR-024 to6300 to ONC VOR and hold.

E-7

MISSED APPROACH

5000

t6000 LNV

c @ f )

MISSED APPROACH

4000

t4900 <£> CD

MISSED APPROACH

9100

PEB R-347 3 CD

MISSEDAPPROACH

7900

t8500 (pna) ( ")

MISSED APPROACH

8300

DNC R-024

4/dnc\ CD

Examples of Missed Approach Instructions

MISSEDAPPROACH

CSmb to 4000. thenclimbing tallturn to intercept ESI R-001 lo6400, then climbing lefi turn loIntercept LNA R-320 lo 8000 loLNA VORTAC and hold.

MISSEDAPPROACH

Climb to6100 direct ASNVORTAC,Wonclimbing lot! tumto IntorceptASN R.297 to 7100. than reverseoourse to the rightedmbingto 8900direct ASN VORTAC and hold.

MISSEDAPPROACHCbmoing right tumonheading 227*to 1000.cross NSR R.270. thenclimbing right tumto Intercept PMBR-100to 2600to PMBVOR/DMEand hold.

MISSEDAPPROACH

Cbmb to 5200, then climbing ngm tumto intercept FPC R-314 to 8200. crossFPC 8 DME.than nghi tum direct GTSVORTAC and hold.

MISSEDAPPROACH

Climb to 11000 direct LCI VORTAC,then climbingtoll tum to InterceptLCI R-073 to 12000. then reversecourse to the left direct LCI VORTACand held.

MISSEDAPPROACH

4000

t6400

ESI R-0018000

LNA R-320 ^ CD

MISSEDAPPROACH

6100

t &7100

ASNR-2978900

^ CD

1000

227*

MISSEO APPROACH

2800

PMB R-100

r-£pmb)

MISSEDAPPROACH

CD

5200

t8200

FPC R-314FPC 8200

r ^ CD

MISSEDAPPROACH

11000

t f=>12000

LCI R-07312000

1$ CD

E-8

APPENDIX F

SELF-TEST FOR ICONIC INSTRUCTIONS

F-1

Missed Approach Instructions Self-Test

MISSEDAPPROACH

Climbing tolltum to intercept NOCR-303 to 2000, then reverse courseto the right dtmbing on heading 110*to 3400 to Mills In and hold.

MISSEDAPPROACH

Climb to 1000.thendirnbmg righttum on heading 160* to 2500,cross NTE 10 DME.then cBmbto3500 direct MEKVORTAC and hold.

MISSEDAPPROACH

Climbing tofttum to interceptOBER-153to3600toCeltsInland hold.

MISSEDAPPROACH

Climb to 3000 directLTA VOR/DME and hold.

MISSEDAPPROACH

Climbingrtghltum to interceptUSNR-0S3to9000toHarapIra and hold.

MISSEDAPPROACH

2000

NOC R-3032000 3400

110"

4

MBs

A ( >

MISSEDAPPROACH

1000

t2500

160'NTE

Do)

3500

t•*Tmek^ CD

F-2

MISSEDAPPROACH

3600

DBER-153Celts

A CD

MISSEDAPPROACH

3000

e CD

MISSEDAPPROACH

9000

USNR-083Karap

A< >

Missed Approach Instructions Self-Test

MISSEDAPPROACH

Climb to 6000 directPMRVOR and hold.

MISSEDAPPROACH

Climbon heading 163*to 1700, thendimbingbit tum on heading144*to 2500 to InterceptCCG R-021toHorse Inland hold.

MISSEDAPPROACH

Climbingtoll tum to 4200direct HRINDB and hold.

MISSEDAPPROACH

Climbing toll tum to InterceptSDR R-189 to 300010 KevinIns and hold.

MISSEDAPPROACH

Climbing righttum to 10000direct Pros LOM and hold.

MISSEDAPPROACH

6000

t/pmr\ CD

MISSEDAPPROACH

1700

163*

4

2500

144*

2500

CC6 R-021

4

Horse

A< >

F-3

MISSEO APPROACH

4200 HRI

6) ( ")

MISSEDAPPROACH

3000

SDR R-189Kevin

A CD

MISSEDAPPROACH

10000

rPfOtt

<__>

Missed Approach Instructions Self-Test

MISSEDAPPROACH

Climbing left tum to 7300direct NNO VORTAC and hold.

MISSEDAPPROACH

Climb to7000direct JEPVORTAC,then dunb viaJEP R-250 to 6500.then reverse courseto me ngm directJEP VORTAC and hold.

MISSEDAPPROACH

Climbto 7000. men dimblngleft tum to 6000 direct ABNNDB and hold.

MISSEDAPPROACH

Cbmbto 4300 directDDLVORTAC,men climbingnghi tum to iraorcepiDDL R-261 to 5700. men reversecourse to the loft efcmbmgto 7000direct DDLVORTAC and hold.

MISSEDAPPROACH

Climb via SIAR-34S to5500 to SIA VOR and hold.

MISSED APPROACH

7300

•CwoT CD

MISSEDAPPROACH

7000

t ^8500

JEP R-250

4

8500

<& CD

MISSEDAPPROACH

7000

t8000 ABN

( ">

MISSEDAPPROACH

4300

t *\°^5700

DDL R-2617000

4& CD

MISSEDAPPROACH

F-4

APPENDIX G

SUBJECTIVE RATINGS QUESTIONNAIRE

G-1

Questionnaire

1. Please circle a number on each scale to show the difficulty of reading the instructions whenthey were displayed in:

text 12 3 4 5 6 7

very easy very difficult

icons 12 3 4

very easy

Please explain your ratings:

6 7

very difficult

2. In single pilot IFR conditions, would you rather have the text or iconic versions of the missedapproach instructions to read?

Please circle one: text iconic Why1]

3. Which icons would you change to make them more clear? How would you change them?

4. When you are flying, what problems do you encounter when reading missed approachinstructions?

G-2

5. What other problems do you encounter when using instrument approach procedure charts?

6. What would you change on the instrumentapproach procedure charts to make them easier touse?

7. Do you write on or mark up your instrument approach procedure charts? If yes, pleasedescribe what you do and why.

G-3/G-4

APPENDIX H

MOST FREQUENT RESPONSES TO QUESTIONNAIRE ITEMS

H-1

Most Frequent Responses to Questionnaire Items

The most frequent responses to each questionnaire item are paraphrased below. Thenumber of Ss giving a particular response is shown on the left.

Please explain your ratings of the difficulty of reading the instructions.

4 Icons were more easily comprehended but more difficult to verbalize than text.

In single pilot IFR conditions, would you rather have the text or iconic versions of themissed approach instructions to read?

3 Iconic was much easier to comprehend and comprehension was faster.

3 Iconic conveyed the individual steps of the procedure more clearly.

2 Would prefer the text with a two-person crew.

Which icons would you change to make them more clear? How would you changethem?

4 The radial icons were very difficult to read.

When you are flying, what problems do you encounter when reading missed approachinstructions?

6 The maneuvers are difficult to visualize and/or remember.

5 The text is difficult to find (or read).

2 I do not regularly brief the missed approach instructions.

What other problems do you encounter when using instrument approach procedurecharts?

3 The NOS booklets are difficult to handle.

3 Small print is difficult to read.

H-2

What would you change on theinstrument approach procedure charts to make themeasier to use?

3 Increase the size and contrast of the print.

Do you write on ormark up your instrument approach procedure charts? If yes, pleasedescribe what you do and why.

7 I do not write on or mark up my IAP charts.

2 Only to indicate NOTAMs or other changes not reflected on the IAP charts.

H-3/H-4

APPENDIX I

VNTSC PROTOTYPE IAP CHART FOR ILS RWY 3 CASPER/NATRONA COUNTY INTL

1-1

VNTSC Prototype Chart - Not for Navigation

LOC ISYO •;

111.3 = •

APP COURSE

032°FAFALT

7200'TDZE

5325'ATS

126.15

CASPER APP CON

120.65

CASPER TOWER'

118.3 (CTAF)GNDCON

121.9

CASPER. WYOMING ILS RWY 3

CASPER/NATRONA COUNTY INTL (CPR)

CASPER RADIO

122.4

CLNCOEL

121.9DME Irom DDY VOR. GS unusable MM inbound.Simuttaneous reception ol ISYOLOCand DDY DMErequired when radar unavailable. RADAR or DMErequired.

missedapproach: Climb to 7500' direct DDYVORTAC and hold.

Apt. Elev 5348'(IAF)

HIKOK

DDY[»)

850010

LOC'ODV [29)1360! (5.3)

and LOC (23)

r-

DOY[|i) J

' CAT 0vitality inqorad1<4 milo tor inoperriv*. MAISR \f&

Amdt5 91150 AL-72(FAA)

1-2

9061

MISSEDAPPROACH

7500

t•Tody?

R-025

R£[LRwyt21and2$HlRLRwys 321 and8-26

032' 56NM Sf/omFAF

TOZE

5325\

42*54^ -106° 2ffWFAFWMAP55NM

Knott 120 150 ISO

MmSec 530 3:40 2:45 2:12 ISO

REFERENCES

Cox, W.J., & Connor, C.W., (1987). Report of safety survey: Human integration ofapproach charts. (Contract No. F33615-85-C3623). Washington, DC: Federal AviationAdministration, Program Engineering and Maintenance Service.

Dewar, R.E., Ells, J.G., &Mundy, G. (1976). Reaction time as an index of traffic signperception. Human Factors, 78(4), 381-392.

Ells, J.G., &Dewar, R.E. (1979). Rapid comprehension of verbal and symbolic traffic signmessages. Human Factors, 21(2), 161-168.

Federal Aviation Administration (1968). Civil useofU.S. government instrument approachprocedure charts (AC NO: 90-1 A). Washington, DC: Department of Transportation,Federal Aviation Administration.

Hansman, Jr., R.J., & Mykityshyn, M.G. (1990). Current issues in the design andinformation content ofinstrument approach charts (Contract No. DTRS-57-88-C-00078).Cambridge, MA: Massachusetts Institute of Technology, Aeronautical SystemsLaboratory.

International Civil Aviation Organization (1985). Instrument approach chart - ICAO. InInternational standards andrecommendedpractices: Aeronautical charts. Annex 4 totheconvention oninternational civil aviation (pp.24-27). Montreal, Quebec: International CivilAviation Organization.

Kantowitz, B.H., &Casper, P.A. (1988). Human workload in aviation. In Wiener, E.L., &Nagel, D.C. (Eds.), Human factors in aviation (pp. 157-187). New York, NY: AcademicPress, Inc.

Kirk, R.E. (1982). Multiple comparisontests. In Experimental design (pp. 90-133,2nd ed.).Belmont, CA: Brooks/Cole Publishing Company.

Multer, J., Warner, M., DiSario, R.M., & Huntley, Jr., M.S. (1990). Design considerationsfor IAP charts: Approach course track and communication frequencies (Report No.DOT/FAA/RD-91/19). Cambridge, MA: U.S. Department of Transportation, Volpe NationalTransportation Systems Center.

Siegel, S. (1956). The one-sample case. In Nonparametric statistics for the behavioralsciences. New York, NY: McGraw-Hill Book Company, Inc.

Stokes, A.F., & Wickens, CD. (1988). Aviation displays. In Wiener, E.L., & Nagel, D.C.(Eds.), Human factors in aviation (pp. 387-431). New York, NY: Academic Press, Inc.

R-1

Stokes, A.F., Wickens, CD., & Kite, K. (1990). Color and pictorial displays. In Displaytechnology -human factors concepts (pp. 65-87). Warrendale, PA: SocietyofAutomotiveEngineers, Inc.

U.S. Departmentof Commerce National Ocean Service (1991). U.S. Terminal Procedures- East Central (EC) VOL 3 of 3. Rockville, MD: Author.

Whitaker, L.A., &Stacey, S. (1981). Response times to left and right directional signs.Human Factors, 23(4). 447-452.

R-2

;j#::. •.!»*•*;;*•>.'•..•?:•••.;'•

• V ,:

. / • •

idiib