development and evaluation of the maintenance error

*Corresponding author. Tel.: #1-206-544-8733; fax: #1-206-544-8502.

E-mail address: [email protected] (W. Rankin).

International Journal of Industrial Ergonomics 26 (2000) 261}276

Development and evaluation of the Maintenance ErrorDecision Aid (MEDA) process

William Rankin*, Rebecca Hibit, Jerry Allen, Robert Sargent

Boeing Commercial Airplane Group, P.O. Box 3707, Seattle, WA 98124-2207, USA

Accepted 1 April 1998

Abstract

The purpose of this study was to evaluate the development and implementation of an airline industry process fordetermining the factors that contribute to maintenance errors and making corrective actions to eliminate or reduce theprobability of future, similar errors. A process like this is useful because maintenance errors have safety and economicconsequences to the airline industry. The Maintenance Error Decision Aid (MEDA) process was developed based on thephilosophy that maintenance technicians do not make errors on purpose, that errors result from a series of relatedcontributing factors, and that these factors are largely under management control and, therefore, can be changed. Theprocess was "eld tested by employees of eight airlines and one repair station. Five surveys, two meetings, and completedMEDA Results Forms were used to evaluate the process. Survey results indicated that: the MEDA process was easy touse, maintenance technicians did not feel intimidated by the process, and management and sta! felt MEDA was usefuland should be continued after the "eld test. Feedback from the meetings was that MEDA had been successfully used tocorrect contributing factors to error, and airline management commitment was the most important factor for successfulMEDA implementation. Suggestions for improving the implementation process were also provided. The completedResults Forms were generally correctly "lled out and indicated an average of 3.4 contributing factors per investigation.Seven of the nine organizations continued to use an error investigation process after the "eld test. Since the end of the"eld test, the authors have provided MEDA implementation consultation to over 60 airplane maintenance organizationsaround the world. Feedback suggests that approximately two-thirds of the organizations are using MEDA.

Relevance to industry

The safety consequences and economic losses to the airline industry due to maintenance errors are very costly.A process for determining the factors that contribute to errors so that they can be corrected should help eliminate future,similar errors. The philosophy that situational factors contribute to error could also be applied in factory settings toinvestigate fabrication, assembly, and operational errors. ( 2000 Elsevier Science B.V. All rights reserved.

Keywords: Maintenance error investigation; Human error; Performance shaping factors; Contributing factors

1. Introduction

Airplane maintenance errors have safety andeconomic costs. A study by Boeing and the U.S.Air Transport Association members (Boeing/ATA,

0169-8141/00/$ - see front matter ( 2000 Elsevier Science B.V. All rights reserved.PII: S 0 1 6 9 - 8 1 4 1 ( 9 9 ) 0 0 0 7 0 - 0

1995) found that maintenance error was one factor,typically among a series of factors, that contributedto 39 of 264 (15%) commercial jet aircraft hullloss accidents where "ve or more people werekilled (or would have been killed if the #ight hadbeen a passenger, rather than cargo, #ight) from1982 through 1991. More speci"cally, in those 39accidents:

f 23% involved an incorrect removal/installationof components,

f 28% involved a manufacturer or vendor main-tenance/inspection error,

f 49% involved an error due to an airline's main-tenance/inspection policy, and

f 49% involved poor design which contributed tothe maintenance error.

In addition, these 39 accidents resulted in 1429on-board fatalities.

One engine manufacturer, discussed by Rankinand Allen (1995), estimated the percentage of speci-"c engine events caused by error and economiccosts of those events to the airlines:

f 20% to 30% of engine in-#ight shutdowns arecaused by maintenance error and can cost anestimated $500,000/shutdown,

f 50% of #ight delays due to engine problems aredue to maintenance errors and can cost an esti-mated $10,000/hour of delay,

f 50%of #ight cancellations due to engine prob-lems are caused by maintenance error and cancost an average of $50,000/cancellation.

But can maintenance error be managed? A Boe-ing analysis of engine in-#ight shut down rates onBoeing airplanes due to maintenance error (com-paring 15 di!erent air carriers with over 1,000,000 hof engine operation) found that the rates di!eredby a factor of sixteen between the lowest (0.0005shutdowns/1000 h) and highest (0.008 shutdowns/1000 h) rates. Clearly, some airlines manage thesetypes of maintenance errors better than others.

Given that maintenance error can have safetyand economic impact, what can be found in theliterature regarding errors and error prevention?The scienti"c study of human error began with

a study of pilot error (Fitts and Jones, 1947). How-ever, major interest in the study of human errorbegan following the Three Mile Island (TMI) nu-clear power plant accident in the spring of 1979.According to Woods et al. (1995), the international,inter-disciplinary study of human error began withthe &clambake' conference on human error in Col-umbia Falls, Maine, in 1980 and with the publica-tions on slips and lapses by Norman (1981) andReason and Mycielska (1982). In addition, work inthe area of human reliability (e.g., Swain and Gutt-man, 1983; Swain, 1987), began in the late 1970sand accelerated following TMI (see Gertman andBlackman, 1994).

The studies dealing with factors that lead to errorlend themselves to the development of a process forinvestigating errors and determining what actionsneed to be taken to prevent or reduce the likelihoodof future, similar errors. However, in order to movein this direction, it is necessary to overcome thenegative connotations about human error, whichcan hinder the in-depth study of the causes of errorand error management (e.g., Lorenzo, 1990; Rea-son, 1990; Woods et al., 1995). Reason (1990) feelsthat people make attributions to the causes of hu-man error, and these attributions most often arerelated to the person rather than to the environ-ment. Reason (1990) discusses this phenomenon asthe &blame cycle'. He believes that we can onlybreak out of the blame cycle if, among other things,we recognize that errors have multiple contributingfactors and that situations or processes are ofteneasier to change than people.

Woods et al. (1995) also are concerned about theprejudicial e!ect that comes from labeling a causeof an accident as human error. One reason is thatattributing an accident to human error is often seenas the causal explanation for the accident. This canrestrict the true investigation that should occur,which is to determine what interaction between theperson, the equipment, and other situational vari-ables lead to the error.

These situational variables have also receivedmuch investigation, especially by Swain and Gutt-man (1983) in their development of human relia-bility analysis tools. They called these situationalvariables performance shaping factors (PSFs), andthey analyzed how PSFs a!ected human error

262 W. Rankin et al. / International Journal of Industrial Ergonomics 26 (2000) 261}276

estimates. They list three major types of PSFs with-in their framework:

1. External PSFs1. C Situational characteristics } e.g., heat, light-

ing, vibration, work hours, shift rotation,organizational structure, and supervisor be-havior.

1. C Job and task instructions } e.g., procedures,shop practices, and organizational policies.

1. C Task and equipment characteristics } e.g., per-ceptual/motor requirements, control/displayrelationships, task complexity, and human/machine interface issues.

2. Internal PSFs } e.g., previous training/experi-ence, personality, intelligence, motivation, atti-tude, gender di!erences, and physical conditionof the person.

3. Stressor PSFs1. C Psychological stressors that directly a!ect

mental stress } e.g., suddenness of onset, taskspeed, task load, duration of stress, high risk,monotony, and distractions.

1. C Physiological stressors that directly a!ectphysical stress } e.g., duration, fatigue, painor discomfort, temperature extremes, radi-ation, vibration, and disruption of circadianrhythm.

Swain (see Lorenzo, 1990) and Bird and Germain(1996) believe that only 15}20% of workplace er-rors can be controlled by individual employees,while the remaining 80}85% are under the controlof management. One important aspect of PSFs isthat they are seen as contributing to the cause ofhuman error. Thus, the concept of PSFs can beused to help break the blame cycle. An obvioussecond important aspect of PSFs is that they helpindicate where changes are needed to reduce hu-man error.

Thus, it is not surprising that the concept ofPSFs is used as a basis of error reduction programs.Lorenzo (1990), in discussing a human error reduc-tion program for the chemical industry, lists theSwain and Guttman (1983) PSFs and then dis-cusses ways to enhance a given PSF in order tominimize human error. McDonald and White(McDonald, 1995; White, 1995a,b) looked at the

PSFs that lead to airport ramp accidents and inci-dents and developed a ramp safety program basedon changes to these PSFs.

The MEDA evaluation was conducted to deter-mine whether a maintenance error reduction pro-cess, based on the PSF concept, could be developedand used to investigate maintenance-error-causedevents and to propose corrective actions to reducefuture, similar maintenance errors within airlinemaintenance organizations.

2. Methods

2.1. Development of maintenance error investigationprocess

Boeing sta!, along with representatives fromBritish Airways, Continental Airlines, United Air-lines, the International Association of Machinists,and the U.S. Federal Aviation Administration, metover a period of 18 months to develop the Main-tenance Error Decision Aid (MEDA) process forinvestigating maintenance errors at airlines. Twoassociated products were developed: a ResultsForm and a User's Guide.

The main investigation tool is the MEDAResults Form. The Results Form consists of "vesections: (1) general information, (2) event, (3) main-tenance error, (4) contributing factors, and (5) cor-rective actions. The general information sectioncontains spaces to report such things as airplaneidenti"cation information, engine type, the MEDAinvestigator, and date of the error and of the errorinvestigation. The Event section contains a listingof potential events, which, if caused by maintenanceerror, would initiate a MEDA investigation. Theevents selected by the development team include#ight delays, #ight cancellations, gate returns, in-#ight engine shut downs, air turn backs, aircraftdamage, #ight diversion, rework, and injury tomaintenance technicians.

The Maintenance Error section lists the errorsthat could occur and lead to an event. The majorerror headings include: improper installation, im-proper servicing, improper/incomplete repair,improper fault isolation/inspection/testing, actionscausing foreign object damage, actions causing

W. Rankin et al. / International Journal of Industrial Ergonomics 26 (2000) 261}276 263

surrounding equipment damage, and actions caus-ing personal injury.

The Contributing Factors section contains situ-ational variables that could contribute to mainten-ance error. The factors were categorized andpresented in a manner recognizable and easily us-able by airline maintenance personnel. Ten catego-ries were used:

1. information } written or computerized in-formation used by maintenance technicians intheir job, e.g., maintenance manuals, servicebulletins, and maintenance tips,

2. equipment, tools, and parts,3. airplane design and con"guration,4. job and task,5. technical knowledge and skills,6. factors a!ecting individual performance } e.g.,

physical health, fatigue, time constraints, andpersonal events,

7. environment and facilities,8. organizational environment issues } e.g., qual-

ity of support from other Maintenance andEngineering organizations, company policiesand processes, and work force stability,

9. leadership and supervision } e.g., planning, or-ganizing, prioritizing, and delegating work,

10. communication } e.g., written and verbal com-munication between people and between or-ganizations.

The Corrective Action section "rst requires theinvestigator to list the existing procedures, pro-cesses, and policies in the maintenance organiza-tion that were intended to prevent the error, but didnot (see Reason's 1990 barriers to error). A secondsection provides space for writing in potential cor-rective actions that would take place where thetechnician does his or her work. A third sectionprovides space for writing in potential correct-ive actions that would take place in other parts ofthe Maintenance and Engineering organization atthe airline (e.g., maintenance planning, mainten-ance engineering, or spare parts ordering).

A typical error investigation process currentlyused by many airplane maintenance organizationsand the eight-step MEDA process are shown inFig. 1. A few words of explanation are needed.

First, the authors have found that in practice thereare actually three variations on the &typical' pro-cess. (1) some organizations stop after the secondstep on the #ow diagram } they simply want toknow which organization to &charge' the error tofor bookkeeping purposes. (2) some organizationsuse the depicted process, although the disciplinecan range from &loss of face' to job termination. (3)some organizations carry out a more in-depth erroranalysis, although, in the authors' experience, theinvestigations are rarely carried out in a consistentmanner or with structured consideration for contri-buting factors. These usually end with the mainten-ance technician promising &never to make thaterror again'.

A few comments are also needed about the pro-posed MEDA investigation process. First, an eventmust occur in order to start a MEDA investigation.Second, although it is necessary to determine whomade the error since a MEDA investigation isdependent upon an interview with this person (orpersons), the MEDA Results Form does not in-clude a place for the maintenance technician's name(the philosophy is &blame the process, not the per-son'). Third, asking the maintenance technicianto suggest potential corrective actions makes themaintenance technician part of the improvementprocess rather than simply the guilty party. Fourth,the interview in the fourth step may determine thatother people or functions within the maintenancesystem contributed to the error (e.g., Stores did nothave the needed replacement part and gave themaintenance technician an incorrect substitutepart), so follow-up interviews may be needed. Fifth,the database mentioned in step six may be nothingmore than a stack of "lled out MEDA ResultsForms, although database software is available forMEDA (Tools for Error Analysis in Maintenance(

from Galaxy Aviation Corporation). Finally, thefeedback mentioned in the last step is needed toshow that something is being done with the invest-igation information and to ensure that everyone inthe organization can learn/bene"t from the process.

In addition to the Results Form, a User's Guidewas developed to explain how to carry out aMEDA investigation using the Results Form and toprovide examples of contributing factors to theinvestigators. Finally, a MEDA presentation was


Fig. 1. Maintenance error decision aid process #ow.

developed to present to airline maintenance man-agement and MEDA investigators.

2.2. Participants

The MEDA process was evaluated at eight air-lines and one repair station to determine its useful-ness for investigating maintenance-error-causedevents. The organizations that were included in the"eld test (along with their MEDA implementationsupport dates) included: All Nippon Airways (De-cember, 1994); America West Airlines (March,1995); British Airways (February, 1995); Continen-tal Airlines (February, 1995); Northwest Airlines(January, 1995); Pemco World Air Services (March,1995); Qantas Airways (March, 1995); Saudi Ara-bian Airlines (December, 1994); and United Airlines(November, 1994).

2.3. Evaluation methodology

Three di!erent methods were used to collectevaluation data on the MEDA process. First, "ve

questionnaires were used to collect opinion data.Before airline employees became MEDA investiga-tors, they "lled out the Field Test Survey, whichcollected baseline opinions about maintenance er-rors and the airline maintenance program. Aftercarrying out a MEDA investigation, the investiga-tor "lled out the Tool Survey, which collected theiropinions about using the Results Form. In addi-tion, the maintenance technician, who made theerror and was interviewed as part of the investiga-tion process, "lled out the Subject Survey, whichcollected their opinions about the MEDA process.Maintenance managers "lled out a ManagementSurvey, which asked about their acceptance ofthe MEDA process and about the perceived im-portance of the contributing factors. Finally, the in-vestigators "lled out the Follow-Up Survey, whichrestated many of the opinion items from the FieldTest Survey.

The second evaluation method was an analysis ofthe completed Results Forms. The forms were re-viewed to see if they were being "lled out in theappropriate manner. In addition, the information


Fig. 2. Field Test Survey, N"248. Percentage of MEDA investigators who selected agree, not sure, or disagree to opinion questionsabout their support environment.

supplied by the forms } event, error type, andcontributing factors } were summarized. The thirdevaluation method was to hold meetings duringand after the "eld test to get feedback from theMEDA contacts from the nine organizations.

3. Results

3.1. Survey results

Each of the "ve surveys contained various typesof opinion questions. Only those questions thatpertain speci"cally to the implementation of theMEDA process are discussed below.

3.1.1. Field Test Survey resultsThe Field Test Survey was "lled out by 248

airline employees before they attended the inves-tigator workshop. Respondents were asked a se-ries of questions about the maintenance supportenvironment (see Fig. 2). A large majority of

respondents (77%) felt that written materials wereavailable when needed, and a plurality of respon-dents agreed that lessons learned from maintenanceerror are shared so that they are less likely to occuragain (48%), that technicians are kept informedabout potential problem areas where errors arelikely to occur (46%), and that written material ispresented in understandable formats (43%). Fewerrespondents agreed than disagreed that techniciansreceive e!ective support from other maintenanceorganizations (34% agree, 37% disagree), that tech-nicians are not afraid to admit to errors (29%agree, 50% disagree), that technicians receive de-tailed feedback from their supervisors about taskperformance (22% agree, 54% disagree), and thatmaintenance technicians are satis"ed with theirworking environment (29% agree, 50% disagree).

In addition, three opinion questions (not shownin Fig. 2) asked about punishment for errors. While45% agreed that punishment is often used to disci-pline technicians for making errors, only 22%agreed that disciplinary actions are fairly applied


Fig. 3. Tool Survey, N"237. Percentage of MEDA investigators who selected agree, not sure, and disgree to opinion questions aboutcarrying out a MEDA investigation.

and justi"ed (43% disagreed), and only 9% agreedthat punishment usually results in improved perfor-mance (63% disagreed).

Finally, one opinion question asked respondentswhat they felt was the biggest obstacle to mainten-ance error investigation in their job environment.Airline processes (32%) was selected most oftenfollowed by airline management (23%) and otheremployees (16%).

In summary, responses regarding the supportenvironment suggested that the MEDA processwould be useful at the airlines. For example, lessthan half the respondents believed that lessonslearned from errors are shared, that written in-formation is presented in an understandable man-ner, that technical support is e!ective, and thattechnicians are satis"ed with their working envi-ronment. However, several opinions suggested thatMEDA would have obstacles to overcome forimplementation. For example, maintenance tech-nicians were afraid to admit to errors, and punish-ment has been used in the past for errors. Inaddition, the respondents also felt that disciplinaryactions were not applied fairly nor did the disciplin-ary actions result in improved performance.

3.1.2. Tool Survey resultsThe Tool Survey was "lled out by 237 respon-

dents after the investigator workshop or after their"rst MEDA investigation. There were six opinionitems regarding whether the MEDA process is easyto use. A majority of the respondents agreed thatMEDA uses familiar words and terms (85%), thatMEDA documentation was understandable (83%),that MEDA documentation was complete (76%),that enough information was provided to learnhow to use the MEDA process (69%), and that it isuseful to list existing barriers to error that failed onthe Results Form (67%).

Eight opinion statements were used regardingusing MEDA for investigations (see Fig. 3). Therewas strong agreement that the results of the MEDAanalysis were understandable (74%) and applicable(74%) and that the MEDA Results Form helpedidentify contributing factors (71%), technicianerror (63%), and corrective actions (57%). Co-operation from other employees during a MEDAinvestigation was not universally strong, as evid-enced by the 52% not sure response to this state-ment. Overall, however, 65% of the respondentsagree that they found MEDA easy to use.


Fig. 4. Tool Survey, N"237. Percentage of MEDA investigators who selected agree, not sure, or disagree to opinion items regradMEDA acceptance and use.

Nine opinion items regarding MEDA acceptanceand likely usage were also asked (see Fig. 4). Theseitems show a generally positive outlook temperedby uncertainty, as shown by the large% of respon-dents who selected the not sure category. There wasstrong agreement that MEDA will yield standard-ized results for analysis (75%), that MEDA couldreplace current maintenance error investigationmethods (72%), that the job environment will im-prove as a result of MEDA (68%), and that main-tenance error will decrease (58%). The respondentsalso believed that maintenance error investigationswill increase (57%) and that MEDA will be sup-ported by maintenance management (57%). How-ever, there was great uncertainty about whetherthere will be less punitive actions for maintenanceerror because of MEDA (54% not sure), thatMEDA would be accepted (53% not sure), and thatMEDA would cause new corrective actions to beput in place (61% not sure).

In summary, 237 respondents "lled out the ToolSurvey. A majority of respondents believed that the

MEDA Results Form helped them with their errorinvestigation, that it was easy to use, and thatMEDA will have a positive impact on their main-tenance organization. However, they are much lesscertain that MEDA will reduce punishment formaking errors or that it will cause new correctiveactions to be taken } issues under managementcontrol.

3.1.3. Subject Survey resultsFollowing a MEDA investigation interview, the

Subject Survey was "lled by the maintenance tech-nician who was interviewed. Seventeen Subject Sur-veys were returned. Nine opinion statements werepresented regarding the maintenance technician'sinvolvement in the MEDA investigation (seeFig. 5). A majority of the respondents agree thatthey did not feel intimidated by the MEDA invest-igation (88%), that it was useful to discuss theexisting barriers to error that failed (75%), that thepurpose of MEDA was made clear to them (65%),that MEDA documentation was made available to


Fig. 5. Subject Survey, N"17. Percent of maintenance technicians who selected agree, not sure, or disagree, on opinion items regardingmaintenance technician involvement in the MEDA process.

them (60%), that MEDA did not create more workfor them (57%), and that the MEDA documen-tation was understandable (53%). There wasless agreement and more uncertainty regardingwhether MEDA helped in identifying contributingfactors to error (50% not sure), whether the resultsof the analysis was understandable (57% not sure),and whether the results of the analysis were madeavailable to the technician (43% not sure). In sum-mary, when maintenance technicians, who madethe error, took part in the MEDA investigation,they usually had a positive experience.

3.1.4. Airline Management Survey resultsThe Airline Management Survey was "lled out

by one or two maintenance managers at each of the"eld test airlines at the end of the "eld test. Thirteenquestionnaires were returned. Nine opinion itemsregarding understanding and acceptance wereasked (see Fig. 6). Ninety-two% of the managers&2fully agree with the MEDA philosophy thatmost maintenance errors are not intentional, but

are mainly a result of factors that contribute toerror' and 67% agreed that they &2fully under-stand how MEDA error investigations were carriedout at my airline during the "eld test'.

Fifty % of the managers agreed that their airlinehad done a good job in implementing MEDA forthe "eld test, although 34% were not sure. Most(85%) agreed that there was strong acceptance ofMEDA by their airline management, although only42% of the managers agreed that there was strongacceptance of MEDA by their airline maintenancetechnicians (and 42% were not sure). Forty-two% agreed that they had seen positive bene"ts totheir airline maintenance function as a result ofusing MEDA, although the remaining 58% werenot sure. Seventy-seven percent of the managersagreed that they strongly support the continued useof MEDA at their airlines, 74% agreed that otherairlines should adopt MEDA, and 69% agreed thatit is important for airlines to share MEDA inves-tigation results with each other. Thirty-eight %of the managers agreed that their airline would


Fig. 6. Airline Management Survey, N"13. Percent of airline managers who selected agree, not sure, or disagree to opinion questionsabout MEDA implementation issues at their airline.

continue to use MEDA after the "eld test, althoughthe remaining 62% were not sure.

In summary, airline management agree with theMEDA philosophy, know how MEDA investiga-tions are carried out, think that MEDA is posit-ively accepted by airline employees, have seenpositive bene"ts from MEDA, and strongly sup-port the continued use of MEDA. However, theyare unsure whether MEDA will continue to be usedat the airline after the "eld test.

3.1.5. Field Test Follow-Up Survey resultsThe Field Test Follow-Up Survey was completed

by 49 MEDA investigators during the last month ofthe "eld test (see Fig. 7). The questionnaire con-tained seven of the same questions that were askedof airline management. In comparing the Fig. 7 re-sults with the results shown in Fig. 6, it is clear thatthe MEDA investigators are not as optimistic asairline management about the acceptance and con-tinued use of MEDA.

Only 18% of the follow-up respondents agreedthat their airline had done a good job in MEDAimplementation, which is much less than the 50%agreement on the Management Survey (p(0.05,one-tailed di!erence of proportions test). Similarly,the follow-up respondents indicated less agreementthan the management on all of the questions: 43%of the managers vs. 17% of the follow-up respon-dents agree that there was strong acceptance ofMEDA by airline technicians (p(0.05, one-taileddi!erence of proportions test); 85% vs. 33% regard-ing strong acceptance of MEDA by airline manage-ment (p(0.05, one-tailed di!erence of proportionstest); 42% vs. 20% regarding having seen positivebene"ts from MEDA (n.s., one-tailed di!erence ofproportions test); 77% vs. 56% on personal strongsupport regarding the continued use of MEDA atthe airlines (n.s., one-tailed di!erence of propor-tions test); 75% vs. 45% regarding whether otherairlines should adopt MEDA (p(0.05, one-taileddi!erence of proportions test); and 38% vs. 22%


Fig. 7. Field Test Follow-up Survey, N"49. Percent of MEDA investigators who selected agree, not sure, or disagree to opinion itemsalso asked to airline maintenance management.

regarding whether their airline will continue usingMEDA after the "eld test airlines (n.s., one-taileddi!erence of proportions test). Thus, MEDA inves-tigators had less positive opinions about MEDAimplementation and acceptance at the airlines com-pared to airline management, although a majorityof the investigators strongly support the continueduse of MEDA at the airlines.

3.2. Results Form results

3.2.1. EventsSeventy-four completed Results Forms were sent

back for analysis. The frequency of the OperationalEvents that were studied were: #ight delay (22events), aircraft damage (17 events), air turn back(11 events), #ight cancellation (7 events), rework (5events), in-#ight shutdown (4 events), gate return (3events), injury (2 events), and other (11 events). The&other' events included workshop errors, vendorproblems, and a few events that probably couldhave been described by an existing event type, but

were coded as &other' by the investigators. Thenumber of events added to more than 74, sincemore than one event could be caused by the error(e.g., in-#ight shut down followed by an air turnback).

3.2.2. Maintenance error typesThe types of errors that lead to the operational

events included: improper installation (26 errors),improper fault isolation/inspection/testing (11 er-rors), improper servicing (9 errors), improper/incomplete repair (3 errors), actions causing foreignobject damage (2 errors), actions leading to per-sonal injury (1 error), other (17 errors), and nomaintenance error reported (5 errors). Of the 17&other' errors, eight were related to errors thatcaused ground damage. The no maintenance errorrecorded was an incorrect use of the Results Form.

3.2.3. Contributing factors typesThe MEDA philosophy is that errors are caused

by a series of contributing factors. The "eld test


results supported this theory. For the 74 error in-vestigations, information was a contributing factorto 37 of the errors, followed by communications(32), job/task (31), environment/facilities (28),factors a!ecting individual performance (26),technician quali"cation/skills (23), airplane design/con"guration (22), equipment/tools/parts (20),organizational environment (19), and supervision(12). Thus, there was an average of 3.4 major cat-egories of contributing factors per error event (250contributing factors divided by the 74 error invest-igations).

3.3. Results of the xeld test feedback meetings

Two feedback meetings were held } one duringthe "eld test and one a month after the "eld testended. The Results Form was felt to work quitewell for error investigation, although several sug-gestions were made for improvements to it, includ-ing the need to rearrange the Results Form so thatthere was more open space on the form for investi-gators to write in.

In general, the major problem discussed was thedi$culty that some of the airlines had in implemen-ting the process. At least one airline had to &restart'the program after their initial implementation. Twoof the organizations involved never got the MEDAprocess implemented during the "eld test. All repre-sentatives agreed that it was very important thatthe process has a management champion to makesure that the program is implemented and con-tinued.

The other important feedback was with regard tothe support provided for MEDA implementation.The airline representatives suggested that three sep-arate implementation sessions be provided. The"rst would be a presentation to senior maintenancemanagement to brie#y explain the investigationprocess and to specify clearly what the manage-ment responsibilities are in implementing the pro-cess. The second session would be a workshop forthe MEDA investigators. It was suggested that theinvestigator workshop be lengthened and includepractice in "lling out the Results Form. The thirdsession would be a meeting with the managementresponsible for implementation in order to lay outan implementation plan.

4. A case study example

4.1. Introduction

The following case study of Going Boeing Air-lines (GBA) was based loosely on one of the "eldtest events. In this case study an airline mainten-ance technician left debris in a fuel cell during fueltank leak checks and repairs, which later resulted ina diversion and #ight delay. A MEDA investigationwas carried out following the event. This examplehas been used extensively by the authors duringtheir MEDA implementation visits.

4.2. Case study

The GBA maintenance personnel involved in-cluded: (1) Scott } A licensed technician on thirdshift, 21 yr of age, two years experience, and aboveaverage height (6@1A) and weight (190 lb.); (2) Dennis} A licensed technician on "rst shift, 32 yr old, 10 yrexperience, and below average height (5@7A) andweight (140 lb.); (3) James } A licensed, lead techni-cian, 41 yr old, and 18 yr experience; (4) Bill } A li-censed technician, now a maintenance qualitycontrol inspector, 52 yr old, and 30 yr experience inmaintenance.

4.2.1. Event summaryA 767 was diverted to the closest airport when

the pilot reported problems with the fuel #owindication system. After a delay, all 210 passen-gers were #own out on another airplane, whichhad been scheduled for an overnight check atthat airport. Extensive troubleshooting, includingdefueling, purging and fuel tank entry for aninspection, revealed debris in the fuel tank. Thedebris included tape, gloves, and several rags. Thesehad clogged some of the fuel lines. The debris hadbeen left during fuel tank leak checks and repairs. Itwas not found by the inspector at the end of thecheck.

4.2.2. MEDA investigationThe MEDA investigation revealed the following

information. Scott and Dennis were the two main-tenance technicians who performed the fuel tank


leak checks and repairs. Scott started the series oftasks during third shift. He used the Aircraft Main-tenance Manual (AMM) as a reference to do thefuel tank purging and entry procedure. Then hestarted the area-by-area leak checks and repairs asshown by GBA work cards, using the AMM asa reference. Scott had trouble moving around in thetank because of his height, and this made him feeluncomfortable. He made minor repairs in someareas of the tank. Scott's shift ended before he"nished the tank. Since Scott wanted to get out ofthe tank as soon as possible, he left the tape, gloves,and rags in the tank for Dennis to use on the nextshift.

Scott checked o! the tasks he had completed onthe sign o! sheets in front of each work card. Healso wrote in the crew shift handover report whichtank areas had been checked and repaired andwhich area he had last worked on. However, he didnot write in the shift handover report nor did he tellhis lead that he had not "nished checking/repairingthe complete tank and that he had left equipment inthe tank.

James was the lead technician for the next shift.He read the shift handover report. He did notnotice that Scott's work card was not signed o!. So,he assumed that Scott's tank was "nished and as-signed the rest of the leak check and repair workcards for the other fuel tanks to Dennis. Dennis wasthe smallest member of his crew, and he found iteasy to work in the fuel tanks.

Dennis completed the leak checks and repairs onthe remaining tanks. Dennis saw that the AMMhad recently been revised. Technicians were nowsupposed to count all the gloves, rags, and otherequipment that were taken into and out of the fueltanks to make sure that all equipment was ac-counted for. Dennis did not think this was needed,but followed the instructions because they wereprobably added for safety reasons. Dennis "nishedthe remaining fuel tanks shortly before the airplanewas due for "nal inspection. He signed o! theremaining work cards and handed them over to hislead, James.

James (following a standard GBA procedure) putall of the fuel tank work cards together in one stack.Then he attached one inspection sign-o! sheet tothe outside of the stack. James handed this and

other stacks of work cards to Bill for the "nalinspection.

The fuel tank access panels were still open whenBill did his inspection. He used a company-pro-vided #ashlight and mirror to inspect as much ofeach fuel tank as he could without going inside thetanks. This was an acceptable level of inspection atthe airline. However, Bill could not see the entirearea inside of each fuel tank from the access panelopenings. Bill stated during his MEDA interviewthat the design of the fuel tanks made it impossiblefor him to see every area using the #ashlight andmirror. He also said that the colors of the gloves,tape, and rags were almost the same color as insidethe fuel tanks. Bill signed o! the inspection sheet onthe top of the stack of work cards. The fuel tankaccess panels were then closed up.

The MEDA investigation also found that theAMM procedures for the fuel tank purging andentry, fuel tank leak checks, and fuel tank repairs allcontained instructions to make sure that all objectswere removed from the tanks when the procedureswere complete.

4.2.3. Completed MEDA Results Form FindingsEvent. The event was a diversion followed by

a #ight delay.Error. The main error was that debris was left in

the fuel tank. A second error was that the inspectormissed the debris on the "nal close-up inspection.

Contributing factors. The factors that contributedto these errors are listed below. In addition, it isindicated whether the factor contributed to leavingthe debris in the fuel tank (debris) or to the inspec-tor not seeing the debris (inspection).

f Information } the work cards had not beenmodi"ed to instruct that everything going intoand out of the fuel tank be counted (debris).

f Equipment/tools/parts } the tape, rags, andgloves were about the same color as the inside ofthe fuel tank (inspection).

f Airplane design/con"guration } the ba%es andstructure in the fuel tank hid the debris from theinspector (inspection).

f Factors a!ecting individual performance } Scottwas too big to move around easily in the fueltank (debris).


f Organizational environment issues } companyprocess allowed stacking of work cards, whichcontributed to no one noticing that Scott hadnot signed o! (debris). Company process allowedless than a full fuel tank inspection (inspection).

f Leadership/supervision } fuel tank work was del-egated to Scott, who was too big to move aroundeasily in the fuel tank (debris).

f Communication } Scott did not tell his lead thathe had not "nished the fuel tank nor that he hadleft the materials in the fuel tank (debris). Inaddition, Scott did not write in the handover logthat he had not "nished the fuel tank nor that hehad left materials in the tank for Dennis (debris).

Corrective actions. There were four failed bar-riers: (1) the "nal close-up inspection, (2) the main-tenance manual, (3) the work cards, and (4) the shifthandover log. Potential corrective actions thatcould be carried out by the inspectors, maintenancetechnicians, leads, and their supervisors (calleda &local' corrective action on the MEDA ResultsForm) include:

f Delegate fuel tank work to smallest maintenancetechnicians.

f Require a full entry inspection.f Encourage better verbal communication and

handover log write-ups.

Potential corrective actions that could be carriedout in other organizations within maintenance andEngineering (called &other' corrective action on theMEDA Results Form) include:

f Order brightly colored rags, gloves, and tape sothat the material can be seen more easily.

f Add the `count things going in/outa steps to thework cards.

f Do not allow stacking of work cards.f Add a sign-o! block for the lead or inspector

next to the maintenance technician's sign-o!block so that a non-signed work card would beseen.

f Design inspection equipment that would allowthe inspector to see all areas of the fuel tank fromthe access panel location.

5. Discussion

The MEDA process was successfully imple-mented at seven of the nine "eld test organizations.The survey data suggested that the MEDA philos-ophy was easy to understand and believed by theparticipants, that the MEDA training was ad-equate, and the MEDA Results Form was under-standable and easy to use. The survey results, alongwith the feedback meetings, suggested that the or-ganizations that implemented MEDA had receivedpositive bene"ts from the error investigation pro-cess.

MEDA was easy to use once it had been imple-mented } the main problem was MEDA processimplementation. Much of the information gained atthe feedback meetings suggested that it was hardfor management to put the various sub-processes(see Fig. 1) in place in order to get the overallMEDA process to work. That is why there wasa strong felt need to have a management championfor MEDA at each airline.

The feedback sessions also suggested that airlinesthat had typically punished maintenance techni-cians for errors found it harder to implementMEDA than airlines that had not carried out(much) discipline for error. Since the MEDA pro-cess is dependent on the erring technician's willing-ness to be interviewed about the error, anythingthat would decrease this willingness, such as a fearof being punished for the error, would have a detri-mental a!ect on MEDA implementation.

After the "nal "eld test meeting in August, 1995,the authors made improvements to the MEDAResults Form, User's Guide, and implementationprocess based on the airline representatives' com-ments. Then, Boeing announced its willingness tohelp customer airlines implement the process (Allenand Rankin, 1995; Rankin and Allen, 1995,1996).Since October, 1995 three of the authors have pro-vided implementation consultation to over 60 addi-tional airplane maintenance programs. Theseorganizations have been encouraged to modify theMEDA Results Form and/or process in order tomake it more useful to them. In January, 1997, theauthors obtained feedback on maintenance organ-ization use of MEDA in order to determine futureimplementation e!orts. The results of the feedback


(Rankin et al., 1997) determined that approxim-ately:

f one-third of the organizations had implementedMEDA as originally designed

f one-third of the organizations modi"ed MEDAand then implemented their modi"ed process,and

f the "nal one-third had not (yet) implementedMEDA.

The organizations using MEDA or their modi-"ed MEDA had all received positive bene"ts fol-lowing implementation. These bene"ts ranged fromsensitizing maintenance management to the causesof error to decreasing #ight departure delays due tomechanical problems by 16%.

6. Conclusions

A maintenance error investigation process basedon the performance shaping factor (contributingfactor) concept can work in the commercial airlineindustry. Implementing the process requires strongmanagement commitment. Carrying out the inves-tigations and making corrective actions is relativelyeasy to do once the process has been put in place.

The results of the "eld test verify that the MEDAphilosophy is correct } i.e., maintenance techni-cians do not make errors on purpose, errors resultfrom a series of contributing factors, and many ofthe contributing factors are under managementcontrol and, therefore, can be managed. This errorphilosophy should also be applicable in otherareas, such as the investigation of fabrication er-rors, assembly errors, and operational errors.

Acknowledgements

Boeing costs for the development of the MEDAprocess were funded by Boeing Internal Researchand Development funds. Airline personnel costsand IAM personnel costs were paid for by theairlines and unions. We would especially like tothank our former colleague, David Marx, forhis help and leadership in developing the MEDA

process. We would also like to thank the MEDAfocals Keven Baines from British Airways and KarlPape from United Airlines for their help in develop-ing the MEDA process. The "eld test was fundedunder FAA contract number DFTA 01-90-X-00055. We would like to thank the FAA O$ce ofAviation Medicine and, speci"cally, the O$ce ofthe Chief Scienti"c Technical Advisor}HumanFactors and our COTR for the project, Thomas M.McCloy, Ph.D.

References

Allen, Jr., J.A., Rankin, W.L., 1995. A Summary of the Use andImpact of the Maintenance Error Decision Aid (MEDA) onthe Commercial Aviation Industry. Proceedings of the FlightSafety Foundation 48th Annual International Air SafetySeminar, International Federation of Airworthiness 25thInternational Conference, and the International Air Trans-port Association &Managing Safety', November 7}9, Seattle,WA, USA.

Bird, F.E., Germain, G.L., 1996. Practical Loss Control Leader-ship. Det Norske Veritas, Inc, Loganville, GA.

Boeing/ATA, 1995. Industry Maintenance Event Review Team.The Boeing Company, Seattle, WA.

Fitts, P.M., Jones R.E., 1947. Analysis of factors contributing to460 `pilot-errora experiences in operating aircraft controls.Memorandum Report TSEAA-694-12. Dayton, OH: AeroMedical Laboratory, Air Materiel Command.

Gertman, D.I., Blackman, H.S., 1994. Human Reliability& Safety Analysis Data Handbook. Wiley, New York.

Lorenzo, D.K., 1990. A Manager's Guide to Reducing HumanErrors: Improving Human Performance in the ChemicalIndustry. Chemical Manufacturers Association, Inc.

McDonald, N., 1995. The Management of Safety on the AirportRamp. The Eighth International Symposium on AviationPsychology, April 24}27, 1995, Columbus, OH.

Norman, D.A., 1981. Categorization of action slips. PsychologyReview 88, 1}15.

Rankin, W.L., Allen Jr., J.P., 1995. Use of the Maintenance ErrorDecision Aid (MEDA) to Enhance Safety and Reliability andReduce Costs in the Commercial Aviation Industry. Pro-ceedings of the International Air Transport Association's1995 Aircraft Maintenance Seminar and Exhibition `TheChanging Visiona, November 14}16, Sydney Conventionand Exhibition Centre, Sydney, Australia.

Rankin, W.L., Allen Jr., J.P., 1996. Boeing introduces MEDA:Maintenance Error Decision Aid. Airliner, April}June, 20}27.

Rankin, W.L., Allen Jr., J.P., Sargent, R.A., 1997. MaintenanceError Decision Aid: Progress Report. Proceedings of theFederal Aviation Administration's Eleventh Meeting on Hu-man Factors Issues in Aviation Maintenance and Inspection`Human Error in Aviation Maintenancea, March 12}13, SanDiego, CA, USA.


Reason, J., 1990. Human Error. Cambridge University Press,New York.

Reason, J., Mycielska, K., 1982. Absent minded? The psychologyof mental lapses and everyday errors. Prentice-Hall, Engle-wood Cli!s, NJ.

Swain, A.D., Guttman, H.E., 1983. Handbook of Human Relia-bility Analysis with Emphasis on Nuclear Power Plant Ap-plications: Final Report. NUREG/CR-1278, SAND80-0200.Prepared by the Sandia National Laboratories for the U.S.Nuclear Regulatory Commission.

Swain, A.D., 1987. Accident sequence evaluation program hu-man reliability analysis procedure. NUREG/CR-4772,SAND86-1996. Prepared by the Sandia National Laborator-ies for the U.S. Nuclear Regulatory Commission.

White, G., 1995a. The Analysis of international accident data onairport ramp accidents. The Eighth International Sympo-sium on Aviation Psychology, April 24}27, 1995, Columbus,OH.

White, G., 1995b. Safety training priorities in aircraft groundhandling. The Eighth International Symposium on AviationPsychology, April 24}27, 1995, Columbus, OH.

Woods, D.D., Johannesen, L., Cook, R., Sarter, N., 1995. BehindHuman Error: Cognitive Systems, Computers, and Hind-sight. Published by Crew Systems Ergonomics Informationand Analysis Center, Wright-Patterson Air Force Base, OH45433-6573.


development and evaluation of the maintenance error

Documents