improving maintenance decision making in the finnish air force through simulation

8/18/2019 Improving Maintenance Decision Making in the Finnish Air Force Through Simulation

1/16

Vol. 38, No. 3, May–June 2008, pp. 187–201issn 0092-2102 eissn 1526-551X 08 3803 0187

informs ®

doi10.1287/inte.1080.0349© 2008 INFORMS

Improving Maintenance Decision Making inthe Finnish Air Force Through Simulation

Ville Mattila, Kai VirtanenSystems Analysis Laboratory, Helsinki University of Technology, FIN-02015 HUT, Finland

{[email protected], [email protected]}

Tuomas RaivioGaia Consulting Oy, Systems Analysis Laboratory, Helsinki University of Technology,

FIN-02015 HUT, Finland, [email protected]

We used discrete-event simulation to model the maintenance of fighter aircraft and improve maintenance-relateddecision making within the Finnish Air Force. We implemented the simulation model as a stand-alone tool that

maintenance designers could use independently. The model has helped the designers to study the impact of maintenance resources, policies, and operating conditions on aircraft availability. It has also enabled the FinnishAir Force to advance the operational capability of its aircraft fleet. We designed the model to simulate bothnormal and conflict operating conditions. The main challenge of the project was the scarcity and confidentialityof data about the fighter aircraft, their maintenance, and various operational scenarios, especially during conflictsituations.

Key words : simulation: applications; military: defense systems; reliability: availability, maintenance/repairs. History : This paper was refereed. Published online in Articles in Advance June 4, 2008.

Afighter aircraft typically requires several hoursof maintenance per hour of flight activity. Thismaintenance involves a diversity of operating poli-

cies, processes, people, and materials. In a fleet of

fighter aircraft, these elements form a complex main-

tenance system. The system performance directly

affects aircraft availability, i.e., the number of aircraft

that can be used in flight missions. Ability to assess

how maintenance-related decisions or operating con-

ditions affect the system is critical in maintaining the

fleet’s operational capability.

We used discrete-event simulation, which has been

widely applied in studying manufacturing systems

(Law and Kelton 2000), to model the maintenance

of the fighter aircraft fleet in the Finnish Air Force(FiAF). It lends itself to analyzing a maintenance sys-

tem because manufacturing and maintenance share

common features, such as workforce considerations,

tasks times, material-handling delays, and equipment

reliability. We also found simulation to be a suitable

method for modeling the FiAF maintenance system

because it enabled us to study the system from many

aspects that the FiAF maintenance designers were

likely to consider. The model describes the essen-

tial features of flight operations and maintenance

including planned and unplanned maintenance, air bases, aircraft repair shops, and maintenance person-

nel. Moreover, it describes both normal and conflict

operating conditions.

Some earlier studies on military operations also

applied simulation to consider the effects of reliability

and maintainability on aircraft operational capability.

For example, Balaban et al. (2000) and Ciarallo et al.

(2005) developed simulation models for availability

of cargo and mobility aircraft, respectively. Upadhya

and Srinivasan (2004) built a simulation model for

availability of generic aircraft and helicopters in com-

bat operations. Rodrigues et al. (2000) used a simula-tion model to assess the spare-parts management for

A-4 aircraft. Kang et al. (1998) considered two simu-

lation models for managing spare-parts and compo-

nent repairs. In a recent paper, Kladitis et al. (2007)

used simulation to analyze the impact of a new

avionics system on the availability of B-52H bombers.

However, these models either considered different

187


2/16

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through Simulation188 Interfaces 38(3), pp. 187–201, © 2008 INFORMS

types of flight operations than our model did or con-

sidered a more narrowly defined problem; they did

not take a system-wide view of maintenance. Pohl

(1991) used operational test data to devise a simula-tion model for operations of the F-15E aircraft. The

model described maintenance in much the same way

as ours did but limited the discussion to consideration

of a fixed-size squadron in a single air base. To the

best of the authors’ knowledge, no previous simula-

tion models in the open literature have considered the

maintenance of a fleet of fighter aircraft at the depth

of the model that we present in this paper.

Our primary challenges in constructing the model

were scarcity and confidentiality of data. In particular,

no data were available for modeling certain elements

of conflict conditions such as battle-damage probabilities or repair-time distributions. Some confidential

data, which FiAF could not share with the authors,

included parts of the contingency plans on conflict-

time maintenance policies. We found two approaches

useful in overcoming these challenges. First, in situa-

tions where data were unavailable, we asked subject

matter experts from different organizational levels

to provide their opinions. Second, we designed the

model such that the confidential information was

included in the input data; the maintenance designers

who performed the corresponding simulation anal-

ysis could thus handle the confidential data inde-

pendently. Implementing the model as a stand-alone

tool with a graphical user interface (GUI) facilitated

our second approach because it made the model

approachable to the designers. The scarcity of data

also affected the validation of the model. We were

able to perform limited comparisons between the sim-

ulation output and actual performance data from the

maintenance system. Therefore, we used subject mat-

ter experts on multiple occasions to assess the under-

lying assumptions as well as the model output.

We introduced the model in the FiAF units that per-

form aircraft maintenance; it has enabled these units

to address many maintenance-related issues. Exam-

ples include the forecasting of aircraft availability, the

analysis of the resource requirements for international

operations, and the feasibility study of a readjusted

periodic maintenance program. The project has also

provided FiAF with new knowledge about possible

applications of simulation techniques. For example,

the Finnish Army subsequently devised a simulation

model for the maintenance system of newly acquired

transport helicopters with collaboration from FiAF.

FiAF Aircraft Maintenance

The FiAF aircraft fleet consists of Boeing F-18 Hor-

net fighters, BAe Hawk Mk51 jet trainers, and other

types of aircraft used in transportation, air surveil-

lance, flight training, and liaison duty. We consid-

ered the flight operations and maintenance of the F-18

Hornet aircraft during normal and conflict conditions

(“conflict” refers to a situation in which the aircraft

fleet is involved in an actual engagement with an

enemy). However, because detailed Hornet informa-

tion is classified, we discuss Hawk maintenance inthis paper. At the modeling level, we found that the

maintenance principles and the appearance mecha-

nisms of unexpected failures are very similar; in gen-

eral, they differ only in model parameters. Hence, the

principles we report here apply to the Hornet as well.

The FiAF aircraft fleet has three primary operational

units that are called air commands (Figure 1). Within

each air command, a fighter squadron is responsible

for aircraft flight operations and specific maintenance

activities. Each air command also has a separate repair

FiAF

Headquarters

Air commands Air command 1

Air command 2

Air command 3

Headquarters

Fighter squadron

Air command’s

repair shop

Depot-level

repair shops

Other units

Other units

Figure 1: The primary operational units for flight operations and aircraft

maintenance of FiAF include three air commands that are further divided

into fighter squadrons and repair shops. Separate, depot-level repair

shops perform the most demanding maintenance.


3/16

Mattila, Virtanen, and Raivio: Improving Maintenance Decision Making in the FiAF Through SimulationInterfaces 38(3), pp. 187–201, © 2008 INFORMS 189

shop for more complex maintenance tasks. Depot-

level maintenance units of the national aerospace

defense industry perform the most demanding main-

tenance. The organization that Figure 1 shows remainsessentially the same during both normal and con-

flict conditions, although the decentralization of the

units during a conflict may change their geographic

locations.

Normal Conditions

During peacetime, the activities of an air command

are centralized at a single air base. The general goals

of aircraft maintenance are to assure that sufficient

numbers of aircraft are available for training and pos-

sible reconnaissance flight operations at all times, and

to preserve the long-term operating condition of theentire fleet. An air command should also be able to

raise the level of preparedness when necessary.

Daily aircraft maintenance consists of flight-

mission-related inspections. The aircraft that perform

flight missions undergo a preflight inspection before

the first mission, whereas a turnaround inspection is

performed after each mission. In these inspections,

the aircraft are checked according to given specifica-

tions and the necessary replenishments are made.

The aircraft periodically undergo more elaborate

maintenance. The frequency of periodic maintenance

is based on accumulated flight hours. The main-tenance intervals as well as the number and con-

tents of periodic maintenance types depend on the

type of aircraft. The Hawk undergoes six different

types of periodic maintenance that are referred to

Maintenance activity Timing Maintenance unit Maintenance level

Preflight inspection Before first flight of the day Fighter squadron OR (Organizat ional-level)

Turnaround inspection After each flight Fighter squadron OR

Periodic maintenance

Type I Every 50 flight hours Fighter squadron OR

Type II Every 125 flight hours Air command’s repair shop IN (Intermediate level)Type III Every 250 flight hours Air command’s repair shop IN

Type IV Every 500 flight hours Depot-level repair shops DE (Depot-level)

Type V Every 1,000 flight hours Depot-level repair shops DE

Type VI Every 2,000 flight hours Depot-level repair shops DE

Failure repairs Unplanned, as required Fighter squadron/Air command’s OR/IN

repair shop

Table 1: The maintenance of aircraft is categorized into different maintenance levels. Each maintenance unit

performs the maintenance of a specific level.

as type I,II, ,VI maintenance. Unplanned mainte-

nance is performed in case of a failure. Some failures

are noncritical—the aircraft are repaired only during

the next periodic maintenance; however, some failuresmust be addressed immediately. A repair typically

involves diagnosing the defect cause and repairing or

replacing the failed component.

The above activities are categorized into different

maintenance levels and the aircraft maintenance units

are categorized according to their capability to per-

form the activities (Table 1).

The organizational-level (OR-level) maintenance

mainly includes turnaround and preflight inspections,

minor periodic maintenance such as type I main-

tenance, and minor failure repairs such as simple

component changes. The fighter squadron operatesthe OR-level maintenance unit, which is located in

the main air base of the air command during normal

conditions. Intermediate level (IN-level) maintenance

includes more complicated periodic maintenance and

failure repairs. The air command’s repair shop, which

is also located in the main air base, performs IN-level

maintenance. Depot-level (DE-level) repair shops,

which are not located within the main air base, handle

major periodic maintenance.

Conflict Situations

In a conflict situation, the aircraft are exposed to battledamage or may be destroyed during flight missions.

Any of the maintenance units may handle battle-

damage repairs during a conflict depending on their

capability and the type of repair. These repairs require


4/16


personnel with specific skills or materials that are

rarely needed during normal conditions.

The overall goals of aircraft maintenance change

as maintenance needs increase. The emphasis is onassuring the availability of aircraft in high-intensity

operations and the restoration of failed or damaged

aircraft to a mission-capable condition in the shortest

possible time. If necessary, periodic maintenance can

be temporarily suspended. However, aircraft perfor-

mance or reliable operation must never be reduced

severely. Changes in flight intensity and in the main-

tenance workload are difficult to anticipate because

they depend on the evolution of the conflict.

In a conflict situation, the FiAF air commands move

their units from the main air base to one or more

decentralized air bases to protect the air bases fromthe enemy. The organization of the air force and the air

commands remains largely the same. The decentral-

ized air bases are located in diverse areas that are typ-

ically sparsely inhabited; they utilize public roads as

a runway. They can typically support the flight oper-

ations and certain maintenance activities of a given

number of aircraft. The maintenance activities that are

allocated to an air base generally depend on the level

of infrastructure that is readily available at the loca-

tion. For example, a given air base may support all

activities that occur in the main air base of an air com-

mand during peacetime. This type of air base wouldhave facilities for all of the OR- and IN-level mainte-

nance. In turn, an air base may support the OR-level

only or merely the daily maintenance of the aircraft.

Decentralization changes the operating environ-

ment of the maintenance units if some of the infras-

tructure is inferior to that found in the main air

base. For example, the hangars in a decentralized

Air command Class 1 air base

Class 2 air base I

Class 2 air base II

Class 2 air base III

IN-level facility

Fighter squadronFacility for daily

maintenance

OR-level facility

Figure 2: In the simulation model, we divide an air command into an IN-level maintenance facility and a fighter

squadron that consists of OR-level maintenance facilities and facilities for daily maintenance.

base may provide less space for larger equipment.

Because of the changes, the durations of different

maintenance tasks can be increased. Decentralization

can also increase the logistic delays involved in trans-ferring materials, tools, and equipment between ware-

houses and air bases.

The Simulation Model

We constructed a simulation model that describes the

flight operations and maintenance of fighter aircraft

during normal and conflict conditions. The model

has three air commands, each with a specific num-

ber of aircraft. The aircraft carry out flight missions,

which bring about different maintenance needs. In

the simulation, maintenance is carried out in facilities

within the air commands and in one DE-level facility

that represents the DE-level repair shops of the actual

maintenance organization. The model input data dic-

tate the exact configuration of the flight operations

and maintenance and also govern whether normal or

conflict conditions are simulated. The model output

consists of aircraft availability and other performance

measures such as queuing times, resource utilization,

and attained flight intensity.

The Structure of the Air Commands

In the simulation model, the functional entities of

an air command include the fighter squadron andan IN-level maintenance facility that represents the

air command’s repair shop. We model two types

of maintenance facility in the fighter squadron, one

for flight-mission-related inspections and one for

other OR-level maintenance (Figure 2). The DE-level

maintenance facility in the simulation model operates

separately from the air commands.


5/16


Each air command can operate in up to four air

bases. The class 1 air base corresponds to the main

peacetime air base of an air command. It includes

a facility for daily maintenance as well as OR- andIN-level facilities. The other three, which are class 2

air bases, represent alternate bases that include facili-

ties for daily and OR-level maintenance. In a simula-

tion of normal conditions, an air command uses the

class 1 air base. However, in a conflict simulation, the

air command typically operates in both class 1 and

class 2 air bases. It has up to four facilities for daily

maintenance, four OR-level facilities, and an IN-level

facility.

The Simulation Logic of Flight Operations and

Maintenance NeedsFrom an individual aircraft perspective, the simula-

tion consists of daily flight operations, daily mainte-

nance, periodic maintenance, and failure and damage

repairs (Figure 3).

First, an aircraft waits in its home air base until

it is assigned to a flight mission. We determine the

number of aircraft required for a mission and the

duration of a mission randomly from suitable prob-

ability distributions (as we discuss later), and select

the required numbers of aircraft from the air bases

of the air command. As an additional criterion, we

select those aircraft that have waited the longest. If amission is generated when no aircraft are available in

the air command, the model records the mission as

noncompleted in the output.

Wait for

flight

mission

Failed,

damaged, or in

need of periodic

maintenance?

Yes

Facility for

daily

maintenance

OR-level

facility

IN-level

facility

DE-level

facility

NoCarry out

flight

mission

Figure 3: In the simulation, an aircraft waits in its home air base until it

is assigned to a flight mission. After completing the mission, it undergoes

any necessary maintenance activities and then returns to wait for the next

mission.

The model assesses the need for maintenance after

a flight mission. It does not include aborting a mis-

sion because of failure or battle damage because

the missions are described as time delays with nospecified objectives. We model periodic maintenance,

failure repairs, and damage repairs using different

maintenance activities that we characterize depend-

ing on the type of activity. Periodic maintenance is

performed on the basis of cumulative flight hours and

a predetermined maintenance interval. Time between

failures is measured in flight hours. To model battle

damage, pass-fail probabilities are used to determine

the type of damage sustained during the mission.

Failures are mutually exclusive, i.e., only one type of

failure can occur at a time. This also applies to differ-

ent types of battle damage. All types of maintenanceneeds can, however, be realized during a mission. For

example, an aircraft may sustain both battle damage

and failure.

Each type of maintenance activity is assigned to

a unique facility where the activity is always car-

ried out. Typically, lower-level activities, such as those

that correspond to type I periodic maintenance, are

assigned to OR-level facilities, and higher-level activ-

ities to IN- and DE-level facilities. If an aircraft

has multiple maintenance needs, maintenance is per-

formed in the highest-level facility required by the

activities. Aircraft are not transferred between facili-

ties. An aircraft that requires maintenance is imme-

diately transferred to the selected facility and will

remain unavailable for flight duty until the mainte-

nance has been completed.

Aircraft daily maintenance involves turnaround

inspections. All aircraft that return from a flight mis-

sion and do not require maintenance, or that return

from maintenance in one of the OR-, IN-, or DE-level

facilities, undergo a turnaround inspection. After an

inspection, an aircraft returns to wait for the next

flight mission. We did not model the preflight inspec-tions because test simulations indicated that their

effect on the performance measures of interest is

negligible.

The Simulation Logic of Aircraft Maintenance

The aircraft downtime consists of the maintenance

in OR-, IN-, and DE-level facilities. The simulation

model considers aircraft that are in a turnaround


6/16


Transfer to the

maintenance

facility

Wait for

material

delivery

Maintenance

Wait for

available

mechanics

Transfer to

home base

Figure 4: The total maintenance delay is the sum of the transfer delay to

the maintenance facility, possible waiting time for materials and person-

nel, duration of maintenance, and the transfer delay back to the home

base.

inspection to be available for flight duty. Thus, the

inspection time does not affect aircraft availability.

The time in maintenance in OR-, IN-, and DE-level

facilities involves the duration of the actual mainte-

nance and logistic delays as Figure 4 illustrates.

The transfer delays to and from a maintenance facil-

ity are specific to the facility. For example, the transfertime to a DE-level facility, which is not located within

the air command, is typically longer than the transfer

time to OR- or IN-level facilities.

A set of material requirements characterizes each

type of maintenance. The materials are modeled as

generic items, which can represent, for example, spare

parts, equipment, or tools. A maintenance activity

cannot begin until the necessary materials are avail-

able in the maintenance facility. The need for materi-

als is assessed when an aircraft arrives in a facility.

The maximum maintenance crew size and the proba-

bility distribution of the duration expressed in main-tenance man-hours, both of which are defined sepa-

rately for each activity, characterize the maintenance

delay. After a possible wait for materials, a mechan-

ics crew gathers to carry out the maintenance. If all

mechanics in the maintenance facility are busy, the

aircraft waits in a first-in-first-out queue until one

becomes available. The number of mechanics allo-

cated to the crew is, by default, the maximum crew

size. If the number of available personnel is less than

the maximum crew size, all available mechanics are

allocated. Finally, the net duration of the maintenance

is its duration in maintenance man-hours divided by

the allocated number of mechanics.

The logic for turnaround inspections differs slightly

from the maintenance in other facilities. The wait

for available materials and personnel and the

actual duration of maintenance determine the total

maintenance delay. Because the transfer delay is neg-

ligible, the model does not include it.

GUI

(VBA)

Simulation

parameters file

(Excel)

Simulation

results file

(Excel)

Input Input

Output

Initial state

file (Excel)

The simulation

model

(Arena)

Figure 5: We implemented the simulation model such that simulation

parameters are either fed through the GUI or through the simulation set-

tings file. The initial state of simulation is defined in the initial state file.The simulation output is written in a results file.

ImplementationWe implemented the simulation model using the

Arena software (Kelton et al. 1998); Arena is based

on the SIMAN language (Pegden et al. 1995) and

is intended for construction and analysis of dis-

crete-event simulation models. Figure 5 depicts the

implementation.

The simulation uses a GUI that we implemented

using Visual Basic for Applications (VBA) (Seppanen2000).

The model input data consist of the simulation

parameters and the initial system state. The simula-

tion parameters define characteristics of the air com-

mands, maintenance needs, and flight operations. The

initial state defines all the data needed to initialize

the system, e.g., the accumulated flight hours and the

location of each aircraft. The output includes aircraft

availability and various flight and maintenance statis-

tics. All external files of the model are Excel spread-

sheets; this makes it easy to manage several sets of

input data and to postprocess the model output.

Distribution Selection and Estimationof Simulation ParametersAn inherent part of the modeling is defining the

model input data as a function of the operating

conditions and air base structure. As we discussed

above, we used Hawk Mk51 unclassified data for the


7/16


simulation parameters of operations during normal

conditions. We also used these parameters as a start-

ing point for determining the parameters in conflict

operations.

Needs and Sources of Data

A FiAF reference data set, which contained either

complete statistics or averaged values of the quan-

tities in question, was available for definition of

the simulation parameters. It included data from

actual flight operations and aircraft maintenance dur-

ing time periods of one to six years. Throughout

the model construction, FiAF project-team members

cooperated with the authors on the model develop-

ment; discussions included the general principles of

flight operations, aircraft maintenance, and differentmodeling solutions. In addition, we convened two

expert panels. The first included FiAF maintenance

personnel. The second included two senior mainte-

nance professionals from a DE-level repair shop. In

open discussions, the experts provided their views on

specific input data and modeling assumptions.

We determined some of the model parameters (e.g.,

the parameters for the duration of daily flight and

maintenance activities, the number of maintenance

personnel in the maintenance facilities, periodic-

maintenance intervals, and transfer delays of aircraft)

easily from the reference data set. Because the actualdata on spare parts and material inventories are clas-

sified, we did not consider material handling. There-

fore, subsequent simulations do not consider material

handling delays.

We needed to estimate the parameters for other

items of input data from statistical data or extract

them based on the opinions of subject matter experts.

These items included:

—Probability density function (p.d.f.) for the times

between failures;

—Probabilities of sustaining each type of damage

during a single flight mission;

—P.d.f. for the duration of each type of periodic

maintenance, failure repair, and damage repair;

—Maximum size of the crew participating in each

type of periodic maintenance, failure repair, and dam-

age repair;

—P.d.f. for the times between flight missions;

—P.d.f. for the duration of a mission.

Distribution Selection and Parameters for

Periodic Maintenance

The reference data included values for the mean and

standard deviation of the duration of type I periodicmaintenance. Because type I maintenance consisted

of relatively straightforward tasks, we could not con-

sider durations longer than the mean duration to be

more likely than short ones. Therefore, we chose a

symmetric triangular distribution as the model.

We collected the maintenance statistics of the dura-

tions of maintenance types II–VI from the IN-level

repair shop of the Air Force Academy, which is the

FiAF primary training unit. This repair shop han-

dles both IN- and DE-level periodic maintenance,

unlike the repair shops in the air commands. Based

on statistical tests and on the histograms of the datasamples, we determined that the maintenance dura-

tions should be modeled with a distribution that has

a longer tail on the right side. The subject matter

experts in both panels agreed with this conclusion.

For type II and IV maintenance durations, three fam-

ilies of distributions, Weibull, Beta, and Gamma, pro-

vided the best fit according to the Chi-square and

Kolmogorov-Smirnov tests (Law and Kelton 2000).

We ultimately chose the Gamma distribution, which

also seemed suitable for maintenance types III, V, and

VI, because the different types of periodic mainte-

nance have many similar tasks.

In choosing the parameter values for the dis-

tributions (Table 2), we first considered type II

maintenance.

We calculated the initial values for the parameters

as maximum likelihood estimates based on the sta-

tistical data, and presented the resulting distribution

to the subject matter experts on the expert panels

and within the project team. They assessed how well

this distribution represented maintenance in the over-

all maintenance organization. Based on their feed-

back, we adjusted the value of the scale parameter

and the constant in the distribution expression repre-

senting the minimum maintenance duration upwards

in the final choice of parameters; we left the shape

parameter unchanged. We also generalized the shape

parameter in type II maintenance to all other main-

tenance types from III to VI. Finally, we selected the

scale parameters and minimum maintenance dura-

tions such that the ratios of the standard deviation


8/16


Maintenance

type Facility Crew size Duration (maintenance man-hours)

I OR-level 4 Tria8 38 68 (Triangularly distributed

with a minimum value of 8 hours,

mode of 38 hours, and maximum of

68 hours)

II IN-level 4 200+Gamma2 50 (Gamma

distributed with shape parameter

equal to 2 and scale parameter

equal to 50)

III IN-level 4 500+Gamma2 125

IV DE-level 5 1,300+Gamma2 300

V DE-level 5 1,500+Gamma2 300

VI DE-level 6 2,000+Gamma2 500

Table 2: We determined the crew sizes and the p.d.f.s of the maintenance

durations for periodic maintenance using statistical data and expert opin-

ion. The assignment of maintenance types to maintenance facilities was

readily available in the reference data.

and distribution mean remained approximately the

same as for type II maintenance.

No data were available on the sizes of maintenance

crews that actually perform the maintenance. There-

fore, we determined the crew sizes with the help of

the subject matter experts.

Distributions and Parameters for Failure Repairs

We used two failure types for modeling all failures

(Table 3).The first type represents the failures that are com-

monly repaired at the OR-level, and the second the

failures that are repaired at the IN-level. Because

detailed knowledge on failure statistics was not

available, we assumed times between failures to be

exponentially distributed. The mean times between

failures were directly available from the reference

data set. For durations of both types of failure repairs,

the reference data included the mean and standard

Duration

Failure Time between Crew (maintenancetype Facility failures (flight hours) size man-hours)

1 OR-level E xp(18.6) (Exponentially 3 4+ gamma2 1

distributed with a

mean of 18.6 hours)

2 IN-level Exp(43.3) 4 78+ gamma2 11

Table 3: The parameters for failure repairs included the assignment to

maintenance facility, time between failures, crew size, and duration.

deviation. We chose the Gamma distribution to rep-

resent the repair durations. We further set the shape

parameters of the distribution equal to those of the

periodic maintenance. We selected scale parametersand minimum maintenance durations so that the

ratios of the mean and standard deviation remained

the same as in the distributions for periodic main-

tenance because both types of maintenance involve

similar tasks and are performed by the same repair

shops. Again, we selected crew sizes based on expert

opinion.

Flight Mission Characteristics

Finally, we derived the parameters for the flight oper-

ations from the statistics of all the Air Force Academy

flight missions during one year. Based on the refer-ence data, we could model times between flight mis-

sions using an exponential distribution with a mean

of 30 minutes. Flight duration, on the other hand, fol-

lows a normal distribution with a mean of 45 minutes

and standard deviation of 12. We assumed that a sin-

gle aircraft is required in each mission.

Model ValidationWe validated the simulation model by comparing its

output with actual performance data. We also con-

ducted a sensitivity analysis of the impact of inputdata to key performance measures of the model and

let subject matter experts assess the underlying mod-

eling assumptions and simulation results.

Comparison to Actual Performance Data

We chose to compare the actual and simulated air-

craft availabilities because availability is the key

performance measure in actual maintenance-related

decision making. The three-month moving average of

availability during a period of four years was avail-

able for the validation. The simulation model con-

tained 51 aircraft divided among three air commands

operating in one class 1 air base. We initialized the

accumulated usage hours of the aircraft with a set of

values that was available but that did not relate to

the situation in the data. We used a warm-up period

of six months to erase the results from the initial

transient phase and to reach the steady state of the

simulation. We compared the simulated availabilities


9/16


1.0

0.9

0.8

0.7

0.6

0.50 1 2 3 4

Time (years)

A i r c r a f t a v a i l a b i l i t y

Simulated

Actual

Figure 6: The simulated availabilities of 10 independent replications are

very close to the actual Hawk Mk51 availability during a four-year time

period. The figure is based on the three-month moving averages of both

actual and simulated availabilities.

from 10 independent simulation replications with the

actual availability data (Figure 6).

The average availability that the model predicted

was approximately 0.72; however, the data showed

an average availability of approximately 0.70. Some

of the difference is because the input data did not

consider material handling as we discussed above. In

addition, the simulation did not seem to reproducea drop in the actual availability just after the second

year because of additional modification work that the

aircraft underwent during the time period. We did

not consider the modification work in the input data

because our purpose was to describe average flight

operations and maintenance. Otherwise, the simula-

tion seemed to reproduce the actual availability well.

Sensitivity Analysis

We can use sensitivity analysis to assess how changes

in input data affect simulation output. The analysis

implies that the model is valid if the simulation out-

put is affected in the same way as the actual system

would be under corresponding changes. Because sets

of reference data from a wide range of operating con-

ditions are not generally available for such analysis,

the sensitivity results are frequently assessed subjec-

tively by both model constructors and subject matter

experts.

Therefore, we conducted the sensitivity analysis by

examining which of a set of 12 input data items

affected the average aircraft availability significantly

in the previously described simulation of the four-year time period. We used design of experiments

(Montgomery 2001) for the analysis and devised a

212−4 fractional factorial design involving 256 simu-

lation runs to estimate the effects of the items. In

the design, we set the simulation parameters corre-

sponding to the items to either −1 or +1 level as

Table 4 describes, but left other simulation parameters

unchanged.

We should also note that in Table 4 the num-

ber of mechanics in the maintenance facilities repre-

sents the effective amount of personnel resources, i.e.,

all mechanics are capable of performing all requiredmaintenance work within the facility. We also con-

sidered flight intensity in terms of the time between

flight missions, but did not include mission duration

in the design because the effects of both variables

are very similar and the exclusion of either variable

helped to limit the required number of simulation

runs. We combined failure types 1 and 2 into the over-

all mean time between failures for the same reason.

Table 4 lists the 95 percent confidence intervals for the

changes in average availability due to the changes in

the input data items. The effect of each item is statis-

tically significant. We selected the −1 and +1 levels

in the design so that the −1 level would presumably

result in lower availability and the +1 level in higher

availability. Because the effects for all items are posi-

tive, the results are consistent with our initial expecta-

tions. The time between flight missions has the largest

effect because flight intensity governs the amount of

all maintenance needs. The model is also sensitive to

the number of DE-level mechanics and the durations

of type IV, V, and VI periodic maintenance that the

DE-level facility performs. The subject matter experts

expected this because many aircraft underwent com-

plex periodic maintenance during the observed time

period. The number of OR-level mechanics and the

duration of type I periodic maintenance performed in

the OR-level facilities have the smallest effect because

the number of mechanics was high relative to the

maintenance needs. Decreasing the number of person-

nel from the +1 to −1 level did not congest the facil-

ities and showed little effect on aircraft availability.


10/16


95 percent confidence

interval of the change in

Input −1 level +1 level average availability

Number of mechanics in maintenance facilities

Daily maintenance 13 17 00065±00011

OR-level 5 7 00016±00010

IN-level 13 17 00081±00009

DE-level 22 28 01578±00039

Time between flight missions Exp(26) Exp(34) 02182±00038

Duration of periodic maintenance

I Tria103 418 733 Tria57 342 627 00018±00007

II 220+gamma22 50 180+gamma18 50 00071±00011

III 550+ gamma22 125 450+gamma18 125 00075±00014

IV 1,430+ gamma22 300 1,170+ gamma18 300 00518±00019

V 1,650+ gamma22 300 1,350+ gamma18 300 00189±00019

VI 2,200+ gamma22 500 1,800+ gamma18 500 00583±00014

Overall mean time between failures 11.7 14.3 00138±00018

Table 4: We conducted a sensitivity analysis by examining the effects of 12 items of input data on simulated

aircraft availability. The effect of each item was statistically significant. Because the changes in availability were

positive for all items, the directions of the effects were also consistent with our initial expectations.

Some interaction effects of two input data items

were also significant. The change in aircraft availabil-

ity resulting from a change in one of the correspond-

ing items depends on the level of the other item.

Therefore, we cannot interpret the effects of single

variables literally. They indicate the relative impor-

tance of the items, however. For brevity, we chose not

to present the interaction effects in this paper.

Expert Validation

In addition to assessing the results of the sensitiv-

ity analysis, subject matter experts were also involved

in other aspects of model validation. The two expert

panels discussed the underlying modeling assump-

tions and output of preliminary versions of the

model, and the FiAF project-team members repeat-

edly addressed both modeling solutions and simula-

tion results.

In the final phase of model construction, we

arranged two user training sessions to introduce both

the model and basic principles of the simulation

approach to FiAF maintenance designers. The train-

ing was necessary because the designers would use

the model independently at a later time. We also

saw the training as an opportunity to further validate

the model. We asked the designers to give feedback

on any of its features. Thus, they contributed to the

validation both as end users and as subject matter

experts.

Because the underlying system of flight activities

and aircraft maintenance is large and multifaceted,

we discussed many issues of wide-ranging scope with

the experts during model validation, e.g., the forma-

tion of maintenance teams and the sequence of activ-

ities during individual maintenance tasks with the

second expert team. We addressed higher-level issues

such as the nature of conflict-time operating condi-

tions or appropriate performance measures of main-

tenance primarily with the project team. Overall, the

need to involve experts with different backgrounds in

model construction and validation was apparent.

In meeting with the experts who were not mem-

bers of the project team, the team members took

part in introducing the background and objectives

of the project. Our impression was that this greatly

helped to make the experts receptive to the simu-

lation approach and committed them to improving

the model. In the meetings, we used the guidelines

of a structured walk-through that Law and Kelton

(2000) describe, and allowed as much time as neces-

sary to discuss the modeling assumptions and simula-

tion results. We also took great care to devote enough

time to introducing the basics of simulation model-

ing to the experts. In the user training sessions, the


11/16


designers were able to explore what the model does;

thus, they gained a far clearer view of its functionality

than they would in a standard classroom presenta-

tion. These actions allowed us to be confident that theassessments the experts ultimately gave were based

on a sufficiently detailed understanding of the model

and the simulation approach.

To summarize, the experts contributed to the val-

idation in two ways. First, they helped us to adjust

the simulation model and its input data during model

construction. Second, they helped to confirm that the

final model described the flight operations and air-

craft maintenance with enough accuracy to make it

sufficient for practical use.

Practical Use of the Model

The simulation model provides FiAF with a quan-

titative analysis tool for the flight operations and

maintenance of its aircraft fleet. In normal operating

conditions, the model can help maintenance design-

ers to allocate appropriate personnel and material

resources for an exercise with high flight intensity.

While this is important to the designers, their ultimate

concern is to learn how to maximize the conflict-time

operational capability of the fleet. As an example of a

conflict-related application of the model, we cooper-

ated with the FiAF project team to simulate a scenarioin which we examined the aircraft periodic mainte-

nance policy. The simulation provided information on

the number of aircraft that can be expected to be avail-

able and the maximum number of flight missions that

can be performed during the conflict.

The Conflict Scenario

The conflict scenario we considered involved four dif-

ferent phases. In the first phase, the level of readi-

ness is increased resulting in higher flight intensity.

In phase two, the flight intensity is further increased;

each air command moves to operate from the nor-

mal main air base into four decentralized air bases

and carries out flight operations and maintenance

24 hours a day. In the third phase, there is actual

conflict in the form of aerial battles and the aircraft

begin to sustain battle damage. In the fourth and final

phase, the flight intensity is decreased as the conflict

approaches an end.

In the scenario, we examined the aircraft periodic

maintenance policy. The maintenance facilities can

become congested at some point during the conflict

because of the increased flight intensity and the needfor battle-damage repairs. The periodic maintenance

can then be temporarily suspended to guarantee that

a sufficient number of aircraft are available for flight

missions. We assume that the decision of whether to

suspend periodic maintenance is made at the begin-

ning of each phase of a scenario. If the maintenance

is suspended, it will not be continued in any of

the remaining phases. However, any ongoing mainte-

nance will be completed. We simulated four alterna-

tive policies:

(1) All periodic maintenance is suspended at the

beginning of the first phase.(2) All periodic maintenance is suspended at the

beginning of the second phase.

(3) All periodic maintenance is suspended at the

beginning of the third phase.

(4) The periodic maintenance is not suspended

during the scenario.

Scenario Input Data

The scenario input data are based on the previously

described set of simulation parameters. We modeled

the different phases of the conflict by changing the

simulation parameters as Table 5 describes. All otherparameters remain unchanged.

The description of battle damage in the third and

fourth phases is an essential part of the simula-

tion. Because no data were available for estimat-

ing the battle-damage parameters, we determined the

parameters with the help of the FiAF project team.

We assumed three types of damage in the scenario

(Table 6).

Number of Daily duration of

operative flight operations Time between

Duration air bases per and maintenance flight missions BattlePhase (days) air command (hours) (min.) damage

1 30 1 8 Exp(24) no

2 30 4 24 Exp(20) no

3 10 4 24 Exp(20) yes

4 30 4 24 Exp(40) yes

Table 5: We modeled the conflict scenario by changing a set of simulation

parameters according to the different phases of the conflict.


12/16


Probability Duration

Damage during a single Crew (maintenance

type Facility flight mission size man-hours)

1 OR-level 0025 4 2.6+ gamma2 07

2 IN-level 0015 4 6.5+ gamma2 18

3 DE-level 001 4 130+ gamma2 357

Table 6: We determined the battle-damage parameters for phases 3 and 4

with the help of FiAF project-team members.

We obtained the initial state of simulation for the

scenario as the final state of a long simulation from a

suitable but artificial initial state.

Simulation Results

We conducted 40 independent simulation replicationsfor each alternative policy. Figure 7 illustrates the

development of aircraft availability averaged across

the replications.

The most critical phase of the conflict is the actual

combat phase, during which the availability decreases

rapidly. If policy 4 is employed, the periodic main-

tenance will use up the maintenance resources and

delay the battle-damage repairs. The availability con-

sequently drops to as low as 0.4; this means that it is

very unlikely that the air commands could meet all

their operational goals. Policies 1 and 2 produce the

highest availability.

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

00 10 20 30 40 50 60 70 80 90 100

Time (days)

A i r c r a f t a v a i l a b i l i t y

Phase 1 Phase 2 Phase 3 Phase 4

Policy 4

Policy 3

Policy 2

Policy 1

Figure 7: Policies 1 and 2 maintain a clearly higher aircraft availability

than policies 3 and 4; this indicates that some periodic maintenance must

be suspended during the conflict.

We concluded that some of the periodic mainte-

nance must be suspended to maintain operational

capability, if maintenance resources, battle damage,

and flight intensity are as the scenario assumed. Itseems that the maintenance policy should be changed

before the actual combat phase. Although some types

of periodic maintenance can be performed in the early

phases, postponing the change of policy can prove

problematic in practice because the phase durations

are not known with certainty. We should also note

that suspending periodic maintenance affects the fail-

ure rate of the aircraft. The impact of periodic main-

tenance on the failure rates of aircraft is a challenging

topic that requires further research. Because no sta-

tistical data on this dependence were available and

the nature of the maintenance policy very preemp-tive, we kept the failure rate unchanged in the simu-

lations. The simulation results therefore represent the

best-case benefit of suspending the maintenance.

We also considered the daily number of completed

flight missions. If mission requests arrive with high

intensity, the air commands may not be able to

respond to all of them because of aircraft unavailabil-

ity (Figure 8).

We averaged the results across 40 independent

replications. Based on the results, it would again be

beneficial to suspend periodic maintenance at some

1.0

0.9

0.8

0.7

0.6

0.5

0.4

D a i l y p r o

p o r t i o n o f c o m p l e t e d f l i g h t m i s s i o n s

0 10 20 30 40 50 60 70 80 90 100

Time (days)

Phase 1 Phase 2 Phase 3 Phase 4

Policy 4

Policy 3

Policy 2

Policy 1

Figure 8: The daily proportion of completed flight missions during the con-

flict scenario indicate, as the availability results did, that operational

capability is best maintained with policies 1 and 2.


13/16


point during the conflict. At the same time, the effect

of each policy on the flight operations became clearer.

The proportion of mission requests to which there

was no response during the busiest phase of theconflict was approximately 10 percent with policies

1 and 2. The ratios were 25 percent and 40 percent,

respectively, for policies 3 and 4.

The above results show that the consideration

of maintenance policies is essential to maintain-

ing the operational capability in a conflict situation.

A maintenance organization that is sized for nor-

mal conditions will have difficulty handling increased

maintenance needs even if the battle damage prob-

abilities are reasonably small. Additional simula-

tions could assess the amount of additional resources

required during a conflict, which maintenance activ-ities should be suspended, and when they should

be suspended. Overall, the model provides valuable

information to support the decision making that is

involved in devising contingency plans for aircraft

maintenance.

Model Construction ChallengesWe faced several challenges in constructing the

model. The primary one was scarcity of data. No sta-

tistical data were available for modeling elements of

the flight operations and aircraft maintenance suchas battle-damage probabilities and repair-time distri-

butions. We found that subject matter experts from

different units and organizational levels needed to

be involved in determining the corresponding model

components. The experts provided their views on

the issues at hand, and the authors explained the

benefits and drawbacks of incorporating a given mod-

eling solution to them. In addition to being essen-

tial in determining given modeling assumptions, the

communication with the experts helped us to refine

our overall view of the flight operations and aircraft

maintenance.

Because we designed a number of components in

the model with the help of subject matter experts,

we needed to carefully assess whether the overall

model validly described the usage and maintenance

of the aircraft. We did extensive sensitivity analyses

to quantify how the output of the model would be

affected by departures from modeling assumptions

or input data. The analyses included the examination

of the structural assumptions associated with several

key components of the model, e.g., the logic of air-

craft maintenance. We also tested the effect of chang-ing the distributions of maintenance durations. As

Table 4 summarizes, we used an experimental design

to examine the effects of input data. We presented

results of our analyses to the experts and allowed

them to use the model. Therefore, we are confident

that we captured the views of the experts correctly in

the final model.

Another challenge we faced was confidentiality

of data. FiAF representatives could not provide the

authors with access to highly classified information.

However, some of this information was necessary to

model scenarios that FiAF ultimately wished to con-sider. For example, it included the contingency plans

on maintenance policies and anticipated flight intensi-

ties, battle-damage rates in various conflict scenarios,

and the statistics from the normal time maintenance

of F-18s in the air commands.

To overcome this difficulty, we implemented the

model such that the choice of input data fully governs

its operations logic. For example, we did not hard-

code the conflict-time maintenance policies. Instead,

we modeled these policies by selecting suitable values

for a set of input parameters. We could isolate con-

fidential information for separate handling by FiAF.

Naturally, determining the structure of the model

did require some discussions on confidential issues

between the authors and the FiAF project team. The

team members described in general terms what the

model should be able to do. Based on their descrip-

tion, we implemented the corresponding model com-

ponents and revised them repeatedly until the model

was satisfactory. Thus, we managed to guarantee that

the model had the right functionality for considera-

tion of any relevant normal or conflict-time scenarios

although we could not use classified informationdirectly.

ConclusionsThe practical use of the simulation model implies

that it offered FiAF a valuable aid in improving

maintenance-related decision making. We first intro-

duced the model and initiated the project in FiAF


14/16


headquarters. In the early phases, the FiAF project-team members were the primary users. They applied

the model to produce short-term forecasts of aircraft

availability. They also used the model to analyze theaccumulation of maintenance needs in some of thelarger exercises and to assess the resource require-ments of a smaller group of aircraft that participated

in a combined exercise with Air Forces from othercountries.

We delivered the model to the air commands aswell as to other units of FiAF. The units applied themodel to analyze the effects of a readjusted periodic

maintenance policy for the F-18s.The project also served as a pilot study to advance

the application of simulation techniques in aircraft

maintenance, air base logistics, and other areas atFiAF. For example, shortly after the completion of

the simulation model that we discussed in this paper,FiAF cooperated with the Finnish Army on a simula-

tion project to analyze the maintenance system of theArmy’s new transport helicopters.

The model is suitable for training purposes.Because it is GUI-based and does not require detailedknowledge of the underlying simulation software, it

is useful in classroom demonstrations or individually by trainees. However, users still need some time to

acquaint themselves with the model. Therefore, train-

ing has thus far been limited to situations where theschedule allows a thorough introduction to the model

functionality. Some of the graduating students of theAir Force Academy have applied the model for sim-

ulation analyses in their theses.The process of constructing the simulation model

has also brought indirect benefits. The subject matter

experts involved in the construction were required todescribe the organization and interaction of given ele-

ments of the maintenance system. The FiAF project-team members and some of the other experts said thatthis involvement helped them to obtain new insights

into the system. They regarded this as an additionalproject benefit.

In the future, FiAF will use simulation to design andcontrol flight operations and aircraft maintenance. Its

research directions include the simulation of smallerelements of the maintenance system, e.g., a singledecentralized air base. We have also begun to pur-

sue the scheduling of aircraft periodic maintenance byusing simulation-based optimization techniques.

AcknowledgmentsWe gratefully acknowledge the help of the people at FiAFwho were involved in this project. In particular, we thankMajor Riku Lahtinen for his invaluable support to the entireeffort.

References

Balaban, H. S., R. T. Brigantic, S. A. Wright, A. F. Papatyi. 2000.A simulation approach to estimating aircraft mission capablerates for the United States Air Force. J. A. Joines, R. R. Barton,K. Kang, P. A. Fishwick, eds. Proc. 2000 Winter Simulation Conf.,Orlando, FL, 1035–1042.

Ciarallo, F. W., R. R. Hill, S. Mahadevan, V. Chopra, P. J. Vincent,C. S. Allen. 2005. Building the mobility aircraft availability fore-casting (MAAF) simulation model and decision support sys-tem. J. Defense Model. Simulation Appl. Methodology Tech. 2(2)57–69.

Kang, K., K. R. Gue, D. R. Eaton. 1998. Cycle time reduction fornaval aviation depots. D. J. Medeiros, E. F. Watson, J. S. Carson,M. S. Manivannan, eds. Proc. 1998 Winter Simulation Conf.,Washington, D.C., 907–912.

Kelton, W. D., R. P. Sadowski, D. A. Sadowski. 1998. Simulation with Arena. McGraw-Hill, Boston.

Kladitis, P. E., J. P. Worden, R. H. Searle. 2007. Estimating and mod-eling inherent availability of the B-52H using SIMPROCESS.Proc. 2007 U.S. Air Force T&E Days, AIAA , Destin, FL.

Law, A. M., W. D. Kelton. 2000. Simulation Modeling and Analysis,3rd ed. McGraw-Hill, Boston.

Montgomery, D. C. 2001. Design and Analysis of Experiments, 5th ed. John Wiley & Sons, New York.

Pegden, C. D., R. E. Shannon, R. P. Sadowski. 1995. Intro-duction to Simulation Using SIMAN , 2nd ed. McGraw-Hill,

New York.Pohl, L. M. 1991. Evaluation of F-15E availability during opera-

tional test. B. L. Nelson, W. D. Kelton, G. M. Clark, eds. Proc.1991 Winter Simulation Conf., Phoenix, AZ, 549–554.

Rodrigues, M. B., M. Karpowicz, K. Kang. 2000. A readiness anal-ysis for the Argentine Air Force and Brazilian Navy A-4 fleetvia consolidated logistics support. J. A. Joines, R. R. Barton,K. Kang, P. A. Fishwick, eds. Proc. 2000 Winter Simulation Conf.,Orlando, FL, 1068–1074.

Seppanen, M. S. 2000. Developing industrial strength simulationmodels using Visual Basic for Applications (VBA). J. A. Joines,R. R. Barton, K. Kang, P. A. Fishwick, eds. Proc. 2000 WinterSimulation Conf., Orlando, FL, 77–82.

Upadhya, K. S., N. K. Srinivasan. 2004. Availability of weapon sys-tems with air-attack missions. J. Defense Model. Simulation Appl.

Methodology Tech. 1(2) 111–121.

Major Riku Lahtinen, Armaments Division, FiAF

Headquaters, PO Box 30, 41161 Tikkakoski, Finland,

writes: “I have acted as the head of the project

team of the Finnish Air Force (FiAF) and as the

primary contact between FiAF and the authors in

the project that is described in the paper ‘Improv-

ing Maintenance Decision Making in the Finnish Air


15/16


Force Through Simulation.’ With this letter, I’d like

to verify that the paper gives an accurate account of

the details of the project and the benefits that it has

provided.“The Armaments Division, together with the Mate-

rial Command, carry out the development of the air-

craft of FiAF to meet operational and airworthiness

requirements. The division also plans and coordinates

the research efforts that are undertaken to support

the development. Our task is to guarantee that the

aircraft are safe, powerful, and properly equipped

at all times: in training, in increasing number of

international operations, as well as in all degrees of

readiness. This requires us to continuously improve

the quality of the maintenance processes. Since the

resources are not unconstrained, the quality must bedeveloped by considering the efficiency of the pro-

cesses as well.

“We had ongoing collaboration with the Systems

Analysis Laboratory, Helsinki University of Technol-

ogy, and asked them to propose how we could study

the effect of maintenance on aircraft availability. The

research team of the Systems Analysis Laboratory

first conducted a pilot simulation study where the

operations of a single airbase were considered. We

regarded this pilot study as a success and decided

on requesting a model of the maintenance of the

entire fighter aircraft fleet. The specifics of the result-

ing model are described in the paper.

“The benefit of the simulation model has been

unquestionable. It has given us a sophisticatedapproach to analyze things with a less labored way

than earlier. Besides the Armaments Division, other

units have benefited from the model in assessing

proposed improvements to maintenance practices.

Although the details of these analyses are mostly con-

fidential and can not be elaborated here, I can state

that they are significant parts in the development of

aircraft maintenance in FiAF.

“Another result of the project has been the emer-

gence of fresh conversation and exchange of ideas

between different branches of aircraft maintenance.

The people with different backgrounds were exposed,in a positive sense, to each others’ viewpoints dur-

ing the discussions that went on in the project. I can

say with confidence, that my understanding of the

different branches has improved and I truly believe

that this is the case for a number of other people.

Since these people are our most important assets, the

impact of the project has been an important one.

“Our experiences from the project have been posi-

tive and we see simulation applications as an integral

part in maintenance-related decision making in the

future.”


16/16

improving maintenance decision making in the finnish air force through simulation

Documents