emj syseng427 project final rev5

8/3/2019 EMJ SysEng427 Project Final Rev5

http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 1/13

Engineering Management Journal


EMJ, 2010 – 12(34): 567-890 – Online ISSN: 1234-5678

Risk Quantification for the Design of Boeing 777 Engine Nacelles

Steven Broussarda, Josh Goldschmid

b, William Harkness

b, Damon Slaughter

c

a The Boeing Company ─ St. Louis, MO

P.O.Box 516, St. Louis, MO 63166, [email protected] bThe Boeing Company ─ Everett, WA

9801 27th Ave W, Everett, WA 98208-4316, [email protected], [email protected]

cUnited States Army-- Fort Leonard, MO

1805 Cooley Avenue, Fort Leonard Wood, MO, 65473, [email protected]

Abstract ─ Identifying and mitigating project risks

are crucial steps in managing successful complex

projects. This article proposes the application of a

Risk Mitigation Quantification methodology

quantify, analyze, and mitigate project risks.Popularity and Success Rubric charts were

developed organize, grade, and communicate our

expectation of the product evaluated. The rubrics

essentially help determine how successful the

product will meet the criteria.

The payoff for management and engineerswill be a process that implements historical risk

mitigation steps with known success.The risk mitigation quantization methodology

proposed enables effective planning to mitigate

any consequences from occurring and thus

providing a safe and reliable product that complies

to high-level government regulations as well as

minimizing unnecessary work, reducing overall

program cost, and keeping programs on schedule.

Index Terms ─ Risk Mitigation Quantification,

Project Management, Risk Severity, EngineNacelles, Design, Risk Rubric.

I. INTRODUCTIONIdentifying and mitigating project risks are

crucial steps in managing successful complex

projects. This article proposes the application of a

Risk Mitigation Quantification and Mitigation

methodology quantify, analyze, and mitigateproject risks.

Imagine the risk of developing a new product,

taking a leap of faith into the design world and

hoping the risk pays off in the end. How would

that risk be handled? In the past, one would findthe smartest person in the room and ask them what

they think the risks of the new product would be.

Unfortunately, not one will have a perfect answer

as complex design escapes human abilities.

Collaborative efforts across engineeringdisciplines such as propulsion, structures,materials, project management, aerodynamics, and

safety are necessary actions to identify and

quantify risk and risk mitigation items. There are

strides being made in risk and the end-product will

help remove some of the ambiguity from the risk

world. Grantham Lough, Stone, and Tumer have

developed a process called The Risk in Early

Design (RED)1 which uses historical data to

identify risk as early in the design phase a possible

and assign a likelihood and consequence to the

risk. But, how would one mitigate the risk?Risk Mitigation Quantification is feeding

off a similar idea. At this point a process will not

be developed, however the first step is gathering

the data needed to create a process. Risk

Mitigation techniques are used in every program,so the idea is to gather as many risk mitigation

techniques as possible, document when the

technique was used, how it was used, how often it

mailto:[email protected]


















was used. Who assigned the technique? Who is

responsible for the technique? Most important

though, was the technique successful for reducing

risk, and by how much?

Management will benefit from this

method the most. With detailed risk data and

mitigation items will provide a manager thecapacity to make intelligent decisions.

As a secondary outcome of using this

strategy management funding will potentially go

further. Earned Value Management has two key

objectives that need to be addressed.

Encourage contractors to use effective

internal cost and schedule management

control system

Provide the customer timely data

produced by those systems for

determining contract status2.

Earned value is a metric that is used to reportto the customer on progress. Technicalperformance measures (TPM) and the risk are

always key program tracking points that must be

reported to the customer. By using this method,

management can quantify what was done in the

past. Estimations for tracking the risk on the

program and meeting mitigation goals will have

more merit and historical data to back up the

approach.

The first step is to gather the historical

data. The intent of this paper is to take the risk

data and provide a means of showing how mucheach mitigation step has reduced the overall risks,

using the example of the design of a Boeing 777

engine nacelle. The function of the nacelle is to

provide housing for the airplane engine, fuel lines,

and electronic equipment. In simple terms, the

engine nacelle is analogous to the hood of a car.

Cylindrically shaped, the nacelle forms the outer,

aerodynamically smooth covering for the engine

(see figure 1).

The design of the Boeing 777 nacelles is

extremely complex and takes approximately eight

(8) months to complete. The nacelle designcomplexity often surpasses human ability in many

aspects- the calculation, modeling, and simulation

of considerable amount of design and analytical

data provides many opportunities for considerable

consequences. Thus, an understanding of

identifying and quantifying risk and risk

mitigation items is a prerequisite for making

intelligent decisions.

Figure 1: Boeing 777 Engine Nacelle

II. RISK MITIGATION

QUANTIFICATIONRisk quantification is a process of the

independent quantification of probability and

severity associated with a particular product,

service or scenario. The product of the probabilityand severity provides a risk value needed to make

decisions as to what should be done about them.

Risk mitigation quantification, however,

involves the quantification of reduction in the

probability and the reduction in severity. Before

we could quantify risk mitigation, we identified

the risk elements associated with the design of the

Boeing 777 engine nacelles, determined the

likelihood and severity values for each risk, and

determined mitigation strategies.

A. Identification and quantification of riskelements of the 777 nacelle designSeveral tools have been used to identify ten (10)

risky scenarios shown in Figure 2 (larger pictures in

Appendix A):

Failure Modes and Effects Analysis

(FMEA)

Physical inspection of components and

systems

Logic models (event trees and fault trees)

Engineering and statistical tools and analyses havebeen used to estimate the likelihood and

consequences in the event that the risk is

materialized. The tools used for quantifying

probabilities/likelihood were:

Failure analysis

Failure rate data and probability analysis




Subject Matter Expert (SME) opinion and

subjective probability analysis

Bayesian probability analysis

Human error analysis

For each risk, the relative likelihood and severity of

consequence lead to the calculation of the risk factor product below:

R = P x S (eq. 1)

Where R is the risk score (chance of an identified

consequence occurring), S is severity of harm

(consequence), P is the likelihood of the occurrence

of that harm.

According to Figure 2, our top four biggest

concerns in the design of the nacelles are:

1. Excess weight2. Excess production cost

3. Non-compliance to regulations

4. Safety requirements not met

Consequences are large if these risk items occur.

Fortunately, there are mitigation strategies (Figures

3, 4, and 6) to reduce the probability and severity of

the consequences. The mitigation steps are

allocated to the risk items to support the goal of a

safe nacelle design.

Figure 2: 10 Risk Elements and Risk Score

B. Waterfall chart and risk mitigation of the

777 nacelle design

For each essential safety planning elements,the SMEs were asked to use their experience

and judgement to identify risk mitigation

strategies and to rate what they believe theaverage reduction in probability and severity

of a disaster. Shown in Figure 3 and 4 is a risk

management plan (the full management plancan be seen in the appendix) with job roles

assigned to each mitigation step; the risk

waterfall chart for a specific risk mitigationitem to reduce the likelihood of the

consequences. Activities are introduced to

progressively reduce risk by increasing

knowledge and confidence in the re-design of the 777 nacelles. Assuming each activity is

successful, risk is shown to decrease from

high to low as each activity is completed.Unfortunately, these quantitative estimates of

risk reduction are subjective basing on

SMEs’s estimates; however with the props of the cross-functional team’s engineering andmanufacturing immeasurable experience and

sophisticated skill, the waterfall chart

communicates a well thought out plan with

each risk assigned to a responsible engineer ormanager. If an activity fails, the chart would

be modified to show increasing risk andadditional activities.

Risk Title:777 Nacelle Design Change0

25

50

75

100

H

i

g

h

M

o

d

e

r

a

t

e

L

o

w

Q3Q1 Q2

P e r c e n t L i k e l i h o o d t h a t t h e 7 7 7 N a c e l l e s D o N o t F a i l O v e r t h e L i f e - C y c l e

1

2

3

7

6

5

4

8

9

10

11

12

13

15

16

17

18

19

20

21

Figure 3: Nacelle Risk Management Plan

Figure 4: Risk Identifier and job role assignment for

the waterfall chart (Figure 3)

C. Risk Plot of the 777 nacelle designThis plot below shows the 10 identified risk

values plotted with potential severity of




consequences on the x-axis and relative likelihood

of consequences on the y-axis.

Figure 5 : Nacelle Risk Plot

D. Assessing the 777 nacelle designBy pulling out the data from the nacelle design

one can begin the process of quantifying the risk

mitigation effort. Figure 6 shows some of the key

points from the risk mitigation effort. The figure

can also be viewed in the appendix.Mitigation Effort.

What does themitigation effortdescribe?

Does it reduce likelihood? If

so, by how much?

Does it reduce consequence? If

so, by how much?

Project proposal for re-design Propose design Yes, 2 No

Integrated, multi-discipline, IntegratedProduct/Process (IPP) team formed

Form team Yes, 2 No

Develop detailed project managementaspects

Develop plan Yes, 2 No

Develop detailed work statement Develop strategy Yes, 4 No

Initial studies and cross-functional researchincluding economics

Conduct research Yes, 4 No

Develop plan for concept design Develop plan Yes, 2 No

Early involvement with leadership andregulatory authority

Assess progress Yes, 10 Yes, 10

Identify applicable regulations andrequirements

Identify limits Yes, 6 Yes, 6

Define plan to meet regulatory and safetyrequirements

Define plan Yes, 6 Yes, 6

Validate certification process Validate process Yes, 4 No

Validate design process Validate process Yes, 6 No

Design the nacelle per the requirements Design product Yes, 2 No

Perform careful materials selection Select material Yes, 4 Yes, 4

Validate build process Validate process Yes, 4 No

Develop prototype for testing Develop prototype Yes, 2 Yes, 2

Supplier management controls in place Implement controls Yes, 6 Yes, 6

Develop conformity and safety test plans Develop Plan Yes, 8 No

Perform conformity and safety tests(flammability, aerodynamics, andappreciable effects of design)

Conduct test Yes, 4 Yes, 4

Identify safety and certification issues-perform failure analyses

Identify concerns Yes, 4 No

Perform technical metrics and audit Conduct checks Yes, 4 No

Conduct production readiness review Conduct review Yes, 4 No

Figure 6: Nacelle Risk Mitigation DataFigure 6 details how each risk mitigation step

affected a particular part of the nacelle design.

The figure also shows the impact to the overall

risk on the redesign. These parts will be important

to determine which mitigation steps are critical to

the success of the risk mitigation effort. For

example one of the mitigation efforts listed is

forming an integrated product process team

(IPPT). Just by forming the team there is a

positive impact on reducing the likelihood a

failure event might occur in the nacelle design,

which is by two (2) percent. The consequence of afailure is not affected by this step. This IPPT is

made up of a core group of engineers and

managers that can make critical and intelligent

decisions on the nacelle design. Early involvement

with leadership and regulatory authorities and

identifying safety and regulatory requirements has

the greatest impact in reducing the likelihood of

the failure consequences as well as reducing the

severity of these consequences. This particular

role goes to regulatory administrators delegated by

the Federal Aviation Administration and the

Boeing Certification Engineers (Figure 4). Other

important risk mitigation strategies include

performing careful materials selection, having

supplier management controls in place, andperforming conformity and safety tests, performed

by Design and Certification Engineers and

Regulatory Administrators.

Figure 7: Risk Mitigation Allocation to ReduceConsequences

Figure 7 shows the allocation of risk mitigation

items from Figure 4 to the risk items to reduce the

severity of the consequences. For instance,

developing a nacelle prototype will ensure design

and safety requirements are met as well as

supporting the understanding of production to

reduce production costs and time.

III. POPULARITY AND SUCCESS

RUBRIC FOR RISK QUANTIFICATION

A. Popularity RubricThe popularity Rubric chart in Figure 8. is

modeled after Dr. Grantham’s work with slightmodifications to meet the generic needs of risk

quantification. See Appendix for clearer picture of

the Rubric Chart. The Rubric organizes, grades,

and communicates our expectation of the product

evaluated.




Figure 8. Popularity Rubric

With popularity, we used the measurement of

good, moderate, and poor which will be convertednumerically into a 1-3 scale (3 being best, and 1

worst). We can evaluate the success of each

criteria to see whether the risk mitigation met the

requirement outlined by the engineers. It has the

advantage of being a top level review of a productbefore we move into lower level review, which is

covered by the next type of rubric, Success Rubric.

B. Success RubricSuccess Rubric Chart (See Figure 9) is nearly

similar to the Popularity Rubric Chart with the

only exception; it actually evaluates the success of

the product that has gone through the Popularity

Rubric Evaluation initially. Success Rubric helps

determines how successful the product itself was

able to meet the criteria outlined at a much lower

level. It takes into consideration Dr. Grantham’sRED methods as a crucial aspect of the design

phase for risk mitigation. See Appendix for a

clearer picture.

Figure 9. Success Rubric

IV. IMPLICATIONS TO THEENGINEERING MANAGER

When we talk about risk quantification,

Bernstein (1998)3 stated that “without numbers,risk is wholly a matter of gut.” To minimize

arbitrary decisions with absolute subjectivity, we

need to collectively form an integrated team with

sufficient level of skill and experience to identify

and quantify all possible associated risks with a

product design. Risks have a major impact on

business decisions that often reflect on the

business case net present value (NPV) or internalrate of return (IRR).

We make business decisions on the basis of

limited data and likelihood of consequences

occurring and thus, in the face of probabilities,

risk management and quantification plays a crucial

role in decision-making.A corporate objective is to maximize value for

the shareholders and itself. The role of engineering

manager is to provide value to the company,

shareholders, and the customer in the form of a

functional product. The goal of the engineering

manager is to provide a product that is reliable andsafe.

The risk mitigation quantization methodology

proposed enables effective planning to mitigate

any consequences from occurring and thus

providing a safe and reliable product that complies

to high-level government regulations.




IV. CONCLUSIONThis method of quantifying risk mitigation can

redefine how risk is managed. There are

important strategies that need to be taken away

from this process:

Collect the data from risk mitigation

plan

Use the Success Rubric to ensure all

necessary data is was recorded in the

risk mitigation efforts.

Use the data for future risk mitigation

efforts

Many people will benefit from the risk

mitigation process in this paper. At the

engineering level, historical data will finally be

used instead of subjectively guessing at an answer.

From a managers perspective effective planning

for risk mitigation will save schedule and cost on

the overall program while having the confidence

corners are not being cut when providing a reliable

product to the customer.

When a process is created from these initial

steps it will save time and money for all of

industry who is looking for innovative ways for

risk mitigation.

ACKNOWLEDGMENT

Dr. Grantham’s Rubric Chart.

REFERENCES[1] Gratham Lough, Stone, Tumer (2008) The Risk in

Early Design (RED) Method

[2] NAVSO P-3686 (1998) Top Eleven Ways to

Manage Technical Risk. Office of the Assistant

Secretary of the Navy (RD&A) Acquisition and

Business Management.

[3] Bernstein, P. (1998) Against The Gods – The

Remarkable Storey of Risk. Published by John

Wiley & Sons, Inc. New York, USA

[4] Project Management Institute, A Guide to the

Project Management Body of Knowledge

(PMBOK® Guide), Project Management Institute

(2000).[5] H. Kumamoto, E. Henley (1996) Probabilistic Risk

Assessment and Management for Engineers and

Scientists. Published by IEEE Press.

About the Authors

Josh Goldschmid received his M.S. in

Engineering Management from DrexelUniversity and a B.S. In Biological

Systems Engineering from UC Davis. Heis a very cool guy with 20 years of experience in skiing and welcomeseveryone for free lessons.

Steve Broussard received his B.S. inMechanical Engineering from The

Pennsylvania State University and an A.S.in Engineering Science from SUNYMorrisville. In 8 years I have worked forNAVAIR, Lockheed Martin and currently

work for The Boeing Co. I hate movingand I personally hate to ski and am not acool guy.

William Harkness received his M.S. inEngineering Management from DrexelUniversity and a B.S. In MechanicalEngineering from California State

University, Chico. He is a very laid back guy with 0 years of experience in skiing

and doesn’t understand why anybodywould want to go up, down, up, down,

up, and down a frozen snow covered hill only to break theirleg or collar bone in the process of doing so.

Damon Slaughter received his B.S. in

Engineering Management from Park University. He has never tried snow

skiing but would like to at some pointas long as he does not break his leg orcollar bone in the process.




APPENDIX A

Figure 2: 10 Risk elements and Risk Score




Figure 3 : Nacelle Risk Management Plan




Figure 4: Risk identifier and assigned job roles for the waterfall chart (Figure 3)




Figure 5 : Nacelle Risk Plot

Figure 6 : Nacelle Risk Mitigation Data

Mitigation Effort.What does themitigation effortdescribe?

Does it reduce likelihood? Ifso, by how much?

Does it reduce consequence? If

so, by how much?

Project proposal for re-design Propose design Yes, 2 No

Integrated, multi-discipline, IntegratedProduct/Process (IPP) team formed

Form team Yes, 2 No

Develop detailed project managementaspects

Develop plan Yes, 2 No

Develop detailed work statement Develop strategy Yes, 4 NoInitial studies and cross-functional researchincluding economics

Conduct research Yes, 4 No

Develop plan for concept design Develop plan Yes, 2 No

Early involvement with leadership andregulatory authority

Assess progress Yes, 10 Yes, 10

Identify applicable regulations andrequirements

Identify limits Yes, 6 Yes, 6

Define plan to meet regulatory and safetyrequirements

Define plan Yes, 6 Yes, 6

Validate certification process Validate process Yes, 4 No

Validate design process Validate process Yes, 6 No

Design the nacelle per the requirements Design product Yes, 2 No

Perform careful materials selection Select material Yes, 4 Yes, 4

Validate build process Validate process Yes, 4 No

Develop prototype for testing Develop prototype Yes, 2 Yes, 2

Supplier management controls in place Implement controls Yes, 6 Yes, 6

Develop conformity and safety test plans Develop Plan Yes, 8 No

Perform conformity and safety tests(flammability, aerodynamics, andappreciable effects of design)

Conduct test Yes, 4 Yes, 4

Identify safety and certification issues-perform failure analyses

Identify concerns Yes, 4 No

Perform technical metrics and audit Conduct checks Yes, 4 No

Conduct production readiness review Conduct review Yes, 4 No




Figure 7 : Risk Mitigation Allocation to Reduce Consequences




Figure 8. Popularity Rubric:

[modeled after Dr. Grantham’s Rubric Charts]




Figure 9: SUCCESS CHART:[modeled after Dr. Grantham’s Rubric Charts]

emj syseng427 project final rev5

Documents