problem management: a system engineering management

Problem Management: A System Engineering Management Framework

By Bill A. Olson

B.S. in Electronics, February 1984, Chapman University M.S. in Program Management, December 2004, Florida Institute of Technology

A Dissertation submitted to

The Faculty of The School of Engineering and Applied Science

of The George Washington University in partial satisfaction of the requirements for the degree of

Doctor of Philosophy

January 31, 2012

Dissertation directed by

Thomas Andrew Mazzuchi Professor of Operations Research and of Engineering Management

Shahram Sarkani

Professor of Engineering Management and Systems Engineering

ii

The School of Engineering and Applied Science of The George Washington University

certifies that Bill Allen Olson has passed the Final Examination for the degree of Doctor

of Philosophy as of 30 December 2011. This is the final and approved form of the

dissertation.


Bill Allen Olson

Dissertation Research Committee:

Thomas Andrew Mazzuchi, Professor of Operations Research and of Engineering Management, Dissertation Co-Director

Shahram Sarkani, Professor of Engineering Management and Systems Engineering, Dissertation Co-Director

Lile Murphree, Jr., Professor of Engineering Management, Chair Person of the Examination Committee

Michael Stankosky, Professor of Engineering Management and Systems Engineering, Committee Member

Timothy Eveleigh, Adjunct Professor of Systems Engineering, Committee Member

iii

Acknowledgements

I am grateful to Dr. Thomas A. Mazzuchi and Dr. Shahram Sarkani, Professors of

Engineering Management and Systems Engineering at the George Washington

University, and to Dr. Kevin Forsberg, Chief Executive Officer and cofounder of the

Center for Systems Management, for supporting me in the development of a problem

management framework and for their insights and guidance throughout the development

and completion of this dissertation.

To my fellow classmates in my PHD cohort, I thank them for their help,

encouragement, and friendship during the course of the program.

I am thankful to Mr. Kevin Topp, Systems Engineering Manager at Huntington

Ingalls Newport News Shipbuilding and adjunct Professor of Engineering Management

and Systems Engineering at the George Washington University, for his insightful

comments and support. I would like to thank Ms. Johanna Lunglhofer Granor for her

amazing technical editing skills and making sure all of my commas are in the right place.

Most of all I would like to thank my loving wife Sheila. She has stood beside me

and believed in me when it seemed that I would never succeed. “Sheila I love you”

Finally, I must recognize my parents, Orville N. Olson, Dorothy Olson and Jean

Olson, who have instilled in me the importance of education and desire to pursue goals

that can only be achieved through hard work. Although my dad is no longer with us,

through faith, I find comfort in knowing that they both also share in the joy of this

accomplishment.

iv

Abstract


Problem management is not recognized as a system engineering process. Unlike risk

management where risks could impact a project at some future date, problems impact the

project now. Therefore, a robust problem management process should be established

(similar to the risk management process described in chapter five of the INCOSE

handbook) [1], early in a project’s formulation. The process should be continuous

throughout all phases of the project life cycle and should be included as a part of the

project System Engineering Management Plan. The process should at a minimum

address: problem identification, problem assessment, problem root cause investigation,

and problem corrective action planning and reporting. It should also include a roll-up of

all current problems, problem closure, and the development of a strong interface with the

project’s knowledge management system.

This paper proposes a problem management systems engineering framework, based

on the foundations of risk, opportunity management and knowledge management, which

could be used by corporations, government agencies, and programs to establish a robust

problem management process for use on their projects.

v

Table of Contents

Acknowledgements .......................................................................................................... iii

Abstract…. ........................................................................................................................ iv

List of Figures ................................................................................................................. viii

List of Tables ..................................................................................................................... x

1 MOTIVATION ......................................................................................................... 1

1.1 Motivation Justification ...................................................................................... 1

1.2 Statement of Purpose .......................................................................................... 2

1.3 Document Organization ...................................................................................... 5

1.4 Background ......................................................................................................... 7

1.5 Goal ..................................................................................................................... 8

1.6 Significance......................................................................................................... 8

1.7 Limitations and Scope......................................................................................... 9

2 Literature Review ..................................................................................................... 9

2.1 Goal ..................................................................................................................... 9

2.2 Systems Engineering ......................................................................................... 11

2.3 Systems Engineering Processes ........................................................................ 12

2.3.1 Architecture ............................................................................................... 13

2.3.2 Systems Engineering Frameworks ............................................................ 14

2.3.3 Technical Coordination ............................................................................. 16

2.3.4 System Integration .................................................................................... 18

2.3.5 Verification and Validation ....................................................................... 21

2.4 Project Planning and Control ............................................................................ 24

vi

2.4.1 Project Planning ........................................................................................ 25

2.4.2 Financial Contract Management ............................................................... 27

2.4.3 Schedule Management .............................................................................. 29

2.5 Overlapping Processes ...................................................................................... 32

2.5.1 Risk Management ..................................................................................... 33

2.5.2 Opportunity Management ......................................................................... 37

2.5.3 Task Definition ......................................................................................... 38

2.5.4 Knowledge Management .......................................................................... 40

2.5.5 Information Management .......................................................................... 44

2.5.6 Configuration Management ...................................................................... 45

2.5.7 Technical Performance Management ....................................................... 49

2.6 Additional other significant processes .............................................................. 50

2.6.1 Decision Management Process ................................................................. 51

2.6.2 Cost Benefit Analysis ............................................................................... 53

2.6.3 Earned Value Management ....................................................................... 54

2.7 Literature Conclusion........................................................................................ 55

3 Problem Management ............................................................................................ 56

3.1 Problem Management ....................................................................................... 56

3.1.1 Overview ................................................................................................... 56

3.2 Problem Management Framework .................................................................... 69

3.2.1 Problem Planning - Step 1 ........................................................................ 70

3.2.2 Problem Identification - Step 2 ................................................................. 73

3.2.3 Problem Analysis and Assessment - Step 3 .............................................. 75

vii

3.2.4 Problem Handling - Step 4 ........................................................................ 78

3.2.5 Problem Monitoring - Step 5 .................................................................... 81

3.2.6 Problem Closure - Step 6 .......................................................................... 83

3.2.7 Problem Knowledge Management – Step 7 .............................................. 84

4 Recommendations and Conclusions ...................................................................... 85

4.1 Recommendation .............................................................................................. 85

4.1.1 Purpose ...................................................................................................... 86

4.1.2 Description ................................................................................................ 86

4.1.3 Outputs from Activities............................................................................. 89

4.1.4 Process Activities ...................................................................................... 90

4.2 Conclusions ....................................................................................................... 94

5 Recommendation for Future Research ................................................................. 95

5.1.1 Opportunity Management Framework ...................................................... 95

5.1.2 Problem, Risk, Opportunity, Realized Opportunities, and Earned Value

Management Integration ........................................................................................... 96

5.1.3 Total Systems Engineering Framework .................................................... 96

6 BIBLIOGRAPHY ................................................................................................... 98

viii

List of Figures

Figure 1-1: Systems Engineering Management Processes Optimal Process Integration

Venn diagram ...................................................................................................................... 7

Figure 2-1: Integrated Academic and Literature Review Roadmap ................................ 10

Figure 2-2: System Architecture Goal[11] ...................................................................... 14

Figure 2-3: Systems engineering and management process model with integration

interface[16]. ..................................................................................................................... 19

Figure 2-4: INCOSE Systems Engineering Vee Model[20] ............................................ 23

Figure 2-5: Typical Project Planning Steps ...................................................................... 26

Figure 2-6: Comparison of Life Cycle Models[1] ............................................................ 31

Figure 2-7: Common Risk Management Process Functions ............................................. 35

Figure 2-8: Common Opportunity Management Functions .............................................. 37

Figure: 2-9: Six Critical Task Definition Requirement Questions .................................. 39

Figure 2-10: Basic Configuration Management Process Steps ........................................ 46

Figure 2-11: Decision Tree for a Bid-No Bid decision[1] ................................................ 52

Figure 3-1: NASA Problem Management Matrix[4] ....................................................... 60

Figure 3-2: Opportunity, Risk, and Problem Management Matrices[2] .......................... 61

Figure 3-3: Five Categories of problems[2] ..................................................................... 63

Figure 3-4: Toyota Stock Price vs. Time[2] .................................................................... 66

Figure 3-5: Problem Management Framework[2] ........................................................... 69

Figure 3-6: Problem Management Framework Step 1 ..................................................... 71



ix


Figure 3-10: Problem Management Framework Step 5 ................................................... 82

Figure 3-11: Problem Management Framework Step 6 ................................................... 84

Figure 3-12: Problem Knowledge Management Process Step 7 ..................................... 85

Figure 4-1: Context Diagram for the Problem Management Process Inputs to Activitie . 88

x

List of Tables

Table 1-1: Systems Engineering/Project Planning and Control Overlap ........................... 6

Table 2-1: Systems Engineering Centric Processes Reviewed in Section 2.3. ................ 12

Table 2-2: Three Common Systems Architecture Frameworks ...................................... 16

Table 2-3: Partial List of Systems Engineering Sub-specialties, Tasks, and Titles ......... 18

Table 2-4: Two examples of problems identified during systems integration ................. 20

Table 2-5: Project Planning Processes Reviewed in Section 2.4. .................................... 25

Table 2-6: Financial Contract Management Software Solutions ..................................... 28

Table 2-7: Project Planning Processes Reviewed in Section 2.5 ..................................... 32

Table 2-8: Alternative Definitions of Risk Management and Risk ................................. 34

Table 2-9: Partial List of Commercial Knowledge Management Software .................... 43

Table 3-1: Sample List of Citations from the INCOSE handbook[1] ............................. 57

1

1 MOTIVATION

1.1 Motivation Justification

Early in the development of risk management processes, project teams justified a lack

of risk management by using excuses such as:

I don’t have time for what-if’s.

Risk Management costs too much money.

Risk Management is a paperwork nightmare.

I do not want to look bad in front of my boss.

This is a new fad – It will pass and then be back to business as usual.

If I state my risks, I might be the next victim of “shoot the messenger

syndrome.”

All of the above have been proven false, and risk management is a proven,

sustainable, cost-saving process that delivers value to projects. They are not adequate

justifications for failure to manage project problems in an open, consistent manner.

Many risks are realized and closed as problems. This is a justifiable option under

most risk management plans and guidebooks. However, once a risk is closed, what

happens to the new problem?

Currently there is no internationally recognized process to deal with problems. It is

time for problems to come out from behind closed-door meetings, get off of PowerPoint

slides, and become a recognized systems engineering discipline which could significantly

benefit a project. Like risk management, the benefits of establishing robust problem

management processes are substantial. This paper proposes a system engineering-based

2

framework intended to provide the backbone of a problem management process for use

by projects, programs, government, and corporate offices to manage their problems in an

open and consistent manner.

1.2 Statement of Purpose

Problem management is not defined as a systems engineering discipline. Yet, the

word ‘problem’ is mentioned 92 times in the International Council on Systems

Engineering (INCOSE) handbook version 3.2.[1] Most System Engineering processes

use the word ‘problem’ to describe the design space of the systems engineering process.

The INCOSE handbook defines ‘system engineering’ as, “an interdisciplinary

approach and means to enable the realization of successful systems. The systems

engineering process focuses on defining customer needs and required functionality early

in the development cycle, documenting requirements, and then proceeding with design

synthesis and system validation while considering the complete problem: operations, cost

and schedule, performance, training and support, test, manufacturing, and disposal.

Systems engineering considers both the business and the technical needs of all customers

with the goal of providing a quality product that meets the user needs”.[1]

Contrastingly, the systems engineering approach taken by most projects today lacks a

problem management process similar to the risk management process. The purpose of

this paper is to fill this gap in the system engineering process by ensuring that a rigorous

problem management process is in place at the beginning of a project’s life cycle. And

that the systems engineering and Project Management teams have a sound management

process in place which will significantly contribute to project success and to customers’

3

satisfaction knowing that the team is executing to the best of their ability while fully

disclosing and addressing all of the problems and how they impact the project’s bottom

line.

When not developing designs to eliminate problems, project teams spend large sums

of money developing risk management plans with the goal of preventing problems from

occurring. Yet, risks are still realized.

Problem management is different from risk management in the probability of

occurrence. While ‘risks’ are potential problems, which, if not dealt with, may become

problems in the future, ‘problems’ exist in the present and need to be addressed

immediately.

There are two types of problems[2]. Anticipated problems are realized risks which

have a 100% likelihood of occurring. In these cases, the risk was known but the project

was unable to keep the risk from being realized. Unplanned events – sometimes called

issues, non-conformances or anomalies – are unknown or unanticipated problems. For

the purposes of this paper, the word ‘problem’ encompasses both of these descriptions.

What happens when problems occur? Are projects prepared to effectively deal with

problems? There is no standard process available which defines categories of problems,

provides a problem management approach, establishes a problem treatment (handling)

approach, and defines a methodical hierarchical scaling of problems.

The National Aeronautics and Space Administration (NASA) recognized the need for

problem management after the November 2003 Space Shuttle Columbia accident [3].

NASA developed a problem management process similar to the risk process that enables

projects to determine which problems to expend valuable resources on and to status the

4

results of problem resolution efforts.[4] However, the primary focus of the NASA

Management Guidebook is on safety and schedule.

Problem Management is defined as a process to identify, report, analyze, develop

resolution plans, assess impacts, and monitor problems as they are identified. The

Problem Management process systematically searches for problems, directs problems

with a potential for occurrence into the Risk Management process, and processes

problems with 100% occurrence into the Problem Management process.

This problem management system engineering framework is based on the

comprehensive INCOSE risk management process, some of the concepts in the NASA

Problem Management Guidebook [4], and concepts developed during the systematic

literature research conducted in chapter 2. This framework draws upon the lessons

learned in the development of risk management to establish problem identification and

problem resolution techniques which, in turn, establish a foundation for a cradle-to-grave

lifecycle approach to problem management for use by projects to better control cost,

schedule, technical, programmatic, and environmental and safety problems. This end-to-

end approach is based on the recognized assumption that problems are going to occur

during the life cycle of a project.

This paper focuses on establishing a requirement to implement a problem

management framework. To establish the need to develop a contractual requirement for a

project problem management plan at the beginning of a project. Implementing a plan

ensures that when problems are identified they are addressed up front and continuously

during the project’s life cycle to minimize related complexities and challenges later on in

the systems engineering life cycle. This paper provides an overview of problem

5

management and the activities included in the proposed problem management

framework.

1.3 Document Organization

This document is organized to describe a proposed problem management framework

that synthesizes the risk, opportunity, knowledge management, earned value systems, and

other systems engineering processes to develop a problem management system from the

proposed framework.

Chapter 2 (literature review) establishes the foundation that the proposed framework

is built upon. The literature review focuses on the current practices and applications that

make up the core systems engineering project processes, starting with INCOSE

processes, with potential interfaces with a problem management framework. Table 1-1 is

an expansion of the INCOSE Project Processes which depicts the overlap between key

project processes.

The INCOSE handbook does not include several key processes which are critical to

the success of a project. It is proposed that the problem management framework will

fall somewhere in the overlap (blue columns) with a strong bidirectional interface with

the risk management, knowledge management, task definition, opportunity management,

and the customer interaction processes.

6

Table 1-1: Systems Engineering/Project Planning and Control Overlap

Chapter 3 provides an in depth overview of problem management concepts, and then

lays out a detailed description of the problem management framework in the format of

the International Council on Systems Engineering Handbook. Chapter 4 provides the

final recommendation and conclusion. Chapter 5 presents several recommendations for

future research as a result of this research.

7

1.4 Background

Figure 1-1: Systems Engineering Management Processes Optimal Process

Integration Venn diagram

Developing and delivering complex systems requires tight control of all of the

systems engineering management processes. The INCOSE handbook defines most of the

project processes; however the handbook leaves out several key project planning

processes. Figure 1-1 Systems Engineering Management Processes Optimal Process

Integration Venn diagram expands the list to include knowledge and opportunity

management. The center -- optimal spot -- is where problem management should reside.

A problem management process developed from the proposed problem management

framework enlarges the optimal spot making it easier for a project to obtain its goal of

providing the best value to their customers and stakeholders. The project processes are

Optimal Integration ‐where all SE project processes overlap and work effectively together toward success of a project

Risk and Opportunity Process

Project Assessment Process

Decision Process

Project Planning

Knowledge Management

Decision Management Process Planning

Problem Management

Problem Management – The missing piece??

8

used to establish and update plans, to execute, to determine progress toward project goals,

and as controllers of the project through fulfillment. Individual project processes may be

invoked at any time in the life cycle and at any level in a hierarchy of projects, as

required by project plans or unforeseen events. The project processes should all be

instituted with the same rigorous discipline and formality. Seven key projects processes

are: Project Assessment and Control Process, Decision Management Process, Risk

Management Process, Opportunity Management Process, Configuration Management

Process, Information Management Process, Measurement Process, and Knowledge

Management[1].

1.5 Goal

The goal of this research is to develop and present a comprehensive Problem

Management Framework that is used to support the system engineering design processes

by integrating proven systems engineering processes. This framework relies on the

familiar concepts of risk and opportunity management while integrating with the seven

key project processes identified in section 1.4.

1.6 Significance

This research is of particular importance because of the lack of an internationally

recognized problem management framework. A framework must be developed that can

be used to plan, quantify, report, and capture the history of the problems encountered

during the systems engineering design life cycle. There are large sums of money at stake

during the development of complex systems for defense, space, and commercial projects.

9

The implementation of a comprehensive problem management framework at the

beginning of a project could be the difference between project success and failure.

1.7 Limitations and Scope

For the purpose of this research, the scope and analysis are limited to the

“International Council on Systems Engineering (INCOSE) project processes as defined in

the INCOSE handbook”[1] and the Systems and Software engineering – systems life

cycle processes standard ISO/IEC 15288:2008[5].

2 Literature Review

2.1 Goal

“Conducting a literature review is a means of demonstrating an authors’ knowledge

about a particular field of study”[6]. The goal of this literature review is to develop an

integrated view across all of the major systems engineering processes and then to

generalize the findings across the various frameworks, treatments, outcomes and settings

to foster and resolve debate about a comprehensive problem management framework.

The secondary goal is to meld the outcomes across the selected frameworks and

processes and to critically analyze previous research to identify the core concepts which

impact, enhance, or substantiate need for the proposed problem management framework.

Figure 2-1 is an integrated view, road map of the literature and academic steps taken to

ensure that exhaustive research was conducted.

10

Figure 2-1: Integrated Academic and Literature Review Roadmap

The review starts with a definition of systems engineering followed by a review of the

core systems engineering processes (Systems Architecture, Systems Engineering

Frameworks, Technical Coordination, Systems Integration, and Validation and

Verification). This section was followed by a review of the project Planning Processes

(Project Planning, Financial Contract Management, and Schedule Management). The

next step of the review was to review processes which overlap the Systems Engineering

and Project Planning and Control Processes (Risk Management, Opportunity

Management, Task definition, Knowledge Management, Information Management,

Configuration Management, and Technical Performance Management). To ensure all

INCOSE Systems Management Processes

Integration of Concepts

Olson, Mazzuchi, Sarkani, Forsberg (Journal of Systems Engineering: Volume 2, 2012)

Problem Management Definition

HRA-INCOSE Conference (2010)Extending Risk Management to

Problem Management

Presented “Problem Management Process, Filling the Gap in the SystemsEngineering Processes between the Risk and Opportunity Processes

GWU Class Work

Identify and Refine Potential Correlations to Selected Dissertation

Research Topic

EMSE 216 “Research Methods for the Engineering Manager”

EMSE 208 “Stochastic Foundations of Operations Research”

EMSE 288 “Technology Issue Analysis”

EMSE 286 “Architecture Description and Enterprise Architecture”

EMSE 288 “Special Topics: Advanced Systems Engineering”

Goal

Identify and validate a Dissertation Topic through Course work and a

comprehensive literature review

Systems Engineering Historical Review

Engineering Project Management

Systems Engineering Architectures and Frameworks

INCOSE

Review INCOSE Process Validate need for a Problem Management

Framework

Literature Review Systems Engineering Processes

Systems Integration

Validate Problem Management Requirement

INCOSE International Conference (2011) Chair of INCOSE KM and Handbook Committee confirmed need and asks for a chapter in the

next Revision of the Handbook (GWU to Lead the effort)

Is Problem Management

Valid?

Systems Engineering Standards Review

Configuration

Knowledge

Risk

OpportunityDecision


Is There a Published

Problem Management Process?

Contract

11

bases are covered the last section of the review addresses other significant processes in

which significant problems are discovered (Decision Management, Cost Benefit

Analysis, and Earned Value Management).

At key points in each subject review, a short discussion on the potential interfaces

with a problem management framework was added to help focus the need for projects to

establish a Problem Management process based on the proposed problem management

framework. The scope of research was narrowed by focusing only on scholarly journals,

reference books, and the INCOSE handbook.

2.2 Systems Engineering

Kossiakoff and Sweet assert that, “The function of systems engineering is to guide the

engineering of complex systems” [7]. INCOSE expands on this point. “Systems

engineering is an interdisciplinary approach and means to enable the realization of

successful systems. It focuses on defining customer needs and required functionality

early in the development cycle, documenting requirements, and then proceeding with

design synthesis and system validation while considering the complete problem:

operations, cost and schedule, performance, training and support, test, manufacturing, and

disposal. Systems Engineering considers both the business and the technical needs of all

customers with the goal of providing a quality product that meets the user needs”[1].

David Hall one of the first pioneers in the field, wrote one of the first definitive books

on systems engineering in 1962 called “A Methodology for Systems Engineering”[8].

Hall was the first of many who saw that a systematic approach to system design

eliminates many problems, saves money and increases the chances of the product being

12

delivered on time. Soon the U.S. military also saw the light and developed Mil-Std 499

and 499A the requirements of which were often included as contractual obligations in

new development projects.

The latest standard ISO/IEC 15288:2008 is a continuation of David Hall’s work. The

ISO standard is the basis of the format and content of the INCOSE Handbook. The best

way to define systems engineering is that it is a problem management system. When a

project is conceived to develop something new, using the systems engineering process is

the best methodology to solve the problem of what, how, when, where, why, how big,

how much cost, et cetera. Table 2-1 highlights the systems engineering centric processes

which are reviewed in the section of the literature review.

2.3 Systems Engineering Processes

Table 2-1: Systems Engineering Centric Processes Reviewed in Section 2.3.

Project ProcessSystems Engineering Over Lapping Processes Project Planning and Control

System Architecture* Risk Management* Task Definition* Project Planning*

Technical Coordination* Customer Interaction*

Opportunity Management

Resource Allocation*

System Integration* Knowledge Management

Configuration Management

Financial Contract Management*

Validation and Verification

Technical Performance Management

Information Management

Schedule Management

* = INCOSE Project Process

13

2.3.1 Architecture

INCOSE describes the Architecture Design Process. “This process encapsulates and

defines the areas of solution expressed as a set of problems of manageable, conceptual

and, ultimately, realizable proportions”[1]. Blanchard and Fabrycky explain that

“Functional analysis is an iterative process of translating system requirements into the

detailed design criteria and subsequent identification of resources required for operation

and support” [9]. Functional analysis is the first of two system engineering processes

used to define the system design. It is here that problem management can initially be

effectively used to identify potential problems before they impact the project. These

potential problems are project risks that should be entered into the project risk register

unless they have a likelihood of occurrence at the time of identification of one[2, 10].

Physical analysis is the next step in which identification of project problems could

occur. Since physical analysis takes place later in the project life cycle, the likelihood

and consequence of a problem impacting the project completion schedule is greater. It is

critical that these steps of the system architecture be used to identify problems. Figure 2-2

“System Architecture Goal”[11], depicts the importance of allocating requirements using

functional and physical analysis/allocation processes.

14

Figure 2-2: System Architecture Goal[11]

These two processes can only be accomplished after the customer requirements are

fully defined. A proven methodology to help organize customer requirements,

stakeholder desires, and the scope of the project’s system architecture is through the use

of systems engineering frameworks.

2.3.2 Systems Engineering Frameworks

Webster’s Dictionary defines a framework as “a basic conception structure (as of

ideas)”[12]. Bernard further expands the definition and describes a framework as “a

structure for organizing information that defines the scope of the architecture and how the

areas of the architecture relate to each other”[13].

Analysis

15

A Systems Engineering Framework is a construct used to frame a portion of the

Systems Engineering processes for specialized purposes to aid in the design and

management of systems. Frameworks originated with the advent of computers.

Specialized architecture was required to develop the complex software required to

operate mainframe computers. Frameworks served as the needed skeleton for

programmers to keep track of the complex code necessary for the machines to operate

within the system design requirements.

John Zachman realized that the processing of information was getting out of control

at IBM. He also understood that in order to sell a product his end users (the customers)

must be able to understand exactly what it was that he was selling. He developed the

“Information Systems Architecture (ISA)”[13] one earliest and best known systems

engineering architectural frameworks. Today there is at least one Systems Engineering

Framework developed for every recognized system development process.

Table 2-2 depicts three common System Architecture Frameworks. One of the

interesting concepts behind frameworks is that the authors tend to extend the ideas of a

basic concept by adding detail and enhancements which re-focus the original idea in a

direction never envisioned by the initial author. That is the case with the frameworks in

table 2-2. Zachman’s concept was extended by Spewak and further extended by Bernard.

Each framework is an original that expands the concept and amplifies a different area of

systems engineering.

16

Table 2-2: Three Common Systems Architecture Frameworks

2.3.3 Technical Coordination

With the Introduction of ‘Systems Thinking,’ “the use of a particular set of ideas,

systems ideas, in trying to understand the world’s complexity. The central concept

‘systems thinking’ embodies the idea of a set of elements connected together to form a

whole, this showing properties which are properties of the whole, rather than the

properties of the component parts”[14] the need for technical coordination became

paramount.

System thinking is a discipline which takes years to master[1, 14]. The systems

thinker must identify and understand the entire set of component elements which make

up a system and understand the processes used to develop the component elements. Once

the elements are assembled, the systems thinker must then be able to see the forest vice

the trees and understand how the parts work together to provide a service greater than the

sum of all of the parts.

Title Of Framework Author Purpose

Information Systems Architecture John Zachman

To depict information systems from different viewpoints

Spewak Enterprise Planning Method

Steven Spewak

Aplanning focused framework which used/extended Zachman’s framework towards business management

EA3 Framework Scott Bernard

Extends the work of Zachman, Spewak, Parsons,Thompson and others by introducing a cube concept which demonstrated that there are multiple levels and dimensions to and Enterprise Architecture.

17

Numerous universities have established Systems Engineering degrees at the

undergraduate, masters, and doctoral levels to teach systems thinking. These systems

engineers are ideal technical coordinators[2].

Technical coordination is the skill used by systems engineers to orchestrate an

iterative process which results in an optimal systems design[1, 14]. Eisner states

“Systems engineering is an iterative process of top down synthesis, development, and

operation of a real-world system that satisfies, in near optimal manner, the full range of

requirements for each system”[15]. The systems engineer is the technical coordinator

and one of his key roles is to understand the problems identified during the systems

development life cycle.

To further complicate the issue, the latest ideas in technical coordination do not stop

with a single system. Today “It is clear the systems engineering of systems of systems is

here to stay” [15]. This means the technical coordinator (the systems engineer) must

understand how a system in California could impact and interact with systems across the

globe.

The magnitude of the original system thinker’s task has expanded significantly. Now

teams of technical coordinators are required, many with specialized skills. Similar to

medicine, wherein doctors specialize in different parts of the human system, systems

engineers specialize in the component elements of the systems engineering process. Each

is a specialized technical coordinator; some examples are depicted in Table 2-3, a list of

systems engineering sub-specialties, tasks, and titles.

18

Table 2-3: Partial List of Systems Engineering Sub-specialties, Tasks, and Titles

2.3.4 System Integration

“An integration perspective is taken in an effort to ensure that the needs of the

customer, the systems engineering team, and existing or legacy systems are considered

or, in effect, integrated”[16] To correctly accomplish systems integration, the customer

and the design staff must understand that integration is much more than physical

components, such as new technology. Integration involves people, projects, companies,

governments, and cultures. It is all about helping them understand information,

education, and collaboration so that the sums of the parts work productively together.

Figure 2-3 demonstrates the iterative nature of systems integration. As each step in the

figure’s simplified three step systems engineering process is accomplished, the

opportunity to identify problems arises.

19

Figure 2-3: Systems engineering and management process model with integration

interface[16].

Table 2-4 is a list of some of the types of problems which could be identified during the

three systems integration steps identified in figure 2-3. Two potential systems integration

problems are identified between systems definition and systems deployment. If systems

integration is rigorously conducted, potential major problems can be identified and solved

[1, 9, 17].

20

System Definition

Integrated Resolution Between the Two Systems

System Deployment

Potential Problem Identification

Customer Requirement

Define what is needed to deploy a system which meets the customers’ expectations

Deployment is what the customer expected

The customer does not understand that what he asked for in writing (Often the design specification) does not meet his vision of the end product

Customer Vision

The form, fit, and function meets the customers vision

The aesthetic portions of the project such as color, size, smell meet what the customer expected

The customer wanted a blue one and the project delivered a red one

Table 2-4: Two examples of problems identified during systems integration

One of the most import points to consider is that even if the systems engineer

identified and solved a problem in the systems definition phase, it does not follow that the

problem has been solved for the system deployment phase. In fact, often a simple

resolution at the beginning drives costs up for the end user[2].

For example: if a product design requires a bolt, the design engineer may select a low

cost bolt that is different from any used on the rest of the design. By using the bolt, the

engineer saves the project fifty cents per bolt. At system manufacturing and deployment

this bolt is cheap but unique. It requires special handling to ensure it does not get mixed

with the other bolts thus driving up the cost to the project to over a dollar a bolt. If

21

system integration had been conducted at the beginning when the bolt was selected the

engineer probably would have selected a standard but slightly more costly bolt which

would have saved the project more in the end.

System integration between the people charged with the design would have identified

the potential problem and potentially a risk would have been submitted with a mitigation

strategy. Later in the product life cycle when the cost of the additional bolt is identified

the risk will have a likelihood of 1 and be considered a problem. Systems integration is

where the problem should have been identified and solved.

2.3.5 Verification and Validation

One of the most obvious times to discover problems is during the verification and

validation phase of a project. It is at this stage in the systems engineering process we

check to make sure we are meeting (verifying) our customers’ requirements and checking

(validating) to ensure the design will satisfy the customer’s needs.

2.3.5.1 Verification

Forsberg defines verification as, “the process of demonstrating (as opposed to

proving) that the product satisfies the user needs, regardless of what the system

specifications require”[18]. Reed Integration’s Professional Systems Engineering

Certificate material explains why “Verification of compliance with requirements is

crucial to ensuring successful delivery of a quality product”[19].

22

This is where the systems engineering team proves that the design is meeting the

customer’s expectation. If the customer expected the design to incorporate the color blue

and the design team used red, the discrepancy is a problem and should be entered into the

risk register if the there is enough schedule left before delivery to mitigate the risk of

delivering a red project. If there is not enough, time to correct the issue, the validation

process has uncovered a problem which the project should address immediately.

2.3.5.2 Validation

Forsberg defines Verification as “the process of determining that the system meets all

specified requirements and survives its intended environment”[18]. Validation is “where

the project proves the project meets the customer’s requirements. This is where the

project proves the design meets the design requirements”[18]. This is often in the project

contract. If the contract required a red color in the design and the end product was red

even if the customer desired blue, the project is valid and other contractual action is

required. This discrepancy is still a problem or risk and needs to be addressed. The most

likely method of resolution is a project change order.

2.3.5.3 Verification and Validation Methods

Testing the device or system developed by the project is not always the most cost

effective method to perform validation and verification. Visual inspection, modeling and

simulation, and analysis are common alternative methods of verification and validation.

A key point to remember is that the project must ensure that the customer agrees to the

method used prior to submitting the results for acceptance. If the customer disagrees at

the time of acceptance, the project has a problem. One of the most common methods

23

used to ensure the project is meets the requirements and the methods of verification and

validation meet the customer needs is the use of Systems Engineering models. One

common and widely accepted model is the Systems Engineering Vee. The Vee model “is

a system development model designed to simplify the understanding of the complexity

associated with developing systems”[1]. Figure 2-4 depicts the layout of Systems

Engineering Vee model[20] and the points in the project development life cycle that

verification and validation take place.

Figure 2-4: INCOSE Systems Engineering Vee Model[20]

24

The keys points of the model are:

Continuous communication with the customer through the whole design life cycle

Verification and validation at every level

Architecture decomposition and definition take place on the left side of the Vee

Architecture integration and verification takes place on the right side of the Vee

Continuous risk and problem identification is a key concept. Verification and

validation done early saves the project money and maintains the schedule by identifying

problems early (as risks), allowing time to mitigate or if the risk has a likelihood of one

informing the project of the problem’s existence and thus allowing the project to focus on

problem resolution[1, 2, 20].

2.4 Project Planning and Control

Project planning and control processes “are used to establish and evolve project plans,

to execute the project plans, to assess actual achievement and progress against the plans

and to control execution of the project through to fulfillment”[1]. Each process adds a

unique prospective on the total project and the potential type of problems identified while

executing the process. Table 2-5 highlights the project planning and control processes

which were reviewed.

25

Table 2-5: Project Planning Processes Reviewed in Section 2.4.

2.4.1 Project Planning

Project planning starts with an idea which is based on some need or requirement. It

can come from a multitude of sources. The customer may want a new product or request

an upgrade to an existing product, or an internal change to a process or product. Planning

a project is not limited to new hardware design. It can be new or updated software, a new

office layout, a new or upgraded organizational chart, or even an overhaul of the existing

office supply cabinet. One rule of thumb is the more complex the project, the greater the

need for projects to adhere to planning and control. Figure 2-5 shows the typical project

planning steps required to execute a project.












Schedule Management


26

Figure 2-5: Typical Project Planning Steps

The first step is the definition of the project. Proposals are made and requirements

are identified and objects are established. A key part of the definition step is the

definition of project boundaries and constraints. Other significant tasks accomplished

during project definition are:

Estimating the project cost

Developing a project proposal

Determining what design model will be used

Defining project-specific procedures (or tailoring existing procedures) such as

risk, configuration, earned value, and other project plans as required.

Next, resources must be identified. This step must be done early in order to ensure

the project can be accomplished in the customer’s required schedule. Resources are not

just material; they can be people, facilities, and transportation. The development of a

work breakdown structure will aid in the identification and allocation of resources.

27

Planning is complete when the documents required are agreed upon and new ones are

written or existing ones are modified to meet the project needs. A systems engineering

management plan is developed during this step. The goal is to minimize problems by

creating the backbone of a project structure which addresses problems long before they

impact the project bottom line.

Final project planning definition takes effect with the signing of a contract. This is

the step that puts the project in place and allows the project team to go to work. The

organization structure has actual names assigned to positions. Material requirements

have been identified and the sources for obtaining them have been identified. Other key

program project requirements such as transportation, permits, and facilities have been

identified and obtained. Perhaps the most crucial part is the customer buying into the

plan.

The controls step is the execution of the procedures put in place during the planning

processes. Managing risks, the configuration, and budget are some examples. During the

controls step, communication is the key. Communication helps eliminate or reduce

conflicts with the customer and ensure the early identification of project risks.

Communication also helps with the identification of problems. This is critical in

order for the management team to avail the right amount of resources to ensure the

project is executed on schedule and to the budget.

2.4.2 Financial Contract Management

Early in the project, risks should be identified to mitigate contractual management

issues [1, 2, 21]. For projects initiated based on making a profit, the goal of contractual

28

management should be maintaining the project profit share line. For the internal (no-fee

projects) an informal contact should be agreed to before execution of the project and may

or may not be committed in a formal document[7, 22].

To prevent confusion and finger pointing if things go wrong, it is highly recommend

that all projects (internal and external) obtain a formal written statement of work with a

financial plan attached. The overall goal is to meet the budget identified and agreed to by

the customer. There are numerous tools available to aid in financial contract

management. Table 2-6 lists three contract management software tools (Perspective[23],

Upside[24], and Contractxlogix[25]) which are available to help manage the financial

aspects of a project.

Table 2-6: Financial Contract Management Software Solutions

All three software tools are available in different sizes to support the budget and scale

of a project. Most are sold by the number of users (sometimes called seats) the project

Select Contract Management Software Tools

Tool Name Developer Description

Perceptive Lexmark “By combining our proven technologies with our industry expertise and best practices, we create specialized sector solutions — such as higher education and healthcare, and departmental solutions like accounts payable and human resources that help organizations: Design, Capture and Process, Collaborate, Protect and Access Information”[18]

Upside Upside Software “Upside Contract is full featured contract management software solution (CLM) that streamlines commitment & contract management including optimization of the sales and supplier contracts. Upside Contract is an integral part of your overall customer and supplier relationship management” [19].

Group, Professional,

Enterprise

Contractxlogix “Contract Logix provides extensive features and tools to help automate your entire contract process” [Logix] Three versions of the software version scaled to user needs [20]

29

expects to need to manage the project’s financial aspects. The software allows the

management team to monitor and report financial progress. The software universally

identifies potential issues if the users are trained enough to understand what the software

is telling them, and has outstanding reporting features.

Reporting is a critical step in problem management[2]. In most cases, reporting does

not solve problems; in fact it can potentially exacerbate the issue. When projects fail to

report a problem, more often than not the problem leads to drastic project damage control

measures. Financial management software facilitates the identification and reporting of

problems.

2.4.3 Schedule Management

The value of applying a systems engineering process to the management of project

schedules has not been fully confirmed. What is known is that, “Schedule overrun

lessens with increasing systems engineering effort and appears to minimize at something

greater than 10% systems engineering effort, although few data points exist to support a

reliable calculation”[1]. INCOSE has undertaken the task of collecting more data about

how systems engineering impacts the overall schedule of a project[1].

If the project is behind schedule, a problem has probably been encountered. The

project risk management process may have identified the risk but been unable to mitigate

it. If a problem management process had been implemented early in the project lifecycle,

the problem might have been identified and resolution plans and alternate plans (work-a

rounds) would have been developed and put into place.

30

Like the financial management process, there are many software tools available to

manage schedule, track delinquencies, and identify/test “what if” scenarios using

simulation techniques. The ability to test potential schedule mitigation actions is a superb

problem management tool and should be included in the project problem management

plan.

One of the most common means to control schedule-related problems is through the

use of the project’s schedule[1]. This is accomplished by scheduling design reviews at

key points during the project life cycle to determine if the project has met all of the

objectives to be completed at a milestone or check point (sometimes called a decision

gate) in the design life cycle.

If the project has problems they are documented and evaluated at this stage. A

decision is made to move forward with reservations, delay until all problems are resolved

or possibly even cancel the project. Figure 2-6 is an INCOSE comparison of various life

cycle models[1]. The comparison was adapted from a figure provided to INCOSE by

Kevin Forsberg[1].

31

Figure 2-6: Comparison of Life Cycle Models[1]

Each model has unique names for the milestones implemented by their model. The

one thing they have in common is that at end of each stage a review is held wherein all of

the progress and problems encountered to date are evaluated. In the DOD model the

three main milestones are labeled A, B and C[1]. In the ISO model the stages are called

Concept, Development, Production, Utilization, and Retirement[1].

The results are the same; if there are too many problems the project cannot continue.

If a problem management system had been in place at the beginning of the project, all

problems would have been available at each decision gate, and resolution plans could

have been developed by the project team[2].

32

2.5 Overlapping Processes

This section reviews processes that overlap the project management and the systems

engineering aspects of a project. Many of the processes are claimed as core concepts and

processes by societies of like-minded individuals such as INCOSE or IEEE. For

instance, risk management is a key process claimed by both the INCOSE and Program

Management Institute (PMI). Neither organization owns the process but each adds value

in slightly different way.

Many times societies come together through the power of integrated product teams

and collaborate to enhance the processes to the betterment of all. The overlapping

processes depicted in table 2-7 are not a comprehensive list but does cover most of the

key processes likely to have significant interface with a problem management

framework[1].

Table 2-7: Project Planning Processes Reviewed in Section 2.5












Schedule Management


33

2.5.1 Risk Management

A risk is the inverse of a problem; however risk management is not the inverse of

problem management. Both management processes are very similar, the major difference

being the goal of the two processes. Risk management seeks to identify problems before

they occur, assess the probability the potential problem may occur, then determine the

consequence if the risk does occur, and lastly, to develop potential mitigation strategies to

prevent the potential problem from occurring. A similar risk management process can be

adapted and used by more than systems engineering and project management[2].

Risk management is also applicable to other fields such as security[21], financial

portfolios[22], actuarial assessments, and public health[26]. Table 2-8 is an interesting

list of different areas risk management is practiced in, the definition of risk management

as defined by the area, and the definition of risk for that area. There are subtle

differences.

34

Table 2-8: Alternative Definitions of Risk Management and Risk

After thorough review of the above industrial areas, it is apparent that each follows

almost the exact same process to identify and manage risk. The specific function of each

industry is irrelevant, and goals of the function are the same.

Figure 2-7 depicts the common risk management process functions.

Industrial Area

Risk Management Definition Risk Definition

Engineering “Is an organized methodology for continuously identifying and measuring the unknowns; developing mitigation options; selecting, planning, and implementing appropriate risk mitigations; and tracking the implementation to ensure successful risk reduction”[1].

“Risk is a measure of future uncertainties in achieving program performance goals and objectives within defined cost, schedule and performance constraints”[1].

Security “enables homeland security leaders to distinguish between and among alternative actions, assess capabilities, and prioritize activities and associated resources by understanding risk and its impact on their decisions”[21].

“The potential for an unwanted outcome resulting from an incident, event, or occurrence, as determined by its likelihood and the associated consequences”[21].

Health “Risk management in health care considers patient safety, quality assurance and patients’ rights”[27].

“is the possibility of a loss or other adverse event that has the potential to interfere with an organization’s ability to fulfill its mandate”[27].

Financial “The probability of loss inherent in financing methods which may impair the ability to provide adequate return”[28].

“The identification, analysis, assessment, control, and avoidance, minimization, or elimination of unacceptable risks. An organization may use risk assumption, risk avoidance, risk retention, risk transfer, or any other strategy (or combination of strategies) in proper management of future events”[28].

35

Figure 2-7: Common Risk Management Process Functions

Planning is “how the project [is] going to conduct risk management”[1]. Planning is

most often captured in a formal risk management plan or as a chapter of the project’s

systems engineering management plan. According to the Software Engineering Institute,

risk planning “is the function of deciding what, if anything, should be done with a

risk”[29]. The decision to fund risk management, develop a local risk register, or

purchase a risk management software tool and the development of a common assessment

criteria are all part of risk planning. A good risk plan will lay out some of the methods of

identifying potential risks. Risk identification is the process of actually identifying risks.

A key point to remember is that unless the risk is written down and communicated to the

program it is not a real risk. After a problem is discovered, many times a project team

member will say, “I thought of that.”

A few potential methods of identifying risks are, brainstorming, lessons learned from

past projects, the reports from the earned value system, word of mouth from suppliers,

and the news media. Expert interview is another way of identifying risk. It is a relatively

36

simple process which consist basically of “identifying the appropriate experts and then

methodically questioning them about risks in their area of expertise”[26] The risk

program should be set up to allow anyone to identify risks, and all risk should be

documented no matter how trivial[10, 21, 26, 29].

Risk assessment is where the project team determines the magnitude of a risk to the

project[1, 10]. A common set of assessment criteria should be established in the risk

management plan and be used to evaluate each risk. Typically risks are assessed against

the likelihood of the risk being realized and the consequences if the risk is realized[10,

21]. After assessment, the project team should determine what the next action will be.

The action taken is called risk mitigation[10, 26, 29]. Risk mitigation can be as

simple as transferring the risk to a vendor or partner (usually money is exchanged for the

service), avoiding the risk by eliminating whatever requirement generated the risk in the

first place, accepting the risk as too minor an impact or unlikely to occur.

The most common mitigation strategy is the development of risk mitigation plans.

“Mitigation plans are probably the most powerful type of action plan you can

develop”[26]. Planning allows the assignment of action steps to individuals, groups,

Integrated Product Teams, companies, and other organizations. When done correctly, a

tight schedule of when each action needs to be finished is developed, allowing the project

team to determine if the risk is going to be realized and turned into a problem.

Tracking the mitigation step is critical to ensure the risk is not realized. Tracking

should be done on individual risks, and reporting should be done to the project office on a

total risk management system basis [10, 21, 29]. The more involvement from senior

management, the more likely the risk program will be successful[10, 21].

37

Once all mitigation steps are complete or the risk has been mitigated to the point that

“reduced the risk exposure below a predetermined point”[29] so it is no longer cost-

effective to manage the risk and it can be closed. A risk can also be closed if it is realized

as a problem or is overcome by events.

2.5.2 Opportunity Management

“Opportunities are events or occurrences that assist a program in achieving its cost,

schedule or technical performance objectives”[17] Like risks there is an uncertainty or

likelihood of achievement component. Opportunities also have a benefit component.

Figure 2-8 Common Opportunity Management Functions are almost exactly the same as

Figure 2-7 Common Risk Management Functions.

Figure 2-8: Common Opportunity Management Functions

38

Smart project teams will try and utilize the same tools and process to manage

opportunities that they use to manage risk[21]. The major difference is with risk the goal

is to mitigate the potential problem (risk). With opportunities, the goal is to enhance the

project’s ability to achieve the benefit. Opportunity management is not as widely

practiced as risk management. As projects fall behind and overruns are incurred,

opportunity management maybe the only way to get the project back on track in times of

tight budgets and stiff competition for project dollars.

2.5.3 Task Definition

At first glance, task definition and project planning appear to be the same process

with different names. This is because task definition is a subset of project planning[1].

“The purpose of the Project Planning Process is to produce and communicate effect

workable project plans”[1]. The subtle difference is project planning puts the tools in

place: systems engineering plan, risk management plan, project organization, assignment

of personnel, et cetera needed to successfully complete a project.

Task definition is more focused on the actual job the project is taking on vice the

framework the project will use to accomplish the task[1]. “The first step is that of

determining what it is that the customer wants. After the customer requirements are

determined, they are translated into a set of technical specifications. The next phase

involves program planning for development of a product that will satisfy the

customer”[16]. Interaction with the customer helps ensure the requirements are

understood by the project team. “It is impossible to develop a meaningful risk

management plan, project execution plan, acquisition strategy, or the alternative design

39

concepts needed for critical decision approval without previously identifying the

requirements associated with the project”[30].

Identifying requirements is a large part of task definition[18]. A good example of this

involves a systems engineer tasked to create a project plan. The job of creating the plan

is task definition. The information or details associated with the deliverable of the task

could be called a task definition. In order to define a task, the project team requires six

critical pieces of information. Figure 2-9 illustrates the six critical task definition

requirement types of questions which need answered in order to develop a task definition

statement. This is not a comprehensive illustration but a basic list of questions to get

started on the task definition process.

Figure: 2-9: Six Critical Task Definition Requirement Questions

40

The order they are answered is not as important as the detail in the answers. The task

definition details (often contained in a work break down structure) are often used to

provide an estimate of cost[7]. The finer the detail, the more accurate the cost estimates.

The second part is the task schedule. Again, the more detailed the task is, the more

accurate the schedule estimate provided to the project negotiating team can be. The task

detail is critical in understanding the problems which might be encountered during the

life cycle of the project. The task definition details of concern are potential risks, should

be entered into the project risk management system and mitigation strategies should be

developed[21].

2.5.4 Knowledge Management

“Knowledge management is a comprehensive term for providing the right piece of

knowledge to the right people at the right time”[31]. Knowledge Management is like any

other systems engineering process. The earlier in the project life cycle the project plans

to actively establish and use knowledge management, the greater the benefits[1].

“Systems may be classed as natural or human-made systems. Human-made systems

may be further classified as physical systems or conceptual and knowledge-focused

systems”[16].

Knowledge Management can be broken into four basic parts:

Known Knowledge: Information already known to humans

Unknown Knowledge: Information unknown to humans

41

Integrated Knowledge: The process of synthesizing known information with

other known information to create a new concepts or products which extend

the human knowledge baseline

Knowledge Planning: Knowledge planning is similar to all of the systems

engineering processes such as risk, opportunity, and configuration

management. It is the development of a strategy for “performing Knowledge

Management as part of the human resources process”[1]. This strategy – the

projects knowledge management plan – must be customized for the project

under development and understood by the whole project team[1].

The project knowledge management plan establishes how the project will maintain the

knowledge known to and created by the project. It also establishes the knowledge

management team, provides project funding, and defines the process for sharing project-

developed knowledge outside of the project. The definition of when and who to share

knowledge with is extremely important due to contractual, copyright, and customer

satisfaction issues[9].

Depending on the size and scope of the project, a formal knowledge management

plan should be part of the project Systems Engineering Management Plan (SEMP) or, for

large projects, a standalone plan referenced by the SEMP[1]. A “key finding about the

knowledge management practitioners is that they have an immense workload. As a result,

they cannot afford the luxury of reading and interpreting lengthy academic

publications”[32].

42

To aide in the organization of knowledge, the project should consider investing in a

knowledge management database[33]. A knowledge management database can be as

simple as a spreadsheet used to capture lessons learned. However, in order to make

knowledge readily available across all projects, companies and organizations (such as

government departments), projects should consider investing in a knowledge

management software suite or hire a knowledge management integration provider to

provide expert assistance in the utilization knowledge management[33].

Table 2-9 is a partial list of list of knowledge management software available on the

commercial market. As more of these types of knowledge management tools sets

become available and the costs associated with completion decrease more and more,

project teams, companies, and organizations will be able to take advantage of the benefits

of applying a robust problem management process to help identify and resolve problems.

43

Table 2-9: Partial List of Commercial Knowledge Management Software

The best reason to develop a knowledge management plan is the reduction of and

resolution to problems identified during the project life cycle[34]. If a problem is

encountered by a project, it has probably been encountered by a different project. If the

resolution has been captured by knowledge management, it may be modified and used as

a solution to the project’s current problems. The re-utilization of knowledge is a key

factor to reduce costs associated with problem management and in the overall reduction

of project life cycle cost by ensuring that reliable information and proven technology is

re-utilized where ever practical during a systems development[31].

Partial List of Commercial Knowledge Management Software

Software Name Vendor Description of Knowledge Management

KPS Solutions “Access to knowledge empowers key support personnel and self service users with the information that they need to quickly resolve queries, thereby increasing service levels at the same time as delivering operational efficiencies”[31]

Knowledgebase Manager Pro (KMP)

“Commonly used to complement a help desk or for sharing information among employees within the organization or business unit. It might store troubleshooting information, articles, white papers, user manuals, or answers to frequently asked questions”[32]

Interspire Knowledge Manager

“Allows you to share information from your website or Intranet with an enterprise-grade knowledge base, reducing customer support, improving staff productivity and eliminating time wasted searching for information across disparate systems such as shared folders and paper documents”[33]

KANA “KCS provides levels of business agility, cost savings and knowledge quality that are essential to dealing with the exploding inquiry volumes, skyrocketing customer expectations and complex technologies that challenge support organizations every day. KANA is a KCS-certified vendor with knowledge management solutions that meet the extensive functionality required to support a KCS-focused support organization”[34]

44

2.5.5 Information Management

“Information integration and management.—It must be possible to access

information of all types easily”[16]. The INCOSE view of the information management

process is as a child of the parent knowledge management process. The INCOSE

purpose statement “the Information Management Process is to provide relevant, timely,

complete, valid and, if required, confidential information to designated parties during

and, as appropriate, after the system life cycle. This process generates, collects,

transforms, retains, retrieves, disseminates and disposes of information. It manages

designated information, including technical, project, organizational, agreement and user

information”[1].

A major concept of information management which helps to resolve many problems

is information storage (vaulting) and version control[1]. Keeping track of who has

accessed information, made modifications, or introduced new information (including the

source of the information) is critical to reducing the number of self-induced problems

encountered during the project’s life cycle[1].

Vaulting includes configuration control of product information, storage of

information used to develop technical manuals and other product information provided to

the end user, blueprints, drawings, test reports, copyright, and other legal documentation.

One the most powerful by-products of information management is keeping track of who

has the information at any given times. Lost documentation can be the source of major

project problems[34].

45

A good knowledge management software suite, such as the ones discussed in Table 2-

9 provides the project with the capability required to manage information. Like

knowledge management, the information management process used by the project should

be formally captured in writing in the projects SEMP or as a standalone instruction[1].

2.5.6 Configuration Management

“There is no engineering of successful systems without any changes; they are the rule

and not the exception in product development”[35] . The Configuration Management

Process is used “to establish and maintain the integrity of all identified outputs of a

project or process and make them available to concerned parties” [4]. Many project

problems are self inflicted by failure to adhere to a sound configuration management

process, by allowing uncontrolled changes in customer requirements, accepting designs

that fail to meet expectations, from material from vendors that does not meet design

expectations, and from software that does not work on newly designed hardware. The

software/hardware problems are an increasingly common problem because “as the usage,

flexibility and complexity of software and electronics increases, and fierce competition

requires shorter time-to-market and customizable products, more frequent releases of

integrated hardware and software configurations become necessary. This evolution

requires adequate configuration management both within the hard- and software

disciplines and across them”[36].

As hardware and software become more and more interactive, it is critical to not only

manage the physical design of the product but also to understand the impacts of design

changes to the software used in and/or developed for the project. To reduce the number

46

of problems encountered by a project during the life cycle of the product, the

configuration must be under strict configuration control[36]. A configuration

management plan should be incorporated in the project’s SEMP or as a standalone

instruction. Configuration management is a relatively simple process but extremely

difficult to execute without a rigid process in place and total project team and (very

important) strong customer support[36]. Figure 2-10 depicts the basic process steps

projects could use to manage the configuration[1].

Figure 2-10: Basic Configuration Management Process Steps

The basic configuration management steps are[1]:

Configuration planning: Planning is similar to the planning steps of the other

systems engineering and project management processes. In this step, the

project configuration management plan is written and formally adopted, the

configuration management organizational structure is developed and

resourced, and the customers are informed of their roles and responsibilities

Planning

Configuration Identification

Control

Verification Validation

Status Reporting

Auditing (Functional and Physical) Execute

47

with regard to change management and should agree to them. Customer

involvement is critical to ensure cost increases are understood and agreed to,

to help decrease problems associated with scope creep, and to help keep the

proposed project delivery schedule[1].

Configuration Identification: At some point during the design phase of a

project, the design is mature enough that the project management team makes

a decision to establish a project baseline[1]. Establishing the baseline

eliminates a whole set of potential problems associated with uncontrolled

changes to the project design. Once the baseline is established, a new set of

problems starts to emerge as engineers, sourcing agents, builders, and the

customer decide to makes changes to the project baseline. The establishment

of the base creates a need to define what can be changed and who can

authorize the change. The items that are controlled through the configuration

management process are designated as Configuration Items (CI’s)[1]. A CI

record is developed. “Configuration item records (includes entries for

hardware, software, documents—including the physical documents, and

change, incident, and problem records) CI attributes and relationships (name,

model, version, location, owner, status, cost, configuration parameters, and

hierarchical relationships with other CI’s). Service performance data,

component usage data, resource performance data, cost data, and service

metric data that needs to be queried by cost centers (e.g., business unit)” [37].

48

Configuration Control: Typically each CI is placed under strict configuration

control. The projects configuration management plan laid out the

organizational structure with the establishment of who can authorize changes

to a CI and when a CI can be changed. Typically, a change management

board is established to review all proposed changes to the project baseline.

Some large projects establish a multi-tiered approval structure. Low cost or

low impact changes may need the approval of a front line manager while high

cost, high risk changes may require the approval of a technical review

authority. One way to overcome scope creep and reduce changes to the

project baseline is to have the customer be a voting member of the change

control or technical review board[1]. This helps ensure the problems

associated with the customer not agreeing to the price escalation of the project

are resolved.

Configuration Status Reporting: The configuration management plan

should lay out when reports should be generated on the status of each CI[1].

Regular reports should be generated on a standardized periodic time table.

Special reports should be created when a problem has been identified to help

define the problem scope and depths. Reports should also be developed and

be presented at design reviews identified in Figure 2-6: Comparison of Life

Cycle Models[1].

49

Configuration Verification and Validation: Ensuring that all configuration

items are in the final product at delivery is part of the validation and

verification process discussed in section 2.3.5 of this paper.

Configuration Auditing: There are basic two types of audits[1];

o Functional audits are used to determine if each CI meets the

requirements of the design (individual attributes such as weight, power

consumption, cost, etc.) Audits also will determine if the sum of the

parts meets the systems requirements for the project design.

o Physical audits are used to demonstrate to the customer that the parts

purchased that are in the design products (technical manuals,

blueprints, etc.) are physically present at delivery.

Configuration Execution: The normal, day-to-day operation of the program as

identified in the project configuration management plan.

2.5.7 Technical Performance Management

“The purpose of the Measurement Process is to collect, analyze, and report data

relating to the products developed and processes implemented within the organization,

to support effective management of the processes, and to objectively demonstrate the

quality of the products”[5]. Keeping track of the performance criteria of each

50

component and the sum of the components’ attributes is critical to delivering a product

which meets the customer’s requirements[38].

Technical performance management is a process that tracks and monitors Technical

Performance Measures (TPM’s). TPM’s are closely related to the CI’s discussed in

section 2.5.6. While CI’s are managed at the component level, TPM’s are measured at

the macro scale and are the roll-up of a single attribute of the project CI’s[7, 14].

For instance, the weight of each component could be added together into one

comprehensive measurement. In a ship construction project, the weight of the ship

would be a technical performance measure used to validate through analysis if the final

weight meets the customer’s requirements. TPM’s are used in this case because it is not

practical to take the ship out the water and weigh it to demonstrate that ship meets the

ship tonnage requirements. The value of using TPM’s is they can be continuously

monitored and adjustments to the design can be made to manage risks and problems

early in the design lifecycle when it is cheaper and easier to make adjustments to the

design[1, 35].

Lewis said it best when she said, “Technical measurement is the process by which the

contractor and the acquisitionist gain insight regarding the definition and development

of technical solutions and continuously assess the risk associated with meeting

performance objectives”[38] .

2.6 Additional other significant processes

There are several key processes outside of the systems engineering and project

management scope which should be reviewed in order to ensure a comprehensive

51

literature review is conducted. The following three processes are instrumental in the

early identification of project problems, in determining what if anything should be done

to address the problem, and in determining if the adjustments the project made are

actually helping correct the problem or have made the situation worse[1, 9].

2.6.1 Decision Management Process

There are hundreds of potential decisions that need to be made over the course of a

project life cycle, which are required to be made just to start-up a project, potentially

thousands over the life cycle of a project. Thus, having a sound decision process in place

at the beginning of a project is of critical importance[1]. Making a decision solves and

creates problems.

Per INCOSE “the purpose of the Decision Management Process is to select the most

beneficial course of project action where alternatives exist”[1]. The most common

decision method is to decide based on expert judgment based on experience, in the form

of “opinions, advice, recommendations, or commentary proffered, usually upon request,

by a person or persons recognized, either formally or informally, as having specialized

knowledge or training in a specific area”[26]. Expert judgment is also the most common

method for assessing the impact of a risk[2].

Project teams do not have to rely on experts though. If the project team has the time

and the money there are commercial and free decision making tools available to aid in the

process.

Figure 2-11 is an INCOSE example of a “Decision Tree for a Bid-No Bid

decision”[1]. The figure illustrates the value of decision trees to a project. Decision trees

52

can be used to make multiple decisions at key points during the life cycle of a system.

One drawback is the outcome must have value, usually in some form of currency (money,

man-hours, etc.) and some probability of occurrence[1]. Often, the probability of

occurrence is based on lessons learned, data mined from the project’s knowledge

management process, or expert opinion of a single person or a team of people[2, 10].

The probability of occurrence must be some value less than one. If it is one, the

decision has already been made and now the project is dealing with problem

management.

Expected Value of Contract Win

10% * $6.05 M + 50% * $4.25 M – 40% * $2.95M, or $1.55M

Expected Value of Making the Bid

60% * $1.55 – 40% * 0.25, or $0.83 M

Figure 2-11: Decision Tree for a Bid-No Bid decision[1]

Other common, methods include classification and regression trees for use when the

variables are continuous and process models which provide a life cycle view of the

53

project[1, 4]. Commercially, there is a large industry dedicated to aiding the customer to

make decisions. Most companies advertise proprietary software which helps the project

team make the right decisions[2].

One aspect of proprietary decision making tools is the logging of the decisions made

[1, 34, 37]. A decision log is an important problem management tool[1]. Whether the

project purchases a commercial product or develops a log using a spreadsheet, a decision

data base typically contains the following types of information: the program foundation

information, the system baselines and the functional architecture, any tradeoffs results

(including assessments of cost, schedule, risk, and growth potential and methods used to

make the decision), the date and time of each action, and records of approval authority

reasoning. The decision log should have a strong interface with the project’s knowledge

management system to ensure lessons learned are available to aid in the risk and problem

management processes on future projects[15, 34, 39].

Decision making is a part of every systems engineering and project management

process. The systems engineer aids in the processes, however, in the end, it is the project

management team’s responsibility to make sound decisions. The project’s problem

management process should be integrated with the decision management system[2, 4].

2.6.2 Cost Benefit Analysis

Cost benefit analysis is a subset of the decision management process discussed in

section 2.6.1. Cost benefit analysis provides a framework which can be used to compare

the pros and cons (in dollars) of project decisions. “It is easy to define benefit-cost-

analysis: simply add up all of the gains from a policy alternatives, subtract all of the

54

losses, and choose the alternative that maximizes net benefits”[40]. It is the decision-

making process after the analysis that creates or solves problems for the project[2].

2.6.3 Earned Value Management

Earned Value Management is a proven process which “provides early insight into

developing trends, indicative of both problems and opportunities, allows a program

manager to focus attention where it is needed and develops and executes corrective

actions to enable the fulfillment of technical and contractual requirements by objectively

measuring the program’s cost, technical and schedule progress”[22].

Objectivity and quality data input are required for earned value management to be a

valuable tool for the project management team[22]. The earned value process uses

collected data to forecast trends in the project’s spending and schedule adherence.

Because of the rear-looking nature of earned value management, when project teams

discover a negative trend they tend to develop risks to mitigate the negative trend.

Earned value management negative trends are in the project’s cost accounting books

thus they have a likelihood of 1 (100%). These results could be indicative of a problem.

The project’s problem management system is the correct place for negative results to be

managed.

The single most important aspect of earned value is measuring and incorporating the

items most important to the project cost and schedule baseline[22]. One way to ensure

the project is using good data is to use technical performance management as discussed in

section 2.5.7.

55

As Solomon states, “EVM can be more effective as a program management tool if it

is integrated with technical performance and if the EVM processes are augmented with a

rigorous systems engineering process”[41]. The day-to-day reports from the earned value

system should be incorporated into the project’s knowledge management system, and

decisions made as a result of the reports should be entered into the project’s decision

database along with the results when known.

2.7 Literature Conclusion

The most notable result of our literature review is the lack of an explicit Systems

Engineering process or framework to manage problems. Government organizations such

as the U.S. Department of Defense’s - Risk Management Guide [10] recommends closing

risks which have been realized. The ISO/IEC Standard for Systems Engineering -

Application and Management of the Systems Engineering Process[5], and the U.S.

Department of Energy Risk Management Guide are prime examples of key documents

which do not address problems at all. Even some of the best books on risk management

such as: Risk Management Fundamentals[21], Continuous risk management

guidebook[29], and Risk management: concepts and guidance[26] note that risks with a

100% likelihood of occurrence should be closed. However, they are silent on what the

next logical step in the Systems Engineering process to resolve problems should be.

Even specialized societies such as IEEE, INCOSE and the Project Management Institute

have not developed a process to identify, manage and resolve problems. There are a

couple of notable exceptions.

56

NASA’s The Procedural Handbook and Project Management of Problems, and

Anomalies[4] and the Information Technology Infrastructure Library (ITIL V3)[42]

contain problem management guides.

NASA’s problem management guidebook was developed as a result of the space

shuttle explosion and is highly safety problem oriented[4]. It can be adapted for other

types of problems and issues. ITIL introduced an update to version 3 of their Services

Handbook in August of 2011. The updated version introduces new Key principles –

including guidance around service requests and request models, and proactive problem

management[42]. ITIL products are Information Systems specific but can easily be

adapted for other types of projects. The follow on chapters of this paper expand upon

our Systems Engineering Problem Management Framework published in the Systems

Engineering Journal[2]. The goal is to provide a universal framework to be used by the

systems engineering community to address risks with a 100% likelihood of occurrence as

well as addressing unplanned or unknowable events which occur regularly during the life

cycle of a program.

3 Problem Management

3.1 Problem Management

3.1.1 Overview

A review of ISO 15288:2008[5] and the INCOSE handbook (V3.2) [1]indicates

there is no direct reference to a problem management process, system, or framework.

However, there are numerous sections of the INCOSE handbook which indicate that a

57

robust problem management process should be developed and implemented. This is a

significant gap in the overall systems engineering process.

Table 3-1 is a sample list of citations from the INCOSE handbook referencing where

problem identification could occur during the development of a system. What is missing

from the INCOSE handbook is a clear definition of a risk, a problem (issue or unplanned

event), what the boundaries between risks and problems are, and how to deal with them.

Table 3-1: Sample List of Citations from the INCOSE handbook[1]

The systems engineering process is designed to identify potential problems and to

develop solutions using a sequential set of processes[1, 2]. Because systems engineers

are often in a unique position between project management and engineering, they are

ideally suited to manage the problem process. This is a logical extension of the systems

engineer’s responsibilities with the risk management process.

Chapter Process Description Comment

3.3.2 Concept Stage Problems identified for individual hardware parts or software modules should be addressed early to minimize the risk that, when these entities are finally designed and verified, they fall short of the required functionality or performance.

Where are the individual hardware and software problems documented when identified?

3.3.4 Production Stage Product modifications may be required to resolve production problems, to reduce production costs, or to enhance product or system‐of interest capabilities.

What is the link to knowledge management?

4.3.1 Architectural Design Process

This process encapsulates and defines areas of solution expressed as a set of separate problems of manageable, conceptual and, ultimately, realizable proportions.

How does the program know which problem to address first? Which is the largest magnitude?

58

To clarify the difference between a risk and a problem: when a problem is identified

at the beginning of a systems engineering process (Concept Stage, Production Stage,

Support Stage, Validation/Concept Etc.), the problem most likely has some likelihood of

occurrence less than 100%. “By INCOSE definition if there is some likelihood of

occurrence less than 100% it is not a problem but a risk”[1].

For instance, if a project team tasked with the development of a new piece of

hardware for a customer identifies a problem that requires a special component to be

developed, the project team may think there is an internal solution and there is adequate

time to complete the component development while still meeting the overall project

timeline. This is a risk and should be entered into the project risk management process.

If the project team is unable to develop the component in time to meet the project

schedule, the likelihood of having the component in time decreases. Once the component

is late, the project now has a problem. Systems engineering literature should be adjusted

to reflect the subtle difference.

Problems could be managed in the risk management system. However, as the Risk

Management Guide for DOD Acquisition points out, “A common misconception, and

program office practice, concerning risk management is to identify and track issues (vice

risks), and then manage the consequences (vice the root causes). This practice tends to

mask true risks, and it serves to track rather than resolve or mitigate risks”[10]. In other

words, it is advisable to separate the unripe apples (risks) from the ripe apples (problems)

because problems are real, costing money now, eating up valuable schedule, and need to

be resolved immediately. Special management, engineering, and supply chain emphasis

should be applied to solve problems because the greater their magnitude, the greater the

59

corresponding threat to the project. Preplanning how to deal with problems will

significantly decrease the risk of a problem scuttling the project.

3.4 Elaboration

3.4.1 Problem Management Concepts Expanded

Murphy’s Law states “If anything can go wrong it will”[43]. A corollary to

Murphy’s Law is O’Malley’s Law “If it can't possibly go wrong, it will”[44]. If Mr.

Murphy and Mr. O’Malley are correct, then projects should create a process to prepare

for the inevitable.

Most projects encounter problems at some point in the project lifecycle. Problems

threaten the ability of the project to successfully achieve the desired end state. The

magnitude of an individual problem significantly increases as the amount of time

available decreases. This is because the closer the project is to scheduled completion

and/or budget exhaustion, the greater the impact on the projects ability to complete on

time and on budget. NASA has established an adaptable problem scoring assessment

format. The NASA Program and Project Management of Problems, Nonconformities,

and Anomalies [4] states problems have two components:

Timeliness of the impact to the project – a scoring which emphasizes time to a

scheduled event such as project completion, a shuttle launch, delivery or other key

project milestone.

The impact to the program – unlike risk consequence, problem impact is realized

as a direct cost and schedule impact to the completion of the project.

60

The Procedural Handbook for NASA Program and Project Management of Problems,

Nonconformance’s, and Anomalies uses a 3 by 5 matrix [4] to manage the assessment

problems, as shown in figure 3-1. Timeframe and consequence class are NASA

constructs which are not clearly defined in the NASA Program and Project Management

of Problems, Nonconformance’s, and Anomalies[4].

Figure 3-1: NASA Problem Management Matrix[4]

Figure 3-2 shows the relationship between Opportunities, Risks, and Problem

assessment methods using the standard 5 by 5 assessment matrix used by many

organizations including the United States Department of Defense, Department of Energy,

and INCOSE. Because risk and opportunity matrices are so common, the problem matrix

is deliberately similar. Thus the same “inherent risk matrix input data biases” [45] are

carried over to the problem matrix. The ease of understanding and common framework

overcomes the inherent biases in assessment.

Timeframe

Consequence Class

A B C

I 1 1 2

II 1 2 3

III 2 3 4

IV 3 4 5

V 3 4 5

High Level

Medium Level

Low Level

61

The major difference in the problem matrix’s x axis is timeliness, a measure of

how soon the project needs to react based on a scoring process which emphasizes time to

scheduled project termination by failure to meet a key event such as project completion, a

shuttle launch, delivery or other key project milestone, and the difference in its y axis is

impact (the worst case severity of the problem). A scoring of impact and timeliness is

conducted and the resulting values are multiplied together to assign the problem a

numerical value. This value is then used (similar to risk scoring) to compare problems

and determine which present the largest threat to the project. If a risk is realized and

moved to the problem management process, ideally the impact should be the

consequence. A validation of the impact should be conducted on all realized risks to

ensure an up-to-date assessment is available for the project management team to use

when determining the amount of project resources to expend and the urgency of response

needed to resolve the problem.

Figure 3-2: Opportunity, Risk, and Problem Management Matrices[2]

62

Opportunity planning works from left to right and up the matrix by increasing the

likelihood of realization of the benefit while increasing the amount of benefit realized.

Risk planning works left to right and down by decreasing the likelihood of the risk being

realized and decreasing the consequence to the program if realized. Problems are

measured with two components:

The impacts the project is currently realizing.

The timeliness of the project impact to the project being terminated.

Problem management works right to left and down by decreasing the impacts to the

project and increasing the amount of time available to deal with the problem[2].

A problem’s impact must be defined within the boundaries of the life cycle stages of

the system. The same or similar problems could be assessed differently depending on

whether the system is in the development, production, utilization and support, or the

retirement phase. For instance, it is easier and less costly to resolve problems with a

faulty design in the concept and development phase than in the utilization phase[2].

The challenge is to understand that problems are going to occur. With this

understanding, the initial system design must be defined so as to allow for risks and

problems that give the best chance of the system meeting project goals. The common

risk process uses two components, likelihood and consequence, to assess the risk.

Consequence is typically broken down into four major categories to further define

risk: technical, cost, schedule, and programmatic. The proposed problem management

framework also uses two components, timeliness and impact, to assess the problem.

Impact is broken down into five categories to further define the problem: technical, cost,

63

schedule, safety/environmental, and programmatic. Figure 3-3 illustrates the interactions

between the five categories.

Figure 3-3: Five Categories of problems[2]

The arrows indicate typical relationships. Rarely will a problem be categorized solely

under just one of the five categories. For example, a technical problem could create

schedule delay problems, which drive cost increase problems, which leads to a lack of

funds to address technical problems which leads to programmatic problems. This is

illustrated by the arrows which indicate the interrelations between the categories. The

hub is the problem management process which defines and ties together all of the

Problem Process

Programmatic Problem

Technical Problem

Cost Problem

Environmental/ Safety Problem

Schedule Problem






64

categories. The project’s knowledge management system is represented as the

foundation process. It is imperative that a knowledge management system be the focal

point for research about past problems similar in nature and the repository for all

information concerning current problems.

Problem Process - The problem management process is the central interface between

the different categories of problems. The problem management process defines the

upper and lower limits of each problem category, weighs the importance of each category

in clearly defined assessment criteria, and defines the reporting thresholds for each. This

allows for consistent assessments so problems are reported in a weighted context in

relation to the technical, cost, schedule, environment/safety, and programmatic impacts to

the project.

The problem process also defines the methodology to capture lessons learned. The

methods and successes of current problem resolution plans along with root causes,

schedule impacts, safety/environmental concerns, and programmatic lessons learned must

be captured and shared with future project teams. A strong interface with the project

knowledge management process is essential to ensure the transfer of knowledge to future

problems’ resolution plans on future projects.

Technical – Technical problems can occur anytime during the life cycle of a project.

Technical problems occur when the system design fails to meet a performance

requirement to meet operability, producability, testability, or integration. Most potential

technical problems ideally would have been identified as technical risks during the

concept phase. Unidentified technical problems can be the root cause of significant

65

safety/environmental, cost, schedule, and programmatic problems in the latter stages of

the project life cycle.

Toyota’s recall associated with sticking accelerator pedals is an excellent example of

a technical problem occurring in the utilization phase of the project. [46] A technical

problem that went undetected through the exploratory, concept, and development phases

of the project was identified deep in the production and utilization phase. The resulting

impact was a recall of 372,000 vehicles due to a major safety problem, which could result

in the loss of life; unplanned costs to the company to resolve the technical problem;

significant legal issues; loss of sales revenue as consumer confidence resulted in less cars

sold; delays in production as the assembly line was halted while a root cause

investigation was conducted; and programmatic problems when government regulators

forced the company to recall all 372,000 vehicles to make repairs.

The overall impact on the company’s stock was enormous. Figure 3-4 is graph

depicting the resulting dip in stock price after the recall was announced. The graph is

purposely smoothed to enhance the dramatic change in stock process. Toyota stock has

still not recovered totally from the problem.

66

Figure 3-4: Toyota Stock Price vs. Time[2]

Schedule – Schedule problems occur as a result of a failure to achieve design

requirements within on allotted timeframe, unanticipated safety/environmental issues,

unexpected cost overruns, or unanticipated programmatic change outside of the project

team’s control. Schedule problems can occur at any time during the project’s life cycle.

NASA, for example, will delay or scrub a launch after detecting an existing or potential

problem. Delaying the launch may allow threats to dissipate or give officials time to

diagnose and repair problems.[47]

Cost – Cost problems occur when the fiscal demand is greater than what was

budgeted. Cost can be measured in man-hours, dollars, Euros, or whatever forms of

65

60

70

80

75

85

95

90

$$$

2010Jul Jul20092008

Impact to Toyota’s stock at the approximate time of disclosure

of the sticking accelerator in 2009

67

currency the project uses to measure the actual amount of resources required to execute

the project. All other problems – technical, schedule, safety/environmental, and

programmatic have cost components. Cost problems may be the number one reason for

project cancellation.

The Federal Bureau of Investigation commissioned Science Application International

Corporation to develop a Virtual Case File software program to upgrade antiquated IT

infrastructure at the bureau.[48] Congress funded the Bureau despite schedule delay after

schedule delay. In the end, the $170 million dollar cost coupled with no foreseeable

software delivery and an external programmatic event on September 11, 2001 was

enough for Congress to halt the project.

Safety/environmental – Safety and environmental problems are not just a class of

technical problems. Safety and environmental problems are assessed by their impact on

human life and the environment. Often, safety and environmental problems are

overlooked as a cost saving initiative in order to get the project off the design board and

into the production and lifecycle phases.

The space shuttle Challenger program provides an example of a safety problem

causing schedule and cost pressures. Safety was considered within acceptable risk. The

ensuing explosion was the result of a safety problem which subsequently grounded the

whole shuttle program for years and cost the American taxpayer’s millions of dollars to

resolve[4]. NASA suffered a severe programmatic (public relations) problem as a result.

Programmatic – Problems produced by events and people beyond the project’s

control are considered to be programmatic. Some programmatic problems are considered

and entered into the project risk management program during the concept development

68

stage of the project life cycle. Alternative plans then typically are discussed. A large

potential problem, such as catastrophic hurricanes on the Gulf Coast, is considered but,

rarely stops a project from going forward. Changes in laws, regulations, and leadership

are some other primary causes of programmatic problems[2].

The timeliness component of problems is the greatest concern to the project. NASA

uses a “timeframe component to categorize identified problems” [4]. The closer

identification of the problem is to the forecasted end of project, significant project

milestone, or other delivery, the greater the magnitude of the problem will be.

During all life cycle phases of the project, timeliness is a measure of how soon the

project needs to react based on a scoring process which emphasizes time to scheduled

project termination. Timeliness of the Toyota accelerator was very high. [46] Once the

problem was formally identified and recognized, production delays, lawsuits, and other

problem impacts were piling up. A timely resolution to the problem was imperative.

Once identified, the accelerator problem was not a risk because the likelihood was

100%. The key is that the problem was initially technical in nature. It became

programmatic as the government got involved via the judicial process. Later a technical

component was re-introduced to resolve the programmatic problem.

Timeliness is a key measurement which allows project managers to categorize all of

the problems in a context that allows for decision making. Those problems, which have a

timing impact of the greatest consequence, are most likely to terminate the project and

should receive most of management’s attention and resources.

69

3.2 Problem Management Framework

Our proposed approach to the problem management framework is based upon the

common concepts of the risk and opportunity risk management processes. This approach

was used because of the familiarity and comfort project management teams have

developed over the past several years as the two approaches have become accepted and

valuable systems engineering and project tools. Figure 3-5 depicts the “Problem

Management Process, Filling the Gap in the Systems Engineering Processes between the

Risk and Opportunity Processes”[2] framework, a step-by-step process to manage

problems.

Figure 3-5: Problem Management Framework[2]

Problem Planning

Problem Identification

Problem Closure

Problem Monitoring

Problem Analysis and assessment

Problem Handling

Put resources in place for effective problem management

Identify problems and issues

Monitor problems and validate/verify effectiveness

of resolution strategy

Decide and implement strategy to handle the

problem

Evaluate Impact and Timeliness. Classify and

prioritize problem

Close problems and pass final history to project KM

process

All Steps of the problem Management Process feed the Project Knowledge Management Process

Step 7

Step 1

Step 3

Step 2

Step 5

Step 4

Step 6

70

3.2.1 Problem Planning - Step 1

Our proposed planning step involves efforts to confirm and/or identify tasks, budget,

and required resources to establish a problem management process. Planning should

consider problem management an important tool within the system engineering and

program management processes. System engineering and program management, based

on sound program management principles, should create a plan which maximizes the

allocation of resources to meet cost, schedule and technical performance objectives which

satisfies all programmatic, safety, and environmental concerns. Problem management

planning incorporates the problem process throughout the entire project life cycle from

conception through retirement. In order to effectively integrate problem management into

a project, periodic re-verification of the plan’s effectiveness should be conducted to

identify gaps in the process and recommend any necessary changes to the project

problem management plan. The problem management planning effort should:

Ensure that problem management is a part of all project reviews.

Identify the need for special problem assessments and reports required to support

major milestones.

Ensure that the problem process is continually improved and integrated with other

program management processes to ensure optimum value to the project.

Ensure that team members receive problem management training.

Ensure that proper guidance is disseminated by project leadership to motivate all

team members to actively participate and practice problem management.

71

3.2.1.1 Problem Planning Flowchart

Figure 3-6 lays out the common steps in a typical problem planning process.

Figure 3-6: Problem Management Framework Step 1

Step 1A is for the project management team to define the project-specific

processes which will be used to manage problems encountered during the life

cycle of the project. Project management with the advice of the systems

engineers should:

o Define the expectations of the problem management process.

o Communicate the expectations to the entire project team.

o Appoint a problem process manager (The project risk manager is often

seen as the most logical person for this job, however the bigger the

project, the more important it may be to separate problems from risks

and opportunities).

o Review past and current projects which are similar in nature, using

knowledge management lessons learned to define a process that will

increase the probability of project success.

Problem Planning: Step 1

1BFunding

Available

1ADefine

Objectives

1EWrite

Problem Management

Plan

1CProcess Defined

1DOrganization

In Place

Stop Stop

y

n

yy

nn

72

o Determine how much the problem management process is likely to

cost the project throughout the life cycle of the project.

o Determine which problem management tool the project is going to use

to manage problems (spreadsheet(s), commercial tool(s), log book(s),

et cetera)

Step 1B is to fund the process. Note that Steps 1A and 1B are best

accomplished as part of the decision by the company or organization to pursue

the project. If a decision is made to utilize problem management prior to

contract negotiations, the costs of the process may be bid in the contract vice

taken on as overhead.

Step 1C is to define the problem management process with particular

emphases on the roles and responsibilities of the project team members and

the interfaces between the other project processes, such as risk and

opportunity management, configuration management, systems requirements

team, et cetera.

Step 1D establishes the problem management team by assigning a team leader

and associate team members. If the project elects to use an integrated product

team approach to develop key pieces of technology, a problem management

team member should be assigned to each team.

Step 1E is to formally write and approve the project’s problem management

plan, publish the plan, and start the formal training of each project team

member, starting with the project management team and the lead systems

engineers.

73

3.2.2 Problem Identification - Step 2

Problem identification is the activity that examines elements of the project used to

identify problems and their root causes, document, and set the stage for successful

problem resolution. Problem identification starts early in successful projects and

continues throughout the project life cycle. Regular reviews and analyses of Technical

Performance Measurements (TPMs), schedule, resource data, life cycle cost information,

EVM data/trends, progress against critical path, technical baseline maturity, safety,

operational readiness, and other project information available to the project team helps to

ensure that problems will be continuously identified from cradle to grave of the project.

3.2.2.1 Who May Identify Problems

Anyone involved with the project may – and is encouraged to – identify and report

problems. Individuals, as well as Integrated Project Team (IPT) members, should review

each element of the Work Breakdown Structure (WBS) under their cognizance and

formally identify any problems. Ideally, they should identify potential problems as risks

and enter them into the risk management process.

The U.S. Army has a unique way to ensure all voices are heard when discussing a

problem. The commanding officer will first ask the opinion of the junior officer present;

discuss the pros and cons of the junior officer’s suggestions and concepts. He will then

move on to the next senior-most officer and repeat the process until every officer has had

a chance to discuss the problem and present his or her recommend solution.

74

Why does the Army do it this way? Lessons learned indicate that when a senior

officer provides input first, the junior officers almost always follow the senior’s lead even

if they have a different opinion. As part of the training message, the Army considers it

important to ensure the project team understands that the early identification of a problem

(or risk) will be rewarded and that all team members both internal and external are

encouraged to identify problems.

3.2.2.2 Problem Identification Flowchart

Figure 3-7 lays out the common problem identification process steps.


Step 2A is the actual identification of problems. Problems can be identified by

anyone at any time during the project’s lifecycle.

Step 2B documents all problems in the problem management tool or database.

Some key points to document are:

o Who discovered the problem

o When the problem was discovered

o What actions have been taken to date

Problem Identification: Step 2

2BDocument Problems

2AProblem

Identification

75

3.2.3 Problem Analysis and Assessment - Step 3

Problem analysis and assessment is the two-process step by which problems are

examined in further detail to determine their extent.[4] Problem analysis involves

investigating the cause of the problem to determine its cause and identify appropriate

proactive steps to correct the problem. Problem assessment then takes the analysis and

evaluates the consequence and timeliness in a context to the project management plan.

The assessment scores the problem in concert with all of the other problems the project

currently faces.

3.2.3.1 Analysis

The first step is to analyze the problem. There are many methods, processes, and

software tools commercially available to aid in the analysis of why a problem occurred, a

process also sometimes called ‘root cause analysis.’ There are even companies that

specialize in problem analysis. For instance, Apollo Associated Services, Ltd. advertises

that they specialize in teaching a process that determines the real root cause and seeks to

identify a solution set that will prevent problem recurrence.[49]

NASA also recognized the importance of a robust problem management process.

NASA developed the “Procedural Handbook for NASA program and Project

Management of Problems, Nonconformities, and Anomalies”[4].

The INCOSE handbook contains excellent appendices which are dedicated to data-

mining and analysis methods to aid in problem identification. When analyzing problems,

program and project managers should at least ask the following questions: [4]

76

How soon do we need to act?

What will happen if we do not act?

How does this problem compare with other problems?

3.2.3.2 Assessment

The second part is to prioritize the problem, then classify or group the problem

with similar problems. This can only be done by using common assessment criteria,

which should be determined by the project systems engineering team at the start of the

project. The criteria should be defined in a chapter of the project Systems Engineering

Management Plan or a standalone problem management plan. Like risk management,

problem management assessment is accomplished in two parts.

Impact: The impact is similar to risk management consequence. It is a

representation of the current and projected impact that the problem is having on

the project.

Timeliness: The timeliness scale is based on how soon the project team needs to

act on the problem. The end scale is based on time to project termination if the

problem is not resolved.

Together the impact and timeliness provide a score which, when compared to

other project problems (assessed using the same common scoring criteria), allows the

determination of which problems need immediate attention and how much valuable

project assets should be expended resolving the problem.

77

3.2.3.3 Problem Analysis and Assessment Flowchart

Figure 3-8 lays out the common problem identification process steps.


Step 3A is where we determine the root cause of the problem. Determining and

understanding the root cause is extremely important to the development of a

problem resolution plan. It also helps the project determine if there are potentially

similar problems in other areas of the project that have not yet been identified yet.

Step 3B is accomplished using the assessment criteria developed and documented

in Step 1. The current assessment is how bad the problem is right now. It is

critical to understand how much effort (cost and schedule) has been expended to

date so that the project management team can use the information in Step 4.

Step 3C is the future impact of the problem, in other words, how bad the problem

impact could be if the project does nothing different. It is also developed using

the criteria developed in Step 1. The future impact is developed from the baseline

established in step 3B.

Problem Analysis and Assessment: Step 3

3EDocument the

Root cause and the

Assessment

3AAnalysis

Determine the root cause of the

problem

3BAssess the

current impact of the problem

3DAssess the Timeliness

of the Problem

3CAssess the

Future Impact of the

Problem

78

Step 3D is the determination of the timeliness of the problem. ‘Timeliness’ is an

assessment of when the problem will cause the project to shut down due to the

impacts of the problem if nothing new is done.

Step 3E is the documentation of the problem in the problem management database

(which is integrated with the project’s knowledge management system) and

facilitates Step 4 by capturing pertinent information for use on future projects.

3.2.4 Problem Handling - Step 4

Once a problem has been identified, assessed, and fully understood, a program

management decision needs to be made about the method to resolve the problem.

Problem resolution strategies are very similar to risk handling methods. The following

are generally accepted problem management options.

3.2.4.1 Problem Acceptance

The problem is acknowledged and the impacts are accepted. Acceptance rationale

should be documented for historical reasons by using the project’s knowledge

management process. Only problems with extremely low impact and ratings are

candidates for acceptance.

3.2.4.2 Problem Avoidance

Problem avoidance is a strategy for problem resolution to eliminate the problem

altogether. It is used when a “lose-lose” situation is likely. An example of avoidance

would be when a project has an unachievable requirement from the customer. Instead of

spending additional time and money trying to resolve the problem, the project team (with

79

the customer’s concurrence), changes the requirement to a more achievable one or

eliminates the requirement altogether. Ideally, this form of problem should be identified

early and documented using the risk management process, fallback options and schedules

should be identified, and the requirement should be changed before it becomes a

problem.

3.2.4.3 Problem Transference

When it is possible to identify another party in a position to efficiently assume the

problem, then an agreement may be reached to transfer the problem to that party. Such

problem transfer is commonly accompanied by the payment of a premium to the

problem-assuming party. Example Problem-assumption mechanisms include warranties

and insurance policies.

One mitigation strategy that includes problem transfer would be to outsource the

entire portion of a contract that contains the problem item, thus making the problem the

responsibility of a subcontractor. The number one issue projects need to remember when

implementing this strategy is that if the subcontractor fails to resolve the problem, the

project is still ultimately responsible. A project risk should be generated using the risk

management process when transferring a problem to another party.

3.2.4.4 Problem Resolution

Problem resolution and risk mitigation are very similar processes. Problem resolution

is a strategy which identifies tasks that, when implemented; reduce the problem to an

80

acceptable level in accordance with the projected problem-assessment level when at the

completion of each task.

Associated with the development of the resolution plan is the determination of

resources and costs to implement the plan. This information is provided with the plan to

support the approval process. Resolution goals can change as circumstances and

conditions improve or deteriorate, or as constraints on the project force acceptance of less

than optimal solutions.

Depending on the assessment of the problem, resolution plans should be intensely

managed by the project team and frequently reviewed to ensure adherence to a sound

resolution plan.

3.2.4.5 Problem Handling Flowchart

Figure 3-9 lays out the proposed problem handling process steps.


Step 4A occurs if the project management team decides to accept the problem

with no additional action for one of two reasons:

Problem Handling: Step 4

4BAvoid the Problem

4CTransfer problem

n

yy

nn4A

Accept the Problem

y

Obtain Customer Concurrence to

Make a Contractual

Change

Transfer the Problem ,

develop Risk to Mitigate if the Transfer was Ineffective

4DDevelop Problem resolution Plan

and document in the Problem

Database

Stop go to step 7A

81

o The problem has very minor consequences, and the project is willing to

accept the added cost and schedule penalties.

o There is nothing further that can be done to mitigate expected

consequences.

Step 4B occurs when the project team avoids the problem by deleting the source

of the problem from the project scope.

Step 4C occurs when the project decides the best was to eliminate the problem is

to transfer it to someone else to resolve.

Step 4D is the most common method of resolution. Based on the information

collected in the steps above the project develops a resolution plan to eliminate the

problem. It must be understood that since the problem has already occurred the

incurred costs and/or schedule setbacks will have to be addressed in the resolution

plan.

3.2.5 Problem Monitoring - Step 5

Problem monitoring is a continuous process to systematically track and evaluate the

performance of problem resolution actions. Effective tracking provides information to

indicate whether problems are increasing beyond project limitations despite handling

actions.

The tracking information must be available in sufficient time to support corrective

actions. Problems should be agenda items at each project management review. Openly

discussing problems provides an opportunity for all stakeholders to suggest approaches

for reducing the impacts of problems to an acceptable level or finding alternative

82

solutions, such as problem avoidance. Communication facilitates early action and

minimizes adverse impacts.

If tracking results indicate an increase in problem impact or timeliness, the level of

the problem should be re-evaluated and adjusted accordingly. Changes to the resolution

plan may be required.

3.2.5.1 Problem Monitoring Flowchart

Figure 3-10 lays out the common problem monitoring process steps.


Steps 5A and B are monitoring steps.

o If the plan is successful, the project team should proceed to step 6.

o If the resolution plan is unsuccessful, a new resolution strategy must be

selected by repeating step 4. A new resolution plan may be developed or

the project may choose to accept, transfer, or avoid the problem.

Steps 5C and 5D are also monitoring steps.

Problem Monitoring: Step 5

5CMonitor Problem transfers

Resolved?

yy

nn5A

Monitor the problem

resolution plan. Resolved?

5DDevelop alternate

resolution by repeating step4D

5BDevelop alternate

resolution by repeating step 4

Stop go to step 7A

Stop go to step 7A

83

o If the party the problem was transferred to resolves the problem, the

project team should proceed to Step 6

o If the resolution plan is unsuccessful a new resolution strategy must be

selected and step 4 is repeated. A new resolution plan could thus be

developed or the project may choose to accept, transfer, or avoid the

problem.

3.2.6 Problem Closure - Step 6

A problem should be closed when:

It meets the target value stated in the resolution plan.

It no longer exists.

It is no longer cost effective to manage.

The project accepts the problem with no resolution.

The project is cancelled.

Information regarding closed problems is maintained in the problem database and

may be used as:

A baseline for monitoring problem resolution actions and verifying the results.

Background material for new projects and programs.

Rationale for future program decisions.

Identification of trends inside and outside of the project.

3.2.6.1 Problem Closure Flowchart

Figure 3-11 lays out the problem closure process steps we propose.

84


Step 6A is a decision point. The problem manager decides the problem can be

closed.

Step 6B occurs when the project management team agrees and the problem is

closed.

Step 6C occurs if the project management team does not agree to close the

problem and returns to Step 4 to restart the process of deciding which additional

steps will be required to resolve remaining issues.

3.2.7 Problem Knowledge Management – Step 7

Knowledge management is a fundamental system engineering requirement[31, 32,

34]. All project documents and records should be captured by the project knowledge

management process though all stages of the project’s life cycle as part of the project’s

information management process. Individual problems along with successful and

Problem Closure: Step 6

yy

n

6AThe problem

has been resolved?

6CPermission Denied,

determine why and return to step 4 to adjust resolution strategy

6BObtain

permission to close the problem Document it in

the project problem database

yStop go to step 7A

6CGo to step 4

n

85

unsuccessful resolution plans are particularly valuable for future projects as the starting

point in the risk management process.

3.2.7.1 Problem Knowledge Management Flowchart

Figure 3-12 lays out our proposed knowledge management capture step.

Figure 3-12: Problem Knowledge Management Process Step 7

Step 7A captures all of the data collected to identify, analyze, assess, resolve,

strategize, monitors, reports, and closure justification for the problem.

4 Recommendations and Conclusions

4.1 Recommendation

The following describes a potential systems engineering process to manage problems

and recommends this process as an addition to the project’s Systems Engineering

Management Plan between the risk management process and the opportunity process.

For ease of understanding, the potential problem management process is formatted as

described in section 1.4 of the INCOSE handbook (V3.2)[1].

Problem Knowledge Management Process: Step 7

7ACollect all data associated with the problem and

document in the knowledge management system

86

4.1.1 Purpose

The following is a recommended purpose statement for a problem management

process:

The purpose of the problem management process is to identify, report, analyze,

develop resolution plans, assess impacts, and monitor problems as they are identified.

The problem management process systematically searches for problems, directs problems

with a potential for occurrence into the risk management process, and forwards risks

with 100% likelihood of occurrence into the problem management process. Each

problem is continuously managed for the life of the problem throughout the life cycle of a

system or service. Problem management can be applied to all phases of the system

engineering life cycle. A history of each identified problem is stored in the project

knowledge management system as a lesson learned for future projects.

4.1.2 Description

The primary objective of the problem management process is the early identification

of problems. To aid in identification, there are customized problem management

processes available for select applications. For instance, trouble call or trouble desks

may be used to address customer issues and capture resolution history as lessons learned

to solve future problems.

NASA created a Problem Management Guidebook called “The Procedural Handbook

for NASA Program and Project Management of Problems, Nonconformance’s, and

Anomalies”[4] which can be used for business related problems. The number one thing

trouble desks and the NASA problem process have in common is the decision to take a

proactive approach to problem management vice a reactive approach.

87

The problem process should be a proactive approach which aggressively identifies

potential problems and processes them in a manner similar to the risk management

process. For those problems identified, a formalized reactive problem management

approach is justified. No matter how the problem is identified (realized risk or stumbled

upon), processes of formalized reporting, root cause analysis, prioritization, resolution

planning and retirement planning should be in place prior to the beginning of the project

to capture and manage problems. The plan should lay out the project’s comprehensive

strategy for dealing with problems.

Like risk management, there are typical strategies for coping with problems,

including transference, avoidance, acceptance, and resolution planning. Strategy

selection may be determined by the scope and impact of the problem. In such instances,

a prioritization schema is critical to make certain that the right problems are addressed

with adequate assets allocated to ensure timely resolution of the problem.

Unlike risks, problems require immediate response to assess and determine their

impact to the project. By having a rapid assessment process in place as part of the

problem management process, a quick initial decision can be made by the project staff.

Those problems with the greatest potential to derail the project will, of necessity, receive

most of the project management team’s attention and project resources.

Figure 4-1 depicts a notional context diagram in the format of INCOSE Handbook

version 3.2 [1] for the proposed problem management process.

88

Figure 4-1: Context Diagram for the Problem Management Process Inputs to

Activities

Inputs to the problem management process include the following:

Identified problems as a result of technical reviews, audits, or other scheduled project

reviews.

Realized risk(s) from the risk management process.

Unplanned events and suspected problems identified by abnormalities in the Earned

Value Management System or Technical Performance Monitoring System.

Frequently problems are identified late in the life cycle of a system. Speciality

engineering activities, cost-effectiveness analysis, environmental impact analysis, life

cycle cost analysis and other activities can also be used to identify problems that

significantly impact the project.

The problem management process is controlled by:

Activities-Problem Management Planning-Manage the Problem Profile-Analyze Problems-Resolve Problems-Execute the Problem Management Process

Controls-Applicable Laws and Regulations-Government and Industry Standards-Contracts and Agreements-Project Procedures, Plans, and Standards Directives- Corporate Code of Ethics-System Engineering Processes

Enablers-Organization/Enterprise Policies, Procedures, Plans, and Standards-Organization /Enterprise Infrastructure-Project Infrastructure

Inputs-Identified Problems-Realized Risks-Unplanned Events

Outputs-Problem Management Plan-Problem Profile-Problems Reports-Knowledge Management Inputs- Problem Resolution Plans

89

Applicable laws and regulations.

Government and industry standards.

Contract agreements.

Project-specific procedures, plans, standards, directives and planning documents.

The corporations code of ethics.

System engineering processes.

The problem management process is enabled by the following:

Organization/Enterprise policies, procedures, plans and standards.

Organization/Enterprise infrastructure

Project infrastructure

4.1.3 Outputs from Activities

Outputs from the problem management process include the following:

Project problem management plan.

Problem profile (Inventory) – A database similar to the project risk register from which a

problem matrix can be extracted to display an inventory of problems (open, closed and

accepted). Problem matrices are similar to risk matrices as they are developed using

cognitive biases [45] which affect the placement of problem points developed by

assigning a numerical value to each level.

Problem Report – the vehicle used to communicate the current problem inventory to the

project stakeholders. Problems reports are highly customized for the particular audience.

For select problems (as determined by the project problem management team), detailed

problem resolution plans, root cause analyses, and impact assessments are developed. In

addition, a list of alternatives solutions is generated for project management decision

making.

90

Problem management knowledge transfers to lessons learned, knowledge management

systems, and other projects.

4.1.4 Process Activities

The Problem management process includes the following activities:

Plan problem management:

Defining the problem identification process by establishing project specific

thresholds (cost, schedule, technical, programmatic, and safety

environmental) to determine problem initial reporting levels.

Defining and documenting the problem management strategy.

Defining and documenting the problem management process.

Establishing criteria for what a problem is and defining the boundaries of

which problems to deal with first – i.e. an assessment criteria to help the

project establish priorities.

Developing and implementing a communications plan to keep stakeholders

informed of the project’s problems.

Problem analysis:

Conducting root cause analysis.

Analyzing the problem to determine the magnitude of the impact and the

timeliness.

Determining a resolution plan and the amount of additional resources to be

allotted to resolving the problem.

Determining the person or organization responsible for resolution plan

execution and continuous reporting of the problem resolution status.

91

Reporting (at a minimum) the problem’s cost to date, cost expected in the

future, schedule impact to date, schedule impacts expected in the future,

safety impacts, technical issues and fallbacks.

Problem resolution:

Using established problem boundary criteria to identify those problems

which require an approach which is inside of the project charter. In an ideal

project, this class of problems would have been already entered into the risk

management system and have a mitigation plan which failed to prevent the

risk from being realized or, if no fallback (work-around) plan has been

prepared, a problem resolution plan and alternative strategies to deal with

the problem would be developed. It is noteworthy that mitigation plans are

developed to prevent risks from being realized (addresses a potential future),

while resolution plans are developed to address the impacts of a problem (in

the present).

For problems identified outside of the risk management process, the

established problem boundary criteria for dealing with problems within the

project charter should be used to develop a problem specific resolution

action plan and alternative strategies to resolve or reduce the problem to an

acceptable level.

Presenting the potential resolution plans to the project management team,

who will select the optimum strategy and authorize execution of the plan.

Management of the problem profile:

92

Developing a comprehensive problem register in which a record of problem

items is kept, including: date identified, personnel involved, actions to date,

root cause analysis, assessment and assessment basis, timeliness, resolution

action plan, and every communication dealing with the problem.

Development of an electronic problem register is recommended. One

potential solution is development of an extension of the project risk

management software to capture problems (and opportunities) in one readily

available central location.

Developing a two-way interface between the problem management register,

the projects risk management register, and the project knowledge

management system.

Evaluation of the problem management process:

Developing a problem management plan or adding a chapter to the project

processes section of the project’s SEMP.

Common approaches and recommendations:

Customizing an established risk management plan to meet the problem management

needs of the project.

Establishing a project-specific document used to aid in the documentation of individual

problems. Like the Risk Information Sheets, a Problem Resolution Sheet documents a

description of an individual problem, assessment (initial and current), priority, resolution

plan, and the primary and secondary points of contact for information regarding the

problem.

93

Extending the risk rule of thumb of using “If <situation> , then <consequence> as a

question to pose for each risk candidate” [1] for problem management, the recommended

rule of thumb is “Currently < to describe the problem as it currently is> and Timeliness

<to describe the problem in the context of time to project termination>” [4] Timeliness is

a concept developed by NASA as part of their “Procedural Handbook for NASA Program

and Project Management of Problems, Non-conformances, and Anomalies”. [4]

A complete record of all actions and persons involved with the problem cause,

identification, analysis of current and future impact, resolution actions to date, and

management decisions will aid in recreating the problem environment, which will

significantly aid the project in planning future resolution actions, determining how much

project assets were expended in problem resolution, and determining who the action

parties were.

Use tools to identify problems and potential resolution actions. Common methods are:

Failure Modes and Effects Analysis (FMEA),a formal process or study in

which a subject is examined in detail and risk is assessed.[50] The FMEA

process is proactive and can be used to determine why, where, and how

failure occurred and to assess the relative impact of different failures in

order to identify the parts of the process that are most in need of change.

Trend Analysis/Data Mining, a process used by NASA to uncover patterns

and trends from existing databases in order to employ problem management.

[4]

Apollo Root Cause Analysis method, simple tools and the knowledge

necessary to find effective solutions. [49]

94

The Procedural Handbook for NASA Program and Project Management of

Problems, Non-conformances, and Anomalies, an excellent source as basic

guidance in the development and understanding of safety- and technical-related

problem management.

ISO/IEC guides and risk assessment techniques, excellent starting points for

common definitions.

4.2 Conclusions

“People understand that bad things can happen, what they do not forgive is

unpreparedness”[2]. The INCOSE system engineering process is the recognized

methodology to develop a system by resolving the inherent design, engineering and other

lifecycle problems using a methodical, proven process. Embedded in the process are sub-

processes to manage risk and opportunities. One key process is missing: problem

management.

The larger the project, the more likely it is that significant problems will occur. Risk

management addresses known potential problems with robust planning and mitigation.

When risk management is unsuccessful, the risk should be closed and problem

management should take over. Unfortunately INCOSE does not provide a framework or

process to aid in the management of problems.

This paper provides a starting point for a problem management process that could be

added to the next revision of the INCOSE handbook. It is recommended for insertion

into Chapter Five, Project Processes as a task between risk and opportunity management.

Overall, it would be best if the risk, opportunity and problem management processes

were combined into one comprehensive framework which could be adapted to each

95

projects use. It is critical that projects understand the full spectrum of problems they face

not only in the beginning of the project but throughout the entire project life cycle. The

management team must be fully informed with the latest information available on all of

the problems at any given moment. The team must communicate with all stakeholders

about what the problems are and what the actions are being undertaken to resolve them.

INCOSE is the appropriate professional authority to sponsor and establish a robust

System Engineering Problem Management process.

5 Recommendation for Future Research

5.1.1 Opportunity Management Framework

The current opportunity management section of the INCOSE handbook lacks clarity.

A review of other literature sources indicates that other, more comprehensive research

has been conducted but fails to address the following situations:

If an opportunity has been successfully achieved what is it called? It should no

longer be called an opportunity to help separate the gains from what has not yet

been realized.

When an opportunity does succeed, does the project management team harvest the

gains from the successful department or does the project redirect the gains to

other areas?

When the gain is on schedule how does the project reward the successful

department? Do they close the department early and lay off people? Do they add

more work to the successful department? The penalties for successfully

96

harvesting a project can be detrimental to the success of the project opportunity

management system.

In many cases there is no sure way to determine if the opportunity has been

realized. What is the method used to determine if the opportunity enhancement

plan was successful needs to be developed?

5.1.2 Problem, Risk, Opportunity, Realized Opportunities, and Earned Value

Management Integration

A review of the INCOSE handbook and the applicable standards indicates there is no

chapter on earned value management. U.S. government and many commercial contracts

require a certified earned management system be in place at the beginning of a project’s

life cycle. Earned value has significant interfaces with many systems engineering and

project management processes. Additional research in the areas below is recommended:

Earned value interfaces with systems engineering processes such as risk,

opportunity, and decision management.

Tailoring earned value based on the systems engineering technical

performance management process.

The effects of aligning the earned value management work breakdown

structure with the systems engineering work breakdown structure.

5.1.3 Total Systems Engineering Framework

Additional research could be conducted to further the development of a total systems

engineering framework which incorporates all of the various systems engineering and

97

management processes reviewed in Chapter 2 of this paper. The goal is a comprehensive

framework that integrates the best of the best of the various frameworks currently in use

by project teams. The framework should, at a minimum, establish a hierarchy of what

processes are needed at what stages of a project life cycle and ensure all needed processes

are included in the SEMP and project planning documents. The comprehensive systems

engineering framework could be used as a guide in contract negotiations to ensure the

project has priced the right work at the right time in the project life cycle.

98

6 BIBLIOGRAPHY

[1] R. D. Hamelin, et al., INCOSE Systems Engineering Handbook vol. 3.2:

INCOSE, 2010. [2] B. A. Olson, et al., "Problem Management Process, Filling the Gap in the Systems

Engineering Processes between the Risk and Opportunity Processes," Systems Engineering, vol. 15, 2012.

[3] P. F. R. a. M. King, "Problem Management: What Happens When Risks Become Problems? ," P. F. Richardson, Ed., ed: NASA, 2005.

[4] NASA, "Procedural Handbook for NASA and Project Management of Problems, Non-conformances, and Anomalies," S. a. M. Assurance, Ed., Baseline ed. Washington DC: NASA, 2008, p. 86.

[5] ISO/IEC, "ISO/IEC Standard for Systems Engineering - Application and Management of the Systems Engineering Process," ISO/IEC 26702 IEEE Std 1220-2005 First edition 2007-07-15, pp. c1-88, 2007.

[6] J. J. Randolph, "A Guide to Writing a Dissertation Literature Review," Practical Assessment Research and Evaluation, vol. 14, p. 13, June 2009 2009.

[7] A. Kossiakoff and W. N. Sweet, "Systems Engineering Principles and Practice," ed: John Wiley & Sons, 2003.

[8] A. D. Hall, A methodology for systems engineering: Van Nostrand, 1962. [9] B. S. Blanchard and W. J. Fabrycky, Systems engineering and analysis: Prentice-

Hall, 1981. [10] D. A. University, Risk management guide for DOD acquisition: Dept. of Defense,

Defense Acquisition University, 2002. [11] J. Wasek, "Introduction to Architectural Frameworks and Enterprise

Architecting," in System Architecture: The Goal, ed. Washington DC: Department of Engineering Management & Systems Engineering (EMSE) George Washington University, 2009.

[12] I. Merriam-Webster, Merriam-Webster's collegiate dictionary: Merriam-Webster, 1993.

[13] S. A. Bernard, An introduction to enterprise architecture: Authorhouse, 2004. [14] P. Checkland, Systems thinking, systems practice: John Wiley, 1999. [15] H. Eisner, Essentials of project and systems engineering management: Wiley,

1997. [16] A. P. Sage and C. L. Lynch, "Systems integration and architecting: An overview

of principles, practices, and perspectives," Systems Engineering, vol. 1, pp. 176-227, 1998.

[17] G. Rebovich. (2010). Enterprise Systems Engineering : Advances in the Theory and Practice (1 ed.). Available: http://gwu.eblib.com/patron/FullRecord.aspx?p=555713

[18] K. Forsberg, et al., Visualizing project management: Wiley, 1996. [19] B. Reed, "Integration, Verification And Validation," in Professional Certificate in

Systems Engineering vol. IV&V, ed. Newport News: Reed Integration INC, 2007.

99

[20] H. M. Kevin Forsberg, "The Dual Vee Illuminating the Management of Complexity " in System Engineering: Shining Light on the Tough Issues, Orlando FL, 2006, p. 32.

[21] R. Beers, "Risk Management Fundamentals," D. o. H. Security, Ed., ed. Washington DC: Homeland Security, 2011, p. 31.

[22] A. B. Kumley, P. ; Coleman, R. ; Abba, W. ; Infanti, G. ; Driessnack, J., "Integrating Risk Management with Earned Value Management: A Statistical Analysis of Survey Results," in 5th:, JOINT INTERNATIONAL CONFERENCE AND EDUCATIONAL WORKSHOP, Denver, 2005, p. 32.

[23] Lexmart, "Perceptive," 6.5 ed. Kansas City: Lexmart, 2011, p. Contract Management Software.

[24] U. Software, "Upside Contract ", 7.1 ed. Alberta, Canada: Upside Software Inc, 2011.

[25] C. Logix, "Contract Management Solutions," ed. Chelmsford, MA, 1997. [26] C. L. Pritchard, Risk management: concepts and guidance: Taylor and Francis,

2005. [27] J. Miller. (2011, The Definition of Risk Management in Health Care. Available:

http://www.ehow.com/about_6619711_definition-risk-management-health-care.html

[28] B. O. Dictionary. (2011, November 22, 2011). Financial Risk [Electronic]. Available: http://www.businessdictionary.com/definition/financial-risk.html

[29] A. J. Dorofee, et al., Continuous risk management guidebook: Carnegie Mellon University, 1996.

[30] N. N. S. Administration, "Managing Design and Construction Using Systems Engineering," vol. DOE G 413.3-1, D. o. Energy, Ed., ed. Washington DC: Department of Energy, 2008, p. 66.

[31] B. Fagerström and L.-E. Olsson, "Knowledge management in collaborative product development," Systems Engineering, vol. 5, pp. 274-285, 2002.

[32] B. Serenko, Hull, "Practical relevance of knowledge management and intellectual capital scholarly research: Books as knowledge translation agents," Knowledge and Process Management, vol. 18, pp. 1-9, Feb 2011 2011.

[33] Interspire. (2011, Software). Interspire Knowledge manager (5.0 ed.). Available: http://www.interspire.com/knowledgemanager/

[34] M. Rao, Knowledge management tools and techniques: practitioners and experts evaluate KM solutions: Elsevier Butterworth-Heinemann, 2005.

[35] E. Fricke, et al., "Coping with changes: Causes, findings, and strategies," Systems Engineering, vol. 3, pp. 169-179, 2000.

[36] R. Krikhaar, et al., "Enabling system evolution through configuration management on the hardware/software boundary," Systems Engineering, vol. 12, pp. 233-264, 2009.

[37] Y. Liu, et al., "Configuration management process design and implementation," in Computing, Communication, Control, and Management, 2009. CCCM 2009. ISECS International Colloquium on, 2009, pp. 4-7.

[38] T. Lewis, "Quantitative approach to technical performance measurement and technical risk analysis utilizing Bayesian methods and Monte Carlo simulation,"

100

The George Washington University Ph.D., The George Washington University, United States -- District of Columbia, 2010.

[39] K. M. Pro. (2011, Knowledge Management (6.0.3 ed.). [40] E. M. Gramlich, A guide to benefit-cost analysis: Waveland Press, 1990. [41] P. J. Solomon, "Integrating Systems Engineering with Earned Value

Management," Defense AT&L, vol. 33, pp. 42-46, 2004. [42] S. Adams, ITIL V3 foundation handbook: Stationery Office, 2011. [43] A. Roe, "Making of a Scientist " in New York City New York, ed: Praeger, 1974,

p. 244. [44] M. H. Roberts. (2010, Project Management Jokes Humour Proverbs and Laws

(2nd ed.). Available: http://www.hraconsulting-ltd.co.uk/project-management-jokes-humour-proverbs-laws.htm

[45] E. D. Smith, et al., "Risk matrix input data biases," Systems Engineering, vol. 12, pp. 344-360, 2009.

[46] Toyota.com. (2011, Toyota Announces Two Voluntary Recalls and Amends Potential Floor Mat Interference Recall Announced in 2009. Available: http://pressroom.toyota.com/safety-recall/

[47] T. V. Wilson. (07 September 2006, Why are space shuttle launches delayed so frequently? [Electronic].

[48] T. Al Neimat. (2005, Why IT Projects Fail. The Project Perfect White Paper Collection [White Paper]. 8. Available: http://www.projectperfect.com.au/downloads/Info/info_it_projects_fail.pdf

[49] A. A. S. LTD, "Apollo Root Cause Analysis," A. LTD, Ed., February 2006, R2 ed: Apollonian Publications LLC, 2005.

[50] M. A. Anleiter, The power of deduction : failure modes and effects analysis for design Milwaukee: ASQ Quality Press, 2010.

problem management: a system engineering management

Documents