probabilistic cost, schedule, and risk management

Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012

1/186

This briefing is an overview of the

probabilistic risk analysis

processes that can be applied to

our program. Although it may not

appear to be a “simple” overview,

this material is the tip of the

iceberg of this complex topic.

Just schedule analysis has been

addressed in detail here. The

cost aspects of forecasting and

simulation must be addressed as

well to complete the connections

between schedule and cost.

Probabilistic cost will be surveyed

here, but an in depth review is for

a later time.

Prepared for NNJ05111915R, by Glen B. Alleman

December 2005


2/186

An important aspect of education and research in our business domain, is “Fair Use” copyright law.

All the material in this briefing is accessible through the internet. Conference proceedings journal articles, company white papers and other public sources form the basis of much of this material and are referenced in the bibliography.

Some materials in this briefing make references to other copyrighted materials in the course of research, investigation, and analysis. These references are solely intended for non–commercial use and as such have no intent to infringe on the copyright holder. All attempts have been made to acknowledge the original copyright holder in pursuit of fair use laws as currently defined in the United States.


December 2005


3/186

The concept that risk and the

management of risk is a

desirable part of our program is

not always appreciated or well

understood

Without risk there can be no

opportunities. The plans for the

program become static and

deterministic.

While risk and opportunity are

related, the management of risk

is not the complement of

opportunity. - even if this is a

popular notions these days.

See the Conrow, AT&L article for

detailed discussion of this

somewhat controversial topic.

The primary opportunity in

Programmatic Risk Management

is the avoidance of being late and

over budget on the planned

launch date.


December 2005


4/186

When we use the term “risk tolerant IMS” it means a plan and supporting that can tolerate risks. Technical risks and programmatic risks. These risks are built into the program by its very nature. These risks must be addressed both technically and programmatically.

The real challenge though is not how to address them, but how to recognize that they are being addressed in a manner that actually reduces the level of risk as the program proceeds along its path to final maturity.

A measure of “increasing maturity” is the reduction of risk made visible to the evaluator of the IMS.

The materials here guide us through the process of building a risk tolerant IMS. But putting it to work still requires practice.


December 2005


5/186

The credibility of the Integrated

Master Schedule (IMS) is the

critical success factor for both

our proposal and our execution

phase after the win.

Without a credible schedule and

the related cost credibility, there

is a low probability of a win.

The effort put into constructing a

credible schedule during the

proposal phase will pay off

(assuming the program structure

remains intact) during the

execution phase.


December 2005


6/186

The skills of creating and

managing a schedule and the

associated cost require special

understanding.

However, the planners are

usually the last in a long line of

“culprits” for finding the cause of

any failure.

This is a “no win” situation.

People skills, project

management skills, and some

level of technical skill is needed.

But most important is the people

skill, since the knowledge of how

to assemble a successful IMS

resides in the minds of others.

Getting this knowledge out and

on paper requires interpersonal

communication as a primary

process, not technical tools and

formal processes.


December 2005


7/186

Understanding the difference

between qualitative and

quantitative risk assessment is

important.

Our first approach is usually

qualitative.

But what is needed is

quantitative.

A specific measure of

programmatic risk, is the impact

of the mitigations or risk

retirement activities and measure

of the increasing maturity of the

program deliverables in the

presence of risk.


December 2005


8/186

Programmatic risk management

makes visible the technical risk

mitigation steps as well as the

alternative programmatic

processes in the presence of

these risks.

Alternative branching in the IMS

must be defined to a level of

detail that instills confidence that

the IMS properly represents a

“risk tolerant” plan.


December 2005


9/186

Since there is quite a bit of

material here, a quick overview

will get us started.

The executive overview should

leave the reader with a sense of

the important topics

• There are no point estimates

allowed in planning. All

estimates must be

probabilistic

• There are core issues with

simple (deterministic) PERT

and it is not to be trusted

• The use of a probabilistic tool

is useful, but understanding

how the underlying statistic

works is critical to its use in

planning and program

execution


December 2005


10/186

When asked “why are we doing

this?” many would answer –

because our customer wants us

to.

This would be too simple an

answer.

The main reason is, most

programs are simply too complex

not to have a better

understanding of how the

programmatic and technical risks

interact.

Not understanding the interaction

between these two types of risk

that creates the biggest risk.

Individually these risk “could” be

managed. But when combined

they behave in unpredictable and

maybe unknowable ways.

This is a core feature of any

system. See Systems Bible


December 2005


11/186

If we get only two concepts out of

this briefing they should be:

• There are multiple critical

paths in any executing

program. Asking “what is the

critical” indicates that the

questioner does not

understand the probabilistic

nature of the program

• PERT is a poor estimating

metric. It has built in biases

which under estimate the total

duration of the program.

Monte Carlo is a better

estimating tool, but it too

needs careful adjustment

before realistic numbers can

be derived.


December 2005


12/186

The DID–MGMT–81650

describes the Integrated Master

Schedule.

Integrating Programmatic and

Technical risk identification and

mitigation adds credibility to the

IMS and therefore to the overall

program.

Applying probabilistic risk

analysis to the IMS is mandated,

but care is needed to interpret

the results.

These tools aid in the evaluation,

but they are not replacements for

good program management

processes.


December 2005


13/186

The idea that uncertainty and the

risk that it produces can be

“programmed out” of the

schedule is a false hope.

Without understanding the

principles of Deming, the

management and the planning

staff will be “chasing their tail,”

trying to control the naturally

occurring variances in the plan.

The first approach is to set the

error bands wide enough to not

trigger an exception report for

these variances.

This approach is “good enough”

but what is missing is the

knowledge of “how wide is wide

enough?” for a specific set of

tasks or during a specific phase

of a program?


December 2005


14/186

The first step in the process of

adding credibility to the IMS is to

recognize that all task completion

times are random variables.

They are not “point” numbers

(scalars) but are “estimates” of

the completion time drawn from a

probability distribution of the

underlying population of all

completion times possible for the

specific task.

Modeling schedule durations are

random variables does not imply

these durations are “random.” It

reflects how a duration’s

uncertainty is influenced by the

underlying probabilistic nature of

the activity network.


December 2005


15/186

Building a credible IMS starts

with identifying the architecture of

the IMP and the supporting tasks

in the IMS.

Although this is restating the

obvious the process to do this is

actually quite hard.

Adding schedule and cost risk

identification and mitigation to the

process is the minimal result for

a winning proposal.

It cannot be emphasized enough

– the architecture of the IMS is

critical to identifying a risk

tolerant schedule. The “rats nest”

approach is simply unacceptable

to the success of any program.


December 2005


16/186

The goal of introducing

probabilistic schedule and cost

analysis is to improve the

probability of a “win” on the

proposal.

While winning is important,

executing the program is even

more important.

What ever “credibility” elements

were in the proposed IMS need

to be carried into the execution

schedule.


December 2005


17/186

The use of Monte Carlo for

assessing the IMS must be

turned into forecasting

performance.

This is done by identifying the

“hot spots” in the IMS through

sensitivity analysis, interventions

for these “hot spots” and the

measure of change resulting

from the intervention.

The important concept is to

connect metrics to measurable

benefits to the program. Without

this the creation of metrics is just

wasted effort.


December 2005



December 2005 18/186


19/186

Using risk and uncertainty as an

integral part of the planning

process is a sign of maturity.

Making decisions on the this risk

information improves maturity.


December 2005


20/186

When we speak of risk management, either technical or programmatic, the term usually has a very localized context.

For the planning context risk management must include both technical and programmatic risk.

The technical risk aspects come from external sources but are directly represented in the IMS.

The programmatic impacts of this technical risk must be explicitly addressed.

This is the easy part.

The hard part is determining the implicit programmatic risk that is derived from the technical risk and the risks that are derived from the “architecture” of the program itself.

This is where the true “risk tolerant” IMS adds value.


December 2005


21/186

There are many approaches to

building a risk tolerant IMS. Our

current approach is to add risk

factors and margin to specific

areas of the IMS

The current approach to use a

Monte Carlo tool to assess where

this margin should be placed.

There are several other steps

along the way. Which steps to

take, how much effort to invest

and how to recognize the value

of this investment are some of

the management challenges as

well as the technical challenges.


December 2005


22/186

The difference between risk and

uncertainty needs to be

understood at some level.

For the most part the differences

are not important in the

beginning.

But once decisions start to be

made about mitigation steps,

branching probabilities for failure

modes, these differences

become more important.


December 2005


23/186

When we use the term

uncertainty or risk it means at

least 4 things.

First let’s sort out “uncertainty”

There are two classes of

uncertainty in large complex

programs.

• Static uncertainty emerges

from the natural variations in

the completion times of tasks.

This is a Deming uncertainty.

http://webserver.lemoyne.edu/

~wright/deming.htm is an

example of this type of

uncertainty

• The dynamic uncertainty is

about the unknowns and the

unknowable


December 2005


24/186

The static uncertainty in a

program can be addressed

directly in the plan with mitigation

tasks.

The dynamic uncertainty arises

from the dynamic interactions

between the tasks of the plan.

This interaction and the

outcomes to the end date cannot

be modeled with static

paradigms.

Monte Carlo simulation is an

approach to modeling these

interactions and their impact on

other elements of the plan


December 2005


25/186

Managing risk in the schedule requires anticipation to identify the risks, but also requires understanding of the source of risk, the impacts of these risks, and the interaction between the risks and the plan.

A process is needed to guide the risk management activities. This process must address both the programmatic as well as technical risk. The interaction between programmatic and technical risks must also be managed.

These interactions must be considered a “first order” interaction.

The common approach is to consider the technical risk as first order and the programmatic risks secondary.

The combination becomes a first order interaction.


December 2005


26/186

As planners our goal must be to

produce a plan that has credibility

and integrity.

Credible plans are believable

plans

Integrity plans are trustworthy

plans.

Both attributes are needed for a

winning proposal and the follow

on execution.

The successful assessment of

the IMS during a proposal or

during execution by the customer

or DCMA depends on how

believable the plan is and how

well it can be assessed to

confirm this believability


December 2005


27/186

The assessment of the credibility

and integrity of the IMS can take

place by asking some questions.

These and similar questions

shine light on the underlying

attributes of the IMS in ways that

simple assessments do not.

These are not technical

assessment, like counting data in

the predecessors field, but are

architectural questions about the

“quality” of the IMS independent

of the technical details.


December 2005


28/186

NASA does risk management in

a specific way. We need to

understand their way as a

starting point.

Reading the NASA materials is a

start, but there is other research

available from conferences and

vendor web sites that needs to

be gathered and read as well.

Other government agencies as

well as civilian firms have similar

risk management approaches.

NASA’s approach is a good

starting because of manned

space flight’s inherent risk. And

NASA’s emphasis on Safety and

Mission Assurance.


December 2005


29/186

The IRMA tool developed at

NASA Johnson Space Center is

the basis of risk management for

a NASA side.

Although this approach is

focused on the technical risks the

programmatic risks appear in the

database.

As well there are other risk

management systems and

paradigms.

Active Risk Manager (ARM) is a

popular one as well,

http://www.strategicthought.com/


December 2005


30/186

The NASA Risk Management

Summary Card calls out

“schedule” impacts in three

places.

Connecting programmatic and

technical risk is a critical success

factor for a proposal as well as

an execution assessment.


December 2005


31/186

Adding probabilistic schedule and risk analysis to the IMS can be done through a structured process.

1. The initiating event of the risk is identified.

2. The result from this event is described

3. The consequence that flow from the scenario are developed

4. The connections, flows, interactions and correlations between the scenarios are modeled

5. The probability of occurrence for each of these scenarios is developed

6. The model of the probability of occurrence and consequences from the occurrence are combined


December 2005


32/186

The Continuous Risk

Management paradigm found in

the technical risk world can be

applied to the programmatic risk

as well.

NASA has adopted Continuous

Risk Management (CRM)

through several guidelines listed

here.

The table summarizes how CRM

is managed in a structured

manner throughout the program

life,


December 2005


33/186

There is a difference between the

design evaluation of the IMS and

the risk evaluation.

The design evaluation describes

how the technical activities

needed to develop and deploy

the product – in this case a

manned spacecraft – must come

together in the right sequence to

make the planned completion

date.

The risk evaluation defines the

probabilistic completion model for

each task, the correlations

between the tasks and the

resulting probabilistic model.

This model is a Bayesian

Network of all the tasks.


December 2005


34/186

To construct an IMS with integrity

and credibility both technical and

programmatic risk must be

connected.

This process starts with the

identification of the technical

risks in ARM and their mitigations

in the IMS. This is the explicit risk

approach.

Next comes the explicit

programmatic risk activities. This

can be the well known margin

needed in front of major

milestones, program events or

deliverables.

Finally comes the implicit risk

mitigation activities that will be

needed to differentiate this IMS

from any other IMS to start to

build confidence that we have a

“risk tolerant” IMS.


December 2005


35/186

A pedagogical literature survey

from the RAND Corporation

supports the notion that

probabilistic risk assessment is

not seen in a favorable light by

management.

• It is too complex.

• The underlying statistic are not

will understood.

• “It’s the customers that are

asking for this.”

• There is little historical data to

calibrate the underlying

probability distribution functions

for task completion times.

All of these gaps must be closed

in some way in order to call our

IMS Risk Tolerant


December 2005


37/186

Managing in the presence of

uncertainty is the core behavior

for any modern program.

Trying to control this uncertainty

requires two basic

understandings:

1. The natural variations in the

schedule cannot be

sufficiently controlled to

remove risk. These are the

Deming variations and the

foreseen uncertainties

2. The unforeseen uncertainties

and the inherent chaos of the

program must be dealt with

through contingencies

Attempting to manage

uncertainty is limited to foreseen

risk. Managing in the presence of

uncertainty deals with unforeseen

and chaotic sources of risk


December 2005


38/186

When estimating the completion times for tasks, there are three primary problems.

1. A number produced by a CAM or an IPT must be a statistical estimate, not a specific duration.

2. The meaning of “best” must be established prior to accepting the statistical estimate

3. The collecting of the “most likely” estimates cannot be added in the sense of adding scalar numbers, since they are probability distributions.

4. The “most likely” is NOT the average completion time, it is the completion time that occurs most often from a large sample of possible completion times.


December 2005


39/186

The first approach to “planning”

the program is to ask the CAMs

or IPT Leads for each task in

their WBS or IMP/IMS area: “how

long with this take to do?”

The numbers that come back are

then entered in the duration field

on the schedule.

These numbers are not only

wrong they are dangerously

wrong.

They are “point” estimates that

live inside a probability

distribution.

The built in bias from the

approach has clinically be shown

to be optimistic or pessimistic,

but rarely “most likely.”


December 2005


40/186

The traditional approach is to roll

up the single point estimates into

a sum of the durations and

search for the longest path.

This is the Critical Path Method

(CPM) for assessing the finish

date of the plan.

The problem of course is these

“numbers” are not actual scalar

values. They are samples drawn

from probability distributions.

Addition is not mathematically

possible in the sense of addition,

defined over the set of natural

numbers (0, 1, 2, … ∞]

These probability distributions

can be “convolved” into a new

probability distribution, but a

better approach is Monte Carlo


December 2005


41/186

When asked “what is the most

likely” or the “best guess”

duration, the variety of answers

removes any chance of getting a

reasonable answer.

The meaning of “best” is

undefined in almost any situation

that has not taken explicit steps

to bound the answers.

Without calibrating the meaning

of “best” the planner cannot

bound the underlying probability

distribution of all the value that

are not “best” but could possibly

occur in the project


December 2005


42/186

When we use a term “best” or

“most likely” there is an implicit

assumption – often not

acknowledged – that other values

than “best” and “most likely” can

occur.

This is the probabilistic nature of

the duration estimate. A single

value cannot exist.

The actual shape of the

probability distributions is what is

needed for generating the “best”

estimate.

Without this knowledge, the

planner is guessing in the dark.


December 2005


43/186

Here are some steps to

producing “educated guesses.”

This is a model based approach

which depends on the maturity of

the data that is the basis of the

model.

While this is a high level

description, it needs raw data

underneath to make it valid.

Without this data the “guess” is of

little value.

What is missing in most cases is

any historical trends for the IMS

elements.


December 2005


44/186

Playing the 20 questions game is

on approach to calibrating the

“guess” for the duration.

This approach will get an answer

to without 10% to 20% in a few

questions.

This is a way to start the

“conversation” about duration

when the participants have

convinced themselves that they

can’t come up with the answer

because there is not enough

information.


December 2005


45/186

Another approach is to classify

the fidelity of the information.

This can be done with a 1, 2, 5

approach.

Gathering estimates by asking

for durations is the preferred

approach.

Instead, making a risk adjusted

estimate – duration and

confidence interval provides a

better approach.

This approach neutralizes the

guessing game by asking a risk

question first, then the duration.

The classification of risk provides

the lower and upper bounds of

the task duration. Along with the

underlying probability distribution,

this forms the basis of

probabilistic schedule analysis


December 2005


46/186

In all cases, uncertainty is the

normal mode for information

gathering.

When we ask a CAM or IPT for

an estimate and do not ask for

the risk associated with that

estimate and the confidence

intervals for that number we are

simply increasing the risk to the

program by absorbing unreliable

numbers.

This unacknowledged risk is

always present . By not making it

visible, the program is

mortgaging the future without

budgeting for the cost of paying

off the mortgage.


December 2005


47/186

Starting with a good topology for

the IMS is important. Not only

because the programmatic

activities need to be well defined,

but the sensitivity of the risk

analysis depends on a “properly

formed” IMS.

If the logic of the IMS is ill–

formed than the results of the risk

analysis will also be ill–formed.


December 2005


48/186

There are several elements of

the probability model for duration.

Not only are the activities from

the IPTs and CAMs important,

but the subcontractors play on

important role.

The data from the subcontractors

includes:

• Durations and the

probabilities

• The internal connectivity of

the activities that produce the

external; “milestones”

conveyed to the prime

contractor.

• The other programmatic risk

factors for the performance of

subcontractor work


December 2005


49/186

Although the formalities of the

probabilistic risk analysis are not

needed for this briefing. Here is

some background on

terminology.

If we are to learn to “speak” in

probabilistic programmatic risk,

these terms should become

familiar.

This is an almost endless topic,

but some understanding of

probability and statistics is

needed.

This of course requires some

effort and patience .


December 2005


50/186

We should not be drawn into the

illusion that the Central Limit

Theorem is operable for the

program.

This is the core assumption of

PERT and CPM based planning.

This requires normally distributed

completion time and

independence between tasks.

Neither can be verified in

practice.

As such the impact of making

these assumptions is “whistling in

the dark.”

The result is that the program is

late before it starts.


December 2005


51/186

The Central Limit Theorem can

be useful in many cases. But it

needs to be understood where it

is not useful.

The assumptions of the CLT

applied to the PERT problem

mask even more problems when

naively applied to estimating the

duration of a program.


December 2005


52/186

The core of the Central Limit

Theorem of the production of a

Gaussian probability distribution

by assembling a collection of

arbitrary probability distributions.

The primary assumptions that

these distributions are

independent provides the basis

of the CLT.

If the activities represented by

the arbitrary distributions are not

statistically independent – which

is hardly ever the case on a real

project – then the assumptions of

the Central Limit Theorem are

false and the probability

distribution of the program

completion time is no long

Gaussian distributed


December 2005


53/186

What happens is the rollup of the

most likely times of the critical

path activities is biased to an

optimistic location in the


project completion distribution.

This is the fundamental reason

PERT is not very useful.

This criticism is only partly true. If

a probabilistic PERT approach is

used or a Bayesian network

approach is used, then the

deterministic issues are

removed.

But it is easier to use a Monte

Carlo simulator since this avoids

gathering all the underlying

probabilistic distribution

information for an initial estimate

of the completion time of the

program


December 2005


54/186

The probability distribution

function describes the frequency

of occurrence of the events in the

underlying statistical process –

say the duration of a task

completion, the roll of a die, or

the time it takes a light bulb to

burn out.

The ordinate of the graph (the y

axis) is normalize to a scale of [0,

1] which represents the

probability percentage 0.10 =

10%

The abscissa represents the

range of values that can be found

in the underlying sample

population. In this case [0.0, …,

5.0]

The mode is the “most likely”

value to occur when samples are

drawn from the distribution.


December 2005


55/186

The standard deviation is a

description of the “spread” of the

probability distribution function

around the mean.

Without understanding the

standard deviation ,a point

estimate or even a sampled

estimate is of little value.

The shape of the probability

distribution is also important in

understanding the confidence in

a single number. These “higher

order moments” will be discussed

later, but for now no estimated

number should be used without

the standard deviation value

being attached.


December 2005


56/186

Looking at the population

statistics of a random process is

not very useful. Humans have a

hard time making any sense from

the graphs.

The Histogram view can show

the frequency of occurrence of

the various values – how often a

specific value occurs in the total

population of value or the

sampled population of values, but

more insight is needed.

The Cumulative Probability

Density is a way to show this.

The CDF shows the probability

that a sampled number drawn

from the population of all possible

numbers


December 2005


57/186

Various Probability Distribution Functions (pdf) have similar Cumulative Density Functions (CDF).

This is important for several reasons:

• The underlying probability distribution function has great influence on how the end point values are weighted. This has impact on the PERT formula

• The cumulative distribution is the source of random numbers in Monte Carlo. For a variety of pdf’s, similar CDF’s are generated, neutralizing the differences in the pdf’s. Monte Carlo isolates these underlying differences. This may be good or bad depending in the need.


December 2005


59/186

Any estimating process must

address the probabilistic

boundaries of the estimate.

Without this, planners and cost

estimators are hopelessly under

or over estimating duration and

associated cost.

The real issue is not over or

under estimate, but not knowing

which one it is or why.

This lack of knowledge about the

underlying statistical process

creates a greater risk.


December 2005


60/186

Making decisions in a risk neutral

manner is not advised.

We should always talk about risk

adjusted decisions, risk adjusted

values, and risk adjusted

outcomes.

The difference between

alternatives, uncertainties and

outcomes also needs to be

understood. They are not

interchangeable concepts


December 2005


61/186

Decision making must address

the different types of uncertainty.

Understanding how these

uncertainties impact the decision

is critical to selecting alternatives


December 2005


62/186

The idea that we can produce

“estimates” about the future in

the absence of models, historical

data, or a methodology for

discovering these models or

historical data is common in the

IMS planning realm.

Forecasting the future is sporty

business.


December 2005


63/186

The IMS contains “branches”

where the path of work makes a

change in direction.

These braches can be modeled

with a decision tree paradigm.

The risk management discipline

uses this approach. And it is

applicable to the construction of

the probabilistic branching found

in the network of tasks in the

IMS.


December 2005


64/186

In the “olde days” the line of

balance chart was used to

forecast the cost at completion.


December 2005


65/186

PERT and the Critical Path

Method are called out as explicit

methods to be used in the

planning process.

The formulas for PERT are

simplified models of the

underlying complexity of

probabilistic networks (Bayesian

Networks)

As such they have little or no

connection to the reality of the

IMS


December 2005


66/186

In deterministic PERT the

durations are defined as a three

point estimate and the PERT

formula is used to compute the

mean and standard deviation for

the program duration as well as

the critical path.

This is the algorithm used in

Microsoft Project when the PERT

tool bar is turned on and the

three point estimates entered into

the appropriate columns.

It is billed as probabilistic but in

fact the 3–point estimates work

against a fixed probability

distribution function with no way

to adjust its shape, bounds or

moments.

As well, there is no way to insert

the correlations that naturally

occur in the IMS.


December 2005


67/186

The estimates produced by the

deterministic three point data can

be used to construct a

probabilistic PERT if the

underlying probability

distributions are defined for each

task completion time.

The development of the

probability distributions requires

historical data as well as an

understanding the behavior of

each node in the network

(coupling).

This is a difficult task without the

proper tools and data sets.

With the Risk+ tool, individual

distribution functions can be

assigned to each task. But the

“tuning” of each function is

difficult.


December 2005


68/186

When we speak of probabilistic

risk analysis, we also need to

speak of the statistical nature of

the activity network.

When we speak of a probabilistic

activity network (a Bayesian

network) we also need to speak

in terms of probability.

A question that can be asked of

the network is – “ what is the

probability of completing this task

by a certain date?”

A second question that can be

asked is – “what are the

underlying statistics of the

activities of the network?”

A final question that needs to be

asked is “what is the inherent

uncertainty in these estimates?”

In other words – how good is our

ability to guess in the presence of

a statistical process?


December 2005


69/186

Once the activity durations are

treated as probability distributions

it can be seen that they can not

be “added” in the normal sense

to produce a program duration.

They must be “summed” in the

probabilistic sense. This can be

done with Monte Carlo or with

convolution of the Cumulative

Distribution Functions.

Again, add to this the correlation

issues (one task influencing the

outcome of another task), and

the simple approach of adding

the durations to come up with a

total duration falls apart.


December 2005


70/186

Here’s another look at distribution

functions.

This approach should be the

standard vocabulary for

discussing the IMS duration

estimates.

The topological integrity of the

IMS is important, but just as

important is our understanding of

how the activity durations have

been developed, their confidence

interval and the underlying

distribution of the values the

durations can take.


December 2005


71/186

A missing element is the

statistics of the “events” that

occur during the execution of the

program.

For example if a fixed date is

defined in the IMS (this is very

usual for things like IBR, PDR,

CDR), what is the underlying


confidence of that date.

The same is true for

subcontractor provided dates,

where the details of the

deliverables is not visible.


December 2005


72/186

With the input probability

distributions, the program

schedule can be treated as a

“system” with a response

function.

The “system” is a Bayesian

network where the elements of

the network are probabilistic and

the driving function is

probabilistic.

The “output” of the system is

therefore probabilistic as well.


December 2005


73/186

One probability distribution

commonly found in scheduling is

the Beta Functions.

This is a “tunable” probability

distribution function that has

been shown to closely match the

behavior of task completion

durations.

The term “closely” needs to be

used with care. The deviations

between actual completion times

and the “model” of completion

times needs to be assessed

before confidence in the

probabilistic results can be

useful.


December 2005


74/186

The Beta distribution is used for

PERT estimates. This use is

many times done with no

understanding of the shape or

the dynamics of the probability

distribution function. Beta is a

selection for Risk+ as well, with

no obvious way to change the

shape of the curve.

Some understanding of the

impact of the Beta function on

the outcome of the PERT formula

or the Monte Carlo simulation is

needed.


December 2005


75/186

There are many alternatives to

Beta. The Triangle distribution is

one. The triangle distribution has

an intuitive appeal due to its

simplicity useful for estimating

task durations.

But the triangle distribution still

has the problem that the most

likely value and the expected

(mean) value are not the same.

So when planning asks for the

“most likely” value many people

respond with the Mean, which

biases to result in the optimistic

direction.


December 2005


76/186

The triangle distribution can

better describe some statistical

processes, but it too needs

“tuning” for specific task duration

processes.


December 2005


77/186

BetaPERT is currently the vogue

in the probabilistic analysis world.

The BetaPERT distribution

provides a “tunable” curve where

the most likely “Mode” is near or

identical to the “mean” of the

distribution.


December 2005


79/186

The challenge to building a risk

tolerant IMS is the initial capture

of the task durations and the

sensitivity of the IMS to

correlations between tasks.

There is a optimism bias created

when a CAM is asked “what is

the duration?”

The answer is usually a “mean”

(average) duration rather than

the “Mode” (most likely).

If the Mean is used in place of

the Mode, then the three point

estimates are biased to start with

without the explicit knowledge of

the planning staff.


December 2005


80/186

The general flow of creating a

risk tolerant IMS looks like this.

The critical aspect is to get the

CAMs to identify the embedded

risks and the mitigation tasks for

those risks.

Once this is done, “planning” can

then assess if the mitigation

processes make sense in terms

of supporting the AC’s and SA’s

of the IMP/IMS.

Constant and continuous

feedback is needed for this to

work properly.

Without this feedback, the IMS is

assembled in the absence of the

knowledge base and the risk

tolerant aspects are lost or

become confused with the

mainline activities.


December 2005


81/186

The capturing of the risk

information is an interactive

process. A Kaizen is one way to

do this and probably the best.

Having the CAM fill out the “most

likely” durations and identify the

risk mitigations cannot be done

without direct contact.

Without this direct contact,

planners have not chance of

intervening in the process and

the IMS becomes a collection of

tasks rather than an “architected”

plan.


December 2005


82/186

The 3 point estimates required by

DiD–81650 have a variety of

uses.

They can be simple values used

for PERT calculations. These

calculations can be “made up” by

the IPT lead and entered into the

schedule.

A risk adjusted value can be

used from the Macro in Risk+.

The CAM or IPT Lead states the

relative risk in a number between

1 and 5. The macro defines the

percentage boundaries for the

classified risk.

Individual risk ranges can be

developed from historical

information. This is the best

approach, since it represents the

past.


December 2005


83/186

There are several classes of

programmatic risk. Although the

Pareto chart shows that scope

change is the most common,

delays are also common. These

come from the customer side

most often as well.


December 2005


84/186

The classification of risk results

in a percentile or quartile

classification scheme. This is a

better approach than asking

someone for the minimum and

maximum durations.

The challenge is to calibrate

these ranges in a meaningful

manner for the specific program.

There can be general

classification ranges, but having

them set for the specific program

is much better.

This of course requires that data

is kept from past programs,

normalized and then made

available in a form useful for

probabilistic risk analysis.


December 2005


85/186

During the data capturing

process where estimates are

extracted from the technical

experts, there is a natural

tendency to accept the numbers

at face value.

Without qualifying the numbers in

some statistical form, this

information is absorbed into the

IMS or Cost and becomes “fact.”

These “facts” then progress

through the program and are

never challenged for their lack of

statistical basis.


December 2005


86/186

The core problem with capturing

estimate from human beings is

they are biased.

Either negatively biased or

positively biased.

There is plenty of literature on

this effect and ways to overcome

it. For now we’ll just live with the

outcome of the bias


December 2005


87/186

Let’s take another tour of the problems with PERT. These issues are well documented in the literature, but poorly understood in practice.

The poor understanding comes from the difficulty of the explanation – statistical conversations are usually not very interesting; and the natural tendency to look for easy answers to complex problems.

The core issue is that without a deep understanding of the errors produced by the PERT equation, the confidence in completion dates and the risk tolerance of the IMS is difficult to build.

When the actual numbers come in (ACWP and BCWP) and they don’t match the expectations – is it the original plan or the underlying performance?


December 2005


88/186

There are several myths about

PERT. The first is that is was

scientifically thought out in detail.

This is not the case. The book

The Management of Projects,

Peter W. G. Morris provides the

background on this development

as well as other project

management histories.

The second historical myth is that

PERT is a general purpose

approach. In fact it is very

specialized and is applicable to a

narrow range of activity

networks. Those with normally

distributed completion times,

statistically independent

relationships, ones where the

critical path does not change and

with the “most likely” estimate

actually representing the “mode”

of the underlying probability

distribution function.


December 2005


89/186

When a manger asks “what is the critical path for this program?” there are several thoughts and actions:

• In a probabilistic activity network there are many critical paths, which change as a function of time, adjustments to the risk profiles, and the completion of work.

• Correlated activities are influenced by off–critical–path activities to place them on the critical path.

So the answer to the management question is “it depends on what you mean by critical and path.”

The real answer only comes by moving away from the static representations of the IMS to a probabilistic representation – and that requires much more effort.


December 2005


90/186

Once we recognize that the

activity network is probabilistic in

nature, the first choice (the naïve

choice) is to apply the PERT

method.

While this may be a useful “first”

choice it produces results that

are overly optimistic and

sometimes overly pessimistic.

Either way they are wrong from a

statistical point of view. They are

wrong because the assumptions

of PERT are wrong. These

assumptions are almost never

found to be true in practice.

Even if they were true, the

probability distribution function

used by PERT does not

represent any useful activity

completion time distribution.


December 2005


91/186

One of the “killer” assumptions in

PERT is the lack of

understanding of “merge bias.”

Merge Bias occurs when two or

more activities are joined at a

merge point. Usually a milestone

or a simple Finish to Start of

several tasks.

The result is the statistical

behavior of the activities prior to

this merge point influence the

statistics of the following

activities in undesirable ways.


December 2005


92/186

Since statistical distributions can

not be simply “added” the

duration of the downstream

activity is not the sum of the

duration of all the upstream

activities (or the longest activity).

Instead it is the statistical sum

(convolution) of the probability

distribution function (pdf)

Without understanding this, the

PERT estimate generates an

optimistic estimate of the

duration, since the PERT formula

simply adds the durations to

arrive at the total duration.

The PERT formula also adds the

individual activity variances to

arrive at a total project variance.

While this provided a simple

method to “guess” the total

duration it produces a poor model

for real analysis of risk.


December 2005


93/186

The PERT approach fails to

consider the “random variable”

nature of the dates in activity

network.

As well the correlation between

each of these random variables

is not considered.

The result is the potential for

large variances in the completion

time estimates – 15% is not

uncommon.


December 2005


94/186

The visual impacts of Merge

Point bias is show here. This is a

small and sample activity

network. A “real” network would

have different outcomes.

It is not important exactly how the

merge point bias impacts the

final completion date, but that the

merge point bias DOES impact

the final completion date.


December 2005


95/186

How the activity network is

arranged has significant impact

on the calculations for PERT.

Here are some examples.

Notice that the PERT mean (the

average) stays the same, while

the “real” mean and the variance

on that mean change

dramatically depending on the

arrangement.


December 2005


96/186

The reason for these changes

involves how the statistics are

“added” in the various

configurations.

The critical concept is that the

PERT calculations are unreliable

as a predictor of the completion

time in a probabilistic model of

the activities.


December 2005


97/186

The effect of the merge bias is

shown in the graph. It is unlikely

in any real plan that only three

parallel paths exists. This number

is usually much larger,

sometimes in the dozens.

All of this discussion is leading to

the suggestion that PERT is not

viable on any complex program.


December 2005


98/186

Use Monte Carlo, don’t use

PERT.

The problem of course is that

DID 81650 and even the

corporate guidelines either

require or strongly suggest the

use of PERT and CPM.

This can be done of course, but

don’t use the numbers for any

real planning processes.


December 2005


99/186

The use of Monte Carlo

simulation is a logical outcome of

the problems with PERT.

What is missing is the

understanding of how Monte

Carlo works, what it’s limitations

are, where it should not be used

and of course how to interpret

the outcomes when they don’t

meet our expectations.

Even though Monte Carlo is a

powerful tool it can produce

unexpected results. This section

is an attempt to give some

background on the mathematics

and stimulate further interest in

applying this tool to the problem

of schedule forecasting


December 2005


100/186

Monte Carlo simulations provide

a useful approach to modeling

schedule risk. But their value is

more than that.

Unlike PERT or other

deterministic approaches – even

though the three point estimates

are billed as probabilistic – Monte

Carlo examines the schedule

network independent of a critical

path, topological constraints or

other “human induced” problems.

It looks at the network as a

collection of nodes and arcs,

independent of the “meaning” of

this information and produces a

model of the behavior of these

nodes and arcs


December 2005


101/186

The concept behind Monte Carlo

is to sample the possible

durations for a task from the

population of all durations and

apply them to the schedule.

The population of possible

samples is defined by the

Cumulative Probability Density

(CDF) function for each task.

This in turn is defined by the 3–

point estimate for the task, which

selects the bounds in the CDF for

sampling.

Since there is no direct concept

of a Critical Path in Monte Carlo,

the near critical path tasks are

considered in the analysis of the

completion time.

As well the PERT biases

produced by the simple minded

PERT formula are avoided as

well.


December 2005


102/186

There are several “components”

to the Monte Carlo process. So

when we speak of Monte Carlo it

is both a process and a product –

in our current case Risk+


December 2005


103/186

Samples drawn from the

underlying distribution function

can produce an “error estimate”

on a completion date.

These error estimates are

different than the fixed

boundaries for PERT, since they

represent the actual probabilities

distribution error bounds


December 2005


104/186

The number of sample runs

needs to be sufficient to cover all

the possibilities in the pdf.

This is usually 500.

A production run for a Monte

Carlo simulation is around 2,000

to 3,000 iterations.

As the iteration count increases

the fidelity of the simulation

increases.

But there is a point where more

samples don’t add value. This

point can be determined by the

statistical performance of the

variance of these sample space.


December 2005


105/186

Since Monte Carlo does not need

to know about the Critical Path, it

is conceptually simpler to use.

A well formed network is needed

and the 3–point estimates need

to represent the proper risk

assessment.


December 2005


106/186

This is a view of how Risk+ sets

up the project file.


December 2005


107/186

The result is a cumulative

distribution and a probability

distribution function.

Interpreting this result is straight

forward.

The confidence of each date is

shown in the table on the right.

This is the probability of

completing the task by the date.


December 2005


108/186

A good IMS is needed.

The risk assessments should be

done with a ranking process

rather than specific 3–point

estimates. This disconnects the

personal opinions from the

assignment of risk.

A 5 level process is one

approach, but any odd numbered

level ranking is best.

The differences between the

levels should be geometric not

linear.


December 2005


109/186

Risk+ generates lots of

information useful for the

analysis of the program.


December 2005



December 2005 110/186


111/186

Constructing a robust IMS means

building a “risk tolerant” plan.

The robustness of the plan

means that it (the plan) can deal

with disruptions that occur

naturally through the course of

execution or un–naturally through

external events.

In either case the “robustness” of

the plan must be visible to the

evaluator without any detailed

explanation, beyond the IMS

narrative. No hand waving

explanations of how the plan

works. The risk tolerant aspects

most be obvious.


December 2005


112/186

Thinking about schedule

contingency is different in a PRA

context. For a simple project,

15% contingency is assumed.

But placing the contingency is

the first problem. The process is:

• Run Risk+ and watch the final

date.

• Compare the 80% confidence

date against the deterministic

date. This difference is the

first cut at the needed margin.

• Assign this duration across

the project in front of the

critical (high risk) milestones.

• Rerun Risk+ and add or

subtract this margin until the

desired confidence date is

achieved.


December 2005


113/186

More detailed statistics and

interpretations of the results an

be produced with Risk+.

This information can them be

used to perform further analysis

of the IMS. The analysis is what

we’re after, not just the date

produced by Risk+.

Like the PERT numbers, the

Risk+ numbers must be

interpreted with the

understanding of how they were

arrived at.

This is one of the purposed of

this briefing – to provide

knowledge of how to use this

approach and what its strengths

and weaknesses are.


December 2005


114/186

Incorporating Technical

Performance Measures (TPM)

with Monte Carlo is a powerful

way of showing how risk is

reduced and maturity increased

in a program.

At each step in the program –

each Program Event – a target

confidence interval for a

completion date can be forecast.

Along with the technical

performance measure, this

programmatic performance

measure approach results in a

“risk tolerant” IMS.


December 2005


115/186

Risk tolerance in the IMS

requires more than just the

planning processes. It requires

the connections to technical and

cost.

This has been stated before, but

it needs to be made not only

visible but actionable.


December 2005


116/186

Using a simple process steps,

risk tolerance can be developed

from the same processes used

by the technical risk engineers.


December 2005


117/186

The goal here is to move the

integrated risk tolerance –

technical, schedule, cost –

forward from a dis–integrated

plan to an integrated plan


December 2005


118/186

Read the chart as follows: The upper horizontal band on the plot is “Ready Early”. “Ready On–time” is the middle band that also spans the launch window. “Ready late” is the lower band, which means a 6–month slip to the next launch window and all associated costs that go with that slip. The upper line plotted is the deterministic

completion date (i.e. no risk) and the lower line plotted with the 20th and 80th percentile confidence bands on the risk–adjusted completion date. The project’s objective is to continue to invest in risk mitigation actions until the band and the area of highest likelihood is no longer in the “Missed Launch Period” area of the chart. Note the improving trend over time indicating the success of the risk mitigation actions as well some “Accepted” risks passing their exposure window without becoming problems.

Taken from [Risk Based Decision Support techniques for Programs and Projects] http://www.futron.com/pdf/RBDSsupporttech.pdf


December 2005


119/186

As Program Events progress the

risk mitigation processes need to

progress as well.


December 2005


120/186

Here’s a 4 step progress for

installing risk in the IMS and

producing a “risk tolerant” plan


December 2005


121/186

The use of branching

probabilities is important for the

assessment of the “risk

tolerance.”


December 2005


122/186


December 2005


123/186

The use of Risk+ and Monte

Carlo replaces the PERT

approach to schedule duration

probability analysis.


December 2005


124/186

The “goodness” of the IMS is

important to the quality of the

results


December 2005


125/186

The distribution to use for a task

depends on the underlying risk

profile.

Triangle is common, but it over

biased the risk on the high end.

Beta can be used, but the simple

Beta distributions in Risk+ may

not represent the real risk profile.

BetaPERT is the better one, but

Risk+ does not support it.


December 2005


126/186

Which tasks drive the sensitivity

of a completion date needs to be

understood. Not all tasks have

the same impact on the outcome.

The “tornado” chart is one way of

showing this.

The Power Law’s behind Pareto’s

rule is worth understanding for

many reasons, not just schedule

and cost modeling. Power Laws

occur across a wide variety of

domains, from moon crater sizes

to the frequency of words in

English.

http://www.nslij–

genetics.org/wli/zipf/ is a good

place to look for the impacts of

Power Laws on everyday life.


December 2005


127/186

In order to build a model of the schedule we have to start with the schedule. But first we have to start with the model of the schedule.

This is the role of the IMP, but the connections between the ACs are needed, not just the list of the IMP elements.

From this model the schedule elements can be arranged to follow the strategy of the IMP rather than represent the passage of time and the consumption of resources.

From there a model of the risk areas, mitigations, parallel development paths, reevaluation points, and “hot spots” (sensitivity analysis) can be extracted. This information can them be used to assess the robustness of the IMS


December 2005


128/186

The primary graphic for an IMS

evaluation is the cumulative

probability of a completion time.

This is technically referred to as

the Cumulative Density Function

(CDF)

This is the format most useful for

answering the question – how

long will this take?


December 2005


129/186

The confidence intervals

produced by the CDF can be

assessed over time against

targets.

These targets can be Technical

Performance Measures or any

other style of metric that is

connected with cost, schedule

and technical performance


December 2005


130/186

Another view is the confidence in

the schedule dates as a function

of time.


December 2005


131/186

It is important to understand the

sensitivity of a completion time to

the various “drivers” of this

sensitivity.

This makes visible the “hot spots”

in the IMS that require attention,

mitigation, or even re–planning to

reduce sensitivity


December 2005


132/186

In Monte Carlo each task can

take on a wide variety of roles. It

can be the driver for the total

schedule duration at one time,

and at another time (in the

simulation) have little effect on

the outcome.

The Criticality of the task is how

“important” it is as a function of

the number of simulation runs.

The higher the criticality of the

task, the more important it is to

look at the details and determine

what mitigations should take

place to keep this task lower in

the criticality index.


December 2005


133/186

When sensitivity and criticality

are combined a sense of the

cruciality. Cruciality is defined as

“a state of critical urgency.”

Although this sounds like a

redundancy term, it can be used

to focus our attention on those

tasks that are both critical and

sensitive.

It is important to understand the

sensitivity aspects, since these

can change and drive the

schedule in non–obvious ways.


December 2005



December 2005 134/186


135/186

Let’s look at some examples of

Monte Carlo


December 2005


136/186

The Monte Carlo simulation

makes use of the three point

estimates generated during the

PERT analysis. This numbers

represent the upper, lower and

most likely durations.

This values are then used to

draw random numbers from the

probability distribution for

evaluating the activity network.

The branching probabilities can

then be added for the

alternatives paths and risk

mitigation activities.


December 2005


137/186

The use of “expert judgment”

itself needs to be calibrated.

The unanswered question on this

program and many others is

“what does a good risk tolerant

IMS actually look like?”

The “units of measure” for risk

tolerance and the confidence in

the probabilistic estimates needs

to be established before the

estimating and modeling process

can be “calibrated”


December 2005


138/186

The ranking of risk or the ranking

of anything needs to be done in a

structured manner.

A geometric progression is a very

useful approach, since it forces

the focus on ranking.


December 2005


139/186

The “sense” of risk and real risk

need to be connected.


December 2005


140/186

Some type of risk ranking needs

to be developed for the IMS

tasks.

One approach is the TRL scale.


December 2005


141/186

When tasks are arranged in

series the cumulative probability

of completion is show in the table

on the right


December 2005


142/186

When the tasks are arranged in

parallel a different completion

profile results.


December 2005


143/186

All of this is very interesting in a

Power Point presentation –

marketecture it's called.

Let’s look at a real schedule and

start to apply some of the things

we’ve learned.


December 2005


144/186

This is a very simple construction

plan. The tasks are networked in

a way to show how the Risk+ tool

works.


December 2005


145/186

The first picture of a completion

time is the PERT assessment.

The task Construction Schedule

Margin (the end of this task is the

end of the planned margin) has a

target date of 2/8/06 and a

forecast PERT date of 3/6/06.

This shows there is not enough

margin by one month for this task


December 2005


146/186

The same task, evaluated with

Risk+ shows a different

completion date.


December 2005


147/186

Project Risk Analysis is part of

any good risk management

activity. This has been said

numerous times and needs to he

repeated daily.

Both the technical and the

programmatic risk aspects of the

program need to be shown in the

IMS.

Any questions, changes,

updates, suggestions – anything

that touches the IMS or the cost

model – needs to be assessed

from the point of view of

programmatic risk.


December 2005


148/186

The accuracy of the dates and

costs in the IMS is a “relative”

term.

±20% to start with is pretty good.

As the program proceeds

accuracy improves but it is

always a statistical estimate until

after the fact.


December 2005


149/186

If we take a deterministic

approach the planning then there

will be built in issues. The first is

that all estimates must include a

confidence interval or they are

wrong.

The “natural” approach to

estimating almost always results

in a bound that is too wide as

well as being optimistic or

pessimistic but hardly never

accurate.


December 2005



December 2005 150/186


151/186

Now that we’ve reached a fairly

detailed level of discussion

regarding programmatic risk

assessment, it’s time to talk

about cost risk assessment.

The first concept to understand is

that cost and schedule are

connected. This is obvious. But

they are not connected in any

linear manner.


December 2005


152/186

The basic principles of cost

estimating start with the

understanding of the uncertainty

in the estimates of cost.

These uncertainties must be

connected to the technical

uncertainties as well as the

programmatic and simple cost

variances.

The arithmetic addition of costs

creates a false number of the not

only the cost but any variance in

this cost.

Monte Carlo simulation is one

starting point, but like the

programmatic simulations, the

underlying probability

distributions must be understood

before the numbers have any

real meaning


December 2005


153/186

The connection between a

technical parameter is its cost is

not only potentially non–linear it

is probabilistic.


December 2005


154/186

A simple 9 step (not so simple

actually) process can be used to

build a cost estimate.

Starting with the “likely” program

in the form of an IMS, the tasks

for delivering that program are

defined.

The underlying probability

distributions for the cost of each

delivering activity are developed.

This is much like the

development of the baseline IMS,

but the next step is much

different.

The correlation between each

WBS element is developed.

These correlations are used to

build a model of sensitivity of the

cost to changes in the tasks.


December 2005


155/186

At this point the Risk+ tool fails to

deliver what is needed. Wither

Crystal Ball or @RISK is needed

to connect these correlations

together.

The technical uncertainty of the

program is used to drive the cost

uncertainty. This is where the

technical and programmatic risk

assessments joins.


December 2005


156/186

The production of the familiar

probability curves for the

likelihood of cost is the result.


December 2005


157/186

The risk margin in dollars is the

result needs to make this

connection.


December 2005


158/186

Resulting in the budget risk


December 2005


159/186

Which produces the estimates for

the management reserve


December 2005


160/186


December 2005


161/186


December 2005


162/186


December 2005


163/186


December 2005



December 2005 164/186


165/186

We’re near the end now, so your

brain is certainly getting full.

This is quite a bit of information

to absorb, but it needs to be

done before we can say we are

building “risk tolerant” plans.


December 2005


166/186

When probabilistic schedule

analysis is used it does not

replace the need for a well

formed project network. It only

replaces the use of PERT for

estimating the completion dates.


December 2005


167/186

The quality of the probabilistic

estimates is the foundation of

confidence.

The next step is to clearly identify

where in the IMS risks are being

mitigated, the impacts of this

mitigation and the overall

confidence in the master plan

resulting from this mitigation


December 2005


168/186

The correlations between cost,

schedule, and technical risk must

be made explicit.

A model of how these elements

interact is the basis for answering

the “what if” questions that occur

when the risk item becomes

active.


December 2005


169/186

Risk based schedule and cost

management is core to

programmatic integrity.


December 2005



December 2005 170/186


171/186

Like any good idea it can be

improved on forever.

These opportunities are much

harder to address than the

process so far. They require care

and effort to build a correlation

matrix for the tasks. They require

detailed understanding of the

underlying statistical processes

and the historical data that was

used to develop these

distributions.

For most projects this is beyond

the scope of the effort and may

be beyond the business interests

as well – since the pay back is

not clearly defined.


December 2005


172/186

The dependencies between tasks

is the basis of the correlation

function. This is very important if

a true model of the network is to

be developed. In the absence of

the correlations it is assumed

tasks are independent, which of

course can not be the case.

Building a Program risk

assessment requires that cost

and schedule be connected as

well – correlated. Cost and

schedule are not linear, so any

simple model of changes in one

linearly effecting the other cannot

work.

Finally the idea of a causal model

– a cause and a set of effects

provides deeper insight into the

risk behaviors of the network.


December 2005


173/186

There is too much information

here for a single digestive

process. The only way to absorb

all this is to start practicing

probabilistic schedule and cost

analysis and make the

knowledge appear in the output

information.


December 2005


174/186


December 2005


175/186


December 2005


176/186


December 2005


177/186


December 2005


178/186


December 2005


179/186


December 2005


180/186


December 2005


181/186


December 2005


182/186


December 2005


183/186


December 2005


184/186


December 2005


185/186

There are nearly unlimited

resources on the web. The

challenge of course is finding

them.

Here’s some know starting

points.


December 2005


186/186

This has been a long journey

over hopefully many weeks of

discussion and hands on

experience with Risk+ and real

project schedules.

Building a risk tolerant IMS is a

“practice” and practices require

proficiency. Proficiency comes

from “doing the work,” looking at

the results and making changes

for improvement.

This is just the beginning.


December 2005

probabilistic cost, schedule, and risk management

Technology

probabilistic cost

management of risk

level of risk

presence of risk

programmatic risk management

reduction of risk

credible schedule

associated cost