benefit study on digital transformation for project

Benefit Study on Digital Transformation for Project Planning &

Control ProcessNipa Phojanamongkolkij and Rob Moreland

August 2018

National Aeronautics and Space Administration 1

National Aeronautics and Space Administration

Background and Motivation (1 of 2)Current collaborative network

• Functional data system (FDS, the disc) for each function. The project’s centralized Technical Data Management System (TDMS, the yellow disc in the center) used mainly for an archival purpose. Coordinative activities (a.k.a. the data collection and preparation activities)Collaborative and innovative activities (the solid linkages) for problem solving, brainstorming, or creating alternative solutions.

•

•

•

•

•

•

2Figure 1. Current collaborative functional network for project management

EPIIC’s Vision(Evolving Project Integration, Innovation & Collaboration)

FDS for each function for uncontrolled and explorative-in-nature models or documents uniquely pertaining to its function. The project’s centralized Digital Data Management and Analytics (DDMA) containing a true source of project data and enabling collaborative and innovative activities. The DDMA includes embedded intelligence, relying upon statistical engineering, machine learning and data analytics, etc. to identify and anticipate trends within the project.

Figure 2. Future collaborative functional network for project management2


Background and Motivation (2 of 2)• This Discrete Event Simulation (DES) model is to characterize

the potential benefits of the EPIIC’s vision with respect to the current collaborative network.

– Specifically, the model will characterize the % time reduction for collecting data, requesting data, reconcile inconsistent data.

• Currently, PP&C process is far-well developed than other functions. As such, the DES model will be demonstrated with the PP&C process.

Current collaborative network EPIIC’s Vision 3


Modeling Process Flow (1 of 3)

Link to the PP&C Handbook:https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20160013229.pdf

Inputs and Outputsby each activity

Inputs (Rows) by Activities (Columns)

Outputs (Rows) by Activities (Columns)

92 data (combined inputs and outputs) and 48 activities

See Appendix A for an example.

4

https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20160013229.pdf



Derive the sequence of activities for DSM analysis

See Appendix B for examples of inconsistent data in the

handbook. 5



1Inputs (Rows) by

Activities (Columns)Outputs (Rows) by

Activities (Columns)

Rearrange Sequence of Activities based on

PP&C Handbook2

VBA script (MakeDSM.xlsm)

Planning Activities

Control Activities

3

Simulation Model

4

4

6

The Sequence of Planning ActivitiesNetwork Diagram View

7

17

Start

14

19

11,12

5,6,7

20

3,4

21,22,23,24

1

2

15

16

9

13

End

18

18

18

18

1810

Activity Name Phase ACT_ID Function_AliasDevelop Acquisition Plan Planning ACT_01 ACM

Support Establishment of Contracts Planning ACT_02 ACMDevelop CM/DM plans Planning ACT_03 CMIdentify items to be under configuration control Planning ACT_04 CM

Prepare cost analysis strategy Planning ACT_05 Cost

Execute cost assessment tasksPlanning ACT_06 Cost

Execute cost estimating and supplementary analytical tasks Planning ACT_07 CostConduct a Joint Confidence Level (JCL) analysis Planning ACT_08 CostPresent analyses to stakeholders Planning ACT_09 Cost

Capture Project Scope of Work Planning ACT_10 PP&C Int.Define Approach/Strategy to Executing Scope of Work (includes Acquisition Strategy) Planning ACT_11 PP&C Int.Provide Guidance, Decisions, Adjustments Planning ACT_12 PP&C Int.Refine Plan Planning ACT_13 PP&C Int.

Activity Name Phase ACT_ID Function_AliasDevelop Plans for Resource Management Planning ACT_14 ResourceDevelop Project Cost Budget based on Available Obligation Authority Planning ACT_15 ResourceImplement EVM System Planning ACT_16 ResourceIdentify Initial Risk Planning ACT_17 RiskExecute an initial Risk Informed Decision Making (RIDM) iteration as a part of project formulation Planning ACT_18 RiskDevelop a Risk Management Plan (RMP) that includes a definition of a Continuous Risk Management (CRM) process Planning ACT_19 RiskDevelop a strategy for schedule estimation and assessment Planning ACT_20 Scheduling

Develop schedule Planning ACT_21 SchedulingAssess & Analyze Schedule Integrity Planning ACT_22 SchedulingValidate schedule consistency with cost & labor plans (if not RLS) Planning ACT_23 Scheduling

Baseline schedule estimate Planning ACT_24 SchedulingProvide JCL Schedule Planning ACT_25 Scheduling

8

25

7


Discrete Event Simulation View

More details in modeling in Appendix C

IN (Inputs) 92 data by 25 activitiesOUT (Outputs) 92 data by 25 activitiesDataOwner 92 data ownershipActivityOwner 25 activity ownershipRequest data time duration

Triangular with (min, mode, max) (95%, 1, 110%)

Revise data time duration (95%, 10, 110%)

Execute activity time duration (95%, 20, 110%)

Minimum = 95% of modeMaximum = 110% of mode

8


Lower Level, Re-Useable Model

Request Input Data

Execute Activity

Generate Output Data

EPIIC

9


Preliminary Results

• Primary Objective:– To quantify benefit of the EPIIC’s w.r.t. the Current Collaborative

Network by means of:• Benefit in terms of % improvement in total time duration

Sensitivity analysis of the benefit on model assumptions•Ø Stakeholder: OCE PPM Strategic Initiative

Lower fidelity model may be sufficient for “relative benefit” of strategic planning.

Ø

• Secondary Objectives: – To gain understanding of process dynamics

To identify process improvement opportunities for the Current Collaborative Network

–

Ø Stakeholders: PP&C Community, OCE PPM Policy, Practice, and Development DivisionHigher fidelity model is preferred for operational/tactical planning.Ø

10 10


Primary Objective (1 of 8)Boxplot to represent Monte Carlo Simulation result for total time duration for single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.

IN (Inputs) 92 data by 25 activitiesOUT (Outputs) 92 data by 25 activitiesDataOwner 92 data ownershipActivityOwner 25 activity ownershipRequest data time duration

Triangular with (min, mode, max) (95%, 1, 110%)

Revise data time duration (95%, 10, 110%)

Execute activity time duration (95%, 20, 110%)

95% Confidence Interval (CI) on the average.

11


Primary Objective (2 of 8)

Determine number of replications•–

12

Curve shows relatively the half-width of the Confidence Interval (CI) converging as number of replications increases. At some points, there is no Return-On-Investment on the half-width for increasing replications.

12



Low-fidelity estimate of the EPIIC’s vs. Current Network shows ~ 10% total time improvement** from (1-10-20)* to (0-10-20).

•

(1-10-20) (0-10-20)

95% Confidence interval on the average total duration, based on 1,000 replications.

** Based on single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.* (Request-Revise-Execute)

13



• Sensitivity analysis on modeling assumptions

•–

–

•

Assumption (Variable)Distribution

Time Durations (or ProcTimeSet)(Request-Revise-Execute)

Case 1Triangular

(1-10-20)

Case 2Uniform

Minimum Maximum

Minimum = 95% of modeMaximum = 110% of mode

(1-5-40)

Total of 8 simulation scenarios:4 Current Network scenarios (each with 1,000 replications) for all assumption combinations (2 distributions x 2 ProcTimeSets)4 EPIIC’s scenarios

Computed (post-processed) % Improvement14



• Sensitivity of % Improvement to modeling assumptions

Statistically sensitive* (P-value <<< 0) to the time durations used.

Practically insensitive to either uniform or triangular distribution (P-value ~ 0.02).

* Overlapping the CI indicates statistically insensitive.

** No statistical evidence that % improvement depends on the combination of both duration and distribution (a.k.a., interaction effect).

15



• Sensitivity analysis to time durations (ProcTimeSetvariable in the previous analysis)

•–

–

•

Variable Case 1 Case 2 Case 3 Case 4 Case 5Request Time 1Revise Time 1 3 5 10 (ReviseTime) (R1) (R3) (R5) (R10)Execute Time 10 20 40 80 100 (ProcTime) (P10) (P20) (P40) (P80) (P100)Distribution Triangular

Total of 40 simulation scenarios:20 Current Network scenarios (each with 1,000 replications) for all assumption combinations (4 Revise x 5 Execute)20 EPIIC’s scenarios

Computed (post-processed) % Improvement

16


Primary Objective (7 of 8)•

•

Sensitivity of % Improvement** to Time Duration Assumptions

•

** Based on single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.

% Improvement decreases as both revise and execute times increase and eventually converges to a minimum region.

With the request time of 1 unit, an operating region with 6-8% improvement** (the gray rectangular region) can be expected for revise time is in the range of (5, 8) and execute time is (40, 100). Other regions can also be determined from the 2D contour plot on the left.

17



Theoretical Minimum % Improvement**•

• Determine the theoretical minimum % improvement by increasing the Execute Time from 100 (P1e2) to 10,000,000 (P1e7) units with Revise Time set at 20% and 100% of the Execute Time.

% Improvement decreases as both revise and execute times increase and eventually convergesto a minimum region.

As the Revise and Execute Times approach a significantly large number (w.r.t Request Time), there is still an average benefit (~3.7%) of the EPIC’s over the Current Network.

There is a benefit (> 0%) with 98% confidence (in other words, 98% of all 12,000 replications show > 0% improvement), with the worst case of -1% benefit (Current is better than EPIC’s).

•

•

•

** Based on single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.18 18


Secondary Objectives (A preliminary example)

• Identify function’s level of effort (LoE) Probable opportunities to•

– reassign activity owners to balance workload;have 1 FTE performing multiple functions;or have 1 FTE performing the same function for multiple projects.

––

19


Total Level of Effort (LoE) Cost

• Low-fidelity estimate of the EPIIC’s vs. Current Network shows ~ 7% total cost improvement** from (1-10-20)* to (0-10-20).

(1-10-20) (0-10-20)

Staffing Cost Assumptions:Function Annual Cost % Fringe

Benefit

PP&C $117K (GS-14 Step 5)

30%

Cost Analyst $99K (GS-13 Step 5)

30%

Risk Manager $180K (WYE) -

Scheduler $160K (WYE) -

Resource Mgmt(EVM)

$160K (WYE) -

CDM $160K (WYE) -

Acquisition & Contract Manager

$160K (WYE) -

** Based on single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.* (Request-Revise-Execute)

20


Repeat RSM for Cost Improvement• Sensitivity of % Improvement** to Time Duration Assumptions

•

•

Execute Time

Revise

Time

% Im

provement

4

6

8

10

12

14

16

Execute Time

Rev

ise

Tim

e

4

6

8

10

12

14

20 40 60 80 100

24

68

10

2

4

6

8

10

12

14

16

18

20

22

24

% Improvement decreases as both revise and execute times increase and eventually converges to a minimum region.

With the request time of 1 unit, an operating region with 4-6% improvement** (the gray rectangular region) can be expected for revise time is in the range of (5, 8) and execute time is (55, 75). Other regions can also be determined from the 2D contour plot on the left.

** Based on single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.21


Theoretical Minimum Cost Improvement

• Theoretical Minimum % Improvement**

•

•

0.0

0.5

1.0

1.5

2.0

2.5

E1e2 E1e3 E1e4 E1e5 E1e6 E1e7ExecuteTime

% Im

prov

emen

t

ReviseTime 20% 100%

Determine the theoretical minimum % improvement by increasing the Execute Time from 100 (P1e2) to 10,000,000 (P1e7) units with Revise Time set at 20% and 100% of the Execute Time.

% Improvement decreases as both revise and execute times increase and eventually converges to zero.

** Based on single-event-driven activities (e.g., gate review), and not including any monthly re-planning activities.22


Future Works

• Increase model fidelity• Secondary Objectives

• Refine modeling assumptions Add iterative monthly re-planning cyclesPerform process improvement through DSM analysis and simulate the proposed process to quantify any benefit

••

• Primary Objective can be repeated with the higher fidelity model

• Extend beyond PP&C (e.g., Systems Engineering and Safety and Mission Assurance functions)

23


Acknowledgments

• Donald E. Shick, Manager, Program Planning & Control Capability Office (PCO), OCFO, NASA Langley Research CenterNani Tosoc, Cost Lead, Proposal Development Office, NASA Langley Research Center

•

24


BACK-UP

25


How sensitive the benefit is to % fringe benefits? Both the delta and the current cost increase proportionally causing the % to be ~7%.

6.0

6.5

7.0

7.5

8.0

30% 40% 50% 60% 70% 80% 90% 100%Fringe Benefit

Perc

ent I

mpr

ovem

ent

Percent LoE Cost Benefit vs. Fringe Benefit

26


The cost benefit decreases as fringe benefit increases due to complex cost benefit relationship.

• Only PP&C and Cost Analyst functions drive the change in % benefit as fringe changes.Let:•

a = delta (between the current and EPIIC) in LoE in hours for PP&C functionb = delta (between the current and EPIIC) in LoE in hours for Cost Analyst functionc = the current LoE in hours for PP&C functiond = the current LoE in hours for Cost Analyst functionC1 = constant delta cost (between the current and EPIIC) for all other 5 functionsC2 = constant current network’s cost for all other 5 functions$_PPC_f = LoE hourly cost at f % fringe for PP&C function$_Cost_f = LoE hourly cost at f % fringe for Cost Analyst function

Function Annual Cost % Fringe Benefit

PP&C $117K (GS-14 Step 5)

30%

Cost Analyst $99K (GS-13 Step 5)

30%

Risk Manager $180K (WYE) -

Scheduler $160K (WYE) -

Resource Mgmt(EVM)

$160K (WYE) -

CDM $160K (WYE) -

Acquisition & Contract Manager

$160K (WYE) -

%Benefit at 30% fringe = !"#$%&'((")$ *%+"#,)" =

%∗$_11&_2345∗$_&6+$_234&78∗$_11&_2349∗$_&6+$_234&:

%Benefit at 100% fringe = %∗$_11&_;3345∗$_&6+$_;334&78∗$_11&_;3349∗$_&6+$_;334&:27

benefit study on digital transformation for project

Documents