cpm-500-c/d : integrating systems engineering with earned ... · integrating system engineering...

presented by:

Integrating System Engineering with Earned Value Management

Pat Barker MCR, [email protected] © MCR, LLC, 2008Presented at the 2009 ISPA/SCEA Joint Annual Conference and Training Workshop - www.iceaaonline.com

What This is all about …• Purpose: This presentation explores the inter-relationship of system

engineering (SE) and earned value management (EVM). – Shows similarities between SE and EVM. – Hands-on requirements generation and work breakdown structure

(WBS) building exercises help students recognize the value of linking the two disciplines together.

• Objectives– Attendees with an EVM/Cost/Scheduling background will recognize the

value of building their next technical baseline, work breakdown structure and performance measurement baseline side by side with lead program engineers.

– Similarly, engineers attending this seminar will hopefully think twice before commencing system engineering planning, design activities, project planning and/or status meetings without “EVM expertise”present.

– All attendees will recognize the value of interdisciplinary work to support program management

© 2008 MCR, LLC 2

Presented at the 2009 ISPA/SCEA Joint Annual Conference and Training Workshop - www.iceaaonline.com

Outline

• Context• Clarifying What the Customer Wants• The Role of Requirements• Planning Considerations• Performance Measurement• Your Turn• So What?• Summary• Acronyms

© 2008 MCR, LLC 3


What SE Does For Us …• SE enables:

– Planning and organization of the technical aspects of the project– Analysis of the problem posed by the stakeholders

• Defining the stakeholders’ problem by converting needs and expectations into validated and integrated technical requirements

• Developing detailed technical requirements to the extent necessary to enable feasible and economical design solutions

– Assessment and evaluation of alternatives which may satisfy these needs and expectations

– Selection of a balanced solution for each system element– A balanced solution for the system as a whole– An implementation of a balanced solution – Verification that the solution satisfies stakeholder requirements

Source: International Council on System Engineering (INCOSE)© 2008 MCR, LLC 4


What EVM Does for Us …

• Why Use Earned Value?– Requirement (on certain contracts)– Way to manage limited resources and

competing priorities– Essential for realistic planning– Critical for decisions that are rooted in reality– Objectively measures work progress– Enables trend analysis and change control

• What is Earned Value Management (EVM)?OMB Definition of EVM: “… a project management tool that effectively integrates the project scope of work with schedule and cost elements for optimum project planning and control.” (OMB Circular A-11, 2002)

ScheduleCost

Scope

© 2008 MCR, LLC 5


SE Examples• Analysis of Alternatives • System Definition• Requirements Analysis• Architectural Design• Requirements Loop• Functional Analysis• Functional Allocation• Design Loop Synthesis• Verification• System Analysis and Control• Trade Studies

What Might EVM and SE Together What Might EVM and SE Together Accomplish?Accomplish?

EVM Examples• WBS Development • OBS Development• RAM Development• Planning• Scheduling• Budgeting• Accounting• Analysis• Reporting• Management• Change Control

We Will Focus On These Two Areas For Today

© 2008 MCR, LLC 6


Where Our Story Begins …..• A hard-to-find Department of Defense organization

decided it wanted a fresh bagel delivery system for its offices and it had $3.5 billion to spend on it.

• It was officially called the “ON-DEMAND FRESH BAGEL DELIVERY SYSTEM, or ODFBDS

• Others just called it the “Hyperbagel”

Using a concept most audience members are readily familiar with (most everyone has toasted a bagel and put cream cheese on it) enables proper focus on SE

and EVM concepts

© 2008 MCR, LLC 7


Outline

• Context• Clarifying What the Customer Wants• The Role of Requirements• Planning Considerations• Performance Measurement• Your Turn• So What? • Summary• Acronyms

© 2008 MCR, LLC 8


Customer Needs: Is This Clear Enough?

© 2008 MCR, LLC 9


A First Step in Translating Requirements

• Measures of Effectiveness (MOEs)– The Key “Operational” measures of success (customer

viewpoint) that meet the mission need

• Measures of Performance (MOPs)– The Key “Operational” measures of success (from the system

performance viewpoint) that enable the MOEs

• Technical Performance Measures (TPMs)– The “Acquisition” measures of success (from designer/developer

viewpoint) that enable the MOPs

© 2008 MCR, LLC

It can get more complicated than this and definitions might differ in other places, but we only

have 45 minutes today... So lets go with this!

10


Exploring the Hyperbagel MOEs

• Hyperbagel MOEs

• So…– How do these relate to each other?– Is there a better way to describe these?– Is there more to this?

GREATTASTING

DELIVEREDFAST

PRESERVESPRODUCTIVITY

DOESN’TINTERFERE

ISAVAILABLE

AFFORDABLE

© 2008 MCR, LLC 11


From MOEs to MOPs

Delivered FAST

One Example: Provide bagel within x minutes/seconds of request

Is AVAILABLE

One Example: Provide bagels between 8am and 5pm every work dayPreserves PRODUCTIVITY

One Example: Delivers bagel directly to employee’s work space

Is GREAT TASTING

One Example: Not burned

Example Questions to Ask:•Where will the ODFBDS be used? Inside? Outside? Both?•How is the delivery of fast, great-tasting bagels envisioned to be accomplished?• What key parameters determine whether of not the bagel delivery is successful (how to measure “great tasting”?)•How long will the system need to last? Days? Months? Years?

GREATTASTING

DELIVEREDFAST

PRESERVESPRODUCTIVITY

DOESN’TINTERFERE

ISAVAILABLEAFFORDABLE

© 2008 MCR, LLC 12


Result: What the Customer Really Wants

© 2008 MCR, LLC 13

MOE MOP

Doesn’t INTERFERE

Delivers bagels to a fully-occupied conference table without disrupting meeting or other employees

Requires no building and/or office modifications to accommodate

Is AFFORDABLEAcquisition Cost not to exceed $3500M (threshold) or $1000M (objective)

O&M cost minimized.

Delivered FAST Consistently (98% of the time) provide bagel within 10 minutes of stated need

Bagel consistently (98% of time) provided within 3 minutes of completing preparation

Is AVAILABLEEnabling technologies at TRL 8 within 8 months of contract award

System experiences little or no (<10%) degradation in inclement weather

Provide bagels between 8am and 5pm every work day

Preserves PRODUCTIVITY

Delivers bagel directly to employee’s work space on flat surface within 1 meter of employee

Entire process (request thru delivery) diverts less than 90 seconds of employee’s time

Is a GREAT TASTING Bagel

Fresh

Reasonably spread cream cheese

Not burned


Outline


© 2008 MCR, LLC 14


What is a Requirement?• According to INCOSE:

– The foundation of the project– The basis for design, build, test, manufacture and operations– State user needs in such a way as to guide development– Enables an understanding between the various interactions

within a system

© 2008 MCR, LLC 15

Each requirement is associated with specific costs that increase over its life if requirement changes


How Do We Get Requirements?

• Some examples (mostly from INCOSE):– External Environment

• Laws and Regulations• Legal liabilities• Social responsibilities• Standards

– Enterprise Environment• Policies• Procedures• Guidelines

– Project Environment• Contract (RFP, SOW)• Users/Operators• Design Reviews• Derivation

Requirement generation and

analysis involves more than “just”the engineers!

© 2008 MCR, LLC 16


What Makes a Good Requirement?

• Achievable• Verifiable – allowing for objective and (preferably)

quantitative verification• Have only one possible meaning• In aggregate, they must be complete (i.e., all the info

needed to understand the customer’s needs)• Expressed in terms of the need, not the solution

– Does not express “how” to do it

• Must be consistent with other requirements• Appropriate to the level so as not to be constraining

– System-level requirements in system-level documents

BAD Requirements do BAD THINGS to cost, schedule and scope!© 2008 MCR, LLC 17


Why Requirements Matter to Cost & Schedule

Source: GAO Cost Estimating and Assessment Guide (March 2009)© 2008 MCR, LLC 18


Outline


© 2008 MCR, LLC 19


Work Breakdown Structure

• Program management is not easy!

• It takes time to define the work and develop a realistic Program baseline and it requires the involvement of key people…

• First…gaining an understanding of the full scope of the Program– Define a Work Breakdown Structure

WBSWBS

© 2008 MCR, LLC 20


What is the Work Breakdown Structure?

• The Work Breakdown Structure (WBS)…– …is a systematic approach to

defining the work – …is product-oriented– …provides the framework for – …provides the necessary

framework for management control

– …is dynamic

•Level 1 – Product – entire product/service 1.2.3 where 1 is Level 1 •Level 2 – Major Element (Segment)/Subsystem 1.2.3 where 2 is Level 2•Level 3 – Subordinate Components to Level 2 1.2.3 where 3 is Level 3•Level 4 to Level X - Sub-products - defines further decomposition until a

manageable unit of work is identified.MCR

WBS Level 1

RADAR

Advanced Radar

TRAINING

RECEIVER SYSTEM ANTENNA

RECEIVER PHASE SHIFTER

POWER SUPPLY APPLICATION S/W

WBS Level 2

WBS Level 3

WBS Level 4

WBS Level 5

© 2008 MCR, LLC 21


Our ODFBDS WBS

© 2008 MCR, LLC

DynamicOrderingSystem

ODFBDS

PM SE

DualSensorSuite

Bagel Prep & Delivery

Subsystem

SystemIntegratio

n

SystemTest

Sensor#1

Mount/Pedestal

Sensor#2

RadarMgt/QA/CM

RadarReqt’s

RadarDesign

RadarIntegration

RadarTest

RadarBuild

Antenna

REX

Signal/DataProcessor

Ancillary/Supt Equipment

ArrayFace

T/RModules

Power

Cooling

Mount/Pedestal

SuiteMgt/SE

Vehicle“Brains”

(software) Payload

O&M

Level 1Level 2

Level 3

Level 4

Level 5

Level 6 Level 7

Dynamic SupplySystem

IPTMgt/SE

SubsystemIntegration

& Test

SuiteIntegration

& Test

QA CM

22


The Control Account

CONTROLACCOUNT

CA Work Packages

WBS Level 2 Delivery Subsytem

Sensor Suite etc

Sensor #1 etc

Radar Build etc

Radar Antenna etc

WBS Level 3

WBS Level 4

WBS Level 5COMPANY

TEST

ENG

MFG

DESIGN

MECHANICALDESIGN

ElectricalDESIGN

DRAFTING/CHECKING

OrganizationBreakdownStructure


ODFBDSWBS Level 1

WBS Level 6

THE CONTROL ACCOUNT

The lowest level of management control within a program that has a unique

•Cost•Schedule•Scope

“Owned” by a Control Account Manager (CAM)

Comprised of 2 or more distinct “work packages”


Control Accounts• Control Accounts are where the action is. They…

– …represent the work assigned to one responsible organizational element on one program work breakdown structure element at the lowest level (intersection of WBS and OBS)

– …are a natural management point for planning and control– …are vital to performance measurement (EV Methodology)

• Rules of Thumb:– Integration of work, cost of the work, and schedule– Homogeneity* of work– What is logical to manage every day– Examine the nature of the work, breakout of labor, span of

control – Typically 6 - 18 months for discrete effort; level of effort can be

longer

*resemblance in structure© 2008 MCR, LLC 24


The Work Package

CONTROLACCOUNT

CA Work Packages

WBS Level 2 Delivery Subsytem

Sensor Suite etc

Sensor #1 etc

Radar Build etc

Radar Antenna etc

WBS Level 3

WBS Level 4

WBS Level 5COMPANY

TEST

ENG

MFG

DESIGN

MECHANICALDESIGN

ElectricalDESIGN

DRAFTING/CHECKING

OrganizationBreakdownStructure


ODFBDSWBS Level 1

WBS Level 6

THE WORK PACKAGE

•Work packages are “natural”decompositions of the CA•Typically includes discrete (product-oriented) and non-discrete•Typically the focal point of performance measurement and resource tracking


Work Packages• Work Packages (WP) contain a discrete segment of

work below the Control Account level that is defined by – a description or brief work statement– starting and ending dates– completion milestone– work-in-process measure– time-phased budget expressed in direct labor (hours and/or

dollars), material, other direct costs and subcontract dollars

• It is important that the duration of a Work Package be a relatively short span of time (normally, but not limited to, six months or less).

© 2008 MCR, LLC

CONTROL ACCOUNT PLAN

Work Pkg #1

Work Pkg #2

Work Pkg #3

$$

$ $

$

$

$

$$

26


Outline


© 2008 MCR, LLC 27


The Need for Objective Performance Data

• Project plans are not perfect nor is it realistic to expect perfect execution.– Consequently, there will always be deviations from the plan.

• Objective progress information removes the guesswork from status on progress and performance.– When information is not objective, project status may not be

accurate and reliable.

• The ability to know the precise status and understand the impact is dependent upon the quality of information available for those tasks.– EV Techniques by themselves are a partial solution– EV Techniques combined with technical performance

measurement (TPM) is a more complete (and realistic) solution

© 2008 MCR, LLC 28


Control Account: Antenna Build

Antenna Work Package Example Without TPM Integration

T/R Module Build WP

Array Face WP

Power WP

Cooling WP

PrototypeComplete

25%

ProductionUnit

Complete50%

Here, completion of WP milestones are typically a binary “yes, it is complete” or “no, it is not complete”measurement.

EV is taken accordingly –25%, 50% then 100% at end.

But we cannot tell if performance goals have been met.

The Performance Measurement does not indicate whether the “built”unit will enable a successful antenna build

Measurement is not as useful to management as it could be

Start Task25%

© 2008 MCR, LLC 29


Attributes of Technical PerformanceMeasurements Common to EVM

• Achieved to Date (sounds like EV)• Current Estimate (sounds like EAC/ETC)• Milestone• Planned (target) value (sounds like PV)• Planned performance profile (sounds like a PMB)• Tolerance band (sounds like reporting thresholds)• Threshold (yep, just what we thought)• Variance (sounds like variance!)

Similar in Language to EVM … Why NotIntegrate Directly With EVM Planning?

© 2008 MCR, LLC 30

PlannedProfile

AchievedTo Date

Tolerance Band

CurrentEstimate

Threshold

Milestones

PlannedValue

Variation10

5

15

Tech

nica

l Par

amet

er V

alue

(e.g

., Ve

hicl

e W

eigh

t, lb

)


Requirements to WPs

© 2008 MCR, LLC 31

Let’s now adjust our CA planning to reflect these requirements


Control Account: Antenna Build

Antenna Work Package Example with TPM Integration

T/R Module Build WP

Array Face WP

Power WP

Cooling WP

Prototype Build

Complete&

TRL6 Achieved

15%

Prototype Test

Complete&

TPM in tolerance

band15%

PilotBuild

Complete&

TRL7Achieved

15%

PilotTest

Complete&

TPM in tolerance

band15%

ProdUnitBuild

Complete&

TRL 8 Achieved

15%

ProdUnitTest

Complete&

TPM meets objective

15%

TRL 8 Enabling Technologies Requirement means that the CAM needs to characterize each WP in terms of TRL achievement. Thus no WP is “finished” until it meets the conditions of TRL8.For the T/R Module WP, the CAM needs to define what TRL is acceptable for Prototype, Pilot and Production Unit in order to assure Antenna of TRL8

Delivery within 1 meter/flat Surface Requirement affects several parts of the radar, but here it means the T/R modules need to achieve a certain power output (to enable a higher S/N ration for signal processing). Therefore the criteria for each test should now include power.

In both cases, CA planning becomes more realistic because it takes SE performance into consideration!

Array Face Integration

Requirements met10%

© 2008 MCR, LLC 32


Outline


© 2008 MCR, LLC 33


Group Assignment• You are now the Toaster Design & Build CAM• Start with the following MOE:

– “Deliver a great tasting bagel”• Examine at least one of the MOPs that shape this MOE:

– Fresh– Reasonable amount of evenly spread cream cheese– Not burned

• Write 2 or 3 RELATED requirements that govern the design of yourtoaster

• Convert the effort required to realize these requirements into aWORK PACKAGE

• Determine at least one TPM that will help comprise an objective performance indicator for EVM

• Be prepared to discuss your planning

© 2008 MCR, LLC 34


ODFBDS

PM SEDynamicOrderingSystem

SystemIntegration

Bagel Prep & Delivery

Subsystem

DualSensorSuite

SystemTest

Propulsion Chassis Electrical

PrepMgt/QA/CM

PrepQA

PrepDesign &

Build

PrepIntegration

PrepTest

ToasterDesign & Build

Loader

Dispenser

Software

BagelPrep

VehicleMgt/SE

“Brains”(software)

Payload

O&M

Level 1

Level 2

Level 3

Level 4

Level 5Level 6

Dynamic SupplySystem

IPTMgt/SE

SubsystemIntegration &

Test

VehicleIntegration &

Test

QA CM

Our ODFBDS WBS

Vehicle

????YOU are now the Toaster Design/BuildCAM

© 2008 MCR, LLC 35


Write your “toaster” performance requirements here that meet the at least one of the above MOPs:

MOP: “reasonable amount of evenly spread cream cheese”

MOP: “not burned”

MOP: “fresh”

Your Requirements

© 2008 MCR, LLC 36


Your Technical PerformanceMeasurement

PlannedProfile

AchievedTo Date

Tolerance Band

CurrentEstimate

Threshold

Milestones

PlannedValue

Variation

10

5

15

Tech

nica

l Par

amet

er V

alue

(e.g

., Ve

hicl

e W

eigh

t, lb

)

© 2008 MCR, LLC 37


The CAM and System Engineer Have Common Challenges

• Objectives• Requirements• Dependencies• WP Breakout• PP Breakout• Resources• Budget• Time• Milestones• Deliverables• Products• Acceptance/Exit Criteria• Constraints• Assumptions• EV Performance

Measurement• Risks

• System Purpose• Technical Performance

Measurement• Performance Requirements• Functional Requirements• Interface Requirements• Architecture• WBS Breakout• Legacy of Similar Systems• Acquisition Strategy• Developmental Testing• Operational Testing• Production• Safety• Training• Resources• Disposal• Budget• Schedule• Milestones• Acceptance/Exit Criteria• Constraints• Assumptions• Risks

CA Scope SE Planning

© 2008 MCR, LLC 40


Consider Performance-Based Earned Value® when Integrating SE with EVM

• Planning:– Link performance measurement baseline (PMB) to:

• Product requirements baseline, SE plan and SE process products– Identify product metrics for performance-based progress measurement

• Planned value profile of TPMs• Planned development maturity to date

– Generate Realistic Success criteria• Reviews• Work products

• Measurement– Objective measurement of interim progress– Progress of requirements through engineering life cycle– EVM linked with

• Indicated quality of end product and/or TPM achieved– EVM used to assess conformance to requirements

• Not just work accomplished

© 2008 MCR, LLC 41For Reference: The PBEV® Website: www.pb-ev.com


Integration Holds Larger Benefits ….

TPM Development

MilestonesBudget

Technical Risk AnalysisOBS

DevelopmentCost Risk Analysis

Schedule Risk Analysis

Control Account Planning (cost/schedule)

Responsibility Assignment Matrix (RAM)

Programmatic Guidance / Constraints

System Requirements Analysis

System Definition

Analysis of Alternatives

WBS Development

Risk-Adjusted PMB

Parametric Cost & Schedule Estimates

MOE and MOP Development

System Engineering Planning

IMS Development

Integrating SE and EVM tends to have a broader effect of “pulling” disparate program planning & baseline development efforts together

RESULT: A Robust risk-adjusted baseline to set the stage for effective program execution

© 2008 MCR, LLC 42


Summary• This presentation explored the inter-relationship of system

engineering (SE) and earned value management (EVM). – Showed similarities between SE and EVM. – Hands-on requirements generation and work breakdown structure

(WBS) building exercises helped you recognize the value of linking the two disciplines together.

• Victory is Claimed If ….– All attendees recognize the value of interdisciplinary work to support

program management

© 2008 MCR, LLC 44


Acronyms

ACWP: Actual Cost of Work PerformedAPB: Acquisition Program Baseline BAC: Budget At CompletionBCWS: Budgeted Cost of Work Scheduled or “planned value”BCWP: Budgeted Cost of Work Performed or “earned value”CA: Control AccountCAIG: Cost Analysis Improvement GroupCFSR: Contract Funds Status ReportCPI: Cost Performance IndexCPR: Contract Performance ReportCRA: Cost Risk AssessmentCWBS: Contractor Work Breakdown Structure CV: Cost VarianceDID: Data Item DescriptionDoD: Department of DefenseDCMA: Defense Contract Management AgencyEAC: Estimate at CompletionETC: Estimate to CompletionEV: Earned ValueEVM: Earned Value ManagementEVMS: Earned Value Management System


Acronyms (cont’d)GAO: Government Accountability OfficeGR&A: Ground Rules & AssumptionsICE: Independent Cost EstimateIEAC: Independent Estimate at CompletionIMS: Integrated Master ScheduleINCOSE: International Council on System EngineeringIPO: Integrated Program OfficeKPP: Key Performance ParameterLCAA™: Linked Crest Assessment & AnalysisLOE: Level of Effort (Earned Value Measurement Technique)LRE: Latest Revised EstimateM: Million(s)MCR: Management, Consulting, Research, LLCMOE: Measures of EffectivenessMOP: Measures of PerformanceMR: Management ReserveNDIA: National Defense Industrial AssociationOBS: Organizational Breakdown StructurePMO: Program Management OfficePM: Program Management


Acronyms (cont’d)PMB: Performance Measurement BaselinePOE: Program Office EstimateRMP: Risk Management PlanSE: System EngineeringSEP: System Engineering PlanSOW: Statement of WorkSRA: Schedule Risk AssessmentTCPI: To-Complete Performance Index (a.k.a. “run-out

efficiency”)TPM: Technical Performance MeasureWBS: Work Breakdown Structure


cpm-500-c/d : integrating systems engineering with earned ... · integrating system engineering...

Documents