evm+agile the darkside

+

Integrating Agile Software Development (Agile) on Earned Value Management ProgramsStarting with an EIA–748–C compliant Earned Value Management System, integrating an Agile Software Development Lifecycle (Agile) is straight forward when there is a Bright Line between the Performance Measurement Baseline (PMB) and the Sprints and Tasks of the Agile Software Development Process.

AGILE AT SCALE

forFAR 34.2 / DFARS 234.2Acquisition Programs

1

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016

+The Dark SideBoth Earned Value Management and Agile have Dark Sides. Things that are not talked about in public.But when they are Integrated, each provides a solution for the problems of the other.Assess current and desired Maturity for Agile and EVM is the starting point for integrating these two processes.

Every tool, process, and practice has a dark side. Knowing these is a Critical Success Factor to the

integration of EVM and Agile at the desired Maturity Level.

338Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016

339All successful software projects must answer three critical questions …

… how much it will cost, when will it be done, and what will be delivered for that time and cost?

Dark Side


+ The Dark Side of EVM and AgileFixed with EVM+Agilen Both EVM and Agile have

Dark Sides.

n EVM can balance some of Agile’s Dark Sides.

n Agile can balance some of EV’s Dark Sides.

n Together they can turn the Dark into Light.

n Agile + EVM = Match Made in Heaven?

340

Dark Side


Being more Agile on Software Development Programs creates Cost Growth to support the emerging requirements in the presence of Uncertainty

+ Let’s Start with the Agile Manifesto 341

Agile Manifesto Impact on EVM Programs

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software (in non–software projects this would refer to product).

This is simply good program management. The plan for delivering the valuable software needs to be encapsulated in the Performance Measurement Baseline

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

This problematic at the EVM level of the program. Constant change prevents EVM form being effective, since the baseline is constantly be changed

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

The period of delivery provides input to the closed Loop Control system. Agile mandates fine grained samples of physical percent complete

Business people and developers must work together daily throughout the project.

This is a key Critical Success Factor for all programs. Agile mandates this

Dark Side


+ Dark Side of EVM

The Dark Side of EVM Agile’s Fix

Uncertainty is inherent and inevitable in software development processes and product †

Incremental and iterative development control exposure to these uncertainties by producing working software every four weeks

For a new software system, the requirements will not be completely known until the users have used the product ‡

Small incremental deliverables reduce exposure to large uncertainties with testable outcomes every four weeks.

Typical implementations expect everything fully defined up front

Requirements emerge as product development proceeds

No formal measures of quality, effectiveness, and performance in EVM

Working software at end of Sprint is the only measure of progress to plan

Earned Value can be claims for intermediate work products

Working software at end of Sprint is the only measure of progress to plan

† Ziv’s Uncertainty Principle, 1996, http://www.ics.uci.edu/~ziv/papers/icse97.ps‡ Humphreys Requirements Uncertainty Principle, 1998

342

Dark Side


+ Dark Side of Agile

The Dark Side of Agile EVM’s Fix

Estimates measured in Story Points. No Story Point fields in DID–81861

Flow down BCWS to Work Package and use Story Points as apportioned effort for each task at the Feature level

All program work is probabilistic and statistical – reducible and irreducible uncertainty. Cost and schedule margin is not a concept in Agile

Risk Register and Schedule Risk Analysis (SRA) are part of DI–MGMT–81861

Story Points are Uncalibrated Ordinal measures of Effort, not duration of cost

Use SP’s as Physical Percent Complete and multiply BCWS to get BCWP

Definition of Done is an inconsistent a check list of activities required to produce software – Passes Unit Tests is typical

MOE, MOP, TPM, and KPP’s have tangible units of measure connected to deliverables

Rebaseling after release to fit Reality causes SPI=1 and CPI=1

Work Packages stay open until the work completes

343

Dark Side


+ Some More Issues with Agile (1)

n Dependencies in the actual work may require planning beyond the next Sprint.n The notion of INVEST is naïve in many Software Intensive System of

Systems development programs

n Percent Complete at the end of the Sprintn What happens is the total number of Features is less than the planned

number of features?

n How can BCWP be stated?

n If uncompleted work results where does it go?

n When CPI=SPI=1.0 at the end of the Sprint in Agile there is no visibility to the actual performance of the programn Fixed duration with unfinished work is moved to the right

n Fixed duration with deprecated deliverables list

344

Dark Side


+

Management Reserve

n “An amount of the total budget withheld for management control purposes, rather than being designated for the accomplishment of a specific task or set of tasks” (ANSI/EIA 748–B) n Meant to address “in scope changes” that were not planned in

baseline n The Contractor has authority and control over discretionary use of

MR budget n Not included in PMB, as it has not been allocated for specific work

scope n Must be formally allocated to work packages through an internal

change control process

Agile and Agile are missing the notion of Management Reserve or Schedule Margin

Some More Issues with Agile (2) 345

Dark Side


+

Schedule Margin

n … is any task not associated with specific scope or resources, and is used to increase the probability of on–time completion of contract events. The terms Contract Events includes major logical integration points, such as, contract events, major test and integration milestones, and end item deliverables.

n Schedule margin, if used, is typically set at the time the baseline is established and set with the baseline and forecast duration equal.

n The baseline without schedule margin is typically not achievable

Agile and Agile are missing the notion of Management Reserve or Schedule Margin

Some More Issues with Agile (3)Dark Side

346


+

Risk Management

n Provide processes, methods, tools, and an infrastructure of resources and organizational responsibilities to identify and assess the risks, determine what to do about them, and implement actions to deal with them.

n Agile is missing the formality of Risk Management that is needed for complex At Scale software development projectsn Agile at Scale technical and programmatic risks

n Interdependencies between work streams

n Interdependencies between deliverables

n Systems integration dependencies

Scrum and other Agile development methods are missing the notion of Risk Management

Some More Issues with Agile (4) 347

Dark Side


+ Epistemic Uncertainty and Aleatory Variability are both risk drivers†

EpistemicUncertainty

§ Epistemicuncertaintyisthescientificuncertaintyduetolimiteddataandknowledgeinthemodeloftheprocess

§ Epistemicuncertaintycan,inprinciple,beeliminatedwithsufficientstudy

§ Epistemic(orinternal)uncertaintyreflectsthepossibilityoferrorsinourgeneralknowledge.

AleatoryVariability

§ AleatoryuncertaintiesarisefromtheinherentrandomnessofavariableandarecharacterizedbyaProbabilityDensityFunction

§ Theknowledgeofexpertscannotbeexpectedtoreducealeatoryuncertaintyalthoughtheirknowledgemaybeusefulinquantifyingtheuncertainty

RandomnessWithKnowableProbabilities RandomnessWithUnknowableProbabilities

Theprobabilityofoccurrence canbedefinedthroughavarietyofmethods.Theoutcomeis

aprobabilityofoccurrenceoftheevent

AProbabilityDensityFunction(PDF)generatesacollectionofrandomvariablesusedto

modeldurationsandcosts† Uncertainty in Probabilistic Risk Assessment: A Review, A.R. Daneshkhan

348

Dark Side


+ Taxonomy of Uncertainty that Drives All Project Risk

Uncertainty

Irreducible(Aleatory)

Reducible(Epistemic)

NaturalVariability

Ambiguity

OntologicalUncertainty

ProbabilisticEvents

ProbabilisticImpacts

PeriodsofExposure

Agile and Agile provides some information to manage risk, but it is not a Risk Management System. Just a piece of Risk Management

349

Dark Side


Technical RiskManagement

TrackingandControllingPerformanceDeviations

Deliberating andrecommendingadecision

alternative

Riskanalysisofdecisionalternatives,performingtradestudiesandranking

Proposingand/oridentifyingdecisionalternatives

FormulationofobjectivesHierarchyandTechnicalPerformanceMeasures

Stakeholderexpectations,requirementsdefinitionandmanagement

Designsolutions,technicalplanning

Designsolution,technicalplanning,

anddecisionanalysis

Technical planninganddecisionanalysis

Decisionanalysis,lessonslearned,knowledgemanagement

Identify

Analyze

Plan

Track

Control

Decideandimplementdecision

alternatives

Communicate

350

Dark Side


+ Now to the REAL Problem with Agile on EVM Programsn When the threshold of $20M is tripped for Self Assessment of the

EVMS and now $100M for DCMA Validation – the program is likely a System of Systems architecture, with complexities well beyond simple Agile teams with a small number of people in the same room with their customer is no longer the paradigm.

n The project is no longer a simple straight forward development effort with Independent work activities prioritized by the customer.n Systems architecture dominates the work processes

n Interface management is a critical success factor

n Interdependences in software, processes, operations, maintenance

n Coupling and Cohesion considerations overwhelm the simple task of writing the software to meet the needs of the on site customer.

351

Dark Side


+ Some Key Concepts for System of Systems†

n System of Systems (SoS)n Very large systems developed by creating a framework or architecture to

integrate constituent systemsn Typically software–intensive and net–centricn SoS constituent systems independently developed and managedn SoS constituent systems have their own purposen Constituent systems can dynamically come and go from SoS

n System of Systems Engineering (SoSE)n “The process of planning, analyzing, organizing, and integrating the

capabilities of a mix of existing and new systems into a system–of–systems capability that is greater than the sum of the capabilities of the constituent parts. This processes emphasizes the process of discovering, developing, and implementing standards that promote interoperability among systems developed via different sponsorship, management, and primary acquisition processes.” [USAF 2005]

† System of Systems Engineering and Process Synchronization, Jo Ann Lane, University of Southern California, Center for Software Engineering

352

Dark Side


+ SoS Compared to Traditional Development (1)n Architecting

n Architecting composability vs. decomposition

n Net–friendly vs. hierarchical

n Prototypes/experimentation/tradeoffsn Early tradeoffs/evaluations of alternatives

n Intense concept phase analysis followed by continuous anticipation; aided by ongoing experimentation

n Modeling and simulation, in particular to better understand “emergent behaviors”

n First order tradeoffs above the component systems level (e.g., more optimal at the SoS level, instead of at the component system level)

n Discovery and application of convergence protocols


353

Dark Side


+ SoS Compared to Traditional Development (2)n Scope and performance

n Added “ilities” such as flexibility, adaptability, composability

n Human as part of the SoS

n Organizational scope defined at runtime instead of at system development time

n Dynamic reconfiguration of architecture as needs change

n Maintenance and evolutionn Component systems separately acquired and continue to be managed as

independent systems


354

Dark Side


+ System of Systems Engineering and Agilen SoSE teams are evolving traditional methods to support the development

and on–going evolution of SoSsn Early feasibility assessments and tradeoff analyses

n Knowing when to engineer and when not to engineer

n In general, leave the constituent systems engineering to the constituent system engineers

n Constituent system monitoring to ensure that constituent system changes are not adversely impacting the SoS

n Combining agile with traditional approaches

n Increases concurrency

n Reduces risk

n Compresses schedules


355

Dark Side


evm+agile the darkside

Technology