avoiding the procrustean bed with the incremental

University of Southern CaliforniaCenter for Systems and Software Engineering

Barry Boehm, USC

Lean-Kanban NA KeynoteJune 8, 2015

Avoiding the Procrustean Bed with the Incremental Commitment Spiral Model (ICSM)

[email protected],http://csse.usc.edu


6/8/2015

Outline• Avoiding the Procrustean Bed

– Of one-size-fits-all process models

• ICSM Overview

• The 4 ICSM Principles– ICSM and Lean-Kanban– Recent Kanban for Systems of Systems Results

• Two Final Meta-Principles© USC-CSSE 2


The Procrustean Bed

• Procrustes: Greek Mythology

– Rogue smith and bandit

– Hostel with one-size-fits-all bed

– Guests too small: stretch them to fit

– Guests too large: lop off the offending parts

6/8/2015 © USC-CSSE 3


Build Your Own Procrustean Bed• Pure Waterfall, Vee: Fixed Price and Spec Contract

– Lop off needed changes as requirements creep• Pure Agile: Easiest First; Dedicated On-Site Customer

– Later scalability and assurance problems; single-failure point• Voice of the Customer: Accept All “Requirements”

– Gold-plating; neglect voices of acquirer, developer, owner• Piling on Incompatible Constraints: No Way Out

– Project Example: Waterfall, COTS, Ada, GOTS Reuse• Inflexible Standards: No Choice But Tailoring Down

– MIL-STD-498: choice of 23, 6, or 1 DID denied • Overconstrained Maturity Models: Excluding Expertise

– Software CMM: Exclude software group from system rqts.

6/8/2015 © USC-CSSE 4


Procrustean Example: DoD Acquisition Process

6/8/2015 © USC-CSSE 5


6/8/2015



• ICSM Overview




6/8/2015

What is the ICSM?

• Risk-driven framework for determining and evolving best-fit system life-cycle process

• Integrates the strengths of phased and risk-driven spiral process models

• Synthesizes together principles critical to successful system development– Stakeholder value-based guidance– Incremental commitment and accountability– Concurrent multidiscipline engineering– Evidence and risk-driven decisions

Principles trump

diagrams…

Principles used by 60-80% of CrossTalk Top-5 projects, 2002-2005© USC-CSSE 7


The Incremental Commitment Spiral Model

6/8/2015

1

2

3

4

5

6

RISK-BASEDSTAKEHOLDER COMMITMENT REVIEW POINTS:

Opportunities to proceed, skip phases backtrack, or terminate

Exploration Commitment Review

Valuation Commitment Review

Foundations Commitment Review

Development Commitment Review

Operations1 and Development2Commitment Review

Operations2 and Development3Commitment Review

Cumulative Level of Understanding, Product and Process Detail (Risk-Driven)

Concurrent Engineering of Products and Processes

2345

EXPLORATION

VALUATION

FOUNDATIONS

DEVELOPMENT1FOUNDATIONS2

OPERATION2DEVELOPMENT3

FOUNDATIONS4

16

Evidence-Based Review Content- A first-class deliverable- Independent expert review- Shortfalls are uncertainties and risks

OPERATION1DEVELOPMENT2

FOUNDATIONS3

Risk

Risk-Based Decisions

AcceptableNegligible

High, butAddressable

Too High, Unaddressable

© USC-CSSE 8


6/8/2015

ICSM Nature and Origins• Integrates hardware, software, and human factors

elements of systems life cycle– Concurrent exploration of needs and opportunities– Concurrent engineering of hardware, software, human aspects– Concurrency stabilized via anchor point milestones

• Developed in response to a variety of issues– Clarify “spiral development” usage

• Initial phased version (2005)– Provide framework for human-systems integration

• National Research Council study, book (2007)• Integrates strengths of current process models

– But not their weaknesses• Facilitates transition from existing practices

– Electronic Process Guide (2009)Copyright © USC-CSSE© USC-CSSE 9


6/8/2015 Copyright © USC-CSSE 10

Incremental Commitment in Gambling

• Total Commitment: Roulette– Put your chips on a number

• E.g., a value of a key performance parameter– Wait and see if you win or lose

• Incremental Commitment: Poker, Blackjack– Put some chips in– See your cards, some of others’ cards– Decide whether, how much to commit to

proceed

© USC-CSSE 10


6/8/2015

ICSM Activity Levels for Complex Systems

© USC-CSSE 11


6/8/2015

Anchor Point Feasibility Evidence Descriptions• Evidence provided by developer and validated by independent

experts that:If the system is built to the specified architecture, it will– Satisfy the requirements: capability, interfaces, level of service, and

evolution– Support the operational concept– Be buildable within the budgets and schedules in the plan– Generate a viable return on investment– Generate satisfactory outcomes for all of the success-critical

stakeholders• All major risks resolved or covered by risk management plans• Serves as basis for stakeholders’ commitment to proceed• Synchronizes and stabilizes concurrent activities

Can be used to strengthen current schedule- or event-based reviews © USC-CSSE 12


6/8/2015

Risk-Driven Scalable Spiral Model: Increment ViewFor each level of systems-of-interest

Agile Rebaselining for

Future Increments

Short, StabilizedDevelopment

of Increment N

Verification and Validation (V&V)of Increment N

Deferrals

Artifacts Concerns

Rapid Change

HighAssurance

Future Increment Baselines

Increment N Transition/Operations and Maintenance

Future V&VResources

Increment N Baseline

Current V&VResources

Unforeseeable Change (Adapt)

ShortDevelopmentIncrements

ForeseeableChange

(Plan)

Stable DevelopmentIncrements

Continuous V&V© USC-CSSE 13


6/8/2015

Agile Change Processing and Rebaselining TriageWell suited to Kanban approach

Assess Changes,Propose Handling

StabilizedIncrement-N

Development TeamChange

ProposersFuture Increment

Managers

Agile Future-Increment

Rebaselining Team

Negotiate change disposition

Formulate, analyze options in context of other changesHandle

AcceptedIncrement-N

changes Discuss, resolve deferrals tofuture increments

ProposeChanges

Discuss, revise,defer, or drop

Rebaselinefuture-increment

Foundations packages

Prepare for rebaselinedfuture-increment

development

Defer some Increment-N capabilities

Recommend handlingin current increment

Accept changes

Handle in current rebaseline

Proposed changes

Recommend no action,provide rationale

Recommend deferrals to future increments

© USC-CSSE 14


6/8/2015 © USC-CSSE 15


6/8/2015



• ICSM Overview




Principles Trump Diagrams: SpiralTry Googling on “spiral process images”

6/8/2015

Increment or

Block 2Increment or

Block 3

Where are stakeholders, commitments, concurrency, risk?© USC-CSSE 17


6/8/2015

Principles Trump Diagrams1. Stakeholder value-based guidance

1. Incremental commitment and accountability

1. Concurrent system engineering

2. Evidence and risk-driven decisions

Counterexample: Bank of America Master Net

Good example: Symbiq Medical Infusion Pump

© USC-CSSE 18


6/8/2015

ICSM Principles Counterexample:Bank of America Master Net

© USC-CSSE 19


6/8/2015

Principles Trump Diagrams: Master Net1. Stakeholder value-based guidance

– Overconcern with Voice of Customer: 3.5 MSLOC of rqts.– No concern with maintainers, interoperators: Prime vs. IBM

2. Incremental commitment and accountability– Total commitment to infeasible budget and schedule– No contract award fees or penalties for under/overruns

3. Concurrent multidiscipline engineering– No prioritization of features for incremental development– No prototyping of operational scenarios and usage

4. Evidence and risk-driven decisions– No evaluation of Premier Systems scalability, performance– No evidence of ability to satisfy budgets and schedules

© USC-CSSE 20


6/8/2015

Example ICSM Commercial Application:Symbiq Medical Infusion Pump

Winner of 2006 HFES Best New Design AwardDescribed in NRC HSI Study Book, Chapter 5

© USC-CSSE 21


6/8/2015

Symbiq IV Pump ICSM Process - I• Exploration Phase

– Stakeholder needs interviews, field observations– Initial user interface prototypes– Competitive analysis, system scoping– Commitment to proceed

• Valuation Phase– Feature analysis and prioritization– Display vendor option prototyping and analysis– Top-level life cycle plan, business case analysis– Safety and business risk assessment– Commitment to proceed while addressing risks

© USC-CSSE 22


6/8/2015

Symbiq IV Pump ICSM Process - II• Foundations Phase

– Modularity of pumping channels– Safety feature and alarms prototyping and iteration– Programmable therapy types, touchscreen analysis– Failure modes and effects analyses (FMEAs)– Prototype usage in teaching hospital – Commitment to proceed into development

• Development Phase– Extensive usability criteria and testing– Iterated FMEAs and safety analyses– Patient-simulator testing; adaptation to concerns– Commitment to production and business plans

© USC-CSSE 23


6/8/2015

Principles Satisfaction: Symbiq IV Pump1. Stakeholder value-based guidance

– Extensive involvement of users, buyers, funders, regulators– Extensive use of prototyping, safety analysis methods

2. Incremental commitment and accountability– Expanding system definition and evidence elaboration– Decision to start with composable 1- and 2-channel pumps

3. Concurrent multidiscipline engineering– Concurrent evaluation of display, alarm, pump suppliers– Concurrent definition, evaluation of safety and business cases

4. Evidence and risk-driven decisions– Evidence-based reviews of technical and business feasibility– Outstanding risks covered by next-phase risk mitigation plans

© USC-CSSE 24


ICSM and Lean Principles• Stakeholder value-based guidance

– See the whole– Empower the team

• Incremental commitment and accountability– Amplify learning– Decide as late as possible

• Concurrent multidiscipline engineering– Deliver as fast as possible– Empower the team

• Evidence and risk-driven decisions– Build integrity in– Eliminate waste

6/8/2015 © USC-CSSE 25


6/8/2015



• ICSM Overview




Recent Kanban for Systems of Systems Results

Scenario Generator KSS Simulator

Scenario

User input

Performanceindicators

Simulation Results

Simulation configuration- Resources allocation,- Prioritization algorithm,- etc.

Scenario Configuration- number of teams,- number of Wis,- complexity of dependencies,- etc.

User input

• System of interdependent systems with ongoing change traffic• Comparing Kanban, last-in-first-out, and random prioritization

6/8/2015 © USC-CSSE 27


Results: complex scenario

0

10000

20000

30000

40000

50000

60000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80

Value

Time

KSS

Value-neutral (random selection)

LIFO

6/8/2015 © USC-CSSE 28


Results: complex scenario

0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80

Number of Suspended

Tasks

Time

KSS

Value-neutral (randomselection)

LIFO

6/8/2015 © USC-CSSE 29


Results: complex scenario - total time spent

61

62

63

64

65

66

67

68

69

70

71

KSS Value-neutral (random selection) LIFO

Total schedule (calendar days)

6/8/2015 © USC-CSSE 30


Results: complex scenario - total effort

440

460

480

500

520

540

560

580

600

KSS Value-neutral (random selection) LIFO

Total effort (person-days)

Effort requiredif there are nointerruptions

6/8/2015 © USC-CSSE 31


Meta-Principle 1: Risk Balancing

• How much (system scoping, planning, prototyping, COTS evaluation, requirements detail, spare capacity, fault tolerance, safety, security, environmental protection, documenting, configuration management, quality assurance, peer reviewing, testing, use of formal methods, and feasibility evidence) are enough?

• Answer: Balancing the risk of doing too little and the risk of doing too much will generally find a middle-course sweet spot that is about the best you can do.

6/8/2015 © USC-CSSE 32


Meta-Principle 2: Choice of PeopleTrumps everything else

• Agile Manifesto Finding #1– Individuals and interactions over processes and tools

• Curtis MCC Study: 19 large projects– Critical success factor: Keeper of the Holy Vision

• Ramo: Best engineers are T-shaped

6/8/2015

Hardware Peopleware Economics Applications Disciplines Software © USC-CSSE 33


6/8/2015

Match the People to the JobsGood at all three: Your chief engineers and future leaders

Agile Rebaselining for

Future Increments

Short, StabilizedDevelopment

of Increment N

Verification and Validation (V&V)of Increment N

Deferrals

Artifacts Concerns

Rapid Change

HighAssurance

Future Increment Baselines

Increment N Transition/Operations and Maintenance

Future V&VResources

Increment N Baseline

Current V&VResources

Unforeseeable Change (Adapt)

ShortDevelopmentIncrements

ForeseeableChange

(Plan)

Stable DevelopmentIncrements

Continuous V&V© USC-CSSE 34


6/8/2015 Copyright © USC-CSSE

List of AcronymsB/L BaselinedC4ISR Command, Control, Computing, Communications, Intelligence, Surveillance,

ReconnaissanceCD Concept DevelopmentCDR Critical Design ReviewCOTS Commercial Off-the-ShelfDCR Development Commitment ReviewDI Development IncrementDoD Department of DefenseECR Exploration Commitment ReviewEVMS Earned Value Management SystemFCR Foundations Commitment ReviewFED Feasibility Evidence DescriptionFMEA Failure Modes and Effects AnalysisFRP Full-Rate ProductionGAO Government Accountability OfficeGUI Graphical User Interface

© USC-CSSE 35



List of Acronyms (continued)

HMI Human-Machine InterfaceHSI Human-System InterfaceHW HardwareICSM Incremental Commitment ModelIOC Initial Operational CapabilityIRR Inception Readiness ReviewIS&SE Integrating Systems and Software EngineeringLCO Life Cycle ObjectivesLRIP Low-Rate Initial ProductionMBASE Model-Based Architecting and Software EngineeringNDI Non-Developmental ItemNRC National Research CouncilOC Operational CapabilityOCR Operations Commitment ReviewOO&D Observe, Orient and DecideOODA Observe, Orient, Decide, ActO&M Operations and Maintenance

© USC-CSSE 36



List of Acronyms (continued)PDR Preliminary Design ReviewPM Program ManagerPR Public RelationsPRR Product Release ReviewRUP Rational Unified ProcessSoS System of SystemsSoSE System of Systems EngineeringSSE Systems and Software EngineeringSW SoftwareSwE Software EngineeringSysE Systems EngineeringSys Engr Systems EngineerS&SE Systems and Software EngineeringUSD (AT&L) Under Secretary of Defense for Acquisition, Technology, and LogisticsVCR Validation Commitment ReviewV&V Verification and ValidationWBS Work Breakdown StructureWMI Warfighter-Machine Interface

© USC-CSSE 37

avoiding the procrustean bed with the incremental

Documents