does use of agile practices support affordability? · does use of agile practices support...

See attached Boeing’s Assignment of Copyright. The work of authorship referred to herein is a work-made-for hire by one or more employees of The Boeing Company

Dr. Shawn Rahmani and Steve Shyman

Boeing Defense, Space/Security

Huntington Beach, California

17th NDIA SE Conference

Oct 29, 2014

Does Use of Agile Practices Support Affordability?

Objective/Outline • Software Affordability

– Objective – Framework – Example outputs

• Model-based approach for quantitatively determining

benefits of agile practices – Approach – Data – Results – Summary

2 Copyright © 2014 Boeing. All rights reserved

3

1. Identifying & Quantifying Affordability Opportunities

2. Performing Affordability Trades • Custom trade features

3. Assessing Risks and Issues • Technical/programmatic topics • Cost/schedule impact

4. Estimating, Capturing, and Evaluating Decision Impacts (similar to Risks)

5. Providing/supporting DtC, DfV, and CAIV Capabilities

Affordability Objective: Develop Process, Techniques & Interactive Tool for


4

Model-based Estimating

Tools

Custom Evaluation

Process, Tool, KB

Affordability /Risk

Evaluation Trade Space

Affordability Assessment Tool

Software Focused, Extendable to Systems Engineering

Programmatic/ Technical Trade Options

Risks

Cost/Schedule Productivity Figures per Trade Option

Cost Targets

Affordability Decision Support (EAC, Security--Cost)

Risk Impacts (Dollarized )

Affordability Framework

Project Data/ Parameters

Historical Data

Trade Drivers


Trading Process and Outputs

Proposal Estimate

Execution Plan to Budget

Execution Plan

Sat3 FSW Probability of Success

Detailed Staffing Plan / % Work Product Complete

SW Affordability Trade Risk Assessment

MS Project Plan/Activity Flow

-0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9

Aug-13 Sep-13 Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14

Test

Program

Design

Req Analyst

QC Lead

CM/RM

Proj Mgr

Staf

f Lev

el

Sat3 Staffing Plan

Build 1 SW I&T

Build 2 SW I&T

FQT Script devlopement & Dry RunsBuild 1 Code

Build 2 Code

TRRFQT

PDR CDRG2U G2S SOR

SRS (Baseline) SRS (TBRs)

SRS (Final)STD STR

IADQDDSPDP STP

C1 C2 C3Ref

11121213131414

24

26

28

30

32

34

0 1 2 3 4 5 6 7 8 9 10 Sche

dule

(M

onth

s)

Effo

rt (

pMon

ths)

Process, tools & automation

Trade CasesBase Estimate with

increased scope using

S&IS standard practices

Increased automated test tool usage

(apply new Launcher tool) during SW

I&T with some target experience

Advance automated test tool usage to

FQT with greater target and process

experience

Base

(Proposal)

Estimate

Sat3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1-Sep 1-Oct 1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May 1-Jun 1-Jul 1-Aug 1-Sep 1-Oct 1-Nov

Analyst SRS (Plan)

Design (Plan)

Program Code (Plan)

Test/QA Scripts (Plan)

Sat3 FSW Schedule

% W

ork

Com

plet

e

Sat3 FSW Product Development Plan

TRRFQT

PDR CDRG2U G2S SOR

SRS (Baseline) SRS (TBRs)

SRS (Final)STD STR

IADQDDSPDP STP

Build 1 SW I&T

Build 2 SW I&T

DVMST/FQT Script devlopement & Dry RunsBuild 1 Code

Build 2 Code

Sat3

10/17/13

SW Reqts Development•SRS Draft

•VCRM Draft

•TBRs identified

9/6/13

Sat 3 Build 1 (code &unit test)• Baseline

• Feature 1

• Feature 2

2/20/14

11/14/13

Preliminary Design

Review

Sat 3 Build 2

(code & unit test)

• Feature 3

• Feature 4

2/24/14

4/22/14

FQT & Dry Run Prep

• Review Test Plan

• Develop FQT Test Cases

4/25/14

6/5/14

8/11/14

FQT

9/1/14

9/19/14

Prep for Selloff

•Release docs in TeamcenterFSW STR

IAD

SVD final FSW buildURMA

QDD

9/19/14

10/2/14

Selloff

PROM Consent

to burn

2/21/14

Build 1 Release

Sat 3 Build 3 code &

unit test

• Feature 5

4/25/14

4/24/14

Build 2 Release

8/29/2014

TRR

10/2/14

12/12/13

Gate 2U SRS

Delivery

STP Release

11/30/13

Dry Runs & FQT Prep

• Execute Dry Runs

• Qual HITL

•IAD draft

• Release docs in TeamcenterHITL STR

STD, SRS VCRM, STP, SVD FQT

Scripts

8/25/14

Build 3 Release

6/5/14

FQT Execution

• Close CSU/Pkg

VCRMs

6/6/14

8/11/14

Sat 3 Build 1 SWIT /

Risk Mitigation

2/24/14

4/16/14Sat 3 Build 2 SWIT

Sat 3 Build 3

SWIT

Build 3 interim

Release

4/25/14 5/30/14

5/8/14

5/12/14

6/4/145/9/14 5/122014

CDR

SRS /VCRM Rev -(TBRs Resolved)

4/18/14


Model for Quantitative

Assessment of Agile Practices


Approach

• Collect and analyze Agile Cost Reduction Data – Identify impact on cost reduction of basic

characteristics provided in the data

• Use statistical analysis


Empirical Data Analysis • The Data

– Cost reduction data collected from 103 projects performing agile practices

– Assuming all projects are utilizing SCRUM to qualify for inclusion

– Source is either Engineering Judgment, Cost data, or Productivity values

• Attempted to quantify 5 potential cost impactors

– Size (SLOC): small, medium, large

– New /Follow-on

– Continuous Integration

– Automated Testing

– Development Environment Change

– Team Capability

• Analysis included

– Correlation matrix

– Subset averages

– Regression Analysis


Regression Analysis

0

0

1

1 10

En

ter

Enter

Enter

9 Boeing Proprietary Copyright © 2014 Boeing. All rights reserved

• These results suggest factors that can be used in a model: SCRUM, Continuous Integration, and Automated Testing

• Strong correlation between Continuous Integration and Automated Testing suggest if adopt one will adopt both

Other benefits: higher quality, customer and employee satisfaction, improved visibility, improved communication,

lower risk, earlier working product

Platform / Source

Average

Cost

Reduction

Domain

Expertise

Follow-on

Developme

nt

Significant

Environme

nt Changes

Automated

Testing

Continuous

Integration

Avionics 32% Very High Most Some Most Most

Cyber Security 20% High Some Most Most Most

Ground Mission Critical 20% High None Most Most Some

Ground Mobile 20% Nominal Most Some All All

Ground Non –Mission Critical 19% High Some Some Some Most

Manned Space 5% High Some Some Some Some

Missile 17% High Some All Some Some

Simulation 20% High All Some Some Some

Unmanned Space 18% High Some Some Some Some

Web Development 52% Very High None All None None

Overall Average (of all projects submitting data) Middle 50th percentile:

24%

12%-32%

Agile Software Data Results (December 31, 2013 version)


11

1.Trade Case Selection - Agile features

1st change

2nd change

3. Model Parameter Adjustment

• Example shows: • Effort reduced -- 193 Person Months (21%) • Schedule reduced -- 3.5 months (8%)

Use of Agile within Affordability Framework (Notional)

2. Model Parameter Selection Copyright © 2014 Boeing. All rights reserved

Summary

Data is sufficient for simple modeling Data allows for subset analysis to assess impact of various factors, e.g., impact of AT/CI

Data does support assumptions of cost reduction Agile reduces cost Small programs have greater reduction Personnel characteristics are significant factor Automated Testing and Continuous Integration tend to be adopted together and do reduce cost

12

Acknowledgement

The authors would like to thank the entire Boeing Agile Software Process (BASP) team members for their contributions to the work presented here. Special thanks go to: Curt Hibbs, Jonathan Kiser, Mark Roberts, Phyllis Marbach, Cindi Walker, Tom Treffner, and Waltraud Taferner.


Backup Information

Abstract

Affordability is a key driver in today’s DOD environment and impacts the DOD contractors and supply chain. Use of agile software engineering practices has shown encouraging results within the community. However, collecting, evaluating, and validating empirical data that shows the quantitative impacts and benefits of using agile practices has been challenging. This presentation describes a framework for performing trades in support of affordability goal settings, assessments and decisions. It then describes how the use of agile practices can be used within the framework. Specifically, it describes (a) the approach for collecting and analyzing the empirical data from a set of agile software development projects, (b) the statistical parametric model used for capturing and presenting the characteristics of the agile projects, and (c) the quantitative results of the study. The presentation also describes the challenges and lessons learned from this study, how quantitative results are used within the affordability framework, and the forward plan for improving the overall capability.


ASSIGNMENT OF COPYRIGHT

Date of Presentation(s):

October 29 , 2014

Copyright Form To:

The 17th National Defense Industrial Association Systems Engineering Conference

Publication Title:

Does Use of Agile Practices Support Affordability?

Author(s): Shawn Rahmani and Steve Shyman

At least one, but possibly not all, of the authors of the Work is an employee of The Boeing

Company or one of its wholly-owned subsidiaries (“BOEING”) preparing the Work within the

scope of his or her employment. The Boeing employees in question are not authorized to

assign or license Boeing’s rights therein, or otherwise bind Boeing. However, subject to the

limitations set forth below, Boeing is willing to assign its copyright in the Work to Publisher.

Notwithstanding any assignment or transfer to the Publisher, or any other terms of this

agreement, the rights granted by Boeing to Publisher are limited as follows: (i) any rights

granted by Boeing to the Publisher are limited to the work-made-for-hire rights Boeing enjoys

in the Work; (ii) Boeing makes no representation or warranty of any kind to the Publisher or

any other person or entity regarding the Work, the information contained therein, or any

related copyright; and (iii) Boeing retains a non-exclusive, perpetual, worldwide, royalty-free

right, without restriction or limitation, to use, reproduce, publicly distribute, display, and

perform and make derivative works from the Work, and to permit others to do so.

Contracts Specialist

Intellectual Property Management

does use of agile practices support affordability? · does use of agile practices support...

Documents