frameworks for assessing it systems engineering acquisition … · 2017. 5. 18. · engineering...

© 2012 Carnegie Mellon University

Frameworks for Assessing IT Systems

Engineering Acquisition Issues and Proposed

Approaches in Support of Public Law 111

Dr. Kenneth E. Nidiffer

Software Engineering Institute

Carnegie Mellon University

Pittsburgh, PA 15213

703-908-1117

15th Annual Systems Engineering Conference

Net Centric Operations/Interoperability Track

National Defense Industrial Association

October 22-25, 2012

2

Frameworks for Assessing IT Systems Engineering Acquisition

Issues and Proposed Approaches in Support of Public Law 111


Overview

• Perspective

• The Problem Space

• The Solution Space (Pre-Decisional)

• What Success Looks Like

Focus: Acquisition of

DoD IT Systems Source: SEI

3




3

Perspective: Cyber Landscape

What are the opportunities?

+ + + + …

Transportation

Infrastructure

Healthcare

Infrastructure

Banking & Financial

Infrastructure

Energy & Utilities

Infrastructure

Communications

Infrastructure

Includes all:

• System of Systems

• Architecture

• Services

• Networked Hardware/ Platforms

• People who digitally connect to

cyberspace

Source: SEI

4




Problem Space: Improving Efficiency and Effectiveness in IT/Cyber Acquisitions in DoD

Source: Director, Command and Control, Programs & Policy (OSD) - Pre-Decisional

5




Problem Space: Current DoD IT Environment

Source: Director, Command and Control, Programs & Policy (OSD) – Pre-Decisional

6




Problem Space: DoD IT Acquisition Cycle-Time - 32 MAIS

Initial

Operational

Capability Planning Phase

Analysis of

Alternatives

Economic

Analysis

Milestone B

MS C

40

48

5

Test

Build Phase

Development

Cycle-Time Driven by Processes Developed to Counter a Cold War Adversary In Industrial Age Society

43

91

Source: Defense Science Board Report, March 2009

7




Problem Space: IT Software Life Cycle Continuum

7 Approaches to Managing Software Development Projects

Gov’t Today

Desired State Industry

Intermediate Adaptive Life Cycle (Example)

Business Need Identified

8




Problem Space: Generic Acquisition Process

Source: Defense Science Board Report, March 2009

9




Year Proportion of software

maintenance costs

Definition Reference

2000 >90% Software cost devoted to system maintenance & evolution / total software costs

Erlikh (2000)

1993 75% Software maintenance / information system budget (in Fortune 1000 companies)

Eastwood (1993)

1990 >90% Software cost devoted to system maintenance & evolution / total software costs

Moad (1990)

1990 60-70% Software maintenance / total management information systems (MIS) operating budgets

Huff (1990)

1988 60-70% Software maintenance / total management information systems (MIS) operating budgets

Port (1988)

1984 65-75% Effort spent on software maintenance / total available software engineering effort.

McKee (1984)

1981 >50% Staff time spent on maintenance / total time (in 487 organizations)

Lientz & Swanson (1981)

1979 67% Maintenance costs / total software costs Zelkowitz et al. (1979)

Problem Space: No Milestone “D” – No Way to Re- Invest

Replacement Savings

10




Problem Space: 2011 DAW Age Analysis

024

3,427

15,24716,549

13,157

16,254

24,928

27,941

19,446

14,702

00

5000

10000

15000

20000

25000

30000

Under 20 20 - 24 25 - 29 30 - 34 35 - 39 40 - 44 45 - 49 50 - 54 55 - 59 Over 60

Empl

oyee

Cou

nt

Def Acq Workforce - Overall Age Distribution - FY11 Q3

[0.02%]

[10.1%][10.9%]

[2.3%]

[10.7%]

[18.4%]

[12.8%]

[9.7%]

[8.7%]

[16.4%]

IT Awareness

Policy Formulation

Data Source: OSD (AT&L Data Mart

11




Problem Space: Four Key Challenges to our Technical Base

Tim

e

Impact

Shift in

Technical

Talent Base

Commercial

Tech Areas

Fo

reig

n

Time

Shift in

Technical

Talent Base

Increasing

Pace of

Innovation

Global

Access to

Technology

Foreign

Time

DoD

Te

ch

nic

al Ta

len

t

Source: DDR&E

12




Problem Space: Technological Rate of Adoption - the Cyber Domain is Hotly Contested

12 UNCLASSIFIED 12

High

Low

1980 1985 1990 1995

Sophistication

Required of Actors

Declining

So

ph

isti

ca

tio

n

cross site scripting

password guessing

self-replicating code

password cracking

exploiting known vulnerabilities

disabling audits

back doors

hijacking

sessions

sweepers

sniffers

packet spoofing

GUI

automated probes/scans

denial of

service

www attacks

“stealth” / advanced

scanning

techniques

burglaries

network mgmt. diagnostics

distributed

attack tools

Staging

sophisticated C2

…next?

Increased GIG Complexity

& Dependence equates to

lower entry barriers and

potential for increased

number of malicious actors

Sophistication

Of Available Tools

Growing

Defensive measures are outpaced by the well resourced sophisticated threat . . .

Source: DoD

13




Problem Space: An Effective Process for Major Defense Systems – But Not Very Agile for IT Systems

Source: Defense Acquisition University

14




Planning,

Programming,

Budgeting &

Execution

(PPBE)

Joint Capabilities

Integration &

Development

System (JCIDS)

Defense

Acquisition

System

Effective Interaction

Essential for Success

Problem Space: Alignment of Three Major DoD

Decision Support Systems

Big A

Source: Defense Acquisition University

15




Problem Space: Software-Reliant Acquisitions Can Be Difficult to Manage

According to Fred Brooks* software projects are difficult because of

accidental and essential difficulties

• Accidental difficulties are caused by the current state of our

understanding

— of methods, tools, and techniques

— of the underlying technology base

• Essential difficulties are caused by the inherent nature of software

— invisibility - lack of physical properties

— conformity

— changeability

— complexity

Dr. Fred Brooks

* Source: The Mythical Man-Month by Fred Brooks, Addison Wesley, 1995

16




Solution Space: Issues Are Well Known and Are Being Addressed


17




Solution Space: Legislative Landscape – 2009 and 2011 National Authorization Acts


18




Solution Space: Section 804 - IT Acquisition Reform Goals


19




Solution Space: IT Reforms in Progress


20




Solution Space: Section 804 Improvement Acquisition Concepts


21




Solution Space: Systems Engineering - Key Upfront Discipline

Source: DDR&E

22




What Success Looks Like: Enabled Agile Capability Delivery


23




What Success Looks Like: Alignment with DoD’s Better Buying Power


24




What Success Looks Like


25




Questions?

26




Contact Information

Dr. Kenneth E. Nidiffer, Director of Strategic Plans for

Government Programs

Software Engineering Institute, Carnegie Mellon University

Office: + 1 703-908-1117

Fax: + 1 703-908-9317

Email: [email protected]

27




NO WARRANTY

THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE

MATERIAL IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY

MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO

ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR

PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM

USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY

WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT,

TRADEMARK, OR COPYRIGHT INFRINGEMENT.

Use of any trademarks in this presentation is not intended in any way to infringe on the

rights of the trademark holder.

Requests for permission to use or reproduce should be directed to the Software

Engineering Institute at [email protected].

This work was created in the performance of Federal Government Contract Number

FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software

Engineering Institute, a federally funded research and development center. The

Government of the United States has a royalty-free government-purpose license to use,

duplicate, or disclose the work, in whole or in part and in any manner, and to have or

permit others to do so, for government purposes pursuant to the copyright license under

the clause at 252.227-7013.

frameworks for assessing it systems engineering acquisition … · 2017. 5. 18. · engineering...

Documents