ideas & products enabling your success! dayton aerospace 1 why revitalize systems engineering?...

1Ideas & Products Enabling Your Success!

Dayton Aerospace

Why Revitalize Why Revitalize Systems Engineering?Systems Engineering?

ororHow We Got To How We Got To This Point TodayThis Point Today

July 21, 2004Tom Hoog

Dayton Aerospace, Inc.

© 2004 Dayton Aerospace, Inc.


Dayton Aerospace

Systems Engineering EulogySystems Engineering Eulogy• SE born in the 1930s as one of RCA Broadcasting Services & Standards• Military development of Ops Research in the 1940s was instrumental in shaping its youth• First formal education attempted at MIT in 1950• SE became the darling of NASA & military systems in the 1960s; processes imposed on

prime contractors• Its many offspring were Chief Systems Engineers, Directors of Engineering, VPs of

Systems Engineering as well as many Systems Engineers• It became bloated & unwieldy in late 1960s; became known as AFSCM 375-5• Attended fat farm & reappeared as Mil-Std-499A; grew in stature & acceptance• SE became check & balance in massive engineering efforts• It suffered an identity crisis when universities taught industrial engineering as systems

engineering• In the 1980s SE attempted to re-identify itself as Mil-Std-499B• Later cashiered out of the military, it found civilian employment as an IEEE standard• It wasted away with an illness brought on by rejection of its processes & bastardizing of its

techniques & methods• Design reviews became re-design meetings• SE died a lingering death in the 1990s & was replace by IPTs, Design Teams & Group

Gropes• SE is survived by & remembered by a cult of old systems engineers who still practice their

now meaningless rituals• SE was almost 70 years old at the time of its death• No services are planned

Full text of this eulogy can be found at http://acc.dau.mil/simplify/ev.php?ID=6174_201&ID2=DO_TOPIC


Dayton Aerospace

OutlineOutline

Background• DoD Transformation

– 5000– JCIDS– Scope of SE

• Systems Engineering Revitalization Initiatives– USD (AT&L) Imperatives– SAF/AQ Initiatives– USD (AT&L) Initiatives

• Health of SE – DAI’s View (Jul 03)


Dayton Aerospace

OutlineOutline






Dayton Aerospace

BackgroundBackgroundDoD Decision Support SystemsDoD Decision Support Systems

• Three parts to DoD Decision Support Systems

• Requirements Generation– Governed by CJCSI 3170.01– reissued 24 June 2003

• Defense Acquisition– Governed by 5000 series– Reissued 12 May 2003

• Financial Management – Governed by multiple sources– Being revised

Financial Management

System

Requirements Generation

System

Defense Acquisition

System


Dayton Aerospace

Functional Area Analysis

ICD

CDD

CPD

JCIDS Analysis

Joint Operating ConceptsJoint Functional ConceptsIntegrated Architectures

Strategic PolicyGuidance

Joint Operations Concepts

DOTMLPF Changes

CJCSI 3180

Process

Functional Solution Analysis

DO

TM

LP

FA

na

lysi

s

Materiel ChangesCJCSI 3170

process

Ideas for Materiel

Approaches

Analysis of Materiel Approaches

Alternative N

Alternative 2

Alternative 1

PostIndependent

Analysis

DOTMLPF Change

Recommendation

Functional Needs

Analysis

BackgroundBackground


Dayton Aerospace

ADAPT INTEGRATE

THE “New” 5000 MODEL

User Needs &

Technology Opportunities

Process entry at Milestones A, B, or C

Start into Increments where appropriate

Increments are interrelated

Continues for development life of system

Adapted From DoDI 5000.2, 12 May 03

IOCBA

Concept

Refinement

Technology

Development

System Development& Demonstration

Production & Deployment

Systems Acquisition

Operations & Support

C

Sustainment

FRP DecisionReview

FOC

LRIP/OT&EDesign

Review

Pre-SystemsAcquisition

(ProgramInitiation)

ConceptDecision

System

IntegrationSystem

Demonstration

TEST AND FIELD

DEV

DEMO

DEMO

DEV DEMO

MATURE

MATURE

MATURE

ADAPT INTEGRATE

ADAPT INTEGRATE

TEST AND FIELD

TEST AND FIELD

B

B

B

C

C

C

Spiral Development - Insert technology into program when it is mature

Readiness


Dayton Aerospace

SE in Evolutionary Acquisition EnvironmentSE in Evolutionary Acquisition Environment

• Multiple concurrent increments lead to increased need for disciplined SE process (vs. single step process)

• Continuous requirements definition & technology maturation tasks– Complete definition of requirements

• Plan to mature requirements• Traceable from top level operational requirements to design

implementation• Inclusion of joint requirements (FoS, SoS, net–centric)• Allocation of requirements to increments

– Mature producibility, affordability, supportability, of technology as well as performance

• Need to cover both government & industry aspects



Dayton Aerospace

5000 Series Extracts5000 Series Extracts

• DoDD 5000.1– Para E 1. 27 Acquisition programs managed through

application of SE approach that optimizes total system performance & minimizes total ownership cost. A modular open systems approach shall be used

• DoDI 5000.2– Para 3.9.2.2 Sustainment uses SE methodology– Para E 7.2 Human factors is part of SE– Para 7.7 Include ESH in SE

• DoD 5000.2-R (Interim Defense Acquisition Guidebook)Para C2.3.2 Acquisition strategy includes SEPara C2.8.2 Support strategy is part of SEPara C2.8.5.5 Human factors is part of SEPara C2.8.6 ESH is part of SEPara C6.7.2.7 Critical Program Information is part of SEPara C5.2 18 pages on SE


Dayton Aerospace

DoD 5000.2-R, Para C5.2DoD 5000.2-R, Para C5.2• Defines what SE does• Defines the SE activities

– Requirements analysis– Functional analysis/allocation– Design synthesis & verification– System analysis & control– Manufacturing & production– Modeling & simulation– Quality– Acquisition logistics– Open systems design– Software management

• Other Design Considerations– WBS– Performance specifications– Metrics– Insensitive munitions

• COTS• Reliability, availability & maintainability• Human systems integration• Environment, Safety & operational health• Interoperability• Survivability• Mission assuredness• Information assurance• Anti-tamper provisions•Other Design considerations

•WBS•Performance Specifications•Metric system•Insensitive munitions•Value engineering•Precise time & time interval•Accessibility•Corrosion prevention & control


Dayton Aerospace

Scope of Systems EngineeringScope of Systems Engineering

Systems Engineering

Adapted from DoDI 5000.2 12 May 2003

AcquisitionRequirements

Acquisition community participation in early requirements process to assist in understanding


Dayton Aerospace

SE DefinitionsSE Definitions

• Systems Engineering (SE) A comprehensive, iterative Technical Management (TM) process that includes translating operational requirements into configured systems, integrating the technical inputs of the entire design team, managing interfaces, characterizing and managing technical risk, transitioning technology from the technology base into program specific efforts, and verifying that designs meet operational needs. It is a life cycle activity that demands a concurrent approach to both product and process development. (11th Glossary of DoD Acqu Terms)

• Systems Engineering - An inter-disciplinary approach and means to enable the realization of successful systems. (EIA 731.1)


Dayton Aerospace

Some Systems Engineering CharacteristicsSome Systems Engineering Characteristics

• Encompasses government and contractor efforts– Either may lead– Some joint

• Iterative process at multiple levels– System level – Subsystem/component level

• Specific tasks depend on program phase– Pre-Acquisition, Acquisition, Sustainment

• Specific focus of tasks/trades depends on type of program– New, modification– Prime or support equipment; system or subsystem– Aircraft, space, electronics, armament– Complete system, subsystem, System of Systems / Family of

Systems



Dayton Aerospace

System Engineering Focus Shifts over System Engineering Focus Shifts over Program Life Cycle Program Life Cycle

• Pre-Acquisition– Operational & system performance requirements – Life cycle affordability requirements

• Acquisition– Design requirements & implementation– Manufacturability, supportability, upgradeability, etc.– System integrity,

• Sustainment– Supportability– System integrity

Modification/Upgrades: Repeat the Cycle



Dayton Aerospace

Requirements AnalysisAnalyze missions & environments

Identify functional requirements

Define/refine performance & design constraint requirements

Functional Analysis/AllocationDecompose to lower level functions

Allocate performance & other limiting requirements to all functional levels

Define/refine functional interfaces

Define/refine/integrate functional architecture

SynthesisTransform architectures

Define alternatives system concepts, configuration items & system elements

Define/refine physical interfaces

Select preferred product & process solutions

System Analysis & Control

Requirements Loop

Design Loop

Verification

Phase dependent

Decision support data

System architecture

Specifications & baselines

Input

Customer needs/objectives/ requirements

Technology base

Outputs from prior phase

Program decision requirements

Requirements applied through specifications & standards

Output

Trade-off studies

Effectiveness analyses

Risk management

Configuration management

Interface Definition

Performance based progress measurements IMP IMS Technical reviews

Adapted from Mil-Std-499B (Draft)


Dayton Aerospace


Dayton Aerospace

OutlineOutline






Dayton Aerospace

USD (AT&L) ImperativesUSD (AT&L) Imperatives

• “Provide a context within which I can make decisions about individual programs.”

• “Achieve credibility and effectivness in the acquisition and logistics support processes.”

• “Help drive good systems engineering practice back into the way we do business.”

From Feb 04 OSD (AT&L) - NDIA SE Division Meeting


Dayton Aerospace

SAF/AQ SE RevitalizationSAF/AQ SE Revitalization

• SAF/AQ Dr. Sambur Memo, Apr 9, 2003– Focus attention on application of SE & elevate these disciplines

commensurate with cost & schedule– Develop SE performance incentives– Include status of SE in future program reviews– Identify key SE processes & practices in acquisition

documentation– Acquisition Strategy Panel briefings must address significant

SE areas• SAF/AQ Memo, Increment 2, Jan 7, 2004

– Intended to institutionalize key attributes of an acceptable SE approach

– Insert the appropriate language into the following acquisition documents

• Solicitations • Award Fee Plan/Incentive Fee Contract • Contracts


Dayton Aerospace

USD (AT&L) InitiativesUSD (AT&L) Initiatives

• AT&L SE Policy Memo, Feb 20, 2004– Effective immediately; included in next DoD 5000 revision– All programs, all ACATs shall apply a robust SE approach– SEP for MDA approval– Drive good SE back into the way we do business

• AT&L SEP Guidance Memo, Mar 30, 2004– Purpose of SEP is to lay out a plan to guide the technical

aspects of an acquisition program– Living document, tailored to the program; roadmap to support

program management– No prescribed format; address integration of technical aspects

to include:• The systems engineering processes to be applied in the program • The systems technical baseline approach• Event-driven timing, conduct, success criteria, and expected

products of technical reviews• The integration of systems engineering into the program’s IPTs


Dayton Aerospace

OutlineOutline




• Health of SE – Briefing to AT&L (SE OPR)– Quick Study


Dayton Aerospace

Health of SE TasksHealth of SE Tasks

• DoD 5000 & JCIDS reissued in May & June 03• Briefing in Jul 03• DAI personnel experiences

– Lack of execution discipline on programs– 18 problem areas cited– Common Denominators– List of Issues– Candidate emphasis areas listed

• DAI asked to provide supporting data• Quick Study Report/Briefing in Sep 03


Dayton Aerospace

Common DenominatorsCommon Denominators

• Program Teams are incentivized by cost and/or schedule - not execution of disciplined SE practices

• Products and Processes are getting out of balance

• Inadequate depth in applied SE capability



Dayton Aerospace

IssuesIssues

Lack of SE Execution Incentives• Contracts reward cost and schedule

performance

Product and Process Imbalance• Emphasis on speedy delivery of product can

De-emphasize “processes”• Misapplication of spiral / evolutionary

development intent is used to justify program shortcomings – “fix it in the next spiral”

• Lack of true multifunctional activities in IPTs



Dayton Aerospace

Issues (cont)Issues (cont)

Inadequate Depth in Applied SE Capability• Sound SE execution requires many trade

studies• Government and industry “de facto” rely on their

personnel to pass along “lessons learned”• Government and their contractors are losing

“critical mass” in some areas• Systems Engineering processes are adequately

defined BUT not followed, not understood, or are short circuited

• Advertised CMMI ratings do not assure effective program implementation



Dayton Aerospace

Summary of Observed ProblemsSummary of Observed Problems

• Observed Problems - Approach• Three Categories of Problems

– Requirements Management (8)– Systems Engineering Process (7)– Engineering Management (6)


Dayton Aerospace

Observed Problems - ApproachObserved Problems - Approach

• Based on analysis of many programs• Each problem area included

– Observed Problem/Issue• Statement of problem or issue• Program examples

– Impact• What happened because of problem

– Recommendations• What can be done to avoid the problem

• Focus was on systems engineering - recognized that systems engineering execution is not independent of overall program management

• Objective was to improve overall program performance



Dayton Aerospace

Observed ProblemsObserved Problems

Requirements Management• Incomplete Definition of Requirements• Requirements Creep• Requirements Volatility• Lack of a Specification Tree• Incomplete/Weak Verification requirements• Late requirements Forced Into Software• Interface requirements and Interface

Management• Failure to Implement and Maintain Requirements

Traceability Tools© 2004 Dayton Aerospace, Inc.


Dayton Aerospace

Observed Problems (cont)Observed Problems (cont)

Requirements Management Example• Inadequate definition of requirements

– Statement of operational requirements– Statement of contractual performance requirements

• Problems– ORD too detailed; includes solution– SRD restates ORD– Contract spec paraphrases SRD; includes design solution– Requirements not refined at lower tiers; incomplete requirements to

subcontractors– Incomplete requirements analysis

• Impacts– Design & implementation cannot proceed effectively– Incomplete design; scrap & rework; expectations not met

• Recommendations– Enforce requirements reviews at Milestone B Review– Establish process to assure the SRD is complete & accurate– Reinforce requirement that IMP have well defined accomplishments

with entrance & exit criteria; SRR must be key event for “opening the gate” to proceed with design

– Develop templates, guides checklists to aid in review processes



Dayton Aerospace


Systems Engineering Process• Systems Engineering Process Not Defined on the

Program• Systems Engineering Process Defined But Not Applied• Lack of Flow Down of Systems Engineering Process

Requirements to Development Subcontractors• Lack of Robust Systems Engineering Applied to Top

Level System Design• Lack of Timely System Integration and Test (I&T)

Planning• Inadequate Planning for Obsolescence and

Sustainment• Design Incorporated into Performance Specifications



Dayton Aerospace

Industry Standard

Corporate Process

Division Process

Program Process

Without a Tailored & Documented Systems Engineering Plan, Program Execution Suffers

Very Generic

Company Approach Top Level Applicable to all Products

Applicable to Product lines

Applicable to Specific Program

Well Documented

Rarely Documented


Dayton Aerospace

Why a Program Specific SEPWhy a Program Specific SEP

• Directly tied to rest of program– Consistency across program plans &

documentation (SOW, WBS, EVMS, etc.)– Readily correlated to IMP & IMS for execution

• Establishes technical approach for stakeholders

• Includes program unique nuances– Established by program requirements– Provides for explanation of unique tailoring



Dayton Aerospace


Engineering Management• Inadequate Schedules and Budget to

Accomplish Development Effort• Assumed Reuse Not Confirmed• Lack of Active Engineering Management of

Development Subcontractor• Integrated Master Plan (IMP) / Integrated

Master Schedule (IMS) Not Adequately Addressed and/or Not Followed on the Program

• No Single Technical focal Point (Chief Engineer)



Dayton Aerospace

ConclusionsConclusions

• Execution of sound SE processes is more important than ever

• Need forcing functions that drive execution of disciplined SE processes

Sound SE execution needs to be key ingredient in contract incentives



Dayton Aerospace

QuestionsQuestions

ideas & products enabling your success! dayton aerospace 1 why revitalize systems engineering?...

Documents

armamentcomplete system

support equipment system

ideas products

operational requirements

configured systems

life cycle activity

limiting requirements

process development