presentation from may 15, 2007 dinner meeting

8/9/2019 Presentation from May 15, 2007 Dinner Meeting

1/39


2/39

2

djGantzer 5-2-07

Objectives

s To give an overall summary of key SE process standardsand models

s To illustrate a top level comparison of them

s To indicate trends and uses (in DoD)

s To briefly address one key process Technical Planning

Disclaimer: The views and information presented here are the

authors alone and do not necessarily represent SAIC or DoD views.


3/39

3

djGantzer 5-2-07

Agenda

s SE Standards and Models

Overview Evolution of Standards & Models

s ISO - 15288

s EIA - 632

s IEEE - 1220

s

CMMI - Devs Defense Acquisition Guide/SE

Correlations across

Application to Tech Planning

Summary

Some Key References


4/39

4

djGantzer 5-2-07

Ohwither models & standards?!The Process Standards / Models Quagmire.Source: Systems and Software Consortium {SSCI}


5/39

5

djGantzer 5-2-07

SE Standards and Models Trends

CMM for SW SW-CMM(97) CMMII v1.1(02)CMMI-Dev v2.0(06)

EIA-731 [SECM](99)

SE-CMM

SECAM

MilStd 499 A/B

95 00 05

GEIA-632 (99) GEIA-632A (07)

IEEE-1220(99)IEEE-1220 IEEE-1220(05)

EIA/IS-632

ISO 15288(02)

INCOSE SE HB v2a (04)INCOSE SE HB v3 (06)

CMM- Capability Maturity Mode

ISO -15288: SE-System Life Cycle Processes

EIA - 632:Process for Engineering a System

IEEE-1220: Application & Management of the SE Process

Sources: INCOSE, SEI

CMMI-Acq (07)CMMI-AMSA-CMM

IEEE-15288 (04)

08?

06+


6/39

6

djGantzer 5-2-07

Overview of SE Standards + CMMI

ISO/IEC - 15288 SE System life cycle processes

Establish a common framework for describing the life cycle of systems 25 processes; incl7 project, 11 technical, and 7 agreement and enterprise; 62 pgs

- it is a hi-level framework [descriptive]

EIA - 632 Processes for Engineering a SystemProvide an integrated set of fundamental processes to aid a developer in the engineering or

re-engineering of a system

- 33 Requirements in 13 processes; 120 pgs

- it is less prescriptive, broader scope [e.g., incl enterprise environment]

IEEE - 1220 Application and Management of the SE Process

Defines the requirements for an enterprises total technical effort related to the developmentof products and processes that will provide life cycle support for the products. It prescribesan integrated technical approach to engineering a system and requires the application ofmanagement of the systems process throughout the product life cycle. - 28 requirements28; incl 14 general, 6 by LC stages, 8 in SE Process ; 87 pgs

CMMI - DEV Capability Maturity Model Integration

- 27 process areas:11 Management, 6 Engineering, 5 Support, and 5 Process; 573 pgs


7/39

7

djGantzer 5-2-07

SE Standards & Models Life Cycle PhasesISO 15288 EIA 632 IEEE 1220 CMMI (informal)

DoD/DAG

[DoDI 5000.2]

Concept Pre-system

Definition

Concept concept,

exploration, vision

Concept

Development

Development System Def.,

Subsystem design,

Detailed design

System Def.,

Subsystem design,

Detailed design;

FAIT

feasibility, design,development

Tech

Development;

System

Development.&

Demonstration

Production End product,physical,

IT&E

Production production,manufacturing,

delivery

Production &

Deployment:

LRIP

Utilization Utilization operations Operations &Support [O&S]:FRIP

Support Support support,

maintenance,

sustainment

O&S:

Sustainment

Retirement Retirement disposal,

phase out

O&S:Disposal


8/39

8

djGantzer 5-2-07

Scope of SE Standards

MIL-STD-499B

EIA/IS 632

ISO/IEC 15288

EIA 632

IEEE 1220

Levelo f

Detail

Breadth of Scope

(source: S. Sheard, SPC and J. Lake, SMi; 2004)


9/39

9

djGantzer 5-2-07

SE/SW Standards Harmonization Road Map

Adapted from:SC7 WG7 N0560 Systems Engineering Study Group Report 11JAN02

EIA 632

ISO/IEC

JTC1/SC7IEEE EIA

Software

Engineering

ISO/IEC 15288 IEEE 1220

ISO/IEC 12207

ISO/IEC 15504

An Harmonization road map

JTC1/SC7

Systems

Engineering

SC7 InternalHarmonization

Joint Standards Body

Harmonization

Assessment

[source: SSCI]


10/39

10

djGantzer 5-2-07

Simple Generic SE Process

source: Mil Std 499A/B and DAU guidance

Requirements

Analysis

Functional

Analysis/allocation

Synthesis

System Analysis

& Control

Inputs:

Outputs:

Note: Applied to AFIT/CSE SE Case Studies; http://www.afit.edu/cse/


11/39

11

djGantzer 5-2-07

IEEE 1220: Systems EngineeringProcess (2005)

2. RequirementsValidation

3. Functional

Analysis

1. RequirementsAnalysis

4. FunctionalVerification

6. DesignVerification

5. Synthesis

7. SystemsAnalysis*

8. Control

Clause 6 The SE ProcessClause 4 - General Requirements1. SE process

2. Polices & procedures for SE

3. Planning the technical effort:Prepare/update engineering plan; schedule;tech plans.

4. Development strategies

5. Modeling & prototyping

6. Integrated repository: data, tools.

7. Integrated data package: HW, SW, LCprocesses, human.

8. Specification tree

9. Drawing tree

10. System breakdown structure

11. Integration of the SE effort: concurrentengr., Int. teams.

12. Technical reviews

13. Quality management

14. Product and process improvement: re-engr., self-assess., LL.

Note: includes

detailed flows for

each activity; and an

example SEMP toc

[source:SSCI + mods from IEEE 1220]

*Requirements/Functional/Design

trade studies &assessments


12/39

12

djGantzer 5-2-07

EIA 632: Processes for Engineering aSystem (1999; reaffirmed 2003)

Supply Process

Acquisition Process

Acquisition and Supply

System Design

Requirements Definition Process

Solution Definition Process*

Product RealizationImplementation Process

Transition to Use Process

Technical ManagementPlanningProcess

ControlProcess

AssessmentProcess

Technical SupportProductVerificationProcess

ProductValidationProcess

RequirementsValidationProcess*

SystemsAnalysisProcess*

CONOPS & Requirements

Architectures/Designs

Products

Outcomes&

Feedback

Plans,Directives,& Status

Acquisition

Request

System

Products

Note: providesdetailed activities

and outcomes for

each process

(source: graph from INCOSE SE Handbook v2)*

Note: under

revision to

harmonize w ISO

15288


13/39

13

djGantzer 5-2-07

Draft update to EIA-632 (out for internal review)EIA 632 [1999] EIA - 632a *{source: GEIA report; R.

Harwell, 11/05}; 4 domains; Systems

Management, System Creation, System

Evaluation, Enterprise Environment

Planning Planning

Assessment Progress Assessment

Control Control

Requirements Definition Concept Definition

Solution Definition System DefinitionProduct Realization

[Implementation/Transition]

System Realization

Systems Analysis Mission & Systems Analysis

Requirements & End products Val.,

System ver.

System V&V

Supply & Acquisition Customer & Supplier Relationship

Management

Enterprise Support Resources & Infrastructure

Enterprise Support Governance

Enterprise support LC Portfolio Mngt.


14/39

14

djGantzer 5-2-07

ISO 15288: System Life-Cycle Processes(2002)

ENTERPRISEAGREEMEN

PROCESSE

ENTERPRISEAGREEMEN

PROCESSE

Source: INCOSE SE Handbook, v3.0Each process has purpose, outcomes, and activitiesEach process has purpose, outcomes, and activities


15/39

15

djGantzer 5-2-07

CMMI Issues

DoD Sponsor Report

s Programs execute at lower maturity levels than their organizations have achieved andadvertised

s High-maturity practices are not consistently applied at the project level after contract

award

s How to ensure new projects will incorporate CMMI processes

s Appraisal sampling procedures how to ensure adequate coverage of theorganizational unit

s Appraiser quality training, consistency

s Lack of agreement on what constitutes Levels 4 and 5

s Need to converge to a single representation

s Content ofappraisal disclosure statements is lacking

s Inadequate training and education for acquirers

s Should CMMI be used forsource selection?

Source: NDIA SE Conf., 10/06 K. Baldwin


16/39

16

djGantzer 5-2-07

Summary Improvements in CMMI-Devv1.2 [August, 2006]

s New material added to CMMI v1.2 to ensure processes usedat project start-up

s Appraisal Disclosure Statement (ADS) improved

s Sampling rules improved

s High maturity appraisercertification instituted

Additionally

s CMMI guide for Acquisition Orgs to leverage Suppliersefforts 3/07

s CMMI Acq in final draft [summer 07?]

Source: NDIA SE Conf., 10/06 K. Baldwin


17/39

17

djGantzer 5-2-07

Capability Maturity Model Integrated(CMMI- Dev)

CMMI Core

SERelated

Examples

Integrated Product and

Process Development

SupplierSourcing

SW

RelatedExamples

V 1.1 (2002)

CMMI Core (now includes SS)

SERelated

Examples

IPPD

SW

RelatedExamples

HardwareRelated

Examples

V 1.2 (2006)

Organizational Goal

(OPD)Project Goal (IPM)

Source: SEI* Sources incl EIA-731(SECM)


18/39

18

djGantzer 5-2-07

[Acq] Requirements Management

[Acq] Requirements Development

[Acq] Technical SolutionProduct Integration [Acq drops this?]

Verification

Validation

Engineering

Project

Management

[addedAcquisition Management]

Project PlanningProject Monitoring and Control

Supplier Agreement Management

[becomesSolicitation & Supplier Agreement

Management ?]

Integrated Project Management)

Risk Management

Quantitative Project Management (4)

Organizational Process Focus; Definition

Organizational Training;

Process Performance (4)

Organizational Innovation and Deployment(5)

ProcessManagement

Configuration Management

Process and Product Quality Assurance

Measurement and Analysis

Decision Analysis and Resolution

Causal Analysis and Resolution (5)

Support

Category ProcessArea

CMMI-Dev 1.2 Process Areas Continuous

Representation

(Hi maturity level)

(boldis for Levels 2&3 capability; []

addresses correlation with draft CMMI-Acq )

D f A i iti G id (DAG) [11/04]


19/39

19

djGantzer 5-2-07

Defense Acquisition Guide (DAG) [11/04] Chapter 4 on SE

[Note: DAG/SE

used ISO

15288, EIA-

632, IEEE

1220, and

DAU 2001 SEHandbook as

key

references]


20/39

20

djGantzer 5-2-07

Example Systems Engineering Process Architecture

(SE Processes taken from DAG, Chapter 4)

Tech

Planning

Tech Decision Risk Requirements

Assessment Analysis Management Management

Requirements Development

Logical Analysis Design Solution

Implementation

Verification & Validation Interface Management Configuration & Data Management

Legend:

Managemen

t

Engineering

Support

Transition

Integration

Technical Management Processes:

Technical Support Processes:

Feedback from each

process

Core Technical Processes:

Note: this illustration was adoptedfrom some PI work done for

DISA_by author

SE St d d /M d l M i M t


21/39

21

djGantzer 5-2-07

SE Standards/Models Mapping - ManagementISO-15288 EIA-632 IEEE-1220 CMMI-DEV DAG/SE Chap. 4

Project Planning Planning Planning tech effort, SEintegration, of SE effort

Project Planning; IntegratedPM;

Quantitative PM

Technical Planning;

Interface Mngt.

Project Assessment Assessment Control Project Monitoring &

Control; M&A;

Quantitative PM

Technical Assessment;

Project Control Control Control Project Monitoring &Control

Interface Mngt.

Decision Making Systems Analysis Systems Analysis Measurement & Analysis(M&A); Decision Analysis &Resolution; QuantitativePM; Causal Analysis &Resolution;

Decision Analysis

Risk Mngt. Systems Analysis Systems Analysis Risk Mngt. Risk Mngt.

Configuration

Management (CM)

CM CM;

Int. repository/ datapkg.

CM;

Requirements Mngt.

CM;

Requirements Mngt.

Information Mngt. Control: infodissemination

Control; IntegratedDB/pkg.

Integrated PM;

CM

Technical Data Mngt.

Agreement : Acquisition

& Supply

Acquisition & Supply Supplier Agreement Mngt.;

see also draft CMMI-Acq

See other DAG chapters

Enterprise:

Environment, Life

Cycle, Resource, &

Quality Mngt.

Environment &

Enterprise Support [e.g.,

investment, resource,

process mngt.]

QM;

Product & ProcessImprovement

Organizational processes;

Process & Product QA

See other DAG chapters


22/39

22

djGantzer 5-2-07

SE Standard/Models Mapping - TechnicalISO-15288 EIA-632 IEEE-1220 CMMI-Dev DAG/ Chap. 4

Stakeholder

Requirements

Definition

Requirements

Definition

RequirementsAnalysis

Requirements

Development

Requirements

Development

RequirementsAnalysis

Systems Analysis (SA)Requirements &

Functional Analysis;

SA

Requirements

DevelopmentLogical Analysis

Architectural Design Solution Definition Synthesis; SA;

modeling, spec tree,SBS

Technical Solution Design Solution

ImplementationImplementation;

production

prototyping,fabrication, assembly,production

Technical SolutionImplementation

Integration integrationProduct Integration

Integration

Verification System Verification Functional and Design

Verification; Tech

reviews

Verification Verification;

[+Chap 9 IT&E]

Validation Requirements andEnd Products

Validation

RequirementsValidation; Tech

reviews, test

Validation Validation;

[+Chap, 9 IT&E]

TransitionTransition to Use

Product Integration;

Org. Innovation &Deployment; Org.Training

Transition

Operation;

Maintenance; and

Disposal

field support support operations, support,maintenance,sustainment, disposal

See other DAG

chapters.


23/39

23

djGantzer 5-2-07

Example: Apply practices to TechnicalPlanning

ISO - 15288EIA - 632IEEE - 1220CMMI - Dev

PM BoK&DoD(AT&L)/SSE Tech Planning considerations


24/39

24

djGantzer 5-2-07

ISO 15288 Project Planning activitiesPurpose is to produce and communicate effective and workable project plans

s Identify the project objectives and constraints

s Define the project scope as established in the agreement

s Establish a WBS based on evolving system architecture

s Define and maintain a project schedule based on project objectives and work estimates

s Project achievement criteria for the life cycle stage decision gates, delivery dates and major

dependencies on external inputs or outputs

s

Define the project costs and plan a budgets Establish the structure ofauthorities and responsibilities for project work

s Define the infrastructure and services required by the project

s Plan the acquisition of materials, goods and enabling system services supplied from outside the

project

s Generate and communicate a plan for technical mgmt. of the project, including the reviews

s Define the project measures to be generated and the associated data to be collected, validated

and analyzed

s Generate a project quality plan

Source: Summarized from ISO 15288


25/39

25

djGantzer 5-2-07

EIA-632 Technical Planning

s Process Implementation Strategy stakeholders, applicable docs, process approaches, LC phases,

integration, reporting requirements, implementation

s Technical Effort Definition

Requirement types, db, risk mngt. process metrics, metrics/quality,cost objectives, TPMs, tasks, methods & tools, technology

s Schedule & Organization

Event& calendar based schedules, resources, staffing/disciplines,team/ org structure

s Technical Plans

Engineering, Risk mngt., Tech Review, V &V, other

s Work Directives

Work packages, work authorizations

Source: EIA-632


26/39


27/39

27

djGantzer 5-2-07

CMMI Project PlanningPurpose is to supply and maintain plans that define project activities.

s Establish Estimates

Estimate scope

Establish Estimates ofwork products/attributes

Define life cycle

Determine effort & cost estimates

s Develop Project Plan

Establish budget & schedule

Identify risks

Plan fordata management,

Plan forresources; Needed knowledge & skills

Plan stakeholderinvolvement

Establish the Plan

s Obtain commitment to the Plan

Review plans that affect project

Reconcile work & resource levels

Obtain commitment s Other key process area relationships: Requirements

Development, Monitoring & Control, Supplier Agreement

Mngt. Integrated PM, Risk Mngt., Measurement &

Analysis,

Source: CMMI-Dev


28/39

28

djGantzer 5-2-07

CMMI Generic Practices for Planningprocess area

s

Perform the planning processs Establish & maintain an Org policy for planning process

s Plan the planning process

s Provide resources

s Assign responsibility

s Train people

s Manage configurations

s Identify and involve relevant stakeholders

s Monitor and control the planning process

s

Objectively evaluate adherence to the planning processs Review status with higher level management [PEO?]

s


29/39

29

djGantzer 5-2-07

PM Book of Knowledge (PM-BoK)(Project Management)

s Scope*

s Integration(charter, scope statement, PMP)

s Communication*

s Risk*

s Quality*

s Human Resources*

s Time (definition, sequencing,estimation)

s Cost (estimation, budgeting)

s

Procurement (purchase,acquisition, contracting)

* ApplyPlanning, Execution &

Control to each area

Source: www.PMI.org ; 3rd Edition, 2004

DoD PMBoK Extension (2003):

SE

SW Acquisition

Logistics

T&E

Manufacturing
http://www.pmi.org/http://www.pmi.org/


30/39

30

djGantzer 5-2-07

Planning Process example [INCOSE SE Handbook v3]

DoD/SSE Technical Planning emphasis


31/39

31

djGantzer 5-2-07

DoD/SSE Technical Planning emphasis

s Manage a Comprehensive Set ofRequirements

Define project scope w key stakeholders [FoS, SoS]

Formulate, assess, select the preferred system concept

Develop explicit and testable system/project requirements

Develop a WBS [products & process]

s Resource & Staffing to the Technical Plan

Organize and staff the project team [TA, PM, LSE, IPTs]

Estimate the time and resource requirements [IMS, EVMS]

Develop a project critical path

Develop a project budget

s Develop and Managing Technical Baselines

Identify, manage, and mitigate project risks [technical]

Manage project changes and customer expectations

s Managing Event-based Technical Reviews

s Integrating Tech Planning into overall Program Planning & ManagementContext [IMP/IMS, EVMS, program Risks]

Source: Dinesh Verma; DoD/SSE Workshop

Note: DoD is updating DAG/SE, SEP

Prep Guide, and DoDI 5000.2


32/39

32

djGantzer 5-2-07

and in summary

s ISO - 15288 is becoming a SE process reference model

IEEE - 1220 updated per ISO15288; IEEE adopted the 15288 w elaboration; furtherupdates planned

EIA - 632 is being harmonized to the ISO 15288 & CMMI

CMMI used EIA 632, 731 as sources; next version will also look at 15288 and 1220

ISO 12207 (SW Engineering processes) is being harmonized with 15288; additionally aLife Cycle Management standard being drafted both [in draft review

INCOSE SE Handbook v3.0, 2006 is based on 15288; SE Certification will be based onit; It is to be released as a ISO tech report

Coordination also underway w ISO 9001

s DoD SE Revitalization:

DAG SE now being revised; one area being beefed up is Software; DAU isimplementing a series of SE courses

DoD SoSE Guide being piloted uses the DAG/Chapter 4

CMMI Guide for Acquirers; SE in Acquisition Contracts Guide

NDIA-SE draft Systems Assurance Guide uses ISO 15288 and EIA-632

s IEEE working on an SE guidance & certification (by specialties)

SE S


33/39

33

djGantzer 5-2-07

SE Sources

Links:

s ISO 15288: http://www.15288.com/

s EIA 632: http://www.geia.org/index.asp?bid=552

s IEEE 1220: http://www.software.org/quagmire/descriptions/ieee1220.asp

s SEI-CMMI: http://www.sei.cmu.edu/cmmi/models/models.html#modules

s Systems & Software Consortium: http://www.systemsandsoftware.org/

s DAU-DAG: http://akss.dau.mil/dag/

s DoD(AT&L) SE: http://www.acq.osd.mil/se/index.html

s AF Center for SE; http://www.afit.edu/cse/

s INCOSE Standards TC http://www.incose.org/practice/techactivities/standards.aspx

References:

s SE Standards & Models Compared; J. Lake (SMi) and S. Sheard (SPC), INCOSE 2004

s Evolution of a Standard EIA-632; R. Harwell, INCOSE 2006

s Cross-Cultural Issues Associated with the Application of ISO/IEC Standard; T. Ferris, INCOSE 2006

s INCOSE Insight: Special Feature: Standards in SE, April 2007; see particularly K. Crowder, D.

Kitterman, T. Doran, R. Harwell, and S. Arnoldarticles.
http://www.15288.com/http://www.software.org/quagmire/descriptions/ieee1220.asphttp://www.sei.cmu.edu/cmmi/models/models.htmlhttp://akss.dau.mil/dag/http://www.afit.edu/cse/http://www.afit.edu/cse/http://akss.dau.mil/dag/http://akss.dau.mil/dag/http://akss.dau.mil/dag/http://www.sei.cmu.edu/cmmi/models/models.htmlhttp://www.software.org/quagmire/descriptions/ieee1220.asphttp://www.15288.com/http://www.15288.com/


34/39

34

djGantzer 5-2-07

Backup slides


35/39

35

djGantzer 5-2-07

Some SE Process References

s ISO/IEC 15288: System Engineering System Life Cycle Processes; 2002

s EIA/IS 632: Processes for Engineering a System; 1998 &?

s IEEE 1220: Application and Management of the Systems Engineering

Process;2005

s CMMI Capability Maturity Model Integration for Systems Engineering,

Software Engineering, and Integrated Product and Process Development,

1.2 (2006)

s Defense Acquisition Guide, Chapter 4 - Systems Engineering; Defense

Acquisition University, 2004

s CMMI Acq: Adapting CMMI for Acquisition Organizations: A Preliminary

Report; 2006

s Understanding and Leveraging a Suppliers CMMI Efforts; 2007

s INCOSE Systems Engineering Handbook, v 3, 2006


36/39

36

djGantzer 5-2-07

NDIA Top 5 SE Issues

Lack of awareness of

SE importance

Lack of adequate

qualified resources

Insufficient SE tools

and environments

Inconsistentrequirements definition

Poor initial program

formulation

2003Inconsistent SE

practices across all life

cycle phases

Insufficient quantity

and quality of SE

expertise

Requirements not wellmanaged or translated

Insufficient SE early in

the life cycle

Inadequate tools and

collaborative

environments

2006

Source: NDIA SE Conference 10/06; M. Schaffer DUSD(A&T) SSE

Taking SE to the Next Level


37/39

37

djGantzer 5-2-07

Taking SE to the Next Level

sMy challenge to you: Instilling upfront and continuous technical rigor

Understanding Systems-of-Systems SE

Fielding capability to the warfightersooner

Increasing combat systems fuel efficiency Addressing growing software concerns

sSincere thanks to our partners

Industry, in particular, NDIA

Service and Agency Engineering Staffs

THE WILL TO CHANGE

Source: Honorable James L. Finely, DUSD(A&T), NDIA SE, 10/06


38/39

38

djGantzer 5-2-07

DUSD(A&T) Vision for SE . . .

sCompetencies Impaired

sElevated Stature

sRaised AwarenesssPositive Influence

sDelivered Product Suites

sWorld class leadership

sBroaden Software Engineering andSystem Assurance, expanding to Systems-of- Systems

sResponsive and agile, proactive tochanging customer needs

s

Focused technical assistance, guidance,and workforce education and training

Systems and SoftwareEngineering

Center of Excellence

. . . the Technical Foundation

that Enables Acquisition Excellence

Systems

EngineeringRevitalization



39/39

Systems Engineering

JCDSPGJoint

ConceptsFNAFAA EOA SDD PDTD &

RR

MS CMS B

FRP DR

CPDCDD

OSD/JCS

COCOM

CD

O&S

MS A

FCB

EOARpt

B1

OSD/JCS COCOMSupport Capability

Based Assessments

Define relationships with

related capabilities,

architectures (e.g., GIG)

Identify alternatives;

trade cost, sched, perf

Identify incremental,

system specifications

Determine system

performance parameters

and verification plans

Develop, test, and assess

increments of capability

Demonstrate capabilities

meet user needs

Assess system

performance against

capability needs

Assess portfolio

performance (CAR)

Integrate SoS;

assess cost, sched, perf

Integrate and test

Soundtechnicalplanning

to supportwarfightin

gcapabilitie

s

ComponentsComponents

EnterpriseEnterprise

SoSSoS

SystemSystem

OperationalOperational

Fully integrated SE approach:Fully integrated SE approach:

technical maturity, costtechnical maturity, cost

realism, risk mitigationrealism, risk mitigation

SE is the technical foundationSE is the technical foundation

for building acquisitionfor building acquisition

knowledge over timeknowledge over time


presentation from may 15, 2007 dinner meeting

Documents