ntsa engineering forum revitalizing systems engineering 21 july 2004 mark adducchio director of...

Ohio ChapterOhio Chapter

NTSA Engineering ForumRevitalizing Systems Engineering

21 July 2004

Mark AdducchioDirector of Engineering (acting)

USAF Training Systems Product Group

[email protected]

(937) 255-7388 x3257

mailto:[email protected]



• Renewed interest in Systems Engineering from AF leadership

• Some policy & guidance rolling in from both OSD & SecAF - pending within ASC and the TSPG

• TSPG & Industry should go forward single-mindedly with what works, what’s useful, and what’s reasonable



• Objectives• Gain a clear idea on ways to apply

systems engineering initiatives to training systems developments

• How best to formulate future training systems acquisitions to reflect these initiatives

• How to stimulate good systems engineering processes that directly relate to delivering the product



• TSA• Contractor SE capabilities/practices evaluated during

source selection• Task orders assume SE in place

• No specific SE requirements• SE assessment of task order proposals integrated into

IMP/IMS evaluation

• For individual orders, contractor determines scope of SE effort

• Typically risk management, reviews, IMP/IMS, Req’t tracking

• SE Incentives schedule milestones

• Nothing precludes additional SE requirements from task orders



• Discrete Programs (non-TSA)• Treat SE as needed and/or per prevailing

policy/climate• SE requirements thoroughly defined on

some programs C-17 MTS• Some dictated by parent program F/A-22



• How is SE managed in the TSPG?• Program specific• Reliant on contractor emphasis & practices

• Individual experience• Sometimes personality driven

• Usually tied to the IMP/IMS• Few institutional guidelines

• IRM, DAU guides and courses• OSS&E

• Gov’t specific indicators based in Tech Health charts



• Technical Health Charts

• Subjective assessment on complexity and progress

• Risk items

• Issues

• TD count


TrainingGov’t DRs Total Critical

Written:

Closed:

Open:

Ready:

Technical Issues

Technical ReviewProgram:

Contractor:Engineer:

As of Date:

SystemComplexity

System ofSystems

SingleElement

DesignDistribution

FragmentedMulti-Site

SingleSite

Vendor/SubManagement

LittleControl

TightControl

Resources OptimumInadequate

GovernmentTechnical Staff

AdequateNotAdequate

DevelopmentAssessment

ExceedsProgramPlan

SignificantImpact on

Program

RequirementsLooselyDefined

Creeping

FirmDocumentedUndisputed

TechnicalData Quality HighLow

LightInsight

HeavyOversight

Technical Access

Oversight/Insight

CompleteNone

Engineer’s Assessment PMR

Last Qtr Current Qtr Next Qtr

Government Status

UserCommunication

ActivelyInvolved

No Dialog

Contract Status

SpecificationCompliance

Meet OrExceed

KPPs WillNot Be Met

Enough Money

Not EnoughMoney

Schedule Impact

Financial Impact

Lots ofTime

Very LateTo Need

Performance

Processes

DisciplineMatureUnproven

Undisciplined Steadfast

OSS&E Level Level 6Level 1

Critical MilestonesLast:

Date:

Criteria Status:

Next:

Est. Date:

Contract Type: Baseline:

Technical

Review

Chart

3 Mar 2004

• Control Loading/Motion Technology

• Facilities Issue- -- Schedule delays due to weather and facility

contract management

• Virtual Flag 04-03

G

ENG

SYSENG SW CM

SW SYS CW

TECHSTAFF

GG

FFP, CPFF Sys Spec

273 None

158 None

88 None

27 None

WST RFT

19Dec03

Complete

WST ATP Start

March 2004

SUBS&DATA

G

DRs

HW

TESTCM/DM



• Current Issues • Obscure requirements & requirements drill-down• Milestone preparation – design review criteria &

expectations• Poorly defined HSI completion criteria – premature

testing SE staying power• Risk identification between contractor & Gov’t

diverge• CM failures• Integration of engineering activities w/

management controls & indicators



• Renewed emphasis on SE processes• Risk Management• Engineering/developmental milestones• Incentives• More Gov’t guidance• Likely no new standards (i.e. MIL-STD-

499B)• CMMI?



• Establish more robust SE policy & guidance within TSPG• Consistent application of SE practices

within TSPG• ASC policies & guidance• Operating instructions• Training

• TSA III emphasis• TSA II Task orders



• How Do We Apply To Our Programs• Contractor defined milestones?• SEP• Leading Indicators/Metrics• Incentives (not disincentives)• Technical Baseline

• Content• Control

• Robust Product Design



• Transitioning to the SE Plan• Provides vehicle for program

standardization• Map expected technical progress• Report technical indicators

• IMP/IMS criteria; integration milestones; test results

• Integrate these w/ incentives

• How do we normalize this with contractor milestones & indicators?



• 3 Discussion Groups• Proposal guidance for effective SE• Policy Implementation • Technical management & leading indicators

• How can industry help this “revitalization”?• Speak up• Put up• Ante up



QUESTIONS?

ntsa engineering forum revitalizing systems engineering 21 july 2004 mark adducchio director of...

Documents