[email protected], [email protected] big u.s. government software projects practical...

[email protected], [email protected] 1

Big U.S. Government Software Big U.S. Government Software ProjectsProjects

Practical Knowledge For ThosePractical Knowledge For ThoseWho Actually Have To Do ItWho Actually Have To Do It

Ralph [email protected]

Alasadar Mullarney

[email protected]

Ralph NebikerRalph Nebiker


AgendaAgenda

IntroductionIntroduction Tough EnvironmentTough Environment The 1The 1stst Executable is Crucial Executable is Crucial Making Change Easy - Making Change Easy - CMMICMMI Out Of Control IndicatorsOut Of Control Indicators Requirements VolatilityRequirements Volatility DocumentationDocumentation Automated DevelopmentAutomated Development SummarySummary


FocusFocus

Practical knowledge for Practical knowledge for front line engineersfront line engineers

andandfront line team leaders.front line team leaders.

IntroductionIntroduction


Tough EnvironmentTough Environment

1995 Cost of software: $35.7 Billion

Confirms a similar 1979 study.

Continues today.

Software Project Outcomes

29%

46%

20%3% 2%

Paid for but neverdelivered

Delivered but neverused

Used or abandoned afterextensive rework

Used after changes

Used as delivered



Troubled Program Performance Issues

48

61

748387

91

264343

52

70

87

Occurrance (% )

Process Capability

Organizational Management

Requirements Management

Product Testing

Program Planning

Product Quality - Rework

System Engineering

Process Adherence

Program Schedule

Interoperability

Decision Making

Configuration Management



Process ElementsProcess Elements

PerformancePerformance

Adherence CapabilityAdherence Capability


“Our analysis showed that nine out of every 10 DoD programs that were assessed exhibited process performance shortfalls - program teams were unable to specify, design, integrate, or execute development processes that met the specific needs of their unique programs.”

“Given the increase in technical and management complexity of future DoD programs, and the trend toward massive systems of systems, our analysis projects that this process-related performance gap will widen.”

Their answer: Educate, motivate, and do better.



After 3+ years the software engineering team had

~500K SLOC that was never compiled or executed.

Why? A core service domain (motion) was never provided by the vendor responsible. The software engineering team never developed their own motion domain for development and test.

Four months after restart we had our first executable, from scratch.

How? We developed our own motion domain to drive our models for development and test.

Lesson: Don’t depend on another vendor to bail you out. The first executable is crucial in establishing a successful production process.

The 1The 1stst Executable is Crucial Executable is Crucial


Making Change EasyMaking Change Easy

Automated software development is a BIG change for the government.

Change management and working with staffs slow to accept change is the biggest problem.

It took two years before our software engineering team could stand alone and we learned how to “abstract well.”

In that two year period we needed additional outside help - John Wolfe and Jeff Thompson.

Even after three years there were in our organization those who still didn’t understand our processes and inhibited forward motion.


Making Change EasyMaking Change Easy

We continuously updated and improved our IPT processes in the four years we existed - CMMI.

Updating and improving our processes was an ongoing struggle. We had to “adapt.”

Making change work - Making change easy:- Leadership provide the vision, scope, and specific goals.- Leadership obtain “consent of the governed.” Those who

will use a new process design the new process.- Getting from A to B made in several small steps, not one

big step.- Never formalize a change before its been tried and tested

for two to six weeks.- Don’t constrain the change process.


The Ugly And The GoodThe Ugly And The Good

The UglyThe Ugly

ExpertSupport

Staff

ExpertProduction

StaffChange

Designthe change

Imposethe change

Project Leadership

The GoodThe Good Project Leadership

ExpertSupport

Staff

ExpertProduction

Staff

Vision, ScopeSpecific Goals

Designthe change

Change

Implement change

the

“Consent ofthe governed.”


Out Of Control IndicatorsOut Of Control Indicators

A B

CD

A B

DC

A B

DC

Step 1 - The System is divided up to build and deliver.

Step 2 - Each developer defines what he is sure he can build and deliver.

Step 3 - Each developer defines his interface with the adjacent components.

Step 4 - Being smart, the developers can see the development of problem areas and redefine what they are going to deliver.

Step 5 - The Prime becomes responsible for delivering, excluding (of course) the out-of-scope functionality, a significant portion of the System.

Step 6 - Lash together what's finally delivered and try to make it work

A B

C DE

A B

C DE

FE

The System

A B

C D


Inability to quantify system impacts of design alternatives or proposed changes.

Continuous Revising of the System Specification (basically to match what is being delivered by Component Developers).

Appearance of “new” components. Redefinition (reduction, never expansion) of

the Scope of the System. Late discovery (especially during integration

tests) of significant design problems.You’re Already Dead

You’re Already Dead

Out Of Control IndicatorsOut Of Control Indicators


Requirements VolatilityRequirements Volatility SPAWAR SC SD: Life cycle strategies: 1) One

pass, 2) Incremental, and 3) Evolutionary.

Only in 3) are requirements refined during development. 1) and 2) = Stable Requirements.

JSIMS: “Just-in-time Requirements.”

JSIMS: “Just-in-time Engineering.”

“Demandingly dynamic environment” doesn’t come close.

Maritime’s Savior: Our Automated Development Environment.

Stable Requirements: NEVER HAPPENS!!!


DocumentationDocumentation “About half of the words created for military

software projects seemed to be due to the very elaborate oversight and status reporting criteria associated with military contract work.”

“Documents are produced to demonstrate contract compliance rather than to add technical content to the project itself.”

“It is not unusual for large defense projects to accumulate roughly 50 percent of total costs in the area of producing and reviewing paper documents.”

One answer: Tools, Tools, Tools and a good documentation plan (e.g. tailoring).


Automated Development EnvironmentAutomated Development Environment

IndividualizedReports

Model DevelopmentBridgePoint

VerifierBridging Files

C++ Hand Code

Compiled & LinkedExecutables

Developer Regression Tests

Test Team Tests

Code GeneratorModel Compiler

Architecture &Tooling

Model CompilerC++ Hand Code

Output LogsAssertion Status

-------------Daily Developer Activity-------------

Newly CompiledDomains

Nightly Status ReportsEmailed to Distribution

Nightly Batch Processing

Daily Feedback LoopDeveloper Review


Automated Tools:

- Pro: Makes work easier.- Pro: Provide automatic metrics.- Pro: Make change implementation easier.- Con: Need engineers to use efficiently.- Con: $$$

“Push” Technology Works“Pull” Technology Doesn’t.- “Push” technology: You get email notice of action items.- “Pull” technology: You have to repeatedly visit web sites

or databases to discover updates.

Use as many eXtreme programming and Agile techniques as you can.

Keep the documentation plan “light” and automate as much as you can.

Automated Development EnvironmentAutomated Development Environment


SummarySummary

The EnvironmentThe Environment The 1The 1stst Executable is Crucial Executable is Crucial Making Change Easy - Making Change Easy - CMMICMMI Out Of Control IndicatorsOut Of Control Indicators Requirements VolatilityRequirements Volatility DocumentationDocumentation Automated DevelopmentAutomated Development CrossTalk - It’s Free!!!CrossTalk - It’s Free!!!

(www.stsc.hill.af.mil/crosstalk)


Questions?Questions?


Backup Backup FreebiesFreebies


Action Item Change ModelAction Item Change ModelOriginator submits

Action Item (AI) form

Gatekeeper

AI Agent

AI Analysis

AI Workforce

Doneon time?

NewVersion in Repository

New VersionNotification

and/orDistribution

AlternateGatekeeper

GatekeeperTracking Agent

(GTA)

Email

Conducts

AcceptsDue DateRefused

Out of Area

Assigns

Assigns

No Yes

Email provides the “push technology” action notification

and record of progress.

Agent

Agent

GTA Closes AI

GTAInforms


Action Item Change ModelAction Item Change Model

Works for all changes not requiring committee approval.- E.g.: Plans, SOPs, Reports, Instructions, WBSs, Tools, etc.- Gatekeeper assignment function is crucial.

Deputies and Assistants make good Gatekeepers. The crux: Responsibility for moving forward can’t be avoided. No agent with responsibility? Gatekeeper can assign to any agent. Originators do not have to figure out to whom an AI must be

submitted, nor monitor its progress. Email feedback is automatic. A project may have one or more Gatekeepers depending on

workload or required breadth of expertise. Fewer is better.

- AI Agents have full authority and responsibility for changes. IPT Leads and staff principals are good AI agents. Agents possess AI domain knowledge. AI assignment is a good way for an IPT Lead to monitor activity. Agents convene stakeholder meetings only if needed. Meetings are

short, episodic, and ad hoc. Agents are responsible for shepherding the change process. The unique requirements for a change is the province of the agent. Agents have assigned alternates to cover periods when they are

unavailable to act as an AI agent.


Action Item Change ModelAction Item Change Model- Email provides the “push technology” action notification and record

of progress. Specific email lists are be developed for the small subset of involved

stakeholders at each transition point. An automated tool such as ClearQuest provides the ability to generate

emails and metrics automatically. Generating emails manually is worth the added effort to achieve “push

technology” and semi-automated tracking and metrics. Originators receive automatic email feedback. Who is added to the email list for each item is determined by the

Gatekeeper and Agent; i.e. the stakeholders.- Gatekeeper Tracking Agent (GTA)

GTA is Cc: on all emails. GTA can be an automated tool. If an individual, the GTA is not the Gatekeeper or the alternate. GTA tracks status of each AI. (ID Number, Short Title, Submit Date, Agent,

Refusal Date, Due Date, Days Overdue, Completion Date, Disposition Notes) GTA provides periodic status to Gatekeepers, Agents, and workforce. Status Report provides AI unique ID Number from which Gatekeeper or

Agent assigns the next number in sequence. A project may have one or more GTAs depending on workload. Fewer is

better.


Action Item Submission FormAction Item Submission FormAI Submission Form

Submitter’s Name: AI ID Number:Submitter’s Phone Number:To: [email protected] (Provided)

[email protected] (Provided)Short Title:Urgency:Importance:Artifact to be Changed or New Idea:Attachment Filenames:Long Description:

Directions1) Submitter does not fill in AI ID Number.2) Fill in and email to Gatekeeper and GTA.3) Provide attachments for longer descriptions and submitted references.4) Short Title: One line maximum length.5) Urgency: Critical: 1-2 Workdays; High: 3-10 Workdays; Medium: 10-20 Workdays;

Low: 20 or more Workdays.6) Importance: High: Fundamental Non-baselined Program Artifact;

Medium: Process/Plan Artifact Update; Low: Supporting Artifact.7) If known, provide specific identification number and version of artifact to be changed.


- Automated tools may contain tracking and in-process information

displayed on the form. This is not needed for manual forms submitted by email. Status is provided by the GTA’s periodic report.

- Don’t confuse urgency with importance. “Priority” is a word and rating to be avoided. Use Urgency and Importance.

- Don’t overload the form. One size NEVER fits all, but a well designed form can fit a number of cases, especially for non-baselined items.

- Everything needed fits on one 7½” by 10” work space, including directions. Longer descriptions and submitted references are provided by attachments.

- Field names are maximally descriptive and self-explanatory. An external reference is not needed to fill out and submit the form.

• Each artifact probably has its own development and update process, which is the responsibility of the AI Agent, and need not be reflected in the AI form.

• If absolutely needed, tailored forms can be provided for different situations. Best method is on a VPO web site if an automated tool in unavailable.

Action Item Submission FormAction Item Submission Form