Product Assurance Revitalization

Presented By:

John DeasProduct Assurance Division

Logistics FIPTJuly 14, 2006

2

Agenda• Background

• Policy & Guidance

• Tools & Initiatives

• Program Support/Systemic Analysis

• Education & Training Initiatives

• Conclusion

3

Overview

• Mission – DCMA Provides Customer-Focused

Acquisition Life Cycle and Combat Support to Ensure

Readiness, Worldwide 24/7

• Represents the Military Services, other Federal

Agencies and Allied Government Buying Agencies at

Defense Contractors Worldwide• Prior to contract award

• After contract award

Agency Goals

1. Transform the Agency into a customer-focused organization

2. Embrace a performance-based culture

3. Equip our employees with the competencies needed to provide extraordinary support to our customers

4. Ensure fiscal responsibility

Agency Goals

1. Transform the Agency into a customer-focused organization

2. Embrace a performance-based culture

3. Equip our employees with the competencies needed to provide extraordinary support to our customers

4. Ensure fiscal responsibility

4

The Way We WereTotal 44 CONUS Major Contract Management Offices (CMO)Total 44 CONUS Major Contract Management Offices (CMO)

DCMA West (21)

DCMAEast (23)

CMOsDistrict HQAgency HQ

DCMA HQ

DCMAE HQ

DCMAW HQ

5

Product Alignment

• Product Divisions:

• Aeronautical Systems - BostonAeronautical Systems - Boston

• Naval Sea Systems - BostonNaval Sea Systems - Boston

• Space and Missile Systems- CarsonSpace and Missile Systems- Carson

• Ground Systems and Munitions - CarsonGround Systems and Munitions - Carson

• Sustainment activities align along product divisions

Space & Missile Division

Ground Systems & Munitions Division

Aeronautical Division

Naval Sea Systems Operations

Defense Supply Centers: Columbus (Maritime

Supply Chain & Ground Supply Chain) &

Philadelphia (Troops Support Supply Chain)

Defense Supply Center: Richmond (Aviation

Supply Chain)

Inventory Control Point: Marine Corps Logistics

Center, Albany

Inventory Control Point: Philadelphia &

Mechanicsburg

Air Logistics Centers: Ogden, Warner-Robins, &

Oklahoma City

6

Key Technical Capabilities Key Technical Capabilities

• Product Assurance (PA)• Enterprise process that provides confidence to our customers

• Used to ensure that program/contract technical requirements are fulfilled

• Focuses on those supplier requirements that poses the greatest risk – Value added oversight

• Early adoption of Performance Based Management

• Supplier Management• Prime Contractor Control of Subcontractors

• Subcontract delegations

• Earned Value Management (EVM)• Ensure contractors use EVM as a management tool instead of

a reporting requirement

7

PA Policy & Guidance PA Policy & Guidance

• PA Policy & Guidance approved for Agency use• Uses a customer outcome-based approach

• Outcomes are translated to DCMA performance measures

• Clearly defined Agency role in different acquisition phases

• System Development & Demonstration

• Production & Deployment

• Sustainment

• Covers Systems Engineering (to include Software Acquisition

Management, Manufacturing & Production), Quality

Assurance, etc.

• Provides an integrated approach to oversight

• Utilizes all technical personnel to get deeper insight

8

Customer Outcomes Customer Outcomes

• Understanding customer outcomes allows us to:• Focus on what the customer needs

• Develop surveillance strategy based on customer priorities

• Fine tune technical expertise in areas most important for

meeting customer requirements

• In the Program Managed environment:• Outcomes are determined through continuing dialog with

Program Management Offices

• Customer outcome Product Assurance support meetings

• PA personnel are a proactive partner in this dialog by

understanding the:

• Contract/technical requirements/deliverables

• Supplier’s performance history and capabilities

9

Program Managed PA

• PA strategy focuses surveillance on:• Areas of moderate and high risk to customer outcomes

• Supplier’s systems – System Audits• Supplier’s processes – Process Reviews• Product characteristics – Product examination

• Criteria for Technical Reviews • Identify level of oversight/mix of personnel• CMMI based Risk Management (CBRM) methodology

• Determine process capability• Based on continuous version of the model

• Predict programmatic outcomes based on process capability, EV data, Technical Performance Measures (TPMs), Technology Readiness Levels (TRLs)

LRIP FRP

Production and DeploymentSystem Development

& Demonstration

System Integration

System Demonstration

10

Program Managed PA

Formulate PA Surveillance Strategy(Interdisciplinary Teaming Approach)

- Systems, Processes, and Characteristics Requiring Monitoring to Ensure customer outcomes are achieved

- Frequency and Intensity Dependent Upon Risk to Customer Outcome

Program Managed Contract

Customer Outcome PA Support Meeting

- Customer Outcomes- Outcome Performance Measures

- Risk Information Effecting the Outcomes

Risk Assessment

- Identification & Analysis of System,Process,and Product Characteristic

Posing Risks to Achieving Outcomes

Process Review

- Initial Reviews SDD to FRP - Recurring Surveillance of Product

Realization Planning Process and Control of Purchased Products

- Other Processes as Risk Warrant - Focused on Process Outputs / Products- Independent Where Possible- Reverse Trail into Systems Audit / Review

System Audit

- Complete QMS (Formal Audit) During SDD, ~3 Year Intervals, or Customer Request

- Recurring Surveillance of System Elements as Risk Warrant

- Product Realization Planning Process and Control of Purchased Products

- Independent Audits

Product Examination

- Inspect- Witness- Test- Verification ( Using Objective Evidence)- Analysis of Supplier Data / Information- Independent Where Possible- Reverse Trail into Process Review

- Document Surveillance Strategy Plan - Perform Surveillance and Document Results- Analyze Surveillance Data- Issue Corrective Action Requests as Necessary- Adjust Strategy based on Analysis of Surveillance

Results and Changes to Customer Outcomes

PDOS W/G

11

DCMA Use of CMMI DCMA Use of CMMI

• Not the “typical” CMMI user• Not seeking a benchmark “Maturity Level”

• Tool to aid program surveillance• Risk identification

• Prioritize activities in support of customer outcomes

• Helps determine contractor performance strengths and

weaknesses

• Impact on the program and customer outcomes

• Determine whether the supplier is maintaining

processes as stated

• Integrated surveillance for Engineering, Software &

Quality Assurance

12

• Contract administration• Contracting functions (other than production surveillance) are

not diminished by performance-based contracts

• For system-level PBL contracts, DCMA provides award fee

board support by:

• ensuring management controls are effective

• Monitoring of performance requirements

• Verification of compliance with Terms & Conditions

• cost savings are being realized

• Payment through WAWF facilitates rapid processing

• DCMA offers WAWF training and support to contractors

• Standardized SOW improves PALT and administration

DCMA PBL SupportDCMA PBL Support

13

• Pricing / business case analysis (BCA)

• DCMA supports by providing input to

• Cost-benefit analysis

• Price negotiation

• Technical evaluation of costs

• Evaluation of capital investments to increase productivity

• Risk assessment at contractor facilities

• DCMA evaluation of cost reasonableness requires specific

requirements established by contract or MOA

DCMA PBL SupportDCMA PBL Support

14

• Product Assurance

• It is essential that PBL contractors have effective quality management

systems

• Enables minimal oversight

• Allow for alternate release procedures

• DCMA assures that contractor quality systems meet requirements

• Normally, DCMA Product Assurance monitors the effectiveness of a contractor’s

quality system through independent analysis of data

• Quality system audits are conducted by specific request

• Credible third-party certification should be relied upon unless evidence exists

indicating that the system is inadequate

• CM becomes a critical process as responsibility is transferred to the

contractor

• Increased surveillance of contractor CM processes is essential

• Production surveillance is limited to verification of performance

DCMA PBL SupportDCMA PBL Support

15

• Property administration• Essential for PBL contractors to have effective property

management systems

• Ideally, establish a rotable pool of spares in contractor custody

• Monitoring of GFP Induction for repairs distinct from recording

receipt in Government property management system

• NAVICP CAV system provides data points for reduction in RTAT

metrics

• PHS&T• Ensure contractor does not use least cost methods for

shipment of high value items

DCMA PBL SupportDCMA PBL Support

16

CMMI Implementation CMMI Implementation

• More prevalent at sites that have a CMMI contract

requirement

• 20 + sites have been trained

• Training includes:• One day of model training

• One day of CMMI Based Risk Management (CBRM) method

with case studies

• ½ day of tool training

• Maintain a core group of trained personnel to support

SCAMPIs

• Full voting member of DoD/NDIA CMMI Steering Group

17

Software Intensive Programs Software Intensive Programs

• New IPT chartered by the Acting Agency Director • Final recommendations due on July 20, 2006

• Looking very closely how the Agency conducts

Program Level Analysis/Program Level Oversight

• Areas of review include:• Integrated Master Plan/Integrated Master Schedule (IMP/IMS)

• Basis of Estimate (BoE)

• Contractor’s Latest Revised Estimate (LRE)

• Independent Estimate At Completion (IEAC)

• Competencies the Agency needs especially with regard to

Software Cost Estimating

18

Education & Training InitiativesEducation & Training Initiatives• Intern and Journeymen Training

• ISO9001/AS9100 Quality Management Systems Auditing Training

• Software Professional Development Program (SPDP)

• Professional Society Certification• INCOSE Systems Engineering

• ASQ Certified Quality Engineer (CQE)

• ASQ Certified Software Quality Engineer (CSQE)

• APICS Certified – Production & Inventory Management (CPIM)

• Tuition Reimbursement

• Sabbatical Programs

• Specialized training from outside sources

19

Other Support ActivitiesOther Support Activities

• Support to Program Status Reviews (PSRs)• Completed/Scheduled this year - 10

• Seven more reviews are being scheduled

• Areas normally covered include:• Earned Value Management

• Manufacturing & Production

• Software Acquisition Management

• Visits to DoD components• Understand priorities, concerns and expectations

• Support to Air Force Acquisition Transformation Advisory Council (ATAC)• Probability of Program Success

• Will be involved in the pilot activities

20

Conclusion

• Product Assurance encompasses all technical functions in DCMA

• Product Assurance focus on customer outcomes will improve the

quality of service provided

• Resource realignment towards higher risk requirements

• Implementation of systems engineering activities continuing across

the Agency

• Professional, mature workforce knowledgeable and embedded in

the supplier base

• Timely acceptance of products and services meeting customer

expectations is the key