Eric FassettJuly 26, 2011

BackgroundDoD prescribed a Single Sampling Plan (SSP) in DAR Supp.

3 Annex II in October 1975 that was replaced by the DoD Property Manual (4161) Double Sampling Plan (DSP) in December 1991

The FAR revisi0n of June 2007 required that contractors shall manage property consistent with voluntary consensus standards/industry leading practices and have a process to report self assessments

Many contractors adopted metrics as replacements for traditional 4161 type assessments

DCMA has questioned the same statistical validity of these self assessments against the 4161 provisions of “90% confidence of rejecting lots with 10% or more defects” DAU AAP 9/22/2010

Background (Cont.)Normally using a sampling plan, the Government's risk

shall not exceed 10% (a 90% confidence level) excepting slight variations due to changes in population sizes.” DCMA Guidebook

DL1.1.111. RISK. A measure of those instances where a sample will not be an accurate representation of the lot. For example, a sampling plan having a 90 percent confidence level means that a sample will not be an accurate representation of the lot 10 out of 100 times. DoD Manual 4161.2-M

DLA Research Report “Statistical Sampling for Property Management” represents a 90% confidence of rejecting lots with 10% or more defects

Issues with 4161 DSP Methodology The sample sizes from the 4161 Double Sampling Plan

(DSP) do not coincide with Table I of Mil Std 105E (e.g. Lot sizes 51-90, 91-150, and 10001-35000 are the only matches)

The Acceptance Quality Limit (AQL) from the Mil Std 105E for Double Sampling (Normal, Reduced, or Tightened) do not coincide with the 4161 DSP defect criteria. The 4161 DSP actual AQL runs between 1.5 and 2.5%

The DSP is based on DLA Research Report “Statistical Sampling for Property Management” DLA-XX-P20258, August 1992

MIL-STD-105E is no longer supported by the Government

Issues with 4161 DSP Methodology (Cont.)ASTM has incorporated it into E2234 via the E11

committee on Quality and StatisticsMet with E11 Vice-chairman Dean Neubauer who is co-

author of “Acceptance Sampling in Quality Control”He stated “The sampling plan that you sent is not a

military sampling plan I have ever seen.  For example, the lot ranges you show are not what is given in the sample size code letter table for General Inspection Level II (the default level to use) in Mil-Std-105E.”

Confidence level is not really the issue as much as using an accepted sampling plan

Key ConceptsConfidence level - means that the sample will be an

accurate representation of the lotProbability of acceptance (rejection) - percentage of

inspection lots estimated to be of acceptable quality when subjected to a specific sampling routine

Process average - the average percent of defective or average number of defects per hundred units of product submitted by the supplier for original inspection

OC Curve for 4161 DSP

DSP Simulation Sample Results – 9% Defect Rate

Sampling PlanTypical AQL levels are 10%, 6.5%, 4%, 2.5%, 1.5%We have proposed using the Mil-Std-105E criteria for doing

Contractor Self Assessments and establishing an Acceptance Quality Limit (AQL) of 6.5%

E2234 charts are statistically soundASTM has incorporated Mil-Std-105E into E2234 via the

E11 committee on Quality and StandardsSample rates are similar to what is being currently

performed

Self Assessment Variables to Consider

Organizational Level of Sample (Sector, Division, Site)Sampling Plan (Single, Double)Frequency (Yearly, Quarterly, Monthly)Each sample must stand on it’s own

StatusBriefed DCMA/HQ on 2-18-11They met to evaluate our proposal and a technical follow up

was held 3-15-11 Recognized in most recent DCMA Standard Operating

Procedure (SOP) Contractor’s Internal Assessment or Self Audit of their Property Management System

ASTM E2234 SSP – AQL 6.5%

Lot Size Single Sample Size Accept if Defects are equal to or less than Reject if Defects are equal to or exceed2-3 All 0 14-15 3 0 1

16-50 8 1 251-90 13 2 391-150 20 3 4

151-280 32 5 6281-500 50 7 8501-1200 80 10 11

1201-3200 125 14 153201-up 200 21 22

"Adapted, with permission, from E2234-09 Standard Practice for Sampling a Stream of Product by Attributes Indexed by AQL, copyright ASTM International, 100 Barr Harbor Drive, West Conshohocken, PA 19428."

ASTM E2234 DSP – AQL 6.5%

"Adapted, with permission, from E2234-09 Standard Practice for Sampling a Stream of Product by Attributes Indexed by AQL, copyright ASTM International, 100 Barr Harbor Drive, West Conshohocken, PA 19428."

Lot Size Sample Size 1

Accept if Defects in Sample 1 are equal to or less

than

Reject if Defects in Sample 1 are

equal to or exceed

Continue with Sample 2 if Defects in

Sample 1 are Sample Size 2

Accept if Sum of Defects in

Samples 1 and 2 are equal to or

less than

Reject if Defects in Samples 1 and 2 are equal to or

exceed2-3 All 0 14-15 3 0 1

16-50 5 0 2 1 5 1 251-90 8 0 3 1-2 8 3 491-150 13 1 4 2-3 13 4 5

151-280 20 2 5 3-4 20 6 7281-500 32 3 7 4-6 32 8 9501-1200 50 5 9 6-8 50 12 13

1201-3200 80 7 11 8-10 80 18 193201-up 125 11 16 12-15 125 26 27