May 12, 2009May 12, 2009Annual EPA QA ConferenceAnnual EPA QA Conference

San Antonio, TexasSan Antonio, Texas

LEADING LEADING CHANGECHANGE

Kenna YarbroughKenna YarbroughUSEPA/NEICUSEPA/NEICDenver, CODenver, CO(303) 462-9256 yarbrough.kenna@epa.gov(303) 462-9256 yarbrough.kenna@epa.gov

ELEVATING ENVIRONMENTAL ELEVATING ENVIRONMENTAL FIELD SAMPLING AND FIELD SAMPLING AND MEASUREMENT ACTIVITIES MEASUREMENT ACTIVITIES TO A HIGHER LEVELTO A HIGHER LEVEL

Leading Change…….Leading Change…….Presentation OverviewPresentation Overview

33rdrd Party Accreditation Party Accreditation Advanced/Alternate Concepts Advanced/Alternate Concepts

for Field Environmental for Field Environmental ProgramsPrograms

Other Program Advancement Other Program Advancement OptionsOptions

12

3

33rdrd Party Accreditation Party AccreditationFor Field ActivitiesFor Field Activities

Based on ISO Standard 17025 - Based on ISO Standard 17025 - General Competence In Carrying General Competence In Carrying Out TestsOut Tests((or ISO Standard 17020 - or ISO Standard 17020 - General Criteria General Criteria for the Operation of Various Types of for the Operation of Various Types of Bodies Performing InspectionBodies Performing Inspection))

Similar to ISO 9000 – Quality Std. Similar to ISO 9000 – Quality Std. but Adds but Adds CompetenceCompetence

1

Accreditation at NEIC Accreditation at NEIC (and Region 4 SESD)(and Region 4 SESD)

Accrediting body: Forensic Quality Services Accrediting body: Forensic Quality Services (www.forquality.org)(www.forquality.org)

Encompasses:Encompasses:– Environmental measurement activities Environmental measurement activities – Forensic activitiesForensic activities

Incorporates EPA Quality Manual and Incorporates EPA Quality Manual and ASCLD/LAB* aspectsASCLD/LAB* aspects

NEIC also has NVLAP** accreditation for NEIC also has NVLAP** accreditation for bulk asbestos determinations bulk asbestos determinations

* American Society of Crime Laboratory Directors/Laboratory Accreditation Board [www.ascld.org]* American Society of Crime Laboratory Directors/Laboratory Accreditation Board [www.ascld.org]

* * National Voluntary Laboratory Accreditation Program* * National Voluntary Laboratory Accreditation Program

Accredited EnvironmentalAccredited EnvironmentalMeasurementsMeasurements

Laboratory Laboratory MeasurementsMeasurements

Field SamplingField Sampling Field Field

Measurement Measurement and Monitoringand Monitoring

Breakdown of Accredited Breakdown of Accredited Field SamplingField Sampling Activities Activities (Scope)(Scope) WastewaterWastewater WaterWater

– Surface WaterSurface Water– GroundwaterGroundwater

AsbestosAsbestos Surface (Wipe)Surface (Wipe) SoilSoil Container and TankContainer and Tank AirAir

* Photos from EPA Office of the Administrator (Multi-media Operations and Services – MOSS) – Eric Vance photographer

Breakdown of Accredited Breakdown of Accredited Field Field Measurement and Measurement and

MonitoringMonitoring Activities Activities (Scope)(Scope)

LDAR MonitoringLDAR Monitoring Site Safety Monitoring Site Safety Monitoring

and Screeningand Screening General Physical Field General Physical Field

MeasurementsMeasurements

Further Breakdown of Further Breakdown of Measurement/MonitoriMeasurement/MonitoringngSite Safety Monitoring and Site Safety Monitoring and

ScreeningScreening Detection of Radiation (Ludlum Geiger Detection of Radiation (Ludlum Geiger

Counter)Counter) Niton Field XRFNiton Field XRF OMNI-4000 Portable Gas DetectorOMNI-4000 Portable Gas Detector Photovac 2020 Photoionization Air MonitorPhotovac 2020 Photoionization Air Monitor ToxiRAE Toxic Gas MonitorToxiRAE Toxic Gas Monitor

How We Got Where We How We Got Where We AreAre

11stst - Strongly Encouraged To Be - Strongly Encouraged To Be AccreditedAccredited

Evaluated Accrediting Options (Lab and Field; 3Evaluated Accrediting Options (Lab and Field; 3rdrd Party; Covered By National/International Party; Covered By National/International Requirements For Accrediting Bodies (ISO Guide Requirements For Accrediting Bodies (ISO Guide 58))58))

Once Selected, Measured Where We Were (Gap Once Selected, Measured Where We Were (Gap Analysis)Analysis)

Developed/Improved Procedures and SystemsDeveloped/Improved Procedures and Systems Implemented the Accreditation SystemImplemented the Accreditation System

How We Maintain How We Maintain What We Have?What We Have?

Technical AspectsTechnical Aspects– ProceduresProcedures– EquipmentEquipment– Personnel Training and Personnel Training and

TestingTesting– Data Package Reviews Data Package Reviews

General AspectsGeneral Aspects

How We Maintain What How We Maintain What We HaveWe HaveTechnical AspectsTechnical Aspects

ProceduresProcedures– Implement Review Schedule of Implement Review Schedule of

Current Procedures (Keep Current Procedures (Keep Current)Current)

– Continue To Evaluate New Continue To Evaluate New ActivitiesActivities

How We Maintain What We How We Maintain What We HaveHaveTechnical Aspects (con’t.)Technical Aspects (con’t.)

EquipmentEquipment– Calibrate/Maintain/Manage Equipment Calibrate/Maintain/Manage Equipment

per SOPs and Instrument Manualsper SOPs and Instrument Manuals– Track Equipment Management with Track Equipment Management with

Equipment Logbooks and an Equipment Logbooks and an Equipment Database Equipment Database (database not (database not required)required)

– Manage and Track Manage and Track Standards/Reagents Used in BranchStandards/Reagents Used in Branch Check standards/gases we receiveCheck standards/gases we receive

How We Maintain What We How We Maintain What We HaveHaveTechnical Aspects (con’t.)Technical Aspects (con’t.)

Personnel Training and Personnel Training and TestingTesting– Competency EvaluationsCompetency Evaluations– Proficiency TestingProficiency Testing– DatabaseDatabase

How We Maintain What We How We Maintain What We HaveHaveTechnical Aspects (con’t.)Technical Aspects (con’t.)

Data Package ReviewsData Package Reviews Performed for all Performed for all fieldfield

analysis presented in reports analysis presented in reports or used in EPA decisionsor used in EPA decisions

Review documentedReview documented Resolution of all issues prior Resolution of all issues prior

to issuance of datato issuance of data

How We Maintain What We How We Maintain What We HaveHaveGeneral AspectsGeneral Aspects Document ControlDocument Control Records ManagementRecords Management PurchasingPurchasing Internal AuditsInternal Audits Perform Corrective and Preventative Actions, as Perform Corrective and Preventative Actions, as

NeededNeeded Evaluate Civil and Criminal TestimonyEvaluate Civil and Criminal Testimony NEW – Determine Measurement Uncertainty of NEW – Determine Measurement Uncertainty of

Our Field Measurements such as TVA and pH dataOur Field Measurements such as TVA and pH data

Confirms Our Commitment to Confirms Our Commitment to QualityQuality

Better Trained and Capable Better Trained and Capable Employees That Can Employees That Can Demonstrate Their ProficiencyDemonstrate Their Proficiency

Better Consistency with Better Consistency with Application of Sampling and Application of Sampling and Measurement TechniquesMeasurement Techniques

How We Feel How We Feel Accreditation Helps Accreditation Helps UsUs

Helps assure legal community/public that Helps assure legal community/public that NEIC capable of generating reliable and NEIC capable of generating reliable and legally defensible datalegally defensible data

Demonstrates that NEIC has taken a Demonstrates that NEIC has taken a recognized and systematic approach to recognized and systematic approach to planning, conducting, documenting, and planning, conducting, documenting, and assessing our environmental data assessing our environmental data collection activities (plan, do, check, collection activities (plan, do, check, act….)act….)

How We Feel How We Feel Accreditation Helps Us Accreditation Helps Us (con’t.)(con’t.)

Advanced/Alternate Advanced/Alternate Concepts for Field Concepts for Field Environmental ProgramsEnvironmental Programs1) Develop and implement 1) Develop and implement fieldfield

measurements and sampling measurements and sampling

competency and proficiency competency and proficiency testing programtesting program

2) Evaluating and reporting 2) Evaluating and reporting

measurement uncertainty for measurement uncertainty for field measurementsfield measurements (and (and sampling?)sampling?)

2

Competency/Proficiency Competency/Proficiency with Field with Field Measurements/SamplingMeasurements/Sampling To qualify staff to perform workTo qualify staff to perform work Thoroughly train staffThoroughly train staff

– Review operating procedure/instrument Review operating procedure/instrument manualsmanuals

– Demonstrate in simulated situations and Demonstrate in simulated situations and On-the-JobOn-the-Job

Competency evaluated by NEIC Competency evaluated by NEIC subject matter expert(s)subject matter expert(s)

Competency evaluation documented Competency evaluation documented and supervisor and supervisor authorizesauthorizes to perform to perform specific workspecific work

Proficiency TestingProficiency Testing

Annually - internally check staff Annually - internally check staff proficiency - demonstrate ability to proficiency - demonstrate ability to perform specific sampling and/or perform specific sampling and/or measurementsmeasurements

Not all – every yearNot all – every year Perform limited external PTs for Perform limited external PTs for

measurements (for areas such as pH, measurements (for areas such as pH, conductivity, total VOCs)conductivity, total VOCs)

Document the testingDocument the testing

Evaluating and Reporting Evaluating and Reporting Measurement Uncertainty Measurement Uncertainty for for FieldField Measurements Measurements Take data from comp/prof tests (and Take data from comp/prof tests (and

instrument precision tests required by instrument precision tests required by Regs) or calibration dataRegs) or calibration data

Evaluate data to assure results within Evaluate data to assure results within NEIC’s and instrument’s specificationsNEIC’s and instrument’s specifications

Can sort by instrument, person tested, Can sort by instrument, person tested, date, etc.date, etc.

Develop MU statement from this data Develop MU statement from this data and/or instrument specsand/or instrument specs

TVA Data ExampleTVA Data Example

Used Statgraphics to analyze data from Used Statgraphics to analyze data from testing testing andand calibration (130 and 620 calibration (130 and 620 points)points)

Individual TVA results well within that Individual TVA results well within that allowed by particular regulation we use allowed by particular regulation we use instrument for (+/- 10%) and the instrument for (+/- 10%) and the instrument stated precision (+/- 25%) – instrument stated precision (+/- 25%) – we found within 3.7% for testingwe found within 3.7% for testing

Report MU calculations in our reports (if Report MU calculations in our reports (if customer wants) – have in our project customer wants) – have in our project recordsrecords

Box-and-Whisker Plot

Pre

c Te

st R

esul

t

0

1

2

3

4

Histogram

Prec Test Result

freq

uenc

y

0 1 2 3 4 50

10

20

30

40

50

Density Trace

Prec Test Result

dens

ity

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

0.6

Various Illustrations of TVA Data

Measurement Measurement UncertaintyUncertaintyTVA Organic DataTVA Organic Data Summary StatisticsSummary Statistics Count = 620Count = 620 Average = 100.302Average = 100.302 Median = 100.0Median = 100.0 Mode = 100.0Mode = 100.0 Standard deviation = Standard deviation =

1.448521.44852 Minimum = 95.0Minimum = 95.0 Maximum = 120.0Maximum = 120.0 Range = 25.0Range = 25.0 Stnd. skewness = 49.0484Stnd. skewness = 49.0484 Stnd. kurtosis = 298.958Stnd. kurtosis = 298.958 Coeff. of variation = Coeff. of variation =

1.44416%1.44416%

Box-and-Whisker Plot

Per

cent

of

know

n va

lue

95

100

105

110

115

120

Sampling Sampling Measurement Measurement Uncertainty???Uncertainty??? Currently collecting analytical data from Currently collecting analytical data from

our triplicate samples into a databaseour triplicate samples into a database Collect information on sampling Collect information on sampling

methodology, sample media (liquid, methodology, sample media (liquid, solid, gas, etc.), sampling tools used, solid, gas, etc.), sampling tools used, sampler name, dates, analysis, etc.sampler name, dates, analysis, etc.

Looking for trends in the data, areas for Looking for trends in the data, areas for improvement, tentative MU calculationsimprovement, tentative MU calculations

No MU calculations reported……..yetNo MU calculations reported……..yet

Other Program Other Program Advancement OptionsAdvancement Options

EPA’s RS&T Director’s Field EPA’s RS&T Director’s Field Operations Group (FOG)Operations Group (FOG)

FOG’s mission….. FOG’s mission….. Adopt five FOG Infrastructure Adopt five FOG Infrastructure

Standards that concern Field Standards that concern Field ActivitiesActivities

Buy-in from all Regions – applied in Buy-in from all Regions – applied in Regions 2 and 4 – commitment from Regions 2 and 4 – commitment from Regions 5, 6, 7, 9Regions 5, 6, 7, 9

3

5 FOG Infrastructure 5 FOG Infrastructure Standards – Relate Standards – Relate to….to…. Evidence ManagementEvidence Management Document ControlDocument Control Records ManagementRecords Management Personnel Training and Records of Personnel Training and Records of

TrainingTraining Equipment and Instrument Equipment and Instrument

InventoryInventory

Another OptionAnother Option

Adopt and/or seek eventual Adopt and/or seek eventual certification through TNI (The certification through TNI (The NELAC Institute) of field NELAC Institute) of field measurements and samplingmeasurements and sampling– Primarily for environmental govt. and Primarily for environmental govt. and

other organization’s labs but writing other organization’s labs but writing some standards for field activities now some standards for field activities now

– Have Drafted “Have Drafted “General Requirements General Requirements For Field Sampling and Measurement For Field Sampling and Measurement OrganizationsOrganizations” Document (2007)” Document (2007)

Many options to go down the road Many options to go down the road of continuous improvementof continuous improvement

Contact me w/ any questions about the Field Ops Group 5 infrastructure standards, becoming ISO accredited, or any other topic discussion items in this presentation!

Kenna Yarbrough, Field Branch QA Representative/InspectorUSEPA/NEIC, Denver, CO (303) 462-9256 yarbrough.kenna@epa.gov