epa’s wet interlaboratory variability study findings marion kelly usepa office of science &...
Post on 17-Jan-2018
214 Views
Preview:
DESCRIPTION
TRANSCRIPT
EPA’s WET Interlaboratory Variability Study Findings
Marion KellyUSEPA Office of Science & Technology
Office of Water
Office of Water
Interlaboratory Variability Study Design Overview
National Study of WET test methods which demonstrated that the methods were scientifically sound
Test methods -12 of the 17 approved WET test methods evaluated in the study
Laboratories - 56 different laboratories involved, each in an average of 3 methods
• Base study design = 9 EPA-sponsored labs, up to 11 non-EPA-sponsored labs
• Extended study design = additional non-EPA-sponsored labs
Office of Water
Study Design Overview (cont.)
Samples - over 700 samples analyzed
• Participant laboratories each tested 3 or 4 blind samples
• Sample types included: effluents, receiving waters, blanks, and reference toxicants
Objectives - study assessed the following parameters for each of the 12 WET test methods:
• Test completion rate
• False positive rate
• Interlaboratory precision
Office of Water
Peer Review Process
Peer review involvement• Study plan - independently peer reviewed 10/9/1998 - 12/9/1998• Study results and report – independently peer reviewed 1/01 – 4/01
Peer reviewers• Three experts in the field of aquatic toxicology and biometrics• Identity blinded to EPA• Experts not associated with the generation of the WET methods
final rule Peer review charge
• Ensure that study design and results are scientifically acceptable within the context of the intended use
Office of Water
WET Test Methods Evaluated
Acute Freshwater acute
• fathead minnow, Pimephales promelas
• cladoceran, Ceriodaphnia dubia
Marine acute• sheepshead minnow, Cyprinodon
variegatus• inland silverside minnow, Menidia
beryillina• mysid, Holmesimysis costata
Chronic Freshwater chronic
• fathead minnow, Pimephales promelas
• cladoceran, Ceriodaphnia dubia• green algae, Selenastrum
capricornutum
Marine chronic• sheepshead minnow, Cyprinodon
variegatus• inland silverside minnow, Menidia
beryillina• mysid, Mysidopsis bahia • red macroalga, Champia parvula
Office of Water
Participant Laboratory Procurement Solicitation
• 319 labs solicited for participation in the study Prequalification
• All participant laboratories were required to prequalify• Prequalification required documentation of capacity and
capabilities, experience and proficiency, and quality assurance / quality control
• Only 8 laboratories failed to prequalify Selection / level of participation
• Dependent upon bid cost, sponsorship, and random selection• Base study design = 9 EPA sponsored labs, up to 11 non-EPA
sponsored labs• Extended study design = additional non-EPA sponsored labs
Office of Water
Participant Laboratories (freshwater)
Test Method EPA-sponsored Non-EPA-sponsored Total
Base Base Extended
Fathead acute 9 11 9 29
Fathead chronic 9 11 7 27
Ceriodaphnia acute 9 11 8 28
Ceriodaphnia chronic 9 11 15 35
Selenastrum chronic 9 2 0 11
Office of Water
Participant Laboratories (marine)
Test Method EPA-sponsored Non-EPA-sponsored Total
Base Base Extended
Silverside acute 9 0 0 9
Silverside chronic 9 1 0 10
Sheepshead acute 7 0 0 7
Sheepshead chronic 7 0 0 7
Mysidopsis chronic 9 2 0 11
Homsesimysis acute 2 Interlaboratory testing canceled
Champia chronic 1 Interlaboratory testing canceled
Developed Study PlanFunded Study
Managed Study
STUDYPLAN
STUDYRESULTS
SOPsCollected, Prepared, & Shipped Samples
ConductedPreliminary Testing
Procured LabsManaged Day-to-Day ActivitiesTracked SamplesReviewed & Analyzed Study Data
TESTDATA(Over700
tests)
Conducted WET Tests on 3 or 4
Blind Samples
7 to 35Participant Labs
per Method
PARTICIPANTLABS SAMPLES
BlankReference Toxicant
EffluentReceiving Water
REFEREELAB
WETVariability
Study Design
Office of Water
Roles and Responsibilities
EPA• Assemble WET technical workgroup to develop study plan and to
provide technical oversight during the study• Provide overall study management
Sample Control Center (SCC) operated by DynCorp I&ET• Procure referee and participant laboratories• Coordinate and provide day-to-day management of referee and
participant laboratories• Track sample shipment and receipt• Review, validate, and analyze study data
Office of Water
Roles and Responsibilities (cont.)
Referee Laboratories• Conduct preliminary testing to determine the appropriateness of
samples for interlaboratory testing• Collect, prepare, package and ship test samples to participant
laboratories Participant Laboratories
• Conduct WET tests on blind samples during interlaboratory testing
Office of Water
Interlaboratory Variability Study Timeline
Task Date
Settlement agreement signed 7/24/98
Preliminary study plan published 8/24/98
Expert peer review conducted on preliminary study plan 10/9/98 – 12/9/98
Final study plan published 6/11/99
Interlaboratory testing conducted 9/28/99 – 4/4/00
Preliminary study results published 10/10/00
Expert peer review conducted on preliminary study results
1/01 – 4/01
Final study report and proposed rule published 9/28/01
Office of Water
Peer Review Process
Peer review involvement• Study plan - independently peer reviewed 10/9/1998 - 12/9/1998• Study results and report – independently peer reviewed 1/01 -
4/01 Peer reviewers
• Three experts in the field of aquatic toxicology and biometrics• Identity blinded to EPA• Experts not associated with the generation of the WET methods
final rule Peer review charge
• Ensure that study design and results are scientifically acceptable within the context of the intended use
Office of Water
Study QA
Laboratory prequalification - EPA required that laboratories prequalify for participation in the study
Test-specific QC criteria - standard quality control measures for WET testing were required in the study, including test acceptability criteria as stated in the methods manual, reference toxicant testing, and test condition monitoring
Data reporting standards - EPA required that laboratories submit all bench-level data electronically in pre-designed standard reporting templates
Independent result recalculation - EPA independently recalculated all test results from reviewed electronic data
Office of Water
Results - Successful Test Completion Rates
Test Method Test completion rateCeriodaphnia dubia acute 95.2%Ceriodaphnia dubia chronic 82.0%Fathead minnow acute 100%Fathead minnow chronic 98.0%Selenastrum capricornutum chronic (with EDTA) 63.6%Mysidopsis bahia chronic 97.7%Sheepshead minnow acute 100%Sheepshead minnow chronic 100%Inland Silverside minnow acute 94.4%Inland Silverside minnow chronic 100%
Office of Water
Results – False Positive Rates
Test Method Survival Endpoint
Sublethal Endpoint
Ceriodaphnia dubia acute 0.00% NACeriodaphnia dubia chronic 0.00% 3.70%Fathead minnow acute 0.00% NAFathead minnow chronic 0.00% 4.35%Selenastrum capricornutum chronic (with EDTA) NA 0.00%Mysidopsis bahia chronic 0.00% 0.00%Sheepshead minnow acute 0.00% NASheepshead minnow chronic 0.00% 0.00%Inland Silverside minnow acute 0.00% NAInland Silverside minnow chronic 0.00% 0.00%
Office of Water
Results – Interlaboratory Precision (CV%)
Test Method LC50 IC25Ceriodaphnia dubia acute 29.0% NACeriodaphnia dubia chronic 21.5% 35.1%Fathead minnow acute 20.0% NAFathead minnow chronic 13.4% 20.9%Selenastrum capricornutum chronic (with EDTA) NA 34.3%Mysidopsis bahia chronic 31.2% 41.3%Sheepshead minnow acute 26.0% NASheepshead minnow chronic 8.72% 10.6%Inland Silverside minnow acute 38.5% NAInland Silverside minnow chronic 40.6% 43.8%
Office of Water
Study Results Summary
Successful test completion rate - • 8 of 10 methods had successful test completion rates greater than
90%
False positive rate - • All 10 methods had false positive rates less than 5% (0.00% -
4.35%)
Interlaboratory variability -• Ranged from 8.72% to 40.6% for LC50s and from 10.6% to 43.8%
for IC25 values• Comparable to chemical methods promulgated at 40 CFR Part 136
• Below interlaboratory variability previously cited for the WET methods
Comparison of WET Method Variabilityat Method Promulgation and Reported
in the WET Variability Study
Reported in WET Variability StudyReported at time of promulgation
Inte
rlab
orat
ory
Vari
abili
ty (
%CV
)
Ceriodaphniadubiaacute
Ceriodaphniadubia
chronic
Fatheadminnowacute
Fatheadminnowchronic
Sheepsheadminnowacute
Sheepsheadminnowchronic
Inlandsilverside
acute
44.2
%
42%
35%
34%
42%
44.2
%
42.2
%
29.0
% 35.0
%
20.0
%
20.9
% 26.0
%
10.5
%
38.5
%
Office of Water
For More Information
Contact:Marion Kelly
U.S. EPA Office of Water Engineering and Analysis Division (4303T)
1200 Pennsylvania Ave., N.W. Ariel Rios Building
EPA WestWashington, DC 20460
(202) 566-1045kelly.marion@epa.gov
top related