Quality Assurance/ Quality Control

Nate Herbst

Southern Ute Indian Tribe

2

Intro to QA/QC

• Getting good data requires many different steps– Data quality objectives (DQOs) developed

• DQOs for ozone being developed• Measurement quality objectives (MQOs) for ozone exist

– Analysis begun (after correct calibration)– QC checks performed– QA conducted

3

Data Quality Objectives (7 steps)

• State problem– Define why monitoring is needed– Create team and purpose

• Identify decision– What decision will be made with data?

• Identify decision inputs– What data necessary to make decision?

• Define boundaries– What are study area boundaries?

4

Data Quality Objectives (cont.)

• Develop decision rule– What conditions will require action (action level)?

• Specify decision error limits– What margin of error is allowable

• Optimize monitoring design– Develop most cost-effective method of reaching

DQOs

• EPA hasn’t yet defined DQOs for ozone analysis

5

DQOs (cont.)Diagram of DQO steps(Diagram courtesy of U.S. Department of Energy – DQO homepage)

6

Measurement Quality Objectives (MQOs)• EPA has ozone analysis MQOs• Use these in element 7 of your QAPP• MQOs in a nutshell

– Shelter temperature kept between 20-30oC ± 2oC– Analyzer must be reference or equivalent method– Lower detectable limit 0.01ppm, noise 0.005ppm– Data completeness 75% of hourly values between

9:01am and 9:00pm (for the ozone season)– Transfer standard certification ±4% or 4ppb

(whichever is greater)

Thanks to Melinda Ronca-Batista (ITEP)

7

MQOs (cont.)

– Transfer standard re-certification to primary std.dev 1.5%

– Local primary standard certification ±5% of reference

– EPA reference photometer regression slope 1.00 ± 0.01

– Zero air free of O3 and anything that might react with O3

8

MQOs (cont.)

– Ozone analyzer calibration • Z/S check zero ±10ppb, span ± 15%• 5pt calibration linearity error ±5%

– Performance (NPAP) mean absolute difference ± 15%

– Precision (quarterly) 95% CI < ±15%– Audits (annually) 95% CI < ±20%

9

Quality Assurance Project Plans (QAPPs)

• Contain 24 “elements” – Element 7 is where MQOs go– Cut and paste from red book

• Ensure data quality• Required by EPA• Developed by program approved by EPA• They must be followed!

– No good if not followed

10

Documentation

• Document everything!!!• Documentation in

– Logbooks– Site folders– QA/QC field forms– Anywhere else you think is appropriate

• QA/QC – document standard values and response

11

Documentation (cont.)

• Document repairs, checks, fine tunes• Document site conditions• Document everything that could ever be

important• Write only in pen (black if possible)• Cross out errors with a single line

12

Linearity

y = x

y = x

R2 = 1

0

1

2

3

4

5

6

0 2 4 6X axis

Y axismb

• Slope = rise over run

• m = slope• b = intercept (where

the trend-line crosses the Y axis)

• r2 close to 1 shows correlation

y = mx+b

y = 2.0829x + 1.9095R2 = 0.997

02468

101214

0 2 4 6X axis

Y axis

13

Instrument Calibration

• Measurements require point of reference• Measurement without standard is impossible• Calibration involves setting instrument to

known level• Calibrations performed fairly regularly

– When monitoring is begun– When repairs or maintenance are performed– When precision checks or audits show need

• Calibrations must be done correctly

14

Calibrations (cont.)

• Calibration = setting analyzer to standard– Data only good within linear range (~0-0.400ppm)

• Calibration followed by a 5-pt check• Analyzer must agree with standard at all 5 pts

– Linearity error < 5%– See next slide on linearity

• Monitoring begins after calibration

Note: Never initiate monitoring without calibration

15

Pre-Calibration Check5pt check

y = 1.1368x + 0.0203R2 = 0.98

0.000

0.100

0.200

0.300

0.400

0.500

0.000 0.100 0.200 0.300 0.400 0.500Standard

Analyzer

• Not always necessary

• Can do 5-pt check • Analyzer must be

calibrated• The r2 value and %

differences for each point are unacceptable

Standard Analyzer % dif.

0.000 0.000 0.0

0.080 0.100 -25.0

0.150 0.220 -46.7

0.250 0.340 -36.0

0.350 0.400 -14.3

0.400 0.460 -15.0

16

Instrument Calibration

• Calibrate instrument to the standard• Use calibration point near URL

– Setting low produces large error at URL

• Set standard to ~0.400 ppm • Let analyzer stabilize• Calibrate analyzer• Do new 5-pt check

17

Post-Calibration 5-pt Check

• Is analyzer response within 7% at each point?

• Would you put this analyzer online?

Calibration

y = 1.0145x - 0.0006R2 = 0.9997

0.000

0.200

0.400

0.600

0.000 0.100 0.200 0.300 0.400 0.500Standard

Analyzer

Standard Analyzer % dif.

0.000 0.000 0.0

0.080 0.081 -1.3

0.150 0.153 -2.0

0.250 0.248 0.8

0.350 0.354 -1.1

0.400 0.408 -2.0

18

Quality Control (QC)

• QC involves “in-house” verifications• Also referred to as precision checks• Verifications are comparisons between

transfer standard and analyzer– Relative % difference within allowable margin?

• Verifications determine monitoring repeatability– Standard deviation

• Different types of verifications

19

QC (cont.)

• Level 1: 40 CFR, Pt. 58, App. A, Table A-1 defines ozone verification requirements (for SLAMS)– Biweekly response check between 0.08 and 0.1

ppm • Comparison between analyzer and standard

– Determines repeatability

• Level 2: “extra” precision checks– Weekly “span level” (~80% URL) checks– Quarterly 5-pt checks– Determines analyzer performance trends

20

Quality Assurance (QA)

• QA involves “external” checks• Referred to as “audits”• Audits involve comparison between transfer

standard and analyzer– Accuracy levels must be within ±10%

• Audits determine how close monitoring gets to actual values

• Different types of audits

21

QA (cont.)• 40 CFR, Pt. 58, App. A, Table A-1 defines

ozone audit requirements (for SLAMS)– Annual (and other) response checks at multiple

points• 0.03-0.08 ppm• 0.15-0.2 ppm• 0.35-0.45 ppm

– Comparison between analyzer and external standard

– Audits should include zero check

22

QA (cont.)

• Different types of audits– By reporting organization (RO) certified by

RO– By RO certified by other than RO– By other than RO certified by other than

RO

23

Precision & Accuracy (P&A) Data

• Precision data come from biweekly precision checks

• Accuracy data come from annual and other audits

• P&A data validate ambient data• P&A data must be included in AQS data

submittals

24

Siting Criteria

• Data quality depends on correct siting of all instrumentation

• Specific instrument siting guidelines• Following guidelines is vital part of quality

assurance and control• We’ll learn more about these guidelines in the

next presentation

25

Summary • Establish DQOs• Develop QAPP

– Get it approved by EPA

• Follow your QAPP• Conduct bi-weekly precision checks

– Conduct level 2 checks to follow monitor trends

• Participate in annual audits and others if possible

• Data quality will be guaranteed