performance audits · 2021. 3. 16. · performance audits - current practice and future activities...

35
PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training, Davis, CA

Upload: others

Post on 15-Aug-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

PERFORMANCE AUDITS -CURRENT PRACTICE AND FUTURE ACTIVITIES

1

PRESENTED BY LAMAR MITCHELL

CARB Quality Assurance Section

2019 PQAO Training, Davis, CA

Page 2: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

2

• Background

• Types of performance evaluations

• Update trace gas assessments

• New equipment

• Future audit technologies

Page 3: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

• Independent verification

• Federally mandated for ambient air data to

be compared to national standards

• Conduct audits using NIST traceable standards

• Adhere to federally established acceptance

criteria

OVERVIEW

• Background

3

Page 4: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

• Background

• Types of Performance Evaluations

4

Page 5: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

2018 AUDITS

5

Particulate Samplers

490Gaseous

Analyzers 270

Meteorological Instruments

231

Mass Analysis Labs 7

Total Audits = 998

Page 6: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

≈ 267 active monitoring

stations in the state.

998 audits in 2018

6

2018 PERFORMANCE AUDITS

Include:

• Mass Analysis Laboratory

• Air Monitoring Stations

Page 7: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

• Background

• Types of Performance Evaluations

• Gases and particulates

7

Page 8: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

8

Through-the-Probe Audits developed by CARB in 1981

and then adopted nationally

Page 9: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

9

For particulate sampler audits we will use:

• BGI DeltaCal for BAMS and FRM2.5

• BGI TetraCal for SASS, Super SASS,

Xontech, & TEOMs

• HiVol Direct read calibrator for HiVol

samplers

PARTICULAR SAMPLER OR

FLOW RATE AUDITS

Page 10: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

10

• Background

• Types of performance evaluations

• Update trace gas assessments

Page 11: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

11

• Audit levels 1 & 2 have an absolute value or percent

difference criteria.

• All other audit levels have a percent difference criteria

• No Corrective action taken at audit levels 1 & 2

U.S. EPA AUDIT RANGES

Page 12: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

TRACE ANALYZER AUDIT ISSUES

• CO analyzer performance

Gas filter correlation technology (GFCT)

Known drift behavior

Intended for continuous sampling

Significant warm-up time

Not ideal for trace auditing

• Explore options for improved confidence and

reliability of assessments.

12

Page 13: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

ATTEMPT TO IMPROVE AUDIT ANALYZER PERFORMANCE

13

Page 14: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

ALTERNATE CO ANALYTICAL METHOD

• Cavity Ring-Down Spectroscopy (CRDS) CO analyzer installedWarm-up time greatly reduced

Consistently near target values (no drift)

Validates CO audit methodology for trace levels

Expensive

• Audited with CRDS and GFCT CO analyzers in parallel

• Likely to further enhance trace bias assessments

14

Page 15: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

15

0.7 0.3 -0.87-0.43 -0.8

0.21

-170

-150

-130

-110

-90

-70

-50

-30

-10

10

Pre Post Pre Post Pre Post

Santa Maria Stockton Fresno

GFCT CRDS

Comparison of Pre and Post Zeros – GFCT vs CRDS

Page 16: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

16

-0.020

-0.010

0.000

0.010

0.020

0.030

0.040

0.050

0.060

0.070C

O A

udit V

alu

es

Gene

rate

d (p

pm

)

Sites

Comparison of CO Audit Level 1 Target Concentration 0.040 ppm

CRDS GFCT

Page 17: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

-18.5

-49.0

246.7

235.0

117.9

19.1

71.1

-95.4

-43.7

-1.1

-6.3

0.7

9.8

-0.5

6.2

-5.7

-150.0 -100.0 -50.0 0.0 50.0 100.0 150.0 200.0 250.0 300.0

Audit Results -- Average Percent DifferenceTrace Level SO2 - 2018

GFCT

CRDS

17

Page 18: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

18

• Background

• Types of performance evaluations

• Update trace gas assessments

• New equipment

Page 19: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

AUDITS IN LATE 2019 OR EARLY 2020

19

•API T640 and T640X

• Scattered light spectrometry

• T640X Measures PM2.5 & PM10

•Thermo 5014i

• Continuous BAM unit measures PM2.5

•API 602 Beta Plus

• Continuous BAM unit measures PM10 & PM2.5 on filters.

•Direct-read NO2 Analyzers

Page 20: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

20

• Background

• Types of performance evaluations

• Update trace gas assessments

• New equipment

• Future audit technologies

Page 21: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

PERFORMANCE AUDITS – FUTURE ACTIVITIES

PRESENTED BY AARON PLASENCIA

CARB Quality Assurance Section

2019 PQAO Training, Davis, CA

Page 22: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

OVERVIEW

•Airpointer

•Remote Audits

•UAV Technology

•Wind Tunnel

22

Page 23: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

AIRPOINTER

23

Page 24: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

AIRPOINTER - CAPABILITIES

•FEM Designation

•Portability & Mounting

•API T640X

•O3 & NOx Analyzers

•Climate Control

24

Page 25: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

TRIALS AND TRIBULATIONS

•Establishing Communication

•Stainless Steel Inlet

•Acceptance Testing

• Through-the-Probe Audit

•Power Requirements

25

Page 26: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

COMPARING O3

26

0.0000

0.0050

0.0100

0.0150

0.0200

0.0250

0.0300

0.0350

QA AirpointerRegulatory Station

Page 27: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

COMPARING NOX

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

Airpointer qa

27

Regulatory Station

Page 28: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

COMPARING PM

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

Airpointer vs EBAM

PM10 640 PM2.5 640 PM10 ebam PM2.5 ebam

28

XX

Page 29: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

DEPLOYMENT

•Community Monitoring

•Areas of Interest

•Data Validation

29

Page 30: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

LOOKING FORWARD:REMOTE AUDITS

•Dedicated/Rotating Audit

System

•Remote Sites

•Automated Audits

30Joshua Tree

Death Valley

Page 31: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

LOOKING FORWARD:REMOTE AUDITS (continued)

•Infrastructure Cost• Calibrator, ZAG, Gas Cylinder

•Limitations• Gas Only

• Siting/Residence Time/Flows

•Referee Instrument for

Troubleshooting

31

Page 32: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

UAV TECHNOLOGY

•Unmanned Aerial Vehicle

•Siting – Birds Eye View

•Layer Profiling / GIS

Mapping

•FAA Requirements

•Restrictions

32

Page 33: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

WIND TUNNEL

•Ultrasonic Meteorological Audits

•Transportability

33

Page 34: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

TAKE-HOME MESSAGE

•Audit program evolves to reflect latest requirements

•Improvements on current practices to maintain integrity

•Possible future use of new technology

34

Page 35: PERFORMANCE AUDITS · 2021. 3. 16. · PERFORMANCE AUDITS - CURRENT PRACTICE AND FUTURE ACTIVITIES 1 PRESENTED BY LAMAR MITCHELL CARB Quality Assurance Section 2019 PQAO Training,

QUESTIONS?

CONTACT INFORMATION:

Aaron PlasenciaAir Resources Engineer

(916) 322-9192

[email protected]

LaMar MitchellAir Resources Engineer

(916) 445-9371

[email protected]

Ranjit BhullarManager, QA Section

(916) 322-0223

[email protected]