ord sensors/applications test bed challenge: investigation of

ORD Sensors/Applications Test Bed Challenge: Investigation of Sensor/Application Response

under Controlled Laboratory Conditions

WA 2-22 and WA 3-01 Air Casting Sensor Evaluation Final Report

TR-RM-13-08

September 2013

Prepared for

Human Exposure and Atmospheric Sciences Division National Exposure Research Laboratory U.S. Environmental Protection Agency

Research Triangle Park, NC 27711

Contract EP-D-10-070

P.O. Box 12313 Research Triangle Park, NC 27709

ORD Sensors/Applications Test Bed Challenge: Investigation of Sensor/Application Response

under Controlled Laboratory Conditions

WA 2-22 and WA 3-01 Air Casting Sensor Evaluation Final Report

TR-RM-13-08

September 2013

by

Sam Garvey Alion Science and Technology


Submitted to

Ronald Williams WA Contracting Officer’s Representative

Human Exposure and Atmospheric Sciences Division National Exposure Research Laboratory U.S. Environmental Protection Agency


Contract EP-D-10-070 Reviewed and approved by Robert A. Mickler Alion Principal Investigator and Technical Supervisor

TR-RM-13-08 September 2013

ii

Foreword

This technical report presents the results of work performed by Alion Science and Technology under contract EP-D-10-070 for the U.S. Environmental Protection Agency, Research Triangle Park, NC. It has been reviewed by Alion and approved for publication. Mention of trade names or commercial products does not constitute endorsement or recommendation for use.


iii

Contents

Section Page

Foreword ...................................................................................................................................... ii

Figures......................................................................................................................................... iii

Tables .......................................................................................................................................... iii

1 Introduction .......................................................................................................................1

2 Definitions.........................................................................................................................2

2.1 General Definitions ...............................................................................................2

2.2 Test Conditions .....................................................................................................2

2.3 Evaluation Parameters ..........................................................................................2

3 Materials and Methods ......................................................................................................4

4 Results ...............................................................................................................................7

5 Discussion .........................................................................................................................8

Appendix: Detailed Project Information Slides ............................................................................9

Figures

Figure 1. Test chamber .................................................................................................................4

Figure 2. Air Casting inside test chamber with heat pad ..............................................................5

Figure 3. Example Air Casting response for NO2 under normal challenge conditions ................7

Tables

Table 1. Calibration Sequence Concentrations and Hold Times ..................................................6

Table 2. Summary of Air Casting Testing ....................................................................................7


1

1. Introduction

EPA’s Office of Research and Development (ORD) recently carried out a sensors/ applications challenge as a result of an EPA-sponsored new technology workshop. This challenge is a high priority for EPA and one in which ORD’s National Exposure Research Laboratory (NERL) was asked to take a leadership role in promoting. Consequently, EPA established as a priority providing critical feedback to groups or individuals developing novel sensors/applications regarding the potential of their devices to be used for community-based (crowd source) data collections. As nitrogen dioxide (NO2) and ozone (O3) represent the primary pollutant gases of interest for evaluation, NERL sought out novel sensor technologies for the measurement of ambient NO2 and O3 through a general appeal to inventors and developers of these technologies with whom to collaborate.

The purpose of this collaboration is to provide NERL an opportunity to examine emerging sensor technologies and then to share technical feedback with the collaborator on the general performance characteristics of their sensor as a means of advancing the general state-of-the-science. The first slides in the Appendix expand on the purpose of this particular sensors/applications evaluation (slide 1) and EPA’s general protocols for technical evaluation of air quality instrumentation (slide 2).

As part of NERL’s Material Cooperative Research and Development Agreements (MCRADAs) with numerous U.S. and international research institutions for performance evaluation of their low-cost sensors, the technology provided by each collaborator is temporarily transferred to NERL custody where its performance is examined under controlled laboratory conditions. Each transferred sensor is being tested under known laboratory/chamber conditions of relative humidity (RH), temperature, pollutant atmosphere, and interfering species concentration. NERL is attempting to establish the following evaluation criteria for each device: (a) linearity of response, (b) precision at each known reference concentration, (c) determination of the lowest established concentration in which a response was detected, (d) concentration resolution, (e) response time, and (f) suggested range of operation to achieve best practical operating conditions. Replicate trials are performed to ensure data quality. These trials are examining the impact of changes in environmental temperature and RH on sensor response, as well as the effect of sulfur dioxide, a common ambient air pollutant that represents a possible interfering response agent.

The work reported here represents work conducted by Alion Science and Technology in support of the evaluation of these novel sensors/applications under Work Assignments (WAs) 2-22 and 3-01 (Contract EP-D-10-070). These evaluation findings and other more general observations are associated with the day-to-day handling of the Air Casting environmental sensor.


2

2. Definitions

2.1 General Definitions

Calibration sequence: The series of input set points for calibrating the sensor beginning with zero air, increasing the concentration of analyte to the maximum the dynamic dilution calibrator can produce (400–500 ppb), and then stepping the concentration down across a total of at least five set points ending with zero air.

FRM/FEM: Federal Reference Method and Federal Equivalent Method, as detailed in slides 3 and 4 of the Appendix.

Test: A series of at least three and up to five calibration sequences run one after the other under a single set of conditions.

2.2 Test Conditions

Normal conditions: Room temperature (between 20 and 25 C) and dry (~ 20% RH).

Hot conditions: Two heaters are used to heat the gas mixture to approximately 45 C, one around the inlet line and the other around the test chamber. Humidity is less than 10% due to the increased solubility of water vapor in hot air.

Cold conditions: Dry ice is used next to the test chamber to chill the gas mixture to approximately 0 C. Humidity is maintained at approximately 50% due to the reduced solubility of water vapor in cold air.

Humid conditions: An impinger is installed in the inlet line and filled with deionized water. The resulting RH is approximately 70%. Testing is performed at room temperature (between 20 and 25 C). This is further detailed on slide 21 of the Appendix.

2.3 Evaluation Parameters

Conditioning: Some sensors show an increased response after completing a calibration sequence. This is hypothesized to be a consequence of analyte gases being absorbed onto the surfaces of the sensor. With prolonged exposure, all active sites on these surfaces become saturated. As a result, analyte gas ceases to be absorbed onto the sensor surfaces and the response appears to rise. This process is referred to as conditioning. Because conditioning frequently happens over the course of hours, and because it can be affected by a test performed the previous day, no direct measurements of conditioning time have been made. Conditioning time can be quantified only by the number of calibration sequences that must be voided because the response is far lower than the response of calibration sequences taken afterwards.

Response: The slope of the best-fit line, which is a means of comparing sensor sensitivity under different conditions.

Linearity: The coefficient of determination (R2) of the best fit line, determined as shown in slide 17 of the Appendix.


3

Precision: At a given set point (generally 100 ppb), the instrument response is measured multiple times (minimum of three replicates). The concentration of gas is calculated from the instrument’s calibration curve and the standard deviation of these measurements (in ppb) is recorded as detailed in slide 17 of the Appendix.

Lower detection limit (LDL): LDL is the difference in sensor output (in ppb) between set points of 0 ppb and 10 ppb (10 ppb is the level specified for LDL calculations in Table B-1 of CFR Part 53, as detailed in slide 18 in the Appendix).

Instrument detection limit (IDL): Zero air is measured multiple times and a calibration curve is used to calculate these measurements (in ppb). The standard deviation of these measurements is multiplied by 3 to produce the IDL.

Res low: 10 data points spaced at least 2 minutes apart are taken at 0 ppb. The lower resolution is the relative standard deviation of the 10 measurements, as detailed in slide 19 of the Appendix.

Res high: 10 data points spaced at least 2 minutes apart are taken at 200 ppb. The 200 ppb set point was chosen because it is the highest set point the Thermo 42C trace-level analyzer (Thermo Electron Corp., Franklin, MA) can accurately measure and the highest set point all tests had in common (see Table 1 in section 3). The higher resolution is the relative standard deviation of these 10 measurements, as detailed in slide 19 of the Appendix.

Lag time: Lag time is the time (in minutes) between the input of a new set point and the first measurement different from baseline by more than twice the resolution, as detailed in slide 20 of the Appendix.

Rise time: Rise time is the time (in minutes) between the end of lag time and when measurements reach 95% of their stable value for the new set point, as detailed in slide 20 of the Appendix. This can be accurately measured only when the sensor has been adequately conditioned for the test analyte.

SO2 int: The false positive interference induced on the sensor by 500 ppb of SO2. This is tested for both NO2 and O3 sensors, as detailed in slide 22 of the Appendix. Note that interference testing was performed only under normal conditions.

O3 int: The false positive interference induced on an NO2 sensor by 500 ppb of O3 as detailed in slide 22 of the Appendix. Note that interference testing was performed only under normal conditions.

NO2 int: The false positive interference induced on an O3 sensor by 500 ppb of NO2 as detailed in slide 22 of the Appendix. Note that interference testing was performed only under normal conditions.

Off scale: The response is too large (high) and too variable to have meaning.


4

3. Materials and Methods

A glass test chamber, pictured in Figure 1 and detailed in slides 6 and 7 of the Appendix, was used to immerse the Air Casting sensor in the test atmosphere. The NO2 test atmosphere was added using a compressed gas cylinder and the dynamic dilution calibrator. Slide 15 of the Appendix provides details of the gas sources used to challenge the sensors.

.

Figure 1. Test chamber.

The test chamber was placed within two layers of coolers in order to insulate it and make temperature controls viable. Humid conditions were achieved by inserting an impinger of Milli-Q water between the calibrator and the test chamber. Hot conditions were achieved through the use of a heat pad wrapped around the test chamber and heat tape wrapped around the inlet line. Cold conditions were achieved through the application of dry ice. The maintenance of these conditions is further explained in slide 8 of the Appendix. The positioning of the Air Casting in the test chamber is shown in Figure 2.


5

Figure 2. Air Casting inside test chamber with heat pad.

Slide 23 of the Appendix shows the basic protocol for testing. To elaborate further, nominally all tests for the Air Casting should have consisted of five calibration sequences. These were to be run as part of a single automated program completed within a 24-hour period. Each time this was attempted, however, the Air Casting monitor ceased recording at some point during the night and failed to save any of the collected data. A workaround for this problem was developed in which the data recording could be stopped manually, saved, and restarted when staff left the building for the day. This ensured that the data taken during business hours was preserved. Later a software patch from the manufacturer allowed the Air Casting monitor to save collected data in the event of a crash, but it did not reduce the number of crashes. The last 5 minutes of data taken at a given set point were averaged to form a single data point per set point per calibration sequence. A Thermo 42C trace-level NO-NO2-NOx FRM/FEM reference analyzer was used to verify that conditions within the test chamber matched the entered set point as shown on slides 9 through 14 in the Appendix.

The maximum concentration for NO2 was selected based on flow rate requirements and the limitations of the dynamic dilution calibrator. The minimum concentration was selected based on the requirements for LDL measurements listed in the CFR and on slide 5 of the Appendix. The concentrations and hold times at each concentration are listed in Table 1.

Temperature/ humidity probe

Reference analyzer sample lines

Heat pad Direction of flow

Air Casting with power cable


6

Table 1. Calibration Sequence Concentrations and Hold Times

NO2 Conc. (ppb) Time (min)

0 30 400 45 200 30 100 30 50 30 10 30 0 30

All testing followed the quality assurance project plan “ORD Sensor/Application Test Bed Challenge,” QAPP-AB-12-03. All data produced were subject to review by Alion QA personnel who were completely independent of the data collection for the project. Raw data and summary Excel worksheets have been provided to the collaborator separately from this report. All data have been included in these Excel worksheets, although conditioning time issues have resulted in some data being excluded from higher order calculations. Excluded data have been highlighted in red on the calculation worksheets. Further details on quality assurance can be found on slide 24 of the Appendix.


7

4. Results

All data were lost in our first attempts to test the Air Casting monitor, so because of time constraints it was tested only under the NO2 normal condition. We also experienced several spontaneous software crashes that resulted in the test being performed in three segments on May 8–9, 2013. In addition, the operator had difficulty establishing and maintaining the Bluetooth connection between the Air Casting monitor and the Android device, which might have been due to the signal being blocked by the walls of the test chamber and the two coolers surrounding it.

The data are summarized below in Table 2. The last row lists the relevant benchmarks for the NO2 FRM/FEM instrument as listed in 40 CFR Part 53 Table B-1 and in slide 5 of the Appendix. An example of the sensor’s response to NO2 under normal challenge conditions is shown in Figure 3.

Table 2. Summary of Air Casting Testing

Analyte Conditions Response (mV/ppb)

Linearity (R2)

Precision(ppb)

LDL (ppb)

IDL (ppb)

Res Low (ppb)

Res High (ppb)

Lag Time (min)

Rise Time (min)

NO2 Normal 2.7993 0.9846 3 11.6 14.6 1.1 1 0 4

CFR NO2 NA NA NA 10 10 10 5 5 20 15

Figure 3. Example Air Casting response for NO2 under normal challenge conditions.

‐100.00

‐50.00

0.00

50.00

100.00

150.00

200.00

250.00

11306

2611

3916

5221

6526

7831

9136

10441

11746

13051

14356

15661

16966

18271

19576

20881

22186

23491

24796

26101

27406

28711

Response

Data Points

Air Casting NO2 Normal

Thermo 42C (ppb) Air Casting (ppb)


8

5. Discussion

Data points were produced every second and included battery voltage; temperature; humidity; sound level; and CO2, NO2, and CO sensor output. Gas sensor data was nominally reported in ppm, but the actual response was extremely low, got lower as the gas concentration in the test chamber increased, and even became negative at high concentrations. The manufacturer was contacted and hypothesized that the unit had been assembled with the sensor chip installed backwards. A software patch was issued such that the unit would report output in mV. For this reason it should be noted that all data and observations in this report were gathered from a defective unit and may not be indicative of a fully functioning Air Casting monitor.

The first calibration sequence run on the Air Casting monitor had to be voided due to conditioning issues. All data points at concentrations above 100 ppb were voided because they lay outside the linear range of the backwards-installed sensor.


9

Appendix: Detailed Project Information Slides


10


11


12


13


14


15


16


17


18


19


20

ord sensors/applications test bed challenge: investigation of

Documents