quality control and measurement uncertainty

Quality Control and Measurement Uncertainty

Ympäristötiedosta palveluihin – seminaari, 23.9.2015

Mauno Rönkkö (UEF), Okko Kauhanen (UEF), Markus Stocker (UEF), Mikko Kolehmainen (UEF), Harri Hytönen (Vaisala), Olli Ojanperä (Vaisala), Esko Juuso (UO), Markku Ohenoja (UO), Ville Kotovirta (VTT), Maija Ojanen (VTT), Petri Koponen (VTT), Teemu Näykki (SYKE), Jari Koskiaho (SYKE), Niina Kotamäki (SYKE), Jari Silander (SYKE), Hanna Huitu (LUKE), Jussi Nikander (LUKE),…

24.9.2015 Mauno Rönkkö 2

Contents

1. Sources for Uncertainty in Environmental Monitoring

2. Quality Flagging by Nordic Meteorological Institutes

3. Extended Quality Flagging Scheme to Environmental Data

4. Automatic Monitoring – case Väänteenjoki

5. Water Quality Monitoring and MUkit

6. Conclusion

Sources for Uncertainty in

Environmental Monitoring


1. Case #1: Incomplete Understanding [1/8]


http://www.mmea.fi

1. Case #2: Indirect Measurements [2/8]

•The Finnish Environmental Institute (SYKE) monitors total phosphorus of lakes and rivers in Finland.

•Currently, there is no device that measures total phosphorus of a lake.

•The amount of total phosphorus is (1) estimated based on the amount of suspended solids (2) which is estimated based on measured turbidity at (3) a given location on a lake/river.


http://wwwi3.ymparisto.fi/i3/sakylapyhajarvi/sakylapyhajarvi.htm

1. Case #3: Heterogeneous Measurement Methods [3/8]


http://www.vaisala.com/en/products/ automaticweatherstations/Pages/default.aspx

http://www.biltema.fi/fi/Toimisto---Tekniikka/Kellot-ja-Lampomittarit/Lampomittari/Langaton-saaasema-84086/

1. Case #4: Sampling [4/8]


Is this a good enough sample?

1. Case #5: Inconsistent Treatment of Measurement Errors [5/8]


ERRORS (random errors + systematic errors): -Devices are individuals -Measurements drift over time -Devices are positioned badly -Devices are used in non-optimal conditions -Measurement noise is too large for measured quantity -Person measuring affects the measurements -Environmental conditions vary and affect measurements -Several different measurement devices used to get a dataset -Spatially and temporally different measurements are not preprocessed -Wrong devices are used in a specific measurement method -Different measurement methods with different devices are used -No calibration -Standards and not used or they are used improperly -Measurement data is treated with wrong statistical methods …

1. Case #5: Inconsistent Treatment of Measurement Errors [6/8]


ERRORS (the less recognized but most important): -Non-synchronized measurement clocks -Cognitive pitfalls

1. Case #6: Semantically Inconsistent Interoperability [7/8]


1. Case #7: Poorly Understood Uncertainties and Validities [8/8]


Quality Flagging by

Nordic Meteorological Institutes


F. Vejen (ed), C. Jacobsson, U. Fredriksson, M. Moe, L. Andresen, E. Hellsten, P. Rissanen, T. Palsdottir, and T. Arason. Quality Control of Meteorological Observa- tions. Automatic Methods Used in the Nordic Countries. Climate Report 8/2002, Norwegian Meteorological Institute, 2002.

2. Why bother?


THIS IS WATER CONSUMPTION!?

2. Why bother?


IS THIS WATER CONSUMPTION!?

2. Why bother?


THIS IS WATER CONSUMPTION!

2. Data to information


Measurement device

Server

Data storage

Data analysis and refinement

Human operator

2. Quality checks


QC0 QC1

QC2

HQC

Measurement device

Server

Data storage


Human operator

2. Quality checks


QC1

QC2

HQC

Measurement device

Server

Data storage


Human operator

QC0: real-time quality control on

individual data points about range, step and

consistency

2. Quality checks


QC2

HQC

Measurement device

Server

Data storage


Human operator


individual data points about range,

step and consistency

QC1: real-time quality control on individual data

points using statistical methods, including missing

and expected values

2. Quality checks


HQC

Measurement device

Server

Data storage


Human operator



step and consistency QC1: real-time quality

control on individual data points using statistical

methods, including missing and expected

values

QC2: non-real-time quality control on data sets

including spatial and temporal analysis with

corrective computations

2. Quality checks


Measurement device

Server

Data storage


Human operator



step and consistency QC1: real-time quality

control on individual data points using statistical

methods, including missing and expected

values

QC2: non-real-time quality control on data sets

including temporal and spatial analysis with

corrective computations

HQC: non-real-time quality inspection

including visualization; the final word

2. Measurement Data


556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42

time

humidity temperature

location

device-id

2. Measurement Data with a Quality Flag


556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330

time quality flag


location

device-id

2. The Flag Values


556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330

time

quality flag


location

device-id

C = 1000EHQC + 100EQC2

+ 10EQC1 + EQC0

2. Multiple Data Points


556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330

C = 1000EHQC + 100EQC2

+ 10EQC1 + EQC0

556 2014-09-27T09:00:30 62.8925, 27.678333 15.3 55 4000

Extending the Quality Flagging Scheme

to Environmental Data


M. Rönkkö, O. Kauhanen, M. Stocker, H. Hytönen, V. Kotovirta, E. Juuso, M. Kolehmainen. Quality Control of Environmental Measurement Data with Quality Flagging. IFIP Advances in Information and Communication Technology, 2015, Volume 448, Environmental Software Systems. Infrastructures, Services and Applications, pages 343-350.

3. Generic Interpretation


Flag Original interpretation Generic interpretation

0 No check performed Value not checked

1 Observation is ok Approved value

2 Suspected small difference Suspicious value

3 Suspected big difference Anomalous value

4 Calculated value Corrected value

5 Interpolated value

Imputed value

6 (Not defined originally) Erroneous value

7 (Not defined originally) Frozen value

8 Missing value Missing value

9 Deleted value Deleted value

3. Example Service Architecture


3. Quality Control of Water Consumption Data


3. Quality Control of Water Consumption Data


QC1: measured and checked once a minute QC2: runs every 2 hours, used for spotting leaks and malfunctions HQC: done once a month, aims at resolving frozen data values

Automatic Monitoring - case Väänteenjoki



4. Automatic monitoring – case Väänteenjoki • The challenge in automating water quality monitoring is that the measurement data

have not only significant seasonal variation, but also erroneous values

• Thus, without proper quality control and reliable uncertainty estimation, the data has little value

• As a solution, we have implemented a computation service based on an Enterprise Service Bus Architecture. The service provides means for online quality control and integration of uncertainty estimation

• Case study: In the Karjaanjoki River Basin the Väänteenjoki site equipped with an OBS3+ turbidity sensor (Campbell Scientific inc.)

• OBS3+ sensor emits a near-infrared

light into the water, measures the light that scatters back from the suspended particles, and transforms this information into turbidity values in Nephelometric Turbidity Units (NTU)


4. Automatic monitoring – case Väänteenjoki • The “raw” turbidity recorded by the OBS3+ sensor had to be calibrated against the

turbidity determined from water samples taken near the sensor • Calibration equation was determined by linear regression between the values of the

water samples and the simultaneous values recorded by the sensor • Then, because turbidity does not denote the content of substance in water, the

calibrated turbidity data had to be converted to concentrations of susp.solids and total P

• We have implemented a computational service that automates and integrates uncertainty estimation to the sequence of operations

Water Quality Monitoring and MUkit


Conclusion


5. Conclusion [1/2]

•What you cannot measure, you cannot control.

•Sources for uncertainties

Incomplete understanding, Indirect measurements, Heterogeneous measurement methods, Sampling, Inconsistent treatment of measurement errors, Semantically inconsistent interoperability, Poorly understood uncertainties and validities

•Quality Flagging

– scheme by the Nordic Meteorological Institutes

– Quality checks at various stages; Real-time and non-real-time checks

•Quality Flagging of Environmental Data

– Generic interpretation

– ESB based architecture


5. Conclusion [2/2]

•Automatic monitoring – case Väänteenjoki

– proper quality control and reliable uncertainty estimation required

– implemented a computation service based on an ESB

•Water quality monitoring and MUkit

– Based on the Nordtest TR 537 guide and on the standard SFS-EN ISO 11352

– Automated turbidity measuring system for ”real-time” uncertainty estimation using AutoMUkit

– Several international publications available!


Mauno Rönkkö [email protected] tel. +358 40 355 2202

www.uef.fi

mailto:[email protected]