Use of Measurement Uncertainty Information in Compliance

Assessment of Chemical Results

D. Theodorou

Athens, October 2008

What is not uncertainty !

• Accuracy: accuracy is not a quantifiable term

• Error: result of a measurement minus the true value of the measurand

idealized concepts

What is uncertainty

a parameter associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand (ISO Guide to the Expression of Uncertainty in Measurement)

Result: 20 ± 3 mg

Uncertainty = tolerance interval

The true amount is between 17 and 23 mg

Uncertainty of Chemical Measurements

There is always experimental variations when we make a measurement

Typical Sources of Uncertainty in Chemical Analyses•SamplingStorage ConditionsInstrument effects (e.g. calibration, accuracy, carry over effects)Reagent purityAssumed stoichiometry (e.g. incomplete or side reactions)Measurement conditions (e.g temperature, humidity)Sample effects (e.g. recovery - matrix effects)Computational effects (e.g calibration model, truncation / round off)Blank correctionOperator effectsRandom effects

Type A &Type B contributions

Overall Estimate of Uncertaintyor Expanded Uncertainty

Eurachem/CITAC Guide

Combined Uncertainty

coverage factor, k

Expanded Uncertainty U, coverage factor k

U=k∙uC

k=295,45%

U U U U

k=399,73%

Typical Uncertainty Statement

Total cadmium content (Cd): 328 μg·kg-1

Measurement Uncertainty: 27 μg·kg-1 (8,2%)

The stated uncertainty is an expanded measurement uncertainty (U). It was obtained by multiplying the combined standard uncertainty uc with a coverage factor k equal to 2. This corresponds approximately to a 95 % confidence interval.

Procedures for the Estimation of Measurement Uncertainty

• ISO Guide to the expression of measurement uncertainty (ISO GUM)

• EURACHEM Guide to quantifying uncertainty in analytical measurement

• Use of collaborative trial data – ISO 5725 critical differences

• ISO/TS 21748 – Guide to the Use of Repeatability, Reproducibility and Trueness Estimates in Measurement Uncertainty Estimation

• Concept established by Commission Decision 2002/657/EC implementing Council Directive 96/23/EC concerning the performance of analytical methods and the interpretation of results

• AOAC INTERNATIONAL approach

• Internal quality control approach

• NMKL (Nordic Committee on Food Analysis) approach

• Microbiological Analyses (ISO/TS 19036, Niemelä Guide)

• Monte Carlo Simulation (ISO GUM Supplement 1)

Why is it important?

10,6

11,6

10,6

11,6

10,6

11,6

10

10,5

11

11,5

12

12,5

Lab A Lab B Lab A Lab B Lab A Lab B

mg

/kg

repeatability

uncertainty

Uncertainty = Confidence Informed Decision

Uncertainty and limiting values

Many analyses are made to assure that limiting values are not exceeded (e.g. for drinking water quality)

Without information about the measurement uncertainty it may appear to be very easy to make decisions, but these decisions may be incorrect

Uncertainty and limiting values

AN EXAMPLE

Drinking water -> Pb content

Parametric value: 10 μg/l (COUNCIL DIRECTIVE 98/83/EC)

Laboratory uncertainty at that level: ± 0,8 μg/l (Expanded uncertainty, 95% confidence interval)

Uncertainty and limiting values

11,2

10,4

9,6

8,8

8

10

12

(i) Result lessuncertaintyabove limit

(ii)Result abovelimit but limit

withinuncertainty

(iii)Result belowlimit but limit

withinuncertainty

(iv)Result plusuncertaintybelow limit

Pb

co

nte

nt

(μg

/l)

Uncertainty and limiting values

Straightforward Approach

1. If possible use a method producing more accurate results

2. Report the result and uncertainty with a statement that compliance (or non-compliance) could not be demonstrated

(for a suggested statement see UKAS Guide, M3003, The Expression of Uncertainty and Confidence in Measurement)

10,4

9,6

8

10

12

(ii) (iii)

Pb

co

nte

nt

(μg

/l)

Uncertainty and limiting values

10,4

9,6

8

10

12

(ii) (iii)

Pb

co

nte

nt

(μg

/l)

Detailed Approach• Set up probability based decision

rules• Select a decision limit (critical

value) taking into account what the end user expects:

i. low probability of false rejection, α

ii. low probability of false acceptance, β

Acceptance zone Rejection zone

Decision Limit

guard band

Upper limit

Acceptance zone Rejection zone

Decision Limitguard band

Upper limit

guard band, g = 2∙uc= U95%

Uncertainty and limiting values

Acceptance Zone Rejection Zone

Upper Limit

U95%

DL

2,5% of the values under the curve are inside the specification limit

High confidence of correct rejectionDecision limit DL is chosen so that the risk of false rejection (α) is less than 2,5%.

guard band, g = 2∙uc= U95%

Uncertainty and limiting values

Upper Limit

U95%

Acceptance Zone

DL

Rejection Zone

2,5% of the values under the curve are outside the specification limit

High confidence of correct acceptanceDecision limit DL is chosen so that the risk of false acceptance (β) is less than 2,5%.

EURACHEM / CITAC Guide

USE OF UNCERTAINTY INFORMATION IN COMPIANCE

ASSESSMENT, Edition 1(2007)

The Guide describes many aspects of the matter.

It covers cases of:

- Simultaneous upper and lower limits

- Uncertainties depending on the value of the measurand

- Asymmetric distributions of the measurand

- Standard uncertainties with effective degrees of freedom

Different decision rules are compared.

Conclusions

-Need for consistent way of reporting test results

-Uncertainty: quantitative measure of the reliability of a result

-Decisions for compliance or non-compliance with a specification

should take into account uncertainty

-When the state of compliance is not clear, appropriate judgments

should be based on probability-based decision criteria

-Decision criteria should be based on risks associated with making

wrong decision i.e. false rejection or false acceptance

Thank you for your attention !!!

Dimitris Theodorou, MSc, MBAdtheodorou@priority.com.gr

PRIORITY S.A. Business Consultant

Laboratory Accreditation Dept.