MARLAP Chapter 19Measurement Uncertainty

Keith McCroan

Bioassay, Analytical & Environmental Radiochemistry Conference 2004

Outline

What you should know The Guide to the Expression of

Uncertainty in Measurement (the “GUM”) Uncertainty in the radiochemistry lab Summary of recommendations

What You Should Know

Chapter 19 of MARLAP is the measurement uncertainty chapter

It’s big and it has lots of equations

What do you really need to know about it?

What You Should Know

It has more than one target audience The first 3 sections present concepts and

terms, with no math They are intended for readers who want to

know what uncertainty means or who want to learn the terminology and notation

The remaining sections contain the mathematical details for lab personnel who need to evaluate and report measurement uncertainty

What You Should Know

At the end of the 3rd section, we summarize our major recommendations

If you don’t like math, you can stop reading after the recommendations

But the fun begins in Section 4

Top Recommendations

Use the terminology, notation, and methodology of the GUM

Report all results – even if zero or negative – unless it is believed for some reason that they are invalid

Report the uncertainty of every result and explain what it is (e.g., 1σ, 2σ ?)

Consider all sources of uncertainty and evaluate and propagate all that are believed to be potentially significant in the final result

All the Rest

The chapter summarizes the GUM General information in Section 3 Mathematical details in Section 4

Section 5 discusses the evaluation of uncertainty for radiochemical measurements

Questions So Far?

Part 1

The GUM

What is the GUM?

It is a guide, published by ISO and available from ANSI

It presents terminology, notation, and methodology for evaluating and expressing measurement uncertainty

It tries to get everyone speaking and writing the same language about uncertainty

The GUM

Published in 1993 by ISO in the name of 7 international organizations

Revised and corrected in 1995 Accepted by NIST and other national

standards bodies Endorsed by MARLAP Gradually being adopted by ANSI & ASTM

Don’t Fight It

The GUM approach is no harder than what you’ve done before

More than anything else, you need to learn its terms and symbols

You’re going to see it more and more (e.g., in ASTM documents)

Resistance is futile

The GUM Approach

What follows is an oversimplified summary of the terminology, notation, and methodology of the GUM

The Measurand

Metrologists define the measurand for any measurement to be the “particular quantity subject to measurement”

For example, if you’re measuring the specific activity of 137Cs in a sample of soil, the measurand is the specific activity of 137Cs in that sample

Uncertainty of Measurement

The GUM defines uncertainty of measurement as a parameter, associated with the result of a

measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand

An uncertainty could be (for example) a standard deviation, a multiple of a standard deviation, or the half-width of an interval having a stated level of confidence

Error of Measurement

Statisticians and metrologists disagree about the meaning of the word “error”

Statisticians use error to mean uncertainty, as in the “standard error” of an estimator

To a metrologist, the error of a measurement is the difference between the result and the true value

Metrological error is a theoretical concept – You can never know what its value is

Mathematical Model of Measurement

Before one ever makes a measurement, one makes a mathematical model of the measurement

Typically the value of the measurand is not measured directly but is calculated from other quantities (input quantities) that are measured

The model is an equation or set of equations that determine how the value of the measurand, Y, is to be calculated from the values of the input quantities X1,X2,…,XN

Mathematical Model of Measurement

When we talk about the model, we may also refer to the measurand Y as the output quantity

Although the model may consist of one or more equations, we’ll denote it here abstractly as a single equation

Y = f(X1,X2,…,XN)

Making a Measurement

To make a measurement, one determines values for the input quantities and plugs them into the model to calculate a value for the output quantity, Y

The values determined for the input quantities in a particular instance of the measurement are called input estimates and may be denoted by x1,x2,…,xN

The value calculated for the output quantity is called the output estimate and may be denoted by y

Uncertainty Propagation

Each input estimate has an uncertainty, and the uncertainties of the input estimates combine to produce an uncertainty in the output estimate

The operation of mathematically combining the uncertainties of the input estimates to obtain the uncertainty of the output estimate is called propagation of uncertainty

Steps in Uncertainty Propagation

Determine values for the input quantities (the input estimates) and calculate the value of the output quantity (the output estimate)

y = f(x1,x2,…,xN) Evaluate the uncertainty of each input estimate

and the covariance of each pair of correlated input estimates

Propagate the uncertainties and covariances of the input estimates to calculate the uncertainty of the output estimate

Standard Uncertainty

Before uncertainties can be propagated, they must be expressed in comparable forms

The standard uncertainty of any measured value is the uncertainty expressed as an estimated standard deviation – i.e., the “one-sigma” uncertainty

The standard uncertainty of an input estimate, xi, is denoted by u(xi)

We express all the uncertainties as standard uncertainties when we propagate them

Evaluating Uncertainties

There are many ways to evaluate the standard uncertainty of an input estimate, xi

For example, one might average the results of several observations and calculate the standard error of the mean, or take the square root of the number of counts observed in a single radiation counting measurement

Or one might make a wild (educated) guess about the maximum possible error in a value and divide it by sqrt(3) or sqrt(6)

Type A and Type B Evaluations

The GUM groups all evaluations of uncertainty into two categories: Type A and Type B A Type A evaluation of uncertainty is a

statistical evaluation based on series of observations

A Type B evaluation of uncertainty is anything else

Type A and Type B

An uncertainty evaluated by a Type A method used to be called a “random uncertainty” (but not anymore!)

An uncertainty evaluated by a Type B method used to be called a “systematic uncertainty” (but not anymore!)

Type A Evaluation

Statistical evaluation of uncertainty involving series of observations

Example: Make a series of observations of a quantity,

then calculate the mean and the “standard error of the mean,” or what metrologists call the “experimental standard deviation of the mean”

Type B Evaluation

Any evaluation that is not a Type A evaluation is a Type B evaluation

Examples: Calculate Poisson counting uncertainty as the

square root of the observed count Use professional judgment to estimate the

maximum possible error in the value, then divide by sqrt(3) or some other constant

Covariance

Correlations between input estimates affect the uncertainty of the output estimate

The estimated covariance of two input estimates, xi and xj, is denoted by u(xi,xj)

The estimated correlation coefficient is denoted by r(xi,xj)

See the MARLAP text, the GUM, or Ken Inn for more information

Uncertainty Propagation

Recall that the output estimate, y, is given by

y = f(x1,x2,…,xN) The following equation shows how the standard

uncertainties and covariances of input estimates are propagated to produce the standard uncertainty of the output estimate

1

1 1

2

2

1c ),(2)()(

N

i

N

ijji

jii

N

i i

xxux

f

x

fxu

x

fyu

Combined Standard Uncertainty

The standard uncertainty of y obtained by uncertainty propagation is called the combined standard uncertainty

Notice that it is denoted here by uc(y), not u(y)

The subscript “c” means “combined”

Sensitivity Coefficients

Each partial derivative is called a sensitivity coefficient

It equals the partial derivative of the function f(X1,X2,…,XN) with respect to Xi , evaluated at X1=x1, X2=x2, …, XN=xn

It represents the sensitivity of y to changes in xi, or the ratio of the change in y to a small change in xi

ixf /

Uncertainty Propagation

All the standard uncertainties of the input estimates are treated alike for purposes of uncertainty propagation

We do not distinguish between Type A uncertainties and Type B uncertainties when we propagate them

The “Law of Propagation of Uncertainty”?

The GUM calls the generalized equation for the combined standard uncertainty the “law of propagation of uncertainty”

MARLAP prefers the less grandiose name “uncertainty propagation formula”

It’s not a “law” – just a first-order approximation formula

Uncertainty Propagation Formula

The uncertainty propagation formula looks intimidating to most people

If you learn examples of particular applications of it, you may be able to use them in many or most situations

If you want to be able to handle any model thrown at you, either you need to know calculus or you need software for automatic uncertainty propagation

Examples

If x1 and x2 are uncorrelated, then

22

222

2

122

12

2

1c

222

1221

221c

22

12

21c

)()(

)()(

)()()(

x

xu

x

x

x

xu

x

xu

xuxxxuxxu

xuxuxxu

Expanded Uncertainty

One may choose to multiply the combined standard uncertainty, uc(y), by a number k, called the coverage factor to obtain the expanded uncertainty, U

The expanded uncertainty is intended to produce an interval about the result that has a high probability of containing the (true) value of the measurand

That probability, p, is called either the coverage probability or the level of confidence

Expanded Uncertainty

Traditionally we have called expanded uncertainties “two-sigma” or “three-sigma” uncertainties

For any number k > 1, what we have called a “k-sigma” uncertainty is an expanded uncertainty with coverage factor k

Expanded Uncertainty

Reporting an expanded uncertainty, especially with k=2, usually suggests that you believe the result has a distribution that is approximately normal When k=2, you are implying that the coverage

probability is about 95 % What are you implying if you use k=1.96?

But reporting only the combined standard uncertainty (an estimated standard deviation) does not imply any particular distribution or coverage probability

Terms to Remember

Measurand, Y Mathematical model of measurement

Y = f(X1,X2,…,XN) Input quantities Xi, output quantity Y Input estimates xi, output estimate y Standard uncertainty, u(xi) Estimated covariance, u(xi,xj) Estimated correlation coefficient, r(xi,xj)

Terms - Continued

Propagation of uncertainty Combined standard uncertainty, uc(y) Coverage probability, or level of

confidence, p Coverage factor, k Expanded uncertainty, U = k × uc(y)

Part 2

Uncertainty in the Radiochemistry Lab

Counting Error

Well, first of all, MARLAP calls it “counting uncertainty,” not “counting error”

We define it as the component of the combined standard uncertainty of the result due to the randomness of radioactive decay [and radiation emission] and radiation counting

It’s only a portion of the total uncertainty of a measurement

Counting Uncertainty

We admit that one can often evaluate the standard uncertainty of a total count, n, by taking the square root of n

It is a convenient Type B method of evaluation, which doesn’t require repeated measurements

It is based on the assumption that n has a Poisson distribution, which may not always be a good assumption

Again, counting uncertainty is only a portion of the total uncertainty of the final result

Non-Poisson Example

One of the best examples of non-Poisson counting statistics comes from alpha-counting 222Rn and its progeny in a Lucas cell

An atom of 222Rn may produce more than one count as it decays through a series of short-lived states from 222Rn to 210Pb

Counts tend to occur in groups The counting uncertainty of n is usually larger

than sqrt(n)

When n is Small

If the Poisson model is valid, and if n, the number of counts, can assume values close to or equal to zero, we recommend evaluating the counting uncertainty as sqrt(n+1), not sqrt(n)

Otherwise you may end up reporting results sometimes as 0 ± 0

Other Uncertainties

MARLAP provides guidance about other uncertainty components

The guidance is intended to be helpful, not prescriptive, and certainly not complete

We deal with uncertainties for volume and mass measurements, which are relatively easy to handle but which also tend to be relatively insignificant

Laboratory Subsampling

We also deal with an uncertainty that is neither insignificant nor easy to handle: the uncertainty associated with subsampling heterogeneous solid material for analysis

Appendix F presents some highlights of Pierre Gy’s sampling theory as it applies to subsampling for radiochemical analysis

We recommend that labs not ignore subsampling uncertainty, although it is hard to evaluate well

Subsampling - Continued

We provide a reasonably simple equation for evaluating the standard uncertainty due to subsampling, which you can use by default if you don’t have a better approach of your own

The equation (next slide) depends on the mass of the sample, the mass of the subsample, and the maximum particle diameter

The Equation

mL = mass of entire sample

mS = mass of subsample d = maximum particle diameter k = 0.4 g/cm3 by default u(FS) = relative standard uncertainty due to

subsampling

3

LSS

11)( dk

mmFu

Why This Equation?

The form of the equation is derived from Gy’s theory

The default value of k is somewhat arbitrary but should give OK results

The equation rightly punishes one for taking too small an aliquant for analysis or failing to grind a lumpy sample before subsampling

Other Uncertainties

Real time and live time Instrument background Radiochemical blank Calibration (detection efficiency) Half-life – easy but usually negligible Gamma-ray spectrometry (MARLAP

chooses to punt this one)

Part 3

Summary of Recommendations

Recommendations

Use the terminology, notation, and methodology of the GUM

Report all results – even if zero or negative – unless you believe they are invalid

Report either the combined standard uncertainty or the expanded uncertainty

Explain the uncertainty – in particular state the coverage factor for an expanded uncertainty

Recommendations- Continued

Consider all sources of uncertainty and evaluate and propagate all that are believed to be potentially significant in the final result

Do not ignore subsampling uncertainty just because it is hard to evaluate

Round the reported uncertainty to either 1 or 2 figures (we suggest 2) and round the result to match

Final Recommendation

Consider all the preceding recommendations to be severable

If you can’t do everything, do as much as you can

But at least use the GUM’s terminology and notation so that we all speak and write the same language

Questions?