evaluation of measurement uncertainties using the monte carlo method

1Standards and Calibration Laboratory, SCL

Evaluation of Measurement Uncertainties Using the

Monte Carlo Method

Speaker: Chung Yin, PoonStandards and Calibration Laboratory (SCL)The Government of the Hong Kong Special

Administrative Region


GUM Uncertainty Framework (GUF)

“Propagation of Uncertainties”Measurement Model: Y= f(X1, X2, XN)Estimate xi of the input quantities Xi

Determine u(xi) associated with each estimate xi and its degrees of freedom

Estimate y = f(xi) of YCalculate the sensitivity coefficient of each xi at Xi = xi

Calculate u(y)Calculate the effective degrees of freedom veff and coverage factor k with

coverage probability pCalculate the coverage interval: yku(y)


GUM Uncertainty Framework (GUF)

Problems:The contributory uncertainties are not of

approximately the same magnitudeDifficult to provide the partial derivatives of the

modelThe PDF for output quantity is not a Gaussian

distribution or a scaled and shifted t-distribution

Y = f (X) y, u(y)

x1, u(x1)

x2, u(x2)

xN, u(xN)


Monte Carlo Method (MCM)“Propagation of Distributions”Measurement Model: Y= f(X1, X2, XN)Assign probability density function (PDF) to each XSelect M for the number of Monte Carlo trialsGenerate M vectors by sampling from the PDF of each X

(x1,1, x1,2, x1,M) (xN,1, xN,2, xN,M) Calculate M model values y = (f(x1,1, xN,1), f(x1,M, xN,M))Estimate y of Y and associated standard uncertainty u(y)Calculate the interval [ylow,yhigh] for Y with corresponding coverage

probability p


Monte Carlo Method (MCM)

Y = f (X)gY(h)

gX1(x1)

gX2(x2)

gXN(xN)


Operation Modes For MCM

• There are three modes of operations– Fixed-Number-of-Trials Mode– Adaptive Mode– Approximated Adaptive (or Histogram) Mode


Adaptive Monte Carlo ProcedureSet number of significant digit

ndig (usually = 2)

Set M = max (100/(1-p), 104)

h = 1

Perform MCM trial

y(h)=(y1(h),y2

(h),...yM(h))

Calculate u(y(h)), ylow(h), yhigh

(h)

h > 1 ?

h = h + 1

Calculate sy, su(y), syhigh, sylow

Use h M model values calculate u(y) and then d

Any 2sy, 2su(y), 2sylow or 2syhigh > d/5?

Use h M model values calculate y,

u(y), ylow, yhighN

Y

Y

N


Validation of GUF

• Calculate: dlow = y – Up – ylow anddhigh = y + Up – yhigh

• If both differences are not larger than d, then the GUF is validated.


Histogram Procedure

• If the numerical tolerance d is small, the value of M required would be larger. This may causes efficiency problems for some computers

• Experiences show that a very precise measurement will require a M of up to 107

• Using histogram to approximate the PDF


Histogram Procedure

1. Build the initial histogram for y withBin = 100,000

2. Continue generate the model Update y and u(y) for each iteration Check stabilization. (Same as the adaptive

procedure, i.e. check the four s values) Update the histogram Store the outliers (i.e. those values beyond the

boundaries of the histogram)


Histogram Procedure

4. When stabilized: Build complete histogram to include the

outliers Transform the histogram to a

distribution function Use this discrete approximation to

calculate the coverage interval


Determine Coverage Intervals• By Inverse linear interpolation [Annex D.5 to D.8 of GS1]

y(r)

ylow

y(r+1)

pr

pr+1

a

y

p GY(h)

y(s)

yhigh

y(s+1)

y

p GY(h)

ps+1

ps

p+a


Shortest Coverage Interval

• Repeat the method to determine a large number of intervals corresponding to(a, p+a) and find the minimum value.E.g. a = 0 to 0.05 for 95 % coverage interval.

• The precision level is related to the incremental step of a in the search.

• The step uses in this software is 0.0001, i.e. total 501 steps.


MCM Software

MCM Code Generator

(VB Codes)

MATLAB Script File

MCM Engine(MATLAB Codes)

- Perform MCM Procedure-GUF Validation

-Graphical Output of Results


GUI of the MCM Code Generator


Results for example 9.4.3.2 of GS1

• PDF for the y values in histogram• GUF Gaussian/t-distribution• Coverage Intervals• MCM and GUF results for y, u(y), ylow and yhigh

• GUF validation result• Number of MCM trials


ExampleCalibration of a 10 V Zener Voltage Reference using

Josephson Array Voltage Standard

• Measurement Model:• PDF parameters input to the software:

ranmOLJ

VVVVKfny

Input Quantity PDF ParameterSymbol Description PDF /

Constant v a b

n Quantum (Step) Number

constant 63968

f Frequency N(, 2) 75.6 GHz 5.13 Hz

KJ Josephson Constant

constant 483597.9 GHz/V

VL Leakage R(a,b) -5 nV 5 nV

VO Offset R(a,b) -0.1 V 0.1 V

Vm Null Voltage R(a,b) 3.722 V 3.712 V 3.732 V

Vran Random Noise tv(, 2) 0 V 30 nV 39


Parameters Input to the MCM Code Generator


Results

Method y (V) u(y) (nV) ylow (nV) yhigh (nV)GUF 10 67 -131 +131

MCM1(Fixed number)

10 66 -120 +120

MCM2(Adaptive)

10 66 -120 +120

MCM3(Histogram)

10 66 -120 +120

Method dlow(nV)

dhigh(nV)

GUF validated?

No. of Trials Computation Time (s)

MCM1(Fixed number)

-11 +11 No 1,000,000 < 2

MCM2(Adaptive)

-11 +11 No 6,210,000 89

MCM3(Histogram)

-11 +11 No 6,270,000 8

Computer Configurations:Windows XP; MATLAB R2008b (version 7.7); CPU: Core Due T5600, 1.83 GHz, 2 GB Ram, 80 GB Harddisk


Thank You

evaluation of measurement uncertainties using the monte carlo method

Documents