1

Determination of KCRV, u(KCRV) & DoE

Final Proposal by KCWG to CCRI(II)

Stefaan Pommé

John Keightley

CCRI(II) Meeting, BIPM, 14-16 May 2013

2

arithmetic weighted

Mandel-Paule Power Moderated Mean

Estimators for mean

3

N

i

ixN

x1

ref

1

N

i

i Nxxxu1

2

refref

2 ).1/()()(

Arithmetic mean

4

101

102

103

104

105

106

107

108

109

0 1 2 3 4 5 6 7 8 9 10

kB

q /

g

Good: ignore wrong unc

5

biased mean

large uncertainty

= not “efficient”

63

68

73

78

83

88

93

98

103

0 1 2 3 4 5

kB

q / g

Bad: inefficient

6

86

91

96

101

106

111

116

0 1 2 3 4 5

kB

q / g

low uncertainty

Bad: low sample variance

7

Remedy against low unc

N

i

iN

ii

NN

xxu

Nxu

1

2

1

2

)1(

)(,

1max)(

Calculate uncertainty from maximum of

- propagated sum of stated uncertainties

- sample variance

8

- measurement uncertainty contains no useful

information

- magnitude error on uncertainty is >2x larger than

magnitude due to metrological reasons

Best solution is close to arithmetic mean if

= the “best” with “bad uncertainty data”

= inefficient with “consistent data”

Arithmetic mean

9

statistical weight = reciprocal variance associated with xi

N

i i

i

u

xxux

12ref

2

ref )(

N

i iuxu 12

ref

2

1

)(

1

Weighted mean

10

63

68

73

78

83

88

93

98

103

0 1 2 3 4 5

kB

q / g

efficient

Good: efficient

11

101

102

103

104

105

106

107

108

109

0 1 2 3 4 5 6 7 8 9 10

kB

q / g

biased mean

low uncertainty

discrepant data set

Bad: sensitive to error

12

- measurement uncertainties are correct

- ‘value’ and ‘uncertainty’ outliers are excluded

Best solution is close to weighted mean if

= the “best” with “perfect data”

= sensitive to “low uncertainty” outliers

Weighted mean

13

,)(1

22ref

2

ref

N

i i

i

su

xxux

N

i i suxu 1,22

ref

2

1

)(

1

s2 is artificially added “interlaboratory” variance to make reduced chi = 1

.)(

1

1~

12

2ref

obs

N

i i

i

u

xx

N

Mandel-Paule mean

14

86

91

96

101

106

111

116

0 1 2 3 4 5

kB

q / g

= weighted mean for a consistent data set

Mandel-Paule mean

15

101

102

103

104

105

106

107

108

109

0 1 2 3 4 5 6 7 8 9 10

kB

q / g

≈ arithmetic mean for an extremely discrepant data set

Mandel-Paule mean

16

63

68

73

78

83

88

93

98

103

0 1 2 3 4 5

kB

q / g

= intermediate between arithmetic and weighted

mean for a slightly discrepant data set

Mandel-Paule mean

17

- measurement uncertainties are informative

- ‘value’ and ‘uncertainty’ outliers are symmetric

Best solution is close to Mandel-Paule mean if

= one of the “best” with “imperfect data”

if no tendency to underestimate uncertainty

Mandel-Paule mean

18

S is a typical uncertainty per datum (max arithmetic or M-P unc)

0<a<2 = power reflects level of trust in uncertainties

Power Moderated Mean - PMM

))(),(max( mp

22 xuxuNS

N

iii xwx

1ref

1

222

ref

2 )(

a

a

Ssuxuw ii

19

= Mandel-Paule mean N

73

78

83

88

93

98

103

108

113

0 1 2 3 4

kB

q /

g

‘understated’ uncertainty?

PMM: a=2

20

N

73

78

83

88

93

98

103

108

113

0 1 2 3 4

kB

q /

g

= closer to arithmetic mean

less weight to this point

PMM: a=1

21

Choice of power a

power reliability of uncertainties

a = 0 uninformative uncertainties

(arithmetic mean)

a = 0 uncertainty variation due to error at least twice

the variation due to metrological reasons

(arithmetic mean)

a = 2-3/N informative uncertainties with a tendency of being

rather underestimated than overestimated

(intermediately weighted mean)

a = 2 informative uncertainties with a modest error;

no specific trend of underestimation

(Mandel-Paule mean)

a = 2 accurately known uncertainties, consistent data

(weighted mean)

22

arithmetic weighted

Mandel-Paule Power Moderated Mean

Test of Estimators by Computer Simulation

23

Efficiency for discrepant data

arithmetic < , M-P < weighted

0 5 10 15 20

1.0

1.1

1.2

1.3

1.4

1.5

AM

M-P

PMM

WM

sta

nd

ard

de

via

tio

n

N

EFFICIENCY

24

Reliability of uncertainty

weighted << M-P < < arithmetic

0 5 10 15 20

0.5

1.0

1.5

2.0

2.5

3.0

RELIABILITY

AM

M-P

PMM

WM

un

ce

rta

inty

/ s

tan

da

rd d

evia

tio

n

N

25

- measurement uncertainties are informative

- uncertainties tend to be understated

- data seem consistent but are not

Best solution is close to PMM if

= one of the “very best” with “imperfect data”

= more realistic uncertainty than Mandel-Paule mean

= more adjustable to quality of data than M-P mean

Power Moderated Mean

26

generally applicable method

Outlier identification

27

CCRI(II) is the final arbiter regarding correcting or

excluding any data from the calculation of the KCRV.

Statistical tools may be used to indicate data

that are extreme.

= a way to protect the KCRV against erroneous data,

data with understated uncertainty, extreme data

asymmetrically disposed to the KRCV

= a way to lower the uncertainty on the KCRV

Outlier identification

28

- valid for any type of mean, using normalised wi

- default k = 2.5

Outlier identification

)1

1)(( )( ref

22 i

iw

xueu

)1

1)(()( ref

22 i

iw

xueu

xi included in mean

xi excluded from mean

|ei| > ku(ei), ei = xi – xref

29

Both options are possible within the method.

→ outlier rejection should be based on technical grounds

Outlier or not?

L

63

68

73

78

83

88

93

98

103

0 1 2 3 4 5

kB

q /

g

L

63

68

73

78

83

88

93

98

103

0 1 2 3 4 5

kB

q /

g

30

generally applicable method

Degree of equivalence

31

- valid for any type of mean

Degree of equivalence

xi included in mean

xi excluded from mean

di = xi – xref, U(di) = 2u(di)

)()21()( ref222 xuuwdu iii

)()( ref222 xuudu ii

32

The Power Moderated Mean keeps a fine balance between efficiency and robustness, while providing also a reliable uncertainty.

Outlier identification and degrees of equivalence are readily obtained.

Conclusions