pseudo-measurement simulations and bootstrap for the ......s. varet ( cea-dam-dif, ens cachan )...

Pseudo-measurement simulations and bootstrapfor the experimental cross-section covariances

estimation with quality quantification

S. Varet1

P. Dossantos-Uzarralde1 N. Vayatis2 E. Bauge1

1CEA-DAM-DIF

2ENS Cachan

WONDER-2012 September 25-28

S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER-2012 1 / 29

Motivations: experimental cross-sections

12 13 14 15 16 17 18 19 200.4

0.5

0.6

0.7

0.8

0.9

1

1.1

Energy (MeV)

n2n

cros

s se

ctio

n

2555Mn n2n cross section

EXFOR data

Available informations:the measurementstheir uncertainty

→ covariance matrix estimation and its inverse

Example

Evaluated cross sections uncertainty: generalized χ2 ([VDUVB11])


Motivations: experimental cross-sections covariances

HOW?

Empirical estimator: only one measurement per energy

cov(Fi, Fj) ≈1n

n∑k=1

(F(k)i − F̄i)(F(k)j − F̄j)

Conventionnal approach: propagation error formula [IBC04],[Kes08]



HOW?

Empirical estimator: only one measurement per energyConventionnal approach: propagation error formula [IBC04],[Kes08]

cov(Fi, Fj) ≈∑

param. exp. (qk, ql)

∂Fi∂qk

.∂Fj∂ql

.cov(qk, ql)

ex. : fission fragments yield, recoil protons yield,

→ the needed informations are rarely available→ linearity assumption



HOW?

Empirical estimator: only one measurement per energyConventionnal approach: propagation error formula [IBC04],[Kes08]

QUALITY OF THE COVARIANCES ESTIMATION?


Outline

1 Notations

2 Experimental covariances estimation: new method

3 Validation

4 Quality measure of the obtained estimation

5 Conclusion


Notations

Outline

1 Notations


3 Validation


5 Conclusion


Notations

Notations

Schematic representation of the experimental cross sections


Experimental covariances estimation: new method

Outline

1 Notations


3 Validation


5 Conclusion



Pseudo-measurements method

Given the available information (the measurements and theiruncertainty)

The idea:

Repeat the available information in order to artificially increase thenumber of measurements

The method:

1 Construction of a regression model h (SVM, polynomial,...)2 Generation of r pseudo-measurements (S(1), ..., S(r)): gaussian

noise centered on h: N ((h(E1), ..., h(EN))t, diag(σ1, ..., σN))3 Σ̂F: empirical estimator of ΣF4 Diagonal terms are imposed



The SVM regression principle [SS04], [VGS97]

Linear case:Assume Fi = h(Ei) = w.Ei + b with Ei ∈ Rs and w ∈ Rs (here s = 1).The svm regression model is a solution of the constraint optimisationproblem:

flat function: minimize12‖w‖2

errors lower than ε: subject to{

Fi − (w.Ei + b) ≤ ε(w.Ei + b)− Fi ≤ ε

Non linear case:Find a map of the initial space of energy, (into a higher dimensionnalspace) such as the problem becomes linear in the new space(mapping via Kernel).




Given the available information (the measurements and theiruncertainty)

The idea:

Repeat the available information in order to artificially increase thenumber of measurements

The method:

1 Construction of a regression model h (SVM, polynomial,...)2 Generation of r pseudo-measurements (S(1), ..., S(r)): gaussian

noise centered on h: N ((h(E1), ..., h(EN))t, diag(σ1, ..., σN))3 Σ̂F: empirical estimator of ΣF4 Diagonal terms are imposed




Σ̂F term at the ith line and jth column, for i and j ∈ {1, ..., N}:

Cij =σiσjn + r

n∑k=1

(F(k)i − h(Ei))(F(k)j − h(Ej))√

V̂i√

V̂j

+σiσjn + r

r∑k=1

(S(k)i − h(Ei))(S(k)j − h(Ej))√

V̂i√

V̂j

and

Cii = σ2i

where n = 1.



Number r of pseudo-measurements?

Constraint on r

0

r too small:Σ̂F non invertible

r too large:Σ̂F diagonal matrix

r = 0

Σ̂F invertible Σ̂F non invertible

no pseudo-measurements r such as Σ̂F invertible


Validation

Outline

1 Notations


3 Validation


5 Conclusion


Validation

Toy model

Goal:Compare Σ̂F with ΣF: real case: ΣF is unknown

⇒ we build a toy model with ΣF fixed ⇒ numerical validation

Toy Model:

M :=

M1

...MN

(≡ F) ↪→ NN(mM,ΣM)number of M realisations n = 1, N = 5

θ =

[email protected]−3.49−0.67−7.41−2.29

1CCCCA ΣF =0BBBB@

9.49 −1.20 2.39 −0.03 0.07−1.20 3.64 −3.21 0.03 −0.092.39 −3.21 5.27 0.48 0.10−0.03 0.03 0.48 6.79 0.010.07 −0.09 0.10 0.01 5.72

1CCCCA


Validation

Pseudo-measurements with the ’Toy Model’

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5−20

−15

−10

−5

0

5

10

Variable (ex : energy)

Mea

sure

men

ts (e

x : X

s)

SVM regression on the toy model

mM


Validation


0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5−20

−15

−10

−5

0

5

10


Mea

sure

men

t (ex

: X

s)


mM

Toy measurement


Validation


0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5−20

−15

−10

−5

0

5

10


Mea

sure

men

t (ex

: X

s)


SVM regressionmM

Toy measurement


Validation


0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5−20

−15

−10

−5

0

5

10


Mea

sure

men

ts (e

x : X

s)


SVM regressionmMToy measurementr=1 pseudo−measurement


Validation


N=5, r = 1 sample of pseudo-measurements

1 2 3 4 5

1

2

3

4

5

1 2 3 4 5

1

2

3

4

5−2

0

2

4

6

8

Real matrix Estimation


Validation

5525Mn: (n,2n) cross section

Values extracted from [MTN11]N=11, r=1, Σ̂F is a 11x11 matrix

12 13 14 15 16 17 18 19 200.4

0.5

0.6

0.7

0.8

0.9

1

1.1

Energy (eV)

Cro

ss s

ectio

n (m

b)

SVM modelExperimental cross sectionsPseudo−measurements

1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

8

9

10

11 −0.4

−0.2

0

0.2

0.4

0.6

0.8


Validation

5525Mn: total cross section

Values extracted from EXFOR [EXF]N=399, r=1, Σ̂F is a 399x399 matrix

0 5 10 15x 106

2.5

3

3.5

4

Energy (eV)

Cro

ss s

ectio

n (m

b)

SVM modelExperimental dataPseudo−measurements

50 100 150 200 250 300 350

50

100

150

200

250

300

350

−0.01

−0.005

0

0.005

0.01

0.015

0.02

0.025

How far is the estimation from the real matrix?


Quality measure of the obtained estimation

Outline

1 Notations


3 Validation


5 Conclusion



Estimator quality

Criterion choice: quality of Σ̂F and/or quality of Σ̂F−1

?⇒ distance criterion

1 Distance between Σ̂F and ΣF: Frobenius norm

IE(‖Σ̂F − ΣF‖fro) = IE

N∑

i=1

N∑j=1

|Ĉij − Cij|2 12

2 Distance between Σ̂F

−1and Σ−1F : Kullback-Leibler distance [LRZ08]

KL(Σ̂F,ΣF) = trace((Σ̂F)−1.ΣF)− log(det(Σ̂F−1

.ΣF))− N

Criterion estimation → parametric bootstrap



Bootstrap principle

F : θF, ΣF: unknown

(F(1), ..., F(n)) : h, Σ̂F: sample measurements

(F′(1), ..., F′(n))1, ...,(F′(1), ..., F′(n))B resample measurements

̂̂ΣF1 ̂̂ΣFBResampling allows to make statistics on Σ̂F:

ÎE(Σ̂F), V̂ar(Σ̂F), B̂ias(Σ̂F),...

Resample strategy:classical: draw with replacement in the initial sample (F(1), ..., F(n))parametric: simulations with a fixed law (here N (h, Σ̂F))



Estimation of IE(‖Σ̂F − ΣF‖fro) and KL(Σ̂F, ΣF)

1st case: with r = 0, Σ̂F invertibleUsual parametric bootstrap:(F′(1), ..., F′(n))1, ..., (F′(1), ..., F′(n))B where F′(i) ↪→ N (H, Σ̂F) withH = (h(E1), ..., h(EN))t.ccΣFb term at the ith line and jth column, for i and j ∈ {1, ..., N} and for b = 1, ..., B:

Cbij =σiσj

n

nXk=1

(F′(k)bi − h(Ei))(F′(k)bj − h(Ej))q bV ′i q bV ′j and C

bii = σ

2i

IE(‖Σ̂F − ΣF‖) ≈1B

B∑b=1

‖̂̂ΣFb − Σ̂F‖KL(Σ̂F,ΣF) ≈

1B

B∑b=1

KL(̂̂ΣFb, Σ̂F)S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER-2012 21 / 29


Estimation of IE(‖Σ̂F − ΣF‖fro) and KL(Σ̂F, ΣF)

2nd case: with r = 0, Σ̂F non invertible2 simultaneous parametric bootstraps:(F′(1), ..., F′(n))1, ..., (F′(1), ..., F′(n))B where F′(i) ↪→ N (H, cΣF) et(S′(1), ..., S′(r))1, ..., (S′(1), ..., S′(r))B where S′(i) ↪→ N (H, diag(cΣF))ccΣFb term at the ith line and jth column, for i and j ∈ {1, ..., N} and for b = 1, ..., B:Cbij =

σiσjn + r

nXk=1

(F′(k)bi − h(Ei))(F′(k)bj − h(Ej))q bV ′i q bV ′j +

σiσjn + r

rXk=1

(S′(k)bi − h(Ei))(S′(k)bj − h(Ej))q bV ′i q bV ′j

and Cbii = σ2i

IE(‖Σ̂F − ΣF‖) ≈1B

B∑b=1

‖̂̂ΣFb − Σ̂F‖KL(Σ̂F,ΣF) ≈

1B

B∑b=1

KL(̂̂ΣFb, Σ̂F)S. Varet ( CEA-DAM-DIF, ENS Cachan ) Experimental covariances WONDER-2012 22 / 29


Bootstrap on the ’Toy Model’

N = 5, n = 1, B = 30, ‖ΣF‖ = 15.6680, r = 1,

0 2 4 6 8 10 1210

12

14

16

18

20

Index of the 1−sample of measures

Em

piric

al m

ean

and

varia

nce

of

||C2 F

− Σ F

|| its

boo

tstra

p es

timat

ion

Empirical mean and variance of||C2F − ΣF|| its bootstrap estimation (B=30).

||C2F − ΣF||

Σb=1B ||C2F

b − C2F||/B



Bootstrap on the ’Toy Model’

N = 5, n = 1, B = 30, ‖ΣF‖ = 15.6680, r = 1,

0 2 4 6 8 10 1210

12

14

16

18

20


Em

piric

al m

ean

and

varia

nce

of

||C2 F

− Σ F

|| its

boo

tstra

p es

timat

ion

Empirical mean and variance of||C2F − ΣF|| its bootstrap estimation (B=30).

||C2F − ΣF||

Σb=1B ||C2F

b − C2F||/B

0 2 4 6 8 10 122

3

4

5

6

7


Em

piric

al m

ean

and

varia

nce

of K

L an

d its

boo

tstra

p es

timat

ion

Empirical mean and variance of KL and its bootstrap estimation (B=30).

KLKL boot



5525Mn: (n,2n) cross section [MTN11]

12 13 14 15 16 17 18 19 200.4

0.5

0.6

0.7

0.8

0.9

1

1.1

Energy (eV)

Cro

ss s

ectio

n (m

b)

SVM modelExperimental cross sectionsPseudo−measurements

1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

8

9

10

11 −0.4

−0.2

0

0.2

0.4

0.6

0.8

IE(‖Σ̂F − ΣF‖) ≈ 0.019 IE(KL(Σ̂F)) ≈ 6.834Var(‖Σ̂F − ΣF‖) ≈ 1.2 10−5 Var(KL(Σ̂F)) ≈ 2.351



5525Mn: total cross section

0 5 10 15x 106

2.5

3

3.5

4

Energy (eV)

Cro

ss s

ectio

n (m

b)

SVM modelExperimental dataPseudo−measurements

50 100 150 200 250 300 350

50

100

150

200

250

300

350

−0.01

−0.005

0

0.005

0.01

0.015

0.02

0.025

IE(‖Σ̂F − ΣF‖) ≈ 2.385 IE(KL(Σ̂F)) ≈ 386.36Var(‖Σ̂F − ΣF‖) ≈ 4.3 10−5 Var(KL(Σ̂F)) ≈ 6.19


Conclusion

Outline

1 Notations


3 Validation


5 Conclusion


Conclusion

Conclusion

ΣF estimation

1 Classical approaches

X one measurement per energyX experimental informations rarely availableX quite unrealistic assumptions

2 Our approach:

X only measurements are neededX quality measure of the estimation

Estimator quality

Numerical validation

Application


Conclusion

Conclusion

ΣF estimation

Estimator quality

Estimation of ‖bΣF − ΣF‖ with BootstrapEstimation of KL(bΣF) with Bootstrap


Application


Conclusion

Conclusion

ΣF estimation

Estimator quality


Toy model

Application


Conclusion

Conclusion

ΣF estimation

Estimator quality


Application

Application to 5525Mn


Conclusion

Future work

New approach

Optimisation: shrinkage approach (in progress)

Other approaches

Kriging: cf poster session


Conclusion

References

Experimental Nuclear Reaction Data EXFOR, http://www.oecd-nea.org/dbdata/x4/.

M. Ionescu-Bujor and D. G. Cacuci, A comparative review of sensitivity and uncertaintyanalysis of large-scale systems −i: Deterministic methods, Nuclear Science andEngineering 147 (2004), 189–203.

Grégoire Kessedjian, Mesures de sections efficaces d’actinides mineurs d’intérêt pour latransmutation, Ph.D. thesis, Bordeaux 1, 2008.

E. Levina, A. Rothman, and J. Zhu, Sparse estimation of large covariance matrices via anested lasso penalty’, The Annals of Applied Statistics 2 (2008), no. 1, 245–263.

A. Milocco, A. Trkov, and R. Capote Noy, Nuclear data evaluation of 55 mn by the empirecode with emphasis on the capture cross-section, Nuclear Engineering and Design 241(2011), no. 4, 1071–1077.

A.J. Smola and B. Schölkopf, A tutorial on support vector regression, Statistics andComputing 14 (2004), 199–222.

S. Varet, P. Dossantos-Uzarralde, N. Vayatis, and E. Bauge, Experimental covariancescontribution to cross-section uncertainty determination, NCSC2 Vienna, 2011.

V. Vapnik, S. Golowich, and A. Smola, Support vector method for function approximation,regression estimation, and signal processing, Advances in Neural Information ProcessingSystems 9 (1997), 281–287.


NotationsExperimental covariances estimation: new methodValidationQuality measure of the obtained estimationConclusion

pseudo-measurement simulations and bootstrap for the ......s. varet ( cea-dam-dif, ens cachan )...

Documents