Response surface methodology for functional data withapplication to nuclear safety

Angelina Rochejoint work with Michel Marques (CEA Cadarache)

47e Journées de Statistique1-5 juin 2015

1 / 19

Outline

1 Motivation

2 Response surface methodology

3 Extension to the functional setting

4 Application to nuclear safety

2 / 19

Motivation

Outline

1 Motivation

2 Response surface methodology

3 Extension to the functional setting

4 Application to nuclear safety

3 / 19

Motivation

Motivation : application to nuclear safety

Behaviour of a nuclear reactor vessel in case of loss of coolant accident.

4 / 19

Motivation

Motivation : application to nuclear safety

Thermo-mecanical code (CEA) : simulate the behaviour of the vessel.

Input:Evolution of the temperature in the vessel t 7→ T(t);Evolution of the pressure t 7→ P(t);Evolution of heat transfer t 7→ H(t).

Output: margin factor Y ∈ R.

Aim: maximise the margin factor Y .

4 / 19

Response surface methodology

Outline

1 Motivation

2 Response surface methodology

3 Extension to the functional setting

4 Application to nuclear safety

5 / 19

Response surface methodology

Response surface methodologyBrief history

Box and Wilson (1950): optimal conditions for chemical experimentation→widely used in industry.

Sacks et. al (1989): Extension to numerical experiments

↪→ Bates et. al (1996): conception of electrical circuit;

↪→ Lee and Hajela (1996): conception of rotor blades...

Recent advances: Facer and Müller (2003), Khuri and Mukhopadhyay (2010),Georgiou, Stylianou and Aggarwal (2014).

6 / 19

Response surface methodology

Methodology

Goal: minimisation of (x1, . . . , xd) ∈ Rd 7→ m(x1, . . . , xd) ∈ R, unknown.

Additional experiments possible but with a cost, output observed with errory = m(x1, ..., xd) + ε.

Idea1 Definition of a departure point (x0,1, ..., x0,d) ∈ Rd and of a surrogate model

Order 1 (steepest ascent): y = b0 +∑d

j=1 bjxj + ε;

Order 2 (optimisation model): y =∑d

j=1 bjxj +∑

1≤i<j≤d bi,j +∑d

j=1 bi,ix2j + ε;

...

2 Realisation of new experiments around (x0,1, ..., x0,d) ∈ Rd ↪→ choice of Design ofExperiments (DoE).

3 Least-squares fit of the model coefficient and optimisation : definition of a new point(x1,1, ..., x1,d).

7 / 19

Extension to the functional setting

Outline

1 Motivation

2 Response surface methodology

3 Extension to the functional setting

4 Application to nuclear safety

8 / 19

Extension to the functional setting

Problems raised by the functional context

First and second-order models can be defined easily but... How to define functional design of experiments ?One possible answer: combine dimension reduction with classicalfinite-dimensional design of experiments

(x(i)0 = (x(i)

0,1, . . . , x(i)0,d) ∈ Rd, i = 1, . . . , n0) d-dimensional design of experiments;

{ϕ1, . . . , ϕd} orthonormal family of H

x(i)0 = x0 +

d∑j=1

x(i)0,jϕj,

−→ functional design of experiments.... How can we define the directions {ϕ1, . . . , ϕd} ?Possible basis of approximation

Fixed basis: Fourier, B-splines, wavelets,...If a training sample exists: data driven basis

PCA basis;PLS basis Wold (1975), Preda and Saporta (2005), Delaigle and Hall (2012): allows totake into account the interaction between x and y.

Random directions...

9 / 19

Extension to the functional setting

Problems raised by the functional context

First and second-order models can be defined easily but... How to define functional design of experiments ?One possible answer: combine dimension reduction with classicalfinite-dimensional design of experiments

(x(i)0 = (x(i)

0,1, . . . , x(i)0,d) ∈ Rd, i = 1, . . . , n0) d-dimensional design of experiments;

{ϕ1, . . . , ϕd} orthonormal family of H

x(i)0 = x0 +

d∑j=1

x(i)0,jϕj,

−→ functional design of experiments.... How can we define the directions {ϕ1, . . . , ϕd} ?Possible basis of approximation

Fixed basis: Fourier, B-splines, wavelets,...If a training sample exists: data driven basis

PCA basis;PLS basis Wold (1975), Preda and Saporta (2005), Delaigle and Hall (2012): allows totake into account the interaction between x and y.

Random directions...

9 / 19

Extension to the functional setting

Problems raised by the functional context

First and second-order models can be defined easily but... How to define functional design of experiments ?One possible answer: combine dimension reduction with classicalfinite-dimensional design of experiments

(x(i)0 = (x(i)

0,1, . . . , x(i)0,d) ∈ Rd, i = 1, . . . , n0) d-dimensional design of experiments;

{ϕ1, . . . , ϕd} orthonormal family of H

x(i)0 = x0 +

d∑j=1

x(i)0,jϕj,

−→ functional design of experiments.... How can we define the directions {ϕ1, . . . , ϕd} ?Possible basis of approximation

Fixed basis: Fourier, B-splines, wavelets,...If a training sample exists: data driven basis

PCA basis;PLS basis Wold (1975), Preda and Saporta (2005), Delaigle and Hall (2012): allows totake into account the interaction between x and y.

Random directions...

9 / 19

Extension to the functional setting

Example of functional design of experimentsFactorial 2d design in H = L2([0, 1])

d = 2, 16 curves d = 4, 32 curves d = 8, 280 curves

Fourier

0.0 0.2 0.4 0.6 0.8 1.0

−2−1

01

2

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−4−2

02

4

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−50

5

t

x i(t)

PCA1

0.0 0.2 0.4 0.6 0.8 1.0

−3−2

−10

12

3

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0−6

−4−2

02

46

t

x i(t)0.0 0.2 0.4 0.6 0.8 1.0

−10−5

05

10

t

x i(t)

PLS2

0.0 0.2 0.4 0.6 0.8 1.0

−2−1

01

2

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−6−4

−20

24

6

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−10−5

05

10

t

x i(t)

X brownian motion, Y = ‖X − f‖2 + ε, f (t) = cos(4πt) + 3 sin(πt) + 10, ε ∼ N (0, 0.01)

1calculated from (Xi)500i=1

2calculated from (Xi, Yi)500i=1

10 / 19

Extension to the functional setting

Example of functional design of experimentsCentral Composite Designs in H = L2([0, 1])

d = 2, 4 curves d = 4, 16 curves d = 8, 256 curves

Fourier

0.0 0.2 0.4 0.6 0.8 1.0

−2−1

01

2

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−4−2

02

4

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−50

5

t

x i(t)

PCA3

0.0 0.2 0.4 0.6 0.8 1.0

−3−2

−10

12

3

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0−6

−4−2

02

46

t

x i(t)0.0 0.2 0.4 0.6 0.8 1.0

−10−5

05

10

t

x i(t)

PLS4

0.0 0.2 0.4 0.6 0.8 1.0

−2−1

01

2

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−6−4

−20

24

6

t

x i(t)

0.0 0.2 0.4 0.6 0.8 1.0

−10−5

05

10

t

x i(t)

X brownian motion, Y = ‖X − f‖2 + ε, f (t) = cos(4πt) + 3 sin(πt) + 10, ε ∼ N (0, 0.01)

3calculated from (Xi)500i=1

4calculated from (Xi, Yi)500i=1

11 / 19

Extension to the functional setting

Properties of the functional DoE

For multivariate inputs, all polynomial models can be written in a matrix form

Y = Xβ + ε. (1)

The design properties are based on the matrix X.

Orthogonality: XtX is diagonal.

Rotatability : Var(y(x)) depends only on ‖x‖.

D-optimality criterion: det(XtX) is maximal....

With our definition of functional DoE, the model can also be written in theform (1), with the same design matrix X as the multivariate DoE.

12 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

−10

−50

510

1520

t

x(t)

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Initial point

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

−10

−50

510

1520

t

x(t)

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Initial pointMinimal point f (t) (target)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

−10

−50

510

1520

t

x(t)

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Initial pointMinimal point f (t) (target)28 factorial design5

5directions : PLS basis calculated from (Xi,m(Xi) + εi)500i=1 (Xi brownian motion)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

−10

−50

510

1520

t

x(t)

Least-squares fit of a firstorder model → estima-tion of direction of steep-est descent

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Initial pointMinimal point f (t) (target)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

−10

−50

510

1520

t

x(t)

Observed response on descent path:

0.0 0.2 0.4 0.6 0.8

01

03

05

0

α

ob

se

rve

d r

esp

on

se

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Initial pointMinimal point f (t) (target)Points of the descent direction

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

−10

−50

510

1520

t

x(t)

Observed response on descent path:

0.0 0.2 0.4 0.6 0.8

01

03

05

0

α

ob

se

rve

d r

esp

on

se

Minimal point of the descent direction

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Minimal point of the descent directionMinimal point f (t) (target)Points of the descent direction

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

05

1015

2025

t

x(t)

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Minimal point of the descent directionMinimal point f (t) (target)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

05

1015

2025

t

x(t)

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Minimal point of the descent directionMinimal point f (t) (target)

} Central Composite Design5

5directions : PLS basis calculated from (Xi,m(Xi) + εi)500i=1 (Xi brownian motion, d = 8)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

05

1015

2025

t

x(t)

Least-squares fit of a sec-ond order model → esti-mation of stationary point

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Minimal point of the descent directionMinimal point f (t) (target)

} Central Composite Design5

5directions : PLS basis calculated from (Xi,m(Xi) + εi)500i=1 (Xi brownian motion, d = 8)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

05

1015

2025

t

x(t)

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Minimal point of the descent directionMinimal point f (t) (target)Stationary point (estimation of the minimalpoint)

13 / 19

Extension to the functional setting

MethodologyAdaptation to a functional context

Goal: minimisation of x 7→ m(x), unknown.

0.0 0.2 0.4 0.6 0.8 1.0

46

810

1214

16

t

x(t)

0

12

0 100 200 300 400 500

02

04

0

number of experiments

va

lue

of

the

re

sp

on

se 0

1 2

Example:m(x) = ‖x− f‖2 with

f (t) = cos(4πt)+3 sin(πt)+10;

ε ∼ N (0, 10).

Legend:Step pointsMinimal point f (t) (target)

13 / 19

Extension to the functional setting

Choice of dimension d : 2d factorial design

Monte-Carlo study of response improvement m(x(0)0 )−m(x(1)

0 )

m(x(0)0 )

after the first descent step.

m(x) = ‖x− f‖2

f (t) = cos(4πt) + 3 sin(πt) + 10

2 3 4 5 6 7 8 9 10 11

98.5

99.0

99.5

100.

0

PLS

f (t) = cos(8.5πt) ln(4t2 + 10)

2 3 4 5 6 7 8 9 10 11

020

4060

8010

0

PLS

14 / 19

Extension to the functional setting

Choice of dimension d : 2d factorial design

Monte-Carlo study of response improvement m(x(0)0 )−m(x(1)

0 )

m(x(0)0 )

after the first descent step.

m(x) = ‖x− f‖2

f (t) = cos(4πt) + 3 sin(πt) + 10

2 3 4 5 6 7 8 9 10 11

98.5

99.0

99.5

100.

0

PLS

f (t) = cos(8.5πt) ln(4t2 + 10)

2 3 4 5 6 7 8 9 10 11

020

4060

8010

0

PLS

Exponential increase of the size of design with the dimension.

14 / 19

Extension to the functional setting

Choice of dimension d : fractional factorial design

Monte-Carlo study of response improvement m(x(0)0 )−m(x(1)

0 )

m(x(0)0 )

after the first descent step.

m(x) = ‖x− f‖2

f (t) = cos(4πt) + 3 sin(πt) + 10

24 2(5−1) 2(6−2) 2(7−3) 2(8−4)

99.2

99.4

99.6

99.8

100.0

factorial design

perce

ntage

of im

prove

ment

f (t) = cos(8.5πt) ln(4t2 + 10)

24 2(5−1) 2(6−2) 2(7−3) 2(8−4)

020

4060

factorial design

perce

ntage

of im

prove

ment

Generation of factorial designs : minimal aberration design (package FrF2, Grömping, 2014).

15 / 19

Application to nuclear safety

Outline

1 Motivation

2 Response surface methodology

3 Extension to the functional setting

4 Application to nuclear safety

16 / 19

Application to nuclear safety

Design of Experiments

Directions : PLS basis of training samples (Ti,Yi)1≤i≤n, (Pi,Yi)1≤i≤n,(Hi,Yi)1≤i≤n, i = 1, ..., n, n = 195.

Limited number of possible additional experiments (about 200) : we fix heren0 = 128 = 27.

Combination of multivariate designs :Temperature 210−5 factorial design;

Pressure 24−2 factorial design;

Heat transfer 23−2 factorial design.

Temperature Pressure Heat transfer

0 1000 2000 3000 4000 5000

5010

020

0

t(s)

Tem

pera

ture

Te(

t)

0 1000 2000 3000 4000 5000

5.0e

+06

1.5e

+07

t(s)

Pre

ssio

n P

(t)

0 1000 2000 3000 4000 5000

050

000

1000

0015

0000

t(s)Tr

ansf

ert t

herm

ique

H(t

)

17 / 19

Application to nuclear safety

Estimation of gradient

First-order surrogate model : Yi = 〈βT ,Ti〉+ 〈βP,Pi〉+ 〈βH,Hi〉+ εi

Least-squares estimate of βP, βT , βH .

Direction of steepest ascent : Ti = T0 + α0βT , Pi = P0 + α0βP andHi = H0 + α0βH .

0 1000 2000 3000 4000 5000

5010

015

020

025

030

0

temps

x0Te

100 200 300 400 500

1.45

1.50

1.55

1.60

α0

Val

ue o

f the

res

pons

e

estimated steepest ascent dir.estimation of the optimum

18 / 19

Conclusion and perspectives

Conclusion

Adaptation of response surface methodology to context where the input are curves.

Definition of functional DoE by combining multivariate DoE and dimensionreduction techniques.

Application to simulated data and to a thermo-mechanical code of the CEA.

PerspectivesSelection of dimension/coefficients :

Theoretical study of the minimisation procedure;

Sensitivity analysis.

Constrained DoE.

More complex surrogate models.

More information: Roche, A (2015). Response surface methodology forfunctional data with application to nuclear safety, prépublication MAP5 2015-11.

Thank you for your attention !

19 / 19