connection between inverse problems and uncertainty quantification problems

Inverse Problems and Uncertainty Quantification

Alexander Litvinenko, Hermann G. Matthies∗, Elmar Zander∗

Center for UncertaintyQuantification


Center for Uncertainty Quantification Logo Lock-up

http://sri-uq.kaust.edu.sa/

∗Institute of Scientific Computing, TU Braunschweig, Brunswick, Germany

http://sri-uq.kaust.edu.sa/

4*

KAUST

Figure : KAUST campus, approx. 7000 people (include 1400 kids),100 nations.




2 / 1

4*

KAUST

Figure : Points of my study and of my work.




3 / 1

4*

Overview of uncertainty quantification

ConsiderA(u; q) = f ⇒ u = S(f ; q),

where S is a solution operator.Uncertain Input:

1. Parameter q := q(ω) (assume moments/cdf/pdf/quantilesof q are given)

2. Boundary and initial conditions, right-hand side3. Geometry of the domain

Uncertain solution:1. mean value and variance of u2. exceedance probabilities P(u > u∗)3. probability density functions (pdf) of u.




4 / 1

4*

Example: Two realisations of random field q(x , ω)




5 / 1

An example:UQ in numerical aerodynamics

(described by Navier-Stokes + turbulence modeling)




6 / 1

4*

Example: uncertainties in free stream turbulence

α

v

v

u

u’

α’

v1

2

Random vectors v1(θ) and v2(θ) model free stream turbulence




7 / 1

4*

Example: UQ

Input parameters: assume that RVs α and Ma are Gaussianwith

mean st. dev.σ

σ/mean

α 2.79 0.1 0.036Ma 0.734 0.005 0.007

Then uncertainties in the solution (lift force and drag force) are

lift force 0.853 0.0174 0.02drag force 0.0206 0.003 0.146




8 / 1

4*

500 MC realisations of cp in dependence on αi and Mai




9 / 1

4*

Example: 3sigma intervals

Figure : 3σ interval, σ standard deviation, in each point of RAE2822airfoil for the pressure (cp) and friction (cf) coefficients.




10 / 1

4*

Part 2: Inverse Problems in Bayesian settings

Inverse Problems in Bayesian settings




11 / 1

4*

Mathematical setup

Consider

A(u; q) = f ⇒ u = S(f ; q),

where S is solution operator.Operator depends on parameter(s) q ∈ Q,

hence state u ∈ U is also function of q:Measurement operator Y with values in Y:

y = Y (q; u) = Y (q,S(f ; q)).

Examples of measurements:a) y(ω) =

∫D0

u(ω, x)dx , D0 ⊂ D, b) u in few points




12 / 1

4*

Inverse problem

For given f , measurement y is just a function of q.This function is usually not invertible⇒ ill-posed problem,

measurement y does not contain enough information.In Bayesian framework state of knowledge modelled in a

probabilistic way,parameters q are uncertain, and assumed as random.

Bayesian setting allows updating / sharpening of informationabout q when measurement is performed.

The problem of updating distribution—state of knowledge of qbecomes well-posed.

Can be applied successively, each new measurement y andforcing f —may also be uncertain.




13 / 1

4*

Conditional probability and expectation

With state u ∈ U = U ⊗ S a RV, the quantity to be measured

y(ω) = Y (q(ω),u(ω))) ∈ Y := Y ⊗ S

is also uncertain, a random variable.Noisy data: y + ε(ω) ∈ Y , where y is the “true” value and a

random error ε.Forecast of the measurement: z(ω) = y(ω) + ε(ω) ∈ Y .




14 / 1

4*

Conditional probability and expectation

Classically, Bayes’s theorem gives conditional probability

P(Iq|Mz) =P(Mz |Iq)

P(Mz)P(Iq),

where Iq some set of possible q’s, and Mz is the informationprovided by the measurements.

Expectation with this posterior measure is conditionalexpectation.

Kolmogorov starts from conditional expectation E (·|Mz),from this conditional probability via P(Iq|Mz) = E

(χIq |Mz

).




15 / 1

4*

Minimum Mean Square Error Estimation (MMSE)

Connection with Minimum Mean Square Error Estimation(MMSE)




16 / 1

4*

Minimum Mean Square Error Estimation (MMSE)

I X : Ω→ X (=: Rn) be the (a priori) stochastic model ofsome unknown QoI (e.g. our uncertain parameter)

I Y : Ω→ Y(=: Rn) be the stochastic model (e.g. ofmeasurement forecast)

I An estimator ϕ : Y → X is any (measurable) function fromthe space of measurements Y to the space of unknowns Xof corresponding measurements

I mean square error eMSE defined by

e2MSE = E[‖X (·)− ϕ(Y (·))‖22]. (1)

I Minimum mean square error estimator ϕ is the one thatminimises error eMSE. Further:

ϕ(Y ) = E[X |Y ], (2)




17 / 1

4*

Properties

I Minimising over the whole space of measurable functionsis numerically also not possible

I restrict to a finite dimensional function space Vϕ with basisfunctions Ψγ , indexed by some γ ⊂ J , J - a multi-index.

I Ψγ can be e.g. a multivariate polynomial set and the γcorresponding multiindices

I other function systems are also possible (e.g. tensorproducts of sines and cosines).

I ϕ has a representation

ϕ = y 7→∑γ∈J

ϕγΨγ(y). (3)




18 / 1

I Minimising (??) for Xi and ϕi gives

∂

∂ϕi,δE[(Xi −

∑γ∈J

ϕi,γΨγ(Y ))2] = 0 (4)

for all δ ∈ J .I Using linearity leads to∑

γ

ϕi,γE[Ψγ(Y )Ψδ(Y )] = E[XiΨδ(Y )]. (5)

I This can be written as a linear system

Aϕi = bi (6)

with [A]γδ = E[Ψγ(Y )Ψδ(Y )], [bi ]δ = E[XiΨδ(Y )] and thecoefficients ϕi,γ collected in the vector ϕi .




19 / 1

4*

Numerical computation of NLBU

Using the same quadrature rule of order q for each element ofA, we can write

Aγδ = E[Ψγ(Y )Ψδ(Y )T

]≈

NA∑i=1

wAi Ψγ(Y (ξi))Ψδ(Y (ξi))T , (7)

bδ = E[XΨδ(Y )] ≈Nb∑i=1

wbi X (ξi)Ψδ(Y (ξi)). (8)




20 / 1

4*

Application of ϕ for parameter update

After all components of MMSE mapping ϕ are computed, thenew model for the parameter is given via

qnew := qapost := ϕ(y + ε(ξ)), (9)

where qnew = Xnew and y + ε(ξ) is a real noisy measurementof Y (ξ).In [1,3,4] we demonstrated that if ϕ is linear, than we obtain thewell-known Kalman Filter.Hypotesis: Mapping ϕ is a polynomial of order n, it converge tofull Bayessian update.




21 / 1

4*

Examples

Examples of computing MMSE




22 / 1

4*

Example 1: ϕ does not exist in the Hermite basis

Assume Y (ξ) = ξ2 and X := q(ξ) = ξ3. The normalized PCEcoefficients are (1,0,1,0)(ξ2 = 1 · H0(ξ) + 0 · H1(ξ) + 1 · H2(ξ) + 0 · H3(ξ))and (0,3,0,1)(ξ3 = 0 · H0(ξ) + 3 · H1(ξ) + 0 · H2(ξ) + 1 · H3(ξ)).For such data the mapping ϕ does not exist. The matrix A isclose to singular.Support of Hermite polynomials (used for Gaussian RVs) is(−∞,∞).




23 / 1

4*

Example 2: ϕ does exist in the Laguerre basis

Assume Y (ξ) = ξ2 and X := q(ξ) = ξ3.The normalized gPCE coefficients are (2,−4,2,0) and(6,−18,18,−6).For such data the mapping mapping ϕ of order 8 and higherproduces a very accurate result.Support of Laguerre polynomials (used for Gamma RVs) is[0,∞).




24 / 1

4*

Lorenz-84 Problem

Lorenz 1984 is a system of ODEs. Has chaotic solutions forcertain parameter values and initial conditions. [LBU is done byO. Pajonk].

x = σ(y − x)

y = x(ρ− z)− yz = xy − βz

Initial state q0(ω) = (x0(ω), y0(ω), z0(ω)) are uncertain.

Solving in t0, t1, ..., t10, Noisy Measur. → UPDATE, solving int11, t12, ..., t20, Noisy Measur. → UPDATE,...




25 / 1

4*

Lorenz-84 Problem

Figure : Partial state trajectory with uncertainty and three updatesCenter for UncertaintyQuantification



26 / 1

4*

Lorenz-84 Problem

10 0 100

0.10.20.30.40.50.60.70.8

x

20 0 200

0.050.1

0.150.2

0.250.3

0.350.4

0.45

y

0 10 200

0.10.20.30.40.50.60.70.80.9

1

z

xfxa

yfya

zfza

Figure : NLBU: Linear measurement (x(t), y(t), z(t)): prior andposterior after one update




27 / 1

4*

Example 5: 1D elliptic PDE with uncertain coeffs

−∇ · (κ(x , ξ)∇u(x , ξ)) = f (x , ξ), x ∈ [0,1]

Measurements are taken at x1 = 0.2, and x2 = 0.8. The meansare y(x1) = 10, y(x2) = 5 and the variances are 0.5 and 1.5correspondingly.




28 / 1

4*

Example 5: updating of the solution u

0 0.2 0.4 0.6 0.8 1−20

−10

0

10

20

30

0 0.2 0.4 0.6 0.8 1−20

−10

0

10

20

30

Figure : Original and updated solutions, mean value plus/minus 1,2,3standard deviations

See more in sglib by Elmar Zander




29 / 1

4*

Example 5: Updating of the parameter

0 0.2 0.4 0.6 0.8 1−1

−0.5

0

0.5

1

1.5

2

0 0.2 0.4 0.6 0.8 1−1

−0.5

0

0.5

1

1.5

2

Figure : Original and updated parameter q.

See more in sglib by Elmar Zander




30 / 1

4*

Conclusion about NLBU

I + Introduced a way to derive MMSE ϕ (as a linear,quadratic, cubic etc approximation, i. e. computeconditional expectation of q, given measurement Y .

I Apply ϕ to identify parameter, i.e. compute E(q|Y = y + ε)

I + All ingredients can be given as gPC.I + we apply it to solve inverse problems (ODEs and PDEs).I - Stochastic dimension grows up very fast.

Future work: 1) Compare with MCMC, 2) compute KLD, 3)compare/test different multi-variate polynomials for ϕ.




31 / 1

4*

I would like to thank

1. Prof. Matthies for the idea and discussions2. Bojana V. Rosic and Elmar Zander from TU Braunschweig,

Germany,3. Oliver Pajonk from Schlumberger4. Olivier Le Maitre for discussions

I used a Matlab toolbox for stochastic Galerkin methods (sglib)https://github.com/ezander/sglib




32 / 1

https://github.com/ezander/sglib

4*

Literature

1. A. Litvinenko and H. G. Matthies, Inverse problems anduncertainty quantificationhttp://arxiv.org/abs/1312.5048, 2013

2. L. Giraldi, A. Litvinenko, D. Liu, H. G. Matthies, A. Nouy, Tobe or not to be intrusive? The solution of parametric andstochastic equations - the ”plain vanilla” Galerkin case,http://arxiv.org/abs/1309.1617, 2013

3. O. Pajonk, B. V. Rosic, A. Litvinenko, and H. G. Matthies, ADeterministic Filter for Non-Gaussian Bayesian Estimation,Physica D: Nonlinear Phenomena, Vol. 241(7), pp.775-788, 2012.

4. B. V. Rosic, A. Litvinenko, O. Pajonk and H. G. Matthies,Sampling Free Linear Bayesian Update of PolynomialChaos Representations, J. of Comput. Physics, Vol.231(17), 2012 , pp 5761-5787




33 / 1

http://arxiv.org/abs/1312.5048

http://arxiv.org/abs/1309.1617

connection between inverse problems and uncertainty quantification problems

Education