polynomial chaos based uncertainty...

Overview UQ Big Picture PCES References

Polynomial Chaos BasedUncertainty Propagation

Lecture 1: Uncertainty and Spectral Expansions

Bert Debusschere†

†[email protected] National Laboratories,

Livermore, CA

Aug 11, 2014 – UQ Summer School, USC

Debusschere – SNL UQ


Acknowledgement

Cosmin Safta Sandia National LaboratoriesKhachik Sargsyan Livermore, CAKenny ChowdharyHabib NajmOmar Knio Duke University, Raleigh, NCRoger Ghanem University of Southern California

Los Angeles, CAOlivier Le Maître LIMSI-CNRS, Orsay, Franceand many others ...

This material is based upon work supported by the U.S. Department of Energy, Officeof Science, Office of Advanced Scientific Computing Research (ASCR), ScientificDiscovery through Advanced Computing (SciDAC) and Applied Mathematics Research(AMR) programs.

Sandia National Laboratories is a multi-program laboratory managed and operated bySandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for theU.S. Department of Energy’s National Nuclear Security Administration under contractDE-AC04-94AL85000.



Goals of this tutorial

• An overview of the key aspects of uncertaintyquantification

• 2 Lectures• Lecture 1: Context and Fundamentals

• Introduction to uncertainty• The various aspects of UQ• Spectral representation of random variables

• Lecture 2: Forward Propagation and InverseProblems

• Please do not hesitate to ask questions!!



Outline

1 Tutorial Overview

2 The Many Aspects of Uncertainty Quantification

3 Spectral Representation of Random Variables

4 References



Aspects of Uncertainty Quantification

• Enabling predictive simulation• Types of uncertainty• UQ methodologies



Predictive simulations enable science-baseddesign

• Empirical design can be inefficient andcostly• Trial and error does not work well for complex

systems• Experiments not always feasible or

permissible• Reliability of nuclear weapons• Climate change mitigation approaches

• Predictive simulations provide insight intothe physics that drive complex systems• Identification of key mechanisms• Allows for rigorous optimization strategies



Model validation requires targeted experimentsModel validation requires targeted experiments

DLR-A Flame: Red = 15,200!Fuel: 22.1% CH4, 33.2% H2, 44.7% N2!

Coflow: 99.2% Air, 0.8% H2O!Detailed Chemistry and Transport: 12-Step

Mechanism (J.-Y. Chen, UC Berkeley)!

Experiment!

LES!

x/d = 10!

MEAN!RMS!

Temperature!

Mixture Fraction!

Comparisons with 1D Raman/Rayleigh/CO-LIF

line images (Barlow et al. )!

H2O Mass Fraction!

CO Mass Fraction!

Large Eddy Simulation (LES) validation on turbulent flame (courtesy J. Oefelein, SNL)



Predictive simulation requires careful assessment ofall sources of error and uncertainty

• Numerical errors• Grid resolution• Time step• Time integration order• Spatial derivative order• Tolerance on iterative solvers• Condition number of matrices

• Uncertainties• Initial and boundary conditions• Model parameters• Model equations• Coupling between models• Sampling noise in particle models• Stochastic forcing terms



Governing equations for LES simulation of multiphasereacting flow



Constitutive models provide closure for governingequations

Smagorinsky sub-grid scale model



Dynamic modeling and reacting flows involveadditional complexity

Does additional complexity provide more accuracy and less uncertainty?Debusschere – SNL UQ


Experimental data is used to calibrate models

Predictive Simulation

Theory

Experiments



UQ methods extract information from all sources toenable predictive simulation


Theory

Experiments

Introduction Introduction UQ Software PCES Summary References

Formulation

dudt

= f (u;λ)

P(λ|D, I) =P(D|λ, I)p(λ, I)

P(D)



Formulation

dudt

= f (u;λ)


P(D)



Formulation

dudt

= f (u;λ)


P(D)



Formulation

dudt

= f (u;λ)


P(D)


Inference

Forward Propagation


Formulation

dudt

= f (u;λ)

P(λ|D, I) =P(D|λ, I)P(λ, I)

P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)

Debusschere – SNL UQ• UQ not just about propagating uncertainties• The term UQ covers a wide range of methods



UQ assesses confidence in model predictions andallows resource allocation for fidelity improvements

• Parameter inference• Determine parameters from data• Characterize uncertainties in inferred parameters

• Propagate input uncertainties through computationalmodels• Account for uncertainty from all sources• Resolve coupling between sources

• Analysis• Sensitivity analysis• Attributions

• Model calibration, validation, selection, averaging



Types of uncertainty

• Epistemic uncertainty• Variable has one particular value, but it is not known• Reducible: by taking more measurements, we can

get to know the value of the variable better• Examples

• The mass of the planet Neptune• Ocean temperature at a particular point and time

• Aleatory uncertainty• Intrinsic or inherent uncertainty: variable is random;

different value each time it is observed• Irreducible: taking more measurements will not

reduce uncertainty in the value of the variable• Examples:

• Collisions interactions in molecular systems• Sampling noise



Point of view matters

• Epistemic versus aleatory sometimes depends onscale level• In fully resolved turbulent simulations, velocity field

near a wall is deterministic• In mesoscale channel flow, near wall velocity field is

turbulent forcing term (aleatory)• In macroscale channel flow, near wall velocity field

provides friction term (epistemic)

• These lectures follow the Bayesian view: probabilityrepresents the degree of belief in the value of avariable



Many UQ approaches are available, filling specificneeds

Estimation of model/parametric uncertainty• Expert opinion, data collection• Regression analysis, fitting, parameter estimation• Bayesian inference of uncertain models/parameters

Forward propagation of uncertainty in models• Local sensitivity analysis (SA) and error propagation• Fuzzy logic; Evidence theory — interval math• Probabilistic framework — Global SA / stochastic UQ

• Random sampling, statistical methods (e.g.Monte-Carlo)

• Spectral Polynomial Chaos (PC) Galerkin methods– Intrusive or non-intrusive

• Collocation, interpolants, regression, ... PC/otherDebusschere – SNL UQ


Outline

1 Tutorial Overview

2 The Many Aspects of Uncertainty Quantification

3 Spectral Representation of Random Variables

4 References



This section focuses on representation of randomvariables with Polynomial Chaos Expansions (PCEs)


Theory

Experiments


Formulation

dudt

= f (u;λ)


P(D)



Formulation

dudt

= f (u;λ)


P(D)



Formulation

dudt

= f (u;λ)


P(D)



Formulation

dudt

= f (u;λ)


P(D)


Inference

Forward Propagation


Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)



Formulation

dudt

= f (u;λ)


P(D)

λ u

P(u) P(λ)


• PCEs are a way to compactly represent randomvariables

• Often used as a way to characterize parametricuncertainties



Polynomial Chaos Expansions (PCEs)

• Representation of random variables with PCEs• Projection of random variables onto PCEs• PC basis types• Convergence of PCEs• Analysis of PCEs



Polynomial Chaos Expansions Represent RandomVariables

u =P∑

k=0

ukψk (ξ)

• u: Random Variable (RV)represented with 1D PCE

• uk : PC coefficients(deterministic)

• ψk : 1D Hermite polynomialof order k

• ξ: Gaussian RV0.0 0.5 1.0 1.5 2.0

u0

1

2

3

4

5

Prob. Dens. [-]

u = 0.5 + 0.2ψ1(ξ) + 0.1ψ2(ξ)Expansion in terms of functions of random variablesmultiplied with deterministic coefficients• Set of deterministic PC coefficients fully describes RV• Separates randomness from deterministic dimensions



PCEs are a functional map from standard RVs to therepresented RV

4 2 0 2 40.5

0

0.5

1

1.5

4 2 0 2 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.5

00.5

11.5

0

0.51

1.52

2.5

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

u = 0.5 + 0.2ψ1(ξ) + 0.01ψ2(ξ)



One-Dimensional Hermite Polynomials

ψ0(ξ) = 1

ψk (ξ) = (−1)keξ2/2 dk

dξk e−ξ2/2, k = 1,2, . . .

ψ1(ξ) = x , ψ2(ξ) = ξ2 − 1, ψ3(ξ) = ξ3 − 3ξ, . . .

The Hermite polynomials form an orthogonal basis over

[−∞,∞] with respect to the inner product

〈ψiψj〉 ≡1√2π

∫ ∞−∞

ψi(ξ)ψj(ξ)w(ξ)dξ = δij⟨ψ2

i

⟩where w(ξ) = e−ξ2/2 is the weight function.Note that e−ξ

2/2√

2πis the density of a standard normal

random variableDebusschere – SNL UQ


Multidimensional Hermite Polynomials

The multidimensional Hermite polynomial Ψi(ξ1, . . . , ξn) isa tensor product of the 1D Hermite polynomials, with asuitable multi-index αi = (αi

1, αi2, . . . , α

in),

Ψi(ξ1, . . . , ξn) =n∏

k=1

ψαik(ξk )

For example, 2D Hermite polynomials:i p Ψi αi

0 0 1 (0,0)1 1 ξ1 (1,0)2 1 ξ2 (0,1)3 2 ξ2

1 − 1 (2,0)4 2 ξ1ξ2 (1,1)5 2 ξ2

2 − 1 (0,2)... ... ... ...



Multidimensional Polynomial Chaos Expansion

u =P∑

k=0

uk Ψk (ξ1, . . . , ξn)

• u: Random Variable (RV) represented with multi-D PCE• uk : PC coefficients (deterministic)• Ψk : Multi-D Hermite polynomials up to order p• ξi : Gaussian RV• n: Dimensionality of stochastic space• P + 1: Number of PC terms: P + 1 = (n+p)!

n!p!

The number of dimensions represents the number ofindependent inputs, degrees of freedom for u• E.g. one stochastic dimension per uncertain model

parameter• Contributions from each uncertain input can be identified



Obtaining PC coefficients for arbitrary randomvariables

• This is a hard problem in general• Random Variable can be specified in a variety of

ways, but often incomplete• Probability Density Function (PDF)• Samples• Expert opinion (e.g. “somewhere between 2 and 4”)

• Particular case of a random variable specified by aPDF is generally tractable



The PC basis functions are orthogonal with respect tothe probability measure of the associated RVs.

〈ΨiΨj〉 ≡∫. . .

∫Ψi(ξ)Ψj(ξ)g(ξ1)g(ξ2) · · · g(ξn)dξ1dξ2 · · · dξn

=n∏

k=1

⟨ψαi

k(ξk )ψ

αjk(ξk )

⟩= δij

⟨Ψ2

i

⟩where,

g(ξ) =e−ξ2/2√

2π



Orthogonality enables a Galerkin projection todetermine the PC coefficients.

u1

E

u

u

uo

ψ1

ψo

1

u1

E

u

u

uo

ψ1

ψo

1

u1

E

u

u

uo

ψ1

ψo

1

u1

E

u

u

uo

ψ1

ψo

1

u1

E

u

u

uo

ψ1

ψo

1

u1

E

u

u

uo

ψ1

ψo

1

u1

E

u

u

uo

ψ1

ψo

�

1

u ≈P∑

k=0

uk Ψk ⇒ 〈Ψiu〉 =P∑

k=0

uk 〈ΨiΨk〉 = ui⟨

Ψ2i

⟩⇒ ui =

〈uΨi〉⟨Ψ2

i

⟩Debusschere – SNL UQ


Galerkin projection requires functional relationshipbetween random variable and germ of PCE.

0 2 4 6 80

1

2

3

4

5

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

u

1

u =P∑

k=0

uk Ψk (ξ) ⇒ ui =〈u Ψi(ξ)〉⟨

Ψ2i

⟩Debusschere – SNL UQ


Cumulative Distribution Function (CDF) maps arbitraryrandom variable to a uniform random variable

• Consider u with PDF p(u)

• CDF of u is given by

F (u) =

∫ u

−∞p(s)ds

• F (u) maps u to η, uniformon [0,1]

0 2 4 6 80

0.2

0.4

0.6

0.8

1

0 2 4 6 80

1

2

3

4

5

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

u

1

ξ

p(ξ)

u(ξ)

p(u)

η

u

1

ξ

p(ξ)

u(ξ)

p(u)

η

1



Inverse CDF mapping enables Galerkin Projection

• η = F (u)

• η = Φ(ξ)maps uniformη to normalRV ξ

• u =F−1(Φ(ξ))

4 2 0 2 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

4 2 0 2 40

0.2

0.4

0.6

0.8

1

0 2 4 6 80

0.2

0.4

0.6

0.8

1

0 2 4 6 80

1

2

3

4

5

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

u

1

ξ

p(ξ)

u(ξ)

p(u)

η

u

1

ξ

p(ξ)

u(ξ)

p(u)

η

1

ξ

p(ξ)

u(ξ)

p(u)

η

1

〈u Ψi(ξ)〉 =

∫F−1(Φ(ξ))︸︷︷︸

u

Ψi(ξ)w(ξ)dξ



PC Illustration: PC Expansion for a Normal RV

• Wiener-HermitePCE constructedfor a Normal RV

• PCE-sampledPDF superposedon true PDF

• Order = 1 0

0.1

0.2

0.3

0.4

0.5

5 6 7 8 9 10 11 12 13 14 15

Normal; WH PC order = 1

u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ

+ u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 2 0

0.1

0.2

0.3

0.4

0.5

5 6 7 8 9 10 11 12 13 14 15


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1)

+ u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 3 0

0.1

0.2

0.3

0.4

0.5

5 6 7 8 9 10 11 12 13 14 15


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ)

+ u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 4 0

0.1

0.2

0.3

0.4

0.5

5 6 7 8 9 10 11 12 13 14 15


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 5 0

0.1

0.2

0.3

0.4

0.5

5 6 7 8 9 10 11 12 13 14 15


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)



PC Illustration: WH PCE for a Normal RV

0

2

4

6

8

10

0 1 2 3 4 5

Normal RV

WH PC order54321

PC mode amplitudesu0–u5

6

7

8

9

10

11

12

13

14

-4 -3 -2 -1 0 1 2 3 4

Normal RV

WH PC order12345

PC function u(ξ)Order 1–5

• First order Wiener-Hermite PCE exact for a normalRV

• Linear function of ξ• Higher order terms are negligible



PC Illustration: WH PCE for a Lognormal RV

• Wiener-HermitePCE constructedfor a LognormalRV


• Order = 1 0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6

Lognormal; WH PC order = 1

u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ

+ u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 2 0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1)

+ u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 3 0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ)

+ u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 4 0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)






• Order = 5 0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5 6


u =P∑

k=0

uk Ψk (ξ)

= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)

+ u5(ξ5 − 10ξ3 + 15ξ)




0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 1 2 3 4 5

Lognormal RV

WH PC order54321

PC mode amplitudesu0–u5

-4

-2

0

2

4

6

8

10

12

14

16

-4 -3 -2 -1 0 1 2 3 4

Lognormal RV

WH PC order12345

PC function u(ξ)Order 1–5

• Fifth-order Wiener-Hermite PCE represents the givenLognormal well

• Higher order terms are negligible



Generalized Polynomial Chaos

PC Type Domain Density w(ξ) Polynomial Free parameters

Gauss-Hermite (−∞,+∞) 1√2π

e−ξ22 Hermite none

Legendre-Uniform [−1, 1] 12 Legendre none

Gamma-Laguerre [0,+∞) xαe−ξ

Γ(α+1)Laguerre α > −1

Beta-Jacobi [−1, 1] (1+ξ)α(1−ξ)β

2α+β+1B(α+1,β+1)Jacobi α > −1, β > −1

Inner product: 〈ψiψj〉 ≡∫ b

a ψi(ξ)ψj(ξ)w(ξ)dξ = δij⟨ψ2

i

⟩• Wiener-Askey scheme provides a hierarchy of possible

continuous PC bases [Xiu and Karniadakis, 2002]• Legendre-Uniform is special case of Beta-Jacobi

• Input parameter domain often dictates the mostconvenient choice of PC

• Polynomials can also be tailored to be orthogonal w.r.t.chosen, arbitrary density



Normal Distribution

4 3 2 1 0 1 2 3 4x

0.0

0.1

0.2

0.3

0.4

0.5

PD

F

• Most commonly used density in PCEs• Support on (−∞,+∞)



Uniform Distribution

1.5 1.0 0.5 0.0 0.5 1.0 1.5x

0.0

0.2

0.4

0.6

0.8

1.0

PD

F

• Appropriate for variables with sharp bounds on theirdistribution

• Support on [−1,1]



Gamma Distribution

0 2 4 6 8 10 12 14 16x

0.0

0.2

0.4

0.6

0.8

1.0

PD

F

α=0

α=1

α=2

α=3

α=4

• Useful to represent quantities that are strictly positive• Support on [0,+∞)



Beta Distribution

1.0 0.5 0.0 0.5 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

PD

F

α=β=−0.5

α=β=0.0

α=β=1.0

α=4,β=0

α=0,β=2

α=1,β=4

• Good for quantities that vary between set boundaries• Can be tailored to preferentially weight some areas• Support on [−1,1]



More Formally: Probability Spaces and RandomVariables

• Let (Θ,S,P) be a probability space• Θ is an event space• S is a σ-algebra on Θ• P is a probability measure on (Θ,S)

• Random variables are functions Ξ : Θ→ R with ameasure corresponding to their image:• if Ξ−1(A) ∈ S, then define µ(A) = P(Ξ−1(A))• p(ξ) = dµ/dξ: the density of the random variable Ξ

(with respect to Lebesgue measure on R)• Expectation: 〈f 〉 =

∫f dµ =

∫f p(ξ) dξ



Convergence of PC expansions

• General convergence theorems are subject ofongoing research

• Depends on the underlying random variables ξ• Wiener-Hermite Chaos has been well-studied• Generalized PC less so

• Ernst et al. 2011:• Let ξ : Θ→ RN such that for i = 1, . . . ,N eachξi : Θ→ R, be a set of random variables

• S(ξ): σ-algebra generated by the set ξ• L2(Θ,S(ξ),P): Hilbert space of real random variables

defined on (Θ,S(ξ),P) with finite second moments• Any random variable in this σ-algebra can be

represented with a Polynomial Chaos expansion withgerm ξ



Convergence of PC expansions

• Proof does not state that any RV with finite variancecan be represented with a PCE to arbitrary precision• In practice, one rarely knows how many degrees of

freedom a RV has• Also, RV specification is rarely complete• In an engineering sense, the choice of the germ and

the PCE order is seen as a model choice to representwhat is known about a RV



How do I know my PCE is converged?

• Approximation error in PCE is topic of a lot ofresearch

• Rules of thumb:• Higher order PC coefficients should decay• Increase order until results no longer change• Not always fail-proof ...





• Moments• Probability Density Functions



Moments of RVs described with PCEs

u =P∑

k=0

uk Ψk (ξ)

• Expectation: 〈u〉 = u0

• Variance σ2

σ2 =⟨

(u − 〈u〉)2⟩ =

⟨(

P∑k=1

uk Ψk (ξ))2

⟩

=

⟨P∑

k=1

P∑j=1

ujuk Ψj(ξ)Ψk (ξ)

⟩

=P∑

k=1

P∑j=1

ujuk 〈Ψj(ξ)Ψk (ξ)〉 =P∑

k=1

u2k

⟨Ψk (ξ)2⟩



Kernel Density Estimation to Get Probability DensityFunction of Random Variable Corresponding to PCEs

• PCE u =∑P

k=0 uk Ψk (ξ) is cheap to sample• Brute-force sampling and bin samples into histogram• Use Kernel Density Estimation (KDE) to get smoother

PDF with fewer samples ui

PDF(u) =1

Nsh

Ns∑i=1

K(

u − ui

h

)K is the kernel, h is the bandwidth.

• Gaussian KDE is commonly used with

K (x) = 1√2π

e−x22 , leading to

PDF(u) =1√

2πNsh

Ns∑i=1

exp(−(u − ui)2

2h2

)Debusschere – SNL UQ


Comparison of histograms and KDE

Source Wikipedia, Drleft; licensed under the Creative Commons Attribution-ShareAlike

3.0 License

• Bandwidth h needs to be chosen carefully to avoidoversmoothing



KDE requires judicious choice of bandwidth h

−5 −4 −3 −2 −1 0 1 2 3 4

Random variable

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

PD

F

True PDFw=wopt=0.16

w=2wopt

w=wopt/2

w=wopt/4

binning

• Scott’s rule-of-thumb optimal for Gaussian RV: h = N− 1d+4

• Smaller h makes KDE more accurate, but more noisy

• Binning is similar to smaller h case, for proper choice of Nbins




−6 −4 −2 0 2 4 6

Random variable

0.00

0.05

0.10

0.15

0.20

0.25

0.30

PD

F

True PDFw=wopt=0.16

w=2wopt

w=wopt/2

w=wopt/4

binning







−10 −5 0 5 10

Random variable

0.00

0.05

0.10

0.15

0.20

0.25

0.30

PD

F

True PDFw=wopt=0.16

w=2wopt

w=wopt/2

w=wopt/4

binning






Extra Material

• Analytical approach to obtain PDF of random variable



Probability Density Function of Random VariableCorresponding to PCEs – 1

• Assume one-dimensional PCE

u =P∑

k=0

uk Ψk (ξ) = f (ξ)

• To evaluate PDF of u: pu(·)

pu(u)du =N∑i

pξ(ξ i)dξ

where ξ1, . . . , ξN are the N roots of f (ξ)− u = 0Many possible ξ may give you this particular u and allof them contribute to the probability density at u.



Probability Density Function of Random VariableCorresponding to PCEs – 2

• More compactly

pu(u) =∑ξ∈Ru

pξ(ξ)

|Df (ξ)|

where Ru = ξ : f (ξ)− u = 0 and |Df (ξ)| = |df/dξ|evaluated at ξ

• Hard to generalize to multi-D PC expansions



PCE to PDF for monotonic mapping between ξ and u

pu(u) =∑ξ∈Ru

pξ(ξ)

|Df (ξ)|

Ru = ξ : f (ξ)− u = 0|Df (ξ)| = |df/dξ|

4 2 0 2 40.5

0

0.5

1

1.5

4 2 0 2 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.5

00.5

11.5

0

0.51

1.52

2.5

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1



PCE to PDF for non-monotonic mapping between ξand u

pu(u) =∑ξ∈Ru

pξ(ξ)

|Df (ξ)|

Ru = ξ : f (ξ)− u = 0|Df (ξ)| = |df/dξ| 4 2 0 2 4

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

02

46

0

0.51

1.52

2.5 4 2 0 2 4

0

2

4

6

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1

ξ

p(ξ)

u(ξ)

p(u)

u

1



Further Reading

• N. Wiener, “Homogeneous Chaos", American Journal of Mathematics, 60:4, pp. 897-936, 1938.

• M. Rosenblatt, “Remarks on a Multivariate Transformation", Ann. Math. Statist., 23:3, pp. 470-472, 1952.

• R. Ghanem and P. Spanos, “Stochastic Finite Elements: a Spectral Approach", Springer, 1991.

• O. Le Maître and O. Knio, “Spectral Methods for Uncertainty Quantification with Applications toComputational Fluid Dynamics", Springer, 2010.

• D. Xiu, “Numerical Methods for Stochastic Computations: A Spectral Method Approach", Princeton U.Press, 2010.

• O.G. Ernst, A. Mugler, H.-J. Starkloff, and E. Ullmann, “On the convergence of generalized polynomialchaos expansions,” ESAIM: M2AN, 46:2, pp. 317-339, 2011.

• D. Xiu and G.E. Karniadakis, “The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations",SIAM J. Sci. Comput., 24:2, 2002.

• Le Maître, Ghanem, Knio, and Najm, J. Comp. Phys., 197:28-57 (2004)

• Le Maître, Najm, Ghanem, and Knio, J. Comp. Phys., 197:502-531 (2004)

• B. Debusschere, H. Najm, P. Pébay, O. Knio, R. Ghanem and O. Le Maître, “Numerical Challenges in theUse of Polynomial Chaos Representations for Stochastic Processes", SIAM J. Sci. Comp., 26:2, 2004.

• S. Ji, Y. Xue and L. Carin, “Bayesian Compressive Sensing", IEEE Trans. Signal Proc., 56:6, 2008.

• A. Saltelli and S. Tarantola and F. Campolongo and M. Ratto, “Sensitivity Analysis in Practice: A Guide toAssessing Scientific Models”. John Wiley & Sons, 2004.

• K. Sargsyan, B. Debusschere, H. Najm and O. Le Maître, “Spectral representation and reduced ordermodeling of the dynamics of stochastic reaction networks via adaptive data partitioning". SIAM J. Sci.Comp., 31:6, 2010.

• K. Sargsyan, B. Debusschere, H. Najm and Y. Marzouk, “Bayesian inference of spectral expansions forpredictability assessment in stochastic reaction networks," J. Comp. Theor. Nanosc., 6:10, 2009.

• D.S. Sivia, “Data Analysis: A Bayesian Tutorial,” Oxford Science, 1996.


polynomial chaos based uncertainty...

Documents