orthogonal polynomials, quadratures & sparse-grid methods ... · orthogonal polynomials,...
TRANSCRIPT
A Tutorial on Sparse Grid Integration, May 2010
Orthogonal Polynomials, Quadratures& Sparse-Grid Methods for Probability
Integrals
Dr. Abebe GeletuMay, 2010
Technische Universität Ilmenau,Institut für Automatisierungs- und SystemtechnikFachgebiet Simulation und Optimale Prozesse
1/31
A Tutorial on Sparse Grid Integration, May 2010
Topics
Interpolatory Quadrature Rules
Orthogonal Polynomials
Embedded Quadrature Rules
Quadrature Rules and Probability Integrals
Full-Grid Tensor Product Multidimensional Integration
Sparse-Grid Tensor Product Multidimensional Integration
Part - I: Quadrature Rules;
Part - II: Sparse-Grid Integration Methods
2/31
A Tutorial on Sparse Grid Integration, May 2010
1. Interpolatory Quadrature Rules
For an integrable function f : ℝ → ℝ to compute the integral
I[f ] =∫ b
af (x)�(x)dx ,
where
- [a,b] ⊂ ℝ is a bounde or unbounde interval;
- � : ℝ → ℝ is a nonnegative weight function.
Sometimes the weight function need to be adjusted toconform to the the properties of the integrand f , e.g.oscillatory behaviors, etc.; and the interval of integrationΩ1.
3/31
A Tutorial on Sparse Grid Integration, May 2010
1.2. Some integrals with standard weight functionsand intervals of integration
(a) I1[f ] =∫ 1
−1f (x)dx ⇒ �(x) = 1; Ω1 = [−1,1]
(b) I2[f ] =∫ 1
−1f (x)(1 − x)�(1 + x)�dx
⇒ �(x) = (1 − x)�(1 + x)� , �, � > −1;
Ω1 = [−1,1]
(c) I3[f ] =∫ 1
−1f (x)
(
1 − x2)− 1
2 dx
⇒ �(x) =(
1 − x2)− 1
2; Ω1 = [−1,1]
4/31
A Tutorial on Sparse Grid Integration, May 2010
(d) I4[f ] =∫ +∞
0f (x)x�e−xdx
⇒ �(x) = x�e−x , � > −1; Ω1 = [0,+∞)
(e) I5[f ] =∫ +∞
−∞f (x)e−x2
dx
⇒ �(x) = e−x2; Ω1 = [0,+∞)
Note:
transform other types of integrals to the standard ones.
Example:
∫ 1
−1sin(x)e−x2
dx −→∫ 1
−1
(
sin(x)e−x2√
1 − x2)
︸ ︷︷ ︸
=f (x)
(
1 − x2)− 1
2 dx
5/31
A Tutorial on Sparse Grid Integration, May 2010
1.3. Interpolatory Quadrature Formulas
To construct an N-point quadrature formula (rule):
Q1N [f ] :=
N∑
k=1
wk f (xk ),
where the integration nodes X = {x1, x2, . . . , xN} and weights{w1,w2, . . .wN} are constructed based on Ω1 = [a,b] and theweight function �.
Question:How to determine the 2N unknowns x1, . . . , xN andw1, . . . ,wN?
Requirement: the weights w1,w2, . . . ,wN should benon-negative to avoid numerical cancelations.
6/31
A Tutorial on Sparse Grid Integration, May 2010
1.4. Efficient interpolatory quadrature rules
1 Gauss qudrature rules and their extensions2 Curtis-Clenshaw quadrature rules
7/31
A Tutorial on Sparse Grid Integration, May 2010
1.5. Polynomial Exactness
One measure of quality for a quadrature rule is its polynomialexactness.
The N-point quadrature Q1N [⋅] is said to be exact for a
polynomial pm of degree m if
I[pm] =
∫ b
apm(x)�(x)dx = Q1
N [pm] =N∑
k=1
wkpm(xk )
the degree of exactness or degree of accuracy d of anN-point quadrature rule is equal to the maximum degreepolynomial for which Q1
N is exact.
8/31
A Tutorial on Sparse Grid Integration, May 2010
1.6.Gauss Quadrature Rules and OrthogonalPolynomials
An N-point Gauss quadrature rule is constructed toachieve the largest possible polynomial exactness.
Theorem (see Davis & Rabinowitz [1])
An N-point quadrature formula Q1N has degree of exactness
d = 2N − 1 if the integration nodes x1, x2, . . . , xN are zeros ofthe N−th degree orthogonal polynomial pN(x) w.r.t. � andΩ1 = [a,b].
Hence, there is a polynomial pm with degree m > 2N − 1,such that I[pm] ∕= Q1
N [pm].
Example:a) A 3-point Gauss quadrature rule Q1
3 has apolynomial exactness d = 2 × 3 − 1 = 5
9/31
A Tutorial on Sparse Grid Integration, May 2010
Orthogonal Polynomials
Two polynomials pn and pm are orthogonal w.r.t. � andΩ1 = [a,b] if
< pn,pm >:=
∫ b
apn(x)pm(x)�(x)dx = 0.
In Theorem 1, the N−th degree orthogonal polynomial pN
is orthogonal to all polynomials pm degree m ≤ N − 1.
Examples:
(a) p0 = 1, p1(x) = x are orthogonal w.r.t.�(x) = 1 and Ω1 = [−1,1].
(b) H1 = 2x , H2(x) = 4x2 − 2 are orthogonal w.r.t.�(x) = e−x2
and Ω1 = (−∞,+∞).
10/31
A Tutorial on Sparse Grid Integration, May 2010
Characterization of Orthogonal Polynomials
Theorem (Recurrence relationship, see Gatushi [2])
Suppose that p0 = 1,p1, . . . are orthogonal polynomials withdeg(pn) = n and leading coefficient equal to 1. For everyinteger n
pn+1(x) = (x − an)pn(x)− bnpn−1(x),n = 1,2 . . . (★)
where
an =
∫ ba xp2
n(x)�(x)dx∫ b
a p2n(x)�(x)dx
,n = 1, . . . ;
bn =
∫ ba xp2
n(x)�(x)dx∫ b
a p2n−1(x)�(x)dx
,n = 1, . . .
11/31
A Tutorial on Sparse Grid Integration, May 2010
(Continued...) Characterization of OrthogonalPolynomials
Theorem (Continued...Recurrence relation)
and
p1(x) = (x − a0)p0(x) = x − a0,
a0 =
∫ ba xp2
0(x)�(x)dx∫ b
a p20(x)�(x)dx
=
∫ ba x�(x)dx∫ b
a �(x)dx.
Remark:
Given a nonnegative weight function � and a set Ω, youcan construct your own set of orthogonal polynomials ifyou can efficiently compute the recurrence coefficientsa0,an,bn,n = 1,2, . . ..
12/31
A Tutorial on Sparse Grid Integration, May 2010
1.7. Algorithms to determine the recurrencecoefficients
Algorithms for the computation of the recurrencecoefficients are commonly known as Steleje’sProcedures.
Currently an efficient and stable algorithm for thecomputation of the coefficients a0,an,bn,n = 1,2, . . . isgiven by Gander & Karp [5].
Software:
FORTRAN: ORTHOPOL by Gatushi [4]
C++ implmentation of ORTHOPOL Fernandes & Atchley[4].
Various Matlab codes by Gatushi [1]
13/31
A Tutorial on Sparse Grid Integration, May 2010
Some known sets of orthogonal polynomials
Jacobi polynomials orthogonal w.r.t.�(x) = (1 − x)�(1 + x)� , �, � > −1 and Ω1 = [−1,1]:
p(�,�)n (x) =
12n
n∑
k=0
(n + �
k
)(n + �n − k
)
(x − 1)n−k (1 + x)k ,
n = 0,1,2, . . .
Lagendre polynomials are the Jacobi polynomials for� = � = 0; i.e. �(x) = 1:
Ln(x) =12n
n∑
k=0
(nk
)(n
n − k
)
(x − 1)n−k (1 + x)k ,
n = 0,1,2, . . .
14/31
A Tutorial on Sparse Grid Integration, May 2010
Hermite Polynomials w.r.t. � = e−x2and Ω1 = (−∞,+∞)
Hn(x) = n![n/2]∑
k=1
(−1)k
k!(n − 2k)!(2x)n−2k ,n = 0,1,2, . . .
The first 6 terms of Hermite polynomials:
H0(x) = 1, H1(x) = 2x , H2(x) = 4x2 − 2
H3(x) = 8x3 − 12x , H4(x) = 16x4 − 48x2 + 12,
H5(x) = 32x5 − 160x3 + 120x
H6(x) = 64x6 − 480x4 + 720x2 − 120,etc.
15/31
A Tutorial on Sparse Grid Integration, May 2010
Chebychev Polynomials orthogonal w.r.t.�(x) = (1 − x2)−
12 and Ω1 = [−1,1]
C0(x) =1√�
T0(x),Cn(x) =
√
2�
Tn(x),n = 1,2, . . .
where Tn(x) = cos(n acrcos(x)), known as Chebychevpolynomials of first kind. Few terms are given by
T0(x) = 1, T1(x) = x , T2(x) = 2x2 − 1,
T3(x) = 4x3 − 3x , T4(x) = 8x4 − 8x2 + 1
T5(x) = 16x5− = 8x4 − 20x3 + 5x ,
T6(x) = 32x6 − 48x4 + 18x2 − 1
Corresponding to each set of orthogonal polynomials wehave the quadrature rules: Gauss-Jacobi,Gauss-Legendre, Gauss-Hermite, Gauss-Chebychev, etc.
16/31
A Tutorial on Sparse Grid Integration, May 2010
1.8. Computation of quadrature nodes and weights
Theorem (see Gatushi [2, 3])
The quadrature nodes x1, . . . , xN and weights w1,w2, . . . ,wN
can be obtained from the spectral factorization
Jn = V⊤ΛV ; Λ = diag(�1, �2, . . . , �N), VV⊤ = IN ;
of the symmetric matrix tridiagonal Jacobi matrix
Jn =
⎡
⎢⎢⎢⎢⎢⎢⎣
a0√
b1√b1 a1
√b2
√b2
. . . . . .
. . . aN−2√
bN − 1√bN − 1 aN−1
⎤
⎥⎥⎥⎥⎥⎥⎦
,
17/31
A Tutorial on Sparse Grid Integration, May 2010
Theorem (continued...)
where ak ,bk , k = 1,2, . . . ,N − 1 are known coefficients fromthe recurrence relation (★). In particular
xk = �k , k = 1,2, . . . ,N;
wk =(
e⊤1 Vek
)2, k = 1,2, . . . ,N.
Observe that: all the weights wk obtained above arenonnegative.
Exercise:Write a Matlab code not longer than 10 lines to compute x ′
k sand w ′
ks.
18/31
A Tutorial on Sparse Grid Integration, May 2010
1.9. Gauss Quadrature Rules with Preassigned nodes
In some application integration nodes need to be prefixed(pre-given), eg. due to boundary conditions, constraints,etc.
Such nodes are usually needed in collocation methods inthe solution of ODEs and PDEs.
Objective: To construct an N−point quadrature rule Q1N with
x1, . . . , xm,m ≤ N, are preassigned (fixed) nodes in [a,b]:
determine the remaining N − m nodes ;
correspoindg weights w1,w2, . . . ,wN
so that Q1N has the maximum possible degree of exactness d .
19/31
A Tutorial on Sparse Grid Integration, May 2010
Classical Gauss quadrature rules with preassignednodes
In a closed bounded interval [a,b]:1 Gauss-Radau rule:
- One end point of [a,b] is prefixed to be a node; i.e. eitherx1 = a or xN = b; the rest of the nodesx2, x3, . . . , xN ∈ (a,b) .- Degree of polynomial exactness d = 2N − 2.
2 Gauss-Lobatto rule:- Both end points of the interval [a,b] are prefixed to benodes; i.e. x1 = a and xN = b. There restx2, . . . , xN ∈ (a,n).- Degree of polynomial exactness d = 2N − 3.
In general, prefixing nodes reduces the degree ofpolynomial exactness by the number of integration nodesprefixed.
20/31
A Tutorial on Sparse Grid Integration, May 2010
Recently, Bultheel et. al. [2] give Gauss qudrature ruleswith prefixed nodes and positive weights in any intervalΩ1 ⊂ ℝ bounded or unbounded.
Software:
See Gatushi [1] for Matlab codes.
21/31
A Tutorial on Sparse Grid Integration, May 2010
1.10 How good are Gauss Quadrature Rules?
Disadvantage:
Different Gauss quadrature rules Q1N1
and Q1N2
have only afew common nodes, for N1 < N2; i.e. the set of nodesX1 = {x1, x2, . . . , xN1
} ⊂ [a,b] andX2 = {z1, z2, . . . , zN2
} ⊂ [a,b].
=⇒ it is difficult to estimate the convergence of the limit:
limN−→∞
∣∣∣I[f ]− Q1
N [f ]∣∣∣ .
Note that: had it been X1 ⊂ X2, we could have obtained∣∣∣I[f ]− Q1
N2[f ]∣∣∣ ≤
∣∣∣I[f ]− Q1
N1[f ]∣∣∣ .
Solution: Either extended Gauss quadrature rules so that thenodes for a lower accuracy can be reused to constructquadrature rules of higher accuracy; or construct embeddedquadrature rules from the start.
22/31
A Tutorial on Sparse Grid Integration, May 2010
1.11. Embedded quadrature rules - Extensions ofGauss quadrature rules
A) Kronord’s extension [6]: Given Gaussquadrature nodes x1, x2, . . . , xN , between everytwo nodes add one new node:
z1 ∈ (a, x1), z2 ∈ (x1, x2), . . . zN ∈(xN−1, xn), zn+1 ∈ (xN ,b) so that theN + N + 1 = 2N + 1 pointsz1, x1, z2, x2, . . . , xN−1, zN , xN , zN+1 are thenodes of the new quadrature rules; andand the new weights w1,w2, . . . ,w2N+1 are allnonnegative.
=⇒ The degree to exactness of a Gauss-Kronordrule is:
d =
{3N + 1, if N is even3N + 2, if N is odd
Hence, XN ⊂ X2N+1. 23/31
A Tutorial on Sparse Grid Integration, May 2010
B) Patterson’s extension [2]:To an existing N Gauss quadrature nodesx1, x2, . . . , xN , add m new integration nodesz1, z2, . . . , zm so the new quadrature rule has themaximum degree of accuracy
d = 2(m + N)− N − 1 = N + 2m − 1
and the weights w1,w2, . . . ,wN+m arenonnegative.Hence, XN ⊂ XN+m.
Note:In general, for a given weight function � and integrationdomain [a,b], the construction of convenient embeddedGauss-quadrature rules is not a trivial task.(see, [3, 3, 5])
Software:FORTRAN, C++, Matlab codes in the book ofKythe & Schäferkotter [1].
24/31
A Tutorial on Sparse Grid Integration, May 2010
Embedded quadrature rules- the Curtis-Clenshawquadrature rule
For integrals on [−1,1] (in general on a closed and boundedinterval [a,b]) the set of nodes
XN =
{
xk ∣ xk = cos(k − 1)�
N − 1, k = 1,2, . . . ,N
}
and the weights are computed from the orthogonal Chebychevpolynomials of first type Tn(x) = cos(n acrcos(x)) usingTheorem 4.Properties:
degree accuracy N − 1;XN ⊂ X2N−1;nodes and weight are very simple to compute; (seeTrefethen [3],Waldvogel[4] for Matlab codes)despite lower polynomial exactness, comparable efficiencywith Gauss quadrature rules ( Trefethen [3]);extensive applications in spectral and pseudo-spectral 25/31
A Tutorial on Sparse Grid Integration, May 2010
1.12. Quadrature Rules and Probability Integrals
For a well behaved nonnegative function �(x) , if∫ ∞
−∞�(x)dx = 1,
then the expression �(x) = �(x)dx defines a probabilitymeasure. And � is termed a probability density function ofx . In particular, for a random set A
Pr{x ∈ A} = �(x ∈ A) =∫ ∞
−∞1A(x)�(x)dx ,
where
1A(x) ={
1, if x ∈ A0, if x /∈ A.
For function f of the random variable x , its expected value is
E [f ] =∫ ∞
−∞f (x)�(x)dx .
26/31
A Tutorial on Sparse Grid Integration, May 2010
Standard probability density functions
Given a nonnegative weight function �(x) with the property
�(x) ={
> 0, for x ∈ [a,b]0, if x /∈ [a,b].
such that �0 =∫ b
a �(x)dx set �(x) := �(x)�0
, then the expression�(x) = �(x)dx defines a probability measure.Example:
For �(x) = 1 on [a,b]:
�0 = b−a ⇒ �(x) =1�0
=1
b − a→ the uniform density function.
27/31
A Tutorial on Sparse Grid Integration, May 2010
For �(x) = e−x2on (−∞,+∞):
�0 =
∫ +∞
−∞e−x2
=√� ⇒ �(x) =
�(x)�0
=e−x2
√�
.
Hence,
F (x) =∫ x
∞
e−z2
√�
dz =
∫ x
∞
e− 12 (√
2z)2
√2√�
√2dz
Setting u =√
2z we have du =√
2dz and
F (x) =∫ x
∞
e−z2
√�
dz =
∫ x
∞
e− 12 u2
√2�
du.
⇒ F defines the standard normal distribution.
28/31
A Tutorial on Sparse Grid Integration, May 2010
Literature
Davis, P. J.; Rabinowitz, P. Methods of numericalintegration. Dover Publications, 2nd ed., 2007.
Bultheel, A.; Cruz-Barroso, R.; Van Barel, M. OnGauss-type quadrature formulas with prescribed nodesanywhere on the real line. Calcolo, 47(2010) 21–48.
Elhay, S.; Kautsky J. Generalized Kronrod Patterson typeembedded quadratures. Aplikace Matematiky, 37(1992)81–103.
Fernandes, A. D.; Atchley, W. R. Gaussian quadratureformulae for arbitrary positive measure. EvolutionaryBioinformatics, 2(2006) 261 – 269.
Gander, M. J.; Karp, A. H. Stable computation of high orderGauss quadrature rules using discretization for measuresin radiation transfer. J. of Molecular Evolution, 53(4-5):47.
29/31
A Tutorial on Sparse Grid Integration, May 2010
Gatushi, W. Orthogonal polynomials (in Matlab). J. Comp.Appl. Math. 178(2005) 215–234.
Gatushi, W. Orthogonal polynomials: computation andapproximation. Oxford University Press, New York, 2004.
Gatushi, W. Orthogonal polynomials and quadrature.Electronic Trans. of Num. Math. 9(1999) 65 – 76.
Gatushi, W. Algorithm 726: ORTHOPOL - A package ofroutines for generating orhtogonal polynomials andGauss-type quadrature rules. ACM Trans. on Math.Software, 20(1994) 21 – 62.
Laurie, D. P. Calculation of Gauss-Kronord quadraturerules. Math. Comp. 66(1997) 1133 – 1145.
Kronord, A. S. Nodes and weights of quadrature formulas.Consultants Bureau, New York, 1965.
30/31
A Tutorial on Sparse Grid Integration, May 2010
Kythe, P. K.; Schäferkotter, M. R. Handbook ofcomputational methods for integration. Chapman &Hall/CRC, 2005.
Patterson, T. N. L. The optimum addition of points toquadrature formulae. Math. Comp. 22(1968) 847 – 856.Errata: Math. Comp. 23(1969) 892.
Trefethen, L. N. Is Gauss Better than Clenshaw-Curtis?SIAM Review 50(2008) 67 – 87.
Waldvogel, J. Fast construction of the Fejer andClenshaw-Curtis quadrature rules. BIT NumericalMathematics 43(2004) 1 – 18.
31/31