fred j. hickernell - iitmeshfree-methods-seminar/presentations/... · 2014. 5. 20. · introduction...

Introduction Integration Problems Open Problems and Ongoing Work References

Automatic Numerical Algorithms

Fred J. Hickernell

Department of Applied Mathematics, Illinois Institute of [email protected] mypages.iit.edu/~hickernell

Joint work with Aleks Borkovskiy, Sou-Cheng Choi, Nick Clancy, Yuhan Ding,Siyuan Deng, Caleb Hamilton, Lan Jiang, Lluıs Antoni Jimenez Rugama, Yuewei Liu,

Art Owen, Martha Razo, Xin Tong, Sunny Yun, Yizhi Zhang, and Xuan ZhouKey: PhD student, MS student, BS student, High School student

With lots of good feedback by this Meshfree Methods Seminar

Supported by NSF-DMS-1115392

May 20, 2014

[email protected] Automatic Algorithms IIT, 5/20/2014 1 / 35

mailto:[email protected]

[email protected]

http://mypages.iit.edu/~hickernell

mypages.iit.edu/~hickernell


Self Introduction

Brief Bio

§ Faculty member at Illinois Institute of Technology (since 2005), Hong KongBaptist University (1985-2005), and University of Southern California(1981-1985)

§ PhD Massachusetts Institute of Technology, BA Pomona College

§ Research in computational mathematics & statistics



MATLAB, LATEX and Beamer

MATLAB, LATEX and Beamer

§ My calculations are mostly done in MATLAB . If you are wondering howgood is your MATLAB, you might try my Matlabtiude Test atmypages.iit.edu/~hickernell.

§ Mathematicians (and computer scientists) use LATEX for writing papers. Youshould learn to use LATEX as well.

§ Presentations like this one and posters can be created using the Beamerpackage. The template that I use is at mypages.iit.edu/~hickernell.







The sin Algorithm

What Can Go Wrong with sin(¨)?

We expect sin(¨) on our calculators or computers to give 15 correct digits, i.e.,|sinpxq ´ sin(x)| ď 10´15. Using MATLAB (The MathWorks, 2014):

>> sin(0)

ans =

0

>> sin(pi/2)

ans =

1

>> sin(pi/4)

ans =

7.071067811865475e-01

>> sqrt(2)/2

ans =

7.071067811865476e-01 100

104

108

1012

1016

1020

10−16

10−12

10−8

10−4

100

N π

|sin(N*pi)|

ActualExpected

We should only expect |sinpxq ´ sin(x)| ď minp2, 10´15 |x|q Ð right



The sin Algorithm


We expect sin(¨) on our calculators or computers to give 15 correct digits, i.e.,|sinpxq ´ sin(x)| ď 10´15. Using MATLAB (The MathWorks, 2014):

>> sin(1e17*pi)

ans =

-8.479696810401983e-01

Why? Because of round-off error

>> 1e17*pi

ans =

3.141592653589793e+17

>> (1e17+1/2)*pi

ans =

3.141592653589793e+17 100

104

108

1012

1016

1020

10−16

10−12

10−8

10−4

100

N π

|sin(N*pi)|

ActualExpected




The sin Algorithm


We expect sin(¨) on our calculators or computers to give 15 correct digits, i.e.,|sinpxq ´ sin(x)| ď 10´15.Ð wrong

>> sin(1e17*pi)

ans =

-8.479696810401983e-01

Why? Because of round-off error

>> 1e17*pi

ans =

3.141592653589793e+17

>> (1e17+1/2)*pi

ans =

3.141592653589793e+17 100

104

108

1012

1016

1020

10−16

10−12

10−8

10−4

100

N π

|sin(N*pi)|

Actual“Expected”Expected




Computing Areas and Averages

Integration

area under the graph of f

“ average of f “

ż 1

0

fpxqdx

E.g.,

ż 1

0

e´5xdx “1

5

`

1´ e´5˘

« 0.19870 0.5 1

0

0.2

0.4

0.6

0.8

1

average=0.1987




Calculus that Needs Numerical Mathematics

Many elementary functions have anti-derivatives that are not elementaryfunctions. E.g.,

ż 1

0

e´5x2

dx “ ?

« 0.3957

erfpxq “2?π

ż x

0

e´t2

dt

We did not solve the problem. We gavethe answer a name.

How do we get numerical values fordefinite integrals?

0 0.5 10

0.2

0.4

0.6

0.8

1






ż 1

0

e´5x2

dx “1

2

c

π

5erfp

?5q

« 0.3957

erfpxq “2?π

ż x

0

e´t2

dt



0 0.5 10

0.2

0.4

0.6

0.8

1






ż 1

0

e´5x2

dx “1

2

c

π

5erfp

?5q « 0.3957

erfpxq “2?π

ż x

0

e´t2

dt



0 0.5 10

0.2

0.4

0.6

0.8

1

average=0.3957



Trapezoidal Rule

How Do We Evaluate Definite Integrals Numerically?

If we only have access to fpxq values, how do we evaluate

ż 1

0

fpxq dx?

E.g.,

ż 1

0

e´5x2

dx “ ?

The trapezoidal rule approximates theintegral (area) by a sum of areas oftrapezoids:

Tn “1

n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨

`f

ˆ

n´ 1

n

˙

`fp1q

2

0 0.5 10

0.2

0.4

0.6

0.8

1

average=0.3957



Trapezoidal Rule



ż 1

0

fpxq dx?

E.g.,

ż 1

0

e´5x2

dx « 0.3957


Tn “1

n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨

`f

ˆ

n´ 1

n

˙

`fp1q

2

0 0.5 10

0.2

0.4

0.6

0.8

1

T1pfq “ 0.5034



Trapezoidal Rule



ż 1

0

fpxq dx?

E.g.,

ż 1

0

e´5x2

dx « 0.3957


Tn “1

n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨

`f

ˆ

n´ 1

n

˙

`fp1q

2

0 0.5 10

0.2

0.4

0.6

0.8

1

T2pfq “ 0.3949



Trapezoidal Rule



ż 1

0

fpxq dx?

E.g.,

ż 1

0

e´5x2

dx « 0.3957


Tn “1

n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨

`f

ˆ

n´ 1

n

˙

`fp1q

2

0 0.5 10

0.2

0.4

0.6

0.8

1

T4pfq “ 0.3954



Trapezoidal Rule



ż 1

0

fpxq dx?

E.g.,

ż 1

0

e´5x2

dx « 0.3957


Tn “1

n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨

`f

ˆ

n´ 1

n

˙

`fp1q

2

0 0.5 10

0.2

0.4

0.6

0.8

1

T50pfq “ 0.3957



Trapezoidal Rule

How Many Trapezoids Are Needed?ż 1

0

fpxqdx « Tnpfq “1

n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨ ` f

ˆ

n´ 1

n

˙

`fp1q

2

Given error tolerance ε, how large should n be to make

∣∣∣∣ż 1

0

fpxq dx´ Tnpfq

∣∣∣∣ ď ε?

0 0.5 10

0.2

0.4

0.6

0.8

1

ş1

0fpxqdx « 0.3957 « T50pfq



Trapezoidal Rule


0


n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨ ` f

ˆ

n´ 1

n

˙

`fp1q

2


∣∣∣∣ż 1

0

fpxq dx´ Tnpfq

∣∣∣∣ ď ε?

Theory says∣∣∣∣ż 1

0

fpxqdx´ Tnpfq

∣∣∣∣ ďş1

0|f2pxq| dx8n2

Guarantees are possible for all

ż 1

0

∣∣f2pxq∣∣ dx ď C

by choosing n “Q

a

C{p8εqU

.

Automatic, not adaptive, impractical

0 0.5 10

0.2

0.4

0.6

0.8

1

ş1




Trapezoidal Rule


0


n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨ ` f

ˆ

n´ 1

n

˙

`fp1q

2


∣∣∣∣ż 1

0

fpxq dx´ Tnpfq

∣∣∣∣ ď ε?

Other -“theory” (Burden andFaires, 2010, p. 223–224) tells us

ż 1

0

fpxqdx´ Tnpfq

« xerrpf, nq :“Tnpfq ´ Tn{2pfq

3

but not how large n must be for “«” tobe valid.

Lyness (1983) warned us about thisproblem.

0 0.5 10

0.2

0.4

0.6

0.8

1

ş1


xerrpf, 50q « 2E´6



Trapezoidal Rule


0


n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨ ` f

ˆ

n´ 1

n

˙

`fp1q

2


∣∣∣∣ż 1

0

fpxq dx´ Tnpfq

∣∣∣∣ ď ε?


ż 1

0

fpxqdx´ Tnpfq


3



0 0.5 1

−0.5

0

0.5

1

ş1

0fpxq dx “ 0.1189,T4pfq “ 0.1070xerrpf, 4qpfq “ 0



Trapezoidal Rule


0


n

„

fp0q

2` f

ˆ

1

n

˙

` ¨ ¨ ¨ ` f

ˆ

n´ 1

n

˙

`fp1q

2


∣∣∣∣ż 1

0

fpxq dx´ Tnpfq

∣∣∣∣ ď ε?


ż 1

0

fpxqdx´ Tnpfq


3



0 0.5 1

−0.2

0

0.2

0.4

0.6

0.8

1

ş1

0fpxqdx “ 0.009050,T50pfq “ 0.008145

xerrpf, 50q “ [email protected] Automatic Algorithms IIT, 5/20/2014 8 / 35


MATLAB’s Best: integral(¨)

integral(¨) Fails for Spiky and Fluky Integrands

0 0.005 0.01

0

0.25

0.5

0.75

1

fspikydata

ż 1

0

fspikypxqdx “1

2

integral(spikef,0,1, ...

’AbsTol’,1e-13,’RelTol’,1e-13)

ans =

0

0 0.5 1−1

−0.5

0

0.5

1

ffluky

data

ż 1

0

fflukypxqdx “ 0.278827

integral(flukef,0,1, ...

’AbsTol’,1e-13,’RelTol’,1e-13)

ans =

0.278799

Unavoidable, but should be quantified Lyness (1983) warned us about [email protected] Automatic Algorithms IIT, 5/20/2014 9 / 35


Our New Algorithm integral g(¨)

A -Based Trapezoidal Rule Error Estimation

§ -based numerical integration requires impractical bounds on

ż 1

0

∣∣f2pxq∣∣ dx§ -based numerical integration can be fooled by fluky functions. Don’t

know which spiky functions can be handled.

§ Instead we develop -based numerical integration (Clancy et al., 2014)

τ“

"

f :

ż 1

0

∣∣f2pxq∣∣dx ď τ

ż 1

0

∣∣f 1pxq ´ fp1q ` fp0q∣∣dx*





τ“

"

f :

ż 1

0


ż 1

0

∣∣f 1pxq ´ fp1q ` fp0q∣∣ dx*Fnpfq :“

nÿ

i“1

∣∣∣∣f ˆ i

n

˙

´ f

ˆ

i´ 1

n

˙

´fp1q ´ fp0q

n

∣∣∣∣Ð data-based

Fnpfq “

ż 1

0

∣∣plinear spline of fq1pxq

´ fp1q ` fp0q∣∣dx

«

ż 1

0

∣∣f 1pxq ´ fp1q ` fp0q∣∣ dx0 0.5 1

0

0.2

0.4

0.6

0.8

1flinear spline





τ“

"

f :

ż 1

0


ż 1

0

∣∣f 1pxq ´ fp1q ` fp0q∣∣dx*Fnpfq :“

nÿ

i“1

∣∣∣∣f ˆ i

n

˙

´ f

ˆ

i´ 1

n

˙

´fp1q ´ fp0q

n

∣∣∣∣Ð data-based

0 ď

ż 1

0

∣∣f 1pxq ´ fp1q ` fp0q∣∣ dx´ Fnpfq ď ş1

0|f2pxq| dx

2nÐ theory

ďτş1

0|f 1pxq ´ fp1q ` fp0q| dx

2nÐ f P

τ

∣∣∣∣ż 1

0

fpxqdx´ Tnpfq

∣∣∣∣ ďş1

0|f2pxq| dx8n2

ďτş1

0|f 1pxq ´ fp1q ` fp0q|dx

8n2

ďτFnpfq

4np2n´ τqÐ data-based




A -Based Trapezoidal Rule Error Algorithm,integral g(f,ε,τ)

§ Start with n “ rpτ ` 1q{2s.

§ Iteratively double n until

τFnpfq

4np2n´ τqď ε,

where Fnpfq :“nÿ

i“1

∣∣∣∣f ˆ i

n

˙

´ f

ˆ

i´ 1

n

˙

´fp1q ´ fp0q

n

∣∣∣∣§ Return Tnpfq as the answer.



Numerical Experiments

Piecewise Quadratic Hump Function

fpxq “1

4a3r4a2 ` px´ zq2 ´ px´ z´ aq|x´ z´ a|

´ px´ z ` aq|x´ z ` a|s,

z ´ 2a ď x ď z ` 2a

ż 1

0

fpxqdx “ 1, z “ center, 4a “ width

a, z random, 10000 replications, ε “ 10´8

τ Probpf P τ q Success

10 0%Ñ 25% 25%integral g(f,ε,τ) 100 23%Ñ 58% 58%

1000 57%Ñ 88% 88%

integral (The MathWorks, 2014) 19%chebfun (Hale et al., 2012) 29%



Summary

Advancing the State of the Art for Integration

We want a guaranteed, adaptive algorithm to approximate

ż 1

0

fpxq dx to within a

user-specified error tolerance ε.Issues Solutions, integral g(¨)

§ Requiring f P is

impractical and leads to automatic,but non-adaptive algorithms.

§ Requiring f P allows one toreliably estimate the size of the

containing f and leads to an

adaptive algorithm.§ Using Tnpfq ´ Tn{2pfq, etc. to

estimate error fails for flukyintegrands.

§ Instead estimate error in terms of adata-driven bound on

ş1

0|f2pxq| dx.

§ Spiky integrands can be hard. § Use a bigger .

§ Existing adaptive algorithms have noguarantees.

§ We provide guarantees that thetolerance will be met and the costwill not be excessive.



Higher Order Univariate Integration

Higher Order Univariate Integration (Yizhi Zhang)

§ The trapezoidal rule will always need at least a cost of O

¨

˝

d

ş1

0|f2pxq| dx

ε

˛

‚.

§ Simpson’s rule has the promise of a smaller cost, namely,

O

¨

˝

«

ş1

0

∣∣f p4qpxq∣∣ dxε

ff1{4˛

‚.

§ Key hurdles:

§ Modify the definition in terms of higher derivatives.§ Derive the error in approximating the weaker norm involving the second

derivative in terms of the data.§ Modify the bump function argument to determine the lower bound on the cost

of the best algorithm (lower complexity bound).



An Open Integration Rule

An Open Integration Rule (You?)

§ The trapezoidal rule is closed because it uses values of the integrand at theendpoints of the interval. Thus, it cannot handle integrands that blow up atthe endpoints.

§ Open rules do not evaluate the integrand at the endpoints of the interval,e.g.,

rTnpfq “1

2n

„

f

ˆ

1

3n

˙

` f

ˆ

2

3n

˙

` f

ˆ

4

3n

˙

` f

ˆ

5

3n

˙

` ¨ ¨ ¨

`f

ˆ

1´2

3n

˙

` f

ˆ

1´1

3n

˙

§ Key hurdles:§ Derive the error bound for this new rule.§ Derive the error in approximating the weaker norm involving the second

derivative in terms of the data.§ Modify the bump function argument to determine the lower bound on the cost




A Different Cone of Integrands

A Different (Martha Razo, Sunny Yun, Me)

§ The of integrands assumed by integral is

τ“

"

f :

ż 1

0


ż 1

0

∣∣f 1pxq ´ fp1q ` fp0q∣∣dx*which leads to a computational cost of O

ˆ

b

τş1

0|f2pxq| dx{ε

˙

. It would be

better without the (large) τ factor.§ What if we changed the of integrands to

τ“

"

f :

ż 1

0

∣∣f2pxq∣∣dx´ F p2qpf, txiuni“0q ď cpδq

ż 1

0

∣∣f2pxq∣∣dxfor all 0 “ x0 ă x1 ă ¨ ¨ ¨ ă xn “ 1, δ “ max

ipxi`1 ´ xiq

)

where F p2qpf, txiuni“0q :“

n´1ÿ

i“1

∣∣∣∣fpxi`1q ´ fpxiq

xi`1 ´ xi´fpxiq ´ fpxi´1q

xi ´ xi´1

∣∣∣∣where cpδq Ó c0 ă 1 as δ Ó 0?





§ What if we changed the of integrands to

τ“

"

f :

ż 1

0


ż 1

0


ipxi`1 ´ xiq

)

where F p2qpf, txiuni“0q :“

n´1ÿ

i“1

∣∣∣∣fpxi`1q ´ fpxiq

xi`1 ´ xi´fpxiq ´ fpxi´1q

xi ´ xi´1

∣∣∣∣where cpδq Ó c0 ă 1 as δ Ó 0? Thenż 1

0

∣∣f2pxq∣∣dx ď F p2qpf, ti{nuni“0q

1´ cp1{nqinstead of

ż 1

0

∣∣f2pxq∣∣dx ď τFnpfq

1´ τ{p2nq

§ New algorithm: Iteratively double n until

F p2qpf, ti{nuni“0q

8n2r1´ cp1{nqsď ε.





τ“

"

f :

ż 1

0


ż 1

0


ipxi`1 ´ xiq

)

§ New algorithm: Iteratively double n until

F p2qpf, ti{nuni“0q

8n2r1´ cp1{nqsď ε.

§ Computational cost of the new algorithm (in # of trapezoids) is

ď 2min

"

n P N : 8n2r1´ cp1{nqs ě

ż 1

0

∣∣f2pxq∣∣dx{ε* .§ Key hurdles:

§ Convince others that this does not make the problem trivial.§ Modify the bump function argument to determine the lower bound on the cost




Relative Error for Univariate Integration

Relative Error for Univariate Integration (You?)

§ The analysis in Clancy et al. (2014) is for absolute error, i.e.,∣∣∣∣ż 1

0

fpxqdx´ Tnpfq

∣∣∣∣ ď ε

§ It would be nice to extend integral g to include relative error, e.g.,∣∣∣∣ż 1

0

fpxqdx´ Tnpfq

∣∣∣∣ ď εr

ż 1

0

fpxqdx or max

ˆ

εa, εr

ż 1

0

fpxqdx

˙

§ Lan Jiang is working on relative error for Monte Carlo methods.§ Key hurdles:

§ Modify the stopping criterion.

§ Get an expression for the cost in terms of εa,

ż 1

0

∣∣f2pxq

∣∣ dx, εr, andż 1

0

fpxq dx.

§ Modify the bump function argument to determine the lower bound on the costof the best algorithm (lower complexity bound).



Local Adaption for Univariate Integration

Local Adaption for Univariate Integration (You?)

§ The sample points in our integral g are evenly spaced.

§ But the sample points in MATLAB’s integral are concentrated where theintegrand is more variable. This makes it more efficient.

§ Yuhan Ding is working on local adaption for univariate functionapproximation.



Monte Carlo Methods for Computing Expectations of Random Variables

Monte Carlo Methods for Computing Expectations ofRandom Variables (Lan Jiang, You?)

§ Sometimes we want to compute EpY q based on samples of a random variableY , such that

Probr|EpY q ´ µ| ď εs ě 99%.

§ The simplest way is to use µ “1

n

nÿ

i“1

Yi, where the Yi are IID.

§ The problem is how large to choose n (Hickernell et al., 2014). We focus on

the of random variables with kurtosis bounded by κmax.

§ The problem of Bernoulli Y is an important special case.

§ This work can also be used to evaluate multidimensional integrals. IfY “ fpXq, where X has probability density function ρ, then

EpY q “ż

Rd

fpxq ρpxqdx.



Multilevel Monte Carlo Methods

Multilevel Monte Carlo Methods (Aleks Borkovskiy)

§ Sometimes we want to compute EpY q based on samples of a random variableY , but getting just one Y costs an infinite amount of time.

§ However, one can write Y p0q “ 0

Y “ pY p1q ´ Y p0qq ` pY p2q ´ Y p1qq ` ¨ ¨ ¨ ` pY pLq ´ Y pL´1qq ` pY ´ Y pLqq

where Y p`q only costs $p`q operations, and Y ´ Y pLq Ñ 0 as LÑ8

§ The multilevel idea is to use a lot of samples to approximate EpY p1q ´ Y p0qq,fewer samples to approximate EpY p2q ´ Y p1qq, . . .

§ How many samples are needed for each piece? Again should look at a

of random variables.



Quasi-Monte Carlo Methods for Computing Multidimensional Integrals

Quasi-Monte Carlo Methods for ComputingMultidimensional Integrals (Siyuan, Tony, You?)

§ Sometimes we want to compute a multidimensional integralż

r0,1qdfpxqdx (perhaps after a suitable variable transformation).

§ Quasi-Monte Carlo methods choose txiu8i“0 where the first 1, 2, 4, 8, . . .

points are evenly distributed on r0, 1qd. Then∣∣∣∣∣ż

r0,1qdfpxqdx´

1

2m

2m´1ÿ

i“0

fpxiq

∣∣∣∣∣ ď Dptxiu8i“0,mqV pfq

where Dptxiu8i“0,mq is the discrepancy (measure of unevenness) of the first

2m points, and V pfq is the variation (measure of roughness) of the integrand.§ We want to choose m to make∣∣∣∣∣

ż

r0,1qdfpxqdx´

1

2m

2m´1ÿ

i“0

fpxiq

∣∣∣∣∣ ď ε.



Quasi-Monte Carlo Methods for Computing Multidimensional Integrals

Quasi-Monte Carlo Methods for ComputingMultidimensional Integrals (Siyuan, Tony, You?)

∣∣∣∣∣ż

r0,1qdfpxqdx´

1

2m

2m´1ÿ

i“0

fpxiq

∣∣∣∣∣ ď Dptxiu8i“0,mqV pfq ď ε for m “?

§ Some results doing an alternative error analysis using Fourier series.

§ Again focus on a of integrands.

§ Key hurdles:§ How well does it work in practice?§ Relating the of integrands to traditional sets of integrands.

§ Determining the lower bound on the cost of the best algorithm (lowercomplexity bound).



Univariate Function Approximation

Approximation or Recovery of Univariate Functions(Yuhan Ding, You?)

§ If you can choose the data sites x1, . . . , xn P ra, bs, how large should n be,where should the xi, and what algorithm A should be used to obtain

supaďxďb

|fpxq ´Apfqpxq| ď ε.

§ We have results for a of functions like those studied for univariate

integration and choosing A to be the piecewise linear spline (connect thedots with line segments) (Clancy et al., 2014).

§ Key extensions being studied: relative error, local adaption, higher ordermethods, etc.



Multivariate Function Approximation Using Kernels

Approximating Multivariate Functions (Xuan Zhou, You?)Given a symmetric, positive definite kernel, K : X ˆ X Ñ R, where X P Rd, thedata sites x1, . . . ,xn P X , and the function values yi “ fpxiq, i “ 1, . . . , n, onemay approximate f by f defined as

fpxq “nÿ

j“1

Kpx,xjqcj ,

¨

˚

˚

˚

˝

c1c2...cn

˛

‹

‹

‹

‚

“

¨

˚

˚

˚

˝

Kpx1,x1q Kpx1,x2q ¨ ¨ ¨ Kpx1,xnqKpx2,x1q Kpx2,x2q ¨ ¨ ¨ Kpx2,xnq

......

. . ....

Kpxn,x1q Kpxn,x2q ¨ ¨ ¨ Kpxn,xnq

˛

‹

‹

‹

‚

loooooooooooooooooooooooooomoooooooooooooooooooooooooon

K

´1¨

˚

˚

˚

˝

y1y2...yn

˛

‹

‹

‹

‚

∣∣∣fpxq ´ fpxq∣∣∣2 ď «

Kpx,xq ´ÿ

i,j“1

Kpx,xiqkp´1qij Kpxj ,xq

ff

‖f‖2K ,

where K´1 “

´

kp´1qij

¯n

i,j“1, and ‖f‖K denotes norm of f in the Hilbert space

with reproducing kernel [email protected] Automatic Algorithms IIT, 5/20/2014 27 / 35


Multivariate Function Approximation Using Kernels

Approximating Multivariate Functions (Xuan Zhou, You?)

fpxq “nÿ

j“1

Kpx,xjqcj , c “ K´1y

∣∣∣fpxq ´ fpxq∣∣∣2 ď «

Kpx,xq ´ÿ

i,j“1

Kpx,xiqkp´1qij Kpxj ,xq

ff

‖f‖2K ,

§ How large should n be to make

supxPX

∣∣∣fpxq ´ fpxq∣∣∣ ď ε?

§ We know that∥∥f∥∥

Kď ‖f‖K , but we need an upper bound on ‖f‖K .

§ What is the right of functions? E.g.,

“ tf : ‖f‖K ď τ ‖f‖weaku or tf : ‖f‖K ´∥∥f∥∥

Kď cpδq ‖f‖Ku



Guaranteed Automatic Integration Library (GAIL)

After Publishing the Theorems, Then What?

My collaborators and I believe that good numerical algorithms should bepublished. They should be

§ Theoretically justified to provide clear warnings, prices (computational costs),and low price guarantees (complexity bounds),

§ Efficiently implemented so that they use as little time and memory aspossible,

§ Accessible to users, have clear documentation, provide an intuitiveuser-interface, and be available for improvement, and

§ Robust in the sense of providing reproducible results, catching erroneousinputs, and being thoroughly tested.

We have tried to fulfill these ideals with the Guaranteed Automatic IntegrationLibrary (GAIL) (Choi et al., 2013–2014)




Guaranteed Automatic Integration Library (GAIL) (Lots ofFolks, Including You?)

§ Version 1.3 (February 14, 2014) includes integral g.m, meanMC g.m,cubMC g.m, and funappx g.m

§ Version 2.0 (exp. September 2014) will hopefully include some of thefollowing:

§ quasi-Monte Carlo—cubSobol g.m, cubLattice g.m§ Monte Carlo for Bernoulli—meanBernoulli g.m§ multi-level Monte Carlo—meanMLMC g.m§ multivariate function approximation§ relative error criterion§ higher order methods (for one dimensional problems)§ local adaption (for one dimensional problems)

§ Theory developed in Hickernell et al. (2014) and Clancy et al. (2014). Shouldapply to other problems where

solutionpcfq “ c solutionpfq for all c ą 0.




The GAIL Site, code.google.com/p/gail/


http://code.google.com/p/gail/

code.google.com/p/gail/



GAIL’s Implementation of integral g

function [q,out param] = integral g(varargin)% INTEGRAL G 1 D guaranteed function integration using trapezoidal ...

rule%% Description%% q = INTEGRAL G(f) computes q, the definite integral of function f% on the interval [0,1] by trapezoidal rule with% in a guaranteed absolute error of 1e´6. Default starting number of% sample points taken is 100 and default cost budget is 1e7. ...

Input f is a% function handle. The function y = f(x) should accept a vector ...

argument% x and return a vector result y, the integrand evaluated at each ...

element% of x.




GAIL’s Implementation of integral g cont’d

%% in param.nlo ´ ´ lowest initial number of function values used, ...

default% value is 10%% in param.nhi ´ ´ highest initial number of function values used,% default value is 1000%% in param.nmax ´ ´ cost budget (maximum number of function values),% default value is 1e7%% q = INTEGRAL G(f,a,b,abstol,nlo,nhi,nmax) computes q, the definite% integral of function f on the finite interval [a,b] by ...

trapezoidal rule% with the ordered input parameters, guaranteed absolute error ...

tolerance% abstol, lower bound of initial number of points in param.nlo, ...

higher




GAIL’s Implementation of integral g cont’d

% more points than cost budget, false otherwise.%% out param.tauchange ´ ´ it is true if the cone constant has been% changed, false otherwise. See [1] for details. If true, you may ...

wish to% change the input in param.ninit to a larger number.%% out param.npoints ´ ´ number of points we need to% reach the guaranteed absolute error tolerance abstol.%% out param.errest ´ ´ approximation error defined as the differences% between the true value and the approximated value of the integral.%% out param.nlo ´ ´ lowest initial number of function values%% out param.nhi ´ ´ highest initial number of function values%% out param.ninit ´ ´ initial number of points we use, computed ...

by nlo




References

Burden, R. L. and J. D. Faires. 2010. Numerical analysis, Ninth Edition, Cengage Brooks/Cole,Belmont, CA.

Choi, S.-C. T., Y. Ding, F. J. Hickernell, L. Jiang, and Y. Zhang. 2013–2014. GAIL: GuaranteedAutomatic Integration Library (versions 1, 1.3).

Clancy, N., Y. Ding, C. Hamilton, F. J. Hickernell, and Y. Zhang. 2014. The cost ofdeterministic, adaptive, automatic algorithms: Cones, not balls, J. Complexity 30, 21–45.

Hale, N., L. N. Trefethen, and T. A. Driscoll. 2012. Chebfun version 4.

Hickernell, F. J., L. Jiang, Y. Liu, and A. B. Owen. 2014. Guaranteed conservative fixed widthconfidence intervals via Monte Carlo sampling, Monte Carlo and quasi-Monte Carlo methods2012, pp. 105–128.

Lyness, J. N. 1983. When not to use an automatic quadrature routine, SIAM Rev. 25, 63–87.

The MathWorks, Inc. 2014. MATLAB 8.3, Natick, MA.


fred j. hickernell - iitmeshfree-methods-seminar/presentations/... · 2014. 5. 20. · introduction...

Documents