chebfun: numerical computing with functions

.

.

. ..

.

.

Chebfun: Numerical computing withfunctions

Alex TownsendOxford University

IMACS: Nonlinear Evolution Equations and Wave Phenomena

Alex Townsend Chebfun

.. Chebfun is about Chebyshev technology

Chebyshev technology is a powerful approach for computing withfunctions. It’s all based on polynomial interpolation at Chebyshevpoints:

xj = cos(jπ/n), 0 ≤ j ≤ n.

There exist fast, accurate algorithms for integration,differentiation, rootfinding, minimization, solution of ODEs, etc

www.chebfun.org


www.chebfun.org

.. The Chebfun team

Chebyshev technology can also bring us together. My friends:

Chebfun has resulted in a book: ATAP by L. N. Trefethen (2013).Alex Townsend Chebfun

.. The Chebfun philosophy

There are two visions behind Chebfun [Trefethen 2007]:

Continuous analogue of vectors Overload Matlab vectors(and recently matrices) to functions. sum, max,roots, norm, qr,. . .

Floating point analogy Extend IEEE arithmetic from doubles tofunctions. In IEEE doubles are rounded to 16 relativedigits, and in Chebfun functions are approximated to16 digits.

f = chebfun( @(x) sin(pi*x) );

length( f )

ans = 20

length( f.*f ) % without truncation length = 40

ans = 29


.. Interpolation at Chebyshev points

Key point 1: Polynomial interpolants in equispaced points in[−1, 1] can have very poor approximation properties, but inter-polants in Chebyshev points are excellent.

Runge function: f (x) =1

1 + 25x2, x ∈ [−1, 1]

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1Degree 20 equally−spaced interpolant

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1Degree 20 Chebyshev interpolant


.. The Runge region

At equally-spaced points a function on [−1, 1] must be analyticinside the Runge region, and there is a convergence rate vs.ill-conditioning trade-off [Platte, Trefethen, Kuijlaars 2011].

0.5255249145i0.5255249145i0.5255249145i

Runge region for equispaced interpolation

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

In contrast, if f is analytic in a Bernstein ellipse Eρ and boundedthere then its Chebyshev interpolant satisfies [ATAP 2013]

‖f − pn‖∞ ≤4Mρ−n

ρ− 1, M ∈ R.


.. Fast Fourier transform

Key point 2: Chebyshev points are equally-spaced when projectedonto the unit semi-circle.

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1Chebyshev points for n = 17

This fact allows us to convert between n + 1 values on the grid ton coefficients in O(n log n) operations:

values ←→︸︷︷︸DCT

coefficients


.. Integration

Key point 3: Gauss quadrature is optimal, but Clenshaw–Curtisquadrature is better.

I =

∫ 1

−1f (x)dx ≈

N∑k=0

wk f (xk) = IN .

Gauss–Legendre quadrature

|I − IN | ≤ 4E ∗2N+1

Clenshaw–Curtis quadrature

|I − IN | ≤ 4E ∗N

Gauss is “twice as good” as Clenshaw–Curtis.


.. Integration

The full factor of 2 is rarely seen [Trefethen 2008].

0 20 40 60 80 10010

−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

n

Err

or

1/(1+16x2)

C−CGauss

E2n+1*

En*

Kinkphenomenon

Gauss–Legendre requires O(n2) operations (due only to polynomialevaluation [Hale & T. 2013]) and Clenshaw–Curtis requiresO(n log n) operations (using the DCT for evaluation).


.. Rootfinding

Key point 4: Rootfinding for large degree polynomials is very prac-tical and remarkably important.

The roots of the polynomial

p(x) =n∑

k=0

akTk(x), Tk(x) = cos(k cos−1(x)),

are the eigenvalues of the colleague matrix [Specht 1960, Good1961]

C =

0 112

0 12

12

0 12

. . .. . .

. . .12

12

0

−

1

2an

a0 a1 a2 . . . an−1

.

For large degree polynomials we subdivide the interval for O(n2)complexity algorithm [Boyd 2003].


.. Rootfinding and global optimization

Two important consequences of robust rootfinding:...1 Locating breakpoints: Useful for commands such as abs,sign, max.

...2 Global optimization: Computed via roots(diff(f)).

x = chebfun(’x’, [0 10]);

h = max( sin(x) + sin(x.^2), 1-abs(x-5)/5 );

mx = max(h);

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2


DEMO


.. Chebop: Differential operators

Key point 1: Overload the Matlab backslash command \ foroperators [Driscoll, Bornemann & Trefethen 2008].

L = chebop( @(x,u) diff(u,2) - x.*u, [-30 30] );

L.lbc = 1; L.rbc = 0; u = L \ 0; plot(u)

−30 −20 −10 0 10 20 30−8

−6

−4

−2

0

2

4

6Solution to Airy equation


.. Spectral method basics

Given values on a Chebyshev grid, what are the values of thederivative on the same grid?:

x1

u1

u’1

x2

u2

u’2

x5

u5

u’5

x9

u9

u’9

Dn

u1...un

=

u′1...u′n

, Dn = diffmat(n).

For example, u′(x) + cos(x)u(x) is represented as

Ln = Dn + diag (cos(x1), . . . , cos(xn)) ∈ Rn×n


.. Imposing boundary conditions

Key point 2: Imposing boundary conditions by boundary borderingcan be automated.

Values Coefficients

BoundaryBordering

x . . . x

.... . .

...

x . . . x

x x

x. . .

. . .. . .

. . . x

x x

Can beautomated

Basis Re-combination

x . . . x

.... . .

...

x . . . x

x x

x. . .

. . .. . .

. . . x

x x

Preservesstructure

Dense, efficient fornonlinear

Sparse and can bewell-conditioned


.. Rectangular projection

Key point 3: Projection is essential for treating less standardboundary conditions.

x1

u1

x2

u2

x5

u5

x9

u9

y1

u’1

y2

u’2

y5

u’5

y5

u’5

Projection if there are four bcs

The second grid is chosen so the solution is interpolated on a gridof size n −m [Hale & Driscoll 2013]. Also allows us to imposenon-boundary conditions:

0.1u′′(x) + xu(x) = sin(x), u(−1) = 1,

∫ 1

−1u(s)ds = 0.


.. Four examples of ODEs

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−8

−6

−4

−2

0

2

4

6Potential barrier

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−2

−1.8

−1.6

−1.4

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0ODE with imposed breakpoint

x

0 5 10 15 20 25 30 35 40 45 50−50

0

50

100

150

200profit/loss versus underlying share price

share price in pounds

prof

it

0 2 4 6 8 10 12 14 16 180.7

0.8

0.9

1

1.1

1.2

1.3

1.4(non−linear) Lotka−Volterra predator−prey

x

RabbitsFoxes


.. AD and nonlinear ODEs

Key point 4: Automatic differentiation and Newton’s method al-lows for nonlinear differential equations.

Newton’s method for finding f (x) = 0,

x (k+1) = x (k) −[f(x (k)

)]−1f(x (k)

).

If N is an operator then the analogous iteration for findingN (u) = 0 is

u(k+1) = u(k) + v (k), N ′(u(k)

)v (k) = −N

(u(k)

),

where N ′(u) is the Frechet derivative of N .We use automatic differentiation to compute the Frechet derivative[Birkisson & Driscoll 2012].


.. A nonlinear ODE

% A solution to Carrier equation

N = chebop(@(x,u) 0.01*diff(u,2) + 2*(1-x.^2).*u + u.^2);

N.lbc = 0; N.rbc = 0;

N.init = chebfun(@(x) 2*x.*(x.^2-1).*(1-2./(1+20*x.^2));

u = N \ 1;

1 4 7 10 13 16 1910

−10

10−8

10−6

10−4

10−2

100

Norm of updates

Iteration number−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

−1.5

−1

−0.5

0

0.5

1

1.5

2A solution to Carrier equation

x

u

If no initial condition is supplied then chebop computes a smalldegree polynomial satisfying the linearized boundary conditions.


.. A fast and well-conditioned spectral method

Key point 5: Spectral methods do not have to result in dense,ill-conditioned matrices [Olver & T. 2013].

The idea is to use simple relations between Chebyshev polynomials:

dTk

dx=

{kUk−1 k ≥ 1

0 k = 0,Tk =

12 (Uk − Uk−2) k ≥ 212U1 k = 1

U0 k = 0.

Dn =

0 1

2. . .

n − 10

, Sn =

10 1

2−1

2 0 12

. . .. . .

. . .

−12 0 1

2

.


.. A fast and well-conditioned spectral method

Well-conditioned The condition number of the N × N linearsystem obtained by discretization is typically O(1).

Fast Solution The linear systems are sparse and can sometimesbe solved in O(N) operations (details in [Olver & T.2013]).

High accuracy The solution is typically accurate to 15-16 digits.

−1 −0.5 0 0.5 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

x

u(x)

0 0.5 1 1.5 2

x 104

10−15

10−10

10−5

100

105

||u1

.01n

−u

n|| 2

n


DEMO


.. Chebfun2 (released on 4th of March 2013)

Chebfun2 is a software project to extend Chebfun to scalar- andvector-valued functions of two variables [T. & Trefethen 2013a].

Based on low rank function approximation using Gaussianelimination.

Exploits 1D Chebyshev technology to perform 2Dcomputations. (Extremely important)

“Hello World”: My first example (back in October 2011):

Rank 1 Rank 3

Rank 5 Rank 10


.. Chebfun2: Chebfun in two dimensions

Key point 1: Gaussian elimination can be used for low rank functionapproximation.

The standard point of view:

A L U

A different, equally simple point of view [T. & Trefethen 2013b]:

A←− A− A(j , :)A(:, k)/A(j , k) (GE step for matrices)

f ←− f − f (x , :)f (:, y)/f (x , y) (GE step for functions)

Each step of GE is a rank-1 update.


.. SVD versus GE

The Singular Value Decomposition (SVD) can be used to find thebest rank-k approximation to a sufficiently smooth function f :

‖f − fk‖2L2 =∞∑

j=k+1

σ2j <∞, (continuous analogue of ‖A− Ak‖2F )

However, for analytic and differentiable functions GE gives anear-optimal rank-k approximation:

0 5 10 15 20 25 3010

−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

Rank of approximant

Rel

ativ

e er

ror

in L

2

SVD

GE

γ = 10γ = 1

γ = 100

0 50 100 150 20010

−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100

Rank of approximant

Rel

ativ

e er

ror

in L

2

SVD

GE

φ3,1

∈ C2

φ3,3

∈ C6

φ3,0

∈ C0


.. Exploiting 1D technology in 2D computations

Key point 2: Two dimensional computations can efficiently exploit1D technology.

f (x , y) ≈k∑

j=1

σjuj(y)vj(x)

Operations such as sum2, diff, and f(x,y) can be computedusing 1D Chebsyhev technology. For example:∫ 1

−1

∫ 1

−1f (x , y)dxdy ≈

k∑j=1

σj

∫ 1

−1uk(y)dy

∫ 1

−1vk(x)dx

F = @(x,y) exp(-(x.^2 + y.^2 + cos(4*x.*y)));

QUAD2D: I = 1.399888131932670 time = 0.0717 secs

SUM2: I = 1.399888131932670 time = 0.0097 secs


.. Zero contours and bivariate rootfinding

Key point 3: The existing Matlab command contourc can beused for rootfinding.

We can solve two very different rootfinding problems (both byusing contourc):

f (x , y) = 0, and f (x , y) = g(x , y) = 0.

−5 −4 −3 −2 −1 0 1 2 3 4 5−4

−3

−2

−1

0

1

2

3

4Zero contours

−1 −0.5 0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1The fourth SIAM digit challenge problem

Zero contours are approximated by complex-valued chebfuns!


.. Parametric surfaces (for fun)

Functions F : R2 → R3 can be used to represent surfaces witheach component of the vector-valued function represented by a lowrank approximant.

Parametric surfaces in Chebfun2 was an idea from Rodrigo Platte(Arizona State University).


DEMO


.. Chebfun References:

[Trefethen 2007] Computing numerically with functions instead ofnumbers (Math. Comp. Sci., 2007).

[Platte, Trefethen, Kuijlaars 2011] Impossibility of fast stableapproximation of analytic functions from equispaced samples,(SIREV 2011).

[Trefethen 2008] Is Gauss quadrature better than Clenshaw–Curtis,(SIREV, 2008).

[Hale, T. 2013] Fast and accurate computation of Gauss–Legendreand Gauss-Jacobi quadature nodes and weights, (to appear inSISC, 2013).

[Driscoll, Bornemann, Trefethen 2008] The chebop system forautomatic solution of differential equations (BIT Numer. Math.,2008).


.. Chebfun References:

[Hale, Driscoll 2013] Rectangular projection, (in preparation, 2013).

[Birkisson, Driscoll 2012] Automatic Frechet differentiation for thenumerical solution of boundary-value problems, (ACM Trans.Math. Softw., 2012).

[Olver, T. 2013], A fast and well-conditioned spectral method, (toappear in SIREV, 2013).

[T., Trefethen 2013a], An extension of Chebfun to two dimensions,(submitted, 2013).

[T., Trefethen 2013b], Gaussian elimination as an iterativealgorithm, (SIAM news, 2013).


chebfun: numerical computing with functions

Documents