cis 541 – numerical methods
DESCRIPTION
CIS 541 – Numerical Methods. Mathematical Preliminaries. Derivatives. Recall the limit definition of the first derivative. Partial Derivatives. Same as derivatives, keep each other dimension constant. (1,f’(t)). f(x). t. Tangents and Gradients. - PowerPoint PPT PresentationTRANSCRIPT
CIS 541 – Numerical Methods
Mathematical Preliminaries
April 22, 2023 OSU/CIS 541 2
Derivatives
• Recall the limit definition of the first derivative.
0
( ) ( )'( ) limh
dy f x h f xf xdx h
April 22, 2023 OSU/CIS 541 3
Partial Derivatives
• Same as derivatives, keep each other dimension constant.
0
( , , ) ( , , )limxh
f f x h y z f x y zfx h
April 22, 2023 OSU/CIS 541 4
Tangents and Gradients
• Recall that the slope of a curve (defined as a 1D function) at any point x, is the first derivative of the function.
• That is, the linear approximation to the curve in the neighborhood of t is l(x) = b + f’(t)x
f(x)
t
(1,f’(t))
x
y
April 22, 2023 OSU/CIS 541 5
Tangents and Gradients
• Since we also want this linear approximation to intersect the curve at the point t.
l(t) = f(t) = b + f’(t)t• Or, b = f(t) - f’(t)t• We say that the line l(x) interpolates the
curve f(x) at the point t.
April 22, 2023 OSU/CIS 541 6
Functions as curves
• We can think of the curve shown in the previous slide as the set of all points (x,f(x)).
• Then, the tangent vector at any point along the curve is
, ( ) 1, '( )d x f x f xdx
April 22, 2023 OSU/CIS 541 7
Side note on Curves
• There are other ways to represent curves, rather than explicitly.
• Functions are a subset of curves (x,y(x)).• Parametric equations represent the curve by the
distance walked along the curve (x(t),y(t)).Circle: (cos, sin)
• Implicit representations define a contour or level-set of the function: f(x,y) = c.
April 22, 2023 OSU/CIS 541 8
Tangent Planes and Gradients
• In higher-dimensions, we have the same thing:
• A surface is a 2D function in 3D:Surface = (x, y, f(x,y) )
• A volume or hyper-surface is a 3D function in 4D:
Volume = (x, y, z, f(x,y,z) )
April 22, 2023 OSU/CIS 541 9
Tangent Planes and Gradients
• The linear approximation to the higher-dimensional function at a point (s,t), has the form: ax+by+cz+d=0, or z(x,y) = …
• What is this plane?
April 22, 2023 OSU/CIS 541 10
Construction of Tangent Planes
Images courtesy of TJ Murphy: http://www.math.ou.edu/~tjmurphy/Teaching/2443/TangentPlane/TangentPlane.html
April 22, 2023 OSU/CIS 541 11
Construction of Tangent Planes
April 22, 2023 OSU/CIS 541 12
Construction of Tangent Planes
April 22, 2023 OSU/CIS 541 13
Tangent Planes and Gradients
• The formula for the plane is rather simple:
• z(s,t) = f(s,t) - interpolates• z(s+dx,t) = f(s,t) + fx(s,t)dx = b + adx
– Linear in dx
• Of course, the plane does not stay close to the surface as you move away from the point (s,t).
( , ) ( , ) ( , )( ) ( , )( )x yz x y f s t f s t x s f s t y t
April 22, 2023 OSU/CIS 541 14
Tangent Planes and Gradients
• The normal to the plane is thus:
• The 2D vector:is called the gradient of the function.
• It represents the direction of maximal change.
( , ), ( , ),1x yN f s t f s t
( , ), ( , )T
x yf s t f s t
April 22, 2023 OSU/CIS 541 15
Gradients
• The gradient thus indicates the direction to walk to get down the hill the fastest.
• Also used in graphics to determine illumination.
( , ) ( , ), ( , )T
f ff x y x y x yx y
April 22, 2023 OSU/CIS 541 16
Review of Functions
• Extrema of a function occur where f’(x)=0.• The second derivative determines whether
the point is a minimum or maximum.• The second derivative also gives us an
indication of the curvature of the curve. That is, how fast it is oscillating or turning.
April 22, 2023 OSU/CIS 541 17
The Class of Polynomials
• Specific functions of the form:
2 3 4 50 1 2 3 4 5
0
0 1 2 3 4 5
( )
( ( ( ( ( ) )))))
ii
i
p x a a x a x a x a x a x
a x
a x a x a x a x a x a
April 22, 2023 OSU/CIS 541 18
The Class of Polynomials
• For many polynomials, the latter coefficients are zero. For example:
p(x) = 3+x2+5x3
April 22, 2023 OSU/CIS 541 19
Taylor’s Series
• For a function, f(x), about a point c.
• I.E. A polynomial
2 3
( )
0
( ) ( )( ) ( ) ( )( ) ( ) ( )2! 3!
( ) ( )!
kk
k
f c f cf x f c f c x c x c x c
f x x ck
April 22, 2023 OSU/CIS 541 20
Taylor’s Theorem
• Taylor’s Theorem allows us to truncate this infinite series:
( )
10
( 1)1
1
( )( ) ( )!
( ) ( )( 1)!
( ) ( , )
knk
nk
nn
n
f cf x x c Ek
fE x cn
where x c x
April 22, 2023 OSU/CIS 541 21
Taylor’s Theorem
• Some things to note:1. (x-c)(n+1) quickly approaches zero if |x-c|<<12. (x-c)(n+1) increases quickly if |x-c|>>13. Higher-order derivatives may get smaller (for
smooth functions).
April 22, 2023 OSU/CIS 541 22
Higher Derivatives
• What is the 100th derivative of sin(x)?
• What is the 100th derivative of sin(3x)?– Compare 3100 to 100!
• What is the 100th derivative of sin(1000x)?
April 22, 2023 OSU/CIS 541 23
Taylor’s Theorem
• Hence, for points near c we can just drop the error term and we have a good polynomial approximation to the function (again, for points near c).
• Consider the case where (x-c)=0.5
• For n=4, this leads to an error term around 2.6*10-4 f()• Do this for other values of n.• Do this for the case (x-c) = 0.1
( 1)
1 1
( ) 1( 1)! 2
n
n n
fEn
April 22, 2023 OSU/CIS 541 24
Some Common Derivatives
1( )
(sin ) cos
(cos ) sin
( )
1(ln )
n n
x x
d ax naxdxd x xdxd x xdxd e edxd xdx x
1( )
(sin ) cos
(cos ) sin
( )
1(ln )
n n
u u
d duau naudx dxd duu udx dxd duu u chain ruledx dxd due edx dxd duudx u dx
April 22, 2023 OSU/CIS 541 25
Some Resulting Series
• About c=02 3 4
1 2 2 3 3
1 2 2 3 3
2 3 4
12! 3! 4!
( 1) ( 1)( 2)( )2! 3!
1 2 31 1 1 1
1
x
n n n n n
n n n n
x x xe x
n n n n na x a na x a x a x
n n na a x a x a x
x x x x xx
April 22, 2023 OSU/CIS 541 26
Some Resulting Series
• About c=03 5 7
2 4 6
2 3 4 5
3 5 7
sin3! 5! 7!
cos 12! 4! 6!
ln(1 ) 1 12 3 4 5
1ln 2 1 11 3 5 7
x x xx x
x x xx
x x x xx x x
x x x xx xx
April 22, 2023 OSU/CIS 541 27
Book’s Introduction Example
• Eight terms for first series not even yielding a single significant digit.
• Only four for second serieswith foursignificantdigits.
3 5 7
ln 2 0.6931471801 1 1 1 1 1 1ln(1 1) 12 3 4 5 6 7 8
0.63452
1 1 111 1 3 3 33ln 21 3 3 5 713
0.69313
April 22, 2023 OSU/CIS 541 28
Mean-Value Theorem
• Special case of Taylor’s Theorem, where n=0, x=b.
• Assumes f(x) is continuous and its first derivative exists everywhere within (a,b).
1( ) ( )( ) ( ) ( ) ( , )
f b f a Ef a b a f a b
April 22, 2023 OSU/CIS 541 29
Mean-Value Theorem
• So what!?!?! What does this mean?• Function can not jump away from current value
faster than the derivative will allow.
f(x)
a b secant
April 22, 2023 OSU/CIS 541 30
Rolles Theorem
• If a and b are roots (f(a)=f(b)=0) of a continuous function f(x), which is not everywhere equal to zero, then f’(t)=0 for some point t in (a,b).
• I.e., What goes up, must come down.
f(x)
a b
f’(t)=0
t
April 22, 2023 OSU/CIS 541 31
Caveat
• For Taylor’s Series and Taylor’s Theorem to hold, the function and its derivatives must exist within the range you are trying to use it.
• That is, the function does not go to infinity, or have a discontinuity (implies f’(x) does not exist), …
April 22, 2023 OSU/CIS 541 32
Implementing a Fast sin()
const int Max_Iters = 100,000,000;float x = -0.1;float delta = 0.2 / Max_Iters;float Reimann_sum = 0.0;for (int i=0; i<Max_Iters; i++){
Reimann_Sum += sinf(x);x+=delta;
}Printf(“Integral of sin(x) from –0.1->0.1 equals:
%f\n”, Reimann_Sum*delta );
April 22, 2023 OSU/CIS 541 33
Implementing a Fast sin()
const int Max_Iters = 100,000,000;float x = -0.1;float delta = 0.2 / Max_Iters;float Reimann_sum = 0.0;for (int i=0; i<Max_Iters; i++){
Reimann_Sum += my_sin(x); //my own sine funcx+=delta;
}Printf(“Integral of sin(x) from –0.1->0.1 equals:
%f\n”, Reimann_Sum*delta );
April 22, 2023 OSU/CIS 541 34
Version 1.0
my_sin( const float x ){
float x2 = x*x;float x3 = x*x2;
return (x – x3/6.0 + x2*x3/120.0 );}
April 22, 2023 OSU/CIS 541 35
Version 2.0 – Horner’s Rule
Static const float fac3inv = 1.0 / 6.0f;Static const float fac5inv = 1.0 / 120.0f;
my_sin( const float x ){
float x2 = x*x;
return x*(1.0 – x2*(fac3inv - x2*fac5inv));}
April 22, 2023 OSU/CIS 541 36
Version 3.0 – Inline code
const int Max_Iters = 100,000,000;float x = -0.1;float delta = 0.2 / Max_Iters;float Reimann_sum = 0.0;for (i=0; i<Max_Iters; i++){
x2 = x*x;Reimann_Sum += x*(1.0–x2*(fac3inv-x2*fac5inv);x+=delta;
}Printf(“Integral of sin(x) from –0.1->0.1 equals:
%f\n”, Reimann_Sum*delta );
April 22, 2023 OSU/CIS 541 37
Timings
• Pentium III, 600MHz machineTime in seconds Result Max(
|sin(x)-my_sin(x)| )
Using sinf 27 -0.0041943
Version 1.0 20 -0.0041943 1.93765*10-11
Version 2.0 13 -0.0041943 8.0495*10-12
Version 3.0 2 -0.0041943 8.0495*10-12
April 22, 2023 OSU/CIS 541 38
Observations
• Is the result correct?• Why did we gain some accuracy with
version 2.0?• Is (–0.1,0.1) a fair range to consider?• Is the original sinf() function optimized?• How did we achieve our speed-ups?• We will re-examine this after Lab1.
April 22, 2023 OSU/CIS 541 39
Homework
• Read Chapters 1 and 2 for next class.• Start working on Lab 1 and Homework 1.