Lecture 6 - Single Variable Lecture 6 - Single Variable Problems & Systems of Problems & Systems of

EquationsEquationsCVEN 302

June 14, 2002

Lecture’s GoalsLecture’s Goals

• Introduction of Solving Equations of 1 Variable

– Newton’s Method– Muller’s Method– Fixed Point Iteration

Lecture’s GoalsLecture’s Goals

• Introduction to Systems of Equations• Discuss how to solve systems

– Gaussian Elimination– Gaussian Elimination with Pivoting– Tridiagonal Solver

• Problems with the technique• Examples

Newton’s MethodNewton’s Method

Do while |x2 - x1| >= tolerance value 1

or |f(x2)|>= tolerance value 2

or f’(x1) ~= 0

Set x2 = x1 - f(x1)/f’(x1)

Set x1 = x2;

END loop

Newton’s MethodNewton’s Method

• Same example problem f(x) = x5 + x3 + 4x2 - 3x - 2

and f’(x) = 5x4 + 3x2 + 8x - 3

• roots are between (-1.7,-1.3), (-1,0), & (0.5,1.5)

Newton’s MethodNewton’s Method x1 f(x1) f’(x1) f(x1)/f’(x1) x2

-2.0000 -20.0000 73.0000 -0.27397 -1.7260

-1.7260 -5.3656 36.5037 -0.14699 -1.5790

-1.5790 -1.0423 22.9290 -0.04546 -1.5335

-1.5335 -0.0797 19.4275 -0.00410 -1.5294

-1.5294 -0.0005 19.1381 -0.00003 -1.5294

etc. etc. etc. etc. etc.

Muller’s MethodMuller’s Method

• Muller’s method is an interpolation method that uses quadratic interpolation rather than linear.

• A second degree polynomial is used to fit three points in the vicinity of the root.

Muller’s MethodMuller’s MethodDo while |x2 - x1| >= tolerance value 1

or |f(x3)| >= tolerance value 2

c1 = (f(x2)-f(x1))/(x2 - x1)

c2 = (f(x3)-f(x2))/(x3 - x2)

d1 = (c2-c1)/(x3 - x1)

s = c2 + d1 *(x3 - x2)

x4 = x3 - 2*f(x3)/

[s + sign(s)*sqrt( s^2 -4*f(x3)*d1]

Set x1 = x2;

Set x2 = x3;

Set x3 = x4;

END loop

Muller’s MethodMuller’s Method

This method uses a Newton form of an interpolating polynomials.

Example ProblemExample Problem

f(x) = x5 + x3 + 4x2 - 3x - 2

The roots are around (-2,-1),(-1,0) and (0,1)

Look at (1,0) choice three points, 1.5, 0.5, 0.25

Muller’s MethodMuller’s Method

x1 f(x1) x2 f(x2) x3 f(x3) c1 c2 d1 s1 x4

1.5000 13.4688 0.2500 -2.4834 0.5000 -2.3438 12.7617 0.5586 12.2031 3.6094 0.8145

0.2500 -2.4834 0.5000 -2.3438 0.8145 -0.8900 0.5586 4.6205 7.1938 6.8839 0.9300

0.5000 -2.3438 0.8145 -0.8900 0.9300 0.1698 4.6205 9.1854 10.6157 10.4101 0.9134

0.8145 -0.8900 0.9300 0.1698 0.9134 -0.0049 9.1854 10.5319 13.6309 10.3057 0.9139

etc. etc. etc. etc. etc. etc. etc. etc. etc. etc. etc.

Fixed-Point Iteration MethodFixed-Point Iteration Method

• The method uses an iterative scheme to find the root.

• The equation is rewritten to obtain new equation in terms of x.

f(x) = a3 x3 + a2 x2 + a1 x + a0 = 0

g(x) = - [( a3/a1 ) x3 + (a2 /a1) x2 + (a0 /a1) ]

Fixed-Point Iteration MethodFixed-Point Iteration Method

• Problem is that the method only converge for a small range.

| g’(x) | < 0.5

Fixed-Point Iteration MethodFixed-Point Iteration MethodRewrite f(x) -> g(x)

IF | g’(x) |< 0.5 Do while |xk+1 - xk| >= tolerance

value xk+1 = g(xk);

k = k+1; END loopENDIF

Fixed Point IterationFixed Point Iteration

From the book:f(x) = 5x3 -10x + 3

g(x) = 0.5x3 + 0.3g’(x) = 1.5x2

for 0.1<x<0.5 | g’(x)| < 0.5

Functional Plot

-20

-15

-10

-5

0

5

10

15

-3 -2 -1 0 1 2

x value

f(x)

Fixed Point Iteration MethodFixed Point Iteration Method

• The program “demoFixedPoint” test the convergence of the point.

• The program is limited to small range of values

• demoFixed(0.1,0.0,0.5)

Systems of EquationsSystems of Equations

Simple Linear OscillatorSimple Linear Oscillator

The spring-mass system canbe described with a series ofequations to model thephysical behavior. Thedisplacement of the massesare given as “u”, k representsthe stiffness of the springsand M represents the mass ofeach member.

Simple Linear OscillatorSimple Linear Oscillator

The free body diagrams ofcomponents of the springmass system can berepresented. Using theequilibrium equations, thestatic behavior of the modelcan be determined.

Simple OscillatorSimple Oscillator

• The equations can be written from the free body diagrams.

• The matrix and vectors can be obtained from the equations.

123212

112312

2 uukukPWWuukuk

2

1

2

1

313

332

2 WPW

uu

kkkkkk

Simple 3-D frameSimple 3-D frame

A force is applied to the apexof a simple 3-D frame. Wewould like to determine theforces in each of themembers of the frame. Witha FBD, the static equilibriumequations can be derived.

Simple 3-D FrameSimple 3-D Frame

The set of 3 equilibrium equations and 3 unknowns can beobtained from the FBD of the frame. Place the equations in a

matrix format.

0llF

llF

llFcosF 0F

0ll

Fll

FcosF 0F

0llF

llFcosF 0F

3

3z3

2

2z2

1

1z1z

3

3y3

2

2y2y

2

2x2

1

1x1x

Simple 3-D FrameSimple 3-D Frame

The frame is represented as a matrix with 3 unknowns. Theforces are normalized with respect to the applied force, F.

Basic PrinciplesBasic Principles

• The general description of a set of linear equations in the matrix form:

[A]{x} = {b}

– [A] ( m x n ) matrix– {x} ( n x 1 ) vector– {b} (m x 1 ) vector

Description of the linear set of Description of the linear set of equationsequations

• Write the equations in natural form• Identify unknowns and order them• Isolate unknowns• Write equations in matrix form

Types of Matrix FormulationTypes of Matrix Formulation

( m x n ) Array

If m = n The solution of [A]{x} ={b} with n unknowns and m equations

If m > n The system is overdetermined system (Least Square Problems)

If m < n The system is underdetermined system (Optimization Problems)

Matrix RepresentationMatrix Representation

n

2

1

n

2

1

nnn2n1

n22221

1n1211

b

bb

x

xx

aaa

aaaaaa

bxA

ConsistencyConsistency

[A]{x} = {b}

• The problem is consistent, if a solution exists for the problem.

• The problem is inconsistent, if there is no solution for the problem.

Rank of MatrixRank of Matrix

• If rank(A) = n and is consistent, A has an unique solution exists

• If rank(A) = n and is inconsistent, A has no solution exists

• If rank(A) < n and system is consistent, A has an infinite number of solutions

Rank of a matrix is the number of linearly independent column vectors in the matrix. For n x n matrix, A:

MatrixMatrix

For an n x n system, rank(A) = n automatically guaranteesthat the system is consistent.• The columns of A are linearly independent• The rows of A are linearly independent• rank(A) = n• det(A) ~= 0• A-1 exists;• The solution to [A]{x} ={b} exist and is unique.

Matrix DefinitionMatrix Definition

Consider

y = -2.0 x + 6 y = 0.5 x + 12 unknowns x , y andrank is 2

Consider

y = -2 x + 6 y = -2 x + 52 unknowns x , y and rank is

1 and is inconsistent

1

615.012yx

56

1212yx

Gaussian EliminationGaussian Elimination

Gaussian elimination is a fundamentalprocedure for solving “linear” sets ofequation general it is applied to a squarematrix.

Gaussian EliminationGaussian Elimination

There are two phases to the solving technique• Elimination --- use row operations to convert

the matrix into an upper diagonal matrix. (The elimination phase, which takes the the most effort and most susceptible to corruption by round off)

• Back substitution -- Solve x using a back substitution.

Gaussian Elimination AlgorithmGaussian Elimination Algorithm

[A]{x} ={b}

• Augment the n x n coefficient matrix with the vector of right hand sides to form a n x (n+1)

• Interchange rows if necessary to make the value a11 with the largest magnitude of any coefficient in the first row

• Create zero in 2nd through nth row in first row by subtracting ai1 / a11 times first row from ith row

Gaussian Elimination AlgorithmGaussian Elimination Algorithm

• Repeat (2) & (3) for second through the (n-1)th rows, putting the largest magnitude coefficient in the diagonal by interchanging rows (consider only row j to n ) and then subtract times the jth row from the ith row so as to create zeros in all positions of jth column below the diagonal at conclusion of this step the system is upper triangular

• Solve for n from nth equation xn = an,n+1 / ann

• Solve for xn-1 , xn-2 , ...x1 from the (n-1)th through the first xi =(ai,n+1 - j=i+1,n aj1 xj ) / aii

Example 1Example 1

X1 + 3X2 = 5

2X1 + 4X2 = 6

65

XX

4231

2

1

Example 2Example 2

-3X1 + 2X2 - X3 = -1

6X1 - 6X2 + 7X3 = -7

3X1 - 4X2 + 4X3 = -6

671

XXX

443766123

3

2

1

Computer ProgramComputer Program

The program GEdemo(A,b) does the Gaussianelimination for a square matrix (nxn). It does not do any pivoting and works for only one {b} vector.

Test the ProgramTest the Program

• Example 1• Example 2• New Matrix

2X1 + 4X2 - 2 X3 - 2 X4 = - 4

1X1 + 2X2 + 4X3 - 3 X4 = 5

- 3X1 - 3X2 + 8X3 - 2X4 = 7

- X1 + X2 + 6X3 - 3X4 = 7

Problem with Gaussian Problem with Gaussian EliminationElimination

• The problem can occur when a zero appears in the diagonal and makes a simple Gaussian elimination impossible.

• Pivoting changes the matrix so that it will become diagonally dominate and reduce the round-off and truncation errors in the solving the matrix.

Computer ProgramComputer Program

• GEPivotdemo(A,b) is a program, which will do a Gaussian elimination on matrix A with pivoting technique to make matrix diagonally dominate.

• The program is modification to handle a single value of {b}

Question?Question?

• How would you modify the programs to handle multiple inputs?

• What is diagonal matrix, upper triangular matrix, and lower triangular matrix?

• Can you do a column exchange and how would you handle the problem if it works?

Question?Question?

• What happens with the following example?

0.0001X1 + 0.5 X2 = 0.5

0.4000X1 - 0.3 X2 = 0.1• What happens is the second equation

becomes: 0.4000X1 - 2000 X2 = -2000

Gaussian EliminationGaussian Elimination

• If the diagonal is not dominate the problem can have round off error and truncation errors.

• The scaling will result in problems

SummarySummary

• Newton’s - need to know f(x) and f’(x) and an initial guess.

• Muller’s method -need to know the f(x) and 3 values.

• Fixed Point method looking at a point a iterate.

SummarySummary

• Matrix properties; consistency, rank, diagonally dominate, upper triangular, lower triangular

• Gaussian elimination is a method to solve for a set of linear equations

• Non-pivoting problems• Setting up the system of linear equations to avoid

problems

HomeworkHomework

• Check the Homework webpage