simultaneous linear equationsfaculty.msmc.edu/daven/numerical/pivoting1.pdfgaussian elimination one...
TRANSCRIPT
Simultaneous Linear Equations
3/7/2008 http://numericalmethods.eng.usf.edu
1
Topic: Gaussian Elimination
Gaussian Elimination
One of the most popular techniques for solving simultaneous linear equations of the form [ ][ ] [ ]CXA =
Consists of 2 steps
1. Forward Elimination of Unknowns.
2. Back Substitution
Forward Elimination
The goal of Forward Elimination is to transform the coefficient matrix into an Upper Triangular Matrix
1525 ���� 1525
7.00056.18.40
1525
���
�
�
���
�
�
−−→���
�
�
���
�
�
11214418641525
Forward Elimination
Linear EquationsA set of n equations and n unknowns
11313212111 ... bxaxaxaxa nn =++++ 11313212111 nn
22323222121 ... bxaxaxaxa nn =++++
nnnnnnn bxaxaxaxa =++++ ...332211
. .
. .
. .
Forward Elimination
Transform to an Upper Triangular MatrixStep 1: Eliminate x1 in 2nd equation using equation 1 as the pivot equation
1Eqn ��)(
121
11
aa
Eqn ��
���
�
Which will yield
111
211
11
21212
11
21121 ... b
aa
xaaa
xaaa
xa nn =+++
Forward Elimination
Zeroing out the coefficient of x1 in the 2nd equation.Subtract this equation from 2nd equation
121
2121
221221
22 ... baa
bxaaa
axaaa
a nnn −=���
�
�−++��
�
�
�− 1
1121
112212
1122 aaa nnn �
��
��
�
'2
'22
'22 ... bxaxa nn =++
nnn aaa
aa
aaa
aa
111
212
'2
1211
2122
'22
−=
−=
�
Or Where
Forward Elimination
Repeat this procedure for the remaining equations to reduce the set of equations as
11313212111 ... bxaxaxaxa nn =++++ 11313212111 ... bxaxaxaxa nn =++++'2
'23
'232
'22 ... bxaxaxa nn =+++
'3
'33
'332
'32 ... bxaxaxa nn =+++
''3
'32
'2 ... nnnnnn bxaxaxa =+++
. . .
. . .
. . .
Forward Elimination
Step 2: Eliminate x2 in the 3rd equation.Equivalent to eliminating x1 in the 2nd equation using equation 2 as the pivot equation.
)(2
3 3222
aa
EqnEqn �
�
���
�−
Forward Elimination
This procedure is repeated for the remaining equations to reduce the set of equations as
11313212111 ... bxaxaxaxa nn =++++'2
'23
'232
'22 ... bxaxaxa nn =+++
"3
"33
"33 ... bxaxa nn =++
""3
"3 ... nnnnn bxaxa =++
. .
. .
. .
Forward Elimination
Continue this procedure by using the third equation as the pivot equation and so on.
At the end of (n-1) Forward Elimination steps, the system of equations will look like:
'2
'23
'232
'22 ... bxaxaxa nn =+++
"3
"3
"33 ... bxaxa nn =++
( ) ( )11 −− = nnn
nnn bxa
. .. .. .
11313212111 ... bxaxaxaxa nn =++++
Forward Elimination
At the end of the Forward Elimination steps
���
���
���
���
���
���
2
1
n
n
b
b
x
x
aaa
aaaa'2
1'2
'23
'22
1131211
�
�
�����
������
�
=
�����
������
������
������
�− )-(n
nn
3
2
nnn
n
n
b
b
b
x
x
x
a
aa
aaa
1
"3
2
)1(
"3
"33
22322
����
�
�
Back Substitution
The goal of Back Substitution is to solve each of the equations using the upper triangular matrix.
��
��
��
��
��
�� 11131211 x baaa
���
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
�
3
2
1
3
2
1
33
2322
131211
xx
000
b
b
a
aa
Example of a system of 3 equations
Back Substitution
Start with the last equation because it has only one unknown
)1( −
=n
nbx )1( −=
nnn
nn a
bx
Solve the second from last equation (n-1)th
using xn solved for previously.
This solves for xn-1.
Back Substitution
Representing Back Substitution for all equations by formula
( ) ( )11
+=
−− −n
ji
iji
i xab
( )1
1
−+= −
=i
ii
ijjiji
i a
xabx For i=n-1, n-2,….,1
and
)1(
)1(
−
−
=n
nn
nn
n ab
x
Example: Rocket Velocity
The upward velocity of a rocket is given at three different times
Time, t Velocity, v
s m/s
5 106.85 106.8
8 177.2
12 279.2
The velocity data is approximated by a polynomial as:
( ) 12.t5 , 322
1 ≤≤++= atatatv
Find: The Velocity at t=6,7.5,9, and 11 seconds.
Example: Rocket Velocity
Assume( ) 12.t5 ,atatatv ≤≤++= 32
21
��
��
��
��
��
�� 11
21 1 vatt 1
Results in a matrix template of the form:
���
����
�
=���
����
����
����
� 3
2
323
222
11
v
v
a
a
tt
tt
3
2
Using date from the time / velocity table, the matrix becomes:
���
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
�
2.2792.1778.106
11214418641525
3
2
1
a
a
a
Example: Rocket Velocity
Forward Elimination: Step 1
=���
��
�− )64(25
12
RowRow
YieldsYields
���
�
�
���
�
�
−=���
�
�
���
�
�
���
�
�
���
�
�
−−2.27921.9681.106
aaa
11214456.18.40
1525
3
2
1
Example: Rocket Velocity
=���
��
�− )144(25
13
RowRow
Yields
Forward Elimination: Step 1
���
�
�
���
�
�
−−=
���
�
�
���
�
�
���
�
�
���
�
�
−−−−
0.33621.968.106
aaa
76.48.16056.18.40
1525
3
2
1
Yields
Example: Rocket Velocity
Yields
=−×���
��
�
−− )8.16(
8.42
3Row
Row
Forward Elimination: Step 2
Yields
���
�
�
���
�
�
−=���
�
�
���
�
�
���
�
�
���
�
�
−−735.0
21.968.106
aaa
7.00056.18.40
1525
3
2
1
This is now ready for Back Substitution
Example: Rocket Velocity
Back Substitution: Solve for a3 using the third equation
735.07.0 3 =a
7350.70
7350.
.a 3 =
0501. a 3 =
Example: Rocket Velocity
Back Substitution: Solve for a2 using the second equation
21.9656.18.4 32 −=−− aa
56.121.96 +− a8.4
56.121.96 32 −
+−=
aa
( )84
05015612196.-
...-a 2
+=
7019.a 2 =
Example: Rocket Velocity
Back Substitution: Solve for a1 using the first equation
8.106525 321 =++ aaa
58.106 32 aaa
−−=
2558.106
321
aaa
−−=
( )25
050.170.1958.1061
−−=a
2900.01 = a
Example: Rocket Velocity
Solution:The solution vector is
���
�
�
���
�
�
=���
�
�
���
�
�
050.170.19
2900.0
3
2
1
a
a
a
The polynomial that passes through the three data points is The polynomial that passes through the three data points is then:
( ) 322
1 atatatv ++=
125 ,050.170.192900.0 2 ≤≤++= ttt
Example: Rocket Velocity
Solution:Substitute each value of t to find the corresponding velocity
( ) ( ) ( )./ 69.129
050.1670.1962900.06 2
sm
v
=++=
( ) ( ) ( ).s/m1.165
050.15.770.195.72900.05.7v 2
=++=
( ) ( ) ( ).s/m8.201
050.1970.1992900.09v 2
=++=
( ) ( ) ( ).s/m8.252
050.11170.19112900.011v 2
=++=
./ 69.129 sm=
Pitfalls
Two Potential Pitfalls-Division by zero: May occur in the forward elimination steps. Consider the set of equations:
7710 32 =− xx
655
901.33099.26
7710
321
321
32
=+−=++
=−
xxx
xxx
xx
- Round-off error: Prone to round-off errors.
Pitfalls: ExampleConsider the system of equations:
Use five significant figures with chopping
���
�
�
���
�
�
−−
−
5156099.230710
���
�
�
���
�
�
2
1
x
x
x
���
�
�
���
�
�
6901.37
=���� − 515 ���� 3x ���� 6
At the end of Forward Elimination
���
�
�
���
�
�
−−
15005006001.000710
���
�
�
���
�
�
3
2
1
x
x
x
���
�
�
���
�
�
15004001.67
=
Pitfalls: Example
Back Substitution
99993.01500515004
3 ==x
��
��
=��
����
��
−−
001.67
6001.000710 1
x
x
5.1 001.0
6001.6 32 −=
−−= x
x
3500.010
077 321 −=
−+=
xxx
���
����
�
=���
����
����
����
�
−15004
001.61500500
6001.00
3
2
x
x
Pitfalls: Example
Compare the calculated values with the exact solution
[ ] ���
���
−=���
���
= 10
2
1
x
x
X exact
[ ]���
�
�
���
�
�
−−
=���
�
�
���
�
�
=99993.0
5.135.0
3
2
1
x
x
x
X calculated
[ ]����
��
−=����
��
=11
3
2
x
xX exact
Improvements
Increase the number of significant digitsDecreases round off error
Does not avoid division by zero
Gaussian Elimination with Partial PivotingAvoids division by zero
Reduces round off error
Partial Pivoting
Gaussian Elimination with partial pivoting applies row switching to normal Gaussian Elimination.
How?
At the beginning of the kth step of forward elimination, find the maximum of
pka
At the beginning of the k step of forward elimination, find the maximum of
nkkkkk aaa .......,,........., ,1+
If the maximum of the values is In the pth row, ,npk ≤≤
then switch rows p and k.
Partial Pivoting
What does it Mean?
Gaussian Elimination with Partial Pivoting ensures that each step of Forward Elimination is performed with the pivoting element |akk| having the largest absolute value.
Partial Pivoting: Example
Consider the system of equations
6x5xx5
901.3x3x099.2x3
7x7x10
321
321
21
=+−=++−
=−
6x5xx5 321 =+−
In matrix form
���
�
�
���
�
�
−−
5156099.230710
���
�
�
���
�
�
3
2
1
x
x
x
���
�
�
���
�
�
6901.37
=
Solve using Gaussian Elimination with Partial Pivoting using five significant digits with chopping
Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to follow the rules of Partial Pivoting, we switch row1 with row1.
���
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
�
−−
6901.37
5156099.230710
3
2
1
x
x
x
���
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
�
−−
5.2001.67
55.206001.000710
3
2
1
x
x
x
�
Performing Forward Elimination
Partial Pivoting: Example
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is switched with row 3
���
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
�
−−
5.2001.67
55.206001.000710
3
2
1
x
x
x
����
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
�
−
−
001.65.2
7
6001.0055.200710
3
2
1
x
x
x
Performing the row swap
Partial Pivoting: Example
Forward Elimination: Step 2
Performing the Forward Elimination results in:
���
�
�
���
�
�
=���
�
�
���
�
�
���
�
�
���
�
� −
002.65.2
7
002.60055.200710
3
2
1
x
x
x
Partial Pivoting: Example
Back Substitution
Solving the equations through back substitution
1002.6002.6
3 ==x
���
���
=���
���
���
��� −
5.27
55.200710 1
x
x002.6
15.255.2 2
2 =−
=x
x
010
077 321 =
−+=
xxx
���
����
�
=���
����
����
����
� 002.65.2
002.60055.20
3
2
x
x
Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it does illustrate the advantage of Partial Pivoting
[ ]���
�
�
���
�
�
−=���
�
�
���
�
�
=11
0
3
2
1
x
x
x
X exact[ ]���
�
�
���
�
�
−=���
�
�
���
�
�
=11
0
3
2
1
x
x
x
X calculated
Summary
-Forward Elimination
-Back Substitution
-Pitfalls
-Improvements
-Partial Pivoting