sec. 7.3: systems of linear algebraic equations; linear

Sec. 7.3: Systems of Linear Algebraic Equations; LinearIndependence, Eigenvalues, Eigenvectors

MATH 351

California State University, Northridge

April 13, 2014

MATH 351 (Differential Equations) Sec. 7.3 April 13, 2014 1 / 25

Systems of Linear Algebraic Equations

A set of n simultaneous linear algebraic equations in n variables

a11x1 + a12x2 + · · ·+ a1nxn = b1,...

an1x1 + an2x2 + · · ·+ annxn = bn,

(1)

can be written asAx = b, (2)

where the n × n matrix A and the vector b are given, i.e.,

A =

a11 a12 · · · a1n

......

. . ....

an1 an2 · · · ann

, and b =

b1

...bn

(3)

and the components of x are to be determined.

If b = 0, the system (1) (or (2)) is said to be homogeneous;

otherwise, it is homogeneous.


Possible solutions to the system (2)

If the coefficient matrix A is nonsingular, that is, if detA 6= 0, then there is aunique solution of the system (2). This is because

A is nonsingular =⇒ A−1∃ =⇒ A−1Ax = Ab =⇒ x = Ab (4)

If A is singular, that is, if detA = 0, then solutions of (2) either do not exist, or

do exist but not unique.

The homogeneous system Ax = 0 has (infinitely many) nonzerosolutions in addition to the trivial solution.The nonhomogeneous system Ax = b (with b 6= 0) has no solutionunless the vector b satisfies a certain further condition. This conditionis that

(b, y) = 0, (5)

for all vectors y satisfying A∗y = 0, where A∗ is the adjoint of A. Ifcondition (5) is met, then the system (2) has (infinitely many)solutions. These solutions are of the form

x = x(0) + ξ (6)

wherex(0) is a particular solution of Eq. (2),ξ is the most general solution of the homogeneous system Ax = 0.


To solve particular systems,

it is generally best to use row reduction to transform the system into a much simplerone from which the solutions, if there are any, can be written down easily.

To do this efficiently, we can form the augmented matrix

A|b =

a11 · · · a1n | b1

...... |

...an1 · · · ann | bn

(7)

We now perform row operations on the augmented matrix so as to transform A into anupper triangular matrix—that is, a matrix whose elements below the main diagonal areall zero.


Example 1

Solve the system of equations

x1 − x3 = 0,3x1 + x2 + x3 = 1,−x1 + x2 + 2x3 = 2.

(8)

Solution.


Example 2

Discuss solutions of the system

x1 − 2x2 + 3x3 = b1,−x1 + x2 − 2x3 = b2,2x1 − x2 + 3x3 = b3,

(9)

for various values of b1, b2 and b3.

Solution.


Remark: Row reduction is also useful in solving homogeneous systems and systems in

which the number of equations is different from the number of unknowns.


Linear Dependence and Independence

linearly dependent

A set of k vectors x(1), . . . , x(k) is said to be linearly dependent if there exists a set ofreal or complex number c1, . . . , ck , at least one of which is nonzero, such that

c1x(1) + · · ·+ ckx

(k) = 0. (10)

In other words, x(1), . . . , x(k) are linearly dependent if there is a linear relation amongthem.

linearly independent

If the only set c1, . . . , ck for which Eq. (10) is satisfied is

c1 = c2 = · · · = ck = 0, (11)

then x(1), . . . , x(k) are said to be linearly independent.


Consider now a set of n vectors, each of which has n components.

Let xij = x(j)i be the ith component of the vector x(j), and let X = (xij).

Then Eq. (10) can be written as

Thus, the set of vectors x(1), . . . , x(n) is linearly independent if and only if detX 6= 0.


Example 3

Determine whether the vectors

x(1) =

210

, x(2) =

010

, x(3) =

−120

(12)

are linearly independent or linearly dependent. If they are linearly dependent, find alinear relation among them.

Solution.


Remark:

The column (or row) vectors are linearly independent if and only if detA 6= 0.

If C = AB, and if the columns (or rows) of both A and B are linearly independent,then the columns (or rows) of C are also linearly independent.


Linear dependence and independence of a set of vector functions

A set of vector functions x(1)(t), . . . , x(k) defined on an interval α < t < β.

linear dependence and independence

The vectors x(1)(t), . . . , x(k) are said to be linearly dependent on α < t < β if thereexists a set of constant c1, . . . , ck , not all of which are zero, such that

c1x(1)(t) + · · ·+ ckx

(k)(t) = 0 (13)

for all t in the interval. Otherwise, x(1)(t), . . . , x(k) are said to be linearly independent.


Remark:

If x(1)(t), . . . , x(k) are linearly dependent on an interval, they are linearly dependentat each point in the interval.

If x(1)(t), . . . , x(k) are linearly independent on an interval, they may or may not belinearly independent at each point; they may, in fact, be linearly dependent at eachpoint, but with different sets of constants at different points.


Eigenvalues and Eigenvectors

The equationAx = y (14)

can be viewed as a linear transformation that maps (or transforms) a given vector x intoa new vector y.We are interested in vectors that are transformed into multiples of themselves.To find such vectors, we set

y = λx, (15)

where λ is a scalar proportionality factor, and seek solutions of the equation

Ax = λx, (16)

or(A− λI)x = 0. (17)

Eq. (17) has nonzero solutions if and only if λ is chosen so that

det(A− λI) = 0. (18)

Eq. (18) is called the characteristic equation of the matrix A

Values of λ that satisfy Eq. (18) may be either real- or complex-valued and arecalled eigenvalues of A.

The nonzero solutions of Eq. (16) or (17) that are obtained by using such a valueof λ are called the eigenvectors corresponding to that eigenvalue.


When A is a 2× 2 matrix,

then Eq. (17) is

and Eq. (18) becomes


Example 3

Find the eigenvalues and eigenvectors of the matrix

A =

(5 −13 1

). (19)

Solution.


Remark:

Eigenvectors are determined only up to an arbitrary nonzero multiplicativeconstant.

a normalized vector x: ‖x‖ = (x, x)1/2 = 1.


algebraic multiplicity and geometric multiplicity

If a given eigenvalue appears m times as a root of Eq. (18), then that eigenvalueis said to have algebraic multiplicity m.

Each eigenvalue has at least one associated eigenvector, and an eigenvalue ofalgebraic multiplicity m may have q linearly independent eigenvectors. The integerq is called the geometric multiplicity of the eigenvalue.

We have 1 ≤ q ≤ m.

If each eigenvalue of A is simple (has algebraic multiplicity 1), theneach eigenvalue also has geometric multiplicity 1.


Remark:

If λ1 and λ2 are two eigenvalues of A, and if λ1 6= λ2, then their correspondingeigenvectors x(1) and x(2) are linearly independent.

For any set λ1, . . . , λk of distinct eigenvalues: their eigenvectors x(1), . . . , x(k) arelinearly independent.

If each eigenvalue of n × n matrix A is simple, then the n eigenvectors of A, onefor each eigenvalue, are linearly independent.

If A has one or more repeated eigenvalues, then there may be fewer than n linearlyindependent eigenvectors associated with A, since for a repeated eigenvalue wemay have q < m. (in Sec. 7.8, we will see this fact may lead to complications inthe solution of systems of differential equations.)


Example 5

Find the eigenvalues and eigenvectors of the matrix

A =

3 2 42 0 24 2 3

(20)

Solution.


self-adjoint or Hermitian matrices: A∗ = A

The eigenvalues and eigenvectors of Hermitian matrices always have the following usefulproperties:

All eigenvalues are real;

There always exists a full set of n linearly independent eigenvectors, regardlessof the algebraic multiplicities of the eigenvalues;

If x(1) and x(2) are eigenvectors that correspond to different eigenvalues, then

(x(1), x(2)) = 0.

Corresponding to an eigenvalue of algebraic multiplicity m, it is possible to choosem eigenvectors that are mutually orthogonal. Thus the full set of n eigenvectorscan always be chosen to be orthogonal as well as linearly independent.


sec. 7.3: systems of linear algebraic equations; linear

Documents