lect2 linear algebra - carleton...
Post on 05-Aug-2020
7 Views
Preview:
TRANSCRIPT
Review of Linear Algebra
Dr. Chang Shu
COMP 4900C
Winter 2008
Linear Equations
A system of linear equations, e.g.
1114
242
21
21
=+
=+
xx
xx
can be written in matrix form:
=
1
2
114
42
2
1
x
x
or in general:
bAx =
Vectors
=
nx
x
x
xM
2
1
=
5
3
2
xe.g.
The length or the norm of a vector is
22
2
2
1 nxxxx +++= L
38532 222 =++=xe.g.
Vector Arithmetic
+
+=
+
=+
22
11
2
1
2
1
vu
vu
v
v
u
uvu
−
−=
−
=−
22
11
2
1
2
1
vu
vu
v
v
u
uvu
Vector addition
Vector subtraction
Multiplication by scalar
=
=
2
1
2
1
u
u
u
uu
α
ααα
Dot Product (inner product)
=
5
3
2
a
−=
2
3
4
b
[ ] 925)3(342
2
3
4
532 =⋅+−⋅+⋅=
−==⋅ babaT
nn
Tbababababa +++==⋅ L2211
Linear Independence
• A set of vectors is linear dependant if one of
the vectors can be expressed as a linear
combination of the other vectors.
• A set of vectors is linearly independent if
none of the vectors can be expressed as a
linear combination of the other vectors.
nnkkkkk vvvvv αααα +++++= ++−− LL 111111
Vectors and Points
Two points in a Cartesian coordinate system define a vector
P(x1,y1)
Q(x2,y2)
x
yv
−
−=
12
12
yy
xxv
P(x1,y1)
Q(x2,y2)
x
yv
A point can also be represented as a vector, defined by the point
and the origin (0,0).
PQv −=
=
−
=
1
1
1
1
0
0
y
x
y
xP
=
−
=
2
2
2
2
0
0
y
x
y
xQ
or vPQ +=
Note: point and vector are different; vectors do not have positions
Matrix
A matrix is an m×n array of numbers.
[ ]ij
mnmm
n
n
a
aaa
aaa
aaa
A =
=
L
MMM
L
L
21
22221
11211
Example:
−=
111070
9314
4532
A
Matrix Arithmetic
Matrix addition
[ ]nmijijnmnm baBA
××× +=+
Matrix multiplication
pmpnnmCBA ××× = ∑
=
=n
k
kjikijbac
1
Matrix transpose
[ ]ji
TaA =
( ) TTTBABA +=+ ( ) TTT
ABAB =
BAAB ≠
Matrix multiplication is not commutative
Example:
=
1119
2917
51
26
13
52
=
1017
3218
13
52
51
26
Symmetric Matrix
We say matrix A is symmetric if
AAT =
Example:
=
54
42A
A symmetric matrix has to be a square matrix
Inverse of matrix
If A is a square matrix, the inverse of A, written A-1 satisfies:
IAA =−1 IAA =−1
Where I, the identity matrix, is a diagonal matrix with all 1’s
on the diagonal.
=
10
012I
=
100
010
001
3I
Trace of Matrix
The trace of a matrix:
∑=
=n
i
iiaATr
1
)(
Orthogonal Matrix
A matrix A is orthogonal if
IAAT = 1−= AA
Tor
Example:
−=
θθ
θθ
cossin
sincosA
Matrix Transformation
A matrix-vector multiplication transforms one vector to another
11 ××× =mnnm
bxA
=
26
16
41
7
3
24
13
52
Example:
Coordinate Rotation
−=
y
x
y
x
r
r
r
r
φφ
φφ
cossin
sincos
'
'
Eigenvalue and Eigenvector
We say that x is an eigenvector of a square matrix A if
xAx λ=
λ is called eigenvalue and is called eigenvector.
The transformation defined by A changes only the
magnitude of the vector x
Example:
x
=
=
1
15
5
5
1
1
41
23
−=
−=
−
1
22
2
4
1
2
41
23and
5 and 2 are eigenvalues, and
1
1
−1
2and are eigenvectors.
Properties of Eigen Vectors
• If λ1, λ2,…, λq are distinct eigenvalues of a
matrix, then the corresponding eigenvectors
e1,e2,…,eq are linearly independent.
• A real, symmetric matrix has real eigenvalues
with eigenvectors that can be chosen to be
orthonormal.
Least Square
When m>n for an m-by-n matrix A, bAx = has no solution.
In this case, we look for an approximate solution.
We look for vector such that x
2bAx −
is as small as possible.
This is the least square solution.
Least Square
Least square solution of linear system of equations
bAx =
bAAxATT =
AAT is square and symmetric
bAAAxTT 1)( −=
Normal equation:
2bxA −
The Least Square solution
makes minimal.
Least Square Fitting of a Line
x
y
mmydxc
ydxc
ydxc
=+
=+
=+
M
22
11
=
mm y
y
y
d
c
x
x
x
MMM
2
1
2
1
1
1
1
The best solution c, d is the one that minimizes:
.)()( 22
11
22
mm dxcydxcyAxyE −−++−−=−= L
Line equations:
yAx =
Least Square Fitting - Example
x
y
P1=(-1,1), P2=(1,1), P3=(2,3)
Problem: find the line that best fit these three points:
Solution:
32
1
1
=+
=+
=−
dc
dc
dc
=
−
3
1
1
21
11
11
d
c
bAAxATT =
=
6
5
62
23
d
c
or
is
The solution is and best line is74
79 , == dc yx =+ 7
479
SVD: Singular Value Decomposition
TUDVA =
An m×n matrix A can be decomposed into:
U is m×m, V is n×n, both of them have orthogonal columns:
IUU T =
D is an m×n diagonal matrix.
IVV T =
Example:
−=
−10
01
00
30
02
100
010
001
00
30
02
top related