Matrix Algebra Usefulin Statistics
Leif E. Peterson, Ph.D.
Dept. of MedicineBaylor College of Medicine
Scalars
• A scalar is a single data point
• A scalar is real-valued (negative, positive, zero)
• Continuous type of data point (integer with decimal component)
• Assumed infinitely precise
• Examples: 2.1384, -14.801,1E-12
Vectors• In statistics, a vector is an n × 1 matrix (column
vector)• In statistics, a vector commonly represents n
measurements for a single variable• Vector elements are assumed infinitely precise and are
continuously scaled
1
21 2 or [ , ,..., ]n
n
xx
x x x
x
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ ′= =⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
x x
Columnvector
Rowvector
1 2( , )P x x
O(0,0)
Length of a vector x between origin (0,0) and point P
2 2 21 2
2 2 21 2
2 2 21 2
2
1
1 1 2 2
( ,0) ( 0) ( 0) ( 0)
( 0) ( 0) ( 0)
+ + =
n
n
n
n
ii
n n
L d P x x x
x x x
x x x
x
x x x x x x=
= = = − + − + + −
= − + − + + −
= + + +
=
= +′
∑
x x
x x
x
y
(age)
(weight)
2 2( , )Q x y
1 1( , )P x y
O
2 21 2 1 2( , ) ( ) ( )d P Q x x y y= − + −
( , )d P Q
Geometric interpretation of two-dimensional vectors (one for each patient)
x
y
z
(age)
(weight)
(BP)
2 2 2( , , )P x y z1 1 1( , , )Q x y z
2 2 21 2 1 2 1 2( , ) ( ) ( ) ( )d P Q x x y y z z= − + − + −
O
Geometric interpretation of three-dimensional vectors (one for each patient)
( , )d P Q
(1,0)
Unit circle
(30 )6π
(45 )4π
(60 )3π(90 )
2π
2 (120 )3π
3 (135 )4π
5 (150 )6π
(180 )π
3 (270 )2π
2 (360 )π
2 2 1x y+ =
θ(radians) θ(degrees) cos(θ) 0 0 1
6π
30 0.866
4π
45 0.707
3π
60 0.5
2π
90 0
34π
135 -0.707
π 180 -1
Trigonometric functions of angles
cos( )x L θ= a
sin( )y L θ= aa
x
y
( , )P x y
θ
cos( )xL
θ =a
sin( )yL
θ =a
1
21 2(1 ) ( 1)
1 1
1 1 2 2
[ , ,..., ]
= + +
nn n
n
n n
yy
x x x
yx y x y x y
× ×
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥′ = ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦+
x y
Inner product between two vectors x and y
Inner product between two vectors x and y
1
21 2 1 1[ , ,..., ] gives n
n
y
yx x x x y
y
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ →⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
1
21 2 2 2[ , ,..., ] n
n
y
yx x x x y
y
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ →⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
1
2
1 2[ , ,..., ] n n n
n
yy
x x x x y
y
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ →⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
Step 1:
Step 2: Step n:
Outer product between two vectors x and y
1
21 2( 1) (1 )
1 1 1 2 1
2 1 2 2 2
1 2
[ , ,..., ]
=
nn n
n nn
n
n
n n n n
x
x y y y
x
x y x y x y
x y x y x y
x y x y x y
× ×
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥′ = ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
x y
θ
Trigonometric functions of angles
1θ 2θ
1x
2x2y
1y
x y
11cos( )
xL
θ =x
= 12cos( )
yL
θy
= 21sin( )
xL
θx
22sin( )
yL
θ =y
x-component
y-component
For x:
For y:x-component
y-component
Geometric Interpretation of inner product= − = +
⎛ ⎞ ⎛ ⎞⎛ ⎞ ⎛ ⎞⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜= +⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜⎟ ⎟⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜⎝ ⎠ ⎝ ⎠⎝ ⎠ ⎝ ⎠+
=
1 2 2 1 2 1
1 1 2 2
1 1 2 2
cos( ) cos( ) cos( )cos( ) sin( )sin( )
y x y xL L L Lx y x yL L
θ θ θ θ θ θ θ
y x y x
x y
1 1 2 2x y x y′ = +x y
1 1 2 2L x x x x′= = +x x x
cos( )L L
θ′ ′
= =′ ′x y
x y x yx x y y
′ = cos( )L L θx yx y
Correlation as angle between deviation vectors
1
2
j j
jj
n j
x xx x
x x
⎡ ⎤−⎢ ⎥⎢ ⎥−⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥−⎢ ⎥⎣ ⎦
d
=
= =
− −′ ′= = = =
′ ′ − −
∑
∑ ∑1
2 2
1 1
( )( )cos( )
( ) ( )j k
n
ij j ik kj k j k jki
n njj kk
j j k k ij j ik ki i
x x x xs
L L s sx x x x
θd d
d d d d
d d d d
1
1 =1,2,...,n
j iji
x x j pn =
= ∑
== − −∑
1
1( )( )
n
jk ij j ik ki
s x x x xn
== = −∑2 2
1
1( )
n
j jj ij ji
s s x xn
== −∑ 2
1
1( )
n
j ij ji
s x xn
Correlation as angle between deviation vectors
41
3
⎡ ⎤⎢ ⎥⎢ ⎥= −⎢ ⎥⎢ ⎥⎣ ⎦
x13 5
⎡ ⎤⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
y
4 2 21 2 3
3 2 1
⎡ ⎤ ⎡ ⎤−⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= − − = −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦
xd1 3 23 3 05 3 2
⎡ ⎤ ⎡ ⎤− −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= − =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦
yd
[ ]2
2 3 1 3 141
⎡ ⎤⎢ ⎥
′ ⎢ ⎥= − − =⎢ ⎥⎢ ⎥⎣ ⎦
x xd d [ ]2
2 0 2 0 82
⎡ ⎤−⎢ ⎥
′ ⎢ ⎥= − =⎢ ⎥⎢ ⎥⎣ ⎦
y yd d
Correlation as angle between deviation vectors
[ ]2
2 3 1 0 22
⎡ ⎤−⎢ ⎥
′ ⎢ ⎥= − =−⎢ ⎥⎢ ⎥⎣ ⎦
x yd d
′ −= = = = −
′ ′
2cos( ) 0.189
14 8xyr θ x y
x x y y
d d
d d d d
−
= = = −2 2
23 0.189
14 83 3
xyxy
x y
sr
s s
Matrix definition
11 12 13 1 1
21 22 23 2 2
( ) 1 2 3
1 2 3
j p
j p
n p i i i ij ip
n n n nj np
x x x x x
x x x x x
x x x x x
x x x x x
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X
Matrix Operations: Transpose11 12 13 1 1
21 22 23 2 2
( ) 1 2 3
1 2 3
j p
j p
n p i i i ij ip
n n n nj np
x x x x x
x x x x x
x x x x x
x x x x x
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X
11 12 13 1 1
21 22 23 2 2
( ) ( ) 1 2 3
1 2 3
i n
i n
T
p n p n j j j ji jn
p p p pj pn
x x x x x
x x x x x
x x x x x
x x x x x
× ×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥′= = ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X X
Transpose of a Matrix
(5 4)
1 123 33 3.5
1 99 30 2.7
1 102 45 1.4
0 212 39 4.6
0 187 57 2.8
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥
= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X
(4 5)
1 1 1 0 0
123 99 102 212 187
33 30 45 39 57
3.5 2.7 1.4 4.6 2.8
T
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X
Matrix Addition
(2 3)
4 1 7
3 2 6×
⎡ ⎤⎢ ⎥=⎢ ⎥⎣ ⎦
X
(2 3)
2 3 6
5 4 1×
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
Y
× ×
+ + +⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥+ = =⎢ ⎥ ⎢ ⎥+ + +⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦(2 3) (2 3)
4 2 1 3 7 6 6 4 13
3 5 2 4 6 1 8 6 7X Y
Matrix-vector Multiplication
1
211 12 1
321 22 2
( ) ( 1)
1 2( )
( 1)
( 1)
n
nT
p n ni
p p pnp n
nn
p
y
yx x x
yx x x
yx x x
y
× ×
×
×
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥
⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
X y
Matrix-vector Multiplication
(2 3) (3 1)
32 1 4
25 3 7
5
T
× ×
⎡ ⎤⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎣ ⎦
X y
2 3 3 1
32 1 4
2 2(3) 1(2) 4(5) 6 2 20 285 3 7
5
28
T
× ×
⎡ ⎤⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + = + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎣ ⎦
X y
Step 1:
Matrix-vector Multiplication
Step 2:
(2 3) (3 1)
32 1 42 5(3) 3(2) 7(5) 15 6 35 56
5 3 75
28
56
T
× ×
⎡ ⎤⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + = + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
X y
Matrix Multiplication
11 12 1 11 12 1
21 22 2 21 22 2
( ) ( )
1 2 1 2( ) ( )
( )
n p
n pT
p n n p
p p pn n n npp n n p
p p
x x x x x x
x x x x x x
x x x x x x
× ×
× ×
×
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
X X
Matrix Multiplication
Step 1:
(2 4) (4 2)
2 6
1 42 1 3 42(2) 1(1) 3(3) 4(4) 4 1 9 16 30
3 26 4 2 7
4 7
30
T
× ×
⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎣ ⎦
X X
Matrix MultiplicationStep 2:
Step 3:
(2 4) (4 2)
2 6
2 1 3 4 1 46(2) 4(1) 2(3) 7(4) 12 4 6 28 50
6 4 2 7 3 2
4 7
30
50
T
× ×
⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
X X
(2 4) (4 2)
2 6
1 42 1 3 42(6) 1(4) 3(2) 4(7) 12 4 6 28 50
3 26 4 2 7
4 7
30 50
50
T
× ×
⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
X X
Matrix Multiplication
Step 4:
(2 4) (4 2)
2 6
2 1 3 4 1 46(6) 4(4) 2(2) 7(7) 36 16 4 49 105
6 4 2 7 3 2
4 7
30 50
50 105
T
× ×
⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦
X X
Matrix Multiplication
Step 1:
(2 4) (4 2)
2 6
1 42 1 3 42(2) 1(1) 3(3) 4(4) 4 1 9 16 30
3 26 4 2 7
4 7
30
T
× ×
⎡ ⎤⎢ ⎥⎢ ⎥⎡ ⎤ ⎢ ⎥⎢ ⎥= → + + + = + + + =⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦ ⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⎢ ⎥⎣ ⎦
X X
Mean vector
1
11 12 11
2 21 22 22
1 2
1
11 11
1
n
nT
p p pnpp
n x x xxx x xx
nn nx x xx
n
′⎡ ⎤⎢ ⎥⎢ ⎥ ⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥′⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= = = =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦ ⎢ ⎥ ⎣ ⎦′⎢ ⎥⎢ ⎥⎣ ⎦
x 1
x 1x X 1
x 1
Mean matrix
1 2
1 2
( 1) (1 )
1 2( )
1
p
p
n p
pn p
x x x
x x x
x x x
× ×
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥′ = ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
x
Deviation matrix
11 1 12 2 1
21 1 22 2 2
( ) ( 1) (1 )
1 1 2 2( )
1
p p
p p
n p n p
n n np pn p
x x x x x x
x x x x x x
x x x x x x
× × ×
×
− − −⎡ ⎤⎢ ⎥
− − −⎢ ⎥⎢ ⎥′− = ⎢ ⎥⎢ ⎥⎢ ⎥− − −⎢ ⎥⎣ ⎦
X x
Variance-Covariance Matrix
( ) ( )11 1 21 1 1 1 11 1 12 2 1
12 2 22 2 2 2 21 1 22 2 2
1 2 1 1 2 2( ) ( )
( )
11 1
n p p
n p pT
p p p p np p n n np pp n n p
p p
x x x x x x x x x x x x
x x x x x x x x x x x x
n
x x x x x x x x x x x x× ×
×
− − − − − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥
− − − − − −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥′ ′= − − = ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− − − − − −⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
S X x X x
11 12 1
21 22 2
1 2
p
p
p p
p p pp
s s s
s s s
s s s
×
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
S
Diagonal matrices: sample standard deviation matrix (inverse)
11 1
22 21/2
0 0 0 0
0 0 0 0
0 0 0 0
p p
pp p
s s
s s
s s
×
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
D
11 1
1/2 22 2
1 10 0 0 0
1 10 0 0 0
1 10 0 0 0
p p
pp p
s s
s s
s s
−×
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
D
Obtaining correlation matrix by sandwiching covariance matrix between inverted standard deviation matrices
11 1111 12 1
21 22 21/2 1/2 22 22
1 2
111 12
11 11 11 22 11
1 10 0 0 0
1 10 0 0 0
1 10 0 0 0
p
p
p p pp
pp pp
p
p
s ss s s
s s ss s
s s s
s sss s
s s s s s s
− −
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥= = ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎣ ⎦⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
=
R D SD
12 1221 22
21 211 22 22 22 22
1 21 2
11 22
1
1
1
p pp
ppp
p pp p pp
pp pp pp pp
r rss s
r rs s s s s s
r rs s ss s s s s s
⎡ ⎤⎢ ⎥⎢ ⎥ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ = ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎣ ⎦⎢ ⎥⎢ ⎥⎣ ⎦
Identity matrices1 0 0
0 1 0
0 0 1
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥= ⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
I
× × × × ×= =
( ) ( ) ( ) ( ) ( )n p n p n p n p n pI X X I X
1 0 3 1 2 3 1 2
0 1 6 7 4 6 7 4
⎡ ⎤ − −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦⎢ ⎥⎣ ⎦
Trace of a matrix
11 221
tr( )n
nn iii
x x x x=
= + + + = ∑X
2 9 7
tr 5 -1 3 2 1 7 8
3 4 7
⎡ ⎤⎢ ⎥⎢ ⎥ = − + =⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
tr( ) tr( )T =X X
1 5 6 1 4 -2
tr 4 6 7 tr 5 6 4 1 6 9 16
-2 4 9 6 7 9
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= = + + =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
Multiple Linear Regression
+y = Xb e
01 11 12 1 1
2 21 22 2 21
1 2( 1) ( ) ( 1)( 1)
( 1)
p
p
n n n np npn n p np
n
y x x x
y x x x
y x x x
β εεβ
εβ× × ××
×
⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥= +⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦⎢ ⎥⎣ ⎦
Multiple Linear Regression
1( )T T−=b X X X y
( )−
× × × × ×× ×
× ×
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢= = ⎢ ⎥ ⎢⎢ ⎥ ⎢⎢ ⎥ ⎢⎢ ⎥ ⎢⎣ ⎦ ⎣ ⎦
11 12 1 11 12 1
1 21 22 2 21 22 2
( 1) ( )( ) ( )( 1)( ) ( 1) 1 2 1 2
( 1) ( )
n p
n pT T
p p n n p p n np p p p p pn n n np
p p n
x x x x x x
x x x x x x
x x x x x x
b X X X y
× × ×
× ×
×
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
11 12 1 1
21 22 2 2
1 2( ) ( ) ( 1)
( ) ( 1)
( 1)
n
n
p p pn nn p p n n
p p p
p
x x x y
x x x y
x x x y
References
• Gentle, J.E. Numerical Linear Algebra for Applications in Statistics. New York(NY), Springer-Verlag, 1998.
• Morrison, D.F. Multivariate Statistical Methods. New York(NY), McGraw-Hill, 1990.