session 9 linear algebra and rec
TRANSCRIPT
-
7/27/2019 Session 9 linear algebra and rec
1/18
Linear algebra and regression
-
7/27/2019 Session 9 linear algebra and rec
2/18
Solving linear equations
The simplest system of linear equations hastwo equations and two variables, for example:
This system can be represented using matrices
and vectors in the form Ax = b
-
7/27/2019 Session 9 linear algebra and rec
3/18
Solving linear equations
Solving this system in Matlab is straightforward:
octave:9> A = [1 -1 ; 3 1]
A =
1 -1
3 1octave:10> b = [-1 ; 9]
b =
-1
9
octave:11> x = A\bx =
2
3
-
7/27/2019 Session 9 linear algebra and rec
4/18
Solving linear equations
We can now verify that x1 = 2, x2 = 3 is a solutionby calculating Ax
octave:12> A*x
ans =-1
9
octave:13> A*x-b
ans =0
0
-
7/27/2019 Session 9 linear algebra and rec
5/18
Exercise
Solve the following linear system:
2x 3y = 3
4x 5y + z = 72x -y -3z = 5
-
7/27/2019 Session 9 linear algebra and rec
6/18
Determinants
-
7/27/2019 Session 9 linear algebra and rec
7/18
Determinants
octave:15> det(A)
ans = 4
octave:16> A(1,1)*A(2,2)-A(1,2)*A(2,1)ans = 4
-
7/27/2019 Session 9 linear algebra and rec
8/18
Linear independence
Consider the following system:
The determinant of this matrix is zero.
(check this in octave)
-
7/27/2019 Session 9 linear algebra and rec
9/18
Linear independence and rank
The rank of a matrix is simply the number of rows
which are not linearly dependent, or linearly
independent rows.
It can be shown that the rank with respect to
rows is equal to the rank with respect to columns,
i.e. the rank of a matrix is also equal to the
number of linearly independent columns. In Matlab we can use the rank() function to
compute the rank.
-
7/27/2019 Session 9 linear algebra and rec
10/18
Underdetermined vs overdetermined
systems
The matrix A can have dimension mn with mn.
If m < n then there are more variables than
equations. Here it will usually be impossible to
find an unique exact solution. This is an
underdeterminedsystem.
If m > n then there are more equations than
variables. It may be impossible to satisfy allequations simultaneously. This is an
overdeterminedsystem.
-
7/27/2019 Session 9 linear algebra and rec
11/18
Least squares and Linear regression
When you use the Matlab backslash operator in thecontext of an overdetermined system it automaticallyreturns a least squares solution.
This feature can be used for linear regression. Forexample, if we want to find the best fit line through aset of points (x1, y1), (x2, y2), ..., (xm, ym) then one of thesimplest forms of regression we can come up withinvolves a single estimate () for the slope of a line.
We redefine the problem as finding the beta valuesthat minimize the sum of square differences:
-
7/27/2019 Session 9 linear algebra and rec
12/18
Exercise
Download the incomplete script
simpleReg.m and finish it by implementing
the function S(x,y,beta). The output should be:
-
7/27/2019 Session 9 linear algebra and rec
13/18
Linear regression
-
7/27/2019 Session 9 linear algebra and rec
14/18
Linear regression
We can define this in matrix form too. Here,
whereXis a matrix withXi1 = 1 andXi2 =xi , i.e.
a column of ones followed by the columnx.
-
7/27/2019 Session 9 linear algebra and rec
15/18
Linear regression
Within Matlab, we simply write this as anoverdetermined system
X = y
to obtain our parameter estimates by leastsquares.
Matlab performs the minimization itself. This
is the simple linear regression.
-
7/27/2019 Session 9 linear algebra and rec
16/18
Exercise
Start by loading the file agevbp.txt from the
course site. These are data comparing age (1st
column) vs systolic blood pressure.
1. Plot age (x-axis) vs blood pressure (y-axis)
2. Build the two-column matrix X as defined
above.
3. SolveX = y
4. Add the regression line to the plot
-
7/27/2019 Session 9 linear algebra and rec
17/18
Quadratic regression
We then solveX = y as before
-
7/27/2019 Session 9 linear algebra and rec
18/18
Exercises
1. Download the file qdata.txt of x,y pairs and
perform a quadratic regression. Plot the data
and the regression curve.
2. For the following data:
Perform an exponential regression using the
model:
Hours 1 2 3 4 5 6 7
Bacteria 25 38 58 89 135 206 315