professor dr. s. k. bhattacharjee department of statistics university of rajshahi, bangladesh...
TRANSCRIPT
PROFESSOR DR. S. K. BHATTACHARJEEDEPARTMENT OF STATISTICS
UNIVERSITY OF RAJSHAHI, BANGLADESH
G-inverse and Solution of System of Equations
and their Applications in Statistics
System of Linear Equations
Linear Combination of Vectors
One extremely helpful view is that each unknown is a weight for a column vector in a linear combination.
Vector Form
The vector equation is equivalent to a matrix equation of the form
where A is an m×n matrix, x is a column vector with n entries, and b is a column vector with m entries.
Solution of Linear System
A solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that each of the equations is satisfied. The set of all possible solutions is called the solution set.
A linear system may behave in any one of three possible ways:
The system has infinitely many solutions.The system has a single unique solution.The system has no solution.
Geometric InterpretationFor a system involving two variables (x and y),
each linear equation determines a line on the xy-plane. Because a solution to a linear system must satisfy all of the equations, the solution set is the intersection of these lines, and is hence either a line, a single point, or the empty set.
For three variables, each linear equation determines a plane in three-dimensional space, and the solution set is the intersection of these planes. Thus the solution set may be a plane, a line, a single point, or the empty set.
For n variables, each linear equations determines a hyperplane in n-dimensional space. The solution set is the intersection of these hyperplanes, which may be a flat of any dimension
Solution of Two Vectors
Solution of Three EquationsBelow is a picture of three planes that have no solution. There is no single point at which all three planes intersect, therefore this system has no solution.
Solution of Three Equations
Each plane intersects the other two planes. However, there is no single point at which all three planes meet. Therefore, the system of 3 variable equations below has no solution.
Solution of Three Equations One Solution of three variable systems
I f the three planes intersect as pictured below then the three variable system has 1 point in common, and a single solution represented by the black point below.
Solution of Three Equations I nfinite Solutions of three variable systems
I f the three planes intersect as pictured below then the three variable system has a line of intersection and therefore an infinite number of solutions.
Solution of Two Equations in Three Variables
The solution set for two equations in three variables is usually a line.
Solution of Linear System In general, the behavior of a linear system is
determined by the relationship between the number of equations and the number of unknowns:
Usually, a system with fewer equations than unknowns has infinitely many solutions. Such a system is also known as an underdetermined system.
Usually, a system with the same number of equations and unknowns has a single unique solution.
Usually, a system with more equations than unknowns has no solution. Such a system is also known as an overdetermined system.
Examples The system has exactly 1 solution.
Systems have 1 and only 1 solution when the two lines have different slope. Think about it, if the two lines have different slopes then eventually at some point they must meet. After all the lines are not parallel.
system has no solutions Systems have no solution when the lines are parallel (ie have the same
slope) and the lines have different y-intercepts. As an example look at the following two lines
Line 1: y = 5x +13 Line 2: y = 5x + 12
The system has infinite solutions Systems have infinite solutions when the lines are parallel and the lines
have the same y-intercept. If two lines have the same slope (ie are parallel) and the same y-intercept, they are actually the same exact line. In other words, systems have infinite solutions when the two lines are the same line! As an example consider the following two lines
Line 1: y = x +3 Line 2: 2y = 2x +6
These two lines are exactly the same line. If you multiply line 1 by two you get line 2.
Solution of Linear System
The following pictures illustrate this in the case of two variables:
One Equation Two Equations Three Equations
Two Variables Three Eq.
The equations x − 2y = −1, 3x + 5y = 8, and 4x + 3y = 7 are not linearly independent.
Example
The equations 3x + 2y = 6 and 3x + 2y = 12 are inconsistent.
Methods of SolutionThe Methods of finding the solution to
systems of linear equations: graph : by looking at where lines intersect (meet)
on a graph algebraic equation : by setting the equations of
the system equal to each other then solving this equation.
substitution : by solving for one of the variables and substituting its value in to the other equation.
Elimination : Elimination involves algebraic manipulations of two or more equations. The end goal is to eliminate a variable by creating opposite coefficients (The examples below should clarify this straightforward approach).
Graphical Method
The Graph Method On the left, the system of linear equations is the following two lines:
y=x+1 y=2x
What is the solution? answer: The point (1,2) is where the two lines intersect.
Algebraic Equation Method
The Algebraic Equation Method
Let's take another look at the system of equations from above:
y=2x+1 y=4x-1
By examining the graph we can see that the point of intersection, or the solution, is the point (1,3) where the lines intersected.
Steps for the algebraic method: make sure that each linear equation is
reduced to slope intercept form o (ie y=3x+2 is good but 2y=6x+4 is NOT)
set the two equations equal to each other
o 2x+1=4x-1 Solve for X
o 2x+1=4x-1 o 2=2x o x= 1
insert x value into either equation to determine y coordinate of solution
o 4(1)-1=3 The solution is the ordered pair you've
just calculated o (1,3)
Substitution MethodThe Substitution Method The substitution method involves algebraic
substitution of one equation into a variable of the other.
A quick refresher on algebraic substitution:
Refresher:Substitution Equation 1 : x = 5 Equation 2: y = x +2 How to Substitute
1) Use equation 1( x= 5) to substitute 5 for x in second equation y = (5) + 2
2) So love for Y y = 5+ 2 = 7
Example
Substitution Example Two Line 1 : y=2x+1 Line 2 : 2y=3x-2
Step 1: Substitute one equation into the other 2(2x+1)=3x-2
Step 2: Now that you have a single variable equation, solve for that variable's equation 4x+2 = 3x-2 x+2= -2 x= –4
Step 3 : Once you have solved for the one variable insert that variable back into either equation to obtain the value of y at the solution. Insert x= –4 to find y value y = 2(–4)+1= –7
This example's solution is ( –4, –7).
Elimination Method
Elimination method is an algebraic method for solving systems. To use elimination you perform an operation on 1 equation then add the two equations so that one of the variables cancels.
Example of Elimination Line 1: y = x + 1 Line 2: y = –x
Elimination Method
Matrix Methods of SolutionCramer's rule is an explicit formula for the solution of a system of linear equations, with each variable given by a quotient of two determinants. For example, the solution to the system
is given by
Inverse Method
The Matrix form of a linear system of equations is
Ax= b If the coefficient matrix is non-
singular then the solution is: x = A-1b
Elementary Row Transformation
Solving a system of linear equations by reducing the augmented matrix of the system to row canonical form
Example
Example. Solve the system
The augmented matrix is
Example
Homogeneous System
A system of linear equations is homogeneous if all of the constant terms are zero:
A homogeneous system is equivalent to a matrix equation of the form
where A is an m × n matrix, x is a column vector with n entries, and 0 is the zero vector with m entries.
Homogeneous System
Homogeneous Systems are always ConsistentEvery homogeneous system has at least one
solution, known as the zero solution (or trivial solution), which is obtained by assigning the value of zero to each of the variables. The solution set has the following additional properties:
If u and v are two vectors representing solutions to a homogeneous system, then the vector sum u + v is also a solution to the system.
If u is a vector representing a solution to a homogeneous system, and r is any scalar, then ru is also a solution to the system.
Homogeneous System
Suppose that a homogeneous system of linear equations has m equations and n variables with n>m. Then the system has infinitely many solutions.
Example
Example
Example
Generalized Inverse
Generalized Inverse
Generalized Inverse
Method of Obtaining g-Inverse
Example
Method of Obtaining g- Inverse
Example
Properties of g Inverse of X’X
The matrix X’X has an important role in statistics where it arises in least square equations X’Xb = X’y. When G is a generalized inverse of X’X:
1.G’ is also a generalized inverse of X’X2.XGX’X=X, i.e., GX’ is a generalized inverse of X3.XGX’ is invariant to G4.XGX’ is symmetric whether G is or not.Let X+ be the Moore – Penrose inverse of X. ThenXX+ = XGX’ but X+ may not be equal to GX’.
g-Inverse
g-Inverse
Properties
Every singular symmetric matrix (of order two or more) has both symmetric and non-symmetric generalized inverses.
Let A represent a matrix of full column rank and B a matrix of
full row rank. Then, (1) a matrix G is a generalized inverse of A if and only if G
is a left inverse of A. And, (2) a matrix G is a generalized inverse of B if and only
if G is a right inverse of B.
Properties
For any matrix A and any nonzero scalar k, (1/k)A− is a generalized inverse of kA.
For any matrix A,−A− is a generalized inverse of −A.
For any matrix A, (A−)’ is a generalized inverse of A’ .
For any symmetric matrix A, (A−)’ is a generalized inverse of A’ .
Properties
A linear system Ax = b is consistent if and only if
AA- b = bor, equivalently, if and only if(I − AA-)B = 0
The Moore-Penrose Inverse
Given any matrix A, there is a unique matrix M such that
(i) AMA=A (ii) MAM=M (iii) AM is symmetric (iv) MA is symmetricThen matrix M is called the Moore-Penrose
inverse of A.M = L’(K’AL’)-1 K’
The Moore-Penrose Inverse
Example
Example