Welcome

What does the term Regression literally mean?

The term regressions literally mean stepping back towards the average.

What is Regression?

Regression is a statistical tool to estimate (or predict) the unknown values of one variable from known values of another variable.

Example: If we know that advertising and sales are correlated, we may find out the expected amount of sales for a given advertising expenditure or the amount of expenditure for achieving a fixed sales target.

What is the utility of studying?

The regression analysis is a branch of statistical theory that is widely used in almost all the scientific disciplines. In economics it is the basic technique for measuring or estimating the relationship among economic variables that constitute the essence of economic theory and economic life.

Example: If two variables price (X) and demand (Y) are closely related one can find out the most probable value of X for a given value of Y or the most probable value of Y for a given value of X.

Similarly, if the amount of tax and the rise in the price of a commodity are closely related, it is possible find out the expected price for a certain amount of tax levy.

Distinguish between correlation and regression analysis?

The points of difference between correlation and regression analysis are:

While correlation co-efficient is a measure of degree of relationship between X and Y, the regression analysis helps study the nature of relationship between the variables. The cause and effect relation is more clearly indicated through regression analysis than by correlation. Correlation is merely a tool of ascertaining the degree of relationship between two variables and, therefore, one can not say that one variable is the cause and the other the effect, while regression shows the extent of dependence of one variable on another.

What are Regression Lines?

If we take the case of two variables X and Y, we shall have two regression lines as regression line of X on Y and the regression line of Y on X.

The regression line of Y on X gives most probable values of Y for given values of X and the regression line of X on Y gives most probable values of X for given values of Y. Thus we have two regression lines.

Under what conditions can there be one regression line?

When there is either perfect positive or perfect negative correlation between the two variables, the two correlation lines will coincide i.e. we will have one line.

The further the two regression lines are from each other, the lesser is the degree of correlation and the nearer the two regression lines to each other, the higher is the degree of correlation. If variables are independent, r is zero and the lines of regression are at right angles i.e. parallel to X axis and Y axis.

What are regression equations?

Regression Equations are algebraic expressions of regression lines. Since there are two regression lines there are two regression equationsthe regression equation of X on Y is used to describe the variations in the values of X for given change in Y and the regression equation of Y on X is used to describe the variation in the values of Y for given changes in X.

Why is the line of best fit?

The line of regression is the line which gives the best estimate to the value of one variable for any specific value of other variable. Thus the line of regression is the line of best fit and is obtained by the principles of least squares.

What is the general form of the regression equation of Y on X?

The general form of the linear regression equation of Y on X is expressed as follows:

Ye = a+bX, where

Ye = dependent variable to be estimated

X = independent variable.

In this equation a and b are two unknown constants (fixed numerical values) which determine the position of the line completely. The constants are called the parameters of the line. If the value of either one or both of them are changed, another line is determined.

What is the general form of the regression equation of Y on X?

The parameter a determines the level of the fitted line (i.e. the distance of the line directly, above or below the origin). The parameter b determines the slope of the line i.e. the change in Y for unit change in X.

To determine the values of a and b, the following two normal equations are to be solved simultaneously.

Y=Na+bX, XY=aX +bX2.

What is the general form of the regression equation of X on Y?

The general form of the regression equation of X on Y is expressed as follows:

X = a+bY

To determine the values of a and b, the following two normal equations are to be solved simultaneously.

X = Na+bY, XY = aY+bY2.

Continued

What is the general form of the regression equation of X on Y?

These equations are usually called the normal equations. In the equations X, Y, XY, X2, indicate totals which are computed from the observed pairs of values of the two variables X and Y to which the least squares estimating line is to be fitted and N is the total number of observed pairs of values.

The geometrical presentation of the linear equation , Y = a+bX is shown in the diagram below:

Y=a+bX

b

1 UNIT IN X

Y

X

o

It is clear from this diagram, the height of the line tells the average value of Y at a fixed value of X. When X=0, the average value of Y is equal to a . The value of a is called the Y- intercept since it is the point at which the straight line crosses the Y- axis. The slope of the line is measured by b, which gives the average amount of change of Y per unit change in the value of X. The sign of b also indicates the type of relationship between Y and X.

How are the values of a and b obtained to determine a regression completely?

The values of a and b are obtained by the method of least squares which states that the line should be drawn through the plotted points in such a manner that the sum of the squares of the vertical deviations of the actual Y values from the estimated Y values is the least, or in other words, in order to obtain a line which fits the points best, (YYe)2 should be minimum. Such a line is known as line of best fit.

How can the normal equations be arrived at?

Continued

Continued

Let S = (Y Ye)2 = (Y a bX)2

( Ye = a+bX)

Differentiating partially with respect to a and b,

Example:

The following data give the hardness (X) and tensile strength (Y) of 7 samples of metal in certain units. Find the linear regression equation of Y on X.

Continued

Solution:

Regression equation of Y on X is given by

Y = a+bX

The normal equations are:

Y = Na + bX.. (1)

XY = aX+ bX2 (2)

X:146152158164170176182Y:65787789828586

Calculation of regression equations

Continued..

XYX2Y2XY14675213165625109501527823104608411856158772496459291216616489268967921145961708228900672413940176853097672251496018286331247396156521148(=X)572(=Y)189280(=X2)46904(=Y2)94120(=XY)

Continued..

Substituting the values in equations (1) and (2), we get

572 =7a +1148 b . (5)

94120 = 1148 a +189280 b (4)

Multiplying the equation (3) by 164, we get

93808 =1148 a +188272 b ..(5)

Subtracting this equation from (4), we get

b = 0.31

Putting this value of b in equation (3), we have

Here, N = 7

The linear regression equation of Y on X is Y = 30.87 + 0.31X

Continued..

Calculate the regression equations of X on Y and Y on X from the following data

Solution:

X:12345Y:25387XYX2Y2XY121422542510339994816643257254935X=15Y=25X2 = 55Y2 =151XY = 88

Regression equation of X on Y is given by X= a +bY.

The normal equations are X = Na+bY

and XY=aY+bY2

Substituting the values, we get

15 =5a+25b .. (1)

88= 25a+151b (2)

Solving (1) and (2), we get =0.5 and b =0.5

Continued..

Hence, the regression equation of X on Y is given by

X = 0.5+ 0.5Y

The Regression equation of Y on X is : Y = a+bX

The normal equations are

Y= Na+ bX

XY = aX + bX2

Substituting the values, we get

25 = 5a +15b (iii)

88 = 15 a + 55b (iv)

Solving (iii) and (iv), we get

a = 1.10 and b = 1.3

Hence, the regression equation of Y on X is given by

Y =1.10 +1.30X

What will be the forms of regression equation of X on Y and regression equation of Y on X on the basis of deviations taken from arithmetic Means of X and Y?

If we take the deviations of X and Y series from their respective means, the regression equation of Y on X will take the form

The value byx can be easily obtained as follows:

The two normal equations in terms of x and y will then become

Since x =y =0 (deviations being taken from means)

Equation (1) reduces to

Na = 0 a = 0

Equation (2) reduces to

After obtaining the value of byx , the regression equation can easily be written in terms of X and Y by substituting for y, and for x, .

Similarly, the regression equations X= a+bY is reduced to = and the value of bxy can be similarly obtained as

Example:

The following data give the hardness (X) and tensile strength (Y) 7 samples of metal in certain units. Find the linear regression equation of Y on X.

Continued..

X:146152158164170176182Y:75787789828586

Continued..

Hardness (X)Strength(Y)(x)(y)x2y2xy114675- 18- 6.714232445.08120.86215278- 12-3.712214413.8044.57315877- 6- 4.71423622.2228.2941648907.2858053.08051708260.2853360.081.71617685123.285814410.8039.43718286184.285832418.3777.14N=7X=1148Y=572x=0y=0x2= 1008y2=163.43xy =312

What are regression coefficients?

The quantity b in the regression equations (Y= a+bX and X = a+bY) is called the regression coefficient or slope co-efficient. Since there are two regression equations, therefore, there are two regression co-efficients regression coefficient of X on Y and regression coefficient of Y on X.

What is regression coefficient of X on Y?

The regression coefficient of X on Y is represented by the symbol bxy or b1 . It measures the amount of change in X corresponding to a unit change in Y. The regression coefficient of X on Y is given by

When deviations are taken from the means of X and Y, the regression coefficient is obtained by

Continued..

What is regression coefficient of X on Y?

When deviations are taken from assumed means, the value of bxy is obtained as follows:

What are the assumptions of constructing a regression modal?

The value of the dependent variable, Y, is dependent in some degree upon the value of the independent variable, X. The dependent variable is assumed to be a random variable, but the values of X are assumed to be fixed quantities that are selected and controlled by the experimenter. The requirement that the independent variables assumes fixed values, however, is not a critical one. Useful results can still be obtained by regression analysis in the case where both X and Y are random variables. The average relationship between X and Y can be adequately described by a linear equation Y= a+bX whose geometrical presentation is a straight line.

The height of the line tells the average value of Y at a fixed value of X. When X= 0, the average value of Y is equal to a. The value of a is called the Y intercept, since it is the point at which the straight line crosses the Y-axis. The slope of the line is measured by b, which gives the average amount of change of Y per unit change in the value of X. The sign of b also indicates the type of relationship between Y and X.Associated with each value of X there is a sub population of Y. The distribution of the sub population may be assumed to be normal or non specified in the sense that it is unknown. In any event, the distribution of each population Y is conditional to the value of X.

The mean of each sub- population Y is called the expected value of Y for a given X: .

Furthermore, under the assumption of a linear relationship between X and Y, all values of must fall on a straight line. This is

Which is the population regression equation for our bivariate linear model. In this equation a and b are called the population regression co efficient.

An individual value in each sub-population Y, may be expressed as:

It is assumed that the variances of all subpopulations, called variances of the regression, are identical.

Where e is the deviation of a particular value of Y from and is called the error term or the stochastic disturbance term. The errors are assumed to be independent random variables because Ys are random variables and independent. The expectations of these errors are zero; E(e) = 0. Moreover, if Ys are normal variables, the error can also be assumed to be normal.

Continued..

What is regression coefficient of Y on X ?

The regression Coefficient of Y on X is represented by byx or b2. It measures the amount of change in Y corresponding to a unit change in X. The value of byx is given

When deviations are taken from the means of X on Y,

When deviations are taken from assumed mean

What are the properties of regression coefficients?

The coefficient of correlation is the geometric mean of the two regression coefficients . Symbolically If one of the regression coefficients is greater than unity, the other must be less than unity, as the value of the coefficient correlation cannot exceed unity.

Example: if bxy =1.2 and byx =1.4,

Continued

What are the properties of regression coefficient?

Both the regression coefficients will have the same sign i.e. they will be either positive or negative.The coefficient of correlation will have the same sign as that of regression coefficient.

Example:

Continued

What are the properties of regression co-efficient?

The average value of the two regression coefficients would be greater than the value of coefficient of correlation. Symbolically,

Example:

r

Continued

Regression coefficients are independent of change of origin but not scale.

Prove that the coefficient of correlation is the geometric mean of the regression coefficient

Proof: Let bxy be co efficient of X on Y and byx be coefficient of Y on X.

The coefficient of correlation is the geometric mean of the two regression coefficients.

Example:

Prove that Regression coefficients are independent of change of origin but not scale.

Example:

The following figures relate to advertisement expenditure and corresponding sales

Estimate

i) The sales for advertisement expenditure of Tk. 80 lakhs and

ii) The advertisement expenditure for a sales target of Tk. 25 crores

Advertisement(in lakhs of Taka)60626570737571Sales ( in crores of Taka)10111315161914

Let the advertisement expenditure be denoted by X and sales by y calculation of regression equations

X= xx2Y= yy2xy6086410416326263611391865391311370241511273525162410757491952535713914000X=476x= 0x2=196Y=98y= 0y2= 476xy= 100

When advertisement expenditure, X= 80 lakhs,

Sales,Y= 0.5180 20.68

= 40.8 20.68

= 20.12

The likely sales for advertisement expenditure of Tk. 80 lakhs = Tk. 20.12 crores.

ii) Regression Equation of X on Y:

Putting the values in equation (2), we get

When sales target, Y= 25 croes,

Advertisement expenditure , X= 1.79 25+43

= 44.75+43

= 87.75

The likely advertisement expenditure for a sales target of Tk.25 crores = Tk. 87.75 Lakhs.

The following data relate to advertising expenditure (in lakhs of Taka) and their corresponding sales (in crores of Taka):

Example:

Estimate i) the sales corresponding to advertising expenditure of Tk 30 lakhs and

ii) the advertising expenditure for a sales target of Tka. 35 crores.

Advertising Expenditure1012152320Sales1417232521

.

Calculation of regression equations

X(x)x2Y(y)y2xy10- 63614- 636+3612- 41617- 39+1215- 1123+39- 323+74925+525+3520+41621+11+4X= 80x= 0x2=118Y=100y=0y2=80xy=84

From equation (1) , we have

When the advertisement expenditure is Tk. 30 lakhs,

Sales, Y= 0.712 X +8.608= 21.36+8.608=29.968

Thus the likely sales corresponding to advertisement expenditure of Tk.30 lakhs is Tk 29.968 crores.

Regression equation of X on Y is given by

when the sales target is Tk. 35 crores, the

advertising expenditure, X = - 5 +1.05 35

= - 5 +36.75 = 31.75

The advertising expenditure for a sales target of

Tk. 35 crores is Tk. 31.75 lakhs.

What is the standard error of estimate?

The measure of variation of the observations around the computed regression line is referred to as the standard error of estimate.

Just as the standard deviation is a measure of the scatter of observations in a frequency distribution around the mean of that distribution, the standard error of estimate is a measure of the scatter of the observed values of Y around the corresponding computed values of Y on the regression line. It is computed as a standard deviation, being also a square root of the mean of the squared deviation. But the deviations here are not deviations of the items from the arithmetic mean; they are rather the vertical distances of every dot from the line of average relationship.

What are the formulae for calculating the standard error of estimate?

where Syx= S.E. of estimate of regression equation of Y on X.

The standard error of estimate can be easily calculated with the help of the following formula.

Continued.

The standard error of estimate measures the accuracy of the estimated figures. The smaller the value of standard error of estimate, the closer will be dots to the regression line and the better the estimates based on the equation for this line. If standard error of estimate is zero, then there is no variation about the line and the correlation will be perfect. With the help of standard error of estimate, it is possible for us to ascertain how good and representative the regression line is as a description of the average relationship between two series.

Significance of standard error

What is coefficient of determination?

The ratio of the unexplained variation to the total variation represents the proportion of variation in Y that is not explained by regression on X. Subtraction of this proportion from 1.0 gives the proportion of variation in Y that is explained by regression on X. The statistic used to express this proportion is called the co-efficient of determination and is denoted by R2. It may be written as follows:

Continued.

The value of R2 is the proportion of the variation in the dependent variable Y explained by regression on the independent variable X.

(

)

X

X

-

}

a

yx

m

e

Y

X

E

Y

+

=

yx

or

X

Y

E

m

yx

X

Y

E

m

=

87

30

7

12

216

12

216

7

88

355

572

7

88

355

7

572

31

0

1148

7

572

=

=

=

-

=

+

=

+

=

a

a

a

a

a

squares

of

sum

Total

squares

of

sum

Error

R

Y

in

Total

X

on

regression

after

remaining

Y

in

Variation

R

-

=

-

=

1

1

2

2

variation

(

)

Y

Y

-

(

)

(

)

X

Y

c

i

X

Y

X

Y

X

Y

X

Y

Y

X

X

b

Y

Y

as

given

is

X

on

Y

of

regression

linear

the

x

xy

b

N

Y

Y

X

yx

yx

31

0

87

30

.

.

87

30

31

0

71

81

84

50

31

0

71

81

84

50

31

0

71

81

164

31

0

31

0

309

0

1008

312

71

81

7

572

164

7

1148

2

+

=

+

=

+

-

=

+

-

=

-

-

=

-

-

=

-

\

=

=

=

\

=

=

=

=

=

(

)

(

)

)

1

.......(

,

,

2

Y

Y

y

and

X

X

x

where

x

xy

b

yx

-

=

-

=

=

yx

xy

yx

xy

x

y

x

y

yx

xy

x

y

xy

y

x

xy

b

b

r

b

b

r

or

r

X

r

b

b

r

b

r

b

now

=

\

=

=

=

\

=

=

2

2

,

.

;

.

,

s

s

s

s

s

s

s

s

(

)

Y

Y

b

xy

-

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

+

=

+

=

+

=

+

=

=

-

-

=

-

-

=

-

-

-

=

-

-

-

=

2

2

,

2

,

1

,

1

2

..

..........

0

1

.....

..........

0

,

0

1

X

b

X

a

XY

bX

aX

YX

have

we

From

X

b

Na

bX

a

Y

have

we

From

X

bX

a

Y

bX

a

Y

Or

X

bX

a

Y

b

S

bX

a

Y

a

S

d

d

d

d

(

)

-

-

=

2

2

dy

y

d

N

dy

dx

dy

dx

N

b

xy

y

x

xy

r

b

s

s

.

=

=

2

y

xy

b

xy

(

)

-

-

=

2

2

x

x

y

x

y

x

yx

d

d

N

d

d

d

d

N

b

yx

xy

b

b

r

=

(

)

X

X

-

=

2

x

xy

b

yx

.

6

0

566

0

4

0

8

0

.

6

0

2

4

0

8

0

,

4

0

8

0

=

=

+

=

=

than

less

is

which

be

would

r

of

value

The

be

would

values

two

the

of

average

the

byx

and

b

If

xy

=

\

=

2

2

x

xy

b

or

b

x

b

xy

yx

4

0

8

0

2

0

,

8

0

2

0

-

=

-

=

-

=

-

=

r

b

and

b

If

yx

xy

(

)

(

)

(

)

(

)

68

20

51

0

14

68

34

51

0

68

34

51

0

14

68

51

0

14

,

,

51

0

196

100

,

)

........(

..........

:

14

7

98

68

7

476

;

7

,

2

-

+

-

-

=

-

-

=

-

=

=

=

-

=

-

=

=

=

=

=

=

=

X

X

X

Y

X

Y

have

we

i

from

Now

x

xy

b

Here

i

X

X

b

Y

Y

X

on

Y

of

equation

i

N

Y

Y

and

N

X

X

N

Here

yx

yx

Regression

2

yx

xy

b

b

+

.

29

1

4

1

2

1

possible

not

is

which

be

would

r

=

(

)

608

8

712

0

20

392

.

11

712

0

16

712

0

20

+

=

+

-

=

-

=

-

X

Y

X

Y

X

Y

Q

(

)

712

0

118

84

,

)

........(

:

)

1

(

2

=

=

=

-

=

-

X

XY

b

Now

i

X

X

b

Y

Y

X

on

Y

of

equation

yx

yx

Regression

(

)

43

79

1

68

9998

24

79

1

9998

24

79

1

68

14

79

1

68

+

=

+

-

=

-

=

-

-

=

-

Y

X

Y

X

X

Y

X

(

)

X

X

-

=

2

y

xy

b

xy

(

)

2

,

2

2

2

-

+

-

=

-

-

=

N

xy

b

X

a

X

S

Or

N

X

X

S

xy

e

xy

(

)

05

1

80

84

2

=

=

=

-

=

-

y

xy

b

Y

Y

b

X

X

xy

xy

20

5

100

16

5

80

,

=

=

=

=

=

=

N

Y

Y

N

X

X

Here

x

y

r

b

yx

s

s

.

=

(

)

X

X

b

Y

Y

yx

-

=

-

(

)

79

1

7857

1

56

100

,

2

=

=

=

-

=

-

y

xy

xy

xy

b

Here

Y

Y

b

X

X

(

)

Y

X

Y

X

Y

X

Y

X

05

1

5

05

1

21

16

21

05

1

16

20

05

1

16

+

-

=

+

-

=

-

=

-

-

=

-

\

(

)

2

,

2

2

2

-

+

-

=

-

-

=

N

YX

b

Y

a

Y

S

Or

N

Y

Y

S

yx

e

yx

(

)

(

)

+

=

+

=

2

....

1

.

..........

2

x

b

x

a

xy

x

b

Na

y

2

2

1

)

1

)

r

S

S

ii

r

S

S

i

x

yx

y

xy

-

=

-

=

bX

a

yx

X

Y

E

+

=

=

m

(

)

X

X

-

(

)

Y

Y

-

(

)

Y

Y

-

(

)

Y

Y

-

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

)

.

,

,

..........

2

2

2

2

2

2

uv

xy

vu

yx

yx

yx

b

k

h

b

that

shown

be

can

it

Similary

b

h

k

u

u

v

v

u

u

h

k

u

u

h

v

v

k

u

u

h

b

get

we

formula

above

the

in

values

these

ng

Substituti

v

v

k

Y

Y

and

u

u

h

X

X

get

we

g

Subtractin

v

k

b

Y

and

u

h

a

X

and

kv

b

Y

and

hu

a

X

k

b

Y

v

and

h

a

X

u

Let

i

X

X

Y

Y

X

X

b

or

X

X

N

Y

X

XY

N

b

=

=

-

-

-

=

-

-

-

=

-

=

-

-

=

-

+

=

+

=

+

=

+

=

\

-

=

-

=

-

-

-

=

-

-

=