1

Outline

• Linear Regression• The Method of Least Squares

LESSON 21: REGRESSION ANALYSIS

2

• This lesson addresses the problem of finding a relationship between two population. The goal is to predict values for one population on the basis of observations taken from the other.

• Engineers often encounter such problems whenever they need to fit a curve to experimental data. Recall that a scatter diagram is a graph that plots points representing relationship between two variables, an independent variable and a dependent variable . A curve can be fitted to the scatter plot using a regression analysis.

• A linear regression assumes a linear relationship and fits a straight line. The method of least square is a method that finds the particular line.

XY

Linear Regression

3

Linear Regression

Dep

end

ent

vari

able

Independent variableX

Y

The scatter diagram on this slide shows a linear relationship between two variables.

4

Dep

end

ent

vari

able

Independent variableX

Y Regressionequation:Y = a + bX

Linear Regression

A straight line of the form Y = a+bX nicely fits the points. Linear regression provides values of parameters a and b

5

Dep

end

ent

vari

able

Independent variableX

Y

Actualvalueof Y

Estimate ofY from regressionequation

Value of X usedto estimate Y

Deviation,or error

{

Regressionequation:Y = a + bX

The least square method finds a and b such that the sum of the squares of errors is minimum

The Method of Least Squares

6

The Method of Least Squares

• Consider n observations

• For the ith observation Xi, the predicted value

• The least square method finds a and b to minimize the sum of the squares of deviations of the predicted values from the actual values

ii bXaXY ˆ

2

1

2

1

ˆ

n

iii

n

iii bXaYXYY

niYX ii ,,2,1, for

7

The Method of Least Squares

• The following a and b minimize the sum of the squares of deviations

XbYa

XnX

YXnXYb

XXn

YXXYnb

2222 or,

8

• The standard deviation of the individual Y observations:

• The standard error of the Y estimate

• Note: The divisor is n minus the number of regression coefficients.

2

ˆ1

2

.

n

XYYs

n

iii

XY

1

1

2

n

YYs

n

ii

Y

Linear Regression

9

Sales AdvertisingMonth (000 units) (000 $)

1 264 2.52 116 1.33 165 1.44 101 1.05 209 2.0

Linear Regression Using Least Squares

Future sales are unknown, but future advertising expenses are given by marketing plan. A known value of advertising expense is used to forecast sales. For such a forecast, we need the relationship between advertising and sales.

Since sales depends on advertising, sales is the dependent variable and shown on the Y-axis. Advertising is the independent variable and shown on the X-axis.

10

a = Y - bX b = XY - nXY

X 2 - nX 2

Sales, Y Advertising, XMonth (000 units) (000 $) XY X 2

1 264 2.52 116 1.33 165 1.44 101 1.05 209 2.0

Total Y= X =

Linear Regression Using Least Squares

11

a = Y - bX b = XY - nXY

X 2 - nX 2

Sales, Y Advertising, XMonth (000 units) (000 $) XY X 2

1 264 2.5 660.0 6.252 116 1.3 150.8 1.693 165 1.4 231.0 1.964 101 1.0 101.0 1.005 209 2.0 418.0 4.00

Total 855 8.2 1560.8 14.90Y= 171 X = 1.64

Linear Regression Using Least Squares

12

a = - 8.136 b = 109.229

Sales, Y Advertising, XMonth (000 units) (000 $) XY X 2

1 264 2.5 660.0 6.252 116 1.3 150.8 1.693 165 1.4 231.0 1.964 101 1.0 101.0 1.005 209 2.0 418.0 4.00

Total 855 8.2 1560.8 14.90Y = 171 X = 1.64

Linear Regression Using Least Squares

13

300 —

250 —

200 —

150 —

100 —

50

Sal

es (

000s

)

| | | |1.0 1.5 2.0 2.5

Y = - 8.136 +109.229(X)

Interpretation: For each $1000 increase in advertising, sales increases by 109,229 units.

Linear Regression Using Least Squares

14

The regression equation can be used to forecast sales of Month 6 from a known value of advertising expenditure in Month 6.

Forecast for Month 6:

Let advertising expenditure = $1750

Y =

Linear Regression Using Least Squares

15

Forecast for Month 6:

Let advertising expenditure = $1750

Y =-8.136+109.229(1.75) = 183.015 thousand units

Linear Regression Using Least Squares

16

Example 1: The following sample observations have been obtained by a chemical engineer investigating the relationship between weight of final product Y (in pounds) and volume of raw materials X in gallons. Find a and b:

Example

X Y

14 68

23 105

9 40

17 79

10 51

17

X Y XY X2

14 68

23 105

9 40

17 79

10 51

Example

XbYa

XXn

YXXYnb 22

18

Example 2: Consider Example 1. Compute the sample standard deviation for final product weight.

Example

19

Example 3: Consider Example 1. Compute the standard error of estimate for final product weight.

Example

20

Lesson 21

Reading:

Section 4-1 pp. 90-102

Exercises:

4-1, 4-2

READING AND EXERCISES