chapter 12. simple linear regression and...

Chapter 12. Simple Linear Chapter 12. Simple Linear Chapter 12. Simple Linear Chapter 12. Simple Linear Regression and CorrelationRegression and CorrelationRegression and CorrelationRegression and Correlation

12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter ββββ1111

NIPRLNIPRLNIPRLNIPRL 1

12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter ββββ111112.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.10 Supplementary Problems12.10 Supplementary Problems12.10 Supplementary Problems12.10 Supplementary Problems

12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1.1 Model Definition and Assumptions(1/5)12.1.1 Model Definition and Assumptions(1/5)12.1.1 Model Definition and Assumptions(1/5)12.1.1 Model Definition and Assumptions(1/5)

• With the simple linear regression modelyi=β0+β1xi+εithe observed value of the dependent variable yi is composed of a linear function β0+β1xi of the explanatory variable xi, together with an error termerror termerror termerror term εi. The error terms ε1,…,εn are generally taken to be


error termerror termerror termerror term εi. The error terms ε1,…,εn are generally taken to be independent observations from a N(0,σ2) distribution, for some error error error error variancevariancevariancevariance σ2. This implies that the values y1,…,yn are observations from the independent random variablesYi ~ N (β0+β1xi, σ2)as illustrated in Figure 12.1

12.1.1 Model Definition and Assumptions(2/5)12.1.1 Model Definition and Assumptions(2/5)12.1.1 Model Definition and Assumptions(2/5)12.1.1 Model Definition and Assumptions(2/5)


12.1.1 Model Definition and Assumptions(3/5)12.1.1 Model Definition and Assumptions(3/5)12.1.1 Model Definition and Assumptions(3/5)12.1.1 Model Definition and Assumptions(3/5)• The parameter β0 is known as the intercept parameter, and the

parameter β0 is known as the intercept parameterintercept parameterintercept parameterintercept parameter, and the parameter β1 is known as the slope parameterslope parameterslope parameterslope parameter. A third unknown parameter, the error variance σ2, can also be estimated from the data set. As illustrated in Figure 12.2, the data


in Figure 12.2, the data values (xi , yi ) lie closer to the liney = β0+β1xas the error variance σ2

decreases.

12.1.1 Model Definition and Assumptions(4/5)12.1.1 Model Definition and Assumptions(4/5)12.1.1 Model Definition and Assumptions(4/5)12.1.1 Model Definition and Assumptions(4/5)• The slope parameterslope parameterslope parameterslope parameter β1 is of particular interest since it indicates how

the expected value of the dependent variable depends upon the explanatory variable x, as shown in Figure 12.3

• The data set shown in Figure 12.4 exhibits a quadratic (or at least nonlinear) relationship between the two variables, and it would make no sense to fit a straight line to the data set.


12.1.1 Model Definition and Assumptions(5/5)12.1.1 Model Definition and Assumptions(5/5)12.1.1 Model Definition and Assumptions(5/5)12.1.1 Model Definition and Assumptions(5/5)

• Simple Linear Regression ModelSimple Linear Regression ModelSimple Linear Regression ModelSimple Linear Regression ModelThe simple linear regressionsimple linear regressionsimple linear regressionsimple linear regression modelyi = β0 + β1xi + εi

fit a straight line through a set of paired data observations (x1,y1),…,(xn, yn). The error terms ε1,…,εn are taken to be


(x1,y1),…,(xn, yn). The error terms ε1,…,εn are taken to be independent observations from a N(0,σ2) distribution. The three unknown parameters, the intercept parameterintercept parameterintercept parameterintercept parameter β0 , the slope slope slope slope parameterparameterparameterparameter β1, and the error varianceerror varianceerror varianceerror variance σ2, are estimated from the data set.

12.1.2 Examples(1/2)12.1.2 Examples(1/2)12.1.2 Examples(1/2)12.1.2 Examples(1/2)• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage

The manager of a car plant wishes to investigate how the plant’s electricity usage depends upon the plant’s production.

The linear model

will allow a month’s electrical0 1y xβ β= +


will allow a month’s electricalusage to be estimated as afunction of the month’s pro-duction.

12.1.2 Examples(2/2)12.1.2 Examples(2/2)12.1.2 Examples(2/2)12.1.2 Examples(2/2)


12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2.1 Parameter Estimation(1/4)12.2.1 Parameter Estimation(1/4)12.2.1 Parameter Estimation(1/4)12.2.1 Parameter Estimation(1/4)

0 1 1 1 The regression line is fitted to the data points ( , ), , ( , )

by finding the line that is "closest" to the data points in some sense.

As Figure 12.14 illustrates, the fitted line is

n ny x x y x yβ β= +� K

�

2

1 0 1

chosen to be the line that

the sum of the squares of these vertical deviations

( ( ))

and this is referred to as

n

i i i

minimizes

Q y xβ β== ∑ − +


and this is referred to as

the fit.least squares

12.2.1 Parameter Estimation(2/4)12.2.1 Parameter Estimation(2/4)12.2.1 Parameter Estimation(2/4)12.2.1 Parameter Estimation(2/4)

2

1

2

0 1

1

2

ˆ ˆ With normally distributed error terms, and are maximum

likelihood estimates.

( ) The joint density of the error terms , , is

1 .

2

This likelihoo

n

i

i

n

n

e

ε

σ

β β

ε ε

πσ

=−

∑

�

Q K

2 2

d is maximized by minizing

( ( ))y x Qε β β∑ = ∑ − + =


2 2

0 1

1 0 1

0

1 0 1

1

0 1 1

1 0 1 1 1

( ( ))

2( ( )) and

normal equati

2 ( ( ))

the

ˆ ˆ and

ˆ ˆ

on

s

i i i

n

i i i

n

i i i i

n

i i i

n n

i i i i i i

y x Q

Qy x

Qx y x

y n x

x y x

ε β β

β ββ

β ββ

β β

β β

=

=

=

= = =

∑ = ∑ − + =

∂= − ∑ − +

∂

∂= − ∑ − +

∂

⇒

∑ = + ∑

∑ = ∑ + ∑

�

n 2

ix

12.2.1 Parameter Estimation(3/4)12.2.1 Parameter Estimation(3/4)12.2.1 Parameter Estimation(3/4)12.2.1 Parameter Estimation(3/4)�

� � �

n

1 1 11 n 2 n 2

i=1 i=1

1 10 1 1

2 2 2

1 1

2

( )( )

n ( )

and then

where

( )

( )

n n

i i i i i i i XY

i i XX

n n

i i i i

n n

XX i i i i

n

n x y x y S

x x S

y xy x

n n

S x x x nx

x

β

β β β

= = =

= =

= =

∑ − ∑ ∑= =

∑ − ∑

∑ ∑= − = −

= ∑ − = ∑ −

∑

�


22 1

1

1 1

( )

and

( )( )

nn i ii i

n n

XY i i i i i i

xx

n

S x x y y x y nxy

==

= =

∑= ∑ −

= ∑ − − = ∑ −

� �*

1 11

*

*

0 1

( )( )

For a specific value of the explanatory variable , this equation

ˆ provides a fitted value | for the dependent variable , as

illustrated in Figure 12.15.

n nn i i i ii i i

x

x yx y

n

x

y x yβ β

= ==

∑ ∑= ∑ −

= +

�

12.2.1 Parameter Estimation(4/4)12.2.1 Parameter Estimation(4/4)12.2.1 Parameter Estimation(4/4)12.2.1 Parameter Estimation(4/4)

�

2 The error variance can be estimated by considering the

deviations between the observed data values and their fitted

values . Specifically, the sum of squares for error SSE is

defined t

i

i

y

y

σ�

� � �

� �

2 2

11 1 0

2

1 0 1 1 1

o be the sum of the squares of these deviations

SSE ( ) ( ( ))n n

i i i i i i

n n n

i i i i i i i

y y y x

y y x y

β β

β β

= =

= = =

= ∑ − = ∑ − +

= ∑ − ∑ − ∑


�

1 0 1 1 1

2

and the estimate of the error variance is

SSE

2

i i i i i i i

nσ

= = =

=−


• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage

12

1

12

For this example 12 and

4.51 4.20 58.62

2.48 2.53 34.15

i

i

i

n

x

y

=

=

= + + =

= + + =

∑

∑

L

L


1

122 2 2

1

122 2 2

1

12

1

2.48 2.53 34.15

4.51 4.20 291.2310

2.48 2.53 98.6967

(4.51 2.48) (4.20 2.53) 169.2532

i

i

i

i

i

i

i i

i

y

x

y

x y

=

=

=

=

= + + =

= + + =

= + + =

= × + + × =

∑

∑

∑

∑

L

L

L

L


� 1 1 11

2 2

1 1

2

The estimates of the slope parameter and the intercept parameter :

( )( )

( )

(12 169.2532) (58.62 34.15)0.49883

(12 291.2310) 58.62

34.15

n n n

i i i i

i i i

n n

i i

i i

n x y x y

n x x

β = = =

= =

−=

−

× − ×= =

× −

∑ ∑ ∑

∑ ∑

58.62


� �0 1

34.15(0.49883

12y xβ β= − = − ×

� �

$

0 1

5.5

58.62) 0.4090

12

The fitted regression line :

0.409 0.499

| 0.409 (0.499 5.5) 3.1535

y x x

y

β β

=

= + = +

= + × =

12.2.2 Examples(4/5)12.2.2 Examples(4/5)12.2.2 Examples(4/5)12.2.2 Examples(4/5)Using the model for production values outside this range is known

as and may give inacce uxtrapolatio resra un te lts.

x



�

� �

�

2

0 12

1 1 1

2

98.6967 (0.4090 34.15) (0.49883 169.2532)0.0299

10

0.0299 0.1729

n n n

i i i i

i i i

y y x y

n

β βσ

σ

= = =

− −=

−− × − ×

= =

⇒ = =

∑ ∑ ∑


12.3 Inferences on the Slope Parameter 12.3 Inferences on the Slope Parameter 12.3 Inferences on the Slope Parameter 12.3 Inferences on the Slope Parameter ββββ111112.3.1 Inference Procedures(1/4)12.3.1 Inference Procedures(1/4)12.3.1 Inference Procedures(1/4)12.3.1 Inference Procedures(1/4)Inferences on the Slope Parameter Inferences on the Slope Parameter Inferences on the Slope Parameter Inferences on the Slope Parameter ββββ1111

� � � �

2

1

1 1 / 2, 2 1 1 / 2, 2 1

ˆ , ).

A two-sided confidence interval with a confidence level 1 for the slope

parameter in a simple linear regression model is

( . .( ), . .( ))

XX

n n

S

t s e t s eα α

σβ β

α

β β β β β

1

− −

Ν(

−

∈ − × + ×

� �

�


1 1 / 2, 2 1 1 / 2, 2 1( . .( ), . .( ))

wh

n nt s e t s eα αβ β β β β− −∈ − × + ×

��

��

��

��

/ 2, 2 / 2, 2

1 1 1

, 2 , 2

1 1 1 1

ich is

( , )

One-sided 1 confidence level confidence intervals are

( , ) and ( , )

n n

XX XX

n n

XX XX

t t

S S

t t

S S

α α

α α

σ σβ β β

α

σ σβ β β β

− −

− −

∈ − +

−

∈ −∞ + ∈ − ∞

�

12.3.1 Inference Procedures(2/4)12.3.1 Inference Procedures(2/4)12.3.1 Inference Procedures(2/4)12.3.1 Inference Procedures(2/4)

�

�

0 1 1 1 1

1

1 1

The two-sided hypotheses

: versus :

for a fixed value of interest are tested with the -statistic

A

XX

H b H b

b t

bt

S

β β

β

σ

•

= ≠

− =


The -value is

-value 2 ( | |)

where the random

p

p P X t

= × >

/ 2, 2

variable has a -distribution with 2 degrees of freedom.

A size test rejects the null hypothesis if | | .n

X t n

t tαα −

−

>


0 1 1 1 1

, 2

The one-sided hypotheses

: versus :

have a -value

-value ( )

and a size test rejects the null hypothesis if .

A

n

H b H b

p

p P X t

t tα

β β

α −

•

≥ <

= <

< −

NIPRLNIPRLNIPRLNIPRL 20Slki Lab.Slki Lab.Slki Lab.Slki Lab.

The one-sided hypotheses

H

•

0 1 1 1 1

, 2

: versus :

have a -value

-value ( )

and a size test rejects the null hypothesis if .

A

n

b H b

p

p P X t

t tα

β β

α −

≤ >

= >

>

12.3.1 Inference Procedures(4/4)12.3.1 Inference Procedures(4/4)12.3.1 Inference Procedures(4/4)12.3.1 Inference Procedures(4/4)• An interesting point to notice is that for a fixed value of the error

variance σ2, the variance of the slope parameter estimate decreases as the value of SXX increases. This happens as the values of the explanatory variable xi become more spread out, as illustrated in Figure 12.30. This result


is intuitively reasonable since a greater spread in the values xi provides a greater “leverage” for fitting the regression line, and therefore the slope parameter estimate should be more accurate.

�1β

12.3.2 Examples(1/2)12.3.2 Examples(1/2)12.3.2 Examples(1/2)12.3.2 Examples(1/2)• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage

��

122

2122 1

1

1

( )58.62

291.2310 4.872312 12

0.1729. .( ) 0.0783

4.8723

i

iXX i

i

XX

x

S x

s eS

σβ

=

=

= − = − =

⇒ = = =

∑∑


�

�( )

0 1

1

1

The -statistic for testing : 0

0.498836.37

0.0783. .

The two-sided -value

value 2 ( 6.37) 0

t H

ts e

p

p P X

β

β

β

=

= =

− = × > �


� � � �

( )

0.005,10

1 1 1 1 1

With 3.169, a 99% two-sided confidence interval for the

slope parameter

( critical point . .( ), critical point . .( ))

0.49883 3.169 0.0783, 0.49883 3.169 0.0783

0.251,

t

s e s eβ β β β β

=

∈ − × + ×

= − × + ×

= ( )0.747


12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4.1 Inference Procedures(1/2)12.4.1 Inference Procedures(1/2)12.4.1 Inference Procedures(1/2)12.4.1 Inference Procedures(1/2)

Inferences on the Expected Value of the Dependent VariableInferences on the Expected Value of the Dependent VariableInferences on the Expected Value of the Dependent VariableInferences on the Expected Value of the Dependent Variable

� �

*

0 1

*

* *

A 1 confidence level two-sided confidence interval for , the

expected value of the dependent variable for a particular value of the ex-

planatory variable, is

( . .(

x

x

x x t s eα

α β β

β β β β −

− +

+ ∈ + − × � � *),xβ β+


� �0 1 0 1 / 2, 1 ( . .(nx x t s eαβ β β β −+ ∈ + − × � �

� � � �

� � �

0 1

* *

0 1 / 2, 2 0 1

* 2*

0 1

),

. .( ))

where

1 ( ) . .( )

n

XX

x

x t s e x

x xs e x

n S

α

β β

β β β β

β β σ

−

+

+ + × +

−+ = +


� � � �

� � � �

* * *

0 1 0 1 , 2 0 1

* * *

0 1 0 1 , 1 0 1

One-sided confidence intervals are

( , . .( ))

and

( . .( ), )

n

n

x x t s e x

x x t s e x

α

α

β β β β β β

β β β β β β

−

−

+ ∈ −∞ + + × +

+ ∈ + − × + ∞


� � � �0 1 0 1 , 1 0 1

*

0 1

( . .( ), )

Hypothesis tests on can be performed by comparing the -statistic

(

nx x t s e x

x t

t

αβ β β β β β

β β

β

−+ ∈ + − × + ∞

+

=� �

� �

* *

0 1 0 1

*

0 1

) ( )

. .( )

with a -distribution with 2 degrees of freedom.

x x

s e x

t n

β β β

β β

+ − +

+

−



� � �( )2* * 2

*

0 1

*

0.025,10 0 1

* 2

1 1 ( 4.885). .( ) 0.1729

12 4.8723

With 2.228, a 95% confidence interval for

1 ( 4.885)

XX

x x xs e x

n S

t x

x

β β σ

β β

− −+ = + = × +

= +

−


* 2* *

0 1

*

1 ( 4.885)(0.409 0.499 2.228 0.1729 ,

12 4.8723

10.409 0.499 2.228 0.179

1

xx x

x

β β−

+ ∈ + − × × +

+ + × ×* 2

*

0 1

( 4.885))

2 4.8723

At 5

5 (0.409 (0.499 5) 0.113,0.409 (0.499 5) 0.113)

(2.79,3.02)

x

x

β β

−+

=

+ ∈ + × − + × +

=

12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5.1 Inference Procedures(1/2)12.5.1 Inference Procedures(1/2)12.5.1 Inference Procedures(1/2)12.5.1 Inference Procedures(1/2)

• Prediction Intervals for Future Response ValuesPrediction Intervals for Future Response ValuesPrediction Intervals for Future Response ValuesPrediction Intervals for Future Response Values*

*

A 1 confidence level two-sided prediction interval for | , a future value

of the dependent variable for a particular value of the explanatory variable,

is

xy

x

α−


� � �*

* 2*

0 1 / 2, 1

is

1 ( )| 1nx

X

x xy x t

n Sαβ β σ−

−∈ ( + − + +

� � �* 2

*

0 1 / 2, 2

,

1 ( )1

X

n

XX

x xx t

n Sαβ β σ−

−+ + + + )


� � �*

* 2*

0 1 , 2

One-sided confidence intervals are

1 ( ) | ( , 1 )

and

nxXX

x xy x t

n Sαβ β σ−

−∈ −∞ + + + +


� � �*

* 2*

0 1 , 1

and

1 ( ) | ( 1 , )nx

XX

x xy x t

n Sαβ β σ−

−∈ + − + + ∞



*

*

0.025,10

* 2*

With 2.228, a 95% confidence interval for |

13 ( 4.885)| (0.409 0.499 2.228 0.1729 ,

xt y

xy x

=

−∈ + − × × +


*

* 2*

*

5

| (0.409 0.499 2.228 0.1729 ,12 4.8723

13 ( 4.885)0.409 0.499 2.228 0.179 )

12 4.8723

At 5

| (0.409 (0.499 5) 0.401,0.409

xy x

xx

x

y

∈ + − × × +

−+ + × × +

=

∈ + × − (0.499 5) 0.401)

(2.50,3.30)

+ × +

=

12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6.1 Sum of Squares Decomposition(1/5)12.6.1 Sum of Squares Decomposition(1/5)12.6.1 Sum of Squares Decomposition(1/5)12.6.1 Sum of Squares Decomposition(1/5)


12.6.1 Sum of Squares Decomposition(2/5)12.6.1 Sum of Squares Decomposition(2/5)12.6.1 Sum of Squares Decomposition(2/5)12.6.1 Sum of Squares Decomposition(2/5)



SourceSourceSourceSource Degrees of freedomDegrees of freedomDegrees of freedomDegrees of freedom Sum of squaresSum of squaresSum of squaresSum of squares Mean squaresMean squaresMean squaresMean squares FFFF----statisticstatisticstatisticstatistic pppp----valuevaluevaluevalueRegressionRegressionRegressionRegression

ErrorErrorErrorError1111

NNNN----2222SSRSSRSSRSSRSSESSESSESSE

MSR=SSRMSR=SSRMSR=SSRMSR=SSR=MSE=SSE/(=MSE=SSE/(=MSE=SSE/(=MSE=SSE/(nnnn----2222))))

FFFF=MSR/MSE=MSR/MSE=MSR/MSE=MSR/MSE P( FP( FP( FP( F1,n1,n1,n1,n----2 2 2 2 > F )> F )> F )> F )

TotalTotalTotalTotal nnnn----1111

�2

σ


F I G U R E 12.41 F I G U R E 12.41 F I G U R E 12.41 F I G U R E 12.41 Analysis of variance table for simpleAnalysis of variance table for simpleAnalysis of variance table for simpleAnalysis of variance table for simple

linear regression analysislinear regression analysislinear regression analysislinear regression analysis

12.6.1 Sum of Squares Decomposition(5/5)12.6.1 Sum of Squares Decomposition(5/5)12.6.1 Sum of Squares Decomposition(5/5)12.6.1 Sum of Squares Decomposition(5/5)Coefficient of Determination RCoefficient of Determination RCoefficient of Determination RCoefficient of Determination R2222

2

1

The total variability in the dependent variable, the total sum of squares

SST ( )

can be partitioned into the variability explained by the regression line,

t

n

i iy y=

•

= ∑ −

�

�

2

1he regression sum of squares SSR ( )

and the variability about the regression line, the error sum of squares

n

i iy y= = ∑ −


� 2

1 SSE ( ) .

The proportion of the tot

n

i i iy y= = ∑ −

•

2

coefficient of determination

al variability accounted for by the regression line is

the

SSR SSE 1 1

SSESST SST1

SSR

which takes a value between zero and one.

R

= = − =+



MSR 1.212440.53

MSE 0.0299F = = =


2 SSR 1.21240.802

SST 1.5115R = = =

12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.7.1 Residual Analysis Methods(1/7)12.7.1 Residual Analysis Methods(1/7)12.7.1 Residual Analysis Methods(1/7)12.7.1 Residual Analysis Methods(1/7)

• The residualsresidualsresidualsresiduals are defined to be

so that they are the differences between the observed values of the dependent variable and the corresponding fitted values .

• A property of the residuals

� , 1i i ie y y i n= − ≤ ≤

0n e∑ =

iy�iy


• Residual analysis can be used to– Identify data points that are outliersoutliersoutliersoutliers,– Check whether the fitted model is appropriateappropriateappropriateappropriate,– Check whether the error variance is constantconstantconstantconstant, and– Check whether the error terms are normallynormallynormallynormally distributed.

1 0n

i ie=∑ =

12.7.1 Residual Analysis Methods(2/7)12.7.1 Residual Analysis Methods(2/7)12.7.1 Residual Analysis Methods(2/7)12.7.1 Residual Analysis Methods(2/7)• A nice random scatter plot such as the one in Figure 12.45

⇒ there are no indications of any problems with the regression analysis

• Any patterns in the residual plot or any residuals with a large absolute value alert the experimenter to possible problems with the fitted regression model.


12.7.1 Residual Analysis Methods(3/7)12.7.1 Residual Analysis Methods(3/7)12.7.1 Residual Analysis Methods(3/7)12.7.1 Residual Analysis Methods(3/7)• A data point (xi, yi ) can be considered to be an outlier if it does not appear

to predict well by the fitted model.• Residuals of outliers have a large absolute value, as indicated in Figure

12.46. Note in the figure that is used instead of • [For your interest only] ˆ

ies

.ie 2

2( )1( ) (1 ) .ii

XX

x xVar e

n Ss

-= - -


12.7.1 Residual Analysis Methods(4/7)12.7.1 Residual Analysis Methods(4/7)12.7.1 Residual Analysis Methods(4/7)12.7.1 Residual Analysis Methods(4/7)• If the residual plot shows positive

and negative residuals grouped together as in Figure 12.47, then a linear model is not appropriate. As Figure 12.47 indicates, a nonlinear model is needed for such a data set.


such a data set.

12.7.1 Residual Analysis Methods(5/7)12.7.1 Residual Analysis Methods(5/7)12.7.1 Residual Analysis Methods(5/7)12.7.1 Residual Analysis Methods(5/7)• If the residual plot shows a “funnel

shape” as in Figure 12.48, so that the size of the residuals depends upon the value of the explanatory variable x, then the assumption of a constant error variance σ2 is not valid.


valid.

12.7.1 Residual Analysis Methods(6/7)12.7.1 Residual Analysis Methods(6/7)12.7.1 Residual Analysis Methods(6/7)12.7.1 Residual Analysis Methods(6/7)• A normal probability plot ( a normal score plot) of the residuals

– Check whether the error terms εi appear to be normally distributed.• The normal score of the i th smallest residual

– 1

3

81

4

i

n

−

− Φ

+


• The main body of the points in a normal probability plot lie approximately on a straight line as in Figure 12.49 is reasonable

• The form such as in Figure 12.50 indicates that the distribution is not normal

4

12.7.1 Residual Analysis Methods(7/7)12.7.1 Residual Analysis Methods(7/7)12.7.1 Residual Analysis Methods(7/7)12.7.1 Residual Analysis Methods(7/7)


12.7.2 Examples(1/2)12.7.2 Examples(1/2)12.7.2 Examples(1/2)12.7.2 Examples(1/2)• Example : Nile River FlowrateNile River FlowrateNile River FlowrateNile River Flowrate



$

�

�

5

3.88

| 0.470 (0.836 3.88) 2.77

4.01 2.77 1.24

1.243.75

0.1092

i i i

i

x

y

e y y

e

σ

=

= − + × =

⇒ = − = − =

= =


�

�

�

0.1092

6.13

5.67 ( 0.470 (0.836 6.13)) 1.02

1.023.07

0.1092

i i i

i

x

e y y

e

σ

σ

=

= − = − − + × =

= =

12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.8.1 Intrinsically Linear Models(1/4)12.8.1 Intrinsically Linear Models(1/4)12.8.1 Intrinsically Linear Models(1/4)12.8.1 Intrinsically Linear Models(1/4)


12.8.1 Intrinsically Linear Models(2/4)12.8.1 Intrinsically Linear Models(2/4)12.8.1 Intrinsically Linear Models(2/4)12.8.1 Intrinsically Linear Models(2/4)


12.8.2 Examples(1/5)12.8.2 Examples(1/5)12.8.2 Examples(1/5)12.8.2 Examples(1/5)• Example : Roadway Base AggregatesRoadway Base AggregatesRoadway Base AggregatesRoadway Base Aggregates


12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.9.1 The Sample Correlation Coefficient12.9.1 The Sample Correlation Coefficient12.9.1 The Sample Correlation Coefficient12.9.1 The Sample Correlation Coefficient

Sample Correlation CoefficientSample Correlation CoefficientSample Correlation CoefficientSample Correlation Coefficient

1

2 2

The for a set of paired data observations

( , ) is

( )( )

( ) ( )

sample correlation coeffici tne

i i

n

i i iXY

n nXX YY

r

x y

x x y ySr

S S x x y y

=∑ − −= =

∑ − ∑ −


2 2

1 1

1

2 2 2 2

1 1

( ) ( )

It measures the st gr n te

i

n nXX YY i i i i

n

i i i

n n

i i i

S S x x y y

x y nxy

x nx y ny

= =

=

= =

∑ − ∑ −

∑ −=

∑ − ∑ −

h of association between two variables and can

be thought of as an estimate of the correlation between the two associated

random variable and .

linear

X Y

ρ

0

Under the assumption that the and random variables have a bivariate

normal distribution, a test of the null hypothesis

: 0

can be performed by comparing the -statistic

X Y

H

t

ρ =


2

can be performed by comparing the -statistic

2

1

with a -dis

t

r nt

r

t

−=

−

0 1

tribution with 2 degrees of freedom. In a regression framework,

this test is equivalent to testing : 0.

n

H β−

=

12.9.2 Examples(1/1)12.9.2 Examples(1/1)12.9.2 Examples(1/1)12.9.2 Examples(1/1)• Example : Nile River FlowrateNile River FlowrateNile River FlowrateNile River Flowrate

2 0.871 0.933r R= = =


chapter 12. simple linear regression and...

Documents