11 - 1 © 2003 pearson prentice hall chapter 11 simple linear regression

11 - 11 - 11

Chapter 11Chapter 11

Simple Linear Regression Simple Linear Regression

11 - 11 - 22

Learning ObjectivesLearning Objectives

1.1. Describe the Linear Regression ModelDescribe the Linear Regression Model

2.2. State the Regression Modeling StepsState the Regression Modeling Steps

3.3. Explain Ordinary Least SquaresExplain Ordinary Least Squares

1.1. Understand and check model assumptionsUnderstand and check model assumptions

4.4. Compute Regression CoefficientsCompute Regression Coefficients

5.5. Predict Response VariablePredict Response Variable

6.6. Interpret Computer OutputInterpret Computer Output

11 - 11 - 33

ModelsModels

11 - 11 - 44

ModelsModels

1.1. Representation of Some PhenomenonRepresentation of Some Phenomenon

2.2. Mathematical Model Is a Mathematical Mathematical Model Is a Mathematical Expression of Some PhenomenonExpression of Some Phenomenon

3.3. Often Describe Relationships between Often Describe Relationships between VariablesVariables

4.4. TypesTypes Deterministic ModelsDeterministic Models Probabilistic ModelsProbabilistic Models

11 - 11 - 55

Deterministic Deterministic ModelsModels

1.1. Hypothesize Exact RelationshipsHypothesize Exact Relationships

2.2. Suitable When Prediction Error is Suitable When Prediction Error is NegligibleNegligible

3.3. Example: Force Is Exactly Example: Force Is Exactly Mass Times AccelerationMass Times Acceleration FF = = mm··aa

11 - 11 - 66

Probabilistic ModelsProbabilistic Models

1.1. Hypothesize 2 ComponentsHypothesize 2 Components DeterministicDeterministic Random ErrorRandom Error

2.2. Example: Sales Volume Is 10 Times Example: Sales Volume Is 10 Times Advertising Spending + Random ErrorAdvertising Spending + Random Error YY = 10 = 10X X + + Random Error May Be Due to Factors Random Error May Be Due to Factors

Other Than AdvertisingOther Than Advertising

11 - 11 - 77

Types of Types of Probabilistic ModelsProbabilistic Models

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

11 - 11 - 88

Regression ModelsRegression Models

11 - 11 - 99

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

11 - 11 - 1010

Regression ModelsRegression Models

1.1. Answer ‘What Is the Relationship Answer ‘What Is the Relationship Between the Variables?’Between the Variables?’

2.2. Equation UsedEquation Used 1 Numerical Dependent (Response) Variable1 Numerical Dependent (Response) Variable

What Is to Be PredictedWhat Is to Be Predicted 1 or More Numerical or Categorical 1 or More Numerical or Categorical

Independent (Explanatory) VariablesIndependent (Explanatory) Variables

3.3. Used Mainly for Prediction & EstimationUsed Mainly for Prediction & Estimation

11 - 11 - 1111

Regression Modeling Regression Modeling Steps Steps

1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component

2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters

3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error

4.4. Evaluate ModelEvaluate Model

5.5. Use Model for Prediction & Estimation Use Model for Prediction & Estimation

11 - 11 - 1212

Model SpecificationModel Specification

11 - 11 - 1313

3.3. Specify Probability Distribution of Random Specify Probability Distribution of Random Error TermError Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error

11 - 11 - 1414

Specifying the Specifying the ModelModel

1.1. Define VariablesDefine Variables

2.2. Hypothesize Nature of RelationshipHypothesize Nature of Relationship Expected Effects (i.e., Coefficients’ Signs)Expected Effects (i.e., Coefficients’ Signs) Functional Form (Linear or Non-Linear)Functional Form (Linear or Non-Linear) InteractionsInteractions

11 - 11 - 1515

Model Specification Model Specification Is Based on TheoryIs Based on Theory

1.1. Theory of Field (e.g., Sociology)Theory of Field (e.g., Sociology)

2.2. Mathematical TheoryMathematical Theory

3.3. Previous ResearchPrevious Research

4.4. ‘Common Sense’‘Common Sense’

11 - 11 - 1616

Advertising

Thinking Challenge: Thinking Challenge: Which Is More Which Is More

Logical?Logical?

11 - 11 - 1717

Types of Types of Regression ModelsRegression Models

11 - 11 - 1818

RegressionModels

11 - 11 - 1919

RegressionModels

Simple

1 Explanatory1 ExplanatoryVariableVariable

11 - 11 - 2020

RegressionModels

2+ Explanatory2+ ExplanatoryVariablesVariables

Simple Multiple

11 - 11 - 2121

RegressionModels

Linear

Simple Multiple

11 - 11 - 2222

RegressionModels

LinearNon-

Linear

Simple Multiple

11 - 11 - 2323

RegressionModels

LinearNon-

Linear

Simple Multiple

Linear

11 - 11 - 2424

RegressionModels

LinearNon-

Linear

Simple Multiple

Linear

Non-Linear

11 - 11 - 2525

Linear Regression Linear Regression ModelModel

11 - 11 - 2626

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

11 - 11 - 2727

Y = mX + b

b = Y-intercept

Changein Y

Change in X

m = Slope

Linear EquationsLinear Equations

YY XXii ii ii 00 11

Linear Regression Linear Regression ModelModel

1.1. Relationship Between Variables Is a Relationship Between Variables Is a Linear FunctionLinear Function

Dependent Dependent (Response) (Response) VariableVariable(e.g., income)(e.g., income)

Independent Independent (Explanatory) (Explanatory) Variable Variable (e.g., education)(e.g., education)

Population Population SlopeSlope

Population Population Y-InterceptY-Intercept

Random Random ErrorError

11 - 11 - 2929

Population & Population & Sample Regression Sample Regression

ModelsModels

11 - 11 - 3030

ModelsModels

PopulationPopulation

11 - 11 - 3131

ModelsModels

Unknown Relationship

PopulationPopulation

Y Xi i i 0 1

11 - 11 - 3232

ModelsModels

PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

11 - 11 - 3333

ModelsModels

PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

Y Xi i i 0 1Y Xi i i 0 1

11 - 11 - 3434

Population Linear Population Linear Regression ModelRegression Model

iXYE 10 iXYE 10

ObservedObservedvaluevalue

Observed valueObserved value

ii = Random error= Random error

11 - 11 - 3535

Sample Linear Sample Linear Regression ModelRegression Model

Y Xi i 0 1 Y Xi i 0 1

Unsampled Unsampled observationobservation

ii = Random = Random

errorerror

Observed valueObserved value

11 - 11 - 3636

Estimating Parameters:Estimating Parameters:Least Squares MethodLeast Squares Method

11 - 11 - 3737

5.5. Use Model for Prediction & EstimationUse Model for Prediction & Estimation

11 - 11 - 3838

0204060

0 20 40 60

ScattergramScattergram

1.1. Plot of All (Plot of All (XXii, , YYii) Pairs) Pairs

2.2. Suggests How Well Model Will FitSuggests How Well Model Will Fit

11 - 11 - 3939

0204060

0 20 40 60

Thinking ChallengeThinking Challenge

How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

11 - 11 - 4040

0204060

0 20 40 60

11 - 11 - 4141

0204060

0 20 40 60

11 - 11 - 4242

0204060

0 20 40 60

11 - 11 - 4343

0204060

0 20 40 60

11 - 11 - 4444

0204060

0 20 40 60

11 - 11 - 4545

0204060

0 20 40 60

11 - 11 - 4646

Least SquaresLeast Squares

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Actual Y Values & Predicted Y Values Are a MinimumAre a Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative

11 - 11 - 4747

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Actual Y Values & Predicted Y Values Are a MinimumAre a Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative

iii YY

11 - 11 - 4848

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimuma Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative

2.2. LS Minimizes the Sum of the Squared LS Minimizes the Sum of the Squared Differences (SSE)Differences (SSE)

iii YY

11 - 11 - 4949

Least Squares Least Squares GraphicallyGraphically

Y X2 0 1 2 2 Y X2 0 1 2 2

Y Xi i 0 1 Y Xi i 0 1

LS minimizes ii

11 - 11 - 5050

Coefficient Coefficient EquationsEquations

Sample SlopeSample Slope

Sample Y-interceptSample Y-intercept

Prediction EquationPrediction Equation

xy 10 ˆˆ

yyxxSS

ii xy 10 ˆˆˆ

11 - 11 - 5151

Computation TableComputation Table

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

Xi2 Yi

2 XiYi

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

Xi2 Yi

2 XiYi

11 - 11 - 5252

Interpretation of Interpretation of CoefficientsCoefficients

11 - 11 - 5353

1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 for Each 1

Unit Increase in Unit Increase in XX If If 11 = 2, then Sales ( = 2, then Sales (YY) Is Expected to ) Is Expected to

Increase by 2 for Each 1 Unit Increase in Increase by 2 for Each 1 Unit Increase in Advertising (Advertising (XX))

11 - 11 - 5454

1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 for Each 1

Unit Increase in Unit Increase in XX If If 11 = 2, then Sales ( = 2, then Sales (YY) Is Expected to Increase ) Is Expected to Increase

by 2 for Each 1 Unit Increase in Advertising (by 2 for Each 1 Unit Increase in Advertising (XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Value of Average Value of YY When When XX = 0 = 0

If If 00 = 4, then Average Sales ( = 4, then Average Sales (YY) Is Expected to ) Is Expected to Be 4 When Advertising (Be 4 When Advertising (XX) Is 0) Is 0

11 - 11 - 5555

Parameter Parameter Estimation ExampleEstimation Example

You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You gather the following data:You gather the following data:

Ad $Ad $ Sales (Units)Sales (Units)11 1122 1133 2244 2255 44

What is the What is the relationshiprelationship between sales & advertising?between sales & advertising?

11 - 11 - 5656

0 1 2 3 4 5

Scattergram Scattergram Sales vs. AdvertisingSales vs. Advertising

Advertising

11 - 11 - 5757

Guess The Parameters!Guess The Parameters!

11 - 11 - 5858

0 1 2 3 4 5

Scattergram Scattergram Sales vs. AdvertisingSales vs. Advertising

Advertising

11 - 11 - 5959

Parameter Parameter Estimation Solution Estimation Solution

TableTableXi Yi Xi

2 Yi2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

11 - 11 - 6060

Parameter Parameter Estimation SolutionEstimation Solution

10.0370.02ˆˆ

11 - 11 - 6161

Coefficient Coefficient Interpretation Interpretation

SolutionSolution

11 - 11 - 6262

SolutionSolution1.1. Slope (Slope (11))

Sales Volume (Sales Volume (YY) Is Expected to Increase ) Is Expected to Increase by .7 Units for Each $1 Increase in by .7 Units for Each $1 Increase in Advertising (Advertising (XX))

11 - 11 - 6363

SolutionSolution1.1. Slope (Slope (11))

Sales Volume (Sales Volume (YY) Is Expected to Increase ) Is Expected to Increase by .7 Units for Each $1 Increase in Advertising by .7 Units for Each $1 Increase in Advertising ((XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Value of Sales Volume (Average Value of Sales Volume (YY) Is ) Is

-.10 Units When Advertising (-.10 Units When Advertising (XX) Is 0) Is 0 Difficult to Explain to Marketing ManagerDifficult to Explain to Marketing Manager Expect Some Sales Without AdvertisingExpect Some Sales Without Advertising

11 - 11 - 6464

Parameter EstimatesParameter Estimates

ParameterParameter Standard T for H0: Standard T for H0:

VariableVariable DF DF EstimateEstimate Error Param=0 Prob>|T| Error Param=0 Prob>|T|

INTERCEPINTERCEP 1 1 -0.1000-0.1000 0.6350 -0.157 0.8849 0.6350 -0.157 0.8849

ADVERTADVERT 1 1 0.70000.7000 0.1914 3.656 0.0354 0.1914 3.656 0.0354

Parameter Parameter Estimation Computer Estimation Computer

OutputOutput

11 - 11 - 6565

Derivation of Derivation of Parameter Parameter EquationsEquations

Goal: Minimize squared errorGoal: Minimize squared error

ˆˆˆ

xy 10 ˆˆ

Derivation of Derivation of Parameter Parameter EquationsEquations

ˆˆˆ

yyxxxxxx

yyxxxx

11 - 11 - 6767

Parameter Parameter Estimation Thinking Estimation Thinking

ChallengeChallengeYou’re an economist for the county You’re an economist for the county cooperative. You gather the following data:cooperative. You gather the following data:

Fertilizer (lb.)Fertilizer (lb.) Yield (lb.)Yield (lb.) 4 4 3.03.0 6 6 5.55.51010 6.56.51212 9.09.0

What is the What is the relationshiprelationship between fertilizer & crop yield?between fertilizer & crop yield?

11 - 11 - 6868

0 5 10 15

Scattergram Scattergram Crop Yield vs. Crop Yield vs.

Fertilizer*Fertilizer*

Yield (lb.)Yield (lb.)Yield (lb.)Yield (lb.)

Fertilizer (lb.)Fertilizer (lb.)Fertilizer (lb.)Fertilizer (lb.)

11 - 11 - 6969

Parameter Parameter Estimation Solution Estimation Solution

Table*Table*

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

11 - 11 - 7070

Parameter Parameter Estimation Solution*Estimation Solution*

80.0865.06ˆˆ

11 - 11 - 7171

Solution*Solution*

11 - 11 - 7272

Solution*Solution*

1.1. Slope (Slope (11)) Crop Yield (Crop Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase in Fertilizer by .65 lb. for Each 1 lb. Increase in Fertilizer ((XX))

11 - 11 - 7373

Solution*Solution*

1.1. Slope (Slope (11)) Crop Yield (Crop Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase in Fertilizer by .65 lb. for Each 1 lb. Increase in Fertilizer ((XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Crop Yield (Average Crop Yield (YY) Is Expected to Be ) Is Expected to Be

0.8 lb. When No Fertilizer (0.8 lb. When No Fertilizer (XX) Is Used) Is Used

11 - 11 - 7474

Probability Distribution Probability Distribution

of Random Errorof Random Error

11 - 11 - 7575

11 - 11 - 7676

Linear Regression Linear Regression Assumptions Assumptions

1.1. Mean of Probability Distribution of Mean of Probability Distribution of Error Is 0Error Is 0

2.2. Probability Distribution of Error Has Probability Distribution of Error Has Constant VarianceConstant Variance1.1. Exercise: Constant across what?Exercise: Constant across what?

3.3. Probability Distribution of Error is Probability Distribution of Error is NormalNormal

4.4. Errors Are Independent Errors Are Independent

11 - 11 - 7777

Error Error Probability Probability DistributionDistribution

X 1X 2

11 - 11 - 7878

Random Error Random Error VariationVariation

11 - 11 - 7979

1.1. Variation of Actual Variation of Actual YY from Predicted from Predicted YY

11 - 11 - 8080

2.2. Measured by Standard Error of Measured by Standard Error of Regression ModelRegression Model Sample Standard Deviation of Sample Standard Deviation of , , ss^

11 - 11 - 8181

2.2. Measured by Standard Error of Measured by Standard Error of Regression ModelRegression Model Sample Standard Deviation of Sample Standard Deviation of , , ss

3. 3. Affects Several FactorsAffects Several Factors Parameter SignificanceParameter Significance Prediction AccuracyPrediction Accuracy

11 - 11 - 8282

Evaluating the ModelEvaluating the Model

Testing for SignificanceTesting for Significance

11 - 11 - 8383

5.5. Use Model for Prediction & EstimationUse Model for Prediction & Estimation

11 - 11 - 8484

Test of Slope Test of Slope CoefficientCoefficient

1.1. Shows If There Is a Linear Relationship Shows If There Is a Linear Relationship Between Between XX & & YY

2.2. Involves Population Slope Involves Population Slope 11

3.3. Hypotheses Hypotheses HH00: : 1 1 = 0 (No Linear Relationship) = 0 (No Linear Relationship)

HHaa: : 11 0 (Linear Relationship) 0 (Linear Relationship)

4.4. Theoretical Basis Is Sampling Distribution Theoretical Basis Is Sampling Distribution of Slopeof Slope

11 - 11 - 8585

Sampling Sampling Distribution Distribution

of Sample Slopesof Sample Slopes

11 - 11 - 8686

Population LineX

Sample 1 Line

Sample 2 Line

Population LineX

Sample 1 Line

Sample 2 Line

11 - 11 - 8787

Population LineX

Sample 1 Line

Sample 2 Line

Population LineX

Sample 1 Line

Sample 2 Line

All Possible All Possible Sample SlopesSample Slopes

Sample 1:Sample 1: 2.52.5

Sample 2:Sample 2: 1.6 1.6

Sample 4:Sample 4: 2.12.1 : : : :Very large number of Very large number of sample slopessample slopes

11 - 11 - 8888

Population LineX

Sample 1 Line

Sample 2 Line

Population LineX

Sample 1 Line

Sample 2 Line

All Possible All Possible Sample SlopesSample Slopes

Sample 2:Sample 2: 1.6 1.6

Sample 4:Sample 4: 2.12.1 : : : :Very large number of Very large number of sample slopessample slopes

Sampling DistributionSampling Distribution

11 - 11 - 8989

Slope Coefficient Slope Coefficient Test StatisticTest Statistic

11 - 11 - 9090

Test of Slope Test of Slope Coefficient ExampleCoefficient Example

You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You find You find bb00 = -.1 = -.1,, bb11 = .7 = .7 & & ss = .60553= .60553..

Is the relationship Is the relationship significantsignificant at the at the .05.05 level? level?

11 - 11 - 9191

Solution TableSolution Table

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

11 - 11 - 9292

Test of Slope Test of Slope Parameter Parameter

SolutionSolutionHH00: : 11 = 0 = 0

HHaa: : 11 0 0

.05.05

df df 5 - 2 = 35 - 2 = 3

Critical Value(s):Critical Value(s):

Test Statistic: Test Statistic:

Decision:Decision:

Conclusion:Conclusion:

t0 3.1824-3.1824

Reject Reject

t0 3.1824-3.1824

Reject Reject

0 70 001915

3 656tS

0 70 001915

Reject at Reject at = .05 = .05

There is evidence of a There is evidence of a relationshiprelationship

11 - 11 - 9393

Test StatisticTest StatisticSolutionSolution

1915.0

60553.0

656.31915.0

070.0ˆ

1915.0

60553.0

656.31915.0

070.0ˆ

11 - 11 - 9494

Test of Slope Test of Slope ParameterParameter

Computer OutputComputer Output Parameter EstimatesParameter Estimates

Parameter Standard Parameter Standard T for H0:T for H0:

VariableVariable DF Estimate Error DF Estimate Error Param=0 Prob>|T|Param=0 Prob>|T|

INTERCEP 1 -0.1000 0.6350 -0.157 0.8849INTERCEP 1 -0.1000 0.6350 -0.157 0.8849

ADVERTADVERT 1 0.7000 0.1914 1 0.7000 0.1914 3.6563.656 0.03540.0354

t = k / S

P-Value

11 - 11 - 9595

Measures of Measures of Variation Variation

in Regression in Regression 1.1. Total Sum of Squares (SSTotal Sum of Squares (SSyyyy))

Measures Variation of Observed Measures Variation of Observed YYii Around the MeanAround the MeanYY

2.2. Explained Variation (SSR)Explained Variation (SSR) Variation Due to Relationship Between Variation Due to Relationship Between

XX & & YY

3.3. Unexplained VariationUnexplained Variation (SSE) (SSE) Variation Due to Other FactorsVariation Due to Other Factors

11 - 11 - 9696

Variation MeasuresVariation Measures

Y Xi i 0 1 Y Xi i 0 1

Total sum Total sum

of squares of squares

(Y(Yii - -Y)Y)22

Unexplained sum Unexplained sum

of squares (Yof squares (Yii - -

YYii))22

Explained sum of Explained sum of

squares (Ysquares (Yii - -Y)Y)22 ^

11 - 11 - 9797

1.1. ProportionProportion of Variation ‘Explained’ by of Variation ‘Explained’ by Relationship Between Relationship Between XX & & YY

Coefficient of Coefficient of DeterminationDetermination

Variation Total

Variation Explained

Variation Total

Variation Explained

0 r2 1

11 - 11 - 9898

Coefficient of Coefficient of Determination Determination

ExamplesExamplesY

r2 = 1 r2 = 1

r2 = .8 r2 = 0

11 - 11 - 9999

Coefficient of Coefficient of Determination Determination

ExampleExampleYou’re a marketing analyst for Hasbro You’re a marketing analyst for Hasbro

Toys. You find Toys. You find 00 = -0.1 & = -0.1 & 11 = 0.7. = 0.7.

Interpret a Interpret a coefficient of coefficient of determination determination ofof 0.8167.0.8167.

11 - 11 - 100100

r r 22 Computer Output Computer Output

Root MSE 0.60553Root MSE 0.60553 R-square 0.8167R-square 0.8167

Dep Mean 2.00000 Dep Mean 2.00000 Adj R-sq 0.7556Adj R-sq 0.7556

C.V. 30.27650 C.V. 30.27650

r2 adjusted for number of explanatory variables & sample size

11 - 11 - 101101

Using the Model for Using the Model for Prediction & EstimationPrediction & Estimation

11 - 11 - 102102

11 - 11 - 103103

Prediction With Prediction With Regression ModelsRegression Models

1.1. Types of PredictionsTypes of Predictions Point EstimatesPoint Estimates Interval EstimatesInterval Estimates

2.2. What Is PredictedWhat Is Predicted Population Mean Response Population Mean Response EE((YY) for ) for

Given Given XX Point on Population Regression LinePoint on Population Regression Line

Individual Response (Individual Response (YYii) for Given ) for Given XX

11 - 11 - 104104

What Is PredictedWhat Is Predicted

Mean Y, E(Y)

Y i= 0

^Y Individual

Prediction, Y

E(Y) = 0 + 1X

Mean Y, E(Y)

Y i= 0

^Y Individual

Prediction, Y

E(Y) = 0 + 1X

11 - 11 - 105105

ConfidenceConfidence Interval Interval Estimate of Mean Estimate of Mean YY

StYYEStY

ˆ2/,2ˆ2/,2

ˆ)(ˆ

StYYEStY

ˆ2/,2ˆ2/,2

ˆ)(ˆ

11 - 11 - 106106

Factors Affecting Factors Affecting Interval WidthInterval Width

1.1. Level of Confidence (1 - Level of Confidence (1 - )) Width Increases as Confidence IncreasesWidth Increases as Confidence Increases

2.2. Data Dispersion (Data Dispersion (ss)) Width Increases as Variation IncreasesWidth Increases as Variation Increases

3.3. Sample SizeSample Size Width Decreases as Sample Size IncreasesWidth Decreases as Sample Size Increases

4.4. Distance of Distance of XXpp from Mean from MeanXX Width Increases as Distance IncreasesWidth Increases as Distance Increases

11 - 11 - 107107

Why Distance from Why Distance from Mean?Mean?

Sample 2 Line

XX1 X2

Y_ Sample 1 Line

Sample 2 Line

XX1 X2

Y_ Sample 1 Line

Greater Greater dispersion dispersion than than XX11

11 - 11 - 108108

ConfidenceConfidence Interval Interval Estimate ExampleEstimate Example

You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You find You find bb00 = -.1 = -.1,, bb11 = .7 = .7 & & ss = .60553= .60553..

Estimate the Estimate the meanmean sales when sales when advertising is advertising is $4$4 at the at the .05.05 level. level.

11 - 11 - 109109

Solution TableSolution Table

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

11 - 11 - 110110

ConfidenceConfidence Interval Interval Estimate SolutionEstimate Solution

7553.3)(6445.1

3316.01824.37.2)(3316.01824.37.2

3316.010

160553.

7.247.01.0ˆ

ˆ)(ˆ

ˆ2/,2ˆ2/,2

StYYEStY

7553.3)(6445.1

3316.01824.37.2)(3316.01824.37.2

3316.010

160553.

7.247.01.0ˆ

ˆ)(ˆ

ˆ2/,2ˆ2/,2

StYYEStY

XX to be predicted to be predictedXX to be predicted to be predicted

11 - 11 - 111111

YYnPYYn

StYYStY

ˆ2/,2ˆ2/,2

YYnPYYn

StYYStY

ˆ2/,2ˆ2/,2

PredictionPrediction Interval Interval of Individual of Individual

ResponseResponse

Note!Note!

11 - 11 - 112112

Why the Extra ‘SWhy the Extra ‘S’’??

Expected(Mean) Y

Y i= 0

+ 1X i

Y we're trying to predict

Prediction, Y

E(Y) = 0 + 1X

^Expected(Mean) Y

Y i= 0

+ 1X i

Y we're trying to predict

Prediction, Y

E(Y) = 0 + 1X

11 - 11 - 113113

Interval Estimate Interval Estimate Computer OutputComputer Output

Dep Var Pred Std Err Dep Var Pred Std Err Low95% Upp95% Low95% Upp95%Low95% Upp95% Low95% Upp95%

Obs SALES Value Predict Obs SALES Value Predict Mean Mean Predict PredictMean Mean Predict Predict

1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037 1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037

2 1.000 1.300 0.332 0.244 2.355 -0.897 3.4972 1.000 1.300 0.332 0.244 2.355 -0.897 3.497

3 2.000 2.000 0.271 1.138 2.861 -0.111 4.1113 2.000 2.000 0.271 1.138 2.861 -0.111 4.111

4 2.000 4 2.000 2.700 0.332 1.644 3.755 0.502 4.897 2.700 0.332 1.644 3.755 0.502 4.897

5 4.000 3.400 0.469 1.907 4.892 0.962 5.8375 4.000 3.400 0.469 1.907 4.892 0.962 5.837

Predicted Predicted YY when when XX = 4 = 4

Confidence Confidence IntervalInterval

SSYYPrediction Prediction IntervalInterval

11 - 11 - 114114

Hyperbolic Interval Hyperbolic Interval BandsBands

Y i= 0

+ 1X i

Y i= 0

+ 1X i

11 - 11 - 115115

Correlation ModelsCorrelation Models

11 - 11 - 116116

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

11 - 11 - 117117

Correlation ModelsCorrelation Models

1.1. Answer ‘Answer ‘How Strong How Strong Is the Linear Is the Linear Relationship Between 2 Variables?’Relationship Between 2 Variables?’

2.2. Coefficient of Correlation UsedCoefficient of Correlation Used Population Correlation Coefficient Denoted Population Correlation Coefficient Denoted

(Rho) (Rho) Values Range from -1 to +1Values Range from -1 to +1 Measures Degree of AssociationMeasures Degree of Association

3.3. Used Mainly for UnderstandingUsed Mainly for Understanding

11 - 11 - 118118

1.1. Pearson Product Moment Coefficient of Pearson Product Moment Coefficient of Correlation, Correlation, rr::

Sample Coefficient Sample Coefficient of Correlationof Correlation

ionDeterminat oft Coefficien

11 - 11 - 119119

Coefficient of Correlation Coefficient of Correlation ValuesValues

-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

11 - 11 - 120120

-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

No No CorrelationCorrelation

11 - 11 - 121121

-1.0-1.0 +1.0+1.000

Increasing degree of Increasing degree of negative correlationnegative correlation

-.5-.5 +.5+.5

No No CorrelationCorrelation

11 - 11 - 122122

-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

Perfect Perfect Negative Negative

CorrelationCorrelationNo No

CorrelationCorrelation

11 - 11 - 123123

-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5

Increasing degree of Increasing degree of positive correlationpositive correlation

11 - 11 - 124124

-1.0-1.0 +1.0+1.000

Perfect Perfect Positive Positive

-.5-.5 +.5+.5

11 - 11 - 125125

Coefficient of Coefficient of CorrelationCorrelation ExamplesExamples

r = 1 r = -1

r = .89 r = 0

11 - 11 - 126126

Test of Test of Coefficient of Coefficient of Correlation Correlation

1.1. Shows If There Is a Linear Relationship Shows If There Is a Linear Relationship Between 2 Numerical VariablesBetween 2 Numerical Variables

2.2. Same Conclusion as Testing Same Conclusion as Testing Population Slope Population Slope 11

3.3. Hypotheses Hypotheses HH00: : = 0 (No Correlation) = 0 (No Correlation)

HHaa: : 0 (Correlation) 0 (Correlation)

11 - 11 - 127127

ConclusionConclusion

1.1. Described the Linear Regression ModelDescribed the Linear Regression Model

2.2. Stated the Regression Modeling StepsStated the Regression Modeling Steps

3.3. Explained Ordinary Least SquaresExplained Ordinary Least Squares

4.4. Computed Regression CoefficientsComputed Regression Coefficients

5.5. Predicted Response VariablePredicted Response Variable

6.6. Interpreted Computer OutputInterpreted Computer Output

End of Chapter

Any blank slides that follow are blank intentionally.

11 - 1 © 2003 pearson prentice hall chapter 11 simple linear regression

pearson prentice hall

pearson prentice hall

simple linear regression

explanatory variable

advertising slide

prediction estimation

common sense slide

computer output slide

Documents

12a - 1 © 2000 prentice-hall, inc. statistics multiple...

chapter 11: regression - pepperdine university

11 dynamic regression

statistics for business and economics 7 th edition chapter...

11. multiple regression

chapter 11 nonlinear regression models - nyu

chapter 11 -- regression

copyright © 2010 pearson education, inc., publishing as...

prentice hall, 2003 1 chapter 11

lecture 11 introduction to nonparametric regression

regression analysis © 2007 prentice hall17-1. © 2007...

© 2010 pearson prentice hall. all rights reserved least...

lecture 11: regression methods i (linear...

11 ssgb amity bsi regression

11 multiple regression

business statistics: a first course, 5e © 2009...

10 - 1 © 2001 prentice-hall, inc. statistics for business...

11 multiple regression part1

chapter 11 regression analysis

introduction to regression analysis grade 11 and...