epi 809/spring 2008 1 chapter 11 regression and correlation methods

EPI 809/Spring 2008EPI 809/Spring 2008 11

Chapter 11Chapter 11

Regression and Correlation Regression and Correlation methodsmethods


Learning ObjectivesLearning Objectives

1.1. Describe the Linear Regression ModelDescribe the Linear Regression Model

2.2. State the Regression Modeling StepsState the Regression Modeling Steps

3.3. Explain Ordinary Least SquaresExplain Ordinary Least Squares

4.4. Compute Regression CoefficientsCompute Regression Coefficients

5.5. Understand and check model assumptionsUnderstand and check model assumptions

6.6. Predict Response VariablePredict Response Variable

7.7. Comments of SAS OutputComments of SAS Output


Learning Objectives… Learning Objectives…

8.8. Correlation ModelsCorrelation Models

9.9. Link between a correlation model and a Link between a correlation model and a regression modelregression model

10.10. Test of coefficient of CorrelationTest of coefficient of Correlation


ModelsModels


What is a Model?What is a Model?

1.1. Representation of Representation of

Some PhenomenonSome Phenomenon

Non-Math/Stats ModelNon-Math/Stats Model

1.1. Representation of Representation of

Some PhenomenonSome Phenomenon

Non-Math/Stats ModelNon-Math/Stats Model


What is a Math/Stats Model?What is a Math/Stats Model?

1.1. Often Describe Relationship between Often Describe Relationship between VariablesVariables

2.2. TypesTypes- Deterministic Models (no randomness)Deterministic Models (no randomness)

- Probabilistic Models (with randomness)Probabilistic Models (with randomness)


Deterministic ModelsDeterministic Models

1.1. Hypothesize Exact RelationshipsHypothesize Exact Relationships2.2. Suitable When Prediction Error is NegligibleSuitable When Prediction Error is Negligible3.3. Example: Body mass index (BMI) is measure of Example: Body mass index (BMI) is measure of

body fat basedbody fat based

Metric Formula: BMI = BMI = Weight in KilogramsWeight in Kilograms (Height in Meters) (Height in Meters)22

Non-metric Formula: BMI = BMI = Weight (pounds)x703Weight (pounds)x703 (Height in inches)(Height in inches)22


Probabilistic ModelsProbabilistic Models

1.1. Hypothesize 2 ComponentsHypothesize 2 Components• DeterministicDeterministic• Random ErrorRandom Error

2.2. Example: Systolic blood pressure of newborns Example: Systolic blood pressure of newborns Is 6 Times the Age in days + Random ErrorIs 6 Times the Age in days + Random Error

• SBPSBP = 6xage(d) = 6xage(d) + + • Random Error May Be Due to Factors Random Error May Be Due to Factors

Other Than age in days (e.g. Birthweight)Other Than age in days (e.g. Birthweight)


Types of Types of Probabilistic ModelsProbabilistic Models

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels


Regression ModelsRegression Models


Types of Types of Probabilistic ModelsProbabilistic Models

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels


Regression ModelsRegression Models

Relationship between one Relationship between one dependentdependent variablevariable and and explanatory variable(s)explanatory variable(s)

Use equation to set up relationshipUse equation to set up relationship• NumericalNumerical Dependent (Response) Variable Dependent (Response) Variable• 1 or More Numerical or Categorical Independent 1 or More Numerical or Categorical Independent

(Explanatory) Variables(Explanatory) Variables

Used Mainly for Prediction & EstimationUsed Mainly for Prediction & Estimation


Regression Modeling Steps Regression Modeling Steps

1.1. Hypothesize Deterministic Hypothesize Deterministic ComponentComponent

• Estimate Unknown ParametersEstimate Unknown Parameters

2.2. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term

• Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error

3.3. Evaluate the fitted ModelEvaluate the fitted Model 4.4. Use Model for Prediction & Use Model for Prediction &

Estimation Estimation


Model SpecificationModel Specification


Specifying the deterministic Specifying the deterministic componentcomponent

1.1. Define the dependent variable and Define the dependent variable and independent variableindependent variable

2.2. Hypothesize Nature of RelationshipHypothesize Nature of Relationship Expected Effects (i.e., Coefficients’ Signs)Expected Effects (i.e., Coefficients’ Signs) Functional Form (Linear or Non-Linear)Functional Form (Linear or Non-Linear) InteractionsInteractions


Model Specification Model Specification Is Based on TheoryIs Based on Theory

1.1. Theory of Field (e.g., Theory of Field (e.g., Epidemiology)Epidemiology)

2.2. Mathematical TheoryMathematical Theory 3.3. Previous ResearchPrevious Research 4.4. ‘Common Sense’‘Common Sense’


Thinking Challenge: Thinking Challenge: Which Is More Logical?Which Is More Logical?

Years since seroconversion

CD+ counts

CD+ counts




CD+ counts

CD+ counts


OB/GYN Study OB/GYN Study


Types of Types of Regression ModelsRegression Models



RegressionModels



RegressionModels

Simple

1 Explanatory1 ExplanatoryVariableVariable



RegressionModels

2+ Explanatory2+ ExplanatoryVariablesVariables

Simple Multiple




RegressionModels

Linear


Simple Multiple




RegressionModels

LinearNon-

Linear


Simple Multiple




RegressionModels

LinearNon-

Linear


Simple Multiple

Linear




RegressionModels

LinearNon-

Linear


Simple Multiple

Linear


Non-Linear


Linear Regression Linear Regression ModelModel



RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable


Y

Y = mX + b

b = Y-intercept

X

Changein Y

Change in X

m = Slope

Linear EquationsLinear Equations

© 1984-1994 T/Maker Co.

YY XXii ii ii 00 11

Linear Regression ModelLinear Regression Model

1.1. Relationship Between Variables Is Relationship Between Variables Is a Linear Functiona Linear Function

Dependent Dependent (Response) (Response) VariableVariable(e.g., CD+ c.)(e.g., CD+ c.)

Independent Independent (Explanatory) Variable (Explanatory) Variable (e.g., Years s. serocon.)(e.g., Years s. serocon.)

Population Population SlopeSlope

Population Population Y-InterceptY-Intercept

Random Random ErrorError


Population & Sample Population & Sample Regression ModelsRegression Models



PopulationPopulation



Unknown Relationship

PopulationPopulation

Y Xi i i 0 1




PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1




PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

Y Xi i i 0 1Y Xi i i 0 1


Y

X

Y

X

Population Linear Regression Population Linear Regression ModelModel


iXYE 10 iXYE 10

ObservedObservedvaluevalue

Observed valueObserved value

ii = Random error= Random error


Y

X

Y

X


Sample Linear Regression Sample Linear Regression ModelModel

Y Xi i 0 1 Y Xi i 0 1

Unsampled Unsampled observationobservation

ii = Random = Random

errorerror

Observed valueObserved value

^̂


Estimating Parameters:Estimating Parameters:Least Squares MethodLeast Squares Method


0204060

0 20 40 60

X

Y

Scatter plotScatter plot

1.1. Plot of All (Plot of All (XXii, , YYii) Pairs) Pairs

2.2. Suggests How Well Model Will FitSuggests How Well Model Will Fit


Thinking ChallengeThinking Challenge

How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’? ‘fits best’?

0204060

0 20 40 60

X

Y


Thinking ChallengeThinking ChallengeHow would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

0204060

0 20 40 60

X

YSlope changed

Intercept unchanged


Thinking ChallengeThinking Challenge

How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

0204060

0 20 40 60

X

Y

Slope unchanged

Intercept changed


Thinking ChallengeThinking ChallengeHow would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

0204060

0 20 40 60

X

YSlope changed

Intercept changed


Least SquaresLeast Squares

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimum. a Minimum. ButBut Positive Differences Off- Positive Differences Off-Set Negative onesSet Negative ones



1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values is a Actual Y Values & Predicted Y Values is a Minimum. Minimum. ButBut Positive Differences Off-Set Positive Differences Off-Set Negative ones. Negative ones. So square errors!So square errors!

n

ii

n

iii YY

1

2

1

2ˆˆ

n

ii

n

iii YY

1

2

1

2ˆˆ



1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimum. a Minimum. ButBut Positive Differences Off- Positive Differences Off-Set Negative. So square errors!Set Negative. So square errors!

2.2. LS Minimizes the Sum of the LS Minimizes the Sum of the Squared Differences (errors) (SSE)Squared Differences (errors) (SSE)

n

ii

n

iii YY

1

2

1

2ˆˆ

n

ii

n

iii YY

1

2

1

2ˆˆ


Least Squares GraphicallyLeast Squares Graphically

2

Y

X

1 3

4

^^

^̂2

Y

X

1 3

4

^^

^^

Y X2 0 1 2 2 Y X2 0 1 2 2

Y Xi i 0 1 Y Xi i 0 1

LS minimizes ii

n2

112

22

32

42

LS minimizes ii

n2

112

22

32

42


Coefficient EquationsCoefficient Equations

Prediction equationPrediction equation

Sample slopeSample slope

Sample Y - interceptSample Y - intercept

ii xy 10ˆˆˆ

21̂

xx

yyxxSS

SS

i

ii

xx

xy

xy 10 ˆˆ


Derivation of Parameters (1)Derivation of Parameters (1)

Least Squares (L-S): Least Squares (L-S):

Minimize squared errorMinimize squared error

xy 10 ˆˆ

220 1

0 0

0 1

0

2

i i iy x

ny n n x

220 1

1 1

n n

i i ii i

y x


Derivation of Parameters (1)Derivation of Parameters (1)

Least Squares (L-S): Least Squares (L-S):

Minimize squared errorMinimize squared error

220 1

1 1

0 1

1 1

0

2

2

i i i

i i i

i i i

y x

x y x

x y y x x

1

1

1̂

i i i i

i i i i

xy

xx

x x x x y y

x x x x x x y y

SS

SS


Computation TableComputation Table

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

XiYi

Xi2 Yi

2 XiYi

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

XiYi

Xi2 Yi

2 XiYi


Interpretation of CoefficientsInterpretation of Coefficients



1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 Unit for Each 1 Unit

Increase in Increase in XX• If If 11 = 2, then = 2, then YY Is Expected to Increase by 2 for Is Expected to Increase by 2 for

Each 1 Unit Increase in Each 1 Unit Increase in XX

^̂

^̂

^̂



1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 Unit for Each 1 Unit

Increase in Increase in XX• If If 11 = 2, then = 2, then YY Is Expected to Increase by 2 for Is Expected to Increase by 2 for

Each 1 Unit Increase in Each 1 Unit Increase in XX

2.2. Y-Intercept (Y-Intercept (00)) Average Value of Average Value of YY When When XX = 0 = 0

• If If 00 = 4, then Average = 4, then Average YY Is Expected to Be Is Expected to Be

4 When 4 When XX Is 0 Is 0

^̂

^̂

^̂

^̂

^̂


Parameter Estimation ExampleParameter Estimation Example Obstetrics:Obstetrics: What is the What is the relationshiprelationship between between

Mother’s Estriol level & Birthweight using the Mother’s Estriol level & Birthweight using the following data?following data?

EstriolEstriol BirthweightBirthweight

(mg/24h)(mg/24h) (g/1000)(g/1000)

11 1122 1133 2244 2255 44


0

1

2

3

4

0 1 2 3 4 5 6

Scatterplot Scatterplot Birthweight vs. Estriol level Birthweight vs. Estriol level

Birthweight

Estriol level


Parameter Estimation Solution Parameter Estimation Solution TableTable

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37


Parameter Estimation SolutionParameter Estimation Solution

10.0370.02ˆˆ

70.0

515

55

51015

37ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii

10.0370.02ˆˆ

70.0

515

55

51015

37ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii


Coefficient Interpretation Coefficient Interpretation SolutionSolution



1.1. Slope (Slope (11)) Birthweight (Birthweight (YY) Is Expected to Increase by .7 ) Is Expected to Increase by .7

Units for Each 1 unit Increase in Estriol (Units for Each 1 unit Increase in Estriol (XX))

^̂



1.1. Slope (Slope (11)) Birthweight (Birthweight (YY) Is Expected to Increase by .7 ) Is Expected to Increase by .7

Units for Each 1 unit Increase in Estriol (Units for Each 1 unit Increase in Estriol (XX))

2.2. Intercept (Intercept (00)) Average Birthweight (Average Birthweight (YY) Is -.10 Units When ) Is -.10 Units When

Estriol level (Estriol level (XX) Is 0) Is 0• Difficult to explainDifficult to explain• The birthweight should always be positiveThe birthweight should always be positive

^̂

^̂


SAS codes for fitting a simple linear SAS codes for fitting a simple linear regressionregression

DataData BW; /*Reading data in SAS*/ BW; /*Reading data in SAS*/ input input estriol birthwestriol birthw@@;@@; cards;cards; 11 11 2 2 1 1 33 22

44 2 2 5 5 44 ; ; runrun;;

PROC REGPROC REG data=BW data=BW; /*Fitting linear regression ; /*Fitting linear regression models*/models*/

model model birthwbirthw==estriolestriol;; runrun; ;


Parameter EstimatesParameter Estimates

Parameter Standard

Variable DF Estimate Error t Value Pr > |t|

Intercept 1 -0.10000 0.63509 -0.16 0.8849

Estriol 1 0.70000 0.19149 3.66 0.0354

Parameter Estimation Parameter Estimation SAS Computer OutputSAS Computer Output

0^̂ 1

^̂


Parameter Estimation Thinking Parameter Estimation Thinking ChallengeChallenge

You’re a Vet epidemiologist for the county You’re a Vet epidemiologist for the county cooperative. You gather the following data:cooperative. You gather the following data:

Food (lb.)Food (lb.) Milk yield (lb.)Milk yield (lb.) 4 4 3.03.0 6 6 5.55.51010 6.56.51212 9.09.0

What is the What is the relationshiprelationship between cows’ food intake and milk yield?between cows’ food intake and milk yield?

© 1984-1994 T/Maker Co.


02468

10

0 5 10 15

02468

10

0 5 10 15

Scattergram Scattergram Milk Yield vs. Food intake*Milk Yield vs. Food intake*

M. Yield (lb.)M. Yield (lb.)M. Yield (lb.)M. Yield (lb.)

Food intake (lb.)Food intake (lb.)Food intake (lb.)Food intake (lb.)


Parameter Estimation Solution Parameter Estimation Solution Table*Table*

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218


Parameter Estimation Solution*Parameter Estimation Solution*

80.0865.06ˆˆ

65.0

432

296

42432

218ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii

80.0865.06ˆˆ

65.0

432

296

42432

218ˆ

10

2

1

2

12

11

11

XY

n

X

X

n

YX

YX

n

i

n

ii

i

n

ii

n

iin

iii


Coefficient Interpretation Coefficient Interpretation Solution*Solution*



1.1. Slope (Slope (11)) Milk Yield (Milk Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase in Food by .65 lb. for Each 1 lb. Increase in Food intake (intake (XX))

^̂



1.1. Slope (Slope (11)) Milk Milk Yield (Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase inby .65 lb. for Each 1 lb. Increase in Food Food intakeintake ( (XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Milk yield (Average Milk yield (YY) Is Expected to Be 0.8 ) Is Expected to Be 0.8

lb. When Food intake (lb. When Food intake (XX) Is 0) Is 0

^̂

^̂

epi 809/spring 2008 1 chapter 11 regression and correlation methods

Documents

linear regression model

regression models relationship

explanatory variable

correlation slide

counts slide

randomness slide

birthweight slide

correlation models