vbm683 machine learning - hacettepepinar/courses/vbm683/lectures/regression.pdf · epi 809/spring...
TRANSCRIPT
![Page 1: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/1.jpg)
VBM683
Machine Learning
Pinar Duygulu
Slides are adapted from
Dhruv Batra,
Aarti Singh, Barnabas Poczos,
Wenjiang Fu
Aykut Erdem
![Page 2: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/2.jpg)
![Page 3: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/3.jpg)
Regression
![Page 4: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/4.jpg)
![Page 5: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/5.jpg)
![Page 6: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/6.jpg)
![Page 7: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/7.jpg)
![Page 8: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/8.jpg)
![Page 9: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/9.jpg)
Wenjiang Fu
What is a Model?
1. Often Describe Relationship between
Variables
2. Types- Deterministic Models (no randomness)
- Probabilistic Models (with randomness)
![Page 10: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/10.jpg)
Wenjiang Fu
Deterministic Models
1. Hypothesize Exact Relationships
2. Suitable When Prediction Error is Negligible
3. Example: Body mass index (BMI) is measure of body
fat based
– BMI = Weight in Kilograms
(Height in Meters)2
![Page 11: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/11.jpg)
Wenjiang Fu
Probabilistic Models
1. Hypothesize 2 Components
• Deterministic
• Random Error
2. Example: Systolic blood pressure of newborns Is 6
Times the Age in days + Random Error
• SBP = 6 x age(d) +
• Random Error May Be Due to Factors Other Than age
in days (e.g. Birthweight)
![Page 12: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/12.jpg)
EPI 809/Spring 2008 12
Types of
Probabilistic Models
Probabilistic
Models
Regression
Models
Correlation
Models
Other
Models
![Page 13: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/13.jpg)
Simple Regression
• Simple regression analysis is a statistical tool that gives us the ability to estimate the mathematical relationship between a dependent variable (usually called y) and an independent variable (usually called x).
• The dependent variable is the variable for which we want to make a prediction.
• While various non-linear forms may be used, simple linear regression models are the most common.
![Page 14: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/14.jpg)
Introduction
• The primary goal of quantitative analysis is to use current information about a phenomenon to predict its future behavior.
• Current information is usually in the form of a set of data.
• In a simple case, when the data form a set of pairs of numbers, we may interpret them as representing the observed values of an independent (or predictor or explanatory) variable X and a dependent ( or response or outcome) variable Y.
lot size Man-hours
30 73
20 50
60 128
80 170
40 87
50 108
60 135
30 69
70 148
60 132
![Page 15: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/15.jpg)
Introduction
• The goal of the analyst who studies
the data is to find a functional
relation
between the response variable y
and the predictor variable x.
)(xfy
0
20
40
60
80
100
120
140
160
180
0 10 20 30 40 50 60 70 80 90
Ma
n-H
ou
r
Lot size
Statistical relation between Lot size and Man-Hour
![Page 16: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/16.jpg)
Pictorial Presentation of Linear Regression Model
![Page 17: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/17.jpg)
EPI 809/Spring 2008 17
Types of
Regression Models
Regression
Models
LinearNon-
Linear
2+ ExplanatoryVariables
Simple Multiple
Linear
1 ExplanatoryVariable
Non-
Linear
![Page 18: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/18.jpg)
Wenjiang Fu
Linear Regression Model
![Page 19: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/19.jpg)
![Page 20: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/20.jpg)
Assumptions
• Linear regression assumes that… – 1. The relationship between X and Y is
linear
– 2. Y is distributed normally at each value of X
– 3. The variance of Y at every value of X is the same (homogeneity of variances)
– 4. The observations are independent
![Page 21: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/21.jpg)
Y
Y = mX + b
b = Y-intercept
X
Change
in Y
Change in X
m = Slope
Linear Equations
© 1984-1994 T/Maker Co.
![Page 22: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/22.jpg)
• 1. Relationship Between Variables Is a
Linear Function
Y Xi i i 0 1
Linear Regression Model
Dependent
(Response)
Variable
(e.g., CD+ c.)
Independent
(Explanatory) Variable
(e.g., Years s. serocon.)
Population
Slope
Population
Y-Intercept
Random
Error
![Page 23: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/23.jpg)
Meaning of Regression Coefficients
• The values of the regression parameters 0, and 1 are not
known. We estimate them from data.
• 1 indicates the change in the mean response per unit
increase in X.
• General regression model
1. 0, and 1 are parameters
2. X is a known constant
3. Deviations are independent N(o, 2)
![Page 24: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/24.jpg)
Y
X
Population Linear
Regression Model
Y Xi i i 0 1
iXYE 10
Observed
value
Observed value
i = Random error
![Page 25: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/25.jpg)
Estimating Parameters:
Least Squares Method
![Page 26: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/26.jpg)
0
20
40
60
0 20 40 60
X
Y
Scatter plot
• 1. Plot of All (Xi, Yi) Pairs
• 2. Suggests How Well Model Will Fit
![Page 27: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/27.jpg)
Thinking Challenge
How would you draw a line through the
points? How do you determine which line
‘fits best’?
0
20
40
60
0 20 40 60
X
Y
![Page 28: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/28.jpg)
Thinking ChallengeHow would you draw a line through the
points? How do you determine which line
‘fits best’?
0
20
40
60
0 20 40 60
X
Y
Slope changed
Intercept unchanged
![Page 29: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/29.jpg)
Thinking ChallengeHow would you draw a line through the
points? How do you determine which line
‘fits best’?
0
20
40
60
0 20 40 60
X
Y
Slope unchanged
Intercept changed
![Page 30: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/30.jpg)
Thinking ChallengeHow would you draw a line through the
points? How do you determine which line
‘fits best’?
0
20
40
60
0 20 40 60
X
Y
Slope changed
Intercept changed
![Page 31: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/31.jpg)
What is the best fitting line
![Page 32: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/32.jpg)
Prediction Error
![Page 33: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/33.jpg)
Least Squares
• 1. ‘Best Fit’ Means Difference Between Actual Y
Values & Predicted Y Values Are a Minimum. But
Positive Differences Off-Set Negative. So square
errors!
• 2. LS Minimizes the Sum of the Squared
Differences (errors) (SSE)
n
i
i
n
i
ii YY1
2
1
2
ˆˆ
![Page 34: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/34.jpg)
Least Squares Graphically
2
Y
X
1
3
4
^^
^^
Y X2 0 1 2 2
Y Xi i 0 1
LS minimizes i
i
n2
1
12
22
32
42
![Page 35: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/35.jpg)
EPI 809/Spring 2008 35
Coefficient Equations
• Prediction equation
• Sample slope
• Sample Y - intercept
ii xy 10ˆˆˆ
21xx
yyxx
SS
SS
i
ii
xx
xy
xy 10ˆˆ
![Page 36: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/36.jpg)
EPI 809/Spring 2008 36
Derivation of Parameters (1)
• Least Squares (L-S):
Minimize squared error
xy 10ˆˆ
22
0 1
0 0
0 1
0
2
i i iy x
ny n n x
22
0 1
1 1
n n
i i i
i i
y x
![Page 37: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/37.jpg)
EPI 809/Spring 2008 37
Derivation of Parameters (1)
• Least Squares (L-S):
Minimize squared error
22
0 1
1 1
0 1
1 1
0
2
2
i i i
i i i
i i i
y x
x y x
x y y x x
1
1
1ˆ
i i i i
i i i i
xy
xx
x x x x y y
x x x x x x y y
SS
SS
![Page 38: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/38.jpg)
EPI 809/Spring 2008 38
Computation Table
Xi Yi Xi2
Yi2
XiYi
X1 Y1 X12
Y12
X1Y1
X2 Y2 X22
Y22
X2Y2
: : : : :
Xn Yn Xn2
Yn2
XnYn
Xi Yi Xi2
Yi2
XiYi
![Page 39: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/39.jpg)
EPI 809/Spring 2008 39
Interpretation of Coefficients
• 1. Slope (1)
– Estimated Y Changes by 1 for Each 1 Unit Increase in X
• If 1 = 2, then Y Is Expected to Increase by 2 for Each 1 Unit Increase
in X
• 2. Y-Intercept (0)
– Average Value of Y When X = 0
• If 0 = 4, then Average Y Is Expected to
Be 4 When X Is 0
![Page 40: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/40.jpg)
EPI 809/Spring 2008 40
Parameter Estimation Example
• Obstetrics: What is the relationship between
Mother’s Estriol level & Birthweight using the
following data?
Estriol Birthweight
(mg/24h) (g/1000)
1 1
2 1
3 2
4 2
5 4
![Page 41: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/41.jpg)
EPI 809/Spring 2008 41
0
1
2
3
4
0 1 2 3 4 5 6
Scatterplot
Birthweight vs. Estriol level
Birthweight
Estriol level
![Page 42: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/42.jpg)
EPI 809/Spring 2008 42
Parameter Estimation Solution
Table
Xi Yi Xi2
Yi2
XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
![Page 43: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/43.jpg)
EPI 809/Spring 2008 43
Parameter Estimation Solution
10.0370.02ˆˆ
70.0
5
1555
5
101537
ˆ
10
2
1
2
12
11
11
XY
n
X
X
n
YX
YX
n
i
n
i
i
i
n
i
i
n
i
in
i
ii
![Page 44: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/44.jpg)
How to estimate parameters
![Page 45: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/45.jpg)
Estimating the intercept and slope: least
squares estimation
** Least Squares Estimation
A little calculus….
What are we trying to estimate? β, the slope, from
What’s the constraint? We are trying to minimize the squared distance (hence the “least squares”)
between the observations themselves and the predicted values , or (also called the “residuals”, or left-
over unexplained variability)
Differencei = yi – (βx + α) Differencei2 = (yi – (βx + α)) 2
Find the β that gives the minimum sum of the squared differences. How do you maximize a function? Take
the derivative; set it equal to zero; and solve. Typical max/min problem from calculus….
From here takes a little math trickery to solve for β…...0))((2
)))(((2))((
1
2
11
2
n
i
iiii
n
i
iii
n
i
ii
xxxy
xxyxyd
d
![Page 46: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/46.jpg)
The standard error of Y given X is the average variability around the regression line at any given
value of X. It is assumed to be equal at all values of X.
Sy/x
Sy/x
Sy/x
Sy/x
Sy/x
Sy/x
![Page 47: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/47.jpg)
![Page 48: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/48.jpg)
C A
B
A
yi
x
y
yi
C
B
*Least squares
estimation gave us the
line (β) that minimized
C2
ii xy
y
A2 B2 C2
SStotal
Total squared distance of observations from naïve mean of y
Total variation
SSreg
Distance from regression line to naïve mean of
y
Variability due to x (regression)
SSresidual
Variance around the regression line
Additional variability not explained by
x—what least squares method aims to
minimize
n
i
ii
n
i
n
i
ii yyyyyy1
2
1 1
22 )ˆ()ˆ()(
Regression Picture
R2=SSreg/SStotal
![Page 49: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/49.jpg)
Regression Line
• If the scatter plot of our sample data suggests a linear relationship between two variables i.e.
we can summarize the relationship by drawing a straight line on the plot.
• Least squares method give us the “best” estimated line for our set of sample data.
xy 10
![Page 50: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/50.jpg)
Regression Line
• We will write an estimated regression line based on sample
data as
• The method of least squares chooses the values for b0, and
b1 to minimize the sum of squared errorsxbby 10
ˆ
2
1
10
1
2)ˆ(
n
i
n
i
ii xbbyyySSE
![Page 51: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/51.jpg)
Regression Line
• Using calculus, we obtain estimating formulas:
or
n
i
n
i
ii
n
i
n
i
n
i
iiii
n
i
i
n
i
ii
xxn
yxyxn
xx
yyxx
b
1 1
22
1 1 1
1
2
11
)()(
))((
xbyb 10
x
y
S
Srb 1
![Page 52: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/52.jpg)
EPI 809/Spring 2008 52
Types of
Probabilistic Models
Probabilistic
Models
Regression
Models
Correlation
Models
Other
Models
![Page 53: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/53.jpg)
EPI 809/Spring 2008 53
• Both variables are treated the same in correlation; in regression there is a predictor and a response
• In regression the x variable is assumed non-random or measured without error
• Correlation is used in looking for relationships, regression for prediction
Correlation vs. regression
![Page 54: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/54.jpg)
EPI 809/Spring 2008 54
Correlation Models
• 1. Answer ‘How Strong Is the Linear Relationship
Between 2 Variables?’
• 2. Coefficient of Correlation Used
– Population Correlation Coefficient Denoted
(Rho)
– Values Range from -1 to +1
– Measures Degree of Association
• 3. Used Mainly for Understanding
![Page 55: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/55.jpg)
Covariance
1
))((
),(cov 1
n
YyXx
yx
n
i
ii
![Page 56: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/56.jpg)
cov(X,Y) > 0 X and Y are positively correlated
cov(X,Y) < 0 X and Y are inversely correlated
cov(X,Y) = 0 X and Y are independent
Interpreting Covariance
![Page 57: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/57.jpg)
Correlation coefficient
Pearson’s Correlation Coefficient is standardized covariance (unitless):
yx
yxariancer
varvar
),(cov
![Page 58: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/58.jpg)
Correlation
• Measures the relative strength of the linear relationship between two variables
• Unit-less
• Ranges between –1 and 1
• The closer to –1, the stronger the negative linear
relationship
• The closer to 1, the stronger the positive linear
relationship
• The closer to 0, the weaker any positive linear
relationship
![Page 59: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/59.jpg)
EPI 809/Spring 2008 59
• 1. Pearson Product Moment Coefficient of Correlation
between x and y:
Sample Coefficient
of Correlation
yyxx
xy
n
i
i
n
i
i
n
i
ii
SSSS
SS
YYXX
YYXX
r
1
2
1
2
1
![Page 60: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/60.jpg)
EPI 809/Spring 2008 60
Coefficient of Correlation
Values
-1.0 +1.00-.5 +.5
![Page 61: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/61.jpg)
EPI 809/Spring 2008 61
Coefficient of Correlation
Values
-1.0 +1.00-.5 +.5
No
Correlation
![Page 62: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/62.jpg)
EPI 809/Spring 2008 62
Coefficient of Correlation
Values
-1.0 +1.00
Increasing degree of
negative correlation
-.5 +.5
No
Correlation
![Page 63: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/63.jpg)
EPI 809/Spring 2008 63
Coefficient of Correlation
Values
-1.0 +1.00-.5 +.5
Perfect
Negative
Correlation
No
Correlation
![Page 64: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/64.jpg)
EPI 809/Spring 2008 64
Coefficient of Correlation
Values
-1.0 +1.00-.5 +.5
Perfect
Negative
Correlation
No
Correlation
Increasing degree of
positive correlation
![Page 65: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/65.jpg)
EPI 809/Spring 2008 65
Coefficient of Correlation
Values
-1.0 +1.00
Perfect
Positive
Correlation
-.5 +.5
Perfect
Negative
Correlation
No
Correlation
![Page 66: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/66.jpg)
Scatter Plots of Data with Various Correlation
CoefficientsY
X
Y
X
Y
X
Y
X
Y
X
r = -1 r = -.6 r = 0
r = +.3r = +1
Y
Xr = 0
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 67: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/67.jpg)
Y
X
Y
X
Y
Y
X
X
Linear relationships Curvilinear relationships
Linear Correlation
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 68: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/68.jpg)
Y
X
Y
X
Y
Y
X
X
Strong relationships Weak relationships
Linear Correlation
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 69: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/69.jpg)
Linear Correlation
Y
X
Y
X
No relationship
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 70: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/70.jpg)
Calculating by hand…
1
)(
1
)(
1
))((
varvar
),(covˆ
1
2
1
2
1
n
yy
n
xx
n
yyxx
yx
yxariancer
n
i
i
n
i
i
n
i
ii
![Page 71: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/71.jpg)
Simpler calculation formula…
yx
xy
n
i
i
n
i
i
n
i
ii
n
i
i
n
i
i
n
i
ii
SSSS
SS
yyxx
yyxx
n
yy
n
xx
n
yyxx
r
1
2
1
2
1
1
2
1
2
1
)()(
))((
1
)(
1
)(
1
))((
ˆ
yx
xy
SSSS
SSr ˆ
Numerator of covariance
Numerators of variance
![Page 72: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/72.jpg)
Least Square estimation
Slope (beta coefficient) =
)(
),(ˆxVar
yxCov
),( yx
x-yˆ :Calculate Intercept=
Regression line always goes through the point:
![Page 73: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/73.jpg)
Relationship with correlation
y
x
SD
SDr ˆ
In correlation, the two variables are treated as equals. In regression, one variable is considered independent (=predictor) variable
(X) and the other the dependent (=outcome) variable Y.
![Page 74: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/74.jpg)
Residual Analysis: check
assumptions
• The residual for observation i, ei, is the difference between its observed and predicted value
• Residuals are highly useful for studying whether a given regression model is appropriate
for the data at hand.
• Check the assumptions of regression by examining the residuals
– Examine for linearity assumption
– Examine for constant variance for all levels of X (homoscedasticity)
– Evaluate normal distribution assumption
– Evaluate independence assumption
• Graphical Analysis of Residuals
– Can plot residuals vs. X
iii YYe ˆ
![Page 75: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/75.jpg)
Residual =
observed - predicted
14ˆ
34ˆ
48
ii
i
i
yy
y
y
X=95 nmol/L
34
![Page 76: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/76.jpg)
Residual Analysis for
Linearity
Not Linear Linear
x
resid
ua
ls
x
Y
x
Y
x
resid
ua
ls
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 77: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/77.jpg)
Residual Analysis for Homoscedasticity
Non-constant variance Constant variance
x x
Y
x x
Y
resid
ua
ls
resid
ua
ls
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 78: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/78.jpg)
Residual Analysis for Independence
Not Independent
Independent
X
X
resid
ua
ls
resid
ua
lsX
resid
ua
ls
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall
![Page 79: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/79.jpg)
Example: weekly advertising expenditure
y x y-hat Residual (e)
1250 41 1270.8 -20.8
1380 54 1411.2 -31.2
1425 63 1508.4 -83.4
1425 54 1411.2 13.8
1450 48 1346.4 103.6
1300 46 1324.8 -24.8
1400 62 1497.6 -97.6
1510 61 1486.8 23.2
1575 64 1519.2 55.8
1650 71 1594.8 55.2
![Page 80: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/80.jpg)
Estimation of the variance of the error terms, 2
• The variance 2 of the error terms i in the regression
model needs to be estimated for a variety of purposes.
– It gives an indication of the variability of the probability
distributions of y.
– It is needed for making inference concerning regression function
and the prediction of y.
![Page 81: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/81.jpg)
Regression Standard Error
• To estimate we work with the variance and take the square root to obtain the standard deviation.
• For simple linear regression the estimate of 2 is the average squared residual.
• To estimate , use
• s estimates the standard deviation of the error term in the statistical model for simple linear regression.
222
. )ˆ(2
1
2
1iiixy yy
ne
ns
2
.. xyxy ss
![Page 82: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/82.jpg)
Regression Standard Error
y x y-hat Residual (e) square(e)
1250 41 1270.8 -20.8 432.64
1380 54 1411.2 -31.2 973.44
1425 63 1508.4 -83.4 6955.56
1425 54 1411.2 13.8 190.44
1450 48 1346.4 103.6 10732.96
1300 46 1324.8 -24.8 615.04
1400 62 1497.6 -97.6 9525.76
1510 61 1486.8 23.2 538.24
1575 64 1519.2 55.8 3113.64
1650 71 1594.8 55.2 3047.04
y-hat = 828+10.8X total 36124.76
Sy .x 67.19818
![Page 83: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/83.jpg)
Residual plots• The points in this
residual plot have a
curve pattern, so a
straight line fits poorly
![Page 84: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/84.jpg)
Residual plots• The points in this plot
show more spread for
larger values of the
explanatory variable x,
so prediction will be
less accurate when x is
large.
![Page 85: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/85.jpg)
Variable transformations
• If the residual plot suggests that the variance is not constant, a transformation can be used to stabilize the variance.
• If the residual plot suggests a non linear relationship between x and y, a transformation may reduce it to one that is approximately linear.
• Common linearizing transformations are:
• Variance stabilizing transformations are:)log(,1
xx
2,),log(,1
yyyy
![Page 86: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/86.jpg)
2 predictors: age and vit D…
![Page 87: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/87.jpg)
Different 3D view…
![Page 88: VBM683 Machine Learning - Hacettepepinar/courses/VBM683/lectures/regression.pdf · EPI 809/Spring 2008 54 Correlation Models • 1. Answer ‘How Strong Is the Linear Relationship](https://reader030.vdocuments.mx/reader030/viewer/2022040109/5e693d00a6499169e4262775/html5/thumbnails/88.jpg)
Fit a plane rather than a line…
On the plane, the slope
for vitamin D is the
same at every age;
thus, the slope for
vitamin D represents
the effect of vitamin D
when age is held
constant.