binary, ordinal, and multinomial logistic regression …...ordinal logistic regression 18 the model:...
TRANSCRIPT
Binary, Ordinal, and Multinomial Logistic Regression for Categorical Outcomes
1
Karen Grace-Martin
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
2
1. Linear Model: Quantitative Dependent Variable • The Model
• Interpreting Coefficients
2. Binary Logistic Model: Binary Dependent Variable • The Model
• Interpreting Coefficients
3. Multinomial Logistic Model: Unordered Multi-category Dependent Variable • The Model
• Interpreting Coefficients
4. Proportional Odds Logistic Model: Ordered Multi-category Dependent Variable
• The Model
• Interpreting Coefficients
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Outline
3 ©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
1. Linear Model: Quantitative Dependent Variable
4
Dependent Variable is
• Continuous
• Unbounded
• Measured on an interval or ratio scale
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Linear Model
Linear Regression
5
The Model:
𝐸 𝑌|𝑋 = 𝛽0 + 𝛽1X1 + ⋯ + 𝛽𝑘X𝑘
Parameter Estimation: Ordinary Least Squares
Model Fit: R2
Assumptions: εi ~ i.i.d N(0, σ2)
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
ikikiii XXXY 122110 ...
6 ©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
SAT math
800700600500400300
Colle
ge
GP
A S
core
s
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
Linear Regression
E(College GPA) = -.03 + .20*HSGPA + .003*SATV + .002*SATM -.15*Sports -.26*Male
How to interpret coefficients:
7 ©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
2. Binary Logistic Model: Binary Dependent Variable
8
Two possible outcome values on each trial:
Success/Failure
Academic Warning/Not
1/0
P is the probability of success for any given student at any given value of Xs
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Binary Dependent Variable
P
Binary Logistic Regression
9
The Model:
Appropriate when: Y is binary
Parameter Estimation: Maximum Likelihood
Model Fit: Deviance (-2LL)
Assumptions: trials are independent; probability of success is the same on any given trial with the same values of X
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
kk XXXP
PLn
...
122110
Binary Logistic Regression
10
The Model:
How to interpret coefficients:
eβ, the odds ratio, is the ratio of the odds of two values of X one unit apart
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
kk XXXeYOdds
...22110)1(
kk XXXP
PLn
...
122110
http://thecraftofstatisticalanalysis.com/understanding-probability-odds-ratios-regression/
11 ©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
3. Multinomial Logistic Model:
Unordered Multi-category Dependent Variable
12
More than two unordered possible outcome values on each trial:
1 = Academic Warning
2 = Passed classes and transferred out or withdrew
3 = Passed classes and remained in Good Standing
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Unordered Multi-category Dependent Variable
Multinomial Logistic Regression
13
The Model:
Where j is the number of categories
h=1 to j-1
k is the number of predictors
Appropriate when: Y is categorical, more than 2 categories
Parameter Estimation: Maximum Likelihood
Model Fit: Deviance (-2LL)
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
kkhhhh
j
h XXXP
P
...ln 22110
Multinomial Logistic Regression
14
The Model:
How to interpret coefficients:
Odds Ratios all compared to the reference outcome
Academic warning vs.
Good Standing
Transferred vs. Good Standing
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
kkhhhh
j
h XXXP
P
...ln 22110
kk XXXP
P222211202
3
2 ...ln
kk XXXP
P122111101
3
1 ...ln
Multinomial Logistic Regression
15
The Model:
How to interpret coefficients:
Odds Ratios all compared to the reference outcome
Academic warning vs.
Good Standing
Transferred vs. Good Standing
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
kkhhhh
j
h XXXP
P
...ln 22110
kk XXXP
P222211202
3
2 ...ln
kk XXXP
P122111101
3
1 ...ln
Entire set of coefficients is different
16 ©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
4. Proportional Odds Logistic Model:
Ordered Multi-category Dependent Variable
17
More than two unordered possible outcome values on each trial:
1 = Academic Warning
2 = Good Standing
3 = Dean’s List
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Ordered Multi-category Dependent Variable
Ordinal Logistic Regression
18
The Model:
Where j is the jth ordered category
k is the number of predictors
Appropriate when: Y is ordered categories
Parameter Estimation: Maximum Likelihood
Model Fit: Deviance (-2LL)
Assumptions: Proportional odds/Parallel Lines
©2018 Karen Grace-Martin| http://TheAnalysisFactor.com
kkj
ij
ijXXX
F
F
...
1ln 22110
j
m
imij PF1
Ordinal Logistic Regression
19
The Model:
How to interpret coefficients:
Each Odds Ratio compares to all higher ordered categories
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Academic Warning and Good Standing vs. Dean’s List (1 and 2 vs. 3)
kk
i
i XXXP
P
...
1ln 221101
1
1
kk
ii
ii XXXPP
PP
...
)(1
)(ln 221102
21
21
kkj
ij
ijXXX
F
F
...
1ln 22110
j
m
imij PF1
Academic Warning vs. Good Standing and Dean’s List (1 vs. 2 and 3)
Ordinal Logistic Regression
20
The Model:
How to interpret coefficients:
Each Odds Ratio compares to all higher ordered categories
©2018 Karen Grace-Martin | http://TheAnalysisFactor.com
Academic Warning and Good Standing vs. Dean’s List (1 and 2 vs. 3)
kk
i
i XXXP
P
...
1ln 221101
1
1
kk
ii
ii XXXPP
PP
...
)(1
)(ln 221102
21
21
kkj
ij
ijXXX
F
F
...
1ln 22110
j
m
imij PF1
Academic Warning vs. Good Standing and Dean’s List (1 vs. 2 and 3)
Only the intercept is different