ordinal regression - the statistical collaboration ... · pdf fileoutline introduction basics...

OutlineIntroduction

BasicsApplication

Typical analysis

Ordinal Regression

LISA short course

July 22, 2009

LISA short course Ordinal Regression

OutlineIntroduction

BasicsApplication

Typical analysis

IntroductionCategorical Variable

BasicsMultinomial VariableLogit Models for Multinomial Variable

ApplicationExample 1Example 2Example 3

Typical analysis


OutlineIntroduction

BasicsApplication

Typical analysis

Categorical Variable


A Categorical Variable has a measurement scale consisting of a setof categories.

I In politics: Liberal, Moderate, or Conservative.

I In medicine: Benign, Probably Benign, Suspicious, orMalignant.

I Opinion: Extremely Disagree, ..., Extremely Agree.

In general it measures attitudes, opinions, effectiveness, etc.


OutlineIntroduction

BasicsApplication

Typical analysis


Response variable distinction

I Nominal. Measures having categories without a naturalordering.

I Catholic, Protestant, Jewish, Muslims.I Automobile, Bicycle, Bus, Subway, Walk.I Classical, Country, Folk, Jazz, Rock.I Apartment, Condominium, House, other.

I Ordinal. Measures having categories with natural ordering.I Size: Small, Medium, Large.I Social class: Upper, Middle, Lower.I Medical Condition: Good, Fair, Serious, Critical.I Intervals: < 10 years, 10-12 years, >12 years.


OutlineIntroduction

BasicsApplication

Typical analysis

Multinomial VariableLogit Models for Multinomial Variable

Multinomial Variable

Let Y=j be a random variable having more than two possibleoutcomes (j = 1, 2, . . . , J). The probability P(Y = j) = πj forj = 1, . . . , JEx. Suppose the following question with 5 categories response (Y):The current health plan is awesome...

1. I completely disagree

2. I disagree

3. Neutral (It does not bother me at all)

4. I agree

5. I completely agree

And suppose the respondent chooses number 3, then Y = 3Further, you assume all the categories have the same probability ofbeing selected, then πj(=3) = 1/5


OutlineIntroduction

BasicsApplication

Typical analysis


Probability and its cumulative

Let Y be the response and setting x, the probability can beexpressed as P(Y = j |x) = πj(x). This equation relates theprobability with the actual response and the regressors expressedby x.For ordinal variables, we are more interested in modeling the

P(Y ≤ j |x) = π1(x) + . . .+ πj(x)

Ex. From the last question:

1. P(Y ≤ 1|x) = 1/5

2. P(Y ≤ 2|x) = 1/5 + 1/5 = 2/5

3. P(Y ≤ 3|x) = 1/5 + 1/5 + 1/5 = 3/5

4. P(Y ≤ 4|x) = 1/5 + 1/5 + 1/5 + 1/5 = 4/5

5. P(Y ≤ 5|x) = 1/5 + 1/5 + 1/5 + 1/5 + 1/5 = 1


OutlineIntroduction

BasicsApplication

Typical analysis


The Cumulative Logits

The cumulative odds are defined as:

P(Y ≤ j |x)

1− P(Y ≤ j |x)

and the cumulative logits are defined as:

logit[P(Y ≤ j |x)] = logP(Y ≤ j |x)

1− P(Y ≤ j |x)

logit[P(Y ≤ j |x)] = logπ1(x) + . . .+ πj(x)

πj+1(x) + . . .+ πJ(x)

for j = 1, . . . , J − 1.


OutlineIntroduction

BasicsApplication

Typical analysis


Proportional odds model

logit[P(Y ≤ j |x)] = αj + x′β, for j = 1, . . . , J − 1.

Notice that each cumulative logit has its own intercept, αj .The usual model assumes that the vector β has the same effect foreach logit. The same impact on the logit regardless the category.


OutlineIntroduction

BasicsApplication

Typical analysis


Proportional odds model

The cumulative logit model satisfies:

logit[P(Y ≤ j |x1)]− logit[P(Y ≤ j |x2)] = β′(x1 − x2)

The odds of making Y ≤ j at x = x1 are

exp[β′(x1 − x2)]

times the odds at x = x2

So, it is clear that the log of odds ratio is proportional to thedistance. This property gives the name of this model, proportionalodds model.


OutlineIntroduction

BasicsApplication

Typical analysis


Other cumulative link functions

1. Probit Model:

Φ−1(P(Y ≤ j |x)) = αj + βx

2. Complementary log-log link (Proportional Hazards Model)

log{-log [1− P(Y ≤ j |x)]} = αj + βx.

A property of this link is:

P(Y > j |x1) = [P(Y > j |x2)]exp[β′(x1−x2)]


OutlineIntroduction

BasicsApplication

Typical analysis

Example 1Example 2Example 3

Example

(Agresti, 2002). It is a study of mental health for a random sampleof adult residents of Alachua County, Fl. It relates mentalimpairment to two explanatory variables.

I Mental (response). Mental impairment is an ordinal responsewith categories well, mild symptom formation, moderatesymptom formation, and impaired.

I Life (regressor 1). The life events index is a measure of thenumber of important life events: birth of child, new job,divorce, or death in family, within the past 3 years.

I SES (regressor 2). Socioeconomic status is measured here asbinary: 0-low and 1-high.


OutlineIntroduction

BasicsApplication

Typical analysis


Example

For this first example, let’s considered only one regressor, Life.Then the models is:

logit[P(Mental ≤ j |Life)] = αj + β ∗ Life

The task is to find values for αj and β on the model, for j =Well,Mild, Moderate.A code on the response outcomes scale is exercised: 1:Well, 2:Mild, 3: Moderate, 4: Impaired.


OutlineIntroduction

BasicsApplication

Typical analysis


Example procedure

In JMP:I Download the data set from the following link: http:

//filebox.vt.edu/users/cvelasco/statwww/MentalImpairment.jmp

I Make sure that the response variable has the ORDINAL data type: rightclick on the response’s column and select Column info, then change toOrdinal.

I Go to JMP’s main menu, choose ANALYZE99KFIT MODEL. Then, selectMind from the list of columns, and click on the Y’s icon. Also select Lifeand click on the Add’s icon.

I Make sure that on the Personality box is written Ordinal Logistic(otherwise, go back to the second step)

I Click on the Run Model icon.


http://filebox.vt.edu/users/cvelasco/statwww/MentalImpairment.jmp

http://filebox.vt.edu/users/cvelasco/statwww/MentalImpairment.jmp

OutlineIntroduction

BasicsApplication

Typical analysis



OutlineIntroduction

BasicsApplication

Typical analysis


Example output

The intercept in the analysis is in the plot the following:

e0.2613

1 + e0.2613= 0.565

e1.656

1 + e1.656= 0.84

The final model for the probability under the proportional Oddslogit model is:

P(Y ≤ j |Life) =eα̂j−0.2879∗Life

1 + eα̂j−0.2879∗Life


OutlineIntroduction

BasicsApplication

Typical analysis


Example 2

From the last data set, adding other variable to the model

logit[P(Mental ≤ j |Life,SES)] = αj + β1 ∗ Life + β2 ∗ SES

We also consider the model with a interaction of those twovariables:

logit[P(Mental ≤ j |Life,SES)] = αj+β1∗Life+β2∗SES+β3∗Life∗SES

At this point there is not a plot as part of the output. This isbecause there are two variable into the model, a surfacerepresentation is hard to present. Thus, a summary table pop upas output.


OutlineIntroduction

BasicsApplication

Typical analysis



OutlineIntroduction

BasicsApplication

Typical analysis


Example 3

Male and female subjects received an active or placebo treatmentfor their arthritis pain, and the subsequent extent of improvementwas recorded as marked, some, or none*. (To download the data:http://filebox.vt.edu/users/cvelasco/statwww/Arthritis.jmp)

* Stokes, Maura. 2000. Categorical data analysis using SAS system


http://filebox.vt.edu/users/cvelasco/statwww/Arthritis.jmp

OutlineIntroduction

BasicsApplication

Typical analysis


Example 3


OutlineIntroduction

BasicsApplication

Typical analysis

Questionnaire, having 10 questions (Yij , for i = 1, . . . , n andj = 1, . . . , 10). All of them on the Likert scale (coded from 1 -5).People create scores for each person interviewed:

Zi =10∑j=1

Yij , for i =, . . . , n

This new variable Zi will have a different range. It is not longerordinal. Associating this one to a set of regressor turns to be aregular regression procedure.


ordinal regression - the statistical collaboration ... · pdf fileoutline introduction basics...

Documents