been there / done that: stata logistic regression (……) conjoint analysis coming up:
DESCRIPTION
Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up: Multi-level analyses. AMMBR course design. CONTENT METHOD. Y is 0/1. logistic regression. conjoint analysis. multi-level methods. Intro: multi-level analysis Logistic regression: finale (?). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/1.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Been there / done that:• Stata• Logistic regression (……)• Conjoint analysis
Coming up:• Multi-level analyses
![Page 2: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/2.jpg)
Advanced Methods and Models in Behavioral Research – 2014
AMMBR course design CONTENT METHOD
Y is 0/1
conjoint analysis
logistic regression
multi-level methods
![Page 3: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/3.jpg)
• Intro: multi-level analysis
• Logistic regression: finale (?)
Advanced Methods and Models in Behavioral Research – 2014
![Page 4: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/4.jpg)
Advanced Methods and Models in Behavioral Research – 2014
MULTI – LEVEL ANALYSIS
![Page 5: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/5.jpg)
In with the (multi-level) statistics...
Advanced Methods and Models in Behavioral Research – 2014
Y = student gradeX = introversion
Y = manager gradeX = age
![Page 6: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/6.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Multi-level models or ...
dealing with clustered data.One solution: the variance component model
• Bayesian hierarchical models • mixed models (in SPSS)• hierarchical linear models • random effects models • random coefficient models • subject specific models • variance component models • variance heterogeneity models
![Page 7: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/7.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Clustered data -> multi-level models
• Pupils within schools (within regions within countries)
• Firms within regions (or sectors)
• Vignettes within persons
• Employees within stores (our fastfood.dta example)
![Page 8: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/8.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Two issues with clustered data
• Your estimates will (in all likelihood) be too precise: you find effects that do not exist in the population
[make sure you get that]
• You will want to distinguish between effects within clusters and effects between clusters
[see next two slides]
![Page 9: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/9.jpg)
Advanced Methods and Models in Behavioral Research – 2014
On individual vs aggregate data
For instance: X = introversion X = age of McDonald’s employee Y = student grade(s) Y = like the manager
![Page 10: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/10.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Had we only known, that the data are clustered!
So the effect of an X within clusters can be different from the effect between clusters!
Using the school example: lines represent schools. And within schools the effect of being introvert is positive!
![Page 11: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/11.jpg)
Advanced Methods and Models in Behavioral Research – 2014
![Page 12: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/12.jpg)
Advanced Methods and Models in Behavioral Research – 2014
MAIN MESSAGESBe able to recognize clustered data and deal with it appropriately (how to do that will follow)
Distinguish two kinds of effects: those at the "micro-level" (within clusters) vs those at the aggregate level (between clusters). They need not be the same!
(and ... do not test a micro-hypothesis with aggregate data)
![Page 13: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/13.jpg)
Multi-level analysis:
variance at different levels
Advanced Methods and Models in Behavioral Research – 2014
![Page 14: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/14.jpg)
Advanced Methods and Models in Behavioral Research – 2014
A toy example – two schools, two pupils
Overall mean(0)
Two schools each with two pupils. We first calculate the means.
Overall mean= (3+2+(-1)+(-4))/4=0
32
-1
-4
exam
scor
e
School 2School 1
(taken from Rasbash)
![Page 15: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/15.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Now the variance
Overall mean(0)
32
-1
-4
exam
scor
e
School 2School 1
The total variance is the sum of the squares of the departures of the observations around the mean, divided by the sample size (4) =
(9+4+1+16)/4=7.5
![Page 16: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/16.jpg)
Advanced Methods and Models in Behavioral Research – 2014
The variance of the school means around the overall mean
32
-1
-4
exam
scor
e
School 2School 1
Overall mean(0)
2.5
-2.5
The variance of the school means around the overall mean=
(2.52+(-2.5)2)/2=6.25 (total variance was 7.5)
![Page 17: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/17.jpg)
Advanced Methods and Models in Behavioral Research – 2014
The variance of the pupils scores around their school’s mean
32
-1
-4
exam
scor
e
School 2School 1
2.5
-2.5
The variance of the pupils scores around their school’s mean=
((3-2.5)2 + (2-2.5)2 + (-1-(-2.5))2 + (-4-(-2.5))2 )/4 =1.25
![Page 18: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/18.jpg)
Advanced Methods and Models in Behavioral Research – 2014
-> So you can partition the total variance in individual level variance and school level variance
How much of the variability in pupil attainment is attributable to factors at the school and how much to factors at the pupil level?
In terms of our toy example we can now say
6.25/7.5= 82% of the total variation of pupils attainment is attributable to school level factors
1.25/7.5= 18% of the total variation of pupils attainment is attributable to pupil level factors
And this is important; we want to know how
to explain (in this example)
school attainment,and appararently thedifferences are at theschool level more than
the pupil level
![Page 19: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/19.jpg)
In a multi-level analysis,
we would like to have an estimateof the amount of variance at the
aggregate level vs at the individual level
Advanced Methods and Models in Behavioral Research – 2014
![Page 20: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/20.jpg)
Advanced Methods and Models in Behavioral Research – 2014
![Page 21: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/21.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Standard multiple regression won't doY D1 D2 D3 D4 D5 id …+4 -1 -1 0 1 0 1-3 1 1 1 0 -1 1+2 0 0 1 0 -1 20 1 0 -1 1 0 2+1 … … … … … 3+2 … … … … … 3-3 … … … … … 4+4 … … … … … 4… … … … … … …
So you can use all the data and just run a multiple regression, but then you disregard the clustering effect, which gives uncorrect confidence intervals and cannot distinguish between effects at the cluster vs at the school level
Possible solution (but not so good) You can aggregate within clusters, and then run a multiple regression on the aggregate data. Two problems: no individual level testing possible + you get much less data points.
So what can we do?
![Page 22: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/22.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Multi-level models
The standard multiple regression model assumes
... with the subscript "i" defined at the case-level.
... and the epsilons independently distributed with covariance matrix I.
With clustered data, you know these assumptions are not met.
![Page 23: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/23.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Solution 1: add dummy-variables per cluster• Try multiple regression, but with as many dummy
variables as you have clusters (minus 1)
... where, in this example, there are j+1 clusters.
IF the clustering differences are (largely) due to differences in the intercept between persons, this might work.
BUT if there are only a handful of cases per person, this necessitates a huge number of extra variables
![Page 24: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/24.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Solution 2: split your micro-level X-varsSay you have:
then create:
and add both as predictors (instead of x1)
Make sure that you understand what
is happening here,and why it is of use.
![Page 25: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/25.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Solution 3: the variance component model
In the variance component model, we split the randomness
in a "personal part" and a "rest part"
![Page 26: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/26.jpg)
• NB solution 1 en 3 gaan niet samen
Advanced Methods and Models in Behavioral Research – 2014
![Page 27: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/27.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Now: how do you do this in Stata?
<See Stata demo> [note to CS: use age and schooling as examples to split at restaurant level]
relevant commandsxtset and xtregbys <varA>: egen <meanvarB> = mean(<varB>)gen dvarB = <varB> - <meanvarB>
convenience commandstab <var>, gen() droporder desedit sum
![Page 28: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/28.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Up next
• How do we run the "Solution 1”, "Solution 2”, and “Solution 3” analysis and compare which works best? What about assumption checking?
• Random intercept we now saw, but how about random slopes?
![Page 29: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/29.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Non-response
![Page 30: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/30.jpg)
Non-response analysis
• Not all of the ones invited are going to participate
• Think about selective non-response: some (kinds of) individuals might be less likely to participate.
How might that influence the results?
sample
![Page 31: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/31.jpg)
Non-response scenarios: things to try
• Compare sample with population on several characteristics …
• … for instance by trying to ask questions to your whole sampling frame
• Compare earlier response with later response
Advanced Methods and Models in Behavioral Research – 2014
![Page 32: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/32.jpg)
What is still missing from these logit do files?
Advanced Methods and Models in Behavioral Research – 2014
![Page 33: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/33.jpg)
What is still missing from these logit do files?
Advanced Methods and Models in Behavioral Research – 2014
• Try all the available tools: outliers, transformations, interactions, dummy-variables, analyze subsets, assumption checking …
• Add more comments (in general) + add interpretation of the findings
• In the end: conclude. A “final model” or …
• … and that can also be a sequence of models
![Page 34: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/34.jpg)
Advanced Methods and Models in Behavioral Research – 2014
Check out:My logistic regression run on auto.dta
(Not easy to explain / thinking out loud /
there is more than one correct answer)
![Page 35: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/35.jpg)
This Friday latest:
• Deliver: per data file: one do-file
• Make sure that it is complete. Questions / stuck? Ask!
Advanced Methods and Models in Behavioral Research – 2014
![Page 36: Been there / done that: Stata Logistic regression (……) Conjoint analysis Coming up:](https://reader035.vdocuments.mx/reader035/viewer/2022062305/568165c1550346895dd8c9c6/html5/thumbnails/36.jpg)