using observation-oriented modeling to examine daily patterns and predictors of post- traumatic...
TRANSCRIPT
USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND
PREDICTORS OF POST-TRAUMATIC STRESS SYMPTOMATOLOGY
IN A SAMPLE OF FEMALE RAPE VICTIMS
Amy M. Cohn, James Grice, Brett Hagman, and Liz Schlimgen
November 17, 2012ABCT
Aim of Presentation
Test a mediation model with daily diary data
Compare multi-level modeling (MM) to Observation Orientated Modeling (OOM)
Daily post-traumatic
stress symptoms
Daily negative
affect
Daily alcohol involvement
Assumptions of MM have limitations
Homogeneity between individuals Within-person fluctuations in behavior represented
as aggregated, over-time association (slope) Linear monotonic changes in behavior over time
Associations between variables are additive, not (necessarily) dynamic
Random sampling Normal population distribution (for differences)
Abstract population parameters that have little (or no) empirical basis
The population is completely theoretical
What’s wrong with the p-value? Not a new argument Relies on population statistics that may
not represent the data The sample is different, the way you collect
the data is different, the questions you ask are different….
Creates a “false belief” in the validity and generalizability of findings Many study results cannot be replicated
Observation Orientated Modeling
“Why is it that the patterns of phenomena are the way they are?” (Harre, 1986)
“Fundamentally incompatible with prevailing research tradition in Psychology”(Grice, 2012)
OOM Incompatible with MM
Variable-based approach (such as MM) is linear, causal, and based on aggregate statistics such as betas and variances
OOM approach is integrative and focused at the level of the individual
Independent variable
Dependent variable
OOM
Non-parametric, idiographic Examines qualitative pattern in the data Rooted in Aristotle’s notion that most things in
nature are not produced by people The researcher does not control everything in a
study Eschews null hypothesis significance testing
(NHST) Results based on probabilities found within the
data, not comparison to population distribution Variables not described in a cause-effect format
OOM describes how the effect conforms to the cause
OOM
To Repeat……. EFFECTS SHOULD CONFORM TO THEIR
CAUSES What the @#*&$?
We do not always know why participants do what they do
Effects are never truly “causal” Unmeasured pieces of “error” or “garbage” in
the data collection process With OOM, patterns of observations reveal
what are in the data – The EFFECTS
Study 1 Hypotheses
NA will be greater on days characterized by greater PTSD
Craving and consumption will be greater on days characterized by more intense PTSD and NA
Daily PTSD
symptoms
Daily NAa
Alcohol Involvem
ent
b
c (c’)
Sample characteristics (n = 54)
Characteristic Statistic
Age 26 (SD = 9.08)
Some college or post high school education
70% (n = 38)
Employed (full or part-time) 25% ( n = 15)
Single 60% (n = 34)
Caucasian 70% ( n = 38)
Income (Median) $9,000 (SD = $20,032)
54 untreated female rape victims who completed at least one day of daily interactive voice response (IVR) monitoring
IVR Assessment
1x a day in the evening (6pm to 12am) Alcohol use, negative affect intensity,
craving intensity, and PTSD symptoms (presence/absence)
Since previous phone call 93% compliance rate
13/14 calls were completed
HLM Analysis DV’s = Number of drinks consumed and
intensity of craving (850 observations) Controlled for day of week Poisson distribution with log link function
for drinking Examined relationship of one variable
EACH DAY to the outcome variable ON THAT SAME DAY
Figure 1. Mediation of NA on the PTSD-alcohol link.
Daily NA
Daily PTSD symptoms
Number of standard
drinks/day
0.13*** 0.42***
-0.14
*** p < .001
(-0.02)
In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)
Figure 2. Mediation of NA of the PTSD-craving link.
Daily NA
Daily PTSD symptoms
Daily craving intensity
0.13*** 0.39***
-0.10
Note. Covariates included day of the week, baseline PTSD symptom severity, baseline alcohol use. *** p < .001
(-0.12)
In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)
OOM Analysis: Mediation Steps
Daily PTSD symptoms
Daily NA
Number of standard
drinks/day
Step 1: Because the effect conforms to the cause, we first examine the probability that number of standard drinks consumed each day conforms to daily ratings of NA intensity
OOM Results
Accuracy rate: % observations correctly classified out of total number of observations Missing data is not a problem
Randomization test Out of1000 trials of randomized versions of the
same observations, what number of instances do we obtain a result high or higher than percent correct classification?
Binomial p-value or chance value should be small (less than .01) Indicates pattern is unique
Results for individual and group-level patterns
Perfect Ordinal Matches for 14 Occasions
Proportion of Matches = 1.00; Binomial p-value = .00012
Weak Ordinal Matches for 14 Occasions
Proportion of Matches = .15; p-value = .99
Overall Results (n = 54 women) :
Number of Matches : 123 Number of Observations : 399 Proportion of Matches : 0.31
Randomization Results :
Observed Proportion of Matches : 0.31
Number of Randomized Trials : 5000.00 Minimum Random Proportion of Matches : 0.24 Maximum Random Proportion of Matches : 0.36 Values >= Observed Proportion : 1758.00 Matching c-value : 0.35
Proportion of matches is unimpressive at .31
C-value of the Randomization Test indicates that .31 is not an unusual aggregate outcome compared to randomized versions of the same observations
Aggregate Results for all 54 Women
Aggregate Results for all 54 Women 1.Proportion of Matches > .50 for only 9
women(5 of these women had 7 or fewer data points)
2.Fourteen women (26%) showed no variability in their drinking across the 14 days
3.An additional 6 women drank on only one day
Conclusions
Women showing no variability in drinking and those who did not drink across 14 days are “swept” into HLM aggregates Should this disturb us?
OOM recognizes women with no variability in their drinking Since OOM not based on means and
variances, impact of these women does not adversely effect the overall percent matches
Conclusions
OOM “effect sizes” are proportions of matches that are readily interpretable and linkable to individual women No need for interpretations- such as Cohen’s effect
sizes Idealized p-values are primary in HLM, even over
effect sizes Even if effect is small, if p < .05 we say “YES”!
Proportions of matches consistent with causal hypotheses are primary in OOM Distribution free p-values (from binomial and
randomization tests) are secondary
Erroneously enticed to posit a mediation mechanism that operates successfully for every woman with HLM OOM treats the women and their individual observations as primary Does not rely on p-values, means, or
variances estimated from a theoretical population
OOM develops integrated models More accurately explains patterns of
observations
Summary
Acknowledgments
Participants who dedicated their time and effort
Research assistants: Jessica Mitchell, Stephanie Bramm, Sarah Ehlke, Ruschelle Leone, Joanne Wang
Grants: NIDA P30DA028807; USF 582000 / MHBCSG
Thank you!
Questions?
Dr. James GriceDepartment of PsychologyOklahoma State University
Stillwater, OK [email protected]
Deep Structure Transformation
M F
0 10 11 01 0
0 1 2 3 4 5 6 7 8 9 10
0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 1 00 0 1 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 1 00 0 1 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0
0 10 11 01 0
0 10 11 01 0
ConformedEffectObservations
EffectObservations
CauseObservations
EffectObservations
3. Accuracy is our central judgment (not statistical significance) and shows how many observations were correctly classified by the algorithm, or how many observations match the pattern.
1. Observations are transformed into their “deep structure”
2. Rotate deep structure effect observations into “conformity” with deep structure observations