LECTURE 3EPSY 642FALL 2009

CODING STUDIESDependent variable(s)

Construct(s) representedMeasure name and related characteristicsEffect size and associated calculations

Independent variablesPopulationSampleDesignPotential Mediators and ModeratorsBias mechanisms and threats to validity

CODING STUDIES- Dependent Variables

Construct name(s): eg. Receptive or Expressive Vocabulary

Measurement name: Willson EV TestRaw score summary data (mean, SD for each group or

summary statistics and standard errors for dep. var):Exp Mean= 22 Exp SD = 5 n=100, Con Mean = 19 Con

SD = 4, n=100Effect size (mean difference or correlation)

e = (22-19)/(20.5)Effect size transformation used (if any) for mean

differences: t-test transform ( e = t (1/n1 + 1/n2)½ ), F-statistic transform (F)½ = t for df = 1, 198 probability transform to t-statistic: t(198) = [probt(.02)] point-biserial transform to t-statistic, regression coefficient t-statistic

Effect size transformations used (if any) for correlations: t-statistic to correlation: r2 = t2 / (t2+ df) Regression coefficient t-statistic to correlation

CODING STUDIES- Independent variablesPopulation(s): what is the intended

population, what characterizes it?Gender? Ethnicity? Age? Physical characteristics, Social characteristics, Psychosocial characteristics? Cognitive characteristics?

Sample: population characteristics in Exp, Control sampleseg. % female, % African-American, % Hispanic, mean IQ, median SES, etc.

CODING STUDIES- Independent variablesDesign (mean difference studies):1.Random assignment, quasi-experimental, or

nonrandom groups2.Treatment conditions: treatment variables of

importance (eg. duration, intensity, massed or distributed etc.); control conditions same

3.Treatment givers: experience and background characteristics: teachers, aides, parents

4.Environmental conditions (eg. classroom, after-school location, library)

CODING STUDIES- Independent variablesDesign (mean difference studies)5. Time characteristics (when during the year,

year of occurrence) 6. Internal validity threats: maturation, testing, instrumentation, regression, history, selection

CODING STUDIES- Independent variablesMediators and ModeratorsMediators are indirect effects that explain part

or all of the relationship between hypothesized treatment and effect:

(T) eM

In meta analysis we establish that the effect of T on the outcome is nonzero, then if M is significantly related to the effect e. We do not routinely test if T predicts M

CODING STUDIES- Independent variablesMediators and ModeratorsModerators are variable for which the

relationship changes from one moderator value to the next:

(T) e for M=1(T) e for M=2

In meta analysis we establish that the effect of T on the outcome is nonzero, then if M is significantly related to the effect e. We do not routinely test if T predicts M

.7

.3

Coding Studies- Bias MechanismsResearcher potential bias- membership in publishing

cohort/groupResearcher orientation- theoretical stance or

backgroundType of publication:

Refereed vs. book chapter vs. dissertation vs. project report: do not assume refereed articles are necessarily superior in design or analysis- Mary Lee Smith’s study of gender bias in psychotherapy indicated publication bias against mixed gender research showing no effects by refereed journals with lower quality designs than non-refereed works

Year of publication- have changing definitions affected effects? Eg. Science interest vs. attitude- terms used interchangeably in 1940s-1950s; shift to attitude in 1960s

Journal of publication- do certain journals only accept particular methods, approaches, theoretical stances?