experimental designs
TRANSCRIPT
Experimental Research Designs
Experimental Design
AdvantagesBest establishes cause-and-effect
relationships Disadvantages
Artificiality of experimentsFeasibilityUnethical
Causality
Temporal precedence Covariation between IV and DV Eliminate alternative explanations
Types of Experimental Designs
Simple True Experimental Complex True Experimental Quasi-Experimental
Types of Experimental Designs
Simple True Experimental Complex True Experimental Quasi-Experimental
Simple True Experimental
Characteristics Types Variations
Characteristics of True Designs
Manipulation (treatment) Randomization Control group
Characteristics of simple true designs One IV with 2 levels (T, C) One DV
Types
Randomized posttest control group design Randomized pretest-posttest control group
design
Randomized posttest control group design
R T Post
R C Post
Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Advantages & Disadvantages
Advantages of pretest designEquivalency of groupsCan measure extent of changeDetermine inclusionAssess reasons for and effects of mortality
Disadvantages of pretest designTime-consumingSensitization to pre-test
Solomon four-group design
R Pre T Post
R Pre C PostR T PostR C Post
Variations
Independent groups (between groups) Repeated measures (within groups)
Repeated Measures Design
Advantages:Fewer subjects needed (less costly)Sensitive to finding statistical differences
Disadvantages:Order effect (practice, fatigue, carry-over)
Dealing with Order Effects
Counterbalancingn!Latin squares
Latin Squares
1 2 3 4
Row 1A
(60)
B
(0)
D
(120)
C
(180)
Row 2B
(0)
C
(180)
A
(60)
D
(120)
Row 3C
(180)
D
(120)
B
(0)
A
(60)
Row 4D
(120)
A
(60)
C
(180)
B
(0)
Dealing with Order Effects
Counterbalancingn!Latin squaresRandomized blocks
Time interval between treatments
Variations
Independent groups (between) vs. repeated measures (within) designs
Consider external validity when deciding which design to use.
Types of Experimental Designs
Simple True Experimental Complex True Experimental Quasi-Experimental Pre-Experimental
Characteristics of True Designs
Manipulation (treatment) Randomization Control group
Characteristics of simple true designs One IV with 2 levels (T, C) One DV
Complex True Experimental
Randomized matched control group design
Increased levels of IV Factorial design Multiple DVs
Complex True Experimental
Randomized matched control group design
Increased levels of IV Factorial design Multiple DVs
Randomized matched control group design
M R T Post
M R C Post
• Used in small samples cost in time & money
Complex True Experimental
Randomized matched control group design
Increased levels of IV Factorial design Multiple DVs
Increased Levels of IV
Provides more complete information about the relationship between the IV & DV
Detects curvilinear relationships Examines effects of multiple treatments
Per
form
ance
leve
l (%
com
plet
e)
Amount of reward promised ($)
DV
IV
$0 $1 $2 $3Reward Amount
Increased Levels of IVP
erfo
rman
ce le
vel (
% c
ompl
ete)
Amount of reward promised ($)
DV
IV
Complex True Experimental
Randomized matched control group design
Increased levels of IV Factorial design Multiple DVs
Factorial Design
>1 IV (factor) Simultaneously determine effects of 2 or
more factors on the DV (real world) Between Factor vs. Within Factor ID’d by # of factors and levels of factors
2X2
Do differing exercise regimens (hi, med, lo intensity) have the same effect on men as they do on women?
3 X 2 (Exercise Regimen X Gender)2 factorsExercise Regimen – 3 levelsGender – 2 levelsBetween factorsDV?Experimental IVs or Participant IVs?
Male Female
High
Medium
LowExe
rcis
e In
tens
ity
Gender
Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo.
2 X 4 (Gender X Time)? factorsTime – 4 levelsGender – 2 levelsBetween or within factors?DV?Experimental IVs or Participant IVs?
0 mo. 2 mo. 4 mo. 6 mo.
Male
FemaleGen
der
Time
0 mo. 2 mo. 4 mo. 6 mo.
Male 50 70 90 130 85
Female 30 60 75 90 64
40 65 83 110 74
Gen
der
Time
Cell means, Margin meansMain Effects, Interactions
Cell means
Margin means Grand mean
0 mo. 2 mo. 4 mo. 6 mo.
Male 50 70 90 130 85
Female 30 60 75 90 64
40 65 83 110 74
Gen
der
Time
Parallel lines indicate no interaction.
Interaction of Exercise Intensity and Gender
35
40
45
50
55
60
65
70
High Medium Low
Exercise Intensity
VO
2 M
ax (
ml/k
g/m
in)
Male
Female
Is there
a main
effect?
Interaction of Exercise Intensity and Gender
35
40
45
50
55
60
65
70
Male Female
Gender
VO
2 M
ax
(m
l/kg
/min
)
High
Medium
Low
Is there
a main
effect?
Non-parallel lines indicate an interaction.
Interaction of Gender and Time
0
20
40
60
80
100
120
140
0 mo. 2 mo. 4 mo. 6 mo.
Time
We
igh
t L
ifte
d (
lbs
.)
Male
Female
Is there
a main
effect?
Interaction Between Gender and Time
0
20
40
60
80
100
120
140
Male Female
Gender
We
igh
t L
ifte
d (
lbs
.)
0 mo.
2 mo.
4 mo.
6 mo.
Is there
a main
effect?
Interpretation
Always interpret the interaction first (graphical)
If no significant interaction, interpret main effects
Advantages of factorial designs:Greater protection against Type I errorMore efficient Can examine the interaction
Disadvantages: subject # for between factor designs
Consider external validity when deciding which design to use.
IV A: Exposure to Violence – violent vs. nonviolent videoIV B: Gender – male vs. femaleDV: # ads recalled (0-8)
1 2
1 1 5 3
2 9 5 7
5 5
A
B
1
5
9
1 2
A
B1
B2
IV A: Exposure to Violence – violent vs. nonviolent videoIV B: Gender – male vs. femaleDV: # ads recalled (0-8)
1
5
9
1 2
A
B1
B2
A: YesB: NoAxB: Yes
Complex True Experimental
Randomized matched control group design
Increased levels of IV Factorial design Multiple DVs
0 mo. 2 mo. 4 mo. 6 mo.
Male 50 70 90 130 85
Female 30 60 75 90 64
40 65 83 110 74
Gen
der
Time
Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo.
Types of Experimental Designs
Simple True Experimental Complex True Experimental Quasi-Experimental
Characteristics of True Designs
Manipulation (treatment) Randomization Control group
Less control More real-world Program evaluation
Randomized posttest control group design
R T Post
R C Post
Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Quasi-experimental Designs
One group posttest-only design One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Quasi-experimental Designs
One group posttest-only design One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Randomized posttest control group design
R T Post
R C Post
One group posttest-only design (One shot study)
T Post
No control of IV threatsUse?
Quasi-experimental Designs
One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
One group pretest-posttest design
Pre T Post
•History•Maturation•Testing•Instrument decay•Regression
Use control group
Quasi-experimental Designs
One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Randomized posttest control group design
R T Post
R C Post
Non-equivalent control group design
(Static group comparison design)T Post
C Post
•Selection bias
Quasi-experimental Designs
One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Randomized pretest-posttest control group design
R Pre T Post
R Pre C Post
Non-equivalent control group pretest-posttest design
Pre T Post
Pre C Post
•Can check selection bias
Quasi-experimental Designs
One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Time series
Pre Pre Pre Pre T Post Post Post Post
Quasi-experimental Designs
One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Quasi-experimental Designs
One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest
design Time series Single subject designs (Case study) Developmental designs
Developmental Research Designs
Longitudinal
Powerful (within subject)
Time consuming Attrition Testing effect
Cross Sectional
Less time consuming Cohorts problem
Choosing a Research Design
Best addresses the problem Ethics Cost in time and money Validity (internal & external)