assuming that there is a relationship in this study, is ...probabilistic equivalence . random...
TRANSCRIPT
![Page 1: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/1.jpg)
Internal Validity & Research Design
Internal Validity: Assuming that there is a relationship in this study, is the relationship a causal one?
![Page 2: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/2.jpg)
![Page 3: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/3.jpg)
Problem: How to increase pet adoptions from the center? Solution: Establish a Facebook presence!
![Page 4: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/4.jpg)
![Page 5: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/5.jpg)
Results (note – this is a fictitious example): Dogs Cats TTL May adoptions (no Facebook page) 100 120 220
Facebook page created June 1
June adoptions 115 150 265
Did the additional communication from the Facebook page cause the increase in pet adoptions?
![Page 6: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/6.jpg)
How do we Establish a Cause-Effect Relationship? 3 criteria
• Temporal Precedence
• Covariation of the Cause and Effect
• No Plausible Alternative Explanations
![Page 7: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/7.jpg)
How do we Establish a Cause-Effect Relationship?
• Temporal Precedence
• Covariation of the Cause and Effect
• No Plausible Alternative Explanations
![Page 8: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/8.jpg)
Research Design R O X O R O O
Notation: O = Observations / Measures Osubscript = measure taken on that occasion X = Treatments / Programs Rows = Groups R = Random assignment to group N = Nonequivalent groups C = Assignment by cutoff Time = moves left to right
![Page 9: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/9.jpg)
![Page 10: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/10.jpg)
Types of Designs
![Page 11: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/11.jpg)
Types of Designs
![Page 12: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/12.jpg)
Single Group Threats to Internal Validity The Single Group Case
Posttest only Pretest-Posttest X O O X O
History Threat History Threat
Maturation Threat Maturation Threat
Testing Threat (primed)
Instrumentation Threat (change test)
Mortality Threat
Regression Threat (regression to the mean)
![Page 13: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/13.jpg)
Regression to the Mean a statistical phenomenon that occurs whenever you have a nonrandom sample from a population and two measures that are imperfectly correlated.
It is a statistical phenomenon. It is a group phenomenon. It happens between any two variables. It is a relative phenomenon. You can have regression up or down. The more extreme the sample group, the greater the regression to the mean. The less correlated the two variables, the greater the regression to the mean.
![Page 14: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/14.jpg)
A real example: Exam 1 Exam 2 Pop. Mean 2.36 2.24 Standard Dev. .79 .80 Correlation rExam1•Exam2 = 0.44 Exam 1 Bottom 10% Mean 1.2 1.4 Distance (Std. Devs.) 1.5 1.0 Exam 1 Top 10%
Mean 3.6 3.3 Distance (Std. Devs.) 1.6 1.3
![Page 15: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/15.jpg)
A real example: Exam 1 Exam 2 Pop. Mean 2.36 2.24 Standard Dev. .79 .80 Correlation rExam1•Exam2 = 0.44 Exam 1 Bottom 10% Mean 1.2 1.4 Distance (Std. Devs.) 1.5 1.0 Exam 1 Top 10%
Mean 3.6 3.3 Distance (Std. Devs.) 1.6 1.3
![Page 16: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/16.jpg)
How do we deal with single group threats to internal validity?
Most common: change the research design Most common change: add a control group (Of course, when you add a control group, you no longer have a single group design.)
![Page 17: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/17.jpg)
Multiple Group Threats to Internal Validity There is one multiple group threat to internal validity: that the groups were not comparable before the study.
A selection bias or selection threat: any factor other than the program that leads to posttest differences between groups.
Pretest-Posttest with Control O X O O O
Selection-History Threat
Selection-Maturation Threat
Selection-Testing Threat (primed)
Selection-Instrumentation Threat (change test)
Selection-Mortality Threat
Selection-Regression Threat (regression to the mean)
![Page 18: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/18.jpg)
Experimental Design • probably the strongest design with respect to
internal validity.
• key to the success of the experiment is in the random assignment (achieves probabilistic equivalence).
![Page 19: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/19.jpg)
Probabilistic Equivalence
![Page 20: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/20.jpg)
Random Selection
vs.
Random Assignment
Sampling Design
External Validity Internal Validity
![Page 21: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/21.jpg)
Social Interaction Threats to Internal Validity Social pressures in the research context that can lead to posttest differences
Groups Research Admin.
Diffusion or Imitation of Treatment ✔
Compensatory Rivalry ✔
Resentful Demoralization ✔
Compensatory Equalization of Treatment ✔ ✔
![Page 22: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/22.jpg)
The simplest of all experimental designs: two-group posttest-only randomized experiment.
![Page 23: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/23.jpg)
Remember this?
![Page 24: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/24.jpg)
Remember this?
![Page 25: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/25.jpg)
This is an Extremely Simple Design Only One Independent Variable (Factor):
Advertising Exposure
![Page 26: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/26.jpg)
What about a design with more than one independent variable?
Independent Variable 1 (Factor 1): Ad Color Two Levels: B&W (1) / CMYK (2) Independent Variable 2 (Factor 2): Humor in Ad Two Levels: Not Humorous (1) / Humorous (2)
Research Design Notation (Post-Test Only) = 4 Groups
![Page 27: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/27.jpg)
R X11 O B&W / Non-humorous R X12 O B&W / Humorous R X21 O CMYK / Non-humorous R X22 O CMYK / Humorous
![Page 28: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/28.jpg)
2 x 2 Factoral Design
Humor
B&W Non-humorous
X11
B&W Humorous
X12 C
olor
CMYK
Non-humorous X21
CMYK Humorous
X22
![Page 29: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/29.jpg)
Understanding Factoral Design Numbering Notation
Design Factors Levels Groups 2 x 2 2 2 / 2 4 2 x 3 2 2 / 3 6 3 x 3 2 3 / 3 9
2 x 2 x 2 3 2 / 2 / 2 8 3 x 2 x 2 3 3 / 2 / 2 12
![Page 30: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/30.jpg)
How Factoral Designs Work
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
Group 1 Mean
Group 2 Mean
Mean Across
Factor 2 Level 2
Group 3 Mean
Group 4 Mean
Mean Across
Mean Down
Mean Down
![Page 31: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/31.jpg)
No Effects (Null Outcome)
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
3
3
3
Factor 2 Level 2
3
3
3
3
3
![Page 32: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/32.jpg)
Main Effect of Factor 1
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
2
4
3
Factor 2 Level 2
2
4
3
2
4
![Page 33: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/33.jpg)
Main Effect of Factor 2
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
2
2
2
Factor 2 Level 2
4
4
4
3
3
![Page 34: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/34.jpg)
Main Effect of Each Factor
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
2
4
3
Factor 2 Level 2
4
6
5
3
5
![Page 35: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/35.jpg)
Interaction Effect (one group differs from all others)
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
3
3
3
Factor 2 Level 2
3
5
4
3
4
![Page 36: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/36.jpg)
Interaction Effect (Crossover)
Factor 1 Level 1
Factor 1 Level 2
Factor 2 Level 1
3
5
4
Factor 2 Level 2
5
3
4
4
4
![Page 37: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/37.jpg)
Adding a Control Group
R X11 O B&W / Non-humorous R X12 O B&W / Humorous R X21 O CMYK / Non-humorous R X22 O CMYK / Humorous R O Control
2 x 2 Factoral Design with Control (not a fully-crossed design)
![Page 38: Assuming that there is a relationship in this study, is ...Probabilistic Equivalence . Random Selection vs. Random Assignment Sampling Design External Validity Internal Validity](https://reader034.vdocuments.mx/reader034/viewer/2022051605/600a8b3c54a747692767532c/html5/thumbnails/38.jpg)
Incomplete Factoral Design Factor 1
Level 1 Factor 1 Level 2
Factor 2 Level 1
Group 1 Mean
Group 2 Mean
Mean Across
Factor 2 Level 2
Group 3 Mean
Group 4 Mean
Mean Across
Mean Down
Mean Down
Control Mean