psy2004 research methods psy2005 applied research methods week five
TRANSCRIPT
![Page 1: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/1.jpg)
PSY2004 Research Methods PSY2005 Applied Research
Methods
Week Five
![Page 2: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/2.jpg)
![Page 3: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/3.jpg)
TodayGeneral principleshow it works, what it tells you etc.
Next weekExtra bits and bobsassumptions, follow-on analyses, effect sizes
![Page 4: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/4.jpg)
ANalysis Of VAriance
A ‘group’ of statistical tests
Useful, hence widely used
![Page 5: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/5.jpg)
used with a variety of designs
start with independent groups & 1 independent variable.
other designs later
![Page 6: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/6.jpg)
revision[inferential stats]
trying to make inference about POPULATION on basis of SAMPLE
but sampling ‘error’ means sample not quite equal to population
![Page 7: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/7.jpg)
Two hypotheses for e.g., difference between group means in your sample
H0 - just sampling error, no difference
between population means
H1 - a difference between the population
means
Decide on the basis of probability• of getting your sample were H0 to be true
![Page 8: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/8.jpg)
if that probability is low
if such our difference between sample means would be unlikely were H0 true
if it would be a rare event
we reject H0 (and so accept H1)
low / unlikely / rare = < 0.05 (5%)
![Page 9: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/9.jpg)
what is ANOVA for?despite its name (analysis of variance)
looks at differences between means
looks at differences between groups means in sample
to make inference about differences between group means in the population
![Page 10: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/10.jpg)
but we already have the t-test
• used to compare means• e.g., PSY1016 last year: difference between males’ &
females’ mean Trait Emotional Intelligence scores
• can only compare two means at a time
• what if we have more than two groups/means?
![Page 11: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/11.jpg)
e.g.,
comparison of drug treatments for offenders
12 step programme
cognitive-behavioural motivational intervention
standard care
DV: no. of days drugs taken in a month
![Page 12: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/12.jpg)
e.g.,
comparison of coaching methods
professional coaching
peer coaching
standard tutorial
DV: self-reported goal attainment (1-5 scale)
![Page 13: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/13.jpg)
we can use the t-test here
• just lots of them
• multiple comparisons• 12-step vs cog-behavioural• 12-step vs standard care• cog-behavioural vs standard care
• bit messy / longwindedbut computer does the hard work
• far more serious potential problem …
• professional coaching vs peer• professional vs standard tutorial• peer vs standard tutorial
![Page 14: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/14.jpg)
increased chance of making a
mistake
![Page 15: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/15.jpg)
statistical inference based on probability
not certainalways a chance that we will make the wrong
decision
two types of mistaketype I – reject H0 when it is in fact true
decide there’s a difference between population means when there isn’t
[false positive]
type II – fail to reject H0 when it is in fact false[false negative]
![Page 16: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/16.jpg)
type I error[false positive]
we reject H0 when p < 0.05i.e., less than 5% chance of getting our data
(or more extreme) were H0 to be true
5% is small, but it isn’t zero
still a chance of H0 being true and getting our data
still a chance of rejecting H0 but it being true
![Page 17: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/17.jpg)
alpha (α)[criterion for rejecting H0 - typically 5%]
sets a limit on probability of making type I error
if H0 true we would only reject it 5% of the time
but multiple comparisons change the situation ….
![Page 18: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/18.jpg)
russian roulette[with six chamber revolver]
one bullet, spin the cylinder
muzzle to temple, pull trigger
one-in-six chance of blowing your brains out
![Page 19: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/19.jpg)
russian roulette
with three guns
each gun on its owna one-in-six chance of blowing your brains out
for the ‘family’ of three gunsthe probability of you getting a bullet in your brain is worryingly higher
![Page 20: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/20.jpg)
russian roulette[with twenty chamber revolver]
one-in-twenty (5%) chance of blowing your brains out
with three such gunsprobability = 1 – (.95 x .95 x .95) = .14
![Page 21: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/21.jpg)
just the same with multiple comparisons[with one obvious difference]
each individual t-testmaximum type I error rate (α) of 5%
for a ‘family’ of three such t-testserror rate = 1 – (.95 x .95 x .95) = .14
![Page 22: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/22.jpg)
controlling family-wise error
various techniques
e.g., Bonferroni correction
adjust α for each individual comparison
where k = number of comparisons
![Page 23: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/23.jpg)
for our ‘family’ of three comparisons
use adjusted α of .0167 for each t-testfamily-wise Type I error rate limited to 5%
and all is well ….
![Page 24: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/24.jpg)
… actually it isn’t.
Such ‘corrections’ come at a price
increased chance of making a type II error
failing to reject H0 when it is in fact false[false negative]
less chance of detecting an effect when there is one
aka low ‘power’ [more of this in Week 10]
![Page 25: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/25.jpg)
Moving from comparing two means to considering three has complicated matters
we seem to face either
increased type I error rateor
increased type II error rate[lower power]
![Page 26: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/26.jpg)
This [finally] is where ANOVA comes in
It can help us detect any difference
between our 3 (or more) group means
withoutincreasing type I error rateor
reducing power
![Page 27: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/27.jpg)
ANOVA is another NHST[Null Hypothesis Significance Test]
need to know what your H0 and H1 are
H0 – all the population means are the same,
any differences between sample means are simply due to sampling error
H1 – the population means are not all the same
NB one-tailed vs two-tailed doesn’t apply
![Page 28: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/28.jpg)
How does ANOVA work?
the heart of ANOVA is the F ratio
a ratio of two estimates of the population variance, both based on the sample
what’s that got to do with differences between means?Be patient.
![Page 29: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/29.jpg)
e.g.,
comparison of drug treatments for offenders
12 step programme
cognitive-behavioural motivational intervention
standard care
DV: no. of days drugs taken in a month
![Page 30: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/30.jpg)
Random data generated by SPSS[just like PSY1017 W09 labs last year]
3 samples (N=48)
all from the same population
H0 [null hypothesis]
(no difference between population means)
TRUE
![Page 31: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/31.jpg)
the sample means are not all the same
due to ‘sampling error’
they vary around the overall mean of 6.61
between-group variability
![Page 32: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/32.jpg)
the standard deviations show how varied individual scores are for each group
within-group variability
![Page 33: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/33.jpg)
both
between-groups variabilityand
within-groups variability
can be used to estimate the population variance
Don’t worry [for now] how this is done
![Page 34: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/34.jpg)
estimate of population variance based on
between-groups variability(differences of groups means around overall mean)
= 3.07
estimate of population variance based on
within-groups variability(how varied individual scores are for each group)
= 2.02
![Page 35: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/35.jpg)
F ratio = between-groups estimatewithin-groups estimate
= 3.072.02
= 1.52
estimates unlikely to be exactly the same, but similar, and so F ratio will be approximately = 1 WHEN H0 IS TRUE
![Page 36: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/36.jpg)
Random data generated by SPSS3 samples (N=48), all as before but
+1 to all ‘Standard Care’ scores
H0 [null hypothesis]
(no difference between population means)
FALSE
![Page 37: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/37.jpg)
previous exampleH0 true
newexampleH0 false
within-groups variability UNCHANGED
only between-groups variability affected
![Page 38: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/38.jpg)
estimate of population variance based on
between-groups variability(differences of groups means around overall mean)
= 3.07[previous], = 9.79 [new]
estimate of population variance based on
within-groups variability(how varied individual scores are for each group)
= 2.02[previous], = 2.02 [new]
![Page 39: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/39.jpg)
F ratio = between-groups estimatewithin-groups estimate
= 3.07[previous], = 9.79[new]
2.02 2.02
= 1.52[previous], =4.85[new]
F ratio will tend to be larger WHEN H0 IS FALSE
as only between-groups estimate affected by differences between means.
![Page 40: PSY2004 Research Methods PSY2005 Applied Research Methods Week Five](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56649f175503460f94c2e64a/html5/thumbnails/40.jpg)
ANOVA is another NHSTprobability of getting F-ratio (or more extreme) if H0 true
If p < 0.05, reject H0
H0 – all the population means are the same
and so accept
H1 – the population means are not all the same
NB this doesn’t say anything about which means are different to which other ones