power analysis (1) slides available via
TRANSCRIPT
Power Analysis (1)
slides available via http://www.kuleuven.ac.be/psystat/
,36 and 9XX N n
example:
violent movies, aggression difference scores X
9 -6 3 8 4 9 -4 10 12
ℋ0: µX=0 vs. ℋ1: µX 0
optional: =.05
•derivation of sampling distribution of statistic under
assumption that ℋ0 is true: ,4XX N
•calculate value of statistic for observed data:
5x
Procedure hypothesis tester:
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
X
X•choice of statistic:
0,4X N
• compare p with
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
X
x
p
0,4X N 2p P X x with .012p
Ronald A. Fisher
Jerzy Neyman
Egon Pearson
• Decision:
accept ℋ0 if p > .05, reject ℋ0 if p .05
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
X
x
p
0
0,05
0,1
0,15
0,2
0,25
X
-3.92 3.92
reject accept reject
• Decision rule:
• Quality of decision rule?
truth
ℋ0 true ℋ0 false
reject ℋ0 wrong correct
decision Type I
accept ℋ0 correct wrong
Type II
true sampling distribution (µX=1)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
X
Truth: ℋ0 is false
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 X
true sampling distribution (µX=1)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
P (Type II-error)
3.92 3.92P Y with 1,4Y N
3.92 1 1 3.92 12 2 2
YP
with 1,4Y N
2.46 1.46P Z
2.46 .007P Z
1.46 .928P Z
P (Type II-error)
with
0,1Z N
.928 .007 .921
Power = P (ℋ0 rightly rejected) = 1 - .921 = .079
power very low!
true X=1 under ℋ0: X=0
1"small"
6X
X
P (Type II-error) = .68 Power = .32
true sampling distribution (µX=3)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
0
0,05
0,1
0,15
0,2
0,25
reject accept reject X
true sampling distribution (µX=0)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
X
Truth: ℋ0 is true
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
true sampling distribution (µX=0)
decision rule hypothesis tester
P(Type I-error)=.05
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
how to meet .20 (.80) criterion?
Option 1: ensure that true µX-value deviates further from µX-value under ℋ0
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 X
true sampling distribution (µX=1)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
P(Type II-error) = .92
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
Option 1: ensure that true µX-value deviates further from µX-value under ℋ0
true sampling distribution (µX=3)
decision rule hypothesis tester
P(Type II-error) = .68
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
reject accept reject XP (Type II-error) = .68
true sampling distribution (µX=3)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
=.05
-3.92 3.92
Option 2: increase
P (Type II-error) = .56
=.10
0
0,05
0,1
0,15
0,2
0,25
reject accept reject X
true sampling distribution (µX=3)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
-3.30 3.30
Option 2: increase
Option 3: increase sample size n=9
(=.05)
0
0,05
0,1
0,15
0,2
0,25
XP (Type II-error) = .68
true sampling distribution (µX=3)
decision rule hypothesis tester
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
-3.92 3.92
reject accept reject
[ 3,4 ]X N
[ 0,4 ]X N
0
0,05
0,1
0,15
0,2
0,25
0,3
X
Option 3: increase sample size n=18
(=.05)
P (Type II-error) = .44
true sampling distribution (µX=3)
decision rule hypothesis tester
[ 3,2 ]X N
[ 0,2 ]X N
0
0,05
0,1
0,15
0,2
0,25
0,3
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
X
reject accept reject-2.77 2.77
Option 4: choice of suitable test statistic
four elements are inherently associated:
(1) size of true effect
(2)
(3) power of chosen test statistic
(4) n (sample size)
0
0,05
0,1
0,15
0,2
0,25
reject accept rejectX-3.92 3.92
0
0,05
0,1
0,15
0,2
0,25
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6X
true sampling distribution (µX=3)
P(Type II-error) = .68
decision rule hypothesis tester
[ 3,4 ]X N
[ 0,4 ]X N