04/19/23 Basics of Significance Testing 1
Chapter 15
Tests of Significance: The Basics
04/19/23 Basics of Significance Testing 2
Significance Testing
• Also called “hypothesis testing”
• Objective: to test a claim about parameter μ
• Procedure: A.State hypotheses H0 and Ha
B.Calculate test statistic
C.Convert test statistic to P-value and interpret
D.Consider significance level (optional)
Basic Biostat 9: Basics of Hypothesis Testing 3
Hypotheses• H0 (null hypothesis) claims “no difference”
• Ha (alternative hypothesis) contradicts the null
• Example: We test whether a population gained weight on average…
H0: no average weight gain in populationHa: H0 is wrong (i.e., “weight gain”)
• Next collect data quantify the extent to which the data provides evidence against H0
04/19/23 Basics of Significance Testing 4
One-Sample Test of Mean• To test a single mean, the null hypothesis is
H0: μ = μ0, where μ0 represents the “null value” (null value comes from the research question, not from data!)
• The alternative hypothesis can take these forms: Ha: μ > μ0 (one-sided to right) orHa: μ < μ0 (one-side to left) or Ha: μ ≠ μ0 (two-sided)
• For the weight gain illustrative example:H0: μ = 0 Ha: μ > 0 (one-sided) or Ha: μ ≠ μ0 (two-sided)
Note: μ0 = 0 in this example
04/19/23 Basics of Significance Testing 5
Illustrative Example: Weight Gain• Let X ≡ weight gain • X ~N(μ, σ = 1), the
value of μ unknown
• Under H0, μ = 0
• Take SRS of n = 10
• σx-bar = 1 / √(10) = 0.316
• Thus, under H0 x-bar~N(0, 0.316)
Figure: Two possible xbars when H0 true
04/19/23 Basics of Significance Testing 6
Take an SRS of size n from a Normal population. Population σ is known. Test H0: μ = μ0 with:
One-Sample z Statistic
x
μxz
nσ
μxz
00
ly Equivalent
For “weight gain” data, x-bar = 1.02, n = 10, and σ = 1
3.23
101
01.020
nσ
μxz
04/19/23 Basics of Significance Testing 7
P-value • P-value ≡ the probability the test statistic would
take a value as extreme or more extreme than observed test statistic, when H0 is true
• Smaller-and-smaller P-values → stronger-and-stronger evidence against H0
• Conventions for interpretation
P > .10 evidence against H0 not significant.05 < P ≤ .10 evidence marginally significant
.01 < P ≤ .05 evidence against H0 significant
P ≤ .01 evidence against H0 very significant
04/19/23 Basics of Significance Testing 8
P-Value
Convert z statistics to P-value :
• For Ha: μ> μ0
P = Pr(Z > zstat) = right-tail beyond zstat
• For Ha: μ< μ0 P = Pr(Z < zstat) = left tail beyond zstat
• For Ha: μμ0 P = 2 × one-tailed P-value
04/19/23 Basics of Significance Testing 9
Illustrative Example
• z statistic = 3.23
• One-sided P = P(Z > 3.23) = 1−0.9994 = 0.0006
• Highly significant evidence against H0
04/19/23 Basics of Significance Testing 10
• α ≡ threshold for “significance”• We set α• For example, if we choose α = 0.05, we require
evidence so strong that it would occur no more than 5% of the time when H0 is true
• Decision ruleP ≤ α statistically significant evidenceP > α nonsignificant evidence
• For example, if we set α = 0.01, a P-value of 0.0006 is considered significant
Significance Level
04/19/23 Basics of Significance Testing 11
Summary
04/19/23 Basics of Significance Testing 12
Illustrative Example: Two-sided test
1. Hypotheses: H0: μ = 0 against Ha: μ ≠ 0
2. Test Statistic:
3. P-value: P = 2 × Pr(Z > 3.23) = 2 × 0.0006 = 0.0012 Conclude highly significant evidence against H0
3.23
101
01.020
nσ
μxz
04/19/23 Basics of Significance Testing 13
Relation Between Tests and CIs• For two-sided tests, significant results at the α-
level μ0 will fall outside (1–α)100% CI
• When α = .05 (1–α)100% = (1–.05)100% = 95% confidence
• When α = .01, (1–α)100% = (1–.01)100% = 99% confidence
• Recall that we tested H0: μ = 0 and found a two-sided P = 0.0012. Since this is significant at α = .05, we expect “0” to fall outside that 95% confidence interval … continued …
04/19/23 Basics of Significance Testing 14
Relation Between Tests and CIs
1.64 to40.0
62.002.110
196.102.1
n
σzx
Recall: xbar = 1.02, n = 10, σ = 1. Therefore, a 95% CI for μ =
Since 0 falls outside this 95% CI the test of H0: μ = 0 is significant at α = .05
04/19/23 Basics of Significance Testing 15
Example II: Job Satisfaction
The null hypothesis “no average difference” in the population of workers. The alternative hypothesis is “there is an average difference in scores” in the population.
H0: = 0 Ha: ≠ 0
This is a two-sided test because we are interested in differences in either direction.
Does the job satisfaction of assembly workers differ when their work is machine-paced rather than self-paced? A matched pairs study was performed on a sample of workers. Workers’ satisfaction was assessed in each setting. The response variable is the difference in satisfaction scores, self-paced minus machine-paced.
04/19/23 Basics of Significance Testing 16
Illustrative Example II Job satisfaction scores follow a Normal
distribution with standard deviation = 60. Data from 18 workers gives a sample mean
difference score of 17. Test H0: µ = 0 against Ha: µ ≠ 0 with
1.20
1860
0170
nσ
μxz
04/19/23 Basics of Significance Testing 17
Illustrative Example II
• Two-sided P-value = Pr(Z < -1.20 or Z > 1.20) = 2 × Pr (Z > 1.20)= (2)(0.1151) = 0.2302
• Conclude: 0.2302 chance we would see results this extreme when H0 is true evidence against H0 not strong (not significant)
04/19/23 Basics of Significance Testing 18
Example II: Conf Interval Method
This 90% CI includes 0. Therefore, it is plausible that the true value of is 0 H0: µ = 0 cannot be rejected at α = 0.10.
40.26 to 6.26
23.261718
601.64517
n
σzx
Studying Job Satisfaction
A 90% CI for μ is