Chapter 8: Hypothesis Testing and Inferential Statistics

• What are inferential statistics, and how are they used to test a research hypothesis?• What is the null hypothesis?• What is alpha?• What is the p-value, used in most hypothesis test?• What are Type 1 and Type 2 errors, and what is the relationships between them• What is beta, and how does beta relate to the power of a statistical test?• What is the effect size statistic, and how is it used?

Sampling Distribution

Sampling Distribution

The distribution of all of thepossible values of a statistic.

Example. To examine your friend’s ESP aptitude, you ask your friend to guess on ten coin flips (heads and tails)

)!1(!

!

−rrn

nCr =

10C5 = )12345(12345

12345678910

××××××××××××××××××

= 252

210 = 1024

Binomial distribution

nCrrnr −

⎟⎠⎞

⎜⎝⎛

⎟⎠⎞

⎜⎝⎛

21

21

10C5

5105

2

1

2

1−

⎟⎠⎞

⎜⎝⎛

⎟⎠⎞

⎜⎝⎛

=10

2

1252 ⎟

⎠⎞

⎜⎝⎛×

= ...0009765.0252×

= 24.6

See Figure 8.2 on page 132…...

Task 1. Calculate the other possibilities and their distribution.

The null hypothesis

The assumption in which the variableA is not statistically differ from the variable B.

Example 1. The coin guessing experimentH0 is that the probability of a correct guess is chance level ( = .5)

Example 2. A correlational designH0 is that there is no correlation between the two measured variables(r = 0). (the correlation between SAT and College GPA.)

H0

Example 3. An experimental designH0 is that the mean score on the dependent variable is the same in allexperimental group (helping behavior between men and women)

Reject null hypothesis and fail to reject null hypothesis

Reject null hypothesis = There is a significant statistical difference between A and B.

Example 1. Observed data is statistically differ from the chance level

Fail to reject null hypothesis = there is no significant statistical difference between A and B

Example 1. Observed datais not differ from the chancelevel.

Example 2. Variable A is nocorrelation to Variable B

Example 2. Variable A is statisticallycorrelate with Variable B.

Testing for Statistical Significance

Significance Level (alpha = )

The level in which we are allowedto reject the null hypothesis.

Who decides the level?

The researcherBy convention, alpha is normally set to = .05Probability value (p value)

The likelihood of an observed statistic occurring on the basis of the sampling distribution.

If P value is less than alpha (p < .05) Reject null hypothesis

If P value is greater than alpha (p > .05) Fail to reject null hypothesis

StatisticallySignificant

Statisticallynonsignificant

Comparing the P-value to Alpha

Example. The coin guessing experiment(Take a look at Figure 8.2!)

P value for 10 correct guesses = .001P value for 9 and 10 correct guesses = .01 + . 001 = .011P value for 8, 9, and 10 correct guesses = .044 + .01 + .001= .055P value for 7, 8, 9, and 10 correct guesses = .117 + .044 + .01 + .001 = .172 P > .05

P > .05

P < .05P < .05

Two-sided p-value

P value for number of guesses as extreme as 10

P value for number of guesses as extreme as 9

P value for number of guesses as extreme as 8

P value for number of guesses as extreme as 7

002.2001. =×=

022.2011. =×=

11.2055. =×=

344.2172. =×=

Type 1 Error & Type 2 Error

Type 1 Error Correct Decisionprobability = Probability = 1-

Correct decision Type 2 Errorprobability = 1 - probability =

Scientist’s DecisionReject null hypothesis Fail to reject null hypothesis

Null hypothesisis true

Null hypothesisis false

Type 1 Error Type 2 Error

Cases in which you rejectnull hypothesis when it isreally true

Cases in which you fail toreject null hypothesis when it is false

= =

Statistical Significance and the Effect Size

ES =

2

)()(

21

2211

21

1 2

−+

−+−

−

∑ ∑nn

yyyy

yyn

i

n

iii

Statistical Significance = Effect Size (ES) X Sample Size

Hypothesis Testing Flow ChartDevelop research hypothesis & null hypothesis

Set alpha (usually .05)

Calculate power to determine the sample size

Collect data & calculate statistic and p

Compare p to alpha (.05)P < .05 P > .05

Reject null hypothesis Fail to reject null hypothesis