chapter 4 hypothesis testing, power, and control: a review of the basics

Chapter 4Chapter 4Hypothesis Testing, Hypothesis Testing,

Power, and Control: A Power, and Control: A Review of the BasicsReview of the Basics

From Question to From Question to HypothesisHypothesis

Finding the TRUTH starts with asking a question Finding the TRUTH starts with asking a question that comes fromthat comes from– CuriosityCuriosity– NecessityNecessity– Past ResearchPast Research

As scientists we PREDICT the answer fromAs scientists we PREDICT the answer from– TheoryTheory– Past Research Past Research – Common SenseCommon Sense

That prediction is EDUCATED not randomThat prediction is EDUCATED not random– An educated prediction is a HYPOTHESISAn educated prediction is a HYPOTHESIS

To ANSWER the question we TEST the To ANSWER the question we TEST the HYPOTHESISHYPOTHESIS

Conceptual hypothesesConceptual hypotheses– State expected relationships among State expected relationships among

concepts.concepts. Research hypothesesResearch hypotheses

– Concepts are operationalized so that they Concepts are operationalized so that they are measurable.are measurable.

Statistical hypothesesStatistical hypotheses– State the expected relationship between State the expected relationship between

or among summary values of populations, or among summary values of populations, called parameters.called parameters. Null hypothesis (HNull hypothesis (H00) ) Alternative hypothesis (HAlternative hypothesis (H11))

Three levels of hypothesesThree levels of hypotheses

ExampleExample

Question – What is the role of Question – What is the role of neurotransmitters in memory?neurotransmitters in memory?

Conceptual – Increasing certain Conceptual – Increasing certain neurotransmitter will increase memoryneurotransmitter will increase memory

Research – Smoking 1 crack rock before Research – Smoking 1 crack rock before testing will increase performance on a testing will increase performance on a standard test of memory compared to standard test of memory compared to placebo controlplacebo control

Statistical – HStatistical – HOO: M: MTT = M = MCC H HAA: M: MTT >M >MCC

Testing the null hypothesisTesting the null hypothesis Null hypothesisNull hypothesis

– The hypothesis being statistically tested The hypothesis being statistically tested when you use inferential statistics.when you use inferential statistics.

– The researcher hopes to show that the null The researcher hopes to show that the null is not likely to be true (i.e.. hopes to nullify is not likely to be true (i.e.. hopes to nullify it).it).

Alternative hypothesisAlternative hypothesis– The hypothesis the researcher postulated at The hypothesis the researcher postulated at

the outset of the study.the outset of the study.– If the researcher can show that the null is If the researcher can show that the null is

not supported by the data, then he or she is not supported by the data, then he or she is able to accept the alternative hypothesis.able to accept the alternative hypothesis.

Testing the null hypothesisTesting the null hypothesis

Steps in testing a research Steps in testing a research hypothesis:hypothesis:

1.1. State the null and the alternative.State the null and the alternative.

2.2. Collect the data and conduct the Collect the data and conduct the appropriate statistical analysis.appropriate statistical analysis.

3.3. Reject the null and accept the Reject the null and accept the alternative or fail to reject the null.alternative or fail to reject the null.

4.4. State your inferential conclusion.State your inferential conclusion.

Statistical significanceStatistical significance

Statistical differenceStatistical difference– The probability that the groups are the The probability that the groups are the

same is very low.same is very low. Significance levels (Significance levels (αα))

– Alpha Alpha ((αα) is the level of significance ) is the level of significance chosen by the researcher to evaluate chosen by the researcher to evaluate the null hypothesis.the null hypothesis.

– 5% or 1%5% or 1%

Inferential Errors: Inferential Errors: Type I and Type IIType I and Type II

Type I ErrorType I Error– Rejecting a true null.Rejecting a true null.

– Probability is equal to alpha (Probability is equal to alpha (αα).).

Type II ErrorType II Error– Failing to reject a false null.Failing to reject a false null.

– Probability is beta (Probability is beta (ββ).).

PowerPower – – our ability to reject false nullsour ability to reject false nulls..

Inferential Errors: Inferential Errors: Type I and Type IIType I and Type II

True State of AffairsTrue State of Affairs

Null is trueNull is true Null is falseNull is false

Reject Reject the nullthe null Type I error (Type I error (αα))

Correct Correct inference inference (power)(power)

Fail to Fail to reject reject the nullthe null

Correct Correct inferenceinference

Type II error Type II error ((ββ))O

ur d

ecis

ion

Why Power is Important Why Power is Important

A powerful test of the null is more A powerful test of the null is more likely to lead us to reject false nulls likely to lead us to reject false nulls than a less powerful test.than a less powerful test.

Powerful tests are more sensitive than Powerful tests are more sensitive than less powerful tests to differences less powerful tests to differences between the actual outcome (what you between the actual outcome (what you found) and the expected outcome (null found) and the expected outcome (null hypothesis).hypothesis).

Power, or the probability of rejecting a Power, or the probability of rejecting a false null, is 1 – false null, is 1 – ββ..

Power and How to Increase Power and How to Increase itit

How one measures variablesHow one measures variables– Interval or ratio scales are betterInterval or ratio scales are better

In testing the effects of alcohol intoxication In testing the effects of alcohol intoxication on aggression…on aggression…

– Intoxication – BAC better than # of drinksIntoxication – BAC better than # of drinks– Aggression – Level of shock (1-10) as opposed to Aggression – Level of shock (1-10) as opposed to

shock or no shockshock or no shock


Use more powerful statistical analysesUse more powerful statistical analyses Parametric vs. NonparametricParametric vs. Nonparametric

ANOVA vs. Chi-SquareANOVA vs. Chi-Square


Use designs that provide good control over Use designs that provide good control over extraneous variables.extraneous variables. Remove unintended variationRemove unintended variation

Experimental vs. Correlational DesignsExperimental vs. Correlational Designs Laboratory vs. FieldLaboratory vs. Field


Restrict your sample to a specific group of Restrict your sample to a specific group of individuals.individuals. Use selection procedures to reduce nuisance Use selection procedures to reduce nuisance

variablesvariables


Increase your sample size Increase your sample size reduces error reduces error variancevariance


Maximize treatment manipulationMaximize treatment manipulation PrecisionPrecision SeparationSeparation

Effect sizeEffect size – a measure of the – a measure of the strength of the relationship strength of the relationship between/among variables.between/among variables.

Effect size helps us determine if Effect size helps us determine if differences are not only statistically differences are not only statistically significant, but also whether they are significant, but also whether they are important.important.

Powerful tests should be considered Powerful tests should be considered to be tests that detect large effects.to be tests that detect large effects.

Effect sizeEffect size

Effect sizeEffect size

Ways to calculate effect size:Ways to calculate effect size:– Cohen’s Cohen’s d – use with t-tests.d – use with t-tests.– Coefficient of determination (Coefficient of determination (rr22) – ) – use use

with correlations.with correlations.– eta-squared (eta-squared (ηη22) – ) – use with ANOVAs.use with ANOVAs.– Cramer’s v – Cramer’s v – use with Chi-square use with Chi-square

analyses.analyses.

Power and the role of Power and the role of replication in researchreplication in research

Power increases when we replicate Power increases when we replicate findings in a new study with different findings in a new study with different participants in a different setting.participants in a different setting.

External and internal External and internal validityvalidity

External validityExternal validity– When the findings of a study can be When the findings of a study can be

generalized to other populations and generalized to other populations and settings.settings.

Internal validityInternal validity– Refers to the validity of the measures Refers to the validity of the measures

within the study.within the study.– The internal validity of an experiment is The internal validity of an experiment is

directly related to the researcher’s control directly related to the researcher’s control of extraneous variables.of extraneous variables.

Confounding and Confounding and extraneous variablesextraneous variables

Extraneous variableExtraneous variable– A variable that may affect the outcome of a A variable that may affect the outcome of a

study but was not manipulated by the study but was not manipulated by the researcher.researcher.

Confounding variableConfounding variable– A variable that is systematically related to A variable that is systematically related to

the independent and dependent variable.the independent and dependent variable. Spurious effectSpurious effect

– An outcome that was influenced not by the An outcome that was influenced not by the independent variable itself but rather by a independent variable itself but rather by a variable that was confounded with the variable that was confounded with the independent variable.independent variable.

Confounding and Confounding and extraneous variablesextraneous variables

Controlled variableControlled variable

– A variable that the researcher takes into A variable that the researcher takes into account when designing the research account when designing the research study or experiment.study or experiment.

Nuisance variablesNuisance variables

– Variables that contribute variance to our Variables that contribute variance to our dependent measures and cloud the dependent measures and cloud the results.results.

Controlling extraneous Controlling extraneous variablesvariables

EliminationElimination– Get rid of the extraneous variables Get rid of the extraneous variables

completely (e.g.. by conducting research in completely (e.g.. by conducting research in a lab).a lab).

ConstancyConstancy– Keep the various parts of the experiment Keep the various parts of the experiment

constant (e.g.. instructions, measuring constant (e.g.. instructions, measuring instruments, questions).instruments, questions).

Secondary variable as an IVSecondary variable as an IV– Make variables other than the primary IV Make variables other than the primary IV

secondary variables to study (e.g.. gender).secondary variables to study (e.g.. gender).

Controlling extraneous Controlling extraneous variablesvariables

Randomization: Random assignment of Randomization: Random assignment of participants to groupsparticipants to groups– Randomly assigning participants to each of Randomly assigning participants to each of

the treatment conditions so that we can the treatment conditions so that we can assume the groups are initially equivalent.assume the groups are initially equivalent.

Repeated measuresRepeated measures– Use the same participants in all conditions.Use the same participants in all conditions.

Statistical controlStatistical control– Treat the extraneous variable as a Treat the extraneous variable as a

covariate and use statistical procedures to covariate and use statistical procedures to remove it from the analysis.remove it from the analysis.

chapter 4 hypothesis testing, power, and control: a review of the basics

Documents