1 of 38

PSYC 4310/6310 Advanced Experimental Methods and Statistics © 2012 Kalsher

Michael J. KalsherDepartment of

Cognitive Science

Adv. Experimental Methods & Statistics

PSYC 4310 / COGS 6310

Inferential Statistics III: ANCOVA

2 of 38

• When and Why do we use ANCOVA?

• Partitioning Variance

• Doing ANCOVA using SPSS

• Interpretation– Main Effects– Covariates

Analysis of Covariance

3 of 38

What is ANCOVA?

ANOVA can be extended to include one or more continuous variables, termed covariates, that are not part of the main experimental manipulation, but have an influence on the DV.

ANCOVA tests whether manipulated factors (IVs) have a significant effect on the DV after removing--or partialing out--the variance accounted for by the covariates.

4 of 38

Accounting for Variance: With and Without a Covariate

SSR without Covariate

SSRSSM

SSM

SSR

=Systematic Variance

Unsystematic Variance

SSR with Covariate

Variance explained by the covariate reduces the overall amount of unexplained (error) variance.

SSRSSM

SSCov

5 of 38

Purpose of Covariates in ANOVAReduce error variance

– The F-ratio in ANOVA compares the amount of variability explained by the experimental manipulation (MSM), against the variability that it cannot explain (MSR). What we hope is that the covariate explains some of the variance that was previously unexplained. This has the effect of reducing the unexplained variance (i.e., MSR becomes smaller) and so our F-ratio gets bigger. In real terms this means we obtain a more sensitive measure of the experimental effect.

Eliminate Confounds– In any experiment, there may be unmeasured variables that confound the results

(i.e., a variable that varies systematically with the experimental manipulation). If any variables are known to influence the dependent variable being measured, then ANCOVA can help to remove the bias of these variables. Once a possible confounding variable has been identified, it can be measured and entered into the analysis as a covariate.

ANCOVA is like an ANOVA on the values of the DV, after removing the influence of the covariate, rather than on the original values.

6 of 38

Example 1: Using ANCOVA to evaluate the relative effectiveness of competing math textbooks

• An educational psychologist is interested in evaluating the relative effectiveness of two textbooks for improving math skills.

• A literature review reveals that general IQ is related to math skills.

• She uses this information to make the test of the association between math skills and textbook type more sensitive (i.e., more powerful).

• Procedure– In each one of the two textbook groups, we compute the correlation

coefficient between IQ and math skills.

– We then estimate the amount of variation in math skills that is accounted for by IQ, and the amount of residual variation (the variance in math skills not explained by IQ).

– We use the residual variance in an ANOVA as an estimate of the true unexplained variance (error) after controlling for IQ.

7 of 38

Example 2: Evaluating the effectiveness of the types of treatment for treating a fatal disease

• A researcher compares the effectiveness of three treatments ("Placebo",

"Drug 1", and "Drug 2”) for treating a fatal disease in terms of average survival time.

• ANOVA could be used to analyze these data. But, suppose that supplementary information is available to him (i.e., each patients’ age).

• Age in this case is a "covariate" - it is not related to treatment, but it might be related to survival time.

• He uses the information to make the test of the association between survival time and treatment type more sensitive (more powerful).

– In each of the treatment groups, he computes the correlation coefficient between age and survival time.

– He estimates the amount of variation in survival time accounted for by age, and the amount of residual variation that is not explained by age.

– He uses the residual variance in an ANOVA as an estimate of the true error after controlling for age.

8 of 38

Choosing Covariates

In general, it is preferable to have a small number of covariates.

Covariates should be correlated with the DV, but NOT with each other.

– When covariates are significantly correlated:• they contribute very little to error reduction

• they can cause computational difficulties such as multicollinearity.

• one or the other should be removed since they are statistically redundant.

9 of 38

Assumptions of ANCOVA

ANCOVA has the same basic assumptions of all of the parametric tests, but also has two additional requirements:

1.independence of the covariate and treatment effect.

2.homogeneity of regression slopes.

10 of 38

Assumptions of ANCOVA: Independence of the covariate and treatment effect

We assume the covariate is correlated with the outcome variable (DV), so that as scores on the covariate change, scores on the DV change by a similar amount.

Important to ensure that:

• the covariate explains previously unexplained variance in the DV—not variance explained by the treatment.

• the effect that the covariate has on the outcome should be the same for all of the experimental groups.

– Randomizing participants to experimental groups or matching experimental groups on the covariate can help.

– Important to test this before running the ANCOVA.

11 of 38

Basic ANOVA showing that the total variance in the DV can be partitioned into treatment variance (SSM) and error or unexplained variance (SSR).

Ideal situation for ANCOVA in which the covariate shares variance with the currently “unexplained” variance, but is independent from the treatment effect (no overlapping variance).

Situation in which ANCOVA should NOT be used since the effect of the covariate overlaps with the experimental effect. Thus, the experimental effect is confounded with the effect of the covariate.

12 of 38

What ANCOVA actually does is to create a single regression equation, ignoring groups, to predict dependent variable (Y) scores using the covariate(s) (X1, X2, X3). Then, in essence, an ANOVA, not ignoring groups, is performed using the residualized scores of each person computed by subtracting each person’s predicted score based on the single equation, from the participants’ actual scores.

These computations are legitimate if the regression equations that predict the Y scores, computed separately in the ANOVA groups, have parallel slopes. This is the homogeneity of regression assumption, which requires that the “b” weight applied to the covariates are reasonably equal across each group.

Assumptions of ANCOVA: Homogeneity of Regression Slopes

13 of 38

The slope of the line predicting the DV from the CV should be relatively the same for each level of the IV. – In other words the regression coefficient (bi) relating a particular

CV to the DV should be the same for each group.

– In still other words, this means no IV x DV interaction.

Assumptions of ANCOVA: Homogeneity of Regression Slopes

14 of 38

Sample ANCOVA Problem: Viagra and Libido

Previously, we considered an example looking at the effects of Viagra on libido. Viagra was shown to influence libido, but other factors could also play a role in explaining a person’s libido, including their health, mood, and their partner’s libido.

If one or more of these variables is measured, it is possible to control for the influence they have on the dependent variable by including them in the analysis.

In the context of hierarchical regression, we would enter one or more of these variables into the model in the first block, followed by entry of the “dummy variables” representing the experimental manipulation in the second block. As such, we “partial out” the effect of the covariates (e.g., their partner’s libido) to see what effect the independent variable has after the effect of the covariate.

15 of 38

Via

gra

Co

vari

ate.

sav

DoseParticipant’s

LibidoPartner’s

Libido

Placebo 3.22 (1.79) 3.44 (2.07)

Low Dose 4.88 (1.46) 3.12 (1.73)

High Dose 4.85 (2.12) 2.00 (1.63)

16 of 38

Initial Considerations: Testing Independence of the IV and Covariate

17 of 38

Initial Considerations: SPSS Output – Independent Samples t-test

The non-significant F-test (p>.05) tells us that the treatment groups do not differ significantly on the covariate (Partner’s libido).

18 of 38

ANCOVA: Main Analysis

19 of 38

ANCOVA: Contrasts

Note that the “Post-Hoc” option isn’t available for analyses using covariates.

Be sure to “click” on the “Change” button!

20 of 38

ANCOVA: Options

You can perform “Post-Hoc” tests through “Options”.

1.Move the IV, “Dose”, into the “Display Means for” box.

2.“Click” on the “Compare main effects” box.

3.The “Confidence interval adjustment” option becomes available.

4.The resulting output will be a table of estimated marginal means (the means adjusted for the effect of the covariate).

21 of 38

SPSS Output: Descriptive Statistics and Homogeneity test

22 of 38

Testing for Equal Variance: Different Approaches

• Levene’s Test– If Levene’s test is significant, a more stringent alpha can be

used (.01) or the variable can be dropped from the analysis. – This is a very conservative test and is not necessarily the

best way to judge whether variances are unequal enough to cause problems.

• Alternative Procedure: Highest and lowest variances

– Obtain standard deviation values for each of the groups; square these values to obtain the variances.

– Take the largest value and divide it by the smallest value. If the resulting value is less than 2, then we shouldn’t worry too much. If it is larger than 2, then we do (see next slide).

23 of 38

The ratio of the highest variance (4.49) to lowest variance (2.13) = 2.11. Look up the critical value for 10 subjects, and 3 variances.

Critical value = 5

24 of 38

SPSS Output: Main Analysis

ANOVA

ANCOVA

25 of 38

SPSS Output: Parameter Estimates

Dummy Coded VariablesDose = 1: Represents the difference between the Placebo and High Dose groups.Dose = 2: Represents the difference between the Low Dose and High Dose groups.Dose = 3: The High Dose reference group.

26 of 38

SPSS Output: Planned Contrasts

Low dose vs. Placebo

High dose vs. Placebo

Adjusted Means

27 of 38

SPSS Output: Post-Hoc Tests

Note: Difference between Placebo and Low Dose groups is no longer significant (p>.05).

28 of 38

SPSS Output: ANCOVA as Regression

29 of 38

SPSS Output: ANCOVA as Regression

30 of 38

Assumption of Homogeneity of Regression Slopes

We assume the relationship between the DV and covariate is the same in each treatment group.

31 of 38

Re-run the ANCOVA: Custom Model

The purpose: To test the Covariate by Outcome Interaction. If the interaction is significant, the homogeneity of regression slopes has been violated.

32 of 38

Step 1 Step 2

Step 3

33 of 38

SPSS Output: Testing the Dose*Partner_Libido Interaction

34 of 38

Calculating Effect Size for ANCOVA

Eta squared (2 ) is essentially r2. Derived by dividing SSM by SST. Assesses the proportion of total variance explained by a variable.

Partial eta squared (partial 2 ) assesses the proportion of variance that a variable explains that is not explained by other variables in the analysis.

2 = SSEffect

SSTotal

Partial 2 = SSEffect

SSEffect + SSResidual

35 of 38

Calculating Effect Size for ANCOVA

Partial 2 Dose

= SSDose

SSDose + SSResidual

Partial 2 Partner Libido = SSPartnerLibido

SSPartnerLibido + SSResidual

= 25.19 25.19 + 79.05

= .24

= 15.08 15.08 + 79.05

= .16

These values show that Dose explained a bigger proportion of the variance not attributable to other variables than Partner_Libido

36 of 38

ANCOVA: Reporting the Results

The covariate, partner’s libido, was significantly related to the participant’s libido, F(1,26) = 4.96, p<.05, r = .40. There was also a significant effect of Viagra on levels of libido after controlling for the effect of partner’s libido, F(2,26) = 4.14, p<.05, partial 2 = .24.

Planned contrasts revealed that having a high dose of Viagra significantly increased libido compared to having a placebo, t(26) = -2.77, p<.05, r=.48, but not compared to having a low dose, t(26) = -0.54, p>.05, r = .11.

(see pp. 416-417 for specifics of results reporting).

37 of 38

Sample Problem #1: Hangover Cures

A marketing manager for a well-known beverage manufacturer (Alka-Seltzer) believes his product is more effective than other recommended hangover “cures.” To test his product against competitor cures he invites 15 people to join him at a bar and proceeds to get them drunk.

The next morning, he gives 5 of them strong coffee to drink (which is assumed to have no beneficial effect), 5 of them Alka Seltzer, and 5 of them Mimosas (champagne and orange juice—”hair of the dog that bit them”). This variable is called “drink”. He measures how well they feel two hours later on a Likert-type scale with the following anchors: 0 = “I feel terrible” to 10 “I feel terrific”. This variable is called “well”.

The marketing manager realizes that it is important to control for how drunk the person got the night before, and so he measures this on another Likert-type scale with the following anchors: 0 = “Completely sober” to 10 = “Completely intoxicated”. This variable is called “drunk”.

38 of 38

Hangover Cures Dataset

Your task:Conduct the appropriate analyses to see whether the drinks differ in their ability to treat hangovers when controlling for how much was drunk the night before. (Hint: Run the ANOVA first, then the ANCOVA).Write up your results. Is Alka Seltzer better than the other cures? What will you conclude from the study?

Strong Coffee Alka Seltzer Mimosas

Well Drunk Well Drunk Well Drunk

5 5 5 6 5 2

5 3 4 6 6 3

6 2 6 4 6 2

6 1 8 2 6 3

3 7 6 3 6 2