One Factor Repeated Measures Design on SPSS for Windows The Multivariate Approach The Example An experimenter wants to evaluate the relative merits of four versions of an instrument used to display altitude in a helicopter. Eight helicopter pilots, with from 500 to 3000 flight hours, are available as participants. Accuracy in reading the altimeter at low altitudes is of prime importance, so the dependent variable is the amount of reading error. It is anticipated that the amount of previous flying experience may affect pilots' performance with the experimental altimeters. So in order to isolate this nuisance variable, a repeated measures design was used. Each participant made 100 readings under simulated flight conditions with each of the altimeters. The sequence in which the four altimeters were presented was randomised for each pilot. Do differences exist between the altimeters? Altimeter 1 1 2 3 4 5 6 7 8 3 6 3 3 1 2 2 2 2 7 8 7 6 5 6 5 6 3 4 5 4 3 2 3 4 3 4 7 8 9 8 10 10 9 11

Pilot

For a repeated measures design, SPSS requires that we enter the data for each level of the factor into a different column of its data editor. We can do this and give each column a reasonable name like alt1, alt2, alt3, and alt4.

To run the one way repeated measures ANOVA, we click on Analyze, General Linear Model, Repeated Measures:

1

This opens up the Repeated Measures Define Factor(s) dialogue box:

SPSS has provided a default factor name, factor1, which I suggest you should replace with something a bit more meaningful, say, altimeter. Once you have done that, you need to indicate that there are four levels of the factor by entering 4 in the Number of Levels box. As soon as you do this, the Add box will become emboldened, and if you click on this, altimeter and 4 will disappear, and altimeter(4) will appear in the box to the right of Add:

2

You can give the dependent variable a name by clicking in the box next to Measure Name and typing in an appropriate name. Were measuring the amount of error with each altimeter, so type error in the Measure Name box and click on the (now emboldened Add). The name error appears in the bottom box.

Now you need to click on Define to open up the Repeated Measures dialogue box:

3

Click on alt1 in the box on the left and drag down to alt4, then click on the right arrow to paste these into the _?_(1,error), _?_(2,error), _?_(3,error) and _?_(4,error) gaps in the WithinSubjects Variables box.

Click on Post Hoc and confirm that the usual post hoc methods are not available for repeated measures factors on SPSS. Now click on Options and click on Descriptive statistics. In the top part of the Options dialogue box click on altimeter and on the right arrow to put it under Display Means for: When you do this the small box beside Compare main effects becomes active.

Click in the Compare main effects box and the Confidence interval adjustment box becomes active. Click on the down arrow and select Bonferroni.

4

Click on Continue and then, click on OK in the Repeated Measures dialogue box. Heres the output:

General Linear Model[DataSet1] G:\PY0701\One Way Repeated Measures Design.savWithin-Subjects Factors Measure: error altimeter 1 2 3 4 Dependent Variable alt1 alt2 alt3 alt4

Descriptive Statistics alt1 alt2 alt3 alt4 Mean 2.7500 6.2500 3.5000 9.0000 Std. Deviation 1.48805 1.03510 .92582 1.30931 N 8 8 8 8

5

b Multivariate Tests

Effect altimeter

Pillai's Trace Wilks' Lambda Hotelling's Trace Roy's Largest Root

Value .971 .029 33.276 33.276

F Hypothesis df 55.461a 3.000 55.461a 3.000 a 55.461 3.000 55.461a 3.000

Error df 5.000 5.000 5.000 5.000

Sig. .000 .000 .000 .000

a. Exact statistic b. Design: Intercept Within Subjects Design: altimeter

b Mauchly's Test of Sphericity

Measure: error Epsilon Within Subjects Effect Mauchly's W altimeter .072 Approx. Chi-Square 15.077 df 5 Sig. .011 Greenhous e-Geisser .419a

Huynh-Feldt .468

Lower-bound .333

Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a. May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the Tests of Within-Subjects Effects table. b. Design: Intercept Within Subjects Design: altimeter

Tests of Within-Subjects Effects Measure: error Source altimeter Type III Sum of Squares 194.500 194.500 194.500 194.500 28.500 28.500 28.500 28.500 df 3 1.257 1.403 1.000 21 8.798 9.818 7.000 Mean Square 64.833 154.750 138.678 194.500 1.357 3.239 2.903 4.071 F 47.772 47.772 47.772 47.772 Sig. .000 .000 .000 .000

Error(altimeter)

Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound Sphericity Assumed Greenhouse-Geisser Huynh-Feldt Lower-bound

Tests of Between-Subjects Effects Measure: error Transformed Variable: Average Source Intercept Error Type III Sum of Squares 924.500 12.500 df 1 7 Mean Square 924.500 1.786 F 517.720 Sig. .000

Estimated Marginal Means altimeter6

Estimates Measure: error altimeter 1 2 3 4 Mean 2.750 6.250 3.500 9.000 Std. Error .526 .366 .327 .463 95% Confidence Interval Lower Bound Upper Bound 1.506 3.994 5.385 7.115 2.726 4.274 7.905 10.095

Pairwise Comparisons Measure: error Mean Difference (I-J) Std. Error -3.500* .267 -.750 .313 -6.250* .881 3.500* .267 2.750* .250 -2.750* .726 .750 .313 -2.750* .250 -5.500* .707 6.250* .881 2.750* .726 5.500* .707 95% Confidence Interval for a Difference Lower Bound Upper Bound -4.472 -2.528 -1.889 .389 -9.454 -3.046 2.528 4.472 1.841 3.659 -5.389 -.111 -.389 1.889 -3.659 -1.841 -8.071 -2.929 3.046 9.454 .111 5.389 2.929 8.071

(I) altimeter 1

2

3

4

(J) altimeter 2 3 4 1 3 4 1 2 4 1 2 3

Sig. .000 .288 .001 .000 .000 .041 .288 .000 .001 .001 .041 .001

a

Based on estimated marginal means *. The mean difference is significant at the .05 level. a. Adjustment for multiple comparisons: Bonferroni.

The first table simply identifies the four levels of the within-subjects factor ALTIMETER. The second table gives the mean, standard deviation and sample size for each of the levels. They would be included in a Results section. Something like this:

7

Table of means (and standard deviations) of the errors made with four altimeters. (n=8) Altimeter 1 2.75 (1.49) The Multivariate Approach We used to have to analyse repeated measures designs using the univariate anova approach. Unfortunately this involved us in having to make some rather unlikely assumptions about the data, or in testing the assumptions and trying to allow for violations of them by adjusting the univariate anovas degrees of freedom. Much of SPSSs output is related to this approach. In recent years it has become clear that it is possible to analyse repeated measures designs using the class of statistical tests known as multivariate analysis of variance which do not make such unrealistic assumptions. This is the approach we now take to analysing all repeated measures factors. The results of such tests are shown in the third table of output. The degrees of freedom associated with such tests are often quite small, but this is a small price to pay for the freedom from restrictive assumptions the multivariate tests bring with them. SPSS presents four different multivariate tests, but they all lead to the same conclusion here. We will report Wilkss Lambda as it seems to be the most frequently used as well as being the one recommended in most textbook. Using this approach we are led to the conclusion that the number of errors made did depend significantly on the altimeter used, Wilks Lambda = .029, F(3,5) = 55.461, p < .001. Post Hoc Analysis Using Bonferroni Corrected Repeated Measures t Tests When post hoc comparisons are carried out following a significant independent groups anova (we used Tukey earlier in the module), these employ an error term that is derived from all the conditions in the study, not just the two conditions we are comparing with a particular comparison. Things are very different in repeated measures designs. The received wisdom is that it is safer to use only the data in the two conditions being compared when conducting post hoc comparisons in repeated measures designs. The approach is to conduct repeated measures t tests between every pair of conditions, and to control for inflation of type one error by dividing the family-wise significance level equally between the tests (using the Bonferroni method). In terms of the p value for one of the tests conducted, following Bonferroni correction, the p value will be j times bigger than the p value that would be given if we were conducting just one t test (instead of j t tests). The next table of output gives the mean and standard error for each altimeter (and a confidence interval for each mean). The table following shows the results of the Bonferroni corrected repeated measures t tests that have been conducted to compare every altimeter with every other one (six tests in all, though SPSS give each one twice!). The table below shows the results in a more user-friendly way. Altimeter 2 6.25 (1.04) Altimeter 3 3.50 (0.93) Altimeter 4 9.00 (1.31)

8

Table showing mean differences between errors made for all pairs of altimeters, and p values resulting from Bonferroni corrected repeated measures t tests. Alt 1 2.75 Alt1 2.75 Alt2 6.25 Alt3 3.50 Alt 2 6.25 -3.500 p < .001 Alt3 3.50 -0.750 p = .288 2.750 p < .001 Alt4 9.00 -6.250 p = .001 -2.750 p = .041 -5.500 p = .001

The p values above are all six times bigger than they would have been if we had only been conducting one repeated measures t test. For example a repeated measures t test to compare altimeters 1 and 3 yields a p value of .048. Six times this value is .288, the Bonferroni corrected p value in the table above for the comparison of altimeters 1 and 3. Conclusions If you look at the right hand side of the table, youll see that all the three tests involving altimeter 4 are significant. We can conclude from this (and from looking at the condition means) that, following Bonferroni correction: Altimeter 4 (mean = 9.00) leads to significanly more error than Altimeter 1 (mean 2.75) and Altimeter 3 (mean 3.50), p = .001, and than Altimeter 2 (mean 6.25), p = .041. The other two significant results both involve altimeter 2. We can conclude that, following Bonferroni correction: Altimeter 2 (mean 6.25) leads to significantly more error than Altimeter 1 (mean 2.75) and Altimeter 3 (mean 3.50), p < .001. Following Bonferroni correction, Altimeter 1 (mean 2.75) and Altimeter 3 (mean 3.50) do not lead to significantly different amounts error, p = .288. (Note that if the only test we had conducted had been between Altimeters 1 and 3, we would have concluded that they did lead to significantly different amounts of error, as the uncorrected p value was .048, as mentioned above.)

9