two-way factorial anova€¦ · web viewfactorial designs topic: the analysis and interpretation of...

42
Factorial Designs Topic: The analysis and interpretation of designs employing two factors. Backing up . . . In designs with only one factor, what do we do about other factors? Other factors are 1) held constant, 2) randomized, 3) matched, etc. 4) Ignored 5) Included in research Caveat: This is about multiple factors in the same research. Multiple quantitative variables next sem. Suppose you decide to include a second factor in the research. Question How should we combine the levels of the second factor with those of the first? Example Suppose two Types of Training are being compared – Lecture vs. CAI are being compared. The situation involves teaching new employees the basic facts they need to know working in an organization. The training period lasts for one week. Our interest is in Type of Training, but it might be that the specific job in which an employee would affect how much they learned. So, Job is an extraneous variable. So we decided to include the factor, Job, in the research. Job has four levels – Clerical, Receptionist, Maintenance, and Managerial. So this research involves two factors: Factor 1: Type of Training with two levels – Lecture and CAI. Factor 2: Job with four levels – Clerical, Receptionist, Maintenance, and Managerial Factorial Designs - 1 4/29/22

Upload: others

Post on 23-Jul-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Factorial Designs

Topic: The analysis and interpretation of designs employing two factors.

Backing up . . .

In designs with only one factor, what do we do about other factors?

Other factors are 1) held constant, 2) randomized, 3) matched, etc. 4) Ignored 5) Included in researchCaveat: This is about multiple factors in the same research. Multiple quantitative variables next sem.Suppose you decide to include a second factor in the research.

Question How should we combine the levels of the second factor with those of the first?

Example

Suppose two Types of Training are being compared – Lecture vs. CAI are being compared.

The situation involves teaching new employees the basic facts they need to know working in an organization. The training period lasts for one week.

Our interest is in Type of Training, but it might be that the specific job in which an employee would affect how much they learned. So, Job is an extraneous variable.

So we decided to include the factor, Job, in the research. Job has four levels – Clerical, Receptionist, Maintenance, and Managerial.

So this research involves two factors:

Factor 1: Type of Training with two levels – Lecture and CAI.

Factor 2: Job with four levels – Clerical, Receptionist, Maintenance, and Managerial

Suppose the dependent variable is a score on a test of amount learned during a training session.

The most efficient way to conduct research involving two different factors is a design called a Factorial Design. It’s also called a completely crossed design.

In a factorial (or completely crossed) design, data are gathered at all combinations of levels of both factors.

This design is best conceptualized using a two way table, with each dimension of the table representing one of the factors . . .

Clerical Receptionist Maintenance Managerial

Lecture Data Data Data Data Lecture mean

CAI Data Data Data Data CAI mean

Clerical mean

Receptionist mean

Maintenance mean

Management mean

Factorial Designs - 1 5/23/23

Page 2: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Any researcher would certainly have two common questions concerning the research:

1. Is there any overall difference in performance of those taught using Lecture vs. performance of those taught using CAI?

This question compares performance of participants in the first row of the two way table with that of participants in the second row of the table. The difference between row means is called the Main Effect of the row factor in the above design.

Clerical Receptionist Maintenance Managerial

Lecture Data Data Data Data Lecture mean

CAI Data Data Data Data CAImean

Cler mean Rec mean Maint mean Manag mean

2. Are there any overall differences in performance of Clerical workers, Receptionists, Maintenance workers, and Managers?

This question compares performance in of participants in the columns of the table. The difference between column means is called Main Effect of the column factor in the above design.

Clerical Receptionist Maintenance Managerial

Lecture Data Data Data Data Lecture mean

CAI Data Data Data Data CAImean

Cler mean Rec mean Maint mean Manag mean

These two questions are about what are called the Main Effects of the factors.

The effect of each factor by itself is called a Main Effect.

Factorial Designs - 2 5/23/23

versus

versus versus versus

Page 3: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

A Third Question

There is a 3rd question, called the interaction question, one that is a little less obvious, but important nonetheless.

This question is emergent – it exists only because we’ve included two factors in our research and included them in a factorial arrangement.

It can be asked in two equivalent ways. Both ways are about differences associated with one factor across levels of the other factor.

3) Version 1: Does the difference between Lecture and CAI change across levels of the Job factor?

3) Version 2: Do the differences between Clericals, Receptionists, Maintenance and Managers change across levels of the Type of Training factor?

This is a question about what is called the Interaction of the Row and Column factors.

An interaction exists when the row differences change across levels of the column factor or equivalently, when the column differences change across levels of the row factor

Clerical Receptionist Maintenance Managerial

Lecture L-C Mean L-R Mean L-M Mean L-B Mean Lecture mean

CAI C-C Mean C-R Mean C-M Mean C-B Mean CAImean

Cler mean Rec mean Maint mean Manag mean

Or

Clerical Receptionist Maintenance Managerial

Lecture L-C Mean L-R Mean L-M Mean L-B Mean Lecture mean

CAI C-C Mean C-R Mean C-M Mean C-B Mean CAImean

Cler mean Rec mean Maint mean Manag mean

Factorial Designs - 3 5/23/23

versus versus versus

versus

Page 4: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Formal statistical tests of the effects

1) Test of Row Main Effect – the Row Main Effect is the difference between row means – difference between overall performance in Row 1 vs. overall performance in Row 2.

Each row is viewed as a group.

The mean of all scores in each row is computed.

The Row main effect is tested by assessing the significance of differences between the marginal means of each row.

2) Test of Column Main Effect – the Column Main effect is the difference between column means – the difference in overall performance in Col 1 vs Col 2 vs. Col 3 vs. Col 4.

Each column is viewed as a group.

The mean of all scores in each column is computed.

The Column Main Effect is tested by assessing the significance of differences between the marginal means of each column.

3) Test of interaction effect.

The differences between cell means within each column are compared with differences between corresponding cell means within every other column.

If the differences between cell means within each column change from one column to the next, the Interaction is significant.

Or

The differences between cell means within each row are compared with differences between corresponding cell means within every other row.

If the differences within cell means within each row change from one row to the next, the Interaction is significant.

If there is an interaction, we say that the effect of each factor depends on the level of the other factor.

If there is NO interaction, we say that the effect of each factor does not depend on the level of the other factor – each factor’s effect on the dependent variable is independent of the level of the other factor.

Examples

Coffee in the morning brightens my day, regardless of what day it is.Coffee in the morning brightens my day on weekends, otherwise not so much. – The effect of coffee interacts with day of the week.

Factorial Designs - 4 5/23/23

Page 5: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Using graphs to visualize main effects and interactions.

The recommended graph

Plot Cell means vs. levels of the Column factor

Connect means of cells within the same row with a line.

Example using artificial data with no interaction

Hypothetical data with 2 scores per cell

Clerical Receptionist Maintenance ManagerialMarginal

Lecture 40,50M=45

50, 60M=55

60,70M=65

70,80M=75

60

CAI 30,40M=35

40,50M=45

50,60M=55

60,70M=65

50

Marginal 40 50 60 70 55

The plot of Cell Means

COL

4. 003. 002. 001. 00

Me

an

DV

80

70

60

50

40

30

ROW

1. 00

2. 00

Graphical Representation of Row Main Effect: The average difference in height of the two lines.

Note that in the example, the continuous line is above the dashed line, so there is (if significant) a Row Main Effect.

Factorial Designs - 5 5/23/23

CAI

Lecture

Cler Rec Maint Man

Page 6: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Graphical Representation of Column Main Effect: The difference in average heights of points at each column level.

COL

4. 003. 002. 001. 00

Me

an

DV

80

70

60

50

40

30

ROW

1. 00

2. 00

The column main effect is assessed by comparing the heights of the filled ellipses added to the figure above. There are clear differences in the heights of the ellipses, sugg esting (if significant) that there is a Column Main effect.

Graphical Representation of The Interaction Effect

COL

4. 003. 002. 001. 00

Me

an

DV

80

70

60

50

40

30

ROW

1. 00

2. 00

The Interaction Effect is tested by comparing the differences between rows – represented by the lengths of the arrows above – at each column. The arrows all look like they’re about the same length, suggesting that the row differences are the same from column to column. This means that there is no interaction. Note that the lack of an interaction means that the lines for the different rows will be parallel.

Factorial Designs - 6 5/23/23

CAI

Lecture

Average of all scores in Column 1

Difference in Row means for Column 1

Lecture

CAI

Page 7: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Example using artificial data with an interactionClerical Receptionist Maintenance Managerial

MarginalLecture 40,50

M=4550, 60M=55

50,60M=55

40,50M=45

50

CAI 30,40M=35

40,50M=45

50,60M=55

60,70M=65

50

Marginal 40 50 55 55 50

Graph illustrating interaction example

COL

4. 003. 002. 001. 00

Me

an

DV

80

70

60

50

40

30

ROW

1. 00

2. 00

Graphical Representation of Row main effect: Compare “average” heights of lines.

We can plainly see that the continuous line is above the dashed line for 3 columns but below the dashed line for the 4th column (Managers). Many times when there is an interaction, the issue of whether there is a main effect may be in question, as it may be here.

Graphical Representation of Column main effect: Compare “average” heights of points at each column

It seems that mean performance goes up as we move from Job 1 to Job 2 to Job 3, but then it levels off between Job3 and Job 4. But we can see that the differences between the columns are not the same for the dashed line as they are for the continuous line. Again, this is an instance in which there might be a question concerning whether or not there is a main effect of the column (Job) factor.

Graphical Representation of Interaction effect: Compare differences between heights of the line at each column.

The differences between heights of the lines are not the same from column to column. So if confirmed by the appropriate statistical test, an interaction may be present.

Factorial Designs - 7 5/23/23

CAI

Lecture

Page 8: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Graphs of Types of Outcomes of Factorial Designs

Based on Aron & Aron, p. 374, Table 13-7.1.

C1 C2 C3 MarginalMeans

R1 10 10 10 10R2 20 20 20 20MarginalMeans

15 15 15 15

RowMain Effect

ColumnMain Effect Interaction

Yes No No

------------------------------------------------------------------------------------------------------------------------------

2.

C1 C2 C3 MarginalMeans

R1 10 20 30 20R2 10 20 30 20MarginalMeans

10 20 30 20

RowMain Effect

ColumnMain Effect Interaction

No Yes No

------------------------------------------------------------------------------------------------------------------------------

3.

C1 C2 C3 MarginalMeans

R1 10 20 30 20R2 20 30 40 30MarginalMeans

15 25 35 25

RowMain Effect

ColumnMain Effect Interaction

Yes Yes No

Factorial Designs - 8 5/23/23

Page 9: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

4.

C1 C2 C3 MarginalMeans

R1 10 20 30 20R2 10 20 60 30MarginalMeans

10 20 45 25

RowMain Effect

ColumnMain Effect Interaction

Yes?? Yes Yes

The performance in R2 increases more from C1 to C2 than does performance in R1.The difference between R1 and R2 is changes as we go from C1 to C3.

------------------------------------------------------------------------------------------------------------------------------

5.

C1 C2 C3 MarginalMeans

R1 10 20 30 20R2 30 20 10 20MarginalMeans

20 20 20 20

RowMain Effect

ColumnMain Effect Interaction

No No Yes

This is a classic crossed interaction. Neither the Row nor the Column Main Effect is important here. The interaction is the key feature. But we would say that each factor has an effect, just not a main effect.-----------------------------------------------------------------------------------------------------------------------------

6.

C1 C2 C3 MarginalMeans

R1 10 20 30 20R2 20 40 60 40MarginalMeans

15 30 45 30

RowMain Effect

ColumnMain Effect Interaction

Yes Yes Yes

This is a situation that I would interpret as representing both Main Effects and an interaction.

-----------------------------------------------------------------------------------------------------------------------------

Factorial Designs - 9 5/23/23

Page 10: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Two Way Factorial ANOVAWorked Out Example Based on Minium et al. p. 359

The data

The data are from a hypothetical Verbal Learning Experiment in which the effects of two factors, Anxiety and External Pressure, on performance in the task were investigated.

Two groups of participants, were selected, one group with generally Low Anxiety levels and the other group with generally High Anxiety levels.

One third of each Anxiety group was given instructions to induce little pressure.A second third was given instructions to induce moderate pressure to perform. The final third was given instructions to induce strong pressure to perform well. The dependent variable is number correct on the verbal learning task.

The data presumably illustrate the classic inverted U relationship of performance to motivation. id verblearn anxiety pressure

1 40 1 1 2 64 1 1 3 46 1 1 4 56 1 1 5 46 1 1 6 46 1 1 7 39 1 1 8 38 1 1 9 44 1 1 10 69 1 1 11 61 1 2 12 54 1 2 13 55 1 2 14 40 1 2 15 43 1 2 16 47 1 2 17 57 1 2 18 51 1 2 19 40 1 2 20 55 1 2 21 50 1 3 22 48 1 3 23 60 1 3 24 63 1 3 25 83 1 3 26 63 1 3 27 53 1 3 28 60 1 3 29 73 1 3 30 69 1 3

31 41 2 1 32 34 2 1 33 37 2 1 34 48 2 1 35 57 2 1 36 47 2 1 37 55 2 1 38 33 2 1 39 42 2 1 40 38 2 1 41 48 2 2 42 58 2 2 43 42 2 2 44 40 2 2 45 49 2 2 46 49 2 2 47 56 2 2 48 41 2 2 49 35 2 2 50 57 2 2 51 56 2 3 52 35 2 3 53 43 2 3 54 39 2 3 55 29 2 3 56 32 2 3 57 54 2 3 58 43 2 3 59 49 2 3 60 49 2 3

Conceptualization: As a 2 (Anxiety) x 3 (Pressure) Factorial

Low pressure: 1 Moderate Pressure: 2 High Pressure: 3Low Anxiety: 1 X X XHigh Anxiety: 2 X X X

The interests are:

1. Is there a Main Effect of Anxiety? On average do high anxious persons perform better or worse than low anxious?

2. Is there a Main Effect of Pressure? On average, do persons under different amounts of pressure perform this task differently?

3. Is there an Interaction of Anxiety and Pressure? Do performance differences between anxiety levels change at different levels of pressure? Equivalently do the effects of different levels of pressure differ for people with high anxiety vs. low anxiety?

Factorial Designs - 10 5/23/23

Perfor-mance

Motivation

Page 11: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

The analysis in SPSS

Analyze -> General Linear Model -> Univariate

When you put the names of two or more factors in the Fixed Factor(s) field, the program assumes that the data represent a factorial design.

Specifying Plots

Make the factor that you’re conceptualizing as the Column Factor the Horizontal Axis.

Make the Row Factor the one represented by Separate Lines.

Factorial Designs - 11 5/23/23

Column factor

Row factor

Page 12: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Post Hoc Tests.

You can specify post hoc tests for any main effect with more than 2 levels –the column main effect in this example.

Pressure has 3 levels, so we can specify post hocs for it.

Anxiety has only two levels, so if we find a difference, we know which means are different, and no post hocs need be specified for it.

The Output

UNIANOVA verblearn BY anxiety pressure /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /POSTHOC = pressure ( BTUKEY ) /PLOT = PROFILE( pressure*anxiety ) /PRINT = DESCRIPTIVE ETASQ OPOWER HOMOGENEITY /CRITERIA = ALPHA(.05) /DESIGN = anxiety pressure anxiety*pressure .

Univariate Analysis of Variance

G:\MdbT\P510 511\P511L13-Factorial\FactorEGBasedOnMinP359.sav

Factorial Designs - 12 5/23/23

Syntax, in case you’re interested.

Page 13: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Between-Subjects Factors

30

30

20

20

20

1

2

anxiety

1

2

3

pressure

N

Descriptive Statistics

Dependent Variable: verblearn

48.80 10.685 10

50.30 7.409 10

62.20 10.758 10

53.77 11.206 30

43.20 8.351 10

47.50 7.906 10

42.90 9.183 10

44.53 8.472 30

46.00 9.766 20

48.90 7.594 20

52.55 13.885 20

49.15 10.894 60

pressure1

2

3

Total

1

2

3

Total

1

2

3

Total

anxiety1

2

Total

Mean Std. Deviation N

Levene's Test of Equality of Error Variances a

Dependent Variable: verblearn

.328 5 54 .894F df1 df2 Sig.

Tests the null hypothesis that the error variance of thedependent variable is equal across groups.

Design: Intercept+anxiety+pressure+anxiety * pressurea.

Tests of Between-Subjects Effects

Dependent Variable: verblearn

2489.350b 5 497.870 5.958 .000 .356 29.791 .990

144943.350 1 144943.350 1734.579 .000 .970 1734.579 1.000

1278.817 1 1278.817 15.304 .000 .221 15.304 .970

430.900 2 215.450 2.578 .085 .087 5.157 .493

779.633 2 389.817 4.665 .014 .147 9.330 .762

4512.300 54 83.561

151945.000 60

7001.650 59

SourceCorrected Model

Intercept

anxiety

pressure

anxiety *pressure

Error

Total

Corrected Total

Type III Sumof Squares df Mean Square F Sig.

Partial EtaSquared

Noncent.Parameter Observed Power

a

Computed using alpha = .05a.

R Squared = .356 (Adjusted R Squared = .296)b.

The Main effect of Anxiety was significant, and the estimate of effect size, eta-squared was .221, huge.

The Main effect of Pressure was not significant, although eta-squared equals .087.

Factorial Designs - 13 5/23/23

Effect sizes are presented in terms eta-squared (2).

Small Medium Largef .1 .25 .42 .01 .059 .138

LowPressure

Moderate Pressure

High Pressur

eLow A 48.8 50.3 62.2 53.77High A 43.2 47.5 42.9 44.53

46.0 48.90 52.55 49.15

The means as a two way table.

Page 14: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

When the estimated effect size is large but the F is not significant, it’s a sign that due to small sample size, the test was not powerful enough to detect the fairly large difference. Shame on the researcher!!

The Interaction was significant, with eta-squared equal to .147.

Whenever you have a significant interaction, you should be very cautious in interpreting and reporting main effects. The significant interaction indicates that there is an effect of a variable, but that it is not a MAIN effect – it is a specific effect, specific to one or more levels of the other factor.

Post Hoc Tests

pressure

Homogeneous Subsetsverblearn

Tukey Ba,b

20 46.00

20 48.90

20 52.55

pressure1

2

3

N 1

Subset

Means for groups in homogeneous subsets are displayed.Based on Type III Sum of SquaresThe error term is Mean Square(Error) = 83.561.

Uses Harmonic Mean Sample Size = 20.000.a.

Alpha = .05.b.

Factorial Designs - 14 5/23/23

As might have been expected from the nonsignficant Main Effect of Pressure, there are no significant overall differences between any of the means at each pressure level.

Page 15: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Profile Plots

There IS an effect of pressure in these data, but it is not a MAIN effect. Instead, it would be best characterized as an “anxiety specific” effect. For low anxiety participants, increasing pressure lead to increasing performance.

But for high anxiety participants, increasing pressure led to increasing performance only up to a point. After that, further increases lead to a decrease in performance.

Factorial Designs - 15 5/23/23

Test question illustrated here:Is there an effect of pressure?Yes, but it’s not a MAIN effect.

Low

High

Pressure marginal means not significantly different.

For Low Anxiety persons, more pressure leads to higher performance

For High Anxiety persons, more pressure leads to higher performance up to a point, then to decreased performance.

Page 16: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Rcmdr Analysis of the data. Start here on 11/28/17

R Rcmdr Import Data from SPSS data set P5100 verblearn by anxiety pressure anova

Data Manage variables in active data set convert numeric variables to factors . . .

Statistics Means Multi-way ANOVA

Factorial Designs - 16 5/23/23

Page 17: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Rcmdr Two-way ANOVA Output

> library(foreign, pos=14)

> verblearn <- + read.spss("G:/MDBT/InClassDatasets/P5100 verblearn by anxiety pressure anova example.sav",+ use.value.labels=TRUE, max.value.labels=Inf, to.data.frame=TRUE)

> colnames(verblearn) <- tolower(colnames(verblearn))

> verblearn <- within(verblearn, {+ anxiety <- as.factor(anxiety)+ pressure <- as.factor(pressure)+ })

> AnovaModel.1 <- lm(verblearn ~ anxiety*pressure, data=verblearn, + contrasts=list(anxiety ="contr.Sum", pressure ="contr.Sum"))

> Anova(AnovaModel.1)Anova Table (Type II tests)

Response: verblearn Sum Sq Df F value Pr(>F) anxiety 1278.8 1 15.3040 0.0002581 ***pressure 430.9 2 2.5784 0.0852166 . anxiety:pressure 779.6 2 4.6650 0.0135256 * Residuals 4512.3 54 ---Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

> with(verblearn, (tapply(verblearn, list(anxiety, pressure), mean, + na.rm=TRUE))) # means 1 2 31 48.8 50.3 62.22 43.2 47.5 42.9

> with(verblearn, (tapply(verblearn, list(anxiety, pressure), sd, + na.rm=TRUE))) # std. deviations 1 2 31 10.685400 7.409453 10.7579432 8.350649 7.905694 9.182713

> xtabs(~ anxiety + pressure, data=verblearn) # counts pressureanxiety 1 2 3 1 10 10 10 2 10 10 10

Factorial Designs - 17 5/23/23

From SPSS

Page 18: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

A factorial between-group and repeated measures design:Comparison of Change Between Pre- and Post-test Performance

All employees of two buildings of an organization were given a pretest measuring productivity. Then employees in Building A were assigned to work in teams while those in Building B

performed essentially the same work as before. After 6 months, a posttest measure of productivity was obtained for each employee. Thus, each person was measured twice using the same test. The interest was on determining the

effect of time – pretest vs posttest and type of work (team vs no team) and the interaction of the two (time X type of work). This is a Pretest-posttest with nonequivalent groups design – Lecture 8, p. 12.)

The Data Editor . . . (Data are hypothetical.)

id bldg pre post

1 1 52 48 2 1 50 49 3 1 62 54 4 1 63 56 5 1 35 26 6 1 82 72 7 1 39 46 8 1 39 38 9 1 60 64 10 1 59 53 11 1 66 67 12 1 35 26 13 1 64 56 14 1 37 37 15 1 59 49 16 1 49 48 17 1 39 34 18 1 44 35 19 1 53 50 20 1 46 35 21 1 18 21 22 1 75 74 23 1 64 54 24 1 58 64 25 1 29 29 26 1 56 46 27 1 67 71 28 1 60 50 29 1 42 41 30 1 24 21 31 1 61 69 32 1 17 8 33 1 34 35 34 1 58 54 35 1 69 68 36 1 62 69 37 1 46 37 38 1 45 45 39 1 56 57 40 1 36 43 41 1 46 43 42 1 61 60 43 1 45 35 44 1 73 77 45 1 62 61 46 1 54 51 47 1 35 35 48 1 46 41 49 1 51 58

50 1 50 54 51 1 37 26 52 1 50 54 53 1 32 27 54 1 79 73 55 1 49 46 56 1 48 42 57 1 36 38 58 1 34 42 59 1 30 23 60 1 64 62 61 1 54 50 62 1 52 56 63 1 58 50 64 1 23 29 65 1 56 53 66 1 43 45 67 1 51 53 68 1 35 37 69 1 46 39 70 1 43 44 71 1 29 33 72 1 63 63 73 1 45 41 74 1 49 38 75 1 62 50 76 1 49 53 77 1 51 51 78 1 31 20 79 1 71 76 80 1 72 78 81 1 50 53 82 1 31 21 83 1 19 25 84 1 66 71 85 1 42 42 86 1 65 56 87 1 47 39 88 1 35 29 89 1 40 30 90 1 36 39 91 1 56 45 92 1 48 42 93 1 47 47 94 1 69 58 95 1 44 35 96 1 58 59 97 1 20 20 98 1 35 25 99 1 35 31 100 1 57 61

101 0 42 44 102 0 35 39 103 0 42 62 104 0 61 75 105 0 49 51 106 0 55 67 107 0 54 72 108 0 44 51 109 0 39 59 110 0 66 70 111 0 43 63 112 0 48 53 113 0 48 54 114 0 44 51 115 0 43 56 116 0 62 77 117 0 36 51 118 0 44 47 119 0 53 65 120 0 47 56 121 0 59 76 122 0 52 71 123 0 50 59 124 0 55 70 125 0 39 56 126 0 39 48 127 0 56 65 128 0 50 68 129 0 74 94 130 0 48 67 131 0 58 73 132 0 53 69 133 0 45 58 134 0 55 72 135 0 58 63 136 0 50 61 137 0 54 62 138 0 49 65 139 0 57 67 140 0 47 64 141 0 30 36 142 0 41 46 143 0 58 63 144 0 51 70 145 0 54 69 146 0 50 65 147 0 64 79 148 0 66 77 149 0 33 35 150 0 69 72 151 0 45 53

152 0 40 57 153 0 54 66 154 0 55 65 155 0 45 67 156 0 50 70 157 0 41 56 158 0 42 62 159 0 59 72 160 0 56 64 161 0 63 71 162 0 49 57 163 0 41 56 164 0 51 71 165 0 62 65 166 0 56 78 167 0 85 90 168 0 54 61 169 0 44 55 170 0 51 63 171 0 39 57 172 0 31 37 173 0 57 71 174 0 45 56 175 0 40 43 176 0 34 48 177 0 45 66 178 0 34 42 179 0 52 61 180 0 33 50 181 0 46 63 182 0 46 51 183 0 40 49 184 0 47 66 185 0 67 76 186 0 60 77 187 0 44 54 188 0 59 71 189 0 61 81 190 0 62 78 191 0 47 56 192 0 57 60 193 0 56 77 194 0 57 77 195 0 54 70 196 0 58 62 197 0 48 53 198 0 52 71 199 0 51 56 200 0 48 52

Factorial Designs - 18 5/23/23

Page 19: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

The expected result – from Lecture 8 . . .

6. A salvage design: The Pretest-Posttest with Nonequivalent Groups Design

It is important to note that the pretest and posttest must be the same instrument.

One of the most frequently employed designs in the social sciences.

Some outcomes lead to defensible arguments for treatment differences.

Others do not.

The ideal outcome:

If this pattern of results occurs – no difference on the pretest, difference favoring the treatment group on the posttest, most researchers would argue that it is evidence for the existence of a treatment effect.

Using our new knowledge of factorial designs, we can recognize this as such a design . . .

Type of work – Teams vs. no Teams – is the Row factor.

Time – Pre vs. Post – is the Column factor.

In most applications of this design, the hoped-for result is a significant interaction with small, even nonsignificant row and column main effects.

Factorial Designs - 19 5/23/23

MeanPerformance

No Teams

Pre Post

Teams

Page 20: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

SPSS menu sequence: Analyze -> General Linear Model -> Repeated Measures

Factorial Designs - 20 5/23/23

Make up a name for the repeated measures factor and enter it here.

Click Add, then Click Define

Enter the number of levels of the repeated measures factor here.

Page 21: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

The main dialog box

Factorial Designs - 21 5/23/23

Click on Plots

Click on Options

Note that the Teams vs no Teams factor is called bldg here.

Page 22: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Specifying Plots

If Time is one of the factors, always put the Time factor (prepost in this case) on the horizontal axis.

Specify the Between-subjects factor (bldg) to be represented by separate lines.

Factorial Designs - 22 5/23/23

Pre PostBldg A 50.32 62.33Bldg B 48.75 46.34

Two way table of means drawn by hand.

Page 23: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

The multivariate tests require the fewest restrictive assumptions.

In this case, all tests led to the same conclusions.

Note that ALL effects – both MAIN effects and the INTERACTION were significant. But, as mentioned above, when the interaction is significant, it is advisable to interpret main effects cautiously.

Factorial Designs - 23 5/23/23

Page 24: Two-Way Factorial ANOVA€¦ · Web viewFactorial Designs Topic: The analysis and interpretation of designs employing two factors.Backing up . . . In designs with only one factor,

Results . . .

1. There is an overall difference between Pre and Post-test means.But that may just be due to the huge increase in Building A.

2. There is an overall difference between Building A and Building B means.But that may be an artifact of the huge increase in Building A.

3. There is an interaction of Building and Prepost. Thus, the difference between Pre- and Post-test means depends on which building is considered or equivalently,

the effect of Teams vs no Teams changed from Pretest to Posttest, becoming larger on the Posttest..

Use the plots to interpret the interaction in greater detail.

This shows, confirmed by the significant interaction shown on the previous page that Performance in Building A increased from Pre to Post while performance in Build B decreased.

The conclusion is that assigning people to work in teams may have lead to increases in individual performance.

Factorial Designs - 24 5/23/23