chapter 10 - part 1 factorial experiments. nomenclature we will use the terms factor and independent...

Chapter 10 - Part 1

Factorial Experiments

Nomenclature

• We will use the terms Factor and Independent Variable interchangeably. They mean the same thing.

• The term “factorial analysis of variance” simply means the analysis of variance when there are multiple factors (multiple independent variables.)

• I will sometimes use the phrase Factor 1 or 2 interchangeably with Independent Variable 1 or 2. But to prevent confusion, whenever the term is abbreviated I will use IV1 or IV2, not F1 or F2.

Two-way Factorial Experiments

In Chapter 10, we are studying experiments with two independent variables, each of which will have multiple levels.

We call each independent variable a factor. The first IV is called Factor 1 or IV1. The second IV is called Factor 2 or IV2.

So, in this chapter we will study two factor, unrelated groups experiments.

Conceptual Overview

• In Chapter 9 you learned to do the F test comparing two estimates of sigma2, MSB and MSW. That is what you do in simple experiments, those with only one independent variable.

• In the single IV, unrelated groups experiments in Ch. 9, F = MSB/MSW

• That is one version of a more generic formula. The generic formula tells us what to do in the two (or more) factor case.

• Here is the basis for the generic formula

Generic formula for the unrelated groups F test

• F = SAMP.FLUC. + (?) ONE SOURCE OF VARIANCE)(SAMPLING FLUCTUATION)

• Which can also be written as

• F = (ID + MP + (?) ONE SOURCE OF VARIANCE)(ID+ MP)

Let me explain why.

The denominator of the F test• The denominator in the F test reflects variation within

each group because of random individual differences (ID) and measurement problems (MP). Since everyone in the same group is treated the same, only ID + MP can contribute to within group variation. So, MSW can not reflect the effect of any independent variable or combination of independent variables.

• Random sampling fluctuation is based on random individual differences and measurement problems (ID + MP).

• Thus, in the unrelated groups F and t test, our best index of the random effects of individual differences and measurement problems, the basis of random sampling fluctuation, is MSW, our least squares, unbiased estimate of sigma2.

Denominator = MSW

• In the F tests we are doing, F tests for unrelated groups, MSW serves as our best estimate of sigma2.

• To repeat, in computing MSW, we compare each score to the mean of its own, specific group. Everyone in each specific group is treated the same.

• So, the only reasons that scores differ from the other scores in their group and their own group means is that people differ from each other (ID) and there are always measurement problems (MP).

• MSW = ID +MP

Numerator of the F ratio: Generic formula

• Numerator of the F ratio is an estimate of sigma2 that reflects sampling fluctuation + the possible effects of one difference between the groups.

• In Ch. 9, there was only one difference among the ways the groups were treated, the different levels of the independent variable (IV)

• MSB reflected the effects of random individual differences (there are different people in each group), random measurement problems, and the effects of the independent variable.In Ch. 9, we could write that as MSB = ID + MP + (?)IV

To repeat, in the single factor analysis of variance F = (ID + MP + ?IV)

(ID + MP)

• Both the numerator and denominator reflect the same elements underlying sampling fluctuation

• The numerator includes one, and only one, systematic source of variation not found in the denominator.

This allows us to conclude that:• IF THE NUMERATOR IS SIGNIFICANTLY

LARGER THAN THE DENOMINATOR, THE SIZE DIFFERENCE MUST BE MUST BE CAUSED BY THE ONE ADDITIONAL THING PUSHING THE MEANS APART, the IV.

• But notice there can’t be more than one thing in the numerator that does not appear in the denominator to make that conclusion inevitable.

Why we can’t use MSB as the numerator in the multifactor analysis of variance

• In the two factor analysis of variance, the means can be pushed apart by:

– The effects of the first independent variable (IV1).

– The effects of the second independent variable (IV2)

– The effects of combining IV1 and IV2 that are above and beyond the effects of either variable considered alone (INT)

– Random sampling fluctuation (ID + MP)

So if we compared MSW to MSB in a two factor experiment, here is what we would have.

• F = (ID + MP + ?IV1 + ?IV2 + ?INT)

(ID + MP)

That’s not an F test. In an F test the numerator must have one and only one source of variation beyond sampling fluctuation. HERE THERE ARE THREE OF THEM!

Each of these three things besides sampling fluctuation could be pushing the means apart.

So, the F ratio would be meaningless.

To use the F test with a two factor design, we must create 3 numerators to compare to MSW, each comprising ID + MP + one other factor.

HOW? To obtain our 3 numerators for the F test, we divide (analyze) the sums of squares and degrees of freedom between groups (SSB & dfB) into component parts. Each part must contain only one factor along with ID and MP. Then each component will yield an estimate of sigma2 that can be compared to MSW in an F test.

Write out the answer to these two questions without reading the

answer from the slides:

• Why can’t you compare MSB to MSW in the two factor, unrelated groups F test?

• What must you do instead?

ANSWER 1: If we compared MSW to MSB in a two factor experiment, here is what we would have.

• F = (ID + MP + ?IV1 + ?IV2 + ?INT)

(ID + MP)

That’s not an F test. In an F test the numerator must have one and only one source of variation beyond sampling fluctuation. HERE THERE ARE THREE OF THEM!

Each of these three things besides sampling fluctuation could be pushing the means apart.

So, the F ratio would be meaningless.

ANSWER 2:We must take apart (analyze) the sums of squares and degrees of freedom between groups (SSB & dfB) into component parts. Each part must contain only one factor along with ID and MP. Then each component will yield an estimate of sigma2 that can be compared to MSW in an F test.

Here is how we divide SSB and dfB into their component parts:

• First, we create a way to study the effects of factor 1 alone.

• To do that, we combine groups so that the resulting, larger aggregates of participants differ only because they received different levels of the first independent variable, IV1.

• Each such combined group will include an equal number of people who received the different levels of IV2.

• So the groups are the same in that regard. • They differ only on how they were treated on the first

independent variable, IV1.

Computing MSIV1, one of the three numerators in a two factor F test

• If we find the differences between each person’s combined group mean and the overall mean, square and sum them, we will have a sum of squares for the first independent variable (SSIV1).

• Call the number of levels of an independent variable L. df for the combined group equals the number of levels of its IV minus one (LIV – 1).

• An estimate of sigma2 that includes only ID + MP + (?) IV1 can be computed by dividing this sum of squares by its degrees of freedom, as usual.

• MSIV1 = SSIV1/dfIV1 = (ID + MP + ?IV1)

Once you have MSIV1, you have one of the three F tests you do in

a two factor ANOVA

•F = MSIV1/MSW

Then you do the same thing to find MSIV2

• You combine groups so that you have groups that differ only on IV2.

• You compare each person’s mean for this combined group to the overall mean, squaring athe differences for each person and then summing them for the entire sample to get SSIV2.

• Degrees of freedom = the number of levels of Factor 2 minus 1 (dfIV2 = LIV2 – 1).

• Then MSIV2 = SSIV2/dfIV2

• FIV2 = MSIV2/MSW

What’s left is the interaction.• Remember, we are subdividing SSB and dfB into their

three component parts.• We have already computed SSIV1, SSIV2, dfIV1, and dfIV2.

• WHAT’S LEFT? The part of SSB and dfB that hasn’t been accounted for is the sum of squares and degrees of freedom for the interaction.

• The interaction involves the means being pushed apart by the two independent variables having a different effect when present together than either has by itself alone.

• For example, a moderate dose of alcohol can make you intoxicated. A moderate dose of barbituates can make you sleepy. Taken together they multiply each others’ effects and the interaction of the two drugs can easily make you dead.

How to compute the interaction.

• To compute the sum of squares and df for the interaction, we find that part of the sum of squares between groups and degrees of freedom between groups that are not accounted for by Factor 1 (IV1) and Factor 2 (IV2)

• THAT IS, Y0U SUBTRACT THE SUMS OF SQUARES AND df YOU’VE ALREADY ACCOUNTED FOR (SSIV1, SSIV2, dfIV1, and dfIV2) FROM THE SUM OF SQUARES AND DEGREES OF FREEDOM BETWEEN GROUPS (SSB & dfB).

• WHAT’S LEFT IS SSINT & dfINT, THE SUM OF SQUARES AND DEGREES OF FREEDOM FOR THE INTERACTION.

Look at that another way: The whole is equal to the sum of its parts.

• SSB and dfB are the between group sum of squares and degrees of freedom. Each is composed of three parts, SSIV1, SSIV2, SSINT and dfIV1, dfIV2, and dfINT.

• So if we subtract the SS and df for factors 1 & 2 from SSB and dfB, what is left is the sum of squares and df for the interaction.

• SSINT = SSB-(SSIV1+ SSIV2)=SSB- SSIV1- SSIV2

• dfINT = dfB-(dfIV1+ dfIV2)=dfB- dfIV1- dfIV2

Designs of 2 factor studies

Analysis of Variance• Each possible combination of IV1 and IV2 creates

an experimental group. Participants are randomly assigned to each of the treatment groups.

• Each experimental group is treated differently from all other groups in terms of one or both factors.

• For example, if there are 2 levels of the first variable (Factor 1or IV1) and 2 of the second (IV2), we will need to create 4 groups (2x2). If IV1 has 2 levels and IV2 has 3 levels, we need to create 6 groups (2x3). If IV1 has 3 levels and IV2 has 3 levels, we need 9 groups. Etc.

Some nomenclature• Two factor designs are identified by simply stating

the number of levels of each variable. So a 2x4 design (called “a 2 by 4 design”) has 2 levels of IV1 and 4 levels of IV2. A 3x2 design has 3 levels of IV1 and 2 levels of IV2. And so on.

• Which factor is called IV1 and which is called IV2 is arbitrary (and up to the experimenter).

Example of 2 x 3 designTo make it more concrete, assume we are testing new

treatments for Generalized Anxiety Disorder. In a two factor design we examine the effects of cognitive behavior therapy vs. a social support group among GAD patients who receive Ativan, Zoloft or Placebo. Thus, IV1 has 2 levels (CBT/Social Support) while IV2 has 3 levels (Ativan/Zoloft/Placebo) So, we would form 2 x 3 = 6 groups to do this experiment. Half the patients would get CBT, the other half get social support. A third of the CBT patients and one-third of the Social Support patients also get Ativan. Another third of those who receive CBT and one-third of those who get social support also receive Zoloft. The final third in each psychotherapy condition get a pill placebo.

A 2 x 3 design yields 6 groups. Let’s say you have 24 participants. Four are randomly assigned to each group.

• Here are the six treatment groups:– CBT + Ativan– CBT + Zoloft– CBT + Placebo– Social support + Ativan– Social support + Zoloft– Social support + Placebo

Example: Experiment Outline• Population: Outpatients with Generalized Anxiety Disorder• Subjects: 24 participants divided equally among 6 treatment

groups.• Independent Variables:

– Factor 1: Psychotherapy: CBT or Social Support (SoSp)– Factor 2: Medication: Ativan, Zoloft, or Placebo

• Groups: 1=CBT + Ativan; 2=CBT + Zoloft; 3=CBT + Placebo; 4=SoSp + Ativan; 5=SoSp + Zoloft; 6=SoSp + Placebo.

• Dependent variable: Anxiety remaining after treatment. Lower scores equal less anxiety and a better outcome.

A 3X2 STUDYType of drug

Type of therapy

Ativan Zoloft Placebo

SoSp

CBT

PlaceboXZoloftXAtivanX

SoSpX

CBTX

M

CBTAX

SoSpAX SoSpZX SoSpPX

CBTZX CBTPX

Effects

• We are interested in the main effects of type of psychotherapy and type of drug. Do participants get better with CBT and not with SoSp or the reverse? Do people get better when they get a mild tranquilizer (Ativan) an SSRI (Zoloft) or Placebo.

• We are also interested in assessing how combining different levels of both factors affect the response in ways beyond those that can be predicted by considering the effects of each IV separately. So, we are interested in the interaction of the independent variables.

MSWDrug Given

Type of therapy


SoSp

CBT


SoSpX

CBTX

M

CBTAX


CBTZX CBTPX

Compare each score to the mean for its group.

XX

Mean Squares Within Groups1.11.21.31.4

2.12.22.32.4

3.13.23.33.4

#S4.14.24.34.4

5.15.25.35.4

6.16.26.36.4

3489

7342

15141611

12776

69

1213

14181921

X #S X 2XX 9449

9104

1049

16114

16149

16019

2XX 6666

4444

14141414

8888

10101010

18181818

XX X-3-223

3-10

-2

102

-3

-4-1-1-2

-4-123

-4013

XX X

18624 kndfW

MSW

• SSw = 132.00

• dfW = 18

• MSW = 132/18 = 7.33

Then we compute a sum of squares and df between groups

• This is the same as in Chapter 9

• The difference is that we are going to subdivide SSB and dfB into component parts.

• Thus, we don’t use SSB and dfB in our Anova summary table, rather we use them in an intermediate calculation.

Sum of Squares Between Groups (SSB)

Type of Drug

Type ofTherapy


SoSp

CBT


SoSpX

CBTXCBTAX


CBTZX CBTPX

Compare each group meanto the overall mean.

MX

M

Means for GAD study

Type of Drug

Diet Type


Social Support

Cognitive BT 6 4

10

14

8 18

00.10M16.007.007.00

12.00

8.00

6666

4444

14141414

8888

10101010

18181818

X

5161 kdfB

Sum of Squares Between Groups (M=10.00, SSB=544 , dfB=5)

10101010

10101010

10101010

10101010

10101010

10101010

M X 2MX 16161616

36363636

16161616

4444

0000

64646464

2MX -4-4-4-4

-6-6-6-6

4444

-2-2-2-2

0000

8888

MX MX M

Next, we answer the questions about each factor having an overall effect.

• To get proper between groups mean squares we have to divide the sums of squares and df between groups into components for factor 1, factor 2, and the interaction.

• Let’s just look at factor 1. Our question about factor 1 was “Do people undergoing different therapies have differential responses to any task?”

• We can group participants into all those who were treated with CBT and those treated with Social Support.

Forming Groups that Differ only on Factor 1

• Pretend that the experiment was a simple, single factor experiment in which the only difference among the groups was the first factor (that is, the type of therapy given each group). Create groups reflecting only differences on Factor 1.

• So, when computing the main effect of Factor 1 (type of psychotherapy), ignore Factor 2 (type of drug). Divide participants into two groups depending solely on whether they were given CBT or Social Support. Then, find the mean of each of the two combined groups (CBT and Social Support).

Computing SS for Factor 1

Next, find the deviation of the mean of the CBT participants from the overall mean. Then sum and square those differences.

Then, find the deviation of the mean of the Social Support participants from the overall mean. Then sum and square those differences.

The total of the summed and squared deviations the mean of each of the combined groups from M, the overall mean, is the sum of squares for Factor 1. (SSIV1).

dfIV1 and MSIV1

• Compute a mean square that takes only differences on Factor 1 into account by dividing SSIV1 by dfIV1.

• For example, in this experiment, type of therapy was either CBT or Social Support. The two ways participants are treated are called the two “levels” of Factor 1.

• dfIV1= L1 – 1 = 2-1 = 1 [where L1 equals the number of levels (or different variations) of the first factor (IV1)].

SSIV1: Main Effectof Therapy Type

Type of drug

TherapyType


SoSp

CBT


SoSpX

CBTXCBTAX


CBTZX CBTPX

Compare each score’s therapy mean to the overall mean.

MX

M

Means for GAD study

Type of New Drug

Diet Type


Social Support

Cognitive BT 6 4

10

14

8 18

00.10M16.007.007.00

12.00

8.00

CBT/SoSp Means, M=10.00Means: CBT=8.00; SoSp=12.00

CBT1.11.21.31.42.12.22.32.43.13.23.33.4

#SSoSp4.14.24.34.45.15.25.35.46.16.26.36.4

group888888888888

group121212121212121212121212

X #S X

12n 12n

888888888888

X

SSIV1=96.00; dfIV1=2-1=1;MSIV1=96.00

101010101010101010101010

-2-2-2-2-2-2-2-2-2-2-2-2

2MX MX 444444444444

M X 2MX MX M121212121212121212121212

101010101010101010101010

222222222222

444444444444

Factor 2

Then pretend that the experiment was a single experiment with only the second factor.

Proceed as you just did for

Factor 1 and obtain SSIV2 and MSIV2 where dfIV2=LIV2 – 1=3-1=2.

SSIV1: Main Effectof Drug Type

Drug Type

TherapyType


SoSp

CBT


SoSpX

CBTXCBTAX


CBTZX CBTPX

Compare each score’sDrug Type mean

to the overall mean.

MX

M

Means for GAD study

Type of New Drug

Diet Type


Social Support

Cognitive BT 6 4

10

14

8 18

00.10M16.007.007.00

12.00

8.00

Means: Ativan=7.00, Zoloft=7.00, Placebo = 16.00, M=10.00

1.11.21.31.44.14.24.34.4

3.13.23.33.46.16.26.36.4

77777777

n=8

1616161616161616

n=8

2.12.22.32.45.15.25.35.4

77777777

n=8

#S X #S X #S X

SSIV2=442; dfIV2=2; MSIV2=221

Ativan1.11.21.31.44.14.24.34.4

#S

.77777777

X

1010101010101010

2MX MX M

-3-3-3-3-3-3-3-3

99999999

No3.13.23.33.46.16.26.36.4

Emb.1616161616161616

1010101010101010

66666666

3636363636363636

Mild2.12.22.32.45.15.25.35.4

Emb.77777775

1010101010101010

-3-3-3-3-3-3-3-3

99999999

21311 FE Ldf

#S X 2MX MX M

#S X 2MX MX M

Computing the sum of squares and df for the interaction.

• SSB contains all the possible effects of the independent variables in addition to the random factors, ID and MP. Here is that statement in equation form

• SSB= SSIV1 + SSIV2 + SSINT

• Rearranging the terms:

SSINT = SSB - (SSIV1+SSIV2) or SSINT = SSB- SSIV1-SSIV2

SSINT is what’s left from the sum of squares between groups (SSB) when the main effects of the two IVs are accounted for.

So, subtract SSIV1 and SSIV2 from SSB to obtain the sum of squares for the interaction (SSINT).

Then, subtract dfIV1 and dfIV2 from dfB to obtain dfINT).

Means Squares for InteractionMSINT

REARRANGE

644296544 nInteractioSS

nInteractioSS 44296544

6nInteractioSS

DifficultyentEmbarrassmBnInteractio dfdfdfdf

00.32/00.6 nInteractioMS

2125 2

Testing 3 null hypotheses in the two way factorial Anova

Hypotheses for Therapy Type

• Null Hypothesis - H0: There is no effect of Drug Type. The means for anxiety will be the same whether the GAD patients were given CBT or SocialSupport.

• Experimental Hypothesis - H1: The type of psychotherapy a group gets, considered alone, will affect their anxiety.

Hypotheses for Type of Drug

• Null Hypothesis - H0: The three drug types, including the placebo, will have similar effects on reported anxiety.

• Experimental Hypothesis - H1: The three drug types, including the placebo, will have different effects on reported anxiety.

Hypotheses for the Interaction of Type of Therapy and Type of Drug

• Null Hypothesis - H0: There is no interaction effect. Once you take into account the main effects of type of therapy and type of drug, there will be no differences among the groups that can not be accounted for by sampling fluctuation.

• Experimental Hypothesis - H1: There are effects of combining CBT and Social Support with specific drugs can not be predicted from either IV considered alone.

Computational steps• Outline the experiment.

• Define the null and experimental hypotheses.

• Compute the Mean Squares within groups.

• Compute the Sum of Squares between groups.

• Compute the main effects.

• Compute the interaction.

• Set up the ANOVA table.

• Check the F table for significance.

• Interpret the results.

Steps so far• Outline the experiment.• Define the null and experimental hypotheses.• Compute the Mean Squares within groups.• Compute the Sum of Squares between groups.• Compute the main effects.• Compute the interaction.

What we know to this point

• SSIV1=96.00, dfIV1=1; MSIV1=96.00

• SSIV2=442.00, dfIV2=2; MSIV2=221.00

• SSINT=6.00, dfINT=2; MSINT=3.00

• SSW=132.00, dfW=18; MSW=7.33

Steps remaining

• Set up the ANOVA table.• Check the F table for significance.• Interpret the results.

Therapy Type – F(1,18)=13.10, p<.01; Drug Type – F(2,18)=30.14, p<.01

Interaction-F(2,18)=0.41, n.s.

Type of Therapy

Type of Drug

132 18 7.33

96.00 1 96.00 13.10 .01

SS df MS F p

Interaction

Error

442 2 221.00 30.14 .01

6 2 3.00 0.41 n.s.

55.305. 01.601.

State Results

• The main effects for CBT vs. Social Support was significant– F(1,18)=13.10, p<.01.

• The main effect for Ativan vs. Zoloft vs. Placebo was significant - F(2,18)=30.14, p<.01

• The interaction was not significant (F(2,18)=0.41, n.s.)

Interpret Significant Results

• CBT participants were less anxious than were those who received Social Support.

• Participants who received the two active medications (Ativan and Zoloft had the same mean anxiety ratings after treatment, while those who received the placebo were much more anxious.

Describe pattern of means.

Interpret Significant Results

• These findings are consistent an additive effect of appropriate medication and CBT for Generalized Anxiety Disorder. Both lower anxiety among these patients.

Reconcile statistical findingswith the hypotheses.

chapter 10 - part 1 factorial experiments. nomenclature we will use the terms factor and independent...

Documents