intro to factorial designs the importance of “conditional” & non-additive effects the...

31
Intro to Factorial Designs The importance of “conditional” & non- additive effects The structure, variables and effects of a factorial design 5 terms necessary to understand factorial designs 5 patterns of factorial results for a 2x2 factorial designs Thinking about the meaningfulness of main effects Descriptive & misleading main effects

Upload: alisha-edwards

Post on 03-Jan-2016

216 views

Category:

Documents


1 download

TRANSCRIPT

Intro to Factorial Designs

• The importance of “conditional” & non-additive effects

• The structure, variables and effects of a factorial design

• 5 terms necessary to understand factorial designs

• 5 patterns of factorial results for a 2x2 factorial designs

• Thinking about the meaningfulness of main effects

• Descriptive & misleading main effects

The importance of “conditional” & “non-additive” effects…

Brownies – great things… worthy of serious theory & research!!!

The usual brownie is made with 4 blocks of chocolate and 2 cups of sugar. Replicated research tells us that the average rating of brownies made with this recipe is about 3 on a 10-point scale.

My theory? People don’t really like brownies! What they really like is fudge! So, goes my theory, making brownies more “fudge-like” will make them better liked.

How to make them more fudge-like, you ask?

Add more sugar & more chocolate!!!

So, we made up several batches of brownies and asked people to taste a standardized amount of brownie after rinsing their mouth with water, eating an unsalted saltine cracker and rinsing their mouth a second time. We used the same 10-point rating scale; 1 = this is the worst plain brownie I’ve ever had, 10=this is the best plain brownie I’ve ever had.

Our first study:

2-cups of sugar 4-cups of sugar

3 5

So, far so good!

Our second study:

4 blocks of choc.

3 2

8 blocks of choc.

What???? Oh – yeah! Unsweetened chocolate…

Then the argument started..

One side: We have partial support for the theory – addingsugar helps, but adding chocolate hurts!!!

Other side: We have not tested the theory!!!

What was our theory?

Add more sugar & more chocolate!!! We need a better design!

4 blocks of choc.

3 2

8 blocks of choc.

2-cups of sugar

4-cups of sugar

5

What do we expect for the 4-cup & 8-block brownies?

standard brownie

+ sugar effect

+ chocolate effect

expected additive effect of choc & sugar

expected score for 4&8 brownies

3

+ 2

- 1

1

4

4 blocks of choc.

3 2

8 blocks of choc.

2-cups of sugar

4-cups of sugar 5

How do we account for this ?

9

There is a non-additive joint effect of chocolate and sugar!!!!

The joint effect of adding chocolate and sugar is not predictable as the sum of the effects of adding each!!!

Said differently, there is an interaction of chocolate and sugar that emerges when they are added simultaneously.

The effect of adding both simultaneously is 6 … not 1???

This leads to the distinction between two “kinds” of interactions…

“Augmenting” Interaction

10

# practices10 30

~FB

FB 20 45

15

The combined effect is greater than would be

expected as the additive effect!

“Interfering” Interaction

10~Aud

Aud

~Rew Rew

25 15

20

The combined effect is less than would be expected as

the additive effect!

Practice effect = 5Feedback effect = 10Expected additive effect = 15Joint effect = 35

“Augmenting” Interaction

45

Reward effect = 10Audience effect = 15Expected additive effect = 25Joint effect = 5

“Interfering” Interaction“Interfering” Interaction“Interfering” Interaction

Introduction to factorial designs

Factorial designs have 2 (or more) Independent Variables

An Example…Forty clients at a local clinic volunteered to participate in a research project designed to examine the individual and combined effects of the client’s Initial Diagnosis (either general anxiety or social anxiety) and the Type of Therapy they receive (either group or individual). Twenty of the participants had been diagnosed with general anxiety and 20 had been diagnosed as having social anxiety. One-half of the clients with each diagnosis were assigned to receive group therapy and one-half received individual therapy. All clients underwent 6 months of the prescribed treatment, and then completed a battery of assessments which were combined into a DV score of “wellness from anxiety”, for which larger scores indicate better outcome. Here is a depiction of this design.

Type of Therapy

Initial Diagnosis Group Individual

General clients diagnosed w/ clients diagnosed w/ Anxiety general anxiety who general anxiety who

received group therapy received individual therapy

Social clients diagnosed w/ clients diagnosed w/ Anxiety social anxiety who social anxiety who

received group therapy received individual therapy

Participants in each “cell” of this design have a unique combination of IV conditions.

Showing this design is a 2x2 Factorial

What’s involved in a 2x2 factorial design ?

There are 3 variables examined …1-- the DV (dependent, outcome, response, measured, etc. variable)

2 -- one IV (independent, treatment, manipulated, grouping, etc. variable)

3 – second IV (independent, treatment, manipulated, grouping, etc. variable)

There are 3 effects examined …

1 -- the main effect of the one IV -- how it relates to the DV independently of the interaction and the other main effect

2 -- the main effect of the other IV -- how it relates to the DV independently of the interaction and the other main effect

3 -- the interaction of the two IVs -- how they jointly relate to DV

For the example…

1 -- the “main effect” of Initial Diagnosis2 -- the “main effect” of Type of Therapy3 -- the “interaction” of Initial Diagnosis & Type of Therapy

The difficult part of learning about factorial designs is the large set of new terms that must be acquired. Here’s a summary;;

cell means -- the mean DV score of all the folks with a particular combination of IV treatments

marginal means -- the mean DV score of all the folks in a particular condition of the specified IV (aggregated across conditions of the other IV)

Main effects involve the comparison of marginal means.

Simple effects involve the comparison of cell means.

Interactions involve the comparison of simple effects.

• An interaction is defined as “different simple effects”

• when the simple effects of one variable are different in direction and/or size across the conditions of the other variable

Identifying Cell Means and Marginal Means

Type of Therapy

Initial Diagnosis Group Individual

General Anxiety 50 50 50

Social 90 10 50 Anxiety

70 30

Cell means mean DV of subjects in a design cell

Marginal means average mean DV of all subjects in one condition of an IV

Identifying Main Effects -- difference between the marginal meansof that IV (ignoring the other IV)

Type of TherapyInitial Diagnosis Group Individual

General Anxiety 50 50 50

Social Anxiety 90 10 50

70 30

Main effect of Initial Diagnosis

Main effect of Type of Therapy

Identifying Simple Effects -- cell means differences between conditions of one IV for a specific level of the other IV

Type of TherapyInitial Diagnosis Group Individual

General Anxiety 50 50 1

Social Anxiety 90 10 2

a b

Simple effects of Initial Diagnosis for each Type of Therapy

a Simple effect of Initial Diagnosis for group therapy

b Simple effect of Initial Diagnosis for individual therapy

Identifying Simple Effects -- cell means differences between conditions of one IV for a specific level of the other IV

Type of TherapyInitial Diagnosis Group Individual

General Anxiety 50 50 1

Social Anxiety 90 10 2

a b

Simple effects of Type of Therapy for each Initial Diagnosis

1 Simple effect of Type of Therapy for general anxiety patients

2 Simple effect of Type of Therapy for social anxiety patients

#1 Task Presentation

Paper ComputerTask Difficulty

Easy 90 > 70 simple effects are

Hard 40 < 60 opposite directions

There is an interaction of Task Presentation and Task Difficulty as they relate to performance. Easy tasks are performed better using paper than using computer (90 vs. 70), whereas hard tasks are performed better using the computer than using paper (60 vs. 40).

Here are the three basic patterns of interactions

#2 Task Presentation Paper Computer

Task Difficulty

Easy 90 = 90 one simple effect “null”

Hard 40 < 70 one simple effect

There is an interaction of Task Presentation and Task Difficulty as they relate to performance. Easy tasks are performed equally well using paper and using the computer (90 vs. 90), however, hard tasks are performed better using the computer than using paper (70 vs. 40).

#3 Task Presentation

Paper ComputerTask Difficulty

Easy 80 < 90 simple effects in the same

direction,

Hard 40 < 70 but of different sizes

There is an interaction of Task Presentation and Task Difficulty as they relate to performance. Performance was better using the computer than using paper, however this effect was larger for hard tasks (70 vs. 40) than for easy tasks (90 vs. 80).

Here are the two basic patterns of NON-interactions

#1 Task Presentation

Paper ComputerTask Difficulty

Easy 30 < 50 both simple effects are in thesame direction and are

Hard 50 < 70 the same size

There is no interaction of Task Presentation and Task Difficulty as they relate to performance. Performance is better for computer than for paper presentations (for both Easy and Hard tasks).

#2 Task Presentation

Paper ComputerTask Difficulty

Easy 50 = 50 both simple effects

Hard 70 = 70 are nulls

There is no interaction of Task Presentation and Task Difficulty as they relate to performance. Performance is the same for computer and paper presentations (for both Easy and Hard tasks).

So, there are 5 basic patterns of results from a 2x2 Factorial

Three patterns that have an interaction:

1. < vs. > simple effects in opposite directions

2. = vs. < one null simple effect and one simple effect

3. < vs. < simple effects in same direction, but different sizes

Two patterns that have no interaction:

4. < vs. < simple effects of the same size in the same direction

5. = vs. = both null simple effects

Identifying Main Effects

Patterns of data that include main effects can be identified by looking at the differences among the marginal means for a specific IV (the main effect of each IV must be examined and described separately !!!)

• When there is an interaction, each main effect (null or significant) must be carefully examined to determine if that main effect is

•“descriptive” (unconditional, that is, descriptive for all levels of the other IV) or is

• “potentially misleading (conditional, that is, descriptive for only some or none of the levels of the other IV)

• You must determine whether the pattern of each main effect (direction of any difference between the marginal means) is equivalent to each of the corresponding simple effects of that variable at the various levels of the other IV

Importance or Main Effects ??

It is not uncommon to hear the advice to “ignore main effects if there is an interaction.”

My best guess is that this is based on the correct idea that the pattern of some interactions can render the pattern of one or both main effects to be potentially or completely misleading.

However, it is also possible that there can be an interaction and that one or both of the main effects can be descriptive.

Discerning whether main effects are descriptive or misleading is a critical step in the examination of data from a factorial design! You must ensure that the reader has a thorough understanding of the pattern of your data!

You must give a complete accounting of each of the three effects involved in the factorial design, the interaction and each of the main effects!

Interpreting main effects … When there is an interaction, the pattern of the interaction may influence the interpretability (generality) of the description of the marginal means.

Task Presentation Paper Computer

Task Difficulty There is a main effect for Easy 80 < 90 Task Presentation, overall

performance was better using computer presenta-

Hard 40 < 70 tion than using paper presentation.

60 < 80

Notice: that the pattern of the main effect is consistent with both the simple effect of Task Presentation for easy tasks and the simple effect of Task Presentation for hard tasks.

Another example …

Task Presentation Paper Computer

Task Difficulty

Easy 90 = 90

Hard 40 < 70

65 < 80

There is a main effect for Task Presentation, overall performance was better using computer presentation than using paper presentation. However, while this pattern is descriptive for hard tasks, it is not descriptive for easy tasks, for which there was no simple effect of Task Presentation.

Yet another example …

Task Presentation Paper Computer

Task Difficulty

Easy 80 > 60

Hard 20 < 70

50 < 65

There is a main effect for Task Presentation, overall performance was better using computer presentation than using paper presentation. However, while this pattern is descriptive for hard tasks, it is not for easy tasks, for which performance was better using paper presentations than using computer presentation.

“Null” main effects can also be misleading….

Task Presentation Paper ComputerTask Difficulty

Easy 90 > 70

Hard 40 < 60

65 = 65

There is no main effect for Task Presentation, overall performance was equivalent using computer presentation and using paper presentation. However, this pattern is descriptive for neither hard tasks, for which computer presentations worked better than paper, nor for easy tasks, for which performance was better using paper presentations than using computer presentation.

There is no main effect for Task Presentation, overall performance was the same using computer and paper presentation.

1. < vs. > simple effects in opposite directions

2. = vs. < one null simple effect and one simple effect

3. < vs. < simple effects in same direction, but different

sizes

4. < vs. < simple effects of the same size in the same direction

5. = vs. = both null simple effects

Another look at the 5 basic patterns of results from a 2x2 Factorial – thinking about how interaction pattern relates to meaningfulness of main effects

Interaction

-- simple effects of different size and/or direction

Misleading main effects

Descriptive main effects

No Interaction

-- simple effects are null or same size

Related to this is the very important issue of whether or not the main effects “mean anything to us” ???

It all goes back to “representation & inference” !!!

Remember – the purpose of any design condition is to represent some population so we can infer that the difference between those conditions or values in the design represent differences between the populations we really care about!

The “cells” in the 2-way each represent a specific population and so, comparisons between them are comparisons between our target populations.

But the marginal means examined by the main effects are “aggregates” – who do they represent???

Here’s an example to help to consider this…

We know what population is represented by each of the four cell means!

What about the marginal mean for “Paper Presentation” the aggregate of Easy & Hard Difficulty…

65

Does it represent “any difficulty”… “medium difficulty” ???

What about the marginal mean “Hard Task Difficulty”???

What population is represented by the aggregate of Paper & Computer Task Presentations????

55

Another way that Main effects can be “meaningless”…

Age 5 25 Sex

Female 90 70 80

Male 80 60 70

85 65

There is no interaction, so the main effects are “unconditional”.

But are they “meaningful” ???

Consider the Sex ME – those marginal means are aggregated across 5 & 25 year olds. Who are represented 15 year olds? Not unless there is a linear relationship between age and the DV, which we’ve certainly not tested for !!!

Consider the Age ME – those marginal means are aggregated across males and females. Who is the average of females and males?

Main effects often don’t represent any existing population. So, ME patterns are most useful if they describe SE patterns !!!