lecture 9: experimental methods

Lecture 9: Experimental Methods

Research Design

Gale M. Lucas, PhD

…or “How to run a study”

Schacter & Singer• Claim we don't automatically know when we are

happy, angry. Instead, we label our emotions by considering situational cues. This labeling process depends on two factors:– something in world triggers general, nonspecific

arousal marked by increased heart rate, tightening of the stomach, and rapid breathing.

– people examine perceptual cues and their thoughts about the situation for clues for what has caused the emotion.

• Essentially 2 factor model– Appraisal a purely cognitive process that provides

context for interpreting body state

Overview of “how to run a study”

• Choose your variables, make them concrete

– IV, DV

• Choose your method

– Correlational or experimental

• Analyze data

– Depends on “type” of measurement

Choose your variables, make

them concrete• What do you want to study?

• Variables (or constructs)

• Operational definitions

– A clear, measurable definition of a construct

Our example

• Belonging is a fundamental need, like hunger

• But people feel need to different degrees…

Construct: Need to belong (NTB)

Our example

• Operational definition: Need to Belong

(NTB) Scale (Leary et al., 2001)

• High scores on items 2, 4, 5, 6, 8, 9, 10

• Low scores on items 1, 3, 7

Practice

• Come up with operational definitions for each of the following:

– Trust

– Aggression

– Depression

Choose your variables, make

them concrete• What do you want to study?

• Variables (or constructs)

• Operational definitions

– A clear, measurable definition of a construct

– Specify operational definition for IV and DV

IVs and DVs

• IV: Independent Variable (“manipulated”)

– Variable that you think causes the effect on

another variable

– Variable that is manipulated in an experiment

• DV: Dependent Variable (“measured”)

– The outcome variable (you think it has effect on)

– Variable measured in an experiment

Warning: some

foreshadowing



– IV, DV



– Depends on your question

?

Our example

• What would that hunger make us do?

– And socially sensitive/aware

– Could ask two questions here

Our example

• “Is NTB associated with social sensitivity?”

• “Does NTB cause heightened sensitivity?”

Correlational

Experimental

NTB Sensitivity

NTB Sensitivity

Correlational vs experimental

• Recall, there was something different

about IVs and DVs in experimental…?

• Correlational

– Measure IV and measure DV

– Our example?

• Experimental

– Manipulate IV and measure DV

– Our example?

?

Is NTB associated with sensitivity?

• What kind of research tests this question?

• Correlational research

• Picket, Gardner, and Knowles

– NTB scale

– DANVA: Number of facial expressions correctly

identified

r = .38What does that mean?

Correlation Coefficient -- r

Correlation

coefficient

Indicates direction

of relationship

(positive or negative)

Indicates strength

of relationship

(0.00 to 1.00)

r = 0.63+r = +0.50

Problems with correlational research

• Only shows there’s an association

• We can’t say NTB causes social sensitivity,

why not…?

• It could be some other variable causing both

– Third variable problem

• Sensitivity could cause NTB

– Directionality problem

NTB Sensitivity

NTB Sensitivity

Third Variable


• Third variable problem

NTBSocial

sensitivity

Motivation

Please

experimenter

Else?


• Directionality problem

NTB Social

sensitivity


• …and there will always another third variable

that you didn’t think of!

• So, conduct experimental research!

– What makes an experiment? What do you have to

do?

– Random assignment to conditions that only

differ on (thus manipulate) one variable

• Our example?Internal

validity!

Experimental NTB research

• Picket, Gardner, and Knowles

– What did they have to do here?

– Manipulate belonging needs

• Measure social sensitivity

Experimental NTB research

• NTB causes an increase in social sensitivity!

0

20

40

60

80

100

120

Control Rejection

So

cia

l s

en

sit

ivit

y

Benefit of experimental research

• Experimental research wins!

• Right?

• Do we only care about outcomes these

kinds of manipulations can effect?

• What important outcomes could NTB

effect? Think big! How might hunger affect

us?

Benefits of correlational research

• Real world outcomes

that really matter

0

0.05

0.1

0.15

0.2

Low High

Males

Females

Age-a

dju

ste

d m

ort

alit

y r

ate

Belonging

External

validity!

Problems with experimental research

• Hard to make manipulations powerful enough

to impact these kinds of outcomes

• Would you really want to?

– Ethical operationalizations?

– Ethically affect those outcomes?

• Even if didn’t have ethics….

– Can’t practically manipulate some things



– IV, DV



• Analyze data


Our example

• Please access the link that was emailed to

you and complete the study…

– ---once you are done, you can get up and stretch

• What did you experience?

• What kind of study was this?

• What am I testing? And why these 3 groups?

Data has been collected…

From the raw data it is impossible to tell if the groups differ!

Population

of

students

Sample

Control Life alone

4 5 6 1 3 6

2 7 7 3 2 7

4 6 7 2 5 5

2 3 7 5 7 3

7 6 7 2 3 7

6 1 7 4 2 5

3 1 2 2 2 4

1 1 3 4 2 5

6 1 4 6 5 1

2 7 7 2 7 2

2 7 2 7 1 3

4 7 6 7 5 4

7 2 4 6 4 7

3 6 5 6 3 4

1 7 7 3 5 5

Accident prone

…so run a statistical test

• Which one?

• It depends. …. On whether you’ve run a

correlational or experimental study, right?

– Calculate a correlation coefficient, r, if we ran

a correlational study, right?

– WRONG!

• It depends… on “type” of IV and DV



– IV, DV



• Analyze data


“Type” of IVs and DVs

• Both IV and DV can either be “categorical”

or “continuous”

– “groups” vs “how much?”

• Think continuous, think “continuum”

___________________

1 2 3 4 5 6 7

– Ask yourself: “can I put these in order?”

Continuous IV

(e.g., how much)

Categorical IV

(e.g, exp vs control)

Continuous

DV (e.g.,

how much)

Categorical

DV (e.g.,

yes/no)

Continuous IV

(e.g., how much)

Categorical IV


Continuous

DV (e.g.,

how much)t-test

Categorical

DV (e.g.,

yes/no)

Continuous IV

(e.g., how much)

Categorical IV


Continuous

DV (e.g.,

how much)

correlation t-test

Categorical

DV (e.g.,

yes/no)

Continuous IV

(e.g., how much)

Categorical IV


Continuous

DV (e.g.,

how much)

correlation t-test or ANOVA

Categorical

DV (e.g.,

yes/no)

Continuous IV

(e.g., how much)

Categorical IV


Continuous

DV (e.g.,

how much)

correlation or

regressiont-test or ANOVA

Categorical

DV (e.g.,

yes/no)

Continuous IV

(e.g., how much)

Categorical IV


Continuous

DV (e.g.,

how much)

correlation or


Categorical

DV (e.g.,

yes/no)

chi-square test

Continuous IV

(e.g., how much)

Categorical IV


Continuous

DV (e.g.,

how much)

correlation or


Categorical

DV (e.g.,

yes/no)

logistical

regression

chi-square test

What test would you use?

• Let’s go back:

– Recall Picket, Gardner, and Knowles’ r = .38

– Was that correct? Why?

– What test here?

0

20

40

60

80

100

120

Control Rejection

So

cia

l s

en

sit

ivit

y


• Practice question:

– I ran a study to test whether people with high

need to belong smile more

– What do you need to know to figure out what

test I should use?

– IV: NTB scores

– DV: count of smiles during 5 minute

conversation

– If you had high vs low NTB?

– If you had high, med, low NTB?

– Are these last 2 experimental or correlational?


• Practice question:

– I ran a study to test whether people with high

need to belong smile more

– What do you need to know to figure out what

test I should use?

– IV: NTB scores

– DV: smile or not in photo

– If you had high vs low NTB?


• …to analyze our data?• ---while I get the data ready, practice & then take a break

• Let’s run that test!

• ...what if I had only randomly assigned you to

“live alone” vs “acceptance” conditions?

• …what if I had you provide your NTB scores

instead of assigning you to conditions?

Could we publish our data?

• Write down: at least one reason why we

could NOT publish our results?

• Lacking IRB approval & informed consent

• Analyze each item separately increases

chance of “false positive”

• Concerns about our sample

– Too small “N” (sample size)

– Findings may not generalize

– Demand characteristics

Summary of “how to experiment”


– IV, DV



• Analyze data


Recap

• Which should you choose when?

• Correlational research

– Important, “hard to manipulate” real-life outcomes

– Could not ethically or practically manipulate

• Experimental research

– Make causal claims

– Can manipulate cleanly

Variable Outcome Internal

validity!

• What does it mean to “cleanly” manipulate…?

• Hard to create conditions that only differ on one variable

– Why is it hard? What have you noticed about manipulations?

– …but its important to try…. Why?

Challenge in experimental research

Internal

validity!

• If conditions differ on more than one variable

– Those other differences are potential threats to internal validity

– what things differ between rejected & walk to school other than increased need to belong?

• Pain

• Negativity

• Mood

• Anxiety

– These could be alternative explanations• If rejected were more socially sensitive, could be….

Challenge in experimental research

Solutions in experimental research

• But, are ways to help ensure internal validity

• 1. Control condition similar to experimental

• The conditions are very much alike, helps to control

for some factors (alone vs with others, anxiety, etc)

– But still might differ on more than rejection: pain,

negativity, mood…


• 2. Other additional conditions that manipulate

these confounds– Pain

– Negativity

– If they don’t increase social sensitivity

but rejection condition does…

– Mood?

– Anxiety?


• 3. Measure alternative explanation– Pain

– Negativity

– Mood

– Anxiety

– If the rejection condition doesn’t increase these

compared to control…

– Even if it does, can control for them statistically,

and if still find a difference in social sensitivity…


• 4. Measure variable intended to manipulate

– If experimental condition is higher on that

variable…

– Manipulation check

– Mediation analysis

0

1

2

3

4

5

6

7

Control Rejected

Be

lon

gin

g n

ee

ds

Rejection

manipulationBelonging needs Sensitivity


• 5. Different manipulations across studies

– If always find difference…

– Multiple ways to increase NTB

• Reliving task

• Cyberball

• Living alone

• Chatroom

• More!

Convergent

validity!


• 6. Combine with other ways of asking the

question

• Does NTB cause heighten social sensitivity?

• A) NTB scores associated with sensitivity

• B) Heighten need (starve) increases sensitivity

• C) Reduce need (feed) decreases sensitivity?

Convergent

validity!

And, if its like hunger, you could also…?

Solutions in experimental research• Lucas et al.

• Priming acceptance to “satiate”– Acceptance - e.g., together went us they there

– Control - e.g., pretty was chair the skiing

• Social sensitivity

0

1

2

3

4

5

6

7

Acceptance Control

So

cia

l s

en

sit

ivit

y

Questions?

SparkNotes reading for today

• Its either available

online for free

(http://www.sparkn

otes.com/psycholo

gy/psych101/resea

rchmethods/),

where you have to

click through all the

sections yourself

• or download easier

-to-read (for < $5)

• http://www.barnesa

ndnoble.com/w/res

earch-methods-in-

psychology-

sparknotes-

editors/101822024

3?ean=978141149

1663

http://www.sparknotes.com/psychology/psych101/researchmethods/

http://www.barnesandnoble.com/w/research-methods-in-psychology-sparknotes-editors/1018220243?ean=9781411491663

lecture 9: experimental methods

Documents