1

Chapter 13: Interpreting Research Results

• Describing Results

• Inferences in Behavioral Science Research

• Null Results

• Integrating Results of Research

• Summary

2

Describing Results

• Nature of relationships– Types of Relationships

• Linear v. Curvilinear

• Mediators and Moderators (partial corr or MR)

• Interaction (factorial experiments)

– Predicted and Observed Relationships• Cf results (observed) to expected (hypothesis)

– Table 13-1, p. 428, Table 13-4, p. 429

3

• Real v. Chance Relationships– Inferential Stats (what alpha level? Why p <.05?)

• Type I and Type II error trade offs

– Testing the Proper Statistical Hypothesis• Multiple tests (what effect on alpha level?)• Omnibus (MANOVA) v. Planned comparisons

– What’s the benefit of Planned comparisons?

• Effect Size and Importance of Effect Size– Effect size (always include effect size)

• http://web.uccs.edu/lbecker/Psy590/es.htm#II.%20independent• Pearson r; Cohen’s d

– Practical Significance (small, medium, large?)

4

Effect size

• When effect size is small)– Prentice & Miller (’92) – “Minimal group effect” (Tajefel et al. ’71)– What is important about this small effect size?

• Weak manipulation -> any effect, important

– Hatfield & Sprecther, (’86) – physical attractiveness– What is important about this small effect size?

• When everything else is equal, it may play an important role

5

Practical SignificanceSmall effect sizes

• Clinical significance (a value judgment)– Abelson (’85) skill and batting average (r = .06)

• Important over a whole season

• Fishbein & Ajzen (’75) religiosity and religious behavior

• Small effect size and large populations– Framington study (Rosenthal & Rosnow, ’91)

• Asprin and avoid heart attack (r = .03)

• Population of 750k people = decrease of 3.4% heart attack rate

• Theory testing v. Applied research– Which is effect size more important for? (Chow, ’88)

• Applied research

6

Inference in Behavioral Science Research

• Knowledge as a Social Construction– Constructionist viewpoint

• Do we build our own reality? Or• Is logical positivism a real possibility?

– How do we view the cause of racial prejudice now?• What zeitgeist are we in now?

– Blank slate? Or biological evolution (cognitive)?

• Bias in Interpreting Data– Theoretical bias (e.g. Mony & Ehrhardt, ’72)

• Which interpretation is correct?

– E.O. Wilson (’78) sociobiologist or– Mackie (’83) cultural influence to explain results

7

Inferences: Bias

• Personal Bias (tenacity)– Sherwood & Nataupsky (’68) study of 82 psychologists’ beliefs

about racial differences in IQ• Environmentalists

• Hereditarians

• Middle-of-the-roaders (inconclusive)

– Statistical sig differences (Bias shows up)• Larry Summers (What happened to him? Why?)

– Assuming group differences are biological / environmental• Correlational data make it hard to decide

– “Victim blame” (look beyond the group for theory)– Behavior labeling (aggressive v. assertive)

8

Inferences:Making Valid Ones

• Measurement and Statistics– Know the level of measure– Recognize the “fallacy of the mean”

• E.g. distributions overlap

– State correlational results and group means appropriately• Corr: state direction and strength

– E.g. “positively related” – “high scores on X were associated with high scores on

Y”• Group means:

– “mean for group A was significantly higher than the mean for group B”

– Don’t forget to show group means (ANOVA table doesn’t)

9

Valid Inferences

• Empiricism– Stay close to the actual statistical findings, don’t speculate until

the discussion– Clarify (or qualify) the relationship between the hypothetical

construct and op definition• E.g. how is race (hypothetical construct) defined operationally?

– Describe, avoid unwarranted evalutations• E.g. do women underestimate the credit they deserve or do men

overestimate? (you know the truth!)

• Causality– Don’t infer causality from correlational findings

• Generalization– Theory and or findings

10

Inferences:3 Uses of the Null

& Prejudice against Null

• Testing hypotheses• Research validity • Testing generalizability

• Null findings don’t get published (despite the fact they may be well done)

– If the null is, in fact true, What does this imply about the published studies?

• They may be Type I errors!

• Researchers unlikely to test the null directly– Why?

11

Possible Sources of type II Errors

• IV– Construct valid?

– Manipulation effective?

– strong enough?

• DV– Construct valid?

– Sensitive enough?

– Unrestricted range?

• Design– Curvilinear relationship? (inspect the distribution)

– Extraneous vars controlled?

– Moderators or mediators operating?

– Large enough sample (power test)

12

Accepting the Null

• Common criteria– Proper design and Sufficient power

• Predicted null results– Based on good theory

• Unexpected null results– Theory could be wrong! (believe it or not)– Suppose it is a Type II error?

• Cold Fusion: Another chance. Does theory matter? Cost of Type II error

13

Integrating Results

• Identifying Implications for Theory– Comparison with prior research– Comparison with theoretical prediction

• Identifying Implications for Research– Research procedures– New research questions

• Identifying Implications for Application

14

Chapter 13: Interpreting Research ResultsSummary

• Describing Results

• Inferences in Behavioral Science Research

• Null Results

• Integrating Results of Research

• Summary