Lecture 11: Modes of Measurement

How Should We Measure Things?

S-data: Self-Reports or Self-Judgments I-data: Informant Reports (People who know the

target) B-data: Behavioral Data (Naturalistic Observation,

Laboratory Measures, Certain Personality Tests like the TAT, Physiological Recordings)

L-data: Life Data (e.g., Court Reports, School Records)

Hint: BLIS Source: Funder (2007)

Self-Reports (S-Data)

Structured: a standardized list of questions and set way of responding is given to all participants

Open-Ended (e.g., How do you act in social situations?)

When using self-reports the psychologist believes the answer (more or less)

Aggression: S-DataBuss & Perry (1992) Aggression Questionnaire

Rate each item from 1 (“Extremely uncharacteristic of me”) to 5 (“Extremely characteristic of me”)– Given enough provocation, I may hit another

person.– If somebody hits me, I hit back.– I have become so mad that I have broken things.– I get into fights a little more than the average

person. Physical aggression scale correlated .33 with the

number of minutes hockey players spent in the penalty box for aggressive penalties (Bushman & Wells, 1998).

Comments on S-Data

Nothing tricky about this approach. Can get self-reports using paper surveys, phone

calls, the internet, or via face to face interviews Each of these specific methods has particular

advantages and disadvantages. (See Textbook) General Disadvantages

– Will people tell you the truth?– Can people even tell you the truth?– Do psychologists use S-Data too much?– Do we overwhelm our research participants?

One Relatively Big Problem …

Sometimes people interpret the meaning of questions differently than researchers intend…

Would You Say You “Had Sex” If …

Source: Sanders & Reinisch (1999) - JAMA Sample: 599 Undergraduates from Midwestern

Universities (59% Female) Average Age: 20.7 (SD = 3.1) 79% described themselves as “Moderate to

Conservative” Question: “Would you say you ‘had sex’ with

someone if the most intimate behavior you engaged in was ....”

Selected Results

99.5%99.2%99.7%P-V Intercourse

40.2%43.9%37.7%Oral Sex (Receiver)

39.9%43.7%37.3%Oral Sex (Actor)

2.0%2.9%1.4%Deep Kissing

OverallMenWomenBehavior

New S-Data: Experience Sampling

Daily diary keeping– What are you doing, thinking, or feeling right

now?– Prompted by a random buzzer, pager, or a PDA

alarm.– Asked to immediately write down your answer

Increasingly popular because there is the sense that these provide more “real time” reports of thoughts, feelings, and behavior

Comments on I-Data - Judgments about another person

Advantages– Based on large amount of information– Raters use common sense

Is Sally an outgoing person?

Disadvantages

Disadvantages– Some dimensions are easier to judge than others.– Letter of Recommendation Problem– Limited perspective – Do people act differently

around different people?– Judgments can be wrong or biased– We tend to remember behaviors that are extreme,

unusual, and/or emotionally provocative– What to do when raters do NOT agree?

Correlations between Self and Informant Reports of Internalizing Symptoms (N = 326)

.19 Anxious Arousal

.41 Anhedonic Depression

.39 General Distress

CorrelationDimension

Thinking More Deeply About Informant Reports…

“Our minds are forever sealed off from each other. We can only know each other from

watching what we do.”

David Funder

Self-Peer Correlations

Source: Watson (1989) – “Hello, my name is…”

Extraversion: .41 Conscientiousness: .16 Openness: .10 Agreeableness: .08 Neuroticism: -.01

Realistic Accuracy Model (Funder, 1999)

Aspect of Target

Accurate Judgment

Relevance

Detection

Availability

Utilization

Realistic Accuracy Model (RAM)

How do we get accurate informant reports?– Relevance: Person must do something relevant to

dimension being judged– Availability: Observer has to be able to see it– Detection: Observer has to actually see it– Utilization: Observer must remember the action

and interpret that piece of information correctly

Is Sally Courageous?

Ratings of Sally’s Courageousness

R: Is there a burning building?

D: Did I see her run into the building?

A: Can I see her run into the building?

U: Do I use this observation when it comes time to make my rating?

Is Judging Yourself Easy?

Some Reasons for Difficulties

Our thoughts, feelings, behaviors are always there (“Fish and Water Effect”): Fish don’t know they are wet because they are always surrounded by water.

People show a “false consensus” effect. They believe their own behavior is more common than it really is.

People believe their behavior was a natural response to the situation. “What else could I do?”

B-data: Behavioral Data/Observational Data

Examples: Record physiological responses to violent media (e.g., heart rate) or measure testosterone levels in the blood.

Measure aggression in the lab: Hot Sauce Paradigm (Lieberman et al., 1997)

– How much hot sauce will an individual give to a person who has “insulted” them?

– Individuals are told to “Put as much or as little hot sauce as you want” into a cup for another person to consume

Precisely measure how much hot sauce is placed in the cup

Comments on B-Data

Often collected in a laboratory or other settings where there is control

Advantages– Control is a good thing – we can

draw out certain behaviors such as aggression

– Quantifiable and objective

Comments on B-Data Part 2

Disadvantages– Well, what are we really measuring?

What do scores mean?– Tendency to ignore validity issues --

rely on “face validity”– Laboratories can be artificial

New B-Data: Implicit Measureshttps://implicit.harvard.edu/implicit/demo/

Goal is to assess automatic or unconscious thoughts and feelings. Usually attitudes.

Implicit measures are based on reaction times to pairings.

– Make judgments about the similarity between two pairings. How similar are the terms TV and good?

– Faster reactive time indicates a more tightly knit associative network.

Faster responses to negative pairings negative implicit attitude

Faster responses to positive pairings positive implicit attitude

Projective Techniques as B-Data

Assumption: How an individual interprets ambiguous stimuli says something about their psychological functioning

Different from “true” self-reports because projective techniques are seen as tests of personality – how you answer questions is interpreted.– Examples: Rorschach Ink Blots, Thematic

Apperception Test (TAT)

L-Data

L-data are life records. L-data for Aggression: School records (Disciplinary

Actions) and Court Records (Arrests and Convictions)

Advantages– Intrinsic Importance (Crime, Mortality, Marital

Status) Disadvantages

– Life outcomes are complicated and multidetermined

Summary Points

Nature of the research question often dictates which modes of measurement are most appropriate

Multiple sources of data help us “triangulate” on the true score.

Recurring Theme: Multiple methods and approaches are good!– Gordon Allport (1947): “No doors should

be closed in the study of personality” (p. 133-134)