shackman psyc210 module04 measuring 021015

Nuts & Bolts Plan for Today Cumula4ve review

Lecture (Block 95; Tomarken 95) Measuring T&P

Some problems with words Some problems with physiological measures of T&P,

Just Because Youre Measuring the Brain (or HPA axis, or facial musculature, etc) Doesnt Mean You Can Stop Using Your Head John Cacioppo, past present of the Assoc for Psychol Science

Take-home cri4cal thinking ques4ons

Nuts & Bolts Plan for Today Lecture (Block 95; Tomarken 95) Measuring T&P

Problems with words Problems with physiological measures of T&P

Just Because Youre Measuring the Brain (or HPA axis, or facial musculature, etc) Doesnt Mean You Can Stop Using Your Head John Cacioppo, past present of the Assoc for Psychol Science


Nuts & Bolts Plan for Today Lecture (Block 95; Tomarken 95) Measuring T&P

Problems with words Problems with physiological measures of T&P

Just Because Youre Measuring the Brain Doesnt Mean You Can Stop Using Your Head

John Cacioppo, past present of the Assoc for Psychol Science (APS)


PSYC 210:

How Can We Discover and Measure Individual Dierences in T&P?

AJ Shackman 10 February 2015

Todays Conceptual Roadmap Where did the fundamental dimensions of T&P come from? How were they discovered? What are their limita4ons?

How should we measure T&P? What is the value of adop4ng biological measures?

What sta4s4cal proper4es do we need to assess if we turn to biological measures of T&P?

Todays Conceptual Roadmap Where did the fundamental dimensions of T&P come from? How were they discovered? What are their limita4ons?

How should we measure T&P? What is the value of adop4ng biological measures?

What sta4s4cal proper4es do we need to assess if we use biological measures of T&P?

From Measures to Models The Case of the 5 Factor Model, or,

How did we get the factors that we were given?

Why 5 Factors? Words, words, words

Lexical Hypothesis,"All aspects of human personality which are or have been of importance, interest, or u4lity have already become recorded in the substance of language Ca[ell Common speech is a poor guide to psychological subtle4es Allport


Allport 1930s 400,000 words in the unabridged dicJonary 18,000 trait-relevant adjecJves

4,500 key traits Ca[ell 1940s

Used a combinaJon of ad hoc subjecJve and staJsJcal techniques to whiQle it down to 35 scales

Used factor analysis to reduce it to 12 Set the stage for the discovery of the Big 5 (OCEAN)


Allport 1930s 400,000 words in the unabridged dicJonary 18,000 trait-relevant adjecJves

4,500 key traits Ca[ell 1940s

Used a combinaJon of subjecJve and staJsJcal techniques to whiQle it down to 35 scales

Used factor analysis to reduce it to 12 Set the stage for the discovery of the Big 5 (OCEAN)


Factor Analysis StaJsJcal technique for compressing or reducing the number of dimensions in a

dataset

e.g., 100 items on a quesJonnaire 5 latent factors or dimensions (95% compression)

Lossy compression technique: RetenJon of a small number of dimensions typically captures

Factor Analysis StaJsJcal technique for compressing or reducing the number of dimensions in a

dataset

e.g., 100 items on a quesJonnaire 5 factors or dimensions (95% compression)

Lossy compression technique: RetenJon of a small number of dimensions typically captures

Non-MathemaCcal Example

Recall the survey that you completed in class

Choices, choices, choices


CumulaJve: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors


CumulaJve: % variance explained by 1, 1+2, 1+2+3, 1+2+3+4 all 18 factors

1st factor explains a great deal of variance in scores

2nd factor explains much less

Factors 3-18 account for less and less variance


Varying degrees of informaCon loss, depending on the number of factors we choose to retainfewer factors, more loss

Loss: 100 CumulaJve

Loss of 56.3%

Loss of 13.5%

Loss of 0.0%


Varying degrees of informaCon loss, depending on the number of factors we choose to retainfewer factors, more loss Pick one big factor, four factors, many factors?

Loss: 100 CumulaJve

Loss of 56.3%

Loss of 13.5%

Loss of 0.0%

If we retain 4 factors (~2/3 variance)

F1: Distress

F2: Worry

F3: Depression

F4: Junk

Limita4ons of Factor Analysis Subjec4ve, the Analyst Has to Choose The number of factors to retain The rotaJon, should the factors be staJsJcally uncorrelated

(orthogonal) or correlated (oblique) "...each orientaJon is equally acceptable mathemaJcally. But dierent

theories proved to dier as much in terms of the [choice of rotaJon] as in terms of anything else, so that model rng did not prove to be useful in disJnguishing among theories." (Sternberg, 1977).

All rotaJons are equally valid mathemaJcal outcomes. There is no unique or opJmal soluJon. Dierent interpretaJons are equally valid.

Garbage In/Garbage Out Results are data-dependent. If the analyst includes 30 anxiety

items and 2 thought-disturbance items, the factor analysis will likely tell you that a single anxiety dimension is sucient, but this simply reects the choice of inputs


(orthogonal) or correlated (oblique) "...each orientaJon is equally acceptable mathemaJcally. But dierent

theories proved to dier as much in terms of the [choice of rotaJon] as in terms of anything else, so that model rng did not prove to be useful in disJnguishing among theories." (Sternberg, 1977).

All rotaJons are equally valid mathemaJcal outcomes. There is no unique or opJmal soluJon. Dierent interpretaJons are equally valid.

Garbage In/Garbage Out Results are data-dependent. If the analyst includes 30 anxiety

items and 2 thought-disturbance items, the factor analysis will likely tell you that a single anxiety dimension is sucient, but this simply reects the choice of inputs

More concretely, we have to choose whether we think that F1 (distress) and F2 (worry) are related or unrelated, in terms of their underlying psychology and neurobiology

F1 F2


(orthogonal) or correlated (oblique) All rotaJons are equally valid mathemaJcal outcomes there is nothing inherent in the mathemaJcs of factor analysis that tells you which rotaJon is the right one

Garbage In/Garbage Out Results are data-dependent. If the analyst includes 30

anxiety items and 2 thought-disturbance items, the factor analysis will likely tell you that a single anxiety dimension is sucient, but this simply reects the choice of inputs


(orthogonal) or correlated (oblique) All rotaJons are equally valid mathemaJcal outcomes there is nothing inherent in the mathemaJcs of factor analysis that tells you which rotaJon is the right one

Garbage In/Garbage Out Results are data-dependent. If you include 30 anxiety items but

only 2 thought-disturbance items, the factor analysis will indicate that a single anxiety dimension is sucient to describe T&P, but this one dimensional soluJon is an arJfact of the inputs

Back to Blocks history lesson

Tupes & Christal 1940s personnel selecJon psychologists employed by the Air Force to improve ocer

selecJon and promoJon procedures factor analyses based on ocers raJngs of one another Can the psychological percep4veness of Air Force ocers and ocer candidates, as

quickly expressed by 3-point ra4ngs on 30 or so scales in an ocially required research program regarding 12 to 30 of their peers known for such short periods [as few as 3 days], provide a fundamental data basis for discerning the essen4al dimensions for the scien4cally sucient descrip4on of personality? Block 95

5 factors: Surgency/Extroversion, Agreeableness, Dependability, EmoJonal Stability, and

Culture.

Norman 1960s Again, various subjecJve (but herculean) ad hoc approaches to turning very large lists

of words (adjecJves or trait descriptors) into a manageable set of dimensions



selecJon and promoJon procedures factor analyses based on ocers raJngs of one another Can the psychological percep4veness of Air Force ocers and ocer candidates, as

quickly expressed by 3-point ra4ngs on 30 or so scales in an ocially required research program regarding 12 to 30 of their peers known for such short periods [as few as 3 days], provide a fundamental data basis for discerning the essen4al dimensions for the scien4cally sucient descrip4on of personality? Block 95

5 factors: Surgency/Extroversion, Agreeableness, Dependability, EmoJonal Stability, and

Culture.




Students Any concerns with this approach?


selecJon and promoJon procedures factor analyses based on ocers raJngs of one another Discovered 5 factors:

Surgency/Extroversion, Agreeableness, Dependability, EmoJonal Stability, and Culture.





selecJon and promoJon procedures factor analyses based on ocers raJngs of one another Discovered 5 factors:

Surgency/Extroversion, Agreeableness, Dependability, EmoJonal Stability, and Culture.

Norman 1960s Again, various subjecJve approaches for turning very large lists of words (adjecJves

or trait descriptors) into a manageable set of dimensions


Culminated in the work of Goldberg 1970s 1990s and Costa & McCrae 1980s 1990s Similar concerns

In designing their quesJonnaire, Costa and McCrae disJnguished and permanently xed upon [case closed!] a half dozen facets each for their broad constructs of NeuroJcism, Extraversion, and Openness. The facet dis4nc4ons they oered were not rooted in factor analysis, formal theorizing, or ineluctable empirical ndings. Rather, the facets derived from their personal thinking about how the three domains could be further ar4culated. The six facets Costa and McCrae nominated to represent the NeuroJcism domain were Depression, Impulsiveness, Anxiety, HosJlity, Self-consciousness, and Vulnerability.


The Lexical Hypothesis

Jack Blocks Cri4que

Students: What are some potenCal problems with the lexical hypothesis?

#1. The Lexical Hypothesis

Jack Blocks Cri4que

Students: What are some potenCal problems with the lexical hypothesis?

#1. The Lexical Hypothesis The premise is false

Meaningful aspects of T&P may not be captured by single word adjecJves

No guarantee that our language naturally includes every scienJcally crucial aspect of T&P If you were trying to reverse engineer cars, radios, computers would you use this

strategy?

Jack Blocks Cri4que

#2. Reliance on Untrained Lay Raters to Specify Personality Descriptors and to Provide Personality Descrip4ons

Jack Blocks Cri4que


Jack Blocks Cri4que

Students: What are some potenCal problems with using untrained raters (e.g., students enrolled in college psychology courses)?


Lay language may not make scienJcally important disJncJons E.g., Fear (imminent threat) vs. anxiety (apprehension/distress when threat is

distal or uncertain)

Lay language is relaJvely gross, casual, unreecJve, and surprisingly inconsistent

E.g., "aggressive" - asserJve vs. hosJle E.g., "criJcal" analyJc vs. hosJle

Jack Blocks Cri4que


Lay language may not make scienJcally important disJncJons E.g., Fear (imminent threat) vs. Anxiety (apprehension/distress when threat is

remote, uncertain, or diuse)


E.g., "aggressive" - asserJve vs. hosJle E.g., "criJcal" analyJc vs. hosJle

Jack Blocks Cri4que


Lay language may not make scienJcally important disJncJons E.g., Fear (imminent threat) vs. Anxiety (apprehension/distress when threat is

remote, uncertain, or diuse)


E.g., "aggressive" - asserJve vs. hosJle

E.g., "criJcal" analyJc vs. hosJle

Jack Blocks Cri4que

#3. Factor Analysis

Jack Blocks Cri4que

#3. Factor Analysis

Jack Blocks Cri4que

Students: Can factor analysis be used to objecCvely discover the fundamental dimensions of T&P?

#3. Factor Analysis Big 5 advocates suggest that the ve factors are real because they emerge for very

large lists of adjecJves across dierent factor analyJc methods

Jack Blocks Cri4que



Block counter-argues that if you closely scruJnize the results of these most compelling demonstraJons The big list was actually chosen by earlier invesJgators (Norman) to reect a

preconceived model of personality

There is sJll a good deal deal of subjecJve choices about which words to use, how they should be clustered, what is important, how many factors are sucient, and so on basically, he argues that the Big 5 advocates oversell the claim that they have objecJvely discovered The Most Important Factors

The sequence of empirical procedures that repeatedly issued similar ve-factor structures may have been constrained to produce the results obtained; and the ve-factor soluJon is not as stable as oyen claimed

Jack Blocks Cri4que



Block counter-argues that if you closely scruJnize the results of these most compelling demonstraJons The big list was actually chosen by earlier invesJgators (Norman) to reect a

preconceived model of personality

SubjecJve, undocumented choices about which words to use, how they should be clustered, what is important, how many factors are sucient, and so on basically, he argues that the Big 5 advocates oversell the claim that they have objecJvely discovered The Most Important Factors

The sequence of empirical procedures that repeatedly issued similar ve-factor structures may have been constrained to produce the results obtained; and the ve-factor soluJon is not as stable as oyen claimed

Jack Blocks Cri4que

Not Just Block [The purported fundamental dimensions of T&P]are derived top-down from pools of lexically-chosen quesJonnaire itemsnot from biological anchors. They also depend on factor analysis, which determines the number of dimensions, but not locaGon of trait axes of the personality space that items occupyIt is liQle more than an act of faith to believe that the causal [i.e., real] structure of personality is isomorphic with its lexical factor structure. Further, not only is there no reason to suppose that biologically accurate scales should have simple structure but also current scale systems, even though designed to have this, oyen do not...

McNaughton & Corr FronJers in Sys Neurosci 2014

Not Just Block It is liale more than an act of faith to believe that the causal [i.e., real scienCc] structure of personality is isomorphic with [i.e. the same as] its lexical factor structure. to have this, oyen d


But what does this even mean? It is liale more than an act of faith to believe that the causal [i.e., real scienCc] structure of personality is isomorphic with [i.e. the same as] its lexical factor structure. to have this, oyen d


An explanaCon by means of a automobile analogy

Epstein Psychol Inquiry 1994

Like people, we can measure individual dierences in car phenotypes Parameters like horsepower, maximum speed, fuel consumpJon, o-road clearance, deep-snow performance, crash raJngs, polluJon We could use factor analysis to crunch these parameters into broad-band factors e.g. sport-performance


Like people, we can measure individual dierences in car phenotypes Parameters like horsepower, top speed, fuel consumpJon, o-road clearance, deep-snow performance, crash raJngs, polluJon, size, and reliability We could use factor analysis to crunch these parameters into broad-band factors e.g. sport-performance


Like people, we can measure individual dierences in car phenotypes Parameters like horsepower, top speed, fuel consumpJon, o-road clearance, deep-snow performance, crash raJngs, polluJon, size, and reliability We could use factor analysis to crunch these parameters into broad-band factors that summarize the phenotype e.g. sport-performance


But a sport-performance score tells you nothing about the systems that CAUSE car-to-car dierences in the phenotype

Power plant Powertrain and Transmission Suspension Emissions

Its unlikely that a factor analysis would discover the key systems that are familiar to every mechanic and automobile engineer




Its unlikely that a factor analysis of self-reported car raJngs would discover the key systems that are familiar to every mechanic and automobile engineer




Its unlikely that a factor analysis of self-reported car raJngs would discover the key systems that are familiar to every mechanic and automobile engineer The fundamental dimensions are descripJve and supercialnot explanatory. They give you no clues about how a car works or what to do when it breaks down


[Factor-analyGc] Traits do notexplain behavior. They describe paPerns and consistencies in behavior, but they don't explain where those paPerns and consistencies come from either developmentally or in terms of their proximal causaGon [i.e., underlying psychological or neural systems]. What traits do not tell you, in parGcular, is why somebody is doing something

David Funder (Psychol Inquiry, 1994)

Block oers a number of sugges4ons for moving beyond the FFM, among them: personality psychologists [should] not limit their thinking and research by considering only what is or will be convenient to index via simple self-report or layperson-report measures. to study [T&P, we]will need to turn, or return, to more complicated and complex ways of studying personsfor example, behavioral observaJons, psychophysiological measures, individual dierences in various standardized situaJonal contexts, the garnering of life facts about the persons studied, truly inJmate interviews, and the longitudinal study of personality development.

Where Do We Go From Here?

Block oers a number of sugges4ons for moving beyond the FFM, among them: personality psychologists [should] not limit their thinking and research by considering only what is or will be convenient to index via simple self-report or layperson-report measures. [That is, do not consider the case of How many dimensions or Which Dimensions closed] to study [T&P, we]will need to turn, or return, to more complicated and complex ways of studying personsfor example, behavioral observaJons, psychophysiological measures, individual dierences in various standardized situaJonal contexts, the garnering of life facts about the persons studied, truly inJmate interviews, and the longitudinal study of personality development.


Block oers a number of sugges4ons for moving beyond the FFM, among them: personality psychologists [should] not limit their thinking and research by considering only what is or will be convenient to index via simple self-report or layperson-report measures. [That is, do not consider the case of How many dimensions or Which Dimensions closed] to study [T&P, we]will need to turn, or return, to more complicated and complex ways of studying personsfor example,

ObjecJve behavioral observaJons by trained raters

Psychophysiological and neurobiological measures Individual dierences in response to standardized challenges


Where do we go from here? How should we measure T&P?

One Way to Measure Trait-Like Individual Dierences

What Are Some Poten4al Advantages of Using Physiology to Assess T&P?

Students???


ObjecJve, not subject to biases in reporJng, introspecJon, memory Repor4ng Bias: Desire to look good (social desirability); outright lying or malingering

Mnemonic Bias: Peak-End Rule Limits of introspec4on: some processes may occur outside of the focus of aQenJon (e.g., habits) or be opaque to introspecJon we may only be aware of the downstream product or read-out of more elementary processes E.g., how much corJsol is in your blood stream? How exactly did you drive from campus to your home?

Behavior is normally guided by both conscious and pre-conscious processes (lie outside of awareness)



Mnemonic Bias: Peak-End rule of retrospecJve raJngs Limits of introspec4on: some processes may occur outside of the focus of aQenJon (e.g., habits) or be opaque to introspecJon we may only be aware of the downstream product or read-out of more elementary processes E.g., how much corJsol is in your blood stream? How exactly did you drive from campus to your home?






Rather than the average experience

peak

end

Some mental processes are too fast for self-report or lie outside of conscious awareness

Some mental processes are too fast for self-report or lie outside of conscious awareness Students??? Whats a possible soluCon?

Shackman et al. 2007; Shackman et al. 2011; Shackman Shackman et al. under review

Note: We will discuss some other interes4ng aspects of the unconscious mind/brain in Module 5

Implica4on Understanding aspects of T&P (N/NE, E/PE, C/SC) that lie outside of conscious awareness mandates the use of behavioral or physiological assays

Tasks

Res4ng Physiology

Task-Evoked Physiology

Tasks

Res4ng Physiology


Cog Tasks

Standardized Behl Challenges

Tasks

Res4ng Physiology PET

rs-fMRI

EEG

MRS


Cog Tasks


Tasks

Res4ng Physiology PET

rs-fMRI

EEG

MRS


Cog Tasks


Startle Reex

ERPs

PET During Behl Challenge

Just because youre measuring the brain

Students?

psychometric = staJsJcal

Like quesConnaire measures of T&P, we need to establish that biological measures are: - Reliable - Stable over Cme (trait-like; test-retest)

psychometric = staJsJcal

Degree to which a scale measures a single latent construct Items hang together Necessary but not sucient condiJon for unidimensionality requires factor analysis Very high ICR is undesirable: items are redundant

2 Kinds of Reliability

Internal Consistency Reliability (Cronbachs Alpha)

Degree to which a scale measures a single latent construct Items hang together Necessary but not sucient condiJon for unidimensionality requires factor analysis Very high ICR is undesirable: items are redundant



Degree to which a scale measures a single latent construct Items hang together (covary) or provide the same rank order of Ss Necessary but not sucient condiJon for unidimensionality requires factor analysis Very high ICR is undesirable: items are redundant



Degree to which a scale measures a single latent construct Items hang together (covary) or provide the same rank order of Ss Very high Alpha is undesirable: items are redundant


Brief examples


Degree to which a scale measures a single latent construct Items hang together or provide the same rank order of Ss Necessary but not sucient condiJon for unidimensionality requires factor analysis Very high Alpha is undesirable: items are redundant

Temporal Stability / Test-Rest Reliability (Correla4on or ICC)

Individual dierences in T&P should be trait-like (stable) Rank order consistency (mean neednt be stable) Over more than ~30 days, however, you might expect genuine change For broad-band traits (e.g., neuroJcism) long-term coecients of ~.70 (.55 to .85) or

about 50% variance





Individual dierences in T&P should be stable and consistent

Over more than ~30 days, however, you might expect genuine change For broad-band traits (e.g., neuroJcism) long-term coecients of ~.70 (.55 to .85) or

about 50% variance





Individual dierences in T&P should be stable and consistent Rank order consistency (not absolute or mean value) Over more than ~30 days, however, you might expect genuine change For broad-band traits (e.g., neuroJcism) long-term coecients of ~.70 (.55 to .85) or

about 50% variance


What does a measure actually measure? (func4onal signicance) What inferences can we legi4mately draw from a measure?

E.g., Is amygdala ac4va4on specically indica4ve of anxiety or is it ac4vated by a range of socially and mo4va4onally signicant s4muli?

E.g., Is electrodermal ac4vity (SCR) indica4ve of stress, arousal, cogni4ve load?

E.g., Is the startle reex sensi4ve to any strong emo4onal state or is it linearly related to valence (NEG > NEU > POS)?

Construct Validity: Linking Physiological Measures to T&P

Students???




E.g., Is the startle reex sensi4ve to any strong emo4onal state or is it linearly related to emo4onal valence (NEG > NEU > POS)?





E.g., Is the startle reex sensi4ve to any strong emo4onal state or is it linearly related to emo4onal valence (NEG > NEU > POS)?


Students??? How might we forge this link ?

In order to establish the func4onal signicance or construct validity of a measure

1. Psychological Sensi4vity E.g., probability of amygdala acJvaJon, given fear-arousing sJmuli AND

2. Psychological Specicity Reverse E.g., Probability of fear, given amygdala acJvaJon OyenJmes, low specicity is not explicitly acknowledged

In both cases, mechanisJc studies are helpful directly manipulate the measure (lesions, TMS, drugs, etc.) and asses consequences for construct of interest (e.g., fear) and other constructs (e.g., aQenJon)




2. Psychological Specicity Reverse inference E.g., Probability of fear, given amygdala acJvaJon OyenJmes, low specicity is not explicitly acknowledged





2. Psychological Specicity Reverse inference E.g., Probability of fear, given amygdala acJvaJon OyenJmes, low specicity is not explicitly acknowledged because researchers

focus on a small number of psychological processes (e.g., emoJon or cogniJon)





2. Psychological Specicity Reverse inference E.g., Probability of fear, given amygdala acJvaJon OyenJmes, low specicity is not explicitly acknowledged because researchers

focus on a small number of psychological processes (e.g., emoJon or cogniJon)

In both cases, mechanisJc studies are helpful directly manipulate the measure (lesions, TMS, drugs, etc.) and assess consequences for construct of interest (e.g., fear) and other constructs (e.g., aQenJon)


Construct Validity Has Fundamental ImplicaCons

As cogniGve neuroscienGsts who use the same brain imaging technology, we know that it is not possible to deniGvely determine whether a person is anxious or feeling connected simply by looking at acGvity in a parGcular brain region. This is so because brain regions are typically engaged by many mental states, and thus a one-to-one mapping between a brain region and a mental state is not possible.

As cogniGve neuroscienGsts who usebrain imaging technology, we know that it is not possible to deniGvely determine whether a person is anxious or feeling connected simply by looking at acGvity in a parGcular brain region. This is so because brain regions are typically engaged by many mental states, and thus a one-to-one mapping between a brain region and a mental state is not possible.

1. There are serious problems with the 5 Factor Model

2. In par4cular, they do not represent an empirically discovered set of natural kinds. They instead strongly reect the theore4cal perspec4ves, methods, and assump4ons of key inves4gators

3. Key assump4ons underlying the 5 Factor Model may not be warranted Words / Lexical Hypothesis Lay assessments and lay language Factor analysis

4. The best level of descrip4on for T&P (wide-band narrow-band) remains unclear 5. Expert behavioral and physiological assessments may prove illumina4ng

6. Especially given other known biases and limita4ons of introspec4ve self-reports Repor4ng biases (e.g., social desirability); Mnemonic biases (peak-end rule) Limits of introspec4on (too fast, outside of conscious awareness)

7. Just because you use biological measures doesnt mean you can forget about psychometric reliability (internal-consistency and test-retest)

8. Determining construct validity (func4onal signicance) of physiological measures is hard Sensi4vity, specicity

Measuring T&P - Key Take-Home Points

Cri4cal Thinking Ques4ons

Pick any 2 opCons

If low on Cme, read at home

Cri4cal Thinking Ques4ons 1. Jack Block argued that, we should not limit [our] thinking and research by considering only what is or will be convenient to index via simple self-report or layperson-report measuresTo study [T&P, we]will need to turn, or return, to more complicated and complex ways of studying persons What do you think? Based on what you know about the Big 3 traits (see the slides from last Jme) or their facets, briey describe a novel experiment that exploits a measure other than self-report to discover and understand some important aspect of T&P. For example, you could think about ways to clarify whether C/SC involves heightened anxiety about order, rules, or deadlines.

Cri4cal Thinking Ques4ons 2. If traits are supercial and descripJve (rather than explanaJons of behavior), what good are they? David Funder (1994) described what I call The NeuroJc Professor, I have a friend and colleague who has served as a faculty member in several departments. In each job, he's been miserable. He can enumerate very persuasively why each of his department chairs was an unfair tyrant, why each teaching load was excessive, how oce space and salary were unfair and paltryHe really is persuasive. But you know what? I think he'd be unhappy anywhere. I think he's got a trait. He exhibits a behavioral paPern (of complaining) from which I infer an emoGonal paPern (of feeling miserable) that I think does explain why he is saying such nasty things about his latest department. To explain a behavior in terms of the broader paPern of which it is a part-as I have just done-can be a legiGmate and useful step in the (innite) explanatory regress. I know, it's not complete. We sGll don't know why my friend is such a negaGvisGc cynic. But idenGfying him as such provides (a) insight into his current acGons (and feelings, which we infer from his acGons), (b) a basis for predicGng his future acGons and feelings, and (c) a useful analyGc rest stop. We have explained his behavior; take a breath; now we need to explain the explanaGon What do you think? Briey describe a strategy or two for addressing the origins of The NeuroJc Professor?

Cri4cal Thinking Ques4ons 2. If traits are supercial and descripJve (rather than explanaJons of behavior), what good are they? David Funder (1994) described what I call The NeuroJc Professor, I have a friend and colleague who has served as a faculty member in several departments. In each job, he's been miserable. He can enumerate very persuasively why each of his department chairs was an unfair tyrant, why each teaching load was excessive, how oce space and salary were unfair and paltryHe really is persuasive. But you know what? I think he'd be unhappy anywhere. I think he's got a trait. He exhibits a behavioral paPern (of complaining) from which I infer an emoGonal paPern (of feeling miserable) that I think does explain why he is saying such nasty things about his latest department. To explain a behavior in terms of the broader paPern of which it is a part-as I have just done-can be a legiGmate and useful step in the (innite) explanatory regress. I know, it's not complete. We sGll don't know why my friend is such a negaGvisGc cynic. But idenGfying him as such provides (a) insight into his current acGons (and feelings, which we infer from his acGons), (b) a basis for predicGng his future acGons and feelings, and (c) a useful analyGc rest stop. We have explained his behavior; take a breath; now we need to explain the explanaGon What do you think? Briey describe a strategy or two for addressing the origins of The NeuroJc Professor, that is, for reverse engineering the systems that underlie N/NE or some other dimension of T&P.

Cri4cal Thinking Ques4ons 3. Andy Tomarken underscored the importance of reliability for behavioral and biological measures of T&P Recent work suggests that the dot-probe task (a widely used measure of anxious individuals vigilance for threat) is not reliable What do you think? Skim the paper (hQp://journal.fronJersin.org/Journal/10.3389/fpsyg.2014.01368/full) and and briey comment on the key take-home points. What are the implicaJons for understanding the cogniJve underpinnings of anxiety and other key facets of T&P

Cri4cal Thinking Ques4ons 3. Andy Tomarken underscored the importance of reliability for behavioral and biological measures of T&P Recent work suggests that the dot-probe task (a widely used measure of anxious individuals vigilance for threat) is not reliable What do you think? Skim the paper (hQp://journal.fronJersin.org/Journal/10.3389/fpsyg.2014.01368/full) and and briey comment on the key take-home points. What are the broader implicaJons for understanding the cogniJve underpinnings of anxiety and other key facets of T&P

Cri4cal Thinking Ques4ons 3. Andy Tomarken underscored the importance of reliability for behavioral and biological measures of T&P Recent work suggests that the dot-probe task (a widely used measure of anxious individuals vigilance for threat) is not reliable What do you think? Skim the paper (hQp://journal.fronJersin.org/Journal/10.3389/fpsyg.2014.01368/full) and and briey comment on the key take-home points. What are the implicaJons for understanding the cogniJve underpinnings of anxiety and other key facets of T&P

Cri4cal Thinking Ques4ons 4. Andy Tomarken underscored the importance of establishing the sensiJvity and specicity (construct validity) of biological measures of T&P (e.g., fMRI, PET, EEG, ERP, GSR, startle, HR, corJsol, etc.) What do you think? a) Briey describe how you might test the specicity of one of these measures,

the probability that it will be acJvated or engaged given a parJcular process or perturbaJon (such as the inducJon of anxiety vs. other.

-or-

a) Play with Neurosynth.org, an on-line tool that allows you to dynamically assess the sensiJvity and specicity of dierent brain regions. Then write a short paragraph describing what you did and what you learned. It need not be a deep insight, its sucient to just go play with it and see what you uncover about sensiJvity (forward inference) and specicity (reverse inference). The next few slides provide some pointers for interested students.


the probability that it will be acJvated or engaged given a parJcular process or perturbaJon (such as the inducJon of anxiety vs. other states).

-or-


Cri4cal Thinking Ques4ons 4. ScienJsts can be sloppy in their use of language. Does fear in a rat exposed to cues paired with shock reect the same processes involved in N/NE? In anxiety disorders? In specic phobias? Is NeuroJcism in adults synonymous with Behavioral InhibiJon in kids? What about Harm Avoidance? In a recent review focused on Theory of Mind, Schaafsma and colleagues (TiCS 2015) described this kind of communicaJon break down: The term X [e.g. self-control] is used interchangeably with Q, R, and S, to name only a few. This diversity of terms used is probably telling: dierent invesGgators have dierent concepts in mind. Dierent levels of descripGon (e.g., neural vs. self-report vs. behavior), together with the dierent terms used, make it dicult even for experts from dierent elds to navigate both what is meant by X and how to study it using scienGc methods...to the uniniGated, the topic becomes bewildering. The problem is that these dierences [are implicit and unspoken] [Thus] it has been [extraordinarily dicult] to triangulate the actual [psychological, geneGc, and neuobiological] processes involved. What do you think? Briey describe a strategy for addressing this road block.


the probability that it will be acJvated or engaged given a parJcular process or perturbaJon (such as the inducJon of anxiety vs. other states).

-or-


Time-Permi|ng Review Ques4ons

Note: NeuroSynth.org slides follow

MoQ et al showed that childhood self-control predicts health, wealth & public safety in midlife. What was one intervening

mechanism during adolescence that parJally explained the link from kid temperament to deleterious adult outcomes?

A. Smoking B. Becoming a parent C. Excessive video game

playing D. Violence in the media E. High-caeine energy

drinks F. A & B G. C & D

Smoking

Becoming a paren

t

Excessive video game pla...

Violence in the m

edia

High-caffeine energy drinks

A & B

C & D

0% 0% 0% 0%0%0%0%

CorrelaJon and variance explained: If two variables are correlated R = .50, the amount

of variance accounted for is:

A. 0.50 * 0.50 = .25 = 25%

B. 0.50 / 0.50 = 1 = 100%

C. Sqrt(.50) = .7071 = 70%

0.50 * 0.50 = .25

= 25%

0.50 / 0.50 = 1 = 100%

Sqrt(.50) = .7071 = 70%

0% 0%0%

T&P reect trait-like individual dierences in emoJonal and cogniJve biases that

A. First emerge early in life B. ConJnue to evolve for many

years C. Account for consistency in

behavior, inner experience, and risk across Jme and contexts. Can be relaJvely simple (e.g., anxious distress) or complex and mulJply determined (orderliness). Excessive video game playing

D. Can be relaJvely simple E. Can be complex and

mulJdimensional F. All of the above

First emerg

e early in life

Continue to evolve for m

...

Account fo

r consisten

cy i...

Can be relatively sim

ple

Can be com

plex and mult...

All of the above

0% 0% 0%0%0%0%

T&P are not dierent in kind (according to Shackman) because they are both

A. Biological B. EmoJonal C. CogniJve D. Somewhat heritable E. All of the above

Biological

Emotional

Cognitive

Somewhat heritable

All of the above

0% 0% 0%0%0%

What are the 3 fundamental dimensions of T&P?

A. N/NE B. P/TA C. E/PE D. S/RE E. C/SC F. A, C, and E G. A, B, and C

N/NE

P/TA E/P

ES/R

EC/SC

A, C, and E

A, B, and C

0% 0% 0% 0%0%0%0%

N/NE can be dissected into which 2 facet traits

A. Distress (fear/anxiety) and IrritaJon (anger)

B. Guilt and Shame

Distress (fear/anxiety) and...

Guilt and Shame

0%0%

Which staJsJcal test is used to quanJfy the conJnuity (temporal stability) of traits

A. Students t test B. ANOVA C. CorrelaJon

Students t test

ANOVA

Correlation

0% 0%0%

T&P is

A. Fixed and immutable

B. Moderately stable (R = 0.4 to 0.6 over periods of one to several years)

C. Completely plasJc and malleable

Fixed and imm

utable

Moderately stable (R = 0....

Completely plastic and ...

0% 0%0%

NeuroSynth.org Slides

You can click around with your mouse on the brain and the coordinates will be autolled!

The End

Material for Future Semesters

This could be incorporated into lecture or- as a take home quesJon

Incorporate some of Grubers material @ hQps://www.youtube.com/watch?v=g1lXq5VIKS4&list=PLh9mgdi4rNewieO9Dsj-OhNBC9bF4FoRp Gruber interview of Mauss could be criJcal q hQps://www.youtube.com/watch?v=dDMdE3MnBFU Note that chapter 1 of maQhews deary and whiteman is preQy good; opJonal or even assigned reading Alex you have the damasio material in the next lecture, #5.copy and paste that version into this version and make it more explicit that if low on Jme, skip.

Future Semester Thought QuesCon

Just out .. earlyview .. Park, G., Schwartz, H.A., Eichstaedt, J.C., Kern, M.L., Kosinski, M., SJllwell, D.J., Ungar, L.H., & Seligman, M.E.P. (2014). AutomaJc personality assessment through social media

language. Journal of Personality and Social Psychology, EarlyView, , 1-20. Abstract Language use is a psychologically rich, stable individual dierence with well-established correlaJons to personality. We describe a method for assessing personality using an open-

vocabulary analysis of language from social media. We compiled the wriQen language from 66,732 Facebook users and their quesJonnaire-based self-reported Big Five personality traits, and then we built a predicJve model of personality based on their language. We used this model to predict the 5 personality factors in a separate sample of 4,824 Facebook users, examining (a) convergence with self-reports of personality at the domain- and facet-level; (b) discriminant validity between predicJons of disJnct traits; (c) agreement with informant reports of personality; (d) paQerns of correlaJons with external criteria (e.g., number of friends, poliJcal artudes, impulsiveness); and (e) testretest reliability over 6-month intervals. Results indicated that language-based assessments can consJtute valid personality measures: they agreed with self-reports and informant reports of personality, added incremental validity over informant reports, adequately discriminated between traits, exhibited paQerns of correlaJons with external criteria similar to those found with self-reported personality, and were stable over 6-month intervals. Analysis of predicJve language can provide rich portraits of the mental life associated with traits. This approach can complement and extend tradiJonal methods, providing researchers with an addiJonal measure that can quickly and cheaply assess large groups of parJcipants with minimal burden.

The key results can be seen in Table 1: From this, the authors conclude (1st sentence of their Conclusions): In this arJcle, we provided evidence that the language in social media can be harnessed to create a valid and reliable measure of personality. Some context: A correlaJon of 0.40, sampling a random normal bivariate distribuJon, 4,000 cases, integer scale varying between 0 and 20, mean = 10, SD = 3, and ploQed as a frequency scaQerplot (the bigger circles indicate higher frequencies of occurrence): What the authors do not do is engage with that word accuracy, using the kinds of analyses (in the metric of the criterion data) that beQer inform the reader about the kinds and

paQerns of errors associated with using a Language Based Assessment (LBA) of Personality rather than a QuesJonnaire Based Assessment. And the authors do make a recommendaJon for usage:

LBAs oer a pracJcal, cost-eecJve alternaJve, allowing assessment of psychological characterisJcs when quesJonnaires are impracJcal. Researchers could reduce parJcipant burden by replacing some quesJonnaires with a single link and sign-in procedure, allowing a research applicaJon to access parJcipant social media language and quickly assess personality and other characterisJcs of interest.

CorrelaJon is not the correct staJsJc to be used here. You need esJmates of errors that are sensiJve to actual score magnitudes, not monotonicity relaJons. You also need to conduct a

fair few actuarial and discrepancy-based analyses/graphics to properly understand both accuracy and error, and where LBAs are more accurate over a range of personality variaJon (do they predict high scorers beQer than low scorers?).

All in all, sophisJcated, innovaJve, and very, very clever work; but, for someone who aspires to greater accuracy of psychological assessment, I am puzzled by the seeming indierence to

accuracy shown within this enJre line of research. Anyway, this is already hot stu.. as Michal Kosinski describes on his website: hQp://www.michalkosinski.com/home/publicaJons Private traits and aQributes are predictable from digital records of human behavior by M. Kosinski, D. SJllwell, T. Graepel, Proceedings of the NaGonal Academy of Sciences

(PNAS), 2013. The 9th most discussed paper of 2013 and the 4th most inuen4al ar4cle ever published by PNAS (Altmetric.com) Regards .. Paul Chief Research ScienGst Cognadev.com

ExisJng theories of personality provide a number of compeJng surface level, lexically-derived, systems with trait measures that relate to approach and avoidance either indirectly via constructs such as Extraversion and NeuroJcism (Eysenck, 1957) or directly via constructs such as Harm Avoidance (Cloninger et al., 1993). Each system is stable, with links to mental disorder (Strelau and Zawadzki, 2011; Gomez et al., 2012; Mullins-SweaQ and Lengel, 2012; Trull, 2012) and brain structure (Gardini et al., 2009; DeYoung et al., 2010). But even when starJng with approach and avoidance as primary constructs, they are derived top-down from pools of lexically-chosen quesJonnaire items (Carver and White, 1994; Elliot and Thrash, 2010) not from biological anchors. They also depend on factor analysis, which determines the number of dimensions, but not locaGon of trait axes of the personality space that items occupy (Lykken, 1971; Corr and McNaughton, 2008). It is liQle more than an act of faith to believe that the causal structure of personality is isomorphic with its lexical factor structure. So, even if we knew for certain that there were only two dimensions within a parJcular measured personality space, one quesJonnaire system could have a single simple trait anxiety dimension (orthogonal to, say, impulsiveness) that was a combinaJon of neuroJcism and introversion in another (Gray, 1970)the two systems diering only on which items from an original pool were used to create scales. Factor analyJcally derived trait measures can also easily meet the criterion of having simple structure (in the sense that a set of items loads highly on only one factor so factors can be clearly idenJed by unique item loadings) while implying improbable causaJon (Lykken, 1971). Further, not only is there no reason to suppose that biologically accurate scales should have simple structure but also current scale systems, even though designed to have this, oyen do not (DeYoung, 2006, 2010).

Goldberg 1970s 1990s and Costa & McCrae 1980s 1990s Similar concerns

In designing their quesJonnaire, Costa and McCrae disJnguished and permanently xed upon [case closed!] a half dozen facets each for their broad constructs of NeuroJcism, Extraversion, and Openness. The facet dis4nc4ons they oered were not rooted in factor analysis, formal theorizing, or ineluctable empirical ndings. Rather, the facets derived from their personal thinking about how the three domains could be further ar4culated. The six facets Costa and McCrae nominated to rep

shackman psyc210 module04 measuring 021015

Documents

biological measures

physiological measures

measuring tp problems

fundamental dimensions

words problems

words lexical hypothesis

ques4ons nuts bolts

head john cacioppo