Assessment of theAssessment of theExecutive FunctionsExecutive Functions

Assessment of Executive FunctionsAssessment of Executive Functions EF are dynamic, fluidEF are dynamic, fluid ““Executive” is often provided by the examinerExecutive” is often provided by the examiner Need intra-individual approach Need intra-individual approach All formal tests and informal tasks are multi-dimensional, All formal tests and informal tasks are multi-dimensional,

requiring both content and EF requiring both content and EF EF deficits should be seen across domainsEF deficits should be seen across domains Need content-matched control tasks for every EF taskNeed content-matched control tasks for every EF task Process method of assessment most functionalProcess method of assessment most functional

Problems with EF AssessmentProblems with EF Assessment

Fluid nature not as amenable to examiner- Fluid nature not as amenable to examiner- driven, pencil and paper testingdriven, pencil and paper testing

Psychometrics of fluid “online” behaviorPsychometrics of fluid “online” behavior Well-structured testing doesn’t provide full Well-structured testing doesn’t provide full

opportunity to observe fluid strategic opportunity to observe fluid strategic problem-solvingproblem-solving

““Test of EF” may not be so if it is familiarTest of EF” may not be so if it is familiar Second administration of EF test reduces Second administration of EF test reduces

EF demandEF demand

Assessment of Executive FunctionsAssessment of Executive Functions

No formal, single test of EFNo formal, single test of EF Many available measures are "adult" Many available measures are "adult" Indirect observation; inferences madeIndirect observation; inferences made IQ: tasks may be too easy to involve EF.IQ: tasks may be too easy to involve EF. Integrity of cognitive processesIntegrity of cognitive processes Need developmental perspective Need developmental perspective

Research-based "Tests" tapping Research-based "Tests" tapping Executive FunctionsExecutive Functions

Visual SearchVisual Search Tower of Hanoi/London/Toronto/CaliforniaTower of Hanoi/London/Toronto/California Tinker Toy TestTinker Toy Test Verbal/Nonverbal Verbal/Nonverbal

Learning-Proactive/Retroactive InhibitionLearning-Proactive/Retroactive Inhibition Matching Familiar Figures TestMatching Familiar Figures Test CHIPASAT CHIPASAT

Visual SearchVisual Search

Tower of Hanoi - 3 ring - StartTower of Hanoi - 3 ring - Start

Tower of Hanoi - 2 moveTower of Hanoi - 2 move

Tower of Hanoi - 3 moveTower of Hanoi - 3 move

Tower of Hanoi - 4 moveTower of Hanoi - 4 move

Tower of Hanoi - 5 moveTower of Hanoi - 5 move

Tower of Hanoi - 6 moveTower of Hanoi - 6 move

Tower of Hanoi - 7 moveTower of Hanoi - 7 move

Tower of Hanoi - 4 Ring - 7 moveTower of Hanoi - 4 Ring - 7 move

Tower of LondonTower of LondonStarting PositionStarting Position

Tower of LondonTower of London

Tower of LondonTower of London

Tower of LondonTower of London

Tower of LondonTower of London

The “Real” Tower of LondonThe “Real” Tower of London

Traditional "Tests" tapping Traditional "Tests" tapping Executive FunctionsExecutive Functions

Verbal Fluency/Figural FluencyVerbal Fluency/Figural Fluency Stroop Color-Word Interference TestStroop Color-Word Interference Test Rey-Osterrieth Complex FigureRey-Osterrieth Complex Figure Trailmaking TestTrailmaking Test Wisconsin Card Sorting TestWisconsin Card Sorting Test Verbal Learning (intrusions, perseverations)Verbal Learning (intrusions, perseverations) MazesMazes

Other means to assess EFOther means to assess EF

parents and teacher interviews (the real experts)parents and teacher interviews (the real experts) behavioral checklists (Conner’s, CBCL, BASC, behavioral checklists (Conner’s, CBCL, BASC,

BRIEFBRIEF)) continuous performance tests (TOVA, Gordon, continuous performance tests (TOVA, Gordon,

Conner’s CPT, TEC)Conner’s CPT, TEC) behavioral observations (classroom, testing)behavioral observations (classroom, testing) Observations during other cognitive testing Observations during other cognitive testing

(Cognitive, Language, Visual Motor, Memory, (Cognitive, Language, Visual Motor, Memory, Motor, Achievement)Motor, Achievement)

EF BatteriesEF Batteries

Delis-Kaplan Executive Function ScalesDelis-Kaplan Executive Function Scales NEPSYNEPSY Cognitive Assessment SystemCognitive Assessment System Welsh, Pennington & Groisser (1991)Welsh, Pennington & Groisser (1991)

Visual Search, Verbal Fluency, Motor Visual Search, Verbal Fluency, Motor Sequencing, WCST, TOH, MFFT)Sequencing, WCST, TOH, MFFT)

Wisconsin Card Sorting TestWisconsin Card Sorting Test

Wisconsin Card Sorting Test - Color MatchWisconsin Card Sorting Test - Color Match

Wisconsin Card Sorting Test - Form Match Wisconsin Card Sorting Test - Form Match

Wisconsin Card Sorting Test - Number MatchWisconsin Card Sorting Test - Number Match

Verbal Fluency Verbal Fluency

F A SF A S

________ ________ ____ ____

________ ________ ____ ____

________ ________ ____ ____

Rey Osterrieth Complex FigureRey Osterrieth Complex Figure

Advantages of EF Performance Tests:

Increased specificity of processesIncreased specificity of processes

Increased task control and internal validityIncreased task control and internal validity

Decades of research on behavior of testsDecades of research on behavior of tests

Limitations to Performance Tests:Limitations to Performance Tests:

Performance tests tap individual Performance tests tap individual components of executive function over a components of executive function over a short time frame and not the integrated, short time frame and not the integrated, multidimensional, relativistic, priority-multidimensional, relativistic, priority-based decision-making that is often based decision-making that is often demanded in real world situations demanded in real world situations

– (Goldberg & Podell, 2000)(Goldberg & Podell, 2000)

“The good Lord did not create us with the Woodcock-Johnson in mind.”

Deborah WaberDeborah Waber

Executive Function Rating Scales

Behavior Rating Inventory of Executive Function

Frontal Systems Behavior Frontal Systems Behavior ScaleScale

DEX (Behavioural Assessment of the Dysexecutive Syndrome)

Advantages of EF ScalesAdvantages of EF Scales

• Opportunity for EF in dynamic actionOpportunity for EF in dynamic action Increased ecological validityIncreased ecological validity Capture multiple perspectivesCapture multiple perspectives Time & cost efficiencyTime & cost efficiency Rapidly developing literatureRapidly developing literature

Limitations to Rating ScalesLimitations to Rating Scales

More global, less process-specific, More global, less process-specific, information: Everyday behavior requires information: Everyday behavior requires integration of EF, e.g., inhibit + working integration of EF, e.g., inhibit + working memory + planning, thus harder to fractionatememory + planning, thus harder to fractionate

Poor control of environmental demands: WM Poor control of environmental demands: WM deficits not noticeable on assembly line but deficits not noticeable on assembly line but problematic at Dunkin’ Donutsproblematic at Dunkin’ Donuts

Limitations to Rating ScalesLimitations to Rating Scales

Rater Bias:Rater Bias:Emotional state, personality of raterEmotional state, personality of raterRater’s context (e.g., math vs lit class)Rater’s context (e.g., math vs lit class)Halo effect: general like/dislike of personHalo effect: general like/dislike of personRater’s annoyance with filling out measuresRater’s annoyance with filling out measuresAwareness of deficit on self report measuresAwareness of deficit on self report measures

LaurenLauren

16 year old 1016 year old 10thth grade girl in regular classes grade girl in regular classes Longstanding problems since K with:Longstanding problems since K with:

Inattention (drifty, lost in a fog)Inattention (drifty, lost in a fog)Anxiety- prefers routines, dislikes changeAnxiety- prefers routines, dislikes changeSocial- ‘very shy’; peers think she is strangeSocial- ‘very shy’; peers think she is strangeLearns lists of facts about one topic at a timeLearns lists of facts about one topic at a timePoor comprehension of reading & math but good Poor comprehension of reading & math but good

basic skills; Very limited written outputbasic skills; Very limited written outputMotor coordinationMotor coordination

BUT functions in regular classes with some BUT functions in regular classes with some learning specialist time; Mostly A studentlearning specialist time; Mostly A student

EL: ReyEL: Rey

Lauren: ReyLauren: Rey

CJ - 16 year old boy with ADHD-ICJ - 16 year old boy with ADHD-I

Medication: Adderall XR since 2002Medication: Adderall XR since 2002““When I don’t take it, I don’t do as well; I feel younger, When I don’t take it, I don’t do as well; I feel younger,

get distracted, go blank, stare at things”get distracted, go blank, stare at things” Anxiety issues - sensitivity to sarcasmAnxiety issues - sensitivity to sarcasm

““I take everything way too seriously”I take everything way too seriously” Parents’ goal: Parents’ goal:

““figuring out how JC can manage all this figuring out how JC can manage all this independently”independently”

CJ Test PerformanceCJ Test Performance

ACT ACT SSSS

9” 9” 100 100

18” 18” 87 87

36” 36” 8787

TOL-DXTOL-DXSSSS

Moves Moves 8282

Correct Correct 78 78

Total Time Total Time 8080

CJ 16 yo male ADHD-ICJ 16 yo male ADHD-I

Johnny-13yo Male: NVLDJohnny-13yo Male: NVLD

Longstanding history of learning and social Longstanding history of learning and social difficulties. (poor effort social impulsivity)difficulties. (poor effort social impulsivity)

Impaired mathematical skillsImpaired mathematical skills Multiple previous evaluations suggested marked Multiple previous evaluations suggested marked

disparity between normal verbal cognitive (and disparity between normal verbal cognitive (and language-based academic skills) and weaker language-based academic skills) and weaker nonverbal/problem-solving abilities (and math)nonverbal/problem-solving abilities (and math)

Overall cognitive scores fell at 5Overall cognitive scores fell at 5 thth percentile so percentile so student identified with Cognitive Disabilitystudent identified with Cognitive Disability

Johnny-13yo Male: NVLDJohnny-13yo Male: NVLD

Child was placed in DH classroom with Child was placed in DH classroom with students with cognitive scores ranging from students with cognitive scores ranging from 55-79.55-79.

Student enjoyed slower pace of classroom Student enjoyed slower pace of classroom and lack of demands (wanted to stay)and lack of demands (wanted to stay)

Student struggling with peersStudent struggling with peers

Johnny-13yo Male: NVLDJohnny-13yo Male: NVLD

Neuropsychological profile similar to previousNeuropsychological profile similar to previous VCI=95 (37VCI=95 (37thth percentile) percentile) Reading=47Reading=47thth percentile) percentile) POI=63POI=63 (<1(<1stst percentile) percentile)

Mathematics=12Mathematics=12thth percentile) percentile) PS=73 (3PS=73 (3rdrd percentile) percentile)

Socioemotional:somewhat hyperemotional and immature, Socioemotional:somewhat hyperemotional and immature, now avoiding others but often complains of wanting now avoiding others but often complains of wanting peers. Active in sportspeers. Active in sports

Does the WISC tap EF problems?Does the WISC tap EF problems?

Verbal tasks knowledge-basedVerbal tasks knowledge-based Performance tasks require more EFPerformance tasks require more EF

Initiate: Initiate: time to respondtime to respondDB > DFDB > DFPoor retrieval on Information vs recognitionPoor retrieval on Information vs recognition

Inhibit: Inhibit: PC or MR impulsive; PC or MR impulsive;

can correct errors can correct errors Stimulus-bound BDStimulus-bound BD

Shift:Shift: Carry-over on verbal Carry-over on verbal

taskstasks Carry-over on DSCarry-over on DS

Organize:Organize: BD vs OA (don’t you miss it?!)BD vs OA (don’t you miss it?!) Comprehension-verbal Comprehension-verbal

organizationorganization

Plan:Plan: MazesMazes Problem solving approach on Problem solving approach on

BD, OABD, OA

Monitor:Monitor:Self-corrects errorsSelf-corrects errorsAble to compare product with stimulus (BD)Able to compare product with stimulus (BD)

Working Memory:Working Memory:Digits Reversed - manipulating digitsDigits Reversed - manipulating digitsArithmetic (repetitions)Arithmetic (repetitions)Letter NumberLetter Number

Does the CELF measure EFDoes the CELF measure EF

Sentence Structure/Semantic Relationships Sentence Structure/Semantic Relationships versus Concepts and Directionsversus Concepts and Directions

Word Classes (working memory)Word Classes (working memory) Recalling sentences (loss of detail or Recalling sentences (loss of detail or

meaning)meaning) Sentence Assembly/Semantic Relationships-Sentence Assembly/Semantic Relationships-

Impulsive respondingImpulsive responding Word AssociationsWord Associations

Do achievement tests tap EF?Do achievement tests tap EF?

Impulsive reader but accurate decoder Impulsive reader but accurate decoder Comprehension of shorter but not longer passagesComprehension of shorter but not longer passages Word retrieval problemsWord retrieval problems Poor self-monitoring when completing arithmetic Poor self-monitoring when completing arithmetic

(misreads signs, misaligns info)(misreads signs, misaligns info) Basic writing mechanicsBasic writing mechanics Longer written work/output (organized or random Longer written work/output (organized or random

thoughts) (WIAT-2, TOWL etc.)thoughts) (WIAT-2, TOWL etc.)

Do memory tests tap EF?Do memory tests tap EF?

CVLT-C (semantic versus serial, intrusions, CVLT-C (semantic versus serial, intrusions, perseverations, proactive/retroactive perseverations, proactive/retroactive interference)interference)

Story memory-organization of recall?Story memory-organization of recall? Visual memory (scanning?, recall of Visual memory (scanning?, recall of

complex versus simpler info)complex versus simpler info)

Motor TestsMotor Tests

PegboardPegboard Finger Tapping TestFinger Tapping Test Stressed Gaits examinationStressed Gaits examination Timed Motor ExaminationTimed Motor Examination Graphomotor speed/precision testsGraphomotor speed/precision tests

Before beginning a task

1. Self-awareness of ability: Ask the child whether the task will be easy or

difficult and to explain the choice of answer. If relevant, ask for a prediction of

performance.

2. Goal-setting, strategic behavior: Ask the child what their goal is Ask him/her to explain plans for achieving his

or her goal.

Observational procedures for assessing EF during task performance (Ylvisaker)

During the task

1. Initiation: If appropriate, create opportunities for initiation (e.g., insufficient materials, requiring the child to initiate a request).

2. Inhibition: If appropriate, create some distractions that would require active inhibition from the child.

3 Self-monitoring: Ask the child how he or she is doing.

4. Strategic behavior/ Problem-solving: If appropriate, create obstacles that would require active, flexible

problem-solving.

After completion of the task

1. Self-evaluation: Ask the child how he or she did and how the results compare to their prediction.

2. Strategic behavior and problem-solving: Ask what the child did to succeed List relevant strategic procedures Ask the child whether he or she used them or

whether they might be useful.