J Head Trauma RehabilCopyright c© 2015 Wolters Kluwer Health, Inc. All rights reserved.

Executive Functions Deficits AfterSevere Traumatic Brain Injury: TheGREFEX Study

Philippe Azouvi, MD, PhD; Claire Vallat-Azouvi, PhD; Pierre-Alain Joseph, MD;Thierry Meulemans, PhD; Celine Bertola, MA; Didier Le Gall, PhD; Anne Bellmann, PhD;Martine Roussel, PhD; Francoise Coyette, MA; Marianne Krier, MA; Catherine Franconie, MA;Claire Bindschadler, PhD; Momar Diouf, MA; Olivier Godefroy, MD, PhD; on behalf of theGREFEX∗ Study Group (Groupe de Reflexion sur l’Evaluation des Fonctions Executives)

Objective: To assess the sensitivity of traditional neuropsychological tests and of a behavioral inventory of executivedisorders in a large sample of patients with chronic severe traumatic brain injury. Methods: A total of 112 patientswere compared with 780 healthy controls from a larger database. The GREFEX battery included 7 widely used testsand the Behavioral Dysexecutive Syndrome Inventory (proxy rating). A previously described statistical methodol-ogy was used, controlling for age, education, and gender. Summary scores were computed and performance wasdichotomized on the basis of 5th percentile cutoffs from controls’ z scores. Results: The frequency of cognitiveimpairment was high (55.4%) but lower than that of behavioral changes (81.5%). Double dissociations were observedbetween cognitive and behavioral assessments. Behavioral changes exhibited larger effect-sizes as compared withcognitive impairments. Logistic regression analysis showed that 3 cognitive tests (verbal fluency, Stroop reading,and Trail Making Test-B) and 3 behavior z scores (hypoactivity, anticipation, and hyperactivity) best discriminatedpatients from controls. Conclusion: Behavioral changes were more frequent and severe than cognitive deficits, atleast as assessed with traditional testing. The present results also suggest that a shortened battery may provide arapid screening method with reasonable sensitivity to detect deficits of executive functions in patients with severetraumatic brain injury. Key words: assessment, behavior, cognition, executive functions, rehabilitation, traumatic brain injury

Author Affiliations: AP-HP, hopital Raymond Poincare, service deMedecine Physique et de Readaptation, Garches (Dr Azouvi); EA 4047HANDIReSP, Universite de Versailles Saint Quentin (Drs Azouvi andVallat-Azouvi); Antenne UEROS et SAMSAH92- UGECAMIDF,Hopital Raymond Poincare, Garches (Dr Vallat-Azouvi); EA 4136Universite de Bordeaux (Dr Joseph); Neuropsychology Unit, University ofLiege, Liege, Belgium (Dr Meulemans); CRF Calve, Fondation Hopale,Berck, France (Ms Bertola); LPPL, EA 4638, and Unite deNeuropsychologie, CHU and Universite d’Angers, Angers, France(Dr Le Gall); Clinique romande de readaptation, Sion, Switzerland(Dr Bellmann); Department of Neurology and Laboratory of FunctionalNeurosciences (EA 4559), University Hospital of Amiens, Amiens, France(Drs Roussel and Godefroy); Saint Luc University Hospital Bruxelles,Brussels, Belgium (Ms Coyette); Institut regional de medecine physique etde readaptation, Nancy, France (Ms Krier); University Hospital, Nimes,France (Ms Franconie); Lausanne University Hospital, Lausanne,Switzerland (Dr Bindschadler); and Department of Biostatistics,University Hospital of Amiens, Amiens, France (Mr Diouf).

We thank the GREFEX Study Group. The following centers and investigatorsparticipated in the GREFEX cooperative study (n = the number of patientsincluded at each center; investigators): Amiens University Hospital (F) (n =183; O. Godefroy and M. Roussel), Angers University Hospital (F) (n = 19;D. Le Gall), Heliomarin Rehabilitation Center Berck (F) (n = 15; C. Bertola),Bordeaux University Hospital (F) (n = 28; J. M. Giroire and P. A. Joseph),Saint Luc University Hospital Bruxelles (B) (n = 6; X. Seron, F. Coyette),Cholet General Hospital (F) (n = 8; E. Bretault and I. Bernard), OttigniesWilliam Lennox Center (B) (n= 3; M. Leclercq), Garches University Hospital(F) (n = 9; P. Azouvi and C. Vallat-Azouvi), Grenoble University Hospital

PATIENTS who survive a severe traumatic braininjury (TBI) often suffer from long-lasting

neuropsychological impairments that impede theirfunctional and social recovery.1–5 Deficits of executivefunctions are usually considered as a core impairment

(F) (n = 24; P Pollack and C. Mosca), Lausanne University Hospital (CH)(n = 9; C. Bindschadler ), Lay St Christophe Rehabilitation Center (F) (n =3; M. Krier), Liege Department of Cognitive Sciences (B) (n = 19; T. Meule-mans and V. Marquet), Lille Stroke Center University Hospital (F) (n = 26;D. Leys and M. Roussel), Nantes University Hospital (F) (n = 8; P. Renouand M. Vercelletto), Nice University Hospital (F) (n = 6; E. Michel and P.Robert ), Nımes University Hospital (F) (n = 15; P. Labauge and C. Fran-conie), Paris-La Salpetriere University Hospital Neurology Department (F)(n = 18; B. Pillon and B. Dubois), Paris-La Salpetriere University HospitalGeriatry Department (F) (n = 13; B. Dieudonnee and M. Verny), Paris-Broca University Hospital (F) (n = 5; H. Lenoir and J. De Rotrou), RouenUniversity Hospital (F) (n = 56; D. Hannequin and S. Bioux), and SionRehabilitation Clinic (CH) (n = 12; J. Fuchs, A. Bellmann and P. Vuadens).

The authors declare no conflicts of interest.

Corresponding Author: Philippe Azouvi, MD, PhD, Department of PhysicalMedicine and Rehabilitation, Raymond Poincare Hospital, 92380 Garches,France (philippe.azouvi@rpc.aphp.fr).

DOI: 10.1097/HTR.0000000000000169

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

1

2 JOURNAL OF HEAD TRAUMA REHABILITATION

after severe TBI,6–11 presumably attributed to lesionsor dysfunctions of anterior prefrontal areas and re-lated brain networks.12,13 “Executive functions” is anumbrella term referring to a wide range of high-levelcognitive abilities involved in novel and goal-directedtasks, mainly mediated by the frontal lobes.14 Indeed,severe TBI has been found associated with deficits ininitiation,15 planning,16 inhibition,17 flexibility and taskswitching,16 sustained attention,18 divided attention,19

and executive aspects of working memory.20 Suchimpairments are frequently associated with slowedinformation processing,21,22 impairment of socialcognition,23 personality and behavioral changes,24 andimpaired awareness of mental changes.25

Neuropsychological evaluation therefore appears ofmajor importance to delineate TBI-related disability andto help clinicians in establishing a prognosis and inrehabilitation planning. However, although there area great number of paper-and-pencil and computerizedtests available to clinicians, little is known on their sen-sitivity and specificity to the effect of TBI and on theirability to detect executive dysfunctions in everyday life.In addition, assessments methods may vary from onestudy to the other, and as a consequence, the profileof executive functions deficits remains relatively poorlyidentified.

Deficits of executive functions are notably difficultto assess, because of a number of methodological is-sues. First, performance on tests of executive functions ishighly dependent on premorbid characteristics, includ-ing age and education, which may act as confounders inneuropsychological assessment. Second, executive func-tions are specifically required in novel, nonroutine, andopen-ended situations, which are quite different fromtypical testing situations. This may explain why somepatients perform within normal limits on standard tests,while demonstrating clinically significant disorders indaily life.26–28 Third, the term “executive functions” un-dercovers a wide range of cognitive and behavioral abil-ities and it may be assumed that a wide range of testsand assessments is required, to tap the different exec-utive subdomains.6 Finally, factors such as motivationand emotions might influence performance.

Wilde et al29 reported the selection of outcome mea-sures by the interagency TBI Outcomes Workgroup toassess outcome after TBI. Among the neuropsychologi-cal domain, core measures (defined as valid, robust, andwidely applicable outcome measures with proven util-ity in TBI) included 1 memory test, the Rey Auditoryverbal Learning test, 1 executive test (the Trail MakingTest), and 1 measure of speed of information processing(the WAIS processing of information speed index). Sup-plemental measures, recommended for consideration inTBI research focusing on specific topics or populations,included a few more cognitive tests. However, no infor-

mation was given on the relative sensitivity and speci-ficity of these measures. Actually, sensitivity/specificityis not available for most neuropsychological measures,thus making difficult reliable assumptions on the patho-logical nature of a given patient’s performance on onegiven test.

The main objective of this study was to assess thesensitivity and specificity of a standardized batteryof executive functions, addressing both cognitive andbehavioral aspects30 in patients with severe TBI at asubacute-chronic stage. It is a part of a larger multi-center study on assessment of executive functions, theGREFEX database.30 For this purpose, we used previ-ously validated diagnostic criteria of executive functionsdeficits and a recently described statistical methodologypermitting to control the effect of confounders on testperformance on a large sample of patients with TBIcompared with healthy individuals.1 Briefly, the trans-formed scores of cognitive and behavioral measureswere entered in a series of linear regression analysiswith age, educational level, gender, and an interactionterm (age × education level) and significant factors inhealthy controls were retained. Regression coefficientscomputed in healthy controls were used to calculatestandardized residuals, that is, z scores. Performancewas then dichotomized (normal, impaired) on the basisof 5th percentile cutoffs determined from z scores ofhealthy controls, thus permitting to compare patientsand controls on each one of the individual scores andon combined summary scores.

METHODS

Participants

Patients referred for cognitive complaint with a di-agnosis of severe TBI, as defined by a lowest Glas-gow Coma Scale score31 of 8 or less were recruitedby 11 rehabilitation and neurological centers partici-pating to the GREFEX study. Inclusion criteria wereage between 16 and 90 years and Mini-Mental StateExamination (MMSE)32 score 16/30 or more. Exclu-sion criteria were (1) severe sensorimotor deficit, hem-ineglect or aphasia precluding cognitive assessment; (2)illiteracy; (3) alcoholism or severe general comorbid-ity; (4) previous neurologic and psychiatric diseases ex-cept for depression or anxiety; (5) recent introductionof psychoactive or antiepileptic medication; and (6)absence of informed consent. A total of 112 patientswere included. Patients’ demographic and baseline neu-ropsychological characteristics are presented on Table1. They were predominantly young males, who sus-tained a severe TBI (as indicated by a mean PTA du-ration of 7 weeks). Most of them were at a chronicstage, more than 2 years postinjury on average. Thegeneral cognitive assessment was performed according

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

Executive Functions Deficits After Severe Traumatic Brain Injury 3

TABLE 1 Demographic and baselineneuropsychological characteristics ofpatientsa

Characteristics(number ofavailable data) Mean (or n) SD (or %)

Age (n = 112) 30.39 10.86Gender (male,

n = 90)68 75.56%

Education level1 26 (23.2%)2 48 (42.8%)3 38 (33.9%)

Time since injury(weeks, n = 59)

129.97 175.87

Coma duration(weeks, n = 61)

3.62 4.65

PTA duration(weeks, n = 44)

7.08 9.30

MMSE score(n = 72)

27.13 2.44

Digit span (n = 108) 5.37 1.22Impairment of oral

comprehension(n = 82)

5 6.1%

Oral expression(n = 85)

9 10.6%

Visuospatial abilities(n = 83)

23 27.7%

Long-term memory(n = 92)

51 55.4%

Abbreviations: MMSE, Mini-Mental State Examination; PTA,posttraumatic amnesia.aEducation level: 1 = primary; 2 = secondary; 3 = higher level.From Folstein et al.32

to a standardized battery varying across centers. Avail-able data showed the prominence of memory impair-ments (see Table 1) followed by visuospatial and lan-guage deficits. A mild to moderate sensorimotor deficitwas reported in 18 patients (27%). Their performancewas compared to the full set of 780 healthy controls ofthe GREFEX database (age: 48.9 ± 19 years; gender fe-male: 45%; education primary/secondary/higher level:21/33/45%; MMSE: 28.8 ± 1.3; digit span: 5.6 ± 1),according to a validated procedure, which was found toprovide the best diagnostic accuracy.1

Procedures

Examination of executive functions used theGREFEX battery and was based on previously validatedcriteria of executive disorders30 Briefly, cognitive ex-ecutive functions deficits were assessed using a Frenchadaptation of 7 tests30: phonemic and categorical Verbalfluency,33 Trail Making test,34 Stroop test,35 ModifiedCard Sorting test,36 6 elements test,26 Brixton test,37

and box-crossing and digit span dual task38 (see theAppendix for a short description of the cognitive tests).Dysexecutive behavioral changes were assessed usingthe Behavioral Dysexecutive Syndrome Inventory, astructured interview of an informant assessing changescompared with previous behavior in 12 domains30: (1)hypoactivity with apathy-abulia; (2) difficulties for an-ticipation, planning, and initiation of activities; (3) dis-interest and indifference to his own concern and others;(4) hyperactivity-distractibility-psychomotor instability;(5) irritability-impulsivity-aggressiveness; (6) euphoria,emotional lability, and moria; (7) stereotyped andperseverative behavior; (8) environmental dependency;(9) anosognosia-anosodiaphoria; (10) spontaneous con-fabulations; (11) social behavior disorders; and (12) dis-orders of sexual, eating, and urinary behavior. For eachitem, the informants were required to state whether therewas a change in comparison to preinjury. If a changewas acknowledged, they were asked to rate its severity(range: 1-3) and frequency of occurrence (range: 1-4) ona Likert-type scale, higher scores indicating more severebehavioral modifications. A global score was computedfor each question (frequency × severity, maximumscore = 12). To be qualified as dysexecutive, behavioraland cognitive impairments must not have been morereadily explained by perceptuomotor, psychiatric, orother cognitive disorders.

Statistics

Performance analysis was based on a validated frame-work for the interpretation of cognitive data that wasfound to provide the best diagnostic accuracy.1 Briefly,the 19 transformed scores (log transformation for thecompletion time in the Trail Making and Stroop tests;Box-Cox transformation for the other scores)39 of the 7cognitive tests were entered in a series of linear regressionanalysis (1 per score) with age, educational level, gender,and an interaction term (age × education level) andsignificant factors in healthy controls were retained. Re-gression coefficients computed in healthy controls wereused to calculate standardized residuals, that is, z scores(poor performance corresponded to a negative z score).An average overall cognitive summary score was computed,following a methodology previously described1 (includ-ing z scores of the following measures: completion time(corrected for the error rate in each test) in the TrailMaking test A and Stroop naming and reading subtests;errors in the interference subtest—errors in the namingsubtest of the Stroop test; perseverations in the CardSorting and Trail Making test B; average of the 2 fluencytests; categories achieved in the Card Sorting test; errorsin the Brixton test; the mu dual task index; the rankscore of the 6 elements task). Scores on the BehavioralDysexecutive Syndrome Inventory were not influenced

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

www.headtraumarehab.com

4 JOURNAL OF HEAD TRAUMA REHABILITATION

by demographic factors30 and were analyzed as z scores.The average of the 12 behavioral z scores was used asoverall behavioral summary score. The dichotomization ofperformance (normal, impaired) was based on 5th per-centile cutoffs determined from z scores of healthy con-trols. Group comparisons (patients vs healthy controls)were performed using the Student t test for continuousvariables and the Fisher exact test or chi-square test forother variables. Effect sizes were analyzed using Cohen’sd statistics (z score) and odds ratio (OR) with 95% con-fidence interval (CI) for dichotomized performance. Alarge effect size was considered for d values ≤−1.645.

A shortened battery for the diagnosis of executivefunctions deficits in TBI was determined using 2 step-wise logistic regression analyses (1 for cognitive scoresand the other for behavioral scores). Regression analyseswere performed on z scores from healthy controls andpatients. To control for multicollinearity, 15 cognitivez scores were selected: both fluency tasks, time com-pletion of 3 subtests of Stroop and 2 parts of the TrailMaking test, error on Stroop interference subtest, partA of the Trail Making test and on Brixton test, perse-verative errors on part B of the Trail Making test andModified Card Sorting test, category achieved on Mod-ified Card Sorting test, mu dual task index, and rankon 6 elements test. Regarding behavioral analysis, the12-component z scores of the inventory were submittedto regression analyses. Multicollinearity was checked ac-cording to usual methods40: Variance inflation factorwas computed and collinearity problem was consideredfor a value more than 0.4, which did not occur.

The diagnostic accuracy of z scores selected by regres-sion analyses was judged according to the scores’ abilityto discriminate between patients and healthy controls bycalculating the area under the receiver operating char-acteristic (ROC) curves (AUC) with the corresponding95% CI, sensitivity, specificity, accuracy, positive predic-tive value (PPV), and negative predictive value (NPV).A false-positive rate of 5% or less (ie, specificity ≥ 0.95)was used as a rule of thumb. The model providing op-timal diagnosis accuracy with the minimal number oftests was retained. The AUC values were compared byusing the Delong et al41 method.

P ≤.05 was considered significant unless otherwiseindicated. Statistical analyses were performed using SASsoftware (SAS Institute, Cary, North Carolina, 2006-10).

RESULTS

Cognitive executive assessment was available in thewhole sample of 112 patients (only 1 patient could notcomplete the full battery because of very severe deficitsand fatigability). However, behavioral assessment wasunfortunately not available for 58 patients, because ofthe lack of any reliable informant. These 58 patients

did not significantly differ from the 54 patients withavailable behavioral assessment regarding age and per-formance on cognitive executive tests (both P > 0.1),but they were significantly less educated (32.7% patientswere at the lowest education category vs 12.9% in thecompleted assessment group; P < .05). To minimizethe risk of reduced statistical power, sensitivity analy-ses reported later have been conducted separately forcognitive and behavioral variables, including all patientswho completed all measures within each set of variables.However, to distinguish the relative importance of cog-nitive and behavioral variables, additional analyses ofcognitive tests were computed in the subsample of 54patients who completed both behavioral and cognitiveassessments.

Frequency of executive functions deficits

Both overall cognitive and behavioral summary scoreswere impaired in the TBI group as compared withhealthy controls (P = .0001, both) (see Table 3). Thefrequency of executive disorders was high (cognitivedeficits: 55.4%, 95% CI: 46-65; behavioral disorders:81.5%, 95% CI: 71-92).

The frequency of combined/dissociated impairmentswas examined in the 54 patients for whom both behav-ioral and cognitive assessments were available. A deficitof executive functions was found in 47 patients (87%;95% CI: 78-96): 23 (49%) had combined behavioral andcognitive disorders, 3 (6.4%), a pure cognitive impair-ment, and 21 (45%), a pure behavioral disorder.

Profile of executive disorders

Analysis of individual cognitive tests z scores showedthat all cognitive z scores were impaired (see Table 2).A large effect size was found only for the Stroop read-ing time (−1.747). Other tests with moderately largeeffect sizes (below −1.5) were Trail Making Test-B (timeof completion) and Stroop naming time. Test z scoreswere impaired, in comparison to healthy controls, in8.2% (TMT-A, errors) to 47.3% (Stroop reading time) ofpatients. Only the 3 subtests obtaining the largest effectsizes were impaired in more than 40% of cases.

As previously mentioned, this analysis was also com-puted in the subsample of 54 patients who completedboth cognitive and behavioral assessments, to allow a di-rect comparison of both sets of measures. These results,which appear in Table 3, were quite comparable withthose obtained in the whole sample. However, thesepatients tended to show a less severe impairment thanthe whole group. Indeed, overall cognitive score was im-paired in 48.1% of cases (vs 55.4% in the whole group),and no individual score obtained an effect size lessthan −1.5. However, the most severely impaired testsscores were the same as in the whole sample (times of

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

Executive Functions Deficits After Severe Traumatic Brain Injury 5

TABLE 2 Cognitive battery: Component mean z scores (± Standard Deviation), effectsize (d), frequency (%) of impairment, and odds ratio in the whole sample (n = 112)

Impairment

Mean SD P d 95% CI % OR 95%CI

Overall cognitive score − 0.907 0.783 .000 − 0.909 − 1.059 − 0.759 55.4 17.8 11.1 28.7Stroop

Naming time − 1.539 1.471 .000 − 1.539 − 1.824 − 1.254 40.2 12.8 7.72 21.2Naming error − 0.799 1.752 .000 − 0.798 − 1.134 − 0.462 25.0 6.45 3.73 11.1Reading time − 1.747 1.669 .000 − 1.747 − 2.068 − 1.426 47.3 17.12 10.4 28.2Reading error − 0.559 2.148 .008 − 0.559 − 0.968 − 0.150 11.6 3.91 1.92 7.97Interference time − 1.280 1.261 .000 − 1.279 − 1.526 − 1.033 33 9.4 5.61 15.8Interference error − 0.622 1.558 .000 − 0.618 − 0.919 − 0.318 15.2 3.41 1.84 6.31

6 elements − 1.007 1.580 .000 − 1.000 − 1.312 − 0.689 24.3 5.96 3.42 10.4Trail Making test

Part A time − 1.287 1.269 .000 − 1.287 − 1.537 − 1.036 31.8 8.89 5.27 14.99Part A error − 0.112 1.104 .318 − 0.112 − 0.332 0.109 8.2 1.8 0.84 3.87Part B time − 1.583 1.605 .000 − 1.583 − 1.895 − 1.271 41.8 13.7 8.26 22.7Part B error − 0.442 1.312 .001 − 0.443 − 0.701 − 0.184 11.8 2.56 1.31 4.99Part B pers. − 0.492 1.557 .002 − 0.493 − 0.799 − 0.188 15.7 3.67 1.98 6.82

Brixton − 0.645 1.060 .000 − 0.646 − 0.850 − 0.442 11.1 2.38 1.2 4.74Dual task − 0.495 1.129 .000 − 0.496 − 0.712 − 0.279 10.3 2.2 1.05 4.61Fluency

Phonemic − 0.956 1.190 .000 − 0.952 − 1.187 − 0.717 30.6 7.76 4.63 13Categorical − 1.278 1.287 .000 − 1.274 − 1.527 − 1.021 39.6 11.8 7.17 19.5

MCSTCategory − 0.847 1.640 .000 − 0.840 − 1.159 − 0.521 18.2 4.2 2.3 7.66Error tot − 0.852 1.083 .000 − 0.849 − 1.056 − 0.642 18.3 4.39 2.4 8.04Pers. − 0.863 1.148 .000 − 0.866 − 1.097 − 0.635 17.4 4.1 2.22 7.56

Abbreviations: CI, confidence interval; MCST, Modified Card Sorting Test; OR, odds ratio.

completion of Stroop reading and naming subtests andof TMT-A). The highest effect size (for the Stroop read-ing time as in the whole group) was −1.467.

Regarding behavior, all behavioral z scores were im-paired as compared with healthy controls (see Table 4).Effect sizes ranged from −0.94 to −3.94. Eight items ob-tained large effect-sizes (≤−1.645): hypoactivity, antic-ipation difficulty, euphoria, hyperactivity, environmen-tal dependency, anosognosia, confabulation, and sexualconduct disorders. Behavioral scores were impaired in13.0% to 63.0% (anticipation) of patients. A frequencyof impairment 50% or more was observed for hypoac-tivity, anticipation difficulty, and anosognosia.

Selection of tests for a shortened battery

Logistic regression analysis showed that the followingcognitive component z scores were associated with TBI:categorical verbal fluency (OR: 1.75; 95% CI: 1.34-2.28;P = .0001), time completion of the Stroop reading (OR:1.98; 95% CI: 1.57-2.5; P = .0001) and Trail Makingtest B subtests (OR: 1.79; 95% CI: 1.4-2.27; P = .0001).On ROC curve analysis, this 3-factor model providedan AUC of 0.864 (95% CI: 0.824-0.91), which did notdiffer (P = 0.7) from the whole battery (AUC = 0.868;

95% CI: 0.832-0.91). The 3-factor battery provided asensitivity of 0.55 (accuracy: 0.9, PPV: 0.63, NPV: 0.93)and did not differ (McNemar test: P = .3) from the fullcognitive battery.

The behavioral z scores associated with TBI in lo-gistic regression analysis were hypoactivity (OR: 1.62;95% CI: 1.26-2.09; P = .0001), anticipation disorder(OR: 1.67; 95% CI: 1.22-2.3; P = .001), and hyperactiv-ity (OR: 1.39; 95% CI: 1.13-1.72; P = .002). On ROCcurve analysis, it provided an AUC 0.894 (95% CI: 0.83-0.958), which did not differ (P = .4) from the wholebattery (AUC = 0.911; 95% CI: 0.857-0.964). The sen-sitivity was 0.72 (accuracy: 0.87, PPV: 0.91, NPV: 0.86)and did not differ (McNemar test: P = .13) from the fullinventory.

In the subgroup of 54 patients with full assessment,the logistic regression analysis selected the same com-ponent z scores associated with TBI for both cognitivescores (categorical verbal fluency [OR: 1.86; 95% CI:1.32-2.61; P = .0001], time completion of the Stroopreading [OR: 1.86; 95% CI: 1.38-2.5; P = .0001], andTrail Making test B subtests [OR: 1.57; 95% CI: 1.17-2.12; P = .003]) and behavioral scores (hypoactivity[OR: 1.624; 95% CI: 1.26-2.09; P = .0001], anticipationdisorder [OR: 1.674; 95% CI: 1.22-2.3; P = .001], and

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

www.headtraumarehab.com

6 JOURNAL OF HEAD TRAUMA REHABILITATION

TABLE 3 Cognitive battery: Component mean z scores (± standard deviation), effect size(d), frequency (%) of impairment and odds ratio in the 54 patients with full assessment

Impairment

Mean SD P d 95%CI % OR 95%CI

Overall Cognitive score − 0.775 0.733 .0001 − 0.777 − 0.979 − 0.575 48.1 13.4 7.26 24.6Stroop

Naming time − 1.352 1.533 .000 − 1.352 − 1.776 − 0.927 40.7 13.1 6.92 24.8Naming error − 0.554 1.582 .014 − 0.553 − 0.991 − 0.115 18.5 4.4 2.04 9.46Reading time − 1.466 1.526 .000 − 1.467 − 1.889 − 1.044 46.3 16.43 8.74 30.9Reading error − 0.198 1.350 .295 − 0.198 − 0.573 0.177 7.4 2.38 0.79 7.16Interference time − 1.190 1.284 .000 − 1.190 − 1.548 − 0.833 29.6 8.02 4.09 15.7Interference error − 0.548 1.305 .004 − 0.545 − 0.908 − 0.181 13 2.84 1.2 6.72

6 elements − 0.658 1.544 .005 − 0.651 − 1.096 − 0.207 16 3.53 1.55 8.08Trail Making test

Part A time − 1.072 1.144 .000 − 1.072 − 1.395 − 0.749 28.3 7.52 3.79 14.93Part A error − 0.162 1.183 .335 − 0.162 − 0.496 0.172 9.4 2.11 0.79 5.64Part B time − 1.173 1.371 .000 − 1.173 − 1.557 − 0.788 35.8 10.7 5.54 20.5Part B error − 0.278 1.134 .053 − 0.279 − 0.599 0.042 7.5 1.56 0.53 4.56Part B pers. − 0.453 1.637 .054 − 0.454 − 0.915 0.007 13.5 3.06 1.29 7.27

Brixton − 0.617 1.141 .000 − 0.617 − 0.940 − 0.295 15.1 3.39 1.49 7.72Dual task − 0.429 1.182 .004 − 0.429 − 0.720 − 0.138 10 2.13 0.79 5.7Fluency

Phonemic − 0.918 1.224 .000 − 0.914 − 1.198 − 0.631 34 9.03 4.7 17.4Categorical − 1.176 1.245 .000 − 1.172 − 1.522 − 0.821 37.7 10.9 5.74 20.8

MCSTCategory − 0.695 1.318 .000 − 0.688 − 1.059 − 0.317 17 3.87 1.74 8.61Error tot − 0.825 1.086 .000 − 0.822 − 1.131 − 0.513 17 3.99 1.79 8.92Pers. − 0.697 1.122 .000 − 0.700 − 0.984 − 0.416 13.2 2.96 1.23 7.08

Abbreviations: CI, confidence interval; MCST, Modified Card Sorting Test; OR, odds ratio.

hyperactivity [OR: 1.39; 95% CI: 1.13-1.72; P = 0.002]).On ROC curve analysis, this 6-factor model providedan AUC of 0.923 (95% CI: 0.869-0.977), which did notdiffer (P = 0.3) from the whole battery (AUC = 0.938;95% CI: 0.902-0.975).

DISCUSSION

The objective of this study was to assess the sensi-tivity of widely used paper-and-pencil tests of executivefunctions and of a behavioral inventory in patients with

TABLE 4 Behavioral executive performance expressed as mean z scores (± standarddeviation), effect size (d), frequency (%) of impairment and odds ratio (n = 54)

Impairment

Behavior Mean SD P d 95%CI % OR 95%CI

Overall score −3.571 3.193 .000 − 3.576 − 4.468 − 2.683 81.5 45.5 16.11 112.3Hypoactivity −3.946 3.576 .000 − 3.948 − 4.943 − 2.953 61.1 48.7 13.6 174.03Anticipation −2.309 2.244 .000 − 2.312 − 2.955 − 1.669 63.0 21.6 8.4 55.7Disinterest −1.505 2.053 .000 − 1.508 − 2.101 − 0.914 46.3 10.9 4.3 27.98Euphoria −2.657 4.335 .000 − 2.659 − 3.858 − 1.460 33.3 11.5 3.64 36.3Irritability −1.486 2.094 .000 − 1.490 − 2.094 − 0.886 42.6 11.1 4.15 29.9Hyperactivity −3.911 5.518 .000 − 3.913 − 5.432 − 2.394 38.9 29.9 6.65 134.5Perseveration −1.289 2.659 .001 − 1.292 − 2.044 − 0.541 29.6 4.63 1.83 11.7Dependency −18.690 56.950 .019 − 18.690 − 34.230 − 3.142 13.0Anosognosia −1.904 2.241 .000 − 1.906 − 2..548 − 1.264 50.0 23.0 7.4 71.5Confabulation −18.948 56.908 .018 − 18.949 − 34.482 − 3.416 13.0Social conduct −0.944 2.170 .003 − 0.946 − 1.570 − 0.323 24 .1 9.83 2.66 36.4Sexual conduct −1.860 4.653 .005 − 1.862 − 3.147 − 0.577 18.5 7.05 1.85 26.9

Abbreviations: CI, confidence interval; OR, odds ratio.

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

Executive Functions Deficits After Severe Traumatic Brain Injury 7

chronic severe TBI. Patients were quite representative ofthe severe TBI population, as they were a large groupof patients from more than 11 different centers, mainlyrehabilitation units. They could be compared to a largesample of healthy controls, using a methodology per-mitting to control for the confounding effects of demo-graphic variables, such as age and education.

It should be emphasized that the main objective ofthis study was not to determine if patients with TBI, asa group, performed poorer than controls on a range oftests, but rather to assess sensitivity and specificity ofthese measures. For this purpose, the first step was toset the most reliable pathological cutoff score, which,according to our previous validation study,1 was the 5thpercentile of the control group. However, sample-sizecalculation indicates that there is a need of about 500healthy controls to obtain a reliable measure of this cut-off, with a 95% CI of no more than 2% (ie, a 95% CIrange of 3%-7%, corresponding to less than 7% false-positive findings).1,42 This is the reason why we decidedto use a large healthy control group, obviously not de-mographically matched to the patient sample, but theeffects of age, education, and gender (when present) weresubsequently adjusted in regression analyses. The largesample size of the healthy control group is a strength ofthis study, permitting to establish a reliable cutoff scorefor each cognitive measure.

The rationale of the size effect analysis was to fit withcurrent methodological recommendations that haveshown the major limitations of the “null hypothesis sig-nificance testing” approach (especially with large sam-ples as in the present study).43,44 In addition, such anal-ysis is mandatory to determine the neuropsychologicalprofile of TBI as previous analyses30 showed that profileof executive functions disorders varies across conditions.Such sensitivity analyses were therefore aimed at exam-ining the sensitivity of tests/questionnaire (and scores)to detect TBI associated with disorders of executive func-tions.

In addition, both OR and frequency of impairmentswere computed. Indeed, OR depends on the frequencyof impairment in the control group (which itself is influ-enced by several factors that may differ across domains).Indeed, some pathological behaviors that never occur inhealthy participants will result in artificially high effectsizes and OR, while still remaining quite exceptionalin patients with TBI. For such items, with very low baserates, frequency of occurrence appears to be a more clin-ically relevant measure. This is why we provided bothOR and frequency of impairment that may be moreclinically relevant.

The results showed that about one half of the patients(55.4%) performed significantly poorer than healthycontrols on the paper-and-pencil battery, while a quitelarger proportion (81.5%) were considered as present-

ing behavioral changes as compared with preinjury bytheir closest relative. This discrepancy was even moremarked if only considering the 54 patients who com-pleted both cognitive and behavioral assessments, as theoverall cognitive score was impaired in only 48.1% ofthis subgroup. Interestingly, double dissociations werefound between cognitive and behavioral assessments.The more frequent dissociation was due to patientsperforming within normal limits on the cognitive bat-tery while demonstrating clinically significant behav-ioral changes. However, a few (n = 3) patients showedthe reverse pattern (impaired cognitive testing withoutdetectable behavioral changes according to relative’srating).

Main cognitive impairments were related to measuresof initiation, information generation, and speed of pro-cessing. Logistic regression analyses showed that 3 cog-nitive tests (verbal fluency, time of completion of theStroop reading subtest, and Trail Making Test-B) and3 behavioral items (hypoactivity, anticipation disorder,and hyperactivity) were best able to discriminate patientsfrom healthy controls. These items provided diagnosticaccuracy of executive functions deficits after TBI simi-lar to that of the whole GREFEX battery. So, a short-ened battery, including these items, may provide a rapidscreening with reasonable sensitivity to detect executivefunctions impairments in patients with chronic severeTBI. However, although this finding may be useful forclinical practice, caution is needed regarding the influ-ence of base rates of this cognitive impairment on posi-tive and negative predictive power. This finding shouldbe replicated in other external patient samples.

These findings raised concerns about the relativelylow sensitivity and effect size of commonly used paper-and-pencil tests, in patients suffering from quite severeTBI. These results are, however, in accordance with pre-vious studies. For example, Levin et al3 found that per-formance on the modified Wisconsin card Sorting Testof patients with severe TBI 1 year after the injury didnot significantly differ from that of matched healthycontrols. More recently, Fortin et al45 also found onlymarginal effects of mild to severe TBI on the num-ber of categories achieved on the modified Wisconsincard Sorting Test at a chronic stage. However, divergentresults have been reported in selected samples, such asolder patients46 or patients at a very early stage after theinjury.21 In a similar way, a number of studies reported anormal performance of patients with TBI on the Stroopor on the Trail Making Test, except for slowness.47–50

Also, the poor sensitivity of the Brixton spatial anticipa-tion test and of the box-crossing and digit span dual-taskare in accordance with previous reports.11,20

In this study, the most discriminating measuresamong cognitive testing appeared to be verbal fluencyand speed of processing of the Stroop and Trail Making

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

www.headtraumarehab.com

8 JOURNAL OF HEAD TRAUMA REHABILITATION

tests. The high sensitivity to the effect of TBI of verbalfluency tests is in accordance with previous studies,15

although there have been negative findings at a very latestage (10 years) after TBI.11 The fact that tests, includ-ing a speed component (such as the Stroop and TrailMaking), were the most sensitive to the effect of TBIis also not surprising and highlights the major effect ofslowed information processing in patients with severeTBI.4,47 Indeed, although this has been a matter of de-bate, previous research found that it may be difficult tofind significant deficits of cognitive functions beyondthe effect of slowed processing after severe TBI.19,22,48

This also raises questions about the nature of underly-ing deficits. Indeed, it is not clear whether an impairedperformance on such tests really reflects an impairmentof executive functions, or if it is actually only a conse-quence of slowed information processing. For example,surprisingly, the interference condition of the Strooptask (assumed to reflect an executive inhibition com-ponent) appeared to be a less discriminative measurethan the nonexecutive reading and color-naming sub-tasks. Unfortunately, no neuroimaging data were avail-able in this study, so any considerations about the un-derlying brain lesions or dysfunctions remain purelyspeculative.

The fact that paper-and-pencil tests had a (relatively)low sensitivity does not mean that patients did not suf-fer from impairments of higher-order cognitive func-tions. It is possible that more complex tasks, such asthe Brown-Peterson task of short-term memory withinterference,20,51 or more demanding dual tasks19 couldbe able to detect significant impairments. However, theobjective of the present study was to assess the sensitiv-ity of tests that are widely used in clinical practice, easilyavailable throughout the world, and for which norma-tive data are available, which is not the case for morecomplex tasks.

This lack of sensitivity is the reason why more ecolog-ically valid, open-ended tasks have been proposed to im-prove testing sensitivity. Examples of such tasks includethe Multiple errands Test,26 the Route Finding test,52,53

the Executive Secretarial Task,54 the Cooking Task,28,55

or the 6 elements task.26 Such tasks have been found toreflect with greater accuracy patients’ difficulties in dailylife. However, because of their multidetermined nature,they are not able to provide precise information aboutthe underlying cognitive impairments. In addition, thesensitivity of 6 elements task was found to be low inthis study and this contrasts to the usual claim that thisopen-ended task might be more sensitive.

Behavioral analyses showed that hypoactivity, antic-ipation difficulty, and hyperactivity were the most dis-criminating changes after TBI. Indeed, lack of drive(apathy) and lack of control (impulsivity) are the 2main behavioral changes that are known to occur

after TBI.56–58 In the present sample, apathy was re-ported in more than 60% of the sample, with anOR > 9. A comparison with another measure ofapathy would be useful. However, apathy seems tobe a quite complex and multidimensional constructand to date, although several scales/questionnaireshave been proposed, there is no consensus on one“gold standard” measure of apathy.57,59 Moreover, hy-poactivity and anticipation disorders seem to corre-spond to the main features of posttraumatic apatheticbehavior.57,59 These findings nevertheless suggest thatthe Behavioral Inventory is very sensitive to detect suchchanges.

The same assessment battery, analyzed with the samestatistical methods, has been recently used in patientssuffering from Alzheimer disease.60 The 2 groups can-not be directly compared, as they differed in terms ofage and global cognitive impairment (the mean MMSEscore was not surprisingly lower in the Alzheimer group:mean = 23.2 ± 2.9 vs 27.1 ± 2.4 in the TBI sample).However, different profiles of executive disorders wereobserved in the 2 groups: indeed, while cognitive im-pairments were much more frequent in the Alzheimergroup (76% vs 55.4% in the TBI group), the frequencyof occurrence of behavioral changes was quite similar(86% vs 81.5%, respectively). This again emphasizes thehigh prevalence of behavioral disorders after severe TBI,even though cognitive processes, at least as assessed withconventional paper-and-pencil testing, may remain rel-atively preserved.

A strength of this study comes from the fact that pa-tients’ performance was assessed with a battery basedon validated diagnostic criterion30 and normalized in alarge sample of healthy controls, permitting to controlfor confounders such as age and education, but also forthe rate of false-positive findings, which, to our knowl-edge, has not been controlled for in previous studies.

There are a few limitations to acknowledge. Patientswere not a random sample but were rather selected bythe participating centers (mostly rehabilitation wards).Patients referred to rehabilitation centers may not be rep-resentative of the whole TBI population, as referral maybe based on both clinical and demographic selectioncriteria.61 This is the case, however, for most neuropsy-chological studies. The present sample may neverthelessbe considered as representative of patients referred torehabilitation centers after a severe TBI. A second lim-itation is related to missing data concerning behavioralassessment, which was not available in 58 patients, dueto the lack of any reliable informant. These patients wereon average less educated than patients who completedthe whole assessment scale. This finding is probablydue to the fact that less educated participants were moreoften isolated. However, their age and their performanceon cognitive executive tests did not significantly differ

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

Executive Functions Deficits After Severe Traumatic Brain Injury 9

from that of patients with full assessment. Nevertheless,we acknowledge that it will be necessary to replicatethese findings on a separate external sample to addressthe issue of representativeness.

A third limitation of this study comes from the factthat the behavioral questionnaire was completed onlyby relatives. The validity of relatives’ ratings may bequestioned, as some of them may overestimate changes,due to an overwhelming burden, while other may tendto deny any kind of difficulties. Nevertheless, previ-ous studies using other comparable behavioral question-naires, such as the Dysexecutive Questionnaire62 for ex-ample, found that proxies’ ratings were more accuratethan patients’ ratings63 or that they did not significantlydiffer from therapists’ ratings.64 Further research shouldlook at the differences between patient’s, proxy’s, andtherapist’s rating to control for sources of bias, such aslack of awareness, denial, or overestimation due to bur-den. Finally, the control group included only healthy

participants. The study design unfortunately did not al-low us to obtain data from orthopedics controls. How-ever, although orthopedic controls are particularly use-ful for studies on mild TBI, this issue seems to be ofless importance in severe TBI, as effect sizes are usuallymuch larger.

A final limitation is related to the fact that the lo-gistic regression analysis conducted in the group of 54patients with full assessment should be taken with cau-tion, because of the relatively low participant-to-variableratio.

To conclude, the main finding of this study in a largegroup of patients with a chronic severe TBI was thattraditional paper-and-pencil tests of executive functionsmay lack sensitivity and that behavioral questionnairesare more able to capture the complex changes that mayoccur after such brain lesions. This should encourageclinicians and researchers to use such behavioral assess-ments in addition to paper-and-pencil testing.

REFERENCES

1. Godefroy O, Gibbons L, Diouf M, et al. Validation of an integratedmethod for determining cognitive ability: implications for routineassessments and clinical trials. Cortex. 2014;54C:51–62.

2. Katz D, Alexander M. Traumatic brain injury: predicting courseof recovery and outcome for patients admitted to rehabilitation.Arch Neurol. 1994;51:661–670.

3. Levin HS, Gary HE Jr, Eisenberg HM, et al. Neurobehavioral out-come 1 year after severe head injury: experience of the TraumaticComa Data Bank. J Neurosurg. 1990;73:699–709.

4. Azouvi P, Vallat-Azouvi C, Belmont A. Cognitive deficits aftertraumatic coma. Prog Brain Res. 2009;177:89–110.

5. Ponsford JL, Downing MG, Olver J, et al. Longitudinal follow-upof patients with traumatic brain injury: outcome at two, five, andten years postinjury. J Neurotrauma. 2014;31(1):64–77.

6. Busch RM, McBride A, Curtiss G, Vanderploeg RD. The compo-nents of executive functioning in traumatic brain injury. J Clin ExpNeuropsychol. 2005;27(8):1022–1032.

7. Levine B, Dawson D, Boutet I, Schwartz ML, Stuss DT. Assess-ment of strategic self-regulation in traumatic brain injury: its re-lationship to injury severity and psychosocial outcome. Neuropsy-chology. 2000;14(4):491–500.

8. Levin HS, Goldstein FC, Willliams DH, Eisenberg HM. The con-tribution of frontal lobe lesions to the neurobehavioral outcomeof closed-head injury. In: Levin HS, Eisenberg HM, Benton AL,eds. Frontal Lobe Function and Dysfunction. New York: Oxford Uni-versity Press; 1991:318–338.

9. Stuss DT. Traumatic brain injury: relation to executive dysfunc-tion and the frontal lobes. Curr Opin Neurol. 2011;24(6):584–589.

10. Wang Y, Chan RC, Shum D. Executive functions impairmentsafter traumatic brain injury. In: Levin HS, Shum D, Chan RC, eds.Understanding Traumatic Brain Injury. New York: Oxford UniversityPress; 2014:99–115.

11. Draper K, Ponsford J. Cognitive functioning ten years follow-ing traumatic brain injury and rehabilitation. Neuropsychology.2008;22(5):618–625.

12. Bigler ED. Traumatic brain injury, neuroimaging, and neurode-generation. Front Hum Neurosci. 2013;7:395.

13. Ham TE, Sharp DJ. How can investigation of network function in-form rehabilitation after traumatic brain injury? Curr Opin Neurol.2012;25(6):662–669.

14. Stuss DT, Shallice T, Alexander MP, Picton TW. A multidisci-plinary approach to anterior attentional functions. Ann N Y AcadSci. 1995;769:191–211.

15. Henry JD, Crawford JR. A meta-analytic review of verbal fluencyperformance in patients with traumatic brain injury. Neuropsychol-ogy. 2004;18(4):621–628.

16. Vilkki J. Cognitive flexibility and mental programming afterclosed-head injuries and anterior or posterior cerebral excisions.Neuropsychologia. 1992;30(9):807–814.

17. Vakil E, Weisz H, Jedwab L, Groswasser Z, Aberbuch S. Stroopcolor-word task as a measure of selective attention: efficiency inclosed-head injured patients. J Clin Exp Neuropsychol. 1995;17:335–342.

18. Whyte J, Polansky M, Fleming M, Coslett HB, Cavalucci C. Sus-tained arousal and attention after traumatic brain injury. Neuropsy-chologia. 1995;33:797–813.

19. Azouvi P, Couillet J, Leclercq M, Martin Y, Asloun S, RousseauxM. Divided attention and mental effort after severe traumatic braininjury. Neuropsychologia. 2004;42(9):1260–1268.

20. Vallat-Azouvi C, Weber T, Legrand L, Azouvi P. Working mem-ory after severe traumatic brain injury. J Int Neuropsychol Soc.2007;13(5):770–780.

21. Spikman JM, Deelman BG, van Zomeren AH. Executive function-ing, attention and frontal lesions in patients with chronic CHI.J Clin Exp Neuropsychol. 2000;22:325–338.

22. Willmott C, Ponsford J, Hocking C, Schonberger M. Factors con-tributing to attentional impairments after traumatic brain injury.Neuropsychology. 2009;23(4):424–432.

23. McDonald S. Impairments in social cognition following se-vere traumatic brain injury. J Int Neuropsychol Soc. 2013;19(3):231–246.

24. Arciniegas DB, Wortzel HS. Emotional and behavioral dyscon-trol after traumatic brain injury. Psychiatr Clin North Am.2014;37(1):31–53.

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

www.headtraumarehab.com

10 JOURNAL OF HEAD TRAUMA REHABILITATION

25. Prigatano GP, Altman IM. Impaired awareness of behaviorallimitations after traumatic brain injury. Arch Phys Med Rehabil.1990;71:1058–1064.

26. Shallice T, Burgess P. Deficits in strategy application followingfrontal lobe damage in man. Brain. 1991;114:727–741.

27. Eslinger PJ, Damasio AR. Severe disturbance of higher cogni-tion after bilateral frontal lobe ablation: patient EVR. Neurology.1985;35:1731–1741.

28. Chevignard M, Pillon B, Pradat-Diehl P, et al. An ecologi-cal approach to planning dysfunction: script execution. Cortex.2000;36(5):649–669.

29. Wilde EA, Whiteneck GG, Bogner J, et al. Recommendations forthe use of common outcome measures in traumatic brain injuryresearch. Arch Phys Med Rehabil. 2010;91(11):1650–1660 e1617.

30. Godefroy O, Azouvi P, Robert P, Roussel M, Legall D, Meule-mans T. Dysexecutive syndrome: diagnostic criteria and validationstudy. Ann Neurol. 2010;68(6):855–864.

31. Teasdale G, Jennett B. Assessment of coma and impaired con-sciousness: a practical scale. Lancet. 1974;2:81–84.

32. Folstein MF, Folstein SE, McHugh PR. Mini-mental state. J Psy-chiatr Res. 1975;12:189–198.

33. Cardebat D, Doyon B, Puel M, Goulet P, Joanette Y. Evocationlexicale formelle et semantique chez des sujets normaux. Perfor-mances et dynamiques de production en fonction du sexe, de l’ageet du niveau d’etude. Acta Neurol Belg. 1990;90:207–217.

34. Reitan RM. Validity of the Trail Making Test as an indicator oforganic brain damage. Percept Mot Skills. 1958;8:271–276.

35. Stroop JR. Studies of interference in serial verbal reactions. J ExpPsychol. 1935;18:643–662.

36. Nelson HE. A modified card sorting test sensitive to frontal lobedefects. Cortex. 1976;12:313–324.

37. Shallice T, Burgess P. The domain of supervisory processes andtemporal organization of behaviour. Philos Trans R Soc Lond B BiolSci. 1996;351(1346):1405–1411; discussion 1411–1412.

38. Baddeley AD, Bressi S, Della Sala S, Logie R, Spinnler H. Thedecline of working memory in Alzheimer’s disease. A longitudinalstudy. Brain. 1991;114(Pt 6):2521–2542.

39. Box G, Cox D. An analysis of transformations. J R Stat Soc B.1964;26:211–252.

40. Glantz S, Slinker B. Primer of Applied Regression and Analysis ofVariance. New York: McGraw-Hill; 2000.

41. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing theareas under two or more correlated receiver operating charac-teristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845.

42. Crawford JR, Garthwaite PH. On the “optimal” size for norma-tive samples in neuropsychology: capturing the uncertainty whennormative data are used to quantify the standing of a neuropsy-chological test score. Child Neuropsychol. 2008;14(2):99–117.

43. Tressoldi PE, Giofre D, Sella F, Cumming G. High impact= high statistical standards? Not necessarily so. PLoS One.2013;8(2):e56180.

44. Wilkinson L, Task Force on Statistical Inference American Psycho-logical Association. Statistical methods in psychology journals:guidelines and explanations. Am Psychol. 1999;54(8):594–604.

45. Fortin S, Godbout L, Braun CM. Cognitive structure of execu-tive deficits in frontally lesioned head trauma patients performingactivities of daily living. Cortex. 2003;39(2):273–291.

46. Rapoport MJ, Herrmann N, Shammi P, Kiss A, Phillips A, Fein-stein A. Outcome after traumatic brain injury sustained in older

adulthood: a one-year longitudinal study. Am J Geriatr Psychiatry.2006;14(5):456–465.

47. Ponsford J, Kinsella G. Attentional deficits following severe closed-head injury. J Clin Exp Neuropsychol. 1992;14:822–838.

48. Spikman J, van Zomeren AH, Deelman BG. Deficits of attentionafter closed-head injury: slowness only? J Clin Exp Neuropsychol.1996;18:755–767.

49. Veltman JC, Brouwer WH, van Zomeren AH, van Wolffelaar PC.Central executive aspects of attention in subacute severe and verysevere closed-head injury patients: planning, inhibition, flexibilityand divided attention. Neuropsychology. 1996;10:357–367.

50. Perianez JA, Rios-Lago M, Rodriguez-Sanchez JM, et al. TrailMaking Test in traumatic brain injury, schizophrenia, and nor-mal ageing: sample comparisons and normative data. Arch ClinNeuropsychol. 2007;22(4):433–447.

51. Brown J. Some tests of the decay theory of immediate memory.Q J Exp Psychol. 1958;10:12–21.

52. Boyd TM, Sautter SW. Route finding: a measure of everydayexecutive functioning in the head-injured adult. Appl Cogn Psychol.1993;7:171–181.

53. Cazalis F, Azouvi P, Sirigu A, Agar N, Burnod Y. Script knowl-edge after severe traumatic brain injury. J Int Neuropsychol Soc.2001;7:795–804.

54. Spikman JM, Boelen DH, Lamberts KF, Brouwer WH, Fasotti L.Effects of a multifaceted treatment program for executive dys-function after acquired brain injury on indications of execu-tive functioning in daily life. J Int Neuropsychol Soc. 2010;16(1):118–129.

55. Chevignard MP, Taillefer C, Picq C, Poncet F, Noulhiane M,Pradat-Diehl P. Ecological assessment of the dysexecutive syn-drome using execution of a cooking task. Neuropsychol Rehabil.2008;18(4):461–485.

56. McAllister TW. Neurobehavioral sequelae of traumatic brain in-jury: evaluation and management. World Psychiatry. 2008;7(1):3–10.

57. Arnould A, Rochat L, Azouvi P, Van der Linden M. A multidimen-sional approach to apathy after traumatic brain injury. NeuropsycholRev. 2013;23(3):210–233.

58. Rochat L, Beni C, Billieux J, Azouvi P, Annoni JM, Van der LindenM. Assessment of impulsivity after moderate to severe traumaticbrain injury. Neuropsychol Rehabil. 2010;20(5):778–797.

59. Robert P, Onyike CU, Leentjens AF, et al. Proposed diagnostic cri-teria for apathy in Alzheimer’s disease and other neuropsychiatricdisorders. Eur Psychiatry. 2009;24(2):98–104.

60. Godefroy O, Martinaud O, Verny M, et al. The dysexecutive syn-drome of Alzheimer’s disease: the GREFEX study. J Alzheimers Dis.2014;42(4):1203–1208.

61. Jourdan C, Bayen E, Bosserelle V, et al. Referral to rehabilita-tion after severe traumatic brain injury: results from the PariS-TBIStudy. Neurorehabil Neural Repair. 2013;27(1):35–44.

62. Wilson BA, Alderman N, Burgess PW, Emslie H, Evans JJ. Be-havioural Assessment of the Dysexecutive Syndrome (BADS). Bury StEdmunds, UK: Thames Valley Test Company; 1996.

63. Burgess PW, Alderman N, Evans J, Emslie H, Wilson B. Theecological validity of tests of executive function. J Int NeuropsycholSoc. 1998;4:547–558.

64. Boelen DH, Spikman JM, Rietveld AC, Fasotti L. Execu-tive dysfunction in chronic brain-injured patients: assessmentin outpatient rehabilitation. Neuropsychol Rehabil. 2009;19(5):625–644.

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

Executive Functions Deficits After Severe Traumatic Brain Injury 11

APPENDIX

Description of the tests used in the cognitive battery

Phonemic and Categorical Verbal Fluency33: Participantswere required to generate in 2 minutes as many wordsas possible (excluding proper nouns or the same wordwith a different suffix) under 2 conditions: phonemicfluency: words beginning by the letter “P”; categoricalfluency: animals. The score was the number of correctitems generated under each condition.

Trail Making Test34: This test is given in 2 parts, Aand B. Part A required the subject to draw as fast aspossible lines to connect consecutively 25 numberedcircles on an A4 paper sheet. In part B, they have toconnect consecutively numbered (1-13) and lettered (A-L) circles by alternating between the 2 sequences. Scoreswere time of completion, the number of errors, and thenumber of perseverative errors (part B).

Stroop Test35: Three conditions are given. The firstone (color naming) required participants to name asfast as possible the color of 100 small colored patches.The second (reading) required participants to read 100color names printed in black ink. In the third condition(interference), participants had to name colors of 100incongruent color names (eg, the word “red” printedin green ink). Scores were time of completion and thenumber of errors for each trial.

Modified Card Sorting Test36: Participants were askedto sort 2 sets of 24 cards with a panel of 4 target cardsaccording to sorting criteria that the patient had to de-duce from the pattern of the examiner’s response to thepatient’s placement of the card. Three successive sort-ing criteria were used: shape, color, and the number ofitems on the card. The examiner changed the responsecriteria after 6 consecutive correct responses, withoutinforming the patient. Scores were the number of cat-egory achieved (maximum = 6, as each criterion wasgiven twice), the number of errors, and of perseverativeerrors.

Six Elements Test26: Participants were instructed tocomplete 6 simple tasks (1 sets of 3 tasks: 2 writtennaming tasks, 2 arithmetic tasks, and 1 task requiringdictation of the route to arrive to and to leave the test-

ing place) within a given time limit (10 minutes). Theywere explicitly instructed that they had to complete atleast some part of each 1 of the 6 subtasks, and thatthey were not allowed to do 2 subtasks of the sametype immediately one after the other. The score was aglobal score (rank score) calculated with the followingformula: the number of tasks correctly attempted minusthe number of tasks with rule break (range: 0-6).

Brixton Test37: This is a spatial anticipation test. Par-ticipants were shown successively 56 cards representingvisual patterns made of 10 (2 rows of 5) circles, 1 black,and the 9 others white. The spatial position of the blackcircle differed from a given card to the next one accord-ing to a predetermined rule. The task was to recognizethe rule, to anticipate the position of the black circle onthe next card. Nine different rules were used through-out the test, without informing the patient when therule changed. The score was the number of errors.

Box-Crossing and Digit Span Dual Task38: Participantscombined crossing out a chain of boxes with repeat-ing span-length sequences of random digits read outby the examiner. As a first step, each subject’s forwarddigit span was determined, by presenting 3 sequences ofdigits at each length. The highest level at which perfor-mance was perfect was selected and then used during a2-minute session during which each subject’s recall wasimmediately followed by the presentation of anothersequence at that length. Performance was measured asthe percentage correct in 2 minutes. The box-crossingtask consisted of a total of 160 1-cm2 boxes joined withlines and arranged along a winding path, printed on anA4 sheet of paper. This task was again given as a singletask during 2 minutes. Then, participants were asked toperform both tasks concurrently during 2 minutes. Acombined score was computed to assess the dual-taskdecrement across the 2 tasks. The combined Mu scorewas defined as follows: Mu = [1 − (pm + pt)/2] ×100, where pm is the proportional loss of memory (digitspan) performance under dual-task condition and pt theproportional loss in tracking score. A score of 100 in-dicates no decrement, whereas a score of less than 100implies impaired performance as a result of combiningthe 2 tasks.

Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

www.headtraumarehab.com