scholarspace at university of hawaii at manoa: home ......normal structural imaging normal or...
TRANSCRIPT
ESTIMATING EMPLOYMENT STATUS IN A SAMPLE OF PARTICIPANTS WITH
TRAUMATIC BRAIN INJURY REFERRED FOR NEUROPSYCHOLOGICAL
ASSESSMENT FOR TREATMENT PLANNING OR FOR LITIGATION PURPOSES
A DISSERTATION SUBMITTED TO THE GRADUATE DIVISION OF THE
UNIVERSITY OF HAWAIʻI AT MĀNOA IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
IN
PSYCHOLOGY
DECEMBER 2014
By
James D. Larsen
Dissertation Committee:
Stephen Haynes, Chairperson
Elaine Heiby
Brad Nakamura
John Meyers
Joe Mobley
EMPLOYMENT STATUS FOLLOWING TBI 2
Abstract
Previous research has identified demographic and neuropsychological variables significantly
related to the amount of time that individuals take before returning to work following traumatic
brain injury (TBI). However, existing models do not identify variables significantly associated
with an individual’s current employment status as a function of time since TBI. The Meyers
Neuropsychological Battery (MNB) is a short battery of neuropsychological tests that assesses
the neuropsychological domains most commonly related to the likelihood that an individual will
be employed following a TBI. The goal of this study was to examine the degree to which scores
from the MNB, in combination with demographic information, predicted an individual’s
employment status as a function of time since TBI. Using archival data from a private practice
neuropsychology database of 192 male and female adults, exploratory and confirmatory
hierarchical regression modeling was used to examine the degree to which neuropsychological
test scores independently and incrementally accounted for variance in an individual’s
employment status, while considering time since injury and demographic variables. Regression
models were created using forward stepwise binary logistic regression on a sample of 96
participants and confirmed on three separate samples of participants taken from the same
database, including samples of litigants and non-litigants. Results showed that regression models
were able to correctly classify the employment status of between 78.6% and 88.5% of study
participants. These correct classification rates are higher than those attained by prediction models
examined in previously published research. The variables that were most consistently identified
as significant predictors of employment status were years of education, independent driving
status, premorbid occupation, Wechsler Adult Intelligence Scale-III Performance IQ score, and
the Overall Test Battery Mean. R2 values ranged from 0.28 to 0.40. Results show that post-TBI
EMPLOYMENT STATUS FOLLOWING TBI 3
employment status in the study sample could be predicted using a combination of scores from
the MNB and demographic information. These findings may be clinically useful when
determining the readiness to return to work of individuals who are recovering from TBI.
EMPLOYMENT STATUS FOLLOWING TBI 4
Table of Contents
List of Tables .................................................................................................................................. 9
List of Abbreviations .................................................................................................................... 10
Introduction ................................................................................................................................... 11
Psychological and Neuropsychological Consequences of TBI ................................................. 12
Difficulty Returning to Work Following TBI ........................................................................... 13
Benefits of Predicting Employment Status Following TBI ...................................................... 14
Demographic Variables Significantly Correlated with Employment Status Following TBI .... 15
Age at time of injury.............................................................................................................. 17
Premorbid occupation ............................................................................................................ 18
Years of education ................................................................................................................. 18
Number of symptoms ............................................................................................................ 19
Ethnic status........................................................................................................................... 20
Activities of daily living ........................................................................................................ 21
Neuropsychological Domains Significantly Correlated with Employment Status Following
TBI ............................................................................................................................................ 21
Memory ................................................................................................................................. 23
Attention ................................................................................................................................ 24
Visuospatial skills .................................................................................................................. 25
Executive functions ............................................................................................................... 25
General cognitive functioning ............................................................................................... 26
Language fluency .................................................................................................................. 27
Motor performance ................................................................................................................ 28
The Meyers Neuropsychological Battery .................................................................................. 29
Suitability of the MNB for Predicting Employment Status ...................................................... 31
Predicting Current Employment Status ..................................................................................... 33
EMPLOYMENT STATUS FOLLOWING TBI 5
Goals ............................................................................................................................................. 35
Method .......................................................................................................................................... 37
Participants ................................................................................................................................ 37
Group 1 .................................................................................................................................. 39
Group 2 .................................................................................................................................. 40
Group 3 .................................................................................................................................. 41
Group 4 .................................................................................................................................. 42
Procedure ................................................................................................................................... 45
Neuropsychological examination .......................................................................................... 45
Assessment domains, instruments and measures .................................................................. 46
Memory .............................................................................................................................. 46
RAVALT ........................................................................................................................ 46
Rey Complex Figure Test .............................................................................................. 48
Attention ............................................................................................................................ 50
Digit-Symbol Coding ..................................................................................................... 50
Trail Making Test, Part B ............................................................................................... 51
Visuospatial skills .............................................................................................................. 52
Judgment of Line Orientation ........................................................................................ 52
Block Design .................................................................................................................. 53
Executive functions ............................................................................................................ 54
Category Test ................................................................................................................. 54
Similarities ..................................................................................................................... 55
General cognitive function ................................................................................................. 56
WAIS-III index scores ................................................................................................... 56
Overall test battery mean ................................................................................................ 56
EMPLOYMENT STATUS FOLLOWING TBI 6
Language fluency ............................................................................................................... 56
COWAT ......................................................................................................................... 56
Motor performance ............................................................................................................ 57
Finger Tapping Test ....................................................................................................... 57
Data Reduction .......................................................................................................................... 58
Demographic Variables ......................................................................................................... 58
Employment status ............................................................................................................. 59
Ethnicity ............................................................................................................................. 59
Premorbid Occupation ....................................................................................................... 60
Independent driving status ................................................................................................. 60
Neuropsychological assessment measures ............................................................................ 61
Data Analysis ............................................................................................................................ 62
Goal 1 .................................................................................................................................... 64
Goal 2 .................................................................................................................................... 65
Goal 3 .................................................................................................................................... 67
Goal 4 .................................................................................................................................... 69
Goal 5 .................................................................................................................................... 70
Goal 6 .................................................................................................................................... 71
Goal 7 .................................................................................................................................... 72
Goal 8 .................................................................................................................................... 73
Results ........................................................................................................................................... 74
Basic Data ................................................................................................................................. 74
Initial Analyses .......................................................................................................................... 76
Study Goals ............................................................................................................................... 77
Goal 1 .................................................................................................................................... 77
EMPLOYMENT STATUS FOLLOWING TBI 7
Goal 2 .................................................................................................................................... 81
Goal 3 .................................................................................................................................... 81
Goal 4 .................................................................................................................................... 84
Goal 5 .................................................................................................................................... 89
Goal 6 .................................................................................................................................... 91
Goal 7 .................................................................................................................................... 94
Model 4 .............................................................................................................................. 94
Model 5 .............................................................................................................................. 97
Goal 8 .................................................................................................................................. 100
Model 6 ............................................................................................................................ 100
Model 7 ............................................................................................................................ 103
Discussion ................................................................................................................................... 109
A Parsimonious Model to Predict Employment Status ........................................................... 109
Sensitivity, Specificity, and Predictive Efficacy of a Parsimonious Model ............................ 111
The Addition of Demographic Predictors ............................................................................... 113
The Additive Value of Neuropsychological Variables ........................................................... 115
An Investigation of Model Performance ................................................................................. 118
Proportion of variance accounted for .................................................................................. 118
Percentage of cases correctly classified .............................................................................. 119
Sensitivity ............................................................................................................................ 120
Specificity ............................................................................................................................ 120
Positive Predictive Value .................................................................................................... 120
Negative Predictive Value ................................................................................................... 121
The Relationship Between Litigation Status and Model Performance ................................... 122
Model Creation in a Sample of Litigants ................................................................................ 122
EMPLOYMENT STATUS FOLLOWING TBI 8
Model Creation Using the Entire Study Sample ..................................................................... 124
The Difficulty of Predicting Employment Status .................................................................... 127
Limitations of the Current Study ............................................................................................. 129
Directions for Future Research................................................................................................ 134
References ................................................................................................................................... 138
EMPLOYMENT STATUS FOLLOWING TBI 9
List of Tables
1 Department of Defense TBI Severity Classification System .............................................11
2 Demographic Variables Significantly Correlated with Employment After TBI ...............16
3 Neuropsychological Domains Significantly Correlated with Employment After TBI ......22
4 Meyers Neuropsychological Battery Tests Organized by Neuropsychological Domain ..32
5 Demographic Variables of Interest in Groups 1-4 and the Full Study Sample .................44
6 Exploratory Stepwise Binary Logistic Regression Models Created ..................................63
7 Neuropsychological Variables of Interest in Groups 1-4 and the Full Study Sample .......75
8 Summary of the Two Steps Completed During Creation of Model 1 ...............................78
9 Summary of the Five Steps Completed During Creation of Model 1b..............................80
10 Summary of the Seven Steps Completed During Creation of Model 2 .............................83
11 Summary of the Seven Steps Completed During Creation of Model 3 .............................87
12 Statistical Performance of Models 1b, 2, and 3 in Groups 1 and 2....................................90
13 Statistical Performance of Models 1b through 3 in Groups 3 and 4 ..................................93
14 Summary of the Six Steps Completed During Creation of Model 4 .................................96
15 Summary of the Five Steps Completed During Creation of Model 5................................99
16 Summary of the Five Steps Completed During Creation of Model 6..............................102
17 Summary of the Eight Steps Completed During Creation of Model 7 ............................105
18 Variables Retained During Regression Analyses in Models 1 through 7 ........................107
19 Statistical Performance of Models 1b through 7 ..............................................................108
EMPLOYMENT STATUS FOLLOWING TBI 10
List of Abbreviations
Abbreviation Definition
COWAT Controlled Oral Word Association Test
FSIQ Full-scale Intelligence Quotient
MNB Meyers Neuropsychological Battery
OTBM Overall Test Battery Mean
PIQ Performance Intelligence Quotient
RAVLT Rey Auditory Verbal Learning Test
SD Standard deviation
TBI Traumatic Brain Injury
VIQ Verbal Intelligence Quotient
WAIS-III Wechsler Adult Intelligence Scale, 3rd Edition
EMPLOYMENT STATUS FOLLOWING TBI 11
Estimating Employment Status in a Sample of Participants with Traumatic Brain Injury Referred
for Neuropsychological Assessment for Treatment Planning or for Litigation Purposes
According to the American Congress of Rehabilitation Medicine (Kay et al., 1993),
Traumatic Brain Injury (TBI) has been defined as any traumatically induced physiological
disruption of brain function as evidenced by any period of loss of consciousness, any loss of
memory for events immediately before or after the accident, any alteration in mental state at the
time of the event, or any focal neurological deficit. While multiple classification systems exist to
rate the severity of TBI, most are comparable. In 2009, the Department of Veterans Affairs and
Department of Defense published Table 1 for use as a tool to rate the severity of a TBI (The
Management of Concussion/TBI Working Group, 2009). In using this classification system, each
injury is assigned the most severe classification for which at least one of the listed criteria is met.
Table 1
Department of Defense TBI Severity Classification System.
Mild TBI Moderate TBI Severe TBI
Normal structural imaging Normal or abnormal structural imaging. Severity based
on other criteria.
LOC = 0-30 minutes LOC > 30 minutes and <
24 hours LOC > 24 hours
AOC = a moment up to 24
hours AOC > 24 hours. Severity based on other criteria.
PTA = 0 to 1 days PTA > 1 and < 7 days PTA > 7 days
GCS = 13-15 GCS = 9-12 GCS < 9
Note: GCS scores represent the best score taken during the first 24 hours after injury.
TBI = Traumatic Brain Injury; LOC = loss of consciousness; AOC = alteration of
consciousness; PTA = posttraumatic amnesia; GCS = Glascow Coma Scale.
EMPLOYMENT STATUS FOLLOWING TBI 12
TBI is a growing concern in the United States. In 2009, approximately 3.5 million
individuals in the United States received inpatient or outpatient medical treatment for a primary
or secondary diagnosis of TBI. In the same year, an additional 52,695 individuals in the United
States died due to causes directly related to the effects of TBI (Coronado et al., 2012). Mild TBI
is thought to account for roughly 75% of all head injuries in the United States (Faul, Wald, Xu &
Coronado, 2010).
Psychological and Neuropsychological Consequences of TBI
The psychological and neuropsychological impact of TBI can be severe. Several notable
meta-analyses (Belanger, Curtiss, Demery, Lebowitz, and Vanderploeg, 2005; Frencham, Fox,
and Maybery, 2005; Rohling, Binder, Demakis, Larrabee, Ploetz, and Langhinrichsen-Rohling,
2011) have demonstrated that mild TBI can affect a spectrum of neuropsychological domains
during the first three months following the injury, including processing speed, working memory,
attention, memory, and executive functions. Additional research (Bigler, Farrer, Pertab, James,
Petrie, and Hedges, 2013; Iverson, 2010; Pertab, James, & Bigler, 2009) has shown that the
cognitive deficits described by Frencham et al. persist beyond three months in as many as 24%
of victims of mild TBI, and may persist for as long as ten years (Ponsford, Downing, Olver,
Ponsford, Acher, Carty, & Spitz, 2014). In a sample of individuals treated at a rehabilitation
hospital, Whelan-Goodinson, Ponsford, Johnston, and Grant (2009) showed a significant
increase in rates of depressive disorders, generalized anxiety disorder, posttraumatic stress
disorder, panic disorder, and phobias following mild to severe TBI when compared to premorbid
rates of those disorders in the same sample. De Guise, LeBlanc, Tinawi, Lamoureux, and Feyz
(2012) found that up to two weeks after injury, victims of mild TBI had significantly elevated
scores on both the Beck Depression Inventory and the Beck Anxiety Inventory when compared
EMPLOYMENT STATUS FOLLOWING TBI 13
to a normative sample. Konrad et al. (2011) showed that when compared to healthy controls,
victims of mild TBI experienced a significantly higher number of depressive symptoms as
measured by the Beck Depression Inventory and a significantly higher level of impairment in
daily life as measured by a 25-item questionnaire derived from the Rivermead Post Concussion
Symptoms Questionnaire (King, Crawford, Wenden, Moss, & Wade, 1995). The identified
between-group differences remained significant up to 6 years post-injury. Approximately 50% of
victims of TBI experience high levels of apathy, which has been related to poor rehabilitation
outcome, loss of social autonomy, financial problems, vocational loss, cognitive decline, and
caregiver distress (Arnould, Rochat, Azouvi, & Van der Linden, 2013). In a sample of 205
individuals with moderate to severe TBI, approximately 65% of individuals were found to
experience sleep disturbance in the month following injury (Nakase-Richardson et al., 2013).
These psychological and neuropsychological consequences can combine to result in an inability
to continue previously established routines, abilities, and roles, including employment
(Grauwmeijer, Heijenbrok-Kal, Haitsma, & Ribbers, 2012; Robertson, 2008).
Difficulty Returning to Work Following TBI
Difficulty returning to work following TBI is a widespread problem. Parks, Diaz-
Arrastia, Gentilello, and Shafi (2010) followed a group of 572 individuals in the State of
Colorado who had suffered a TBI between 1996 and 1999. Of the 381 members of the cohort
who were employed at the time of injury, only 69% were employed one year post-injury and
only 74% were employed three years post-injury. Similarly, the Traumatic Brain Injury Model
Systems estimated that 63% of individuals are employed at the time they experience a TBI,
whereas employment rates drop to 28% after one year (The Traumatic Brain Injury Model
Systems, 2010). Grauwmeijer, Heijenbrok-Kal, Haitsma, and Ribbers (2012) found that in a
EMPLOYMENT STATUS FOLLOWING TBI 14
cohort of 113 patients hospitalized following a moderate to severe TBI, employment rates
climbed steadily to 55% during the first year post-injury, but showed little improvement during
the following two years. Other studies have shown that those who return to work following TBI
often have difficulty maintaining their employment (Fraser, Machamer, Temkin, Dikmen, &
Doctor, 2006; Yasuda, Wehman, Targett, Cifu, & West, 2001), especially when medical
symptoms and emotional dysregulation persist (Artman & McMahon, 2013). Sigurdardottir,
Andelic, Roe, and Schanke (2013) showed that lack of employment during the first 5 years
following TBI is significantly related to an increase in depressive symptoms. In such a case,
depressive symptoms and unemployment would have a reciprocal relationship in which each
increases the likelihood of the other.
There are also collateral effects of not returning to work following a brain injury.
Wehman, Targett, West, and Kregel (2005) remind us that working is tied to other activities that
promote recovery, such as a sense of purpose, a reason to leave the house, and the creation or
maintenance of friendships. Cattelani, Tanzi, Lombardi, and Mazzucchi (2002) point out that a
failure to return to pre-injury employment or work status can result in decreased quality of life
for victims of TBI and their families.
Benefits of Predicting Employment Status Following TBI
Sherer, Novack, Sander, Struchen, Alderson, and Thompson (2002) recognized that one
of the purposes of neuropsychological assessment following TBI is the prediction of the degree
and latency of return to normal cognitive functioning. They proposed that the ability to return to
work was an ecologically valid measure of general neuropsychological functioning. Robertson
(2008) described another goal of neuropsychological assessment when he stated that a prominent
concern for many victims of TBI is how soon they can expect to resume previous roles, such as
EMPLOYMENT STATUS FOLLOWING TBI 15
employment. The ability to use neuropsychological assessment scores to help predict
employment status following a TBI would help to accomplish both of the above-mentioned core
purposes of neuropsychological assessment. However, Robertson points out another, perhaps
more important reason for understanding return to work following TBI: A better understanding
of the factors that contribute to a return to employment may help identify interventions that
enhance the recovery process.
Demographic Variables Significantly Correlated with Employment Status Following TBI
Current models for predicting employment status following TBI are moderately effective,
with correct classification rates between 65% and 77% (Drake, Gray, Yoder, Pramuka, &
Llewellyn, 2000; Fleming, Tooth, Hassell, & Burchan, 1999; Guerin, Kennepohl, Leveille,
Dominique, & McKerral, 2006; Kreutzer et al., 2003; MacMillan, Hart, Martelli, & Zasler, 2002;
Simpson & Schmitter-Edgecombe, 2002), and are improving our understanding of the factors
moderating return to work (Ownsworth & McKenna, 2004; Shames, Treger, Ring, & Giaquinto,
2007). Recent research has identified a number of demographic variables that are significantly
correlated with employment status following TBI, as outlined in Table 2.
EMPLOYMENT STATUS FOLLOWING TBI 16
Table 2
Demographic Variables Significantly Correlated with Employment After TBI
Study
Demographic Variables Identified
Age at
Injury
Premorbid
Occupation
Level of
Education
Number
of
Symptoms
Ethnic
Minority
Status
Activities
of Daily
Living
Arango-Lasprilla (2009) (+) (-)
Arango-Lasprilla (2011) (-)
Chamelian (2004) (-)
Drake (2000) (+) (+) (-)
Flemming (1999) (+) (-) (+)
Gary (2009) (+) (+) (+) (-)
Grauwmeijer (2012) (-) (-)
Guerin (2006) (-) (-)
Hanlon (1999) (-)
Johansson (2001) (+)
Ketchum (2012) (+) (+)
Keyser-Marcus (2002) (-) (+)
Kreutzer (2003) (-) (+) (-)
Machamer (2005) (+)
Ownsworth (2004) (-) (+) (+) (-)
Schonberger (2011) (-) (+) (+)
Shames (2002) (-) (+) (+)
Simpson (2002) (+)
van der Horn (2013) (-)
Walker (2006) (+)
Note: Boxes marked with a (+) signify that the corresponding study found a statistically significant
positive relationship between the corresponding demographic variable and employment status
following TBI. Boxes marked with a (-) signify that the corresponding study found a statistically
significant negative relationship between the corresponding demographic variable and employment
status following TBI. Studies are identified by the name of the first author and year of publication
in order to conserve space. See References for complete article reference information.
EMPLOYMENT STATUS FOLLOWING TBI 17
Age at time of injury. A number of studies have found a significant, inverse relationship
between a person’s age at the time they acquire a TBI and their ability to return to work
afterward, with older individuals taking longer to return to work and having more difficulty
maintaining stable employment following a TBI than younger individuals (Grauwmeijer,
Heijenbrok-Kal, Haitsma, and Ribbers, 2012; Guerin et al., 2006; Hanlon, Demery, Martinovich,
& Kelly, 1999; Jourdan et al., 2013; Keyser-Marcus et al., 2002; Ketchum et al., 2012; Kreutzer,
et al., 2003). Schonberger, Ponsford, Olver, Ponsford, and Wirtz (2011) suggested that the
significant correlation between age and a failure to return to work may be due to the fact that
older victims of TBI chose to retire rather than return to work following their injury. However, in
certain cases a significant positive correlation has been found between a person’s age at the time
they are injured and their ability to return to work afterwards. Drake et al. (2000) found that
older active-duty members of the armed forces were able to return to full duty more quickly
following a TBI. Similarly, Machamer, Temkin, Fraser, Doctor, and Dikmen (2005) found that
older individuals maintained more steady employment following a TBI, an effect the authors
ascribed to the benefits of being well-established in a career at the time of injury.
Guerin et al. (2006) collected demographic, neurological, psychological, and
environmental data from 110 individuals receiving treatment for mild TBI at a major
rehabilitation center in Montreal, Canada. Logistic regression showed that age at time of injury
was the variable most strongly related to whether or not participants were able to return to at
least part-time employment at the end of the treatment program. In another study (Hanlon et al.,
1999), the participant's age at the time of injury was identified as the only demographic variable
significantly correlated with post-injury employment status. Research has also demonstrated a
EMPLOYMENT STATUS FOLLOWING TBI 18
significant correlation between age at the time of injury and the stability of employment
following TBI (Kreutzer et al., 2003; Machamer et al., 2005).
Premorbid occupation. The type of employment held by an individual at the time of
their injury has also been investigated as a possible predictor of whether or not he or she will
return to work afterwards (Shames et al., 2007). Walker, Marwitz, Kreutzer, Hart, and Novack
(2006) studied a Traumatic Brain Injury Model Systems database cohort of 1341 individuals who
had been hospitalized for TBI. They divided participants into three groups based on their pre-
injury employment, categorizing each individual as a professional/managerial worker, a skilled
worker, or a manual laborer. They found that between 10 and 14 months post-injury, skilled
workers were just over 1.5 times as likely to have returned to work as manual laborers, while
professional and managerial workers were nearly 3 times as likely to have returned to work as
manual laborers. Two additional studies of return to work rates following TBI in minority
populations also found that pre-injury occupation significantly predicted post-injury occupation
regardless of minority status (Arango-Lasprilla et al., 2009; Gary et al., 2009). Andelic, Stevens,
Sigurdardottir, Arrango-Lasprilla, and Roe (2012) found that individuals who were unemployed
prior to experiencing a TBI were 95% less likely to find employment following their injury than
those who had previously been employed. Schonberger, Ponsford, Olver, Ponsford, and Wirtz
(2011) also found that premorbid employment (i.e. individuals who were employed vs. those
who were unemployed) was a significant (p < 0.01) predictor of employment status one year
after a TBI.
Years of education. Simpson and Schmitter-Edgecombe (2002) used a discriminant
function analysis to identify which combination of neuropsychological measures and
demographic variables were mostly strongly correlated with employment status following a TBI.
EMPLOYMENT STATUS FOLLOWING TBI 19
The 61 participants in their study were injured an average of ten and a half years before data
collection. Results showed that level of education was the demographic variable most strongly
correlated with employment status at the time of data collection, with unemployed individuals
having significantly (p < 0.05) fewer years of education than those who were employed. Kruetzer
and colleagues (2003) made the observation that those who have graduated from high school
were nearly twice as likely to find employment during the four years following a TBI than those
who did not graduate from high school. Similarly, Keyser-Marcus and colleagues (2002) found
that individuals with higher levels of education were significantly more likely to return to work
within one year following a TBI (p < .01; odds ratio = 1.38). Using structural equation modeling,
Schonberger, Ponsford, Olver, Ponsford, and Wirtz (2011) found that the number of years of
formal education completed before sustaining a TBI was a significant (p < 0.01) predictor of
employment status one year after injury. Similarly, Ketchum et al. (2012) used a logistic
regression model to show that pre-injury employment status accounted for a significant (p <
0.01) amount of variance in employment status 1 year after TBI in a population of 418 Hispanic
individuals hospitalized for TBI between 1990 and 2009.
Number of symptoms. Fleming et al. (1999) used the hospital records of 209 TBI
patients admitted between 1991 and 1995 to identify which variables would be most predictive
of employment status two to five years post-injury. Among the measures collected were scores
from the Disability Rating Scale (Rappaport, Hall, Hopkins, Belleza, & Cope, 1982), an
observational rating scale that quantifies functional disability across categories following TBI.
Discriminant function analysis showed that the Disability Rating Scale total score was a
significant predictor of employment status following TBI (F1,200 = 12.80; p < .01), with lower
scores predicting a greater chance of being employed. In a sample of 242 adults with mild to
EMPLOYMENT STATUS FOLLOWING TBI 20
severe TBI, van der Horn, Spikman, Jacobs, and van der Naalt (2013) found that individuals who
did not return to competitive employment after mild TBI also experienced a significantly (p <
0.01) higher number of self-reported depression- and anxiety-related symptoms. Similar
relationships between the number of subjective symptoms immediately following a TBI and
post-injury employment status have been reported elsewhere (Chamelian & Feinstein, 2004;
Drake et al., 2000; Guerin et al., 2006).
Ethnic status. In a review of studies investigating variables and measures that might
have been significantly correlated with employment status following TBI, Ownsworth and
McKenna (2004) found that studies with robust methodologies all found that ethnic minority
groups were at a significant disadvantage when seeking employment following TBI. Numerous
other empirical studies have found significant correlations between minority status and
unemployment following TBI (Gary et al., 2009; Kreutzer et al., 2003). Notably, Arango-
Lasprilla and colleagues (2009) followed a Traumatic Brain Injury Model Systems cohort of 633
individuals, 219 of whom were ethnic minorities, for three years following TBI. Employment
status was assessed each year during the three-year follow-up period. They found that when
compared to White Americans, members of ethnic minority groups were between two and three
and a half times more likely to be unemployed or unstably employed during the three years
following a TBI. These differences were true even when controlling for other variables such as
premorbid employment status, age, marital status, level of education, cause of injury, loss of
consciousness, and general level of impairment. A follow up study (Arango-Lasprilla, Ketchum,
Lewis, Krch, Gary, & Dodd, 2011) found that Whites continued to find competitive employment
at higher rates than Hispanics and Blacks five years after a moderate to severe TBI. The
EMPLOYMENT STATUS FOLLOWING TBI 21
researchers concluded that it is critical to consider ethnic minority status when predicting
employment status following TBI.
Activities of daily living. In their study of community integration and employment
outcomes following participation in a TBI rehabilitation program in Australia, Fleming and
colleagues (1999) assessed each participant using the Modified Barthel Index (Shah, Vanclay, &
Cooper, 1989), an instrument which rates level of functioning in 10 areas of activities of daily
living including personal hygiene, bathing, feeding, use of the toilet, stair climbing, dressing,
bowel control, bladder control, ambulation, and transferring from a chair to a bed. They found
that individuals with higher scores on the Modified Barthel Index were significantly more likely
to be employed between two and five years post-injury than individuals with low scores, and that
scores on the Index significantly (p < 0.01) predicted post-injury employment status. In another
study, Johansson and Bernspang (2001) showed that occupational therapy assessments were
useful in predicting work status following a TBI, mostly through their collection of in-depth
information regarding activities of daily living. Recently, Forslund, Roe, Arango-Lasprilla,
Sigurdardottir, & Andelic (2013) showed that 2 years after a moderate to severe TBI, individuals
who were driving independently were significantly more likely to be employed than those who
were not driving independently (p < 0.01, odds ratio = 8.4).
Neuropsychological Domains Significantly Correlated with Employment Status Following
TBI
In addition to the demographic variables discussed above, a number of
neuropsychological measures have been investigated in order to examine their validity and
clinical utility in predicting employment status after a TBI. While there has been little
consistency in the assessment instruments used across studies, certain general domains of
EMPLOYMENT STATUS FOLLOWING TBI 22
neuropsychological functioning have often been the focus of these investigations, as outlined in
Table 3. Although the exact definitions of these domains vary across studies, the differences are
not extensive enough to prevent comparisons. Some of the most commonly investigated domains
are discussed below.
Table 3
Neuropsychological Domains Significantly Correlated with Employment After TBI
Study
Neuropsychological Domains Identified
Memory Attention Visuo-
spatial
Executive
Functions
General
Cognitive
Function
Language
Fluency Motor
Cifu (1997) (+)
Drake (2000) (+) (+) (+)
Fraser (2006) (+) (+)
Han (2009) (+) (-)
Hanlon (1999) (+) (+) (+)
Johansson (2001) (+)
Machamer (2005) (+) (+) (+) (+)
Ownsworth (2004) (+) (+) (+)
Ryu (2010) (+) (+)
Sherer (2002) (+) (+) (+)
Sigurdardottir (2009) (+)
Spitz (2012) (+) (+) (+)
Zakzanis (2013) (+)
Note: Boxes marked with a (+) signify that the corresponding study found a statistically
significant positive relationship between the corresponding neuropsychological domain
and employment status following TBI. Boxes marked with a (-) signify that the
corresponding study found a statistically significant negative relationship between the
corresponding neuropsychological domain and employment status following TBI. Studies
are identified by the name of the first author and year of publication only in order to
conserve space. See References for complete article reference information.
EMPLOYMENT STATUS FOLLOWING TBI 23
Memory. Memory impairment is one of the most frequently reported and researched
effects of TBI (Vakil, 2005) and has been frequently studied as a predictor of post-injury
employment status. Scores from a wide variety of commonly used memory tests have been
identified as significant predictors of employment status following TBI. Drake et al. (2000)
found significant (p < 0.05) between group differences in the California Verbal Learning Test-II
(Delis, Kramer, Kaplan, & Ober, 2000) long-delay free recall score when comparing active duty
military members who were or were not able to return to full duty following a TBI, with those
who returned to full duty attaining higher scores. Using a statistical procedure known as optimal
data analysis, Han et al. (2009) found that in a sample of 52 active duty military personnel, the
percentage of change between the California Verbal Learning Test-II short-delay free recall and
long-delay recall scores, as well as the total recognition hits score, accounted for a significant
amount of variance (p = 0.01) in employment status following TBI. Other measures of memory
that have been found useful in predicting employment status following TBI include the Selective
Reminding Test (Buschke, 1973; Machamer et al., 2005; Fraser et al., 2006) the Rivermead
Behavioural Memory Test (Johansson & Bernspang, 2001; Wilson, Cockburn, Baddeley, &
Hiorns, 1988), and certain subtests of the Wechsler Memory Scale – Revised (Cifu et al., 1997;
Hanlon et al, 1999; Wechsler, 1987). While not directly linked to employment status, Spitz,
Ponsford, Rudzki, and Maller (2012) found that the total score and the 20-minute delayed recall
score from the Rey Auditory Verbal Learning Test (RAVLT) and the Immediate Memory Index
score from the Wechsler Adult Intelligence Scale – III (WAIS-III; Wechsler, 1997) were
significantly different between healthy controls and individuals who had experienced a moderate
to severe TBI. These differences were significant (p < 0.01) at 3, 6, and 12 months post-injury,
with the exception of the WAIS-III Immediate Memory Index score at 12 months.
EMPLOYMENT STATUS FOLLOWING TBI 24
Attention. Attention refers to the process by which a person orients to, selects, and
maintains focus on information to make it available for cortical processing (Zillmer, Spiers, &
Culbertson, 2008). It is another neuropsychological domain frequently investigated in studies of
employment status following TBI. In the study mentioned above, Hanlon et al. (1999) found that
scores on the Trail Making Test part B (Reitan, 1958), a commonly used measure of switching
attention between two tasks, were significantly correlated with employment status following TBI
(r = 0.30; p < 0.01). In addition, Machamer and colleagues (2005) found that scores on the Trail
Making Test part B collected one month post-injury were significantly different between groups
of TBI patients who had worked less than 50% of the time, between 50% and 89% of the time,
and 90% of the time or more since their injury.
Ryu, Cullen, and Bayley (2010) assessed 87 TBI patients participating in an inpatient
rehabilitation program. Their test battery included the Digit-Symbol Coding subtest from the
WAIS-III to measure attention and processing speed. Later statistical analysis showed that of all
the neuropsychological tests administered, scores from the Digit-Symbol Coding subtest were
most significantly correlated with employment status at one year post-injury (p = 0.03) and
showed moderate effect size (r = 0.63). This finding has been echoed in other studies, which
have also found performance on the Digit-Symbol Coding subtest to significantly predict
employment status following TBI (Fraser et al., 2006; Machamer et al., 2005).
Fatigue, as measured by the Fatigue Severity Scale (Krupp, LaRocca, Muir-Nash, &
Steinberg, 1989), which is related to deficits in attention, has also been linked to employment
status at one-year post-TBI, with high amounts of fatigue significantly related to poor
employment outcomes, especially when fatigue persists late in the recovery process
(Sigurdardottir, Andelic, Roe, & Schanke, 2009).
EMPLOYMENT STATUS FOLLOWING TBI 25
Visuospatial skills. Visuospatial ability has been described as the ability to process the
position, direction, or movement of objects or points in space (Elias & Saucier, 2006). Two of
the studies mentioned above present evidence that measures of visuospatial performance are
significantly correlated with employment status following a TBI. Hanlon et al. (1999) found that
scores from the Judgment of Line Orientation test (Benton, Varney, & Hamsher, 1978) were
moderately correlated with employment status at one year post-injury (r = 0.25), with higher
scores being associated with more favorable employment outcomes. Additionally, Ryu and
colleagues (2010) found that scores from the WAIS-III Block Design subtest were predictive of
employment status one year after injury. These findings are supported by Ownsworth and
McKenna (2004), whose literature review found moderate support for visuospatial skills as a
significant predictor of employment status following TBI across studies.
Executive functions. Executive functions include higher order regulatory and
supervisory functions, such as planning, mental flexibility, attentional allocation, and inhibitory
control (Zilmer et al., 2008). Empirical studies have shown that executive functions do not
represent a single, unitary construct (Parkin, 1998; Salthouse, 2005; Varney & Stewart, 2004),
and have disputed the claim that they reside primarily in the frontal lobes (Alvarez & Emory,
2006; Meyers & Rohling, 2009). Instead, executive function can be conceptualized as a
“macroconstruct” (Zelazo, Carter, Reznick, & Frye, 1997, p. 219) that describes the coordination
of multiple psychological processes to allow an organism to solve complex problems across a
variety of contexts. The term will be used in the current study to describe a single
neuropsychological domain in order to maintain consistency with previously published research.
In a review of scientific literature published between 1980 and 2003, Ownsworth and McKenna
(2004) found that employment status following TBI was found to share a significant correlation
EMPLOYMENT STATUS FOLLOWING TBI 26
with executive functions more consistently than with any other domain of neuropsychological
functioning addressed in their literature review.
In a study of active-duty military personnel, Drake and colleagues (2000) administered
the Map Planning Test (Ekstrom, French, Harman, & Derman, 1976) to 121 participants an
average of three months following a mild TBI. The Map Planning Test asks participants to find
the shortest path between two spots on a matrix-type grid while avoiding certain obstacles. They
found that the number of maps completed was significantly (p = 0.05) different between service
members who were able and those who were not able to return to full work duties six months
after injury. They also noted that those who were unable to return to full work duties were rated
as having significantly (p < 0.01) higher executive dysfunction by the Neurobehavioral Rating
Scale (Levin et al., 1987). Spitz, Ponsford, Rudzki, and Maller (2012) assessed executive
functioning performance in 111 individuals at 3, 6, and 12 months after a moderate-to-severe
TBI. Their battery included the Zoo Map test, the Trail Making Test Part B, the Controlled Oral
Word Association test, and the Working Memory Index from the WMS-III. They found that
gains in executive functioning performance as measured by these instruments mirrored gains in
functional outcomes as measured by the Mayo-Portland Adaptability Inventory during the first
year following injury.
General cognitive functioning. In their literature review, Ownsworth and McKenna
(2004) found moderate support for using “general intellectual or global cognitive functioning”
(p. 774) to predict employment status following TBI. For the purpose of this paper, all measures
that represent combined neuropsychological performance across domains will be said to
represent general cognitive function. Perhaps the most common measures of general cognitive
function used in the literature to predict employment status following TBI have been the various
EMPLOYMENT STATUS FOLLOWING TBI 27
WAIS-III index scores. Past research has found these scores to be significantly correlated with
post-TBI employment status (Han et al., 2009; Machamer et al., 2005).
In their study of a Traumatic Brain Injury Model Systems cohort of 388 adults with TBI,
Sherer, Sander, Nick, High, Malec, and Rosenthal (2002) combined scores from 16
neuropsychological tests to create a single score which they called “overall cognitive status” (p.
188). The tests comprising this score measured a variety of neuropsychological domains,
including motor performance, verbal fluency, visuospatial performance, memory, attention, and
executive function. Results showed that the overall cognitive status scores obtained one month
after TBI were significantly (p = 0.02) correlated with employment status at one year post-injury,
and remained so even when controlling for other significant predictors of functional outcome
such as severity of injury, education, and pre-injury productivity status.
Language fluency. In their study of employment status in active duty military personnel
with brain injuries, Drake and colleagues (2000) retained the total number of correct words on
the Controlled Oral Word Association Test (COWAT; Benton & Hamsher, 1989) as a measure
of language fluency in their predictive model. Zakzanis, McDonald, and Troyer (2013) recoded
COWAT scores from 28 patients who had incurred severe TBI and 54 healthy controls to
produce both semantic fluency and phonemic fluency scores. They found that both semantic
fluency and phonemic fluency scores were significantly different between victims of TBI and
healthy controls, with semantic fluency showing larger effect sizes (d = 1.53) than phonemic
fluency (d = 0.62). The COWAT was also one of the tests included in the overall cognitive status
score created by Sherer, Sander, Nick, High, Malec, and Rosenthal (2002), which was also
predictive of employment status one year after TBI. Spitz, Ponsford, Rudzki, and Maller (2012)
EMPLOYMENT STATUS FOLLOWING TBI 28
found that individuals who had undergone a moderate to severe TBI had significantly (p < 0.01)
lower COWAT scores than healthy controls at 3, 6, and 12 months post-injury.
Motor performance. Motor performance represents an individual’s ability to
demonstrate bilateral fine muscle control in the upper and lower extremities (Zillmer et al.,
2008). Machamer et al. (2005) investigated motor performance in individuals who had
experienced different levels of job stability following TBI. They found that in the first three to
five years after TBI, dominant-hand scores on the Finger Tapping Test (Reitan & Wolfson,
1993) were significantly (p < 0.01) different between those who had worked 50% of the time or
less, those who had worked 51-89% of the time, and those who had worked 90% of the time or
more, with individuals who achieved higher scores securing more consistent employment.
Sherer, Sander, Nick, High, Malec, and Rosenthal (2002) also included the Grooved Pegboard
Test (Klove, 1963), a test of fine motor control, in the calculation of their overall cognitive status
score.
While there is only moderate agreement across studies on the specific assessment
instruments used, the demographic and neuropsychological categories mentioned above
represent a consensus about which variables have been found to be most significantly associated
with employment status following TBI. Any standardized assessment battery wishing to
successfully predict employment status following TBI would need to measure most or all of
these domains in order to maximize its predictive efficacy. However, cost-efficiency is an
important concern in neuropsychological assessment and the use of a short battery to make valid
predictions could prove to be both efficacious and clinically useful.
EMPLOYMENT STATUS FOLLOWING TBI 29
The Meyers Neuropsychological Battery
The Meyers Neuropsychological Battery (MNB; Volbrecht, Meyers, & Kaster-
Bundgaard, 2000) is a short, flexible neuropsychological battery that takes approximately two
and a half hours to administer. The battery contains 16 core tests covering a variety of
neuropsychological domains. These tests include the Ward seven subtest short form of the
WAIS-III (Ward, 1990), the Animal Naming Test (Spreen & Strauss, 1998), COWAT (Benton &
Hamsher, 1989), One-minute Estimation (Benton, Van Allen, & Fogel, 1964), Dichotic
Listening (Roberts et al., 1994), Sentence Repetition (Spreen & Strauss, 1991), Judgment of Line
Orientation (Benton et al., 1978), Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1983),
Finger Tapping (Reitan & Wolfson, 1993), Finger Localization (Benton, Hamsher, Varney, &
Spreen, 1983), Trail Making Test (Reitan, 1958), Token Test (De Renzi & Vignolo, 1962), the
Victoria Revision of the Category Test (Sherrill, 1987), Rey Complex Figure Test (Meyers &
Meyers, 1995), the RAVLT (Rey, 1964), and a forced-choice effort measure (for a description of
each measure, see the Methods section). The subtests included in the MNB were originally
selected based on the availability and range of normative data, demonstrated sensitivity to brain
injury, and the need for battery that measured multiple domains (Volbrecht et al., 2000). Many of
the subtests included in the MNB are available for public use, which reduces the cost of battery
administration. Lezak, Howieson, and Loring (2004) remind us that the time and cost of a
neuropsychological assessment can reduce access to neuropsychological services for many
patients. The short administration time and low cost of administering the MNB can be listed
among its significant contributions.
Meyers and Rohling (2004) demonstrated the validity and clinical utility of the MNB in
evaluating individuals who have experienced a mild TBI. They showed a correct classification
EMPLOYMENT STATUS FOLLOWING TBI 30
rate of 96.1% when discriminating between individuals with a mild TBI (n = 57) and those who
had been hospitalized for other reasons including chronic pain and depression (n = 103). They
also calculated an Overall Test Battery Mean (OTBM) by calculating the mean of all test T-
scores, which they found to have good test-retest reliability over a period of 12 to 14 months in a
TBI population (r = 0.86, Miller & Rohling, 2001).
Rohling, Meyers, and Millis (2003) demonstrated the discriminative validity of the
MNB’s OTBM score as a measure of general cognitive performance following TBI. A sample of
291 consecutive TBI referrals to a neuropsychological assessment practice was divided into six
groups based on injury severity as measured by length of loss of consciousness following the
injury. Analysis of variance showed that OTBM scores were significantly different between
severity groups (p < 0.01), and linear regression showed that injury severity accounted for 34%
of the variance in OTBM scores (r = -0.58). These results are similar to those reported for the
Halstead-Reitan Battery’s Impairment Index score, which showed a correlation of r = 0.59 with
the time needed to follow simple commands after a TBI (Dikmen, Machamer, Winn, & Temkin,
1995).
Meyers, Volbrecht, and Kaster-Bundgaard (1999) demonstrated the discriminative and
ecological validity of the MNB by using MNB scores to discriminate between those who were
currently driving (n = 230) and those who were unable to drive (n = 82) in a sample of 312
consecutive outpatient referrals to a neuropsychological assessment clinic. Participants
represented a wide range of common neuropsychological diagnostic groups. Discriminant
function analysis was able to distinguish significantly between the two groups (p < 0.01), and
resulted in a 94.4% correct classification rate.
EMPLOYMENT STATUS FOLLOWING TBI 31
The assessment of effort is also an essential part of any neuropsychological battery
measuring cognitive performance following TBI. The MNB contains nine identified measures of
patient effort during neuropsychological testing. These measures have shown 83% sensitivity in
a group of simulated malingerers, as well as 100% specificity in a group of 106 non-litigant TBI
patients and a group of 32 healthy controls (Meyers & Volbrecht, 2003).
Suitability of the MNB for Predicting Employment Status
Recent research (Spitz, Ponsford, Rudzki, & Maller, 2012; Williams, Rapport, Hanks,
Millis, & Greene, 2013) has shown that neuropsychological assessment can make unique
contributions to the prediction of employment status after TBI, even after demographic and
neuroimaging data have been considered. The MNB includes subtests that measure each of the
neuropsychological domains listed above that have been found to be significantly related to
employment status following TBI. A brief list of the tests included in the MNB and the
neuropsychological domains that they assess can be found in Table 4. For more complete
descriptions of individual tests, see the Method section below, Lezak et al. (2004), or Strauss,
Sherman, and Spreen (2006).
EMPLOYMENT STATUS FOLLOWING TBI 32
Table 4
Meyers Neuropsychological Battery Tests Organized by Neuropsychological Domain
Domain MNB Tests
Memory
Rey Auditory Verbal Learning Test
Rey Complex Figure Test (recall and recognition trials)
Digit-Span Backwards (WAIS-III)
Attention
Digit-Span Forward (WAIS-III)
Digit-Symbol Coding (WAIS-III)
Sentence Repetition
Trail Making Test
Visuospatial Skills
Rey Complex Figure Test (copy trial)
Judgment of Line Orientation
Block Design (WAIS-III)
Executive Functions Category Test
Similarities (WAIS-III)
General Cognitive
Functioning
WAIS-III Index Scores (VIQ, PIQ, FSIQ)
Overall test battery mean
Language Fluency Controlled Oral Word Association Test
Animal Naming Test
Motor Finger Tapping Test
Note: WAIS-III = Wechsler Adult Intelligence Scale, Third Edition; VIQ = Verbal
Intelligence Quotient; PIQ = Performance Intelligence Quotient; FSIQ = Full-scale
Intelligence Quotient.
EMPLOYMENT STATUS FOLLOWING TBI 33
The MNB includes several measures of different memory domains. Among these is the
RAVLT, a well-established verbal word-list learning task similar to the California Verbal
Learning Test-II used by Drake et al. (2000) and Han et al. (2009). The battery also contains the
Rey Complex Figure Test, a test of visual memory. Attention is assessed using the Digit Span
Forward and Digit-Symbol Coding subtests of the WAIS-III, the Sentence Repetition Test, and
the Trail Making Test. Measures of visuospatial skills are provided by the Rey Complex Figure
Test copy trial, Judgment of Line Orientation, and the Block Design subtest of the WAIS-III.
The MNB provides a measure of executive functions using the Category Test, a widely used test
of visual concept formation. The Similarities subtest of the WAIS-III, a test of verbal reasoning
and abstract concept formation, provides another measure of executive functions. General
cognitive functioning is represented by the Performance Intelligence Quotient (PIQ), Verbal
Intelligence Quotient (VIQ), and Full-scale Intelligence Quotient (FSIQ) scores from the Ward
Seven-Subtest Short Form of the WAIS-III, as well as the OTBM as described above. The MNB
contains the COWAT and the Animal Naming Test as measures of language fluency, and the
Finger Tapping Test as a test of motor performance. The battery can also be supplemented with a
number of general mental health questionnaires and measures of demographic variables as
needed.
Predicting Current Employment Status
The literature review presented above outlines a number of studies designed to create
models that predict how long it will take an individual to return to work following TBI (Drake et
al., 2000; Fleming et al., 1999; Guerin et al., 2006; Kreutzer et al., 2003; MacMillan et al., 2002;
Simpson & Schmitter-Edgecombe, 2002). However, the methodologies used in these studies
only allow for predictions about an individual’s future employment status to be made soon after
EMPLOYMENT STATUS FOLLOWING TBI 34
a TBI occurs. Furthermore, predictions have only been made about a patient’s employment status
at specific points in time, most commonly at six months or one year after the injury occurs. None
of the studies listed above have investigated the efficacy of neuropsychological assessment
scores to estimate current employment status as a function of time since injury. Such predictions
could be facilitated by assessing individuals at varied time points following a TBI, and
subsequently including the time since injury as a predictor variable in a regression equation used
to predict employment status. Estimates made by this model would be useful in guiding a
clinician’s discussion with their client about whether or not the client is ready to return to work
following TBI, regardless of the amount of time that had passed between the injury and the
assessment. These estimates might also be helpful in identifying neuropsychological domains
that would be appropriate candidates for cognitive intervention. The aim of this study is to create
such a model, in which neuropsychological assessment scores, demographic variables, and time
since injury are used to estimate the current employment status of persons who differ in their
time since TBI.
A study of variables associated with symptoms as well as those who were both referred
and not referred due to litigation. employment status following TBI ideally would involve the
use of cohorts systematically evaluated at designated time points following injury. These
analyses would include those individuals who were both referred and not referred for
neuropsychological evaluations due to persisting cognitive However, the composition of this
sample precludes such an analysis.
EMPLOYMENT STATUS FOLLOWING TBI 35
Goals
Using scores from a short neuropsychological assessment battery administered between 0
and 55 months after a TBI with a sample of persons referred for neuropsychological assessment,
the goals of the current investigation were to:
1. employ exploratory step-wise regression procedures in order to examine the incremental
proportion of variance in employment status accounted for with the step-wise addition of
each neuropsychological assessment variable, and identify the most parsimonious model
(i.e., the model that includes variables beyond which inclusion of additional variables
does not account for a significant proportion of additional variance).
2. calculate the percentage of cases correctly classified, sensitivity, specificity, positive
predictive value, and negative predictive value in order to examine the extent to which
the regression model mentioned above is able to correctly estimate an individual’s
employment status at the time of assessment.
3. examine the degree to which the addition of the demographic variables of age, premorbid
occupation, education, number of subjective symptoms, ethnic status, and independent
driving status increases the proportion of variance accounted for, as well as the
percentage of cases correctly classified, the sensitivity, the specificity, the positive
predictive value, and the negative predictive value of the regression model described
above.
4. examine the degree to which the addition of neuropsychological variables affects the
proportion of variance accounted for, percentage of cases correctly classified, sensitivity,
specificity, positive predictive value, and negative predictive value of a model which uses
only the demographic variables listed in goal three alone.
EMPLOYMENT STATUS FOLLOWING TBI 36
5. calculate the proportion of variance accounted for, percentage of cases correctly
classified, sensitivity, specificity, positive predictive value, and negative predictive value
in order to examine the efficacy of the predictive models identified in goals one, three,
and four in an independent and age-matched sample taken from the same database.
6. calculate the proportion of variance accounted for, percentage of cases correctly
classified, sensitivity, specificity, positive predictive value, and negative predictive value
in order to examine the efficacy of the predictive models identified in goals one, three,
and four in two additional samples of litigants and non-litigants taken from the same
database.
7. examine any changes in the proportion of variance in employment status accounted for,
correct classification rate, sensitivity, specificity, positive predictive value, negative
predictive value, or in the predictor variables that account for a significant amount of
variance in employment status when the models mentioned in goals three and four are
created in a sample of 76 litigants taken from the study database.
8. examine any changes in the proportion of variance in employment status accounted for,
correct classification rate, sensitivity, specificity, positive predictive value, negative
predictive value, or in the predictor variables that account for a significant amount of
variance in employment status when the models mentioned in goals three and four are
created using data from all cases contained in the study sample.
The decision to perform analyses based on litigation status in the current sample was based
on consistent findings in prior studies that involvement in litigation can be negatively associated
with performance on neuropsychological assessment measures (Belanger, Curtiss, Demery,
Lebowitz, & Vanderploeg, 2005; Binder & Rohling, 1996; Davis, McHugh, Axelrod, & Hanks,
EMPLOYMENT STATUS FOLLOWING TBI 37
2012; Meyers, Reinsch-Boothby, & Miller, 2011). Past survey-based research has also shown
that nearly 50% of practicing attorneys agree that they should "always" or "usually" inform their
clients of the presence of effort measures before they participate in a psychological evaluation
(Wetter & Corrigan, 1995), even though such warnings have been shown to alter performance
during neuropsychological assessment (Suhr & Gunstad, 2000). In discussing the assessment of
insufficient effort, a recent consensus conference of the American Academy of Clinical
Neuropsychology also stated that “there is substantial risk that [poor effort] could be present
among the numerous cases assessed by neuropsychologists in a secondary gain setting”, and that
“the presence of problematic effort and response bias can potentially invalidate results”
(Heilbronner, Sweet, Morgan, Larrabee, & Millis, 2009, p. 1121).
It would be possible to conduct further confirmatory analyses by creating additional groups
from the database sample based on important variables such as level of education, ethnic status,
impairment severity, or age. However, due to the limited sample size of this study and the risk of
increased familywise error with multiple analyses, such additional analyses are options for later
exploratory investigations but are beyond the scope of the current study.
Method
Participants
Data were extracted from a private practice neuropsychology database that was collected
in the Midwestern United States. Participants were identified for inclusion in the study if they
carried a diagnosis of TBI and had a complete set of data for all variables of interest. The study
sample included data from 192 participants (130 male and 62 female) between the ages of 18 and
76 (mean = 32.1; SD = 13.1). The study sample consisted of 184 participants who described
themselves as White, 4 participants who described themselves as Hispanic, 3 participants who
EMPLOYMENT STATUS FOLLOWING TBI 38
described themselves as Native American, and 1 participant who self-identified as being from a
mixed or biracial background.
All participants had been referred for neuropsychological assessment following a TBI as
identified by a primary care provider. Participants in the sample experienced loss of
consciousness ranging from no loss of consciousness to 56 days (mean = 3.6 days; SD = 7.7
days), with 84 participants experiencing loss of consciousness less than 30 minutes, and 108
participants experiencing loss of consciousness of 30 minutes or more. The number of years of
formal education ranged from 6 to 20 years (mean = 12.5 years; SD = 2.0 years). Seventy six
participants in the sample were referred for neuropsychological assessment for purposes related
to litigation, and 116 were referred by physicians to assist in treatment planning. Participants
with missing data values for any of the demographic or neuropsychological assessment variables
of interest were not included in the analysis. Consistent with previous research, (Meyers &
Volbrecht, 2003; Meyers, Volbrecht, Axelrod, & Reinsch-Boothby, 2011) data from participants
who failed two or more embedded effort measures during MNB administration were also
excluded. Embedded effort measures included highly irregular scores on the Rey Complex
Figure Test, Reliable Digit Span, the Forced Choice Test, the Token Test, Dichotic Listening,
Sentence Repetition, the Rey Auditory Verbal Listening Test recognition trial, Estimated Finger
Tapping, and Judgment of Line Orientation.
In order to examine the efficacy of predictive models across subgroups as described in
the goals section, study participants were subdivided in two ways. The first subdivision
randomly divided the sample into two equal-sized groups after the sample had been stratified by
age. The second subdivision was made based on the litigation status of study participants, with
participants who were, and were not, involved in litigation at the time of neuropsychological
EMPLOYMENT STATUS FOLLOWING TBI 39
assessment being placed in separate groups. These subdivisions were performed independent of
one another, resulting in four nonorthogonal groups as described below.
Group 1. The first group was created by stratifying the 192 study participants by age and
using a random number generator to select half of the study sample for inclusion in Group 1 (n =
96). Although there are additional variables by which the sample could have been stratified, the
size of the study sample limited the feasibility of multiple stratifications. Data from Group 1
were used to construct the initial prediction models, described in Goals 1 through 4 above.
Participants in Group 1 underwent neuropsychological assessment with the MNB
between 0 and 54 months following TBI (mean = 11.04; SD = 14.50). Group 1 consisted of 65
males and 31 females, with ages ranging from 18 to 61 years (mean = 32.40; SD = 12.68). The
number of years of formal education completed by members of Group 1 ranged from 9 to 20
years (mean = 12.54; SD = 1.72). Participants in Group 1 experienced loss of consciousness at
the time of TBI ranging from no loss of consciousness to 28 days (mean = 3.41 days; SD = 7.40
days). SCL-90 global impairment scale T-scores in Group 1 ranged from 40 to 80 (mean = 64.18;
SD = 11.54). Group 1 included 1 participant who self-described as mixed/biracial, 93
participants who described themselves as White, 1 participant who self-described as Native
American, and 1 participant who self-described as Hispanic. Employment backgrounds in Group
1 included 32 participants who were employed in unskilled positions at the time of their injury,
23 participants who were employed in semi-skilled positions, 14 participants who were not
working, 5 participants who were employed in skilled professions, 5 participants who were
working in manager/office/sales positions, 3 participants who were working as
technical/professionals, and 14 participants who were students at the time of their injury. At the
time of neuropsychological assessment, 79 participants in Group 1 were driving independently, 3
EMPLOYMENT STATUS FOLLOWING TBI 40
were partially driving, and 14 were partially not driving. Forty-nine participants in Group 1 were
involved in litigation at the time of neuropsychological assessment. A summary of demographic
information for Group 1 can be found in Table 5 below.
Group 2. Group 2 consisted of the 96 participants from the study sample who were not
selected for inclusion in Group 1. Data from Group 2 were used to validate the prediction models
created in Group 1. Cross-validation with this sample was done by utilizing a forced-entry
regression model that mimicked the initial prediction models, as described below in Data
Analysis. The purpose of this analysis was to investigate the efficacy of the initial prediction
models in an independent sample as described in Goal 5 above.
Participants in Group 2 underwent neuropsychological assessment with the MNB
between 0 and 55 months following TBI (mean = 9.07; SD = 11.90). Group 2 consisted of 65
males and 31 females, with ages ranging from 18 to 76 years (mean = 31.79; SD = 13.60). The
number of years of formal education completed by members of Group 2 ranged from 6 to 20
years (mean = 12.53; SD = 2.24). Participants in Group 2 experienced loss of consciousness at
the time of TBI ranging from no loss of consciousness to 56 days (mean = 3.85 days; SD = 8.07
days). SCL-90 global impairment scale T-scores in Group 2 ranged from 43 to 80 (mean = 64.17;
SD = 12.18). Group 2 included 91 participants who described themselves as White, 2
participants who described themselves as Native American, and 3 participants who described
themselves as Hispanic. Employment backgrounds in Group 2 included 26 participants who were
employed in unskilled positions at the time of their injury, 15 participants who were employed in
semi-skilled positions, 14 participants who were not working, 6 participants who were employed
in skilled professions, 11 participants who were working in manager/office/sales positions, 6
participants who were working as technical/professionals, and 18 participants who were students
EMPLOYMENT STATUS FOLLOWING TBI 41
at the time of their injury. At the time of neuropsychological assessment, 74 participants in
Group 2 were driving independently, 5 were partially driving, and 17 were partially not driving.
Thirty-five participants in Group 2 were involved in litigation at the time of neuropsychological
assessment. A summary of demographic information for Group 2 can be found in Table 5 below.
Group 3. The third group was composed of 74 participants from the study sample who
were involved in litigation directly related to their head injury at the time of neuropsychological
assessment. As mentioned above in Goals 6 and 7, Group 3 was created to examine the degree to
which litigation status was associated with neuropsychological test performance and employment
following a TBI.
Participants in Group 3 underwent neuropsychological assessment with the MNB
between 0 and 52 months following TBI (mean = 10.72; SD = 11.63). Group 3 consisted of 53
males and 23 females, with ages ranging from 18 to 76 years (mean = 34.08; SD = 12.83). The
number of years of formal education completed by members of Group 3 ranged from 9 to 20
years (mean = 12.61; SD = 1.95). At the time of TBI, participants in Group 3 experienced loss of
consciousness ranging from no loss of consciousness to 28 days (mean = 1.97 days; SD = 4.76
days). SCL-90 global impairment scale T-scores in Group 3 ranged from 50 to 80 (mean = 67.07;
SD = 10.12). Group 3 included 1 participant who self-described as mixed/biracial, 73
participants who described themselves as White, and 2 participants who described themselves as
Native American. Employment backgrounds in Group 3 included 26 participants who were
employed in unskilled positions at the time of their injury, 16 participants who were employed in
semi-skilled positions, 7 participants who were not working, 5 participants who were employed
in skilled professions, 8 participants who were working in manager/office/sales positions, 3
participants who were working as technical/professionals, and 11 participants who were students
EMPLOYMENT STATUS FOLLOWING TBI 42
at the time of their injury. At the time of neuropsychological assessment, 66 participants in
Group 3 were driving independently, 2 were partially driving, and 8 were partially not driving. A
summary of demographic information for Group 2 can be found in Table 5 below.
Group 4. The fourth group was composed of 115 participants from the study sample who
were not involved in litigation directly related to their head injury at the time of
neuropsychological assessment As mentioned above under Goals 6 and 7, Group 4 was created
to make non-statistical comparisons with Group 3 based on the results of exploratory stepwise
logistical regression models, to examine the degree to which litigation status was associated with
neuropsychological test performance and employment following a TBI.
Participants in Group 4 underwent neuropsychological assessment with the MNB
between 0 and 55 months following TBI (mean = 9.50; SD = 14.28). Group 4 consisted of 76
males and 39 females, with ages ranging from 18 to 71 years (mean = 30.73; SD = 13.24). The
number of years of formal education completed by members of Group 4 ranged from 6 to 20
years (mean = 12.45; SD = 1.99). At the time of TBI, participants in Group 4 experienced loss of
consciousness ranging from no loss of consciousness to 56 days (mean = 4.72 days; SD = 9.01
days). SCL-90 global impairment scale T-scores in Group 4 ranged from 40 to 80 (mean = 62.35;
SD = 12.54). Group 4 included 110 participants who described themselves as White, 4
participants who described themselves as Hispanic, and 1 participant who described themselves
as Native American. Employment backgrounds in Group 4 included 32 participants who were
employed in unskilled positions at the time of their injury, 22 participants who were employed in
semi-skilled positions, 21 participants who were not working, 5 participants who were employed
in skilled professions, 8 participants who were working in manager/office/sales positions, 6
participants who were working as technical/professionals, and 21 participants who were students
EMPLOYMENT STATUS FOLLOWING TBI 43
at the time of their injury. At the time of neuropsychological assessment, 86 participants in
Group 4 were driving independently, 6 were partially driving, and 23 were partially not driving.
A summary of demographic information for Group 4 can be found in Table 5 below. Results of
MANOVA analysis were non-significant (λ = 0.96, p = 0.76), indicating that differences
between groups 1, 2, 3, and 4 on the demographic variables of interest did not exceed those
expected by chance.
EMPLOYMENT STATUS FOLLOWING TBI 44
Table 5
Demographic Variables of Interest in Groups 1-4 and the Full Study Sample
Group 1 Group 2 Group 3 Group 4 Full Sample
Variable Mean SD Mean SD Mean SD Mean SD Mean SD
Age 32.4 12.7 31.8 13.6 34.1 12.8 30.7 13.2 32.1 13.1
Education 12.5 1.7 12.5 2.2 12.6 1.9 12.5 2.0 12.5 2.0
SCL-90 64.2 11.5 64.2 12.2 67.1 10.1 62.4 12.5 64.2 11.8
Loss of Consciousness 3.4 7.4 3.8 8.1 2.0 4.8 4.7 8.9 3.6 7.7
Variable n % n % n % n % n %
Sex
Male 65 68% 65 68% 53 70% 76 66% 130 68%
Female 31 32% 31 32% 23 30% 39 34% 62 32%
Premorbid Occupation
Not Working 14 15% 14 15% 7 9% 21 18% 28 15%
Semi-skilled 55 57% 41 43% 42 55% 54 47% 96 50%
Skilled 27 28% 41 43% 27 36% 40 35% 68 35%
Driving Status
Driving 82 85% 79 82% 68 89% 92 80% 161 84%
Not Driving 14 15% 17 18% 8 11% 23 20% 31 16%
Note: Results of MANOVA showed no significant between-group differences on variables of
interest. Cumulative percentages for some demographic variables may add up to more than
100% after rounding. Loss of consciousness is measured in days. SCL-90 = Symptom
Checklist-90 Global Impairment T-score.
EMPLOYMENT STATUS FOLLOWING TBI 45
Procedure
Neuropsychological examination. All participants were referrals to a private practice
neuropsychology clinic located in the Midwestern United States. Referrals were made as part of
routine neuropsychological practice, including referrals from neurologists and physicians to
assist in treatment planning. Each participant underwent neuropsychological assessment using
the MNB between 0 and 55 months following a TBI. All testing was performed during a single
office visit by a board certified clinical neuropsychologist or a trained psychometrist under the
supervision of the same neuropsychologist. Each visit also included a semi-structured interview
conducted by the neuropsychologist. All testing was conducted in an office space specifically
designated for neuropsychological test administration, and scores were entered into a
computerized database at the time of assessment.
Administration of the battery followed standardized MNB procedures, which include the
following subtests in the order presented: the Ward Seven Subtest form of the WAIS-III, a
forced-choice effort measure, Rey Complex Figure Test copy trial, Animal Naming, One Minute
Estimation, Rey Complex Figure Test immediate recall, COWAT, Dichotic Listening, the North
American Adult Reading Test, Sentence Repetition, Rey Complex Figure Test delay recall, Rey
Complex Figure Test recognition trial, RAVLT acquisition, distraction list, and immediate recall
trials, Judgment of Line Orientation, Boston Naming Test, Finger Tapping, Finger Localization,
Trail Making Test parts A and B, RAVLT delayed recall and recognition trials, Token Test, and
the Category Test. During administration, participants were offered a short break following the
WAIS-III, the Rey Complex Figure Test recognition trial, and the Category Test.
Following the Category Test, each participant was asked to complete the Symptom
Checklist-90-R (Derogatis, 1994). Participants also participated in a brief semi-structured
EMPLOYMENT STATUS FOLLOWING TBI 46
interview with the board certified clinical neuropsychologist designed to collect demographic
and background information, including employment status, sex, age in years, years of education
completed, ethnicity, handedness, marital status, and occupation. Data regarding whether or not
each participant was driving independently was also collected during the interview as an
indicator of independent functioning. The number of months since injury at the time of
neuropsychological assessment was also recorded. The variable number of months since injury
was grouped with neuropsychological assessment variables rather than demographic variables in
the current study so that it would be included in all planned analyses.
Assessment domains, instruments and measures. The neuropsychological measures
included in the data analyses represent the neuropsychological domains previously shown to be
significantly related to employment status following TBI. To the extent possible,
neuropsychological domains were measured using the same tests that have been found in
previous studies to be significantly related to employment status following TBI.
Memory.
RAVALT. The RAVLT was used to obtain measures of immediate, delayed, and
recognition memory. The RAVLT is a well-established verbal word-list learning task in which
the participant is asked to memorize a list of 15 words over five learning trials. The
memorization is followed by a distractor list trial, immediate and 20-minute delayed recall trials,
and a recognition trial. The score for each trial is the number of words correctly recalled or
recognized. The numerous recall trials allow the test to assess short term, long term, and
recognition verbal memory.
Factor analysis and structural equation modeling studies have shown that the RAVLT can
be described by two-factor models that represent either acquisition and retention (Vakil &
EMPLOYMENT STATUS FOLLOWING TBI 47
Blachstein, 1993) or short-term memory and long-term memory (Muller, Hasse-Sander, Horn,
Helmstaedter, & Elger, 1997). Factor analytic studies have also shown the RAVLT to be a test of
verbal learning and memory that are distinct from measures of attention, concentration and
intelligence (Ryan, Rosenberg, & Mittenberg, 1984). Scores from the RAVLT have been found
to have moderate convergent validity with scores from other measures of verbal memory,
including correlations ranging from r = 0.50 to r = 0.58 with the Wechsler Memory Scale-
Revised Logical Memory subtest and correlations ranging from r = 0.49 to r = 0.83 with the
California Verbal Learning Test (Delis, Kramer, Kaplan, & Ober, 1987; Johnstone, Vieth,
Johnson, & Shaw, 2000; Stallings, Boake, & Sherer, 1995). Scores from the RAVLT have also
been found to have modest correlations (r = 0.37 – r = 0.44) with scores from the Benton Visual
Retention Test, a test of visual memory (Magalhaes, Malloy-Diniz, & Hamdan, 2012). Scores
from the RAVLT have demonstrated good divergent validity (i.e. low correlation coefficients
ranging from r = 0.01 to r = 0.22) with scores from the Trail Making Test, a test of visual
attention (Magalhaes, Malloy-Diniz, & Hamdan, 2012). It has been found to be a valid measure
of left-hemisphere temporal lobe damage. Ivnik, Sharbrough, and Laws (1988) found that scores
from each of the learning trials, the immediate recall trial, and the delayed recall trial all
significantly discriminated (p < 0.01) between patients with intractable epilepsy who had
undergone temporal lobectomy on either the right or left side, with left-side lobectomy resulting
in lower scores. Similarly, Miceli, Caltagirone, Gainotti, Masullo, and Silveri (1981) found that
non-aphasic patients with left-hemisphere brain lesions scored significantly (p < 0.05) worse
than non-aphasic patients with right-hemisphere brain lesions on both the immediate and delayed
recall trials of the RAVLT.
EMPLOYMENT STATUS FOLLOWING TBI 48
Scores from the RAVLT have also demonstrated moderate test-retest reliability over
periods of 14-35 days, with Pearson r values ranging from 0.68 to 0.78 for the trials 1-5 total
score, 0.68 to 0.76 for the immediate recall trial, and 0.71 to 0.81 for the delayed recall trial
(Lemay, Bedard, Rouleau, & Tremblay, 2004; Magalhaes, Malloy-Diniz, & Hamdan, 2012). At
an interval of three months, test-retest reliability was found to be r = 0.70 for the trials 1-5 total
score, and r = 0.62 for the delayed recall trial (van den Burg & Kingma, 1999). The test has also
been found to have high internal consistency across learning trials as measured by Cronbach’s
coefficient α, with scores for different forms ranging from 0.80 to 0.91 (Magalhaes, Malloy-
Diniz, & Hamdan, 2012; van den Burg & Kingma, 1999).
Rey Complex Figure Test. The Rey Complex Figure Test was used to obtain measures of
visuospatial processing and visual memory. Administration and scoring of the Rey Complex
Figure Test followed the guidelines presented by Meyers and Meyers (1995). During
administration of the Rey Complex Figure Test, the participant is presented with a printed
reproduction of a complicated geometric figure, a pencil, and a blank piece of paper. They are
then instructed to copy the figure to the blank piece of paper and encouraged to make their
drawing as accurate as possible. The test also consists of two recall trials in which the participant
is asked to reproduce the figure from memory, an immediate recall trial that takes place 3
minutes after completing the copy trial and a delayed recall trial that takes place 30 minutes after
completing the copy trial. Following the delayed recall trial the patient participates in a
recognition trial. During the recognition trial, 12 individual elements of the original complicated
figure are presented alongside 12 distractor designs, and the patient is asked to circle all of the
elements that they recognize as part of the original figure.
EMPLOYMENT STATUS FOLLOWING TBI 49
For the copy, immediate recall, and delayed recall trials, the figure is divided into 18
distinct elements, and each element is assigned a score of 0 (not present), 0.5 (misplaced and
inaccurately drawn), 1 (misplaced but accurately drawn or properly placed but inaccurately
drawn), or 2 (properly placed and accurately drawn), resulting in a total possible score of 36
points. The recognition trial is scored by calculating the number of true positive, false positive,
true negative, and false negative responses. A total recognition score is also obtained by adding
the number of true positives and the number of true negatives.
A number of factor analytic studies have demonstrated that variability in scores from the
Rey Complex Figure Test often demonstrate significant convergent validity with scores from
other tests of memory and perceptual organization, while at the same time showing good
divergent validity with scores from tests designed to measure unrelated constructs. In a sample of
260 adults with head injuries, scores from the copy and immediate recall trials of the Rey
Complex Figure Test shared stronger correlations with other measures of perceptual organization
(r = 0.35 for copy trial and r = 0.50 for immediate recall trial), than they did with measures of
verbal comprehension (r = 0.10 for copy trial and r = 0.16 for immediate recall trial) or of
freedom from distractibility (r = 0.18 for copy trial and r = 0.19 for immediate recall trial;
Sherman, Strauss, Spellacy, & Hunter, 1995). In a separate factor analysis of memory
performance scores, measures obtained from the Rey Complex Figure Test immediate and
delayed recall trials loaded onto a nonverbal memory factor (r = 0.88 and r = 0.85, respectively)
along with measures from the Visual Reproduction subtest of the Wechsler Memory Scale
(Wechsler, 1945), another commonly used test of visual memory (Ostrosky-Solis, Jaime, &
Ardila, 1998). Finally, in a recent factor analysis of neuropsychological battery scores conducted
by Ponton, Gonzalez, Hernandez, Herrera, & Higareda (2000), a visuospatial factor emerged
EMPLOYMENT STATUS FOLLOWING TBI 50
which consisted of the Rey Complex Figure Test copy trial (r = 0.80) and immediate recall trial
(r = 0.84), as well as Raven’s Standard Progressive Matrices (r = 0.51; Raven, Raven, & Court,
1993), a test of visuospatial reasoning.
The manual for the Rey Complex Figure Test Meyers and Meyers (1995) scoring system
reports interrater reliabilities ranging from r = 0.93 and r = 0.99 for the copy, immediate recall,
and delayed recall trials, with a median interrater reliability coefficient of r = 0.94. The manual
also reports test-retest reliability coefficients of r = 0.76 for the immediate recall trial, r = 0.89
for the delayed recall trial, and r = 0.87 for the recognition total correct trial. Test-retest
reliabilities for the copy trial were not reported due to the restricted range of patient scores on
this particular subtest, which are usually near perfect. Other research (Taylor, Leung, & Deane,
2011) has suggested that scores from the copy trial of the Rey Complex Figure Test show modest
correlations with functional outcomes following TBI, specifically the ability to drive
independently.
Attention.
Digit-Symbol Coding. The WAIS-III Digit-Symbol Coding test was used to obtain a
measure of attention. During administration of the Digit-Symbol Coding subtest of the WAIS-III,
the patient is asked to pair written symbols with numbers using a key presented at the top of the
page. The test produces a single score, which is the number of symbols correctly copied by the
patient with a 120-second time period. Successful completion of the test depends on motor
persistence, sustained attention, response speed, and visuomotor coordination rather than on
memory or learning (Lezak et al., 2004). Mertens, Gagnon, Coulombe, and Messier (2006) found
that across age groups, scores from the Digit-Symbol Coding subtest correlated more strongly
with scores from the Symbol Search subtest of the WAIS-III, another test of visual-motor
EMPLOYMENT STATUS FOLLOWING TBI 51
attention (r = 0.83), than other subtests (r = 0.27 – 0.48). Ward, Ryan, and Axelrod (2000) used
exploratory factor analysis in the WAIS-III standardization sample to show that in a three-factor
model, scores from Digit-Symbol Coding shared the most variance with scores from other
subtests of attention and working memory such as Arithmetic, Digit-Span, and Letter-Number
Sequencing. In a sample of participants with moderate to severe TBI, scores from the Digit-
Symbol Coding subtest have shown high convergent validity with scores from the Repeatable
Battery for the Assessment of Neuropsychological Status Coding subtest (Randolph, 1998; r =
0.83), a similar test of visual-motor attention (McKay, Casey, Wertheimer, & Fichtenberg,
2007). Scores on the Digit-Symbol Coding subtest have been found to be sensitive to brain
injury, with TBI patients scoring significantly (p < 0.01) lower than age-, gender-, and education-
matched healthy controls, and with Cohen’s d effect sizes ranging from -0.61 to -1.47 depending
on the severity of injury (Langeluddecke & Lucas, 2003). Finally, a functional MRI study
performed by Usui and colleagues (2009) found that performance on the WAIS-III Digit-Symbol
Coding subtest was related to activation of prefrontal cortical areas involved in sustained
attention and performance on tasks of working memory.
Trail Making Test, Part B. The Trail Making Test, Part B was used to obtain a second
measure of attention. The Trail Making Test is a test of visuospatial attention that is conducted in
two parts. During administration of the first part of the test (Part A), the patient is presented with
a sheet of paper containing randomly-placed circles numbered from 1 to 25 and is asked to
connect the circles in numerical order as quickly as possible. Part A is thought to provide a
measure of scanning and visuomotor tracking. During the second part of the test (Part B), the
patient is asked to draw a line connecting numbered and lettered circles both numerically and
alphabetically, alternating between the numbered circles and the lettered circles. Part B is
EMPLOYMENT STATUS FOLLOWING TBI 52
thought to provide a measure of divided attention and cognitive flexibility. Moderate test-retest
reliabilities have been noted over a nine-month period for both Part A (r = 0.79) and Part B (r =
0.89; Mitrushina, Boone, Razani, & D’Elia, 2005). However, test-retest reliabilities are generally
higher for Part A than for Part B (Bornstein, Baker, & Douglas, 1987; Dikmen, Heaton, Grant, &
Tempkin, 1999; Mitrushina & Satz, 1991). A moderate correlation has been noted between
scores from the Trail Making Test Parts A and B (r = .31; Strauss et al., 2006), supporting the
idea that the two tests measure similar but nonequivalent constructs. The Trail Making Test has
also been found to have moderate to strong positive correlations with other tests of attention,
such as the Symbol-Digit Modalities Test (r = 0.39), the Visual Search and Attention Test (r =
0.50), and the Paced Serial Addition Test (r = 0.60; O’Donnell, McGregor, Dabrowski,
Oestreicher, & Romero, 1994; Royan, Tombaugh, Rees, & Francis, 2004).
Acker and Davis (1989) demonstrated the efficacy of the Trail Making Test in predicting
functional outcomes following TBI by using the Social Status Outcome Survey (Cope, 1982) to
track functional outcomes of 148 participants who had undergone neuropsychological
assessment following a TBI. Participants completed the Social Status Outcome Survey an
average of 3.8 years following assessment. Results showed significant correlations between
Social Status Outcome Survey and the Trail Making Test part A (r = 0.31) and Part B (r = 0.33).
Similar correlations with other measures of functional outcome have been noted elsewhere
(Ross, Millis, & Rosenthal, 1997).
Visuospatial skills.
Judgment of Line Orientation. The Judgment of Line Orientation test was used to obtain
a measure of visuospatial skills. Judgment of Line Orientation is a popular test of visual
perception in which a patient is shown 11 numbered radii that form a semicircle. For each test
EMPLOYMENT STATUS FOLLOWING TBI 53
item, the patient is presented with two additional angled radii and asked to identify the numbered
radii that are oriented identically to the item stimulus. Benton, Varney, and Hamsher (1978)
report high split-half reliabilities ranging from 0.89 to 0.94, as well as a test-retest reliability of
0.90 over a period of up to 21 days. Spencer, Wendell, Giggey, Seliger, Katzel, and Waldstein
(2013) found that the Judgment of Line Orientation test showed high internal consistency in both
a sample of 128 undergraduate students (α = 0.84) and 203 stroke- and dementia-free,
community dwelling older adults with a mean age of 66 years (α = 0.81). As a test of
visuospatial processing, Judgment of Line Orientation is widely considered to be sensitive to
right-hemisphere lesions (Lezak et al., 2003; Mitrushina et al., 2005), although this claim has
recently been called into question (Treccani & Cubelli, 2011).
Block Design. The Block Design subtest of the WAIS-III was used to obtain an additional
measure of visuospatial skills. During administration of the Block Design subtest of the WAIS-
III, the patient is asked to use between two and nine red and white colored blocks to recreate
patterns presented in a stimulus book. Similar to other construction tasks, the test measures both
spatial perception as well as motor execution (Lezak et al., 2004). Factor analysis showed that
the Block Design subtest was most strongly correlated with other WAIS-III subtests measuring
visuospatial constructs such as Matrix Reasoning, Visual Puzzles, and Picture Completion (The
Psychological Corporation, 2002). The test has been shown to be sensitive to closed-head
injuries in both hemispheres, but especially in the right hemisphere (Wilde, Boake, & Sherer,
2000).
Scores from the Block Design subtest have shown split half reliability coefficients of r =
0.85 to r = 0.90 in individuals under 75 years of age (The Psychological Corporation, 2002), and
a split half reliability of r = 0.92 in a sample of 22 individuals with TBI (Zhu, Tulsky, Price, &
EMPLOYMENT STATUS FOLLOWING TBI 54
Chen, 2001). Using data from the WAIS-III standardization sample, Iverson (2001) categorized
the Block Design subtest as one of the “most reliable” subtests of the WAIS-III, meaning that
scores from the test demonstrated internal consistency ranging from r = 0.88 to r = 0.99, as well
as test-retest reliability ranging from r = 0.75 to r = 0.99 (p. 186).
Executive functions.
Category Test. The Victoria Revision of the Category Test was used to obtain a measure
of executive functions. The Category Test is a test of visual concept formation, and was one of
the seven tests originally included in the Halstead Neuropsychological Battery. The test contains
seven subtests in which different patterns are presented visually, and the patient is told that each
pattern will remind him or her of a number between one and four. For each subtest, there is a
consistent principle that determines which number is represented, and it is up to the patient to
discover the principle and apply it to each item. The patient is able to test different principles by
using feedback from the examiner, who tells him or her whether each of their answers is correct
or incorrect. Scores on the Category Test show moderate positive correlations with Wisconsin
Card Sort Test, another popular test of visual concept formation (Strauss et al., 2006). While the
Category Test and the Wisconsin Card Sort Test are both thought to be tests of visual concept
formation, the modest correlations between the two are most likely due to the different formats
of the two tests. Adams and Trenton (1981) report that the Category Test is almost as valid as the
complete Halstead Neuropsychological Battery in detecting the presence or absence of brain
damage, citing the fact that 95% of individuals with brain damage scored significantly below
matched controls.
Using coefficient α, Lopez, Charter, and Newman (2000) found high internal consistency
for the Category Test total score (r = 0.97), as well as for subtests III to VII (r = 0.77 - 0.96).
EMPLOYMENT STATUS FOLLOWING TBI 55
The Victoria revision of the Category Test is a condensed version that only contains 86 of
the original 208 test items. In a sample of 86 adults with closed head injuries resulting from
motor vehicle accidents, scores did not differ significantly between 43 individuals who were
assessed using the Victoria revision (mean score = 41.88, SD = 13.03) and 43 individuals were
given the full Category Test (mean score = 41.79, SD = 10.34; Kozel & Meyers, 1998).
Similarities. The Similarities subtest of the WAIS-III was used to obtain an additional
measure of executive functions. The Similarities subtest is an instrument that provides a measure
of verbal reasoning and abstract concept formation. In it, the patient is presented with two words
and asked to identify how the words are similar to one another. The word pairs presented initially
represent very concrete relationships, but later progress to represent relationships which are
much more abstract. More points are awarded for answers that identify an abstract relationship
rather than a concrete likeness. Van der Heijden and Donders (2003) administered the WAIS-III
to 166 participants with TBI who were consecutive referrals to a Midwestern rehabilitation
facility. Confirmatory factor analysis supported a four-factor model for the WAIS-III, where the
Similarities subtest shared more variance with tests of verbal comprehension (p = 0.83) than with
tests of perceptual organization, working memory, or processing speed. The test has also been
found to have high split-half reliability in a sample of 22 TBI patients (r = 0.96; Zhu et al.,
2001). Scores on the Similarities subtest have been found to be sensitive to brain injury, with
TBI patients scoring significantly (p < 0.01) lower than age-, gender-, and education-matched
healthy controls, with Cohen’s d effect sizes ranging from -0.38 to -1.02 depending on the
severity of injury (Langeluddecke & Lucas, 2003).
EMPLOYMENT STATUS FOLLOWING TBI 56
General cognitive function.
WAIS-III index scores. The PIQ, VIQ, and FSIQ scores from the Ward Seven-Subtest
Short Form of the WAIS-III were used to obtain measures of general cognitive function. The
Ward Seven-subtest Short Form of the WAIS-III is a condensed administration of the traditional
WAIS-III that only includes the Information, Digit Span, Arithmetic, Similarities, Picture
Completion, Block Design, and Digit-symbol Coding subtests. Just like its full-form counterpart,
the Ward short form produces PIQ, VIQ and FSIQ scores. Pilgrim, Meyers, Bayless, and
Whetstone (1999) have shown that these short form index scores correlate strongly with the same
index scores obtained during full-form WAIS-III administration. They calculated and compared
PIQ, VIQ, and FSIQ scores from both the full-form and Ward Seven-subtest Short Form of the
WAIS-III in a sample of 111 individuals from a semirural area. They found that scores were
strongly correlated for the PIQ (r = 0.95), VIQ (r = 0.97), and FSIQ (r = 0.98), demonstrating the
usefulness of the short form as a valid measure of general cognitive functioning.
Overall test battery mean. The MNB’s OTBM score was used to obtain another measure
of general cognitive function. An OTBM is a descriptive statistic used to summarize
performance on a complete neuropsychological battery. The score is derived by calculating the
mean of the T-scores from each test included in the battery. A detailed description of the process
used for calculating the OTBM, as well as a discussion of its clinical utility has been provided by
Miller and Rohling (2001). The OTBM for each participant in the current study was calculated
using T-scores from all MNB subtests.
Language fluency.
COWAT. The COWAT was used to obtain a measure of language fluency. The COWAT
is a well-established test of phonemic fluency that has been shown to be a sensitive indicator of
EMPLOYMENT STATUS FOLLOWING TBI 57
brain dysfunction (Strauss et al., 2006). During COWAT administration, the patient is assigned a
letter of the alphabet and told to verbally list as many words as possible beginning with that letter
within a 60 second time limit. Patients are told to specifically avoid proper names as well as
different forms of the same word (i.e. ‘eat’ and ‘eating’). The process is repeated a total of three
times with different letters of the alphabet, and a total score is derived. Correlations between
tests of phonemic fluency using different letters of the alphabet have been shown to be high (r =
0.86; Ross, Furr, Carter, & Weinberg, 2006). The COWAT has also shown moderate correlations
(r = 0.52) with the Animal Naming Test, a measure of semantic fluency (Tombaugh, Kozak, &
Rees, 1999). It has also shown correlations ranging from r = 0.64 to r = 0.87 with the WAIS-III
Verbal Intelligence Quotient Index score, suggesting a strong verbal component to test
performance (Henry & Crawford, 2004).
Ruff, Light, Parker, and Levin (1996) found a high coefficient α for the three letters
presented during COWAT administration (r = 0.83), indicating high internal consistency across
items. They also noted a test-retest reliability of r = 0.74 over a period of six months. Interrater
reliabilities for the COWAT have been described as “near perfect” (Mitrushina et al., 2005, p.
202), and have been reported to be as high as r = 0.98 (Norris, Blankenship-Reuter, Snow-Turek,
& Finch, 1995).
Motor performance.
Finger Tapping Test. The Finger Tapping Test was used to obtain a measure of motor
performance. The Finger Tapping Test is a test of motor speed in which the patient is presented
with a flat wooden board to which a mechanical counter with a small lever is attached. The
patient is asked to rest their hand on the board and tap the lever with their index finger as quickly
as possible for a period of ten seconds. The mechanical counter keeps track of the number of
EMPLOYMENT STATUS FOLLOWING TBI 58
times the patient taps the lever during each trial. The patient repeats the process between five and
ten times with each hand, depending on the consistency of scores across trials. The Finger
Tapping Test was one of the tests used by Ward Halstead in his neuropsychological assessment
battery and is one of the most widely used tests of manual dexterity (Lezak et al., 2004).
The Finger Tapping Test has shown high convergent validity across handedness
asymmetries with the Purdue Peg Placement Test (Tiffin, 1968), a measure of motor dexterity (r
= 0.78; Triggs, Calvanio, Levine, Heaton, & Heilman, 2000). The Finger Tapping Test has also
shown low correlation with the Processing Speed Index of the WAIS-III (r = 0.25), suggesting
that it is able to measure speed of motor performance independent from speed of information
processing (Kennedy, Clement, & Curtiss, 2003). Scores on the Finger Tapping Test have been
found to correlate significantly (p = 0.01) with damage to the splenium of the corpus callosum 6-
months after moderate to severe TBI (Farbota, Bendlin, Alexander, Rowley, Dempsey, &
Johnson, 2012). Finger Tapping scores have also been found to share moderate correlations with
a number of functional outcomes following brain injury, including employment outcomes
(Prigatano, 1999).
Data Reduction
Demographic Variables.
In several cases, demographic variables were coded into multiple categories during data
collection. After coding, the demographic variables employment status, ethnicity, premorbid
occupation, and independent driving status demonstrated low membership in certain categories.
Such cases of low expected frequencies can result in decreased statistical power and inflated
odds ratios (Tabachnick & Fidell, 2007). In order to maximize statistical power, these variables
were re-coded into fewer categories in order to increase membership in each individual category.
EMPLOYMENT STATUS FOLLOWING TBI 59
Employment status. At the time of neuropsychological assessment, the employment
status of each participant was coded into one of nine possible categories. The coding included
seven categories for use with civilians and two categories for use with active duty members of
the armed forces. For civilians the categories were retired, disabled, not employed, volunteer,
non-competitive, below premorbid (i.e. a position that was considered less competitive than
premorbid employment), and same as premorbid (i.e. employment that was considered equally
competitive as premorbid employment). For active duty military the categories were
deployable/full duty, and non-deployable.
For the purposes of the current study, these employment categories were originally
collapsed into three main employment outcome groups based on the estimated
neuropsychological demand of each category. The three employment outcome groups were not
employed, partially employed, or fully employed. The employment outcome group not employed
included those who were rated as retired, disabled, and not employed. The employment outcome
group partially employed included those who were rated as volunteer, non-competitive, below
premorbid, and non-deployable. The employment outcome category fully employed included
those who were rated same as premorbid and deployable/full duty. However, under the
conditions described above, only one participant from the sample qualified as fully employed,
making the category impractical for use in data analysis. Instead, the employment categories
partially employed and fully employed were collapsed into a single group. The result was two
employment categories, not employed and employed, which were used as the criterion measure
for binary logistic regression analyses.
Ethnicity. During the neuropsychological assessment, the self-reported ethnicity of each
participant was coded into one of seven categories: African American, Asian, White, Hispanic,
EMPLOYMENT STATUS FOLLOWING TBI 60
Native American, Pacific Islander, or Mixed/Biracial. However, the ethnic composition of the
sample, which was predominantly White, resulted in low membership in ethnic minority
categories. In order to increase group membership and thus increase statistical power, the
ethnicity of each participant was re-coded into one of two categories: White or Ethnic Minority.
The category Ethnic Minority included the six minority categories that had previously been
coded individually.
Even after re-coding the ethnicity of each participant in the database sample, membership
in the Ethnic Minority category was low (Group 1 n = 3; Group 2 n = 5; Group 3 n = 3; Group 4
n = 5). Low group membership in the Ethnic Minority category resulted in reduced statistical
power and odds ratios which were difficult to interpret (Tabachnick & Fidell, 2007). The
variable ethnicity was therefore removed from data analysis and not considered in the creation of
any of the seven regression models.
Premorbid Occupation. At the time of neuropsychological assessment, the occupation of
each participant was recorded and coded into one of seven categories: technical/professional,
manager/office/sales, skilled, student, semi-skilled, unskilled, and, not working. In order to
maximize statistical power, employment categories were condensed into three groups for
analysis: not working (Group 1 n = 14; Group 2 n = 14; Group 3 n = 7; Group 4 n = 21), semi-
skilled (Group 1 n = 55; Group 2 n = 41; Group 3 n = 42; Group 4 n = 54), and skilled (Group 1
n = 27; Group 2 n = 41; Group 3 n = 27; Group 4 n = 40). The semi-skilled group contained the
categories unskilled and semi-skilled, while the skilled group contained the categories skilled,
manager/office/sales, technical/professional, and student.
Independent driving status. At the time of neuropsychological assessment, the driving
status of each participant was coded as driving, partially driving, partially not driving, or not
EMPLOYMENT STATUS FOLLOWING TBI 61
driving. In order to maximize statistical power, these categories were combined into two levels:
driving, which consisted of the prior categories driving and partially driving (Group 1 n = 82;
Group 2 n = 79; Group 3 n = 68; Group 4 n = 92), and not driving, which consisted of the prior
categories partially not driving and not driving (Group 1 n = 14; Group 2 n = 17; Group 3 n = 8;
Group 4 n = 23).
Neuropsychological assessment measures.
The variable percent change in RAVLT scores between short-delay and long-delay recall
trials was originally planned for inclusion in data analysis, based on evidence provided by Han et
al. (2009) that a similar score taken from the California Verbal Learning Test, Second Edition
(Delis et al., 2000) significantly distinguished between groups of employed and unemployed
individuals following TBI. However, calculation of this score became problematic for
participants who did not recall any words during the short-delay free recall trial of the RAVLT.
Han and colleagues calculated their percent change score by using the following formula:
(long delay free recall score – short delay free recall score) / (short delay free recall score)
For participants who did not recall any words during the short delay recall trial, the formula
required dividing by zero and was therefore unsolvable. This created missing values for 4
participants in Group 1 and 5 participants in Group 2, thus precluding these data from analysis.
After original analyses were conducted, it was noted that the percent change variable had not
reached statistical significance in any of the regression models. In order to maximize sample size
and statistical power, the variable was removed from analysis and all models were recalculated.
In an attempt to minimize the number of predictor variables, specific Symptom
Checklist-90-R subscale scores were not considered for inclusion in analyses. This is consistent
with past research demonstrating that TBI-related symptom endorsements on the Symptom
EMPLOYMENT STATUS FOLLOWING TBI 62
Checklist-90-R are no more predictive of functional outcomes following TBI than non-TBI-
related symptom endorsements (Hoofien, Barak, Vakil, & Gilboa, 2005).
Data Analysis
All analyses were performed using SPSS version 20. Predictor variables were tested for
any violation of linearity in the logit by using a Box-Tidwell approach in which each variable is
entered into a regression model along with an interaction term for the variable and its natural
logarithm. Violations of linearity in the logit are detected when interaction terms make
significant contributions to the final model (Tabachnick & Fidell, 2007). Forward stepwise
binary logistic regression analysis was then used to create eight models in order to accomplish
the eight study goals outlined in the Goals section above. For a summary of all eight models
created during analysis and the relevant groups in which the models were created and confirmed,
see Table 6.
EMPLOYMENT STATUS FOLLOWING TBI 63
Table 6
Exploratory Stepwise Binary Logistic Regression Models Created
Model
Number
Data
Source
Primary
Predictors
Secondary
Predictors
Criterion Variable Confirmation
Group(s)
1 Group 1
Neuropsychological
Assessment
Variables
None Employment Status at
time of assessment None
1b Group 1
Neuropsychological
Assessment
Variables
None Employment Status at
time of assessment Groups 2, 3, 4
2 Group 1
Neuropsychological
Assessment
Variables
Demographic Variables Employment Status at
time of assessment Groups 2, 3, 4
3 Group 1 Demographic
Variables
Neuropsychological
Assessment Variables
Employment Status at
time of assessment Groups 2, 3 4
4 Group 3
Neuropsychological
Assessment
Variables
Demographic Variables Employment Status at
time of assessment Group 3
5 Group 3 Demographic
Variables
Neuropsychological
Assessment Variables
Employment Status at
time of assessment Group 3
6 Entire
Sample
Neuropsychological
Assessment
Variables
Demographic Variables Employment Status at
time of assessment Entire Sample
7 Entire
Sample
Demographic
Variables
Neuropsychological
Assessment Variables
Employment Status at
time of assessment Entire Sample
Note: Data Source = The group from which data was utilized in the creation of the model;
Primary Predictors = variables entered into the regression model in Block 1; Secondary
Predictors = variables entered into the regression model in Block 2; Criterion Variable =
criterion variable that was used in the creation of the model; Confirmation Group = the group in
which the fit of the model was tested.
EMPLOYMENT STATUS FOLLOWING TBI 64
Goal 1. Employ exploratory step-wise regression procedures in order to examine the
incremental proportion of variance in employment status accounted for with the step-wise
addition of each neuropsychological assessment variable, and identify the most parsimonious
model (i.e., the model that includes variables beyond which inclusion of additional variables
does not account for a significant proportion of additional variance).
In order to accomplish Goal 1, Model 1 was created using data from participants in
Group 1. Neuropsychological test scores from the MNB were used as predictor variables, while
the employment status of each participant at the time of neuropsychological assessment was used
as a dichotomous criterion measure. A complete listing of the neuropsychological assessment
measures used in analysis can be found in Table 7. While additional predictor variables could
have been chosen from the MNB, the study sample size and statistical power considerations
precluded their inclusion. The MNB measures listed in Table 7 are those with the strongest
empirical support as predictors of employment outcomes following TBI.
All neuropsychological assessment predictor measures were entered into binary logistic
regression analysis in a single block using a forward conditional stepwise entry method. Default
settings in SPSS were changed to include a probability for stepwise entry of 0.15 and a
probability of stepwise removal of 0.40. The use of less stringent inclusion criteria in exploratory
logistic regression techniques was recommended by Tabachnick and Fidell (2007) to maximize
predictive accuracy by ensuring the inclusion of variables with beta coefficients that are different
from zero. The use of a binary logistic regression analysis was necessary in the creation of all
models due to the reduction of the planned criterion variable employment status to a dichotomous
variable, as described above under Data Reduction.
EMPLOYMENT STATUS FOLLOWING TBI 65
The adjusted R2 value was calculated using SPSS software during model creation as a
measure of the amount of variance in employment status that was accounted for by Model 1. Due
to the use of binary logistic regression analysis, the significance of the χ2 change for each step
was also calculated using SPSS software as a way of determining whether the addition of each
new variable accounted for a significant amount of variance in the criterion variable.
In order to examine the aggregated effect of including additional neuropsychological
variables, one additional model, to be known as Model 1b, was created by repeating the analyses
described above and allowing variable inclusion to continue until the specified selection criteria
did not result in the inclusion of any additional variables. Model creation was allowed to
continue regardless of the significance of the χ2 change statistic for each step. Due to the
hypothesized robustness of this new model, which would theoretically account for a larger
amount of variance in employment status in Group 1, it was planned for use in all subsequent
analyses.
Goal 2. Calculate the percentage of cases correctly classified, sensitivity, specificity,
positive predictive value, and negative predictive value in order to examine the extent to which
the regression model mentioned above is able to correctly estimate an individual’s employment
status at the time of assessment.
In order to accomplish Goal 2, employment status predictions were created for each
participant in the study sample using Model 1b following the initial step-wise, forward entry
regression analysis. The decision to use Model 1b for confirmatory analyses was based on that
model’s inclusion of more predictor variables, making it more robust than Model 1, which
contained only one predictor variable and which accounted for a significantly smaller proportion
of variance in employment status. Predicted employment status was then compared to observed
EMPLOYMENT STATUS FOLLOWING TBI 66
employment status in order to classify each individual prediction as a true positive, a false
positive, a true negative, or a false negative. For the current study, a true positive was defined as
a participant who was employed at the time of neuropsychological assessment and for whom the
model-based prediction was also that the participant was employed. A false positive was defined
as a participant who was not employed at the time of neuropsychological assessment but for
whom the model-based prediction was that the participant was employed. A true negative was
defined as a participant who was not employed at the time of neuropsychological assessment and
for whom the model-based prediction was also that the participant was not employed. A false
negative was defined as a participant who was employed at the time of neuropsychological
assessment but for whom the model-based prediction was that the participant was not employed.
Following the identification of true positive, false positive, true negative, and false
negative predictions, the percentage of cases correctly classified, sensitivity, specificity, positive
predictive value, and negative predictive value of Model 1b for participants in Group 1 were
calculated. The percentage of cases correctly classified was defined as the percentage of model
predictions in Group 1 that correctly estimated the employment status of a participant at the time
of neuropsychological assessment. To calculate the percentage of cases correctly classified in
Group 1, the following formula was used:
true positives + true negatives / n
Reporting the accuracy of a model in terms of the percentage of cases correctly classified has
been a commonly used technique in previous return to work models (Drake et al., 2000; Fleming
et al., 1999; Guerin et al., 2006; Kreutzer et al., 2003; MacMillan et al., 2002; Simpson &
Schmitter-Edgecombe, 2002) and reporting a similar figure in the current study allowed for non-
statistical, subjective comparisons between models and with previous research.
EMPLOYMENT STATUS FOLLOWING TBI 67
Sensitivity was defined as the proportion of participants who were employed at the time
of neuropsychological assessment who were so identified by model predictions. To calculate the
sensitivity of Model 1b for participants in Group 1, the following formula was used:
true positives / (true positives + false negatives)
Specificity was defined as the proportion of participants who were not employed at the
time of neuropsychological assessment who were so identified by model predictions. To
calculate the specificity of Model 1b for participants in Group 1, the following formula was used:
true negatives / (true negatives + false positives)
Positive predictive value was defined as the proportion of participants classified as
employed by model predictions who were employed at the time of neuropsychological
assessment. To calculate the positive predictive value of Model 1b for participants in Group 1,
the following formula was used:
true positives / (true positives + false positives)
Negative predictive value was defined as the proportion of participants classified as not
employed by model predictions who were not employed at the time of neuropsychological
assessment. To calculate the negative predictive value of Model 1b for participants in Group 1,
the following formula was used:
true negatives / (true negatives + false negatives)
The same formulas were used throughout the study for the calculation of the percentage of cases
correctly classified, sensitivity, specificity, positive predictive value, and negative predictive
value.
Goal 3. Examine the degree to which the addition of the demographic variables of age,
premorbid occupation, education, number of subjective symptoms, ethnic status, and
EMPLOYMENT STATUS FOLLOWING TBI 68
independent driving status increases the proportion of variance accounted for, as well as the
percentage of cases correctly classified, the sensitivity, the specificity, the positive predictive
value, and the negative predictive value of the regression model described above.
In order to accomplish Goal 3, Model 2 was created using data from Group 1. Analysis
employed the use of hierarchical, forward-entry stepwise binary logistic regression. As
predictors of employment status at the time of neuropsychological assessment, Model 2 utilized
the same variables as Model 1b, with the addition of the demographic variables age, education in
years, number of subjective symptoms as measured by the Symptom Checklist-90-R Global
Severity Index, independent driving status, and premorbid occupation. Again, the use of Model
1b as the foundation for analysis was based on the robustness of that model when compared to
Model 1, as described above under Goal 2.
Predictor variables were entered into the regression analysis in two blocks, with
neuropsychological assessment variables entered in Block 1 and demographic variables entered
in Block 2. In each block, variables were entered using the same stepwise procedure described
for Model 1b (see Goal 1 above). The employment status of each participant at the time of
neuropsychological assessment was used as the criterion measure. The adjusted R2 value was
calculated using SPSS software during model creation as a measure of the amount of variance in
employment status that was accounted for by Model 2. Following the creation of Model 2, the
percentage of cases correctly classified, positive predictive value, negative predictive value,
sensitivity, and specificity in Group were 1 calculated.
While the ease of collecting demographic information reduces the clinical utility of the
analysis described in Goal 3, the analysis was conducted in order to identify which demographic
variables were most strongly related to returning to work following a TBI.
EMPLOYMENT STATUS FOLLOWING TBI 69
Goal 4. Examine the degree to which the addition of neuropsychological variables affects
the proportion of variance accounted for, percentage of cases correctly classified, sensitivity,
specificity, positive predictive value, and negative predictive value of a model which uses only
the demographic variables listed in goal three alone.
As in Goal 3, the ease of collecting demographic information reduces the clinical utility
of the analyses described in Goal 4. However, past research has suggested that demographic
variables alone may be sufficient to explain differences in employment status following TBI
(Arango-Lasprilla et al., 2009). These analyses were conducted in order to examine the degree to
which neuropsychological assessment variables account for incremental variability in
employment status following a TBI beyond that accounted for by demographic variables alone.
In order to accomplish Goal 4, Model 3 was created using hierarchical, forward-entry
stepwise binary logistic regression. As predictors of employment status at the time of
neuropsychological assessment, Model 3 utilized the same neuropsychological assessment
measures and demographic variable values from Group 1 that were used during the creation of
Model 2 (see Tables 5 and 6). Predictor variables were entered in two blocks, with demographic
variables being entered in Block 1 and neuropsychological assessment variables being entered in
Block 2. In each block, variables were entered using the same stepwise procedure described for
Model 1b (see Goal 1 above). The employment status of each participant at the time of
neuropsychological assessment was used as the criterion measure. The adjusted R2 value was
calculated using SPSS software during model creation as a measure of the amount of variance in
employment status that was accounted for by Model 3. Following the creation of Model 3, the
percentage of cases correctly classified, positive predictive value, negative predictive value,
sensitivity, and specificity of model predictions in Group 1 were calculated.
EMPLOYMENT STATUS FOLLOWING TBI 70
Goal 5. Calculate the proportion of variance accounted for, percentage of cases
correctly classified, sensitivity, specificity, positive predictive value, and negative predictive
value in order to examine the efficacy of the predictive models identified in goals one, three, and
four in an independent and age-matched sample taken from the same database.
The size of the database sample resulted in an undesirably small ratio of participants to
predictor variables, which can increase the probability of errors and decrease the accuracy of
population estimates. Confirmatory analyses were therefore employed as a measure of the
robustness and efficacy of Models 1b, 2, and 3. Following the creation of these models, three
separate employment group membership predictions were created for each participant in Group 2
(i.e. one prediction from each of the three models). Cross tabulations were then created to
compare the observed employment status of each participant in Group 2 to their employment
status as predicted by each model in order to identify true positive, false positive, true negative,
and false negative predictions.
The robustness of each model was measured by calculating the percentage of cases
correctly classified in Group 2. Next, the sensitivity, specificity, positive predictive value, and
negative predictive value of each model’s predictions in Group 2 were also calculated.
Data from participants in Group 2 were then entered into a forced-entry binary logistic
regression procedure. As a criterion variable, the procedure utilized the employment status of
each individual at the time of neuropsychological assessment. As predictors of employment
status at the time of neuropsychological assessment, the forced-entry procedure utilized the
neuropsychological assessment variables retained in Model 1b. This forced-entry procedure
identified the extent to which those same neuropsychological assessment variables accounted for
variance in employment status in Group 2. The same forced-entry procedure was repeated two
EMPLOYMENT STATUS FOLLOWING TBI 71
more times, once using the neuropsychological assessment and demographic variables retained
by Model 2, and once using the neuropsychological assessment and demographic variables
retained by Model 3.
In the case that the performance of the original prediction models was largely different
when confirmed in Group 2, a contingency plan was created to increase statistical power by
removing variables from the initial prediction models that showed low statistical significance and
repeating the confirmatory analysis.
Goal 6. Calculate the proportion of variance accounted for, percentage of cases
correctly classified, sensitivity, specificity, positive predictive value, and negative predictive
value in order to examine the efficacy of the predictive models identified in goals one, three, and
four in two additional samples of litigants and non-litigants taken from the same database.
In order to accomplish Goal 6, additional confirmatory analyses were performed in order
to examine the extent to which Models 1b, 2, and 3 would be confirmed in groups 3 and 4. The
efficacy of Models 1b, 2 and 3 were tested in Groups 3 and 4 by creating cross tabulations that
compared actual employment status to employment status as predicted by each model. These
cross tabulations were used to calculate the number of true positive, false positive, true negative,
and false negative predictions from each model in groups 3 and 4. The percentage of cases
correctly classified in Groups 3 and 4 was then calculated, along with the sensitivity, specificity,
positive predictive value, and negative predictive value of each model’s predictions in the same
groups.
Data from participants in Group 3 were then entered into a forced-entry binary logistic
regression procedure. As a criterion variable, the procedure utilized the employment status of
each individual at the time of neuropsychological assessment. As predictors of employment
EMPLOYMENT STATUS FOLLOWING TBI 72
status at the time of neuropsychological assessment, the forced-entry procedure utilized the
neuropsychological assessment variables retained in Model 1b. This forced-entry procedure
identified the extent to which those same neuropsychological assessment variables accounted for
variance in employment status in Group 3. The same forced-entry procedure was repeated two
more times, once using the neuropsychological assessment and demographic variables retained
by Model 2, and once using the neuropsychological assessment and demographic variables
retained by Model 3. This same forced-entry procedure was then repeated to mimic the creation
of Models 1b, 2, and 3 using data from Group 4.
Goal 7. Examine any changes in the proportion of variance in employment status
accounted for, correct classification rate, sensitivity, specificity, positive predictive value,
negative predictive value, or in the predictor variables that account for a significant amount of
variance in employment status when the models mentioned in goals three and four are created in
a sample of 76 litigants taken from the study database.
In order to accomplish Goal 7, two additional models (Model 4 and Model 5) were
created using data from the 76 litigants comprising Group 3. The same exploratory hierarchical
stepwise binary logistic regression methods described for the creation of Models 2 and 3 were
used for the creation of Models 4 and 5, respectively (see Goal 3 and Goal 4 above). Two
separate employment status predictions were made for each participant in Group 3 using Models
4 and 5, and cross tabulations were created in order to compare the observed employment status
of each participant in Group 3 at the time of neuropsychological assessment with their
employment status as predicted by Models 4 and 5. These cross tabulations allowed for the
calculation of true positive, false positive, true negative, and false negative predictions.
Following these calculations, the percentage of cases in Group 3 correctly classified by Models 4
EMPLOYMENT STATUS FOLLOWING TBI 73
and 5 was determined. The sensitivity, specificity, positive predictive value, and negative
predictive value of each model’s predictions in Group 3 were also calculated. Subjective
comparisons of Models 4 and 5 with Models 2 and 3 were conducted by observing the calculated
correct classification rate, sensitivity, specificity, positive predictive power, and negative
predictive power for each model and determining which models produced higher relative values.
Observational comparisons were also made between Models 2, 3, 4, and 5 by comparing the
demographic and neuropsychological assessment variables that were retained by each model.
This was done in order to provide insight into the effects that litigation status may have on the
neuropsychological and demographic variables that are related to employment status following
TBI.
The adjusted R2 values of Models 4 and 5 were calculated using SPSS software during
model creation as a measure of the amount of variance in employment status that was accounted
for by each model.
Goal 8. Examine any changes in the proportion of variance in employment status
accounted for, correct classification rate, sensitivity, specificity, positive predictive value,
negative predictive value, or in the predictor variables that account for a significant amount of
variance in employment status when the models mentioned in goals three and four are created
using data from all cases contained in the study sample.
In order to accomplish Goal 8 and to increase sample size and thus maximize statistical
power, two additional models were created (Model 6 and Model 7). The creation of Models 6
and 7 mimicked the procedures used for Models 2 and 3, respectively (see Goal 3 and Goal 4
above), but was created using data from the entire sample of 192 participants. Two separate
employment status predictions were made for each participant in the study sample using Models
EMPLOYMENT STATUS FOLLOWING TBI 74
6 and 7, and cross tabulations were created to compare the observed employment status of each
participant in the study sample with their employment status as predicted by Models 6 and 7.
These cross tabulations allowed for the calculation of true positive, false positive, true negative,
and false negative predictions. Following these calculations, the percentage of cases in the study
sample correctly classified by Models 6 and 7 was determined. The sensitivity, specificity,
positive predictive value, and negative predictive value of each model’s predictions in the full
study sample were also calculated. Nonstatistical comparisons of Model 6 and 7 with Models 1b
through 5 were then conducted to provide insight into the effects of increased sample size and
statistical power on the neuropsychological and demographic variables that are related to
employment status following TBI. These comparisons were made using the same methods as
described above under Goal 7. The adjusted R2 values of Models 6 and 7 were calculated using
SPSS software during model creation as a measure of the amount of variance in employment
status that was accounted for by each model.
Following the creation of each model, power analysis was conducted using Statistics
Calculator, Version 3.0 (Soper, 2012) to ensure that the size of the database sample was
sufficient to detect significant contributions from predictor variables.
Results
Basic Data
A summary of the neuropsychological test scores for participants in Groups 1 through 4
can be found below in Table 7. Results of MANOVA analysis were non-significant (λ = 0.90, p
= 0.97) indicating that differences between Groups 1, 2, 3, and 4 on the neuropsychological
variables of interest did not exceed those expected by chance.
EMPLOYMENT STATUS FOLLOWING TBI 75
Table 7
Neuropsychological Variables of Interest in Groups 1-4 and the Full Study Sample
Group 1 Group 2 Group 3 Group 4
Variable Mean SD Mean SD Mean SD Mean SD
Months since injury 11.0 14.5 9.1 11.9 10.7 11.6 9.5 14.3
RAVLT long-delay free recall* 32.6 15.0 30.5 15.1 34.2 14.1 29.6 15.4
RAVLT recognition* 43.0 11.7 39.4 14.1 42.5 13.0 40.4 13.1
RCFT copy* 33.7 19.2 33.0 19.0 34.8 16.9 32.3 20.4
RCFT immediate recall* 38.8 18.1 36.1 19.7 41.2 16.5 34.9 20.1
RCFT delayed recall* 37.8 17.8 35.3 19.3 40.2 17.0 34.1 19.2
RCFT recognition total correct* 39.6 17.4 40.3 16.6 42.1 15.5 38.5 17.8
WAIS-III Digit-Symbol Coding* 39.9 8.7 39.4 9.6 40.5 7.8 39.0 9.9
Trail Making, Part B* 43.0 11.0 39.2 14.9 43.0 11.3 39.8 14.3
Judgment of Line Orientation* 50.3 9.4 49.1 9.5 49.7 8.3 49.6 10.1
WAIS-III Block Design* 47.1 10.1 45.3 9.7 46.7 8.4 45.8 10.8
Category Test* 38.9 11.6 38.9 13.0 37.7 11.6 39.5 12.8
WAIS-III Similarities* 45.1 8.3 44.8 7.8 45.4 8.6 44.6 7.6
WAIS-III FSIQ 90.2 13.2 89.0 12.8 90.1 11.6 89.0 13.6
WAIS-III PIQ 90.9 16.4 88.7 15.9 90.3 14.5 89.2 17.0
WAIS-III VIQ 91.0 11.3 90.6 12.1 91.3 11.1 90.3 12.0
OTBM 41.9 6.5 40.0 7.5 42.0 5.4 40.2 7.9
EMPLOYMENT STATUS FOLLOWING TBI 76
Table 7 (Cont.)
Group 1 Group 2 Group 3 Group 4
Variable Mean SD Mean SD Mean SD Mean SD
COWAT* 37.6 8.6 37.1 9.0 38.6 8.1 36.4 9.1
Finger Tapping: dominant* 40.0 9.9 37.8 11.7 39.6 9.2 38.3 11.8
Finger Tapping: non-dominant* 42.2 11.2 40.4 13.2 41.8 11.5 40.9 12.8
Note: An asterisk (*) indicates a T-score value. Results of MANOVA showed all
between-group differences on the variables of interest to be non-significant. RAVLT =
Rey Auditory Verbal Learning Test; RCFT = Rey Complex Figure Test; WAIS-III =
Wechsler Adult Intelligence Test, Third Edition; FSIQ = Full-scale Intelligence
Quotient; PIQ = Performance Intelligence Quotient; VIQ = Verbal Intelligence Quotient;
COWAT = Controlled Oral Word Association Test.
Initial Analyses
As stated above, statistically significant differences between Groups 1 through 4 on the
demographic, neuropsychological assessment, and employment variables used in analysis were
assessed using MANOVA and found to be non-significant (see Tables 5 and 7). Initial Box-
Tidwell analysis showed no significant contributions to a regression model from the interaction
terms of each predictor and its natural logarithm, indicating that there was no violation of
linearity in the logit (Tabachnick & Fidell, 2007). Results are organized below according to the
eight study goals outlined above under Goals.
EMPLOYMENT STATUS FOLLOWING TBI 77
Study Goals
Goal 1. Employ exploratory step-wise regression procedures in order to examine the
incremental proportion of variance in employment status accounted for with the step-wise
addition of each neuropsychological assessment variable, and identify the most parsimonious
model (i.e., the model that includes variables beyond which inclusion of additional variables
does not account for a significant proportion of additional variance).
In order to accomplish Goal 1, Model 1 was created using data from the 96 participants
comprising Group 1. Significant χ2 goodness-of-fit tests (p < 0.01) justified the addition of
neuropsychological assessment variables to a constant-only model. The regression model was
constructed using a forward selection step-wise procedure with χ2 selection criteria. During Step
1, the model retained the OTBM as a significant predictor of employment status, resulting in a
Cox and Snell R2 value of 0.21 and a χ2 change score of 22.92 (p < 0.01). In Step 2, the Judgment
of Line Orientation T-score was retained in addition to the OTBM, resulting in a Cox and Snell
R2 value of 0.23 (R2 change = 0.02) and a χ2 change score of 2.49 (p = 0.12). The resulting non-
significant χ2 change score during Step 2 indicated that the inclusion of additional variables did
not account for a significant portion of additional variance in employment status, and model
creation was halted following Step 1.
Hosmer and Lemeshow tests indicated a good fit for this one-variable model (p = 0.51),
and power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed an
obtained statistical power of 0.99. A summary of the two steps used to create Model 1, including
the χ2 change for each step, the significance of the χ2 change for each step, Cox and Snell R2
values for each step, the Cox and Snell R2 change value for each step, and the odds ratios for
each variable can be found in Table 8.
EMPLOYMENT STATUS FOLLOWING TBI 78
Table 8
Summary of the Two Steps Completed During Creation of Model 1
Variable χ2change p R2 R2
change β Exp(β)
Step 1 22.92 <0.01 0.21
OTBM 0.19 1.21
Step 2 2.49 0.12 0.23 0.02
OTBM 0.14 1.15
JOLO 0.05 1.06
Note: p values listed in the table represent the statistical significance of the χ2 change
for each step of model creation. JOLO = Judgment of Line Orientation Test.
As mentioned above in Data Analysis, Model 1b was created in order to examine the
aggregated effect of including additional neuropsychological predictor variables to Model 1.
Model 1b was created by repeating the analyses described above and allowing variable inclusion
to continue until the specified selection criteria did not result in the inclusion of any additional
variables. Model creation was allowed to continue regardless of the significance of the χ2 change
statistic for each step. The resulting model for predicting employment status retained three
variables in addition to the constant, the Ward 7-subtest PIQ, the Judgment of Line Orientation
T-score, and the Finger Tapping Test T-score from the dominant hand. The OTBM, which had
been included in Step 1 as described above, was removed during the final step of model creation
due to low statistical significance following the inclusion of additional variables. While χ2
change scores had not reached statistical significance during Step 2 (p = 0.12), Step 3 (p = 0.07),
or Step 4 (p = 0.10), post-hoc analysis showed that the χ2 change score between step 1 and step 4
was significant (p < 0.05), suggesting that the cumulative effect of including three additional
variables had accounted for a significant change in the amount of variance accounted for in
EMPLOYMENT STATUS FOLLOWING TBI 79
employment status. In Step 5, the removal of the OTBM resulted in a non-significant χ2 change
score (p = 0.98).This finding indicated that the amount of variance in employment status
accounted for was not significantly changed, and thus justified the removal of the variable.
Hosmer and Lemeshow tests indicated a good fit for this three-variable model (p > 0.05),
and power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed an
obtained statistical power of 0.95. A summary of the five steps used to create Model 1b,
including the χ2 change for each step, the significance of the χ2 change for each step, Cox and
Snell R2 values for each step, the Cox and Snell R2 change value for each step, and the odds
ratios for each variable can be found in Table 9.
EMPLOYMENT STATUS FOLLOWING TBI 80
Table 9
Summary of the Five Steps Completed During Creation of Model 1b
Variable χ2change p R2 R2
change β Exp(β)
Step 1 22.92 <0.01 0.21
OTBM 0.19 1.21
Step 2 2.49 0.12 0.23 0.02
OTBM 0.14 1.15
JOLO 0.05 1.06
Step 3 3.28 0.07 0.26 0.03
OTBM 0.10 1.10
JOLO 0.07 1.07
Tapping-Dominant 0.06 1.06
Step 4 2.65 0.10 0.28 0.02
OTBM <0.01 1.00
JOLO 0.07 1.07
Tapping-Dominant 0.07 1.07
PIQ 0.05 1.05
Step 5 <0.01 0.98 0.28 0.00
JOLO 0.07 1.07
Tapping-Dominant 0.07 1.07
PIQ 0.05 1.05
Note: p values listed in the table represent the statistical significance of the χ2 change for
each step of model creation. JOLO = Judgment of Line Orientation Test; Tapping-
Dominant = Finger Tapping Test, dominant hand score; PIQ = Performance Intelligence
Quotient.
EMPLOYMENT STATUS FOLLOWING TBI 81
Goal 2. Calculate the percentage of cases correctly classified, sensitivity, specificity,
positive predictive value, and negative predictive value in order to examine the extent to which
the regression model mentioned above is able to correctly estimate an individual’s employment
status at the time of assessment.
Model 1b consisted of the Ward 7-subtest PIQ score, the Judgment of Line Orientation T-
score, and the Finger Tapping Test T-score from the dominant hand. Using the analytic formulas
presented under Data Analysis, Goal 2 above, Model 1b showed a correct classification rate of
83.3% for employment status in Group 1, with 93.8% sensitivity and 61.3% specificity. In
addition, Model 1b demonstrated a positive predictive value of 83.6% and a negative predictive
value of 82.6% within Group 1.
Goal 3. Examine the degree to which the addition of the demographic variables of age,
premorbid occupation, education, number of subjective symptoms, ethnic status, and
independent driving status increases the proportion of variance accounted for, as well as the
percentage of cases correctly classified, the sensitivity, the specificity, the positive predictive
value, and the negative predictive value of the regression model described above.
In order to accomplish Goal 3, Model 2 was created using data from the 96 participants
comprising Group 1. Initial classification tables showed that the creation of a constant-only
model that categorized all participants as employed at the time of their assessment resulted in a
67.7% correct classification rate in Group 1. Chi-squared goodness-of-fit tests were significant (p
< 0.01), thus justifying the addition of neuropsychological assessment variables to the constant-
only model.
EMPLOYMENT STATUS FOLLOWING TBI 82
Model 2 was created in two blocks. The first block was completed in five steps that
mimicked the creation of Model 1b (see Goal 1 above). At the end of the first block, Model 2
was identical to Model 1b.
Following the first block, significant χ2 goodness-of-fit tests (p = 0.01) justified the
inclusion of additional, demographic predictor variables. The second block was completed in two
steps. In the first step, premorbid occupation was retained as a predictor variable, resulting in an
84.4% correct classification rate in Group 1 and a Cox and Snell R2 value of 0.35. In the second
step, independent driving status was retained as an additional predictor variable, resulting in an
86.5% correct classification rate and a Cox and Snell R2 value of 0.36 in Group 1.
The final model consisted of the WAIS-III PIQ (p = 0.03), the Judgment of Line
Orientation T-score (p = 0.12), the Finger Tapping Test dominant hand T-score (p = 0.03),
premorbid occupation (semi-skilled p < 0.01; skilled p = 0.01), and independent driving status
(driving p = 0.09). Hosmer and Lemeshow tests indicated a good fit for the final model (p =
0.09). Using the analytic formulas described above under Data Analysis, Goal 2, Model 2
showed a correct classification rate of 86.5% in Group 1, with 95.4% sensitivity and 67.7%
specificity. In addition, Model 2 demonstrated a positive predictive value of 86.1% and a
negative predictive value of 87.5% within Group 1.
Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed
an obtained statistical power of 0.99 for Model 2. A summary of the seven steps used to create
Model 2, including χ2 change scores, significance of χ2 change scores, Cox and Snell R2 values,
Cox and Snell R2 change scores for each step, and the odds ratios for each variable can be found
in Table 10.
EMPLOYMENT STATUS FOLLOWING TBI 83
Table 10
Summary of the Seven Steps Completed During Creation of Model 2
Variable χ2change p R2 R2
change β Exp(β)
Step 1 22.92 <0.01 0.21
OTBM 0.19 1.21
Step 2 2.49 0.12 0.23 0.02
OTBM 0.14 1.15
JOLO 0.05 1.06
Step 3 3.28 0.07 0.26 0.03
OTBM 0.10 1.10
JOLO 0.07 1.07
Tapping-Dominant 0.06 1.06
Step 4 2.65 0.10 0.28 0.02
OTBM <0.01 1.00
JOLO 0.07 1.07
Tapping-Dominant 0.07 1.07
PIQ 0.05 1.05
Step 5 <0.01 0.98 0.28 0.00
JOLO 0.07 1.07
Tapping-Dominant 0.07 1.07
PIQ 0.05 1.05
Step 6 9.24 0.01 0.35 0.07
JOLO 0.06 1.06
Tapping-Dominant 0.07 1.07
PIQ 0.06 1.06
Premorbid Employment
Semi-skilled 2.42 11.29
Skilled 2.40 11.01
EMPLOYMENT STATUS FOLLOWING TBI 84
Table 10 (Cont.)
Variable χ2change p R2 R2
change β Exp(β)
Step 7 2.92 0.09 0.36 0.01
JOLO 0.06 1.06
Tapping-Dominant 0.07 1.07
PIQ 0.05 1.06
Premorbid Employment
Semi-skilled 2.50 12.19
Skilled 2.49 12.01
Independent Driving 1.46 4.29
Note: p values presented represent the statistical significance of the χ2 change for each
step of model creation. JOLO = Judgment of Line Orientation Test; Tapping-dominant =
Finger Tapping Test, dominant hand; PIQ = Performance Intelligence Quotient.
Goal 4. Examine the degree to which the addition of neuropsychological variables affects the
proportion of variance accounted for, percentage of cases correctly classified, sensitivity,
specificity, positive predictive value, and negative predictive value of a model which uses only
the demographic variables listed in goal three alone.
In order to accomplish Goal 4, Model 3 was created using data from the 96 participants
comprising Group 1. Initial classification tables showed that the creation of a constant-only
model that categorized all participants as employed at the time of their assessment resulted in a
67.7% correct classification rate in Group 1. Chi-squared goodness-of-fit tests were significant (p
< 0.01), thus justifying the addition of demographic variables to the constant-only model.
Model 3 was created in two blocks. The first block was completed in three steps. In the
first step, analysis retained independent driving status as a predictor of employment status,
resulting in a 74.0% correct classification rate in Group 1 and a Cox and Snell R2 value of 0.11.
During the second step, analysis retained premorbid occupation as an additional predictor of
EMPLOYMENT STATUS FOLLOWING TBI 85
employment status, resulting in a 76.0% correct classification rate in Group 1 and a Cox and
Snell R2 value of 0.19. In the third step, education was retained as an additional predictor
variable, resulting in a 75.0% correct classification rate in Group 1 and a Cox and Snell R2 value
of 0.23.
Following the first block, significant (p < 0.01) χ2 goodness-of-fit tests justified the
inclusion of neuropsychological assessment predictor variables. The second block was
completed in four steps. In the first step, the WAIS-III PIQ was retained as a predictor of
employment status, resulting in an 82.3% correct classification rate in Group 1 and a Cox and
Snell R2 value of 0.32. During the second step, the Finger Tapping Test dominant hand T-score
was retained as a predictor variable, resulting in an 86.5% correct classification rate in Group 1
and a Cox and Snell R2 value of 0.36. In the third step, the Rey Complex Figure Test recognition
trial T-score was retained as an additional predictor of employment status, resulting in an 88.5%
correct classification rate in Group 1 and a Cox and Snell R2 value of 0.39. In the fourth step, the
COWAT T-score was retained as a final predictor variable, resulting in an 88.5% correct-
classification rate in Group 1 and a Cox and Snell R2 value of 0.40.
As predictor variables, the final model included education (p = 0.11), premorbid
occupation (semi-skilled p < 0.01; skilled p < 0.01), independent driving status (driving p =
0.04), PIQ (p = 0.01), COWAT T-score (p = 0.12), Finger Tapping Test dominant hand T-score
(p = 0.02), and the Rey Complex Figure recognition trial T-score (p = 0.05). Hosmer and
Lemeshow tests showed a poor fit for the final model (p < 0.01), indicating the possibility of
uneven distribution of model predictions across deciles of risk. Using the analytic formulas
presented above under Data Analysis, Goal 2, Model 3 showed a correct classification rate of
88.5% in Group 1, with 98.5% sensitivity and 67.7% specificity. In addition, Model 3
EMPLOYMENT STATUS FOLLOWING TBI 86
demonstrated a positive predictive value of 86.5% and a negative predictive value of 95.5%
within Group 1.
Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed
an obtained statistical power of 0.99 for Model 3. A summary of the seven steps used to create
Model 3, including χ2 change scores, significance of χ2 change scores, Cox and Snell R2 values,
Cox and Snell R2 change scores for each step, and the odds ratios for each variabe can be found
in Table 11.
EMPLOYMENT STATUS FOLLOWING TBI 87
Table 11
Summary of the Seven Steps Completed During Creation of Model 3
Variable χ2change p R2 R2
change β Exp(β)
Step 1 10.72 <0.01 0.11
Independent Driving 1.98 7.26
Step 2 9.76 0.01 0.19 0.08
Independent Driving 1.92 6.78
Premorbid Employment
Semi-skilled 1.72 5.59
Skilled 2.34 10.35
Step 3 4.85 0.03 0.23 0.04
Independent Driving 1.72 5.59
Premorbid Employment
Semi-skilled 1.94 6.98
Skilled 2.54 12.64
Education 0.39 1.48
Step 4 12.33 <0.01 0.32 0.09
Independent Driving 1.28 3.58
Premorbid Employment
Semi-skilled 2.43 11.37
Skilled 2.65 14.18
Education 0.31 1.36
PIQ 0.07 1.07
Step 5 5.78 0.02 0.36 0.04
Independent Driving 1.39 4.02
Premorbid Employment
Semi-skilled 2.67 14.39
Skilled 2.72 15.25
Education 0.33 1.39
PIQ 0.06 1.06
Tapping-Dominant 0.07 1.07
EMPLOYMENT STATUS FOLLOWING TBI 88
Table 11 (Cont.)
Variable χ2change p R2 R2
change β Exp(β)
Step 6 3.16 0.08 0.39 0.03
Independent Driving 1.31 3.69
Premorbid Employment
Semi-skilled 2.90 18.17
Skilled 2.88 17.84
Education 0.31 1.36
PIQ 0.06 1.06
Tapping-Dominant 0.06 1.06
RCFT-Recognition 0.04 1.04
Step 7 2.69 0.10 0.40 0.01
Independent Driving 2.00 7.40
Premorbid Employment
Semi-skilled 3.13 22.94
Skilled 3.30 27.05
Education 0.36 1.44
PIQ 0.07 1.07
Tapping-Dominant 0.08 1.08
RCFT-Recognition 0.04 1.04
COWAT -0.08 0.92
Note: p values presented represent the statistical significance of the χ2 change for each
step of model creation. PIQ = Performance Intelligence Quotient; Tapping – dominant =
Finger Tapping Test, dominant hand; RCFT = Rey Complex Figure Test; COWAT =
Controlled Oral Word Association Test.
EMPLOYMENT STATUS FOLLOWING TBI 89
Goal 5. Calculate the proportion of variance accounted for, percentage of cases
correctly classified, sensitivity, specificity, positive predictive value, and negative predictive
value in order to examine the efficacy of the predictive models identified in goals one, three, and
four in an independent and age-matched sample taken from the same database.
Initial calculations showed that classifying all members of Group 2 as employed would
result in a 58.3% correct classification rate. Using the analytical formulas presented above under
Data Analysis, Goal 2, Model 1b showed a 69.8% correct classification rate overall, with 85.7%
sensitivity and 47.5% specificity in Group 2. Model 1b had a positive predictive value of 69.6%
and a negative predictive value of 70.4% in Group 2.
Model 2 showed a 72.9% correct classification rate overall in Group 2, with 87.5%
sensitivity and 52.5% specificity. Model 2 had a positive predictive value of 72.1% and a
negative predictive value of 75.0% in Group 2.
Model 3 showed a 71.9% correct classification rate overall in Group 2, with 83.9%
sensitivity and 55.0% specificity. Model 3 had a positive predictive value of 72.3% and a
negative predictive value of 71.0% in Group 2.
Cox and Snell R2 values for Models 1b, 2, and 3 in Group 2 were 0.18, 0.28, and 0.29,
respectively. Table 12 shows the correct classification rates, sensitivity, specificity, positive
predictive value, negative predictive value, and Cox and Snell R2 value for Models 1b, 2, and 3
in Groups 1 and 2. The performance of the initial prediction models in Group 2 was considered
subjectively similar enough to their performance in Group 1 that further modification of the
models was deemed unnecessary.
EMPLOYMENT STATUS FOLLOWING TBI 90
Table 12
Statistical Performance of Models 1b, 2, and 3 in
Groups 1 and 2
Group 1 Group 2
Correct Classification Rate
Model 1b 83.3 69.8
Model 2 86.5 72.9
Model 3 88.5 71.9
Sensitivity
Model 1b 93.8 85.7
Model 2 95.4 87.5
Model 3 98.5 83.9
Specificity
Model 1b 61.3 47.5
Model 2 67.7 52.5
Model 3 67.7 55.0
Positive Predictive Value
Model 1b 83.6 69.6
Model 2 86.1 72.1
Model 3 86.5 72.3
Negative Predictive Value
Model 1b 82.6 70.4
Model 2 87.5 75.0
Model 3 95.5 71.0
R2
Model 1b 0.28 0.18
Model 2 0.36 0.28
Model 3 0.40 0.29
Note: Group 1 includes the 96 cases from which Models
1b, 2, and 3 were created, while Group 2 includes the 96
cases that were not included in the creation of Models
1b, 2, and 3.
EMPLOYMENT STATUS FOLLOWING TBI 91
Goal 6. Calculate the proportion of variance accounted for, percentage of cases correctly
classified, sensitivity, specificity, positive predictive value, and negative predictive value in order
to examine the efficacy of the predictive models identified in goals one, three, and four in two
additional samples of litigants and non-litigants taken from the same database.
Initial calculations showed that classifying all members of Group 3 as employed would
result in a 59.2% correct classification rate, while classifying all members of Group 4 as
employed would result in a 65.2% correct classification rate.
In Group 3, Model 1b showed a 76.3% correct classification rate with 93.3% sensitivity
and 51.6% specificity. Model 1b demonstrated a positive predictive value of 73.7%, and a
negative predictive value of 84.2% in Group 3. In Group 4, Model 1b showed a correct
classification rate of 79.1%, with 88.0% sensitivity and 62.5% specificity. Model 1b also
demonstrated a positive predictive value of 81.5%, and a negative predictive value of 73.5% in
Group 4.
In Group 3, Model 2 showed a 76.3% correct classification rate with 93.3% sensitivity
and 51.6% specificity. Model 2 demonstrated a positive predictive value of 73.7% and a negative
predictive value of 84.2% in Group 3. In Group 4, Model 2 showed a correct classification rate
of 79.1%, with 88.0% sensitivity and 62.5% specificity. Model 2 also demonstrated a positive
predictive value of 81.5% and a negative predictive value of 73.5% in Group 4.
In Group 3, Model 3 showed a 76.3% correct classification rate with 97.8% sensitivity
and 45.2% specificity. Model 3 demonstrated a positive predictive value of 72.1% and a negative
predictive value of 93.3% in Group 3. In Group 4, Model 3 showed a correct classification rate
of 82.6%, with 88.0% sensitivity and 72.5% specificity. Model 3 also demonstrated a positive
predictive value of 85.7% and a negative predictive value of 76.3% in Group 4.
EMPLOYMENT STATUS FOLLOWING TBI 92
In Group 3, Models 1b, 2, and 3 showed Cox and Snell R2 values of 0.18, 0.28, and 0.35,
respectively. In Group 4, the same models showed Cox and Snell R2 values of 0.22, 0.31, and
0.33, respectively. Table 13 outlines the performance of Models 1, 2, and 3 in Groups 3 and 4.
EMPLOYMENT STATUS FOLLOWING TBI 93
Table 13
Statistical Performance of Models 1b through 3 in
Groups 3 and 4
Group 3 Group 4
Correct Classification Rate
Model 1b 76.3 79.1
Model 2 76.3 79.1
Model 3 76.3 82.6
Sensitivity
Model 1b 93.3 88.0
Model 2 93.3 88.0
Model 3 97.8 88.0
Specificity
Model 1b 51.6 62.5
Model 2 51.6 62.5
Model 3 45.2 72.5
Positive Predictive Value
Model 1b 73.7 81.5
Model 2 73.7 81.5
Model 3 72.1 85.7
Negative Predictive Value
Model 1b 84.2 73.5
Model 2 84.2 73.5
Model 3 93.3 76.3
R2
Model 1b 0.18 0.22
Model 2 0.28 0.31
Model 3 0.35 0.33
Note: Group 3 includes the 76 participants from the study
sample who were involved in litigation at the time of
assessment, while Group 4 includes the 115 participants
from the study sample that were not involved in litigation
at the time of assessment.
EMPLOYMENT STATUS FOLLOWING TBI 94
Goal 7. Examine any changes in the proportion of variance in employment status
accounted for, correct classification rate, sensitivity, specificity, positive predictive value,
negative predictive value, or in the predictor variables that account for a significant amount of
variance in employment status when the models mentioned in goals three and four are created in
a sample of 76 litigants taken from the study database.
Model 4. Initial classification tables showed that the creation of a constant-only model
that categorized all participants as employed at the time of their assessment resulted in a 59.2%
correct classification rate for Group 3. Chi-squared goodness-of-fit tests were significant (p <
0.01), thus justifying the addition of neuropsychological assessment variables to the constant-
only model.
Model 4 was created in two blocks. The first block was completed in four steps. In the
first step, the OTBM was retained in addition to the constant as a predictor variable, resulting in
a 72.4% correct classification rate in Group 3 and a Cox and Snell R2 value of 0.20. In the second
step, number of months since injury was retained as a predictor variable, resulting in a 71.1%
correct classification rate in Group 3 and a Cox and Snell R2 value of 0.24. The Rey Complex
Figure Test delayed recall trial T-score was retained as a predictor variable during the third step,
resulting in a 76.3% correct classification rate in Group 3 and a Cox and Snell R2 value of 0.27.
Finally, during the fourth step, the Rey Complex Figure Test recognition trial T-score was
retained as a predictor variable, resulting in a 78.9% correct classification rate and a Cox and
Snell R2 value of 0.30.
Following the first block, significant χ2 goodness-of-fit tests (p = 0.01) justified the
inclusion of additional, demographic predictor variables. The second block was completed in two
steps. In the first step, education was retained as a predictor variable, resulting in an 81.6%
EMPLOYMENT STATUS FOLLOWING TBI 95
correct classification rate in Group 3 and a Cox and Snell R2 value of 0.35. In the second step,
age was retained as an additional predictor variable, resulting in an 81.6% correct classification
rate in Group 3 and a Cox and Snell R2 value of 0.39.
The final model consisted of number of months since injury (p = 0.04), the Rey Complex
Figure Test delayed recall trial T-score (p = 0.17), the Rey Complex Figure Test recognition trial
T-score (p = 0.12), the OTBM (p < 0.01), age (p = 0.04), and education (p = 0.02). Hosmer and
Lemeshow tests indicated a good fit for the final model (p = 0.79). Using the analytical formulas
presented above under Data Analysis, Goal 2, Model 4 showed a correct classification rate of
81.6% in Group 3, with 84.4% sensitivity and 77.4% specificity. In addition, Model 4
demonstrated a positive predictive value of 84.4% and a negative predictive value of 77.4%
within Group 3. Model 4 demonstrated a Cox and Snell R2 value of 0.39.
Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed
an obtained statistical power of 0.97 for Model 4. A summary of the six steps used to create
Model 4, including χ2 change scores, significance of χ2 change scores, Cox and Snell R2 values,
Cox and Snell R2 change scores for each step, and the odds ratios for each variable can be found
in Table 14.
EMPLOYMENT STATUS FOLLOWING TBI 96
Table 14
Summary of the Six Steps Completed During Creation of Model 4
Variable χ2change p R2 R2
change β Exp(β)
Step 1 17.01 <0.01 0.20
OTBM 0.21 1.23
Step 2 4.17 0.04 0.24 0.04
OTBM 0.19 1.21
Months Since Injury 0.05 1.05
Step 3 2.94 0.09 0.27 0.03
OTBM 0.30 1.35
Months Since Injury 0.06 1.06
RCFT-Delayed -0.04 0.96
Step 4 2.82 0.09 0.30 0.03
OTBM 0.27 1.31
Months Since Injury 0.07 1.07
RCFT-Delayed -0.05 0.95
RCFT-Recognition 0.04 1.04
Step 5 6.14 0.01 0.35 0.05
OTBM 0.29 1.33
Months Since Injury 0.06 1.06
RCFT-Delayed -0.06 0.94
RCFT-Recognition 0.03 1.03
Education 0.45 1.56
Step 6 4.55 0.03 0.39 0.04
OTBM 0.28 1.33
Months Since Injury 0.07 1.08
RCFT-Delayed -0.05 0.96
RCFT-Recognition 0.04 1.04
Education 0.52 1.68
Age -0.06 0.94
Note: p values presented represent the statistical significance of the χ2 change for each
step of model creation. RCFT-Delayed = Rey Complex Figure Test delayed recall trial;
RCFT-Recognition = Rey Complex Figure Test recognition trial.
EMPLOYMENT STATUS FOLLOWING TBI 97
Model 5. Initial classification tables showed that the creation of a constant-only model
that categorized all participants as employed at the time of their assessment resulted in a 59.2%
correct classification rate for Group 3. Chi-squared goodness-of-fit tests were significant (p <
0.01), thus justifying the addition of demographic predictor variables to the constant-only model.
Model 5 was created in two blocks. The first block was completed in one step, which
retained education as a predictor variable in addition to the constant. The first block resulted in a
64.5% correct classification rate in Group 3 and a Cox and Snell R2 value of 0.13.
Following the first block, significant (p < 0.01) χ2 goodness-of-fit tests justified the
inclusion of additional, neuropsychological assessment predictor variables. The second block
was completed in four steps. In the first step, the OTBM was retained as a predictor variable,
resulting in a 73.7% correct classification rate in Group 3 and a Cox and Snell R2 value of 0.27.
In the second step, the Rey Complex Figure Test delayed recall trial T-score was retained as an
additional predictor variable, resulting in a 73.7% correct classification rate in Group 3 and a
Cox and Snell R2 value of 0.30. During the third step, the Category Test T-score was retained as
a predictor variable, resulting in a 76.3% correct classification rate and a Cox and Snell R2 value
of 0.34. During the fourth step, number of months since injury was retained as an additional
predictor variable, resulting in an 80.3% correct classification rate and a Cox and Snell R2 value
of 0.36.
The final model consisted of number of months since injury (p = 0.13), the Rey Complex
Figure Test delayed recall trial T-score (p = 0.03), the Category Test T-score (p = 0.10), the
OTBM (p < 0.01), and education (p = 0.01). Hosmer and Lemeshow tests indicated a good fit for
the final model (p = 0.69), indicating an even distribution of model predictions across deciles of
risk. Model 5 showed a correct classification rate of 80.3% in Group 3, with 88.9% sensitivity
EMPLOYMENT STATUS FOLLOWING TBI 98
and 67.7% specificity. In addition, Model 5 demonstrated a positive predictive value of 80.0%
and a negative predictive value of 80.8% within Group 3. Model 5 demonstrated a Cox and Snell
R2 value of 0.36.
Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed
an obtained statistical power of 0.94 for Model 5. A summary of the five steps used to create
Model 5, including χ2 change scores, significance of χ2 change scores, Cox and Snell R2 values,
Cox and Snell R2 change scores for each step, and the odds ratios for each variable can be found
in Table 15.
EMPLOYMENT STATUS FOLLOWING TBI 99
Table 15
Summary of the Five Steps Completed During Creation of Model 5
Variable χ2change p R2 R2
change β Exp(β)
Step 1 10.44 <0.01 0.13
Education 0.48 1.62
Step 2 13.87 <0.01 0.27 0.14
Education 0.46 1.59
OTBM 0.20 1.22
Step 3 3.03 0.08 0.30 0.03
Education 0.51 1.67
OTBM 0.30 1.35
RCFT-Delayed -0.04 0.96
Step 4 3.83 0.05 0.34 0.04
Education 0.63 1.88
OTBM 0.28 1.32
RCFT-Delayed -0.05 0.95
Category Test 0.06 1.06
Step 5 2.63 0.11 0.36 0.02
Education 0.56 1.75
OTBM 0.28 1.33
RCFT-Delayed -0.06 0.94
Category Test 0.05 1.05
Months Since Injury 0.05 1.05
Note: p values presented represent the statistical significance of the χ2 change for each
step of model creation. RCFT-Delayed = Rey Complex Figure Test delayed recall trial.
EMPLOYMENT STATUS FOLLOWING TBI 100
Goal 8. Examine any changes in the proportion of variance in employment status
accounted for, correct classification rate, sensitivity, specificity, positive predictive value,
negative predictive value, or in the predictor variables that account for a significant amount of
variance in employment status when the models mentioned in goals three and four are created
using data from all cases contained in the study sample.
Model 6. In order to accomplish Goal 8, Model 6 was created using data from the entire
sample of 192 participants. Initial classification tables showed that the creation of a constant-
only model that categorized all participants as employed at the time of their assessment resulted
in a 63.0% correct classification rate for the study sample. Chi-squared goodness-of-fit tests were
significant (p < 0.01), thus justifying the addition of neuropsychological assessment variables to
the constant-only model.
Model 6 was created in two blocks. The first block was completed in three steps. In the
first step, the OTBM was retained in addition to the constant as a predictor variable, resulting in
a 76.6% correct classification rate in the study sample and a Cox and Snell R2 value of 0.21. In
the second step, the WAIS-III PIQ was retained as a predictor variable, resulting in a 78.1%
correct classification rate in the study sample and a Cox and Snell R2 value of 0.22. The
COWAT T-score was retained as a predictor variable during the third step, resulting in a 78.1%
correct classification rate in the study sample and a Cox and Snell R2 value of 0.23.
Following the first block, significant (p < 0.01) χ2 goodness-of-fit tests justified the
inclusion of additional, demographic predictor variables. The second block was completed in two
steps. In the first step, premorbid occupation was retained as a predictor variable, resulting in a
77.6% correct classification rate in the study sample and a Cox and Snell R2 value of 0.30. In the
EMPLOYMENT STATUS FOLLOWING TBI 101
second step, independent driving status was retained as an additional predictor variable, resulting
in a 78.6% correct classification rate in the study sample and a Cox and Snell R2 value of 0.32.
The final model consisted of the WAIS-III PIQ (p = 0.15), the COWAT T-score (p =
0.07), the OTBM (p = 0.02), premorbid occupation (semi-skilled p < 0.01; skilled p < 0.01), and
independent driving status (driving p = 0.06). Hosmer and Lemeshow tests indicated a good fit
for the final model (p = 0.26). Model 6 showed a correct classification rate of 78.6% within the
full sample, with 91.7% sensitivity and 56.3% specificity. In addition, Model 6 demonstrated a
positive predictive value of 78.2% and a negative predictive value of 80.0% within the full study
sample. Model 6 demonstrated a Cox and Snell R2 value of 0.32.
Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed
an obtained statistical power of 0.99 for Model 6. A summary of the five steps used to create
Model 6, including χ2 change scores, significance of χ2 change scores, Cox and Snell R2 values,
Cox and Snell R2 change scores for each step, and the odds ratios for each variable can be found
in Table 16.
EMPLOYMENT STATUS FOLLOWING TBI 102
Table 16
Summary of the Five Steps Completed During Creation of Model 6
Variable χ2change p R2 R2
change β Exp(β)
Step 1 45.42 <0.01 0.21
OTBM 0.16 1.18
Step 2 3.20 0.07 0.22 0.01
OTBM 0.11 1.12
PIQ 0.03 1.03
Step 3 2.21 0.13 0.23 0.01
OTBM 0.15 1.16
PIQ 0.03 1.03
COWAT -0.04 0.96
Step 4 18.23 <0.01 0.30 0.07
OTBM 0.15 1.16
PIQ 0.03 1.03
COWAT -0.04 0.96
Premorbid Employment
Semi-skilled 2.13 8.38
Skilled 2.45 11.58
Step 5 3.65 0.06 0.32 0.02
OTBM 0.13 1.14
PIQ 0.03 1.03
COWAT -0.05 0.95
Premorbid Employment
Semi-skilled 2.16 8.69
Skilled 2.51 12.31
Independent Driving 1.11 3.03
Note: p values presented represent the statistical significance of the χ2 change for each
step of model creation. PIQ = Performance Intelligence Quotient; COWAT = Controlled
Oral Word Association Test.
EMPLOYMENT STATUS FOLLOWING TBI 103
Model 7. In order to accomplish Goal 8, Model 7 was created using data from the entire
study sample of 192 participants. Initial classification tables showed that the creation of a
constant-only model that categorized all participants as employed at the time of their assessment
resulted in a 63.0% correct classification rate for the study sample. Chi-squared goodness-of-fit
tests were significant (p < 0.01), thus justifying the addition of demographic predictor variables
to the constant-only model.
Model 7 was created in two blocks. The first block was completed in three steps. The first
step retained premorbid occupation as a predictor variable in addition to the constant, resulting
in a 72.4% correct classification rate in the study sample and a Cox and Snell R2 value of 0.15.
During the second step, independent driving status was retained as an additional predictor
variable, resulting in a 75.5% correct classification rate in the study sample and a Cox and Snell
R2 value of 0.22. In the third step, education was retained as a predictor variable, resulting in a
75.5% correct classification rate in the study sample and a Cox and Snell R2 value of 0.23.
Following the first block, significant (p < 0.01) χ2 goodness-of-fit tests justified the
inclusion of additional, neuropsychological assessment predictor variables. The second block
was completed in five steps. In the first step, the OTBM was retained as a predictor variable,
resulting in a 78.6% correct classification rate in the study sample and a Cox and Snell R2 value
of 0.30. In the second step, the COWAT T-score was retained as an additional predictor variable,
resulting in a 77.6% correct classification rate in the study sample and a Cox and Snell R2 value
of 0.31. During the third step, the Rey Complex Figure Test immediate recall trial T-score was
retained as a predictor variable, resulting in a 79.7% correct classification rate and a Cox and
Snell R2 value of 0.32. During the fourth step, the Rey Complex Figure Test recognition trial T-
score was retained as an additional predictor variable, resulting in a 78.6% correct classification
EMPLOYMENT STATUS FOLLOWING TBI 104
rate in the study sample and a Cox and Snell R2 value of 0.34. Finally, number of months since
injury was retained as a predictor variable, resulting in a 79.2% correct classification rate in the
study sample and a Cox and Snell R2 value of 0.35.
The final model consisted of education (p = 0.52), premorbid occupation (semi skilled p
< 0.01; skilled p < 0.01), independent driving status (driving p = 0.09), number of months since
injury (p = 0.09), the COWAT T-score (p = 0.04), the Rey Complex Figure Test immediate
recall trial T-score (p = 0.05), the Rey Complex Figure Test recognition trial T-score (p = 0.06),
and the OTBM (p < 0.01). Hosmer and Lemeshow tests indicated a good fit for the final model
(p = 0.35), indicating an even distribution of model predictions across deciles of risk. Model 7
showed a correct classification rate of 79.2% within the full sample, with 89.3% sensitivity and
62.0% specificity. In addition, Model 7 demonstrated a positive predictive value of 80.0% and a
negative predictive value of 77.2% within the full study sample. Model 7 demonstrated a Cox
and Snell R2 value of 0.35.
Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed
an obtained statistical power of 0.99 for Model 7. A summary of the eight steps used to create
Model 7, including correct classification rates, χ2 change scores, significance of χ2 change scores,
Cox and Snell R2 values, and Cox and Snell R2 change scores for each step can be found in Table
17. Table 18 outlines the predictor variables retained in each of the 8 models along with their
accompanying significance values. Table 19 shows the correct classification rate, sensitivity,
specificity, positive predictive value, negative predictive value, and R2 value for Models 1b
through 7 in the groups from which they were created.
EMPLOYMENT STATUS FOLLOWING TBI 105
Table 17
Summary of the Eight Steps Completed During Creation of Model 7
Variable χ2change p R2 R2
change β Exp(β)
Step 1 30.31 <0.01 0.15
Premorbid Employment
Semi-skilled 2.22 9.20
Skilled 2.71 14.95
Step 2 16.24 <0.01 0.22 0.07
Premorbid Employment
Semi-skilled 2.14 8.50
Skilled 2.61 13.65
Independent Driving 1.81 6.14
Step 3 2.44 0.12 0.23 0.01
Premorbid Employment
Semi-skilled 2.09 8.10
Skilled 2.43 11.33
Independent Driving 1.76 5.82
Education 0.15 1.17
Step 4 19.01 <0.01 0.30 0.07
Premorbid Employment
Semi-skilled 2.13 8.42
Skilled 2.30 9.97
Independent Driving 0.81 2.25
Education 0.11 1.12
OTBM 0.14 1.15
Step 5 3.61 0.06 0.31 0.01
Premorbid Employment
Semi-skilled 2.15 8.58
Skilled 2.40 11.07
Independent Driving 1.05 2.87
Education 0.11 1.11
OTBM 0.17 1.19
COWAT -0.05 0.95
EMPLOYMENT STATUS FOLLOWING TBI 106
Table 17 (Cont.)
Variable χ2change p R2 R2
change β Exp(β)
Step 6 3.39 0.07 0.32 0.01
Premorbid Employment
Semi-skilled 2.41 11.09
Skilled 2.66 14.29
Independent Driving 1.00 2.71
Education 0.11 1.12
OTBM 0.24 1.27
COWAT -0.05 0.95
RCFT-Immediate -0.03 0.97
Step 7 3.37 0.07 0.34 0.02
Premorbid Employment
Semi-skilled 2.51 12.33
Skilled 2.78 16.09
Independent Driving 1.01 2.76
Education 0.10 1.10
OTBM 0.21 1.23
COWAT -0.05 0.95
RCFT-Immediate -0.03 0.97
RCFT-Recognition 0.03 1.03
Step 8 3.25 0.07 0.35 0.01
Premorbid Employment
Semi-skilled 2.66 14.34
Skilled 3.03 20.66
Independent Driving 1.02 2.76
Education 0.07 1.07
OTBM 0.21 1.23
COWAT -0.06 0.94
RCFT-Immediate -0.03 0.97
RCFT-Recognition 0.03 1.03
Months Since Injury 0.03 1.03
Note: p values presented represent the statistical significance of the χ2 change for each
step of model creation. COWAT = Controlled Oral Word Association Test; RCFT-
Immediate = Rey Complex Figure Test immediate recall trial; RCFT-Recognition = Rey
Complex Figure Test recognition trial.
EMPLOYMENT STATUS FOLLOWING TBI 107
Table 18
Variables Retained During Regression Analyses in Models 1 through 7
Variable Name Model
1
Model
1b
Model
2
Model
3
Model
4
Model
5
Model
6
Model
7
Age 0.04
Education 0.11 0.02 0.01 0.52
Independent driving status 0.09 0.04 0.06 0.09
Occupation (semi-skilled) 0.01 <0.01 <0.01 <0.01
Occupation (skilled) 0.01 <0.01 <0.01 <0.01
Months since injury 0.04 0.13 0.09
PIQ 0.02 0.03 0.01 0.15
COWAT* 0.12 0.07 0.04
Judgment of Line Orientation* 0.05 0.12
Finger Tapping dominant* 0.03 0.03 0.02
Booklet Category* 0.10
RCFT immediate recall* 0.05
RCFT delayed recall* 0.17 0.04
RCFT recognition* 0.05 0.12 0.06
OTBM <0.01 <0.01 <0.01 0.02 <0.01
Note: For each variable in each model, the numbers shown represent the value of p. An
asterisk (*) indicates that the score was represented by a T-score value. Occupation =
premorbid occupation; PIQ = Performance Intelligence Quotient; COWAT = Controlled Oral
Word Association Test; RCFT = Rey Complex Figure Test.
EMPLOYMENT STATUS FOLLOWING TBI 108
Table 19
Statistical Performance of Models 1b through 7
Model Correct Classification Rate Sensitivity Specificity PPV NPV R2
1b 83.3% 93.8% 61.3% 83.6% 82.6% 0.28
2 86.5% 95.4% 67.7% 86.1% 87.5% 0.36
3 88.5% 98.5% 67.7% 86.5% 95.5% 0.40
4 81.6% 84.4% 77.4% 84.4% 77.4% 0.39
5 80.3% 88.9% 67.7% 80.0% 80.8% 0.36
6 78.6% 91.7% 56.3% 78.2% 80.0% 0.32
7 79.2% 89.3% 62.0% 80.0% 77.2% 0.35
Note: All values represent the statistical performance of each model in the group from which
it was created. PPV = positive predictive value; NPV = negative predictive value.
EMPLOYMENT STATUS FOLLOWING TBI 109
Discussion
A Parsimonious Model to Predict Employment Status
Several hierarchical regression models were created to develop a parsimonious model
that utilized scores from a standard administration of the MNB to predict employment status in a
population of individuals who have incurred a TBI. The first model included the OTBM as a
single predictor variable in addition to the constant. The second model was identical to the first
with the addition of the Judgment of Line Orientation T-score as a second predictor variable,
although the addition of this variable resulted in a nonsignificant χ2 change score as illustrated in
Table 8. The model was named Model 1 due to the fact that it was created to accomplish Goal 1.
The initial plan for analysis was that model creation would be halted at this point. However, due
to the exploratory nature of this study, model creation was continued. This resulted in the
creation of three additional hierarchical regression models, as shown in Table 9. The first
included the dominant hand T-score from the finger tapping test, and the second included the
Ward 7-subtest WAIS-III PIQ score. The last model omitted the OTBM due to low statistical
significance following the inclusion of additional predictor variables. The final model was
named Model 1b so that it could be distinguished from Model 1 during subsequent comparisons.
It included three variables in addition to the constant: the Ward 7-subtest PIQ, the Judgment of
Line Orientation T-score, and the Finger Tapping Test T-score from the dominant hand. Model
1b was considered the more robust model due to its inclusion of additional predictor variables
and its significant chi-square change score when compared to Model 1, and it was therefore used
in all subsequent analyses.
There appears to be consistency between the predictor variables that were retained in
Model 1b. The Ward 7-subtest PIQ is a measure of general cognitive functioning that primarily
EMPLOYMENT STATUS FOLLOWING TBI 110
quantifies what the WAIS-III refers to as perceptual organization and processing speed (The
Psychological Corporation, 2002). The Judgment of Line Orientation T-score is another
instrument that is thought to provide a measure of perceptual organization, similar to the
measures that contribute to the PIQ score. The final composition of Model 1b suggests that
measures of processing speed and perceptual organization accounted for a significant amount of
variance in employment status when compared to the other cognitive domains that were
considered during model creation. Even the OTBM, which represents a mean score for all
neuropsychological measures administered, and which can therefore be considered a more global
measurement of cognitive functioning than the PIQ, was removed from Model 1b after the PIQ,
Finger Tapping, and Judgment of Line Orientation scores were retained. This suggests that
scores from instruments that measure domains other than processing speed, perceptual
organization, and motor performance did not significantly account for variance when predicting
employment status.
It is notable that neither Model 1 nor Model 1b retained any measures of memory
performance as significant predictors of employment status following TBI, even though memory
problems are the most often cited and most frequently researched consequence of brain injury
(Vakil, 2005). The amount of time since injury also failed to account for a significant amount of
variance in employment status following TBI in both Model 1 and Model 1b. This suggests that
for participants in Group 1, cognitive performance was more strongly associated with returning
to work than was the amount of time since injury.
Also of note is the R2 value for Model 1b, which remained modest at 0.28. This means
that despite the consideration of neuropsychological test performance and time since injury,
roughly 70% of the variance in employment status in Group 1 was not accounted for. This
EMPLOYMENT STATUS FOLLOWING TBI 111
finding indicates a wide range of factors not accounted for by Model 1b that influenced the
employment status of participants in Group 1. It is difficult to compare the observed R2 value for
Model 1b with previously existing models, as R2 values have not been consistently reported in
the literature.
Sensitivity, Specificity, and Predictive Efficacy of a Parsimonious Model
Model 1b showed an 83.3% correct classification rate, 93.8% sensitivity, and 61.3%
specificity in Group 1. It also showed positive and negative predictive values of 83.6% and
82.6%, respectively (see Table 12).
The correct classification rate obtained in this study is slightly higher than previous
models that have used neuropsychological assessment and demographic variables to predict
employment outcomes following TBI. Prior studies obtained correct classification rates of
between 65% and 77% (Drake et al., 2000; Fleming et al., 1999; Guerin et al., 2006; Kreutzer et
al., 2003; MacMillan et al., 2002; Simpson & Schmitter-Edgecombe, 2002). While differences in
patient samples and assessment methods make it difficult to make direct comparisons between
studies and the predictor models examined, results from this study suggest that estimates of
employment status following TBI made using measures taken from a standard administration of
the MNB result in higher correct classification rates than those achieved by previous models.
The higher correct classification rates found in this study are important given the fact that the
average MNB administration takes approximately two and a half hours, making it a more
parsimonious instrument than most traditional neuropsychological assessment batteries, which
can last an average of 8 hours (Rabin, Barr, & Burton, 2005). The use of a battery lasting less
than half of the time spent on an average neuropsychological assessment could increase access to
EMPLOYMENT STATUS FOLLOWING TBI 112
services and reduce overall cost to the patient without sacrificing the accuracy of model
predictions of employment status following a TBI.
The calculated sensitivity and specificity of Model 1b demonstrated an overall
classification rate that favored the correct identification of participants who were employed at the
time of neuropsychological assessment. As shown in Table 12, the sensitivity was found to be
93.8%, while the specificity of Model 1b was much lower, correctly identifying only 61.3% of
unemployed participants as unemployed. Part of the discrepancy between the sensitivity and
specificity of Model 1b could be due to the demographic composition of Group 1. Almost 70%
of the participants in Group 1 were employed at the time of neuropsychological assessment. In
such a case, a model that favored the correct classification of participants as employed, would
achieve a higher overall correct classification rate.
The positive and negative predictive values for Model 1 in Group 1 were nearly equal at
83.6% and 82.6%, respectively. This means that out of 100 people that the model identified as
employed, 83 of them actually were employed. Similarly, out of 100 people that the model
identified as unemployed, 82 of them actually were unemployed. In other words, there is
approximately a 20% chance that each model prediction is incorrect.
The clinical implications of these values vary by individual, and are influenced partly by
the potential risk of harm in resuming a particular type of employment prior to the return of
premorbid neuropsychological functioning. For instance, it may create a safety risk to return a
patient to work prematurely if they operate heavy equipment or work in a hazardous
environment. In other cases, it may be detrimental to an individual’s career to wait too long
before returning to work due to time missed or experience lost.
EMPLOYMENT STATUS FOLLOWING TBI 113
The Addition of Demographic Predictors
Model 2 was created in order to determine the extent to which demographic variables
could account for variance in employment status in Group 1 beyond that already accounted for
by the neuropsychological assessment measures included in Model 1b. The model was created
by adding the identified demographic variables into a second block of analysis after the
calculation of Model 1b was complete. The new model retained only two demographic variables:
self-reported independent driving status and premorbid occupation (see Table 10). As described
in the Results section above, χ2 goodness of fit calculations showed that the addition of these two
demographic variables added significantly to the amount of variance in employment status
accounted for by Model 1b, from an R2 value of 0.28 to an R2 value of 0.36. Model 2 had only a
slightly higher correct classification rate than Model 1b at 86.5% compared to 83.3%. Model 2
also showed increases in sensitivity, specificity, positive predictive power, and negative
predictive power when compared to Model 1b. While the increases in statistical performance
described above are modest, the inclusion of these measures in predicting employment status
following a TBI is supported by the ease of collecting such information.
The fact that independent driving status was retained as one of two demographic
variables in Model 2 supports the use of this variable as a clinically useful indication of good
overall adjustment and occupational functioning following TBI. This could simply be due to the
fact that individuals who are driving independently are more able to deliver résumés, attend
interviews, and secure transportation to and from work. However, Rapport, Hanks, and Bryer
(2006) found that the amount of social support provided by significant others accounted for more
variance in post-TBI driving status than injury severity, negative affectivity, overall level of
social support, and use of public transportation. These findings suggest that in the current study,
EMPLOYMENT STATUS FOLLOWING TBI 114
each participant’s independent driving status may be directly related to the amount of social
support they receive from significant others, a variable which was not included in the current
analysis.
An alternate explanation for the retention of independent driving status as a predictor
variable in Model 2 is that driving involves many cognitive functions, such as the maintenance of
divided attention, route planning, muscle coordination, and rapid information processing, all of
which may be necessary to secure and retain successful employment. This explanation is
supported by a follow-up study by Rapport, Bryer, and Hanks (2008) who found that individuals
who were not driving following a TBI produced significantly lower scores on a short
neuropsychological battery than those who were driving following a TBI. The relationship
between driving following a TBI and good cognitive functioning as measured by
neuropsychological assessment has also been highlighted in other studies (D’apolito,
Massonneau, Paillat, & Azouvi, 2013; Labbe, Vance, Wadley, & Novack, 2014). When
combined with the research findings described above, the current study supports the continued
use of independent driving status as a good measure of neuropsychological functioning
following TBI.
Premorbid occupation was retained as an additional predictor variable in Model 2. As
stated above in Data Reduction, the variable was defined categorically, and each participant’s
premorbid occupation was labeled as unemployed, semi-skilled, or skilled. The odds ratios for the
categories semi-skilled and skilled represent the increase in odds that a participant who held
semi-skilled or skilled premorbid employment was employed at the time of neuropsychological
assessment, when compared to participants who were unemployed prior to TBI. The observed
odds ratios for the categories semi-skilled and skilled were very similar, at 12.19 and 12.01
EMPLOYMENT STATUS FOLLOWING TBI 115
respectively (see Table 10). An odds ratio represents the odds that a given outcome will occur
when a specific criterion is met, compared to the odds of the same outcome will occur if the
criterion is not met. In the case of the current example, the odds of being employed at the time of
neuropsychological assessment were 12 times greater for participants who held semi-skilled or
skilled employment prior to sustaining a TBI than for participants who were not employed prior
to their injury. Given the fact that the odds ratios are similar for both premorbid employment
categories, it is reasonable to conclude that those participants in Group 1 who were working
before sustaining a TBI are more likely to be working after sustaining a TBI, regardless of the
type of employment they held. While this result seems intuitive, it supports the idea that the
occupational impact of sustaining a TBI may be more severe for unemployed individuals than it
is for those who are employed at the time of injury. The retention of premorbid employment as a
predictor variable in Model 2 may also suggest the presence of numerous premorbid, mediating
variables that impact an individual’s ability to secure employment both before and after
sustaining a TBI.
The Additive Value of Neuropsychological Variables
Model 3 was created in order to determine the extent to which neuropsychological
measures could account for variance in employment status in Group 1 beyond that accounted for
by demographic variables alone. The importance of this model lies in the overall cost, both in
time and money, of a full neuropsychological assessment and the relative ease of collecting
demographic information. If employment status can be predicted equally well using demographic
information or neuropsychological measures, then the collection of demographic information
would be a cost-effective alternative to neuropsychological assessment when predicting
employment status (Yates & Taub, 2003).
EMPLOYMENT STATUS FOLLOWING TBI 116
The model was created by entering variables in two separate blocks, with demographic
variables entered before neuropsychological assessment variables. When analysis was complete,
the model retained the same demographic and neuropsychological variables as Model 2 with the
addition of education as a demographic variable and the COWAT T-score as a
neuropsychological assessment variable, although neither of them reached statistical significance
at the p = 0.05 level (see Table 11). Hosmer and Lemshow tests showed a poor fit for the model
(p = 0.01). These results suggest that model predictions may have been unevenly distributed
when compared to observed employment status in Group 1. However, any unevenness in the
distribution of model predictions was considered to have minimal effect on model performance
due to observed R2 values that were comparable to other models created during this study.
As described in the Results section above, χ2 goodness of fit calculations showed that the
addition of neuropsychological assessment variables added significantly to the amount of
variance in employment status accounted for by the demographic variables alone. Together with
the finding that demographic variables contributed significantly to the amount of variance
accounted for by neuropsychological assessment variables alone during the creation of Model 2,
these results demonstrate that there is statistically significant benefit in using both demographic
and neuropsychological assessment variables together when predicting employment outcomes
following TBI and that any model wishing to predict employment status following TBI should
consider both variable categories.
Comparisons between Models 1b, 2, and 3 show that Model 3 had a slightly higher
correct classification rate than Models 1b and 2 at 88.5% compared to 83.3% for Model 1b and
86.5% for Model 2. The R2 value for Model 3 (R2 = 0.40) was slightly higher than the R2 values
for Models 1b (R2 = 0.28) and 2 (R2 = 0.36). Model 3 also showed increases in sensitivity,
EMPLOYMENT STATUS FOLLOWING TBI 117
positive predictive power, and negative predictive power when compared to both Models 1b and
2. Most notably, Model 3 demonstrated 98.5% sensitivity in Group 1 while maintaining 67.7%
specificity. This represents a 3.1% increase in sensitivity with no change in specificity when
compared to Model 2, and thus better overall predictive accuracy. For a complete explanation of
the performance of Model 3 in Group 1, see Table 12.
The increase in R2 value, correct classification rate, sensitivity, positive predictive value,
and negative predictive value seen in Model 3 when compared to Models 1b and 2 is most likely
due to the addition of two new predictor variables: education and the COWAT T-score. The
variable education showed an odds ratio of 1.44, indicating that for participants in Group 1, the
odds of being employed at the time of neuropsychological assessment increased by 0.44 with
each additional year of formal education. This effect may be related to the idea of cognitive
reserve, the idea that individuals whose brains process information more efficiently may be
better able to compensate for small deficits in cognitive performance (Stern, 2002). It has been
proposed that increasing levels of education may result in more efficient information processing
and thus a larger cognitive reserve, subsequently reducing the amount of time needed to recover
from a TBI (Schneider et al., 2014). In this case, participants who were more highly educated
would have had more ease in locating employment following a TBI due to the fact that minor
cognitive problems would result in less observable change in their workplace performance. Such
a relationship has been demonstrated by Levi, Rassovsky, Agranov, Sela-Kaufman, and Vakil
(2013), who used structural equation modeling to create a three-factor model of cognitive reserve
in victims of TBI that identified intelligence, socioeconomic status, and involvement in leisure
activities as significant factors.
EMPLOYMENT STATUS FOLLOWING TBI 118
An alternate explanation for the inclusion of education as a predictor variable in Model 3
can be found in the previously-cited study conducted by Machamer, Temkin, Fraser, Doctor, and
Dikmen (2005), who found that in their sample, older participants were more likely to return to
stable employment following a TBI than younger participants. The authors concluded that this
was due to the fact that older individuals may be more established in their careers and therefore
more highly valued by their employers, who would in turn be willing to let them return to
employment following an injury. Similarly, individuals with more work experience may possess
skill sets that are less easily replaced and would therefore be more highly encouraged to return to
work following an injury. These observations could also explain the relationship between
education and employment status in Model 3, due to the fact that a participant with more formal
education may be more likely to possess valued skill sets, qualifications, or degrees, which
would in turn make him or her less easily replaced by employers and more competitive for
renewed employment following a TBI.
Also included in Model 3 as a predictor variable was the COWAT T-score. The exact
reason why this variable would be retained in Model 3 but not in Model 2 is unclear, as all other
neuropsychological assessment predictor variables are identical in both models. However, the
fact that it was retained in Model 3 supports earlier claims that the test is a sensitive measure of
brain dysfunction and may be significantly correlated with functional impairment (Lezak,
Howieson, and Loring, 2004).
An Investigation of Model Performance
Proportion of variance accounted for. Decreased model performance was expected
when the regression models created in Group 1 were confirmed in Group 2. Confirmatory and
cross-validation analyses of models routinely results in decreased performance due to the fact
EMPLOYMENT STATUS FOLLOWING TBI 119
that initial data analyses take advantage of chance correlations that are less likely to appear in a
separate sample. As expected, the R2 values for Models 1b, 2, and 3 were consistently higher in
Group 1 than in Group 2. For each model, the between-groups difference in R2 values was
approximately 0.10 (actual differences ranged from 0.08 to 0.11; see table 12). This suggests that
group membership accounted for approximately 10% of the variance in employment status
accounted for by each of the above-mentioned models. It also suggests that when applied to
independent samples, the demographic and neuropsychological variables retained by Models 1b,
2, and 3 may account for less variance in employment status than they did for participants in
Group 1.
Percentage of cases correctly classified. As illustrated in Table 12, Models 1b, 2, and 3
showed lower percentages of cases correctly classified in Group 2 than in Group 1. When
comparing model performance in Group 1 and Group 2, the between-group differences in the
percentage of cases correctly classified were similar for all three models, ranging from 13.5% to
16.6%. Model 3, which demonstrated the highest correct classification rate in Group 1, showed
the largest difference and no longer demonstrated the highest correct classification rate in Group
2. This could be due in part to the fact that Model 3 retained three additional non-significant
predictor variables that were not included in Model 2. Given the fact that these variables were
non-significant in Group 1, they may have accounted for even less variance in employment
status in Group 2 and therefore had a negative impact on the accuracy of predictions.
From a clinical perspective, the between-group differences described above may be
important, as they represent the incorrect classification of approximately 13 to 16 additional
individuals when Models 1b, 2, and 3 were applied to an independent sample. These results are
consistent with the finding that R2 values were consistently lower in Group 2, and support the
EMPLOYMENT STATUS FOLLOWING TBI 120
idea that the statistical performance of Models 1b, 2, and 3 is noticeably impacted by group
membership. However, the correct classification rates of all three models in Group 2 were still
comparable to the correct classification rates of previous models (Drake et al., 2000; Fleming et
al., 1999; Guerin et al., 2006; Kreutzer et al., 2003; MacMillan et al., 2002; Simpson &
Schmitter-Edgecombe, 2002).
Sensitivity. Table 12 shows that the sensitivity of Models 1b, 2, and 3 was lower in
Group 2 than it had been in Group 1. The sensitivity of Models 1b, 2, and 3 in Group 2 ranged
from 83.9% to 87.5%. This means that approximately 84% to 88% of participants in Group 2
who were working at the time of neuropsychological assessment were correctly identified as
such by the 3 models. These outcomes suggest that despite lower sensitivity when applied to an
independent sample, all three models still retain clinical utility when trying to determine whether
or not a particular patient is ready to return to work. While Model 2 showed the highest
sensitivity in Group 2, any between-model differences are considered to be negligible.
Specificity. Table 12 shows that the specificity of Models 1b, 2, and 3 was lower in
Group 2 than it had been in Group 1. On average, the between-group differences in specificity
for each model were greater than the between-group differences in sensitivity. The specificity of
Models 1b, 2, and 3 in Group 2 ranged from 47.5% to 55.0%, indicating that in some cases the
models correctly identified less than half of the participants in Group 2 who were not working at
the time of neuropsychological assessment. However, as described above under percentage of
cases correctly classified, overall predictive accuracy was comparable to previous prediction
models, thus demonstrating the continued utility of Models 1b, 2, and 3 in Group 2.
Positive Predictive Value. The positive predictive value of Models 1b, 2, and 3 in Group
2 ranged from 69.6% to 72.3%, with Model 1b demonstrating the lowest positive predictive
EMPLOYMENT STATUS FOLLOWING TBI 121
value and model 3 demonstrating the highest. This means that of all the participants identified by
each model as being employed at the time of neuropsychological assessment, approximately
70% of them were actually employed at that time. The clinical utility of a positive predictive
value of approximately 70% will depend on the occupational setting, and the specific risk
involved in returning an individual to work too early or too late.
Subjective between group comparisons show that these values are lower in Group 2 than
they had been in Group 1, where the models demonstrated positive predictive values ranging
from 83.6% to 86.5%.
Negative Predictive Value. The negative predictive value of Models 1b, 2, and 3 in
Group 2 ranged from 70.4% to 75.0%, with Model 1b showing the lowest negative predictive
power and Model 2 showing the highest. This means that of all the participants in Group 2 that
were classified as unemployed at the time of neuropsychological assessment, between 70% and
75% of them were actually unemployed at that time, depending on the specific prediction model
used. As was the case with positive predictive value, the clinical utility of the observed negative
predictive power will depend on the specific setting in which each individual is employed.
Subjective between group comparisons show that the negative predictive value of each
model was lower in Group 2 than it had been in Group 1.
In general, between-group comparisons showed that Models 1b, 2, and 3 all demonstrated
reduced performance in Group 2 on all identified performance outcome measures. Any attempt
to generalize the findings of this study to other populations should take into consideration the
fact that model performance is reduced in independent samples. However, in most cases these
differences were not large enough to preclude clinical utility.
EMPLOYMENT STATUS FOLLOWING TBI 122
The Relationship between Litigation Status and Model Performance
Upon initial examination, the performance of Models 1b, 2, and 3 appears to be lower in
litigants than in non-litigants (see Table 13). As a general rule, all three models demonstrated
lower performance on identified outcome measures in Group 3 than they had in Group 4.
However, closer inspection shows that the sensitivity and negative predictive power of all three
models was higher in the group of litigants than in the group of non-litigants. This observation is
likely the result of a low unemployment rate in Group 3, in which only 9% of the sample was
unemployed at the time of neuropsychological assessment. As explained above, all three models
demonstrated moderate levels of specificity in Groups 1 and 2. However, in the group of
litigants, which consisted of only 7 participants who were unemployed as opposed to 69
participants who were employed at the time of assessment, the few participants who were
classified as unemployed would represent almost the entire cohort of unemployed participants.
Despite any differences in model performance, overall correct classification rates in both
Groups 3 and 4 were comparable to the correct classification rates reported in earlier studies.
Also of note is the fact that the R2 values for Models 1b, 2, and 3 were nearly identical in Groups
3 and 4, with an average between-groups difference of 0.02. This suggests that there is no
clinically significant difference in the amount of variance accounted for by the
neuropsychological and demographic variables retained by Models 1b, 2, and 3 when comparing
litigants and non-litigants. The combination of these findings suggests that all three models can
be applied in both populations with subjectively similar results.
Model Creation in a Sample of Litigants
When Models 4 and 5 were created in Group 3, there were several differences in the
predictor variables that were retained when compared to Models 1b, 2, and 3 (see Tables 14 and
EMPLOYMENT STATUS FOLLOWING TBI 123
15). Perhaps most notably, the number of months since injury was retained by both models, and
accounted for a significant (p = 0.04) amount of the variance in employment status in Model 4.
In both models, participants were more likely to be employed when more time had passed since
their injury. The fact that this variable was retained in Models 4 and 5 but not in Models 1b, 2, or
3 suggests that the number of months since injury explains a higher proportion of variance in
employment status for individuals who are involved in litigation at the time of their assessment.
There are several reasons why this difference might appear. One reason is that a more
severe injury would result in more severe cognitive deficits, thus increasing both recovery times
and the chances of being involved in litigation. However, this does not seem to be the case in the
current sample, since the results of between-group MANOVA analysis of neuropsychological
assessment variables did not reach significance (p = 0.76), indicating that litigants in Group 3
and non-litigants in Group 4 were not significantly different on all global measures of cognitive
functioning including the WAIS-III FSIQ, WAIS-III PIQ, WAIS-III VIQ, and OTBM. These
findings indicate that between group differences on the neuropsychological and demographic
variables of interest did not exceed those expected by chance, and that cognitive impairment
following injury was similar in both groups.
Another likely scenario is that the number of months since injury accounts for a
significant amount of variance in employment status in Group 3 as a direct result of the litigation
process itself. Individuals who are involved in litigation resulting from a head injury, and
especially those who are making disability claims, may be suspected of malingering if they
return to work while their cases are being heard. As a result, they would be required to wait until
their case has been settled before returning to work, and individuals assessed soon after their
injury would be less likely to have resumed employment.
EMPLOYMENT STATUS FOLLOWING TBI 124
Another notable way in which Models 4 and 5 differ from Models 1b, 2, and 3 is that the
OTBM, which was not retained in any of the first three models, was highly significant in both
Model 4 (p < 0.01) and Model 5 (p < 0.01). The Rey Complex Figure Test delayed recall trial
was the only other neuropsychological assessment variable that was retained by both Models 4
and 5. This would suggest that while litigants did not necessarily show more severe impairment
in any specific neuropsychological domain, they might have shown a more diffuse pattern of
deficits than those who were not involved in litigation. This finding is interesting in that it
suggests that individuals involved in litigation following a TBI may experience a wider range of
minor deficits that prevent them from returning to work, as opposed to severe deficits in any one
specific neuropsychological domain.
With only one exception, Models 4 and 5 showed lower R2 value, percentage of cases
correctly classified, sensitivity, specificity, positive predictive value, and negative predictive
value in Group 3 than Models 1b, 2, and 3 did in Group 1. This seems to suggest that, given the
demographic and neuropsychological assessment variables used in this study, it is more difficult
to accurately predict the employment status of individuals when they are involved in litigation.
Models 4 and 5 may have both benefited from the inclusion of additional predictor variables that
are more directly related to the litigation itself, such as whether the litigation involved a
disability claim or whether the participant had been assigned public or private legal
representation.
Model Creation Using the Entire Study Sample
When the entire study sample was used to create Models 6 and 7, the variables retained
were very similar to those retained in previous models. Only one variable retained by Model 7,
the Rey Complex Figure Test immediate recall trial T-score, had not been retained by any of the
EMPLOYMENT STATUS FOLLOWING TBI 125
previous models. Although the implications of retaining this variable are unclear, its inclusion
suggests that with larger sample sizes and the resulting increase in statistical power, models may
be more able to identify variables that account for significant amounts of variance in
employment status following TBI. Future research would benefit from the use of larger sample
sizes.
Also notable was the highly significant contribution of the OTBM in both Model 6 (p =
0.02) and Model 7 (p < 0.01). No other neuropsychological assessment variable attained such
high statistical significance in any of the models created in this study. In Step 1 of Model 6, the
inclusion of the OTBM alone resulted in an R2 value of 0.21, nearly equivalent to the R2 value of
Model 1b in Group 1 (R2 = 0.28). This finding demonstrates that with increased sample sizes and
the resulting increase in individual variation across neuropsychological performance, the
importance of global indicators of cognitive performance in the prediction of employment status
is increased. This finding also demonstrates the utility of the OTBM in predicting employment
status following a TBI, and supports its continued use as an outcome measure in TBI research.
The observed utility of global indicators of cognitive performance in predicting
employment status can also help inform treatment recommendations that might increase the
chances of a successful return to work following TBI. Cicerone, Mott, Azulay, Sharlow-Galella,
Ellmo, Paradise, and Friel (2008) tracked the functional outcomes of 68 adults with mild to
severe TBI as they participated in either a traditional neuropsychological rehabilitation program
that focused on the remediation of specific cognitive deficits, or a comprehensive holistic
neuropsychological rehabilitation program that focused on the improvement of general
neuropsychological performance, metacognition, interpersonal functioning, and emotional
regulation. After 16 weeks of treatment, participants from the holistic treatment program showed
EMPLOYMENT STATUS FOLLOWING TBI 126
better community integration and quality of life as measured by the Community Integration
Questionnaire (effect size = 0.59) and the Perceived Quality of Life Scale (effect size = 0.30).
Geurtsen, van Heugten, Martina, and Geurts (2010) reviewed 13 neuropsychological
rehabilitation studies published between 1990 and 2008 and found that comprehensive
neuropsychological rehabilitation programs significantly improved community integration
following severe TBI. This finding was supported by Cicerone and colleagues (2011), who
reviewed 141 cognitive rehabilitation studies published between 2003 and 2008 and found strong
evidence for the use of comprehensive, holistic neuropsychological rehabilitation programs
following mild to severe TBI. Combined with these findings, the current study supports the use
of comprehensive rehabilitation programs that focus on the treatment of a broad range of
neuropsychological and functional deficits following TBI rather than programs which focus on
addressing specific neuropsychological domains.
Interestingly, the percentage of cases correctly classified, sensitivity, specificity, positive
predictive value, and negative predictive value of Models 6 and 7 in the study sample were
consistently lower than the same values for Models 1b, 2, and 3 in Group 1. While the reasons
for this finding are not entirely clear, it may be partly due to the fact that in logistic regression
smaller sample sizes can lead to overly optimistic estimates of model performance (Leeflang,
Moons, Reitsma, & Zwinderman, 2008), partly due to an artificial inflation of odds ratios
(Bohning, Holling, & Patilea, 2010). Therefore, the decreased value of statistical performance
outcomes for Models 6 and 7 when compared to Models 1b, 2, and 3, can be attributed in part to
the increase sample size during model creation.
EMPLOYMENT STATUS FOLLOWING TBI 127
The Difficulty of Predicting Employment Status
The current study examined the degree to which data from a standard MNB
administration correctly predicted the current employment status of individuals who had
previously incurred a TBI. The correct classification rates of the models created in this study
ranged from 78.6% to 88.5%. This is consistent with the predictive efficacy of similar models
from previous research that demonstrated correct classification rates ranging from 65% to 77%
(Drake, Gray, Yoder, Pramuka, & Llewellyn, 2000; Fleming, Tooth, Hassell, & Burchan, 1999;
Guerin, Kennepohl, Leveille, Dominique, & McKerral, 2006; Kreutzer et al., 2003; MacMillan,
Hart, Martelli, & Zasler, 2002; Simpson & Schmitter-Edgecombe, 2002). Thus, we can conclude
that the models created using data from a standard MNB administration were able to correctly
predict employment status as well as models created using a variety of other neuropsychological
measures. However, R2 values of the models created in this study ranged from 0.29 to 0.40. The
fact that only 30% to 40% of variance in employment status was accounted for highlights the
difficulty in predicting employment status following a TBI, and suggests that additional variables
should be considered in order to increase predictive accuracy. In fact, employment is a highly
complex variable that can be influenced by a wide range of individual characteristics and aspects
of the environment that might not be measured by a neuropsychological battery. Skills such as
being able to accurately describe past work experience and being able to accurately evaluate your
own strengths, as well as being able to demonstrate proper social skills and situational judgment
may impact performance during a job interview (Salgado & Moscoso, 2002). Even an
applicant’s accent may affect perceived employability (Rakic, Steffens, & Mummendey, 2011).
These same constructs may also affect the likelihood that an individual will maintain
employment once they have been hired.
EMPLOYMENT STATUS FOLLOWING TBI 128
While many of these skills are related to neuropsychological constructs, a traditional
neuropsychological assessment may lack the ecological validity needed to predict performance
in a situation as complex as a job interview (Chaytor & Schmitter-Edgecombe, 2003). For
example, scores from neuropsychological tests designed to measure verbal fluency would be
expected to share some variance with an individual’s overall communication skills. However, the
ability to perform well on a test of verbal fluency may not be sufficient to succeed in a job
interview, where communication is highly complex and the environment is less controlled and
predictable. Behavioral assessment measures such as the Frontal Systems Behaviour Scale have
been shown to be more effective than traditional neuropsychological assessment in predicting
community integration following TBI (Reid-Arndt, Nehl, & Hinkebein, 2007). The addition of
similar behavioral variables into a model meant to predict employment status may increase
correct classification rates and variance accounted for.
Even a model that accounts for a comprehensive spectrum of behavioral, psychological,
and cognitive performance variables might only have moderate success in predicting
employment status following a TBI. Employment status is affected by a wide range of
environmental variables that are largely unrelated to the individual being tested. These variables
could include the rate at which individuals are entering and exiting the workforce (Shimer,
2012), wage rigidity (Haefke, Sonntag, & van Rens, 2013) the distribution of available jobs
across large and small employers (Moscarini & Postel-Vinay, 2012), the presence of secondary
gain (Harris, Mulford, Solomon, van Gelder, & Young, 2005; Schneider, Bassi, & Ryan, 2011),
and government tax policies (Arnold, Brys, Heady, Johansson, Schwellnus, & Vartia, 2011). All
of these variables would affect an individual’s ability to find work regardless of whether or not
he or she had sustained a TBI and independently of their ability to perform well on a
EMPLOYMENT STATUS FOLLOWING TBI 129
neuropsychological assessment battery. At times when economic conditions are particularly
poor, these environmental factors may have an increased influence on an individual’s ability to
find employment following a TBI due to the fact that the market may become saturated with
highly qualified applicants competing for a limited number of positions. Conversely,
environmental factors may also account for increased variance in employment rates following a
TBI when economic conditions are highly favorable. In such a case, an increase in the number of
jobs available may reduce the effect that of cognitive abilities and limitations have on the ability
to find employment.
Limitations of the Current Study
The current study faced a number of limitations. As is the case with all studies using
archived data sets, the variables included for analysis were limited to those that had been
collected prior to the creation of this research design. While efforts were made to include the
same measures that had been included in previous investigations of employment outcomes
following TBI, this was not always possible. In many cases, neuropsychological instruments
were included which were thought to be conceptually similar to, or which were thought to
provide measures of similar constructs as the instruments used in previous studies. However, no
two neuropsychological tests are identical. Each measures a unique component of the construct
of interest and is associated with unique sources of measurement and inferential error. In the
current study it is difficult to determine the extent to which the use of novel but conceptually
similar assessment instruments affected the performance of the prediction models, or if
predictive efficacy would have improved if different instruments had been used for data
collection. A similar statement can be made for demographic variables, which may have been
EMPLOYMENT STATUS FOLLOWING TBI 130
defined or quantified differently in this study than they had been in prior research due to the
limitations established by data collection methods.
Perhaps the greatest limitation of the current study was the homogeneity of the sample
demographics. The study originally planned to include ethnicity as a demographic variable
containing seven distinct categories. The decision to include ethnicity as a predictor variable had
been made based on previous research showing that an individual’s ethnicity was predictive of
employment outcomes following a TBI even after other demographic and injury-characteristic
variables had been controlled for (Arango-Lasprilla et al., 2009). However, the database sample
used in the current study was predominantly White, leaving the other six planned categories of
ethnicity with insufficient membership to be included in analysis. In an attempt to measure the
impact of ethnic status, six of the categories of ethnicity were combined to create a single ethnic
minority category. However, group membership in the ethnic minority category was still
insufficient for inclusion in analysis and a decision was made to not include the variable in the
final analysis.
While the models created during this study showed higher correct classification rates than
similar models created in previous research studies, the fact that ethnicity was not considered by
any of the final prediction models is considered a major limitation. The results of the current
study can only be considered applicable to a primarily White population, and the extent to which
these results might generalize to other populations is unknown. Any attempt to apply the results
of this study to members of ethnic minority groups should be undertaken with caution, especially
in light of research demonstrating the importance of ethnicity on employment outcomes
following TBI (Arango-Lasprilla, et. al, 2009; Arango-Lasprilla, Ketchum, Lewis, Krch, Gary, &
Dodd, 2011; Forslund, Roe, Arango-Lasprilla, Sigurdardottir, & Andelic, 2013).
EMPLOYMENT STATUS FOLLOWING TBI 131
This study also included two demographic variables that were reduced prior to data
analysis due to low group membership. The variable premorbid occupation was originally
planned to include seven categories. However, due to low membership in some categories, the
variable was reduced to three groups: not employed, partially employed, and fully employed.
While efforts were made to place the original seven categories in the most appropriate group
following the data reduction, there was often no clear distinction between partial employment
and full employment. For instance, the employment category such as student may have included
participants who were attending school full-time, participants who were attending school part-
time, and participants who were working at another job while attending school. While the
employment category student was included in the category fully employed due to the perceived
neuropsychological demands of full-time education, the division between partial employment
and full employment was a subjective one. With a database that included participants from a
greater variety of occupations, employment categories could be more precisely defined. This
could be important due to the fact that not all full-time employment demands the same cognitive
resources, and impairment in a specific neuropsychological domain might be related to poor
employment outcomes for individuals in certain career fields.
The variable independent driving status was also reduced before inclusion in data
analysis. The variable originally contained four categories: not driving, partially not driving,
partially driving, and driving. Due to low group membership, this variable was reduced to two
categories: not driving and driving. The extent to which the inclusion of all four originally
planned categories may have affected the results of this study is unknown. However, the fact that
numerous variables had to be reduced before analysis due to low membership in certain
categories represents a larger problem. Such cases of low expected frequencies can result in low
EMPLOYMENT STATUS FOLLOWING TBI 132
statistical power and inflated odds ratios (Tabachnick & Fidell, 2007), and a more diverse
database sample would have resulted in higher statistical power and more accurate results during
analysis.
A similar limitation to this study exists in the reduction of the outcome variable
employment status. The variable was originally meant to include nine possible categories, seven
of which represented civilian employment categories and two of which represented military
employment categories. For the purposes of the current study, these employment categories were
originally collapsed into three main employment outcome groups based on the estimated
neuropsychological demand of each category. The three employment outcome groups were not
employed, partially employed, and fully employed. However, due to low membership in the
partially employed category, the variable was reduced again to contain only the categories
employed and unemployed. This reduction resulted in the need to change the planned statistical
analysis to a binary regression meant to predict whether or not each participant was employed
following a TBI. However, the change in statistical analyses eliminated the ability to predict
whether or not subjects were employed in the same type of employment that they held before
their injury, or if they had returned to a position that had fewer cognitive demands or potentially
paid a lower salary. In fact, before the variable employment status had been reduced, only one
participant in the study sample qualified for inclusion in the fully employed category, suggesting
that a return to premorbid occupations may be very difficult following a TBI, regardless of
neuropsychological functioning. However, due to sample limitations, conclusive statements
about the difficulty of returning to full vs. partial employment following TBI are beyond the
scope of the current study. This information would be very useful to the clinician who is making
recommendations about when a patient might be ready to return to work following a head injury
EMPLOYMENT STATUS FOLLOWING TBI 133
and who would like to assist the patient in establishing realistic expectations about what type of
employment they might be able to find.
The reduction of the employment status variable also resulted in the loss of the military
employment outcomes full duty and limited duty. The fact that these categories were not included
limits the ability to apply results from this study to a military population due to unique
considerations when working with military members who have sustained a TBI, including the
presence of secondary gain (French, Anderson-Barnes, Ryan, Zazeckis, & Harvey, 2012).
This study was also limited by the fact that no distinction was made between participants
with mild, moderate, and severe TBI. The decision to collapse all three categories of TBI into a
single group was made in order to maximize group membership in the study sample and thus
increase statistical power. However, the inclusion of mild TBI in the same sample as moderate
and severe TBI could be misleading. In many of the prediction models created during this study,
the time since injury at the time of neuropsychological assessment did not account for a
significant amount of variance in employment status. However, other research has shown that the
time since injury is one of the best predictors of outcomes following a mild TBI, with injury
characteristic variables being more predictive of recovery outcomes in moderate and severe TBI
(Iverson, 2005; McCrea, 2008). It is likely that the inclusion of all three categories of TBI in a
single sample has resulted in the non-inclusion of potentially significant predictor variables.
Employment status models created using data only from participants with a history of mild TBI
may find that the time since injury accounts for a more significant amount of variance in
employment outcomes.
A final limitation of this study involves the composition of the comparison groups used to
test the models created during the initial statistical analyses. The comparison groups were created
EMPLOYMENT STATUS FOLLOWING TBI 134
by stratifying the database sample by age and randomly assigning participants into one of two
groups. This was done in order to demonstrate the external validity and generalizability of the
models created within the first group. However, inferences from the comparisons are limited in
that the comparison groups were taken from the same original study sample and therefore to not
represent truly independent groups. It is expected that model performance in a truly independent
sample would be lower than was demonstrated in this study, especially considering the
limitations presented by the composition of this study’s sample as described above.
Directions for Future Research
The prediction of employment outcomes following a TBI is a topic that is likely to be of
continued interest in the future. The results and limitations described above suggest several
directions for future research. The current study provided valuable insight into the variables that
account for a significant amount of variance in employment status following a TBI, especially
broad measures of neuropsychological performance such as the WAIS-III index scales and the
OTBM. It also demonstrates the utility of data collected from a standard administration of the
MNB in making predictions about an individual’s employment status following a TBI. However,
the applicability of these results to ethnic minorities and members of the military is limited.
Future research should be conducted to include data from a higher percentage of participants
representing ethnic minorities as well as active duty service members. Such research would
provide valuable information about demographic and neuropsychological variables that
consistently account for variance in employment status following a TBI and those variables that
account for variance only in certain populations. It would also provide data regarding the utility
of a standard MNB administration in predicting employment status following TBI across
multiple populations.
EMPLOYMENT STATUS FOLLOWING TBI 135
Future research would also benefit from the independent investigation of mild TBI as
opposed to moderate and severe TBI. Given previous findings that injury characteristics are
predictive of recovery outcomes mostly in moderate and severe TBI (McCrea, 2008), it is likely
that the separate analysis of data from individuals with mild TBI would provide a more accurate
description of variables that are related to employment status following a mild head injury.
The current research study utilized predictor variables that were the same as, or
conceptually similar to, predictor variables that had been used in previous research studies. This
was done in order to maintain a strong empirical basis for analysis and to facilitate comparison
between the MNB and other, previously investigated, flexible neuropsychological batteries in the
prediction of employment status following TBI. However, the results of this analysis showed the
utility of large, summative predictor variables such as the WAIS-III index scores and the OTBM
in predicting employment outcomes. Future research could combine scores on individual
measures to create large, aggregate variables representing specific categories of
neuropsychological functioning. For instance, z-score composites or mean T-scores could be
calculated using scores from all neuropsychological tests measuring attention to create an
aggregated attention score, which could then be entered as a predictor variable in logistic
regression analyses. The use of such aggregate measures would increase statistical power and
may increase the accuracy of model predictions by allowing the consideration of data from
multiple instruments in a single predictor variable.
Currently, research attempting to predict employment outcomes following TBI has not
completely recognized the complexity of employment status. While many of these studies
recognize the contribution of demographic and neuropsychological variables that may be related
to employment outcomes following a head injury, other potentially significant variables such as
EMPLOYMENT STATUS FOLLOWING TBI 136
personality style and interpersonal skills have been largely ignored. An argument can be made
that such long-standing character traits would have existed prior to the injury and should
therefore not impact employment status following an injury any more than they did prior to the
injury. However, it is also known that changes in personality characteristics are associated with
injuries to the frontal lobes, and that the frontal lobes are particularly vulnerable to injury in TBI
(Zappala, Thiebaut de Schotten, & Eslinger, 2012). Prediction models that account for
personality traits may be better able to measure the impact of frontal lobe injury on employment
status. Future research would benefit from the inclusion of personality measures and measures of
interpersonal functioning.
The available research on employment outcomes following TBI is also limited in that it
ignores external factors that may impact a person’s ability to find employment following a
serious injury. These external factors could include economic factors such as unemployment
rates and the strength of local and national economies. Lundqvist and Samuelsson (2012)
referred to these external factors as “society factors” (p. 13), and make the point that favorable
society factors must be in place in order for a successful return to work to take place. Future
research would benefit from the inclusion of variables meant to provide measures of these
society factors, such as the general unemployment rate at the time of neurological assessment or
an appropriate measure of the strength of local and national economies. The inclusion of such
variables may help account for additional variance in employment status, increase the accuracy
of model predictions, and improve our understanding of variables that account for variance in
employment outcomes following TBI.
Given the current growth of scientific knowledge and general awareness regarding TBI, it
is likely that the prediction of outcomes following a brain injury will remain highly relevant for
EMPLOYMENT STATUS FOLLOWING TBI 137
many years to come. The continuation of research regarding employment outcomes following
TBI will prove to be vital to our understanding of individual functioning and quality of life
following an injury, and will improve our ability to provide quality care for individuals who have
experienced a TBI.
EMPLOYMENT STATUS FOLLOWING TBI 138
References
Acker, M. B., & Davis, J. R. (1989). Psychology test scores associated with late outcome in head
injury. Neuropsychology, 3(3), 123-133.
Adams, R. L., & Trenton, S. L. (1981). Development of a paper-and-pen form of the Halstead
Category Test. Journal of Consulting and Clinical Psychology, 49(2), 298-299. doi:
10.1037/0022-006X.49.2.298
Alvarez, J. A., & Emory, E. (2006). Executive function and the frontal lobes: A Meta-analytic
review. Neuropsychology Review, 16(1), 17-42. doi: 10.1007/s11065-006-9002-x
Andelic, N., Stevens, L. F., Sigurdardottir, S., Arango-Lasprilla, J. C., & Roe, C. (2012).
Associations between disability and employment 1 year after traumatic brain injury in a
working age population. Brain Injury, 26(3), 261-269. doi:
10.3109/02699052.2012.654589
Arango-Lasprilla, J. C., Ketchum, J. M., Gary, K. W., Kreutzer, J. S., O’Neil-Pirozzi, T. M.,
Wehman, P., … Jha, A. (2009). The influence of minority status on job stability after
traumatic brain injury. PM&R: The Journal of Injury, Function, and Rehabilitation, 1(1),
41-49. doi: 10.1016/j.pmrj.2008.07.001
Arango-Lasprilla, J. C., Ketchum, J. M., Lewis, A. N., Krch, D., Gary, K. W., & Dodd, B. A.
(2011). Racial and Ethnic Disparities in Employment Outcomes for Persons With
Traumatic Brain Injury: A Longitudinal Investigation 1-5 Years After Injury. PM&R,
3(12), 1083-1091. doi: 10.1016/j.pmrj.2011.05.023
Arnold, J. M., Brys, B., Heady, C., Johansson, A., Schwellnus, C., & Vartia, L. (2011). Tax
Policy for Economic Recovery and Growth. The Economic Journal, 121(550), F59-F80.
doi: 10.1111/j.1468-0297.2010.02415.x
EMPLOYMENT STATUS FOLLOWING TBI 139
Arnould, A., Rochat, L., Azouvi, P., & Van der Linden, M. (2013). A Multidimensional
Approach to Apathy after Traumatic Brain Injury. Neuropsychology Review, 23(3), 210 –
233. doi: 10.1007/s11065-013-9236-3
Artman, L. K., & McMahon, B. T. (2013). Functional limitations in TBI and their relationship to
job maintenance following work re-entry. Journal of Vocational Rehabilitation, 39(1),
13-21. doi: 10.3233/JVR-130638
Belanger, H. G., Curtiss, G., Demery, J.A., Lebowitz, B.K., & Vanderploeg, R.D. (2005).
Factors moderating neuropsychological outcomes following mild traumatic brain injury:
a meta-analysis. Journal of the International Neuropsychological Society, 11(3), 215-
227. doi: 10.1017/S1355617705050277
Benton, A., Hamsher, K., Varney N., & Spreen, O. (1983). Contributions to Neuropsychological
Assessment: A Clinical Manual. New York: Oxford University Press.
Benton, A. L., & Hamsher, K. deS. (1989). Multilingual Aphasia Examination (3rd ed.). Iowa
City, IA: AJA.
Benton, A. L., Van Allen, M. W., & Fogel, M. L. (1964). Temporal orientation in cerebral
disease. Journal of Nervous and Mental Disease, 139(2), 110-119. doi:
10.1097/00005053-196408000-00003
Benton, A. L., Varney, N. R., & Hamsher, K. D. (1978). Visuospatial judgment: A clinical test.
Archives of Neurology, 35(6), 364-367.
Bigler, E. D., Farrer, T. J., Pertab, J. L., James, K., Petrie, J., & Hedges, D. W. (2013).
Reaffirmed Limitations of Meta-Analytic Methods in the Study of Mild Traumatic Brain
EMPLOYMENT STATUS FOLLOWING TBI 140
Injury: A Response to Rohling et al. The Clinical Neuropsychologist, 27(2), 176-214. doi:
10.1080/13854046.2012.693950
Binder, L. M., & Rohling, M. L. (1996). Money matters: A meta-analytic review of the effects of
financial incentives on recovery after closed-head injury. American Journal of
Psychiatry, 153(1), 7-10. Retrieved from: http://ajp.psychiatryonline.org/index.dtl
Bohning, D., Holling, H., & Patilea, V. (2010). A limitation of the diagnostic-odds ratio in
determining an optimal cut-off value for a continuous diagnostic test. Statistical Methods
in Medical Research, 20(5), 541-550. doi: 10.1177/0962280210374532
Bornstein, R. A., Baker, G. B., & Douglas, A. B. (1987). Short-term retest reliability of the
Halstead-Reitan Battery in a normal sample. Journal of Nervous and Mental Disease,
175(4), 229-232. doi: 10.1097/00005053-198704000-00007
Buschke, H. (1973). Selective reminding for analysis of memory and learning. Journal of Verbal
Learning and Verbal Behavior, 12(5), 543-550. doi: 10.1016/S0022-5371(73)80034-9
Cattelani, R., Tanzi, F., Lombardi, F., & Mazzucchi, A. (2002). Competitive re-employment
after severe traumatic brain injury: clinical, cognitive, and behavioural predictive
variables. Brain Injury, 16(1), 51-64. doi:10.1080/02699050110088821
Chamelian, L., & Feinstein, A. (2004). Outcome after mild to moderate traumatic brain injury:
The role of dizziness. Archives of Physical Medicine and Rehabilitation, 85(10), 1662-
1666. doi:10.1016/j.apmr.2004.02.012
Chaytor, N., & Schmitter-Edgecombe, M. S. (2003). The Ecological Validity of
Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills.
Neuropsychology Review, 13(4), 181-197. doi: 10.1023/B:NERV.0000009483.91468.fb
EMPLOYMENT STATUS FOLLOWING TBI 141
Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Frass, M.,...Ashman,
T. (2011). Evidence-Based Cognitive Rehabilitation: Updated Review of the Literature
From 2003 Through 2008. Archives of Physical Medicine and Rehabilitation, 92(4), 519-
530. doi: 10.1016/j.apmr.2010.11.015
Cicerone, K. D., Mott, T., Azulay, J., Sharlow-Galella, M. A., Ellmo, W. J., Paradise, S., & Friel,
J. C. (2008). A Randomized Controlled Trail of Holistic Neuropsychologic Rehabilitation
After Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation, 89(12),
2239-2249. doi: 10.1016/j.apmr.2008.06.017
Cifu, D. X., Keyser-Marcus, L., Lopez, E., Wehman, P., Kreutzer, J. S., Englander, J., & Walter,
H. (1997). Acute predictors of successful return to work 1 year after traumatic brain
injury: A multicenter analysis. Archives of Physical Medicine and Rehabilitation, 78(2),
125-131. doi:10.1016/S0003-9993(97)90252-5
Cope, D. N. (1982). Social service data 3 form: Head injury demographic outcome survey. In S.
Berrol (Ed.), Head Injury Rehabilitation Project Final Report (Vol. I, Appendix G, pp. 1-
3). San Jose, CA: Santa Clara Valley Medical Center.
Coronado, V. G., McGuire, L. C., Sarmiento, K., Bell, J., Lionbarger, M. R., Jones, C. D,…Xu,
L. (2012). Trends in Traumatic Brain Injury in the U.S. and the public health response:
1995-2009. Journal of Safety Research, 43(4), 299-307.
D’apolito, A. C., Massonneau, A., Paillat, C., & Azouvi, P. (2013). Impact of brain injury on
driving skills. Annals of Physical and Rehabilitation Medicine, 56(1), 63-80.
Davis, J. J., McHugh, T. S., Axelrod, B. N., & Hanks, R. A. (2012). Performance Validity and
Neuropsychological Outcomes in Litigants and Disability Claimants. The Clinical
Neuropsychologist, 26(5), 850-865. doi: 10.1080/13854046.2012.686631
EMPLOYMENT STATUS FOLLOWING TBI 142
de Guise, E., LeBlanc, J., Tinawi, S., Lamoureux, J., & Feyz, M. (2012). Acute Relationship
between Cognitive and Psychological Symptoms of Patients with Mild Traumatic Brain
Injury. ISRN Rehabilitation, 2012, Article ID 147285. doi: 10.5402/2012/147285
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (1987). California Verbal Learning Test
(Research Edition). San Antonio, TX: Psychological Corporation.
Delis, D. C., Kramer, J. H., Kaplan, K., & Ober, B. A. (2000). California Verbal Learning Test-
Second Edition (CVLT-II). San Antonio, TX: Psychological Corporation.
De Renzi, E., & Vignolo, L. A. (1962). The Token Test: A sensitive test to detect disturbances in
aphasics. Brain, 85, 665-678. doi: 10.1093/brain/85.4.665
Derogatis, L. R. (1994). SCL-R: Symptom Checklist-90-R: Administration, Scoring, and
Procedures Manual (3rd ed.). Minneapolis: National Computer Systems.
Dikmen, S. S., Heaton, R. K., Grant, I., & Tempkin, N. R. (1999). Test-retest reliability and
practice effects of expanded Halstead-Reitan Neuropsychological Test Battery. Journal
of the International Neuropsychological Society, 5(4), 346-356. Retreived from:
http://journals.cambridge.org/action/displayJournal?jid=INS
Dikmen, S. S., Machamer, J. E., Winn, H. R., & Temkin, N. R. (1995). Neuropsychological
outcome at 1-year post head injury. Neuropsychology, 9(1), 80-90. doi: 10.1037/0894-
4105.9.1.80
Drake, A. I., Gray, N., Yoder, S., Pramuka, M., & Llewellyn, M. (2000). Factors predicting
return to work following mild traumatic brain injury: A discriminant analysis. Journal of
Head Trauma Rehabilitiation, 15(5), 1103-1112. Retrieved from: http://journals.lww.com
/headtraumarehab/pages/default.aspx
EMPLOYMENT STATUS FOLLOWING TBI 143
Ekstrom, R. B., French, J. W., Harman, H. H., & Derman, D. (1976). Map Planning Test:
Manual Kit of Factor-Referenced Cognitive Tests. Princeton, NJ: Educational Testing
Service.
Elias, L. J., & Saucier, D. M. (2006). Neuropsychology: Clinical and Experimental Foundations.
Boston, MA: Pearson Education, Inc.
Farbota, K. D., Bendlin, B. B., Alexander, A. L., Rowley, H. A., Dempsey, R. J., & Johnson, S.
C. (2012). Longitudinal diffusion tensor imaging and neurospychological correlates in
traumatic brain injury patients. Frontiers in Human Neuroscience, 6, article 160. doi:
10.3389/fnhum.2012.00160
Faul, M., Wald, M. M., Xu, L., & Coronado, V. G. (2010). Traumatic Brain Injury in the United
States: Emergency Department Visits, Hospitalizations, and Deaths 2002-2006. Atlanta,
GA: Centers for Disease Control and Prevention. Retrieved from: http://www.cdc.gov/tra
umaticbraininjury/tbi_ed.html
Fleming, J., Tooth, L., Hassell, M., & Burchan, W. (1999). Prediction of community integration
and vocational outcome 2-5 years after traumatic brain injury rehabilitation in Australia.
Brain Injury, 13(6), 417-431. doi:10.1080/026990599121476
Forslund, M. V., Roe, C., Arango-Lasprilla, J. C., Sigurdardottir, S. & Andelic, N. (2013).
Impact of personal and environmental factors on employment outcome two years after
moderate-to-severe traumatic brain injury. Journal of Rehabilitation Medicine, 45(8),
801-807. doi: 10.2340/16501977-1168
Fraser, R., Machamer, J., Temkin, N., Dikmen, S., & Doctor, J. (2006). Return to work in
traumatic brain injury (TBI): A perspective on capacity for job complexity. Journal of
EMPLOYMENT STATUS FOLLOWING TBI 144
Vocational Rehabilitation, 25(3), 141-148. Retrieved from: http://iospress.metapress.com
/content/103174/?p=87746e411d9545b9b4d4af586f6e7f98&pi=0
French, L. M., Anderson-Barnes, V., Ryan, L. M., Zazeckis, T. M., & Harvey, S. (2012).
Neuropsychological Practice in the Military. In C. H. Kennedy & E. A. Zillmer (Eds.)
Military Psychology: Clinical and Operational Applications (2nd ed.) New York: The
Guilford Press.
Frencham, K. A. R., Fox, A. M., & Maybery, M. T. (2005). Neuropsychological studies of mild
traumatic brain injury: A meta-analytic review of research since 1995. Journal of Clinical
and Experimental Neuropsychology, 27(3), 334-351. doi: 10.1080/13803390490520328
Gary, K. W., Arango-Lasprilla, J. C., Ketchum, J. M., Kreutzer, J. S., Copolillo, A., Novack, T.
A., & Jha, A. (2009). Racial differences in employment outcome after traumatic brain
injury at 1, 2, and 5 years postinjury. Archives of Physical Medicine and Rehabilitation,
90(10), 1699-1707. doi: doi:10.1016/j.apmr.2009.04.014
Geurtsen, G. J., van Heugten, C. M., Martina, J. D., & Geurts, A. C. H. (2010). Comprehensive
Rehabilitation Programmes in the Chronic Phase after Severe Brain Injury: A Systematic
Review. Journal of Rehabilitation Medicine, 42(2), 97-110. doi: 10.2340/16501977-0508
Grauwmeijer, E., Heijenbrok-Kal, M. H., Haitsma, I. K., & Ribbers, G. M. (2012). A prospective
Study on Employment Outcome 3 Years After Moderate to Severe Traumatic Brain
Injury. Archives of Physical Medicine and Rehabilitation, 93(6), 993-999. doi:
10.1016/j.apmr.2012.01.018
Guerin, F., Kennepohl, S., Leveille, G., Dominique, A., & McKerral, M. (2006). Vocational
outcome indicators in atypically recovering mild TBI: A post-intervention study.
EMPLOYMENT STATUS FOLLOWING TBI 145
Neurorehabilitation, 21(4), 295-303. Retrieved from: http://iospress.metapress.com/conte
nt/103177/?p=e63a4a67a0a74d779d92138ecaefd237&pi=0
Haefke, C., Sonntag, M., & van Rens, T. (2013). Wage rigidity and job creation. Journal of
Monetary Economics, 60(8), 887-899. doi: 10.1016/j.jmoneco.2013.09.003
Han, S. D., Suzuki, H., Drake, A. I., Jak, A. J., Houston, W. S., & Bondi, M. W. (2009). Clinical,
cognitive, and genetic predictors of change in job status following traumatic brain injury
in a military population. Journal of Head Trauma Rehabilitation, 24(1), 57-64. doi:
10.1097/HTR.0b013e3181957055
Hanlon, R. E., Demery, J. A., Martinovich, Z., & Kelly, J. P. (1999). Effects of acute injury
characteristics on neuropsychological status and vocational outcome following mild
traumatic brain injury. Brain Injury, 13(11), 873-887. doi:10.1080/026990599121070
Harris, I., Mulford, J., Solomon, M., van Gelder, J. M., & Young, J. (2005). Association
Between Compensation Status and Outcome After Surgery. JAMA, 293(13), 1644-1652.
doi: 10.1001/jama.293.13.1644
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. L., & Millis, S. R. (2009). American
Academy of Clinical Neuropsychology consensus conference statement on the
neuropsychological assessment of effort, response bias, and malingering. The Clinical
Neuropsychologist, 23(7), 1093-1129. doi: 10.1080/13854040903155063
Henry, J. D., & Crawford, J. R. (2004). A meta-analytic review of verbal fluency performance
following focal cortical lesions. Neuropsychology, 18(2), 284-295. doi: 10.1037/0894-
4105.18.2.284
EMPLOYMENT STATUS FOLLOWING TBI 146
Hoofien, D., Barak, O., Vakil, E., & Gilboa, A. (2005). Symptom Checklist-90 Revised scores in
persons with traumatic brain injury: Affective reactions or neurobehavioral outcomes of
the injury? Applied Neuropsychology, 12(1), 30-39. doi: 10.1207/s15324826an1201_6
Iverson, G. L. (2001). Interpreting change on the WAIS-III/WMS-III in clinical samples.
Archives of Clinical Neuropsychology, 16(2), 183-191. doi: 10.1016/S0887-
6177(00)00060-3
Iverson, G. L. (2005). Outcome from mild traumatic brain injury. Current Opinion in Psychiatry,
18(3), 301-317.
Iverson, G. L. (2010). Mild traumatic brain injury meta-analyses can obscure individual
differences. Brain Injury, 24(10), 1246-1255. doi: 10.3109/02699052.2010.490513
Ivnik, R. J., Sharbrough, F. W., & Laws, E. R. (1988). Anterior temporal lobectomy for the
control of partial complex seizures: Information for counseling patients. Mayo Clinic
Proceedings, 63(8), 783-793.
Johansson, U., & Bernspang, B. (2001). Predicting return to work after brain injury using
occupational therapy assessments. Disability and Rehabilitation, 23(11), 474-480. doi:
10.1080/09638280010010688
Johnstone, B., Vieth, A. Z., Johnson, J. C., & Shaw, J. A. (2000). Recall as a function of single
versus multiple trials: Implications for rehabilitation. Rehabilitation Psychology, 45(1), 3-
19. doi: 10.1037/0090-5550.45.1.3
Jourdan, C., Bosserelle, V., Azerad, S., Ghout, I., Bayen, E., Aegerter, P,…Azouvi, P. (2013).
Predictive factors for the 1-year outcome of a cohort of patients with severe traumatic
brain injury (TBI): Results from the PariS-TBI study. Brain Injury, 27(9), 1000-1007.
doi: 10.3109/02699052.2013.794971
EMPLOYMENT STATUS FOLLOWING TBI 147
Kaplan, E., Goodglass, H., & Weintraub, S. (1983). Boston Naming Test. Philadelphia: Lea &
Febiger.
Kay, T., Harrington, D. E., Adams, R., Anderson, T., Berrol, S., Cicerone, K,…Malec, J. (1993).
Definition of mild traumatic brain injury. Journal of Head Trauma Rehabilitation, 8(3),
86-87.
Kennedy, J. E., Clement, P. F., & Curtis, G. (2003). WAIS-III Processing Speed Index scores
after TBI: The influence of working memory, psychomotor speed and perceptual
processing. The Clinical Neuropsychologist, 17(3), 303-307. doi:
10.1076/clin.17.3.303.18091
Ketchum, J. M., Getachew, M. A., Krch, D., Banos, J. H., Kolakowsky-Hayner, S. A., Lequerica,
A.,…Arango-Lasprilla, J. C. (2012). Early predictors of employment outcomes 1 year
post traumatic brain injury in a population of Hispanic individuals. NeuroRehabilitation,
30(1), 13-22. doi: 10.3233/NRE-2011-0723
Keyser-Marcus L. A., Bricout, J. C., Wehman, P., Campbell, L. R., Cifu, D. X., Englander, J.,…
Zafonte, R. D. (2002). Acute predictors of return to employment after traumatic brain
injury: A longitudinal follow-up. Archives of Physical Medicine and Rehabilitation,
83(5), 635-641. doi:10.1053/apmr.2002.31605
King, N. S., Crawford, S., Wenden, F.J., Moss, N.E.G., Wade, D.T. (1995). The Rivermead Post
Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced
after head injury and its reliability. Journal of Neurology, 242, 587-592.
Klove, H. (1963). Clinical Neuropsychology. In F. M. Forster (Ed.), The medical clinics of North
America. New York: Saunders.
EMPLOYMENT STATUS FOLLOWING TBI 148
Konrad, C., Geburek, A. J., Rist, F., Blumenroth, H., Fischer, B., Husstedt, I.,…Lohmann, H.
(2011). Long-term cognitive and emotional consequences of mild traumatic brain injury.
Psychological Medicine, 41(6), 1197-1211. doi: 10.1017/S0033291710001728
Kozel, J. J., & Meyers, J. E. (1998). A cross-validation study of the Victoria revision of the
Category Test. Archives of Clinical Neuropsychology, 13(3), 327-332. doi:
10.1016/S0887-6177(97)00036-X
Kreutzer, J. S., Marwitz, J. H., Walker, W., Sander, A., Sherer, M., Bogner, J.,…Bushnik, T.
(2003). Moderating factors in return to work and job stability after traumatic brain injury.
The Journal of Head Trauma Rehabilitation, 18(2), 128-138. Retrieved from: http://journ
als.lww.com/headtraumarehab/pages/default.aspx
Krupp, L. B., LaRocca, N. G., Muir-Nash, J., & Steinberg, A. D. (1989). The fatigue severity
scale. Application to patients with multiple sclerosis and systemic lupus erythematosus.
Archives of Neurology, 46(10), 1121-1123. Retrieved from: http://archneur.ama-assn.org/
Labbe, D. R., Vance, D. E., Wadley, V., & Novack, T. A. (2014). Predictors of driving
avoidance and exposure following traumatic brain injury. The Journal of Head Trauma
Rehabilitation, 29(2), 185-192. doi: 10.1097/HTR.0b013e3182795211
Langeluddecke, P. M., & Lucas, S. K. (2003). Wechsler Adult Intelligence Scale – Third Edition
findings in relation to severity of brain injury in litigants. The Clinical
Neuropsychologist, 17(2), 273-284. doi:10.1076/clin.17.2.273.16499
Leeflang, M. M., Moons, K. G., Reitsma, J. B., & Zwinderman, A.H. (2008). Bias in sensitivity
and specificity caused by data-driven selection of optimal cutoff values: mechanisms,
magnitude, and solutions. Clinical Chemistry, 54(4), 729-737. doi:
10.1373/clinchem.2007.096032
EMPLOYMENT STATUS FOLLOWING TBI 149
Lemay, S., Bedard, M. A., Rouleau, I., & Tremblay, P. L. G. (2004). Practice effect and test-
retest reliability of attentional and executive tests in middle-aged to elderly participants.
The Clinical Neuropsychologist, 18(2), 284-302. doi: 10.1080/13854040490501718
Levi, Y., Rassovsky, Y., Agranov, E., Sela-Kaufman, M., & Vakil, E. (2013). Cognitive Reserve
Components as Expressed in Traumatic Brain Injury. Journal of the International
Neuropsychological Society, 19(6), 664-671. doi: 10.1017/S1355617713000192
Levin H. S., High, W., Goethe, K., Sisson, R., Overall, J., Rhoades, H.,…Gary, H. E. (1987).
The neurobehavioral rating scale: Assessment of the behavioral sequelae of head injury
by the clinician. Journal of Neurology, Neurosurgery, and Psychiatry, 50(2), 183–193.
doi: 10.1136/jnnp.50.2.183
Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological Assessment (4th
ed.). New York: Oxford University Press.
Lopez, M. N., Charter, R. A., & Newman, J. R. (2000). Psychometric properties of the Halstead
Category Test. The Clinical Neuropsychologist, 14(2), 157-161. doi: 10.1076/1385-
4046(200005)14:2;1-Z;FT157
Lundqvist, A. & Samuelsson, K. (2012). Return to work after acquired brain injury: A patient
perspective. Brain Injury, 26(13-14), 1574-1585. doi: 10.3109/02699052.2012.698363
Machamer, J., Temkin, N., Fraser, R., Doctor, J. N., & Dikmen, S. (2005). Stability of
employment after traumatic brain injury. Journal of the International Neuropsychological
Society, 11(7), 807-816. doi: 10.1017/S135561770505099X
MacMillan, P. J., Hart, R. P., Martelli, M. F., & Zasler, N. D. (2002). Pre-injury adaptation
following traumatic brain injury. Brain Injury, 16(1), 41-49. doi:10.1080/0269905011008
812
EMPLOYMENT STATUS FOLLOWING TBI 150
Magalhaes, S. S., Malloy-Diniz, L. F., & Hamdan, A. C. (2012). Validity Convergent and
Reliability Test-retest of the Rey Auditory Verbal Learning Test. Clinical
Neuropsychiatry, 9(3), 129-137.
McCrea, M. (2008). Mild Traumatic Brain Injury and Postconcussion Syndrome: The New
Evidence Base for Diagnosis and Treatment. New York: Oxford University Press.
McKay, C., Casey, J. E., Wertheimer, J., & Fichtenberg, N. L. (2007). Reliability and validity of
the RBANS in a traumatic brain-injured sample. Archives of Clinical Neuropsychology,
22(1), 91-98. doi: 10.1016/j.acn.2006.11.00
Mertens, V. B., Gagnon, M., Coulombe, D., & Messier, C. (2006). Exploratory factor analysis of
neuropsychological tests and their relationship to the Brown-Peterson task. Archives of
Clinical Neuropsychology, 21(7), 733-739. doi: 10.1016/j.acn.2006.08.00
Meyers, J., Reinsch-Boothby, L., & Miller, R. (2011). Does the Source of a Forensic Referral
Affect Neuropsychological Test Performance on a Standardized Battery of Tests? The
Clinical Neuropsychologist, 25(3), 477-487. doi: 10.1080/13854046.2011.554442
Meyers, J. E., & Meyers, K. R. (1995). Rey Complex Figure Test and Recognition Trial:
Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc.
Meyers, J. E., & Rohling, M. L. (2004). Validation of the Meyers Short Battery on mild TBI
patients. Archives of Clinical Neuropsychology, 19(5), 637-651. doi: 10.1016/j.acn.2003.
08.007
Meyers, J. E., & Rohling, M. L. (2009). CT and MRI correlations with neuropsychological tests.
Applied Neuropsychology, 16(4), 237-253. doi: 10.1080/09084280903098752
EMPLOYMENT STATUS FOLLOWING TBI 151
Meyers, J. E., & Volbrecht, M. E. (2003). A validation of multiple malingering detection
methods in a large clinical sample. Archives of Clinical Neuropsychology, 18(3), 261-
276. doi: 10.1016/S0887-6177(02)00136-1
Meyers, J. E., Volbrecht, M., Axelrod, B. N., & Reinsch-Boothby, L. (2011). Embedded
Symptom Validity Tests and Overall Neuropsychological Test Performance. Archives of
Clinical Neuropsychology, 26(1), 8-15. doi: 10.1093/arclin/acq083
Meyers, J. E., Volbrecht, M., & Kaster-Bundgaard, J. (1999). Driving is more than pedal
pushing. Applied Neuropsychology, 6(3), 154-164. doi: 10.1207/s15324826an0603_3
Miceli, G., Caltagirone, C., Gainotti, G., Masullo, C., & Silveri, M. C. (1981).
Neuropsychological correlates of localized cerebral lesions in non-aphasic brain-damaged
patients. Journal of Clinical Neuropsychology, 3(1), 53-63. doi:
10.1080/01688638108403113
Miller, L. S., & Rohling, M. L. (2001). A statistical interpretive method for neuropsychological
data. Neuropsychology Review, 11(3), 143-169. doi: 10.1023/A:1016602708066
Mitrushina, M., Boone, K. B., Razani, J., & D’Elia, L. F. (2005). Handbook of Normative Data
for Neuropsychological Assessment. New York: Oxford University Press.
Mitrushina, M., & Satz, P. (1991). Effect of repeated administration of a neuropsychological
battery in the elderly. Journal of Clinical Psychology, 47(6), 790-801. doi: 10.1002/1097-
4679(199111)47:6<790::AID-JCLP2270470610>3.0.CO;2-C
Moscarini, G., & Postel-Vinay, F. (2012). The Contribution of Large and Small Employers to
Job Creation in Times of High and Low Unemployment. The American Economic
Review, 102(6), 2509-2539. doi: http://dx.doi.org/10.1257/aer.102.6.2509
EMPLOYMENT STATUS FOLLOWING TBI 152
Muller, H., Hasse-Sander, I., Horn, R., Helmstaedter, C., & Elger, C. E. (1997). Rey Auditory-
Verbal Learning Test: Structure of a Modified German Version. Journal of Clinical
Psychology, 53(7), 663-671. doi: 10.1002/(SICI)1097-4679(199711)53:7<663::AID-
JCLP4>3.0.CO;2-J
Nakase-Richardson, R., Sherer, M., Barnett, S. D., Yablon, S. A., Evans, C. C., Kretzmer, T,…
Modarres, M. (2013). Prospective Evaluation of the Nature, Course, and Impact of Acute
Sleep Abnormality After Traumatic Brain Injury. Archives of Physical Medicine and
Rehabilitation, 94(5), 875-882. doi: 10.1016/j.apmr.2013.01.001
Norris, M. P., Blankenship-Reuter, L., Snow-Turek, A. L., & Finch, J. (1995). Influence of
depression on verbal fluency performance. Aging and Cognition, 2(3), 206-215. doi:
10.1080/13825589508256598
O’Donnell, J. P., McGregor, L. A., Dabrowski, J. J., Oestreicher, J. M., & Romero, J. J. (1994).
Construct validity of neuropsychological tests of conceptual and attentional abilities.
Journal of Clinical Psychology, 50(4), 596-600. doi: 10.1002/1097-4679(199407)50:4<5
96::AID-JCLP2270500416>3.0.CO;2-S
Ostrosky-Solis, F., Jaime, R. M., & Ardila, A. (1998). Memory abilities during normal aging.
International Journal of Neuroscience, 93(1-2), 151-162.
Ownsworth, T., & McKenna, K. (2004). Investigation of factors related to employment outcome
following traumatic brain injury: A critical review and conceptual model. Disability and
Rehabilitation, 26(13), 765-784. doi:10.1080/09638280410001696700
Parkin, A. J. (1998). The central executive does not exist. Journal of the International
Neuropsychological Society, 4(5), 518-522. Retrieved from: http://www2.psych.ubc.ca/~
pgraf/Psy583Readings/Parkin%201998.pdf
EMPLOYMENT STATUS FOLLOWING TBI 153
Parks, J. K., Diaz-Arrastia, R., Gentilello, L. M., & Shafi, S. (2010). Postinjury employment as a
surrogate for functional outcomes: A quality indicator for trauma systems. Proceedings
(Baylor University Medical Center), 23(4), 355-358. Retrieved from: http://www.baylorh
ealth.edu/Research/Proceedings/Pages/default.aspx
Pertab, J. L., James, K. M., and Bigler, E. D. (2009). Limitations of mild traumatic brain injury
meta-analyses. Brain Injury, 23(6), 498-508. doi: 10.1080/02699050902927984
Pilgrim, B. M., Meyers, J. E., Bayless, J., & Whetstone, M. M. (1999). Validity of the Ward
seven-subtest WAIS-III short form in a neuropsychological population. Applied
Neuropsychology, 6(4), 243-246. doi: 10.1207/s15324826an0604_7
Ponsford, J., Downing, M., Olver, J., Ponsford, M., Acher, R., Carty, M., & Spitz, G. (2014).
Longitudinal follow-up of patients with traumatic brain injury: Outcome at 2, 5, and 10-
years post-injury. Journal of Neurotrauma, 31(1), 64-77. doi: 10.1089/neu.2013.2997
Ponton, M. O., Gonzalez, J. J., Hernandez, I., Herrera, L., & Higareda, I. (2000). Factor analysis
of the Neuropsychological Screening Battery for Hispanics (NeSBHIS). Applied
Neuropsychology, 7(1), 32-39. doi: 10.1207/S15324826AN0701_5
Prigatano, G. P. (1999). Impaired awareness, finger tapping, and rehabilitation outcome after
brain injury. Rehabilitation Psychology, 44(2), 145-159. doi: 10.1037/0090-
5550.44.2.145
Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical
neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA
Division 40 members. Archives of Clinical Neuropsychology, 20(1), 33-65. doi:
10.1016/j.acn.2004.02.005
EMPLOYMENT STATUS FOLLOWING TBI 154
Rakic, T., Steffens, M. C., Mummendey, A. (2011). When it matters how you pronounce it: The
influence of regional accents on job interview outcome. British Journal of Psychology,
102(4), 868-883. doi: 10.1111/j.2044-8295.2011.02051.x
Randolph, C. (1998). RBANS manual: Repeatable Battery for the Assessment of
Neuropsychological Status. San Antonio, TX: The Psychological Corporation.
Rappaport, M., Hall, K. M., Hopkins, K., Belleza, T., & Cope, D. N. (1982). Disability rating
scale for severe head trauma: Coma to community. Archives of Physical Medicine and
Rehabilitation, 63(3), 118-123.
Rapport, L. J., Hanks, R. A., & Bryer, R. C. (2006). Barriers to Driving and Community
Integration After Traumatic Brain Injury. Journal of Head Trauma Rehabilitation, 21(1),
34-44.
Rapport, L. J., Bryer, R. C., & Hanks, R. A. (2008). Driving and Community Integration After
Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation, 89(5), 922-
930. doi: 10.1016/j.apmr.2008.01.009
Raven, J., Raven, J. C., & Court, J. H. (1993). Manual for the Raven’s Progressive Matrices and
Vocabulary Scales. Oxford, England: Oxford Psychologists Press.
Reid-Arndt, S. A., Nehl, C., & Hinkebein, J. (2007). The Frontal Systems Behaviour Scale
(FrSBe) as a predictor of community integration following a traumatic brain injury. Brain
Injury, 21(13-14), 1361-1369. doi: 10.1080/02699050701785062
Reitan, R. M. (1958). Validity of the Trail Making Test as an indicator of organic brain damage.
Perceptual and Motor Skills, 8(3), 271-276. doi: 10.2466/pms.1958.8.3.271
Reitan, R. M., & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery:
Theory and clinical applications (2nd ed.). Tucson, AZ: Neuropsychology Press.
EMPLOYMENT STATUS FOLLOWING TBI 155
Rey, A. (1964). L’examen clinique en psychologie. Paris: Presses Universitaires de France.
Roberts, M. A., Persinger, M. A., Grote, C., Evertowski, L. M., Springer, J. A., Tuten, T.,…
Baglio, C.S. (1994). The Dichotic Word Listening Test: preliminary observations in
American and Canadian samples. Applied Neuropsychology, 1(1-2), 45-56.
Robertson, I. (2008). Traumatic brain injury: Recovery, prediction, and the clinician. Archives of
Physical Medicine and Rehabilitation, 89(12 Suppl. 2), 1-2. doi:10.1016/j.apmr.2008.10.
001
Rohling, M. L., Binder, L. M., Demakis, G. J., Larrabee, G. J., Ploetz, D. M., & Langhinrichsen-
Rohling, J. (2011). A Meta-Analysis of Neuropsychological Outcome After Mild
Traumatic Brain Injury: Re-analyses and Reconsiderations of Binder et al. (1997),
Frencham et al. (2005), and Pertab et al. (2009). The Clinical Neuropsychologist, 25(4),
608-623. doi: 10.1080/13854046.2011.565076
Rohling, M. L., Meyers, J. E., & Millis, S. R. (2003). Neuropsychological impairment following
traumatic brain injury: A dose-response analysis. The Clinical Neuropsychologist, 17(3),
289-302. doi: 10.1076/clin.17.3.289.18086
Ross, S. R., Millis, S. R., & Rosenthal, M. (1997). Neuropsychological prediction of
psychosocial outcome after traumatic brain injury. Applied Neuropsychology, 4(3), 165-
170. doi: 10.1207/s15324826an0403_4
Ross, T. P., Furr, A. E., Carter, S. E., & Weinberg, M. (2006). The psychometric equivalence of
two alternate forms of the Controlled Oral Word Association Test. The Clinical
Neuropsychologist, 20(3), 414-431. doi: 10.1080/13854040590967153
Royan, J., Tombaugh, T. N., Rees, L., & Francis, M. (2004). The Adjusting-Paced Serial
Addition Test (Adjusting-PASAT): Thresholds for speed of information processing as a
EMPLOYMENT STATUS FOLLOWING TBI 156
function of stimulus modality and problem complexity. Archives of Clinical
Neuropsychology, 19(1), 131-143. doi: 10.1016/S0887-6177(02)00216-0
Ruff, R. M., Light, R. H., Parker, S. B., & Levin, H. S. (1996). Benton Controlled Oral Word
Association Test: Reliability and updated norms. Archives of Clinical Neuropsychology,
11(4), 329-338. doi: 10.1016/0887-6177(95)00033-X
Ryan, J. J., Rosenberg, S. J., & Mittenberg, W. (1984). Factor analysis of the Rey Auditory-
Verbal Learning Test. The International Journal of Clinical Neuropsychology, 6(4), 239-
241.
Ryu, W. H. A., Cullen, N. K., & Bayley, M. T. (2010). Early neuropsychological tests as
correlates of productivity 1 year after traumatic brain injury: A preliminary matched case-
control study. International Journal of Rehabilitation Research, 33(1), 84-87. doi:
10.1097/MRR.0b013e32832e6b4b
Salgado, J. F., & Moscoso, S. (2002). Comprehensive meta-analysis of the construct validity of
the employment interview. European Journal of Work and Organizational Psychology,
11(3), 299-324. doi: 10.1080/13594320244000184
Salthouse, T. A. (2005). Relations between cognitive abilities and measures of executive
functioning. Neuropsychology, 19(4), 532-545. doi: 10.1037/0894-4105.19.4.532
Schneider, E. B., Sur, S., Raymont, V., Duckworth, J., Kowalski, R. G., Efron, D. T.,…Stevens,
R. D. (2014). Functional recovery after moderate/severe traumatic brain injury: A role for
cognitive reserve? Neurology, 82(18), 1636-1642. doi:
10.1212/WNL.0000000000000379
EMPLOYMENT STATUS FOLLOWING TBI 157
Schneider, J. C., Bassi, S., & Ryan, C. M. (2011). Employment Outcomes After Burn Injury: A
Comparison of Those Burned at Work and Those Burned Outside of Work. Journal of
Burn Care & Research, 32(2), 294-301. doi: 10.1097/BCR.0b013e31820aaf56
Schonberger, M., Ponsford, J., Olver, J., Ponsford, M., & Wirtz, M. (2011). Prediction of
functional and employment outcome 1 year after traumatic brain injury: a structural
equation modeling approach. Journal of Neurology, Neurosurgery, and Psychiatry, 82(8),
936-941. doi: 10.1136/jnnp.2010.210021
Shah, S., Vanclay, F., & Cooper, B. (1989). Improving the sensitivity of the Barthel Index for
stroke rehabilitation. Journal of Clinical Epidemiology, 42(8), 703-709. doi:
10.1016/0895-4356(89)90065-6
Shames, J., Treger, I., Ring, H., & Giaquinto, S. (2007). Return to work following traumatic
brain injury: Trends and challenges. Disability and Rehabilitation, 29(17), 1387-1395.
doi:10.1080/09638280701315011
Sherer, M., Novack, T. A., Sander, A. M., Struchen, M. A., Alderson, A., & Thompson, R. N.
(2002). Neuropsychological assessment and employment outcome after traumatic brain
injury: A review. The Clinical Neuropsychologist, 16(2), 157-178. doi: 10.1076/clin.16.2.
157.13238
Sherer, M., Sander, A. M., Nick, T. G., High, W. M., Malec, J. F., Rosenthal, M. (2002). Early
Cognitive Status and Productivity Outcome After Traumatic Brain Injury: Findings From
the TBI Model Systems. Archives of Physical Medicine and Rehabilitation, 83(2), 183-
192. doi: http://dx.doi.org/10.1053/apmr.2002.28802
EMPLOYMENT STATUS FOLLOWING TBI 158
Sherman, E. M. S., Strauss, E., Spellacy, F., & Hunter, M. (1995). Construct validity of WAIS-R
factors: Neuropsychological test correlates in adults referred for evaluation of possible
head injury. Psychological Assessment, 7(4), 440-444.
Sherrill, R. E. (1987). Options for shortening Halstead’s category test for adults. Archives of
Clinical Neuropsychology, 2(4), 343-352. doi: 10.1016/0887-6177(87)90003-5
Shimer, R. (2012). Reassessing the ins and outs of unemployment. Review of Economic
Dynamics, 15, 127-148. doi: 10.1016/j.red.2012.02.001
Sigurdardottir, S., Andelic, N., Roe, C., & Schanke, A. K. (2009). Cognitive recovery and
predictors of functional outcome 1 year after traumatic brain injury. Journal of the
International Neuropsychological Society, 15(5), 740-750. doi: 10.1017/S135561770999
0452
Sigurdardottir, S., Andelic, N., Roe, C., & Schanke, A. K. (2013). Depressive Symptoms and
Psychological Distress During the First Five Years After Traumatic Brain Injury:
Relationship With Psychological Stressors, Fatigue, and Pain. Journal of Rehabilitation
Medicine, 45(8), 808-814. doi: 10.2340/16501977-1156
Simpson, A., & Schmitter-Edgecombe, M. (2002). Prediction of employment status following
traumatic brain injury using a behavioral measure of frontal lobe functioning. Brain
Injury, 16(12), 1075-1091. doi:10.1080/02699050210155249
Soper, D. S. (Accessed January 30, 2012). The Free Statistics Calculators Website. Retrieved
from: http://www.danielsoper.com/statcalc3/
Spencer, R. J., Wendell, C. R., Giggey, P. P., Seliger, S. L., Katzel, L. I., & Waldstein, S. R.
(2013). Judgment of Line Orientation: An examination of eight short forms. Journal of
EMPLOYMENT STATUS FOLLOWING TBI 159
Clinical and Experimental Neuropsychology, 35(2), 160-166. doi:
10.1080/13803395.2012.760535
Spitz, G., Ponsford, J. L., Rudzki, D., Maller, J. J. (2012). Association between cognitive
performance and functional outcome following traumatic brain injury: a longitudinal
multilevel examination. Neuropsychology, 26(5), 604-612. doi: 10.1037/a0029239
Spreen, O., & Strauss, E. (1991). A Compendium of Neuropsychological Tests: Administration,
Norms and Commentary. New York: Oxford University Press.
Spreen, O., & Strauss, E. (1998). A Compendium of Neuropsychological Tests: Administration,
Norms, and Commentary (2nd ed.). New York: Oxford University Press.
Stallings, G., Boake, C., & Sherer, M. (1995). Comparison of the California Verbal Learning
Test and the Rey Auditory-Verbal Learning Test in head-injured patients. Journal of
Clinical and Experimental Neuropsychology, 17(5), 706-712. doi: 10.1080/01688639508
405160
Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A Compendium of Neuropsychological
Tests: Administration, Norms, and Commentary (3rd ed.). New York: Oxford University
Press.
Stern, Y. (2002). What is cognitive reserve? Theory and research application of the reserve
concept. Journal of the International Neuropsychological Society, 8(3), 448-460. doi:
10.1017.S1355617701020240
Suhr, J. A., & Gunstad, J. (2000). The effects of coaching on the sensitivity and specificity of
malingering measures. Archives of Clinical Neuropsychology, 15(5), 415-424. doi:
10.1016/S0887-6177(99)00033-5
EMPLOYMENT STATUS FOLLOWING TBI 160
Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics (5th ed.). Boston, MA:
Pearson.
Taylor, J. E., Leung, J., & Deane, F. P. (2011). Neuropsychological assessment of fitness to drive
following acquired cognitive impairment. Brain Injury, 25(5), 471-487. doi:
10.3109/02699052.2011.559609
The Management of Concussion/mTBI Working Group (2009). VA/DoD Clinical Practice
Guideline for Management of Concussion/Mild Traumatic Brain Injury. Journal of
Rehabilitation Research and Development, 46(6), CP1-68.
The Psychological Corporation (2002). WAIS-III/WMS-III: Updated Technical Manual. San
Antonio: The Psychological Corporation.
The Traumatic Brain Injury Model Systems National Data and Statistical Center (2010). TBI
Model Systems Brochure [Brochure]. Retrieved from https://www.tbindsc.org/Documents
/TBIModel%20SystemsBrochure2010.pdf
Tiffin, J. (1968). Purdue Pegboard Examiner Manual. Chicago: Science Research Associates,
Inc.
Tombaugh, T. N., Kozak, J., & Rees, L. (1999). Normative data stratified by age and education
for two measures of verbal fluency: FAS and animal naming. Archives of Clinical
Neuropsychology, 14(2), 167-177. doi: 10.1093/arclin/14.2.167
Treccani, B., & Cubelli, R. (2011). The need for a revised version of the Benton judgment of line
orientation test. Journal of Clinical and Experimental Neuropsychology, 33(2), 249-256.
doi: 10.1080/13803395.2010.511150
EMPLOYMENT STATUS FOLLOWING TBI 161
Triggs, W. J., Calvanio, R., Levine, M., Heaton, R. K., & Heilman, K. M. (2000). Predicting
hand preference with performance on motor tasks. Cortex, 36(5), 679-689. doi:
10.1016/S0010-9452(08)70545-8
Usui, N., Haji, T., Maruyama, M., Katsuyama, N., Uchida, S., Hozawa, A,…Taira, M. (2009).
Cortical areas related to performance of WAIS Digit Symbol Test: A functional imaging
study. Neuroscience Letters, 463(1), 1-5. doi: 10.1016/j.neulet.2009.07.048
Vakil, E. (2005). The effect of moderate to severe traumatic brain injury (TBI) on different
aspects of memory: A selective review. Journal of Clinical and Experimental
Neuropsychology, 27(8), 977-1021. doi: 10.1080/13803390490919245
Vakil, E., & Blachstein, H. (1993). Rey Auditory Verbal Learning Test: Structure analysis.
Journal of Clinical Psychology, 49(6), 883-890. doi: 10.1002/1097-
4679(199311)49:6<883::AID-JCLP2270490616>3.0.CO;2-6
van den Burg, W., & Kingma, A. (1999). Performance of 225 Dutch school children on Rey’s
Auditory Verbal Learning Test (AVLT): Parallel test-retest reliabilities with an interval
of 3 months and normative data. Archives of Clinical Neuropsychology, 14(6), 545-559.
doi: 10.1016/S0887-6177(98)00042-0
van der Heijden, P., & Donders, J. (2003). A confirmatory factor analysis of the WAIS-III in
patients with traumatic brain injury. Journal of Clinical and Experimental
Neuropsychology, 25(1), 59-65. doi: 10.1076/jcen.25.1.59.13627
van der Horn, H., Spikman, J. M., Jacobs, B., van der Naalt, J. (2013). Postconcussive
Complaints, Anxiety, and Depression Related to Vocational Outcome in Minor to Severe
Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation, 94(5), 867-
874. doi: 10.1016/j.apmr.2012.11.039
EMPLOYMENT STATUS FOLLOWING TBI 162
Varney, N. R., & Stewart, H. (2004). Is impaired executive function a single or multidimentional
disability? Applied Neuropsychology, 11(4), 227-232. doi: 10.1207/s15324826an1104_10
Volbrecht, M. E., Meyers, J. E., & Kaster-Bundgaard, J. (2000). Neuropsychological outcome of
head injury using a short battery. Archives of Clinical Neuropsychology, 15(3), 251-265.
doi: doi:10.1016/S0887-6177(99)00016-5
Walker, W. C., Marwitz, J. H., Kreutzer, J. S., Hart, T., & Novack, T. A. (2006). Occupational
categories and return to work following traumatic brain injury: A multicenter study.
Archives of Physical Medicine and Rehabilitation, 87(12), 1576-1582. doi:10.1016/j.apm
r.2006.08.335
Ward, L. (1990). Prediction of verbal, performance and full scale IQs from seven subtests of the
WAIS-R. Journal of Clinical Psychology, 46(4), 436–440. doi: 10.1002/1097-
4679(199007)46:4<436::AID-JCLP2270460411>3.0.CO;2-M
Ward, L. C., Ryan, J. J., & Axelrod, B. N. (2000). Confirmatory factor analyses of the WAIS-III
standardization data. Psychological Assessment, 12(3), 341-345. doi: 10.1037/1040-
3590.12.3.341
Wechsler, D. (1987). Wechsler Memory Scale—Revised. New York: The Psychological
Corporation.
Wechsler, D. (1997). Wechsler Adult Intelligence Scale-III. San Antonio, TX: The Psychological
Corporation.
Wechsler, D. A. (1945). Standardized memory scale for clinical use. Journal of Psychology,
19(1), 87-95.
Wehman, P., Targett, P., West, M., & Kregel, J. (2005). Productive work and employment for
persons with traumatic brain injury: What have we learned after 20 years? Journal of
EMPLOYMENT STATUS FOLLOWING TBI 163
Head Trauma Rehabilitation, 20(2), 115-127. Retrieved from: http://journals.lww.com/he
adtraumarehab/pages/default.aspx
Wetter, M. W., & Corrigan, S. K. (1995). Providing information to clients about psychological
tests: A survey of attorneys’ and law students’ attitudes. Professional Psychology:
Research and Practice, 26(5), 474-477. doi: 10.1037/0735-7028.26.5.474
Whelan-Goodinson, R., Ponsford, J., Johnston, L., & Grant, F. (2009). Psychiatric disorders
following traumatic brain injury: Their nature and frequency. Journal of Head Trauma
Rehabilitation, 24(5), 324-332. doi: 10.1097/HTR.0b013e3181a712aa
Wilde, M. C., Boake, C., & Sherer, M. (2000). Wechsler Adult Intelligence Scale – Revised
block design broken configuration errors in nonpenetrating Traumatic Brain Injury.
Applied Neuropsychology, 7(4), 208-214. doi: 10.1207/S15324826AN0704_2
Williams, M. W., Rapport, L. J., Hanks, R. A., Millis, S. R., & Greene, H. A. (2013).
Incremental Validity of Neuropsycholgoical Evaluations to Computed Tomography in
Predicting Long-Term Outcomes after Traumatic Brain Injury. The Clinical
Neuropsychologist, 27(3), 356-375. doi: 10.1080/13854046.2013.765507
Wilson, B., Cockburn, J., Baddeley, A., & Hiorns, R. (1988). The development and validation of
a battery for detecting and monitoring everyday memory problems. Journal of Clinical
and Experimental Neuropsychology, 11(6), 855-870, doi: 10.1080/01688638908400940
Yasuda, S., Wehman, P., Targett, P., Cifu, D., & West, M. (2001). Return to work for persons
with traumatic brain injury. American Journal of Physical Medicine and Rehabilitation,
80(11), 852-864. Retrieved from: http://journals.lww.com/ajpmr/pages/default.aspx
EMPLOYMENT STATUS FOLLOWING TBI 164
Yates, B. T., & Taub, J. (2003). Assessing the costs, benefits, cost-effectiveness, and cost-benefit
of psychological assessment: we should, we can, and here's how. Psychological
Assessment, 15(4), 478-495. doi: 10.1037/1040-3590.15.4.478
Zakzanis, K. K., McDonald, K., Troyer, A. K. (2013). Component analysis of verbal fluency
scores in severe traumatic brain injury. Brain Injury, 27(7-8), 903-908. doi:
10.3109/02699052.2013.775505
Zappala, G., Thiebaut de Schotten, M., & Eslinger, P. J. (2012). Traumatic brain injury and the
frontal lobes: what can we gain with diffusion tensor imaging? Cortex, 48(2), 156-165.
doi: 10.1016/j.cortex.2011.06.020
Zelazo, P. D., Carter, A., Reznick, J. S., & Frye, D. (1997). Early development of executive
function: A problem-solving framework. Review of General Psychology, 1(2), 198-226.
doi: 10.1037/1089-2680.1.2.198
Zhu, J., Tulskey, D. S., Price, L., Chen, H. (2001). WAIS-III reliability data for clinical groups.
Journal of the International Neuropsychological Society, 7(7), 862 – 866.
Zillmer, E. A., Spiers, M. V., & Culbertson, W. C. (2008). Principles of Neuropsychology (2nd
ed.). Canada: Wadsworth Cengage Learning.