factors affecting athletes' performance on baseline and post concussion cognitive testing

Factors Affecting Athletes' Performance on Baseline and Post Concussion Cognitive TestingPhilip Schatz, PhDSaint Josephs UniversityTracey Covassin, PhD, ATCMichigan State UniversityAnthony Kontos PhDHumboldt State University Elizabeth Larson BSHumboldt State University RJ Elbin MAMichigan State UniversityNational Academy Of Neuropsychology - November 2009

OverviewMethodological Issues Philip SchatzHistory of Concussion RJ ElbinLearning Disabilities Beth LarsonDepression and Anxiety Anthony KontosGender and Age Tracey Covassin

Methodological Issues(keeping the psychology in sports concussion assessment)-Scheduling of baselines -Individual vs. group effects -Environmental distraction -Feedback from test takers

Concussion Testing(It all started with) Barths VA football study: within-subjects comparison changes from baseline performance between-groups comparison matched cohorts Model widely adopted (colleges, NFL, NHL)

Computer-based AssessmentComputer-based tests introduced (circa 2000)ImPACT, CogSport, CRI, ANAM

Advantages: ease of administration automated scoring decreased practice effects increased test-retest reliability Collie,et al. (2003). Br J Sports Med, 37, 556-559

Baseline TestingBaseline pre-injury testing, serial follow-up is a necessary componentCISG: Vienna (2001), Prague (2004), Zurich (2009) AAN: (1997: Neurology, 48, 581-585): recommend neuropsych measures to detect impairment associated w/concussion NATA: (2004: Guskewicz et al., Neurosurg, 55, 891-895) neuropsych data assists ATCs in evaluating recovery

Scheduling of BaselinesQuestion: How often to update baselines? After a concussion, start of academic yearCollie: (2001: BJSM, 35, 297-302) Valovich: (2003: JAT, 38, 51-56) More frequent for youth athletes -compared to older athletes-period of rapid cognitive maturation (8-15)(Buzzini, 2006: Curr Opin Pediatr; Maruff 2004: BJSM; McCrory: 2004: BJSM)

Scheduling of BaselinesHow often to update? Annually, independent of age, until long-term test-retest reliability data are availableRandolph, McCrea, Barr: (2005: JAT, 40, 139-152) Test-Retest Data: (ImPACT)7-day (Iverson, Lovell, Collins: 2003: Clin Neuropsy)45/50-day (Broglio, et al.: 2001: JAT)4-month (Miller, et al.: 2007: AJSM)

Scheduling of BaselinesSchatz: 2009: AJSM 2-year test-retest reliability of ImPACT 117 Varsity Athletes, base-lined 2 years apart-22 concussed between baselines-7 excluded due to high Impulse Control

Resultant sample of 95 (no football players)

Scheduling of BaselinesSchatz: 2009: AJSM Mean age Time 1: 18.8, Time 2: 20.8 54% male, 46% female Mean difference of 1.9 years between baselines

Scheduling of BaselinesComparing Time 1 -> Time 2-Pearsons r, t-test-Intraclass Correlation CoefficientICC can distinguish those sets of scores that are merely ranked in the same order from test toretest from those that are not only ranked in the same order but are in low, moderate, orcomplete agreement; also yields UER (consistency across baselines; measurement instrument as source of error?)-Reliable Change Indices (RCI) Is change between repeated assessments reliable and meaningful; an estimate of the probability that a score was not obtained as a result of measurement error-Regression-based Measures Scores from the first assessment are placed into a regression analysis, using the score at Time 2 as the DV; the resulting equation provides an adjustment for the effect of initial performance level, as well as controlling for any regression to the mean

Scheduling of BaselinesTime 1 -> Time 2-Pearsons r little help (.27-.60)-t-test no significant differences-Intraclass Correlation Coefficient(ICC as well as UER) Processing Speed (.75); Reaction Time (.68), Visual Memory (.66), Verbal Memory (.47), Symptom scores (.44).

Scheduling of BaselinesT1 -> T2: % athletes showing signif. Change-Reliable Change Indices (RCI) (Traditional /Chelune method adjust for practice effects)

80% CI95% CIVerbal Memory 8% / 12%5% / 5%Visual Memory13% / 10%9% / 7%Process Speed7% / 10%4% / 2%Reaction Time7% / 9%5% / 5%Symptoms8% / 8%6% / 6%Nearly identical numbers of athletes showed improvement &declines in performance from Time 1->Time 2

Scheduling of BaselinesT1 -> T2: % athletes within expect. limits-Regression-based Measures (RMB)

80% CI95% CIVerbal Memory 94.7%97.9%Visual Memory94.7%97.9%Process Speed94.7%96.8%Reaction Time96.8%97.9%Symptoms89.5%94.7%Nearly identical numbers of athletes showed improvement &declines in performance from Time 1->Time 2

Scheduling of Baselines: Summary 2-year test-retest data are stronger than published data at 7-50 days 2-year test-retest data show stability Using RCI, only a small % of athletes show significant change Verb Mem, Symptoms Using RBM, no athletes showed signif change Stretching time between baselines from 1 to 2 years may have little effect on concussion management in collegiate athletes

Individual vs. group effectsBenefits of computer-based testing: cost-benefit gains increased security savings in time, money rapid testing of an entire team (French & Beaumont: 1987: Br. J Clin Psy; Barak: 1999: Appl Prev Psych: Collie: 2001 BJSM; Collie &Maruff: 2003: BJSM)

Individual vs. group effectsRole of distraction in group setting? (Ligon: 1942: Educ and Psychol Measurement) Common errors in group tests: misunderstood instructions, carelessness, low motivation, dishonesty of subjects mental confusion due to too great excitement, size of group group inter-distraction. To reduce errors, training of group testers is needed

Individual vs. group effectsRole of distraction in group setting? SATs, GREs, licensing exams=precedent? Ext. distraction may affect performance (McCrory 2005) Test should be administered in conditions without noises or disruptions (Pierce & Pierce: 1963: J Consult Psychol) Wechsler (1955): Conducted in settings that are quiet and disciplined Studies show no difference between individual and group admin (PPVT: Morris: 1960:J Educ Psych; WAIS: Eme & Walker: 1969, J Clin Psych; Bender-Gestalt:Brannigan:1995: Percep Mot Skills; WAIS/IPET: Neva & Salo: 2003)

Individual vs. group effectsRole of distraction in group setting?

Schatz, Neidzwski, Moser, Karpf (in revision):

Evaluated subjective feedback from athletes taking ImPACT Environmental: noise (talking, laughing, cell phones ringing), distractions (keyboards clicking), discomfort (room temperature) Also recorded Computer-based and Instruction-based (problems) feedback 538 out of 1818 HS student provided subjective feedback (32.4%) 161 reported environmental distraction (9.7%)



Evaluated subjective feedback from athletes taking ImPACT Students reporting environmental feedback endorsed signif. moreSymptoms (8.8 vs. 5.9; p

Individual vs. group effectsWhat to do about distraction in group setting? test individually seat athletes every-other computer (Broglio: 2007) make sure someone in room has authority Team captain, Coach, Head ATC explain the process, educate athletes recourse for inappropriate behaviorTeam is re-tested, not cleared to play until testing is complete


Need counterbalanced basic methodological design Group->Group Group->Individual Individual->Group Individual->Individual

Subjective FeedbackBenefits of computer-based testing: the freedom to increasingly focus on treatment or qualitative assessment gained by the automation of data collection

American Psychological Association. (1986). Guidelines for Computer-based Tests and Interpretations. Washington, DC: APA

Subjective FeedbackIf athletes are tested in groups, where is the testing of the limits during baseline exams?

Are (neuro)psychologists even present? Do (neuro)psychologists need to be present? What qualitative data can be obtained?

Subjective FeedbackWho completes baseline assessments?

team physician or web-based (CISG/Vienna, 2002) Neuropsychologists in best position to interpret test data/batteries (CISG/Zurich, 2008) RTP decision is medical results assist RTP decisions (CISG/Zurich, 2008) Non-neuropsychologist personnel can administer tests (Moser, et al/NAN PoP, 2007) Interpretation of results restricted to neuropsychologists (Moser, et al/NAN PoP, 2007)

Subjective FeedbackWhat qualitative data can be obtained?

Subjective feedback from test takers (ImPACT)

Environmental Computer Instructions Completed at the end of the examination

Subjective FeedbackRole of subjective feedback


Evaluated subjective feedback from athletes taking ImPACT Computer-based feedback: mechanical problems with the mouse, other issues (problems with the monitor, pop-up messages). Problems with test instructions: difficulty understanding specific modules of the test, general issues (confusion, failure to read instructions, not devoting enough time).

Subjective FeedbackRole of subjective feedback?


538 out of 1818 HS student provided subjective feedback (32.4%) 199 reported computer problems (12%) 297 reported problems with instructions (17.9%)

Subjective FeedbackStudents reporting computer problems had signif. Faster reaction time (53 vs. 54; p

Subjective FeedbackStudents reporting problems with instructions endorsed signif. more symptoms (7.8 vs. 5.9; p

Subjective Feedback

Increased symptoms may reflect test-taking anxiety, exacerbation of existing issues from testing Incorporating a question and answer session following testing may enhance qualitative data. Findings reflect the importance of understanding concussion-based symptoms and their relationship to baseline symptoms and an athletes personal style

FACTORS THAT AFFECT NEUROCOGNITIVE TESTING:

HISTORY OF CONCUSSION

Overview of Research: History of Concussion(s)Numerous studies have investigated the influence of concussion history on the risk and recovery from future concussionThree questions have emergedAre athletes with a Hx of concussion at a greater risk for future concussion?Do athletes with a Hx of concussion take longer to recovery from subsequent concussion?Are their any long-term effects associated with a Hx of multiple concussions?

Are Athletes With a History of Concussion at a Greater Risk for Future Concussion?Athletes with a history of concussion are at a four to six times greater risk for incident concussion (Zemper, 2003; Wilberger, 1993; Gerberich et al. 1983)

Concussed H.S. and collegiate athletes are 3x times more likely to sustain another concussion in the same season (Guskiewicz et al. 2000)

Dose Response for # of previous concussions and risk for incident concussion (Guskiewicz et al. 2003)3+ (3.4x)2 (2.8x)1 (1.5x)

Hx of 3+ and severity of future concussion (Collins et al. 2002)9.3 times more likely to demonstrate 3-4 abnormal markersOn-field LOC (6.7x), confusion (4.1x), anterograde amnesia (3.8x)

Do Athletes with a History of Concussion Take Longer to Recover from Subsequent Concussion?No NC differences between 1st and 2nd concussion (Macciocchi et al. 2001)Prolonged recovery after two concussions?2+ Hx of concussionLingering memory impairment and slower reaction time up to 5 days post-concussion (Covassin et al. 2008)

3+ Hx of concussionSignificant memory impairment at 2 days post-concussion (Iverson et al. 2004)Prolonged symptom resolution associated with 3+ Hx (Guskiewicz et al. 2003)

Are There Long-Term Effects Associated with a History of Multiple Concussions?Assessing NC function in athletes with prior history of concussion Deficits in executive function and processing speed observed in Hx of 2+ compared to 1 or zero (Collins et al. 1999)No differences on NC performance between H.S. athletes w/Hx 2+ and recently concussed (w/in 2 weeks) (Moser et al. 2005)

Support found in studies using paper-and-pencil NC tests

Are There Long-Term Effects Associated with a History of Multiple Concussions?No support for computerized NC testsStudies using ImPACT and CRI found no differences between groups of athletes with and without hx of concussions (0, 1, 2, 3, 4) (Collie et al. 2006; Iverson et al. 2006)

Are paper-and-pencil better at detecting long-term impairment than computerized tests?No differences between Hx vs. No Hx on computerized and formal NC tests (Bruce & Echemendia, 2009)

If long-term effects do exist, then NC testing may not be sensitive enough to detect residual impairment (Broglio et al. 2006)

Future studyNeuro-imaging

Linking Research to PracticeCurrent literature suggests that athletes with a Hx of concussion are at a greater risk for injury and may take longer to recover from subsequent concussion

Obtaining detailed information on previous concussion history is important!Identify athletes who are at a higher risk for incident concussionEducate them about concussion

What is the best practice to obtain this information?

Do we take the athletes word for it?Limitations of self-report/recall can be misleadingConcussions are under-reported at all athletic levels (Kaut et al. 2003; LaBotz et al. 2005)

Pre-participation screening (PPS) forms?Lacks specific questions regarding concussion hx (McCrory, 2004)Terminology issue (e.g., ding and/or bell-ringer)8.5 % said yes when asked if they have had a concussion25% said yes when using lay terminology (Valovich-McLeod, 2008)

What is the best practice to obtain this information?

Concussion Symptom Scale (CSS)Higher potential for estimating concussion Hx than PPS (Valovich-McLeod, 2008; LaBotz et al. 2005)The endorsement of one Sx does not imply concussionMay over-report concussion HxInterpretation is key!Headache and blacking out strongly associated with concussion Hx on PPSOther sources and questions?Coaches and teammates are unreliable (McCrory, 2004)

Clinician Recommendations:Detailed Concussion HistoryIncludeSpecific questions about previous S/S of concussion NOT just the perceived numberPrevious head, face, neck injuriesConconmitant concussion?Symptom severity vs frequency of impactsIndicate vulnerability?Consider lay terminology E.g., bell-ringerUse concussion symptom scale formsInquire for both sport and non-sport injuries

Gender and Age Differences

Sport-related Concussion Research: Exploring GenderThe majority of concussion research in the past has been conducted on malesFemale sport participation has increased at both the collegiate and high school levelAre there gender differences in:RiskConcussion symptomsPost-concussion outcomes

Gender Differences and Risk of Concussion in Collegiate AthletesFemales have a greater incidence of concussion than males (Covassin et al. 2003; Gessel et al. 2007;Hootman et al. 2007; Colvin et al. 2009)

Concussions represent 5 to 21% of all reported game injuries for NCAA female athletes and 2 to 9% of all reported game injuries for NCAA male athletes (Hootman et al. 2007)

Womens ice hockey 21.6%Mens ice hockey 9.0%

Womens lacrosse 9.8%Mens lacrosse 8.6%

Womens soccer 8.6%Mens Soccer 5.8

Football 6.8%

Gender Differences and Risk of Concussion in High School Athletes Rechel et. al. (2008) reported a greater proportion of competition injuries 12.0% compared to practice injuries 5.9%

Girls basketball 19.0%Girls soccer 18.8%Boys soccer 15.6%

Gender Differences and Concussion SymptomsFemale concussed athletes reported more concussion symptoms than male concussed athletes (Broshek et al. 2005)Poor concentrationIncreased fatigueLightheadednessSeeing stars

Gender Differences and Concussion SymptomsRecent study on high school and collegiate soccer players found concussed female athletes demonstrated more symptoms than concussed male athletes (Colvin et al. 2009)Total concussion symptomsHeadache

Gender Differences and Concussion OutcomeFemale concussed athletes have demonstrated longer recovery patterns than concussed male athletesvisual memory (Covassin et al. 2007)slower reaction time (Colvin et al. 2009; Broshek et al. 2005)

What may Account these Gender Differences?Females have decreased head mass and neck strength compared to males (Tierney et al. 2005;Manshell et al. 2005; Broglio et al. 2003)Females may report their concussion symptoms with a greater frequency than males

Age Difference and ConcussionHigh school athletes have been found to demonstrate a slower neurocognitive and symptom recovery following concussion than collegiate athletes (Field et al., 2003; Lovell et al., 2003; McCrea et al., 2003; Sim et al. 2008)

Age Differences and Recovery from ConcussionHigh school athletes exhibited memory impairment up to 14 days post-concussion (Field et al. 2003; McClincy et al. 2006)Collegiate athletes demonstrate memory impairment up to 5 days post-concussion (Macciocchi et al. 1996;Field et al. 2003)

Age Differences and Concussion Immature or developing brain places high school athletes at risk for more adverse outcomes following concussion Larger head-to-body ratioImmature musculoskeletal systems (i.e., weaker and smaller)Larger subarachnoid space in the cranium (e.g., allowing more room for brain movement)

Clinical ImplicationsConcussed high school athletes should be removed from practice or competition (Guskiewicz et al. 2004)Second impact syndrome

***SEVERITYATHLETES W/HX OF LOC HAVE BEEN FOUND TO BE FOUR TIMES MORE LIKELY TO EXPERIENCE ANOTHER CONCUSSION INVOLVING LOC THAN THOSE W/O HX**AFTER TWO THERE SEEMS TO BE EVIDENCE OF IMPAIRMENT FOLLOWING CONCUSSION30.0% of those with > or =3 previous concussions had symptoms lasting >1 week compared with 14.6% of those with 1 previous concussion**LITERATURE ON HX AND RISK AND RECOVERY IS CONCLUSIVE! CANT SAY THE SAME ABOUT LONG TERM EFFECTS

Collins et al. 99 did not assess time since last concussion and severity of previous concussions*

factors affecting athletes' performance on baseline and post concussion cognitive testing

Documents