Web-Based Cognitive Health Coaching for

Elders in a Home Environment Holly Jimison, PhD

Misha Pavel, PhD ORCATECHOregon Center for Aging & Technology

ORCATECH 2

What Can Be Done about Cognitive Decline?

• Good news– Neural plasticity at all ages– New drugs to delay cognitive decline

• Potentially good news– Monitoring computer interactions may detect

decline earlier– Development of cognitive exercises to delay

decline and possibly remediate

ORCATECH 3

Project Objectives

• Develop & test computer games for continuous cognitive monitoring of elders in a home environment– Sufficiently enjoyable to play frequently– Embedded assessment algorithms– Dynamic user models

• Develop & test a cognitive health coaching system that provides opportunities for cognitive skill building– Assess cognitive deficits– Provide tailored cognitive exercises– Interactive cognitive health coaching

ORCATECH 4

Problems with Current Cognitive AssessmentMethods

• Assessments done only after concern on the part of the patient or caregiver – usually late

• Repeated assessments infrequent – yearly at best

• Large day-to-day variability in cognitive performance with onset of dementia

• Unwanted variability in assessments due to education, language, motivation, etc.

ORCATECH 5

• Verbal Fluency• Word List Learning• Constructional Praxis• Trail Making Test• Symbol Digit Modalities Test• Letter-Number Sequencing• Finger Tap Test

Traditional Cognitive Tests

ORCATECH 6

Adaptive Cognitive Computer Games

• 30 users (all cognitively healthy)– Average age 81.5 yrs

(standard dev= 6.04 yrs)– 13% male, 87% female

• 9 Interactive games developed with Spry Learning as part of NIST grant

• Being evaluated in local elders’ residences

• Neuropsychological data was collected at baseline, 3 month, 6 month, and at a planned 1 year period

Adaptive Cognitive Computer GamesSpry Learning / OHSU (NIST grant)

ORCATECH 7

Elders’ Use of Computers

• Growth in computer use by elders rose 47% since 2000 – fastest growing group

• 70% use email• 59% Web browsing• 35% play computer games (similar to general

population rate of 39%)

ORCATECH 8

In-Home Monitoring of Computer Interactions

• Frequent data trend detection– Individual as their own control– Less influenced by

• Educational background• Language• Cultural differences

• Dynamic algorithms to model user performance• Inexpensive screening, coaching

ORCATECH 9

Game Data Analysis and Interpretation

• Objective: Develop metrics that reflect cognitive capability and function of the players

For Each Game• Ideal Player:

Develop framework capturing the information processing resources and capabilities required by an ideal player

• Cognitive Models and Limits: Describe ways that a human player may differ from an ideal one

• Specific Metrics: Characterize the required cognitive processes by a small set of parameters

ORCATECH 10

Word games:• Word length• Word complexity• Speed of word generation• Total number of words

Compare to:• Verbal Fluency

Letter Lotto – Word Game

ORCATECH 11

Letter Generation Rate as a Game Metric

0 10 20 30 40 50 60 70 80 90 1000

50

100

150

200

250

Total Time to Guess Words (Each Game) vs Number Letters Guessed Correctly (User 1020)

# Letters Guessed Correctly

Tim

e (Sec)

Regression Equation: y= 2.31189 x + 13.2406 r = 0.866714

ORCATECH 12

Letter Time vs. Neuropsychological Scores

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 550

55

60

65

70

75

80

85Neuropsych Average Total Fluency vs Letter Generation Time

Letter Generation Time (Sec/Letter)

Ave

rage

Tot

al F

luen

cy (Le

tter + A

nim

al) Fluency vs Letter Generation

y = -4.44616 x + 75.3999 r = -0.343154

ORCATECH 13

Frequency Generation of Words by Length

• Shorter words are guessed more frequently than longer ones

• Exception is 7 letter word– Try harder because it’s

the goal – Have to use all letters– Hence not necessarily

most difficult

0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Average Frequency of Words Correct by Length

Word Length

Ave

rage

Fre

quen

cy

(Wor

ds G

uess

ed/T

otal

Pos

sibl

e)

ORCATECH 14

Neuropsychological Scores vs. Proportion of Words

30 35 40 45 50 55 60 65 70 7550

55

60

65

70

75

80

85Neuropsych Average Total Fluency vs Game Difficulty

Game Difficulty

Ave

rage

Tot

al F

luen

cy (Le

tter + A

nim

al)

Fluency vs Game Difficulty

y = 0.522433 x + 40.2374 r = 0.403347 (with outliers)

Outliers

y = 0.687054 x + 31.3083 r = 0.659844 (w/o outliers)

Proportion of Generated Words

Neuropsych Score vs Proportion of Words

ORCATECH 15

Memory games:• Short-term memory• Working memory• Spatial memory• Abstract reasoning

On the Flip Side – Memory Game

ORCATECH 16

• Characterize Memory Capacity as a Function of:– Intervening number of events– Intervening time– Memory load

• Simple Memory Model: Discrete Buffer

B

B

A

B

A

C

B

A

C

D

B

A

C

D

A

B

C

D

E

C

D

E

B

C

D

E

B

C

D

E

B

F

D

E

B

F

B

D

E

F

G

D

E

F

G

H

E

F

G

H

D

E

F

G

H

D

G

E

F

H

D

E

F

H

D

I

Deterministic Computational Model of Working Memory

Time, Events

ORCATECH 17

Results

0 5 10 150

0.5

1Subject 1020, N = 8687

Pro

ba

bili

ty o

f Co

rre

ct

Intervening Number of Events

0 5 10 15 20 250

0.5

1

Pro

ba

bili

ty o

f Co

rre

ct

Intervening Time [sec]

ORCATECH 18

Examples of Other Fits

2 4 6 8 10 12 14 160

0.5

1Subject 1021, N = 387

Pro

ba

bili

ty o

f Co

rre

ct

Intervening Number of Events

2 4 6 8 10 12 14 16 18 20 220.2

0.4

0.6

0.8

1

Pro

ba

bili

ty o

f Co

rre

ct

Intervening Time [sec]

0 5 10 150.2

0.4

0.6

0.8

1Subject 1024, N = 892

Pro

ba

bili

ty o

f Co

rre

ct

Intervening Number of Events

0 5 10 15 20 250

0.5

1

Pro

ba

bili

ty o

f Co

rre

ct

Intervening Time [sec]

ORCATECH 19

Weekly Tracking of Working Memory Buffer Size

10 20 30 40 50 60 70 80 900

0.5

1

1.5

2

2.5

Subject 1021, N = 120M

em

ory

Bu

ffer

Siz

e

Time [Days]

Rel

ativ

e B

uffe

r S

ize

ORCATECH 20

Estimation of Working Memory Capacity

0 5 10 15 20 250

0.5

1

1.5

2

2.5

3

3.5

Buf

fere

Siz

e [#

of It

ems]

Subjects

Squ

are

Roo

t of

B

uffe

r S

ize

ORCATECH 21

Assessment of Neuropsychological Test Scores

60 70 80 90 100 110 120 130 140 150 1600

20

40

60

80

100

120

140

160

180

200

TR

AIL

S-B

Sco

res

Predicted Scores from Flipside

Trail-Making Test B R = 0.60

Digit Span R = 0.36

ORCATECH 22

21 Tally (2-D Black Jack)

0

00

0 19

8

18

Dual Taskgames:• Divided attention• Working memory• Spatial memory

ORCATECH 23

Double

Bust

Single

BustAccept

Single

Win

Multiple

Win

Bust-Win

highlow

high lowCost

Quality

Evaluation Metrics

ORCATECH 24

Subject ID:1020

Val

ue

of

sub

ject

mo

ve /

valu

e o

f Id

eal m

ove

Response Time per move (sec)

α = 3

in quality function

FAST SLOW

HIGH-QUALITY

LOW-QUALITY

Response Time vs Quality of Move

ORCATECH 25

Reference Players

• Ideal Player- uniform card distribution- look a single move ahead- choose ‘loc*’ where

Valuec(loc*)= min(Valuec(loci)), i = 1 … # empty loc Valueq(loc*)= max(Valueq(loci)), i = 1 … # empty loc

• Chance Player- place card randomly- all locations are equally likely

ORCATECH 26

Modeling Subject Reaction Time

a= 9.9886

b= 38.4971

To= 1.284 s

Te= 0.683 s

RTmin= 2.9397 s

Avg

RT

per

mo

ve

Number of Occupied Locations

Number of Occupied Locations

Subject ID 1025

Pro

b S

earc

hing

Occ

Loc

“Serial Processing of locations”

ORCATECH 27

ORCATECH 28

Modeling Subject Reaction Time

ORCATECH 29

Modeling Subject Reaction Time

Subject ID a b T_occ T_emp T_min

1020 10.7166 13.3114 1.2583 0.4043 3.3855

 1021 14.1551 1.1685 1.8104 0.2804 1.2226

 1022 11.8667 3.534 0.9403 0.9704 2.9144

 1023 21.4271 0.9 1.2769 0 2.053

1024 10.8912 37.9464 1.2048 1.6135 0.9237

1025 9.9886 38.4971 1.2840 0.6830 2.9397

1026 18.0063 1.348 0.8088 0.2773 1.1533

1027 18.0757 0.9 0.7120 0.0282 1.6898

1028 13.3458 12.3714 1.1771 0 2.5044

1029 13.7456 1.0617 0.5736 0.8603 2.2749

1030 10.9933 19.787 1.2249 9.9950 7.3894

1032 11.5338 4.6529 1.2456 0.1127 1.176

1033 8.7695 12.867 0.5714 0.0385 2.137

1034 12.8849 21.9481 0.6782 9.9996 2.9198

Took more time looking at…

Model shows these subjects don’t spend time searching empty locations

ORCATECH 30

Cost and Benefits of Divided Attention

Forced Bust

Unforced Bust

Forced Bust-Win

ORCATECH 31

Shape / color games:• Divided attention• Speed / accuracy• Visual search• Planning• Working memory

Pyramid Builder – Color / Shape Matching

ORCATECH 32

Response Time vs. Move Number

• Response time (move duration) Increases with the Complexity of the Game– Average move duration

increases with move number

– Variance increases with move number

0 20 40 60 80 100 120 1400

1

2

3

4

5

6

7

8

Move Number

Res

pons

e T

ime

[Sec

]

File: tst3.mouse

ORCATECH 33

Example: Response Time Distribution(Data from a Single Game)

Multimodal distribution suggests

• Fast responses to known or easy decision

• Medium requiring some search and decision

• Long – pondering and planning of execution

0 0.5 1 1.5 2 2.5 3 3.50

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

Response Time[Sec]

Rel

ativ

e F

requ

necy

File: tst2.mouse

ORCATECH 34

Resulting Search Efficiency

0 5 10 15 20 25 302

2.2

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

Mov

e T

imes

[sec

]

Number of Distractors

tst3.move

slope = 24 ms/item

Correlation with

Trail-making B

R = 0.49

ORCATECH 35

Rapid movement games:

• Motor speed• Visual search• Set switching • Speed of processing• Divided attention

Compare to:• Finger Tap Test• Trail Making Test

Scavenger Hunt – Trail Making Test

FreeCell – Solitaire Planning Game

Strategy games:• Planning• Visual search

ORCATECH 37

User Performance Metrics

• Difficultydt = number of moves to solution at time t

• User Performance– Reduction in difficulty

• Outcome of game• Time to completion

11

,t

tD f d d

1t t td d

ORCATECH 38

FreeCell Performance Curve

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90S ubjec t nb1.a ls

Eve nts

Diff

icul

ty

Ac tua lExpe c ted

Diff

icul

ty

Number of moves

subject

solver

ORCATECH 39

Performance Scores for Normals vs MCI

0.550.720.27

Mild

Cognitive

Impairment

0.380.120.58Normals

Average

(Individual

SD(Perf))

St Dev

(Individual

Ave(Perf))

Average

(Individual

Ave(Perf))

ORCATECH 40

Benefits of Cognitive Computer Games

Improvements over standard techniques:– Daily monitoring (not yearly, after-the-fact)– Individual as their own control– Adaptive measurement – more sensitive measure– Understanding of variability, trend detection– Unobtrusive, more representative performance

– Potential for remediation

ORCATECH 41

Cognitive Health Coaching Project Objectives

Remote Cognitive Health Coaching to the Home• Provide tailored cognitive exercises & advice with tailored

feedback• General coaching protocols based on principles of health

behavior change– Using health behavior change techniques (motivational

interviewing)– Tailored action plans– Automated feedback– Facilitate care manager and caregiver communications

• Extend the outreach of health coaches– Tailored messaging / automated dynamic protocols

• Incorporate patient and family members into the care team

ORCATECH 42

Overview of Methods

• Needs Assessment– Interviews & focus groups with patients, family caregivers, home

health workers, clinicians, nurse care managers, payers– Interview researchers/experts with an interest in deploying and

evaluating a health intervention in a home environment

• Clinical / Behavioral Protocol Development (Cognitive)• Design Specifications• System Development• Implementation / Usability Testing / Stakeholder Feedback• Test in Homes of Elders

ORCATECH 43

Elder Focus Group Findings

• “Brain before body”• Most had multiple chronic diseases, but were not as

interested in working on the “medical” aspects of health• Highest priorities were quality-of-life and independence• Advice for others

– Considered depression and socialization to be the big problems for their age group

– Many elders recognized a downward spiral in older people who became increasingly embarrassed about their abilities and appearance, so that they tended to avoid leaving their room or socializing.

ORCATECH 44

Expert Interviews

• Researchers• Government

– Legislators, policy makers

• Industry – Health record developers, interoperability specialists, disease

management system vendors

• Elder Care Professionals– Geriatricians, home health nurses, senior activity coordinators,

health coaches

• Cognitive Health Intervention Experts (continuing)

ORCATECH 45

Shared Care Plan(PHR for Chronic Care)

Monitoring ModulesSensors

Health Coaching System

Medication ListElder-Specific HRA

Cognitive Protocols (Spry Games)

Lab Values

Diagnoses

Allergies

HomeCareNurse /Coach

ElderFamily Support

Cognitive Health Coaching Design

ORCATECH 46

Welcome Ellen! Your Cognitive Exercise PlanEllen – Last week you did a great job with the number of FreeCell and Sudoku exercises played. This week, in addition let’s try adding some verbal exercises,such as the Spelling Bee game featured below.Click here for more information on the importance of verbal exercise.

Reply to Coach

Previous Messages

View My Action Plan

Detail on Progress Goal 2/day 2/day 2/day 2/day 2/day 2/day 2/day 2/day

FreeCell Hunt LLotto Sp Bee FlipSide Pyr Bldr 21Tally Sudoku

Mon *** ** * ** * ** ***

Tue ** ** **

Wed **** * ** ** ** ** ** ***

Th * * *** ****

Fri ** * *

Sat

Sun ****** * *

Ellen’s Cognitive Exercise Progress

Log Off

Cognitive Coachingfor Better Living

Update My Information

Return to PHMS

ORCATECH 47

Review of Methods

• Needs Assessment– Interview researchers/experts with an interest in deploying and

evaluating a health intervention in a home environment– Interviews & focus groups with patients, family caregivers, home

health workers, clinicians, nurse care managers, payers

• Clinical / Behavioral Protocol Development (Cognitive)• Design Specifications• Implementation / Usability Testing / Stakeholder

Feedback• Test in Homes of Elders

ORCATECH 48

Components of Cognitive Health Coaching

• Adaptive Computer Games - Cognitive Exercise– NIST grant with Spry Learning– Adaptive cognitive games– Embedded cognitive assessment algorithms– Suite of 9 games measuring various aspects of cognition

• Health Advice Related to Cognition– Physical exercise– Social Interactions– Nutrition– Sleep– Novelty exercises– Mood management (depression advice, humor)

ORCATECH 49

Sample Message – Novelty Exercises

Non-routine actions and thoughts encourage the growth of new connections in the brain. This week let’s work on novelty exercises that can exercise your brain. Here are some examples to choose from:

□ Turn your desk clock or an illustrated calendar upside down. □ Go to a new park, or a new store.□ Try brushing your teeth with your non-dominant hand.□ Distinguish coins using only your sense of touch.□ Turn the volume off on television and invent the dialog based on

what you see the characters doing.□ Find a magazine you normally wouldn't choose, and read it for a

while.

ORCATECH 50

Sample Message – Novelty Exercises

◙ Turn your desk clock & calendar upside down. When you look at a familiar image right side up, your left brain quickly

labels it and diverts your attention to other things.

When the picture is upside down, the quick-labeling strategy doesn't work and your right-brain kicks in, trying to interpret the shapes, color and relationships of a puzzling picture.

This week, place at least one clock and your calendar upside down. Practice using them this way and see how it feels. Do you get better over time?

Next week we’ll try something new.

ORCATECH 51

User Model

1. Elder Assessment• Health Risk Appraisal• Health Goals• Motivation• Barriers• Readiness-to-Change• Contact Preferences

•Timing / Media• Social Support Contacts• Data Sharing Preferences• Authentication (pswd?)

Web forms – spread timing

Monitorcontinuous

Elder User Model• Goal (Assume

Cognitive Health)

• Motivations• Barriers• Change State• Contact Prefs• Social Support• Display Prefs

3. Create / UpdateTailored Action Plan

Daily goals & activities

Fam/Fr User Model• Contact Info/Prefs• Data Permission• Authentication (pwd)

Coach User ModelPredefined

2. Family / Friend Assessment

• Contact Preferences•Timing / Media

• Level of commitment• Authentication (pswd?)

Web form

4. Tailored Feedback & Recommendations

Based on Spry GamesBased on meeting other goals

Sleep behaviorsDietExercise …

5. Generic News and Articles of Interest

Standard Web content

ORCATECH 52

Architecture for Tailored Messaging

Elder Assessment

MonitoringSensorsComputer InteractionsSelf Report

Elder DBUser ModelRaw dataUser State

State DependentActive Methods

(Coaching Protocols)

Update Action Plan

Elder’s Technology Interface(computer, cell phone, exercise watch)

Content DatabasePreconfigured MessagesTutorialsGeneral ContentDefinitions

Summary ReportsPrioritized Alerts

Recommended Messaging

Health CoachTechnology(computer, cell phone)

Tailored Content Delivery• reminders• encouragement• tailored feedback/education• prompt for assessment

Shared Care Plan

ORCATECH 53

Welcome Ellen! Your Cognitive Exercise PlanEllen – Last week you did a great job with the number of FreeCell and Sudoku exercises played. This week, in addition let’s try adding some verbal exercises,such as the Spelling Bee game featured below.Click here for more information on the importance of verbal exercise.

Reply to Coach

Previous Messages

View My Action Plan

Detail on Progress Goal 2/day 2/day 2/day 2/day 2/day 2/day 2/day 2/day

FreeCell Hunt LLotto Sp Bee FlipSide Pyr Bldr 21Tally Sudoku

Mon *** ** * ** * ** ***

Tue ** ** **

Wed **** * ** ** ** ** ** ***

Th * * *** ****

Fri ** * *

Sat

Sun ****** * *

Ellen’s Cognitive Exercise Progress

Log Off

Cognitive Coachingfor Better Living

Update My Information

Return to PHMS

ORCATECH 54

Review of Methods

• Needs Assessment– Interview researchers/experts with an interest in deploying

and evaluating a health intervention in a home environment– Interviews & focus groups with patients, family caregivers,

home health workers, clinicians, nurse care managers, payers

• Clinical / Behavioral Protocol Development (Cognitive)• Design Specifications• Implementation / Usability Testing / Stakeholder

Feedback• Test in Homes of Elders

ORCATECH 55

Summary

• Scalable approach to delivering health care to the home– Extends outreach of lower cost professionals– Can involve family members– Timely and continuous– Home health management is key to caring for chronic conditions

• Cognitive monitoring and potential remediation– Most important aspect of health– Keeps people independent – (better quality of life & lower costs)

• Cognitive Health Coaching System– Serves as a framework for testing the best delivery of cognitive

interventions

ORCATECH 56

Acknowledgements

• ORCATECH – NIA funding• Intel funding for initial FreeCell Cognitive Monitoring• NIST Funding for Spry Learning Cognitive Computer

Games• Health Coaching funding to OHSU from Intel as part of

the Behavioral Intervention & Assessment Commons

More info: jimisonh@ohsu.edu

ORCATECH 57

Representative FreeCell Performance DataMCI Subject and Healthy SubjectOver Time

MCI Subject FreeCell Efficiency Over Time

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

2/23/04 3/14/04 4/3/04 4/23/04 5/13/04 6/2/04 6/22/04

Healthy Elder FreeCell Efficiency Over Time

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

2/3/04 2/23/04 3/14/04 4/3/04 4/23/04 5/13/04 6/2/04

Subject with Mild Cognitive Impairment Cognitively Health Subject

ORCATECH 58

Median Login Inter-keystroke IntervalsM

illis

eco

nd

s

Date

Subject with mild cognitive impairment

Two cognitively healthy elders

ORCATECH 59

Framework for Data and Inferences

• Data– Movements – trajectory and speed

– Selection of objects – ID and response time

– Placement of objects – position and speed

• Interpretation– Characterization of movements

– Inference of tactics

– Inference of strategy

– Inference of abilities

Raw Data

TacticsStrategies

State Estimation

ORCATECH 60

– Average move duration increases with move number

– Variance increases with move number

0 20 40 60 80 100 120 1400

1

2

3

4

5

6

7

8

Move Number

Res

pons

e T

ime

[Sec

]

File: tst3.mouse

Duration of Move ↑ with Complexity of Game

ORCATECH 61

– Multimodal Trajectory of a mouse movement on a single move.

– The size of the disks indicates the dwell time at a that location

0 100 200 300 400 500200

250

300

350

400

450

500

550

600

650

Move number 53

Mouse Move: Trajectory and Dwell Time

ORCATECH 62

Background on Shared Care Plan

• Personal Health Record specifically designed for people with multiple chronic conditions

• Marc Pierson, MD – Whatcom County, Washington• Pursuing Perfection Project funded by Robert Wood Johnson

Foundation• Further refinement of the application was made possible

through grants from the Agency for Healthcare Research and Quality (AHRQ), the Foundation for eHealth Initiative (HRSA OAT), and PeaceHealth

• Commercial version called PHMS from the Congral Company – Nic Ivancic

ORCATECH 63

Project Team Members

• Holly Jimison – PI• Misha Pavel – Co-PI• Nicole Larimer – Senior Research Associate• Michael Shapiro – Graduate Research Assistant • Bobby Hagerty – Health Educator / OHSU Brain Institute• Nic Ivancic – Programmer, Congral Company• Lori Nichols - Consultant

– Director, Whatcom Health Information Network– Shared Care Plan Personal Health Record

• Marc Pierson – Consultant• Spry Learning Company

ORCATECH 64

Correlation Amongst Standard Test (Animal and Letter Fluency)

• What is the inherent variability within animal and letter fluency?

• Within users averaged over baseline, 3 month, and 6 month test period

• R^2~=.46

• Used sum of animal and letter score as total verbal fluency

0 5 10 15 20 25 30 350

5

10

15

20

25

30

35Average Animals vs Average Letter Fluency (C,F,L)

Animal Fluency

Lette

r Flu

ency

Average Animal vs Average Letter

y = 0.36845 x + 7.78601

R2=0.354482 (with outliers)Outliers

y = 0.378927 x + 7.89503

R2=0.464943 (w/o outliers)

ORCATECH 65

Newly Developed Cognitive Computer GamesSpry Learning / OHSU (NIST grant)