Champlain Assessment/Outcome

Measures Forum

February 22, 2010

WelcomeTable Configurations

Each table has a card with the name of the

discipline(s) and the mix of professions

Select the discipline of interest to you and join that

group using the guidelines for the numbers of

each profession, per table.

Once a table is filled (has it‟s „quota‟) please choose

another table. You will have an opportunity to

work with the tools associated with three (3)

disciplines today.

Please try to mix up the experience around the

table (e.g., community-based, rehabilitation, etc.)

Keynote Speaker

Marc Paquette

Stroke Rehabilitation Coordinator

Regional Stroke Program

The Ottawa Hospital

Canadian Stroke Strategy

2006 Consensus Panel convened

Goal: core set of rehab outcome measures

across continuum

Selection Criteria: published use in stroke

trials, strong psychometric properties, ease of

admin., inter-professional where possible

Domains selected based on ICF (body

structure, function, activity & participation)

Benefits of using a Core Set of Tools

Improve data collection re: stroke rehab

outcomes across regions

Ensure clinicians know which tools are valid,

reliable & support best practice

Promote a common language within teams &

across transitions

Select screening tools which identify when

more assessment/tx is needed

Allow consistency for pts to track progress

Disadvantages of doing nothing

Using tools with poor reliability/validity

Fragmented communication of measures

Duplication of assessment

Increased system costs

Meaningless data

Best Practice for Stroke Care

Canadian BP Recommendations 2008: 4iv & 5viii.

Clinicians should use standardized, valid assessment

tools to evaluate the patient‟s stroke related

impairments & functional status

Ontario Consensus Panel on the Stroke

Rehabilitation system report, 2007: Standard 5,

Stroke related impairments and functional status will

be evaluated by rehab professional trained in stroke

rehab using standardized, valid assessments

Provincial & Regional Supports

Ontario Stroke System

Provincial regions undergoing process

Rehab Network of Champlain (RNOC)

Stroke Rehab Sub-Committee (RNOC)

Champlain Regional Stroke Steering

Committee

Management of your organizations

Objectives of the day

Gain knowledge of CSN domains & tools

Knowledge of Champlain tools in use

Identify enablers & barriers of tools &

implementation

Achieve consensus on some core tools for

adoption

Report back to stakeholders, RNOC &

provincial & national groups

Objectives continued…

Patient centered

Focus on regional standardization

Think system wide

Consider all parts of the rehab continuum

Remember change process takes time

Network with your regional colleagues

Exchange e-mail addresses

Have fun!

Domains for today

Activities of Daily Living

Upper Extremity Structure & Function

Walking/Lower Extremity

Upper & Lower Extremity Impairment

Functional Mobility/Balance

Spasticity

Language

Cognition

Visual Perception

Tools for consideration

FIM & Alpha-FIM

Chedoke-McMaster Assessment Scale

Chedoke Lower Extremity Disability Inventory

Timed “Up & Go” Test

Mini Mental Examination

Montreal Cognitive Assessment

Line Bisection Test

Visual Perception Test Motor-Free

Rivermead Perceptual Assessment Battery

…Tools for consideration

Ontario Society of Occupational Therapists

Perceptual Evaluation

Modified Ashworth Scale

6-Minute Walk Test

Berg Balance Scale

Boston Diagnostic Aphasia Exam

Frenchay Aphasia Screening Test

Domains for future consideration

Measure of Stroke Severity

Medical Comorbidities

Speech Intelligibility

Depression

Functional Communication

Instrumental Activities of Daily Living

Participation

Provincial Outcome Measures

Working Group - Recommendations

Balance: Berg Balance Scale

Self-Care Activities of Daily Living: AlphaFIM® Instrument /FIM® Instrument

Cognition: Montreal Cognitive Assessment (MoCA)

Upper Extremity/Lower Extremity Structure & Function: Chedoke-McMaster Stroke Assessment (CMSA) Impairment Inventory

Language: Boston Diagnostic Aphasia Exam

Results of a Canadian Stroke

Strategy/Heart and Stroke Foundation

National Consensus Panel

CSS Outcome Measures Methodology

Dr. Debbie Timpson,

Physiatrist

Chief of Rehabilitation

Pembroke Regional Hospital

Acknowledgements

Canadian Stroke Network

Canadian Stroke Strategy

Heart and Stroke Foundation of Ontario

Dr. Mark Bayley and Consensus Planning

Group

Considerations for Selecting Responsive

and Interpretable Outcome Measures

Your stroke team indicates they want

to measure the outcomes of their

patients. What are the considerations

in selecting measures?

What measures are Canadian

Clinicians using?

Nicol Korner Bitensky et al surveyed

about 1800 rehabilitation clinicians

in Canada by telephone

Asked to answer scenarios

concerning typical stroke patients

Asked about what measures they

used

Summary of Survey findings

Inconsistent use of measures

Frequently used only at admission and

not at discharge

Not necessarily using measures tested

for responsiveness

Stroke Outcome measures in SREBR

1968-2004 Total number of outcome citations

= 1105

Large number of authors created own study

specific outcomes = 175

Physical assessments not using a single

standardized scale = 178 citations

Citations of previously published scales= 752

Only 35 previously published assessment

scales were cited 5 or more times

Consensus Panel Principles for selection

Tried to select measures that worked

across the continuum

Can be interprofessional administration

Can be administered in reasonable time

at beginning and end of Rehabilitation

Minimize cost of training

Ideally available in English and French.

Ideal Measures for Outcomes

You need a well developed evaluativemeasure with strong psychometric properties for use in clinical practice.

- An evaluative measure assesses an individual or group at baseline and again at one or two points, usually to determine if change has occurred. It needs to be responsive to reflect change in patient status when it occurs.

…continued

What are important

Psychometric properties of

Measures?

Reviewing Reliability

Reliability: the degree to which –

A measure is free from random error

The observed score is different from the true

score

Types of Reliability

**Stability – test-retest (longitudinal)

Inter- and Intra-rater

Assessed by correlation coefficients:

Cronbach‟s alpha, Intra-class Correlation

Coefficient (ICC), Kappa or Weighted Kappa

(K or WK) etc.

Verifying Validity

Validity: the extent to which a measure really

measures what it claims to measure

Reflects an absence of both random and

systematic error (bias)

Not an all or none property –rather a matter

of degree

Responsiveness

What is it? Unlike reliability and validity it is

not a traditional psychometric property.

How is it defined? Many ways / no

consensus.

How is it evaluated? Even more ways.

Terwee et al Qual Life Res 2003;12:349-62

How should it be reported? Quantitatively.

How should it be interpreted? Two main

approaches.

What are we trying to measure?

Meaningful change – to patient, to health

professional, to payer

Minimal Clinically Important Difference

(MCID) (MID)

Some Thoughts on Choosing Outcomes

Less is More – avoid the temptation to want

to answer all the questions on your first

attempt

Consider the time in administration

Consider the time to train people

Consider how much equipment is needed

Consider whether it will change your practice

Some Thoughts on Choosing Outcomes

Pick tools that are trans-disciplinary if possible

Think about who is going to use the Results of all your work in collecting outcomes

Is this for you as a clinician to plan your practice?

Do you want to show those administration people that you make a difference?

Do you want to show the fundors that you make a difference

( What do you think they are interested in?)

Conclusions

1. A core set of rehabilitation outcome measures

for Canada is required as there is variability in

current practice, need to consistently evaluate

system and compare across provinces in

Canadian Stroke Strategy

2. Issues that must be considered when

selecting responsive outcome measures

include the reliability, validity and

responsiveness of the measure. These

properties can now be estimated quantitatively

Today’s Facilitator

Stephanie Burgetz

Learning & Development Specialist

Human Resources

The Ottawa Hospital

Change & TransitionFocus on Future

Clinging to Past

Denial

Anger

Grief

Goodbye

Rituals Fear

Excitement

Commitment:

External

concern for

new job

realities

Internal

Self

InvolvementLetting Go

Neutral Zone

New

beginning

Active Resistance

Exploration

Focus on Future

Clinging to Past

Denial

Anger

Grief

Goodbye

Rituals Fear

Excitement

Commitment:

External

concern for

new job

realities

Internal

Self

InvolvementLetting GoLetting Go

Neutral ZoneNeutral Zone

New

beginning

New

beginning

Active Resistance

Exploration

What is Consensus?

I can live with that and support it

What it is not…

Winner vs. Loser

Us -Them approach

Discussion Guidelines

Listen to understand

Minimize the side conversations

Keep an open mind

Stay focused on the issues

Be tough on the issues but soft on each other

Everyone‟s opinion is valued equally

Participate

Share the air time

Process - AM

In your table groups, by domain:

Review & Discuss each tool

Identify Enablers and Barriers – write on flip charts

Discuss and make recommendations (e.g., guidelines for when it is best used)

Your Goal:

To make a recommendation as to whether this tool should be a “core tool”

Work towards achieving a consensus

Rank order (1, 2, 3…) the tools (if applicable)

Report back to the larger group

Process - PM

Move to another domain – keeping in mind the ratios identified

Work through case studies using the recommended tool(s)

Add any additional enablers/barriers, and recommendations

You will have the opportunity to do this twice (total of two domains)

End of Day Decisions

Going to the three domains that you

looked at today, decide:

I Support the adoption of this Outcome

Measure Tool as a “Core Tool” to be

used in the Champlain LHIN

I am Neutral…

I Reject…

Please place your ‘dot’ sticker in

ONE of the following sections:

I Support… I am Neutral… I Reject…

Wrap Up- Next Steps

Report back to your Directors/Managers

Findings to be shared with regional

stakeholders

Deliverables = Core Tools agreed upon

Start using tools where possible

Training of tools where needed

Follow-up survey of implementation

Review of parking lot items for next year +

Continental breakfast, nutrition breaks & lunch

Compliments of