HOW WILL WE KNOW WE’VE MADE AN IMPROVEMENT: MEASUREMENT
AND DATA COLLECTION
Agenda
• Improvement Measurement Basics– Family of measures– Data over time
• CLeAR outcome measures and data collection tool
• Creating measures for the ideas being tested
Improvement Measurement
• Why do we measure?
To show if the changes we are testing are making
things better
Family of Measures
• Measuring improvement will usually need a “family of measures”:
– Process – Outcome– Balancing
Outcome measures
• What are we trying to accomplish?• What is ultimately better?• What will we tell our residents and their
families?
Process Measures
• What are you changing – is it really happening?
• Voice of the system – what is being done differently?
• Processes improve more quickly than outcomes
Balancing Measures
• What are the unintended consequences that might occur?
• What are we worried about – that we can do something about?
A Run Chart: Building Block of Measurement for Improvement
9
Data displayed in time order
Data is collected and displayed weekly or monthly.
April May June July Aug Sept Oct Nov Dec Jan0%
20%
40%
60%
80%
100%
Percent of residents on less than 5 medications
10
No improvement. Random fluctuation.
Improvement. Trend going up.
Improvement. Trend going down.
The Value of Data in Real Time
11
Run Charts can tell us what is happening in real time.
0%
20%
40%
60%
80%
100%
Percent of residents on less than 5 medications
The Value of Data Over Time
12
Run Charts detect true patterns and trends over time.
Not just what is happening before and after a change.
April May June July Aug Sept Oct Nov Dec Jan Feb MarchApril May June July 0%
20%
40%
60%
80%
100%
Percent of residents on less than 5 medications
Pre and Post Change Bar Chart – What is the Interpretation?
14
Scenario 1 . Data displayed in a run chart over time.
0123456789
10
change made between week 7 and 8
Scenario 1 . Pre-post data.
Scenario 1
15
Wee
k 1
Wee
k 2
Wee
k 3
Wee
k 4
Wee
k 5
Wee
k 6
Wee
k 7
Wee
k 8
Wee
k 9
Wee
k 10
Wee
k 11
Wee
k 12
Wee
k 13
Wee
k 14
0123456789
10
Scenario 2 . Pre-post data.
Scenario 2 . Data displayed in a run chart over time.
Scenario 2
change made between week 7 and 8
16
Scenario 3. Pre-post data.
Scenario 3. Data displayed in a run chart over time.
change made between week 7 and 8
Scenario 3
CLeAR Outcome Measures
1) Percent of residents with physicians order for antipsychotic.
1a) Percent of residents with physician’s order for regular antipsychotic.1b) Percent of residents with physician’s order for PRN antipsychotic.3) Total Number of BPSD related incidents.
Measuring the changes being tested
• Will need to measure each of the ideas being tested.
• A few things to decide:– What to measure– How frequently– How many to measure– How to collect and display– Who will collect the data
Example #1
• Change idea: Enhanced education about dementia and BPSD for all staff
• Test: running a couple of 20 minute education session for staff taught by Chris
Measures1) Ask participants if they learned new information. (outcome)
– What to measure - did participants learn something new– How frequently – each education session– How many to measure - all– How to collect – Raise of hands– Who will collect the data - Chris
2) Number of people who attend (Process)– What to measure - Number of people– How frequently – each education session– How many to measure - all– How to collect – Count at the end of session– Who will collect the data - Chris
Example #2
• Change idea: Reduce number of medications (pill burden)
• Test: Use the polypharmacy algorithm and antipsychotics drug sheet to review the medications of 10 residents.
Measures1) Number of medications discontinued (outcome)
– What to measure – number of medications discontinued for each patient– How frequently – each patient– How many to measure – 10– How to collect – checksheet, during medication review– Who will collect the data - Keith
2) Time to review medications(Process)– What to measure – Time spent on review– How frequently – each patient– How many to measure – 10– How to collect – clock, checksheet– Who will collect the data - Keith
Measures3) Number of medications restarted. (balancing)
– What to measure – number of medications restarted within 1 month– How frequently – each patient– How many to measure – 10– How to collect – Chart review– Who will collect the data - Chris
Your turn!
