Data Collection and Measurement

Diane Abatemarco, PhD, MSWRuth S. Gubernick, MPH

Practicing Safety Learning SessionMay 30, 2009

Disclosures

Diane Abatemarco : I have no relevant financial relationships with the manufacturers of any commercial products and/or provider of commercial services discussed in this CME activity. I do not intend to discuss an unapproved/investigative use of a commercial product/device in my presentation.

Ruth Gubernick: I have no relevant financial relationships with the manufacturers of any commercial products and/or provider of commercial services discussed in this CME activity. I do not intend to discuss an unapproved/investigative use of a commercial product/device in mypresentation.

Measurement, Data Collection & Evidence of Change

Importance of measurement – Why? How will we know that a change is an improvement?

Clarify and be directly linked to goals Seek usefulness over perfection Be integrated into daily work whenever possible Be graphically and visibly displayed For PDSA cycles, be simple and feasible enough to accomplish in close time proximity to tests of change

How will we know that a change is an improvement?

Plan

DoStudy

Act

What are we trying to accomplish?

What changes can we make that will result in improvement?

The Improvement ModelThe Improvement ModelThe Improvement ModelThe Improvement Model

The Improvement GuideAssociates in Process Improvement

Measurement for Quality Improvement

You can’t improve what you can’t (or don’t) measure

A good aim statement provides clear direction. Measures tell a team if the changes they make

are making a difference Measurement tells you where you are and

where you are going

Measures

Need to define Target population Numerator Denominator

Practicing Safety: Example

Target Population: All 2 month old infants seen in a participating practice for

well care visit

Numerator: # of infants with documentation in chart that

parent/caregiver received assessment/screening regarding coping with crying at or by the 2 month well visit

Denominator: All 2 month old infants seen in participating practice for

well care visit whose charts are reviewed

Effective Measurement

Seek usefulness, not perfection

Keep measurement simple, think big, but start small

Effective Measurement: Outcomes

Outcome measures: represents the voice of the customer or patient

Average hemoglobin A1c Hospitalizations or ED visits due to asthma Patient satisfaction with time to getting an

appointment

Effective Measurement: Processes

Process measures: represents the workings of the system Percent of charts with documentation indicating

use of 1 or more tools for maternal depression at or by the 2 month well visit.

Percent of parents/caregivers receiving assessment/screening regarding discipline at or by the 18 month well visit.

Effective Measurement

Build measurement into daily work routine Data should be easy to obtain and timely Small samples over time

Use quantitative and qualitative data Qualitative data is highly informative Qualitative data is easy to obtain

Why Plot Data Over Time

You develop a process to screen mothers for maternal depression using the Edinburgh Postnatal Depression Scale. The 6 months before implementing the process the average % screened using this tool is 10%. Six months after the process is implemented, the average % of mothers screened is 90%.

How will you answer the question: was this change an improvement?

0102030405060708090

100

April

May

June Ju

ly

Augus

tSep

tOct

Nov

Change

Run Charts

0

10

20

30

40

50

60

70

April

May

June Ju

ly

Augus

tSep

tOct

Nov

Change

Chart Documentation Form and Instructions

Include in charts; not used as retrospective chart review form

Use in charts of specified visits (2 month; 18 month; or other as determined by your team)

Leave form in chart to document follow-up

Insert practice ID numbers on Charts Documentation Forms before making copies for charts

Submit on 30th of month to Ruth

Note: Binders include 1 form for each age group per month (totaling 12 forms)

Chart Documentation Form and Instructions

Team Planning Identify which of your infant and toddler patients will be seen for

well care during each month Use Chart Documentation Form included as part of their pre-visit

planning/preparation This might change over the six (6) month period, as your team

conducts PDSA cycles to test the process and the Toolkit Your team will also decide how you will determine which 10

Infant Chart Documentation Forms and 10 Toddler Chart Documentation Forms will be submitted to the Practicing Safety Project team for review. For example:

Your protocol may be to submit copies of the Chart Documentation Forms for those infant and toddler patients seen for a well-child visit one day each week (i.e., each Monday or Friday) that month

Or those infant and toddler patients seen for well-care during the last week of that month, for a total of 10 forms of each cohort.

Example Run Charts from Chart Documentation Form

% Receiving Assessment/ScreeningRe: Maternal Depression and Attachment/Bonding

0%

50%

100%

Apr-09

May June July Aug Sept Oct Nov

Month

% A

chie

ved Mat Depression

Attach/Bonding

Goal

% Receiving Assessment/Screening Re: Discipline

0%

50%

100%

Apr-09 May June July Aug Sept Oct Nov

Month

% A

chie

ved

Monthly Progress

Provides information about Tests of change completed each month Assessment of team progress Other qualitative measures

Each month, staff will email the Monthly Progress Report so it can be completed electronically

Instructions Insert your practice’s Aim statement. Indicate the Practicing Safety bundles you have implemented.

Describe specific changes and tools you have tested. Rate your team’s progress using the scale. Report your team’s

learning. Complete Index. E-mail your monthly progress report to Jill Healy at

jhealy@aap.org

Example Chart from Monthly Progress Report Index

Practice-based Systems Index by Month

012345

6789

10

Apr-0

9M

ayJu

ne July

AugSep

tOct Nov

To

tal

Nu

mb

er o

f B

oxe

s C

hec

ked

Project Workspace

All data and run charts will be available for review on the Project Workspace Web site http://www.aap.org/qualityimprovement/quiin/workspaces/PracticingSafety/PS_Home.html Username: pracsafety Password: r3f5y7

Compare your team’s data to other teams and to the aggregate