nca tbc session 3 dec 14 2016
TRANSCRIPT
![Page 1: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/1.jpg)
We will begin shortly…
Welcome
1
![Page 2: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/2.jpg)
Using Zoom
Turn your webcam on!
Please remember to mute yourself during the presentations.
If you have a question, you may un-mute yourself and ask after each presentation, OR use the Q&A button
![Page 3: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/3.jpg)
Advanced Team Based Care (TBC) Learning Collaborative
Welcome to Session 3December 14, 2016
3:00 - 4:30 EST
Developing core and extended team capabilities for implementing an advanced team based care model.
![Page 4: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/4.jpg)
Introductions
TBC Collaborative Design, Facilitation, Faculty Ann Marie R Hess ANP, MS
National Cooperative Agreement Anna Rogers, Director Reema Mistry, Program Coordinator
Mentors , Coaching Faculty Deborah Ward, RN (1:8) Kasey Harding (1:8)
Evaluation Faculty Kathleen Thies, PhD, RN
Improvement Science Faculty Patti Feeney Mark Splaine, MD
![Page 5: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/5.jpg)
Objectives Session 3
Summarize Action Period 2 Milestones (6 weeks) Learn how to use data for improvement Learn from team Specific Aims and PDSAs
→Healthcare for the Homeless→Carolina Family Health Center→The Children's Clinic
Provide path forward and resources for Action Period 3
![Page 6: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/6.jpg)
Session 2 Feedback
88%-100% Visuals supported TBC content Session met learning needs Can apply information to our practice Learned a moderate to great amount Teaching methods were effective
Improvementso Provide more examples of PDSA cycles and fishbone diagramso Send agenda and next assignments ahead of timeo Provide more examples of what it means to create a team environmento We would like more statistics about what works and what does not worko At the time of day for us, people tired even though sessions very educational
![Page 7: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/7.jpg)
![Page 8: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/8.jpg)
16 Teams : most teams adjusted core and extended
93 participating Core and Extended Team Members Interact daily/weekly with patients and families Roles: MD, NP, LPN, BH, Care Coordinator, CNA, FNP, Care Manager, MA, NP,
Front Desk, RN, LVN, Radiology Tech, LCSW, PNP, Interpreter, Case Manager, Dental Coordinator, Pharmacist, Call Center Rep, Outreach Specialist, MSW
POD structure with multiple providers and dedicated MAs (4)
Other Leadership and Management : CQO, Quality Coord, Dir of Ops, Site Manager, HR Dir, Data Analyst, Ops Specialist, Clinical Support Services Manager, Compliance, COO, CMO, EMR Manager, Clinical Manager, IT, Patient Financial Coord, Dir Patient Services, BH Manager, Medical Director, Chief Clinical Officer, CMO, Referral Manager
![Page 9: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/9.jpg)
![Page 10: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/10.jpg)
Advanced TBD Learning Collaborative
![Page 11: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/11.jpg)
Advanced TBD Learning Collaborative
![Page 12: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/12.jpg)
Advanced TBD Learning Collaborative
![Page 13: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/13.jpg)
Agenda (3:00-4:30)
3:00 5 min Welcome and IntroductionsPutting it All Together : Improvement Ramp
3:05 10 min Action Period 2 Milestones and Challenges
3:15 35 min Using Data for Improvement
3:50 30 min TBC Improvement Work -Healthcare for the Homeless Team-Carolina Family Health Center Team-The Children’s Clinic
4:20 10 min Path Forward and Resources [ Introduce Progress Check List]Action Period 3 Assignments
![Page 14: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/14.jpg)
Improvement Science Theory Bursts (10 min)Developing Capacity for Implementing Advanced TBC Model
Session 1 : Sept 21st o Running effective team meetings using toolso Developing and using a cause and effect diagram to inform PDSAso Writing a global and specific aim statement
Session 2 : Nov 2nd
o Developing a process map or current state workflow o Applying PDSA methodology for improvement
Session 3 : Dec 14o Using data for improvement (run charts, bar graphs, sampling)
Session 4 : Jan 25o Standardizing (SDSAs) and Reliability Science
Session 5 : March 15o Spreading Change
Session 6 : April 26o Gantt Charting : 3-6 month Core Team improvement plan
![Page 15: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/15.jpg)
Define Core and Extended Team
Achieve multiple TBC specific aims
Standardize (SDSAs) roles and key processes (Playbook, Spread Plan)
Improve team and coach skills (improvement science, team work, coaching)
Move Practice Assessment Data toward Level A
Develop a post collaborative team action plan
TBC Learning Collaborative
90 minLearningSessions
Between Session Action Periods (6 weeks)Complete AssignmentsWeekly Team Meetings , Daily HuddlesMonthly ReportingShare Your Work – TBC Website (Moodle)
Developing Effective Meeting and Improvement SkillsImplementing Team Based Care – Small Tests of ChangeLearning from Each Other
Action Period 3
Core and Extended Team Refinements – challenges
TBC Webinars
Effective Meetings and Daily Huddles
Readiness Survey
Role Activity and Cycle Time Data - deeper
Global, Specific Aims
Fishbone : Defining Problem and PDSAs
Process Mapping : Workflow and Roles
Brainstorming and Benchmarking
PDSAs
SDSAs
Between Session Mentoring and Faculty SupportMoodle Resources and Discussion Board
7
Sept 21 Dec 14 Jan 25 Mar 15 Apr 26 June 14Nov 2
![Page 16: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/16.jpg)
Mentors HelpingWeekly Coach : Mentor Group Meetings, Individual as Needed
Teaching skills, reviewing tools (more skill building needed from theory bursts, struggling with tools)
Addressing Online Learning Network Site (Moodle) challenges
Advising difficulties getting meetings off the ground (no time to meet), using roles (resistance)
Clarifying assignments, how to use assessment tools
Providing advice for managing : ‘turmoil’, ‘overwhelming assignments’, ‘team and leadership engagement issues’ , ‘team vs coach ownership’, ‘worried management will roadblock us’
Reminding : Start Where you Are, Use What you Have, Do What you Can….
Weekly Meetings : Action Period 3*no meeting tomorrow 3pm
Rate the Meetings On average 8-9 (scale of 1 -10) .
Most helpful:
hearing report outs from other teams and progress they are making,reinforcing and learning new skills to help our teams, learning from how others are overcoming challenges, staying on track with assignments and getting help with how to completegetting help with how to use data
![Page 17: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/17.jpg)
Action Period 2
Milestones More than half the teams have been: Refining their core and extended team structures Defining roles and communication – both within
core team, and between core and extended team
Challenges:• Keeping same staff in Core Team or a POD• Staff pulled for coverage other shifts or sites• Significant turnover (Providers, MAs, RNs)• RNs more attracted to extended team role• Other team members who cover us do not know
what we are doing…
![Page 18: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/18.jpg)
Action Period 2 : Give Teams Time to Meet (webinar 1)Challenges: Time to meet. Getting time when ‘necessary people’ can get there. Team members not engaged Turnover – significant. Staff pulled for coverage other sites, departments Provider not willing to delegate, when ultimately responsible if does not get done Leadership support (e.g.’buy in’ to the process, communication about the work)
10 Teams (overcoming challenges) working hard at sustaining 3 meetings/month, some 4-5/ month), 45-60 min finding a good time to meet after trials of different times learning who is required to attend – those needed to do the work, invite others when needed meeting even if a few people (discipline and rhythm), start on time and do not wait for people getting leadership support for protected time sending meeting reminders (e.g. text 15 min ahead of meeting) engaging team members
o using fishbone and process mapping (‘have stake in it’)o rotating roles, assigning timekeeper for easy roleo posting work on the wall for input between meetings
6 Teams : meeting 2 or fewer times/month, working on unique challenges, different pace
![Page 19: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/19.jpg)
Assignments : Improvement Ramp for Implementing TBC
Define Team Stru
cture, M
eeting
1
Action Period 6
4-5
2-3
SPECIFIC AIMS AND PDSAs
![Page 20: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/20.jpg)
Aim: Reduce waiting room time from 25 min to 14 min By Dec 30
Weekly Data 5 Patients To Date: 19 min from 25 min Adding observation by Coach: MA Rooming
PDSAs : Redesign and standardize MA Rooming Process – more time with MA (value added)
Team Engagement
Peach Tree Healthcare
Action Period 2Example : Data, Mapping, Aims, PDSAs
![Page 21: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/21.jpg)
Early Stages of Change MA Role, RN Just Getting Started (Use Your Data)
Peach Tree Healthcare
![Page 22: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/22.jpg)
Holyoke Health Center
MA Role
RN Role
Insights 8 hours of Tracking Significant Duplication of Efforts Double Documentation (paper, EHR) No Standards and Protocols for activities
‘Insufficient’ RN Care Management and Coordination time – 16 min
Challenging Questions
What is ideal time for advanced activity? All roles at once? How do you narrow down roles and
activity to work on?
![Page 23: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/23.jpg)
Healthcare for Homeless
![Page 24: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/24.jpg)
Role Activity Challenges (Action Period 3)
Common Challenges
Provider Completion of Role Activity More differences than we anticipated
between what we are ‘currently doing’ and what we identified as ‘ideal’
Fears about giving things up, adding new
Lack of role delineation between LPN and RN – and Provider duplication
Variation between same roles (2-3 MAs) So many inefficiencies to tackle
Some Opportunities
Shadowing provider half day - activity tracking. Shorter periods over days.
Activity analyses helping improve job descriptions, role delineation, optimization
Transparency of work opportunity to ask – We are doing ‘what’? We are putting it ‘where’? Why are we doing ‘that’?
Eliminating duplication of documentation Reducing interruptions by optimizing EHR
messaging, workflows
![Page 25: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/25.jpg)
Using Fishbone Long Cycle Timeto Identify Aims and PDSAs (Action Period 3)
Key Drivers Pre Registered vs Not On site lab delays Early and late arrivals (team on time?)
AIM : Increase Percentage of Patients that are Pre Registered from 26% to 29% by Dec 31st
Team Engagement
High Leverage PDSAs(Action Period 3)
Efficiency, Role Optimization
AM , PM Start Times Rooming Standards Daily Huddles (6 Teams) Pre Visit Planning
Sumter Family Health Center
![Page 26: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/26.jpg)
PDSA Discipline (Action Period 3)
Increase Complexity of Change- Start Small Test, days –weeks- Small Wins- Each PDSA can have a measure
Increase number of patients- Have standards and protocols- Have standard workflows- Hard to implement
Get it right , Fewer Patients- before scaling up to a defined population
![Page 27: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/27.jpg)
Daughters of Charity
October 20160
10
20
30
40
50
60
70
80
90
80
Total Cycle Time - Check In to Check Out
Use Your Data (Cycle Time) Track Over Time (Action Period 3)(5 / week, 15/ month)
How long do we track it? Achieving results you wanted, sustaining results due to standardization and process reliability, new habits in daily work
Check In Time
Waiting Room Time
Support Staff Time
Exam Room Wait
Time
Time with Provider
Check out Time
0
5
10
15
20
25
30
Break Down of Cycle TimeSpecific Aims
Goal: 30 min for 20 min appt type
![Page 28: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/28.jpg)
Using Data for Improvement (Action Period 3)
Gathering an
d Using Data
Skills
Using Eff
ective
Mee
ting Skills
Implemen
ting Daily Huddles
Using Im
provement S
kills
Applying Tea
mwork Skills
020406080
10066
86 76 6891
Team Skills Self-Assessment Summary (N=68)Percentage Strongly Agree or Agree are Competent
% S
tron
gly
Agre
e or
Agr
ee
October 2016
![Page 29: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/29.jpg)
Using Data for Improvement
Mark Splaine, MD3:15 – 3:50
![Page 30: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/30.jpg)
Session on Data & Samplingfor the Team-Based Care Collaborative
Mark E. Splaine, MD, MS
December 14, 2016
![Page 31: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/31.jpg)
Displaying Data & Sampling• Three theory bursts
– Displaying data over time (5 minutes)– Types of variation (5 minutes)– Overview of run charts (5 minutes)
• Application exercise– Interpreting a run chart example (10 minutes)
• Sampling for improvement work (5 mins)• Questions and discussion (5 mins)
31
![Page 32: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/32.jpg)
Daughters of Charity
Check In Time Waiting Room Time
Support Staff Time
Exam Room Wait Time
Time with Provider
Check out Time0
5
10
15
20
25
30
Series1
![Page 33: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/33.jpg)
Time 1 October 26, 2016 33
total time in office
check in time waiting room time
time with the nurse
exam room wait time
time with provider
check out time
0
20
40
60
80
100
120
140
160
180
Mean Minimum Maximum
Cycle Time: Johnson City CHC
![Page 34: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/34.jpg)
Holyoke
MA Role Activity Tracking RN Role Activity Tracking
![Page 35: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/35.jpg)
Diabetics & Flu Shots
35
2012 201346.2
46.4
46.6
46.8
47.0
47.2
47.4
47.6
Year
% R
ecei
ving
Vac
cine
Jan-Mar Apr-Jun Jul-Sep Oct-Dec20253035404550556065
20122013
Quarter
% R
ecei
ving
Vac
cine
Jan-1
2
Mar-12
May-12
Jul-1
2
Sep-12
Nov-12
Jan-1
3
Mar-13
May-13
Jul-1
3
Sep-13
Nov-13
20253035404550556065
Month
% R
ecei
ving
Vac
cine
2012 201320253035404550556065
Year
% R
ecei
ving
Vac
cine
![Page 36: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/36.jpg)
Diabetes MonitoringB
lood
Sug
ar
Days 1-15
36
Days 1-15 Days 16-310
20
40
60
80
100
120
140
160
180
200
> 140< or = 140
> 140< or = 140
Days 16-31Intervention to change diet began on Day 16
Proportion of High
Readings
![Page 37: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/37.jpg)
Time Plot• A graph of data in time order
• Often kept to identify if and when problems appear (proactive)
• Also used to see trends over time (reflection)
• Especially helpful when you implement a change to follow the result
37
![Page 38: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/38.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200 Diabetes Stratified Time Plot
Days
Blo
od S
ugar
(mg/
dl)
Morning ReadingsBefore Bed Readings
38
Goal
![Page 39: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/39.jpg)
Daughters of Charity
October 20160
10
20
30
40
50
60
70
80
90
Total Cycle Time - Check In to Check Out
Total Cycle Time - Check In to Check Out
![Page 40: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/40.jpg)
Two Types of Variation
• Random (common cause) variation
• Non-random (special cause) variation
40
![Page 41: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/41.jpg)
Random Variation• Typically due to a large number of small
sources of variation – Example: Variation in arrival time of a patient
might include: weather, vehicle problems, parking issues
• Usually requires a deep understanding of the process to change
41
![Page 42: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/42.jpg)
Non-Random Variation• Are not part of the process all the time.
Arise from special circumstances – Example: Patients arrive late for appointments
due to a bus strike
• Usually best uncovered when monitoring data in real time (or close to that)
42
![Page 43: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/43.jpg)
How to React to VariationActionProcess result
Process with only random variation
Not satisfied with result: redesign process to get a better result
Reduce variation: make the process even more predictable or reliable
Process with non-random variation
Identify the cause: If positive, then can it be replicated or standardized.
If negative, then cause needs to be eliminated
Target the special causes - to get the process predictable
43
![Page 44: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/44.jpg)
Run ChartsDetecting non-random (special cause) variation
44
![Page 45: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/45.jpg)
45
Anatomy of a Run Chart
Variable “y”
Time
Center line is MEDIAN
![Page 46: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/46.jpg)
Run Chart Example
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200
Days
Fast
ing
Blo
od S
ugar
(mg/
dl)
Median
46
![Page 47: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/47.jpg)
Non-Random Patternson Run Charts
• The presence of a shift in the process– A “run” is one or more consecutive points on the same
side of the median– A run that is too long (6 or more consecutive points on
one side of the median)• The presence of a trend
– A run with consecutive increases or decreases in data (5 or more consecutive points)
• The presence of too much or too little variability – Too few or too many runs (depends on number of
points on the chart)
47Perla, Provost, and Murray. BMJ Qual Saf. 2011;20:46-51
![Page 48: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/48.jpg)
Source: Perla, Provost, and Murray. BMJ Qual Saf. 2011;20:46-5148
Table. Runs Rule Guidance
Number of observations excluding points on the median
Lower limit for the number of runs
Upper limit for the number of runs
13 4 1114 4 1215 5 1216 5 1317 5 1318 6 1419 6 1520 6 1621 7 1622 7 1723 7 1724 8 1825 8 1826 9 1927 10 1928 10 2029 10 2030 11 2131 11 22
![Page 49: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/49.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200
A Run is a point or group of consecutive points that fall on one side of the median
Days
How to Count Runs
49
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 50: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/50.jpg)
Questions1. What does the blue line on the graph
represent?2. How many runs are there?3. How many shifts do you see?4. How many trends are in the data?5. How many non-random patterns (special cause
signals) are met in this run chart?6. What is your interpretation of the chart?
50
![Page 51: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/51.jpg)
Discussion of Answers to QuestionsWhat did you decide?
51
![Page 52: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/52.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200
Q1. What does the blue line represent?
Days
Run charts use the Median as the central tendency measure
The Median
52
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 53: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/53.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200 There are 14 Runs
Days
Q2. How many runs are there?
53
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 54: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/54.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200
There is ONE Shift
Days
Q3. How many shifts do you see?
54
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 55: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/55.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200
There are NO Trends
Days
Q4. How many trends are in the data?
55
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 56: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/56.jpg)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
6080
100120140160180200
Runs = 14 Shifts = 1 Trends = 0
There is ONE Signal
Days
Non-random pattern (Shift)
Q5. How many non-random patterns?
56
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 57: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/57.jpg)
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930316080
100120140160180200
• There is non-random (special cause) variation present. One would need to investigate why this occurred. Since the cause is in the wrong direction, one would ideally like to eliminate this cause from the system.
• Note: upon talking to the patient, the special cause was related to him eating dessert every night while on vacation. Some education about diet could then eliminate the cause.
Days
Non-random pattern (Shift)
Q6. What is your interpretation of the chart?
57
Fast
ing
Blo
od S
ugar
(mg/
dl)
![Page 58: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/58.jpg)
Thoughts on SamplingFramed for Improvement Work
58
![Page 59: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/59.jpg)
Two Type of SamplingRandom (Probability)
Samples
• Think of a pond or lake• Water stays in place
Judgment Samples
• Think of a stream or river• Water constantly moving
59
![Page 60: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/60.jpg)
Improvement Work• Benefits from judgment sampling• What is judgment sampling?
– A nonprobability sample that is selected on the basis of knowledge of the process or a subject matter expert
• Is there a trade-off to using judgment sampling?– “We trade the ability to quantify the precision of estimation
and control the bias of selection of a defined population for learning about variation in the fragments of experience we are most interested in learning about – most often with an eye toward efficiency and getting ‘just enough’ data to guide our learning and subsequent testing”
60
![Page 61: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/61.jpg)
Some Examples
61
Example Situation Probability Sample Judgment Sample
5
Nurse leader wants to test impact of new pressure ulcer bundle
• Obtain list of all units• Randomly select 50%• Assess all patients on
selected units before and once after intervention
• Sample 5 patients each week who are at highest risk on the unit with the highest risk patients
• Track data over time
6
Oncology manager wants to know whether patients get proper education after flowsheet initiation
• Simple random sample of all patients in last 3 months
• Charts reviewed by manager
• Select the most recent patients
• Perform chart review
![Page 62: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/62.jpg)
Summary• Variation over time is intrinsic to all health care &
other work processes.• Displaying data over time can help visualize the
variation present.• Understanding that variation can help monitor,
adjust and improve processes.• Studying variation with run charts can offer insights
about possible cause of that variation and offer clues to the design of change.
62
![Page 63: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/63.jpg)
What haven’t we figured out yet?
Questions or issues that remain unclear?
63
![Page 64: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/64.jpg)
References• Perla RJ, Provost LP, and Murray SK. The run chart: a simple
analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46-51.
• Perla RJ and Provost LP. Judgment sampling: a healthcare improvement perspective. Qual Manage Health Care. 2012;21(3):169-175.
• George ML, Rowlands D, Price M, and Maxley J. The Lean Six Sigma Pocket Toolbook. New York, NY: McGraw-Hill, 2005. Chapters 6 and 7, pp 104-118.
![Page 65: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/65.jpg)
Agenda (3:00-4:30)
3:00 5 min Welcome and IntroductionsPutting it All Together : Improvement Ramp
3:05 10 min Action Period 2 Milestones and Challenges
3:15 35 min Using Data for Improvement
3:50 30 min TBC Improvement Work -Healthcare for the Homeless Team-Carolina Family Health Center Team-The Children’s Clinic
4:20 10 min Path Forward and Resources [ Introduce Progress Check List]Action Period 3 Assignments
![Page 66: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/66.jpg)
Healthcare For The Homeless
Specific Aim and PDSAsReferral Process : RN Role
Optimization Extended Team
![Page 67: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/67.jpg)
![Page 68: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/68.jpg)
Manages Referrals to specialists and community resources, ensuring relevant clinical information is provided
B C
![Page 69: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/69.jpg)
![Page 70: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/70.jpg)
Referral Process Redesign
![Page 71: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/71.jpg)
http://online.ideasontario.ca/wp-content/uploads/2015/10/Slide1.png
![Page 72: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/72.jpg)
Carolina Family Health Centers
Specific Aim : Cycle Time Pre Visit Planning Daily Huddles
![Page 73: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/73.jpg)
![Page 74: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/74.jpg)
Total
Cycle T
ime -
Check In
to Check
Out
Check In
Time
Waiting R
oom Time
Support
Staff Ti
me
Exam Room W
ait Tim
e
Time w
ith Pro
vider
Time a
t Lab
Time a
t Refe
rrals
Check out T
ime
0102030405060708090
Oct-16
Oct-16
Cycle Time : Average 78 minutes
![Page 75: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/75.jpg)
Fishbone
![Page 76: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/76.jpg)
Process Map
![Page 77: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/77.jpg)
![Page 78: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/78.jpg)
![Page 79: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/79.jpg)
http://online.ideasontario.ca/wp-content/uploads/2015/10/Slide1.png
![Page 80: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/80.jpg)
The Children’s Clinic
Daily Huddles
![Page 81: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/81.jpg)
Huddles• What time of day, and how long are your huddles?• What is the focus of your huddles?• Are you using a tool?• How are you including your Care Coordinator?• How are you thinking about including your Patient
Service Representative?
![Page 82: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/82.jpg)
![Page 83: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/83.jpg)
http://online.ideasontario.ca/wp-content/uploads/2015/10/Slide1.png
![Page 84: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/84.jpg)
Agenda (3:00-4:30)
3:00 5 min Welcome and IntroductionsPutting it All Together : Improvement Ramp
3:05 10 min Action Period 2 Milestones and Challenges
3:15 35 min Using Data for Improvement
3:50 30 min TBC Improvement Work -Healthcare for the Homeless Team-Carolina Family Health Center Team-The Children’s Clinic
4:20 10 min Path Forward and Resources [ Introduce Progress Check List]
Action Period 3 Assignments
![Page 85: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/85.jpg)
Improvement Ramp for Implementing TBC
1
Action Period 6
4-5
2-3
![Page 86: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/86.jpg)
Action Period 3 Assignments
1. Meet weekly as a Core Team Problem solve ‘time to meet’ Practice effective meeting skills using tools, with coaching support Define core and extended care team (members and roles)
2. Implement Daily Huddles Work on improving (PDSAs) and standardizing (SDSAs) Align Huddle intervention with a specific aim (e.g. reduce cycle time, increase screening)
3. Write specific aim(s) statements , using data and knowledge of problem Continue Assessments (role activity, cycle time, other) Complete Fishbone diagrams and process mapping
![Page 87: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/87.jpg)
Action Period 3 Assignments
4. Implement PDSAs (small, measurable, rapid) Share your work by uploading TBC website, discussion board Use brainstorming and benchmarking to inform changes
5. Complete readiness survey ( 50 % ), data will be posted on TBD WebsitePurpose : To assess whether an organization is ready and committed to the
implementation of a specific change, from the perspective of care team members.
Still time to complete Team Skills Assessment survey (50%)
6. Post Monthly Reports : Next Due January 10th , 2017
7. Watch Webinars (1-4)
8. Introduce Coaches to Progress Check List for tracking implementation, leadership reporting
![Page 88: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/88.jpg)
Progress Check List
![Page 89: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/89.jpg)
Improvingprimarycare.org
Discussion Board Ask questions or make requests of teams, faculty….
![Page 90: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/90.jpg)
Resources
Improvingprimarycare.orgTBC Website (Moodle)
![Page 91: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/91.jpg)
![Page 92: NCA TBC Session 3 Dec 14 2016](https://reader035.vdocuments.mx/reader035/viewer/2022062522/587ca7c91a28ab356c8b5f05/html5/thumbnails/92.jpg)
Thank You All
Survey Post Session
Thank you for your participation today and feedback : Session Evaluation