program theory and logic models gareth parry phd senior scientist
Post on 22-Feb-2016
32 Views
Preview:
DESCRIPTION
TRANSCRIPT
Program Theory and Logic ModelsGareth Parry PhDSenior Scientist
Program Theory– Content Theory– Execution Theory
Logic ModelsPractical
Welcome and Introduction
Program Theory
What Is A Program Theory?
A conceptual explanation of how the program, as designed, will theoretically result in the desired improvement.
Comprises two major components:– Content Theory– Execution Theory
5
Kirkpatrick Framework
Kirkpatrick Evaluation of Learning Framework has four levels:
1) Experience What was the participants’ experience? Did the participants have an excellent experience working on the improvement project?
2) Learning What did participants learn? Did they learn improvement methods and begin testing?
3) Process/Behavior Did participants modify their behavior? Did they work differently and see change in their process measures?
4) Outcomes Did the organization improve its performance (via outcome measures)?
InputsResources
invested and activities
performed by each
improvementagent/entity
Level 1Participant Experience
Level 2Learning
Level 3 Process/Behavior Changes
Level 4Clinical, personal,
organizational outcomes
Content Theory:What changes will teams make that will result in improved outcomes?
Execution Theory:What will the improvement initiative do that will lead teams to adopt the process changes?
Parry et al. Recommendations for Evaluation of Health Care Improvement Initiatives, 2013, Acad Peds.
Content Theory
Rapid Response Teams
MEWS >=5Use a reliable method to identify
deteriorating patients in real
time.
When a patient is deteriorating,
provide the most appropriate
assessment and care as soon as
possible
MEWS >=4
2 Nurses1 Physician
1 Nurse1 Physician
1 Physician
ActionTheory
Core Concepts Detailed Tasks and Local
Adaptations
PRIMARY DRIVERS SECONDARY DRIVERSAIM
Apply the MEWS Score
Reduce inpatient mortality
Establish appropriate criteria for the MEWS Score
Establish a communication system for an alert
Identify the team who will respond to an alert
Train staff in how to proceed following an alert
Driver Diagram: Example
Use a reliable method to identify
deteriorating patients in real
time.
When a patient is deteriorating,
provide the most appropriate
assessment and care as soon as
possible
Train staff in the use of the MEWS Score
Execution Theory
11
1) Generating the pressure (will) for ICUs to take part2) A networked community3) Re-framing BSIs as a social problem4) Approaches that shaped a culture of commitment5) Use of data as a disciplinary force6) Hard edges
Milbank Quarterly, 2011
SummaryA well described theory:– Provides a common understanding of the intention of the model.– Allows an evaluator to understand the core concepts that
underpin the more detailed tasks carried out by those at the front lines.
– Encourages testing and local amendment, informed by underlying principles.
– Shows how all the pieces of a program are related.– Provides a high-level summary that can be the starting point for
testing in other settings.
Rossi PH, Freeman HE, Lipsey MW. Evaluation: A Systematic Approach, 7th Edition. Sage, California 2004.
Ovretveit J, Leviton L, Parry GJ. Increasing the generalisability of improvement research with an improvement replication programme. BMJ Qual Saf 2011;20:i87-i91.
Logic Models
What Are They?Diagram connection between what we want to accomplish and the way we plan to do that
Teams use them to:– Lay out plan– Expose gaps– Get multiple parties on the same page– Increase intentionality
What Are They?The Basic Logic Model: “If-Then”
Mid-term Outcomes
5
If participants start testing,
then participants will change
their behavior/ processes
Long-term Outcomes
6
If participants change
processes, then you will see changes in patient and organizational
outcomes
Resources/Inputs
Certain resources
are needed to operate
your program
1
Activities
If you have access to
them, then you can
use them to
accomplish your
planned activities
2
Outputs
If you accomplish
your planned activities,
then you will deliver the intended
amount of product/
service to the intended audience
3
Short-term Outcomes
If you accomplish
your planned activities to
the extent you intended, then
participants will see
changes in knowledge and start testing
4
Execution Theory Content Theory
Source: Adapted from WK Kellogg Foundation, Logic Model Development Guide
Project JOINTS: BackgroundAim: To assess whether a rapid spread approach applied at a state-level, will lead to increased adoption of the three interventions to reduce SSI in arthroplastic hip and knee surgery.
1. Preoperative bathing or showering with chlorhexidine
2. Preoperative nasal screening for Staphylococcus aureus carriage followed by decolonization of S. aureus carriers
3. Preoperative skin preparation with a long-acting antiseptic agent in combination
Project JOINTS: Execution Theory Summary
Uptake will occur:
1. When interventions align with local and national priorities
2. By using charismatic leaders, highly respected within their field, to describe the changes.
3. By establishing clear and practical and ambitious aims at a local level.
4. By allowing for and encouraging local tailoring.
5. Through facilitation of a networked community.
Medium Term
In participating hospitals, >90% adoption of:
Preoperative bathing or showering with chlorhexidine for 3 days prior to surgery
Preoperative a)nasal screening for Staphylococcus aureus carriage followed by b) decolonization of S. aureus carriers
Preoperative skin preparation w/a long-acting antiseptic agent in combination w/alcohol
Long Term
Reduce surgical site infections in hip & knee patients in 10 states in the U.S.
Short Term
IHI: Establish & Support Rapid Spread Network
Node- Recruit hospitals-Strengthen/ develop relationships with & among hospitals- Assist hospitals with implementation & questions-Increased capacity to coach hospitals with subject experts-Create links to related initiatives- Track hospital participation- Link hospitals to IHI resources- Provide feedback- Suggest potential exemplar hospitals-Raise awareness of Project JOINTS within state (media)
Hospital- Set aims; form teams- Access and use tools- Increased knowledge of importance of bundle- Increased leadership buy-in- Increased surgeon, admin staff, lab, & patient buy-in- Test & implement new processes & tools- Share and learn from other hospitals- Develop tools to share w/other facilities- Customized implementation (e.g., Pre-ops joints class- Provide feedback to nodes & IHI
External factors: - Other SSI focused projects e.g. Partnership for Patients , Dept of Health HAI reduction efforts and QIO 10th SOW detracted attention due to competing priorities and also prioritized reduction of SSIs- Shifting payment mechanisms for reimbursement for SSIs
Assumptions: SSI reduction is a priority; IHI & nodes are a trusted source of content; to achieve goal must effectively recruit and then engage to support flexible implementation; developing an infrastructure for teams to learn from each other will support successful implementation
Activities
Recruitment-Recruit nodes, hospitals, national partners-Build/strengthen state-level relationships-States assigned to cohort 1 or cohort 2
Ongoing Development & Refinement of Content Materials -Understand and summarize evidence of interventions-Gather existing materials from early adopters -Continuous development & refinement of materials to guide the work
Support ImplementationBuild infrastructure where nodes & hospitals can share learning and adapt implementation to their setting: in-person, phone, web-based support
Revise activities and approach based on learning from cohort 1
Outputs
Nodes in 10 states recruited & assigned to cohorts; hospitals recruited for participation; relationship & support letters from national partners
Tools and communication structure for nodes and hospitals that support implementation• IHI in-person site visits • How-to guide• Improvement tools• Patient/family 1-pager• Surgeon/MD materials &
calls• Measurement tools• Node meetings at 2011 and
2012 forum• Monthly node calls• Webinar call series• State-specific calls• Electronic communications
(website, listserv, email)• Exemplar hospital
application & awardees• Business Case • Hospital stories• Certificate of project
completion • AORN and AAOS Support
Letters• Hospital Town Hall
Meetings
Inputs
$500,000 funding
Project & communication management team: PC, PM, communications, director, field manager, clinical director
Content experts: Surgical, infection control, nursing, and improvement faculty from IHI
Network of state nodes, hospitals & national partners
Experience of sites that have done this before
Communication & technology infrastructure
Learning from campaign tools
Evidence that is ready for spread
Context
3yr federal grant funded project designed in two waves to work with all hospitals in 10 states that perform hip/knee replacements to reduce hip and knee SSIs
The project is designed to use IHI’s strong relationships w/existing nodes, hospitals and nodes’ strong relationships w/hospitals to speed adoption of evidence based practices
Project JOINTS: Logic Model
Practical
ExerciseYou have been asked to design a one year collaborative that aims to improve the uptake of three evidence-based interventions for stroke care:1. Brain imaging within 24 hours of admission to hospital (CT scan) to
confirm stroke type (ischaemic or haemorrhagic) and determine management.
2. Delivery of aspirin or an alternative antiplatelet (for patients where an antiplatelet is clinically indicated) within 24 hours of admission to modulate stroke complications and improve outcomes. For shorthand, we refer to this as “aspirin”.
3. Swallow screen within 24 hours of admission, to prevent unnecessary withdrawal of nutrition, support timely administration or modification of aspirin/antiplatelet delivery and highlight patients who need on-going management of swallow safety.
20
Exercise (30 minutes)
Use the logic model template to begin to map out, how you would describe the:– Inputs– Activities– Outputs– Short, medium and long-term outcomes
Work with others on your tableThink about the outcome you want to impact and work backwards
21
Exercise (15 minutes)10 minutes to talk through your logic models. Reflect on:– What did you find helpful about drafting a logic model?– What was challenging?– What questions do you have?– How can you move forward?
5 minute report out, for each table:– 1 thing that was helpful– 1 challenge– 1 question
22
Thank you!
Please send any follow up questions or suggestions to:
Gareth Parry email: gparry@ihi.orgTwitter: @gjparry03
23
What are you doing?
e.g., training in QI, monthly data analysis calls
What changes in organizational and patient outcomes do you expect ultimately?
e.g., 20% decrease in under-5 mortality
What changes in learning do you expect to come from your activities and outputs?
e.g., teams complete PDSA cycles
What are the reach and products of the activities?
e.g., 3 QI teams trained in improvement methods
What changes in processes do you expect as a result of your short-term outcomes?
e.g., blood protocol implemented with 80% reliability
What are you investing to get the work done?
e.g., staff, funding, space, external partners, materials, tools
External factorsWhat factors outside of the project may be a barrier or facilitator to reaching your desired outcomes?e.g., new incoming government that prioritizes x, high staff turnover in health centers,
AssumptionsWhat are you assuming about what is needed to be able to deliver the program in the above planned way? e.g., leadership is on board and the will is strong, surgeons will make the time to attend trainings and lead their QI team
LOGIC MODEL TEMPLATE: Project Title (Month Year – Month Year)
Context
What is the 2-3 bullet elevator speech? What background info is necessary to understand this project plan?
Inputs Activities Outputs Short term outcomesMedium term outcomes
Long term outcomes
top related