Adaptive Design: The Key Ingredient for Successful Large Scale Improvement Initiatives

Sodzi Sodzi-Tettey MD, MPH, ISQua

Salomey Akparibo MPH

International Forum

on Quality & Safety

Goteburg, Sweden

April 14, 2016

Objectives

Case study format: Project Fives Alive!

1. What are the various design adaptations of PFA!?

2. What did these adaptations result in?

2

Project Fives Alive!2008-2015

AIM:Assist and accelerate Ghana’s efforts to achieve

Millennium Development Goal 4 (66%

reduction in Under-5 mortality to 40/1000 live births by 2015)

through the application of quality improvement methods

Funded by the Bill & Melinda Gates Foundation

COLLABORATORS :

• Ambitious Aims• Systems View • Core Metrics with Feedback • Rapid Cycle Tests of local ideas

Independent Evaluation: UNC

“Adaptive”

Development evaluation approach given different phases

and national scale up

Findings shared with the implementing team in an

ongoing manner

Evaluation strategy adapted to account for program

changes (Patton 2006)

The mixed methods approach; quantitative impact

analysis, qualitative assessments, cost-effectiveness

analysis (CEA) and the analysis of survey data

4

Implementation Scale Up Framework

National Catholic Health Service

– System transformation through QI was a major

strategic focus

Ghana Health Service– Millennium Development Goals 4 & 5 prioritized

– Use of local data for improvement

– QI potential to complement existing QA structure – coaching,

mentoring, learning networks, rapid cycle tests

Will Building & Set UpAlignment with Local Health Priorities

Scale-Up Design

July 2008

Wave 1: Launch. Nov.12 ;PFA initial end date

July 2010 NCE-1 (PNC Policy)

Journey of Adaptive Designing

Mar. 2014. Cost Extension. National Scale Up. August 2015 PFA! end2012 End of Project - initial

May 2011. Referral Supplemental. May 2015, PFA! end

Nov. 2014. NCE2. Dec. 2015 PFA! end

Community Engagement

Reliance on

Routine Data

Standardized QI Capacity –

Building

Refocus on System

Failures, from Diseases

Hospital-Health Centre

Dynamics

What are we trying toaccomplish?

How will we know that achange is an improvement?

What change can we make thatwill result in improvement?

Model for Improvement

PlanAct

DoStudy

Source: Associates for Process Improvement

Change package* of process improvements that had been shown to be effective in similar contexts

Methodology and Strategy

Assessment and Design

Period

Learning Session 1

© Institute for Healthcare Improvement

Learning Session 2

ACTIVITY PERIOD

Repeated improvement

cycles:

Learning Session 3

12 -24 months

Intensive support from project staff & DHMT

ACTIVITY PERIOD

Repeated improvement

cycles:

Improvement Collaborative Network

Health Facilities

Sub-district Change Package

Care Pathway Successful Change Idea(s) Facility

C H

AN

TENA

TAL

1. Registration

in 1st

Trimester

1A. Community stakeholder meetings

1B. Community stakeholder meetings followed by pregnancy registration

X

X

2. At least 4

visits before

delivery

2A. ANC offered more days at static site AND clinic process re-design

2B. ANC offered as outreach AND re-design clinic processes re-design

X

X

X

PER

INA

TAL

3. Skilled

Delivery &

Immediate

Postnatal Care

3A. Video show in communities on the risks of labour & delivery

3B. Male advocacy group in communities to promote skilled delivery

3C. TBA engagement on risks of unskilled delivery and provide incentives

3D. Home visits to ANC clients at 36+ weeks

3E. Domiciliary delivery if needed

3F. Create a welcoming, patient-friendly environment in health facility

3G. Create systems to ensure consistent and correct use of partographs

3H. Create systems for reliable neonatal resuscitation

XXXXXXXXX

XXX

PO

STNA

TAL

4. Care on Day

1 or 2

4A. If facility skilled delivery –observe for ≥24hrs If not, facility or home visit on Day 2

4B. If domiciliary skilled delivery – follow-up on Day 2 with facility or home visit.

4C. If unskilled delivery – health staff notified,. Home or facility visit on Day 1 or 2

X XX

X

5. Care on Day

6 or 7

5A. Make appointment for Day 6/7 visit at facility or home. Reminder systems in place

5B. If woman lives in different area, refer to other sub-district for Day 6/7 visit.

5C. If woman lives too far away, train IMCI volunteers to provide Day 6/7 care.

XXX

XX

DriverArea of

Clinical/

Community

Care Change Concept Package # Description of Successful Change Ideas

Delay in

Seeking

Care

Care–seeking

behaviour

Targeted health

education

1A Targeted health education on early care-seeking using

interactive platforms

1B Community engagement and education via durbar or place of

worship

ReferralEngaging primary

providers1C Engagement with health providers (both traditional and

allopathic)

Prompt

Diagnosis and

Treatment

Triage

2A

Triage system for screening and emergency treatment of

critically ill children

Separate U5 OPD services from adult OPD service

Prioritize U5 outpatient care

Prioritize U5 inpatient care

Delay in

Providing

CareFast Track

Non-

Adherence

to Protocols

Adherence to

Protocols

Training/

Coaching/

Mentoring

3A

Training staff on protocols followed by regular coaching and

mentoring which include ad hoc testing on site with immediate

feedback.

3B

Training postpartum women and other care givers on hygienic

cord care through demonstration, practice and immediate

feedback. Midwives and nurses teach,

3C Mother-to-mother support group on food choices and frequency

of feeding while on admission under mentoring of nurses.

Task-shifting 3D Empowering nurses to start acting on standard treatment

protocols before doctor arrives

Hospital Change Package

Drivers of Hospital Based Deaths % of QI Teams Adopting at least one Change Idea (N=134)

Comments

Early Care Seeking 84.3 Three Change Ideas (H-1A, 1B, 1C)

Prompt Provision of Care 69.4 A Change bundle (H-2A)

Adherence to treatment protocols

69.4 Four Change Ideas (H-3A to 3D)

Change Idea H -1A H- 1B H- 1C H- 2A H- 3A H- 3B H- 3C H- 3D

Proportion of teams testing this change Idea

58.2 23.1 3.0 69.4 43.3 1.5 2.2 22.4

Hospital Change Package Adoption

by October 2014

July 2008

Wave 1: Launch. Nov.12 ;PFA initial end date

July 2010 NCE-1 (PNC Policy)

Results from a Journey of Adaptive

Designing

Mar. 2014. Cost Extension. National Scale Up. August 2015 PFA! end2012 End of Project - initial

May 2011. Referral Supplemental. May 2015, PFA! end

Nov. 2014. NCE2. Dec. 2015 PFA! end

Community Engagement

Reliance on

Routine Data

Standardized QI Capacity –

Building

Refocus on System

Failures, from Diseases

Hospital-Health Centre

Dynamics

14

(Program Level Time Series Analysis)

Tackling Root Causes of Sub-district & Hospital Processes

0.8

0%

20%

40%

60%

80%

100%

% o

f t

ota

l d

eliv

erie

s c

on

du

cte

d b

y s

kill

ed

pe

rso

nn

el

Wave 1 Collaborative - Skilled Delivery CoverageAim: ≥75% of deliveries conducted by skilled personnel

Subgroup Center UCL LCL

Wave 1 – Aggregated ResultsSkilled Delivery Coverage

LS4; spread of successful change ideas

Incorporation into Wave 2

NHI free for maternity & early infant care; Project launch; LS1

0%

20%

40%

60%

80%

100%

% o

f P

NC

regis

trants

receiv

ing c

are

on D

ay 1

or

2

Wave 1 Collaborative - PNC on Day 6 or 7Aim: ≥80% of PNC registrants to receive follow-up

care on Day 6 or 7

Subgroup Center UCL LCL

0%

20%

40%

60%

80%

100%

% o

f neonate

s r

egis

tering f

or

PN

C o

n D

ay

1 o

r 2

Wave 1 Collaborative - PNC on Day 1 or 2Aim: ≥85% of neonates to receive PNC on Day 1 or 2

Subgroup Center UCL LCL

LS2; Testing of early PNC change ideas began

LS4; spread of successful change ideas & incorporation into Wave 2 LS2; Testing of

early PNC change ideas began

LS4; spread of successful change ideas & incorporation into Wave 2

Wave 1 - Aggregated Results Postnatal Care in 1st Week of Life

Reducing Under 5 Deaths in NCHS Hospitals

Delay in Seeking Care

Reliable use of Protocols

Delay in Providing Care

1o Drivers 2o Drivers

Mobilizing Community

Cultural Barriers

Referral from 1o facility

Financial Barriers

Attractiveness of services

Emergency response Syst.

Outpatient services

Staff Issues

Admission Process

Process Measures

Staff Knowledge and Skills

Availability of Drugs, supplies and equipment

Access to Protocols

Outcome

Average cervical dilatation of women in labour arriving at Hospital

Average time of 1st encounter with hospital after onset of symptoms for children U5

Average Time critically ill U5 identified in hospital to

time first treatment is commenced

Percentage adherence to selected protocols

Average stock out for antimalarial, blood and oxygen

Knowledge of 1o caregiver

Average Time spent by woman in labor from registration until assessment by midwife of doctor

Driver Diagram: Under-5 Deaths

in Hospitals

Wave 3: Nine Innovation Hospitals

Inhibiting Factors:

Weak management

support

Poor team dynamics

High Attrition of core QI

team members

Challenged reporting of

process measures

202H

68H

32H

9H

National Scale-up of Hospital

Change Package

140 Hospitals as of August 2015

(Wave 4)

• 35% reduction in under-5 mortality

• 54% reduction in post-neonatal infant mortality

• 38% reduction in under-5 malaria case fatality

15.2

9.9

UCL

LCL

0

2

4

6

8

10

12

14

16

18

20

Jan

-12

Ap

r-12

Jul-1

2

Oct-

12

Jan

-13

Ap

r-13

Jul-1

3

Oct-

13

Jan

-14

Ap

r-14

Jul-1

4

Oct-

14

Jan

-15

Ap

r-15

Jul-1

5

Under 5 Mortality Rate (Collaborative - 140 Hospitals)

in 7 Regions

7.2

4.7

UCL

LCL

0

2

4

6

8

10

12

Jan

-12

Ap

r-12

Jul-1

2

Oct-

12

Jan

-13

Ap

r-13

Jul-1

3

Oct-

13

Jan

-14

Ap

r-14

Jul-1

4

Oct-

14

Jan

-15

Ap

r-15

Jul-1

5

Malaria Case Fatality Rate (Collaborative - 140 Hospitals)

in 7 Regions

Subgroup Center UCL LCL

50.0

23.0

UCL

LCL

0

10

20

30

40

50

60

70

Jan

-12

Ap

r-12

Jul-1

2

Oct-

12

Jan

-13

Ap

r-13

Jul-1

3

Oct-

13

Jan

-14

Ap

r-14

Jul-1

4

Oct-

14

Jan

-15

Ap

r-15

Jul-1

5

1-11 Months Mortality Rate (Collaborative - 140 Hospitals) in 7 Regions

Subgroup Center UCL LCL

21

Independent Evaluation

Creating an Equal Health Worker-Community Platform

Referral: Deeper Community Engagement

Improved Access for Poor Women

0

10

20

30

40

50

60

70

80

90

North-Intervention** North-Comparison+ Central-Intervention *** Central-Comparison

Baseline Midline Endline

Note: Significance from baseline to endline shown: †p < 0.1 *p < 0.05 **p < 0.01 ***p < 0.001

Improving Health Worker Attitude

The midwife here is good and she does not scream at

people in labour. As you know, going to deliver is a very

painful thing and some of the midwives scream or shout

at pregnant woman in labour.

– [Hairdresser, Central Region, 28 yrs.]

24

25

QI Team Dynamics

QI Team Functionality & Achievement

Type of QI Team Key Quotations

High Functioning/High Achieving “They work together as a unit. They all see it

as a responsibility.”

High Functioning/Low Achieving “Certain factors that are outside the facility

cause them not to be able to sustain the gains.”

Low Functioning/High Achieving “Whatever they get is because of the few who

put so much into it.”

Low Functioning/Low Achieving “A team that doesn’t execute. You plan, plan,

plan and no execution.”

Pairing high functioning/high achieving teams with teams who performed less well

Was the QI Intervention Cost-Effective?

Overview of Largest Expense

Categories in Wave 1

33.65%

25.64%

5.97%

24.54%

7.24%

2.96%

ProfessionalTraining

Personnel (full-time project

staff)

Equipment Direct Project Indirect Costs Personnel(support staff)

Capital Costs Recurrent Costs

Expenditures as a Proportion of Wave 1 Budget

Was PFA! Cost-Effective?

“… the large expenditures in the pilot phase paid

off, not only in the pilot phase itself, but also in

the scale-up phases as demonstrated by the

reductions in under-five mortality that were

significant in the Wave 2 and 4 impact analyses”– UNC Independent Evaluation

29

Demographic Health Survey Results

(2015)

The Demographic Health Survey of Ghana coincided with the start (2008) and end (2015) of the Project. Current results show:

– Under-5 mortality in Ghana reducing from 80 to 60 per 1,000 live births

– Child mortality (1 -4 years) reducing from 31 to 19 per 1,000 live births

– Infant mortality reducing from 50 to 41 per 1,000 live births

– Neonatal mortality reducing from 33 to 29 per 1,000 live births

Sustainability

Wide & Deep Capacity Building

~ 400 Improvement Coaches Trained

for Scale Up

10 Regional Quality Advisors

~ 3000 Site Visits

~ 4000 frontline

workers trained in LSs

Africa–based Quality Institute Formed

Objectives:Centre of QI Excellence for regional support

Partnership with IHI

Set global standards for development and

implementation of large-scale QI initiatives

Deliver QI educational content, spur innovation

in QI, and challenge conventional thinking

Facilitate learning opportunities for

organizations, professionals, and students keen

to learn about QI implementation

Offer basic and advanced online education

learning options, and sponsor periodic

benchmarking visits

Ghana QI Sustainability WorkIHI-Ubora Collaboration to Hold the Gains

Objective 2: Strengthen the Ubora Institute to sustainably support the national quality improvement strategy, coordination and programming in Ghana and across the African Region

Objective 1: Strengthen

national-level sponsorship for

a health system that

mainstreams and integrates

QI methodologies

•Funded by the Bill & Melinda Gates Foundation – Nov. 15 – Oct. 2018

Ministry of Health

Lessons Learned

1. Complement QI with QA

2. Plan project with end in mind at the beginning

3. Co-ownership, co-creation, co-design and co-

implementation with health system managers are

crucial for sustainability

4. Complement reliance on routine data systems with

robust data quality improvement protocols

5. Communication / dissemination are part of “the work”

6. Need to design even more cost-effective spread

strategies

Large Scale Initiatives: What Worked, Likely Pitfalls, Useful Lessons

Human Resources QI Capability Measurement Communication

Project Design Relationships Leadership

PFA! Lessons Learned Guide

http://www.ihi.org/resources/Pages/Publications/ProjectFive

sAliveLessonsLearnedGuide.aspx

37

Thank You

Questions & Discussions