incorporating quality of care investments into a health

1

Incorporating Quality of Care Investments into a Health System Pay-for-Performance Framework:

Process Evaluation of Zimbabwe’s Continuous Quality Improvement Innovation

Combined Qualitative and Quantitative Process Evaluation Report

FINAL

This report was prepared by Jed Friedman (Senior Economist/Principal Investigator), Ronald Mutasa (Senior Health Specialist/Team Leader), Tamar Gotsadze (Lead Qualitative Research Consultant), Anna Gage (Consultant), Felicia Takavarasha (Consultant) and Crecentia Gandidzanwa (Consultant). The study and report were conceptualized, conducted and analyzed with technical input from Son Nam Nguyen (Lead Health Specialist), Christine Lao Pena (Senior Economist/ Zimbabwe Health Sector Development Support Project Team Leader), Aditya Khaparde (Consultant), Mildred Pepukai (Consultant) and Chenjerai Sisimayi (Health Specialist). Leah Jones (Knowledge Management Consultant), Farai Sekeramayi (Program Assistant) and Yvette Atkins (Senior Program Assistant). The team benefited from the general guidance of Ernest Massiah (Practice Manager), Monique Vledder (Practice Manager for the Global Financing Facility), Magnus Lindelow (Practice Manager) and Mukami Kariuki (Country Manager, Zimbabwe). The technical leadership of the Ministry of Health and Child Care, Zimbabwe in conceptualizing and implementing the CQI innovation and in guiding the process evaluation is duly acknowledged. Cordaid’s role in conceptualizing the CQI initiative and supporting the roll-out, mentorship of MOHCC staff at various levels is greatly acknowledged. Cordaid’s contribution to the process evaluation is equally acknowledged. The financial support and leadership of the Global Financing Facility and Ministry of Finance is duly acknowledged.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

2

Executive Summary

Introduction

This report provides findings and recommendations based on the Process Monitoring Evaluation of the Continuous Quality Improvement (CQI) intervention being implemented under the World Bank financed Health Sector Development Support (HSDS) Project (P125229). This report includes both qualitative and quantitative components of the process evaluation to understand the extent to which the CQI initiative has improved quality of care structures and processes and facilities, and to identify the enablers and barriers to change.

The CQI initiative was piloted in 2016 as an arm of the Zimbabwe RBF program in order to improve quality of maternal and child health care services. The CQI intervention in Zimbabwe is designed to build capacity of facility health care teams, district managers, and supervisors to continuously improve care and thereby strengthen the capacity of the system to deliver high quality care. CQI regularly monitors quality outcomes to track progress against quality targets set by health facility teams at primary and secondary levels of care and their supervisors at district and provincial levels, a course of action is devised to improve these targets, and necessary actions are taken to achieve these targets.

Key components of the CQI intervention at facility and district management levels include formation of and regular support to CQI teams in participating facilities in order to: i) Define measurable improvement aims for priority best practices; ii) Discuss, review, and prioritize changes to routine care processes to improve adherence with incentivized best practices; iii) Collect and analyze monthly clinical quality measures used in the RBF project to assess progress against defined improvement aims; and iv) Ensure regular posting of results in relevant patient care areas for public information.

In addition, a component of capacity building and mentoring of provincial and district managers and supervisors was designed to: i) Support facility quality improvement teams to regularly carry out CQI activities; ii) Ensure regular competency-based and refresher training of skilled providers; iii) Oversee local supply chain (in close communication with facility managers) to prevent stock outs of essential commodities; iv) Audit medical records and observe simulated performance of priority clinical procedures to assess quality of care during supervision visits; and v) Track facility-specific results for priority clinical quality indicators (incentivized in the RBF program) to tailor support to facilities and clinical areas demonstrating poor performance on priority indicators.

Methodology

This report addresses three primary research questions on the CQI intervention: 1. What is the effect of a quality improvement model on quality of care? 2. What are the factors that influenced observed changes? 3. What is the effect of a quality improvement model on how health facility teams

organize to improve quality of care?

Quantitative methods were used to explore the first question, while qualitative work addresses the second and third questions.

CQI is being piloted in five of the sixteen RBF districts, namely: Mashonaland Central Province-Centenary District, Manicaland Province-Chipinge District, Masvingo Province-Mwenezi District, Matabeleland North Province-Binga District and Matabeleland South Province-Mangwe District. In each of the five districts, one District Hospital and five health clinics were purposively selected for initial CQI implementation.

3

The qualitative process evaluation was implemented in 2 phases. In the first phase the evaluation team i) reviewed project related documents, ii) developed process evaluation methodology and tools, iii) sampled health facilities for site visits and collected qualitative information using various data collection methods, iv) constructed stakeholder map and produced inception report covering evaluation methodology, sampling of provinces, facilities and respondents, implementation phases, and schedule and draft outline of the final evaluation report. The second phase was subject to the availability of quantitative PME results collected separately from the qualitative. In this phase, the research team reviewed preliminary results of the quantitative process evaluation and jointly with the quantitative team elaborated exploratory research questions, respective data collection tools, and research protocols.

For the field data collection, the research team collected qualitative information and shared preliminary findings and recommendations with key stakeholders. Data collection included a mix of desk-based research, key stakeholder in-depth interviews (IDIs) and Focus Group Discussions (FGD). Based on the desk review findings, information gaps were identified, and related questions included in the IDI and FGD guides. IDIs were implemented face to face for key informants from central, provincial, district, and facility levels. IDI interview topic guides based on research questions, helped ensure systematic coverage of research questions and issues. FGDs were carried out for hospital CQI committees. FGD guides included questions specific to the FGD participant group and included 8-10 participants per each group. In health care centers with small number of staffed the FGDs were replaced by small group interviews. 232 in depth interviews were conducted at national, province, district and community levels in selected three provinces at both CQI and non-CQI facilities. Apart from the IDIs, researchers also carried out 15 FGDs and 22 group interviews.

Qualitative data analysis focused on synthesizing and triangulating information from the various data sources and evaluation methods. The evaluation took an iterative approach based on grounded theory that allows themes and findings to emerge from the data.

The quantitative evaluation compares quality-of-care outcomes in CQI intervention and control facilities. Data from the CQI quality checklist was used for the quantitative process evaluation. A baseline quality checklist was collected for all health facilities in the 18 study districts in the last quarter of 2016, and then the intervention began in the first quarter of 2017. The primary healthcare checklist contained data on 153 individual quality-of-care related outcomes. These were grouped into thirteen aggregate measures assessing the quality of inputs and seven aggregate measures assessing the quality of processes. The hospital checklist contained data on 253 indicators, which were grouped into 17 aggregate measures assessing the quality of inputs and nine aggregate measures assessing the quality of processes. All aggregate measures were calculated as the percentage of the underlying individual indicators (equally weighted) achieved by the facility. As the CQI intervention was intended to affect process quality more than inputs, the aggregate process quality indicators are presented as the main results.

For both the primary healthcare and hospital outcomes, an average of the four quarters in 2018 was used as the post-intervention timepoint for most analyses in order to (a) allow for the intervention, begun at the start of 2017, to take hold and achieve its intended aims, and (b) smooth variability, including possible seasonal variability, between quarters.

ANCOVA models were used to assess the effect of CQI on the quality outcomes, comparing CQI versus non-CQI facilities while controlling for the facility’s baseline quality. As this process evaluation is quasi-experimental in nature, two comparison groups are used for the primary healthcare analysis. The first comparison group is facilities that did not receive the CQI intervention within implementation districts, while the second comprises of facilities from the other thirteen RBF districts that did not implement CQI.

4

Qualitative results

The CQI pilot is perceived as one of the most important initiatives for improving the quality of MNCH services in Zimbabwe. There is a clear agreement among actors involved in CQI pilot about the significance of quality improvement as a fundamental tool to improve health outcomes. According to respondents, the CQI pilot was one of the first and few initiatives that ensured staff exposure to quality improvement, and which strategically and deliberately included a component of local problem analysis and planning of corrective measures.

The evaluation team noted several important factors facilitating and or impeding

achievement of CQI program results. Enabling factors included:

1. GoZ political will to promote quality improvement in the health sector set an enabling environment.

2. The CQI intervention increased awareness of and participation in CQI activities and processes, and enabled CQI facilities to accelerate improvements in service provision and clinical quality.

3. External supportive supervision was viewed as important and useful in guiding the health facilities to focus on areas requiring attention and contributed to staff capacity building and knowledge sharing.

4. CQI fostered leadership, teamwork, and joint decision-making at intervention sites, and QI teams use selected CQI tools in their daily practice.

5. Staff pay for performance had marginal effect on staff motivation to improve care quality at both intervention and non-intervention sites. Rather, improved working environment and external supportive supervision had positive effect on staff motivation.

6. CQI encouraged a virtuous cycle in which improved performance encourages staff to further advance their performance.

7. Addition key components that supported staff knowledge, empowerment and motivation included peer review and peer support practices, knowledge exchange platforms and stronger linkages between facilities and their communities.

Impeding factors included both external and internal factors. External factors included:

1. Fragmentation of national and provincial level QA/QI results in inefficiencies and ineffectiveness of interventions. This fragmentation is mirrored at province, district, and health facility levels and CQI is treated as one among other government programs.

2. Attempts to integrate all QI interventions into one comprehensive QI plan failed at province, district, and district hospital levels.

3. National monetary policies introduced in 2019 inhibit effective implementation of CQI at intervention and non-intervention sites.

4. Delays in payment of RBF subsidies result in reduction of purchasing power and shortages in the procurement of goods for quality improvement.

5. Human resource for health planning along with a government freeze of employment resulted in staff shortages and adversely affected quality of care.

Internal factors included:

6. The effectiveness of CQI trainings was challenged by the training content, intensity, and mode of delivery. There was a lack of balance between theoretical knowledge transfer and practical skill development during the training course. Furthermore, due to high turnover staff trained in CQI moved to other facilities and/or locations, and their replacements lacked

5

the knowledge of CQI. The lack of clinical training opportunities to complement the CQI training impedes MNCH quality improvement.

7. The effectiveness of supportive supervision and mentoring/coaching from provincial and district levels was confronted by a number of challenges, including staff shortages at provincial and district levels and limited time spent at the health facilities.

Quantitative results

At baseline, most facilities performed well on the input measures but had poorer performance on the process measures. This is likely due, at least in part, to the participation of all study districts in the RBF program. For example, across all facilities, facilities had on average 79% of the general equipment items such as radio, mosquito nets and adult weighing scales. Primary health centres by far performed the worst on obstetric complications, with facilities only completing 3% of the items on average. The majority of summary outcomes were balanced between CQI and both groups of control facilities at baseline. Where the indicators differ, CQI facilities generally performed better than the control facilities in other districts.

CQI was associated with improvement in the primary health centres for two of the seven process measures: postnatal care (PNC) and maternal care. The intervention was associated with completing one half of an additional PNC process item correctly and one additional maternal process item. The PNC improvement is driven primarily by better assessment in the CQI facilities: facilities were 25 percentage points more likely to have all women checked for their general condition, pulse rate and temperature in comparison to the control facilities. Among the maternal care process indicators, monitoring for women and newborns in the 4th stage of labor, administering immediate postpartum oxytocin, and monitoring women using partographs were the indicators that improved the most between CQI and control facilities.

When comparing the primary care intervention facilities against control facilities in their

districts maternal care quality improvement is no longer significant at standard levels, although the coefficients are positive and of similar magnitude. Post-natal care quality still shows significant improvement, though the magnitude of the improvement is smaller than what was observed using the other districts as comparison. Given the robustness checks conducted, it suggests that the CQI intervention did improve post-natal care quality in the intervention facilities, and likely maternal care-quality as well. However, it did not improve the other process quality measures including ANC, PMTCT, sick child assessment, sick child treatment, or obstetric complications quality. In addition, as expected given the focus of the CQI activities, no improvement in input quality measures was observed.

6

Trend in PNC process quality over study period in PHC CQI and control facilities in 13 other districts

Five hospitals implemented CQI, and data was available for at least some indicators from

sixteen other hospitals. Hospitals overall performed better on the quality of inputs measures than the quality of processes at baseline. Process quality was generally lower than input quality in 2016, although facilities still scored highly on both maternal care processes and paedatric care processes. Hospital performance was generally higher than primary health centre performance at baseline, although the measures are not directly comparable due to different indicators included in the summary measures. There were few differences in outcomes between CQI hospitals and control hospitals in 2016 but where there were, CQI hospitals had better performance.

The study is not powered to find significant associations among the limited number of

hospitals, and so unsurprisingly there are no significant associations. Even given the limited sample size, however, many outcomes have CQI coefficients close to zero, suggesting that there may not be significant effects even with larger sample sizes. The largest positive coefficient is for sick child treatment quality. Despite the lack of associations in the summary outcomes, CQI was significantly associated with a few of the individual input quality outcomes. For example, CQI was associated with a 31 percentage point increased likelihood of having an appropriate waste pit, 19 percentage point increased likelihood of having second line paediatric ART medications, and 47 percentage points more likely to have a functioning ultrasound machine. However, CQI was also associated with a 36 percentage point decreased likelihood of correct treatment for sepsis, and had no significant associations for the vast majority of indicators.

Conclusions There is some evidence that CQI improved maternal care and postnatal care processes in

PHCs, but no evidence of improvement in any inputs, as expected, or any of the hospital processes. Improvements were limited when considering the full breadth of potential outcomes but arise in certain areas of focus of the CQI program.

This absence of broad-based improvement may be due to several factors. First, CQI facilities were already performing better than control facilities at baseline on many indicators. Second, many structural indicators were already very high at baseline. In combination, these suggests that there may

7

have been ceiling effects where CQI facilities were unable to improve anymore but control facilities were. Third, the qualitative results suggest that there was mixed success among different CQI facilities as the intervention interacted differently with the environment, the service, the quality improvement process and the stakeholders. The overall grouping of CQI and control facilities may have masked heterogeneity of improvement, with some facilities performing better.

The mixed success among different CQI facilities is associated with the interaction of features of the environment, the service, the quality improvement process, and the stakeholders, which operated together to produce a set of circumstances that either inhibited or enabled the process of change. Performance improvements would have been impossible without strong leadership of management teams who drive the process of quality improvement and establishment of teamwork culture. Availability of skilled health workforce—and building their capacity with training for continuous knowledge and skills development along with supportive supervision—are the most influential factors behind care quality improvements.

General health system related bottlenecks, including high staff turn-overs and staffing

shortages, challenged effectiveness of the CQI pilot. Furthermore, a sub-optimal medical and nursing education system to produce professional health workforce and limited continuous medical education opportunities. Specific challenges to the implementation of CQI included fragmentation of QA/QI functions, gaps in building health worker knowledge and skills, suboptimal supervision and delays in RBF payments.

The qualitative results indicate that the CQI facilities saw greater leadership, teamwork,

ownership of clinical quality improvement, engagement, and staff motivation. These could be important outcomes in and of themselves but may not necessarily translate into higher quality care. This reveals a major limitation of the CQI approach: some items are not within the power of the facility staff to solve through continuous quality improvement. For example, the CQI methodology cannot address the challenges with staff shortages, high staff turnover and high workload. These problems may greatly affect care processes but need to be addressed at the national or district levels. The quantitative results also show the extremely low performance of the PHCs on managing obstetric and newborn complications. Improved staff motivation and teamwork will not overcome challenges of lack of training or practice on these skills, rather it suggests that to improve quality, only hospitals should handle these complications.

These findings align with a broader literature indicating that micro strategies to improve quality at the level of providers or facilities will likely be insufficient by themselves. Rather, macro and meso level strategies are likely needed at the national, provincial and district levels in order to improve overall health system quality. The large gaps observed in the quality of care cannot be addressed through targeted quality management when the root causes may be high workload, large turnover, inadequate clinical training, or simply providing a service at an inappropriate level of the system. Macro or meso level strategies to address these challenges would include addressing gaps in governance, reorganizing services to be provided at the appropriate level (i.e. moving deliveries to hospitals that can manage obstetric complications), strengthening pre-service clinical education, or establishing facility learning networks. While CQI may contribute to a better working environment and spur improvement in routine care practices that the staff are already well trained in, it should be seen as one tool in a broader health systems quality improvement strategy.

8

Table of Contents

1. Introduction............................................................................................................. 10

2. Country Context ....................................................................................................... 10

2.1 Zimbabwe’s Health Sector................................................................................................. 10 2.1.2 Quality of care challenges ................................................................................................................... 11

2.2 Health delivery system ...................................................................................................... 12

2.3 Quality assurance and improvement in Zimbabwe ............................................................. 12

3. Continuous Quality Improvement............................................................................. 13

3.1 Summary of Evidence on PBF and CQI ............................................................................... 15

3.2 CQI in Zimbabwe .............................................................................................................. 16

3.3 A theory of change for quality improvements at the primary clinic level ............................ 17

3.4 Intervention questions and objectives ............................................................................... 18

4. Methodology ........................................................................................................... 18

4.1 CQI Implementation ......................................................................................................... 18

4.2 Selection of CQI implementing health facilities .................................................................. 19

4.3 Qualitative process evaluation .......................................................................................... 20 4.3.1 Qualitative data collection .................................................................................................................. 21 4.3.2 Qualitative sampling ........................................................................................................................... 22 4.3.2 Qualitative data analysis and triangulation ........................................................................................ 23 4.3.3 Qualitative research ethics ................................................................................................................. 23 4.3.4 Qualitative study limitations ............................................................................................................... 23

4.4 Quantitative process evaluation ........................................................................................ 23 4.4.1 Quantitative data collection and extraction ....................................................................................... 23 4.4.2 Quantitative outcomes adopted in the analysis ................................................................................. 24 4.4.3 Quantitative data analysis .................................................................................................................. 24

5. Qualitative Results ............................................................................................... 26

5.1 Quality improvement results ............................................................................................. 26

5.2 Factors influencing observed changes................................................................................ 28 5.2.1 Enabling factors .................................................................................................................................. 28 5.2.1 Impeding factors ................................................................................................................................. 33

6. Quantitative results .................................................................................................... 39

6.1 Primary healthcare facility results ..................................................................................... 39

6.2 Hospital results ................................................................................................................. 43

7. Conclusions and lessons learned .............................................................................. 48

7.1 Drivers of change .............................................................................................................. 49

7.2 Barriers to change ............................................................................................................. 50

7.3 Role of CQI interventions .................................................................................................. 53

9

7.4 Lessons Learned ............................................................................................................... 54

8. Recommended Approach to CQI Scale-up and Institutionalization ............................ 55

8.1 Essential elements for vertical CQI scale-up and institutionalization ................................... 55

8.2 Recommendations for horizontal CQI scale-up .................................................................. 58

Annex 1: RBF Package of quantity services ...................................................................... 60

Annex 2: Exploratory Research Questions ....................................................................... 61

Annex 3: Additional tables & figures ............................................................................... 63

Annex 4. Continuous Quality Intervention checklists for PHCs and Hospitals .................. 114

10

1. Introduction

In July 2011, the Government of Zimbabwe (GOZ) piloted a health sector results-based financing (RBF) program with grant support from the Health Results Innovation Trust Fund1 and with Ministry of Finance and Economic Development (MOFED) co-funding. The RBF program supports the Ministry of Health and Child Care (MOHCC) to increase the availability, accessibility, and utilization of quality health care to improve maternal, neonatal, and child health. The program piloted in two districts in July 2011 and expanded to 415 health facilities across 16 additional districts in March 2012. In 2014, the MOHCC scaled up RBF to cover 44 rural districts and the two urban districts of Harare and Bulawayo with Health Transition Fund2 support. The program has three components: (i) results-based contracting; (ii) management and capacity building; and (iii) monitoring. Under the first component, a portion of financing received by health facilities depends on the quantity and quality of services, with a focus on maternal and child health (MCH). Annex 1 summarizes the package of services linked to RBF incentives. In addition to introducing incentives, the program abolished user fees on a package of services in each district, with the aim of improving access to care.

Improving the quality of care helps to avert adverse health outcomes among targeted populations and may attract more households and patients to seek health care, thereby increasing utilization of services3. Empirical evidence from the first phase of the Zimbabwe RBF impact evaluation in 2016 showed RBF’s positive impact, but also its mixed effects on quality of care, signaling policy makers and development partners to explore innovations to improve quality of care in line with Zimbabwe’s National Quality Strategy 2016 vision.

In 2016, following results of the first phase of the RBF program impact evaluation, the MOHCC requested World Bank support to design and pilot a continuous quality improvement (CQI) innovation to be rolled out in select facilities implementing the RBF program. The CQI innovation was introduced with the goal of improving overall quality of care outcomes—in particular clinical process measures—that had not been very responsive to the RBF intervention.

As part of the Process Evaluation for Zimbabwe’s RBF program, the World Bank commissioned an assessment of the CQI innovation. This assessment was undertaken with technical input and guidance from the MOHCC and Cordaid. This overview report—which incorporates both qualitative and quantitative evidence on the impact of CQI on health care delivery—was developed at the request of MOHCC senior management. The purpose of the report is to inform future directions of the CQI intervention by documenting implementation factors that might contribute to its success or challenges and by offering a synthesis of the quantitative evidence on the effect of the CQI intervention.

2. Country Context

2.1 Zimbabwe’s Health Sector Zimbabwe’s population is estimated at 13,061,2314, with a life expectancy at birth (LEB) of 61

years in 2017. The total fertility rate is 4.1 children per woman. According to the WHO 2015 country burden of disease profile5, at least three quarters of Zimbabwe’s annual deaths can be attributed to communicable, maternal, perinatal, and nutritional illness. A major area of concern for Zimbabwe has been the high maternal mortality ratio (MMR), which reached 960 deaths per 100,000 live births in

1 This is a multidonor trust fund administered by the World Bank and funded by the Governments of Norway and United Kingdom to pilot innovative approaches to accelerating progress in maternal, neonatal and child health outcomes. 2 This is a multidonor trust fund administered by UNICEF established to support the recovery of the health sector in Zimbabwe. 3 Shroff CZ, Bigdeli M, Meissen B. From scheme to system (part 2): Findings from ten countries on the policy evolution of results-based financing in health systems. Health Systems & Reform. 2017;3:137-47 4 Zimbabwe Population Census 2012, National Report, ZimStat 5 WHO Country Profile: http://www.who.int/nmh/countries/zwe_en.pdf?ua=1

11

2010-2011 but dropped to 462 deaths per live births by 20196. However, the pace of progress is not fast enough to achieve Sustainable Development Goal (SDG) targets. In addition, despite some progress since 2009, infant and under-5 mortality rates were estimated at 52 per 1,0001 live births and 72 per 1,0001 live births, respectively. One-third of children under five are stunted, and there has been little improvement in these figures over the last decade. In addition, the country has in the last decade been increasingly challenged by the dual burden of communicable and non-communicable diseases.

A major financial burden of health care falls on households in the form of out-of-pocket payments—rendering the health system inequitable and inefficient. Access to health services and subsequent outcomes have largely been inequitable with poor and rural populations shouldering a disproportionate burden of disease and health risks.

Cycles of fragility and macroeconomic challenges, compounded by climatic shocks in the last few years have led to cash shortages, affecting the prices of goods and services and, in turn, health providers’ ability to execute planned activities. The worsening economic conditions have also strained the health workforce, a key pillar of the health system. Salaries of the health workforce have been eroded due to inflation, resulting in a demotivated health workforce leading to poor health service delivery and uncertainties in retention of staff. In the last half of 2019, the country experienced the most protracted industrial action by medical doctors it has ever faced, with over 500 junior doctors absconding from duty for over three months citing incapacitation.

Current patterns in the utilisation of health services and quality of care reflect a heightened risk of reversal towards poor health outcomes reminiscent of the pre-2010 era. Although results from the MICS 2019 indicate a continued positive trajectory for the maternal mortality ratio, an increase in the neonatal mortality rate signifies continued gaps in utilization in quality. The underlying patterns of health service utilisation and quality of care in the reference period for the study (2015-2018) are different from those pertaining to 2019 and likely going forward.

The response to the current challenges is limited as both domestic and external funding remain constrained. Whilst the share of Government public health financing has been significant to total health spending, the contributions have been below regional and international thresholds. The sector has continued to experience a financing gap relative to the costed needs, and current forecasts of domestic and external funding streams as reflected in resource mapping data are indicative of limited prospects to address this gap in the short term. In the absence of a coordinated response to address some key priority high impact interventions, the system will fail to activate the essential structural pillars necessary to sustain the gains witnessed to date, let alone withhold the downward spiral in the availability and quality of the critical health services. The sector can leverage on a number of low hanging fruits amongst key interventions. However, the current macro-economic environment and extent of the deterioration in health service delivery suggest modest improvements, at the very best a rebounding to levels to sustain the gains witnessed in the last decade, should these interventions be implemented.

2.1.2 Quality of care challenges Major gaps in quality of health care services in Zimbabwe are substantiated by several

household and facility surveys (DHS 2005, 2010/11, 2016, USAID MCHIP study 2012). A 2016 study of the quality of MCH services demonstrated important quality gaps for both routine maternal new-born services and for the leading causes of maternal, newborn, and child mortality (i.e. post-partum haemorrhage [PPH], eclampsia, maternal and newborn sepsis, newborn asphyxia, newborn prematurity, child pneumonia, diarrhoea, and acute malnutrition)7. In addition to being overstretched

6 2019 Multiple Indicator Cluster Survey, Zimbabwe National Statistics Agency 7 Hill, K, Quality of Maternal, New Born and Child Health in Zimbabwe (World Bank Report, 2016).

12

in understaffed facilities, providers often do not have the skills or the confidence to manage common life-threatening complications. District and regional administrative supervisors often lack up-to-date clinical knowledge and skills to assess and support provider competence in priority clinical areas. There is also no system of clinical certification maintenance.

In many clinics and hospitals, medical records and registers do not document core elements of basic quality care, such as vital signs and physical examination findings. For example, outpatient pediatric and adult registers often include only the patient name, main complaint, diagnosis, and medication prescribed without any results of vital signs, physical exam, or duration and severity of symptoms, etc. Indeed, providers face the burden of completing as many as ten to 15 different clinical registers to meet reporting requirements, yet there is very little integration of essential data. As in many countries, Zimbabwe’s health management information system (HMIS) primarily records coverage information with regard to health care services, with infrequent inclusion of clinical quality measures even for high-impact clinical interventions, such as immediate post-partum oxytocin for prevention of PPH. There is limited MOHCC capacity to support quality improvement (QI) activities at any level of the health system and rare use of QI to influence care in clinics and hospitals.

2.2 Health delivery system Zimbabwe’s public health delivery system, which takes a primary health care centered

approach, is the responsibility of the MOHCC, local governments, and not-for-profit faith-based organizations. Zimbabwe’s current health system delivery has four levels.

(i) Primary level: A total of 1,634 primary care facilities provide basic health promotion, prevention, and curative and rehabilitation services to their catchment populations. These facilities are also responsible for environmental sanitation, water supplies, hygiene, waste disposal systems, control of communicable diseases, nutrition, mental health, and disabilities. They are also the point of reference for village health workers (VHWs). Primary health facilities normally have at least two nurses, one of whom should be a skilled midwife. The staff complement should also include an environmental health technician (EHT). Normally, primary health facilities refer complicated cases to district hospitals.

(ii) Secondary level: This level consists of district hospitals, which take referrals from primary care facilities. These hospitals offer medical interventions that include Caesarean sections and blood transfusions.

(iii) Tertiary level: Zimbabwe has eight provincial hospitals, which comprise the tertiary level of care. One provincial hospital is run and owned by the church, while the GOZ owns and runs the other seven. Provincial hospitals provide health services to patients referred from district hospitals.

(iv) Quaternary level: The quaternary level offers highly specialized hospital services; these hospitals are only found in Harare and Bulawayo. Normally referred to as central hospitals, these provide services to patients with complications as referred from the provincial hospitals.

While in rural areas mission facilities play a considerable service, in urban areas private health providers are major providers of health services. Nationally, mission hospitals and private clinics account for 35% of health care delivery, while about 65% of health care services are provided by the public sector.

2.3 Quality assurance and improvement in Zimbabwe Equity and quality are fundamental MOHCC healthcare principles. In 2010, the MOHCC

established a Quality Assurance and Quality Improvement (QA/QI) Directorate under the division of Policy, Planning, and Monitoring and Evaluation. The department was designed to oversee the

13

MOHCC’s healthcare quality assurance and improvement efforts, including creation and/or adherence to quality policies, guidelines, and standards.

Health sector quality improvement in Zimbabwe is regulated by a QA and QI Policy8 approved by the MOHCC in 2015. The strategy takes a systems approach to improving and sustaining high-quality health services in Zimbabwe, highlighting cross-cutting priority intervention areas related to the World Health Organization’s (WHO) health system building blocks and priority quality domains (e.g. safety, clinical effectiveness), and priority interventions in vertical disease control programs targeting major causes of morbidity and mortality. The policy guides the process of ensuring quality of care as well as CQI in both public and private health sectors; it also provides guidance for capacity building and leadership for QA/QI processes at all levels of the health system through adoption of proven change management methodologies.

Mechanisms for implementing the QA/QI policy include: creation of a National Advisory Committee and technical working groups at MOHCC; creation and support of provincial and district QI focal points; regular monitoring of priority quality measures tailored to specific health system stakeholders; as well as provisions for monitoring of the policy implementation.

At national level, a National QI Committee–with representation from senior management within the MOHCC and other major stakeholders—oversees and guides institutionalization of QA/QI processes to support health care service delivery. The National QI Committee drives quality improvement and ensures that QI becomes a nationwide continuous system-wide approach.

The National QI Committee has five main areas of responsibility:

1. Strategic Planning: Prioritizing goals so the most critical areas are addressed first.

2. Overseeing QI implementation: Ensuring that all quality improvement activities are performed effectively in line with key quality priorities.

3. Guidance: Overseeing activities to ensure that they are on track and responsive to staff, clients, and partners during the improvement process. This guidance includes support and encouragement of the provincial level to maintain QI momentum.

4. Resource mobilization: The national QI committee is responsible for building sustainable infrastructure that fosters a culture of quality service delivery.

5. Enforcing accountability: The committee tracks and reports on implementation progress, using a set of agreed indicators and adjusting implementation course when necessary.

Implementation of the QI strategy utilizes existing structures and systems, as much as possible, enabling QI to take off with minimal additional resources, and encouraging QI activities to be embedded and integrated within the normal MOHCC operational environment.

3. Continuous Quality Improvement

In 2014, the MOHCC approved a National QI Strategy, which provided a basis and rationale for introducing the Continuous Quality Improvement (CQI) innovation embedded within the performance-based financing scheme (Box 1). This approach enabled the GOZ to further strengthen improvements in quality of care, building on the widely successful effects of the RBF mechanism to strengthen coverage and structural quality of services provided in the country. The CQI component of the RBF project was designed to build the capacity of facility health care teams, district managers and supervisors, to continuously improve care and thereby strengthen the capacity of the system to deliver high quality care. The CQI component of the National QI Strategy defined a set of priority conditions for which it seeks to improve the quality of care. Conditions were prioritized based on their contribution to the burden of mortality and morbidity, as well as on evidence of effective health care

8 Quality Assurance and Quality Improvement Policy, 2015, Ministry of Health and Child Care Zimbabwe

14

interventions (preventive and curative). Quality of care measures were integrated into established checklists for various aspects of maternal, newborn, and child care.

Following MOHCC approval of the National QI Strategy, the CQI initiative was piloted in 2016 as an arm of the Zimbabwe RBF program. Continuous quality improvement is “a management philosophy that organizations use to reduce waste, increase efficiency, and increase internal (meaning employee) and external (meaning customer) satisfaction. It is an ongoing process that evaluates how an organization works and ways to improve its processes.”9 CQI regularly monitors quality outcomes to track progress against quality targets set by health facility teams at primary and secondary levels of care and their supervisors at district and provincial levels, a course of action is devised to improve these targets, and necessary actions are taken to achieve these targets.

Box 1: The Continuous Quality Improvement Model

The premise of the CQI model is that a change or series of changes is needed to improve quality of health care services. Managing the change process is therefore central and involves four steps of the Plan-Do-Study-Act (PDSA) cycle. In each cycle, a CQI team collaborates on a series of steps:

• Plan: The team plans a change to improve the quality of services it offers

• Do: The team executes the plan and documents what is working and any unexpected developments

• Study: The team analyzes the results in relation to original objectives

• Act: The team decides on next steps. If the actions were successful, the team may introduce it at a larger scale; if they are not successful, the team may decide to discard the model or adapt changes to make it work more successfully.

The summative effect of the CQI team testing changes, identifying effective ones, and adopting solutions should help improve quality and strengthen the health system overall. CQI teams are made up of supervisors, front-line health care workers, and staff with the knowledge of their local systems necessary to be able to identify and test feasible and sustainable changes to “usual processes” to improve care in their local setting. CQI teams regularly monitor performance with adherence to clinical best practices in the areas of improvement targeted by the project and track the improvement of quality outcomes against a quality target.

This model has two types of CQI clinical audits: (i) audit of maternal-neonatal deaths and near-miss10 events allowing teams to reflect on, understand, and learn from rare, catastrophic (or near-catastrophic) events through peer review of cases that caused concern, affected patient safety, or resulted in an unfortunate outcome; and (ii) monthly clinical audits, which are systematic reviews of patient charts to determine care given in relation to the standard of care. These two types are carried out internally by site for monthly monitoring and externally for data validation.

9 https://study.com/academy/lesson/what-is-continuous-quality-improvement-definition-process-methodologies.html 10 Near miss (World Health Organization): Women who survive life-threatening conditions arising from complications related to pregnancy and childbirth have many common aspects with those who die of such complications. This similarity led to the development of the near-miss concept in maternal health. Exploring the similarities, the differences and the relationship between women who died and those who survived life-threatening conditions provides a more complete assessment of quality of maternal health care.

https://study.com/academy/lesson/what-is-continuous-quality-improvement-definition-process-methodologies.html

15

3.1 Summary of Evidence on PBF and CQI

While performance-based financing schemes are primarily aimed at improving demand and access to health care, PBF schemes implicitly or explicitly address quality of care, especially through their payment formulas11. Studies on PBF schemes in low- and middle-income countries (LMICs) show that quality of care is often included directly or indirectly in program implementation or design12. Quality improvement mechanisms also help prevent health facilities from comprising health care quality as they aim to meet their quantity targets. At the same time, targets set in the SDGs—specifically SDG 3, which measures health using outcome quality indicators—drove the need for health financing programs to prioritize quality of care in health delivery systems13.

Among LMICs, many PBF programs have begun integrating quality into their programs, but not yet in a uniform or systematic manner14 15. While PBF has been linked to mixed results in workers’ morale and demand of health services, there is a general lack of conclusive data to measure the improvement of quality within PBF programs. Evidence on the impact of PBF on quality is scarce as there is no uniformity of documentation of quality in PBF programs. Standardization of data collection around PBF would help researchers better interpret impact estimates and provide practical guidance to policy makers. This dearth of knowledge and evidence on quality improvement, coupled with limited documented success of the PBF in improving quality of care in LMICs, motivated the CQI pilot in Zimbabwe.

One study in Ghana suggests that CQI improved health outcomes. CQI helped to diagnose problems, recommended solutions, and charter progress on a color-coded scoring system. Staff were provided regular feedback on performance targets and achievements. This initiative showed a 34% decrease in maternal mortality despite increased patient admissions to the study hospital. There was a reduction in case fatality rates for pre-eclampsia and hemorrhage from 3.1% to 1.1% (p<0.05) and from 14.8% to 1.9% (p<0.001), respectively. Stillbirths declined by 36% (p<0.05)16. Over a two-year period, the MMR decreased from 496 per 100,000 live births to 328 per 100,000. In South Africa, CQI—specifically of HIV/AIDS services—increased antenatal HIV testing by almost 10 percentage points, i.e., from 89% to 98%. CD4 testing of HIV positive mothers increased from 40% to 97%, maternal nevirapine from 57% to 96%, and infant nevirapine from 15% to 68%17. In Kenya, CQI improved adolescents and young adults’ knowledge on HIV prevention and transmission knowledge, and a trend towards improved intent to retest at one clinic18.

11 Fritsche GB, Sr, Meessen B. Performance-Based Financing Toolkit. Washington, DC: World Bank; 2014. https://openknowledge.worldbank.org/handle/10986/17194 Accessed 23 December 2019. 12 Quality of Care in Performance-Based Financing: How It Is Incorporated in 32 Programs Across 28 Countries. Jessica Gergen, Erik Josephson, Martha Coe, Samantha Ski, Supriya Madhavan, Sebastian Bauhoff Glob Health Sci Pract. 2017 Mar 24; 5(1): 90–107. Published online 2017 Mar 15 13 Sustainable Development Goals. Goal 3: Ensure healthy lives and promote well-being for all at all ages. United Nations; http://www.un.org/sustainabledevelopment/health/ Accessed December 23, 2019. 14 Witter S, Fretheim A, Kessy FL, Lindahl AK. Paying for performance to improve the delivery of health interventions in low- and middle-income countries. Cochrane Database Syst Rev. 2012;(2):CD007899. 10.1002/14651858.CD007899.pub2. 15 Meessen B, van Heteren G, Soeters R, Fritsche G, van Damme W. Time for innovative dialogue on health systems research. Bull World Health Organ. 2012; 90(10):715–715A. 10.2471/BLT.12.112326. 16 E.K. Srofenyoh ,N.J. Kassebaum, D.M. Goodman, A.J. Olufolabi, M.D. Owen. Measuring the impact of a quality improvement collaboration to decrease maternal mortality in a Ghanaian regional hospital. International Journal of Gynecology and Obstetrics 134 (2016) 181–185 17 Bhardwaj S, Barron P, Pillay Y, et al. Elimination of mother-to-child transmission of HIV in South Africa: rapid scale-up using quality improvement. S Afr Med J. 2014;104(3 Suppl 1):239–43. 18 Wagner AD, Mugo C, Bluemer-Miroite S, et al. Continuous quality improvement intervention for adolescent and young adult HIV testing services in Kenya improves HIV knowledge. AIDS. 2017;31 Suppl 3(Suppl 3):S243–S252. doi:10.1097/QAD.0000000000001531

https://openknowledge.worldbank.org/handle/10986/17194 Accessed%2023%20December%202019

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5493453/

http://www.un.org/sustainabledevelopment/health/

16

3.2 CQI in Zimbabwe

For all facilities participating in the Zimbabwe RBF program, quality supervision checklists were revised in 2016 to include clinical quality of care indicators related to delivery of proven best practices for improving MCH in Zimbabwe. These revised quality checklists were linked with incentives in all RBF facilities, including both treatment and control group facilities in the process evaluation, and hence constitute part of the control against which the CQI interventions are assessed.

The CQI intervention study, implemented in addition to the quality of care changes above, included the following steps: (a) introducing CQI processes to MOHCC-selected facilities and districts; and (b) further incentivizing selected CQI process measures to improve use of standardized checklists, charts, and forms by health workers to document the health worker/patient interface. Through implementation of high-impact quality improvement interventions, the CQI initiative aims to decrease morbidity and mortality of: pregnant women during labor and delivery and postpartum; neonates during the early neonatal period; and under-5s from common conditions. CQI was also designed to build capacity of facility health care teams, district managers, and supervisors. By continuously improving care CQI intends to strengthen the capacity of the health system to deliver quality care.

Figure 3.1: Design of the RBF Project with CQI component

Key components (Figure 3.1) of the CQI intervention at facility and district management level

include the formation of, and regular support to, CQI teams in participating facilities in order to:

• Measure improvement for priority best practices (e.g. essential newborn care, prevention/management newborn sepsis and newborn asphyxia, PMTCT priority services)

• Discuss, review, and prioritize changes to routine care processes to improve adherence to incentivized best practices, i.e., quality measures (e.g. division of essential tasks among staff; re-organization of services to be more efficient; posting of job aids; regular clinical update sessions; and modification of patient records/registers to capture essential information)

• Routinely collect and analyze clinical quality measures used in the RBF program to assess progress against defined improvement aims

• Regularly post results in relevant patient care areas for public information

• Conduct a structured audit of every maternal, newborn, or child death and near miss

In addition, a component of capacity building and mentoring by district/municipal managers and supervisors was designed in order to:

• Support facility CQI teams to regularly carry out CQI activities (see above)

RBF PROJECT

CQI FacilitiesNon-CQI Facilities

RBF training

RBF quarterly supervision using

check-lists

RBF subsidies

CQI training

Supportive Supervision, mentoring/coaching

17

• Ensure regular competency-based training and refresher training of skilled providers

• Oversee local supply chain (in close communication with facility managers) to prevent stock-outs of essential commodities (e.g., Gentamycin, functional neonatal bag and mask, rapid HIV diagnostic tests)

• Audit medical records and observe simulated performance of priority clinical procedures (e.g., newborn resuscitation using structured observation checklist) to assess quality of care during supervision visits

• Track facility-specific results for priority clinical quality indicators (incentivized in RBF program) to tailor support to facilities and clinical areas demonstrating poor performance on priority indicators

3.3 A theory of change for quality improvements at the primary clinic level

The theory of change for the CQI intervention is based on the premise that the continuous quality improvement model will lead to enhanced quality of care and client satisfaction (Figure 3.1). At facility level, the intervention was supported by a multi-departmental QI team, which included facility managers and health providers. Supervisors at the district level built QI team capacity in clinical and management competences through training, review of records, and direct observations of the priority clinical quality indicators. The clinical competences consisted of adherence to the national standards of care for select MCH services, e.g., review of timely completion of partograms as stipulated by the MOHCC and international health guidelines, HIV-infected infants initiated on treatment before age two, newborns breastfed within one hour of birth, and child outpatient diarrhea cases correctly treated. Through mentoring and coaching by supervisors along with facilitation of the supply chain, it was assumed that the capacity and motivation of the health workers would be enhanced, which in turn would enable regular implementation of the CQI cycle (i.e., identify services for quality improvement, plan, analyze, and review, implement, and share the results). Regular CQI implementation would result in adherence to standards and hence improved clinical indicators. Finally, better clinical indicators lead to improved quality of care and client satisfaction. Improved clinical indicators also lead to higher incentive payments through the RBF mechanism that will further

motivate the health teams.

Figure 3.2: Theory of change for the quality improvement intervention

Source: World Bank 2017

18

3.4 Evaluation questions and objectives

While there is an emergent body of evidence of the positive impact of CQI (and related quality improvement models) on quality of care, little is known about the effectiveness of incentivizing the implementation of CQI per se. There is limited evidence on the effect of combining RBF interventions with CQI in low-middle income country (LMIC) settings. Therefore, the overarching goal of the qualitative CQI PME study is to help the MOHCC, the Ministry of Finance and Economic Development (MOFED), Cordaid, the World Bank (WB) Task Team and interested local and international stakeholders learn from the CQI implementation process; make course adjustment to the technical design and operational processes; and enhance evidence-based project management decision making by exploring opportunities for the CQI component scale-up.

The primary research questions guiding the process evaluation of the CQI intervention were as follows:

1. What is the effect of a quality improvement model on quality of care?

2. What are the factors that influenced observed changes?

3. What is the effect of a quality improvement model on how health facility teams

organize to improve quality of care?

Quantitative analyses explore the first question, while qualitative work addresses the second and third questions.

4. Methodology

4.1 CQI Implementation

MOHCC QA/QI team staffing was strengthened through Cordaid seconding seasoned international technical staff to work in the MOHCC national QA/QI directorate to support national, provincial, and district level implementation of quality aspects of the RBF program, with added emphasis on the CQI innovation. An MOHCC technical team developed the CQI guidelines with the support of international consultants from Health Quality International. The “how to” guidelines supported the standardization of CQI activities conducted by health facility teams.

A selected team of national and provincial focal persons was trained in the initial phase by a team of international and local CQI experts from Health Quality International and from the University of Zimbabwe. The focal persons were trained in QI methods to serve as trainers-of-trainers (TOT) and further cascade the capacity building process to the districts and facilities. A small technical team from the MOHCC QA/QI Directorate and provincial/district focal points was responsible for the roll-out of the CQI innovation to facilities in selected districts during this initial phase. This process was designed to ensure fidelity of CQI implementation across intervention sites.

The CQI project collected data on MCH service-related indicators from the quality supervision checklists; quality assessments are conducted by Provincial Health Executives (PHEs) and District Health Executives (DHEs). PHEs and DHEs were provided with necessary tools and upskilled in CQI through a training-of-trainers model.

PHE and DHE teams were given a four-day basic training in CQI. The training focused on performance measurement and quality improvement techniques to strengthen: quarterly supervision visits; extraction of data; data analysis and interpretation, including conducting root cause analysis; prioritization of health facility issues; and mentorship and coaching. Cordaid was contracted to train identified MOHCC personnel. To facilitate learning and sharing across facilities and to enhance the skills and knowledge of coaches and quality improvement teams, refresher trainings were planned for

19

a minimum of once a year. Quarterly coaching of coaches by MOHCC central coaches to the provincial and district coaches was integrated into the design.

4.2 Selection of CQI implementing health facilities

Eighteen districts, two in each of eight Zimbabwean provinces, have been implementing the RBF program and constituted the study population. Stratified by province, 9 of these 18 districts were initially randomly selected to participate in the CQI pilot program. However, resource limitations further restricted the pilot to five districts. Therefore five of these nine districts were purposively selected in the following manner: the nine districts were ranked on the basis of baseline quality scores for 2014 and 2015 (taken from the balanced scorecard used in the RBF program in those years) as well as the average catchment population for all facilities. To encompass a range of district situations, the three districts with the lowest ranked average, and the two with the highest composite scores were selected. These five districts received the additional quality improvement efforts through the CQI approach.

The two study arms are:

• Arm 1: Facilities receiving performance-based incentives for quantity and quality, with quality dimensions determined through a revised Balanced Score Card

• Arm 2: Arm 1 activities plus CQI measures and training on management of quality (i.e., quality improvement model)

The five selected districts are listed in Table 4.1 and depicted in the map in Figure 4.2.

Figure 4.2 Map of CQI districts studied

20

Table 4.1: Details of provinces and districts under CQI

ARM 1

Province District District Population Number of Clinics Number of Nurses

Mashonaland Central Centenary 131,219 12 38

Masvingo Mwenezi 166,263 16 50

Manicaland Chipinge 326,467 34 76

Matabeleland North Binga 138,074 14 38

Matabeleland South Mangwe 78,665 11 42

In the original process evaluation plan, 9 randomly selected PHCs and the district hospital that

were surveyed in the first round RBF evaluation facility survey were selected to participate in the pilot CQI program. However, resource constraints subsequently limited, before CQI implementation, the number of participating PHCs to five per district (as well as the district hospital). The five study PHCs were purposively chosen from the nine originally sampled facilities, with greater weight given to larger facilities without prior QI initiatives, and greater weight given to facilities that were closer to other participating facilities in order to make the CQI pilot implementation more tractable. Consequently, the supervising district hospital and the five purposively selected rural health centers (RHCs) were chosen for initial implementation.

Figure 4.3 below shows the CQI timeline.

Figure 4.3: CQI timeline

4.3 Qualitative process evaluation

The qualitative process evaluation was implemented in 2 phases (Error! Reference source not found..3). In the first phase (steps 1-4) the evaluation team i) reviewed project related documents, ii) developed process evaluation methodology and tools, iii) sampled health facilities for site visits and collected qualitative information using various data collection methods (described in Section 2.2.3), iv) constructed stakeholder map and produced inception report (steps 1 & 2) covering evaluation methodology, sampling of provinces, facilities and respondents, implementation phases, and schedule and draft outline of the final evaluation report. The inception report incorporated comments solicited

21

from the MOHCC, the World Bank Task Team, and the Principle Investigator (PI). Final data collection tools were submitted to the MOHCC for Ethical Review Committee approval.

Figure 4.3: Phases of the Qualitative Process Evaluation implementation

For the field data collection (step 3) the research team (one international and one national consultant) visited Zimbabwe for two weeks, collected qualitative information, and shared preliminary findings and recommendations with key stakeholders. Following data collection, researchers completed data analysis and triangulation and produced a draft of the CQI process

evaluation report (step 4). The draft CQI report has been shared with the World Bank for review and comments.

The Initiation of the second phase (steps 6-9) of the evaluation was subject to the availability of quantitative PME results collected separately from the qualitative process evaluation (step 5). In this phase, the research team, which comprised one international and one national consultant, reviewed preliminary results of the quantitative PME and jointly with the quantitative evaluation team elaborated exploratory research questions (Annex 2: Exploratory Research Questions) respective data collection tools, and research protocols (steps 6 and 7). For the field data collection (step 3) the research team (one international and one national consultant) visited Zimbabwe for 2 weeks, collected qualitative information and shared preliminary findings and recommendations with key stakeholders. Following data collection step, researchers completed data analysis and triangulation and produced draft CQI process evaluation report (step 4). The draft CQI report has been shared with the World Bank and comments solicited.

At the end of phase 2, the research team visit the country for the second time to ensure collection of exploratory data in the field, analyze and incorporate findings into the final CQI process evaluation report (steps 8-9). The team presented preliminary findings and recommendations to the MOHCC and the World Bank by end of the country mission.

4.3.1 Qualitative data collection

The qualitative process evaluation methodology comprised a mix of desk-based research, key stakeholder in-depth interviews (IDI), Focus Group Discussions (FGD) and review of quantitative process evaluation results (in phase 2). Review of documents was a major part of the assignment during the inception phase and largely informed PME methodology. The research team consulted with and obtained necessary documents from the Bank and the MOHCC. Based on the desk review findings, information gaps were identified, and related questions included in the IDI and FGD guides.

IDIs are an important source of evidence for many exploratory research questions and aided researchers to: a) understand the range of contextual and operational challenges and opportunities; b) continue analysis started by the review of quantitative PME results for deeper analysis; c) generate findings and lessons learned; d) explore implementation of different strategies, inputs, and tools by health facilities for improvement of the care quality; and e) identify different results/pathways of contribution where feasible. IDIs were implemented face to face for key informants from central,

01

Desk review

02

Development

of PME

methodology

03 04

Data

analysis

and draft

report

Field Data collection

09

Data

Analysis

and final

PME report

08 07 06

Review of the

Qualitative

Data findings

Field Data collection

Qualitative Impact

Evaluation

05

Formulation of

exploratory

research

questions,

data collection

methodology & tools

QU

ALI

TATI

VE

PM

E

QUANTITATIVE PME

Phase 1: 2018

Phase 2: 2019

22

provincial, district, and facility levels. IDI interview topic guides based on research questions, helped ensure systematic coverage of research questions and issues.

FGDs were carried out for hospital CQI committees. FGD guides included questions specific to the FGD participant group and included 8-10 participants per each group. In health care centers with small number of staffed the FGDs were replaced by small Group Interviews.

During Phase 2, the research team reviewed the preliminary results of the qualitative process evaluation and formulated exploratory research questions that guided the team in the development of the data collection methodology and tools and carried out IDIs of FGDs.

4.3.2 Qualitative sampling

A multistage sampling approach was used to select the provinces and facilities. A cascade sampling enabled selection of three of the five provinces in which CQI was piloted. The selection criteria were based on geographic spread to ensure representation from north and south geo-regions and a remoteness from the capital Harare. Then within each selected province, facilities were sorted by two groups: CQI and non-CQI facilities.

Figure 4.4: Provinces, districts, and facilities sampled

Finally, within each group, facilities were sampled ensuring the presence of all types of facilities (hospital, clinic, and rural health center) per each facility group; and quality supportive supervision results to assess facility performance from the 1st quarter of 2018. Sampling for CQI group facilities involved the selection of high and low performing (based on quality scores) facilities, whereas for the comparison non-CQI group, facilities with the highest and lowest quality scores were selected. As a result, in each selected province, six facilities were selected (three CQI group facilities and three comparison group facilities) (Figure 4.4).

Qualitative process evaluation findings are based on the information collected from 232 individuals at national, province, district and community levels in selected three provinces. Overall, apart from individual face-to-face IDIs, researchers also carried out 15 FGDs and 22 group interviews.

CQI Facilities

1. St. Albert Mission Hospital (QS=86%)

2. Chidikamwedzi Clinic (QS=92.9%)

3. Machaya Rural Health Centre (83.2%)

Comparison Facilities

1. Concetional District Hospital (QS= 82%)

2. David Nelson Clinic (QS=94.4%)

3. Chawarura Rural Health Centre (QS=89.4%)

CQI Facilities

1. Neshuro DH (QS=90%)

2. Rutenga Clinic (QS=77.3%)

3. Munyamani Rural Health Centre (67.2%)

Comparison Facilities

1. Chiredzi District Hospital(QS= 92%)

2. Chitsa Clinic (QS=89.4%)

3. Malipati Rural Health Centre (QS=78.7%)

CQI FacilitiesComparison Facilities

1. Chista Clinic (QS=87.7%)

2. Chiredzi District Hospital (QS=92%)

3. Malipati Rural Health Centre (QS=78.7%)

1. Neshuro District Hospital (QS=90%)

2. Rutenga Clinic (QS=77.3%)

3. Chirindi Rural Health Centre (QS=89.7%)

23

4.3.2 Qualitative data analysis and triangulation

The study team entered and analyzed the qualitative data using NVivo 10™ software19. Analysis focused on synthesizing and triangulating information from the various data sources and evaluation methods. The evaluation took an iterative approach based on grounded theory that allows themes and findings to emerge from the data. In general, this evaluation considered data or evidence to be more valid, and therefore gave it more weight, when the analysis identified convergence in opinions and experiences across multiple sources. However, the evaluation team recognizes that CQI implementation will vary in different contexts and therefore have also reflected opinions and experiences that are not widely shared but are illustrative of a particular situation or consideration.

4.3.3 Qualitative research ethics

The study obtained ethical permission from the Medical Research Council of Zimbabwe for the Qualitative process evaluation study as part of the overall RBF impact evaluation research. Before the interviews, participants were informed about the objectives of the study, why they had been chosen, and the risks and benefits of participating in the study. Participants were afforded the opportunity to ask questions or make comments and then decide if they wanted to participate. A consent form which participants were asked to read before the interview, provided further information. Participants then decided to participate voluntarily and were asked to sign two consent forms. The participants were assured that the whole process would uphold the highest standards of confidentiality and that their participation would be anonymized.

4.3.4 Qualitative study limitations

It is important to acknowledge some limitations associated with the process evaluation.

- At the time of the qualitative process evaluation, the CQI pilot was yet at an early stage of

implementation and the findings of the study may not accurately reflect potential

attainments of the pilot. More time would have been desired to see medium to long term

effects.

- The definition of terms “coaching” and “mentoring” were not well understood by

respondents and were used interchangeably during IDIs and FGDs. Thus, the report refers

to term “coaching/mentoring” to accurately reflect information collected.

4.4 Quantitative process evaluation

The quantitative evaluation compares quality-of-care outcomes in CQI intervention and control facilities. All other facilities that did not receive the CQI intervention within the same districts as CQI intervention facilities constituted one possible comparison group of facilities. A second comparison group was made up of facilities in the 13 other study districts that did not receive the CQI pilot.

4.4.1 Quantitative data collection and extraction A baseline survey on quality standards in the form of the revised quality checklist was

collected for all health facilities in the 18 study districts in the last quarter of 2016, and then the intervention began. The quality checklists for both primary and secondary facilities, used for all data capture, are presented in Annex 4.

Data from health facilities’ quality assessments was extracted from the online database. Initially data was captured using the ONA platform, however, starting in 2018, data capture switched

19 NVivo is a qualitative data analysis (QDA) computer software package produced by QSR International. It has been designed for qualitative researchers working with very rich text-based and/or multimedia information, where deep levels of analysis on small or large volumes of data are required. The software allows users to classify, sort and arrange information; examine relationships in the data; and combine analysis with linking, shaping, searching and modeling.

https://en.wikipedia.org/wiki/Qualitative_data_analysis

https://en.wikipedia.org/wiki/Computer_software

https://en.wikipedia.org/wiki/QSR_International

https://en.wikipedia.org/wiki/Qualitative_research

https://en.wikipedia.org/wiki/Reference_(computer_science)

https://en.wikipedia.org/wiki/Search_engine_technology

https://en.wikipedia.org/wiki/Data_model

24

to the District Health Information System 2 (DHIS2) database platform. The DHIS2 RBF database is a national database that stores data on most key indicators and disease surveillance for Zimbabwe.

While the RBF Database makes use of the MOHCC HMIS (DHIS2), it has additional data on key RBF indicators and other key variables. Hospital and primary facility level data was extracted from the entire checklist totaling 153 PHC-level indicators and 253 hospital level indicators. Data from these forms included both CQI and non-CQI facilities to enable comparability of results. Data cleaning involved removal of duplications and merging of data from the 15 different forms into one single dataset. The final dataset contained data for the period between October 2016 and December 2018 – i.e., one quarter before the start of implementation and eight quarters of the implementation period. While Cordaid managed and oversaw the maintenance of the DHIS2, the DHE’s carried out data collection. Data was collected using real time data collection tools on mobile devices. However, due to lack of validation rules in the data collection tool, reporting of complete quality indicators varied among DHEs.

4.4.2 Quantitative outcomes adopted in the analysis

The PHC checklist contained data on 153 individual quality-of-care related outcomes. These were grouped into thirteen aggregate measures assessing the quality of inputs and seven aggregate measures assessing the quality of processes. All aggregate measures were calculated as the percentage of the underlying individual indicators (equally weighted) achieved by the facility. For example, facility infrastructure quality was calculated as the percent of the following eleven items that a facility had: visible sign post when arriving at the clinic, fence/wall in good condition, external appropriate wall finishing, roof intact with well-maintained rain gutters, minimum of three toilets, hand washing facility with soap available near the toilets, electricity for 24 hours a day and 7 days a week, fire extinguishers available and accessible and fire safety plan, appropriate drainage of waste water, shower with either running water or container of at least 100 liters and waiting area with adequate ventilation of waiting area. As the CQI intervention was intended to affect process quality more than inputs, the aggregate process quality indicators are presented as the main results, while the inputs are presented in Annex 3. All individual indicators are presented in their aggregate groups in the Annex 3 Table A1.

The hospital checklist contained data on 253 indicators, which were grouped into 17 aggregate measures assessing the quality of inputs and nine aggregate measures assessing the quality of processes. In addition to the aggregate measures available in the PHC checklist, hospital outcomes include patient amenities; human resources management; surgical, lab and radiology equipment; quality improvement strategy; paediatric care processes; paediatric complications processes, and surgical safety. PMTCT input data was not available from the hospitals.

For both the PHC and hospital outcomes, an average of the four quarters in 2018 was used as the post-intervention timepoint for most analyses in order to (a) allow for the intervention, begun at the start of 2017, to take hold and achieve its intended aims, and (b) smooth variability, including possible seasonal variability, between quarters. A sensitivity check using only the fourth quarter of 2018 to reflect this variability and assess the reliability of the effects is presented in Annex 3.

4.4.3 Quantitative data analysis

The balance of the outcome measures was first assessed prior to the start of the CQI intervention in the fourth quarter of 2016. A T test with unequal variance was used to identify differences between the treated and control facilities.

25

To estimate the effect of the CQI intervention on the input and process outcomes, the ANCOVA model20 below was used:

𝑦𝑖2018 = 𝛽0 + 𝛽1𝐶𝑄𝐼𝑖 + 𝛽2𝑦𝑖

2016 + 𝜖𝑖

Where 𝑦2018 is the value of the outcome in 2018 in facility i, CQI is an indicator for whether the facility implemented the CQI intervention, and 𝑦2016 is the value of the outcome in 2016. Standard errors were clustered by district.

As this process evaluation is quasi-experimental in nature, two comparison groups are used for the PHC analysis. In addition, two estimation approaches to the above equation are explored for robustness purposes. First, an unadjusted model is presented that compares the intervention facilities against all PHCs in the 13 non-CQI comparator districts, as well as a model using coarsened exact matching (CEM) weights. CEM is a method that corrects for imbalances between treatment and control facilities by coarsening a set of facility covariates into bins, creating a stratum per bin and assigning observations to the strata, then dropping any observation whose stratum does not contain at least one treated and one control unit.21 CEM weights are then included in the ANCOVA regression. The covariates used for matching include distance from the PHC to the district hospital, facility designation (clinic versus rural health centre), and volume of new patients prior to the intervention in 2016. The hospital model is unmatched given the already small number of treated and control facilities (5 and 13 respectively).

The second PHC comparator group contrasts CQI facilities against non-intervention facilities in the same districts. With this comparator, the ANCOVA is again estimated first unweighted and then with CEM weights. Difference and difference models are presented as robustness checks in Annex 3. The PHC difference in difference model includes district fixed effects, when compared with facilities in the same districts, and the CEM weights used in the ANCOVA model. In all models, facilities that do not have data in both 2016 and 2018 are removed case-wise. As the process evaluation is exploratory, P-values are not adjusted for multiple hypothesis testing.22

20 McKenzie D. Beyond baseline and follow-up: The case for more T in experiments. Journal of development Economics 2012; 99(2): 210-21. Also see, Frison, Lars, and Stuart J Pocock. 1992. “Repeated Measures in Clinical Trials: Analysis Using Mean Summary Statistics and Its Implications for Design.” Statistics in Medicine 11 (13): 1685–1704. 21 Blackwell M, Iacus S, King G, Porro G. cem: Coarsened exact matching in Stata. The Stata Journal 2009; 9(4): 524-46. 22 Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology 1990: 43-6.

26

5. Qualitative Results

This section of the report summarizes evaluation findings in relation to:

1. Quality improvements of MCH services in intervention health facilities; 2. Factors influencing observed changes, particularly examining CQI system structure, content,

delivery modes, reorganization of the facility teams, effectiveness of CQI trainings, and effects of RBF payments on CQI system advancement; and

3. Options and opportunities for CQI system scale-up.

5.1 Quality improvement results The CQI pilot is perceived as one of the most important initiatives for improving the quality

of MNCH services in Zimbabwe. There is a clear agreement among actors involved in CQI pilot about the significance of quality improvement as a fundamental tool to improve health outcomes. According to respondents, the CQI pilot was one of the first and few initiatives that ensured staff exposure to quality improvement, and which strategically and deliberately included a component of local problem analysis and planning of corrective measures. However, there were some limitations of implementation, including variation in the quality of the CQI training and supervision.

CQI facilities demonstrate improved quality of care compared to non-CQI facilities. To perform comparative analysis of the quality improvements at intervention and non-intervention facilities, results of quantitative process evaluation were summarized according three quality domains (structure, process, and outcomes).

CQI facilities demonstrated better results in improvement of MCH outcomes and enhancement of service delivery processes along with advancement of the structural quality than non-CQI facilities. There are a handful of best practices recorded (see Text box).

Both types of facilities (hospitals and health clinics) show an increased proportion of children under five with diarrhoea being correctly treated and the potential for serious bacterial sepsis among neonates managed as per national protocols. Decreases in post-partum hemorrhage rates at CQI health clinics were more evident than in non-CQI health clinics. Although positive results are observed at intervention hospitals and health clinics, higher achievements were recorded at hospitals compared to health clinics (see results of quantitative study). Hospitals demonstrated higher results in improving process and structural quality (see Figure 5.1).

Figure 5.1: Quality improvement by quality domains at CQI facilities

- When CQI came it was one of the first few initiatives in terms of improving quality…

- For most health workers it was their first exposure to quality improvements cycles …

- I think it facilitated finding local solutions to local problems…

Quotes from Key Informants at Province and District CQI Facilities

Child health

outcomes

Delivery

outcomes

Process

Quality

Structural

quality

Structural

quality

CQI HospitalsCQI Health

Clinics

Child health

outcomes

Delivery

outcomes

Process

Quality

Structural

quality

27

The majority of women interviewed were satisfied with the quality of services they received during childbirth. To learn about their experiences with childbirth and their satisfaction with the quality of care received, the study interviewed women in the postdelivery departments at intervention and non-intervention sites. No significant difference was recorded between CQI and comparison facilities.

Overall, women interviewed for each group of hospitals are satisfied with services received and commended overall good conditions in delivery and postdelivery departments, good hygiene, timely provision of services, and teaching young mothers how to breastfeed their babies. Interviewees unanimously expressed readiness to advise their peers to seek delivery services in these hospitals. Respondents at CQI facilities more frequently highlighted the benevolent attitudes of health personnel, but these results cannot be firmly attributed to only CQI interventions due to the drawbacks of real time patient satisfaction survey methodology used by researchers.

Text Box 1: Selected best practices

Improving immunization coverage: Per month we are supposed to cover 16 kids in each vaccine dose. When we analyzed our immunization coverage, we found it to be below our target of 90%. It’s very difficult to reach out to those children who are not vaccinated. To address this shortcoming, every month we mobilize the community using the village health workers (VHW). VHWs are requested to go door-by door and find children who are due to be immunized. For such children, they write a referral letter to our clinic. Whilst VHW involvement helped to upsurge coverage rates, we were still below the target as not every parent was bringing a child to the clinic for vaccination. Next strategy elaborated by the CQI committee was the EPI outreach and vaccinate children in the communities. At present we mobilize the community to make sure that we have a vehicle to drive to the outreach points and immunize children with the help of VHWs.

Addressing late booking for antenatal services and increasing institutional deliveries: We managed to improve the quality of antenatal services and decrease the share of mothers who deliver at home. To address late presentation for antenatal services and decrease home deliveries, the clinic introduced gift packages with soap, napkins, and Vaseline for those who book for antenatal services before 16 weeks of pregnancy, complete six antenatal visits and deliver at the facility.

Management of delivery and post delivery period: To improve the quality of delivery services, we assessed whether our staff follow standards and found out that partographs are used during deliveries only in 40% of cases, oxytocin is administered later and mothers after delivery are not regularly monitored due to the shortage of staff and high workload. To resolve this issue, we trained our staff in using portograms, placed filled in portogram on a wall as a reminder and regularly monitored its use. This helped us to reach 60% within 5 months.

Timely administration of oxytocin was identified as one of the main deviations from the standard. When we completed root cause analysis, we found out that oxytocin is not always readily available in the delivery room, which causes delays. To improve on this, there was a need to ensure availability of the medicine next to the delivery bed of women. By this intervention we assured that in 98% of cases, oxytocin is administered within the first minute after delivery.

In order to improve regular monitoring of women during the delivery, we procured wall watches with alarms and assigned a nurse. Alarms are set with 30 minutes interval and remind the nurse to check foetal heartbeat, contraction and pulse.

Improving the quality of childhood diseases: We work on improving management of childhood illnesses, by trying to improve correct diagnosis and treatment of cases of diarrhea and pneumonia. Before CQI training we did not know how to assess these variables correctly. When we analyzed only 50% of diarrhoea and 8% of pneumonia cases were managed according to IMNCH guidelines. We had a meeting and strategized how to improve on these indicators. As a result, we provided peer- to peer support and mentoring using those staff who

- Staff during the whole process was very polite, examined regularly...

- I liked staff attitude, they are considerate and responsive…

- During childbirth, the attitude of personnel was very good and caring…

- After I delivered the baby the nurse would come and check me every other hour…

- I am satisfied with the services received …

Quotes from Key Informants at CQI Facility

- I am very satisfied with the services received at the hospital…

- They have blankets, medicines and meals…

Quotes from Key Informants at Non-CQI Facility

28

underwent the IMNCH training. We placed flow charts on the wall, that served as a reminder to staff. For the management of pneumonia, we procured timers to accurately measure respiration, oriented new staff and supplied IMNCHI registers and forms. We regularly supervise our staff to ensure that guidelines are strictly followed. By combination of these efforts we ensured 100% compliance with IMNCH protocols.

Quotes from key informants at CQI facilities

5.2 Factors influencing observed changes

The evaluation team noted several important factors facilitating and or impeding achievement of CQI program results. These factors are discussed in following sections of the report.

5.2.1 Enabling factors

The achievements in improved clinical outcomes and quality of care improvements observed at CQI facilities were facilitated by a number of factors discussed in detail below.

GoZ political will to promote quality improvement in the health sector set an enabling environment to pilot CQI in selected districts and health facilities that resulted in improved quality. Enabling a QA/QI policy environment and political will of the MOHCC to promote quality improvement in the health sector allowed the initiation of the CQI pilot. The pilot demonstrated improvements in the quality of MNCH services, though achievements are not uniform across all targeted facilities.

Availability of additional funding (RBF subsidy) enabled all RBF facilities to improve quality. Both CQI and comparison facilities had equal access to additional funding (RBF subsidies) based on the facility performance. Availability of additional resources allowed CQI and non-CQI facilities to improve infrastructure, procure some basic medical equipment, as well as maintain a stock of essential medicines and supplies for MNCH services, albeit at various degrees across types of facilities.

Volume of additional funding received by hospitals was higher compared to health clinics due to the higher case load and volume of provided services. This allowed hospitals to perform better by improving structural quality than health clinics though these improvements were marginal compared to non-intervention facilities. CQI facilities were more strategic in using available additional funding for infrastructure improvements that potentially helped to enhance their performance in the field of MNCH, whereas comparison facilities thinly spread resources for general hospital priorities.

RBF subsidies occasionally served as bridge funding to ensure availability of government supplied medicines when delays were observed at both intervention and non-intervention sites.

CQI intervention has increased awareness of and participation in CQI activities and processes, and enabled CQI facilities to accelerate improvements in service provision and clinical quality. Nearly all health service managers and staff interviewed at intervention sites were aware of the concept of CQI, could articulate what it meant, and were able to provide examples of CQI activities in which they had participated. Most informants were able to provide examples of changes to practices they had made as a result of recent engagement in CQI processes. Whilst RBF subsidies were

- With RBF subsidies we improved our facility. We procured some basic equipment although we are yet short of some equipment…

- Subsidies that we receive based on our performance often is not sufficient to cover all needs of infrastructure improvement, procurement of medicines and supplies and/or equipment… Compared to hospitals our earning is smaller because of service volumes…

Quotes from Key informants

- Sometimes we used RBF payments to procure some medicines…

- RBF subsidies help when we are short of medicines… We use these funds to procure medicines when there are delays from the government…


29

found to be important to structural quality improvement, CQI interventions empowered CQI facilities to accelerate improvements in processes and clinical quality (see Figure 5.1).

Staff capacity building in CQI principles and methods improved quality at CQI facilities. The CQI component of the RBF project supported cascade training of staff in principles and techniques of CQI at provincial, district and facility levels. PHEs and DHEs were trained as coaches in quality improvement principles and how to teach facility CQI committees in MNCH quality; performance measurement and using data for improvement; how to apply quality improvement in practice through the systematic use of simple tools to monitor, assess, and improve the care. Each trained individual had to train others in the health facilities to transfer knowledge and skills.

External Supportive Supervision (ESS) is viewed as important and useful in guiding the health facilities to focus on areas requiring attention. ESS was noted to be helpful in improving performance. CQI facilities reported ESS teams undertaking supportive supervision by reviewing facility data reported per each CQI indicator against primary source of information, examining progress or lack of achieving the target and provided feedback to staff on findings of the supervision, and advising on ways of improvement. Respondents appreciated ESS using a standardized approach to performance assessment, specifically, being assessed against predefined indicators via check lists. Supervisor feedback on results, discussions on identified weaknesses, and recommendations for improvement after each assessment were considered beneficial. Supportive supervision encouraged health facility management and staff member to openly acknowledge weaknesses and mistakes and to work as a team to overcome them.

While non-CQI facilities also received ESS, they lacked mentoring/coaching opportunities. As per the original project design, non-CQI facilities were not supported with mentoring and coaching from their supervisors. Hence, compared to intervention facilities, respondents from non-CQI facilities reported having no or only vague information about CQI and not receiving any coaching or mentoring from their supervisors.

CQI facilities were shown to be more equipped with skills and knowledge to improve processes (e.g. carrying out root cause analysis), whereas teams at non-CQI facilities mostly depend on how innovative the teams are at defining problems and finding solutions. Respondents from non-CQI sites agreed on the need (i) to be trained in quality improvement and (ii) for mentoring from supervisors who on a quarterly basis supervise their facility and check the accuracy of reported indicators.

- We discuss area of weaknesses and challenges and the next time we visit we track and check we monitor…

- For proper interventions we also advised them to go to the “5 Whys”, process maps and trying to get the root causes. Because without the root cause analysis you will not able to develop proper interventions…

- The supervisors follow on the checklist and after supervision they sit down and tell us how well we have done, the gaps and how we were supposed to do things…

- When they come for the supervision they come as team and analyze our work. then they give us a feedback and give us a %. They call all the staff and we have a short meeting and that is when they give us a feedback on the areas that we are not doing well and also praise us for what we are doing well…

Quotes from Key Informants at CQI facilities

- Nobody coaches and mentors us... - We do not know what is involved in CQI and what is

supposed to be done... - We are trying our best, but we do not have enough

information on what needs to be done to improve quality…

- In CQI facilities quality becomes central to their service delivery as “people who are trained train others”, so there is more knowledge transfer

Quotes from Key Informant at non-CQI facilities

- Good leader makes things happen… - I think basically what makes things work is leadership and

teamwork because as we sit down as a team, we identify our gaps and find solutions…

- Teamwork is number one in improving CQI… - Last year matron mentored us in using partogram, but we

achieved only 10%. Since January 2018, a new nurse has been assigned to our ward. She strictly follows on use of partogram, trains us. Under her leadership in May 2018 we achieved 87% on use of partogram. Leadership is important to improve quality…

Quotes from Key Informants at CQI facility

30

CQI fostered leadership, teamwork, and joint decision-making at intervention sites. Strengthened health facility management has been observed in all health facilities since the introduction of RBF. leadership at facility, department/unit, and individual level were more evident at CQI facilities than at non-CQI sites.

Respondents were appreciative that management now consults and engages staff in monitoring results, identifying root causes, planning, and decision making. Health facility QI teams intensified internal supervision and met once a month. As part of regular internal supervision, the QI committee examines each clinical record (clinical audit) sampled and discusses if and how it complied or not with standards of care for prenatal, delivery, immediate postpartum, newborn care, management of main obstetric complications, and management of diarrhoea and pneumonia cases. This process is not only a monitoring procedure, but also a form of collective supervision over the quality of care. The QI committee determines the results of improvement actions implemented and identifies actions required to solve problems detected in weak areas. Based on these discussions, the QI committee decides if the intervention is successful and deserves to be implemented at a larger scale in the health facility or area, or whether it is preferable to modify the intervention, or choose a new healthcare process for improvement.

QI teams use selected CQI tools in their daily practice. “5S”23 was the most commonly used technique by CQI facilities. This tool focuses on establishing an organized workplace, cleanliness, and standardization to improve service profitability, efficiency, and safety. The evaluation found that 5S has been the starting point for health-care quality improvement at reviewed intervention sites. Workplace environment was characterized as a bottleneck in providing adequate services. Respondents noted that 5S was initially the easiest tool for starting improvement of quality exercises. Almost all CQI facilities visited reported improving workplace organization and getting rid of unnecessary equipment, papers, etc. by using 5S. Respondents believe that 5S helps and stimulates working on the quality improvement.

The “5S” technique was used in combination with other CQI tools in pilot facilities, however, experience and knowledge of tools introduced by the pilot are not spread adequately across CQI health facilities. QI committees often apply flow charts, “Why Tree” root cause analyses, and fishbone

23 5S-CQI (KAIZEN)-TQM, using the 5S principle (sort, set, shine, standardize and sustain)

- Internal supervision here at the hospitals actually assist us…

- We do our surveillance data analysis and discuss. After assessing the progress, we go back to the ward assessed and report. Alternatively, we discuss via “WhatsApp” mobile application. We also send pictures to people to see where their problems are...

- CQI helps to find local solutions to local problems…

Quotes from Key informants at CQI facilities

- The root cause analysis is very helpful, because with the root cause you can plan now...

- To identify causes, we use „5 Whys“. This helps to undestand where are the problems and to plan measures...

- PDSA really assist to test out the issues and design interventions…


- With the 5S you see changes in the department because items are sorted for convenience. It creates a sense of orderliness and give a sign so that that you should not mess up with each other work to be conscious. The 5S makes CQI work…

- We have been sensitized about the 5S on quality and we have removed useless things and retained only needed items and equipment. We learnt a lot from this. We had kept so many things in our department over so many years, old chairs, and old BP machines. We repaired some and disposed others. We filed our material and covered our files. Things are now in order…

- Using the 5S we cleaned up the department. Before the program there was no labelling... We also removed old registers that we did not need…


31

diagrams, and follow PDSA cycle. However, some respondents also noted having inadequate knowledge to effectively use these tools. The evaluation was not able to observe robust evidence on the regular application of PDSA cycle in studied facilities. Respondents indicated need for additional training and more intense mentoring from PHEs and DHEs.

Employee motivation and satisfaction differs across health facilities. Motivation at work is widely believed to be a key factor for performance of individuals and organizations24. RBF rewards health facilities with supplemental payments based on facility performance, 25% of which is used to motivate personnel and the remaining funds to cover other needs for improving performance. The CQI pilot assumes that financial incentives on top of low salaries will increase staff motivation and promote CQI.

The evaluation attempted to understand how staff financial incentives motivates personnel in improving the quality of MNCH services.

Staff pay for performance had marginal effect on staff motivation to improve care quality at both intervention and non-intervention sites. Views of respondents differed between their peers from the same health facilities and across facilities from different study groups. While few respondents considered financial incentive as a motivation to perform better, a majority thought that financial incentives had marginal effect. Staff at both intervention and non-intervention sites believe that incentives were not sufficient given the increased workload—and that the amount received was insignificant compared to their base salaries. Several facilities reported collectively making decisions on how to use staff performance pay for structural improvements.

Health worker views also varied in relation to the staff incentive distribution formula. All health facilities reported distributing financial incentives to all, clinical and non-clinical personnel using “staff incentive calculator” that considers type, level/grade of staff member, experience, and attendance. Findings suggest that because hospitals have more staff than health clinics, the portion of RBF supplemental payments for staff motivation are thinly spread across all staff and are too small to motivate personnel. Few respondents expressed their concerns regarding the fairness of the “staff incentive calculator”. On the other hand, some personnel consider that paying all staff is important to encourage and promote team-work for quality improvement. In summary, while a few respondents considered the financial

24 Hornby, P. & Sidney E. (1988). Motivation and Health Systems Performance. WHO, WHO/EDUC/88.196

- We have over 100 staff members and a portion of payment we receive from RBF project, is distributed among all staff based on staff grades. We share this little cake and staff get small incentives. Staff in smaller facilities get more incentives than us… This is discouraging…

- We also do not want it to be equal because of the differences in roles and responsibilities…

- We are not happy. It’s too little… - Partly its fair considering that the differences are not much and

partly unfair because these people directly concerned with the RBF issues working hard have sleepless nights are on the lower rate…

- If everyone is not paid it ends up creating frictions and they are not happy to work with others. It demotivates…

Quotes from Key Informant Interviews

- I believe quality should be internal. One should not be motivated by financial incentives to deliver the best quality services. Of course, incentives motivate, but quality should be part of our daily work…

- Of course, the financial benefit is there but whatever I do, I do it to my best capacity and knowledge…

Quotes from Key informants at CQI and comparison facilities

- To ensure electricity supply for the facility, we had to purchase electricity meters. As RBF subsidies were not sufficient to procure electricity meters in addition to already planned procurements, we collectively decided to seek DHE approval for using the budget dedicated for staff incentives...

Quote from Key informants at CQI facility

32

incentive as motivating them to perform better, the majority thought that financial incentives had marginal effect.

To underpin the causal pathways of marginal effect of financial incentives on staff motivation, the evaluation attempted to understand other factors that powered staff motivation for improved care quality.

Improved working environment had positive effect on staff motivation. Respondents from all studied facilities talked about experiencing changes in their working conditions since the introduction of RBF, including staff dynamics, structural changes due to the RBF subsidies, and changes in supervision and training. A majority of respondents considered improvements in physical working conditions to be the most crucial change with the introduction of RBF. With the subsidies received, facilities in both study groups had improved infrastructure, availability of required essential medicines, and equipment. In addition, facilities were also able to improve sanitary conditions and hygiene. These improvements in working conditions had a positive influence on staff morale and ability to improve performance.

ESS contributed to staff capacity building, knowledge sharing and consequently to personnel motivation. The ESS approach introduced and encouraged by the RBF project influenced capacity of facility management and staff. The benefits of ESS—with its enhanced mentoring/coaching element—are widely recognized by CQI facility respondents that helped to understand how to improve internal supervision practices, apply different CQI tools, and find effective ways to process improvements. ESS is perceived as an influential tool contributing towards knowledge and skills building, thus motivating staff performance.

Improved performance in its term encourages staff to further advance their performance. Attaining better results was found to be another factor encouraging facilities to further improve their performance. Accomplishments on targets encouraged facilities to attend to other priority areas. Improvements resulting from implementation of streamlined procedures led by individual champions, are another motivating factor increasing staff confidence and self-esteem, and promoting personnel dedication and buy-in.

- The DHE comes and supervise us. We sit together and discuss… They do not force us to do things, rather encouraging us to work harder. This motivates us as a team…

- It’s not a fault-finding mission. They are coming to support and to motivate us in a good way…

Quotes from key informant interviews

- Our patients are happy with improvements we introduced at our facility… Patient satisfaction motivates us to continuously improve…

- When we see that our results improve from quarter to quarter, we do not stop there, but try to achieve higher results in coming months…

- At out facility we achieved 100% use of partographs and administration of oxytocin. Now we will shift to the next priority to improve management of post-partum period…

- I am proud of being responsible to lead the work on these indicators and will continue to closely monitor these indicators to make sure that they do not deteriorate…

Quotes from key informant Interviews

- We do enjoy having clean working are where all items are easily accessible…

- Availability of medicines and supplies is encouraging… Now we can provide better quality to women and children… This brings more self-satisfaction and motivates…

- RBF subsidies help to introduce innovative approaches… Now we can afford to organize outreach immunization sessions, vaccinate more children and show better results… This is really encouraging…we are enthusiastic to explore other innovative ways how to improve quality…

Quotes from Key Informant interviews

33

Peer review and peer support practices observed in some CQI facilities contributed to staff empowerment and motivation. Intra-hospital cross-departmental peer assessment of the performance was found to be mutually beneficial for participating hospital departments, catalyzing collective learning, teamwork, staff empowerment, and motivation. Examples of peer support between senior and junior staff and/or between best and poorly performing staff members were found to be another effective practice to capacitate staff and motivate behavior for better performance. Lessons learned from these facilities demonstrate a need for system-wide institutionalization of these peer review and support practices.

Knowledge exchange platforms created by CQI districts promoted staff knowledge and motivation. To inspire CQI, knowledge exchange platforms were established at provincial and district levels. Monthly CQI meetings, held at province level, are attended by all DHEs, where issues related to performance are discussed, common challenges identified, remedial measures planned, and best practices presented. At the district level, the social media platform “WhatsApp” is used for knowledge sharing, peer support, and collective problem solving. Respondents noted benefits of opportunities to: discuss issues with their peers and the DHE, learn how other facilities are performing and/or resolve common problems, and advise and support others when needed. There is a handful of good examples observed in CQI districts. The formation of a social interaction network as an unintended consequence of the CQI pilot is an important aspect for health facility employees because it fosters knowledge exchange required for a particular task. This model also serves as an important way to connect often remote and isolated facilities and providers to the wider health system.

Enhanced social control from the communities is another influential factor keeping health facility management and personnel devoted to continuous quality improvement. The CQI pilot fostered stronger linkages between health facilities and community-based organizations (CBOs) by engaging CBOs in the assessment of patient satisfaction, communicating community concerns to QI improvement committees, and in some cases, participating in joint planning of remedial measures. As reported by CQI facilities and CBO representatives interviewed, the given arrangement empowers a sense of community ownership and creates an obligation and motivation to effectively respond to community needs. While this is a commendable development, the evaluation cannot conclude unanimously that social control mechanisms equally influenced facility and staff motivation and overall facility performance at all intervention and non-intervention sites.

5.2.1 Impeding factors

While there is progress in improving clinical quality of MNCH services at CQI facilities, achievement was undermined by external and internal barriers discussed below.

External Factors:

Fragmentation of national and provincial level QA/QI results in inefficiencies and ineffectiveness of interventions. At the national level, QAD operations are undermined by a

- FCH staff come to maternity ward and maternity goes to FCH and we help each other…

- In the team we use peer reviews. e.g. nurse in charge after analysis of partographs, identifies the best performer and calls other nurses to come and see how partograph has to be filled in…


- Close relationship with CBOs helps to understand what we are not doing well, as they regularly undertake patient satisfaction exist surveys… They share main issues with us, and we seek ways of improvements together…

- Even if we do not prioritize some aspect of care, CBOs demand to address them as a first priority… This keeps us alerted to quality improvement…


34

combination of different factors such as fragmentation of QA/QI functions among MOHCC Divisions and lack of horizontal coordination, insufficient staffing, and absence of operational budget. The QI strategy aimed at establishing the QAD with the responsibility for overall operational oversight and coordination for QI, but in reality, the QI function is distributed to different MOHCC structural units such as the Curative Care Division (CCD), the Preventive Care Division (PCD), and QAD—with weak horizontal coordination among them. The CCD and PCD are heavily supported by development partners, whose program/project provides technical and financial support to these divisions, as well as assist in operationalization of QI plans for vertical programs (e.g. HIV/TB, EPI, MCH QA/QI initiatives). Apart from external financial support, these divisions are also well resourced by the MOHCC budget.

In contrast, there is limited support available for QAD, which in the MOHCC hierarchy falls below the CCD and PCD (See Figure 2). Whilst these divisions directly report to Permanent Secretary (PS), QAD reports to Chief Director of Policy, Planning, Monitoring and Evaluation Division and lacks an assigned operational budget.

QAD staffing is another factor limiting QAD to administer its assigned functions. At present, the department has three full time staff: Department Head; Deputy Head; and Secretary, and one part time international consultant seconded to QAD and fully financed by the World Bank project. With the given staffing ratio, location in the hierarchy of the MOHCC, and absence of an operational budget, QAD is not well positioned to administer leadership function in improving health services quality. An imbalance of high responsibility with low power underpins the shortcomings of the QAD’s performance.

According to the Quality Improvement Policy25, each MOHCC technical department is requested to work in collaboration with relevant professional associations and councils as well as the QAD to define a 5-year operational improvement strategy in their respective technical areas, which are aligned to MOHCC strategic objectives. Individual thematic TWGs had to develop a detailed operational plan that prioritizes and sequences specific areas of improvement, indicators, training, and supervisory support for the high-burden conditions in their technical area. However, at the time of the process evaluation, Zimbabwe lacks a comprehensive 5-year quality improvement plan that should be guiding MOHCC operations in improving service quality.

Fragmentation of QA/QI functions at the MOHCC level is mirrored at province, district, and health facility levels and CQI is treated as one among other government programs. At each level there are different teams and QI committees overseeing quality of services within different government programs such as TB, HIV/AIDS, PMTCT, OI, MNCH, etc.

CQI is perceived as another government program, rather than approach to service quality improvement for which there is a separate group of people and committee working on CQI issues.

25 Quality Assurance and Quality Improvement Policy, 2015, Ministry of Health and Child Care Zimbabwe

- Each Division have its own QI plan and budget for implementation. Although we tried to coordinate and elaborate a comprehensive QI plan and budget, where all divisions are assigned to particular tasks, it did not work out. In summary, we fail to have one, streamlined QI plan to guide QI process within the system…Other divisions are supported by development partners and programs allowing these divisions to have budget in support of vertical QI initiatives…

- Being layer below other divisions, undermines our importance and influencing power…

- We are only 2 full-time staff and one part-time quality specialist… - The department does not have budget. In order to work on clinical

protocols, we mostly rely on DP support… We cannot travel regularly for M&E to see how CQI functions at province and district levels, what are the challenges and best practices…


- At the province level we have number of programs which have its own QI component. Supervision visits are paid to facilities separately by different program teams using program specific QI check-lists…

- For CQI there is a team of three individuals who only check performance against CQI indicators…”


35

Consequently, the presence of several QI committees increases the administrative burden and makes both the system and processes inefficient and ineffective.

Attempts to integrate all QI interventions into one comprehensive QI plan failed at province, district, and district hospital levels. Inefficiency of QI committee operations at all levels is well-acknowledged. Integration of all QI interventions into one comprehensive QI plan using CQI has been discussed and probed at province, district, and CQI facility levels, but without success. Respondents noted having limited authority to make changes. Without adjustments of QI structure at the central level and standardization of approaches and processes, they are not in a position to ensure improvements in QI committee performance.

New national monetary policies inhibit effective implementation of CQI at intervention and non-intervention sites. Before 2019, financial transactions using USD were permitted and RBF project used USD for disbursing RBF subsidies to health facilities. In June 2019, the Reserve Bank of Zimbabwe abolished the multiple currency system and replaced it with a new Zimbabwe dollar (RTGS) Dollar26 and set exchange rate 1 RTGS to 1 US$. The new monetary policy greatly affected the volume of RBF subsidies received by facilities and restricted their ability to ensure adequate supply of medicines and/or implementation of planned quality improvement activities at the initial phase of policy implementation. Though the GoZ’s approach has changed lately to use inter-bank exchange rate instead of original proposed one, but at the time of the qualitative process evaluation (Phase 2) facilities have not yet received their RBF funding.

Delays in payment of RBF subsidies result in reduction of purchasing power and shortages in the procurement of goods for quality improvement. In January 2018, responsibility for the payment of RBF subsidies was handed to the MOHCC from Cordaid. Respondents at CQI and non-CQI facilities alike flagged problems related to the delays in performance-based payments as well as lower levels of payments due to the recently introduced government monetary policy.

Another impediment was decreasing purchasing power of the subsidies due to observed delays in payments by the MOHCC. Estimation of RBF quarterly subsidies is performed by the MOFED based on the intrabank exchange rate (starting from the quarter 2, 2019) on the day of budget preparation. Variations in the exchange rate until subsidies reach the facility negatively affect purchasing power of subsidies allocated for the procurement of commodities in the quantities planned. The situation is further worsened by changing prices of health commodities in the market due to the frequent fluctuation of RTGS/US$ exchange rate.

26 The RTGS Dollar, is Zimbabwe’s new currency since 21 February 2019 and came into effect after the February 2019 Monetary Policy by the Governor of the Reserve Bank of Zimbabwe. The currency is made up of bond coins, bond notes and RTGS balances. The original bond notes came in to become part of Zimbabwe's multicurrency system which was introduced in 2009. They were renamed RTGS dollars in 2019, and in June 2019 became the only legal currency in Zimbabwe, replacing the multicurrency system

- CQI is a vertical program and at times integration is not possible because some programs are better paying, and some more are willing to spend only an hour at a health facility…

- Everything should start from the top. Everything we are doing as a way for measuring CQI just ends there… That’s how QI is organized at central level and cascaded to levels below…

- TB only looks at mentoring and spends less time at a facility. So, the vertical disease specific QI initiatives cannot be integrated with CQI because by the time CQI is finished the TB team will have completed their work already...

Quotes from Key Informants at Provincial, District and CQI facilities

- Subsidies were calculated at exchange rate of 3 RTGS:1 US$, but when funds reached us the exchange rate was already 10.4 RTGS to 1 US$

- We often fail to complete our procurement process, because although we collect 3 quotations, at the date funds are received, prices are changed by suppliers and we have to start the whole process again. This delays procurement of medicines and supplies and affects our performance …


36

Observed challenges adversely affect their performance in the next reporting period. The latter ultimately results in lower pay and further limitations for quality improvement.

Human resource for health planning along with a government freeze of employment resulted in staff shortages and adversely affected quality of care. The evaluation observed improvement of staff posting in both groups of studied health facilities. However, the evaluation discovered that post establishments were set around the time of independence in the country and have never been revised since that time. Along with inadequate staff establishment, other challenges are the maldistribution of health workers and the associated difficulty of filling health worker posts, particularly in rural areas, which has long been recognized as a serious challenge to the equitable provision of healthcare.

Health sector staffing was further affected by recruitment freeze introduced in 2010. In response to deteriorated economic growth and in search of efficiency gains, an employment freeze was introduced to control the public service wage bill. Though the freeze was occasionally temporarily lifted and short windows of hiring were provided between 2011 and 2013, staffing levels continued to be inadequate. The combination of low establishments and staff shortages caused managers of studied health facilities to complain of the difficulty of managing the workload with insufficient staff. The recent amnesty for nurses has slightly improved workloads as posts have been filled by returning nurses and/or junior nurses (though most junior nurses lack practical experience). The gradual loss of skills among returning nurses and insufficient knowledge and lack of hands-on experience of juniors, has required more mentoring and training from already practicing senior staff.

Internal Factors:

The effectiveness of CQI trainings was challenged by the training content, intensity, and

mode of delivery. While cascade trainings

have been beneficial in the early stages of

the CQI implementation as a mechanism for

getting CQI ‘off the ground’, supporting

staff at all levels to become familiar with

CQI concepts and tools and increasing

awareness and buy-in to CQI was hampered

by:

Content of the training – Respondents commented on the lack of balance between theoretical knowledge transfer and practical skill development during the training course. Training participants also acknowledged complexity of issues and recommended simplification of the training and/or extend the duration from the current five days.

Inability and/or unwillingness to transfer knowledge – Considering the complexity of the training, not all trained professionals were able to fully absorb knowledge delivered during the training and/or are able or willing to further transfer it to other staff members.

- Training was more theoretical than practical. The practical side was lacking…

- The course is very difficult to understand… - The training is too congested and theoretical delivered in 5

days. Because of its intensity after the training we could only remember the 5s. I think we need to have to redesign the training programme to be easily digested because this becomes complicated…


- After CQI training we came back and give feedback, but it was very difficult for people to grasp the concepts…

- We were trained in CQI but it’s very difficult to get someone who will explain everything to you, because that person also has her own way of understanding…


37

High staff turnover – Staff trained in CQI moved to other facilities and/or locations, and their replacements lacked the knowledge of CQI, hence challenging effective quality improvement.

High staff workload – In the context of observed staff shortages, staff have high workloads that limit transfer of CQI knowledge to newcomers. The evaluation found that at the time of the qualitative process evaluation, most of the newly deployed staff have not yet been trained in CQI.

Lack of clinical training opportunities impedes MNCH quality improvement. Provision of the clinical training was not originally planned under the CQI pilot. Nonetheless, some health workers at CQI facilities have attended different training courses supported by the MOHCC with financial support from developing partners. Respondents noted a need for more trainings due to the high staff turnover and sub-optimal in-house transfer of knowledge by trained staff. A need for more clinical trainings was also mentioned by respondent of non-CQI facilities.

The effectiveness of supportive supervision and mentoring/coaching from provincial and district levels was confronted by a number of challenges: According to respondents, supervision is not always followed by mentoring/coaching of health facilities. The effectiveness of supervision from provincial and district levels is hampered by a number of factors:

Staff shortages at provincial and district levels and high workload limit effective supervision, coaching, and mentoring by provincial and district health authorities. Respondents highlighted a difficulty in assembling a team for CQI supervision visits. Often there is a shortage of staff and high staff turnover, and those deployed have a high workload.

Time spent at health facilities restricts mentoring and coaching. Time spent at health facilities during the supervision is defined by the PHE/DHE teams’ plan on number of facilities they have to visit that day. PHEs and DHEs report that on average they spent 3-4 hours per facility. Time required for checking data and meeting with the staff for feedback often does not leave the room for coaching and mentoring, as teams have to move to another health facility. Supervisory teams acknowledge that they are supposed to supervise every department and coach staff but are not always able to allocate sufficient time for the given task. A need for more mentoring and coaching was also confirmed by key informants at CQI facilities. The majority noted that supervisory teams mostly come for supervision, but not for coaching and mentoring.

Observed delays in RBF funded performance payments impede quality improvement at CQI facilities. Respondents at CQI and non-CQI facilities complained about delays in performance-based

- Two health workers from each CQI health facility were trained. But because of staff turnover some of the health facilities now have none of its staff trained…

- I have not been trained for the CQI yet. I have been here for about four months…

- When the team from the district came, we did a little verification that is when I picked one or two things…

- There are now 5 nurses trained in the clinic and the other nurses are not trained. This compromises on quality…

- We need training.

Quotes from Key informants at CQI facility

- Duration of supervisory visit is based on the CQI needs of the facility. it’s just estimated time…

- We suppose to supervise department by department, but we fail due to time limitations…

Quotes from Key informants at PHE, DHE

- Most of the time the PHE team comes for supervision but there is little time left for mentoring…

- Out mentors come, supervise, provide feedback and advise us how to improve. But we need more mentoring and coaching to increase our knowledge and skills in applying CQI…

- They don’t make time for coaching…


- Due to staff shortages and workload it takes a while to have a full complement of the team for supportive supervision hence usually when we go out the team fails to have all representatives and at most only three members go…

- There is need to add more members to the team. There are times when the DMO and DNO are not there and the pharm tech is on leave…

Quotes from key Informants at CQI facilities

38

payments. Absence of sufficient funds to ensure quality improvements adversely affects their performance in the next reporting period. The latter ultimately results in lower pay and further limitations for quality improvement.

39

6. Quantitative results

6.1 Primary healthcare facility results

Available data on at least some indicators in 2016 and 2018 covered 24 PHC facilities that received the CQI intervention, 230 control facilities in the 13 comparator districts, and 63 control facilities in the same districts. At baseline, most facilities performed well on the input measures but had poorer performance on the process measures. This is likely due, at least in part, to the participation of all study districts in the RBF program. For example, across all facilities, facilities had on average 79% of the general equipment items such as radio, mosquito nets and adult weighing scales. Of the input measures, the facilities performed the worst on the HMIS completion and correctness indicators and the highest on the privacy indicators. Performance on process measures was generally worse than input measures at baseline. For example, all facilities completed only 62% of ANC processes on average. PHC facilities by far performed the worst on obstetric complications, with facilities only completing 3% of the items on average. There were several individual indicators, including correct management of intra- or post-partum sepsis and resuscitation of newborns with APGAR scores below 5 that no PHCs had completed in the fourth quarter of 2016 (Annex 3). This suggests PHCs should not be handling obstetric complications.

Table 6.1: Balance of PHC summary outcomes in 2016

Outcome CQI Control 1

Control 2

T test p-value 1

T test p-value 2

N CQI

N control 1

N control

2

Input measures

General infrastructure: % of 11 items 73% 69% 63% 0.23 0.02 24 230 63

Cleanliness: % of 6 items 88% 87% 85% 0.60 0.41 24 211 62

Safety: % of 4 items 80% 66% 80% 0.01 0.93 24 230 63

Privacy: % of 3 items 96% 95% 93% 0.73 0.33 24 205 60

General equipment: % of 7 items 82% 78% 81% 0.17 0.74 24 214 62

Pharmacy management: % of 9 items 79% 76% 84% 0.35 0.17 24 203 61

Pharmacy stock: % of 25 items 78% 79% 80% 0.77 0.47 24 213 61

Availability of guidelines: % of 6 items 90% 90% 88% 1.00 0.57 24 204 60

Vaccine management: % of 17 items 86% 88% 88% 0.51 0.66 19 178 37

Maternity supplies: % of 11 items 77% 70% 80% 0.08 0.45 23 168 59

HMIS: % of 6 items 66% 62% 68% 0.51 0.77 23 209 59

OPD triaging knowledge: % of 4 items 95% 87% 96% 0.07 0.74 24 207 60

TB screening knowledge: % of 5 items 96% 92% 96% 0.25 0.95 19 182 50

Processes measures

ANC process quality: % of 5 items 65% 60% 69% 0.25 0.47 19 217 46

PNC process quality: % of 5 items 69% 71% 66% 0.59 0.61 20 226 48

PMTCT process quality: % of 4 items 46% 38% 42% 0.36 0.71 17 212 49

Sick child assessment quality: % of 3 items 57% 38% 73% 0.05 0.12 17 225 44

40

Sick child treatment quality: % of 5 items 32% 35% 42% 0.53 0.08 17 225 44

Maternal process quality: % of 10 items 68% 59% 65% 0.11 0.74 16 203 46

Obstetric complications quality: % of 7 items 2% 3% 1% 0.59 0.32 19 211 47

Where Control 1 are PHC facilities in the 13 non-intervention districts and Control 2 are non-intervention facilities within the CQI districts

The majority of summary outcomes were balanced between CQI and both groups of control facilities at baseline. Where the indicators differ, CQI facilities generally performed better than the control facilities in other districts (Control 1). For example, CQI facilities on average scored 80% on safety indicators in comparison to 66% in control facilities from other districts. Similarly, CQI facilities on average scored 57% on assessing sick child care for pneumonia, diarrhea and malaria, while control facilities from other districts scored 38%. Differences between the CQI facilities and control facilities in their own districts (Control 2) were minor for most summary measures, although general infrastructure was higher in CQI facilities at baseline. There are select differences between treatment and control facilities in other districts at the individual outcome level, in these cases the CQI facilities also tend to score better than the control facilities (Annex 3 Table A1). For example, 96% of CQI facilities had handwashing facilities with soap available for outpatients, in comparison with 76% of control facilities. Taken overall, there are few differences between treatment and comparison facilities, especially with regard to the process measures which are the focus of the CQI activities.

CQI and control facilities were largely balanced at baseline on the identified covariates used in the matching estimator (Annex 3 Table A2). A greater percentage of control facilities are clinics (versus RHCs) than CQI facilities. Both CQI and control facilities are a mean distance of 41 km from the district hospital and had similar volumes of new patients in 2016.

The unadjusted estimates for the effect of the CQI intervention on the process quality measures using the ANCOVA model and the other 13 districts as comparison are presented in Table 5.2. CQI was associated with improvement for two of the seven process measures: postnatal care and maternal care. The intervention was associated with completing one half of an additional PNC process item correctly and one additional maternal process item. The individual indicator results reveal that the PNC improvement is driven primarily by better assessment in the CQI facilities: facilities were 25 percentage points more likely to have all women checked for their general condition, pulse rate and temperature in comparison to the control facilities. Among the maternal care process indicators, monitoring for women and newborns in the 4th stage of labor, administering immediate postpartum oxytocin, and monitoring women using partographs were the indicators that improved the most between CQI and control facilities (Annex 3 Table A10). As maternal and post-natal care were two focus areas of the CQI program, these results suggest that CQI related efforts may have improved process quality of care in these areas. As expected, the intervention was not associated with improvements of any of the input quality measures (Annex 3 Tables A3 and A4).

Table 6.2: Unadjusted PHC summary ANCOVA results

CQI Baseline Total N

Process quality Coef. p-val Coef. p-val

ANC process quality: % of 5 items -0.03 0.35 0.17 0.04 236

PNC process quality: % of 5 items 0.11 0.02 0.21 0.00 246

PMTCT process quality: % of 4 items -0.03 0.75 0.12 0.03 229

41

Sick child assessment quality: % of 3 items 0.02 0.75 0.33 0.00 242

Sick child treatment quality: % of 5 items 0.11 0.16 0.34 0.00 242

Maternal process quality: % of 10 items 0.12 0.00 0.32 0.00 218

Obstetric complications quality: % of 7 items -0.03 0.07 0.27 0.08 230

After matching on the facility covariates, CQI is still significantly associated with improvement

in these two indicators (at a p-value of .10 or less), suggesting that the results are robust to covariate balancing and a narrowing of the comparison set. As before, no other process indicator is significantly improved in the CEM ANCOVA analysis.

Table 6.3: Adjusted PHC summary ANCOVA results

CQI Baseline N





Sick child assessment quality: % of 3 items -0.02 0.75 0.35 0.01 107




Figure 6.1. Trend in maternal process quality over study period in PHC CQI and control facilities in 13 other districts

42

In the difference in differences model, the PNC process result was robust, but the maternal quality process improvement was no longer statistically significant. Standard errors are typically higher in difference-in-difference models, especially in the presence of measurement error in the differenced variable. In addition, when using the difference in differences model, assessment of sick child care was worse among CQI facilities in comparison to the control facilities. The graph of this trend (Annex 3 graphs) suggests this is due to particular poor performance in sick child assessment in control facilities at baseline rather than poor performance among CQI facilities.

When comparing the intervention facilities against control facilities in their districts (Tables 6.4 and 6.5), maternal care quality improvement is no longer significant at standard levels, although the coefficients are positive, of similar magnitude, and attain p-values of .13 (unmatched) and .15 (matched). Post-natal care quality still shows significant improvement, though the magnitude of the improvement is smaller than what was observed using the other districts as comparison. Sick child treatment quality is improved in the unmatched analysis, but the coefficient is very small and this association disappears in the matched analysis. There is no change in any of the other process measures. Given all of these robustness checks, it suggests that the CQI intervention did improve post-natal care quality in the intervention facilities, and likely maternal care-quality as well, but no other process quality measures. In addition, as expected given the focus of the CQI activities, no improvement in input quality measures was observed.

Table 6.4. Unadjusted PHC summary ANCOVA results using same-district comparators

CQI Baseline N


ANC process quality: % of 5 items 0.02 0.54 -0.06 0.53 65






Obstetric complications quality: % of 7 items -0.02 0.32 -0.08 0.60 66

Table 6.5. Adjusted PHC summary ANCOVA results using same-district comparators

CQI Baseline N


ANC process quality: % of 5 items 0.05 0.17 -0.10 0.28 52







43

When looking at the descriptive trends of input or process quality measures, there appears to be large heterogeneity by quarter rather than smooth trends in many of the summary indicators (Annex 3 graphs). PNC process quality seems to improve in both CQI and control facilities just after the CQI intervention began but then revert down by the end of 2018. CQI facilities did not consistently perform better than control facilities over this period.

Figure 6.2 Trend in PNC process quality over study period in PHC CQI and control facilities in 13 other districts

The qualitative section of the report found that the funding through RBF was important in improving inputs or structural items in both CQI and control facilities, while the CQI intervention was thought to have a larger impact on processes of care. These quantitative results are partially consistent with this conclusion. Although the CQI intervention did improve maternal and post-natal care processes, these are limited when considering the full breadth of potential MNCH process quality items.

6.2 Hospital results Five hospitals implemented CQI, however data was unavailable from one of the facilities in Q4

of 2016, and it was therefore dropped from the analysis. Data was available for at least some indicators from sixteen other hospitals. Similar to the PHCs, hospitals overall performed better on the quality of inputs measures than the quality of processes at baseline. For example, nearly all facilities had all appropriate guidelines available and staff scored an average of 97% on knowledge of outpatient triaging. Process quality was generally lower than input quality in 2016, although facilities still scored highly on both maternal care processes and paediatric care processes. Hospital performance was generally higher than PHC performance at baseline, although the measures are not directly comparable due to different indicators included in the summary measures.

44

Table 6.6: Balance of hospital summary outcomes in 2016

Outcome CQI Control T test p-value N CQI

N Control

Inputs

General infrastructure: % of 18 items 83% 84% 0.82 4 14

Cleanliness: % of 7 items 86% 88% 0.58 4 14

Safety: % of 24 items 89% 78% 0.14 4 16

Privacy: % of 4 items 94% 81% 0.15 4 13

Patient amenities: % of 6 items 95% 78% 0.06 4 13

General equipment: % of 19 items 81% 79% 0.72 4 15

HR management: % of 13 items 88% 83% 0.47 4 17

Surgical, lab and radiology equipment: % of 16 items 88% 84% 0.68 3 15

Pharmacy management: % of 16 items 75% 78% 0.86 3 15

Pharmacy stock: % of 21 items 76% 77% 0.97 4 16

Guidelines availability: % of 8 items 100% 96% 0.19 3 10

Vaccine management: % of 16 items 97% 83% 0.05 2 10

Maternity supplies: % of 17 items 88% 82% 0.46 2 13

HMIS indicators: % of 9 items 83% 84% 0.90 4 16

Quality improvement strategy: % of 22 items 71% 48% 0.15 3 10

OPD triaging knowledge: % of 2 items 100% 90% 0.17 3 10

TB screening knowledge: % of 10 items 80% 91% 0.18 3 9

Processes

ANC process quality: % of 7 items 76% 43% 0.09 3 12

PNC process quality: % of 5 items 80% 70% 0.54 3 14

Sick child assessment quality: % of 3 items 67% 39% 0.51 3 11

Sick child treatment quality: % of 5 items 53% 51% 0.91 3 11

Maternal process quality: % of 10 items 87% 73% 0.16 3 14

Obstetric complications quality: % of 8 items 79% 61% 0.34 3 14

Paediatric care processes: % of 4 items 100% 69% 0.02 3 12

Paediatric complications processes: % of 4 items 75% 52% 0.47 3 11

Surgical safety: % of 2 items 83% 59% 0.31 3 11

Also similar to PHCs, there were few differences in outcomes between CQI hospitals and

control hospitals in 2016 but where there were, CQI hospitals had better performance. All CQI hospitals completed all of the recommended basic paediatric care items, while control facilities scored 69% on average. CQI facilities also had better maternal care quality than control facilities at baseline. Although not statistically significant, it is worth noting that quality improvement plans and strategies

45

were already higher in CQI facilities before the start of the intervention, with an average of 71% of the 22 quality improvement items in comparison with 48% in control facilities.

The estimates for the effect of the CQI intervention on the outcome measures in hospitals using the ANCOVA model are presented in Table 6.7. The study is not powered to find significant associations among the limited number of hospitals, and so unsurprisingly there are no significant associations. Even given the limited sample size, however, many outcomes have CQI coefficients close to zero, suggesting that there may not be significant effects even with larger sample sizes. The largest positive coefficient is for sick child treatment quality. The coefficient for CQI on paediatric care processes was large and negative, however that is due to the high performance of CQI facilities on this outcome at baseline as described above. The intervention was also not associated with any of the input quality measures in hospitals (Annex 3 Table A11).

Table 6.7: Hospital summary ANCOVA results

CQI Baseline N



PNC process quality: % of 5 items -0.05 0.73 0.05 0.77 17




Obstetric complications quality: % of 8 items -0.01 0.95 -0.15 0.47 17

Paediatric care processes: % of 4 items -0.22 0.15 0.38 0.04 15

Paediatric complications processes: % of 4 items -0.12 0.45 -0.05 0.78 14

Surgical safety: % of 2 items -0.01 0.98 0.42 0.07 14

Despite the lack of associations in the summary outcomes, CQI was significantly associated with a few of the individual input quality outcomes (Annex 3 Table A14). For example, CQI was associated with a 31 percentage point increased likelihood of having an appropriate waste pit, 18 percentage point increased likelihood of having second line paediatric ART medications, and 47 percentage points more likely to have a functioning ultrasound machine. However, CQI was also associated with a 34 percentage point decreased likelihood of correct treatment for intra-partum or post-partum sepsis, and had no significant associations for the vast majority of indicators.

The results are robust to using the fourth quarter of 2018 as the outcome instead of all of 2018 (Annex 3 Table A12). The difference in differences model does reveal several significant associations, however, all of them show that CQI was negatively associated with the outcome. ANC process quality and paediatric complications processes were both negatively associated with the CQI intervention. The number of ANC items completed correctly decreased by about 2 out of 7 items as a result of the CQI intervention. The graph of these trends shows that although ANC process quality was improving in both CQI and control hospitals, control facilities had more rapid improvement. In contrast, paediatric complications processes increased slightly in control facilities but fell in CQI facilities, particularly in the 4th quarter of 2018. Given large heterogeneity in the data from month to month, it is possible that these results are spurious.

46

Figure 6.3 Trend in ANC process quality over study period in CQI and control hospitals

Figure 6.4 Trend in paediatric complications processes over study period in CQI and control hospitals

The hospital results also differ somewhat than those presented in the qualitative section of the report. First, the qualitative section suggests better MNCH related process quality among CQI facilities. However, the results here suggest that the CQI intervention did not result in better quality improvement processes in hospitals, in part because these processes were already better in CQI facilities prior to start of the intervention. The difference in difference analysis shows that quality improvement processes improved by 32 percentage points in control facilities over the study period, in comparison the CQI facilities improved by a much smaller amount. Second, the qualitative section reported structural improvements in both CQI and control facilities particularly in hospitals due to the higher RBF payments. However, the trends of structural measures in Annex 3 reveal that many inputs actually declined in both CQI and control facilities over the study period. General infrastructure, cleanliness, patient amenities, human resources management, and maternity supplies all saw broad declines from 2016-2018. The structural indicators that saw improvement, including pharmacy

47

management, HMIS indicators and quality improvement, do not require large increases in funding. Finally, the qualitative results report that hospitals demonstrate greater results in improving process and structural quality than PHCs. This finding is not directly comparable in the quantitative results because of the differences in indicators and power to detect differences, however many of the coefficients appear to be the same size or smaller than those seen in the PHC analysis.

48

7. Conclusions and lessons learned

The Zimbabwe health system recently piloted a CQI intervention in select PHCs and district hospitals in five rural districts that had already been implementing a related RBF program over the four previous years. This process evaluation leverages both qualitative and quantitative data to understand whether CQI had an impact on quality of care structures and processes, and factors that contributed to or inhibited its success. The qualitative section drew on key stakeholder in-depth interviews and focus group discussions to identify and understand enablers and barriers to change from the CQI intervention. The quantitative section uses routine data collected by the RBF program in both CQI and comparison facilities to explore possible differential gains in process quality measures in the facilities that participate in CQI activities.

Uptake of CQI is a complex process that engages multiple stakeholders in new relationships that can operationalize key concepts and tools, develop and embed new practices into service systems and routines. At the time of the process evaluation, the CQI pilot is in its experimental phase of implementation. Hence, results achieved so far should be treated with caution.

The progress of internalization of CQI is slowly gaining momentum and has exhibited positive changes in management processes but has shown limited improvement of quality of care processes. Table 7.1 below schematically presents both qualitative and quantitative process evaluation findings on the changes in quality processes, management processes, structural quality and client satisfaction. The pilot demonstrated some improvements in the quality of MNCH services, though achievements are not uniform across all targeted facilities nor all services.

Table 7.1: Comparison between CQI and non-CQI facilities

Domains` CQI Facilities Non-CQI Facilities

District

Hospital

PHC District

Hospital

PHC

Quality processes

ANC quality

PNC quality

PMTCT quality (PHC only)

Sick child care quality

Routine maternal care quality

Obstetric complications quality

Management processes

Leadership

Teamwork

MNCH QI processes

Ownership of clinical quality improvement

Staff engagement

Capacity building

Staff motivation (financial)

Staff motivation (non-financial)

Structure

Client Satisfaction

49

There is some evidence that CQI improved maternal care and postnatal care processes in PHCs, but no evidence of improvement in any inputs, as expected, or any of the hospital processes. Improvements were limited when considering the full breadth of potential outcomes but arise in certain areas of focus of the CQI program.

This absence of broad-based improvement may be due to several factors. First, CQI facilities were already performing better than control facilities at baseline on many indicators. Second, many structural indicators were already very high at baseline. In combination, these suggests that there may have been ceiling effects where CQI facilities were unable to improve anymore but control facilities were. Third, the qualitative results suggest that there was mixed success among different CQI facilities as the intervention interacted differently with the environment, the service, the quality improvement process and the stakeholders. The overall grouping of CQI and control facilities may have masked heterogeneity of improvement, with some facilities performing better.

The limited improvement may also be due to limitations in the checklist data: First, there was a change in measurement system in Q2-Q4 2018. Measurement likely improved non-differentially, but ANCOVA results do not fully adjust for baseline outcomes if there was measurement error. Second, missingness was high, particularly for vaccine management and maternity supplies forms. We assume that missingness was completely at random, however missingness could be related to data management ability. Regardless of bias, it limits the power to detect differences.

The mixed success among different CQI facilities is associated with the interaction of features of the environment, the service, the quality improvement process, and the stakeholders, which operated together to produce a set of circumstances that either inhibited or enabled the process of change. The following section of the report summarizes enablers and barriers influencing effectiveness of the CQI pilot from the qualitative analysis.

7.1 Drivers of change To summarize key drivers of change at health facility level, as seen by health service providers,

the evaluation explored perceptions of respondents on the internal key factors influencing care quality. Respondents were asked to name factors by priority and answers were recorded on a five-point Likert scale and transformed into unidirectional measures with the highest level of influence “5” and the lowest “1”. Results of this exercise are schematically presented in Figure 1.1 below.

Figure 1.1: Prioritization of factors influencing quality improvements

Performance improvements would have been impossible without strong leadership of management teams who drive the process of quality improvement and establishment of teamwork culture. The findings suggest that the extent to which top management becomes directly involved in quality improvement activities determines the degree of quality improvement implementation.

High

Influence

Low

influence

3.4

3.2

3.0

2.9

2.5

Leadership and

teamwork

Staff capacity

building

Supportive

Supervision

Staff financial

incentives

Addition subsidies

50

Management teams at CQI facilities proactively lead the quality improvement initiatives and demonstrated tangible results. Despite the widespread recognition of the importance of effective leadership among respondents, there are some considerable barriers to participation in leadership. Such barriers are noted extensively and include shortage of confidence and poor preparation for leadership/managerial roles. Deeper analysis is required to inform future strategies addressing these barriers to enhance the quality of management and leadership. The findings suggest that a “teamwork” culture, introduced by CQI pilot, provides an important foundation for implementing a quality improvement initiative. CQI facilities, by internalizing teamwork culture, show some quality improvement results compared to non-CQI facilities.

Availability of skilled health workforce—and building their capacity with training for continuous knowledge and skills development along with supportive supervision—are the most influential factors behind care quality improvements. Evidence suggests that with adequate staffing, tailored training, and regular supportive supervision, health care quality can improve even without additional funding. Evidence-based best practices are the foundation upon which successful quality improvement initiatives are built. Developing and integrating evidence-based best practices is not enough; healthcare organizations also need to have ways to measure how consistently best practices are being used and their impact on outcomes. External and internal supportive supervision system introduced in these health facilities have crucial roles in continuous quality improvement. On the other hand, systematic staff shortages undermined attainment of better results.

Monitoring of quality of care is an important and legitimate part of the government’s stewardship role enabling quality improvements at health facilities. Organizational processes to ensure quality and continuous improvement, such as supervision from PHEs and DHEs and within health organizations, is essential to promoting quality improvement at CQI facilities. According to key

informants, the CQI pilot improved communication and cooperation between facility management and PHEs and DHEs facilitated adoption of a shared goal of improving facility performance. This has helped to promote a localized level of stewardship for service delivery and improved data capturing and analysis, and implementation of remedial actions.

Staff financial incentives are considered to have less impact on performance. Health workers interviewed ranked staff financial incentives as the lowest priority for improvement in performance. While they unanimously agree that higher staff remuneration is needed to incentivize staff, they believe that non-financial incentives are more powerful to influence staff motivation for CQI. Informants reported to be ‘awakened’ by improved working environments, routine supportive supervision, opportunities to improve knowledge and skills, changed relationships among staff and managers, more cohesive teams, and availability of essential drugs and equipment.

The core to sustained quality improvement lies in good leadership, teamwork, and availability of skilled labor. All respondents agree that adequate funding allowing regular maintenance of facilities, procurement of medical equipment, medicines and consumables is important, but without sound leadership, teamwork, and skilled staff, availability of equipment, medicines, and supplies cannot ensure quality improvement.

7.2 Barriers to change Attainment of better results was hampered by external and internal factors detailed below

and schematically presented on Figure 7.2.

51

Figure 7.2: External and Internal factors negatively affecting effectiveness of CQI

External Barriers

General health system related bottlenecks challenged effectiveness of the CQI pilot. These include:

- Freeze on employment: The government-wide policy of freezing of posts in 2010, initiated by the MOFED, compounded the problem of the insufficient quantity of established posts. The freeze affected vacancy rates for doctors and Environmental Health Professionals negatively, in spite of the three episodes of the temporary unfreezing in 2011, 2012, and 2013. For nursing posts, vacancy rates improved as a result of the temporary unfreezing of posts.

- Staff to population ratio, staff shortage, high staff turn-over and high workload: Staff deployment to some facilities has not been consistent with the catchment population and staffing plans. The staffing situation in both the government-run facilities and faith-based health facilities is the result of the number of posts that the government funds and the availability of staff willing and able to take up the posts. As in other sectors, the number of funded – or ‘established’ – posts per health facility has been reduced in line with overall spending cuts, rather than increased in line with health service expansion or the upgrading of facilities – for example, of health centers to hospitals. In the absence of adequate staff, facility personnel are overstretched, and high workload often limits them to focus on the quality improvement. Frequent staff movement is another challenge faced by the system. Specifically, respondents reported gradually losing people knowledgeable in CQI and/or clinical practices at province, district, and facility levels that evidently affect continuity in quality improvement. Addressing structural factors affecting health human resource supply is definitely beyond the project reach as it requires multi-sectoral and bold government action. The challenge, by far, is not unique to Zimbabwe; most countries around the globe face similar problems,27 which has led the WHO to propose a global strategy on human resources for health28. Consequently, the evaluation team thinks, it would not be fair to expect RBF project to resolve this problem.

- Sub-optimal medical and nursing education system to produce professional health workforce and limited continuous medical education opportunities: Inadequate knowledge and skills of

27 Evans TG, Correia Araujo E, Herbst C, Pannenborg. Transforming Health Workers’ Education for Universal Health Coverage: Global Challenges and Recommendations. World Health & Population 17(3) September 2017: 70-80. 28 WHO 2016. Global strategy on human resources for health: workforce 2030

Fragmentation of QA/QI

function

INTERNALEXTERNAL

New monetary policy and delays in RBF subsidy payments

Health human resource planning and freeze on

employment

Sub-optimal medical and nursing education, limited access to

continuous medical education

Effectiveness of CQI training

Quality of mentoring/coaching

Delays in RBF subsidy payments

Staff ability and willingness to transfer knowledge and skills

52

new graduates entering the system slows quality improvements. The latter is further aggravated by limited professional development and medical education opportunities enabling filling in the knowledge gap. To a large extent, access to such trainings is mostly granted by donor financed projects.

- Many facilities, including CQI facilities, have space shortages, which undermines quality of care. While health clinics were able to expand space availability through construction of new wings using RBF funding, not all hospitals could afford new construction.

- New monetary policy with observed delays in RBF funded performance payments along with the lowered costs of performance indicators impede quality improvement at CQI facilities. Since the transfer of the RBF subsidy payment function from Cordaid to the MOHCC, respondents in both groups of studied health facilities flagged problems related to delays in performance-based payments, as well as lower payments due to the recently introduced government monetary policy. Although the government adjusted the exchange rate, at the time of the study, PME facilities still awaited subsidies denominated in US$ and recalculated based on the intra-bank exchange rate. Another important impediment extensively discussed by respondents was the decreasing purchasing power of the subsidies due to observed delays in payments by the MOHCC and instability of commodity prices due to variation of exchange rate. These resulted in the absence of sufficient funds to ensure quality improvements, which adversely affected their performance for the next reporting period. The latter ultimately resulted in lower pay and further limited quality improvements.

Internal Barriers:

- Inadequate leadership from the central level in developing management structures, flow of resources, monitoring and evaluation, and providing adequate guidance to the CQI pilot challenged attainment of better results. Fragmentation of QA/QI function among different divisions and directorates of the MOHCC, a less empowered QAD due to insufficient staffing and lack of operational budget, impede effectiveness and efficiency of the QA/QI system. The country lacks an integrated health quality improvement program and plan that translates QA/QI strategy into action. Shortage of staff and absence of an operational budget restrict the QAD from adequate monitoring, developing/updating clinical guidelines, or providing training and regular support to QI structures at provincial, district, and facility levels.

- Fragmentation of QA/QI functions at the central level is mirrored at sub-national and local levels and reduces the effectiveness and efficiency of supervision. Supervision is another important planned intervention under CQI pilot. However, fragmentation of teams and QI committees overseeing quality of services within different government programs hinders regularity of supervision visits and decreases time spent at the facility for coaching and mentoring.

- Gaps in implementation of effective strategies to build health worker knowledge and skills obstructs CQI. Large numbers of health workers, a short time period, and financial restrictions forced the GoZ to opt for cascade training strategy to introduce CQI. This strategy undermined the quality of services delivered at the local levels. CQI training mostly relies on didactic teaching, with limited on-the-job follow-up and practical skills-building. However, didactic training does not effectively ensure the translation of theoretical knowledge into practice or address system-level barriers29, neither does the cascade training approach promote transfer of knowledge and skills. The given approach failed to ensure transfer of knowledge and development of skills in CQI and

29 Pariyo GW, Gouws E, Bryce J, Burnham G, Uganda IMCI Impact Study Team. Improving facility-based care for sick children in Uganda: training is not enough. Health Policy Plan. 2005 Dec; 20 Suppl 1():i58-i68.

53

undermined achievement of better results. For example, insufficient knowledge in data analysis and planning has been named as a challenge to quality improvement at CQI facilities.

- The quality of supervision has been also affected by deficient knowledge and hands-on experience of the supervisors to mentor and coach. Problem solving, clinical supervision, and community involvement were less commonly part of the quarterly supervision. Coaching and mentoring require skills that the project tried to develop through training. However, not all supervisors attained knowledge and skills required for quality supervision. In a few instances, respondents reported confusing or even useless advice from supervisors.

- Observed delays in RBF funded performance payments along with the lowered costs of performance indicators impeded quality improvement at CQI facilities from the start of the project. CQI and non-CQI facilities complained about delays in RBF subsidy payments experienced from the start of the project (when RBF payments were processed by Cordaid), which limited their ability to carry out planned quality improvement activities. These delays unfavorably affected their performance for the next quarter and resulted in lower pay for the next reporting period, consequently constraining further quality improvement. Other than late payments, respondents also mentioned lowered costs of performance indicators affecting volume of funds received by health facility - funds normally used for quality improvement and staff motivation purposes.

7.3 Role of CQI interventions

The qualitative results indicate that the CQI facilities saw greater leadership, teamwork, ownership of clinical quality improvement, engagement, and staff motivation. These could be important outcomes in and of themselves but may not necessarily translate into higher quality care. This reveals a major limitation of the CQI approach: some items are not within the power of the facility staff to solve through continuous quality improvement. For example, the CQI methodology cannot address the challenges with staff shortages, high staff turnover and high workload. These problems may greatly affect care processes but need to be addressed at the national or district levels. The quantitative results also show the extremely low performance of the PHCs on managing obstetric and newborn complications. Improved staff motivation and teamwork will not overcome challenges of lack of training or practice on these skills, rather it suggests that to improve quality, only hospitals should handle these complications.

These findings align with the conclusions of the recent Lancet Global Health Commission on High Quality Health Systems.30 The Commission argues that micro-level strategies to improve quality at the level of providers or facilities will likely be insufficient by themselves. Rather, macro and meso level strategies are likely needed at the national, provincial and district levels in order to improve overall health system quality. The large gaps observed in the quality of care cannot be addressed through targeted quality management when the root causes may be high workload, large turnover, inadequate clinical training, or simply providing a service at an inappropriate level of the system. Macro or meso level strategies to address these challenges would include addressing gaps in governance, reorganizing services to be provided at the appropriate level (i.e. moving deliveries to hospitals that can manage obstetric complications), strengthening pre-service clinical education, or establishing facility learning networks. While CQI may contribute to a better working environment and spur improvement in routine care practices that the staff are already well trained in, it should be seen as one tool in a broader health systems quality improvement strategy.

30 Kruk, M. E., Gage, A. D., Arsenault, C., Jordan, K., Leslie, H. H., Roder-DeWan, S., ... & English, M. (2018). High-quality health systems in the Sustainable Development Goals era: time for a revolution. The Lancet Global Health, 6(11), e1196-e1252.

54

7.4 Lessons Learned There are several key lessons learned that can be useful to any country who plans to introduce and/or enhance QA/QI system in the health sector.

Lesson 1: Optimal CQI benefits to overall quality of care are attained through a health systems approach that uses macro and meso level strategies in addition to CQI.

Lesson 2: Tangible improvements in quality of care are dependent on a streamlined, yet adequately staffed and well-budgeted CQI organizational structure at all levels, along with well-defined and enforced procedures.

Lesson 3: Supervision produces greater improvements. However, issues relating to frequency, intensity (only supervision vs. supervision and coaching), depth, duration and quality of mentoring/coaching (including capacity of mentors and coaches) must be addressed to optimize the return on the CQI investment.

Lesson 4: Development of mentoring and coaching capabilities at all levels of the health care system should be integral to the design of QA/QI initiatives. The selection of people with the right knowledge and skills for the mentor/coach role -and strategic plans to train and support them - are important to consider when initiating and especially when scaling up a CQI intervention.

55

8. Recommended Approach to CQI Scale-up and Institutionalization

Experience has demonstrated that technical interventions shown to be effective at a small scale under tightly controlled conditions cannot naturally be assumed to be successful at scale-up, despite intervention scale-up being essential for population-level impact. Scale-up is challenging, and not always successful31. CQI scale-up and institutionalization is an ongoing process where activities related to defining, measuring, and improving quality become formally and philosophically integrated into the structure and functioning of a healthcare organization and system. It is not a linear process, but rather a fluid one in which the essential elements may mature in sequence or in a less coordinated fashion. There is no one formula or set of steps an organization should follow to successfully institutionalize CQI. Nevertheless, considering Zimbabwe’s current context and factors that impeded attainment of the better results in CQI pilot sites, the process evaluation team suggests that the Government applies a two-prong CQI strategy should it wish to scale up CQI implementation: vertical scale-up followed by horizontal scale-up (see Figure 8.1).

Figure 8.1: Strategies for CQI pilot scale-up and institutionalization

The vertical scaling up strategy involves institutionalization of CQI through policy, legal, budgetary or other health systems change, whereas the horizontal scaling up strategy implies expansion or replication of the CQI approach in different geographic locations and/or healthcare facilities and/or roll-out of CQI pilot

to departments other than MNCH wards of the pilot facilities.

8.1 Essential elements for vertical CQI scale-up and institutionalization

Many factors affect a health system’s and organization’s ability to institutionalize CQI and a culture of quality, but there are key elements defined essential for implementing and sustaining the core quality improvement activities32 grouped into three main categories, depicted on Figure 8.2 and detailed below. Consequently, recommendations for the scale-up and institutionalization of the CQI are grouped per each category of essential elements for implementing and sustaining the core quality improvement activities. The category “Enabling Environment” refers to the necessary conditions within the health

system’s and organization’s internal environment, which benefit and facilitate the process of

CQI institutionalization. The specific essential factors to be considered in this category are: a)

institutional policies which are conducive and help guide the process; b) development of

leadership which establishes priorities and guides the staff; c) availability of human, financial,

and material resources for the implementation of CQI; and d) values that emphasize quality

31 Jeffrey Michael Smith et al, scaling up high-impact interventions: How is it done? International Journal of Gynecology and Obstetrics, 2015, 130, S4-S10 32 ibid

Institutionalization of CQI

through policy, political,

legal, budgetary and other

health systems change

VERTICAL SCALE UP

Expansion or replication of

the CQI pilot in different

geographic locations or

healthcare facilities or to

other departments of pilot

health facilities

HORIZONTAL SCALE-UP

01

02

Strategies for

CQI scale-up

56

and improvement. More specifically, for enhancement of the enabling environment, the

MOHCC is advised to consider the following recommendations.

Figure 8.2: Essential elements for CQI institutionalization

Recommendation 1: Enforce integrated health Quality programming, planning, and implementation. The absence of the integrated quality improvement program along with fragmentation of CQI processes impede potential gains of CQI. Hence, it is strongly recommended that a cohesive health quality program and plan that integrates and harmonizes all quality programs of every health service across vertical

programs is developed and enforced.

Recommendation 2: Develop CQI operational manual and standard operating procedures. To ensure consistency in application of the CQI approach during the scale-up, the MOHCC should facilitate the development of the CQI operational manual and standard operating procedures (SOP) to guide national, sub-national, and health facility staff in CQI.

Recommendation 3: Enforce legislation supporting health facility autonomy. The PME revealed that health facilities with flexibility to plan for quality improvements along with access to funds at their disposal to be used as needed, contributed to improvement of quality. While everybody appreciates the benefits of autonomy, they also realize that autonomy is granted only for the RBF project and will be lost after the project closure. To ensure that quality improvements are sustained, the government is advised to enforce legislation supporting health facility autonomy.

Recommendation 4: Streamline and strengthen CQI governance and leadership at national, provincial and district levels to drive change. Streamlining and strengthening CQI governance at national level should be given a priority before the scale-up. To ensure adequate leadership of the CQI scale-up and institutionalization, it is recommended that the MOHCC: i) Revisits placement of the QA Directorate in the organizational structure of the MOHCC: It is highly recommended that QAD is elevated to the level of division to report to the Permanent Secretary. Right placement of the QA/QI function will give more power and leverage to administer stewardship function; ii) Considers expanding the staffing of the QA/QI division (currently Quality Assurance Department) to allow provision of needed guidance and oversight to central and local levels; and iii) Allocates sufficient resources for effective operation of the QAD. Streamlining and strengthening governance is also required at provincial, district, and facility levels. It is believed that health organizations equipped with CQI department or dedicated staff responsible for CQI would find it much easier to conduct CQI activities.

Recommendation 6: Advancement of supportive supervision mechanisms. The effectiveness of the CQI scale-up process will largely depend on the quality of supervision, which incorporates mentoring/coaching. Enhancement of mentoring and coaching skills of supervisors can transform traditional supervision into a more effective intervention to improve care quality. Furthermore, effective delivery of mentoring and coaching can potentially fill the knowledge gap faced by the health organizations and service providers at the initial stage of CQI scale-up.

Policy Leadership

Core values

Resources

ENABLING ENVIRONMENT

Structure, clear definition of roles and

responsibilities and

accountability mechanisms.

ORGANIZING FOR QUALITYCapacity building Information and

Communication

Rewarding quality

SUPPORT FUNCTIONS

57

Recommendation 7: Build requisite clinical and competitive capabilities. The gap in implementation of effective strategies to build staff skills and knowledge, identified by PME, reflect a need to define and invest in effective approaches to better train and support health workers to deliver quality care. The cascade model, a mechanism delivering training messages from trainers at the central level to trainees at the local level through several layers, is largely used for in-service training, as it can deliver many trained health workers quickly and economically33. However, despite these advantages, this model is often criticized for its ineffectiveness. Its main weakness is the distortion of the messages transferred during the training, because they are passed down through many different levels of personnel. The intended messages are often altered, and their effects are diluted through miscommunication and different interpretations of the same messages34,35,36. Therefore, where possible and resources permit, opportunities for more effective training modalities should be explored by the GoZ.

To ensure that health workforce capacity building efforts are sustained, the GoZ is advised to incorporate these trainings into the pre- and post-service and continuous medical education programs. It is also important to upskill trainers so that they can teach quality improvement methods robustly. More specifically, the PME recommends: i) Integrating CQI principles and tools in the pre- and post-service training programs to generate a cadre of health workers equipped with basic knowledge of CQI; ii) Refining the training syllabus developed as part of the CQI pilot to balance theoretical knowledge with practical skills development and assign a clinical base where trainees can practice acquired knowledge and practical skills; and iii) Integrating CQI training into the continuous medical education program as a mandatory module for all health workers. Capacity building should include, but not be limited to only, CQI principles and tools, clinical training, as well as training of health administrators (particularly at health facility level) in basic management and strategic planning. Particular emphasis should be placed on developing specific training programs for incapacitating supervisors at central, provincial, district, and facility levels in mentoring and coaching.

Recommendation 8: Enhance knowledge exchange. In support of CQI scale-up and institutionalization, develop and institutionalize effective knowledge exchange platforms and encourage collection and dissemination of best practices. Capitalize on already available best practices and institutionalize system-wide.

Recommendation 9: Develop appropriate and affordable incentive structures and mechanisms. The development of incentive structures for the health facilities and within individual health-care institutions is important to influence organizational strategy and individual decision making. Therefore, the MOHCC is advised to develop incentive structures, which, in addition to financial incentives, will also consider modalities of non-financial staff motivation.

Recommendation 10: Establish a national evaluation, quality monitoring, measurement, and reporting system. The country should have a well-established QA/QI evaluation, monitoring, and measurement system to inform on the strength and weakness of the current system. Evaluation refers to the formal way in which information is gathered, interpreted, and evaluated in relation to the set standards and criteria and requires monitoring and measurement. Thus, the MOHCC is advised to develop: i) a QA/QI monitoring framework that is adequately reflected in the National Health Sector Strategy M&E Plan; ii) monitoring instruments; iii) people to ensure collection of the data that shows a true reflection of the reality in terms of the performance being monitored ; iv) supporting system in the form of technology and information system, as well as the manpower to enable the transmission of the collected data in the most

33 L. A.Dove, Teachers and Teacher Education in Developing Countries, New Hampshire: Croom Helm, 1986. 34 Takako SUZUKI, The Effectiveness of the Cascade Model for In-service Teacher Training in Nepal, Graduate School of International Cooperation Studies, Kobe University 2-1 Rokko-dai, Nada-ku, Kobe, 657-8501 Japan 35 Department of Education and Science, A critique of the implementation of the cascade model used to provide inset for teachers in preparation for the introduction of the general certificate of secondary education, Stanmore, Middlesex, 1988 36 J.Mezirow, Transformative Dimensions of Adult Learning, San Francisco, Jossey-Bass Publishers, 1991

58

logical and workable manner, as well as the analysis of the data in the most objective manner to prevent any distortion of the data, complying with all the principles of trustworthiness; and v) a system which enables adequate reporting and feedback of the results to the health facilities and practitioners concerned.

8.2 Recommendations for horizontal CQI scale-up

Should the Government decide to horizontally expand CQI to additional facilities, it is recommended to use a phased approach.

Phase 1: Integration of MNCH, EPI, HIV, TB CQI approaches in pilot facilities. State-level programs have operational QI components in the studied health facilities, albeit in silos. It is advised to: i) integrate the QI components of all state programs into one “extended” QI component; ii) develop an extended list of indicators; iii) harmonize monitoring tools; iv) provide training of staff at provincial, district, and facility levels; and v) monitor performance of facilities.

Phase 2: Phased scale-up of integrated (MNCH, EPI, HIV, TB) CQI approaches to all facilities in the pilot provinces. When institutional policies and regulatory mechanisms are developed so that quality assurance mechanisms are integrated into the health system’s normal operations at all levels, the MOHCC can consider scaling up integrated CQI to all facilities in original pilot districts.

Phase 3: Phased scale-up of integrated (MNCH, EPI, HIV, TB) CQI approaches to other provinces. When integrated CQI is formally integrated into the structure and functions of the health care system and health organizations, the model can be rolled out to other provinces.

Phase 4: Expansion of the CQI to other government priority programs and phased scale -up to all facilities in the pilot provinces. At the last instance, the CQI approach can be extended to other government priorities in the health sector, e.g., non-communicable diseases, oncology, surgery, etc.

To make the scale-up/expansion process smooth, the GoZ is encouraged to adhere to the steps detailed below.

Table 8.1: Steps in the CQI scaling up process37

Action Description

Step1: Develop a scaling up plan: outline a vision of scale-up and a compelling case for action

1.1 Rationale for scale-up

Draw up a rationale for scaling up from the information in Step 1, noting that further investigation and analysis may be necessary to provide a compelling case for action.

1.2 Description of the intervention

Describe ‘what’ will be scaled up and where possible the original intervention should be simplified and streamlined.

1.3 Situation and stakeholder analysis

Map the social, political, and organizational environment(s) in which the intervention will be scaled up and identify potential barriers and enablers to scale-up.

1.4 Role and function of stakeholders in the scale-up process

Consider who might perform key functions when the intervention is scaled up by mapping key functions and matching them to those who will potentially be involved.

1.5 Approach to scale up

There are two main scale-up approaches. A vertical approach involves the introduction of an intervention simultaneously across a whole system and results in institutional change through policy, regulation, financing, or health systems change. A horizontal approach involves the introduction of an intervention across different sites or groups in a phased manner. These

37 Adapted from Andrew J Milat et al, A guide to scaling up population health interventions, 2016, Public Health Research and Practice, January 2016; Vol. 26(1):e2611604

59

approaches are not mutually exclusive, and a combination of approaches can be used.

1.6 Monitoring and Evaluation framework of the intervention scale-up

Determine what variables are important to measure over time and determine feasibility and associated cost of these systems.

1.7 Resources required for scale-up

Estimate the human, technical, and financial resources needed to scale up the intervention.

1.8 Scale up plan The plan should present a clear and concise case for scaling up the intervention, as well as an overview of how this will be brought about, including a vision of what scaling up will look like if successfully completed.

Step 2. Prepare for scale-up: secure resources and build a foundation of legitimacy for the scaling up plan

2.1 Consult with stakeholders

Assess the appropriateness and acceptability of the intervention and the scaling up plan and use this information to design advocacy and communication strategies.

2.2 Legitimize change

Gain the support of decision makers who must be convinced that scaling up the intervention is: a credible and superior solution to a pressing problem; for a priority population; and is affordable.

2.3 Build a constituency

Mobilize the broader community required to successfully scale up an intervention.

2.4 Realign and mobilize resources

Mobilize financial resources through existing channels or through new funding streams. Ensure that resources are directed to address skill and other capacity deficits in delivery organizations.

Step 3. Scale up the intervention: implement the scale-up plan, making necessary adjustments based on performance data

3.1 Modify and strengthen organizations

Manage organizational change through processes such as staff re-training, mentoring, leadership development and coaching

3.2 Coordinate action and governance

Develop and implement concrete and detailed agreements about how, when, where and by whom resources are to be used, and the governance structures that will be used to identify issues and resolve any disputes that may arise.

3.3 Monitor performance and efficiency

Develop systems that have an ongoing focus on measuring effectiveness, reach, fidelity, acceptability, and costs, with a particular focus on the efficiency of intervention delivery.

3.4 Ensure sustainability

Implement organizational and cultural changes to institutionalize an intervention so that it becomes part of routine practice.

60

Annex 1: RBF Package of quantity services

District Hospital

1. Normal deliveries in district hospital 2. Deliveries with complications (caesareans excluded) and postpartum

complications 3. Caesareans performed 4. Family planning: Tuba Ligations 5. Counter referral note arrives at RHC

Rural Health Center

1. OPD new consultations

2. First ANC visit during the first 16 weeks of pregnancy (October 2012)

3. Ante natal care 4 visits completed

4. Post-natal care 2 or more

5. Normal deliveries

6. HIV VCT in ANC

7. Syphilis RPR test

8. IPT (x2 doses)

9. Tetanus TT2+

10. ARVs to HIV+ preg. Women (PMTCT)

11. Family planning short and long term methods

12. High risk perinatal referrals

13. Vitamin A supplementation

14. Children fully immunized

15. Growth monitoring, children < 5yrs

16. Cure discharged acute malnutrition children < 5yrs

61

Annex 2: Exploratory Research Questions

The qualitative research in the Phase 2 focused on gaining insights into different performance observed between hospitals and Health Centers at intervention sites and between intervention and non-intervention sites. More specifically, the PME attempts to answer the following questions: 1. Consistency of potential gains and staff perceptions that received CQI intervention:

1.1. What were the general perceived benefits of the CQI intervention? 1.2. Are these potential gains consistent with the qualitative perceptions of the PHC staff

that received the CQI intervention? 1.3. Have the staff noticed more organized care/more attention to detail/more feedback

and supervision related to these issues? 1.4. What were the perceived costs/detriments if any? 1.5. How has the provision of care changed under the CQI intervention? 1.6. What are the constraints that prevent further improvement in the quality of care? 1.7. In comparison facilities, has there been any attempt in the previous two years to

improve the clinical quality of care and how? 1.8. What are the constraints that prevent further improvement in the quality of care?

2. Structural quality 2.1. What are some of the reasons that account for the mixed results on structural quality? 2.2. What are the main structural quality changes that health facility teams ascribe to the

CQI (qualitatively)? 2.3. What implementation or design changes do health facility teams recommend to further

enhance the CQI’s effects on structural quality? 2.4. For non-CQI facilities (PHC and hospital levels), has there been any attempt in the

previous two years to improve the structure of care? 2.5. What are the constraints that prevent further improvement in the quality of care?

3. Health Facility Team Organization and Management: 3.1. How did teams in CQI facilities organize themselves differently to deliver better quality

services? 3.2. What worked under the new team set up/organization, if any? 3.3. What would CQI facility teams do differently to achieve better results in the future?

4. Mentorship/coaching: 4.1. Extent to which coaching and mentoring influenced attainment of better results? 4.2. What are the main differences between CQI facilities and non-CQI facilities in terms

of access to mentorship and coaching? 4.3. How do health facility teams in CQI facilities perceive the nature and quality of this

mentorship by supervisors? 4.4. How do district and provincial managers perceive the nature and quality of the

mentorship? 4.5. What could be done differently from a health systems management perspective going

forward?

5. Health Systems Management: 5.1. How do field level CQI mechanisms and processes (from Province to facility level)

influence management decisions related to health system elements (refer across the pillars)?

5.2. What CQI related factors play a dominant role in determining decision making by district and provincial managers and follow up on those decisions at health facility level?

62

5.3. What are the dynamics between Health Facility CQI Teams, Facility Management and District/Province supervisors in the process of facilitating changes at facilities?

6. Staff capacity building and motivation: 6.1. How well and in what way is attained knowledge and skill to assess quality and

improve clinical processes internally transferred amongst staff (from the nuclei- CQI Teams to other staff and departments)?

6.2. To what extent staff capacity building and motivation played a role in improvement or not improving the care quality?

6.3. Extent to which RBF performance payments influenced care quality improvement and how?

7. Factors influencing care quality improvement: 7.1. What were the factors that supported or inhibited improvement of care quality across

system, process and outcome levels (external and internal)? 7.2. What influenced better performance of non-CQI facilities across all three domains

(where applicable)? 7.3. Extent to which CQI content and delivery modes were appropriate to achieve care

quality improvement at CQI facilities?

8. Scale -up: What are the critical elements to be considered for the scale-up?

63

Annex 3: Additional tables & figures

Table A1. Organization of individual PHC outcomes into aggregate measures and balance in 2016

Summary measure Indicator CQI Control

T test p-value N CQI

N control

General infrastructure

Visible sign post when arriving at the clinic 92% 96% 0.52 24 201

Fence/wall in good condition and gate with clearly written emergency contact num 42% 58% 0.14 24 201

External appropriate wall finishing 100% 99% 0.08 24 201

Roof intact, presence of well-maintained rain gutters 54% 64% 0.37 24 201

Minimum of 3 toilets 100% 96% 0.00 24 201

Hand washing facility with soap available near the toilets 75% 50% 0.02 24 201

Electricity for 24 hours a day, and 7 days a week 63% 83% 0.07 24 201

Fire extinguishers available, accessible, functional and serviced AND a clearly 29% 25% 0.69 24 230

Appropriate drainage of waste water 79% 88% 0.35 24 201

Shower with either running water or container of at least 100 litres 75% 81% 0.56 24 201

Waiting area with adequate ventilation of waiting area 96% 96% 0.95 24 205

Cleanliness Ground clean with no litters and/or stagnant water and the grass cut 83% 72% 0.17 24 201

No waste and dangerous objects in courtyard such as needles – syringes –gloves – 96% 99% 0.54 24 201

Recently clean without visible fecal material and without smell 83% 90% 0.41 24 201

Bin with lid accessible to clients 88% 85% 0.69 24 201

Building clean and with good ventilation without bad smell? 100% 97% 0.01 24 205

Walls, Floors and Ceiling / roof clean and in good condition (no leaks, cobwebs) 79% 80% 0.88 24 205

Safety Bin with plastic liners + sharps containers available and Bin not more than ¾ fu 75% 80% 0.64 24 205

Hand washing facilities with soap available at accessible points for outpatient 96% 76% 0.00 24 197

Safe drinking water is accessible 92% 80% 0.07 24 197

Ottoway pit or rubbish pit and incinerator 58% 51% 0.52 24 230

Privacy Doors with locks and closing properly 96% 95% 0.79 24 205

Curtains on windows or non- transparent glass and screen 96% 95% 0.87 24 205

Sufficient space between beds (at least 1m between beds) 96% 95% 0.84 24 197

General equipment

Radio or mobile phone with airtime or landline for communication 92% 91% 0.92 24 201

Transport plan for emergency referrals and/or contingency plan 83% 85% 0.83 24 201

64

Beds, mattresses (covered in plastic), bed sheet and bed side lockers available 88% 84% 0.66 24 197

Mosquito nets available and in good state? 96% 64% 0.00 24 197

Adult Weighing Scale and standard pediatric scale available 96% 96% 0.96 24 204

BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmo 54% 54% 0.98 24 204

Pharmacy management

Important Accessories: cannula, giving sets, syringes and needles, drip stand, s 67% 69% 0.81 24 204

Monthly physical counts conducted with min, max and emergency order levels recor 67% 66% 0.91 24 203

The physical stock level corresponds with that on the stock card: supply on stoc 83% 85% 0.82 24 203

Staff completes and send MIS forms to district each month 75% 58% 0.09 24 203

Stored correctly in a locked secured storeroom 100% 96% 0.00 24 203

Clean place, well ventilated with cupboards, labelled shelves, no incident light 92% 81% 0.11 24 203

Medicines stored in alphabetical order also observing the First Expiry First Out 88% 62% 0.00 24 203

Expired Medicine Register available 42% 52% 0.34 24 203

No expired products in stock: supervisor verifies randomly 3 medicines and 2 con 79% 95% 0.08 24 203

Lockable trolleys available with working lock 88% 88% 0.93 24 203

Pharmacy stock

Doxycycline capsules 100mg 88% 80% 0.34 24 203

Ciprofloxacin tablets 500mg 38% 27% 0.33 24 203

Metronidazole tablets 200mg Oral 71% 73% 0.84 24 203

Diazepam injection 5mg/ml 63% 55% 0.50 24 203

Benzathine Penicillin injection 33% 28% 0.61 24 203

Benzyl Penicillin 88% 88% 0.98 24 203

Amoxycillin suspension 125mg/5ml (dispersible tablets) 75% 81% 0.51 24 203

Ferrous sulphate tablets/Folic Acid 67% 82% 0.14 24 203

Zinc Sulphate tabs 83% 87% 0.68 24 203

Paracetamol 500mg tablets 92% 97% 0.42 24 203

Paracetamol syrup or dispersible tablets 92% 86% 0.39 24 203

Dispensing envelopes 79% 72% 0.43 24 203

Latex gloves 96% 97% 0.78 24 203

Oxytocin 10IU/ml Injection, 1ml Ampoule. 88% 86% 0.86 24 203

Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule 96% 97% 0.78 24 203

Gentamycin 40mg/ml Injection, 2ml Ampoule 88% 72% 0.05 24 203

Artemether 20mg + Lumefantrine 120mg Tablets, 24's 88% 95% 0.29 24 203

Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial 100% 98% 0.02 24 203

Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100 Cycles 83% 94% 0.19 24 203

65

Lignocaine 96% 88% 0.11 24 203

PEP kits available and accessible. Contents of PEP kit: Zidovudine 300 mg + lami 71% 87% 0.11 24 203

Adult first line ART. Preferred: TDF +3TC+NVP, alternate: TDF+3TC+EFV or ZDV+3TC 100% 100% 0.32 24 203

Paediatric first line ART. Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( 96% 94% 0.62 24 203

Emergency tray with all the necessary un expired medicines available 38% 44% 0.57 24 204

Un expired ringer lactate, 5all dextrose, normal saline available 42% 63% 0.06 24 204

Guidelines available

Guidelines/protocols: National Malaria guidelines for diagnosis and treatment of 88% 90% 0.76 24 204

PEP policy and guidelines: Available in OPD: posted on the wall and up to date 83% 89% 0.47 24 204

Opportunistic Infection and ART guidelines: Available, accessible in all consult 92% 96% 0.51 24 204

STI Management protocol: Displayed in all consultation rooms and up to date 96% 89% 0.17 24 204

IMNCI guidelines: Flowcharts displayed in all consultation areas and up to date 92% 82% 0.13 24 204

Focused ANC protocol: Displayed I all consultation rooms an up to date 92% 96% 0.46 24 204

Vaccine management

Surveillance line listing and case definitions displayed 68% 73% 0.69 19 178

Updated EPI schedule, and a contingency plan displayed 74% 81% 0.48 19 178

EPI graphs showing trends displayed and staff member is able to interpret the g 79% 79% 0.98 19 178

EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI 100% 84% 0.00 19 178

Fridge with a temperature booklet available and filled twice a day 89% 94% 0.52 19 178

The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (S 84% 79% 0.55 19 178

The following Antigens are available: BCG, MR (measles and Rubella), polio, Pen 79% 92% 0.19 19 178

The physical stock and the amount in the stock cards match ( Supervisor verifies 63% 76% 0.30 19 178

Vaccines correctly stored in fridge with compartments as follows in fridges with 95% 96% 0.89 19 178

No expired vaccines 89% 97% 0.34 19 178

The Vaccine Vial Monitor (VVM) status is kept 95% 94% 0.95 19 178

There are readable labels on vials with matching diluents 79% 95% 0.12 19 178

The number of syringes available matches the number of vaccines in the stock car 89% 92% 0.78 19 178

Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 100% 98% 0.05 19 178

Vaccine carriers, cold box, gas regulator, gas cylinder and scissors 100% 95% 0.00 19 178

66

AEFI investigation forms, case investigation forms for EPI targeted diseases an 89% 83% 0.42 19 178

Vaccine order forms and stock cards available 89% 94% 0.57 19 178

Maternity supplies

IV fluids and giving sets 74% 68% 0.55 23 168

Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and 57% 33% 0.04 23 168

Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant 91% 85% 0.36 23 168

PPH kits available and complete 70% 51% 0.09 23 168

Eclampsia kit available and complete 48% 47% 0.94 23 168

Uterotonic Medicines stored at correct temperature available in delivery room or 87% 87% 0.99 23 168

Fetoscope, baby blanket Baby scale and tape measure 91% 83% 0.21 23 168

Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord 83% 68% 0.11 23 168

Penguin suction, neonatal bag and mask and suction tube in delivery room 78% 91% 0.17 23 168

At least 2 obstetric sterilized delivery standard packs 74% 76% 0.87 23 168

At least 5 pairs of sterile gloves should be available 96% 86% 0.06 23 168

HMIS Standard referral forms (at least 10) and register available and properly filled 87% 77% 0.19 23 209

A referral feedback documented in the register for every referral made and/or re 52% 53% 0.96 21 183

The following T Series forms available and fully completed: T1, T2, T3, T5, T6, 87% 82% 0.51 23 209

T5 completed and sent timely (by the 7th of the following month) for previous mo 83% 94% 0.17 23 209

All selected registers complete and correct 39% 34% 0.67 23 209

T5 correct according to HMIS 48% 34% 0.24 23 209

OPD triaging knowledge

Consultations being done by appropriately qualified staff: PCN and/or RGN 100% 86% 0.00 24 204

Triage: Emergency signs requiring immediate attention 83% 90% 0.70 6 82

Triage: Priority signs (requiring priority in the queue) 67% 89% 0.34 6 82

Triage: Non-urgent cases 67% 90% 0.32 6 82

TB screening knowledge

TB sign: Weight loss 100% 95% 0.00 19 182

TB sign: Fever for more than 3 weeks 95% 85% 0.12 19 182

TB sign: Cough for more than 14 days 100% 98% 0.05 19 182

TB sign: Night sweats 95% 95% 0.97 19 182

TB sign: TB contact exposure. 89% 86% 0.68 19 182

ANC process quality

all of first visit ANC bookings who had documented: BP, Height, Weight measurement 79% 65% 0.17 19 217

all of first visit ANC bookings who received the standard laboratory test accord 37% 42% 0.67 19 217

all of first ANC visits who received TT vaccine 100% 94% 0.00 19 217

all of first ANC visits who received iron supplementation 100% 90% 0.00 19 217

67

all of first visit ANC who had documented pregnancy test results 11% 12% 0.90 19 217

PNC process quality

all PNC visits with assessment: General condition of the infant, NAD recorded if 58% 76% 0.15 19 217

all PNC visits with assessment: General condition ,Pulse rate, B/P and temperature 32% 37% 0.62 19 217

all PNC visits documenting infant feeding (BF) status (exclusive, mixed or not B 95% 96% 0.77 19 217

all women post-partum counselled and offered any of the modern FP method at fol 63% 72% 0.47 19 217

all facility births seen for day 3 PNC visit in any one month in the last quarte 94% 74% 0.01 16 203

PMTCT process quality

all NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE 59% 58% 0.98 17 212

all of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of b 59% 67% 0.55 17 212

all of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks o 29% 23% 0.57 17 212

all of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 35% 5% 0.02 17 212

Sick child assessment

all children treated as outpatient for pneumonia in any one of month in last qu 65% 38% 0.05 17 225

all children with diarrhoea correctly assessed for signs of dehydration), persi 71% 48% 0.07 17 225

all children diagnosed with malaria that have RDT + or laboratory confirmation i 35% 29% 0.61 17 225

Sick child treatment

all children correctly treated as an outpatient for pneumonia in any one of mont 59% 59% 0.99 17 225

all children correctly treated as an outpatient (ambulatory) for diarrhoea in an 47% 64% 0.20 17 225

all Children with uncomplicated malaria correctly treated according to national 35% 24% 0.37 17 225

all Children with severe malaria correctly treated according to national guideli 0% 4% 0.00 17 225

all of 6-59 months old children with un complicated severe acute malnutrition (S 18% 24% 0.53 17 225

Maternity process quality

all women who delivered in the facility monitored using partographs as per crite 19% 14% 0.67 16 203

all deliveries performed by skilled personnel in any one month in the last quart 100% 92% 0.00 16 203

all total births in any one month in the last quarter documenting administration 88% 66% 0.03 16 203

all births with placental status documented at birth in any one month in the las 100% 93% 0.00 16 203

all newborns BF within one hour of birth in any one month in the last quarter 69% 75% 0.60 16 203

all newborns received Vitamin K in the any one month in the last quarter 63% 40% 0.10 16 203

all newborns received eye care (Tetracycline) in the any one month in the las 81% 86% 0.67 16 203

68

all newborns received first vaccination (BCG) in the any one month in the last q 50% 71% 0.13 16 203

all women monitored in early post-partum period (4th stage) per guideline (birth 63% 24% 0.01 16 203

all newborns monitored in early post-partum period per guideline (birth to disch 44% 25% 0.18 16 203

Obstetric complications

all women with prolonged labour in last quarter referred to higher level 5% 11% 0.33 19 211

all women with prolonged labor or Rupture of Membranes and without chorioamnioni 0% 1% 0.16 19 211

all women with PPH managed per guideline in the last quarter 0% 6% 0.00 19 211

all women with signs of intra- or post-partum sepsis (fever, foul-smelling disch 0% 0% 19 211

all pregnant women with severe pre-eclampsia and/or eclampsia managed according 5% 0% 0.33 19 211

all of neonates who had an APGAR score of ≤5 in the first minute after birth for 0% 0% 19 211

all neonates with possible serious bacterial sepsis managed per standard in last 5% 3% 0.66 19 211

Table A2. Balance between treated and control PHCs on covariates

CQI Control P-value of T test N CQI

N control

Percent Clinic (in comparison to RHC) 46% 70% 24 230

Distance from district hospital 41 41 0.94 24 206

2016 Total New Patients 4068 4480 0.61 24 230

Table A3. Unadjusted PHC summary ANCOVA results on input quality measures

CQI Baseline N

Coef. p-val Coef. p-val

Inputs

General infrastructure: % of 11 items 0.07 0.11 0.31 0.00 254

Cleanliness: % of 6 items 0.03 0.35 0.16 0.16 235

Safety: % of 4 items 0.04 0.29 0.26 0.00 254

Privacy: % of 3 items 0.01 0.63 0.07 0.08 229

General equipment: % of 7 items 0.04 0.46 0.02 0.81 238

Pharmacy management: % of 9 items 0.02 0.54 0.32 0.00 226

Pharmacy stock: % of 25 items -0.03 0.33 0.12 0.02 237

Availability of guidelines: % of 6 items -0.01 0.79 0.16 0.09 228

Vaccine management: % of 17 items -0.02 0.65 0.15 0.09 197

Maternity supplies: % of 11 items 0.05 0.29 0.03 0.57 190

HMIS: % of 6 items 0.01 0.93 0.20 0.00 231

OPD triaging knowledge: % of 4 items -0.04 0.33 0.02 0.19 231

TB screening knowledge: % of 5 items -0.01 0.65 0.08 0.18 201

69

Table A4. Adjusted PHC summary ANCOVA results on input quality measures

CQI Baseline N


Inputs



Safety: % of 4 items 0.04 0.42 0.19 0.11 114

Privacy: % of 3 items 0.04 0.20 0.25 0.02 103

General equipment: % of 7 items 0.04 0.50 -0.01 0.96 106




Vaccine management: % of 17 items 0.00 0.97 0.10 0.39 92


HMIS: % of 6 items -0.01 0.88 0.26 0.03 102


TB screening knowledge: % of 5 items 0.01 0.62 0.02 0.83 89

Table A5. Adjusted PHC summary ANCOVA results using Q4 2018 as outcome

CQI Baseline N


General infrastructure: % of 11 items -0.03 0.57 0.06 0.53 114

Cleanliness: % of 6 items -0.03 0.57 0.26 0.09 106

Safety: % of 4 items -0.02 0.71 0.05 0.34 114

Privacy: % of 3 items -0.03 0.43 0.14 0.33 97


Pharmacy management: % of 9 items -0.03 0.66 0.32 0.14 96



Vaccine management: % of 17 items -0.08 0.27 -0.07 0.50 91


HMIS: % of 6 items 0.06 0.21 0.43 0.00 95





PMTCT process quality: % of 4 items -0.01 0.98 -0.06 0.72 101



Maternal process quality: % of 10 items 0.03 0.46 -0.01 0.83 93


70

Table A6. Adjusted PHC summary difference in differences results

DnD estimator Post N


Input measures


Cleanliness: % of 6 items 0.03 0.41 -0.02 0.59 176

Safety: % of 4 items -0.09 0.33 0.10 0.26 182

Privacy: % of 3 items 0.06 0.09 -0.04 0.06 173



Pharmacy stock: % of 25 items 0.01 0.79 0.03 0.29 176


Vaccine management: % of 17 items 0.03 0.58 -0.01 0.81 167

Maternity supplies: % of 11 items -0.02 0.83 -0.01 0.92 168

HMIS: % of 6 items -0.07 0.32 0.06 0.03 171



Process measures


PNC process quality: % of 5 items 0.15 0.02 -0.01 0.84 177


Sick child assessment quality: % of 3 items -0.15 0.05 0.22 0.00 174




Table A7. Adjusted PHC ANCOVA results for all facility outcomes

CQI Baseline N


Visible sign post when arriving at the clinic 0.02 0.71 0.13 0.07 100 Fence/wall in good condition and gate with clearly written emergency contact num 0.08 0.49 -0.20 0.05 100

External appropriate wall finishing 0.00 0.97 -0.05 0.14 84

Roof intact, presence of well-maintained rain gutters 0.11 0.34 0.14 0.34 100

Minimum of 3 toilets 0.04 0.24 0.66 0.00 100

Hand washing facility with soap available near the toilets 0.10 0.08 -0.02 0.57 100

Electricity for 24 hours a day, and 7 days a week 0.07 0.37 0.19 0.01 100 Fire extinguishers available, accessible, functional and serviced AND a clearly 0.09 0.27 0.04 0.62 114

Appropriate drainage of waste water 0.03 0.51 0.24 0.00 100

Shower with either running water or container of at least 100 litres 0.02 0.80 0.18 0.01 100

Waiting area with adequate ventilation of waiting area 0.01 0.80 -0.04 0.10 103

Ground clean with no litters and/or stagnant water and the grass cut 0.02 0.66 0.13 0.10 100 No waste and dangerous objects in courtyard such as needles – syringes –gloves – 0.08 0.06 0.39 0.02 100

Recently clean without visible fecal material and without smell 0.02 0.73 0.02 0.84 100

Bin with lid accessible to clients 0.07 0.15 0.03 0.60 100

71

Building clean and with good ventilation without bad smell? 0.02 0.02 0.07 0.42 103 Walls, Floors and Ceiling / roof clean and in good condition (no leaks, cobwebs) -0.06 0.44 0.18 0.06 103 Bin with plastic liners + sharps containers available and Bin not more than ¾ fu -0.01 0.85 0.04 0.59 103 Hand washing facilities with soap available at accessible points for outpatient -0.05 0.51 -0.05 0.18 101

Safe drinking water is accessible -0.10 0.29 -0.02 0.68 99

Ottoway pit or rubbish pit and incinerator 0.13 0.20 0.24 0.02 114

Doors with locks and closing properly -0.02 0.39 0.26 0.05 103

Curtains on windows or non- transparent glass and screen 0.02 0.27 -0.03 0.05 103

Sufficient space between beds (at least 1m between beds) 0.09 0.26 0.34 0.13 99

Radio or mobile phone with airtime or landline for communication 0.02 0.74 0.23 0.09 100

Transport plan for emergency referrals and/or contingency plan 0.01 0.94 0.14 0.01 100 Beds, mattresses (covered in plastic), bed sheet and bed side lockers available 0.02 0.77 0.07 0.32 99

Mosquito nets available and in good state? 0.00 0.98 0.18 0.05 99

Adult Weighing Scale and standard pediatric scale available 0.00 0.87 -0.02 0.03 105 BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmo 0.17 0.14 -0.06 0.39 105 Important Accessories: cannula, giving sets, syringes and needles, drip stand, s -0.01 0.91 0.02 0.82 105 Monthly physical counts conducted with min, max and emergency order levels recor 0.07 0.46 0.04 0.61 102 The physical stock level corresponds with that on the stock card: supply on stoc -0.09 0.20 -0.01 0.89 102

Staff completes and send MIS forms to district each month 0.03 0.77 0.34 0.01 102

Stored correctly in a locked secured storeroom 0.02 0.15 0.17 0.01 102 Clean place, well ventilated with cupboards, labelled shelves, no incident light 0.00 1.00 0.43 0.01 102 Medicines stored in alphabetical order also observing the First Expiry First Out -0.01 0.85 0.13 0.25 102

Expired Medicine Register available -0.02 0.81 0.15 0.06 90 No expired products in stock: supervisor verifies randomly 3 medicines and 2 con -0.08 0.26 -0.12 0.04 102

Lockable trolleys available with working lock 0.05 0.33 -0.08 0.16 102

Doxycycline capsules 100mg 0.01 0.76 -0.02 0.63 102

Ciprofloxacin tablets 500mg 0.00 0.97 0.17 0.00 102

Metronidazole tablets 200mg Oral 0.01 0.79 0.04 0.17 102

Diazepam injection 5mg/ml -0.09 0.34 0.12 0.22 102

Benzathine Penicillin injection 0.01 0.91 -0.05 0.61 90

Benzyl Penicillin -0.05 0.44 0.13 0.46 102

Amoxycillin suspension 125mg/5ml (dispersible tablets) -0.02 0.72 0.02 0.67 102

Ferrous sulphate tablets/Folic Acid -0.02 0.12 -0.02 0.01 102

Zinc Sulphate tabs -0.01 0.89 -0.05 0.10 102

Paracetamol 500mg tablets -0.11 0.29 0.04 0.73 102

Paracetamol syrup or dispersible tablets 0.00 0.99 -0.20 0.04 102

Dispensing envelopes -0.04 0.25 0.07 0.16 102

72

Latex gloves 0.00 0.92 0.14 0.21 102

Oxytocin 10IU/ml Injection, 1ml Ampoule. -0.03 0.61 0.05 0.51 102

Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule 0.09 0.37 -0.06 0.37 102

Gentamycin 40mg/ml Injection, 2ml Ampoule -0.01 0.85 0.07 0.15 102

Artemether 20mg + Lumefantrine 120mg Tablets, 24's 0.01 0.64 -0.02 0.13 102

Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial -0.01 0.71 0.53 0.01 102

Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100 Cycles 0.06 0.39 0.14 0.31 102

Lignocaine -0.02 0.66 -0.05 0.05 102 PEP kits available and accessible. Contents of PEP kit: Zidovudine 300 mg + lami -0.10 0.38 0.18 0.45 102 Adult first line ART. Preferred: TDF +3TC+NVP, alternate: TDF+3TC+EFV or ZDV+3TC 0.01 0.10 0.00 102 Paediatric first line ART. Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( 0.00 0.96 0.12 0.04 102

Emergency tray with all the necessary un expired medicines available -0.03 0.77 0.02 0.84 105

Un expired ringer lactate, 5all dextrose, normal saline available -0.07 0.59 -0.11 0.12 105 Guidelines/protocols: National Malaria guidelines for diagnosis and treatment of 0.01 0.77 -0.02 0.23 105 PEP policy and guidelines: Available in OPD: posted on the wall and up to date -0.08 0.34 0.03 0.67 105 Opportunistic Infection and ART guidelines: Available, accessible in all consult -0.01 0.74 -0.03 0.05 105 STI Management protocol: Displayed in all consultation rooms and up to date 0.07 0.05 0.14 0.23 105 IMNCI guidelines: Flowcharts displayed in all consultation areas and up to date 0.02 0.16 0.16 0.00 105

Focused ANC protocol: Displayed I all consultation rooms an up to date -0.03 0.32 0.07 0.38 105

Surveillance line listing and case definitions displayed 0.03 0.37 0.12 0.03 92

Updated EPI schedule, and a contingency plan displayed -0.07 0.40 -0.02 0.58 92 EPI graphs showing trends displayed and staff member is able to interpret the g 0.06 0.54 -0.01 0.93 92 EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI -0.13 0.28 0.26 0.13 92

Fridge with a temperature booklet available and filled twice a day 0.02 0.74 0.09 0.27 92 The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (S 0.03 0.66 0.16 0.33 92 The following Antigens are available: BCG, MR ( measles and Rubella), polio, Pen -0.17 0.01 -0.04 0.64 92 The physical stock and the amount in the stock cards match ( Supervisor verifies 0.00 0.96 0.18 0.10 92 Vaccines correctly stored in fridge with compartments as follows in fridges with 0.00 0.88 -0.06 0.00 92

No expired vaccines -0.02 0.70 0.00 0.95 92

The Vaccine Vial Monitor (VVM) status is kept 0.04 0.38 -0.07 0.03 92

There are readable labels on vials with matching diluents -0.06 0.43 0.07 0.35 92 The number of syringes available matches the number of vaccines in the stock car 0.10 0.22 0.13 0.36 92 Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 0.02 0.08 -0.02 0.08 92

73

Vaccine carriers, cold box, gas regulator, gas cylinder and scissors -0.02 0.75 -0.07 0.20 92 AEFI investigation forms, case investigation forms for EPI targeted diseases an 0.09 0.07 0.09 0.19 92

Vaccine order forms and stock cards available 0.02 0.60 -0.05 0.01 92

IV fluids and giving sets 0.08 0.38 -0.04 0.41 91 Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and 0.02 0.69 0.00 0.94 91

Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant -0.01 0.92 0.12 0.22 91

PPH kits available and complete 0.24 0.02 0.04 0.67 91

Eclampsia kit available and complete 0.22 0.07 0.10 0.26 91 Uterotonic Medicines stored at correct temperature available in delivery room or 0.06 0.08 0.08 0.11 91

Fetoscope, baby blanket Baby scale and tape measure -0.04 0.48 0.14 0.20 91 Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord 0.05 0.71 -0.03 0.70 91 Penguin suction, neonatal bag and mask and suction tube in delivery room -0.06 0.44 0.12 0.30 91

At least 2 obstetric sterilized delivery standard packs 0.10 0.52 0.08 0.32 91

At least 5 pairs of sterile gloves should be available -0.05 0.31 0.02 0.58 91 Standard referral forms (at least 10) and register available and properly filled -0.05 0.48 -0.01 0.79 102 A referral feedback documented in the register for every referral made and/or re -0.10 0.40 0.17 0.08 92 The following T Series forms available and fully completed: T1, T2, T3, T5, T6, 0.02 0.85 0.29 0.10 102 T5 completed and sent timely (by the 7th of the following month) for previous mo -0.03 0.57 0.26 0.01 102

All selected registers complete and correct 0.03 0.77 0.06 0.25 102

T5 correct according to HMIS 0.13 0.37 0.01 0.87 102 Consultations being done by appropriately qualified staff: PCN and/or RGN -0.05 0.41 0.10 0.01 105

Triage: Emergency signs requiring immediate attention 0.05 0.34 -0.05 0.34 33

Triage: Priority signs (requiring priority in the queue) 0.02 0.34 -0.02 0.34 33

Triage: Non-urgent cases 0.00 0.00 33

TB sign: Weight loss -0.01 0.58 -0.01 0.25 89

TB sign: Fever for more than 3 weeks 0.01 0.85 -0.03 0.32 89

TB sign: Cough for more than 14 days 0.02 0.53 -0.03 0.18 89

TB sign: Night sweats 0.02 0.22 0.02 0.48 89

TB sign: TB contact exposure. 0.02 0.51 0.00 0.99 89 all of first visit ANC bookings who had documented: BP, Height, Weight measureme -0.02 0.83 0.00 0.96 105 all of first visit ANC bookings who received the standard laboratory test accord 0.03 0.56 0.07 0.10 105

all of first ANC visits who received TT vaccine -0.05 0.27 0.04 0.32 105

all of first ANC visits who received iron supplementation 0.00 0.98 -0.02 0.09 105

all of first visit ANC who had documented pregnancy test results -0.09 0.24 -0.11 0.18 105 all PNC visits with assessment: General condition of the infant, NAD recorded if 0.18 0.11 0.21 0.04 105

74

all PNC visits with assessment: General condition ,Pulse rate, B/P and temperatu 0.25 0.02 0.15 0.15 105 all PNC visits documenting infant feeding (BF) status (exclusive, mixed or not B 0.03 0.22 -0.03 0.29 105 all women post-partum counselled and offered any of the modern FP method at fol 0.00 0.98 0.07 0.26 105 all facility births seen for day 3 PNC visit in any one month in the last quarte 0.01 0.76 0.15 0.06 98 all NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE 0.02 0.84 0.17 0.02 105 all of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of b -0.15 0.09 0.02 0.70 105 all of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks o -0.08 0.57 0.06 0.49 105 all of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 -0.07 0.59 0.02 0.84 105 all children treated as outpatient for pneumonia in any one of month in last qu 0.02 0.79 0.19 0.07 107 all children with diarrhoea correctly assessed for signs of dehydration), persi -0.07 0.37 0.10 0.29 107 all children diagnosed with malaria that have RDT + or laboratory confirmation i 0.08 0.52 0.49 0.00 107 all children correctly treated as an outpatient for pneumonia in any one of mont 0.09 0.34 0.25 0.00 107 all children correctly treated as an outpatient (ambulatory) for diarrhoea in an 0.13 0.31 0.07 0.41 107 all Children with uncomplicated malaria correctly treated according to national 0.11 0.47 0.38 0.00 107 all Children with severe malaria correctly treated according to national guideli 0.13 0.41 -0.05 0.68 107 all of 6-59 months old children with un complicated severe acute malnutrition (S 0.13 0.19 0.07 0.39 107 all women who delivered in the facility monitored using partographs as per crite 0.15 0.14 -0.19 0.26 98 all deliveries performed by skilled personnel in any one month in the last quart 0.01 0.13 0.58 0.01 98 all total births in any one month in the last quarter documenting administration 0.13 0.00 0.04 0.29 98 all births with placental status documented at birth in any one month in the las 0.03 0.01 0.68 0.00 98 all newborns BF within one hour of birth in any one month in the last quarter 0.05 0.47 0.12 0.33 98

all newborns received Vitamin K in the any one month in the last quarter 0.11 0.36 0.20 0.03 98 all newborns received eye care (Tetracycline) in the any one month in the las 0.07 0.11 0.30 0.11 98 all newborns received first vaccination (BCG) in the any one month in the last q 0.03 0.51 0.09 0.12 98 all women monitored in early post-partum period (4th stage) per guideline (birth 0.18 0.09 0.09 0.21 98 all newborns monitored in early post-partum period per guideline (birth to disch 0.28 0.01 0.01 0.93 98

75

all women with prolonged labour in last quarter referred to higher level -0.05 0.10 0.11 0.32 103 all women with prolonged labor or Rupture of Membranes and without chorioamnioni -0.03 0.19 -0.05 0.03 103

all women with PPH managed per guideline in the last quarter 0.00 0.93 -0.03 0.03 103 all women with signs of intra- or post-partum sepsis (fever, foul-smelling disch 0.00 0.17 0.00 103 all pregnant women with severe pre-eclampsia and/or eclampsia managed according 0.01 0.44 -0.01 0.25 103 all of neonates who had an APGAR score of ≤5 in the first minute after birth for -0.04 0.05 0.00 103 all neonates with possible serious bacterial sepsis managed per standard in last -0.04 0.36 -0.06 0.02 103

Table A8. PHC unmatched ANCOVA using same-district comparators on input quality measures

CQI Baseline N


Input measures



Safety: % of 4 items 0.01 0.76 -0.01 0.91 87

Privacy: % of 3 items 0.00 0.86 0.06 0.45 84





Vaccine management: % of 17 items -0.01 0.64 -0.05 0.39 56


HMIS: % of 6 items -0.01 0.65 0.17 0.15 82

OPD triaging knowledge: % of 4 items -0.04 0.33 -0.03 0.78 84


Table A9. PHC matched ANCOVA using same-district comparators on input quality measures

CQI Baseline N


Input measures



Safety: % of 4 items -0.01 0.62 0.08 0.48 71

Privacy: % of 3 items -0.01 0.63 0.05 0.57 69







HMIS: % of 6 items -0.02 0.65 0.19 0.32 67 OPD triaging knowledge: % of 4 items -0.06 0.33 0.07 0.63 69

76

TB screening knowledge: % of 5 items 0.01 0.42 -0.02 0.06 56

Table A10. PHC matched ANCOVA using same-district comparators on individual indicators

CQI Baseline N


Visible sign post when arriving at the clinic 0.00 0.96 0.30 0.01 70 Fence/wall in good condition and gate with clearly written emergency contact num 0.11 0.24 0.19 0.09 70

External appropriate wall finishing 0.16 0.02 0.34 0.29 67

Roof intact, presence of well-maintained rain gutters 0.06 0.42 0.41 0.01 70

Minimum of 3 toilets -0.03 0.26 0.10 0.18 70

Hand washing facility with soap available near the toilets 0.05 0.08 0.03 0.33 70

Electricity for 24 hours a day, and 7 days a week -0.01 0.73 0.07 0.54 70 Fire extinguishers available, accessible, functional and serviced AND a clearly 0.17 0.01 0.06 0.38 71

Appropriate drainage of waste water -0.02 0.58 0.18 0.01 70

Shower with either running water or container of at least 100 litres -0.04 0.72 0.20 0.08 70

Waiting area with adequate ventilation of waiting area -0.02 0.44 -0.01 0.12 69

Ground clean with no litters and/or stagnant water and the grass cut 0.02 0.75 -0.02 0.69 70 No waste and dangerous objects in courtyard such as needles – syringes –gloves – 0.02 0.35 -0.01 0.87 70

Recently clean without visible fecal material and without smell -0.03 0.26 0.05 0.58 70

Bin with lid accessible to clients 0.03 0.17 0.15 0.13 70

Building clean and with good ventilation without bad smell? 0.02 0.23 0.02 0.61 69 Walls, Floors and Ceiling / roof clean and in good condition (no leaks, cobwebs) -0.03 0.48 0.09 0.18 69 Bin with plastic liners + sharps containers available and Bin not more than ¾ fu 0.05 0.31 0.10 0.10 69 Hand washing facilities with soap available at accessible points for outpatient -0.06 0.41 0.21 0.01 69

Safe drinking water is accessible 0.08 0.10 0.01 0.87 69

(Ottoway pit OR Rubbish pit) AND Incinerator 0.05 0.39 0.29 0.09 71

Doors with locks and closing properly -0.03 0.26 0.09 0.48 69

Curtains on windows or non- transparent glass and screen 0.03 0.37 -0.04 0.13 69

Sufficient space between beds (at least 1m between beds) -0.01 0.66 0.04 0.39 69

Radio or mobile phone with airtime or landline for communication 0.01 0.76 0.15 0.32 70

Transport plan for emergency referrals and/or contingency plan -0.08 0.08 0.06 0.05 70 Beds, mattresses (covered in plastic), bed sheet and bed side lockers available 0.12 0.16 0.20 0.06 69

Mosquito nets available and in good state? 0.13 0.04 0.20 0.33 69

Adult Weighing Scale and standard pediatric scale available 0.00 0.72 -0.03 0.12 69 BMI calculator, Gluco meters and strips and ophthalmoscope, stethoscope, sphygmo 0.08 0.50 0.10 0.09 69 Important Accessories: cannula, giving sets, syringes and needles, drip stand, s 0.04 0.53 -0.06 0.02 69 Monthly physical counts conducted with min, max and emergency order levels recor 0.07 0.08 0.16 0.04 69

77

The physical stock level corresponds with that on the stock card: supply on stoc 0.01 0.92 0.07 0.29 69

Staff completes and send MIS forms to district each month 0.04 0.74 -0.13 0.09 69

Stored correctly in a locked secured storeroom 0.01 0.12 -0.01 0.12 69 Clean place, well ventilated with cupboards, labelled shelves, no incident light 0.03 0.68 0.04 0.59 69 Medicines stored in alphabetical order also observing the First Expiry First Out 0.00 0.99 0.06 0.43 69

Expired Medicine Register available -0.08 0.14 0.19 0.05 55 No expired products in stock: supervisor verifies randomly 3 medicines and 2 con -0.02 0.82 -0.06 0.38 69

Lockable trolleys available with working lock 0.05 0.08 0.31 0.07 69

Doxycycline capsules 100mg 0.03 0.32 -0.01 0.88 69

Ciprofloxacin tablets 500mg 0.05 0.46 0.10 0.20 69

Metronidazole tablets 200mg Oral 0.03 0.50 -0.06 0.14 69

Diazepam injection 5mg/ml -0.08 0.39 0.00 0.97 69

Benzathine Penicillin injection 0.20 0.18 0.12 0.04 55

Benzyl Penicillin 0.02 0.50 -0.04 0.42 69

Amoxycillin suspension 125mg/5ml (dispersible tablets) 0.00 0.99 -0.03 0.46 69

Ferrous sulphate tablets/Folic Acid -0.02 0.21 -0.02 0.10 69

Zinc Sulphate tabs -0.02 0.54 -0.04 0.02 69

Paracetamol 500mg tablets -0.02 0.52 0.01 0.89 69

Paracetamol syrup or dispersible tablets -0.11 0.33 0.00 0.99 69

Dispensing envelopes -0.02 0.59 0.07 0.18 69

Latex gloves 0.00 0.62 -0.01 0.36 69

Oxytocin 10IU/ml Injection, 1ml Ampoule. -0.02 0.66 -0.06 0.08 69

Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule 0.01 0.94 -0.11 0.29 69

Gentamycin 40mg/ml Injection, 2ml Ampoule 0.03 0.63 0.19 0.40 69

Artemether 20mg + Lumefantrine 120mg Tablets, 24's 0.02 0.34 -0.02 0.13 69

Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial 0.05 0.02 0.44 0.31 69

Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100 Cycles 0.01 0.91 0.08 0.63 69

Lignocaine -0.03 0.56 -0.04 0.03 69 PEP kits available and accessible. Contents of PEP kit: Zidovudine 300 mg + lami -0.08 0.37 0.16 0.42 69 Adult first line ART. Preferred: TDF +3TC+NVP, alternate: TDF+3TC+EFV or ZDV+3TC 0.01 0.44 -0.01 0.44 69 Paediatric first line ART. Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( -0.04 0.24 -0.07 0.05 69

Emergency tray with all the necessary un expired medicines available 0.01 0.83 0.02 0.76 69

Un expired ringer lactate, 5all dextrose, normal saline available 0.04 0.68 0.02 0.85 69 Guidelines/protocols: National Malaria guidelines for diagnosis and treatment of -0.01 0.89 0.05 0.29 69 PEP policy and guidelines: Available in OPD: posted on the wall and up to date -0.08 0.51 0.06 0.55 69 Opportunistic Infection and ART guidelines: Available, accessible in all consult 0.01 0.37 0.11 0.20 69 STI Management protocol: Displayed in all consultation rooms and up to date 0.02 0.63 -0.01 0.92 69

78

IMNCI guidelines: Flowcharts displayed in all consultation areas and up to date 0.03 0.22 0.04 0.37 69

Focused ANC protocol: Displayed I all consultation rooms an up to date -0.01 0.77 0.20 0.01 69

Surveillance line listing and case definitions displayed 0.11 0.17 0.00 0.97 45

Updated EPI schedule, and a contingency plan displayed 0.02 0.81 0.01 0.89 45 EPI graphs showing trends displayed and staff member is able to interpret the g -0.03 0.42 -0.04 0.73 45 EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI -0.15 0.23 0.11 0.19 45

Fridge with a temperature booklet available and filled twice a day -0.01 0.87 0.03 0.57 45 The temperature is within the recommended range of + 2 and+ 8 degrees Celsius (S -0.07 0.18 0.00 0.89 45 The following Antigens are available: BCG, MR ( measles and Rubella), polio, Pen -0.03 0.48 0.09 0.53 45 The physical stock and the amount in the stock cards match ( Supervisor verifies -0.04 0.46 0.09 0.37 45 Vaccines correctly stored in fridge with compartments as follows in fridges with 0.01 0.72 0.01 0.89 45

No expired vaccines -0.04 0.35 0.04 0.69 45

The Vaccine Vial Monitor (VVM) status is kept -0.02 0.23 -0.04 0.18 45

There are readable labels on vials with matching diluents -0.09 0.28 -0.05 0.26 45 The number of syringes available matches the number of vaccines in the stock car 0.00 0.95 -0.10 0.06 45 Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 0.00 0.00 45

Vaccine carriers, cold box, gas regulator, gas cylinder and scissors 0.01 0.84 0.00 45 AEFI investigation forms, case investigation forms for EPI targeted diseases an -0.04 0.62 0.02 0.60 45

Vaccine order forms and stock cards available 0.04 0.48 0.18 0.03 45

IV fluids and giving sets -0.05 0.36 0.02 0.76 68 Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate and -0.04 0.43 0.14 0.05 68

Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant 0.03 0.74 0.04 0.70 68

PPH kits available and complete 0.03 0.56 0.05 0.52 68

Eclampsia kit available and complete 0.02 0.86 0.09 0.44 68 Uterotonic Medicines stored at correct temperature available in delivery room or 0.04 0.07 0.05 0.43 68

Fetoscope, baby blanket Baby scale and tape measure 0.00 0.98 -0.11 0.16 68 Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for umbilical cord 0.08 0.30 -0.19 0.04 68 Penguin suction, neonatal bag and mask and suction tube in delivery room -0.03 0.65 0.05 0.71 68

At least 2 obstetric sterilized delivery standard packs 0.08 0.28 0.07 0.44 68

At least 5 pairs of sterile gloves should be available 0.01 0.81 -0.05 0.21 68 Standard referral forms (at least 10) and register available and properly filled 0.07 0.30 0.13 0.27 67 A referral feedback documented in the register for every referral made and/or re -0.04 0.63 0.41 0.06 63 The following T Series forms available and fully completed: T1, T2, T3, T5, T6, -0.04 0.38 0.13 0.16 67

79

T5 completed and sent timely (by the 7th of the following month) for previous mo -0.03 0.38 0.12 0.05 67

All selected registers complete and correct -0.18 0.02 0.03 0.72 67

T5 correct according to HMIS -0.07 0.58 0.09 0.38 67 Consultations being done by appropriately qualified staff: PCN and/or RGN -0.08 0.34 -0.01 0.22 69

Triage: Emergency signs requiring immediate attention 0.00 0.00 10

Triage: Priority signs (requiring priority in the queue) 0.00 0.00 10

Triage: Non-urgent cases 0.00 0.00 10

TB sign: Weight loss -0.01 0.71 0.00 56

TB sign: Fever for more than 3 weeks -0.02 0.58 -0.07 0.24 56

TB sign: Cough for more than 14 days -0.01 0.34 0.00 56

TB sign: Night sweats 0.02 0.07 -0.01 0.06 56

TB sign: TB contact exposure. 0.01 0.80 -0.07 0.12 56 all of first visit ANC bookings who had documented: BP, Height, Weight measureme 0.08 0.45 0.00 1.00 52 all of first visit ANC bookings who received the standard laboratory test accord -0.01 0.78 0.12 0.01 52

all of first ANC visits who received TT vaccine -0.04 0.35 0.00 52

all of first ANC visits who received iron supplementation 0.02 0.23 -0.03 0.04 52

all of first visit ANC who had documented pregnancy test results 0.06 0.58 -0.33 0.13 52 all PNC visits with assessment: General condition of the infant, NAD recorded if 0.02 0.69 0.02 0.78 52 all PNC visits with assessment: General condition ,Pulse rate, B/P and temperatu 0.15 0.05 0.03 0.78 52 all PNC visits documenting infant feeding (BF) status (exclusive, mixed or not B 0.04 0.29 0.04 0.69 52 all women post-partum counselled and offered any of the modern FP method at fol 0.14 0.04 -0.03 0.65 52 all facility births seen for day 3 PNC visit in any one month in the last quarte 0.00 0.90 -0.07 0.12 51 all NEWLY IDENTIFIED HIV + pregnant women initiated on ART in MNCH (ANC) ON THE 0.10 0.10 0.18 0.01 53 all of infants born to HIV+ women who had a DNA PCR sample within 6-8 weeks of b -0.09 0.27 0.08 0.36 53 all of HIV exposed infants who had A DNA PCR SAMPLE COLLECTED within 6-8 weeks o 0.06 0.56 0.00 0.98 53 all of confirmed HIV positive infants initiated on ART in last quarter WITHIN 21 -0.02 0.72 0.12 0.31 53 all children treated as outpatient for pneumonia in any one of month in last qu 0.09 0.27 0.15 0.29 50 all children with diarrhoea correctly assessed for signs of dehydration), persi 0.03 0.42 0.21 0.06 50 all children diagnosed with malaria that have RDT + or laboratory confirmation i -0.01 0.96 0.56 0.05 50 all children correctly treated as an outpatient for pneumonia in any one of mont -0.02 0.87 0.16 0.09 50 all children correctly treated as an outpatient (ambulatory) for diarrhoea in an -0.02 0.85 0.07 0.01 50

80

all Children with uncomplicated malaria correctly treated according to national -0.03 0.63 0.56 0.04 50 all Children with severe malaria correctly treated according to national guideli -0.10 0.11 -0.16 0.27 50 all of 6-59 months old children with un complicated severe acute malnutrition (S 0.04 0.58 0.12 0.47 50 all women who delivered in the facility monitored using partographs as per crite 0.11 0.27 0.01 0.86 51 all deliveries performed by skilled personnel in any one month in the last quart 0.01 0.41 0.00 51 all total births in any one month in the last quarter documenting administration 0.06 0.03 -0.02 0.34 51 all births with placental status documented at birth in any one month in the las 0.06 0.40 0.03 0.45 51 all newborns BF within one hour of birth in any one month in the last quarter 0.09 0.06 0.10 0.35 51

all newborns received Vitamin K in the any one month in the last quarter 0.11 0.26 0.11 0.07 51 all newborns received eye care (Tetracycline) in the any one month in the las 0.07 0.05 0.09 0.12 51 all newborns received first vaccination (BCG) in the any one month in the last q 0.01 0.83 0.07 0.09 51 all women monitored in early post-partum period (4th stage) per guideline (birth 0.04 0.50 0.17 0.08 51 all newborns monitored in early post-partum period per guideline (birth to disch 0.08 0.30 0.20 0.09 51

all women with prolonged labour in last quarter referred to higher level -0.06 0.03 0.15 0.38 55 all women with prolonged labor or Rupture of Membranes and without chorioamnioni -0.03 0.12 0.00 55

all women with PPH managed per guideline in the last quarter -0.01 0.79 0.00 55 all women with signs of intra- or post-partum sepsis (fever, foul-smelling disch -0.01 0.11 0.00 55 all pregnant women with severe pre-eclampsia and/or eclampsia managed according 0.00 0.84 -0.01 0.34 55 all of neonates who had an APGAR score of ≤5 in the first minute after birth for -0.02 0.24 0.00 55 all neonates with possible serious bacterial sepsis managed per standard in last 0.02 0.53 -0.03 0.37 55

Table A11. Hospital summary ANCOVA results of input quality measures

CQI Baseline N


Input measures



Safety: % of 24 items 0.00 0.91 0.27 0.01 20

Privacy: % of 4 items -0.02 0.69 0.04 0.75 17

Patient amenities: % of 6 items -0.01 0.78 -0.01 0.90 17

General equipment: % of 19 items -0.04 0.36 0.16 0.28 19

HR management: % of 13 items 0.05 0.38 0.09 0.56 21

81

Surgical, lab and radiology equipment: % of 16

items 0.11 0.20 0.35 0.05 18

Pharmacy management: % of 16 items -0.02 0.70 0.16 0.16 18

Pharmacy stock: % of 21 items 0.04 0.30 0.16 0.17 20

Guidelines availability: % of 8 items 0.00 1.00 0.17 0.27 13


Maternity supplies: % of 17 items -0.06 0.31 0.38 0.02 15

HMIS indicators: % of 9 items 0.01 0.73 0.24 0.03 20

Quality improvement strategy: % of 22 items -0.06 0.45 0.29 0.04 13

OPD triaging knowledge: % of 2 items 0.02 0.63 0.09 0.23 13


Table A12. Hospital summary ANCOVA results using Q4 2018 as outcome

CQI Baseline N


Input measures


Cleanliness: % of 7 items -0.16 0.25 -0.46 0.37 18

Safety: % of 24 items -0.01 0.85 0.36 0.12 20

Privacy: % of 4 items -0.09 0.38 -0.06 0.85 15

Patient amenities: % of 6 items -0.13 0.12 0.31 0.07 15


HR management: % of 13 items 0.03 0.81 -0.01 0.97 21 Surgical, lab and radiology equipment: % of

16 items 0.14 0.22 0.07 0.74 17

Pharmacy management: % of 16 items 0.06 0.38 -0.04 0.79 18

Pharmacy stock: % of 21 items 0.05 0.36 -0.18 0.23 20

Guidelines availability: % of 8 items -0.01 0.80 1.66 0.00 13

Vaccine management: % of 16 items 0.02 0.79 0.05 0.75 12

Maternity supplies: % of 17 items -0.13 0.28 0.62 0.05 15

HMIS indicators: % of 9 items 0.06 0.43 0.09 0.62 20

Quality improvement strategy: % of 22 items -0.20 0.25 0.20 0.45 12

OPD triaging knowledge: % of 2 items 13

TB screening knowledge: % of 10 items -0.02 0.82 0.05 0.90 12

Process measures ANC process quality: % of 7 items -0.13 0.27 0.16 0.34 14

PNC process quality: % of 5 items -0.42 0.20 -0.38 0.43 15


Sick child treatment quality: % of 5 items 0.33 0.05 -0.56 0.08 13

Maternal process quality: % of 10 items -0.20 0.29 0.05 0.92 15

Obstetric complications quality: % of 8 items 0.03 0.84 -0.31 0.29 17

Paediatric care processes: % of 4 items -0.29 0.15 0.38 0.12 14 Paediatric complications processes: % of 4

items -0.23 0.42 0.36 0.31 13

Surgical safety: % of 2 items -0.24 0.37 0.50 0.10 13

82

Table A13. Hospital summary difference in differences results

DnD estimator Post CQI N

Coef. p-val Coef. p-val Coef. p-val

Inputs

General infrastructure: % of 18 items 0.05 0.29 -0.11 0.00 -0.01 0.81 39

Cleanliness: % of 7 items -0.14 0.24 -0.16 0.00 -0.02 0.43 39

Safety: % of 24 items -0.09 0.18 0.01 0.91 0.11 0.07 41

Privacy: % of 4 items -0.23 0.01 0.10 0.04 0.13 0.09 35

Patient amenities: % of 6 items -0.24 0.00 -0.12 0.00 0.17 0.03 35

General equipment: % of 19 items -0.01 0.92 0.00 0.99 0.03 0.70 40

HR management: % of 13 items -0.03 0.70 -0.07 0.41 0.06 0.46 42 Surgical, lab and radiology equipment:

% of 16 items 0.10 0.21 -0.10 0.09 0.04 0.61 38

Pharmacy management: % of 16 items 0.08 0.56 0.09 0.06 -0.03 0.83 39

Pharmacy stock: % of 21 items 0.05 0.62 -0.04 0.62 0.00 0.96 41

Guidelines availability: % of 8 items 0.01 0.73 -0.04 0.04 0.04 0.32 34

Vaccine management: % of 16 items -0.10 0.04 0.05 0.22 0.14 0.01 33

Maternity supplies: % of 17 items -0.08 0.37 -0.01 0.76 0.06 0.29 36

HMIS indicators: % of 9 items 0.07 0.54 0.05 0.10 -0.02 0.88 41 Quality improvement strategy: % of 22

items -0.29 0.08 0.32 0.00 0.23 0.07 32

OPD triaging knowledge: % of 2 items -0.10 0.14 0.10 0.14 0.10 0.14 33

TB screening knowledge: % of 10 items 0.08 0.40 0.06 0.31 -0.11 0.13 31

Processes ANC process quality: % of 7 items -0.40 0.01 0.43 0.00 0.34 0.04 35

PNC process quality: % of 5 items -0.38 0.27 -0.17 0.46 0.10 0.52 36 Sick child assessment quality: % of 3

items -0.27 0.44 0.11 0.54 0.27 0.44 33 Sick child treatment quality: % of 5

items 0.03 0.95 -0.31 0.02 0.02 0.89 33 Maternal process quality: % of 10

items -0.23 0.21 -0.04 0.79 0.14 0.15 36 Obstetric complications quality: % of 8

items -0.10 0.56 -0.20 0.11 0.18 0.22 38

Paediatric care processes: % of 4 items -0.42 0.08 -0.01 0.91 0.31 0.04 32 Paediatric complications processes: %

of 4 items -0.57 0.03 0.07 0.49 0.23 0.37 32

Surgical safety: % of 2 items -0.28 0.19 0.08 0.58 0.24 0.20 33

83

Table A14. Hospital ANCOVA results for all facility outcomes

CQI Baseline N


Visible sign post -0.02 0.68 0.00 17

Fence/wall without holes and a gate that can be closed -0.07 0.45 0.63 0.00 17 Functional guard room with boom gate and functional light at the gate 0.17 0.27 0.34 0.02 17

Direction signs with visiting times displayed -0.09 0.55 0.15 0.36 17

Clean ground and grass cut -0.22 0.09 0.00 0.99 17

Garden well maintained -0.02 0.91 0.39 0.06 17

Clearly marked parking area for staff and clients 0.11 0.25 0.01 0.88 17

Connected to local sewage system or septic tank 0.02 0.60 0.00 17

Available incinerator, functional, fenced and being used 0.18 0.49 0.20 0.38 17

Waste pit with hole 0.31 0.06 0.13 0.40 17

Electricity for at 24 hours a day, and 7 days a week 0.02 0.60 0.00 17

The hospital has a functional infection control committee 0.04 0.68 -0.11 0.40 17

National Infection control guideline available 0.04 0.66 -0.10 0.54 17 The committee assesses implementation of infection control guidelines 0.02 0.82 0.17 0.08 17

Functional Autoclave/ Steam steriliser available 0.04 0.42 0.96 0.00 17 Sensitive tape available on sterilized packs and cords not used to tie packs 0.00 0.00 17 Are the following SoPs available in each department/service area? Hand Hygiene, 0.04 0.77 0.17 0.24 17 Kitchen staff members receiving quarterly in house training on food handling 0.01 0.95 0.08 0.60 17

health workers trained on IPC 0.00 0.99 -0.08 0.39 17

Duty roster fully completed for medical and support staff on call -0.02 0.93 -0.23 0.49 13 Response time for staff on call displayed and a review of calls made to ascertai -0.08 0.64 0.32 0.05 13 Suction machine, adult and paediatric laryngoscope, bag and mask (ambubag) (adul -0.27 0.16 0.19 0.32 13

If not functional was a report made? -0.13 0.26 -0.04 0.83 13 Emergency tray with all the necessary drugs (as from EDLIZ). 50% dextrose, adren 0.01 0.94 0.15 0.36 13

Emergency tray book up to date, signed daily, no expired drugs -0.03 0.73 -0.06 0.66 13

Emercency tray is labelled. -0.02 0.90 0.00 13

Cannula, syringes and needles, specimen bottles, -0.14 0.04 0.00 13

Swabs, strapping, disinfectant, gloves, face mask -0.10 0.53 0.35 0.18 13

Functional laryngoscope -0.05 0.65 0.21 0.06 13 Ringer lactate, 5% dextrose, and normal saline available and not expired and giv -0.22 0.20 -0.07 0.70 13

Blood and blood products 0.06 0.65 0.05 0.69 13 OR: Walls of durable material and easily washable walls, Non transparent windows 0.20 0.29 0.35 0.08 14

OR: Good appropriate air ventilation system -0.34 0.03 0.07 0.59 14

Surgical register available and up to date 0.00 0.67 14

84

Operating table: In good state with easy to clean mattress covered with waterpro 0.08 0.47 0.26 0.13 14

Operating table: Functional hand rests with handcuffs & stir ups? 0.20 0.33 0.47 0.16 14

Operating table: The table can be tilted and raised 0.12 0.45 0.22 0.38 14 Anaesthetic machine with: Patient monitor, ECG and ETCO2, Ambu Bag, Laryncoscope 0.09 0.43 0.53 0.00 14 Air way equipment: Endotracheal tubes size 3.5-8.5, Laryngeal mask sizes 2-5, Or 0.26 0.21 0.12 0.60 14 Efficient suction machine, fluid warmer, pressure pump infusion gadget, all rang 0.05 0.84 0.04 0.86 14 Medicines: ketamin, pethidine/morphine, metclorpromide, suxamethanium, atracuriu -0.15 0.43 0.10 0.54 14 Emergency drugs: adrenaline, atropine, NaHCO3, hydrocortisone, promethazine -0.17 0.46 0.21 0.25 14 Inhalation anaesthetic agents: halothane and/or isoflurane, N2O and O2 cylinder 0.22 0.12 0.15 0.35 14

Available trolley with working locks? 0.06 0.73 0.35 0.13 14 Dangerous drugs cupboard (double locked) each with a different lock, with DDA re 0.11 0.23 0.89 0.00 14 At least 5 kits for each of the packs i.e. general and caesarean available -0.11 0.51 0.57 0.01 14

Available and has adequate washing and scrubbing space? 0.00 1.00 0.50 0.00 14

Has Pedal or elbow tap with disinfection device? -0.08 0.33 0.17 0.05 14

Running water and anti-septic available? -0.02 0.93 0.01 0.96 14 Surgical blouse, trousers, masks, hats, sandals and gumboots, goggles and gownin -0.13 0.31 0.30 0.06 14 Qualified (certified) staff (laboratory technician and/or lab scientist)? 0.09 0.61 0.53 0.02 16

Functional and available for emergencies after working hours? 0.02 0.63 -0.02 0.77 16

Laboratory register correctly and completely filled -0.03 0.73 0.00 16 Lab personnel wash dirty pipes in containers with antiseptic (except disposable 0.02 0.65 0.00 16

Internal and external quality controls done 0.03 0.87 0.41 0.12 16

Blood smear: Vivax, Oval, Falciparum, Malariae 0.00 0.98 0.13 0.48 16

Stools: Ascaris, entamoebae, ankylostome, schistosome 0.04 0.82 0.00 16

Available & functional microscope 0.00 0.00 16 Centrifuge, full blood count machine, chemistry analyser machine, incubator and 0.03 0.90 -0.03 0.87 16

Reagents for microscope an centrifuge -0.13 0.21 -0.12 0.14 16

No expired reagents -0.03 0.87 -0.11 0.61 16

Expiry and disposal reagents register available -0.14 0.47 0.16 0.33 16 Gloves, specimen containers, appropriate protective clothing in laboratory 0.06 0.07 0.15 0.00 16

Pharmacy being manned by qualified staff 0.09 0.66 0.53 0.01 18

Specimen signatures for prescribers available at pharmacy 0.01 0.97 0.00 18

Medicines and Allied Substances Act available 0.06 0.64 0.08 0.36 18

Dangerous Drugs Act available 0.08 0.57 0.11 0.32 18

Magnesium sulphate -0.02 0.92 -0.18 0.51 18

Gentamycin, Amoxicillin: select one and check its availability 0.05 0.52 0.00 1.00 18

85

Oxytocin 0.05 0.43 0.00 18 Contraceptives (implant, injectable, post-operative IUD, progestone- oral contra 0.09 0.36 0.66 0.00 18

RHZE: rifampicin + isoniazid + pyrazinamide+Ethambutol -0.23 0.10 -0.02 0.89 18 Hydrochlorthiazide, Metformin, Insulin: select one and check its availability 0.08 0.36 -0.01 0.90 18

Paediatric first line ART: 0.05 0.70 0.09 0.48 18

Adult Second line ART -0.06 0.39 -0.08 0.40 18

Paediatric second line ART: 0.18 0.06 0.12 0.28 18

V Medicines available at 100% in the last three months -0.11 0.40 0.08 0.47 18

E Medicines available at 80% in the last three months -0.03 0.84 0.13 0.24 18

N Medicines available at 60% in the last three months -0.01 0.95 0.03 0.84 18 Monthly physical counts conducted with min, max and emergency order levels reco -0.01 0.95 -0.02 0.94 18

The physical stock level corresponds with that on the stock card -0.04 0.73 -0.10 0.45 18 Stored correctly in a locked secured storeroom (e.g burglar bars on windows and 0.00 0.00 18 Clean place, well ventilated with cupboards, labelled shelves, no incident light -0.08 0.49 0.23 0.12 18 Medicines stored in alphabetical order also observing the First Expiry First Out 0.12 0.24 0.21 0.04 18

Expired medical items disposed according to guidelines -0.15 0.37 0.08 0.61 18

Prescriptions made according to latest edition of EDLIZ 0.00 0.33 18 The hospital sent adverse drug reaction report to PHE and/or MCAZ/MoHCC HQ in th -0.07 0.67 0.18 0.17 18 The average number of antibiotics prescribed to a patient is less than two-three 0.00 0.00 18

Radiological department manned by qualified staff 0.33 0.40 0.01 0.97 12 Radiology department registered with radiation authority of Zimbabwe -0.13 0.65 0.35 0.12 12

Staffs monitored for radiological exposure 0.35 0.34 0.28 0.32 12

X-ray machine Available and working 0.00 0.00 12

Ultrasound scan machine available and working 0.47 0.05 0.57 0.01 12

Protective clothing available for each X-Ray room and in place 0.16 0.42 -0.16 0.42 12

X-ray films 0.09 0.52 0.24 0.23 12

X ray fixers available 0.20 0.36 0.13 0.55 12

X ray developers available 0.04 0.85 0.04 0.85 12 OPD consultations are done by appropriately qualified staff NURSE (RGN)/DOCTOR 0.00 0.00 8 National Malaria guidelines for diagnosis and treatment of uncomplicated and sev 0.10 0.43 0.00 8

PEP policy and guidelines 0.08 0.42 0.00 8

Opportunistic Infection and ART guidelines 0.10 0.43 0.00 8

STI Management protocol -0.03 0.84 0.00 8

IMNCI guidelines -0.06 0.67 0.18 0.33 8

Adult weighing scale and Standard Paediatric Weighing 0.00 0.25 8 BMI calculator, glucometers and strips, peak flow meter, ophthalmoscope, otoscop -0.38 0.20 0.25 0.48 8

Functional thermometer and OPD register in place 0.00 0.00 8

86

PEP kit readily available in the event of a needle stick injury or other acciden 0.18 0.21 0.15 0.41 8

Focused ANC protocol in ANC care area 0.05 0.44 0.00 13

PMTCT guidelines and charts 0.06 0.42 -0.06 0.61 13

PMTCT medicine -0.08 0.08 0.00 1.00 13 Standard Paediatric Weighing (SALTER) Scale, length/height board and MUAC tape -0.08 0.08 0.00 1.00 13

OTP register and case sheet 0.06 0.74 0.45 0.07 13 RUTF (ready to use therapeutic food) (adequate for at least three months: based -0.18 0.38 0.41 0.11 13 Trained health worker on IMAM (Integrated Management of Acute Malnutrition) -0.08 0.30 0.64 0.00 13

EPI surveillance line listing and case definitions displayed 0.02 0.92 0.10 0.54 12

Updated EPI schedule, and a contingency plan displayed 0.04 0.66 -0.04 0.75 12 EPI graphs showing trends displayed and staff member is able to interpret the g 0.12 0.35 0.03 0.72 12 EPI reference materials: EPI Policy, (e.g. multi dose vial policy (MDVP) and EPI 0.03 0.82 0.00 12

Fridge with a temperature booklet available and filled twice a day 0.00 0.00 12 The temperature is within the recommended range of + 2 and+ 8 degrees Celsius 0.00 0.00 12 The following antigens are available: BCG, MR (measles and Rubella), polio, Pent -0.03 0.86 0.00 12 The physical vaccine stock and the amount in the stock cards match 0.15 0.40 0.05 0.71 12

Vaccines correctly stored in fridge 0.00 0.00 12

No expired vaccines 0.00 0.00 12

The Vaccine Vial Monitor (VVM )status is kept 0.08 0.68 0.00 12

There are readable labels on vaccine vials with matching diluents 0.03 0.66 0.22 0.03 12 The number of syringes available matches the number of vaccines in the stock car -0.32 0.11 0.07 0.66 12 Sharps boxes available in immunisation room/corner/area and not more than 3/4 fu 0.03 0.66 -0.03 0.75 12

Vaccine carriers, cold box, gas regulator, gas cylinder and scissors -0.20 0.26 -0.30 0.23 12 AEFI investigation forms, case investigation forms for EPI targeted diseases an 0.07 0.61 0.10 0.43 12

Vaccine order forms and stock cards available -0.10 0.24 -0.03 0.79 12 IV fluids (ringer lactate, 5% dextrose, normal saline) and giving sets 0.03 0.81 0.05 0.61 15 Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium Sulphate, cal 0.07 0.70 0.25 0.05 15 Cannula, syringes and needles, drip stand, swabs, strapping, disinfectant 0.08 0.65 0.50 0.04 15 At least 10 pairs of sterile gloves available, face mask, specimen bottles -0.06 0.68 0.02 0.92 15

PPH kits available and complete -0.03 0.86 0.28 0.16 15

Eclampsia kit available and complete -0.15 0.50 0.32 0.11 15

Fetoscope, baby blanket, Baby scale and tape measure -0.21 0.20 0.04 0.85 15 Sterile cord clamps/ties for umbilical cord, Eye ointment ( Tetracycline) 0.06 0.69 0.10 0.36 15

87

Neonatal bag and mask, penguin suction, resuscitator and suction bulb (at least -0.08 0.32 -0.04 0.71 15

Electric heater, and wall clock 0.13 0.33 -0.13 0.47 15

At least 5 obstetric sterilized delivery standard packs 0.18 0.35 -0.09 0.61 15 All beds in the maternity ward/delivery room are in good state (not broken, matt -0.25 0.17 -0.12 0.50 15

KMC bed 0.05 0.68 -0.05 0.68 15

KMC heat source -0.10 0.16 -0.03 0.66 15

KMC wrap for baby ( mbereko) -0.05 0.74 0.10 0.40 15

KMC Clothes for baby ( hat, nappy and socks) 0.15 0.40 -0.27 0.13 15

KMC register 0.00 0.00 15

Standard referral forms (at least 10) available 0.03 0.61 -0.03 0.51 15

Referral register available and properly filled -0.33 0.40 0.02 0.95 13

The T Series forms are available and fully completed -0.12 0.32 -0.05 0.71 15

The T5 and HS3/5completed and sent timely 0.05 0.65 0.20 0.05 15

N complete registers -0.05 0.89 -0.01 0.95 15 Are the information in each column of the selected registers complete and correc 0.08 0.77 0.22 0.34 15

N correct registers 0.85 0.10 0.53 0.00 15 Are the figures reported in any one month of the last quarter correct according 0.40 0.18 0.53 0.02 15

Triaging of patients at OPD waiting area during all clinic shift 0.05 0.57 0.08 0.44 12

TB screening: weight loss 0.00 0.00 12

TB screening: fever -0.21 0.05 -0.04 0.78 12

TB screening: cough 0.03 0.66 0.00 1.00 12

TB screening: night sweats 0.06 0.51 0.00 1.00 12

TB screening: TB contact exposure -0.38 0.08 -0.22 0.35 11 % of TB presumptive (TB symptom positive) that have sputum results documented in -0.13 0.78 -0.50 0.48 11 % of TB patients (SS + and SS-) that have HIV test results documented in any mo 0.17 0.49 -0.17 0.54 11 % TB patients diagnosed in any one month in the last quarter that are receiving 0.29 0.36 -0.29 0.54 11 % TB patients diagnosed in any one month in the last quarter that were TB notifi 0.00 1.00 0.50 0.04 11 % TB patients diagnosed in any one month in the last quarter that TB contact tra 0.22 0.49 0.84 0.04 11

Total OPD and TB -3.75 0.25 0.44 0.11 12

ANC assessment -0.07 0.65 0.12 0.42 10

ANC lab tests -0.29 0.46 0.00 9

ANC iptp 0.11 0.49 -0.11 0.60 10

ANC TT vaccine 0.03 0.65 0.00 10

ANC iron 0.09 0.35 0.13 0.17 10

ANC pregnancy test -0.46 0.04 0.25 0.35 10

PNC newborn assessment 0.00 0.00 9

PNC mother assessment -0.50 0.24 0.67 0.10 9

PNC infant feeding 0.00 1.00 9

PNC family planning -0.48 0.06 -0.23 0.23 9

88

subtotal_points_scored_2c -5.65 0.38 -0.35 0.54 9

Maternity waiting home monitoring -0.25 0.06 0.90 0.00 12

group_ge5cd64/complete_records_5c1 0.55 0.44 0.14 0.35 14

Pneumonia assessment -0.05 0.81 0.21 0.27 14

group_jk5pk73/complete_records_5c2 0.00 0.00 14

Pneumonia treatment 0.09 0.66 -0.09 0.66 13

group_ez5ry68/complete_records_5c3 0.00 0.00 14

Diarrhoea assessment 0.07 0.75 0.13 0.45 13

group_qq8do67/complete_records_5c4 0.00 0.00 13

Diarrhoea treatment 0.15 0.39 0.35 0.06 13

group_ad1im39/complete_records_5c5 0.00 0.00 13

Malaria diagnosis 0.24 0.23 0.13 0.48 14

group_yc2gx93/complete_records_5c6 0.00 0.00 14

Uncomplicated malaria treatment 0.23 0.24 0.23 0.24 14

group_tx3ii64/complete_records_5c7 0.00 0.00 14

Severe malaria treatment -0.15 0.39 0.35 0.06 13

group_oq0on51/complete_records_5c8 0.00 0.00 14

Acute malnutrition treatemtn 0.03 0.93 0.21 0.50 13

Total Sick child care 4.25 0.62 0.48 0.10 14 deliveries performed by skilled personnel in any one month in the last quarter -0.07 0.23 0.00 17 women who delivered in the facility monitored using partographs as per criteria -0.02 0.92 0.22 0.19 17 total births in any one month in the last quarter documenting administration of -0.18 0.20 -0.15 0.49 17 births with placental status documented at birth in any one month in the last qu 0.01 0.90 -0.10 0.57 17 newborns BF within one hour of birth in any one month in the last quarter 0.04 0.76 0.00 17 newborns received Vitamin K in the any one month in the last quarter -0.04 0.89 0.20 0.35 17 newborns received eye care (Tetracycline) in the any one month in the last quart 0.10 0.30 -0.10 0.25 17 newborns received first vaccination (BCG) in the any one month in the last quart -0.01 0.96 -0.06 0.70 17 women delivered monitored in early post-partum period (4th stage) per guideline 0.23 0.37 0.11 0.58 17 newborns monitored in early post-partum period per guideline (birth to discharge 0.14 0.52 0.14 0.41 17 facility births seen for day 3 PNC visit in any one month in the last quarter -0.13 0.58 -0.11 0.53 17

Total maternity care 2.31 0.60 0.19 0.26 15 women with prolonged labor or Rupture of Membranes and without chorioamnionitis -0.16 0.43 0.03 0.86 17

women with PPH managed per guideline last quarter -0.03 0.85 -0.27 0.07 17 women with signs of intra- or post-partum sepsis that were treated per standard -0.34 0.05 0.18 0.21 17 pregnant women with severe pre-eclampsia and/or eclampsia managed according to -0.08 0.60 0.24 0.05 17

89

neonates who had an APGAR score of ≤5 in the first minute after birth for whom r 0.08 0.57 -0.47 0.00 17 neonates who had an APGAR score of ≤5/10 at 1 minute after delivery and were suc 0.20 0.21 -0.18 0.18 17 neonates with possible serious bacterial sepsis managed per standard in last qua 0.12 0.57 0.29 0.10 17 low birth weight (LBW) newborns admitted to KMC unit in the last quarter -0.06 0.69 0.08 0.55 17

Total complications -6.43 0.48 -0.29 0.36 17 hospitalized patients with correctly completed admission medical record any one -0.15 0.60 0.19 0.42 15 Written record of administration of patient medications up to date any one month -0.14 0.35 0.31 0.05 15 hospitalized patients with documentation of vital signs every 6 hours and every -0.19 0.15 0.25 0.07 15 hospitalized patients with documentation of daily progress note by doctor any on -0.26 0.15 0.43 0.05 15

Total: pediatric best practices -2.79 0.25 0.36 0.02 15 hospitalized children treated for pneumonia any one month in the last quarter wh -0.04 0.80 -0.03 0.78 14 hospitalized children treated correctly (all criteria met) for pneumonia among t -0.09 0.59 -0.02 0.90 14 hospitalized children treated for diarrhoea in any one month in the last quarter -0.33 0.11 -0.08 0.65 14 hospitalized children treated for diarrhoea correctly (all criteria met) in any -0.02 0.93 -0.10 0.56 14

Total: pediatric complications -0.55 0.83 -0.08 0.50 14

Surgical safety checklist utilization rate -0.08 0.73 0.58 0.01 14

Surgical site infection 0.07 0.75 0.26 0.27 14

Total: surgery safety 0.90 0.81 0.73 0.05 14 Does the QIC have terms of reference (ToR) with the following components 0.00 1.00 0.33 0.20 13 Are all service areas, including administration, represented in the QIC committe 0.03 0.82 0.14 0.47 13

QI plan: Action plan for improvement 0.08 0.65 0.35 0.05 13

QI plan: Targets 0.02 0.94 0.25 0.20 13

QI plan: Areas for improvement -0.08 0.72 0.29 0.15 13

Are reports on QI activities sent to PMDs quarterly? -0.28 0.12 0.09 0.63 13 Are processes like waiting time, appointment system, and patient flow discussed 0.23 0.28 0.04 0.82 13

Is the QIC meeting quarterly? 0.04 0.71 0.00 1.00 13 Are status of QI plan and other improvement plans discussed during the meeting? -0.04 0.78 0.13 0.33 13 Completeness and correctness of the information in the reviewed patient files 0.17 0.35 0.15 0.33 13 Comprehensive patient management given to patients in the reviewed files -0.12 0.49 0.28 0.08 13 Consistency of information between the reviewed patient files and registers -0.08 0.65 0.40 0.02 13

Does the QIC receive feedback from the supervision? -0.18 0.24 0.03 0.81 13

90

Has the QIC developed action plan for improvement to address the identified gaps 0.02 0.92 -0.01 0.95 13 Quarterly client satisfaction surveys: Does the QIC have survey tool? 0.05 0.81 0.19 0.34 13

Does the QIC conduct quarterly surveys? -0.05 0.82 0.45 0.07 13

Are survey analysis reports available? -0.15 0.39 0.47 0.02 13

Does the facility have labelled suggestion boxes? -0.07 0.62 0.03 0.85 13 Is instruction on how to use the boxes posted on or above the suggestion boxes? -0.04 0.84 0.07 0.66 13

Does the facility analyse the findings of suggestion boxes -0.16 0.45 0.01 0.94 13 Does the hospital conduct audit meetings or maternal mortality review meetings a 0.00 0.00 13

Is guideline for audit/mortality meetings available? -0.22 0.20 0.04 0.83 13

91

PHC summary outcome figures

101

Hospital summary outcome figures

114

Annex 4. Continuous Quality Intervention checklists for PHCs and Hospitals

HOSPITAL QUALITY SUPERVISION CHECKLIST

February 2016

115

HOSPITAL QUALITY SUPERVISION CHECKLIST Questionnaire for a Provincial/District/Mission Hospital Quality Review Province: ___________________________________________________________________________ District: ___________________________________________________________________________ Hospital: ___________________________________________________________________________ Number of beds: ________ Catchment area population: __________________________________________ Date of supervision: ______________________________________________________________________ Name of supervisors and designation

No. Name of supervisor Designation

1

2

3

4

5

6

7

8

Date received by ____________________________________

For RBF use: % Structural score (35%Weight): ………….... % Management & planning score % Clinical care score (65% weight): ……….... Final Combined Score from Database: ……....

116

I. Hospital Staffing

STAFF Establishment In post

Vacant Duration of vacancy

District Medical Officer

Government Medical Officer

Clinical Officer

Health Promotion Officer

District Nursing Officer

Matron

Sister in Charge

Sister in Charge Community

Principal State Certified Nurse

Sister General

State Certified Nurse

District TB Coordinator

District Environmental Health Officer

Environmental Health Officer

Environmental Health Technician

Pharmacist

Pharmacy Technician

Dispensary Assistant

Nutritionist

Assistant Nutritionist

Radiographer

X-ray Operator

Dark Room Assistant

Physiotherapist

Medical Laboratory Scientist

District Health Service Administrator

Human Resource Officer

Health Information Assistant

Accountant

Accounting Assistant

Government Dental Officer

Dental Therapist

Dental Surgery Assistant

Rehabilitation Technician

CCSD Packer

Nurses with midwifery

Up skilled PCN

PCN

Operating theatre nurses

Nurse anaesthetist

Nurse aides

General hands

Non-medical staff or unqualified staff : Cook, ,

117

Non-medical staff or unqualified staff : driver

Non-medical staff or unqualified staff : Laundry Hand , Senior Hand)

ASSESSMENT SUMMARY

I. STRUCTURAL SECTION

Available Points

Number of composite indicators

Applicable valid points this quarter

Number of applicable composite indicators this quarter

Total points scored

General Structure and auxiliary services

11

6

Structure in clinical departments

44 18

TOTAL 55 24

II, MANAGEMENT & PLANNING SECTION

Available Points




Total points scored

Administration, finance, planning

17 8

Infection control 8 4

Emergency services 12 5

Operating theatre 19 6

Laboratory 13 10

Pharmacy 28 11

Radiological services 9 4

Outpatient department (OPD)

10 3

Family and Child Health (FCH)

7 2

Extended Program Immunization (EPI)

17 7

Maternity ward 17 7

HMIS 14 4

TOTAL 171 71

III. CLINICAL MANAGEMENT SECTION

Available Points




Total points scored

OPD/ consultation area

18 3

TB management 24 4

ANC-PNC Best practices

54 9

Maternity waiting home

6 1

HIV-PMTCT 30 5

119

Ambulatory management diarrhoea, pneumonia

48 8

Delivery best practices

66 11

Management obstetric complications

54 8

Inpatient Pediatric best practices

28 4

Inpatient management diarrhea, pneumonia

28 4

Postoperative infection control

12 2

Quality Improvement/Assurance

22 7

TOTAL 384 66

120

STRUCTURAL SECTION

I.1 Structural indicators in the general compound of the Hospital, mortuary, Operating theatre and maternity waiting home

*Please give a score for each of the criteria under each indicator as per the criteria in the right column *N.B. The items highlighted in bold are the indicators

Score: 1: if all criterion have been met/ recorded 0: if all criterion have not been met/ not recorded N/A: not applicable

1S Outside appearance (when arriving at hospital):

1S.1 -Visible sign post

1S.2 -Fence/wall without holes and a gate that can be closed

1S.3 -Functional guard room with boom gate and functional light at the gate

1S.4 -Direction signs with visiting times displayed

2S Maintenance of the ground:

2S.1 -Clean ground and grass cut

2S.2 -Garden well maintained with flower beds, trees or lawn, resting places (benches in shade),and with no animal excrement, no litter or dangerous objects such as needles, syringes, gloves, used cotton wool, etc.,

2S.3 -Clearly marked parking area for staff and clients

3S Waste water drainage system

3S.1 -Connected to local sewage system (if not, septic tank must be available)

4S Incinerator within the premises:

4S.1 -Available, functional, fenced and being used

5S Waste pit for non-contaminated objects:

5S.1

-Waste pit with hole of minimum 3 metres depth fenced, with no contaminated and non-decomposable (non – biodegradable) objects available (Waste pit is only required in hospitals where the city municipality is not collecting the non-contaminated objects)

6S Lighting system

6S.1 -Electricity for at 24 hours a day, and 7 days a week (Source of electricity: ZESA with backup system of either generator or solar energy and/or inventors).

Total points this quarter (MAXIMUM AVAILABLE Points: 11 POINTS)

121

I.2 Structural Indicators in Selected Departments:

Indicators

Randomly select one department each quarter for indicators listed below. Please vary the department to be selected quarterly *Please give a score for each of the criteria under each indicator as per the criteria in the right column *N.B. The items highlighted in bold are the indicators *if you write N/A for an indicator in a selected department, please deduct the points for the indicator from the maximum available points

Wrap

Please select one department among the following four departments ( encircle the selected department) and assess indicators 7S-16S 1. OPD 2. FCH 3. Maternity Ward 4. Paediatric Ward

7S Outside appearance of buildings

7S.1 -External appropriate wall finishing (painting, bricks or rough plastering)

7S.2 -Roof intact with well-maintained gutters, window panes

8S Inside appearance of building and its cleanliness

8S.1

-Walls and Floors- Clean, without cracks and floors-polished ( if required) and Ceiling / roof intact without leaks neither cobwebs or mould growth

8S.2 -Doors with locks and closing properly

9S Firefighting System:

9S.1 -Functional and serviced fire extinguishers with or without water hoses available and accessible

9S.2

-Fire exits clearly marked and is there a clearly marked firefighting assembly area and evacuation plan: Ask staff member on firefighting plan

10S Garbage bins in ground

10S.1

-Labelled Bins with lids and plastic lining available and not more than ¾ full without hazardous waste (e.g. sharps, used cotton wool e.t.c) at accessible points

11S Presence of sufficient well-maintained latrines/toilets

122

11S.1 -One water closet for every 6 inpatients or One water closet for every 20 outpatients

11S.2 -The toilet entries should be clearly marked for each sex and the male toilets should also have a urinal

11S.3 -Recently cleaned without visible faecal matter or urine

12S Water and soap for hand washing:

12S.1 -Hand washing facilities with running water and soap available at accessible points

13S Waste collection

13S.1 -Waste collected and disposed daily or within 4 to 5 hours ( only for maternity ward) (Ask staff member)

13S.2

Waste segregated with colour coding: -Red/yellow for non-sharp infectious waste -Yellow or other puncture proof containers for sharp objects -Black for communal non-sharp non-infectious wastes and/or service area *the bins and/or sharp boxes should not be more than ¾ full

14S Protective clothing and disinfectant use by cleaners: * Ask the available cleaners during the assessment period

14S.1

-Cleaners have appropriate protective clothing (Heavy duty gloves, Uniforms, Dustcoats, Plastic Aprons, Gumboots, Face Mask)

14S.2

-Cleaners know how to appropriately use disinfectants? Soap with water for general cleaning and 1 part jik to 4 parts for spillages and/or depending on the concentration of the bleach i.e. blood and mainly body fluids (after containing and cleaning the spills): Ask the available cleaners during the day of assessment

15S Professional staff appropriately dressed :

15S.1 -Clean with standard uniform, identification tag and lace up shoes.

123

16S Staff duty roster, staff leave calendar and clock in register (per department) including maintenance and cleaning duty roster with timeline and signature column for cleaner and supervisor:

16S.1

-Duty roster clearly visible including assigned nurse(s) and doctors(s) covering each ward for each shift, including night and weekend coverage

16S.2 - The DOCTOR, NURSES actually present in the ward (Check the person/s on duty on the day of visit)

16S.3 -Staff leave calendar complete and up to date, and displayed where all staff can see

16S.4

-Clock in register including maintenance and cleaning with timeline and supervisor signature column

Sub-total points (maximum available sub-total points: 20)

Please select one department among the following two departments ( encircle the selected department) and assess indicators 17S-20S 1. OPD 2. FCH

17S Good conditions in waiting area, meeting minimum standards:

17S.1 -With sufficient benches and / or chairs (according to daily attendance): calculate the average daily attendance from the weekly attendance

17S.2 - Well/adequately ventilated with sufficient light adequate ventilation of waiting area:

• If Open space with a shade or roof supported by brick or metal pillars or

• If closed space: windows should measure at least 1/10 of floor area and at least ½ of window area should be openable.

18S Consultancy rooms in good condition, meeting minimum standards

18S.1 -Windows non-transparent glass and screen around bed

18S.2 -Functional doors with lock

18S.3 -Furniture (at least one chair and table for nurse/doctor, one chair for patient)

124

positioned to reduce transmission of infection from patient

18S.4 -Hand washing facility with running water and/or alcohol based hand rub

18S .5 -Examination bed in good condition and covered with clean linen

19S OPD fees, and medical aid companies:

19S1

-OPD fees and list of accepted medical aid companies accepted displayed in local vernacular (when applicable) and easily visible for patients

20S Hygienic and aseptic conditions in wound dressing:

20S.1 -Bench and foot rest covered with Macintosh available

20S.2 -Bins for infected and contaminated objects with lid, plastic lining and foot pedal available and not more than ¾ full

20S.3 -Sharp box well positioned and not more than ¾ full?

Sub- total points (maximum sub-total points: 11)

Please select one department among the following two departments ( encircle the selected department) and assess indicators 21S-26S 1. Maternity Ward 2. Paediatric Ward

21S Availability and status of furniture and other items

21S.1 -Beds, and mattresses with sheets, blankets, bedside lockers, benches on bed side available and in good state (not broken, torn and clean)

21S.2 -Mosquito nets (in malaria endemic areas) available and in good state ( Not Applicable in Non-Malaria Endemic area)

21S.3 -Fan and heater available when required and cleaned on a regular basis

22S Bucket or basin for dirty linen:

22S.1 -Bins covered with a lid and not overflowing

22 S.2 - SOP for linen segregation available as per standard National IPC guideline

125

23S Hygienic condition, access to water and space:

23S.1 -Clean and regular cleaned ( ask staff member or look at the weekly plan for cleaning)

23S.2 -Safe drinking water available and accessible

23S.3 -Enough space between beds (at least 1m between beds)

23S.4 -Well ventilated without bad smells and/or all windows operational and open

23S.5 -Shower with running water, and/ or container with at least 100 litres for patients to bath with hot water available during winter?

24S Movable lockable drug trolleys:

24S.1 -Available with working locks?

24S.2 -Dangerous drugs cupboard (double locked) each with a different lock available

24S.3 -DDA registers available

Sub-total ( maximum sub-total available points: 13)

Total points this quarter (MAXIMUM AVAILABLE Points: 44 POINTS)

126

MANAGEMENT AND PLANNING: STAFF, POLICY, GUIDELINES, Medicines &

SUPPLIES and Vaccines

II.1 ADMINISTRATION, FINANCE AND PLANNING Score:

1: if all criterion have been

met/ recorded

0: if all criterion have not

been met/ not recorded

Indicat

ors

*Assess at District Medical officer’s office/Administration and/or

Finance Department /Matron office

*Please give a score for each of the criteria under each indicator as

per the criteria in the right column

*N.B. The items highlighted in bold are the indicators

1M Mission statement, Vision, Values and patient charter

1M.1 -Displayed in public places and clearly visible

2M Catchment area map, spot map, monitoring graphs, demographic data:

* Assess the indicator in the office of the DMO/Matron or administrator office

2M.1 -Catchment area maps with current catchment population target

population for services calculated correctly and displayed

2M.2 -Spot map showing recent or suspected out breaks with clear markings

displayed

2M.3 -Monitoring graphs for different services displayed

3M Documentation of activities/ Operational Plan

3M.1 -Staff minute book/file available, well filed and up to date

3M.2 -Quarterly review reports, annual operational plan and annual progress

report ( of previous year) available, well filed and up to date

4M Management book, inventory register, maintenance book, returns( human, material and finance)

4M.1 -Available and up to date?

5M Service and maintenance plan for hospital equipment and vehicles

5M.1 -Plan available and being followed (Ask for reports and cross check with

the plan)

6M Purchasing of medicines, equipment and consumables

6M.1 -MoHCC tender and procurement procedure documents available?

6M.2 -Functional CBU and PTC available

6M.3 -Medicines, equipment and consumables purchased as per the procedure?

(ask the procurement committee to provide the documents for an item

which was purchased and cross check whether it was bought as per the

MoHCC procedures)

7M Finance and accounting system

7M.1 -Financial and accounting documents available and well-kept in clearly

labelled files including bank statements, payment vouchers with attached

support documents.

7M.2 -Document showing budget and revenues (GoZ, HSF, RBF, Donations)

available

7M.4 -Managed by qualified staff (accountant/ accounts assistants)?

7M.5 -Financial reports which show expenditure of proper utilization of funds

according to statutory requirements and minutes of finance meetings

available

8M Ambulance and communication systems

8M.1 -The hospital has a functional (24/7) ambulance

8M.2 -The hospital has a functional communication system (land line/cell phone/radio)

127

Total points this quarter: (Maximum Available Points: 17)

II.2 . INFECTION CONTROL




*If N/A, please deduct the points for the indicator from the

maximum available points

Score:


met/ recorded



9M Infection control committee

*Assess the indicator by asking the infection control committee/focal person

9M.1 -The hospital has a functional infection control committee (stand alone

or as part of the quality improvement committee) ( check for the minutes

of quarterly meetings)

9M.2 -National Infection control guideline available with a facility risk

assessment tool including TBIC

9M.3 -The committee assesses implementation of infection control guidelines

and takes measures according to the guideline? (check in reports/minutes)

10M Sterilization of instruments:

Assess the indicator at CSSD (Central Steam Sterilizing Department) as all sterilizations are done in this

department.

10M.1 -Functional Autoclave/ Steam steriliser available

10M.2 Sensitive tape available on sterilized packs and cords not used to tie

packs ( check at least two packs)

11M Basic Infection Prevention and Control (IPC) SOPS

11M.1 -Are the following SoPs available in each department/service area? Hand

Hygiene, Environmental Cleaning; waste segregation and management;

decontamination.

Please assess in one selected department quarterly.

12M Regular training of kitchen staffs on food handling and health workers on IPC: Check for training reports

from Hospital Food Services Supervisor and IPC focal person/Committee chair and ask two kitchen staff

members and/or health workers

12M.1 -Kitchen staff members receiving regular in-house training (quarterly) on

food handling

12 M.2 -health workers trained on IPC


II.3 EMERGENCY SERVICES

Indicat

ors *Please give a score for each of the criteria under each indicator as



*

Score:


met/ recorded



128

13M. Staff Duty Roster for On call Medical doctor and support staff

13M.1 -Duty roster fully completed and posted in public place for medical and

support staff on call through the month (doctor, lab, x-ray, anaesthetist,

theatre nurse and observed on the day of the visit)?

13M.2 -Response time for staff on call displayed and a review of calls made to

ascertain response time

SUPPLIES

14M Are the following emergency airway equipment available and functional?

14M.1 -Suction machine, adult and paediatric laryngoscope, bag and mask

(ambubag) (adult -and paediatric), oxygen.

14M.2 - If not functional was a report made? *Equipment should be easily

accessible not stored under lock and key.

15M Emergency tray:

15M.1 -With all the necessary drugs (as from EDLIZ). 50% dextrose,

adrenaline, lignocaine, diazepam, atropine,

15M.2 -Emergency tray book up to date, signed daily, no expired drugs

15M.3 -The tray is labelled. (should be including the drugs and accessories, with

clearly legible and durable labels)

16M Important Accessories:

16M.1 -Cannula, syringes and needles, specimen bottles,

16M.2 -Swabs, strapping, disinfectant, gloves, face mask

16M.3 -Functional laryngoscope

17M IV fluids and blood:

17M.1 -Ringer lactate, 5% dextrose, and normal saline available and not expired

and giving sets, drip stand

17M.2 Blood and blood products


II.4. OPERATING THEATRE

Indicat

ors




*If N/A, please deduct the points for the indicator from the


Score:


met/ recorded



N/A: not applicable

18M Structure and Ventilation system, and register

18M.1 -Walls of durable material and easily washable walls, Non transparent

windows and functional doors with floor paved with vinyl/ceramic tiles

without cracks, ceiling in good state

129

18M.2 -Good appropriate air ventilation system: small meshed windows to let

air in ( if only natural ventilation is used) and/or air conditioner fitted

without/with sealed windows ( if mechanical ventilation is used), and

doors to the aisle closed

18M.3 -Surgical register available and up to date?

SUPPLIES

19M Operating table

19M.1 -In good state with easy to clean mattress covered with waterproof

material?

19M.2 - Functional hand rests with handcuffs & stir ups?

19M.3 -The table can be tilted and raised

20M Basic equipment and consumables:

-Are the following basic equipment & consumables available?

20M.1 Anaesthetic machine with: Patient monitor, ECG and ETCO2, Ambu

Bag, Laryncoscope size 0-4 and O2 cylinder for back up, functional

failure alarm and7 ventilator

20M.2 Air way equipment: Endotracheal tubes size 3.5-8.5, Laryngeal mask

sizes 2-5, Oral air way sizes 00-6, Intubating intruder and all range sizes

of face masks

20M.3 Efficient suction machine, fluid warmer, pressure pump infusion gadget,

all range sizes of cannulas, syringes and needles and defibrillator

20M.4 Medicines: ketamin, pethidine/morphine, metclorpromide,

suxamethanium, atracurium, ephedrine, neostigmine, diclophenac

IM/supporitory

20M.5 Emergency drugs: adrenaline, atropine, NaHCO3, hydrocortisone,

promethazine

20M.6 Inhalation anaesthetic agents: halothane and/or isoflurane, N2O and O2

cylinder

21M Movable lockable drug trolleys

21M.1 -Available with working locks?

21M.2 -Dangerous drugs cupboard (double locked) each with a different lock,

with DDA registers available?

22M Emergency surgical packs (general and caesarean ):

22M.1 -At least 5 kits for each of the packs i.e. general and caesarean available

23M Gowning area and theatre clothing:

23M.1 -Available and has adequate washing and scrubbing space?

23M.2 -Has Pedal or elbow tap with disinfection device?

23M.3 - Running water and anti-septic available?

23M.4 -Surgical blouse, trousers, masks, hats, sandals and gumboots, goggles

and gowning packs available


130

II.5. LABORATORY SERVICES

Indicat

ors




Score:


met/ recorded



24M Staffing of Laboratory:

24M.1 -Qualified (certified) staff (laboratory technician and/or lab scientist)?

25M Functionality of Laboratory after working hours:-

25M.1 -Functional and available for emergencies after working hours? * Verify

after hour activities in the laboratory register for the last quarter

26M Recording of results:

26M.1 -Laboratory register correctly and completely filled:

Check record of any month in the last quarter

27M Washing dirty pipettes:

27M.1 -Lab personnel wash dirty pipes in containers with antiseptic (except

disposable pipettes)?

28M Internal and External quality assurance services:

28M.1 -Internal and external quality controls done

*Check for copy of report of internal and External quality assurance

assessment report (by ZINQAP)

c

SUPPLIES

29M Parasite demonstration on plastic paper, in a colour book, or put on wall

29M.1 Blood smear: Vivax, Oval, Falciparum, Malariae

29M.2 Stools: Ascaris, entamoebae, ankylostome, schistosome

30M Microscope:

30M.1 -Available and in working condition (functional) with functional

objectives - immersion oil – mirror or electricity and – blades, cover

glass, slides, GIEMSA

31M Equipment:

31M.1 -Centrifuge, full blood count machine, chemistry analyser machine,

incubator and fume cup board (also serviced) available and serviced and

functional:

( Check in the maintenance register if the assessments were done

monthly and signed for)

32M Reagents

32M.1 -Available for the equipment mentioned under 30M.1 and 31M.1?

(Check stocks against minimum levels)

32M.2 -No expired stocks

32M.3 -Expiry and disposal register available

33M Are the following items available?

131

33M.1 -Gloves, specimen containers, appropriate protective clothing


II.6. PHARMACY (MEDICINES AND SUNDRIES STOCK

MANAGEMENT)

Indicato

rs




Score:


met/ recorded



34M Staffing and Specimen Signature:

34M.1 -Pharmacy being manned by qualified staff (Pharmacist and/or

Pharmacy technician)?

34M.2 -Specimen signatures for prescribers available at pharmacy?

35M Statutory instruments:

35M.1 -Medicines and Allied Substances Act available

35M.2 -Dangerous Drugs Act available

36M Availability of essential medicines: -Availability for at least 90 days ( Minimum stock)

36M.1 Magnesium sulphate

36M.2 Gentamycin, Amoxicillin: select one and check its availability

36M.3 Oxytocin

36M.4 Contraceptives (implant, injectable, post-operative IUD, progestone-

oral contraceptive, combined oral contraceptive pills)* select one among

the list and check its availability

36M.5 RHZE: rifampicin + isoniazid + pyrazinamide+Ethambutol

36M.6 Hydrochlorthiazide, Metformin, Insulin: select one and check its

availability

37M HIV and AIDS medicines: Availability for at least 90 days.

37M.1 Adult first line ART:

Preferred: TDF +3TC+EFV, alternate: TDF+3TC+NVP or

ZDV+3TC+EFV/NVP ( could available in Dual or triple FDCs)

37M.2 Paediatric first line ART:

Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( <3 yrs)

37M.3 Adult Second line:

AZT + 3TC + ATV/r or LPV/r or TDF + 3TC + ATV/r or LPV/r

37M.4 Paediatric Second line:

ABC+3TC+LPV/r

38M VEN medicines-

Please ask for the MIS report for any one month in the last quarter and then check in the report

whether the VEN medicines were available as required

38M.1 V Medicines available at 100% in the last three months

38M.2 E Medicines available at 80% in the last three months

38M.3 N Medicines available at 60% in the last three months

39M Stock Cards:

132

39M.1 -Monthly physical counts conducted with min, max and emergency

order levels recorded and updated

39M.2 - The physical stock level corresponds with that on the stock card

40M Storage of drugs:

40M.1 -Stored correctly in a locked secured storeroom (e.g burglar bars on

windows and doors)?

40M.2 -Clean place, well ventilated with cupboards, labelled shelves, no

incident light

41M.3 -Medicines stored in alphabetical order also observing the First Expiry

First Out rule

41M Expired products:

41M.1 -Separated from stock

41M.2 -Expired medical items disposed according to guidelines(Check for the

presence of expired medicine register disposal register and certificate.

verify randomly 3 medicines and 2 consumables (check stock cards)

42M Prescriptions:

* check last 3 prescriptions made during the day of assessment from OPD register and compare

with EDLIZ

42M.1 -Prescriptions made according to latest edition of EDLIZ

43M Adverse events report:-

*Ask the DMO or at the pharmacy

43M.1 -The hospital sent adverse drug reaction report to PHE and/or

MCAZ/MoHCC HQ in the last quarter (Check for presence of copy of

adverse event) ( if there was no adverse event, check for the presence of

the reporting forms)

44M Average number of antibiotics prescribed to a patient

•Assessment: measured by considering the previous 30

patients/prescription/T12 (pharmacy register) and then tallying the

number of antibiotics per prescription and divide by 30. (Acceptable

range is 2-3)

*Source of data: Pharmacy register and/or T12

44M.1 -The average number of antibiotics prescribed to a patient is less than

two-three antibiotics in any one month in the last quarter


II.7 RADIOLOGICAL SERVICES

Indicato

rs *Please give a score for each of the criteria under each indicator as



Score:


met/ recorded



45M Staffing of Radiological department:

45M.1 -Manned by qualified staff? (radiographer or x-ray operator

46M Registration and monitoring of staff for exposure:-

46M.1 -Radiology department registered with radiation authority of Zimbabwe

46M.2 -Staffs monitored for radiological exposure

SUPPLIES

133

47M Radiological equipment:

47M.1 - X-ray machine Available and working

47M.2 -Ultrasound scan machine available and working ( if it is available in

maternity ward, consider it as available, but check its functionality)

48M Protective clothing and necessary safety precautions:

48M.1 -Available for each X-Ray room and in place

49M Consumables : minimum level

49M.1 -X-ray films

49M.2 X ray fixers available

49M.3 X ray developers available


II.8. OUTPATIENT DEPARTMENT (OPD)

Indicato

rs




Score:


met/ recorded



50M Staffing

50M.1 -Consultations are done by appropriately qualified staff NURSE

(RGN)/DOCTOR

51M Guidelines/protocols

51M.1 National Malaria guidelines for diagnosis and treatment of

uncomplicated and severe malaria

-On wall, accessible to staff and up to date

51M.2 PEP policy and guidelines:

-Available in OPD

51M.3 Opportunistic Infection and ART guidelines:

-AVAILABLE AND ACCESSILBE in all consultation rooms

51M.4 STI Management protocol:

-Displayed in all consultation rooms and up to date

51M.5 IMNCI guidelines:

-Available and flowcharts displayed in all consultation areas

SUPPLIES

52M Equipment and PEP kit

52M.1 Adult weighing scale and Standard Paediatric Weighing (SALTER)

Scale available and functional , height meter,

52M.2 BMI calculator, glucometers and strips, peak flow meter,

ophthalmoscope, otoscope, stethoscope, otoscope, sphygmomanometer,

HC meter

52M.3 Functional thermometer and OPD register in place

52M.4 PEP kit readily available in the event of a needle stick injury or other

accidents


II.9 FAMILY AND CHILD HEALTH (FCH)

53M Guidelines/protocols, medicines and equipment

134

53M.1 Focused ANC protocol in ANC care area:

-Available, displayed and up to date

53M.2 PMTCT guidelines and charts:

-Available and accessible

53M.3 PMTCT medicine:

-Available according to guidelines?

53M.4 Standard Paediatric Weighing (SALTER) Scale, length/height

board and MUAC tape

-Available and functional

54 M Availability of functional OTP equipped with (outpatient therapeutic center at health facility )

54M.1 OTP register and case sheet

54M.2 RUTF (ready to use therapeutic food) (adequate for at least three

months: based on previous records/admissions/utilization)

54M.3 Trained health worker on IMAM (Integrated Management of Acute

Malnutrition)

Total points this quarter: (Maximum Available Points:7)

II.10. EXPANDED PROGRAM ON IMMUNIZATION (EPI)

Indicato

rs




* N.B. Please assess all the indicators requiring opening of the

refrigerator at once in order to avoid frequent opening of the

refrigerator

Score:


met/ recorded



55M POLICY & GUIDELINES

55M.1 -Surveillance line listing and case definitions displayed

55M.2 -Updated EPI schedule, and a contingency plan displayed

55M.3 -EPI graphs showing trends displayed and staff member is able to

interpret the graphs

55M.4 -EPI reference materials: EPI Policy, (e.g. multi dose vial policy

(MDVP) and EPI modules available and easily accessible

SUPPLIES & STORAGE

56M Cold Chain Mechanism:

56M.1 -Fridge with a temperature booklet available and filled twice a day

56M.2 -The temperature is within the recommended range of + 2 and+ 8

degrees Celsius (Supervisor should verify functionality of thermometer)

57M Availability of vaccines:

57M.1 -The following antigens are available: BCG, MR (measles and Rubella),

polio, Penta, tetanus, pneumococcal and rota virus vaccine

57M.2 -The physical stock and the amount in the stock cards match (Supervisor

verifies physical stock in the fridge by selecting three different vaccines

quarterly)

58M Vaccines storage

58M.1 -Correctly stored in fridge with compartments as follows in fridges with

compartments:

-Freezing compartment: ice packs well frozen

-None freezing compartment: top shelf BCG, OPV, measles

135

-Lower shelf: DPT+HEPB, TT, etc

N.B. the new type of refrigerator i.e. Domestic fridge do not have

compartments and the live vaccines are stored in the lower tray (colder

zone)

58M.2 -No expired vaccines

58M.3 -The Vaccine Vial Monitor (VVM )status is kept

58M.4 -There are readable labels on vials with matching diluents

59M Syringes:

59M.1 -The number of syringes available matches the number of vaccines in

the stock cards

60M Sharps boxes:

60M.1 -Sharps boxes available in immunisation room/corner/area and not more

than 3/4 full)

61M EPI accessories: the following EPT accessories should be available and functional

61M.1 -Vaccine carriers, cold box, gas regulator, gas cylinder and scissors

62M Forms:

62M.1 - AEFI investigation forms, case investigation forms for EPI targeted

diseases and vaccine wastage monitoring forms available

62M.2 -Vaccine order forms and stock cards available


II.11 MATERNITY Paediatric Services ; LABOUR, DELIVERY

POST-NATAL CARE FOR MOTHER AND NEWBORS and

Children

Indicators

*Please give a score for each of the criteria under each

indicator as per the criteria in the right column


Score:

1: if all criterion

have been met/

recorded

0: if all criterion

have not been met/

not recorded

63M Medicines on Emergency tray:

Are the following medicines available on the emergency tray and not expired?

63M.1 -IV fluids (ringer lactate, 5% dextrose, normal saline) and giving

sets

63M.2 -Adrenaline, lignocaine, diazepam, oxytocin, ergometrine,

Magnesium Sulphate, calcium gluconate

63M.3 -Cannula, syringes and needles, drip stand, swabs, strapping,

disinfectant

63M.4 -At least 10 pairs of sterile gloves available, face mask, specimen

bottles

64M PPH kit (please open one kit and check for its completeness)

64M.1 -PPH kits available and complete:

please refer to the annex section in the checklists guideline for list

of items that should be available in the PPH kit

*refer annex section in the checklist guideline for the list of items

in PPH kit

136

65M Eclampsia kit (please open one kit and check for its completeness)

65M.1 Eclampsia kit available and complete:

please refer to the annex section in the checklists guideline for list

of items that should be available in the eclampsia kit

*refer annex section in the checklist guideline for the list of items

in Eclampsia kit

66M Equipment/supplies for care of newborn and monitoring FHB: Are the following

equipment available?

66M.1 -Fetoscope, baby blanket, Baby scale and tape measure

66M.2 -Sterile cord clamps/ties for umbilical cord, Eye ointment (

Tetracycline)

66M.3 -Neonatal bag and mask, penguin suction, resuscitator and suction

bulb (at least two sets) in a “ready newborn resuscitation” area

next to delivery bed

66M.4 Electric heater, and wall clock:

67M Obstetric sterilised delivery packs: ( open one pack to see whether all the items are

present and check for expiry date )

67M.1 -At least 5 obstetric sterilized delivery standard packs with -2

wrapping towels, 6 drapes, A galipot with 10 swabs, 5 gauze

swabs, A receiver, 2 Artery Forceps, Cord Scissor, Episiotomy

Scissor, Drying towel for hands, Gown, Cord ties, sanitary pads

available

68M Delivery bed:

68M.1 -All beds in the maternity ward/delivery room are in good state

(not broken, mattress not torn) and covered with a clean sheet

69M Availability of functional equipped KMC ( Kangaroo Mother Care) unit

69M.1 KMC bed

69M.2 Heat source

69M.3 KMC wrap for baby (mbereko)

69M.4 Clothes for baby ( hat, nappy and socks)

69M.5 KMC register


II12 HEALTH INFORMATION MANAGEMENT SYSTEM

Indicator

s




Score:

1: if all criterion have been met/

recorded

0: if all criterion have not been

met/ not recorded

N/A: Not applicable

70M Referral and feedback system:

* review referral made in any one month in the last quarter ( if there was no referral made in the selected

month, extend the period of assessment to any of the two months in the last quarter)

70M.1 -Standard referral forms (at least 10) available

70M.2 - Referral register available and properly filled ( Applicable only

if there was referral in the last quarter)

71M T Series forms and timely reporting :

137

* check the following two items in in any one month in the last quarter

71M.1 -The T Series forms are available and fully completed (T1, T3, T5,

T6, T11, and T12)

71M.2 -The T5 and HS3/5completed and sent timely (by the 21st of the

following month) for previous months

For the following two indicators requiring review of registers and/or reported figures/indicators:

Score each register/reported figure as:

1: if all criterion that have been met/ recorded

0: if the criteria has not been met/ not recorded

And then give an overall score as shown below:

5 Points: if 5 (100%) of registers/reported figures are complete and/or correct

3 Points: If 3-4( 60-80%) of registers are complete and/or correct

0 Point: if ≤2 (≤40%) of registers are complete and/or correct

72M Completeness and correctness of information in registers :

*Randomly select 5 registers to assess the indicators listed below

*Please review the annex section of the checklist guideline for the list of registers available in a hospital

setting

72M.1 Are the information in each column of the

selected registers complete and correct in any one

month in the last quarter (Select different registers

quarterly)?

R1 R2 R3 R4 R5 Complete

registers

Overall

score

73M Correctness of reported figures:

73M.1 Are the figures reported in any one month of the

last quarter correct according to the HMIS age

groups in the T5 and HS3/5?

* Randomly select five indicators, verify for

accuracy and correctness

(selected different indicators quarterly)

i1 i2 i3 i4 i5 Accurate

and

correct

figures

Overall

score

Total points this quarter: (Maximum Available

Points: 14)

138

CLINICAL MANAGEMENT PRIORITY AREAS

1C OPD/CONSULTATION AREA

For indicator 1C1: assess by reviewing 5 files of

patient who visited the clinic during the day of

assessment. If there are no enough records, Score each

file as 1or 0 as per the criterion and then give an over

score/points for the indicator. If there are no records

for review, please assess the indicator by asking the

nurse at OPD who is doing the triaging during the day

of assessment (( if she/he is able to identify the three

group, please give 100% ( 6 points; if not, please give

0 points)

For indicator 1C2: assess by asking at one of the

nurses on duty during the day of assessment. Score

each criterion as 1 or 0 as per the response of the

health care provider and then give an over score/point

for the indicator.

For indicator 1C3: assess by reviewing OPD register

and/or TB presumptive register

PATIENT

’S

RECORD

S

Or TB

Symptom

screening

1: if all

criterion

have been

met/

recorded

0: if all

criterion

have not

been met/

not

recorded N/A: not

applicable

if there is

no record

for review

5 records /symptoms

(100%): 6 points

4 records /symptoms

(80%): 4 points

3 records /symptoms

(60%): 2 points

≤2 records /symptom

(≤40%): 0 points

1 2 3 4 5 Complet

e records

POINTS

1C1 Triaging of patients at OPD waiting area during all

clinic shift:

-Patients are classified into three groups and given

due attention accordingly:

Assess by reviewing patient files if patients are

available at OPD during the day of assessment. If not,

assess by asking the nurse on how she/he conducts

triaging of patients (her/his answers should match with

the points listed below)

Emergency signs requiring immediate attention

Priority signs (requiring priority in the queue)

Non-urgent cases

1C2 % of adult TB screening symptoms correctly

stated by health center/clinic OPD provider (Select

one provider randomly on shift during the day of

assessment)

• Assessment: Ask one health care provider at OPD to

name criteria for TB testing(please observe whether

139

her/his answers match with the list mentioned below

and give score accordingly):

1) Weight loss

2) fever for more than 3 weeks

3) cough for more than 14 days

4) Night sweats

5) TB contact exposure.

1C3 % of TB presumptive (TB symptom positive) that

have sputum results documented in any month in

the last quarter

AMBULATORY MANAGEMENT OF TB

*Source of data: TB Register

*Review TB register and select 5 cases with TB in the

any one month in the last quarter. If more than 5 cases

found, randomly select 5 cases for each disorder. If the

number of cases is not enough extend the search to the

last quarter to gather 5 cases for each disorder. If

there were less than 5 cases in the last quarter assess

the cases found

*Write Not Applicable (N/A) if there are no TB cases

for review

* Indicators 1C4-7 should be assessed at TB clinic

and/or OI/ART clinic

1C4 % of TB patients (SS + and SS-) that have HIV test

results documented in any month in the last

quarter

1C5 % TB patients diagnosed in any one month in the

last quarter that are receiving correct treatment

with DOTS,


last quarter that were TB notified


last quarter that TB contact tracing was conducted

SUBTOTAL Points – ( Maximum available

points:42 )

2C FAMILY AND CHILD HEALTH (FCH)

AMBULATORY (ANC, PNC) BEST

PRACTICES

Source of Data:

*ANC register for ANC Best Practise indicators

* PNC register/follow up notes for

PNC/Postpartum best practise indicators

*Assess 10 cases/records in any one month in the

last quarter.

*If there are no enough cases for review, please

extend the review period to a quarter.

PATIENT’S

RECORDS/Registers


met/ recorded):


met/ not recorded

N/A: not applicable: if there

are no records for review

*Score each case/record as 1

or 0 and then give a score for

9-10

records (≥90%):

6

points

8

records (80%):

4

points

7

records (70%):

2

points

140

*If there are less than 10 cases for review after

extending the review period to a quarter, assess

the available cases/records.

* If there are more than 10 cases for review,

select 10 cases by using either simple/systematic

random sampling

*For indicators 2C1.6 i.e. pregnancy test,

resampling or extending the review period to a

quarter may be required as the number of women

with ≤16 weeks of gestations is usually low.

*If there are no records for review/the indicator

is not applicable in the set up being assessed,

please do not assess and not score the indicator;

rather write N/A and deduct the available points

for the indicator from the maximum available

points.

items per patient record, when

applicable.

* At last, please write the

number of records with

complete information as

required and give an over

score/points as the per the

criteria in the right column

≤6

records (≤60%):

0

points

2C1 ANC BEST PRACTICES 1 2 3 4 5 6 7 8 9 1

0

Complete

records

2C1.1 % of first visit ANC bookings in any one month in last quarter who had

documented:

BP

Height

Weight measurements

Fundal height measurements (if pregnancy >16

weeks of gestation)

ALL ITEMS PER PATIENT RECORD

2C1.2 % of first visit ANC bookings in any one month in last quarter who received the

standard laboratory test according to the ANC guideline:

Blood group and RH

HIV test

Haemoglobin

RPR (Rapid plasma regain for syphilis diagnosis)

Urine analysis


2C1.3 % of first ANC visits in any one month in last

quarter who received IPTp (if pregnancy >16

weeks of gestation if women living in malaria

area)

* Write Not-Applicable (N/A) if it is not a

malaria endemic area


quarter who received TT vaccine


quarter who received iron supplementation

2C1.6 % of first visit ANC bookings with ≤ 16 weeks

of gestation in any one month in last quarter

who had documented pregnancy test results

2C2 POSTNATAL AND/OR POSTPARTUM BEST

PRACTICES

1 2 3 4 5 6 7 8 9 1

0

Complete

records

141

*Source of data: PNC register and/or post-partum

follow up notes/sheet/register

Indicator 2C2.4 needs a separate sampling since

the denominator is women who are 6 weeks post-

partum during the assessment period.

2C2.1 % PNC visits in any one month in last quarter

documenting assessment for the following

conditions of the infant:

General condition of the infant;

NAD recorded if abnormality was not detected



documenting assessment for the following

conditions of the mother:

General condition, Pulse rate, B/P and

temperature




documenting infant feeding (BF) status

(exclusive, mixed or not BF)

2C2.4 % women post-partum counselled and offered

any of the modern FP method (below)at follow

up PNC visit within 6 weeks of delivery in any

one month within the last quarter( this

indicator should be assessed at 6 weeks post-

partum visit)

POP (progesterone-only contraceptive safe

with BF)

injectable,

Implant

IUCD

Tubal ligation

Decline

SUBTOTAL: ( Maximum available points: 60)

3C MATERNITY WAITING HOME

Follow up of pregnant mothers in maternity waiting

home

*Write Non-Applicable (N/A) in clinics without

maternity waiting homes and/or if there are no mothers

in maternity waiting homes during the assessment

period and deduct the available points for the indicator

from the maximum available points.

PATIENT’S

RECORDS

1: All criterion

have been

met/

recorded):

0: if all

criterion have

not been met/

recorded

5 records (100%):

6 points

4 records (80%) :

4 points

3 records (60%) :

2 points

≤2 records

(≤40%):

0 points

142

*If there are less than 5 mothers in the maternity

waiting home,, assess indicator with mothers available

during the day of assessment

N/A: if there

are no mothers

in the

maternity

waiting home ANC Best Practices: follow up of pregnant mothers

in maternity waiting home

1 2 3 4 5 Complet

e record

POIN

TS

3C1 % of mothers in maternity waiting homes monitored

for BP, FHR, and assessed for danger signs daily

*Source of data: ANC cards of pregnant mothers

*Danger signs: vaginal bleeding, headache/blurred

vision, fetal movement, sudden release of water from

vagina

* if there was no danger signs, it should be recorded as

no danger signs

SUBTOTAL; ( Maximum available points: 6)

4C HIV–PMTCT

Source of data: ANC, ART, Delivery and DNA PCR register

*Review ANC register and select 5 newly identified HIV women for

indicator 4C1.

*Review ART/ANC register to identify pregnant women initiated on

ART before 6 months or indicator 4C2

*Review delivery register and select 5 HIV exposed new-borns for

indicators 4C2-4C4 in the last quarter.

*If more than 5 cases found, select 5 cases using simple/systematic

random sampling for each condition. If less than 5 cases in the last

quarter, assess all the cases found.

*N/A (Not Applicable) if there are no HIV+/HIV exposed cases for

review and deduct the available points for the indicator from the


PATIENT’S

RECORDS

1: if all criterion

have been met/

recorded):

0: if all criterion

have not been

met/ not recorded N/A: not

applicable: if there

are no records for

review

5

records

(100%):

6 points

4 records

(80%) :

4 points

3 records

(60%)

2 points

≤2 records

(≤40%):

0 points

1 2 3 4 5

Complete

records

4C1 % NEWLY IDENTIFIED HIV + pregnant women initiated on

ART in MNCH (ANC) ON THE SAME DAY in the last quarter

*Source: ANC and ART register

4C2 % of HIV+ women retained on ART 6 months after initiation in

ANC in the last quarter

*Source: ART register

4C3 % of infants born to HIV+ women who had a DNA PCR sample

within 6-8 weeks of birth in the last quarter

*source of data: delivery register, PNC register and DNA PCR register

4C4 % of HIV exposed infants who had A DNA PCR SAMPLE

COLLECTED within 6-8 weeks of age and received results within

one month in last quarter

*Source of data: DNA PCR register

143

4C5 % of confirmed HIV positive infants initiated on ART in last

quarter WITHIN 21 DAYS OF RECEIPT OF RESULTS

*Source of data: DNA PCR register and ART register


5C AMBULATORY MANAGEMENT OF DIARRHEA,

PNEUMONIA, MALARIA and Severe Acute Malnutrition in

CHILDREN

*Source of data: OPD/ IMCI register/CMAM register

*Review OPD/ IMCI register and select 5 cases with pneumonia, 5

cases with diarrhea and 5 cases with malaria in the last month. If

more than 5 cases found, randomly select 5 cases for each

disorder. If the number of cases is not enough extend the search to

a quarter to gather 5 cases for each disorder. If there were less

than 5 cases in the last quarter assess the cases found

*Not Applicable (N/A) if there are no cases for review

PATIENT’S

RECORDS

YES (all

criterion that

have been met/

recorded):

1

No (if the criteria

has not been

met/ not

recorded) :

0 NA (not

applicable):

N/A

5 records

(100%):

6 points

4 records

(80%):

4 points

3 records

(60%):

2 points

≤2 records

(≤40%):

0 points

1 2 3 4 5 Compl

ete

record

s

POINTS

5C1 % children treated as outpatient for pneumonia in any one of

month in last quarter who were correctly assessed

*Source of data: OPD/ IMNCI register

Absence of general danger signs recorded: able to drink/feed,

vomiting, consciousness

Duration of fever and cough/difficult breathing and child’s age

recorded

Respiratory rate, and presence/absence of chest in drawing, stridor

and wheezing recorded

Classified Correctly


5C2 % children correctly treated as an outpatient for (ambulatory)

pneumonia in any one of month of the last quarter among

those correctly assessed

Treatment: Oral Amoxicillin 50mg/kg divided thrice per day x 5

days; caretaker counselling and follow up or admitted into hospital

5C3 % children with diarrhoea correctly assessed for signs of

dehydration), persistent diarrhoea and dysentery in any one of

month of the last quarter

Assessment of dehydration: Using IMNCI guidelines (Integrated

Management of Neonatal and Childhood Illnesses) IMNCI Flow

diagram available and applied,

Duration of diarrhoea and presence of blood recorded

144

General condition of the child recorded: lethargy, consciousness

and/or restless or irritability

Presence of sunken eyes, drinking status ( thirsty/drinking eagerly

or un able to drink/drink poorly) and skin pinch

Classified correctly


5C4 % Children correctly treated as an outpatient (ambulatory) for

diarrhoea in any one month of the last quarter among those

correctly assessed

Treatment : ORS, Zinc supplements and continued feeding and

advise when to return

5C5 % children diagnosed with malaria that have RDT + or

laboratory confirmation in any one month of the last quarter

5C6 % Children with uncomplicated malaria correctly treated

according to national guidelines in any one month of the last

quarter

Treatment: ARTEMETHER (20mg)-LUMEFANTRINE

(120mg)(C0ARTEMETHER) during 3 days (See treatment protocol

in appendix 2 of the checklist guideline)

5C7 % Children with severe malaria correctly treated according to

national guidelines in any one month in the last quarter

Treatment: PARENTERAL ARTESUNATE IS THE MEDICINE OF

CHOICE at a Dose of 2.4mg/kg body weight for 7 days(See

treatment protocol in appendix 2 of the checklist guideline)

5C8 % of 6-59 months old children with un complicated severe

acute malnutrition (SAM) who were managed as per the

national protocol in any month in the last quarter

A 6 to 59 months old child with any one of the following criteria is

classified as SAM :

Weight for height <-3SD (WHO)

MUAC <115mm

MUAC <125mm and HIV positive

Bilateral pitting oedema

Out Patient management of SAM:

RUTF

Routine Medicine

Health and nutrition counseling and continued follow up

*see annex section of checklist guideline for treatment details

SUBTOTAL: (Maximum available points: 48)

DELIVERY BEST PRACTICES

*Source of data: delivery register and partographs

*Review delivery register and randomly select 10

deliveries in any month in last quarter. If the number

of deliveries is not enough extend the search to the

last quarter to gather 10 deliveries. If there were less

9-10 records (≥90%):

6 points

8 records (80%):

4 points

7 records (70%):

2 points

6C

MATERNITY SERVICES; LABOUR, DELIVERY POST-NATAL CARE FOR MOTHER AND

NEWBORN

145

than 10 deliveries in the last quarter assess the cases

found. If there were more than 10 cases in a

month/quarter then randomly select 10 deliveries and

assess the partographs for the deliveries selected to

assess the following indicators

* Write Not Applicable (N/A) if there are no cases for

review and deduct the available points for the

indicator from the maximum available points.

≤6 records (≤60%):

0 points

1 2 3 4 5 6 7 8 9 10 Complet

e records

POIN

TS

6C1

% deliveries performed by skilled personnel in any

one month in the last quarter

•Assessment: Identification of the nurse/ midwife by

names in the delivery register

6C2

% women who delivered in the facility monitored using partographs as

per criteria *Please take into account the cervical dilatation at admission while assessing the

following items

Assumption: Partograph should be opened for all pregnant women in labor unless

they come with fully dilated cervix with head visible. Women who present in

advanced labor should have at least one measurement of the items mentioned below.

If urine analysis was not done, the reason should be noted/documented i.e. lack of

supplies/women did not produce urine.

Fetal heart tones plotted every 30 minutes

State of membranes every 4 hours presence/absence

meconium

Descent of presenting part every 4 hours

Contractions plotted every 30 minutes

Maternal BP every 4 hours

Maternal pulse every 30 minutes

Maternal temperature every 4 hours

Urinalysis documented at admission


6C3

% total births in any one month in the last quarter

documenting administration of immediate

postpartum oxytocin 10 units IM (within one

minute of delivery of baby) (AMSTL: Active

management of third stage of labour)

6C4

% births with placental status documented at birth

in any one month in the last quarter

•Assessment: complete or ragged, retained placenta

6C5

% newborns BF within one hour of birth in any


•Assessment: Time of BF initiation documented

6C6 % newborns received Vitamin K in the any one

month in the last quarter

6C7 % newborns received eye care (Tetracycline) in

the any one month in the last quarter

6C8 % newborns received first vaccination (BCG) in

the any one month in the last quarter

146

*source of data: Delivery and PNC register

6C9

% women delivered monitored in early post-partum period (4th stage) per

guideline (birth to discharge) in the any one month in the last quarter

*Source of data: partographs and/or post-partum follow up notes/sheets/register

Vaginal bleeding, at least every 30 minutes 1st 2 hrs

after birth and then four hourly until discharge

Uterine contraction at least every 30 minutes 1st 2 hrs

after birth and then four hourly until discharge

BP at least every 30 minutes 1st 2 hrs after birth and

then four hourly until discharge

Pulse at least every 30 minutes 1st 2 hrs after birth

and then four hourly until discharge

Temperature at least every 30 minutes 1st 2 hrs after

birth and then four hourly until discharge


6C10

% newborns monitored in early post-partum period per guideline (birth to

discharge) in the any one month in the last quarter

*source of data: partographs and/or post-partum follow up notes/sheets/register

Temperature documented at least every 30 minute

first 2 hours after birth then four hourly until

discharge

Respiratory Rate documented at least every 30 minute


discharge

Breast feeding status documented at least every 30

minute first 2 hours after birth then four hourly until

discharge

Colour documented at least every 30 minute first 2

hours after birth then four hourly until discharge


6C11

% facility births seen for day 3 PNC visit in any


*Source of data: PNC register


7C OBSTETRIC, NEONATAL and Childhood

COMPLICATIONS

*Source of data: in patient and/or delivery registers.

*Review in patient and/or delivery register in maternity ward

and randomly select 5 cases of patients with the following

conditions in the last quarter:

PROM; PPH; Postpartum sepsis; and severe Pre-

eclampsia/eclampsia,

PATIENT’S

RECORDS

1: if all

criterion have

been met/

recorded):

0: if all

criterion have

5 records

(100%):

6 points

4 records

(80%) :

4 points

3 records

(60%) :

2 points

≤2 records

(≤40%):

147

*Review in patient and/or delivery register in paediatric and

randomly select 5 cases of patients with the following

conditions in the last quarter:

Neonatal asphyxia; neonatal sepsis; low birth weight and

severe acute malnutrition

*Then review their records and assess whether they were

managed as per the national protocol..

* Write Not Applicable (N/A) if there are no cases for review

and deduct the available points for the indicator from the

maximum available points.

not been met/

not recorded

N/A: not

applicable: if

there are no

records for

review

0 points

1 2 3 4 5

Complet

e records

POINTS

7C1 % women with prolonged labor or Rupture of Membranes

and without chorioamnionitis that were administered

antibiotics as per protocol in the last quarter

Treatment with oral erythromycin (or amoxicillin) if ROM > 6

hours or active labor > 12 hours without signs of chorio-

amnionitis; first dose antibiotic

*Review in patient register and select those in which rupture of

membrane documented > 6 hours (at any time in course of

labour and delivery) or active

labour >12 hours (at any time) without documentation of other

signs of maternal sepsis in the last quarter

(maternal fever or foul-smelling discharge) is documented

7C2 % women with PPH managed per guideline

last quarter

*Review in patient register and select 5 cases of PPH fulfilling the following

criteria:

PPH documented (EBL > 500 cc or VB and tachycardia > 100 bpm or hypotension

SBP < 100 or DBP <50) and check whether they were managed according to the

guideline and check whether the following three items listed below were done for

each identified case

•Assessment: See annex section of checklist guideline for specific audit criteria

and management of PPH

1. PPH Cause documented (atony, tear, retained placenta,

other)

2. secure two IV lines with two 16 G cannulas or any large size

available, and run normal saline (NS) or ringer lactate (RL)

3. Management according to the cause:

-Uterine atony: 60 IU Oxytocin in 1L NS or RL solution at 60

drops/minute until uterus is firmly contracted or

-Retained placenta: controlled cord traction. If failed, manual

removal or

-Vaginal/cervical laceration: sutured


7C3 % women with signs of intra- or post-partum sepsis :fever

temperature ≥38⁰C, foul-smelling discharge, ≥38⁰C) or

148

Membranes were ruptured for ≥18 hours before delivery

that were treated per standard in last quarter

*Treatment with triple antibiotic given IV:

Benzyl penicillin 5 mega units IV stat, then 2.5 mega units IV

hourly,

Metronidazole 500mg IV 8 hourly and

Chloramphenicol 500mg IV 6 hourly until patient is fever-free

for 48 hours

** see annex section of hospital checklist guidelines for

maternal sepsis(chorioamnionitis/puerperal sepsis) case

management


7C4 % pregnant women with severe pre-eclampsia and/or eclampsia managed

according to the guideline in last quarter

Severe Pre-eclampsia:

-Diastolic BP 100mm HG or more

-proteinuria 3+ or more

Eclampsia:

-Unconsciousness or Convulsions (fits)

-dBP 110 mmHg or more

-Proteinuria 2+ or more in a pregnant woman or a woman who has recently given

birth

-Check whether the following three items listed below were at least done for each

identified case:

*refer Annex section of the checklist guideline for details on the management of

severe pre-eclampsia/eclampsia

1- blood pressure checked every 5 minutes until diastolic BP is

90 - 100mmHg

2-Blood pressure monitored (if diastolic blood pressure (dBP) is

≥110 mmHg, Nifedipine 10 mg provided. If inadequate response

after 20 minutes following first dose:

Repeat 10mg dose orally every 20 to 30 minutes until adequate

dBP response is achieved, to a maximum of 40 mg given. Then

10-20 mg orally every 4-6 hours to maintain dBP 90-100 mmHg

or

Hydralazine 5mg IV slowly every 20 minutes for a maximum of

20 mg

* applicable only if dBP was ≥110mm Hg

3-Magnesium sulphate 20% solution, 4gm IV over 5 minutes

given. Followed promptly with 10g of 50% magnesium sulphate

solution, 5gm in each buttock as deep IM injection with 1 ml of

2% lignocaine in the same syringe.

4. Deliver within 12hours for Severe Pre-Eclampsia and 6 hours

for Eclampsia


149

7C5 % of neonates who had an APGAR score of ≤5 in the first

minute after birth for whom resuscitation was initiated in

last quarter

*Source of data: partographs and notes

*refer checklist guideline for details

7C6 % neonates who had an APGAR score of ≤5/10 at 1 minute

after delivery and were successfully resuscitated i.e. 5th

minute APGAR score ≥6/10 in last quarter

*refer checklist guideline for details

7C7 % neonates with possible serious bacterial sepsis managed

per standard in last quarter

•Assessment of possible neonatal sepsis: Review all cases of

newborn sepsis (pre-discharge or re-admitted to paediatric

ward) in in patient register in last quarter; and select 5 records

for review that meet any of following probable sepsis criteria:

-if documented temperature >380 C or < 250 C (and not

warming);

- RR > 60 or <30 breaths per minute;

-chest in-drawing or convulsion;

-no movement on stimulation;

- poor feeding/sucking or

-umbilical redness,

*see annex section of checklist guideline criteria for chart audit

and for treatment details

7C8 % of low birth weight (LBW) newborns admitted to KMC

unit in the last quarter

Definition of LBW: infant with birth weight lower than

2500gregardless of gestational age

Criteria for providing Kangaroo Mother Care (KMC):

LBW neonates weighing >1500 and <2500g

Baby‘s condition is stable to permit KMC

The mother is in good health to start KMC

SUBTOTAL: ( Maximum Available Points: 48)

8C PAEDIATRIC WARD

PAEDIATRIC BEST PRACTICES

*Select 5 cases from registers (randomly if there are more

than 5 cases) and then obtain and review patient files

* Write Not Applicable (N/A) if there are no cases for review



PATIENT’S

RECORDS

1: if all criterion

have been met/

recorded):

0: if all criterion

have not been met/

not recorded

N/A: not applicable:

if there are no

records for review

5 records

(100%):

6 points

4 records

(80%) :

4 points

3 records

(60%) :

2 points

≤2 records

(≤40%):

150

0 points

1 2 3 4 5

Complet

e records

POINTS

8C1 % hospitalized patients with correctly completed admission medical record any


Admission medical record for hospitalisations available and documenting at a

minimum:

Vital signs (RR, HR, BP, temperature);

history of illness;

physical exam;

laboratory/radiology results (if applicable);

admission diagnosis and treatment


8C2 % Written record of administration of patient medications

up to date any one month in the last quarter

8C3 % hospitalized patients with documentation of vital signs

every 6 hours and every half hour for critical patients any


8C4 % hospitalized patients with documentation of daily

progress note by doctor any one month in the last quarter.

SUBTOTAL - ( Maximum available points 24 )

9C PAEDIATRIC COMPLICATIONS: PNEUMONIA,

DIARRHEA,

Review 5 cases in registers with the diagnosis of pneumonia,

and 5 with diagnosis of diarrhoea. * increase the assessment

period to a quarter if there are less than 5 cases in the

selected month

* Not Applicable (N/A) if there are no cases and deduct the

available points for the indicator from the maximum points.

PATIENT’S

RECORDS

1: if all criterion

have been met/

recorded):

0: if all criterion

have not been met/

not recorded


if there are no

records for review

5 records

(100%):

6 points

4 records

(80%) :

4 points

3 records

(60%) :

2 points

≤2 records

(≤40%):

0 points

1 2 3 4 5 Complet

e records

POINTS

9C1 % hospitalized children treated for pneumonia any one month in the last quarter

who were correctly assessed for pneumonia in any one month in the last quarter

Vitals: child’s age recorded; weight recorded; Temperature,

respiratory rate

Symptoms & duration recorded (at a minimum

absence/presence and duration of fever, cough, ability to

drink/feed)

Pulmonary exam results recorded stridor, wheezes, chest in-

drawing


151

9C2 % hospitalized children treated correctly (all criteria met)

for pneumonia among those correctly assessed in any one

month in the last quarter

•Assessment: see checklist guideline for criteria

IV ceftriaxone 50 mg/Kg per day OR oral amoxicillin 50

mg/Kg divided TID x 7 days (if taking fluids and no severe

respiratory distress

Antipyretic for fever control

Oxygen (per nasal cannula or paediatric mask) if : saturation <

94% Or breathing (intercostal retractions and/or respiration

rate > 50, if 2months to 1 year; > 40 if 1 yr or older)

9C3 % hospitalized children treated for diarrhoea in any one month in the last quarter

correctly assessed for signs of severe dehydration

Was general condition (abnormally sleepy or difficult to wake

up, restless and irritable, or well and alert) assessed and

documented?

Were eyes checked for dehydration signs and documented

(such as sunken and dry, sunken, normal);

Was thirst assessed by offering fluid (drinks poorly or not able

to drink, drinks eagerly - thirsty, drinks normally not thirsty)?

Was skin turgor assessed by pinch of abdomen or thigh (goes

back very slowly - longer than 2 seconds?


9C4 % hospitalized children treated for diarrhoea correctly (all criteria met) in any

one month in the last quarter among those correctly assessed. See checklist

guideline for criteria

If able to drink: Low osmolarity Oral Rehydration Solution

(continue breastfeeding and feeding)

If unable to drink: NS (or Ringer Lactate if NS not available)

IV Or If unable to drink and unable to star IVF, administer

ORS via NGT

Zinc 10-20 mg/kg/day x 10 days given


SUBTOTAL - ( Maximum available points 24 )

152

10 C POST OPERATIVE INFECTION CONTROL

Review operating Theatre register and randomly select 5

patients who have undergone major surgical procedures in

the last quarter and review their files to assess

indicators10C1-2

*If there are less than 5 cases, assess the indicators for the

available number of cases. But if there are more than 5 cases

for review in the last quarter, select 5 cases for review using

either simple random/systematic random sampling

** Write Not Applicable (N/A) if there are no cases for

review and deduct the available points for the indicator from

the maximum available points.

*See the WHO surgical safety checklist and the definitions of

surgical site infections in annex section of the checklist

guideline

PATIENT’S

RECORDS

1: if all criterion

have been met/

recorded):

0: if all criterion

have not been met/

not recorded


if there are no

records for review

5 records

(100%):

6 points

4 records

(80%) :

4 points

3 records

(60%) :

2 points

≤2 records

(≤40%):

0 points

1 2 3 4 5 Complete records POIN

TS

10C1 Surgical safety checklist utilization rate:

-% of patients with major surgical procedures on whom

safe surgical checklist was completed in the last quarter

10C2 Surgical site infection:

-% of post major surgical procedures free of surgical site

infections in the last quarter


QUALITY PROCESS MANAGEMENT

11C Quality Improvement/Assurance:

*Please give a score for each of the criteria under each indicator as per the

criteria in the right column *N.B. The items highlighted in bold are the indicators

* Assess asking the QI focal person and/or QIC chairperson

Score:

1: if all criterion have

been met/ recorded


not been met/

not recorded

11C1 Presence of a Quality Improvement/Assurance Committee/team (QIC) with clear structure and

responsibilities:

-Does the QIC have terms of reference (ToR) with the following components:

Structure & leadership, known responsibilities, meeting frequency, reporting system

and list of committee members?

-Are all service areas, including administration, represented in the committee?

11C2 Presence of quality improvement plan:

-Is a quality improvement (QI) plan with the following items present, as part of the overall plan of the hospital?

-Action plan for improvement

-Targets

-Areas for improvement

11C3 Quarterly committee meetings and reports to PMDs:

-Are reports on QI activities sent to PMDs quarterly? (Check copy of report)?

- Are processes like waiting time, appointment system, and patient flow discussed

during the meeting? (Check in the minute)

-Is the QIC meeting quarterly? (Check for the presence of minutes)

153

- Are status of QI plan and other improvement plans discussed during the meeting?

(Check in the minute)

11C4 Quarterly review of patient files:

* Check in the minute and/report of QIC meetings.

Does the QIC review at least 10 patient files from at least each of three service areas and assess the following

items? :

-Completeness and correctness of the information in the reviewed patient files

-Comprehensive patient management given to patients in the reviewed files

-Consistency of information between the reviewed patient files and registers

11C5 Feedback and action on quality supervision checklist assessment findings:

-Does the QIC receive feedback from the supervision?

(Check for the presence of copy of feedback of previous quarter supervision)

-Has the QIC developed action plan for improvement to address the identified gaps

from the quality supervision? ( Check for the presence of action plan for

improvement)

11C6 Feedback mechanism from Clients

Quarterly client satisfaction surveys:

-Does the QIC have survey tool? ( Check for the presence of the survey tool)

-Does the QIC conduct quarterly surveys? ( check for copies of filled questionnaires

)

-Are survey analysis reports available?( check for the presence of analysis report)

Suggestion box:

-Does the facility have labelled suggestion boxes?

-Is instruction on how to use the boxes posted on or above the boxes?

- Does the facility analyse the findings?

(Check for the presence of analysis report of previous quarter)

11C7 Clinical audit and/or maternal-perinatal mortality audit meetings:

-Does the hospital conduct audit meetings or maternal mortality review meetings at

least once in a quarter? ( check for minutes/report)

-Is guideline for audit/mortality meetings available?( check for the presence of

guidelines for clinical audit meetings)

Total Points- ( Maximum available points: 22)

VERIFY THAT ALL QUESTIONS ARE FILLED IN

Supervisor thanks the staff

Signature:

PHE ………………………………………………………..

DMO/MED. Sup/MATRON…………………….........................

Counter verification………………………………………...

ASSESTMENT FEEDBACK

154

I. Summary Comments on Results. Please note any trends, problems, exceptional or creative

changes and results that you saw during your visit assessment

II. Noteworthy Improvement. Please note any improvement and include a few details of what they

are doing and why it is unique

III. Difficulties/ Challenges. Please note any assessment area that seem to be having an especially

difficult time in improving. Please include a few details about the problem, how it might be solved, and

who might be involved

IV. Recommendations and suggestions for improvement. Please note that the feedback is more

effective when emphasizes features of the clinical task to be performed (e.g. specifies a target

performance, presents information on how target performance can be attained, and address change in

performance observed since previous feedback

V. Follow up. Please review previous recommendations provided and assess if they were followed or

not

155

HEALTH CENTER/CLINIC

QUALITY SUPERVISION CHECKLIST

February 2016

156

HEALTH CENTER/CLINIC QUALITY SUPERVISION CHECKLIST

Province:

______________________________________________________________________________

District: ______________________________________________________________________________

Health facility:

___________________________________________________________________________

Number of beds: ________ Catchment area population:

__________________________________________

Date of supervision:

______________________________________________________________________

Name of supervisors and designation

No. Name of supervisor Designation

1

2

3

4

5

6

Facility Staffing

STAFF Establishment In

post

Vacant Duration of vacancy

Number of Months Number of

Years

RGN

RGN with midwifery

PCN

EHT

Nurse aides

General hands

Other (Non-medical staff

or unqualified staff)

157

ASSESSMENT SUMMARY

I. STRUCTURAL

SECTION

Available

Points

Number

of

composite

indicators

Applicable

valid points

this quarter

Number of

applicable

composite

indicators this

quarter

Total

points

scored

Structural indicators in

general compound of

the clinic

20 11

Structure indicators in

OPD, Labour

ward/Maternity and

Inpatient/Observation

departments

29 8

TOTAL 49 19

II, MANAGEMENT

& PLANNING

SECTION

Available

Points

Number

of

composite

indicators

Applicable

valid points

this quarter

Number of

applicable

composite

indicators this

quarter

Total

points

scored

Administration, finance

and planning

14 6

Community services 5 3

Environmental health

services

9 4

Infection control and

waste management

10 5

Pharmacy 32 4

Outpatient department

(OPD)

12 4

Extended Program

Immunization (EPI)

17 8

Maternity ward 11 7

Observation/in patients

services

1 1

Health Information

Management System

14 4

TOTAL 125 46

III. CLINICAL

MANAGEMENT

SECTION

Available

Points

Number

of

composite

indicators

Applicable

valid points

this quarter

Number of

applicable

composite

indicators this

quarter

Total

points

scored

1C.OPD/ consultation

area 18

3

2C. ANC-PNC Best

practices 60

10

158

3C. Maternity waiting

home 6

1

4C. HIV-PMTCT 24 4

5C. Ambulatory

management diarrhoea,

pneumonia

48

8

6C. Delivery best

practices 66

11

7C. Management

obstetric complications 42

7

TOTAL 264 44

159

STRUCTURAL SECTION

Indicators

I.1. Structural indicators in the general

compound of the clinic

*Please give a score for each of the criteria under

each indicator as per the criteria in the right column

*N.B. The items highlighted in bold are the

indicators

Score:


recorded

0: if all criterion have not been met/

not recorded

N/A: not applicable

1S Outside appearance (when arriving at the clinic ):

1S.1 -Visible sign post

1S.2 -Fence/wall: in good condition and gate with clearly written

emergency contact number/s and service hours

2S Maintenance of the ground:

2S.1 -Ground clean with no litters and/or stagnant water and the

grass cut

2S.2 -No waste and dangerous objects in courtyard such as needles

– syringes –gloves – used cotton wool, etc

3S Outside appearance of buildings:

3S.1 -External appropriate wall finishing (painting/bricks/rough

plastering)

3S.2 -Roof intact, presence of well-maintained rain gutters

4S Availability of a garbage bin in ground:

4S.1 -Bin with lid accessible to clients and not more than ¾ full

5S Presence of sufficient and clean latrines/toilets:

5S.1 -Minimum of 3 toilets: 1 male, 1 female, 1 staff offering

privacy

5S.2 -Recently clean without visible fecal material and without

smell

5S.3 -Hand washing facility with soap available near the toilets

6S Lighting system

6S1 -Electricity for 24 hours a day, and 7 days a week:

Source of electricity: ZESA with backup system of either

generator or solar energy and/or inventors.

7S Firefighting System:

7S.1 -Fire extinguishers available, accessible, functional and

serviced

7S.2 -A clearly marked firefighting assembly point and procedure

( well understood by a staff member): Ask staff member on

firefighting procedure

160

8S Waste Management system: Fenced and lockable disposal area

N.B. Ottoway pit and rubbish pit are not applicable for facilities where the rubbish is collected by

city municipality

8S.1 -Ottoway pit with lid, not full and functional

8S.2 -Incinerator ( lined with bricks) and functional

8S.3 -Rubbish pit: 2-3 metres deep without infected non

decomposable objects (non – biodegradable)

9S Bathing facilities and waste water drainage system

9S.1 Appropriate drainage of waste water (presence of septic tank

or connected to local sewage)

9S.2 Shower with either running water or container of at least 100

litres

10S Referral service-Availability of means of communication:

10S.1 -Radio or mobile phone with airtime or landline for

communication

11S Transport plan for emergency referrals:

11S.1 -Transport plan for emergency referrals and/or contingency

plan (in case of unavailability of ambulances from hospitals)

included in the plan

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS:

20)

Indicators

I.2. STRUCTURAL Indicators In OPD,

Maternity/Labour ward and

Inpatient/Observation Departments

Score


met/recorded

0: if all criteria have not

been met/not recorded

N/A: Not Applicable




OPD

12S Good conditions in waiting area, meeting minimum standards:

12S.1 -With sufficient benches and / or chairs (according to average daily

attendance calculated using attendance over a week)

12S.2 -Adequate ventilation of waiting area:

If open space: with a shade or roof supported by brick or metal

pillars or

If closed space: windows should measure at least 1/10 of floor area

and at least ½ of window area should be openable.

13S Displaying of free service sign and/or service fee charges:

161

MANAGEMENT AND PLANNING: STAFF, POLICY, GUIDELINES, MEDICINES &

SUPPLIES & Vaccines

13S.1 -Free services sign or service fee charges (if any) displayed and

easily visible for patients

13S.2 -Displayed in local vernacular?

OPD Labour

ward

Inpatient

14S Inside appearance of building and its cleanliness

14S.1 -Walls, Floors and Ceiling / roof clean and in good condition (no

leaks, cobwebs) and floors polished ( when applicable)

14S.2 -Doors with locks and closing properly

14S.3 -Curtains on windows or non- transparent glass and screen

14S.4 -Clean and with good ventilation without bad smell?

OPD Labour ward

15S Availability of waste management supplies and their utilization:

15S.1 -Bin with plastic liners + puncture proof sharps containers available

and not more than ¾ full

16S Hand washing facilities:

16S.1 -Hand washing facilities with soap available at accessible points for

outpatient and/or a functional water point and/or at least 50 litres

reserve with soap available in delivery room

Labour ward

17S Delivery bed:-

17S.1 -In good state (not broken, mattress not torn) and covered with a

clean sheet and Macintosh

Labour ward Inpatient

18S Availability and status of furniture:

18S.1 -Beds, mattresses (covered in plastic), bed sheet and bed side lockers

available and in good state

18S.2 -Mosquito nets available and in good state? (In malaria endemic

areas)?

N/A in non- malaria endemic area

19S Access to drinking water and space between beds:

19S.1 -Safe drinking water is accessible

19S.2 -Sufficient space between beds (at least 1m between beds)

\

TOTAL POINTS THIS QUARTER: (MAXIMUM AVAILABLE POINTS: 29)

162

Indicators II.1. ADMINISTRATION, FINANCE AND PLANNING




Score: 1: if all criterion have been

met/recorded

0: if all criteria have not been

met/ not recorded

1M Mission statement, vision, values and patient charter

1M.1 -Displayed in accessible area and easily readable ( should be health

facility specific)

2M Catchment area map, spot map, monitoring graphs, demographic data and list and mapping of

community based workers:

2M.1 -Catchment area maps with current catchment population target

population for services calculated correctly and displayed

2M.2 -Up to date monitoring graphs for different services showing trends

displayed and health care providers know their interpretations ( ask

the health care provider on duty)

2M.3 -Up to date list of community based workers (VHWs, HBC givers)

showing those that are active available and displayed

3M Finance, accounting and procurement

3M.1 -Bank statements, receipts, invoices, etc available and filed in

clearly labelled files

3M.2 -Procurement procedures being followed while purchasing items

(Ask for one purchased item and check whether it was purchased

according to the procedure)

3M.3 -Management book, inventory/asset register, maintenance book

available and up to date

4M Staff duty roster, current practising certificates (nurses and EHT), staff leave calendar and

clock in register:

4M.1 -Staff duty roster, staff leave calendar and clock in register

complete, up to date and displayed on the wall where all staff can

see

4M.2 -Current practising certificates for nurses and EHTs available in the

HR files

5M Documentation of activities/ Operational Plan

5M.1 -Staff minute book/file and HCC meetings available, well filed and

up to date

5M.2 -Quarterly review and annual/operational plan and annual progress

report available and up to date

6M Health education plan/ diary, and reports at the facility

6M.1 -Health education plan/diary with clear topics, target audience (in

which number of audience are clearly stated and disaggregated by

sex) and dates when they will be presented available

6M.2 -Topics related to current health problems

6M.3 -Report/s on health education session delivered available

163


Indicators II.2 COMMUNITY SERVICES




Score:


been met/ recorded


not been met/

not recorded

7M School health programme:

7M.1 -School health programme available: check for the availability of the

plan

7M.2 -Reports and data on activities conducted available

8M VHW, HBC activity reports:

8M.1 -Monthly reports and minutes of meetings available

9M Domiciliary visits to patients:

9M.1 -Reports on visits available in register

9M.2 -Visits categorized according to patients’ condition (e.g. chronically ill,

PNC and ANC)


II.3 ENVIRONMENTAL HEALTH SERVICES

10M Participatory community health activities

10M.1 -Availability of plan

11M Hygiene education activities

11M.2 -Report of any sessions of education conducted

12M Epidemics and disease surveillance:

12M.1 -Contact tracing forms, and disease surveillance protocols available

12M.2 -Spot map showing recent or suspected out breaks with clear markings

displayed

12M.3 -Follow ups and contact tracing (look in registers) on possible or

confirmed outbreaks made: Compare reports with weekly statistics

(RDNS)

13M Inspections for public premises (including clinics and hospitals) and trading places:

13M.1 -Inspections done (Check availability of inspection reports)

14M Water and sanitation activities:

14M.1 -Coverage statistics displayed

164

14M.2 -Community health clubs formed

14M.3 - Were water tests conducted: check for reports


Indicators II 4. INFECTION CONTROL AND WASTE MANAGEMENT Score:


been met/ recorded 0: if all criterion have

not been met/

not recorded

15M Infection control guideline:

15M.1 -Available and staff members know about it: Ask a staff member about

it

16M Sterilisation of instruments:

16M.1 -Functioning steam steriliser

16M.2 -Guidelines for sterilization of instruments available

16M.3 -Sensitive tape available on sterilized packs and cords not used to tie

packs

17M Hygienic and aseptic conditions in wound dressing and injection room:

17M.1 -Bench and foot rest covered with macintosh

17M.2 -Labelled Bins for infected and contaminated objects with lid, plastic

lining and foot pedal available and not more than ¾ full

17M.3 -Puncture proof Sharps boxes well positioned and not more than ¾ full

18M Protective clothing and disinfectant use by cleaners:

18M.1 -Cleaners have appropriate protective clothing (Heavy duty gloves,

Uniforms, Dustcoats, Gumboots, Plastic Apron, Face Mask)

18M.2 - Cleaners know how to appropriately use disinfectants? Soap with

water for general cleaning and 1 part jik to 4 parts for spillages and/or

depending on the concentration of the bleach i.e. blood and mainly

body fluids (after containing and cleaning the spills): Ask the available

cleaners during the day of assessment

19M Appropriate dressing of consulting staff:

19M.1 -Dressed with clean standard uniform (princess liner, Brown lace up

shoes), and functioning nurses watch

165


Indicators II 5. PHARMACY (MEDICINES AND SUNDRIES STOCK

MANAGEMENT) Score:


been met/ recorded

0: if the criterion have

not been met/

not recorded 20M Stock Management:

20M.1 -Monthly physical counts conducted with min, max and emergency

order levels recorded and updated in stock cards

20M.2 -The physical stock level corresponds with that on the stock card:

supply on stock card corresponds with physical count, (use sample of

three medicines)

20M.3 -Staff completes and send MIS forms to district each month (check for

copies remaining at institution)

21M Storage of drugs:

21M.1 -Stored correctly in a locked secured storeroom (e.g burglar bars on

windows and doors)

21M.2 -Clean place, well ventilated with cupboards, labelled shelves, no

incident light

21M.3 -Medicines stored in alphabetical order also observing the First Expiry

First Out rule

22M Expired products:

22M.1 -Expired Medicine Register available

22M.2 -No expired products in stock: supervisor verifies randomly 3

medicines and 2 consumables (check stock cards)

23M VEN Medicines (according to EDLIZ) adequately stocked 3-6 months’ supply

23M.1 Doxycycline capsules 100mg

23M.2 Ciprofloxacin tablets 500mg

23M.3 Metronidazole tablets 200mg Oral

23M.4 Diazepam injection 5mg/ml

23M.5 Benzathine Penicillin injection

23M.6 Benzyl Penicillin

23M.7 Amoxycillin suspension 125mg/5ml ( dispersible tablets)

23M.8 Ferrous sulphate tablets/Folic Acid

166

23M.9 Zinc Sulphate tabs

23M.10 Paracetamol 500mg tablets

23M.11 Paracetamol syrup or dispersible tablets

23M.12 Dispensing envelopes

23M.13 Latex gloves

23M.14 Oxytocin 10IU/ml Injection, 1ml Ampoule.

23M.15 Magnesium Sulphate 500mg/ml Injection, 2ml Ampoule

23M.16 Gentamycin 40mg/ml Injection, 2ml Ampoule

23M.17 Artemether 20mg + Lumefantrine 120mg Tablets, 24's

23M.18 Depo Medroxyprogesterone 150mg/Ml Injection, 1Vial

23M.19 Levonorgestrel + Ethinyl Oestradiol (0.15+0.03)mg, 28Tablets, 100

Cycles

23M.20 Lignocaine

23M.21 PEP kits available and accessible:

Contents of PEP kit: Zidovudine 300 mg + lamivudine 300 mg

+Atazanavir (300 mg)/Ritonavir (100mg)

23M.22 Lockable trolleys available with working lock

23M.23 Adult first line ART:

Preferred: TDF +3TC+EFV, alternate: TDF+3TC+NVP or

ZDV+3TC+EFV/NVP ( could available in Dual or triple FDCs)

23M.24 Paediatric first line ART:

Preferred:AZT+3TC+NVP (3-10 yr old), AZT+3TC+LPV/r ( <3 yrs)


167

Indicators II6. OUTPATIENT DEPARTMENT (OPD) Score:


met/ recorded


been met/ not recorded 24M Consultations:

24M.1 -Consultations being done by appropriately qualified staff: PCN and/or

RGN

25M Guidelines/protocols

25M.1 National Malaria guidelines for diagnosis and treatment of

uncomplicated and severe malaria

- Posted on the wall, accessible to staff and up to date

25M.2 PEP policy and guidelines:

-Available in OPD: posted on the wall and up to date

25M.3 Opportunistic Infection and ART guidelines:

-Available, accessible in all consultation rooms and up to date

25M.4 STI Management protocol:

-Displayed in all consultation rooms and up to date

25M.5 IMNCI guidelines:

-Flowcharts displayed in all consultation areas and up to date

25M.6 Focused ANC protocol:

-Displayed I all consultation rooms an up to date

26M SUPPLIES: Medicines in Emergency tray and Accessories

26M.1 -Emergency tray with all the necessary un expired medicines (as from

EDLIZ) available: adrenaline, lignocaine, diazepam, MgSO4, atropine?

26M.2 -Un expired ringer lactate, 5% dextrose, normal saline available

26M.3 -Important Accessories: cannula, giving sets, syringes and needles, drip

stand, swabs, strapping, disinfectant, gloves, face mask, specimen

bottles available, as part of emergency service

27M Availability of equipment at OPD

27M.1 -Adult Weighing Scale and standard pediatric scale available and

functional: inspect in comparison with a known weight, after weighing

the indicator should come to zero and height meter

27M.2 -BMI calculator, Gluco meters and strips and ophthalmoscope,

stethoscope, sphygmomanometer, thermometer, tape measure,

fetoscope


168

Indicators II 7. EXPANDED PROGRAM ON IMMUNIZATION (EPI);

N.B. Please assess all the indicators requiring opening of the

refrigerator at once in order to avoid frequent opening of the

refrigerator

Score:


met/ recorded


been met/ not recorded 28M POLICY & GUIDELINES

28M.1 -Surveillance line listing and case definitions displayed

28M.2 -Updated EPI schedule, and a contingency plan displayed

28M.3 -EPI graphs showing trends displayed and staff member is able to

interpret the graphs

28M.4 -EPI reference materials: EPI Policy, (e.g. multi dose vial policy

(MDVP) and EPI modules available and easily accessible

29M Cold Chain Mechanism:

29M.1 -Fridge with a temperature booklet available and filled twice a day

29M.2 -The temperature is within the recommended range of + 2 and+ 8

degrees Celsius (Supervisor should verify functionality of

thermometer)

30M Availability of vaccines:

30M.1 -The following antigens are available: BCG, MR ( measles and

Rubella), polio, Penta, tetanus, pneumococcal and rota virus vaccine

30M.2 -The physical stock and the amount in the stock cards match (

Supervisor verifies physical stock in the fridge by selecting three

different vaccines quarterly)

31M Vaccines storage

31M.1 -Correctly stored in fridge with compartments as follows in fridges

with compartments:

-Freezing compartment: ice packs well frozen

-None freezing compartment: top shelf BCG, OPV, measles

-Lower shelf: DPT+HEPB, TT, etc

N.B. the new type of refrigerator i.e. Dometic fridge do not have

compartments and the live vaccines are stored in the lower tray (

colder zone)

31M.2 -No expired vaccines

31M.3 -The Vaccine Vial Monitor (VVM )status is kept

31M.4 -There are readable labels on vials with matching diluents

32M Syringes:

32M.1 -The number of syringes available matches the number of vaccines in

the stock cards

33M Sharps boxes:

33M.1 -Sharps boxes available in immunisation room/corner/area and not

more than 3/4 full)

34M EPI accessories: the following EPT accessories should be available and functional

169

34M.1 -Vaccine carriers, cold box, gas regulator, gas cylinder and scissors.

35M Forms

35M.1 - AEFI investigation forms, case investigation forms for EPI targeted

diseases and vaccine wastage monitoring forms available

35M.2 -Vaccine order forms and stock cards available


Indicato

rs II8. MATERNITY SERVICES; LABOUR, DELIVERY POST-

NATAL CARE FOR MOTHER AND NEWBORN

Score:


met/ recorded


been met/ not recorded 35M Medicines on Emergency tray: the following medicines available on the emergency tray and not expired

(Check expiry date when applicable)

35M.1 -IV fluids (ringer lactate, 5% dextrose, normal saline) and giving sets

35M.2 -Adrenaline, lignocaine, diazepam, oxytocin, ergometrine, Magnesium

Sulphate and calcium gluconate

35M.3 -Cannula, syringes and needles, drip stand, swabs, strapping,

disinfectant

36M PPH kit ( please open one kit and check for its completeness)

-PPH kits available and complete:

*please refer to the annex section in the checklists guideline for list of

items that should be available in the PPH kit

37 M Eclampsia kit ( please open one kit and check for its completeness)

37M.1 Eclampsia kit available and complete:

*please refer to the annex section in the checklists guideline for list of

items that should be available in the eclampsia kit

38M Uterotonic Medicines:

38M.1 -Stored at correct temperature available in delivery room or immediate

vicinity to delivery room: oxytocin requiring refrigeration at 2-8 oC and

the ones that do not require refrigeration below 25 oC

39M Equipment/supplies for care of newborn and monitoring of FHB: Are the following equipment

available?

39M.1 -Fetoscope, baby blanket Baby scale and tape measure

39M.2 -Eye ointment (tetracycline), Vit K Sterile cord clamps/ties for

umbilical cord

39M.3 -Penguin suction, neonatal bag and mask and suction tube in delivery

room

40M Obstetric sterilised delivery packs: ( open one pack to see whether all the items are present and

check for expiry date )

40M.1 -At least 2 obstetric sterilized delivery standard packs with -2 wrapping

towels, 6 drapes, A galipot with 10 swabs, 5 gauze swabs, A receiver, 2

Artery Forceps, Cord Scissor, Episiotomy Scissor, Drying towel for

hands, Gown, Cord ties (2: if no cord clamps), sanitary pads available

170

41M Sterile gloves:

41M.1 -At least 5 pairs of sterile gloves should be available


II9. OBSERVATION/INPATIENT SERVICES

40M In-patient register:

40M.1 -Proper or improvised in patient register available and well maintained:

check whether observations documented, identity and hospital bed days


Indicato

rs

II.10. HEALTH INFORMATION MANAGEMENT SYSTEM Score:


met/ recorded



N/A: Not applicable

41M Referral and feedback system:

review referral made in any one month in the last quarter ( if there was no referral made in the selected

month, extend the period of assessment to any of the two months in the last quarter)

41M.1 -Standard referral forms (at least 10) and register available and

properly filled

41M.2 - A referral feedback documented in the register for every referral

made and/or referral feedback notes available ( Applicable only if

there was referral in the last quarter)

42M T Series forms and timely reporting : check the following two items in in any one month in the last

quarter

42M.1 -The following T Series forms available and fully completed: T1,

T3, T5, T6, and T12

42M.2 -T5 completed and sent timely (by the 7th of the following month)

for previous month/s

For the following two indicators requiring review of registers and/or reported figures/indicators:

Score each register/reported figure as:

1: if all criterion that have been met/ recorded 0: if the criteria has not been met/ not recorded

And then give an overall score as shown below:

5 Points: if 5 (100%) of registers/reported figures are complete and/or correct

3 Points: If 3-4( 60-80%) of registers/reported figures are complete and/or correct

0 Point: if ≤2 (≤40%) of registers/reported figures are complete and/or correct

43M Completeness and correctness of information in registers :

Randomly select 5 registers to assess the indicators below

*Please review the annex section of the checklist guideline for the list of registers available in a

Health Center setting

43M.1 -The information in each column of the selected registers is

complete and correct in any one month in the last quarter: select

different registers quarterly

R

1

R

2

R

3

R

4

R

5

Complete

registers

Ove

rall

scor

e

171

44M Correctness of reported figures:

44M.1 Are the figures reported for the last month of the last quarter

correct according to the HMIS age groups in the T5?

* Randomly select five indicators , verify for accuracy and

correctness

(selected different indicators quarterly)

i1 i2 i3 i4 i5 Accurate

and correct

figures

O

ve

ral

l

sc

or

e


172

CLINICAL MANAGEMENT PRIORITY AREAS

1C OPD/CONSULTATION AREA

AMBULATORY MANAGEMENT :

For indicator 1C1: assess by reviewing 5 files of patient who

visited the clinic during the day of assessment. If there are no

enough records, Score each file as 1or 0 as per the criterion

and then give an over score/points for the indicator. If there

are no records for review, please assess the indicator by

asking the nurse at OPD ( if she/he is able to identify the

three group, please give 100% ( 6 points; if not, please give 0

points)

For indicator 1C2: assess by asking one PCN/RGN on duty

during the day of assessment. Score each criterion as 1 or 0

as per the response of the health care provider and then give

an over score/point for the indicator.

PATIENT’S

RECORDS

Or TB

Symptom

screening

1: if all

criterion

have been

met/

recorded

0: if all

criterion

have not

been met/ not

recorded N/A: not

applicable if

there is no

record for

review

5 records/symptoms

(100%): 6 points

4 records/symptoms

(80%): 4 points

3 records/symptoms

(60%): 2 points

≤2 records/symptom

(≤40%): 0 points

1 2 3 4 5 Complete

records

/No. of

symptoms

correctly

identified

POINTS

1C1 Triaging of patients at OPD waiting area during all clinic

shift:

-Patients are classified into three groups and given due

attention accordingly:

Assess by reviewing patient files if patients are available at

OPD during the day of assessment. If not, assess by asking

the nurse on duty on how they conduct triaging of patients (

her/his answers should match with the points listed below)

Emergency signs requiring immediate attention

Priority signs (requiring priority in the queue

Non-urgent cases

1C2 % of adult TB screening symptoms correctly stated by

health center/clinic OPD provider (Select one provider

randomly on shift during the day of assessment)

• Assessment: Ask the nurse at OPD to name criteria for TB

screening (please observe whether her/his answers match

with the list mentioned below and give score accordingly)

173

1) Weight loss

2) fever for more than 3 weeks

3) cough for more than 14 days

4) Night sweats

5) TB contact exposure.

1C3 % of TB presumptive (TB symptom positive) patients that

have sputum results documented in any month in the last

quarter

*source of data: OPD register/TB suspect register

SUBTOTAL Points – ( Maximum Available points:18)

2C FAMILY AND CHILD HEALTH (FCH)

AMBULATORY (ANC, PNC) BEST

PRACTICES

Source of Data:

*ANC register for ANC Best Practise

indicators

* PNC register/follow up notes for

PNC/Postpartum best practise indicators

*Assess 10 cases/records in any one month in

the last quarter.

*If there are no enough cases for review, please

extend the review period to a quarter.

*If there are less than 10 cases for review after

extending the review period to a quarter, assess

the available cases/records.

* If there are more than 10 cases for review,

select 10 cases by using either simple/systematic

random sampling

*For indicators 2C1.6 i.e. pregnancy test,

resampling or extending the review period to a

quarter may be required as the number of

women with ≤16 weeks of gestations is usually

low.

*If there no records for review/the indicator is

not applicable in the set up being assessed,

please do not assess and not score the

indicator; rather write N/A and deduct the

available points for the indicator from the


PATIENT’S RECORDS/Registers


recorded):

0: if all criterion have not been met/

not recorded

N/A: not applicable: if there are no

records for review

*Score each case/record as 1 or 0

and then give a score for items per

patient record, when applicable.

* At last, please write the number

of records with complete

information as required and give

an overall score/points as the per

the criteria in the left column

9-10 records

(≥90%):

6 points

8 records

(80%):

4 points

7 records

(70%):

2 points

≤6 records

(≤60%):

0 points

2C1 ANC BEST PRACTICES:

1 2 3 4 5 6 7 8 9 1

0

Complet

e records

POINTS

2C1

.1

% of first visit ANC bookings in any one month in last quarter who had

documented:

BP

Height

Weight measurements

174

Fundal height measurements (Applicable only if

pregnancy >16 weeks of gestation)


2C1

.2

% of first visit ANC bookings in any one month in last quarter who received the

standard laboratory test according to the FNAC protocol:

Blood group and RH

HIV test

Haemoglobin

RPR (Rapid plasma regain for syphilis

diagnosis)

Urine analysis


2C1

.3

% of first ANC visits in any one month in last

quarter who received TT vaccine

2C1

.4


quarter who received iron supplementation

2C1

.5


quarter who received IPTp (if pregnancy >16

weeks of gestation and women living in

malaria area)

* Write Non-Applicable (N/A) if it is not a

malaria endemic area

2C1

.6

% of first visit ANC bookings with ≤ 16

weeks of gestation in any one month in last

quarter who had documented pregnancy test

results

2C2 POSTNATAL AND/OR POSTPARTUM

BEST PRACTICES

*Source of data: PNC register and/or post-

partum follow up notes/sheet/register

Indicator 2C2.4 needs a separate sampling

since the denominator is women who are 6

weeks post- partum during the assessment

period.

1 2 3 4 5 6 7 8 9 1

0

Complet

e records

POINTS

2C2

.1

% PNC visits in any one month in last quarter documenting assessment for the

following conditions of the infant:

General condition of the infant;



2C2

.2

% PNC visits in any one month in last

quarter documenting assessment for the

following conditions of the mother:

General condition ,Pulse rate, B/P and

temperature



175

2C2

.3

% PNC visits in any one month in last

quarter documenting infant feeding (BF)

status (exclusive, mixed or not BF)

2C2

.4

% women post-partum counselled and

offered any of the modern FP method

(below)at follow up PNC visit within 6 weeks

of delivery in any one month within the last

quarter ( this indicator should be assessed at 6

weeks post-partum visit)

POP (progesterone-only contraceptive safe

with BF)

injectable,

Implant

IUCD

Tubal ligation

Decline

SUBTOTAL: ( Maximum available points:

60)

3

C

MATERNITY WAITING HOME

Follow up of pregnant mothers in maternity waiting

home

*Write Non-Applicable (N/A) in clinics without maternity

waiting homes and/or if there are no mothers in

maternity waiting homes during the assessment period



*If there are less than 5 mothers in the maternity waiting

home,, assess indicator with mothers available during the

day of assessment

PATIENT’S

RECORDS

1: if all

criterion have

been met/

recorded):

0: if all

criterion have

not been met/

not recorded

N/A: not

applicable: if

there are no

mothers in the

maternity

waiting home

or if the facility

does not have

maternity

waiting home

5 records (100%):

6 points

4 records (80%) :

4 points

3 records (60%) :

2 points

≤2 records (≤40%):

0 points

3

C

1

ANC Best Practices: follow up of pregnant mothers in

maternity waiting home

1 2 3 4 5 Complet

e

Records

POINT

S

3

C

1.

1

% of mothers in maternity waiting homes monitored

for BP, FHR, and assessed for danger signs daily

*Source of data: ANC cards of pregnant mothers

176

* Danger signs: vaginal bleeding, headache/blurred

vision, fetal movement, sudden release of water from

vagina

*if there was no danger signs, it should be recorded as

no danger signs

SUBTOTAL; ( Maximum Available Points:6)

4C HIV–PMTCT

Source of data: ANC, ART, Delivery and DNA PCR

register

*Review ANC register and select 5 newly identified

HIV women for indicator 4C1.

*Review delivery register and select 5 HIV exposed

new-borns for indicators 4C2-4C4 in the last quarter.

*If more than 5 cases found, select 5 cases using

simple/systematic random sampling for each condition.

If less than 5 cases in the last quarter, assess all the

cases found.

*N/A (Not Applicable) if there are no HIV+/HIV

exposed cases for review and deduct the available

points for the indicator from the maximum available

points.

PATIENT’S RECORDS


been met/ recorded):



N/A: not applicable: if

there are no records for

review

5 records (100%):

6 points

4 records (80%) :

4 points

3 records (60%) :

2 points

≤2 records (≤40%):

0 points

1 2 3 4 5

Complete records POIN

TS

4C1 % NEWLY IDENTIFIED HIV + pregnant women

initiated on ART in MNCH (ANC) ON THE SAME

DAY in the last quarter

*Source: ANC and ART register

4C2 % of infants born to HIV+ women who had a DNA

PCR sample within 6-8 weeks of birth in the last

quarter

*source of data: delivery register, PNC and DNA PCR

registers

4C3 % of HIV exposed infants who had A DNA PCR

SAMPLE COLLECTED within 6-8 weeks of age

and received results within one month in last

quarter

*Source of data: DNA PCR register

4C4 % of confirmed HIV positive infants initiated on

ART in last quarter WITHIN 21 DAYS OF

RECEIPT OF RESULTS

*Source of data: DNA PCR register and ART register


5C AMBULATORY MANAGEMENT OF DIARRHEA,

PNEUMONIA ,MALARIA and Un complicated Severe

Acute Malnutrition IN < 5 CHILDREN

*Source of data: OPD/ IMNCI/CMAM registers

*Review OPD/ IMCI register and select 5 cases with

pneumonia, 5 cases with diarrhea, 5 cases with malaria, 5

PATIENT’S

RECORDS

1: if all criterion

have been met/

recorded):

5 5 records (100%):

6 points

4 records (80%) :

4 points

3 records (60%) :

2 points

177

Cases of SAM in any one month in the last quarter. If more

than 5 cases found, randomly select 5 cases for each dis order.

If the number of cases is not enough, extend the search period

to a quarter to gather 5 cases for each disorder. If there are

less than 5 cases in the last quarter, assess the indicator based

on the available cases.

*Write Not Applicable (N/A) if there are no cases for review



0: if all criterion

have not been

met/ not recorded

N/A: not

applicable: if

there are no

records for review

≤2 records (≤40%):

0 points

1 2 3 4 5 Complet

e

Records

POINTS

5C1 % children treated as outpatient for pneumonia in any one of month in last

quarter who were correctly assessed

*Source of data: OPD/ IMNCI register

Absence of general danger signs recorded: able to drink/feed,

vomiting, consciousness

Duration of cough/difficult breathing and child’s age recorded

Respiratory rate, and presence/absence of chest in drawing,

stridor and wheezing recorded

Correctly Classified


5C2 % children correctly treated as an outpatient for

pneumonia in any one of month in the last quarter among


Treatment: Oral Amoxicillin 50mg/kg divided thrice per day x

5 days; caretaker counselling and follow up specified or

admitted into hospital

5C3 % children with diarrhoea correctly assessed for signs of dehydration),

persistent diarrhoea and dysentery in any one of month in the last quarter

Assessment of dehydration: Using IMNCI guidelines (Integrated Management of

Neonatal and Childhood Illnesses)

Duration of diarrhoea and presence of blood recorded

General condition of the child recorded: lethargy,

consciousness and/or restless or irritability

Presence of sunken eyes, drinking status ( thirsty/drinking

eagerly or un able to drink/drink poorly) and skin pinch

Correctly Classified


5C4 % children correctly treated as an outpatient (ambulatory)

for diarrhoea in any one month in the last quarter among


Treatment :

-ORS, Zinc supplements and continued feeding and advise

when to return

5C5 % children diagnosed with malaria that have RDT + or

laboratory confirmation in any one month in the last

quarter

178

5C6 % Children with uncomplicated malaria correctly treated

according to national guidelines in any one month in the

last quarter

Treatment: ARTEMETHER (20mg)-LUMEFANTRINE

(120mg)(C0ARTEMETHER) during 3 days (See treatment

protocol in appendix 2 of the checklist guideline)

5C7 % Children with severe malaria correctly treated

according to national guidelines in any one month in the

last quarter

(See treatment protocol in the Annex section of the checklist

guideline)

5C8 % of 6-59 months old children with un complicated severe

acute malnutrition (SAM) who were managed as per the

national protocol in any one month in the last quarter

A 6 to 59 months old child with any one of the following

criteria is classified as SAM :

Weight for height <-3SD (WHO)

MUAC <115mm

MUAC <125mm and HIV positive

Bilateral pitting oedema

Out Patient management of SAM:

RUTF

Routine Medicine

Health and nutrition counseling and continued follow up

*see annex section of checklist guideline for treatment details

of SAM

SUBTOTAL points: ( Maximum available points: 48)

DELIVERY BEST PRACTICES

*Source of data: delivery register and

partographs

*Review delivery register and randomly select

10 deliveries in any month in last quarter. If

the number of deliveries is not enough extend

the search to the last quarter to gather 10

deliveries. If there were less than 10 deliveries

in the last quarter assess the cases found. If

more than 10 cases in a month/quarter then

randomly select 10 deliveries and assess the

partographs for the deliveries selected to

assess the following indicators

* Write Not Applicable (N/A) if there are no

cases for review and deduct the available

points for the indicator from the maximum

available points.

PATIENT’S RECORDS


recorded):


met/ not recorded

N/A: not applicable: if there are

no records for review

9-10 records (≥90%):

6 points

8 records (80%):

4 points

7 records (70%):

2 points

≤6 records (≤60%):

0 points

1 2 3 4 5 6 7 8 9 1

0

Complete

records

POINTS

6C

MATERNITY SERVICES; LABOUR, DELIVERY POST-NATAL CARE FOR MOTHER

AND NEWBORN

179

6C1

% deliveries performed by skilled

personnel in any one month in the last

quarter

•Skilled provider: RGN with mid wifery

training/RGN/PCN

6C2

% women who delivered in the facility monitored using partographs as

per criteria *Please take into account the cervical dilatation at admission while assessing the

following items

Assumption: Partograph should be opened for all pregnant women in labor unless

they come with fully dilated cervix with head visible. Women who present in

advanced labor should have at least one measurement of the items mentioned below.

If urine analysis was not done, the reason should be noted/documented i.e. lack of

supplies/women did not produce urine.

Fetal heart rate plotted every 30 minutes

State of membranes every 4 hours

presence/absence meconium ( if ruptured

membrane)

Cervical dilatation every 4 hours

Descent of presenting part every 4 hours

Contractions plotted every 30 minutes

Maternal BP every 4 hours

Maternal pulse every 30 minutes

Maternal temperature every 4 hours

Urinalysis documented at admission


6C3

% total births in any one month in the last

quarter documenting administration of

immediate postpartum oxytocin 10 units

IM (within one minute of delivery of baby)

(AMSTL: Active management of third

stage of labour)

* administration oxytocin 10 units IM within

one minute of delivery of fetus (or misoprostol

or ergomertrine, if BP normal, and oxytocin

unavailable)

6C4

% births with placental status documented

at birth in any one month in the last

quarter

•Assessment: complete or ragged , retained

placenta

6C5

% newborns BF within one hour of birth in

any one month in the last quarter

•Assessment: Time of BF initiation

documented

6C6 % newborns received Vitamin K in the any


6C7

% newborns received eye care

(Tetracycline) in the any one month in the

last quarter

180

6C8

% newborns received first vaccination

(BCG) in the any one month in the last

quarter

*source of data: delivery and/or PNC registers

6C9

% women monitored in early post-partum period (4th stage) per guideline (birth to discharge)

in any one month in the last quarter. *source of data: partographs and/or post-partum follow up

notes/sheets/register

Vaginal bleeding, at least every 30 minutes

1st 2 hrs after birth and then four hourly until

discharge

Uterine contraction at least every 30 minutes

1st 2 hrs after birth and then four hourly until

discharge

BP at least every 30 minutes 1st 2 hrs after


Pulse at least every 30 minutes 1st 2 hrs after


Temperature at least every 30 minutes 1st 2

hrs after birth and then four hourly until

discharge


6C10

% newborns monitored in early post-

partum period per guideline (birth to

discharge) in the any one month in the last

quarter

*source of data: partographs and/or post-

partum follow up notes/sheets/register

Temperature documented at least every 30

minute first 2 hours after birth then four

hourly until discharge

Respiratory Rate documented at least every

30 minute first 2 hours after birth then four

hourly until discharge

Breast feeding status documented at least

every 30 minute first 2 hours after birth then

four hourly until discharge

Colour documented at least every 30 minute


discharge


6C11

% facility births seen for day 3 PNC visit in

any one month in the last quarter

*Source of data: Delivery register ( to identify

list of deliveries) and PNC register

SUBTOTAL: (Maximum available points:

66)

7C OBSTETRIC & NEONATAL COMPLICATIONS

PATIENT’S

RECORDS

5 records (100%):

6 points

181

*Source of data: delivery register and/or partographs.

*Review partographs/follow up notes and randomly select 5

cases with PROM, 5 cases with PPH, and 5 cases with

Postpartum sepsis, 5 cases with Pre-eclampsia/eclampsia, 5

with neonatal asphyxia and 5 with neonatal sepsis in the last

quarter. If there were less than 5 for review in the last quarter,

assess the cases found. If there were more than 5 cases in a

quarter then randomly select 5 cases to assess the following

indicators

*Write Not Applicable (N/A) if there are no cases for review

PATIENT’S

RECORDS


been met/ recorded):


not been met/ not

recorded

N/A: not applicable: if

there are no records

for review

4 records (80%) :

4 points

3 records (60%) :

2 points

≤2 records (≤40%):

0 points

* 1 2 3 4 5

Complete

Records

POIN

TS

7C1 % women with prolonged labour in last quarter referred to

higher level facility

-obstructed labour criteria in any one month in last quarter:

active labour > 12 hours (from admission at minimum 4 cm

dilation or per patient-reported labour onset if admitted > 4

cm)

* Source of data: partographs and/or referral register or

referral notes/

7C2 % women with prolonged labor or Rupture of Membranes

and without chorioamnionitis that were administered

antibiotics as per protocol

Treatment with oral erythromycin (or amoxicillin) if ROM > 6

hours or active labor > 12 hours without signs of chorio-

amnionitis; first dose antibiotic and referral to hospital if signs

of sepsis (intra partum fever, foul-smelling discharge)

*Review all partographs of women who delivered in last

quarter and select those in which rupture of membrane

documented > 6 hours (at any time in course of labour and

delivery) or active

labour >12 hours (at any time) without documentation of other

signs of maternal sepsis

(maternal fever or foul-smelling discharge) is documented

7C3 % women with PPH managed per guideline in the

last quarter

*Review all partographs of women who delivered in last quarter and select those

partographs fulfilling the following criteria: PPH documented (EBL > 500 cc or VB and

tachycardia > 100 bpm or hypotension SBP < 100 or DBP <50) and check whether the

following three items listed below were done for each identified case

- See the annex section of the checklist guideline for the details

-PPH cause documented (atony, tear, retained placenta, other)

-Resuscitation for all PPH cases irrespective of the cause:

secure two IV lines with two 14/16 G cannulas or any large size

available, and run normal saline (NS) or ringer lactate (RL)

182

-Management according to the cause:

-Uterine atony: add 40 IU Oxytocin in 1L NS or RL solution

and run at 60 drops/minute until uterus is firmly contracted or

insert misoprostol 600-1000 mcg rectally or

-Retained placenta: controlled cord traction. If failed, manual

removal or

-Vaginal/cervical laceration: sutured

-Referral to hospital with IV access irrespective of the cause


7C4 % women with signs of intra- or post-partum sepsis (fever,

foul-smelling discharge) that were given pre-referral

treatment and referred to hospital in last quarter

Pre-referral treatment: X-pen 5 mega units IV immediately,

then 2.5 units IV 6 hourly, plus metronidazole 500 mg IV 8

hourly and chloramphenicol 500mg IV 6 hourly (Use ampicillin

1g IV stat, then 500mg IV 6 hourly instead of X-pen if

available)

7C5 % pregnant women with severe pre-eclampsia and/or eclampsia managed according

to the guideline in last quarter

Review partographs of women who delivered in the last quarter and select those who fulfill

the following criterion

Severe Pre-eclampsia:

-Diastolic BP 100mm HG or more

-proteinuria 3+ or more

Eclampsia:

-Unconsciousness or Convulsions (fits)

-dBP 110 mmHg or more

-Proteinuria or 2+ or more in a pregnant woman or a woman who has recently given birth

-Check whether the following three items listed below were done for each identified case:

*see the annex section in the checklist guideline for details on management of severe pre-

eclampsia and/or eclampsia

- blood pressure checked every 5 minutes until diastolic BP is

90 - 100mmHg

-Blood pressure monitored (if diastolic blood pressure (dBP) is

≥110 mmHg give:

Nifedipine 10 mg provided. If inadequate response after 20

minutes following first dose and repeat 10mg dose orally every

20 to 30 minutes until adequate dBP response is achieved, to a

maximum of 40 mg given. Then 10-20 mg orally every 4-6

hours to maintain dBP 90-100 mmHg or

Hydralazine 5mg IV slowly every 20 minutes for a maximum

of 20 mg

* applicable only if dBP was ≥110mm Hg

-Magnesium sulphate 20% solution, 4gm IV over 5 minutes

given. Followed promptly with 10g of 50% magnesium

sulphate solution, 5gm in each buttock as deep IM injection

with 1 ml of 2% lignocaine in the same syringe.

183

-Progress of labour including fetal well being monitored

-referral after stabilizing the mother


7C6 % of neonates who had an APGAR score of ≤5 in the first

minute after birth for whom resuscitation was initiated in

last quarter

*Source of data: partographs and notes

*see the annex section in checklist guideline for details

7C7 % neonates with possible serious bacterial sepsis managed

per standard in last quarter

•Assessment of possible neonatal sepsis: Review all cases of

newborn sepsis in last quarter from partographs and/or PNC

register; and select 5 records for review that meet any of

following probable sepsis criteria. *see checklist guideline

criteria for chart audit

-if documented temperature >380 C or < 250 C (and not

warming);

- RR > 60 or <30 breaths per minute;

-chest in-drawing or convulsion;

-no movement on stimulation;

- poor feeding/sucking or

-umbilical redness, newborn treated with stat dose antibiotics

and referred to hospital:

Treatment: Benzyl Penicillin 100,000 units / and Gentamycin

5 mg/kg first dose antibiotic and referred to hospital

*Source of data: partographs and/or PNC register


VERIFY THAT ALL QUESTIONS ARE FILLED IN

Supervisor thanks the staff

Signature:

DMO/Representative………………………………………………………..

Nurse in Charge…………………….........................

Counter verification………………………………………...

184

ASSESTMENT FEEDBACK

I. Summary Comments on Results. Please note any trends, problems, exceptional or creative

changes and results that you saw during your visit assessment

II. Noteworthy Improvement. Please note any improvement and include a few details of what they

are doing and why it is unique

III. Difficulties/ Challenges. Please note any assessment area that seem to be having an especially

difficult time in improving. Please include a few details about the problem, how it might be solved, and

who might be involved

IV. Recommendations and suggestions for improvement. Please note that the feedback is more

effective when emphasizes features of the clinical task to be performed (e.g. specifies a target

performance, presents information on how target performance can be attained, and address change in

performance observed since previous feedback

V. Follow up. Please review previous recommendations provided and assess if they were followed or

not

incorporating quality of care investments into a health

Documents