KARUNA (COMPASSION) PROJECT

Mid-Term Evaluation

Authors: Dr. Shantidani Minz and Dr. Harika Siddabathula

Project Delivered by: EMMS International and Duncan Hospital

Purpose of the report The purpose of the mid-term evaluation was to assess how well the project is meeting its objectives as per the modified plan and what are the main learning points. Christian Hospital, Chhatarpur and Duncan Hospital, Raxaul are two EHA hospitals with project funding from EMMS with similar broad goals, and the purpose was also to identify common indicators for both.

The questions specified in the TOR for mid-term evaluation were:

Are the objectives/issues stated in the project plan being addressed adequately?

Will the inputs through the project have a long term effect on the health of the community and are these expandable to more areas served by Duncan Hospital?

What are the learning points from this project for project staff and Duncan hospital management and administration that will help project staff to be more relevant and effective in addressing the health needs of the local communities?

Are data for publication and research from the project being captured?

Biographies of the authors Dr. Shantidani Minz, MD, MPH, is a professor of Community Health from the Community Health Department of Christian Medical College (CMC) Vellore. She has worked in rural north India in a mission hospital and with the Christian Medical Association of India and is a faculty in CMC. She has 20 years experience in the field of community health, programmes and training and has been involved with programme evaluations in north and south India.

Dr. Harika Siddabathula (MD) is a resident in Community Medicine in CMC and has worked in rural mission hospitals for 2 ½ years in north India.

Acknowledgments The evaluation team would like to thank EMMS International and Duncan Hospital, Raxaul for the opportunity to evaluate this unique project in one of the most needy districts of Bihar. The team would also like to thank all the staff of Karuna project and other community health projects of Duncan hospital, groups and individuals met, especially the doctors and nurses at the three PHCs for their support and valuable information provided to the team during the course of the evaluation. Special thanks go to Dr. Vandana, Dr. Sharon and Mr. Sumit for their valuable inputs towards planning and interpretation of the findings.

The cover photograph is of the Karuna project team in Duncan Hospital.

2

Contents Purpose of the report ............................................................................................................ 2 Biographies of the authors .................................................................................................... 2 Acknowledgments ................................................................................................................. 2 Contents ............................................................................................................................... 3 Acronyms .............................................................................................................................. 4 Executive Summary .............................................................................................................. 5

Major Findings ................................................................................................................... 5 Recommendations............................................................................................................. 6

Introduction ........................................................................................................................... 8 Project background............................................................................................................ 8 Methodology and limitations ............................................................................................ 11 Context analysis and project planning ............................................................................. 11

Findings .............................................................................................................................. 15 Project implementation and possible modifications .......................................................... 16 Project objectives and assumptions ................................................................................. 18 Implementation, achievements and performance of overall objectives............................. 20 SWOT Analysis and brainstorming by project staff .......................................................... 27

Compliance with EMMS International Operational Framework ............................................ 29 Coherence with the vision and mission of EHA ................................................................ 29 Compliance with overall Operational Framework ............................................................. 29 Sustainability of the initiative ............................................................................................ 29

Learning and Conclusions ................................................................................................... 30 Recommendations .............................................................................................................. 31 Annexes .............................................................................................................................. 33

Annex 1: Terms of Reference .......................................................................................... 33 Annex 2: Methodology ..................................................................................................... 37 Annex 3: Duncan Hospital Community Projects Timeline ................................................ 38 Annex 4: People met during evaluation ........................................................................... 50 Annex 5: Evaluation Programme ..................................................................................... 53

3

Acronyms

ANM - Auxiliary Nurse Midwife

ASHA - Accredited Social Health Activist

AWC - Anganwadi Centre (day care centre run by the government)

AWW - Anganwadi Worker

AYUSH - Physician trained in Indian systems of medicine

CHC - Community Health Centre (Government health care institution)

EHA - Emmanuel Hospital Association

IEC - Information, Education, Communication

ICDS - Integrated Child Development Services

NGO - Non-Governmental Organisation

NRHM - National Rural Health Mission

PO - Project Officer

PHC - Primary Health Centre

RCH - Reproductive and Child Health

SC - Subcentre

SWOT - Strengths, Weaknesses, Opportunities, Threats

VHND - Village Health and Nutrition Day

VHSC - Village Health and Sanitation Committee

4

Executive Summary EHA and EMMS International have worked together to bring transformation in the lives of

people from many underdeveloped parts of North India for a long period. Duncan Hospital

has had a long history of engaging with the local communities based on the needs of the

time and vision of the institution, and currently has eight community health projects in

various stages of their existence. The KARUNA project is the first of its kind where

engagement with the community for improving Maternal and Child Health is linked with

clinical services of the hospital through special funding for increasing the capacity of the

clinical department.

The project plan was modified based on the findings of a situation analysis carried out at the

start of the project. The mid-term evaluation was proposed to assess the progress made and

lessons learned that would influence the future direction of the Karuna project and guide all

projects with community engagement.

Major Findings The project is focussing on a change in key behaviours that lead to demand for preventative

and curative services of the healthcare delivery system under the banner of the National

Rural Health Mission (NRHM), and matching that with quality improvement in the public

health system. The Karuna project's role is facilitatory and catalytic, with Duncan hospital

providing tertiary level clinical services for mothers and children. This is a daunting task in

Purba (East) Champaran District of Bihar, which has never been a focus for development.

Being on the border with Nepal, the blocks close to Duncan Hospital are difficult for

development programmes, with high migration and trafficking and a great shortage of

indigenous workforce for health programmes.

The project's effort towards improving the quality of health services through skills training of

government functionaries has been limited, since few ANM (Auxiliary Nurse Midwife) training

sessions have been done so far. ASHA (Accredited Social Health Activists) and ICDS

(Integrated Child Development Services) workers are more available. Permissions and

access to ANMs is difficult, because of their preoccupation with immunisations in recent

months and because CARE India is currently the official partner in Bihar for quality

improvement. The project is supporting the work of ANM and meeting with the subcentre

level group to improve their knowledge. Input at the PHC level has not been possible and

monitoring of PHC functions has not been done.

5

The activities towards changing key behaviours for healthy lifestyles are done through

community action groups, of which the adolescent boys’ and girls’ groups are the most

promising. Peer educators from adolescent groups are coming forward to address issues

that will result in social change. Community workers at the grassroots level who come from

the target villages are the key change agents.

The referral system for very sick children or complicated pregnant women is working well,

since PHCs refer patients to Duncan Hospital readily. However, the cost of care at Duncan

hospital is a cause of concern and the management is working on a system for providing

concessional treatment to patients from project areas.

Recommendations The government healthcare delivery system is deficient in human resources, infrastructure

and skills in delivering the standard of care set by the NRHM. While the government health

department works towards bridging the gap, many NGOs are involved in supporting the

work. Since Duncan Hospital has contributed for a long time to this district's healthcare and

gained the trust of local people, the Karuna project is well positioned to contribute actively to

change in the health status of the community.

In order to improve the quality of services, the Karuna project will need:

→ To go into a collaborative relationship with other participating NGOs in the district and

the government programme

→ To fill in the gaps that exist in planning, implementing and monitoring field

programmes. Project staff will need to complement the work of ANM and ASHA in the

field.

→ Efforts for community mobilisation and behaviour change need to concentrate on the

groups that are working well and show promise. These groups should be built up as

hubs with skills to initiate newer groups for dissemination of knowledge and skills, and

action. They should also be seen and promoted as agents for challenging the whole

community together for action to achieve health and long-term social change.

6

7

Introduction Project background The Duncan Hospital, Raxaul is a member hospital of Emmanuel Hospital Association

(EHA). It is situated in East Champaran District of Bihar which has amongst the poorest

health and development indicators in India. Being on the border of India and Nepal, this

hospital serves approximately 11 million people from surrounding parts of Bihar and

Southern Nepal. Founded by Dr. H. Cecil Duncan in 1930 and shaped by Drs. Mathew and

Joanna Peacock and by Drs. Trevor and Patricia Strong, this hospital developed into a

referral centre for providing essential medical and surgical services in this remote location. It

was managed by the Regions Beyond Missionary Union until 1974 and later handed over to

EHA.

The hospital has been challenged by the needs of the local community, and many

community projects have evolved over time in response. The relationship with EHA has led

to alignment with the vision and mission of EHA in responding to community needs around

the hospital. EHA has identified focus areas for all its hospitals working in the community,

and so Duncan Hospital’s community initiatives are aligned to these.

The Community Health projects of Duncan hospital have addressed issues such as

immunisation, mental health, community-based rehabilitation, HIV/AIDS, and capacity-

building for improving access to health services. Many of these projects link to the hospital

for referral of patients and technical support. However, Karuna project is the first to have a

defined collaboration with Duncan Hospital’s clinical services and a linkage for improving

maternal and child health care.

EMMS International is a charity based in Edinburgh, Scotland, working with EHA to address

maternal and child health (MCH) in north India. Duncan hospital is the implementing partner

for a time-bound project, Karuna, to improve maternal and child health in three Community

Development Blocks of East Champaran District of Bihar state.

History of Community Health initiatives of the Duncan Hospital Duncan Hospital is situated in a challenging environment. Being at the Indo-Nepal border, it

has been catering to the citizens of both countries, most recently during the massive

earthquake of April 2015. Traditionally, Duncan Hospital has provided high quality obstetric,

surgical and medical care to people from Bihar, UP and Nepal. In many ways it has been the

hospital for complicated cases in this region. The need for high volume clinical services has

lead to a persistent need for more and more doctors and nurses. Alongside this, there has

been awareness of the need to address health issues beyond the boundaries of the hospital.

A summary response of the hospital is presented in Annex 2. It reflects the development of

community health response in the country as a whole, starting with an outreach approach for

8

clinical services, specific disease responses and then widening out to address other

determinants of health such as livelihoods. While community engagement or

communitisation became an important area for NGO workthe to support National Rural

Health Mission, the Duncan Community Health team also addressed vulnerable groups in

society, including the disabled, mentally ill and those at risk of trafficking.

While many issues directly and indirectly related to health are of great concern in this area,

mothers and children of East Champaran District of Bihar continue to be at a disadvantage,

with inadequate preventative and curative services. CHETNA (Centre for Health, Education,

Training and Nutrition Awareness) and RCH (Reproductive and Child Health) projects have

been working in this field, trying different approaches and resulting in improved awareness

and use of services, but most projects have had a limited area for implementation, resulting

in valuable learning but limited impact.

Despite strides made with NRHM and many RCH programmes, quality care during

pregnancy and childbirth and better health of women and children in the region has not yet

been ensured. The Bihar government is collaborating with many NGOs to address this, but

East Champaran still lags behind with poorer health indicators. The Duncan Hospital team

identified as a challenge to build the capacity of the public health system through skills

enhancement and an effective network for referrals, to reduce morbidity and mortality among

mothers and babies.

Uniqueness of the Karuna Project The KARUNA project is envisaged to address two important issues leading to poor health

outcomes for women and children:

1. Lack of good quality health services and

2. Lack of demand for quality health services by the communities.

The project is building on the experiences of earlier community projects to increase

awareness and demand generation for preventative and curative health services and change

in behaviours leading to better health. The strategy combines community organisation and

awareness for positive health through action groups. The expectation is that small

homogenous community groups of older women (mothers-in-law), pregnant women, men

and adolescents will develop into action groups which will ensure promotional, preventative

and appropriate health-seeking behaviours for and by women and children.

In order to meet the demand thus generated, the Karuna project expects to match it with

quality preventative and curative health services, through the combined efforts of the

government health system and Duncan Hospital. Duncan Hospital is currently providing

tertiary level referral care for obstetrics cases and for sick children for many blocks of

districts. The Karuna project’s strategy is to ensure adequate skills for health personnel from

9

primary to higher level, appropriate facilities at each level and a functioning referral system

from the village and PHC to Duncan Hospital, for cases needing this.

This is the first community project addressing capacity-building within Duncan Hospital to

meet the needs generated by a community health project. The project is contributing to the

training of specialists and additional equipment for complicated obstetric and paediatric

patients.

Previous project input in the Karuna project area All three blocks have had some input through previous community projects of Duncan

Hospital. Of the three, Adapur block has had the most input, in the areas of HIV/AIDS, RCH,

Community Based Rehabilitation and trafficking, and Ramgarhwa has had the least input,

even though Champapur clinic has been running in this block for a long time and the school

of nursing makes home visits in this block. (See Annex 3.) The most beneficial effect of

previous projects is the rapport with the community. While Adapur and Raxaul villages are

easier to work with and the relationship with health staff is good, Ramgarhwa is a new area

for the hospital, with new challenges. Duncan Hospital has worked on RCH and VHSC

(Village Health and Sanitation Committee) formation and training in Adapur block for a long

time, under its CHETNA project, and established two RCH clinics there. The RCH nurse in

charge of these RCH clinics was well known in the community and had built a strong

relationship with the government PHC in Adapur. Under the Karuna project, these RCH

clinics were merged and moved to a more central location, at the request of patients. The

RCH clinics run by CHETNA have contributed greatly to the rapport with the community with

government healthcare providers in Adapur.

Relevance of the Karuna Project The current situation of the health system and gaps in human resources and services make

it necessary for Duncan Hospital to contribute to addressing some of these through

supportive action and bridging the gaps in care. East Champaran District has very poor

health indicators, which have not improved even with inputs from the NRHM, as the district

has not been able to implement the recommended changes and upgrade services. Being on

the border brings its own problem of issues with, most recently, the effects of Nepal's and

India’s somewhat confrontational relations. This prevents implementation of many

development activities. Migration is a challenge for government systems, an area to which a

private player like Duncan hospital can contribute significantly. The area has poor roads and

poor transport infrastructure, making healthcare inaccessible to most of the population, and

making it essential for projects to complement the government, to reach remote villages for

health service delivery.

10

Methodology and limitations The Karuna project has been functioning for only one and a half years, and so a deliberate

decision was taken to concentrate on processes rather than on outcomes and indicators.

Therefore, data related to the project was not considered in this evaluation. The final

methodology for the evaluation was decided based on ideas evolving from discussion with

the project leaders and field staff, some of whom have been involved in other community

health projects of Duncan Hospital. The experiences of the hospital’s earlier community

projects was incorporated, to identify and describe the specific contribution of the Karuna

project.

The methodology followed is represented in a flow diagram in Annex 2.

The evaluators reviewed all project documents and reports provided by the project

implementers before the evaluation, to draw up a tentative plan for field work and interaction

with specific groups. After collection of data from the field, the evaluators reviewed the

documents again, to clarify issues, inputs and strategies for interpretation and report writing.

Additional documents, such as maps, data collection format, field reporting format and

monthly reports, were made available for the team when asked.

The evaluators observed facilities for maternal and child care in the 3 blocks’ PHCs, 1

subcentre, 2 ICDS centres, Duncan hospital's Adapur and Champapur centres, and the Out-

Patient and In-Patient facilities at Duncan hospital. We used IPHS (Indian Public Health

Standards) as the guideline for expected standards.

It was found imperative that the Karuna project be seen as a component or a part of Duncan

Hospital's (and EHA's) response to the needs of the larger communities served by the

hospital. For this the evaluators needed to see the larger picture, and Dr. Sharon Cynthia put

together a comprehensive document to decipher this complex picture. (See Annex 3.)

Both evaluators met with the coordinating people, Dr. Vandana Kanth, Dr. Sharon Cynthia

and Mr. Sumit Khalkho for an overview and discussion on the direction of the evaluation.

The evaluation’s final plan was based on their input and that of other project staff,

concerning feasibility of meetings and observations in the three blocks under the project.

Details are given in Annex 4.

Context analysis and project planning Health system in Project Area: Challenges and gaps Purba (east) Champaran District headquarters is in Motihari, which is 40 to 70 km by road

from three block headquarters. The district hospital is located in Motihari. The three blocks in

the project are adjacent to each other, each with a designated block PHC (equivalent to

CHC) and all belonging to Raxaul subdivision. The health infrastructure is poorly developed

in all three blocks, with only one functional PHC in each block, although additional PHCs

11

exist in the official plan. Some conduct Out-Patient services, when doctors can be spared

from their main PHC work.

Each of these blocks has a population close to 200,000, with each ANM covering a

population of nearly 10,000, compared to the recommended 5,000. There is a shortage of

human resources, with 2 to 3 MBBS doctors and 4 PHC nurses per PHC and a shortage of

ANMs for public health work. Although male MPHWs are supposed to work alongside ANMs,

most of these posts are vacant. Subcentre and PHC infrastructure is inadequate. This

results in inadequate coverage of the population by ANM and concentration on a few service

activities such as immunisations and VHND (Village Health and Nutrition Days).

Until recently, immunisation coverage was very low in the project area and the programme

has concentrated on improving it. ANM, AWW and ASHAs come together fortnightly to plan

activities and generate a due list for immunisation. Most of the ANMs are currently occupied

with catching up on a backlog of primary immunisations. This has also become the main

activity for ASHAs who are primarily responsible for making the due list for every VHND and

going house to house to motivate beneficiaries. ASHAs are also responsible for

accompanying women to the 24x7 PHCs which provide delivery care.

ANMs’ time is taken up fully in organising these activities, reporting, attending reviews and

attending training run by many NGOs who are involved with NRHM programmes.

Ayush doctors in PHCs are expected to oversee school health programmes, but are unable

to fill the need of PHCs’ clinical work.

The new PHC building in Adapur has a good space and 30 beds, but additional nursing or

support staff to maintain the structure are not available.

Raxaul is located on the border with Nepal and this border town has a hospital with multiple

specialities. However, complicated cases are referred to Duncan hospital.

Basic training of ANMs is lacking in quality, since they are trained by different private

institutions which lack skills training. Most of them come from other districts, as the

educational achievement of women and girls is very low in East Champaran. This also

results in lower motivation.

Apart from poor infrastructure and quality of services, government PHCs are plagued with

corruption. Patients are required to pay to receive care. This prevents many poor people

going to PHCs.

Preparatory phase and modifications in proposal The first proposal was prepared by Dr. Vandana Kanth with inputs from EMMS. This was

modified subsequently, based on findings of a needs assessment which used qualitative

methods and a baseline survey report which brought out huge gaps in antenatal and child

care. The decision was taken to address behaviours that will lead to long term change in

health status rather than influencing short-term but measurable indicators. Based on the plan

12

of project personnel, the direct target population was reduced. However, training input for

quality improvement in health services was maintained for all government health workers. Hierarchy and responsibilities Dr. Vandana Kanth is the director of the Community Health Department and oversees all

planning, implementation and monitoring of the Karuna project. Dr. Sharon Cynthia and Mr.

Sumit Khalkho are responsible for detailed implementation, monitoring, review and revision

of plans and activities.

RCH nurses support clinics at Adapur and Champapur and the Block team with

implementation of activities at community level, and training of government staff,

complementing service delivery at VHND. They also interact with Duncan Hospital

functionaries for referred patients.

Mobilisers and motivators work as a team for community mobilisation, community education

and implementation of all activities in the 10 villages of the block adopted by the project.

Mobilizers interact with block level government health system functionaries for planning,

collaboration for service delivery and feedback to the PHC team. They attend the block level

meetings at the PHC.

Motivators are voluntary workers from the community. They have designated villages and

may belong to one of the project villages. Male and female motivators tend to work together,

covering 4 to 5 villages rather than individually covering 2 to 3 villages. They are more

comfortable working with same-gender groups. Reporting is primarily done by the motivators

and mobilisers depending on the level of event or activity. Reporting of vital events is done

by the motivators. They are the functional link between project staff, community and the

village-level functionaries, ASHA and AWW.

Mobilisers are supervisory workers in principle, but as the project activities are in the

developing stage, they work mostly along with motivators.

Review of all Community Health Projects of Duncan Hospital to date It was found imperative that the Karuna project be seen as a component or a part of Duncan

Hospital's (and EHA's) response to the needs of the larger communities served by the

hospital. For this the evaluators needed to see the larger picture, and Dr. Sharon Cynthia put

together a comprehensive document to decipher this complex picture. (See Annex 3.)

Persons/groups met Both evaluators met with the coordinating people, Dr. Vandana Kanth, Dr. Sharon Cynthia

and Mr. Sumit Khalkho for an overview and discussion on the direction of the evaluation.

The evaluation’s final plan was based on their input and that of other project staff,

concerning feasibility of meetings and observations in the three blocks under the project.

Details are given in Annex 4.

13

Role of Duncan Hospital The hospital administration is part of planning the needs at hospital level for maternal and

child services. The requirements for equipment and staff were forecast based on current

utilization from the project area and type of service needs. The hospital supports the

Community Health Director with all administrative and management-related responsibilities.

The hospital management team is directly involved in reviewing progress and changes in the

project with the Community Health Director.

Recent plans are to bring all community health projects into a common platform to

complement and support each other. There is also a plan for a common review system,

which has already started.

14

Findings This is the first project linking a community health project to clinical service development or

enhancement at Duncan Hospital. The plans for input to the hospital systems were based on

the experience of the clinical department. Documentation of outputs and related outcome

needs were thought of by the team. The administration of the hospital is tuned towards

bringing the strengths of all community initiatives together and ensuring an active link

between hospital and local communities through community health projects.

The project modification reduced substantially the quantum of proposed activities and

population to be covered. The plan and details of activities were reorganised to follow the

logical framework’s original objectives and strategies. This has made the project more

doable and the objectives more likely to be achieved.

Duncan Hospital and its community health projects have developed a very good rapport with

the government health system and many other departments related to health. The project is

getting involved with skills training of ASHA, ANM and ICDS workers. There are hurdles in

addressing specific gaps in skills of workers and basic equipment to improve the quality of

care, as the need is high.

The healthcare delivery system in Purba Champaran District is not yet developed as per the

recommended standards under NRHM. The population covered by a PHC and ANM is much

more than the target maximum 30,000 and 5,000 respectively. There is a shortage of health

functionaries who belong to the district. Most of the staff are from other districts, creating

additional challenges for workers. Infrastructure development will take a long time.

Many other international and national NGOs are working in Bihar to support NRHM activities.

This is creating extra demands on staff for reporting and creating avenues for monitoring

needs of each. The Karuna project and Duncan Hospital could channel the efforts of all

groups, bringing the required and appropriate level of quality in training, monitoring and

reporting acceptable to all. The challenge is also not to burden staff.

The service delivery component of the Karuna project is supplementing basic MCH care for

a limited population in a few project villages around two centres in Adapur and Champapur.

While high quality of clinical care is provided, some of the components of antenatal care are

not available in the clinics. Linkages with the PHCs and Subcentre functionaries are not

clear. However, referral from PHCs to Duncan Hospital for maternal care is actively done.

Sick children are referred by all government health workers and Karuna project workers to

Duncan Hospital. However, the cost of care is an important deciding factor for most families.

The Karuna project team is skilled and highly motivated. The commitment and involvement

of community motivators is inspiring. The grassroots workers are well accepted and have a

good rapport with the different action groups being organised to build capacity of the

15

communities. Only one project officer is employed full-time in the project and needs to

supervise a wide geographic area and 2 levels of workers. The skills and involvement of

male and female community organisers is different and RCH nurses contribute to this level

of supervision.

Many community action groups are organised for building capacity for better health. There

are different levels of success, with different groups, with the smost active and promising

group being the adolescent boys’ and girls’ groups. There is a possibility of social change

with this change in youth.

Documentation and reporting of activities are done at many levels, but mainly by grassroots

workers (motivators). There are many reports and the need for each needs a review.

All community health projects of Duncan Hospital have different strengths and experiences.

At present each one is working almost independently and sometimes parallel to others in the

field. There is room for sharing capacity and maximising benefit to the community.

Project implementation and possible modifications Skills and Capacity Building for project team

All full-time staff underwent training for updating and upgrading their skills for the needs of

the Karuna project. New staff had in-house training and orientation to the project. Motivators

were trained separately. The teams carried out needs assessment activities in their block to

gather baseline information. This included the baseline survey in approximately 24,000

households and documentation of felt needs using participatory methods. These processes

were useful in orienting staff to the community, their needs and difficulties, barriers to overall

development and challenges for the Karuna project. It also provided an opportunity for

developing data collection skills.

Recommendation: The project needs people with skills in community organization and participatory methods for

improving ownership by the community and health communication. Special training courses

for PO and community mobilizers (who will now be appointed as Community Coordinators)

will help better implementation of the activities.

Development of the baseline and revision in plan and budget A baseline study of over 23,000 households was carried out in all three blocks. The

objectives were to estimate infant, neonatal and maternal deaths in the population and to

study the causes of them. This confirmed the previous understanding of the needs for

changing maternal and child health status. It became clear that health services were not

accessed for preventative maternal care and curative child-care.

There are a few issues regarding the quality of data. 60% of respondents were male

respondents and all information was not collected for all households. Vital information such

16

as total population was not captured in the survey, and as a result some rates cannot be

calculated. Therefore we recommend that the survey results be used only for internal

circulation, planning and monitoring purposes and not for publication.

Modifications were made in the logframe, budget and Gantt chart in February 2015 and June

2015.

Recommendation: Programme data should show the changes in specific indicators so that repeated surveys

are not necessary. If an end line survey is done, it should only look for validating the data on

a sample and in addition study behaviour changes identified as key indicators in the modified

plan.

Project Organogram (Field Component) and monitoring system Community Health Director

KARUNA

Project Coordinator

Project Officer

RCH nurses - 2

Block teams Raxaul Ramgarhwa Adapur

(Clinic at Champapur with SoN) (Clinic at Adapur)

Community mobilisers (6)

Male & Female

Sub-Block level Motivators (Male and Female-12)

Observations: This project needs more supervisory and technical input at the level ofme Project Officer for

an effective programme with community action groups. Motivators usually work in pairs.

The Project Officer oversees programme implementation activities, but the supervision is

shared with the Coordinator. RCH nurses also supervise community motivators and help

community mobilisers with the activities. Since the community level activities are not

17

specifically divided between mobilisers and motivators, there is overlap and resulting

confusion in reporting.

Women motivators work with women groups while men work with boys and men.

Vital event reporting is done by the motivators, but there is no formal system of verification or

validation.

Recommendations: Clear responsibility for supervision must be given to different levels of staff. Verification of

vital data collected by the motivator is necessary and can be done by RCH nurses, while

community activity reports must be authenticated by the community organisers. Cause of

death is not collected using verbal autopsy methods, which should be done to monitor

changes in child deaths.

Collaboration with other NGOs Being one of the least developed districts, East Champaran has attracted a large number of

NGOs and international organisations, the leading one being CARE India, which is involved

in skills training to improve the quality of care in health centres and hospitals. There are

many others, including the WHO, Path Finder and UNCEF, who are involved in different

aspects of the NRHM programme. All representatives attend review meetings at the PHC

and visit the field for either monitoring or training activities. There is a difference in level of

skills and commitment of each of the staff at the district level, but each one has a voice at

higher levels in the hierarchy of the heath system that Duncan Hospital lacks.

Recommendation: The Karuna project should take the initiative to bring all of them together and form a network

for exchange of ideas, reduce duplication and work to complement the efforts of each other.

There is a need for advocacy at higher levels for a change in the functioning of the PHC

system, and a need for a better image and removal of corruption. This network can advocate

collectively at all levels, to encourage accountability and quality improvement, which is in any

case everyone’s agenda.

Special efforts need to be made to collaborate with CARE India for skills training of

healthcare workers and for advocacy to address corruption at all levels of healthcare. Since

CARE India is a trusted long-term partner with the Government of Bihar, it carries good

leverage with government organisations.

Project objectives and assumptions The broad objectives of the Karuna project are:

1. To provide good quality maternal and infant care in 3 blocks of East Champaran District

18

a) To network with government stakeholders to facilitate implementation of

government guidelines for antenatal, delivery, postnatal and infant care services in

the PHCs and HSCs in 3 blocks.

b) Capacity-building of government and private healthcare providers in maternal and

child healthcare.

c) Strengthening mother and child healthcare services in Duncan Hospital and the

primary health care services in Duncan Rural Health Centres at Champapur and

Adapur and mobile clinics.

2. To empower communities to adopt recommended behaviours to improve the survival of

mothers and children through awareness-generation and advocacy.

The objectives are in line with the mission and vision of EHA and EMMS International and

there is commitment from Duncan Hospital to consolidate the community engagement efforts

to bring cumulative benefit from all project inputs.

One of the important assumptions was that the Karuna project would be able to engage with

NRHM efforts in improving the quantum and quality of health services. This has been

difficult, since more than one non-governmental organisation is involved in training in various

aspects and components of the NRHM. CARE India is the main partner with NRHM Bihar

for skills development of nurses and ANMs for improvement in quality of care. However, their

effort is to work from the higher level to the lower level of service delivery, resulting in poorer

quality of primary care services while tertiary care services are being upgraded. While there

is wisdom in this approach, a very low quality of services persisting at primary level with non-

existent secondary level care (no functioning CHC) does not develop confidence in people

who need to use the government health system. Project staff have anecdotal information of

language barriers with trainers and more theoretical training than skill building for subcentre

ANMs by CARE. Duncan Hospital is a site for skills training and could offer more

involvement with CARE training. Training of ASHAs has been more possible but all areas of

need cannot be addressed.

The project plan also had a component for monitoring quality at health centres according to

Indian Public Health Standards (IPHS). Without an official agreement or MOU, this role

cannot be played by Duncan Hospital.

Monitoring of changes expected through the Karuna project interventions would not have

been possible if the whole block was covered uniformly with activities. The modified plan

restricts the implementation area to 10 villages in each block, and therefore changes in

these villages collectively can be compared to others. Since the some training input for

health and nutrition workers (ANM, ASHA and AWW) is at the block level with PHC, a

certain level of change is expected in the non-target villages also. This will be reflected in the

change at block and district level indicators.

19

Implementation, achievements and performance of overall objectives MCH Services

I. Training of ICDS and health workers Formal training sessions with health staff and community level nutrition and health workers

are limited. There is difficulty in gaining permission to train ANMs and PHC nurses. ASHAs

are the more willing and enthusiastic group as a whole, but some are uninterested, chosen

arbitrarily and therefore ineffective. AWWs are well trained and knowledgeable. They need

encouragement and acknowledgement of their work to go beyond their call of duty and

collaborate with other functionaries. They are happy to work with the Karuna project, as they

feel appreciated and supported when specific needs for health services arise.

RCH nurses use any opportunity to train the ANMs on care of antenatal women, but this is

usually limited to VHND.

Recommendation: The project could make separate plans for formal and informal training. Informal training in

the field during VHND, visits to AWW and SC could concentrate on skills while formal

training when possible could address programme improvement, information quality, quality

assurance and special skills training. Karuna project leaders need to identify specific

monitoring needs for the government-managed services to which they can contribute and

make a formal agreement for Duncan Hospital to provide services.

Each block team could develop its own list of training gaps to address and follow it, since

each one faces different challenge and barriers.

Depending on the skills and knowledge of mobilizers and motivators, they should start as

training assistants and later take on training of topics related to community organisation,

participation and communication.

II. Collaboration among grassroots functionaries of nutrition and health, and linkage with project

ASHAs, ANM and ICDS workers are brought together at the Health Sub Centre (HSC) level

for training and planning for VHND. This has forced two hitherto independently functioning

programmes to come together. Anganwadi workers are fairly knowledgeable, know their

work and responsibilities and document on a regular basis. However, they may not cover the

whole population. They are now starting to share information but still are not cooperating

well, since ANMs are at a higher level, while ASHA and AWW are at a lower level and have

better knowledge of the community. Community mobilisers and motivators are working on

the ground with these groups, but their involvement is tentative at present. Motivators can be

20

part of village teams, actively working to supplement and complement ASHAs’ and AWWs’

work.

Fortnightly meetings of this group is used as an opportunity by the Karuna project to interact

and add inputs to improve knowledge, assess availability of services and plan for bridging

the gap, such as training ASHA to check blood pressure, and organizing VHND activities

with them.

Recommendation: Government health workers and Karuna project workers are not fully utilising the strengths of

the ICDS programme. Karuna could work to strengthen this linkage and bring them closer to

the community through interaction with Saasbahu and adolescent girls’ groups. Motivators

and mobilisers need to work closely with AWW and plan complementary activities with

beneficiary groups.

Motivators and mobiliser need to work with AWW to reach mothers of small children.

III. Contribution during VHND At present the role of community motivator, mobiliser and RCH nurse during VHND is not

well defined. Raxaul block had additional input through another Duncan Hospital project,

Sexual and Reproductive Health Rights (SRHR), in which intensive input on antenatal

examination, investigations, IFA supply and counselling was given. The ANMs are able to

organise the activities better and able to provide all services with the help of ASHAs and

AWWs. RCH nurses occasionally help in examination of antenatal women, but are in a

dilemma, thinking they might replace ANM rather than build their capacity.

Recommendation: VHND involvement/contribution needs to be planned and accepted by ANM, MOIC.

Mobilisers and motivators must use the opportunity for interpersonal education. RCH nurses

can help in examinations to start with and transfer responsibility over time when the backlog

of immunisations is over.

IV. Referral linkage

Although a formal referral linkage with the government health system may not be possible,

the informal system is active. It was observed that this open referral to Duncan Hospital is

misused by PHC staff. The other barrier to early referral to Duncan hospital for acute care or

severe illness is the cost of care. For a large section of society, the cost of care at Duncan

Hospital is prohibitively high, causing either delay in seeking care or choosing a less

appropriate mode of treatment.

Recommendation: Agreement on a protocol-based referral and encouraging protocol-based management at

PHC will improve the value of referral, reduce delays and reduce inappropriate referrals to

higher centres and Duncan Hospital.

21

The referral system followed by all community health projects of Duncan Hospital should be

uniform and an assessment of patients’ paying capacity with recommendations for

concession should be part of the referral. The hospital needs to develop a feasible plan for

this in consultation with Community Health and Finance departments. There should be a

clear communication to potential beneficiaries for a hassle-free referral. Since the project is

contributing to human resources and equipment at the hospital, a specific indicator showing

improvement in usage of Duncan Hospital’s maternal and child health services is needed to

justify the investment.

Institutional (Duncan Hospital) capacity building

The project aims to support postgraduate training for one doctor, who will return to work in

the community health projects. A suitable candidate has been identified for this. Additional

equipment for obstetrics and paediatric care have also been bought with project funding.

This has enabled the hospital to provide higher quality services for a higher number of

patients coming to Duncan Hospital. An estimated 70% of the patients coming for obstetric

care are from the project blocks. Therefore improving quality of services at the hospital will

directly benefit the community.

Comment: The role of doctors receiving support for training on their return needs to be discussed. While

it is important that they contribute to services in the project area, they will not be competent

in programme-based interventions for maternal or child care. They will require additional

training if they are to help in project management. Therefore, it is recommended that they

be used by the hospital primarily for speciality care and not be given project responsibility. If

any one of them is interested and has a leaning towards community approaches then s/he

may be encouraged to take more responsibilities.

Peripheral Health Centres at Adapur and Champapur Champapur centre building is owned by Duncan Hospital and Adapur health centre is a

rented building. Both have been used as peripheral outreach centres for clinics and use by

other community health projects. The facilities in the centres are adequate for running a

fully-fledged 24-hour dispensary and community centre. Alternatively, they can be used as a

place for community training. At present, both the buildings are underutilized. At present,

Adapur centre is used for conducting a general antenatal clinic and the clientele is from the

surrounding villages, majority Muslim and not necessarily from the project villages. It does

not provide the full range of antenatal services as per NRHM guidelines and requires

patients to buy medicines at hospital cost. This form of service can only be considered an

extension of Duncan Hospital services, not really fulfilling the need for bridging the service

gap in the project population. A service at the community level must be following norms set

by NRHM or higher, and should be available for a minimum or subsidised cost.

22

The location of Champapur centre has more possibilities for being relevant and bridging the

service gap in healthcare of women and children in addition to providing care for other

illnesses, especially of the elderly. This centre was a vibrant place when run around the

clock with availability of trained health professionals and a fair level of care for most

illnesses. Over time, this has changed and it now functions only as a stop for nursing

students going for field practice. Doctor- or nurse-managed clinics in the centre are therefore

not regular, and do not contribute significantly to healthcare of local people.

Recommendation:

Adapur centre is very close to the 30-bed block PHC which has a 24-hour delivery facility.

Therefore, if this centre is maintained, it should identify useful service activity for itself, such

as a high risk antenatal or newborn clinic, or a special clinic for diabetes, hypertension and

similar illnesses which are not yet a priority of the government health system.

Champapur clinic is located in Ramgarhwa block, which is the most backward among the

three project blocks. It could function truly as an extension of Duncan Hospital, with

comprehensive services for specific illnesses. The School of Nursing and the hospital could

develop a relevant model of clinical services that could be provided through this centre as

local people find it difficult to go to PHC or the nearest town for healthcare. This would mean

a specific budgetary allocation and personnel planning by Duncan Hospital. Of the two

centres, Champapur could still play a positive role for improving access to healthcare, but

this will require a policy decision by Duncan Hospital.

Both buildings can be utilized as peripheral centres for all community health project

activities. While new projects are planned by Duncan Community Health Department,

utilization of these centres should be kept in mind. As Ramgarhwa block is relatively

underdeveloped, future projects can be concentrated in this area.

Community empowerment for behaviour change The Karuna project has identified multiple community groups for focusing on behavior

change communication in 26 villages, namely Siswa, Kowadhangar, Parsauna Tapsi, Gadh

Bahuari, Nayakatola, Lakshmipur Lakshmanwa in Raxaul block, Bakhri, Kurmeniya, Koraiya,

Latihanwa, Dhabdhabwa, Bishunpurwa, Pokhariya, Manghurhara, Chikani, Chiljhapti in

Adapur block

Tilkahoni, Belahiya, Kalikapur, Gulariya, Jaitapur, Sakrar, Mushahri, Bahuari, Mangalpur and

Nandlali in Ramgarhwa block. Community organization also takes place through

organization of these smaller groups. The most active and functional of them all are the

adolescent boys’ and girls’ groups and older women’s groups (Saas). The team has found it

difficult to organize new fathers’ groups and develop an active mothers’ group. The reason

for this is migration of young men for long periods of time and younger women (daughters-in-

law) having no voice in the family and community. It is a challenge to form and maintain

23

adolescents’ groups and older women’s groups too, since this society has very low regard

for women, put no value on education, exercises caste dominance and contains very little

political will to change this current status. Some of the comments and findings from the field

are given in the boxes below.

Due to poor educational achievement in general and lack of access to information, the

communities in the study area are not aware of their entitlements or rights. They do not

question the lack of opportunities for education, healthcare, food security or livelihoods,

maintaining a low level of nutrition and health with very high distress migration by able-

bodied persons. Previous Community Health projects by Duncan Hospital have addressed

OLDER WOMEN

• New daughters-in-law can come out of their house only with their mother-in-law

• All agree that hospital delivery is a good thing • Most older people are willing to postpone marriage for girls if they are

interested in studying • Villages understand the relationship between early marriage and

maternal deaths • Older women are proud to belong to a group, learning many things, e.g.,

how to feed children and prevent diarrhoea • Some are not sure of the messages but trust the motivators’ guidance

ADOLESCENT GIRLS

• See motivators as role models who share knowledge and experience • Have learned new things, e.g. menstrual hygiene and use of pads instead

of cloths • Are not isolated during their menstrual period • Value knowledge and education • Are more confident of taking care of themselves, e.g., “If we can read and

write we will not be fooled by others (traffickers) who may try to take us to other places (we can read signs)”.

• All want to continue their education and marry later than is traditional

ADOLESCENT BOYS

• Have learned about harmful habits, e.g., substance abuse; some of them have stopped smoking and drinking.

• Are proud to be peer educators • Have understood the consequences of early marriage and the need for

care of mothers and children. They are encouraging their sisters to get care.

• Are keen on education and looking for alternative ways to use their energy

24

some of these issues, but the input has been scattered in many blocks and has reached only

a few villages and beneficiaries. Often there is a lack of continuity in the focus of one project

to the next in the same geographical area. This has resulted in poor sustainability of

changes achieved by individual projects and lack of measurable social change.

Adolescent boys and girls groups have benefitted most from the input of the project. Peer

educators have been identified and trained. There is interest and enthusiasm to embrace the

knowledge imparted through the training sessions. Members of the groups are becoming

vocal and have developed positive self-esteem, and some of them have taken action at the

community and family level to initiate change. Major change is seen in stopping underage

marriages and decisions to continue schooling. The examples of a few bold individuals, both

boys and girls, is an encouragement to others. Older members of the community are

accepting and support such decisions, which is a big change in this society.

Boys and girls are actively seeking ways to build their lives with positive behaviours moving

away from harmful or useless behaviours and practices.

Older women are influential and probably gatekeepers for any positive health-seeking

behaviors for the care of women and children. They also make decisions about their

adolescent sons’ and daughters’ education and marriage. They have a louder voice than

their sons, who may be absent from home or decision-making most of the time. The younger

women do not have a voice in a combined group. It was very difficult to get them to talk in

the large group. Even individually they were not able to repeat any message learned through

activities of the project or ICDS programme. They depend very much on ASHA for any

health service. Motivators in the community are an additional source of information on health

services for them. It is possible that the women motivators have developed a rapport with

this group of women but the evaluators did not have the opportunity to verify this.

Older men, though very influential in community level decision-making, seem to agree with

the older women’s decision to change health-seeking practices. They also are benevolent

towards adolescent boys’ and girls’ decisions on schooling and marriage. So far the project

staff have not faced any resistance from traditional leadership or the older age group in

educating the younger generation on their rights and possibility of different futures.

The organization of the task force has not been easy and in light of the modification of the

project plan, with only 10 target villages per block, the role of the task force is in question.

Ideally, it would help the project implementers in planning and monitoring project activities.

Motivators met by the evaluators were very well accepted in the community and were greatly

appreciated for their contribution, especially to the young people. Some boys and girls

considered them as their role models.

25

Recommendations: The younger generation is ready and eager for change. It would be wise to invest in this

group and give lower priority to young men’s group and the Task Force. Activities to

encourage wider participation, building character and encouraging a competitive spirit could

be brought, in the form of Melas or inter-village competitions, for example, for young people

to showcase their talents. This would also help in giving positive direction to their energy.

Non-functional groups should be either removed from activities or made low priority.

Develop a community-initiated collective action There should be an active move to engage villagers of all ages in discussion to address their

rights, entitlement and access to health and development services. This should be initiated

by the adolescent boys’ and girls’ groups, making it a community effort for change. Ekjut

NGO’s strategy in Jharkhand has been a successful experiment in markedly bringing down

maternal and child deaths, due to community decisions and ownership of the action for

accessing health services.

Social change needs change in the mindset of the powerful, especially when gender issues

are to be addressed. This will need collective decision and action by the whole community.

The process can be tried for MCH care, and if successful can become a vehicle for social

change.

Behavioural change needs an enabling environment, which needs community support. So,

older men who are making most decisions need to be brought into the discussion when older

women are comfortable with this.

Health Communication for behaviour change

The Health Communication strategy is not well defined. Multiple methods and media have

been sporadically used for different groups and for mass health education. As community

action groups are the most receptive, the whole strategy for behaviour change

communication can be focused on them. The project team should study appropriate

messages and media needs and plan specific strategies for this input. Specific indicators for

this component need to be in place before input is planned. Sporadic mass health education

programmes are strongly discouraged, and instead young people in the community should

be used as a vehicle to reach the wider community. They can be formed into health

education groups, taking messages to schools, peers, family and the village.

Documentation and health information system

Currently, a large amount of information is captured in various reports by motivators and

organizers. Apart from vital events, most of the other documentation is only related to

activities such as meetings and visits.

Two sets of indicators need to be developed:

26

1. Informing the change in health status, e.g. births, perinatal deaths, causes of

deaths, outcome of childhood illnesses and maternal deaths.

2. Assessing progress towards achievement of outcomes listed in 1, e.g. sick children

identified and referred by ASHA, motivators and even community members,

community initiatives in making ICDS centres function, near-miss cases

referred and managed in Duncan Hospital, and number of ANMs providing complete

antenatal care package.

We suggest that the project team revisits all process and outcome indicators and retains

only those that will directly lead to outcome indicators even beyond the project period. After

identifying this, the process should be to work backwards and identify the information that

needs to be collected. Some of these will need to be done on a regular basis prospectively

(e.g., births, antenatal care, perinatal deaths and birth weight), while some other indicators

will need only periodic collection of specific information. A set of suggested indicators is

given in a separate document.

SWOT Analysis and brainstorming by project staff The group identified a happy, strong and safe community as a healthy community where

livelihoods are ensured. This project is contributing towards this broad goal. All three block

teams worked on challenges and opportunities for achieving this goal.

Common barriers identified were

• Poor quality of ANMs’ work, non-functional ASHA, dominance and interference

of husbands (AWW, ANM and ASHA).

• Restrictions on girls

• Poor attitude towards girls’ education

• Difficulty of changing attitudes towards care of women

• Too many NGOs

• Entitlements which do not reach the people

• Bribe-taking in PHCs (Specific to Ramgarhwa)

• Inactive and uninterested motivators

• Poor access to communities, e.g. bad roads

• Elections and the monsoon interfere with activities, especially with monitoring

• Unavailability of adequate transport

• Poorly functioning health services

• Lack of equipment in PHCs

Facilitating aspects were:

• Acceptance and support of community, PRI and MOIC

• Ability to work with ASHA, AWW and ANM

27

• Support of CARE and UNICEF

• Change in adolescent girls, now taking decisions about their own lives and

dreaming

• Willingness of parents to change

• Mothers-in-law thinking about their daughters-in-law

• Increasing awareness of some task force members

• Different NGOs working on different issues and so multiple inputs.

28

Compliance with EMMS International Operational Framework Coherence with the vision and mission of EHA EHA has identified specific areas for community engagement. MCH remains one of the

important areas for intervention. Duncan Hospital's community health projects are

addressing some of the key areas of interest, and recent projects have moved to issues

which are not traditionally considered "health", for example human trafficking.

Compliance with overall Operational Framework EMMS International articulates its values and principles very clearly in its policy document

called its Operational Framework. These are translated in the Karuna project through

engaging with the vulnerable (women), government health providers, empowering women

and adolescents who otherwise have no voice, and making quality health care and access to

it a rights issue. EMMS International puts a major emphasis on gender, and sees all its

Maternal and Child Health (MCH) work as concerning primarily gender.

Sustainability of the initiative Sustainability of project activities depends on developing a long-term Duncan-Community

intersection, where an open communication channel is created for communities to develop

their skills in exercising their rights to achieve good health. Duncan Hospital may bring in

different ideas in the form of community projects, but these should all address the basic

need to develop social capital for long-term and sustainable development.

Focus on adolescents is a strategy with a long-term effect on social change, much needed in

this society. Addressing issues of the disabled and mental health issues are also efforts

towards social justice. All community projects of Duncan Hospital should develop a common

agenda for social change and incorporate these in project plans.

29

Learning and Conclusions Duncan Hospital has a long history of engaging with local communities to address needs

arising out of interaction with them. This has lead to a large number of community projects

addressing a range of issues, including inequity, injustice and right to life with dignity in a

small way. The team has gained valuable experience of engaging with the community and

building trust. They are respected and people have faith in programmes initiated by Duncan

Hospital. The Karuna project has gone a step further to address deep-rooted social

behaviours to bring change in ordinary people’s lives and health. This is a difficult task since

the outcome of activities towards this will only be visible in the long term. However, it is a

start and the experience of the Karuna project in changing mindsets and behavior should be

used for all future community initiatives by Duncan Hospital.

• The project is targeting a small number of villages in each block for intensive inputs.

This should be used for learning and fine-tuning strategies. If possible, this lesson

should be extended to other villages through the community action groups.

• East Champaran District will always remain in the peripheries of political interests. It

is in a difficult area, being on the Nepal border. People here will always need non-

governmental organisations to advocate for them, for the foreseeable future.

Therefore, Duncan Hospital’s intervention in the district will always be needed.

• Collaboration with NGOs with different value bases will always be a challenge for an

institution such as Duncan Hospital with a strong Christian ethos. The challenge is to

find the meeting point with such agencies for the benefit of the common man and

woman.

• People are waiting for a change. Developing long-term partnerships with

communities will sustain even a small input by a short-term project. Therefore, short-

term projects should invest in community organisation, building social capital and

trust-based partnerships with local communities.

• The face of a project is identified by the staff and workers. Having a large number of

local workers will increase credibility with local communities and indirectly invest in

capacity building.

30

Recommendations Due to the delay in effective implementation of NRHM programmes in the project district, the

contribution of the Karuna project and Duncan Hospital is needed to bridge the gap.

However, the needs are in the areas of generating demand through community mobilisation,

improving quality of services through training of government functionaries, monitoring

services and bridging the gap in services. This will need the following:

• Develop the current status of good rapport with government health system into a

trust relationship and work with the PHCs for image-building for better utilisation of

public health services. (Improve Quality)

• Work towards forming a collaborative network with other NGOs and the government

health system in the district and leverage good will for Duncan Hospital for better

participation in skills training of government health workers. (Improve Quality)

• Link all community health projects of Duncan Hospital together and with the

communities, to complement and supplement the health services. (Improve access

to Quality Care and better Demand Generation)

• Build on the success in reaching adolescents and youth and invest in this group,

even though the outcomes of this intervention will be seen much later than the

project period. Drop the non-functional, or difficult to reach community groups such

as the new fathers’ group and Task Force. However, joint community action is

needed for improvement of maternal and child care where villagers of all strata and

ages need to be involved. Bring the interested groups of adolescents, older and

vocal women and men, and those who make decisions in the community together

for a joint effort for lasting change in health-seeking behaviour.

• Match the demand for quality healthcare with an efficient and trusted referral system

to public health system (PHCs) and Duncan Hospital

• Review the organisational structure to increase the number of animators

(community motivators), or reorganise their work responsibilities since they are key

to the success of the main objectives. Institute higher level supervision (level of

Project Officer), since there are 3 blocks with different needs. Improve the skills and

supervision of community level workers and redefine RCH nurses’ role to give more

responsibilities based on their knowledge and experience.

• Review the project document, plans and strategies to identify process and outcome

indicators relevant for tracking the efficiency of the inputs and change. Simplify and

restrict documentation of events, programmes and reports from the field to those

required for indicators. Link community data to hospital data.

31

Make a policy decision on involving clinical staff in community projects and improve

involvement of the School of Nursing. Institute a two-way integration of community health

activities and hospital-based work for long-term ownership of community engagement

initiatives by Duncan Hospital staff. Referral and feedback systems, monthly review of

activities by a common group, hospital data feeding into programmes and vice versa will

promote this.

32

Annexes Annex 1: Terms of Reference Terms of Reference for Mid-Term Evaluation of Duncan Hospital’s Karuna (Compassion) – MCH in East Champaran, June 2014-March 2017

1. Introduction/background EMMS International is a charity based in Edinburgh, Scotland, which works to improve

healthcare, through partners in north India, Malawi and Nepal. Its main partner in India is

Emmanuel Hospital Association (EHA), and its largest initiatives with EHA concern Maternal

and Child Health (MCH). This mid-term evaluation is of the above-named project, run from

Duncan Hospital in Raxaul, Bihar.

This project, with direct costs of £343,343, is intended to improve maternal and child health

in three blocks around Duncan Hospital. It is funded from EMMS International’s fundraising.

The purpose of the mid-term evaluation is to find out how well the project is meeting its

objectives, make recommendations to improve it and provide a report to be published by

EMMS International.

The report will be the copyright of EMMS International. EMMS International and Duncan

Hospital will circulate the report to all their staff and key stakeholders of the project, and will

aim to present key findings in conferences and in academic research papers.

2. Scope of mid-term evaluation to be carried out by the consultant 1. Review attached programme documentation. (1 day)

2. In Duncan Hospital, interview the Project Manager (Community Health

Consultant, Dr. Sharon Cynthia), Director of Community Health and Development

at Duncan Hospital (Dr. Vandana Kanth), and any other relevant staff

recommended by Dr. Cynthia and Dr. Kanth, Conduct a Focus Group Discussion

with the whole Karuna team in Duncan Hospital. (1/2 day)

3. View all statistics and relevant documentation in the Karuna office. (1/2 day)

4. Visit at least one village in each of the three blocks in the project area, villages to

be chosen by the evaluator. Include in these visits Duncan Hospital’s Adapur

Clinic. During these site visits, in each block, conduct the following:

a. view all premises relevant to the Project and associated services,

b. interview and hold discussions with staff working on the project,

c. interview government healthcare staff and others involved in the project,

33

d. conduct focus group discussions with each of the groups involved,

including pregnant women/girls, lactating women/girls, fathers, mothers-

in-law, adolescent girls and boys. (3 days)

5. Interview relevant senior government healthcare staff in Raxaul. (1/2 day)

6. Consolidate and analyse data, write draft evaluation report using the format

attached, send to EMMS International and Duncan Hospital for review, and

incorporate feedback into final evaluation report (2.5 days total)

7. Conduct all the above in relation to the attached project documents.

8. Travel from home to Duncan Hospital and back home. (2 days)

Total: 10 working days, to be done within 2 months from start to end.

3. Main purpose and questions to be answered in the report - To what extent is the project meeting its objectives?

- Is the project working to address the issues raised in the baseline survey?

- What have staff learned through working on this project?

- Will systems instituted by the project continue after it, so that health and healthcare

continue to improve?

- Is there scope to replicate the project in other locations covered by Duncan Hospital?

- Could Duncan Hospital usefully institute more advocacy, to improve the project’s

outcomes?

- Has the project yielded data which could be used in research to be presented or

published?

- What changes could help Karuna achieve its objectives better?

4. Fees, expenses and budget A consultant based in India is sought, to minimise costs of travel.

Fees: £100/day = £1,000, to be paid after all work is completed, upon submission of an

invoice. EMMS International will reimburse reasonable public transport, accommodation and

subsistence costs, upon submission of receipts.

Budget = 10 days @ £100 = £1,000 + reasonable expenses. If the evaluation is conducted by an institution, the expenses policy of this institution will be reimbursed. If the consultant is accompanied by a more junior person for learning purposes, that person’s expenses may be reimbursed. 5. Timing

November/December 2015, with report to be finalised within 2 months of start date.

34

6. Logistics

EHA will arrange the consultant’s travel to Duncan Hospital and back home. Duncan

Hospital will provide accommodation and meals for the consultant, and will organise all local

travel needed.

7. Confidentiality and publication The report is for publication, and therefore will comply with EMMS International’s Child and

Vulnerable Adult Protection Policy, in terms of which people to name in the report, and

gaining permission from such people.

EMMS International and Duncan Hospital will distribute the mid-term evaluation report

internally, to relevant stakeholders in India and to potential donors to healthcare in India.

EMMS International will put the mid-term evaluation report on its website and on IATI

(International Aid Transparency Initiative). EMMS International or Duncan Hospital may

present the report at conferences, and will inform the other parties before doing so. Any

party may reference the report, and in so doing will always refer to EMMS International, EHA

and Duncan Hospital as co-implementers.

EMMS International will use the mid-term evaluation report as a learning document amongst

its staff. Duncan Hospital will use the report to debrief all the project’s stakeholders

(including EHA staff, volunteers, government staff, community groups, senior and junior

government health staff) about challenges and successes.

8. Profile of the Consultant An independent consultant with minimal prior experience of Duncan Hospital and the Karuna

staff, who has conducted at least 5 external evaluations, has at least 5 years of experience

of working in community medicine and maternal and child health in India, and has a

demonstrable grounding in and academic knowledge of the issues surrounding gender and

maternal and child health in India.

9. Outputs

• A draft mid-term evaluation report, using the format attached, to be submitted

to EMMS International and Duncan Hospital within a month after the

evaluation visit.

• A final draft of the mid-term evaluation report, still using the format attached,

to be submitted to EMMS International and Duncan Hospital within 2 weeks

after receiving feedback from EMMS International and Duncan Hospital on

the draft evaluation report.

10. Attachments

• Logframe (version revised June 2015), budget (version revised February 2015),

Gantt chart (version revised February 2015), Additional Notes (version revised

35

February 2015) and Project Agreement signed 25th March 2014 between EMMS

International and EHA.

• EMMS International Child and Vulnerable Adult Protection Policy

• EMMS International Operational Framework

• Project baseline survey report

• All quarterly narrative, financial and monitoring reports from Duncan Hospital to

EMMS International, as finalised by EMMS International for language and clarity.

11. Other documents, to be prepared or provided by Duncan Hospital before the evaluation As requested by the consultant.

12. Signatures Consultant Signature

Signed:

Name:

Date:

Position in Organisation:

EMMS International Signature

Signed:

Name: Catherine Ratcliff

Date: 25th October, 2015

Position in Organisation: Director of International Programmes

36

Annex 2: Methodology

37

Annex 3: Duncan Hospital Community Projects Timeline

Duration Funding Partner

Collaborating Partners

Main theme

Project name

Strategies Population covered

Sagauli

Adapur

Banjariya

Chhaoradano

Raxaul

Ramgarhwa

Ghodasahan

Problems Main Output/

impact

1988

2006

Medical camps Bhelahi, Chikkni, Jaitapur

X

1995

2007

CHAMPAK

Medical camps, Literacy programs

5 villages

38

1997

Ongoing

DIFAEM, Duncan Hospital,

BSACS AIDS Control and Treatment

ACT Outpatient and inpatient care of HIV affected patients, improving access to ART, intravenous drug users, HIV surveillance - ICTC, community awareness on HIV prevention among CBOs, youth groups, local health practitioners and high risk groups,

Around 6 villages

X X X X X X Stigma reduction in patient families, improved access to ART and government schemes for HIV patients

2001

2007

SIM Community empowerment

CHETNA Immunization, Literacy, self help groups

Migration

39

2003

Ongoing

CBM, EHA Canada

Other CBR units in EHA, Mobility India, SAGE

CBR CBR Continuum of health care(both medical and rehabilitative), learning centers, life skill trainings, capacity building for improving employment, improving access to government entitlements for differently abled, mainstreaming disability, developing community leaders, community mobilisation for barrier free environment

8 panchayats, 4 blocks

X X X X Lack of systems to improve access to education for children with disabilities and lack of services such as ENT and resource centers in the government system

Improved access of differently abled persons to health services from Duncan, improved awareness in the community regarding government schemes for PwDs and stigma related to PwDs

2008

2010

Geneva Global

BVHA RCH CHETNA Community clinics, saas mandali, adolescent groups - Badte Kadam, CHASSNI (HIV awareness), Village Development Committee, children club, immunization

16 panchayats

X X X X 2007 - Major flood in East Champaran

Improvement in immunization, antenatal care

40

2009

2013

EHA Canada

DIFAEM

Urban health

ROSHNI (Raxaul Overall Social and Health Needs Initiative)

Awareness on communicable and non communicable disease, poverty reduction

52854 (Raxaul Urban)

X

2011

2013

Tear Fund UK

EFICOR - technical assistance

DRR CHETNA PADR - Participatory Assessment of Disaster Risk, VDMC - Village Disaster Management Committee - Rescue, relief, medical, early warning, mitigation plans, District Disaster Management Committee

10 villages

X X

41

2011

2013

Geneva Global

Indo Nepal Human Liberty Network

Human trafficking

Indo Nepal Human Liberty Initiative

Mass awareness on prevention of trafficking, identification of statutory bodies working on human trafficking, rapport building with Child Welfare Committee, Juvenile Justice Board

X X X X Child labour and trafficking could not be reduced by community awareness and empowerment alone. Need for strong networking with other partners and government in India and across the border was identified

Formation Anti Human Trafficking Unit at district level, District Child Protection Committee, District level Labour Superintendent, District Task Force (Dhaba dhal) on child labour rescue operations

42

2011

2013

SIM Livelihood improvement

IGPs, SHG group formation and capacity building, skill development - tailoring, livestock rearing, pottery, adult literacy, children's clubs, adolescent groups - BCC

250 villages

X X X X In spite of improving livelihood, inequalities in health access could not be reduced. The key gap which was identified absent government health services

Empowerment among women's groups and adolescent girls, improved literacy

43

2012

2015

SIMAVI BVHA Sexual and reproductive health rights

SRHR Improving awareness on reproductive health rights among newly wed and eligible couples, adolescent boys and girls. Creating enabling environment for SRHR in the community by interacting with self help group women, local elected body members, rural health practitioners and religious leaders. Advocating for improvement in service delivery at Village Health Sanitation and Nutrition Days, Health subcentres and Primary Health center.

13 panchayats in Raxaul block

X Improved knowledge and changed attitudes among target groups with regards to SRHR. 18 health sub centers which were previously closed became functional at the end of 3 years. 30% increase in the number of deliveries in primary health centre, Raxaul.

44

2014

2015

SIMAVI BVHA Prevention of child marriage

Unite against Child Marriage

Improving awareness on child marriage and its dangers among adolescent boys and girls, community leaders, religious leaders and health care providers through peer education, school level awareness and community level awareness programs. Improving access to school education through interaction with school management committee and parents and community members

13 panchayats in Raxaul block

129 child marriages stopped by youth clubs and 195 drop out children re enrolled in school

45

2014

2017

SIM Improving access to government health services through empowerment of VHSNCs

CHETNA VHSNC capacity building and advocacy, community awareness through mass awareness programs

42 panchayats

X X X X Main gap identified was need for capacity building in terms of technical knowledge for ASHA, ANM and program level knowledge government health officials

Progressing towards standardization of VHSNC functioning - monthly meetings, minutes register, health plans, advocacy applications, use of untied funds, maternal death reporting

46

2014

2017

TearFund UK & Geneva Global as part of Freedom Fund

Human Liberty Network, Justice Venture International, Free the Slaves

ASISH (Action Against Social Injustice and Promoting Social Harmony)

Reduction of vulnerability to slavery (trafficking, bonded labour, child labour), Information sharing on slavery incidents in the target community with other partners working on rescue and prosecution, Joint advocacy initiatives to strengthen political will

X X Appointment of Additional Director of Child Protection, East Champaran in second position in the state (2015) in rescue and rehab initiatives under leadership of labour superintendent Mothihari

47

2014

2017

EMMS International

Mother and child health

KARUNA

Strengthening government MCH services at the block and village level. Community awareness and empowerment on MCH

3 blocks 30 villages

X X X Gender inequality found to be a strong underlying social factor for poor MCH

Strong networking with government health officials and CAE India at district and block level. Improved skills and knowledge among ANMs and ASHAs. Village level information related to delivery of MCH services being used as powerful tool of advocacy

48

2014

2019

GLRA & EU

Improving access to government entitlements for people with disabilities (PwDs)

Sammalit Vikas Yojana

Listing, formation and strengthening Disabled Peoples Groups and Organizations to access government entitlements for PwDs and government stakeholders

10 blocks

X X X X X X Empowerment of PwDs to demand for their rights in an organized manner is challenging

Listing and group formation of PwDs who were unidentified so far, certification of persons with intellectual and mental impairment for the first time in the entire district.

2014

2017

EHA Canada & DIFAEM

Mental health

Nayi ROSHNI

Community awareness on mental health and prevention and treatment of mental illnesses, identification, treatment and follow up care of persons with mental illnesses

1 block X Absence of mental health services in the government hospitals at block and district level

Improved care and follow up of persons with mental illnesses in Raxaul.

49

Annex 4: People met during evaluation

People Met

Group Issue/ topic Comment

Dr. Vandana Kanth

Dr. Sharon Cynthia

Mr. Sumit Khalkho

Project overview

Brief on other community project of Duncan hospital

Plan for evaluation

Top Duncan hospital administrators -

MD

CEO

MS

Perspective on the community engagement of the hospital and long term commitment for change in health status of the local communities

Plan for better integration of Hospital and Community work. Bringing all community projects together as one functional system.

Project field staff & RCH nurses

Understanding of the project goal, their part and challenges

Participatory activity using modified force-field analysis

Medical Officer -In-Charge - Raxaul PHC

Karuna project input through training and participation in monitoring programs

Block PHC nurses, helpers of Raxaul, Ramgarhwa and Adapur

Availability of MCH services, challenges, protocol based management, referrals, interaction with Karuna Project staff

Observation of facilities also done

ANMs, ASHAs, AWWs Their activities, challenges, relationship with Karuna project

1 subcenter and 2 ICDS centers visited

6 Community level motivators Informal discussion on their role and responsibility, vital event reporting, division of work among male and female motivators

Observed interaction with community members, adolescents.

Formal interview was not possible due to lack of time and availability

50

People Met

One ASISH project teacher at the community.

Roles and responsibility, link to Karuna project

Adolescent boys group -2 Approximately 20 members and 3 peer educators

Age at marriage, education, migration, self worth

Motivators present

Adolescent girls group -3

approximately 12 in each

Learning by joining the group, information on care of self, value of education, early marriage, self worth

Some girls have become vocal, interested in showcasing themselves, consider Karuna project motivator as a role model.

Women's group- 3 mixed groups were met that included younger and older women. Some of these were pregnant.

Understanding of health and care of women and children, need to seek care, preventive serves. Knowledge of government health services, workers. Issues with accessing care at PHCs. Rapport with Karuna project workers

Unable to meet younger/ pregnant women separately. Older women dominated the discussions

Men (mostly older) Gave opinion on health issues and access to care.

Were present in the periphery of the older women's group and joined in some discussions.

Community informal leaders- 1 Role of Karuna Project in improving health care access

Observations

ICDS center 3 Lack of building

AWW with good knowledge and performing well

Good relation with Karuna

Subcenter Poor facility and few equipments

PHC Adapur - new building with Protocols on the wall

51

People Met

adequate space, few staff

Ramgarhwa

Raxaul- old building with limited space

New coming up

Ultra sound done by a male technician

Adapur center Use as OP and Chetna project uses for meetings/ lunch stop

Almost opposite Adapur PHC

Champapur center

Duncan hospital faciltities Additional equipments for maternal and child health emergency and acute care are purchased with project finding.

PHC review meeting

Raxual

ANMs, MO-IC and representatives of many NGOs (WHO, CARE, Pathfinders, UNICEF etc. & Karuna project coordinators) also attend

Lot of input in this PHC with equipments, Hb and urine test kits, support in organising VHND etc. Different NGOs monitoring/contributing to different aspects of primary health care

PHC review meeting

Ramgarhwa

ANMs, MO-1, MO-IC, UNICEF rep, CARE rep.

52

Annex 5: Evaluation Programme Day 1, AM The evaluation team had detailed discussions with the Director

of Community Health Department and Project Coordinator, Dr. Sharon, on Day 1. With input from the team, the field visits were finalised.

Day 1, PM, Day 2 and Day 4

Field visits to all three blocks, visit to PHCs, HSC and ICDS centre.

Visit to the Hospital especially maternal and neonatal service.

Informal interaction with doctors’ team.

Day 3 Meeting with Duncan Hospital administrators and project leaders.

Group activity and discussion with the supervisory team.

Day 5 Debriefing and clarifications with project leaders.

53

© EMMS International April 2016

This publication is copyright but the text may be used free of charge for the purposes of advocacy, campaigning, education, and research, provided that the source is acknowledged in full. The copyright holder requests that all such use be registered with them for impact assessment purposes. For copying in any other circumstances, or for re-use in other publications, or for translation or adaptation, permission must be secured and a fee may be charged.

The information in this publication is correct at the time of going to press.

EMMS International 7 Washington Lane Edinburgh EH11 2HA

0131 313 3828 www.emms.org info@emms.org

EMMS International is a charity, registered in Scotland (SC032327) and a company limited by guarantee, registered in Scotland (SC224402).

54