Alyssa L. DavisHealth Advisor, Concern Worldwide Burundi

Operations Research: Methods, Challenges, Emerging Lessons and OpportunitiesCORE Group Spring Meeting

May 12, 2011

The Care Group ModelThe Care Group ModelOperations Research in Burundi

Unrealistic Workloads

Inadequate Training

Insufficient Supervision

Lack of Support Systems

A Common

Community Health Worker or Volunteer

Basic Description of a Care GroupA Care Group is a group of 10-15 volunteers, community health

educators who meet regularly with a project staff person for training, supervision and support.

• Care Groups are different from typical “Mother’s Groups” in that each volunteer is responsible for regularly visiting 10-15 of her neighbors, sharing what she has learned and facilitating behavior change at the household level. .

• Household visits are targeted to the primary child caregiver, usually a mother, but all family members are invited to participate in the home visits.

Source: Tom Davis, Melanie Morrow and Carolyn Wetzel. Presentation “Care Groups: The Essential Ingredients.” April 2010.

Care Groups create a multiplying effect to equitably reach every family with women of reproductive age and/or children under five years of age with individualized

Behavior Change Communication (BCC) and social support.

Basic Description of a Care Group

• As interventions are phased in, Care Group volunteers become neighborhood resource people for an integrated package of health education and referral services.

• Packaging several interventions through a single delivery strategy has been noted to make economic sense, but there is evidence that this can lead to increased inequities unless population coverage of the implementation strategy is very high (Source: Victora, C.G., Fenn, B., Bryce, J. Kirkwood, B.R. Co-coverage of prevention interventions and implications for child survival strategies: evidence from national surveys. Lancet 2005, Volume 366, Issue 9495, Pages 1460-1466). Source: Tom Davis, Melanie Morrow and Carolyn Wetzel.

Presentation “Care Groups: The Essential Ingredients.” April 2010.

Under-five deaths could be could be preventing in 42 countries with 90% of worldwide child deaths in 2000

through achievement of universal coverage with individual interventions.

Source of Table: Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS. The Bellagio Child Survival Study Group. How many child deaths can we prevent this year? Lancet 2003; 362: 65-71.

The Care Group Model has been used to promote

and support highly effective child survival

interventions—and could be used to promote and support even more in the

future.

Focus on High Impact Interventions in Child Survival

  Project Country1. Health facility

improvement

2. Governance

3. Behavior Change – one-on-one

4. Outreach Services

5. Comm. treatment

6. Comm. Finance

7. Behavior change -

mass media

  Nepal x x x x   x x  Mali x x x x   x    Benin x   x       x  Rwanda 2   x x x x      Ethiopia x x x x   x    Cambodia 3   x x x x      Madagascar x         x x  Guinea x x x x   x    Senegal x x x x x x x  Cambodia 2 x x x x   x    Rwanda 1 x x x   x x    Kenya x              Haiti x x   x x   x  Mozambique x x   x        Malawi x x   x x x    Cambodia 1 x x x x       

  More successful 82% 82% 91% 73% 36% 73% 36%  Less successful 100% 80% 20% 80% 40% 20% 20%  Odds Ratio 0.8 1.0 4.6 0.9 0.9 3.6 1.8  Fisher Exact Test NS NS 0.05 NS NS 0.08 NS 

Significant at the level p < 0.05  Significant at the level p < 0.10  

Contact Intensity IndexSource: Jim Ricca, Senior Technical Advisor, MCHIP

The Care Group network of volunteers extends the

REACH of the formal

health system.

Health Center

Care Group Volunteers

Community Health Workers

Household Child Caregivers

26 Project Staff Train 200 Care Groups → 2,600 Care Group Volunteers Train 34,000 Households → Total Target Population Reached: 40,000 Women of Reproductive Age

24,000 Child under Five Years.

Mabayi Health District Child Survival Project: The Integrated Care Group Model

• Concern Worldwide Burundi identified the Care Group Model as a powerful strategy for community mobilization for widespread behavior change that would be suitable to the context of Mabayi Health District.

• However, the project staff structure was recognized to be unrealistic for the MOH to maintain after the life of a donor/NGO project.

• The idea for an Integrated Care Group Model that could be tested through Operations Research was developed.

Source: Philip Wegner, Concern Worldwide

22 Health Center Titulaires Train 153 Community Health Workers →153 Community Health Workers Train 300 Care Groups →

2,952 Care Group Volunteers Train 44,000 Households →Total Target Population Reached: 53,000 Women of Reproductive Age

41,000 Child under Five Years

Contributions to National Health Policy

• The National MOH is currently revising its Community Health Policy, including Community-Integrated Management of Childhood Illness roll-out strategy

• Community Health Workers have existed as a component of the MOH community health policy on paper, but an effective Community Health Worker strategy has not been successfully operationalized at scale in the country

• There is widespread acknowledgement within the MOH and its partners that CHWs currently have unrealistic workloads, inadequate training, insufficient supervision and lack of support systems

• The Integrated Care Group Model has been identified in national MOH discussions as a promising model for implementing a realistic Community Health Worker strategy

Source: Alyssa L. Davis, Concern Worldwide

Operations Research Objective

To test the effectiveness and sustainability of an adapted,

lower-input Care Group Model to improve both knowledge and practice of key child health and nutrition behaviors as compared

to the traditional, higher-input Care Group Model.

Source: Alyssa L. Davis, Concern Worldwide

Research QuestionsThere are four primary research questions:

• Does the adapted, lower-input Care Group Model achieve the same improvement in the knowledge of key child health and nutrition behaviours among caregivers of children 0-23 months as the traditional, higher-input Care Group Model?

• Does the adapted, lower-input Care Group Model achieve the same improvement in the practice of key child health and nutrition behaviours among caregivers of children 0-23 months as the traditional, higher-input Care Group Model?

• Does the adapted, lower-input Care Group Model achieve the same level of Care Group functionality as the traditional, higher-input Care Group Model?

• Does the adapted, lower-input Care Group Model achieve the same level of Care Group sustainability as the traditional, higher-input Care Group Model?

Hypothesis 1: There is no significant difference in the coverage of the knowledge and practice of key health and nutrition behaviors among caregivers of children 0-23 months reached by the Integrated Care Group Model as compared to the Traditional Care Group Model after two years of implementation.

– Hypothesis 1 will be tested by data collected through a Knowledge Practice Coverage (KPC) survey of caregivers of children 0-23 months of age at baseline and endline (non-inferiority testing; sample size of 320 in each study arm).

Hypothesis 2: There is no significant difference in the functionality or sustainability of Care Groups implemented through the adapted Care Group Model as compared to Care Groups implemented through the traditional Care Group Model after two years of implementation.

– Hypothesis 2 will be tested by data collected through regular monthly monitoring systems (Key Operational Monitoring Indicators).

Supportive Qualitative Data: In-depth interviews and focus groups will be conducted with District Health Team members and community stakeholders. Benefits and challenges associated with implementation of each Care Group Model will be identified.

Hypothesises and Methods

Key Operational Monitoring IndicatorsOperation Research Indicator

Target Value

Percentage of beneficiary households that received at least one Care Group Volunteer visit per month.

80%

Percentage of Care Groups that held a meeting at least once per month.

80%

Percentage of Care Group Volunteers that reported registration and C-HIS data to their supervising Promoter/CHW per month.

80%

Percentage of CHWs that reported registration and C-HIS data to their associated Health Center per month.

80%

Percentage of CHWs that attended at least one COSA meeting per month.

80%

Percentage of Care Group Volunteers who received at least one visit from a Promoters/CHW per month.

80%

Percentage of Care Group meetings that are conducted according to minimum standards of quality per month.

80%

Percentage of Care Group registers that meet minimum standards of quality per month.

80%

Percentage of CHW supervision visits to Care Group Volunteers conducted according to minimum standards of quality per month.

80%

Challenges• Defining an Integrated Care Group Model that would be feasible to implement within the current

health system structure

• Getting buy in of key MOH stakeholders in a constantly changing environment—both in MOH appointments and policies

• Changing guidance on IRB process; absence of a functional IRB in Burundi; delay in attaining IRB approval

• Identifying an appropriate sampling methodology for the baseline survey that would allow for statistical significance and would be feasible to implement within a project context

• Investing massive time and effort in the Operations Research baseline survey (640 households selected through simple random sampling)

• Lacking exclusive control over monitoring data collection with the Integrated Care Group Model implementation structure

Source: Alyssa L. Davis, Concern Worldwide

Lessons Learned• Attaining IRB review and approval is challenging in a post-conflict setting where there is

a dearth of ethics review institutions and systems

• Measuring indicators with statistical significance might require statistical tests, sampling methodologies and sample sizes that are unusual for child survival projects

Source: Concern Worldwide Burundi

• Conducting a separate Operations Research baseline survey is expensive in terms of budget, human resource and project effort costs

• Operational indicators change over the course of implementation, especially when working in partnership with the MOH

Tips• Identify an IRB well in advance, understand the submission protocol and consider the costs of the

IRB review

• Explore the possibility of establishing a relationship with an academic institution

Source: Alyssa L. Davis, Concern Worldwide

• Critically assess the necessity of a separate Operations Research baseline/endline survey

• Integrate Operations Research data collection into the regular project M&E systems as much as possible

• Consider how Operations Research M&E burdens will be managed throughout project implementation: “If the Operations Research stops operations, it isn’t Operations Research!”

Murakoze chane!(Thank you very much)

Source: Alyssa L. Davis, Concern Worldwide

Source: Philip Wegner, Concern Worldwide…et bonne chance!