Case management versus M&E in the context of OVC programs:
What have we learned?
Jenifer Chapman, PhDFutures Group/MEASURE Evaluation
Overview
Case management tools 101
Update on the Child Status Index
Key questions
What is a case mgmt tool?
Paper-based instrument
Completed by direct service providers (generally low-literate volunteers)
Assesses child’s well-being along priority dimensions (e.g. health, social relations, etc.)
Care plan documentation
Purpose of a case mgmt tool
Purpose: To improve quality of care Highest priority = Case workers’ needs
o Untested hypothesis: case workers will make better decisions if they use a tool
May also support M&E / reporting needs
What is included?
Information collected Client contact information / demographics
Wellbeing information that is changeable over time
Care plan: services & referrals provided
Information flow Most important use is at local/SDP level
Some information may flow up regional level
A plethora of tools...
Child Status Index (MEASURE Evaluation)
Child Support Index (Pact)
OVC Wellbeing Tool (CRS)
Child Status Matrix (FHI)
Parenting Map (TSA)
Etc.
We need to be cautious
Some CM tools are being applied for purposes beyond case management Targeting (identifying beneficiaries)
Program monitoring (recording services provided)
Evaluation (aggregating wellbeing scores)
Exercise caution in using a CM tool for other purposes
Child Status Index Assessment & Update
About the CSI 5 years ago, CSI was designed for low-literate
home visitors to capture children’s status across the 6 domains of PEPFAR OVC programming
Early hopes that CSI could meet a range of information needs
CSI has been implemented for different purposes: case management to program evaluation
CSI is used in at least 16 countries O'Donnell K, Nyangara F, Murphy R, Nyberg B. Child Status Index. A Tool for Assessing the Well-Being of Orphans and Vulnerable Children—MANUAL. Chapel Hill, NC: MEASURE Evaluation; 2009.
CSI Assessment: Phase I
Rationale: to systematically assess how programs are implementing & using the CSI and understand OVC program field needs for additional tools to meet care, support, and M&E demands
Study questions For what purposes are OVC programs using CSI?
What are the advantages and limitations of CSI?
What are the unmet M&E needs of OVC programs?Cannon & Snyder. 2012. The CSI Usage Assessment. Chapel Hill, NC: MEASURE Evaluation.
Summary of Findings: Phase 1
25 interviews with senior program staff in 13 countries
Program staff find the CSI useful
CSI implemented mainly by volunteers
Information collected via CSI is rarely used by volunteers except for targeting (not recommended)
Care plans and referral protocols are inadequate
Summary of Findings: Phase 1
Variation in CSI implementation, data use due to: Unclear purpose & guidance on CSI use with
desire to assess impact
Variability in training approaches
Insufficient support/funding for technical assistance, follow-up, and training (data management, analysis)
CSI is important, but one tool in the toolkit
So now what?
Study Phase II Problem: Lack of information on the utility of the CSI at the
community-level as a job aid & input from CCWs
Purpose: To understand how CCWs and care teams make decisions about children (including role of job aid / data)
Methods: Interviews with CCWs and team leads in five countries, among organizations using/not using the CSI
Revision of CSI Guidance
6 Core CSI functions (we think)
1. Builds rapport between service provider and beneficiaries
2. Orients service provider to the holistic needs of the child and encourages referrals
3. Strengthens informed care decisions by systematically considering and documenting child’s needs
4. If applied regularly with the same child, may show a child’s progress over time in particular domains
5. May be helpful in community-level planning and resource allocation decision making
6. May reveal emergency situations (a score of 1 in any outcome area)
6 Core CSI functions (we think)
Probably inappropriate CSI uses
Targeting Unnecessarily complex
May lead to expectations of action/enrolment
1st contact with child may not be reliable
Evaluating regional or national program impact Children’s needs/status are assessed relative to their
local community, and not to national standards
Probably inappropriate CSI uses
Producing a single combined score for the child CSI assessment should be presented as 12
independent measures
Risk varies across domains
CSI scale values are not equal-interval, but ordinal
Evaluating implementing organizations
Back to Case Management tools generally…
Some key questions Are case management tools effective at improving
care decision making? For all types of case workers? Formal? Informal?
How does training in both case management and tool use factor?
What specifically about a case management tool improves care decision making? For whom?
Does the benefit outweigh the burden?
Some key questions II
Are CM tools useful in managing case workers?
Are some CM tools useful also for targeting beneficiaries, monitoring outputs (services delivered, and evaluating impact? What are the risks?
The research presented here has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement GHA-A-00-08-00003-00. Views expressed are not necessarily those of PEPFAR, USAID or the United States government.
MEASURE Evaluation is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University.