Nordlund 01

Samantha NordlundLast Mile Project; Advisory Report13 November 2013

Using Technology to Monitor & Evaluate

Project CollaboratorsDAT: Jamie Pfaff, Andrew Whittington, Woohyun ChunIndependent International Consultant: Gopalan Ba lagopalIndian Institute of Management, Kozhikode: Pranav Mehta, Durga lakshmi , Sudhansu Dutta , Dipankar Biswas , andShaheen Mal ik

Macro-- Team Research: Assessments on experiences in other parts of the world in reaching hardto reach communities.

Micro-- My Research: International examples where modern technology (i.e. internet and cellphones) is used to monitor and track progress of social sector projects in hard to reachcommunities.

Context: The topography of the tribal communities in Wayanad are mountainous and forested,making monitoring and evaluation difficult. Wayanad has 177 Anganwadi Centres through whichthe child nutrition program is extended into the community, and there is information that needsto be regularly collected from these centres (e.g. how many beneficiaries of Anganwadi Centerservices, performance of the preschool education component, administrative and logistical issuesaffecting the Center, etc). Our international consultant on the ground, Gopalan Balagopal, explainsthat “all reporting that flows from [...] centres is paper based and is consolidated at the Project,district, state and national levels. This is inevitably a slow process and the use of the consolidatedinformation for analysis and action at the Project and district level is marginal.” He remarks thatreporting is “transitioning to an electronic database though the systems are yet to be fullyestablished.” As the centres in Wayanad and across Kerala look for new ways to collectinformation, the Notre Dame DAT has been asked to provide research of international examples ofhow modern technology can be used for monitoring and evaluation of social sector projects insimilar hard to reach communities.

Key terms useful for further research:M&E: Monitoring and eva luation of socia l sector projectseHealth/e-Health: Heal thcare practice supported by electronic processes and the internetmHealth/m-Health/mobile health: A subsegment of eHealth, mHealth i s more speci fica l ly the use of mobi le devicesin col lecting aggregate and patient level hea l th datamDevelopment: The use of mobi le devices in the development sphereICT/ICT4D: Information and communication technology/information & communication technology for developmentTEME: Technology Enabled M&E

CASE 1: CLEAR South Asia ReportIn conjunction with the Abdul Latif Jameel Poverty Action Lab (J-PAL), Innovations for

Poverty Action (IPA), Fieldata, and IFMR Trust, the Regional Center for Learning on Evaluation andResults of South Asia (CLEAR - South Asia) has put together a publication by the name“Mobile-Based Technology for Monitoring and Evaluation.” In the words of CLEAR, this publicationis “intended to inform would-be users on technology options” and act as a reference guide forunderstanding the “application and operationalization of TEME to their projects and programs.”

Nordlund 02

The guide highlights the “latent potential of TEME to improve data quality, speed andtransparency, leading to better implementation of projects, evidence-based decision making, andeventual development outcomes. ”1

I have included reference to this report for a number of reasons. Not only does it boast anumber of internationally esteemed research institutions as contributors, the CLEAR report iswonderfully thorough, informative, and concise. It is broken down into three sections: A QuickStart section on the applicability of mobile technology to projects and how to initiatemobile-based monitoring and data collection tools; a section on Mobile Technology Options andOpportunities which lists the many components of a mobile-based data collection system,including the possible devices, features and functionalities, from low-end phones to smartphonesand tablets; a final section on Implementing Mobile Technology in M&E which detailsimplementation issues, including but not limited to estimating costs, planning timelines, trainingfield staff and ensuring data quality. CLEAR’s report also mentions practical considerations thatcan dampen the success of digital data collection, such as limited network coverage, lowbandwidth difficulties, literacy issues, dialects, and gender disparity. The publication supportsresearch with case studies that highlight TEME initiatives in various countries and contexts.

One such case study details the use of mobile phones by the Community AssistantsInitiative in Ghana, launched by the Ghana Education Service in 2009. The project was undertakento improve educational quality in primary schools, and entailed the tracking of about 25,000students over the course of two years. Over one hundred field enumerators were equipped withphones to photograph and track the children each year. In an effort that resonates with the LastMile Project, “a database with individual pictures of the students by school was collated inreal-time on a web-server. The list with photographs became an audit as well as tracking tool forthe project.” This 2009 initiative in Ghana has components similar to the Last Mile Project’s child2

nutrition programs in Wayanad, which hopes to regularly collate beneficiary information fromAnganwadi Centres scattered over the district. Furthermore, the 2013 CLEAR report would be anintegral resource and guide should the Last Mile Project choose to move forward in implementingmobile technology in its M&E.

Links :http://www.theclearinitiative.org/mobile-basedtechnology.html

http://www.theclearinitiative.org/mobile-based-tech.pdf

CASE 2: MOTECHThe Mobile Technology for Community Health Project (MOTECH) in Ghana is an initiative

to decrease the infant mortality rate in Ghana through mobile technology. In addition to sendingmessages to pregnant women throughout their pregnancy and during the child’s first year, theservice “enables community health workers to use mobile phones to enter information about thepatients they have seen and the services they have delivered, increasing efficiency and accuracy onthe healthcare side of the equation.” The program now boasts over 25,000 people registered for3

the service, and about 300 community health workers equipped with phones to track patients.Through CARE’s programs in Bihar, India, pregnant women and children under one are

enrolled in the MOTECH system and “tracked along the continuum of care.” Mobile phones alsoprovide “built-in checklists to help [field workers] follow the most appropriate protocol during

1 http://www.theclearinitiative.org/mobile-basedtechnology.html2 http://www.theclearinitiative.org/mobile-based-tech.pdf3 The Stanford Social Innovation Reviewhttp://www.ssireview.org/blog/entry/scale_can_happen_the_motech_experience

Nordlund 03

their interactions with women.” Furthermore, “mobile phones of health workers in the samecatchment area are synchronized with each other, enabling information sharing and improvedcoordination. The application generates a list of patients in each health worker’s catchment areawho are due for immunization, and supervisors receive real-time reports on services delivered byhealth workers.“4

The MOTECH intervention is notable for its “eye toward long-term scale and replication,”and emphasis on building components that could be “reused in other geographies and otherhealth domains.” The Grameen Foundation notes that “there is nothing maternal-health orGhana-specific about the technology components we are building,” and worked with Dimagi,InSTEDD and other organizations that had “complimentary [sic] technologies” to “make ourservices interoperable,” all of which resulted in the MOTECH Suite. The Suite maintains core5

funding from the Bill and Melinda Gates Foundation, as well as project-specific funding fromorganizations such as Johnson & Johnson and USAID. This suite of technology services hasexpanded the use of MOTECH to other areas of Ghana, as well as development initiatives acrossIndia. MOTECH also notably helps health workers track clients in World Vision programs in sevencountries, from Afghanistan to Zambia. The technology can be locally customized as needed, and6

supports multiple devices (from low-budget phones to tablets and PCs) as well as messageformats (SMS, voice, data), suggesting the possibility for easy incorporation into the Last Mile’s7

projects in India. The preceding information was found in part through the Stanford SocialInnovation Review, which has a special series on scaling up mobile health initiatives that can beaccessed here.

Links :http://www.ss i review.org/blog/entry/sca le_can_happen_the_motech_experience#related_storieshttp://www.impatientoptimists .org/Posts/2013/07/Making-a-Di fference-with-Mobi le-Technology

http://www.motechsui te.org/http://www.motechsui te.org/images/stories/MOTECH_Suite_Overview_Nov2012.pdf

CASE 3: Water for People and Akvo’s FLOWWater for People and Akvo have joined forces to address the issue of monitoring

development projects through the creation Akvo FLOW (Field Level Operations Watch) in 2010.Created for the purpose of “addressing the need to replace cumbersome paper-based monitoringsurveys and the delay in manually compiling the information,” FLOW was designed “to provideaccountability and transparency to donors and the public through fast data collection, surveyflexibility, analytical tools for data-driven decision making, and map-based reporting of results.”8

Through using Android cell phones, GPS and Google Earth software, FLOW allows volunteers,partners, and community members to record vital data from water points around the world. Thisdata includes location, pictures, and the service status of the wells/pumps. This information isthen displayed online, signaling whether a project is functional, broken, or in need of maintenance.

FLOW maintains partners such as the World Bank, Water for People, A Child’s Right, andthe International Water and Sanitation Center IRC, and is currently used around the world in 17

4 http://www.motechsuite.org/images/stories/MOTECH_Suite_Overview_Nov2012.pdf5 http://www.ssireview.org/blog/entry/scale_can_happen_the_motech_experience6 http://www.impatientoptimists.org/Posts/2013/07/Making-a-Difference-with-Mobile-Technology7 http://www.motechsuite.org/images/stories/MOTECH_Suite_Overview_Nov2012.pdf8 http://www.waterforpeople.org/flow-mapping/

Nordlund 04

countries: Ethiopia, Ghana, Liberia, Malawi, Rwanda, Senegal, Sierra Leone, Uganda, Nepal, India,Bolivia, Dominican Republic, Ecuador, Guatemala, Honduras, Nicaragua, Peru.9

The FLOW project map is impressive, as one can view approximately 40 data pieces abouta single well point in Bolivia, from numerical factors (e.g. “How many people use this watersource?”...“98”) to the type of community or name of local contact. The format of FLOW is worthlooking into for disseminating information regarding the Last Mile Project’s interventions inWayanad, Palghat and Idukki. So far, the primary applications of FLOW have been in thesanitation, hygiene and water sector, but “smaller projects outside the sector have demonstratedits flexibility and potential for diverse applications,” potentially opening the door for the Last MileProject’s type of nutritional interventions in India.10

Links :http://www.waterforpeople.org/flow-mapping/

http://watermapmonitordev.appspot.com/http://www.waterforpeople.org/assets/pdfs/flow/akvo-flow.pdf

http://www.waterforpeople.org/everyone/monitoring-eva luation.html

Additional Resources for Further Research:● A 2013 World Bank Study: “Challenges and Opportunities of Mobile Phone-Based Data

Collection: Evidence from South Sudan”● The 2012 Journal of Health Informatics in Developing Countries report titled “A Review on

Challenges in Implementing Mobile Phone Based Data Collection in Developing Countries”● A 2013 paper published by the Global Impact Investing Network in partnership with

Acumen, the Aspen Network of Development Entrepreneurs, Echo Mobile, and GrassrootsBusiness Fund entitled “Collecting Impact Data Using Mobile Technology”

● The United Nations Global Pulse Project, an online directory of projects that use mobilephones for data collection. The inventory exists as a living document that is regularlyupdated, accessible in a Google Spreadsheet here.

9 http://www.waterforpeople.org/assets/pdfs/flow/akvo-flow.pdf10 http://www.waterforpeople.org/assets/pdfs/flow/akvo-flow.pdf