Sustainability Check: An Innovative Approach to WASH Sustainability

Monitoring

Rochelle Rainey, USAIDMonitoring WASH Service Delivery

Addis Ababa, Ethiopia April 9-11, 2013

Objectives of the Sustainability Check• Quantification of qualitative factors • Flag critical areas of strengths and

weakness and trends to inform decision-making

• Allow comparison within same interventions across different locations

• A re-usable/adaptable framework for different types of WASH interventions

• Balance ease of use with intellectual rigor for validity and reliability

Design and development of tool

Service provider level

District or municipal

level

National level

Impact on sustainability at different levels

Literature review to identify indicators and criteria - 4 factors of sustainability

INSTITUTIONAL MANAGEMENT FINANCIAL TECHNICAL

INSTITUTIONAL

MANAGEMENT

FINANCIAL

TECHNICAL

INDICATOR 2

Q A

= 100

INDICATOR 3

INDICATOR 1Q B

Q C

Q D

Indicators defined for different scales:

1. Service Provider2. Service Authority (District)3. National

Points for questions for each indicator must sum

to 100

Component overview of the tool

Example of Indicators: Community piped water system

Statistical design and sampling

• Identify number and location of interventions and beneficiaries• Tradeoff between resources and data quality • Try to capture diversity of communities over ensuring

“statistical significance” of results

Application of Sustainability Check

• Dominican Republic - 5 interventions/63 indicators• Ghana - 4 interventions/61 indicators• Philippines - 7 interventions/103 indicators

• Surveys in 144 (out of 496) communities and 2,332 households

• Four month field process including development and testing of country specific frameworks

• One gnarly spreadsheet

Sustainability Check Tool

Community

HY-HWP-M-SP1a Is there a health promoter?

HY-HWP-M-SP1c HP makes visits

HY-HWP-M-SP1d HP give talks

HY-HWP-F-SP1a Use soap to wash hands

HY-HWP-F-SP1b House has soap

HY-HWP-F-D1a Community sells soap

HY-HWP-F-D1b Buy femine products

HY-HWP-F-D1c Dry dish appropriately

HY-HWP-F-D1d Buy other hygiene products

HY-HWP-T-SP1a Correctly wash hands

HY-HWP-T-SP1b Knows when to wash hands

QA-1 Quality of Responses

QA-2 Understood questions

San Jose de Ocoa SI SI SI SI NO SI SI N/A SI SI SI A A

San Jose de Ocoa SI SI SI SI SI SI SI N/A SI SI NO A A

San Jose de Ocoa SI SI SI SI SI SI SI N/A SI SI SI A A

San Jose de Ocoa NO NO NO SI SI SI SI N/A SI SI SI A A

San Jose de Ocoa NO SI SI SI NO SI SI N/A SI SI NO A A

San Jose de Ocoa NO NO NO SI SI SI SI N/A SI SI SI A A

San Jose de Ocoa SI NO SI SI NO SI SI N/A SI SI SI A A

San Jose de Ocoa SI SI SI SI SI SI SI N/A SI SI SI A A

San Jose de Ocoa SI SI SI SI SI SI SI N/A SI SI NO B A

San Jose de Ocoa SI SI SI SI SI NO SI N/A SI SI NO B B

San Jose de Ocoa SI SI SI SI NO SI SI N/A SI SI NO A A

San Jose de Ocoa SI SI SI SI SI SI SI N/A SI SI SI B C

San Jose de Ocoa SI SI SI SI SI SI SI N/A SI SI SI A B

San Jose de Ocoa SI SI SI SI SI SI SI N/A NO SI NO A A

Batey 3 SI SI SI SI SI SI SI SI SI SI SI A A

Batey 3 SI SI SI SI NO SI SI NO SI SI SI A A

Batey 3 SI SI SI SI SI SI SI SI SI SI SI A A

Batey 3 SI SI SI SI NO SI SI SI SI SI NO A A

Batey 3 SI SI SI SI SI SI SI NO SI SI SI A A

Batey 3 SI SI SI SI SI SI SI NO SI SI NO A A

Batey 3 SI SI SI SI SI SI SI SI SI SI SI A A

Batey 3 SI SI SI SI SI SI SI NO SI SI SI A A

Batey 3 SI SI SI SI SI SI SI NO SI SI SI A A

Batey 3 SI SI SI SI NO SI SI SI SI SI NO A A

TRANSCRIBED INTO PIVOT TABLES AND COMPOSITE

DATA SHEETS

ENTRY OF RAW DATA SCORES PER INDICATOR

QUESTION FROM H/HOLD AND HIGHER LEVELS

Source Code Description AB-4 AB-6 Batey 3 Batey Margarita Bonagua Cienfuegos Clavijo Arriba Consuelito

1-HH HY-HWP-F-D1a Community sells soap 96% 74% 100% 100% 100% 100% 95% 100%

1-HH HY-HWP-F-D1b Buy femine products 96% 21% 100% 100% 100% 97% 89% 98%

1-HH HY-HWP-F-D1c Dry dish appropriately 36% 0% 59% 17% 75% 52% 53% 28%

1-HH HY-HWP-F-D1d Buy other hygiene products 96% 58% 100% 100% 100% 100% 89% 100%

1-HH HY-HWP-F-SP1a Use soap to wash hands 96% 74% 100% 97% 94% 97% 95% 94%

1-HH HY-HWP-F-SP1b House has soap 80% 26% 78% 67% 81% 87% 84% 80%

1-HH HY-HWP-M-SP1a Is there a health promoter? 80% 74% 100% 100% 13% 71% 100% 88%

1-HH HY-HWP-M-SP1c HP makes visits 76% 63% 93% 97% 13% 71% 100% 80%

1-HH HY-HWP-M-SP1d HP give talks 80% 68% 100% 100% 6% 71% 100% 88%

1-HH HY-HWP-T-SP1a Correctly wash hands 96% 68% 96% 93% 100% 100% 89% 96%

1-HH HY-HWP-T-SP1b Knows when to wash hands 56% 16% 74% 27% 50% 77% 32% 40%

1-HH QA-1 Quality of Responses 92% 63% 74% 60% 75% 94% 68% 86%

1-HH QA-2 Understood Questions 92% 68% 70% 57% 75% 90% 79% 76%

2-SP Code SP SP respondent code 19 22 2 30 11 13 12 25

2-SP HY-HWP-M-D1a HP are supervised 100% 100% 100% 100% N/A 100% 100% 100%

2-SP HY-HWP-M-D1b Is there support to HP when requested? 100% 100% 100% 100% N/A 100% 100% 100%

2-SP HY-HWP-M-D1c Is there support given systematically? 100% 100% 100% 100% N/A 100% 100% 100%

2-SP HY-HWP-M-D1d Is there retraining performed? 100% 100% 100% 100% N/A 100% 100% 100%

2-SP HY-HWP-M-SP1a Is there a health promoter? 100% 100% 100% 100% 0% 100% 100% 100%

2-SP HY-HWP-M-SP1b Health promotors monitor health practices in home? 100% 100% 100% 100% N/A 100% 100% 100%

3-DL District District San Pedro San Pedro Bahoruco San Pedro Repadero Santiago 0rthwest Region San Pedro

3-DL DL Code Person Interviewed 12 12 17 12 7 8 16 12

3-DL HY-HWP-F-N1a Local budget for implementing programs 0% 0% 100% 0% 0% 100% 100% 0%

3-DL HY-HWP-F-N1c Social program for hygiene products 0% 0% 100% 0% 0% 100% 0% 0%

3-DL HY-HWP-I-D1a Local govnt involved in HWP 100% 100% 0% 100% 0% 100% 0% 100%

3-DL HY-HWP-I-D2b Local govn't is linked to Ministry of Health 100% 100% 0% 100% 0% 100% 0% 100%

3-DL HY-HWP-I-D3cLocal govn't activities coordinated with federal ministry activies 100% 100% 0% 100% 0% 100% 0% 100%

3-DL HY-HWP-I-D4d Govn't provides personell or resources for HWP 100% 100% 100% 100% 0% 100% 0% 100%

3-DL Level Regional Govt, Provincial Govt, Municipality Govt, NGO

NGO (MUDE-Save)

NGO (MUDE-Save)

Provincial

NGO (MUDE-Save)

Municipal

NGO

PCDR

NGO (MUDE-Save)

4-NL HY-HWP-F-N1b

Are there supplementary National funds available for hygiene and handwashing promotion?

0% 0% 0% 0% 0% 0% 0% 0%

4-NL HY-HWP-I-D2b

Does the district authority liaise with relevant ministries, such as ministry of health regarding hygiene promotion? ) 100% 100% 100% 100% 100% 100% 0% 100%

4-NL HY-HWP-I-N1aIs hygiene promotion a recognized government policy, overseen by specified ministry (Ministry of Health)? 0% 0% 0% 0% 0% 0% 0% 0%

4-NL HY-HWP-I-N1b

Is handwashing part of the government's hygiene policy?

0% 0% 0% 0% 0% 0% 0% 0%

4-NL HY-HWP-I-N2a

Is there a national hygiene promotion/behavior change program?

100% 100% 100% 100% 100% 100% 100% 100%

4-NL HY-HWP-I-N2b

Does the national hygiene promotion/behavior change program clearly designate responsibilities in national ministries?

50% 50% 50% 50% 50% 50% 50% 50%

GRAPHICS PRODUCED TO SHOW AGGREGATE SCORES BY FACTOR OR

INTERVENTION COMMUNITY

Service Provider Service Authority /District Central

Service Provider Service Authority /District Central

Service Provider Service Authority /District Central

Service Provider Service Authority /District Central

Aggregation by scale- Ghana

Thanks to:The partners and communities in the 3 pilot countriesThe Rotary Foundation and the many Rotary Clubs

involvedAguaConsultUSAID colleagues in Washington and the 3 pilot

countries

Rochelle Rainey, USAIDrrainey@usaid.gov