post construction support colombia
TRANSCRIPT
Impacts of post-construction support on the performance of rural water supply in Colombia
7 March 2012Water services that last
Water MDG
7 March 2012Water services that last
Background• IADB re-entering into rural water supply and sanitation• Encountering sustainability issues around rural water supply in Latin America
– Lack of statistics and data-bases on the state of rural water supply systems– Recognition of the need for post-construction support, but lack of detailed insight
into key characteristics: how, what, when, who?– Lack of quantitative data to support claims for post-construction support
• Request to IRC and CINARA to support this through research in Colombia:• Assess the effectivity and efficiency of various modalities for post-construction
support to community-based water providers on the quality and sustainability of the water services delivered
29 Agosto 2011Water services that last
Context• Different rural coverage figures depending on
definitions: 57% - 73%; if water quality is included in the definition, only 12%
• Decentralization since 1991:– Community-based service providers– Municipalities as guarantors of sustainable service
provision• Role of post-construction support not made explicit
– Some municipalities have taken this up; others not– National and regional government have set-up support
programmes– Civil society initiative (AQUACOL and coffee growers
association)
7 March 2012Water services that last
Methodology• Quantitative analysis to link characteristics of post-construction support to
service levels and performance and governance of service providers• Applied to 40 rural water systems, selected using stratified sampling in
Caldas, Cauca and Valle del Cauca, supported by 7 different post-construction support models:– Business Culture Programme – national government– Housing Secretariat of Caldas – departmental government– Health Secretariat of Cali – municipal government– Aguas de Manizales – urban utility– Aguas Manantiales de Pacora – urban utility– AQUACOL – association of community-based service providers– Coffee Growers Association Caldas – No post-construction support
7 March 2012Water services that last
Indicator sets
• Indicator sets for: service levels, governance and performance of service providers, support models
• Scoring tables– Comparison – Aggregation– Quantifying qualitative data– Data collection: 1 day per system
7 March 2012Water services that last
Service ladderCoverage (%)
Continuity of supply
Quality Net quantity received(l/p/d)
User satisfaction Total score Service level
>90 Equivalent to >23 hours/day
IRCA between 0 and 5% (No risks)
Between 130-170 l/p/d
80% of respondents satisfied with quality, quantity, continuity and tariff
More than 4,5 High
80-90 Between 20 and 23 hours
IRCA between 5,1% - 14% (low risk)
Between 100 -129 l/p/d, or between 171-200 l/p/d
70% of respondents satisfied with at least three of the indicators
Between 3,75 and 4,5
Acceptable
60-79 Between 12 and 19 hours
IRCA is measured but not met : 14,1% - 80 (Medium to high risk)
Bewteen 50-99 l/p/d or between 201 -250 l/p/d
50% of respondents satisfied with at least three of the indicators
Between 3 and 3,74
Deficient
<59 Less then 12 horas
No water quality analysis or IRCA very high, >80%
Less then 50 or more than 250 l/p/d, or no water quantity analysis
Less than 50% of respondents satisfied with three indicators
Below 3 Very deficient
7 March 2012Water services that last
Indicators for the governance and performance of service providers
• Three sub-categories– Internal governance and legality:
compliance with legal requirements, organisational structure, decision-making and accountability
– Administrative management (incl commercial aspects and accounting)
– Technical and operational management (including water resources management tasks)
• Total 21 indicators, each with a maximum score of 1
• Weighing factor per category
Performance level Score
High performance More than 80Acceptable performance
Between 60 and 79,9
Deficient performance
Between 40 and 59,9
Very deficient performance
Below 40
7 March 2012Water services that last
Indicators for post-construction support
• 3 characterizing variables without score (direct/indirect, demand/supply-driven, costs)
• 10 indicators with score to measure degree of formality
• Based on semi-structured interviews and survey
Score Performance levelMore than 6 HighBetween 4 and 6 MediumLess than 6 Low
7 March 2012Water services that last
Findings: service levels• Most limiting factors are
water quality and quantity:– Lack of information (22
systems had no data on quality; 9 had no data on quantity)
– Very high consumption levels
• Not necessarily a problem for users; their main reason for (dis)satisfcation is continuity
Qualification Number of systemsHigh 4Acceptable 16Deficient 15Very deficient 5
7 March 2012Water services that last
Findings: service provider performance
• Less than half (16) has an acceptable of high score for performance
• Above all high scores in administrative management (non-payment rate of only 15%)
• Low scores in “advanced” technical management, particularly catchment protection
Governance and legality
Administrative management
Technical management
Most common low scores
Customer relations
Gender balance in water committee
Register of materials
Capacity of the personnel of the water committee
Water metering
Catchment management
Most common high scores
Inter-institutional relationships
Renewal of water committee
Non-payment rates
Accounting
State of the infrastructure
Autonomous operations
7 March 2012Water services that last
Contextual factors• Settlement size has no impact on service
level; performance differs significantly according to settlement size
• Technology type impacts on water quality and performance: more complex systems have better management
• Age of systems mainly affects performance: older systems have significantly better management
7 March 2012Water services that last
Menos de 100 (n=14) 101-300 (n=16) Mas de 300 (n=10)
Calificacion promedia en gobernanza in-terna y legali-dad
15.1836666666667 19.7665416666667 23.7754
Calificacion promedia en gestion admin-istrativa
17.7193163265306 20.8907678571429 25.9974
Calificacion promedia en gestion tec-nico-operativa
13.9986 16.4566875 23.19768
5.0
15.0
25.0
35.0
45.0
55.0
65.0
75.0
Number of user families
Sco
re o
n p
erf
orm
an
ce
7 March 2012Water services that last
Findings: relation between performance of service provider and service level
• Low level of correlation• Mainly at the extremes: best
performing service providers have best services and worst performers have poorest service, but blur in the middle
• Performance of service providers closest related to the indicator of water quality
20 30 40 50 60 70 80 90 1000
0.51
1.52
2.53
3.54
4.55
R² = 0.156410744594906
Score of service provider performance
Scor
e on
serv
ice
leve
l
7 March 2012Water services that last
muy deficiente (n=6)
deficiente (n=18)
aceptable (n=13)
alto (n= 3)0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
Calificacion promedia de satisfaccion de usuarios
Calificacion promedia de calidad
Calificacion promedia de cantidad
Calificacion promedia de continuidad
Calificacion promedia de Cobertura
Performance of service provider
Scor
e on
serv
ice
leve
l
7 March 2012Water services that last
Findings: impact of post-construction support
• Except for 2, all systems had received some external support the last year, some ad hoc, some structural post-construction support
• Analysis with original classification and after re-classifying• Post-construction support does lead to statistically significant better
performance of service providers, but not to better service levels• But, even with support the average performance is barely acceptable
Number of systems Average performance score of the service
provider
Average score for service level
Systems linked to post-construction support model
27 61.1 3.63
Systems without structured post-construction support
13 48.1 3.52
7 March 2012Water services that last
Findings: impact of post-construction support
Sin mod-elo de apoyo
Aguas Manan-
tiales Pa-cora
Comite de Cafeteros
UES Rural Cali
Aguas de Manizales
AQUACOL Secretaria Vivienda Caldas
Programa Cultura
Empresar-ial
Califi-cacion pro-media de gober-nanza y le-gali-dad
17.4056666666667
14.9985 17.6834166666667
18.2944666666667
17.776 23.7013333333333
25.1826666666667
25.553
Califi-cacion pro-media en ges-tion admin-istra-tiva
18.5695714285714
20.7122142857143
20.1170357142857
19.9027714285714
19.5218571428571
25.5926785714286
21.5851428571429
30.9492857142857
Califi-cacion pro-media en ges-tion tec-nico-opera-tiva
14.6652 14.6652 17.16495 17.06496 18.9981 17.6649 20.2202 26.8862
5.0
15.0
25.0
35.0
45.0
55.0
65.0
75.0
85.0
Ca
lifi
ca
cio
n p
ro
me
dia
de
de
se
mp
en
o d
el
pre
sta
do
r
7 March 2012Water services that last
Findings: impact of post-construction support
• Factors explaining degree of impact of post-construction support:– None of the originally identified factors appeared to be
significant (neither scored nor other explicative variables)– Low correlation with the degree of formality of the
support model (staff profiles and institutionalization)– Frequency of support activities– Inter-institutional set-up of models (models acting as node,
referring to dedicated entitites)
7 March 2012Water services that last
0 (n=2) 1-4 (n=16) 5-8 (n=13) 9-12 (n=4) Más de 12 (n=5)0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
• Performance of service provider in relation to frequency of support activities
7 March 2012Water services that last
Conclusions on the use of indicator sets
• The sets at system level worked well; above the use of scoring tables was considered positive both by water committees and officials:– Main drawback is that standard regression analysis is not possible with
categorised data– 5 + 21 indicators requires over 75 sub-indicators! – easy to collect much more
information than needed– Key indicator missing is information management
• The set of variables for the post-construction support model can identify the degree of formality of the model and describe it, but is not a set that can explain or predict its possible impact
7 March 2012Water services that last
Conclusions• Half of the surveyed systems had acceptable
levels of service; more than half had under-performing service providers
• Low scores particularly due to lack of data on water quality and quantity, and poor technical management
• Relatively high scores in financial management and organisation – result of high emphasis given to this in Colombia
• In different degrees explained by contextual factors (settlement size, type of technology, age)
• Low correlation between performance of service provider and service received
7 March 2012Water services that last
Conclusions• Nearly all systems receive some support, albeit it ad hoc• Those with structured post-construction support have
significantly better performing service providers; impact on service levels is positive but not significant
• High variability within and between support models: not clear that one model works better than another; rather look at the underlying factors that explain effectivity of models: – degree of instutionalisation– frequency of support– inter-institutional character of support model
• No clear data on costs or personnel to draw conclusions on efficiency
7 March 2012Water services that last
Recommendations
• Strengthen performance of providers and indirectly service levels, partially through post-construction support
• Clarify and specify mandates for post-construction support, whilst recognising the variety of mechanisms that already exists, with key role for municipalities
• Focus on institutionalizing support roles of those models that score low currently
• Use monitoring as tool to identify both generic (water quality, technical management) and provider-specific support needs
• Strengthen devolution of information to service providers
