# ex post evaluation of cohesion policy intervention 2000-2006 financed by the cohesion fund (incl....
TRANSCRIPT
#
Ex Post evaluation of cohesion policy intervention 2000-2006 financed by the Cohesion Fund (incl. ISPA)
WP C – CBA of Environmental Projects
Workshop with Member States,Brussels 3 February 2011
Session 2: Introduction and findings of 10 Environment Projects of WP C
Christina van Breugel, Tine Skyggebjerg, Szabolcs Szekeres, Birgitte Holt Andersen (COWI)
Davide Sartori, Silvia Vignetti (Csil)
13 Feb 2011 Ex post eval cohesion fund_DG Regio
#
Overview of the 10 case studies
23 Feb 2011 Ex post eval cohesion fund_DG Regio
Waste management
Water management
Waste water management
01 Bulgaria
03 Pilsen
17 Barcelona
06 Crete
09 Zaragosa
27 Hungary
22 Dublin
13 Madrid
29 Poland
50 Portugal
#
Outline of presentation
1. Presentation of the Ex ante situation
2. Reporting of results of ex post CBA
3. Main differences from ex ante to ex post
4. Typical components and typical benefits
Waste management projects
Water Management projects
Waste water projects
5. Main findings related to:
Ex ante CBA
Ex post CBA
33 Feb 2011 Ex post eval cohesion fund_DG Regio
#
1. Ex ante situationMain driver for project initiation and project context
43 Feb 2011 Ex post eval cohesion fund_DG Regio
CASES Lega
l com
plia
nce
Envi
ronm
enta
l
Heal
th
Risk
miti
gatio
n
Oth
er
Self
cont
aine
d
Upg
radi
ng e
xisti
ng
infr
astr
uctu
re
Part
of
'Mas
ter P
lan'
01 Bulgaria/Waste Mngt X X X X03 Czeck Rep/Pilsen: Combined water X X X X06 Greece/Crete: Waste Mngt X X09 Spain/Zaragoza: Water supply X X X X13 Spain/Madrid: Waste/Energy recovery X X17 Spain/Barcelona: Waste water X X X X22 Ireland/Dublin: Waste water X X X X27 Hungary/Szeged: Waste water X X29 Poland/Szczecin: Waste water X X X X X50 Portugal/Lipor: Waste management X X
8 6 1 1 2 4 3 8
Main driverProject context
MAIN DRIVER
Legal compliance 8Environmental 6
Health 1Risk mitigation 1
Other 2
No of projects
PROJECT CONTEXT
Self contained 4Upgrading existing 3
Part of 'Master Plan' 8
No of projects
#
1. Ex ante situationEx ante assumptions, financial and economic analysis
53 Feb 2011 Ex post eval cohesion fund_DG Regio
CASES01 Bulgaria/Waste Mngt03 Czeck Rep/Pilsen: Combined water06 Greece/Crete: Waste Mngt09 Spain/Zaragoza: Water supply13 Spain/Madrid: Waste/Energy recovery17 Spain/Barcelona: Waste water22 Ireland/Dublin: Waste water 27 Hungary/Szeged: Waste water29 Poland/Szczecin: Waste water50 Portugal/Lipor: Waste management
QUALITY OF FINANCIAL ANALYSIS
High quality 2Medium quality 7
Poor quality 1
No of projects
Very
reas
anab
le
Reas
onab
le
Wea
k or
misl
eadi
ng
High
qua
lity
Med
ium
qua
lity
Poor
qua
lity
High
qua
lity
Med
ium
qua
lity
Poor
qua
lity
X X XX X X
X X XX X X
X X XX X XX X X
X X XX X X
X X X2 6 2 2 7 1 3 2 5
Ex ante assumptions
Quality of ex ante financial
analysis
Quality of ex ante
economic analysis
EX ANTE ASSUMPTIONS
Very reasanable 2Reasonable 6
Weak or misleading 2
No of projects
QUALITY OF ECONOMIC ANALYSIS
High quality 3Medium quality 2
Poor quality 5
No of projects
#
1. Ex ante situationThe use of CBA for project formulation and decision
One thing is the quality of the ex ante – another thing is HOW the CBA was actually used for project formulation and decision making
CBA used for project formulation?
– NO - Why not?
(a) since most projects emerged to comply with legislation
(b) project is part of a larger context (Master Plan)
CBA used as basis for decision making?
– Not really, perhaps due to the timing of CBA
63 Feb 2011 Ex post eval cohesion fund_DG Regio
'The city authorities opted
for this project largely
because they were told
that only such an integrated project would be
eligible for financing'
#
1. Ex ante situationThe timing of the CBA
73 Feb 2011 Ex post eval cohesion fund_DG Regio
CBA
Project identificationProject formulation
EIA process/develpment consentsApplication
TenderingConstruction Operation
3-5 years 3-5 years 30 years
The project is usually already a part of a process or part of a regional or local Master Plan
The Project needs to be seen in this context
#
2. Reporting on results of ex post analysis
Most projects investigated appear to be sensible investments as they provide for fundamental EU environmental infrastructures ('needs to have'/legal compliance)
In economic terms however only one project generate positive ex post ENPV meaning that economic benefits justify the costs
Few problems with either over-capacity or under-capacity
The implemented technical solutions are with one exception sensible and reasonable
Size of investment costs appear more or less reasonable according to our technical experts
There are however unrealised benefits in some of the projects
Wider benefits include: improved environmental awareness among citizens, political benefits, enabling benefits, etc.
83 Feb 2011 Ex post eval cohesion fund_DG Regio
#
2. Reporting on results of ex post analysisOutcome of the projects (1/2)
93 March 2010 Ex post eval cohesion fund_DG Regio
Project outcomes
Fulfillment of objectivesScore 1-5
Financial sustainability Direct benefits Unrealised benefitsHindrances for
realizing benefits
Other project outcomes/unintended
effects
01 Bulgaria 4
Only the operational cost can be financed by the waste management tax No re-investement are included for e.g. sealing of the cells
Reduction of illegal waste disposal _ _A first step toward a modern waste handling management system
03 Czeck Rep/Pilsen: Combined water
4
Yes due to the tariffs and since the affordability analysis of the total tariffs showed no risk of payment problems among household consumers.
Much improved quality of drinking water + reliability of supply of tapped waterReduced pollution of the river due to reduced risk of overspill of sewage systemConnection of approximately 1000 households to the central sewage system
Upgrading of the river environment (underway)
Perhaps slowed down now due the financial crisis
Enabled other projects to emerge, e.g. GreenwaysInstitutional learning, e.g. establishing of UKEP
06 Greece/Crete: Waste Mngt
4UncertainOP cost much higher than budgetRevenues lower than budget
Reduction of illegal waste disposal of 90000 tonnes of waste a yearRecycling of 30.000 tonnes of waste a year
Use of compost for soil enrichmentReduced use of pesticidesRefined sorting of waste100% waste management
Lack of awareness among farmersLack of education of households to sort waste at source
Waste management know-how to benefit rest of CreteIncreased environmental awareness among citizens
09 Spain/Zaragoza: Water supply
5The project is sustainable thanks to full cost recovery tariffs
Significant improvement in water quality _ _
Reduction of potabilization treatment costs for municipalities. the project has also indirectly induced the municipality of Zaragoza to undertake actions that have radically reduced water losses and modified the behavior of the users, with beneficial effects on environment and safety of water supply.
13 Spain/Madrid: Waste/Energy recovery
5
The energy generation component generate a surplus. While the other components require operational funding form the municipality.
Closing off illegal landfill - stopping negative environmental impactsEnergy generation
Unutilized synergies (energy plant) with other facilities of the combined Madrid management facility
Contractual bindings of current contract with different operators
An important part of the combined Madrid Waste Management facility
#
2. Reporting on results of ex post analysisOutcome of the projects (2/2)
103 March 2010 Ex post eval cohesion fund_DG Regio
Project outcomes
Fulfillment of objectivesScore 1-5
Financial sustainability Direct benefits Unrealised benefitsHindrances for realizing
benefits
Other project outcomes/unintended
effects
17 Spain/Barcelona: Waste water
5
Based on the financial indicators, the project is not sustainable. No income is generated. However; the economic assessment shows a sustainable investment.
Reduced risk of flooding (giving positive effects on quality of bathing water, traffic congestion and reduction of damage costs)Improved know how of water management. Qualitative benefits includes access to ground water for irrigation and street cleaning, improved ecosystem in marine water, public information activitiesand cleaner streets.
_ _
A lot of knowledge and experience has been gathered. The project has many visits from delegations from other cities and countries. There has been a lot of awareness rising among the population.
22 Ireland/Dublin: Waste water
3
The Project capacity is based on the information from the time of the application - and now it requires an upgrading to cater for unexpected increased demand - therefore it is not financially sustainable and requires the identification of further investment funds. Further, a revision of the user fee structure is needed.
The direct benefits include an increase in housing permissions and recreational benefits, in particular in connection with cleaner Dublin Bay beaches.
_
The design capacity of the new infrastructure is too low and it does not comply with the requirements of the newly designated sensitive waters. Hence, further investments are needed to fully reap the benefits.
_
27 Hungary/Szeged: Waste water
5
The project is financially sustainable despite its poor financial results, attributable to the high capital costs. But the fixed assets are not in the books of the operator. Tariffs are sufficent to cover operating and reinvestment costs and are inflation indexed.
The direct benefits are the extension of the sewage system to new users, who therefore avoid the cost of dealing with their own individual sewage solutions.
The objective of a cleaner Tisza River is unrealized because the dillution effects of the river are so strong (2400:1). The effects of the project might be more directly felt in the future, when incoming river water will be cleaner (icluding that of the Maros, which arrives from Romania)
_
The project is excellently run. There has been some awareness rising among the population.
29 Poland/Szczecin: Waste water
5
The project appears to be financially sustainable as user fees have been allowed to increase in line with the increase in the quality of service provision.
Direct benefits to the inhabitants of Szczecin due to improved quality of waste water and water supply services, and from improved environmental conditions of the Oder River.
Since the new infrastructure has only been in operation for aroumnd a year, it is still early days for estimating e.g. possible impacts on business and housing developments.
_Szczecin is no longer a HELCOM hotspot
50 Portugal/Lipor: Waste management
5
Financial sustainability is insured either by green or "guaranteed" administrative prices and close control of costs
Comprehensive and efficient management of urban solid waste generated in the second largest conurbation of the country with a population in excess of one million inhabitants
-
Usual implementation delays of a fairly large and multisectoral investmentAcceptance of compost quality by farmersAwareness to waste source separation + recycling of the public at large
Not detected
#
3. Main differences from ex ante to ex post
Financial results
– higher operational costs
– lower operational income
– project delays
– investment costs overrun
113 Feb 2011 Ex post eval cohesion fund_DG Regio
Economic results
– overestimated or arbitrary economic benefits
– unrealised benefits
#
3. Main differences from ex ante to ex postResults of financial and economic analysis (1/2)
123 Feb 2011 Ex post eval cohesion fund_DG Regio
EX ANTE
•The investments has risen due to budget overruns – some ex-post CBA are only done for some of the component and are therefore not comparable
•The FNPV are small or even negative – the negative results are larger in the ex-post analyses - no good business cases have been identified
•The ENPV are more positive in the ex-ante analyses than the ex-post CBAs – When complying with legislation a less positive ENPV could be accepted as it is part of a higher level objective of generally improving the environment
#
3. Main differences from ex ante to ex postResults of financial and economic analysis (2/2)
133 Feb 2011 Ex post eval cohesion fund_DG Regio
• Few B/C ratios above 1
• A number of the benefits have only been qualitatively described in the CBAs due to lack of benefit estimates
•A number of projects have experienced overcapacity
• Barcelona – improved marine water quality
•Hungary – covers just a section the Tisza river
•Bulgaria - is part of a master plan of national waste handling
#
4. The typical components and typical benefits of the different types of projects
Waste management
Drinking water management
Waste water treatment
143 Feb 2011 Ex post eval cohesion fund_DG Regio
# 153 Feb 2011 Ex post eval cohesion fund_DG Regio
Waste Management
# 163 Feb 2011 Ex post eval cohesion fund_DG Regio
Water supply
# 173 Feb 2011 Ex post eval cohesion fund_DG Regio
Waste Water
#
5. A) Main findings related to ex ante CBA
The main CBA ex ante issues:
not integrated in the decision process
not looking at individual components
missing the bigger pictures (e.g. synergies, risk un-realised benefits)
183 Feb 2011 Ex post eval cohesion fund_DG Regio
Recommendations:
Do the CBAs much earlier in the process. A solid CBA should precede doing the final technical design of the project
CBA thinking should apply to the selection of alternatives, prior to the final design of the project
CBA to be related to the Master Plan context
Cost efficiency analysis could be considered for 'need to have' projects
#
5. B) Main findings related to ex post CBA
This study has used CBA for ex post impact assessment and have drawn some lessons:
1) starting from individual components is the easiest way to identify the benefit elements
2) concentrate on valorisation of the main benefit elements, if too uncertain the result is altered unreasonable
3) the wider benefits are often important outcomes but are difficult to quantify
193 Feb 2011 Ex post eval cohesion fund_DG Regio
Considerations
1) break-even analysis might be more convincing to illustrate economic surplus/deficit
2) develop a price and benefit catalogue (Excel tool) to support CBA in practice
3) CBA to be combined with other qualitative impact assessment methodologies to improve the capturing of wider benefits
# 203 Feb 2011 Ex post eval cohesion fund_DG Regio
THANK YOU FOR YOUR ATTENTION!