1 ex post evaluation of the cohesion fund (including former ispa) work package b: cost - benefit...
TRANSCRIPT
1
Ex post evaluation of the Cohesion Fund (including former
ISPA)
Work Package B: cost - benefit analysis of selected transport
projects
Jurate Vaznelyte, Evaluation Network MeetingBrussels, 20 October, 2011
2
Objectives of the study
Cohesion Fund and ISPA (2000 – 2006):
254 individual transport projects co-financed
Terms Of Reference for this study:
103 individual transport projects
10 schemes considered for ex post evaluation(38 individual projects out of 260 or 15%)
● Question 1: What were the impacts of these projects?
● Question 2: How can ex post CBA contribute to the practice of ex ante CBA?
● Question 3: What are the potential and limits of ex post CBA?
3
AVE Madrid-Barcelona
A23 Motorway
Lisbon – Algarve railway
M1 Motorway
IX B Corridor + Vilnius bypass
A2 Motorway
Bratislava Railway Upgrade
M0 Motorway
Agiou Konstantinou bypass
Thriasso-Pedio-Eleusina-Korinthos railway
Road
Rail
4
Ex ante stated objectives
Reduce travel time (6)
Increase capacity/reduce congestion (4)
Reduce operating costs (2)
Improve safety (8)
Improve connectivity (2)
Projects2 4 6 8 10
5
Ex ante impacts taken into account in CBA
Impacts on safety (10)
Impacts on the environment (4)
Impacts on transport operators (3)
Impacts on users (Time Savings, Vehicle Operating Costs) (10)
Other impacts (2)
Projects2 4 6 8 10
6
Ex ante CBA – its role in decision making
Compliance with EC CF application requirements (9)
Ensure value for money (6)
Choose alignment (2)
Choose design standards (2)
Prioritise elements of national transport strategy (1)
Allocate budget and optimise timing (0)
Projects2 4 6 8 10
7
Ex post economic evaluation – benefit-cost ratio (BCR)
Most projects show BCR larger than 1, i.e. the net project impact, as captured by the CBA is positive
Due to high capital costs, the AVE shows a BCR < 1. However, the wider socio-economic impact (not captured in the CBA) are significant
8
Ex post financial analysis – financial NPV
The financial NPV (before EC contributions) of seven of the ten projects is negative – not surprising as most projects do not generate direct commercial revenues.
The fact that most projects have an economic BCR > 1, but a negative financial NPV indicates that that the EC contributions help ‘unlock’ the economic benefits of these projects
9
Sources of benefits – railway projects
Railway projects
-20%
0%
20%
40%
60%
80%
100%
120%
Railway - Spain Railway - Portugal Railway - Greece Railway - Slovakia
Travel time VOCs Transport operator fares
Safety Carbon Air pollution
Noise Other
10
Sources of benefits – road projects
Road projects
-40%
-20%
0%
20%
40%
60%
80%
100%
120%
140%
Road - Poland Road - Spain Road - Greece Road - Ireland Road - Lithuania Road - Hungary
Travel time VOCs Transport operator fares
Safety Carbon Air pollution
Noise Other
11
Other key findings
Transparency and accountability
● The regular publication of ex post findings, whatever the outcome, generally increases transparency and accountability. It also help identify key learning points for future project planning
Evidence base
Feedback
● Average difference between ex ante and ex post was 13.4% - relatively low
● Factors explaining the difference: time lag, project delays, scope alterations and unforeseen circumstances
● Generally in line with objective to build in sufficient spare capacity to cover project lifetime. Two exceptions (one too high, one too low)
● Ex ante NPV generally higher than ex post: the main driver for discrepancy was the demand forecast
● Together with capital cost and utilisation rates findings, it is indicative of optimism bias
● Ex ante demand modelling would benefit from improvement in study area definition, modal coverage, inclusion of induced traffic and variable demand
● CBA practice would benefit from more consistency in scope, more granularity (e.g. business vs. non-work users) and harmonisations of assumptions, parameters and approaches
Capital costs
Utilisation rates
NPV
Modelling practice
CBA practice
12
Key issues considered
Strengths and weaknesses of CBA methodologies as applied by the Member States
1 2 3 4 5
Effectiveness of CBA as a tool to support project generation and decision of Member States and Commission
Utility of ex post CBA from the point of view of project promoters, Member States and the Commission
Ex post CBA as a tool for evaluating project impacts
Relevance and potential utility of CBA for macro-economic modelling
13
Strengths and weaknesses of CBA1 2 3 4 5
● There is a basic level of consistency in CBA methodology used for transport infrastructure projects across EU Member States
● CBA framework used in the ten projects is both consistent with DG REGIO’s guidelines and with good practice
Scope of the ex ante analysis sometimes narrower than what
would be ideal (externalities not always
considered)
Different appraisal parameters used across
EU Member States
(e.g. HEATCO values)
Weaknesses in demand modelling, counterfactual definition and modelling
of network effects
14
Effectiveness of CBA in decision-making
1 2 3 4 5
● The current contribution of CBA in the decision making process is limited
● Mainly used to confirm project’s value of money and to support funding application
CBA results Regional development and
environmental concerns
CBA does not fully take into account other key impacts Time that would be required to fully embed CBA into planning process
CBA offers more potential as decision-making tool
Technically Politically
CBA is strong in comparing projects with similar impacts. Could help in choosing design, alignment and in prioritization
CAB can add transparency to the decision-making process and improves political accountability
15
Utility of ex post CBA1 2 3 4 5
Ex post CBA can add significant value to planning process
Adds transparencyStrengthens evidence
base
Provides feedback on methods and techniques used for ex ante design
and appraisal
1 2 3
A key contribution of ex post evaluation has been a
better understanding of optimism bias and its causes at the planning
stage
16
Ex post CBA as a tool for impact evaluation
1 2 3 4 5
Advantages Limitations
Holistic approach
Delivers an unambiguous indicator of the project’s economic worth
Considers both direct impacts (e.g. time savings) as well as associated
externalities (e.g. safety benefits)
1
2
3
Due to long life of infrastructure, ex post CBA should be carried out as late as
possible
Counter-balancing the above, there is a risk of institutional memory loss
The analysis requires the definition of a counterfactual – for projects with a long
expected economic life, this may be problematic
1
2
3
17
CBA relevance and utility for macro-economic modelling
1 2 3 4 5
Ex post CBA covers a wider range of impacts than those focus of macro-
economic models (i.e. non-work traffic savings, safety and environment)
but
It provides a ‘net’ benefit indicator and does not indicate
who actually and ultimately receives this benefit.
It measures the added value of bringing new resources to use, while macroeconomic modeling
measures the gross value.
In practice
Macroeconomic impacts are best modeled by linking transport
accessibility models directly with macroeconomic models
18
Thank you for your attentionThank you for your attention
http://ec.europa.eu/regional_policy/sources/docgener/evaluation/rado_en.htmhttp://ec.europa.eu/regional_policy/sources/docgener/evaluation/rado_en.htm