“Analytical Tools and Data Collection” Workshop

15 July 2009EuropeAid/125317/D/SER/TR

Session 3

Assessing Impacts

Stages of Impact Analysis

Assess Impacts

and Compare Options

Identify Impacts

Gather Evidence

RIA Informs Decision

• Governments should not act unless the results (benefits) are worth the cost

• Governments should find the lowest cost solution to solve problems

• Benefits and costs must be sufficiently clear to identify better and worse solutions

Policy Analysis Serves Decision Criteria

Cost Benefit Analysis (CBA) = Benefits minus CostsCompares overall negative and positive impacts to ask IF options should be

considered based on whether they produce more benefits than costs..

Cost Effectiveness Analysis (CEA) =

Compares costs per metric of benefit for different options and asks WHAT KIND of action should be taken to maximize results per unit cost.

Least Cost Analysis (LCA) = lowest cost to reach a FIXED target. Compares options that reach the same result and asks WHICH action reaches

a fixed target at lowest cost.

Costs

Effects (lives saved, accidents reduced)

Use the methods together for a good IA

• They are most effective when used together

• CEA is based on the comparison of options. It is valuable only if we consider the right options

• Cost-effectiveness and least cost analysis are best seen as an essential supplement to cost benefit analysis

CEA Example:

Reduction of Injuries in Minibuses

Costs (TL)

No. of injuries prevented

CEA

Option 1 4,000,000

1,200 3,333

Option 2 5,000,000

1,800 2,778

Option 3 7,000,000

2,000 3,500

LCA Example:Reducing Emissions

Fixed Target Option Cost

Reduce emissions by 30 %

Fixed caps for all emissions

€ 300 million

Tradable permits € 100 million

More stringent regulation on high volume emissions

€ 200 million

Three Elements of CBA

• Identification of potential benefits and costs

• Quantification and valuation of costs and benefits

• Spread of costs and benefits over time

Identification of Costs and Benefits

• Set baseline

• Consider ‘Do nothing option’

• Avoid double counting

Example of wrong baselineIssue: Re-equipped Minibuses causing injuries and discomfort

Option: Do nothing Option: Quality requirements

Benefits:• None

Benefits:• Decrease in healthcare expenses• Increase in productivity of passengers

Costs:• High healthcare expenses• Lost productivity of passengers

Costs:• Investment in new minibuses

Example Double CountingOption: Ban of smoking in public places

Benefits:•Health benefits•Productivity benefits

Costs:•Decrease in sales for importers•Decrease in sales for producers•Decrease in sales for bars

Quantification and Valuation

• Whenever possible, quantify (measure) impacts using different metrics (kilos, metres etc)

• Whenever possible, value impacts in a single metric to enable adding them together and easy comparison

• Monetary valuation is also referred to as ‘market valuation’ or ‘economic valuation’

• Use different scenarios whenever appropriate

Estimating Prices

• Market value = willingness to pay• Some prices do not have market value:

– Health benefits– Time savings– Environmental benefits and costs– Human life

• The basic approach is to estimate the ‘willingness to pay’ (or ‘willingness to accept compensation’) for a particular outcome

Methods for Estimating Prices

1. Use market prices information• Change in productivity• Human capital cost valuation

2. Valuation using information on individuals’ preferences

• Replacement cost or preventive expenditure method • Contingent valuation method • Surrogate market valuation method

3. Benefit Transfer • Use estimates of shadow prices from other studies

Spread of Costs and Benefits over Time

• The value of 1TL received tomorrow is less than the value of 1TL received today

• Individuals expect to receive a rate of interest on savings to compensate for giving up consumption today

• The rate of interest gives us a measure of an individual’s rate of time preference

• Using rate of time preference to convert future value into equivalent value today is called discounting. This enables comparison of benefits and costs that occur in different time periods

Calculating Net Present Value/Discounting

Net Present Value =• (Benefits – Costs) * Discount Factor, for year 0 +• (Benefits – Costs) * Discount Factor, for year 1 +• (Benefits – Costs) * Discount Factor, for year 2 +• etc

Discount Factor =(use discount tables)

Benefits justify the costs if the NPV is positive (NPV>0)

1

(1+ interest rate)year

Example of NPV Calculation

Year 0 1 2 3 4

Benefits 0 0 0 1500 1500

Costs 1500 300 300 300 300

Net Benefits -1500 -300 -300 1200 1200

Discount Factor (for 10%)

1.000 0.909 0.826 0.751 0.683

Discounted Net Present Value -1500 -272.7 -247.8 901.2 819.6

Cumulative Net Benefits = – 1500 – 300 – 300 + 1200 + 1200 = 300

Net Present Value = – 1500 – 272.7 – 247.8 + 901.2 + 819.6 = – 299.7

Risk Assessment

• In assessment of impacts consider risks and likelihood of their occurrence

• Describe the consequences of risks – could they stop the regulation from meeting its objectives?

• Risk analysis provides an overall judgement on whether the risk-adjusted benefits exceed the risk-adjusted costs

Must We Quantify Everything?

No, but….• Governments should not act unless the results

are worth the costs• Governments should find the lowest cost solution

to solve problems• Good qualitative comparisons may be better than

very uncertain quantitative comparisons• Use quantitative and qualitative metrics to

compare benefits and costs• The important point is that benefits and costs are

sufficiently clear that the public can understand

Qualitative Weighting

• Where it is difficult to value impacts, describe them by looking at other impact measures

• You might provide information such as: – how many people will be affected– what type of people might be affected– the nature and impact of the effects

• Effects may be categorized as ‘small’, ‘medium’ or ‘large’ to compare and weight various impacts

Qualitative Presentation: EC Sugar Quotas IA

21

Advantages Drawbacks

Price F

all

1. Facilitates restructuring and improved competitiveness of the sector.

2. Reduces production surpluses and world market distortion.

3. Reduces distortions in competition.

4. Reduces inequalities among producers.

5. Reduces cost to consumers and users.

6. Promotes diversification in the market for sweeteners.

7. Facilitations WTO negotiations.

8. Promotes application of cross-compliance.

9. Reduces budget cost of the regime slightly.

1. Producer incomes fall.

2. Revenue of ACP countries falls.

3. Cost of possible accompanying measures for restructuring

Qualitative Presentation: EC Sugar Quotas IA

Qualitative Presentation: UK RIA on Implementation of EU Directive on Maximum Levels of Organochlorine Compounds in Animal Feed

Qualitative presentation: EU Postal Services (2006)

Quantitative presentation: UK RIA for Directive on Minimum Rules for the Protection of Chicken Kept for

Meat

Quantitative presentation: UK RIA for Directive Laying Down Minimum Rules for the Protection

of Chicken Kept for Meat

Continuation of the previous example

Combining Quantitative and Qualitative Presentations

Example: UK RIA on Gambling Act

Source: www.evidence-based-medicine.co.uk

Option 1Retaining the status quo

Option 2 Implement the Gambling Act 2005

Total cost per annum

No additional costs

£1.9 – 2.6m

Total benefit per annum

No additional benefits

• Greater employment opportunities • Higher revenue generation as a result of additional

websites being based in the UK and non-UK residents visiting UK-based websites

• Improved regulation• Better choice, information and safeguards for

customers

Other costs • Voluntary contributions to the Responsibility In Gambling Trust

• Additional software to protect the vulnerable and prevent crime

• Training to identify problem gambling

Presenting Results and Recommendation: Decision Criteria

• Assessment of costs and benefits informs decision and provides evidence for recommended option

• Do the benefits justify the costs?• Is the approach the least costly option for

achieving the objective?

Presenting Results and Recommendation

• Result of assessment should be clearly described and explained in RIA

• Sources of evidence should be provided to build credibility• Quantify as many impacts (costs and benefits) as possible• Present results in tabular form: show positive impacts (benefits)

and negative impacts (costs) separately and, if possible, net benefits are calculated

• Present separately any significant distributional impact