An interdisciplinary approach to evaluating environmental policy: the case

of biodiversity offsetting

Megan C. Evans

Fenner School of Environment and Society, Australian National University

megan.evans@anu.edu.au @megcevans

Talk outline

• Research motivation

• Background to biodiversity offsetting

• Environmental policy evaluation

• Why an interdisciplinary approach is needed

• Moving forward

A global problem:

Biodiversity loss

Hoffman et al. 2010 Science

with conservation actionwithout conservation action

Case study:

Biodiversity offsetting• Broad goal is to achieve no net loss or net gain of biodiversity

• Rapid growth in uptake: policies exist in 45 countries, and in development in another 27• Wetland banking and species banking (USA), BioBanking (NSW), Bushbroker

(Vic), EPBC policy (Australia), EU ‘no net loss by 2020’ , UK pilots, Germany

4Gibbons and Lindenmayer 2007 EMR Madsen et al. 2010 State of the Biodiversity Markets Report

Total Wetland and Stream Mitigation Banks (USA) 1980-2009

Dealing with biodiversity loss:

What role for economic policy instruments?

Opportunity• More efficient and effective

than ‘command and control’ regulation (?)

• ‘Win-win’ outcomes • Leverage the market for

multiple benefits

Outcomes?

Risk• Failure to capture

externalities• Moral and ethical objections• Successful “symbolic

policies”

Need for evaluation

What is evaluation?

“…the process of determining the merit, worth, or value of something, or the product of that process”

Scriven, M., 1991. Evaluation Thesaurus

Summative evaluation approaches examine the effects or outcomes of programs

What was the impact/outcome of a policy intervention?

Formative evaluation approaches aim to strengthen or improve the program being evaluated

Why did these outcomes occur?

How can outcomes be improved?

Rossi, et al. 2004. Evaluation: A Systematic Approach

Evaluation of environmental policy

“…evaluation can cut through and resolve data-free debates about the relative merits of different approaches”

Bottrill and Pressey 2012 Conservation Letters

…but rare

important…

“Our review confirms previous claims that causal evidence of the effectiveness of conservation interventions…is rare”

Miteva et al 2012 Oxford Review of Economic Policy

“…ecological scientists are often not familiar with impact evaluation methodologies; and environmental policy is one of the most difficult areas in which to conduct credible evaluations”

Ferraro 2009 New Directions for Evaluation

Biodiversity offsetting:

What evidence exists?United States• Wetland banking:

• 25% permit conditions not met, compensatory mitigation seldom result in wetlands with optimal condition Ambrose et al 2007

• 74% of offsets achieve no net loss goal [by acreage] Brown and Lant (1999) Environmental Management

• Species banking: • Fox and Nino-Murcia (2005) and Bunn et al. (2014) have examined drivers of

bank establishment and their ecological values– but no evaluation of adequacy of compensation

• “If the agencies are not monitoring the bank sites, there is no assurance that the properties are providing the intended habitat for the species. Meanwhile, credits are being sold and impacts are taking place”

Fleischer, D. & Fox, J. (2008) in Conservation and Biodiversity Banking: A Guide to Setting Up and Running Biodiversity Credit Trading Systems

Biodiversity offsetting:

What evidence exists?Australia• Senate inquiry into environmental offsets, June 2014

“…there is currently no way to tell whether the ‘no net loss’ policy objective of environmental offsetting is being achieved in Australia”

Evans MC (2014) Submission to Australian Senate Inquiry into Environmental Offsets

“The committee recommends…. that the Department of the Environment expedite the development of a publicly available nationally coordinated register of environmental offsets…

…that the compliance audit program be extended to include an evaluation of the progress of offsets … in achieving their intended environmental outcomes.

Australian Senate Environment and Communications References Committee (2014)

Research questions

1. What environmental outcomes are being delivered by biodiversity offsetting policies?

2. Why is this the case, and how could outcomes be improved?

How to evaluate?

Ecology

Political science

Economics

Program evaluation

Public policy

How to evaluate?• Systematic review

• “Evidence-based conservation”, ConservationEvidence.com and EnvironmentalEvidence.org

• Relies on an existing evidence base to draw upon

• Long term experiments• …long term• Expensive

Ecology

How to evaluate?• Impact evaluation

• Strong emphasis on establishing causality• Experimental, quasi-experimental design• More amenable to cases where data is readily available (e.g water

quality, air quality, point-source pollutants)

Economics

Program evaluation

How to evaluate?• Qualitative methods

• Case study method (Yin 2009) • Comparative method (Collier 1993)• “Lesson-drawing” (Rose 1991) “systematic study of small N”

Data collation:• open-ended survey questions, interviews or

focus groups (Patton 2001)

• Focusses on answering “why?” questions• Issues re. validity, reliability, generalising findings

Political science

Program evaluation

Public policy

Research questions

1. What environmental outcomes are being delivered by biodiversity offsetting policies?

1a) Is no net loss possible? Evaluating policy instruments for reducing deforestation with a growing economy• Evaluate the impact of regulation and offset policies on deforestation

rate (1972-2011), account for climatic and economic drivers • Hierarchical modeling, bent-cable regression (Chiu et al. 2006)• Synthetic control method

(Abadie et al. 2010)

Research questions

1. What environmental outcomes are being delivered by biodiversity offsetting policies?

1b) Doing but not knowing? Evaluating the outcomes of biodiversity offsetting policies• At a policy level, evaluate to what extent biodiversity offsets are

compensating for impacts• Australia, USA

Research questions

2. Why is this the case, and how could outcomes be improved?2a) Comparative analysis of species compensatory policies in the United States and Australia• Similar legislative frameworks

• Endangered Species Act (1973)• Environmental Protection and Biodiversity Conservation Act (1999)

• Banking/offset policies or practice in place for ~15 years in each• Is the policy practical and desirable? (Rose 1991)• What lessons can be learnt?

Research questions

2. Why is this the case, and how could outcomes be improved?2b) Understanding barriers and opportunities to improve environmental outcomes from biodiversity offsetting• Semi-structured interviews with industry representatives and proponents

• Are there issues with policy that if changed could improve both business bottom line as well as environmental outcomes?

Final thoughts

“…we believe the most fundamental design challenge trading poses to environmental policy lies not in currency adequacy or exchange adequacy ….

… but in review adequacy [mechanisms for evaluation] and confronting the pressures trading places on the institution of environmental permitting.”

Salzman, J., Ruhl, J.B., 2000. Stanford Law Review

Thank-you

20

E: megan.evans@anu.edu.au

T: @megcevans

W: economical-ecology.com

Thanks to my supervisors:

Karen Hussey, Stuart Whitten, Phil Gibbons, Tara Martin & Grace Chiu