countryside stewardship facilitation fund monitoring and ...randd.defra.gov.uk ›...

FINAL

REPORT

July 2015

Collingwood

Environmental Planning Limited

in partnership with

GeoData Institute

Defra project code: WC 1061

Countryside Stewardship Facilitation

Fund Monitoring and Evaluation Scoping

Study

A project for the

Department for

Environment,

Food and Rural

Affairs in

collaboration with

Natural England

Final Report July 2015

CSFF M&E Scoping Study Collingwood Environmental Planning i

Project title: Countryside Stewardship Facilitation Fund (CSFF) Monitoring and Evaluation (M&E) Scoping Study - Project to explore uses of data and indicators for reporting on the CSFF

[Variation to the Monitoring and Evaluation of Nature Improvement Areas (NIAs) Phase 2 Project]

Contracting organisation:

The Department for Environment, Food and Rural Affairs (Defra)

Defra project code: WC 1061

Lead contractor: Collingwood Environmental Planning Limited

Address: 1E The Chandlery, 50 Westminster Bridge Road, London, SE1 7QY, UK

Contacts: Dr William Sheate (Project Director) [email protected] Teresa Bennett (Project Manager) [email protected]

Tel. 020 7407 8700

Website: www.cep.co.uk

Partner organisation:

GeoData Institute:

Contact: Chris Hill

Report details: Report title: Countryside Stewardship Facilitation Fund Monitoring and Evaluation Scoping Study – Final Report

Date issued: 10th July 2015

Purpose: Final report of the study

Author(s): Teresa Bennett, Peter Phillips, William Sheate, Ric Eales and Jonathan Baker.

mailto:[email protected]

mailto:[email protected]

http://www.cep.co.uk/


CSFF M&E Scoping Study Collingwood Environmental Planning 1

Contents

Executive Summary ....................................................................................... 2

1. Introduction ............................................................................................. 6

1.1 Background and objectives of the Countryside Stewardship Facilitation Fund ..................... 6 1.2 Requirement / purpose of this scoping study ......................................................................... 8 1.3 Structure of the report ............................................................................................................ 8

2. Approach ................................................................................................. 9

2.1 Understanding of the CS scheme and, in particular, the CSFF ............................................... 9 2.2 Developing a Countryside Stewardship Facilitation Fund logic model ................................... 9 2.3 Identifying existing information / data sources of use in monitoring and evaluating the effects of facilitation and undertaking gap analysis ........................................................................... 9 2.4 Literature review on the timescales for detectable outcomes from biodiversity and ecosystem services enhancements .................................................................................................... 9 2.5 Scoping the Countryside Stewardship Facilitation Fund evaluation design ......................... 10

3. Potential scope of the CSFF evaluation design ....................................... 12

3.1 Introduction .......................................................................................................................... 12 3.2 Evaluation objectives ............................................................................................................ 12 3.3 Options for the evaluation design ........................................................................................ 13 3.4 Sample size and sampling strategy ....................................................................................... 17 3.5 Additional resource implications for CSFF M&E ................................................................... 17

4. Context .................................................................................................. 23

4.1 Nature Improvement Areas monitoring and evaluation ...................................................... 23 4.2 Lessons from NIA M&E ......................................................................................................... 24 4.3 The counterfactual for NIA M&E and parallels for CSFF M&E ............................................. 24 4.4 Background literature ........................................................................................................... 25

5. The Logic Model and Theory of Change .................................................. 30

5.1 Overall approach ................................................................................................................... 30 5.2 Development of the logic model .......................................................................................... 31 5.3 Logic model for CSFF ............................................................................................................. 32 5.4 Additional logic model issues ................................................................................................ 38

6. Data and information ............................................................................. 39

6.1 Discussion with Defra, Natural England and the Environment Agency ................................ 39 6.2 Review of data used in monitoring and evaluation of NIAs ................................................. 39 6.3 Information and data requirements based on the CSFF logic model ................................... 43

7. Conclusions ............................................................................................ 51

8. References and bibliography .................................................................. 54

Appendix 1: Note on scope and nature of the Countryside Stewardship Facilitation Fund .......................................................................................... 57

Appendix 2: Summary of data sources and analysis used for the M&E indicators ..................................................................................................... 63

Appendix 3: Theory of change literature review .......................................... 66

Appendix 4: Review of Biodiversity 2020 Outcome 1C Self-Assessment Criteria ......................................................................................................... 72

Appendix 5: Potential approaches to CSFF M&E based on the logic model... 75



Executive Summary

The purpose of this study was to use knowledge and methods gained in the Nature Improvement Area (NIA) Monitoring and Evaluation (M&E) Phase 2 project to help scope the design of the M&E for the Countryside Stewardship Facilitation Fund (CSFF). The CSFF has been established as part of Countryside Stewardship (CS) to support cooperation between farmers and land managers at the landscape scale. The scheme builds on the principles of NIAs and the Game & Wildlife Conservation Trust (GWCT) Farm clusters. The fund was launched on 17 March 2015 and has a value of £7.2 million over five years.

The fund provides for facilitators to coordinate action amongst farmers and other land managers working in collaboration. It is expected that a variety of people / organisations will apply for funding, such as, NIAs, lead farmers (under the GWCT model), National Parks and non-governmental

organisations (NGOs). The purpose behind the CSFF is to deliver shared environmental outcomes that go beyond what could be delivered by individual holdings acting in isolation, specifically:

The achievement of CS priorities more effectively than without facilitation.

Delivery of CS priorities with greater spatial coherence at the landscape scale.

It is anticipated that the CSFF can deliver enhanced benefits by helping to ensure:

Greater intensity of CS option uptake.

Better management of uptake.

Better spatial coordination and coherence of uptake.

In scoping the CSFF M&E design therefore, the objective has been to ensure that the adopted M&E design can focus on:

Whether the investment in facilitation has delivered the anticipated outcomes of more effective delivery of Countryside Stewardship (CS) with more spatial coherence at the landscape scale.

Whether up skilling of group members and combined cooperation over several land holdings has led to additional delivery compared to what can be achieved at the holding scale.

Whether the benefits from supporting groups of farmers / land managers to cooperate at landscape scale justifies the additional costs of facilitation

The study has identified three main options for evaluating the CSFF as set out in the following sub-sections.

Option 1 – experimental / counterfactual approach

The adoption of a counterfactual or experimental evaluation approach seeks to provide a clearer demonstration of what happened with the intervention (CSFF), compared to what would have happened anyway (no CSFF). For CSFF, the outcome and impact metrics likely to be used are mainly quantitative – e.g. trend in farmland bird populations, compliance with WFD targets etc under facilitated and non-facilitated management.

This option is likely to be the highest cost option (compared to Options 2 and 3) given the requirement for fresh ecological survey work (at a scale unlikely to be already available) to evidence the delivery of outcomes and impacts in facilitated and non-facilitated areas. Option 1 would constitute a more ‘deep and narrow’ approach to evaluation design whereby limited resources for M&E are focused on exploring a smaller (non-representative) sample in greater detail. Furthermore, an assessment of the costs and benefits of facilitation in Option 1 would increase the overall costs of M&E further, potentially resulting in an even narrower design (smaller sample size).



This option includes two possible sub-approaches (it would also be possible to combine both):

Option 1(a) paired case studies / comparative approach

This would identify paired groupings of holdings where one group is under facilitation and the other isn’t, in order to compare the outcomes and impacts of facilitated vs non-facilitated management (there are also crucial context factors to consider). The paired approach would be designed, where possible, to eliminate all variables except for facilitation. Paired groups of holdings would be selected on the basis that they are delivering similar CS options, so that the objectives and anticipated outcomes / impacts of management would be similar also.

Identifying paired / suitably similar groups of holdings may be difficult (or impossible) given the need to eliminate all variables except for facilitation, e.g. how would key biophysical differences be accounted for?

Option 1(b) trajectories assessment

The trajectories assessment approach would collate data on outcome and impact metrics, before and after facilitation, to identify trajectories (trends) for the metrics considered. By comparing the trajectories before and after facilitation, it would be possible to identify where facilitation may be delivering different (e.g. enhanced) outcomes and impacts – e.g. decrease in agricultural pollutant loading, site / landscape level response of key species groups etc.

Option 2 – theory of change approach

A theory of change ‘model’ can be used to map out theoretical causal processes between activities, outputs, outcomes and impacts. Using M&E to demonstrate an understanding of what is happening ‘on the ground’ in terms of activities and outputs can then evidence the delivery of outcomes and impacts – i.e. using the theory of change model to demonstrate links between known activities / outputs and theoretical outcomes / impacts.

This option is likely to be the least cost option, because many of the necessary data will already be collected under the facilitation agreement and through existing monitoring. Additional qualitative data collection through interviews or surveys could be scaled proportionately to CSFF uptake and to available budget for evaluation.

This option would focus CSFF M&E on activities and outputs which are easier to attribute to facilitation – e.g. the facilitator will need to collect basic data on activities as part of their facilitation agreement with Natural England. By tracking the activities and outputs delivered in facilitated areas therefore, theoretical links to outcomes and impacts could be made, using the theory of change model, to evaluate the overall success of CSFF projects. Data collation under the theory of change approach would be primarily through facilitator reporting, interviews with participating land managers etc.

Whilst a degree of primary data collation will be required, this is likely to be much less onerous than in Option 1 (i.e. no ecological survey data will be required as outcomes and impacts are evidenced through theory rather than measurement) and therefore sample size could potentially be larger and more representative / generalisable – i.e. Option 2 would constitute a more ‘broad and shallow’ approach.

Option 3 – combined approach

Option 3 would seek to bring the key strengths of Options 1 and 2 together in a combined approach – i.e. the detailed insights of the ‘deep and narrow’ approach in Option 1 combined with the more representative and generalisable nature of Option 2’s ‘broad and shallow’ approach. The exact nature of the combined approach would come down to the budget available for CSFF M&E as well as



the uptake of the CSFF and therefore the population size of CSFF projects from which a sample would be drawn. Option 1 M&E would produce data that can also help to validate the theory of change model in Option 2 – i.e. where outcomes and impacts are being evidenced as anticipated (or not) under Option 1. This will support (or contradict) the theoretical causal processes identified in the model underpinning Option 2, leading to refinements.

From these Options, three practical Sampling Strategies are suggested that combine elements of these Options. All the strategies include the need for a theory of change model (and therefore Option 2) as the anchor for understanding the mechanisms by which change is brought about by the activities undertaken. Relative and indicative costs for each sampling strategy are summarised in the table. The indicative costs of these sampling strategies range from 1.5% to 8.5% of the total budget for CSFF of £7.2 million. The CSFF represents 0.75% of the total budget available for Countryside Stewardship of £925 million.

Summary of indicative costs for suggested sampling strategies

Elements Range of indicative costs

Strategy 1 Strategy 2 Strategy 3 Option 2 (Theory of Change) £60,000-

£75,000 £60,000-£75,000

£60,000-£75,000

Project management, reporting etc (fixed sum for all strategies)

£50,000-£70,000

£50,000-£70,000

£50,000-£70,000

Additional ecological monitoring of facilitation agreements and options (stratified sample) at the beginning, middle and end [if 10-30 holdings sampled, say £3,000-£5,000 per sample x 3 (beginning, middle and end) = £90,000-£450,000]

£90,000-£450,000

Additional ecological modelling of outcomes and impacts for those holdings / options sampled [if 10-30 holdings / options sampled, say 1 researcher day per site using an existing model – c. £500 per day + 5 days data management]

£7,500- £17,500

Case study agreement selection and identification of embedded case study holdings (i.e. sites within the case study agreements) [e.g. 3 CSFF agreement areas x 3 holdings in each = 9)]

£10,000-£15,000

In-depth semi-structured interviews with stakeholders involved in the case study holdings [if 9 case studies x c. 6 interviewees = 54 interviews]

£25,000-£30,000

Ecological surveys of case study sites [if 9 case studies, say £5,000 per site at beginning, middle and end – survey plus reporting (£5,000 x 9 x 3 = £135,000)]

£135,000

Total indicative costs £207,500-£612,500

£110,000-£145,000

£280,000-£325,000

Conclusions

The study sought to identify where data sources already exist and how these could be used to inform lower cost options for the CSFF M&E, as well as identify what other data sources could be used to furnish more elaborate and therefore costly options for evaluation, i.e. where additional new data collection would be required.

Context will have an important influence over the success of CSFF projects – e.g. previous attempts at partnership working, biophysical context factors, access to technology / machinery and skills etc. This is a critical issue for Option 1 in particular, and a ‘deep and narrow’ approach is less likely to be representative due to the resource implications of carrying out fresh ecological survey work across paired groups of holdings. Sample size for an M&E design under Option 2’s ‘broad and shallow’ approach may be more representative due to the less onerous data collection and reliance on more secondary data.



Whichever option / sampling strategy is adopted, it will need to be proportionate to the scale of the intervention being evaluated, i.e. the available CS facilitation fund, the uptake of that fund in practice and the relative spatial extent of land holdings under facilitated management, compared to non-facilitated.



1. Introduction

The purpose of this study is to use knowledge and methods gained in the Nature Improvement Area (NIA) Monitoring and Evaluation (M&E) Phase 2 project to help scope the design of the M&E of the Countryside Stewardship Facilitation Fund (CSFF).

The NIA M&E Phase 2 project was initiated to evaluate delivery of NIA objectives at the landscape scale. Knowledge and experience gained on the effectiveness of different approaches to monitoring the delivery of biodiversity, landscape connectivity and ecosystem services has been transferred from the NIA M&E Phase 2 project to scoping the design of the CSFF evaluation.

Defra’s objectives for the scoping of the CSFF evaluation were to focus on:

Whether the investment in facilitation has delivered the anticipated outcomes of more effective delivery of Countryside Stewardship (CS) with more spatial coherence at the landscape scale.

Whether up skilling of group members and combined cooperation over several land holdings has led to additional delivery compared to what can be achieved at the holding scale.

Whether the benefits from supporting groups of farmers / land managers to cooperate at landscape scale justifies the additional costs of facilitation

Countryside Stewardship facilitation M&E will be developed from this scoping project during subsequent work. The facilitation monitoring will build on and be additional to the proposed monitoring of Countryside Stewardship to assess the added-value of facilitation.

1.1 Background and objectives of the Countryside Stewardship Facilitation Fund

A facilitation fund has been established as part of Countryside Stewardship (CS) to support cooperation between farmers and land managers at the landscape scale. The scheme builds on the principles of NIAs and the Game & Wildlife Conservation Trust (GWCT) Farm clusters. The fund was launched on 17 March 2015.

The fund provides for facilitators to coordinate action amongst farm and land managers working in collaboration. It is expected that a variety of people / organisations will apply for funding, such as, NIAs, lead farmers (under the GWCT model), National Parks and NGOs.

From discussion with the Steering Group, the purpose behind the CSFF is to deliver shared environmental outcomes that go beyond what could be delivered by individual holdings acting in isolation. This includes:

The achievement of CS priorities more effectively than without facilitation.

Delivery of CS priorities with greater spatial coherence at the landscape scale.

It is anticipated that the CSFF can deliver enhanced benefits by helping to ensure:

Greater intensity of option uptake.

Better management of uptake.

Better spatial coordination of uptake.

These concepts have been used to produce a set of objectives for the CSFF (Table 1). These objectives have been collated from relevant references in the absence of an explicit set of objectives for CSFF. The objectives of the specific interventions need to be known to be able to effectively scope the design of its M&E.



Table 1: Anticipated objectives of the Countryside Stewardship Facilitation Fund

Overall objective To support greater cooperation amongst farmers and others to deliver effectively the Countryside Stewardship Priorities at a landscape scale in a spatially coherent way. (pers. comm. Webster, undated)

Headline objective Sub-objectives (specific) Sub-objectives

(generic) A. To protect and enhance biodiversity at

a landscape scale (Defra, 2014b & 2015b)

To protect, enhance and restore priority habitats (TBG, undated)

To reduce habitat fragmentation, increase habitat connectivity and increase the overall extent of priority habitats at a landscape scale (TBG, undated)

To promote ecosystems approach based species and habitat management at a landscape scale (Defra et al., 2014)

To promote greater uptake of integrated land management to help safeguard biodiversity and ecosystem services at a landscape scale (TBG, undated)

To promote land management approaches that deliver synergies and multiple benefits across, for example, biodiversity, water quality and flood risk management (Defra et al., 2014; and Hardiman & Cathcart, 2013)

B. To protect and enhance the water environment at a landscape scale (Defra, 2014b & 2015b)

To reduce point source and diffuse pollution from agriculture at a landscape scale (Defra et al., 2014 )

To reduce soil erosion at a landscape scale (Defra et al., 2014)

To promote sustainable flood management at a landscape scale where it contributes to the protection and enhancement of river, wetland and coastal habitats (Defra et al., 2014)

C. To support people and organisations that bring farmers, foresters and other land managers together to improve the local natural environment at a landscape scale (Defra, 2015b)

D. To support partnership and collective approaches across holdings to deliver shared environmental outcomes that go beyond what could be delivered by individual holdings acting in isolation (Defra, 2015b)

E. To ensure that land is managed in a spatially coherent manner at a landscape scale (Webster, undated)

F. To support the concentrated and effective uptake of relevant options to help deliver Countryside Stewardship statement of priorities at landscape scale (Natural England, 2014)

Facilitation work can be for up to five years and will cover a variety of tasks. Facilitators will help land managers mainly with their Mid Tier agreements, which are designed to deliver at the landscape scale, and also help with Upper Tier, existing HLS, ELS and other agreements, as well as seek other resources. They will help farmers and land managers work cooperatively together to deliver environmental priorities across a number of holdings. Facilitators will also be expected to provide or buy-in training for skills development amongst collaborating land managers.

The area of land involved in facilitation must cover at least 2,000 hectares (unless a smaller environmental boundary can be justified) and be spread across at least four adjoining holdings.



Facilitators will be required to report on their facilitation activities (although monitoring is not payable due to constraints on the nature of EU funding regulations).

Further background on the scope and nature of the Countryside Stewardship Facilitation Fund is provided in Appendix 1.

1.2 Requirement / purpose of this scoping study

This study scopes the potential approaches to M&E that could be used to judge whether Countryside Stewardship facilitation makes a difference to biodiversity (e.g. habitat restored, farmland birds) and selected ecosystem services (e.g. water quality, pollination services) outcomes compared with non-facilitated Countryside Stewardship. It also considers approaches to assessing landscape scale delivery of habitat connectivity and synergies between biodiversity and water quality options. It takes into account the existing monitoring data / information capture methods that can be used to support such an evaluation, and the value and practicalities of options for collecting additional information (measures and indicators) for reporting on facilitated Countryside Stewardship vs non-facilitated Countryside Stewardship. It provides suggestions of data / information that could be gathered either as part of the Countryside Stewardship monitoring or by the facilitators or ecologists to support the evaluation of what difference the facilitated approach is making. Where appropriate, suggestions are made for modelling to be used. The relative potential costs and benefits of these options are considered. A consideration in designing the M&E of the Countryside Stewardship facilitation has been to ensure it is proportionate in terms of the overall scale of the scheme and that it is fit for purpose.

Objectives of the study

The objectives of this study were to:

Identify gaps in the CS M&E programme (under development) with respect to CSFF.

Focus work on addressing data gaps in relation to inputs and outputs (since mechanisms for monitoring outcomes are more or less in place).

Set out the ecological outcomes from having the right things done in the right places and whether this results in, for example, better water quality, farmland birds, soil structure etc.

Taking account of target statements and digitised data sets, determine which ecosystem services to focus on e.g. water quality, pollination, carbon storage, soils (provisioning and regulating), flood risk (including soil management to prevent flood risk), and erosion control.

Link with the work on the counterfactual for NIAs.

Incorporate lessons learnt from NIAs; check the NIA M&E framework indicators for relevance and identify links with CS priorities, data availability, gaps and efficiencies.

1.3 Structure of the report

The report is structured as follows:

Chapter 2 provides an overview of the approach to the research adopted.

Chapter 3 then provides the overall findings of the study upfront, i.e. the scoping of the possible options for CSFF M&E design.

Subsequent chapters provide the thinking and detail that underpin the scoping of options, including Chapter 4 (Context), Chapter 5 (Logic Model and Theory of Change) and Chapter 6 (Data requirements).

Chapter 7 provides overall conclusions.



2. Approach

The approach to this study involved:

Developing understanding of the scope and nature of the Countryside Stewardship Facilitation Fund.

Developing and evidencing a Countryside Stewardship Facilitation Fund logic model.

Identifying existing information / data sources of use in monitoring and evaluating the effects of facilitation and undertaking a gap analysis.

Undertaking a literature review on the timescales for detectable outcomes from biodiversity and ecosystem services enhancements.

Scoping the Countryside Stewardship Facilitation Fund evaluation design.

2.1 Understanding of the CS scheme and, in particular, the CSFF

Information was gathered from Defra web pages and a variety of documents available online on the new CS scheme and the facilitation fund. Other relevant background information was provided by Defra and Natural England. There was no one document covering CS or CSFF.

Information was brought together to provide a summary note on the scope and nature of the Countryside Stewardship Facilitation Fund (Appendix 1).

2.2 Developing a Countryside Stewardship Facilitation Fund logic model

An initial draft logic model was shared with the Steering Group in the Interim Report (March 2015) and has subsequently been further refined. The methodology for developing the logic model and the associated Theory of Change is described in detail in Chapter 5.

2.3 Identifying existing information / data sources of use in monitoring and evaluating the effects of facilitation and undertaking gap analysis

Three approaches were taken for identifying information and data sources for use in monitoring and evaluation of facilitation:

1) Discussion with representatives involved in monitoring and evaluation from Defra, Natural England and the Environment Agency.

2) Review of data used in monitoring and evaluation of NIAs.

3) Identification of information needs for monitoring and evaluation of inputs, activities and outputs, based on the logic model.

2.4 Literature review on the timescales for detectable outcomes from biodiversity and ecosystem services enhancements

A focused literature review was undertaken to help support the logical model development, and this is described in Chapter 4.



2.5 Scoping the Countryside Stewardship Facilitation Fund evaluation design

Scoping the potential approaches to the M&E of the CSFF has drawn on inputs from the other tasks undertaken in this study. The approach has taken account of the need to identify potential challenges which influence the design (for example, issues of selection bias for the CSFF, how to define the counterfactual and whether it will be possible to quantify the level of difference that facilitation makes to outcomes / impacts).

The Magenta Book1 sets out a series of issues or components to be considered in planning and designing an evaluation. These and lessons learnt during the NIA M&E project including the counterfactual work, have been drawn on to identify the considerations and components used to inform the scoping of the design of the CSFF M&E:

Clear understanding of the details of the CSFF (Appendix 1) to define the objectives of the CSFF and the intended outcomes and impacts, including a logic model supported by a theory of change model that involves a mapping of how outcomes and impacts are expected to come about. This approach is useful where it is not possible to track all outcomes and impacts at the CSFF level, as the theory of change (where practicable) provides a robust causal chain supported by relevant literature (e.g. evidence from landscape scale approaches elsewhere that have delivered similar impacts) to help predict likely outcomes and impacts.

Develop a clear set of evaluation objectives and/or questions for the CSFF evaluation to focus on and to help determine its scope.

Clearly define the audience for the CSFF evaluation and its outputs.

Develop and adopt a set of key guiding principles to inform the approach to the evaluation and criteria for its design.

Develop an understanding of the broad approach to the evaluation of the CSFF that will be appropriate and proportionate.

Review the broad information sources and data needs required to support the evaluation (any potential gaps), including an assessment of existing available information and sources and identification of where additional information gathering / monitoring may be required.

Identify potential evaluation methods, both qualitative and quantitative, including related to the counterfactual that could be used in undertaking the CSFF evaluation and their relative strengths and weaknesses.

Consider the proposed programme for the M&E and who will be responsible for undertaking it.

Figure 1 illustrates these components that need to be taken into account in designing the evaluation.

Costing of possible evaluation options and additional data gathering/monitoring has not proved possible in this scoping phase since the scale of likely facilitation fund uptake is as yet unknown. It is not therefore possible to assess the geographical scale and extent of any additional data gathering that might be required. We have, however, identified a range of options from relatively low to high cost, based on whether suitable data are likely to be readily available (low cost) or where new data gathering activities would be required (higher cost).

1 H.M. Government (2011b) The Magenta Book: Guidance for evaluation. London, HM Treasury.



Figure 1: Illustration of the components considered in designing the evaluation



3. Potential scope of the CSFF evaluation design

3.1 Introduction

This chapter scopes out what a possible evaluation of the CSFF might look like. It identifies a set of possible objectives for the evaluation, along with some initial headline evaluation questions. It then considers possible options for the design of the evaluation, including possible counterfactual analyses and use of a ‘theory of change’ approach. It then identifies the additional resource implications of different sampling strategies for CSFF M&E that utilise components of the different options. It also recognises some of the challenges inherent in an evaluation of this nature and makes suggestions as to how to deal with these.

3.2 Evaluation objectives

Based on the scoping work undertaken, a number of possible objectives have been identified for the evaluation of the CSFF. The evaluation objectives selected for the adopted CSFF M&E will depend on the overall purpose of the evaluation – e.g. a summative evaluation design will focus on the objectives of CSFF and evidencing the delivery of anticipated outcomes and impacts; a formative evaluation may attempt to capture wider considerations such as costs and benefits of facilitation, land manager awareness and learning etc.

Possible evaluation objectives include:

1) To assess the degree to which CSFF objectives (see Table 1) have been met at the project and programme level.

o To compare the outcomes and impacts of facilitated vs non-facilitated land management.

o To identify any unexpected outcomes and impacts of facilitation.

o To examine and evidence the processes (methods and approaches) used to develop and deliver facilitated management projects at the landscape scale.

o To identify any barriers to facilitated management at the landscape scale and possible reasons for these barriers.

o To identify strengths, weaknesses and areas of good-practice in landscape scale, partnership based land management.

2) To assess the costs and benefits of facilitation.

o To provide evidence and views from participating land managers and other stakeholders in facilitated areas on whether the benefits of facilitation justify the costs.

The audience for any CSFF evaluation is likely to be primarily policy makers in Defra and Natural England, while lessons learnt will also be of use to facilitators and land managers for future partnership working and landscape scale initiatives.

Evaluation questions

The evaluation objectives provide the basis for developing evaluation questions that will need to be answered through the gathering of evidence from different sources of data: literature, quantitative monitoring data and qualitative data through interviews or surveys with relevant stakeholders. The mix of data gathering techniques will depend upon the final choice of evaluation type, e.g. whether it is formative, summative, impact or process focused, or some combination. Evaluation questions



are likely to be developed around the key elements of the logical model, in order to elicit evidence for those elements, i.e. input, activities/process, output, outcome, impact questions.

Some initial headline evaluation questions could be:-

Inputs: What advice, support and training were provided by facilitators? How much did this cost? What existing knowledge, skills and resources were held by land managers? Were there any barriers to collaborative/partnership working under the CSFF?

Activities/Process: What type of meetings and how many were held between facilitators and land managers? What type of on-the-ground activities were supported by facilitators? What were the costs of these activities? What were the barriers, if any, to implementing activities in practice?

Outputs: What advice was given to land managers by facilitators and how was this acted upon? Which priority habitats were placed under which CS management options?

Outcome/impact: What are the biodiversity outcomes/impacts of facilitated land management (e.g. in terms of habitat or ecosystem service change)? What changes have there been in agricultural pollution loading of water bodies? How do these compare with non-facilitated land management? Were there any unexpected outcomes from facilitation?

Further questions would need to be developed as part of the full evaluation of the CSFF.

Guiding principles

The guiding principles in Box 1 result from lessons from the NIA M&E. This learning should be taken into account in developing the CSFF M&E.

Box 1: The guiding principles identified for the design of the CSFF M&E

Focus on helping to provide evidence to answer the key policy questions (including the benefits of delivery at the landscape scale).

Approach to draw on, and be consistent with, the methods proposed in the Magenta Book.

Method to:

o Use a logic model

o Reflect causal links / theory of change

o Recognise the importance of context

o Include clear evaluation questions

o Develop a counterfactual, where appropriate

Approach to be consistent with the overall CS M&E plan.

Existing information to be used where possible, with optional data and monitoring proposed where appropriate.

Minimise burden on the facilitators in undertaking M&E and overall be proportional in effort and cost to the CSFF.

Enable learning on the effectiveness of facilitation from the overall summative evaluation of the CSFF approach.

3.3 Options for the evaluation design

There are three broad theoretical options that form the basis for the CSFF M&E design. Regardless of which option (or combination of elements of these options) is pursued, the CSFF logic model (see



Chapter 5) provides the overall structure for the M&E design and defines the range of activities and outputs that CSFF will undertake / deliver as well as the anticipated outcomes and impacts of facilitation, focusing on biodiversity and water environment related outcomes. The three broad options are:

1) Experimental / counterfactual approach.

2) Theory of change approach.

3) Combined approach.

Option 1 – experimental / counterfactual approach

Comments on the Interim Report from the CSFF M&E scoping study Steering Group suggested that consideration of the counterfactual should be an important focus for the CSFF M&E. Adoption of a counterfactual or experimental evaluation approach has key strengths in this regard as it can help to provide a clearer demonstration of what happened with the intervention (CSFF), compared to what would have happened anyway (no CSFF). For CSFF, the outcome and impact metrics likely to be used are such that this comparison could be represented quantitatively in many cases – e.g. trend in farmland bird populations, compliance with WFD targets etc under facilitated and non-facilitated management.

There are two possible sub-approaches within the overall counterfactual / experimental design. Discussed further in Boxes 2 and 3, these are: Option 1(a) paired case studies / comparative approach; and Option 1(b) trajectories assessment. As explained further below, both approaches would focus on outcome and impact elements of the CSFF logic model using metrics from the existing Countryside Stewardship (CS) M&E regime2, noting that the nature of these metrics is such that a quantitative comparison should be possible.

Box 2: Option 1(a) The paired case studies / comparative approach

The paired case studies / comparative approach would seek to identify paired groupings of holdings where one group is under facilitation and the other isn’t, in order to compare the outcomes and impacts of facilitated vs non-facilitated management (there are also crucial context factors to consider). The paired approach would be designed, where possible, to eliminate as many variables as possible except for facilitation. In particular, the paired groups of holdings would be selected on the basis that they are delivering similar CS options, so that the objectives and anticipated outcomes / impacts of management would be similar also.

This pairing could be done using expert knowledge (referred to as ‘judgemental mapping’ or ‘matched comparisons’) or it could be supported by using regression discontinuity designs (RDD). RDD is a structured analysis of the applicants which results in a defined benchmark, applicants are then ranked against set criteria (potentially those defined by Defra). It would then be possible to identify an ‘intervention’ and ‘control’ group by selecting applicants that are close to this threshold on either side. RDD allows for greater confidence if intervention treated and control groups are similar and it allows for an understanding of the main feature on which they differ (their score on the selection criteria).

There are some key issues with the paired case / comparative studies approach that would need to be carefully considered in M&E design. In particular, identifying paired / suitably similar groups of holdings may be exceptionally difficult given the need to eliminate all variables except for facilitation, which in practice is probably not possible. For example, how would key biophysical differences be accounted for – e.g. soil type, elevation, aspect, local climatic factors etc. The paired cases would probably be located in the same broad area (e.g. the same NCA or ideally the same catchment, landscape etc); however there will still be biophysical differences that are impossible to control for, as well as any influence by past facilitated management (not only NIAs); all may influence the delivery of outcomes and impacts. These challenges apply in both RDD and judgemental mapping.

2 CAP Pillar 2 Rural Development Programme Environmental Land Management Schemes M&E Plan 2015-2020 (Defra et al, 2014): [not available online]



Box 3: Option 1(b) The trajectories assessment approach

The trajectories assessment approach would collate data on outcome and impact metrics, before and after facilitation, to identify trajectories / trends for the metrics considered. By comparing the trajectories before and after facilitation, it would then be possible to identify where facilitation may be delivering different (e.g. enhanced) outcomes and impacts – e.g. a decrease in agricultural pollutant loading, site / landscape level response of key species groups etc.

Costs of delivering Option 1 M&E are anticipated to be higher than for Options 2 and 3 given the requirement for fresh ecological survey work (at a scale unlikely to be already available) to evidence the delivery of outcomes and impacts in facilitated and non-facilitated areas. Option 1 would constitute a more ‘deep and narrow’ approach to evaluation design whereby limited resources for M&E are focused on exploring a smaller (non-representative) sample in greater detail. Furthermore, an assessment of the costs and benefits of facilitation in Option 1 would increase the overall costs of M&E further, potentially resulting in an even narrower design (smaller sample size). It would also be possible to combine the Options 1(a) and 1(b).

The M&E regime for the wider CS scheme is well developed and includes a range of proposed metrics for anticipated outcomes and impacts of CS3. Recognising that M&E for CSFF needs to be practical and pragmatic (as well as proportionate in terms of cost), the M&E within Option 1 would focus on the use of these existing outcome and impact metrics. In Option 1(a) paired case studies / comparative approach (Box 2), once as many variables as possible other than facilitation have been controlled for in paired groups of holdings (notwithstanding the difficulty of this challenge and high burden in terms of data demand), data from CS M&E outcome and impact metrics could be compared like for like to assess the difference (or not) made by facilitation. In Option 1(b) trajectory assessment approach (Box 3), data on CS M&E outcome and impact metrics would be collated for the facilitated area, pre and post facilitation, to assess the difference made. Both approaches would likely necessitate the collation of fresh survey data to evidence the biodiversity, water environment and related ecosystem service outcome and impact metrics.

CSFF M&E under Option 1 would make good use of existing outcome and impact metrics from the CS M&E regime. However, it may also be the case that Option 1 involves the collation of some data and information on CSFF activity and output metrics / questions. In particular, where an assessment of costs and benefits of facilitation is an evaluation objective, Option 1 M&E would need to assess the costs associated with delivering CS related land management activities and outputs, with and without facilitation. By comparing the total costs under facilitation and non-facilitation with the outcomes and impacts being delivered an assessment of costs and benefits could then be made – e.g. if the costs of delivering CS related land management activity under facilitation decrease but the benefits (outcomes and impacts) stay the same or are enhanced, an overall benefit of facilitation could be evidenced. This sort of cost-benefit could occur, for example, where more coordinated land management under facilitation reduces costs – e.g. applying a more targeted (as opposed to whole farm) approach to nutrient management.


Chapter 5 outlines the rationale behind a ‘theory of change’ based evaluation approach. A theory of change ‘model’ (see Figure 3) is used to map out theoretical causal processes between activities, outputs, outcomes and impacts. Using M&E to demonstrate an understanding of what is happening ‘on the ground’ in terms of activities and outputs can then evidence the delivery of outcomes and impacts – i.e. using the theory of change model to demonstrate links between known activities / outputs and theoretical outcomes / impacts. This sort of approach can be useful where the links between activities and outcomes are not straightforward and / or where providing comprehensive

3 Ibid



evidence on final impacts may not be possible4. As explained at Chapter 5, robustness to the CSFF theory of change model is given by evidencing links between activities / outputs and outcomes / impacts with literature sources where relevant and appropriate.

The Option 2 theory of change approach would focus CSFF M&E on activities and outputs which are easier to attribute to facilitation – e.g. the facilitator will need to collect basic data on activities as part of their facilitation agreement with Natural England and that data on outputs may be available through BARS5 (though BARS data would need to be ‘tagged’ as CSFF to differentiate it from other biodiversity outputs being delivered). By tracking the activities and outputs delivered in facilitated areas therefore, theoretical links to outcomes and impacts could be made, using the theory of change model, to evaluate the overall success of CSFF projects.

Data collation under the theory of change approach would be primarily through facilitator reporting, interviews with participating land managers, etc. Whilst a degree of primary data collation will be required, this is likely to be much less onerous than in Option 1 (i.e. no ecological survey data will be required as outcomes and impacts are evidenced through theory rather than measurement) and therefore sample size could potentially be larger and more representative / generalisable – i.e. Option 2 would constitute a more ‘broad and shallow’ approach. To increase the robustness of the theory of change model (and therefore the robustness of the M&E design under this option), further validation of the model would be required through either or both of the following: 1) further literature review to evidence the theoretical causal processes identified in the model; and 2) workshop with experts (e.g. relevant personnel from Defra, Natural England, Environment Agency etc) to validate the model overall and suggest enhancements.

Option 2 is likely to be the least cost option, because many of the necessary data will already be collected under the facilitation agreement and through existing monitoring. Additional qualitative data collection through interviews or surveys could be scaled proportionately to CSFF uptake and to available budget for evaluation. The focus of Option 2 is on process issues, and consideration could be given to the inclusion (within the logic model / wider CSFF M&E design) of the additional activities identified through the review of the Biodiversity 2020 Outcome 1C self-assessment guide (see Section 5.4 and Appendix 4). These activities provide a comprehensive picture of integrated / ecosystems approach based land management and therefore where these activities are evidenced in facilitated projects, they can add weight to the theory of change approach evidencing the delivery of relevant ecosystem / landscape scale outcomes and impacts.


Option 3 would seek to bring the key strengths of Options 1 and 2 together in a combined approach – i.e. the detailed insights of the ‘deep and narrow’ approach in Option 1 combined with the more representative and generalisable nature of Option 2’s ‘broad and shallow’ approach. The exact nature of the combined approach would come down to the budget available for CSFF M&E as well as the uptake of the CSFF and therefore the population size of CSFF projects from which a sample would be drawn. Option 1 M&E would produce data that can also help to validate the theory of change model in Option 2 – i.e. where outcomes and impacts are being evidenced as anticipated (or not) under Option 1. This will support (or contradict) the theoretical causal processes identified in the model underpinning Option 2, leading to refinements.

4 Logic mapping – hints and tips for better transport evaluations (Tavistock Institute, 2010a): https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/3817/logicmapping.pdf [accessed 08/05/15] 5 BARS homepage: http://ukbars.defra.gov.uk/ [accessed 13/05/15]

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/3817/logicmapping.pdf

http://ukbars.defra.gov.uk/



The importance of context

Context will have an important influence over the success of CSFF projects – e.g. previous attempts at partnership working, biophysical context factors (see Box 2), access to technology / machinery and skills etc. This is critical for Option 1(a) which is premised, as far as possible, on a ‘like for like’ comparison between paired groupings of facilitated and non-facilitated holdings – i.e. controlling and / or accounting for contextual differences between the two groups of holdings will be critical to the success of this design, including, for example, which CS priorities (e.g. biodiversity, water quality, other ecosystem services) are established as part of each CSFF agreement. This assertion was supported by the CSFF M&E scoping Steering Group in their comments on the Interim Report.

Contextual factors could be incorporated in CSFF M&E through the use of a more ‘realist’ approach to evaluation design, where relevant. This might mean, for example, that local contextual factors could be investigated for each CSFF project sampled and the logic model and M&E framework ‘tweaked’ accordingly, or that any pairing (as per Option 1) is informed by an understanding and evidence of context.

3.4 Sample size and sampling strategy

It is now known that the ‘population size’ of CSFF projects in the first year of the fund (five years) is c.20 with wide variation in the number of holdings (c.5-80), area covered (c.750-9,000 ha) and agreements lasting over 3-5 years. The nature and size of the population influences the sampling strategy and therefore the resource implications of possible strategies, which are discussed in Section 3.5. If further rounds of applications are likely to be accepted in future years this would need to be factored into the design of the CSFF M&E as well.

The characteristics of this policy make statistical analysis challenging because of the likely scale of impact (relatively small), large number of confounding factors (signal to noise ratio), long time scales and small population size. It also means that an explicit theory of change model is needed as a common element to all sampling strategies for CSFF M&E, to underpin the logic model and to understand the links between activities (CS Options) and ultimate outcomes / impacts on biodiversity and ecosystem services, since such outcomes and impacts are not possible to measure directly and meaningfully over a short five year period.

An M&E design using Option 1’s ‘deep and narrow’ approach is less likely to be representative due to the resource implications of carrying out fresh ecological survey work across paired groups of holdings (though there may be some efficiencies to be gained by aligning CSFF survey work with that undertaken for CS). Sample size for an M&E design under Option 2’s ‘broad and shallow’ approach may be more representative due to the less onerous data collection and reliance, to a degree, on secondary data. While qualitative data gathering (e.g. through interviews) would still be required this could be more readily tailored to available resources. Where possible, a stratified sampling strategy would be appropriate. For all three M&E options, the sample could be stratified on the basis of relevant criteria – e.g. National Character Area (NCA) type, number of holdings facilitated, size of facilitated area, CS priorities delivered, existing land manager awareness / training etc.

3.5 Additional resource implications for CSFF M&E

This section seeks to outline the possible additional resource elements and possible sampling strategies for monitoring and evaluation due to the existence of the Countryside Stewardship Facilitation Fund (CSFF) Monitoring and Evaluation (M&E). It is made in light of discussions with the Steering Group and with Natural England.

These additional resource implications are intended to support Defra / Natural England in the development of proposals for the CSFF M&E design. The section is structured by Sampling Strategies that broadly relate to the theoretical Options as presented in Section 3.3, although these strategies



present a more pragmatic approach that reflects the practical suitability of the components in the theoretical options to the CSFF M&E, and in light of the actual CSFF agreements having now been made.

Some elements have not been costed in detail because they are likely to form a common part of any of the suggested sampling strategies and will be proportionate to the scale of sampling and surveying actually undertaken. These include: contractor and Defra / Natural England project management, steering group time, background document review and analysis, specific analysis of results and reporting costs, support to facilitators etc. However, these might amount to a further £50,000-£70,000 at least over a five year period.

Note that none of the sampling strategies presented are mutually exclusive and a ‘pick and mix’ approach would be possible / desirable, depending on available resources. The potential costs identified for the components of each sampling strategy use numbers that are not entirely arbitrary, but have the potential to be highly variable, e.g. ecological survey costs depend upon the nature, scale and complexity of specific agreements and options being surveyed. The figures do however provide a relative basis for comparison and an understanding of what might be achievable for a certain budget. All figures exclude VAT.

Sampling strategy 1: Sampling / experimental / counterfactual approach

We know from the ongoing NIA counterfactual work, using existing Environmental Stewardship data, that the trajectory/trend analysis of NIA areas, while possible, can tell us only about the uptake of options and the extent of areas under options in relation to specific landholdings, and compared to the national picture, but cannot provide information in relation to habitat condition.

As it is now known that around 20 facilitation projects will be funded, with a very wide diversity in the number of holdings (from c.5-80), timescale (three or five years), area covered (c. 750–9,000 ha), with and without previous facilitation arrangements (NIA links), over a very diverse range of landscapes and habitats and applying a wide variation of options, it would not be possible to sample these (i.e. a representative proportion of the 20 only) in any statistically meaningful way. In effect, these are 20 unique case studies and should therefore be treated as such.

Across the 20 projects there will be, however, up to c.400 individual CS agreements / individual land holdings with a very wide variation in the number and nature of CS options taken up. The CS M&E programme, as we understand it from Natural England, includes sample monitoring of the ecological status (condition assessment) of 3-5% of all CS agreements. All agreements are mapped at the start for the location of options, but for 3-5% of agreements, on a rolling basis, additional condition assessment, and assessment of appropriateness of options and opportunities missed are also undertaken, and then re-survey undertaken at a later date. Clearly, good baseline data is essential.

It may be possible, therefore, to apply a similar level of sampling to the facilitated agreements, i.e. 3-5% of the c.400 agreements within facilitated areas (which would = c.20 agreements) or perhaps more (10% = c.40 agreements) in order to try to cover the range (as much as possible) of the six NCA clusters Natural England suggests using for categorising landscapes to reduce variability. Condition assessments would be required for these samples at baseline, middle and end of the facilitation period, although it is questionable whether it would be worthwhile including agreements covering only three years due to the time lag for habitat outcomes. It is possible of course that some NCA clusters will simply be unrepresented in the 20 CSFF projects. Within this sample it would be logical, given the focus of the CSFF, to further stratify the sample by focusing on only those options that are relevant for landscape scale conservation or landscape scale ecosystem services, such as water quality, and therefore only areas under one or more of such options would be surveyed at the start, middle and end of the period (see Figure 2). However, even data collected through this sampling approach would not provide meaningful outcomes/impacts because of the short timescale and the near impossibility of distinguishing the influence of other factors/variables, e.g. weather. Even over



five years, it would be quite a stretch to conclude causality due to facilitation in the light of any results. Comparison to the national picture, and therefore a form of non-facilitated counterfactual, provided by the wider CS 3-5% M&E would be possible, i.e. it would be possible to conclude whether the uptake of options was different under facilitation, and whether implementation/activity was different, but not outcomes/impacts.

Figure 2: Possible stratified sampling strategy for Sampling strategy 1

As a consequence, even with a sampling approach such as this, a theory of change model is necessary to understand whether the activities undertaken are likely, in the longer term, to give rise to positive outcomes/impacts as intended. Quantitative modelling could be undertaken of outcomes and impacts6, but again this would necessitate having a theory of change to underpin it. The theory of change would help to specify what needs to be modelled, i.e. which specific outcomes and impacts should be modelled in order to attempt to quantify them. The baseline/beginning, middle and end (b, m, e) habitat condition sampling would help to validate the result of any numeric models.

It would not now, in our view, be possible, desirable or cost effective to try to find comparable areas for paired comparisons, given so many variables are uncontrolled and uncontrollable, the noise to signal ratio is so great, and the timescales (3-5 years) for evaluation of these agreements are insufficient to be able to draw any real conclusions about causality, i.e. even if it were possible to find comparable areas (which is unlikely) it would not be possible to conclude that facilitation was the cause of change in outcomes over such a short timescale, because outcomes would not be apparent. If all one is doing with paired areas is therefore confirming or otherwise a difference in uptake of certain options or numbers of options and the spatial extent of area under options (compared to non-facilitated comparator areas), then such knowledge will already be available from

6 The proposed modelling element would provide quantitative predictions of environmental outcomes and impacts of facilitated land management. This would also act to provide a quantification of key elements within the theory of change model. The modelled predictions would then be validated (where possible) through the evaluative habitat condition monitoring. This would also support validation and refinement of the theory of change model.

400 holdings

5-10 % sample – 20-40 holdings

Stratify sample by NCA cluster (up to 6 clusters: X, Y, Z etc)

(20-40 holdings) X holdings Y holdings Z holdings

Stratify sample by those holdings with relevant groups of landscape scale options

(say 10-30 holdings)

Landscape Options only in X holdings

group

Landscape Options only in Y holdings

group

Landscape Options only in Z holdings

group



national CS monitoring and trend/trajectory analysis, comparing uptake under CS facilitated agreements against the national picture for non-facilitated CS agreements. The difficulty with spatially paired comparisons using current data sources7 is that each parcel of land (polygon) in the ES database can have many options stacked upon it. So even if you were able to have two paired sites Xa and Xb, it is likely that they would have very different numbers and types of options on those sites, e.g. Xa might have five options, Xb might have 35 options. Any difference therefore in terms of extent of habitat between Xa and Xb may have nothing to do with facilitation, but plenty to do with the number and types of options present on that parcel of land, and/or other uncontrollable variables, e.g. enthusiasm of land manager.

In practice this means that the old Option 1 now needs to be supplemented, to include ecological sampling allied to the sampling applied to the rest of CS, but at the facilitation agreement level, PLUS ecological modelling where that is possible or desirable, PLUS a theory of change component (i.e. Option 2).

Key elements would be:

Elements of sampling strategy 1 Range of indicative

costs

Option 2 (Theory of Change) £60,000- £75,000


£90,000- £450,000


£7,500- £17,500

Project management, reporting etc (fixed sum for all strategies) £50,000- £70,000

Total indicative cost £207,500- £612,500

Sampling strategy 2: Theory of change

This Theory of Change strategy is the simplest and cheapest option and directly reflects Option 2 in Section 3.3. It would require a number of key elements (these would also apply to sampling strategies 1 and 3):

Additional literature review to support / refine / validate the theory of change.

Workshop with experts to validate the theory of change.

Ground-truthing the theory of change with any available quantitative data (this could include historical data on habitat creation, restoration and data from modelling and habitat condition monitoring). (Optional)

Qualitative surveys of facilitators and land managers (semi-structured interviews with facilitators; interviews or questionnaires with land managers, e.g. the latter where large number of land holdings involved) – 3 rounds: beginning, middle and end of facilitated project / all CSFF projects.

A feature of this sampling strategy is that it is explicit in NOT requiring quantitative counterfactual data; it accepts the difficulties of proving causality since quantitative data still requires expert 7 We are still working the paired comparison experimental approach for the NIA counterfactual study, using the same data as we have been using for the trajectory/trend analysis. While we are attempting paired comparisons as part of this experimental approach it should be noted that there are significant challenges in identifying suitable scales at which to compare, for all the reasons already mentioned.



interpretation in light of all the uncontrolled variables. Instead it relies on developing a robust theory of change, supported by experts in habitat creation, management and restoration, and on the views of those on the ground that are able to comment on how things are different under facilitation compared to their own experience were facilitation not to have occurred.

Assuming the M&E to run over a five-year period this sampling strategy could be delivered for a cost in the region of: c. £60,000-£75,000 (excluding all reporting, project management, facilitator support, VAT etc).

Key elements would be:


costs




Sampling strategy 3: Combined approach with Case Studies

This sampling strategy would include all the elements of Option 2 (Theory of Change), plus a selected number of the 20 agreements as detailed case studies to support the theory of change. In light of the number of agreements made (20) we would suggest something like three of the 20 could be selected as case studies, NOT as representative of the CSFF agreements, since that is not possible, but to reflect a range of scale and landscape types to provide ground truthing of the theory of change. It may make sense to exclude the three year agreements from the case study selection and select only from those with five year agreements. The case studies would include additional in-depth qualitative and quantitative surveying and monitoring through ecological survey of specific options / locations (embedded case study holdings) under management as part of the wider facilitation agreement. The nature of these case study holdings would need to be agreed with facilitators and land managers within the selected case studies.

This is the only sampling strategy that would provide detailed longitudinal ecological monitoring of selected case study holdings within facilitated agreement areas, providing rich data of the practical implementation of actual (multiple) options in specific locations to support the theory of change.

Key elements would therefore include:


costs



£10,000- £15,000


£25,000- £30,000


£135,000





Overview of the sampling strategy costs compared with the CSFF budget

Given that the total budget for CSFF is £7.2million, the range of costs suggested for evaluation across sampling strategies 1-3 range from 1.5% to 8.5% of the total CSFF budget. This compares with the total spend on Countryside Stewardship in England over five years of £925 million, so CSFF budget equates to 0.75% of total spend.



4. Context

4.1 Nature Improvement Areas monitoring and evaluation

The establishment of the Nature Improvement Area (NIA) programme was announced in the Natural Environment White Paper (NEWP) – Natural choice – securing the value of nature (HM Government, 2011a). NIAs are large, discrete areas where a local partnership has a shared vision for their natural environment that are intended to deliver a ‘step change’ in nature conservation. The programme takes forward the recommendations of the Lawton review, Making space for nature (Lawton et al., 2010).

The NIA M&E Phase 2 project is supporting the delivery of NEWP commitment 11 “[Defra] will capture the learning from NIAs, and review whether further action is needed in planning policy, regulation or capacity building, to support their development”.

The 12 initial Nature Improvement Areas (NIAs) started work in April 2012 and received three years of grant funding. Their aim was to provide better places for wildlife, to improve the natural environment for people, and to unite local communities, land managers and businesses through a shared vision. They have been trying out different approaches. The variety of landscapes, objectives, and partnerships seen across the NIAs reflects this.

A consistent approach for monitoring and evaluation (M&E) was adopted to assess the NIAs including what works well, and potentially not so well, and to take stock overall. The overall approach to the evaluation of the NIA programme draws on guidance in the Magenta Book8 (HM Government 2011b). A logic model9 approach was used to provide the overall framework for the design of the evaluation.

The approach is a combination of a process and impact evaluation – focusing on both how the NIA partnerships are delivering their objectives, as well as on what and how much they are delivering for biodiversity, ecosystem services and social and economic wellbeing.

The NIA partnerships are applying several concepts where the practical use of science is still developing, for example relating to restoration of habitat connectivity and ecosystem services.

The final evaluation report covering all three years of the NIAs is due to be completed by November 2015.

The NIA monitoring and evaluation framework

The 12 initial NIA partnerships were required to undertake monitoring and evaluation using a framework which includes four themes: biodiversity; ecosystem services; social and economic benefits and contributions to wellbeing; and partnership working. Each theme includes several sub-themes. Within each of the themes there are a number of indicators that seek to measure the NIAs individual and combined progress (Appendix 2).

The framework includes ‘core’ standardised indicators that are reported on by all the NIA partnerships, while the rest are optional, yet standardised and local indicators that allowed the NIA to evaluate specific outputs. The framework is intended to provide a consistent approach across all the NIAs which allows for aggregated analysis whilst recognising the variability across the programme.

8 HM Government (2011) The Magenta Book: Guidance for evaluation. London, HM Treasury. 9 A logic model seeks to understand the complexity of a policy intervention and the relationship between an intervention’s inputs, activities, outputs, outcomes, and impacts.



The NIA partnerships use an online reporting tool specifically developed to record and report their annual monitoring data. The NIA partnerships also report on progress quarterly, including financial monitoring and progress against their agreed objectives and outputs. Additional qualitative research (surveys, interviews and case studies) has been undertaken to complement the results of the framework which primarily reports outputs and outcomes.

4.2 Lessons from NIA M&E

Some lessons from the NIAs are relevant to scoping the evaluation of the CSFF. Facilitation and coordination of activities by partners have been central to the NIA programme, including the facilitation of applications by land holders for Environmental Stewardship funding. The CSFF is seen by Defra as a form of follow-on from the NIAs (although CSFF is sector specific and predominantly rural) as a means of continuing some of the partnership-type activity in relation to landscape-scale habitat management, while extending it beyond existing NIA areas. Lessons from the logic model developed for the NIAs have been used in developing the logic model for the CSFF scoping. The datasets and indicators developed as part of the NIAs are also likely to be useful in future evaluation of the CSFF, including the counterfactual.

Particular lessons found to be useful / important are:

Having clear evaluation objectives from the outset.

An evaluation framework with a logic model to drive the evaluation effort.

Both quantitative outcome indicators and qualitative date.

Understanding the processes occurring (especially in process based policies like NIA and CSFF).

Baseline information is a prerequisite for any monitoring system.

Use national data sets where possible (using centralised analysis where possible).

Look for opportunities for triangulation of data.

Consider the counterfactual from the beginning.

The NIA partnerships are all very different and have locally specific objectives and work programmes. This means that comparative and cumulative reporting is not always appropriate or possible. Typically, given the nature of the activities involved, many of the NIA partnerships’ activities will result in impacts that will only be fully realised in the long-term. It is a challenge for both existing monitoring systems and those developed specifically for the NIA programme – and therefore also for the CSFF – to provide a direct measure of their outcomes and impacts during the specific funding period. Given that many of the indicators rely on the extent of changes in land cover resulting from biodiversity options implemented during the programme the general lack of a consistent, full extent baseline land cover / land use geospatial data has constrained the repeat change mapping and analysis. Direct monitoring of actions and the cross-tabulation of these actions as contributing to specific objectives (e.g. area contributing to water quality, contribution to ecosystem services) has been an easier approach to adopt for the NIAs than where indicator protocols proposed modelling (e.g. diffuse pollution modelling). Although the modelled approaches offer much, they have only been successful when run external to the project and there was often no repeat modelling to support monitoring and evaluation.

4.3 The counterfactual for NIA M&E and parallels for CSFF M&E

As part of the NIA evaluation three exploratory approaches to the counterfactual are being addressed and analysed alongside each other. These are experimental approaches because the



counterfactual for complex environmental policy evaluations is challenging; it is rarely possible to control for what would have happened in the absence of the policy intervention and many other factors may be influential in bringing about change. The three approaches are being used together in order to provide triangulation and check for consistency of results:-

Approach 1: Interviews and questionnaire survey of NIAs partners to understand at the individual NIA partnership level how partners may, or may not, have worked together in the absence of the NIA initiative and the potential impact of this on related outcomes.

Approach 2: Trends and trajectory analysis of key indicators/datasets before and after the NIAs came into existence, using historical datasets for the NIA areas covering at least three years prior to the NIAs. A relatively simple spreadsheet format is being used to compare data within (individually) and across (collectively) NIA partnerships.

Approach 3: This approach is focusing on a comparison of the delivery of actions within NIAs with similar ‘non-NIA’ areas or landscapes, based on the uptake of schemes by their objectives within the sample areas and requires spatially referenced scheme data. This is an experimental approach and will make comparison of similar attributes across broadly similar areas, based on agreed comparative criteria.

These approaches to the counterfactual have informed the scoping of the counterfactual for the CSFF, i.e. what would/would not have happened in the absence of facilitation provided through the CSFF fund. Since the purpose of the CSFF is to facilitate both applications and delivery of management activities at a landscape scale, counterfactuals are likely to relate to land holdings that previously have not been involved in facilitated arrangements – a trajectory approach, and land holdings entering into facilitated arrangements compared with those that have not, in similar comparative areas/landscapes.

4.4 Background literature

Task 5 of the project was to provide additional literature to support the project as a whole.

Background on NIA M&E

The Year 1 and Year 2 NIA M&E Progress Reports10 have provided the experience to date of the NIA partnerships, along with the Method Note11 developed for the NIA Counterfactual work, which provides a description of the methodology for the three counterfactual approaches being undertaken.

Timescales for detectable outcomes from biodiversity and ecosystem services enhancements

For the purposes of this short literature review ‘enhancements’ have been taken to encompass habitat (not just priority habitats) and ecosystem creation, restoration and enhancement works. The literature review (Table 2) was based on documents produced by Defra for the biodiversity offsetting pilot12 and internet searches on keywords and phrases including: timescale for habitat creation and ecological services restoration timescales. While some reviewed papers indicated timescales (in years) for development of habitats and ecosystems others were less specific. Most papers that specified timescales referred to the time taken to achieve ecosystems of comparable quality to

10 Collingwood Environmental Planning (2013; 2014a) NIA M&E Progress Reports Year 1, Year 2 http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&Completed=2&ProjectID=18555 11 Collingwood Environmental Planning (2014b) Evaluating the NIAs Against the Counterfactual – Method Statement and Work Plan, 19 December 2014. 12 Biodiversity offsetting is an approach to delivering conservation activities designed to give biodiversity benefits to compensate for losses (due to development) in a measurable way. The biodiversity offsetting pilot, which ran from April 2012 to March 2014) tested the approach in six pilot areas.

http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&Completed=2&ProjectID=18555



historic sites. Knowledge of historical ecosystems provide a basis for identifying restoration targets (Table 3). Figure 3 shows the feasibility and timescales of restoring different ecosystem types (TEEB, 2009) and has been reproduced in the Biodiversity Offsetting Pilots Technical Paper (Defra, 2012).

Figure 3: Feasibility and timescales of restoring European ecosystems to a resilient, self-sustaining state

Source: Defra (2012) Biodiversity Offsetting Pilots Technical Paper: the metric for the biodiversity offsetting pilot in England (extracted from TEEB 2009).

Biodiversity and ecosystem services enhancements at the landscape scale

Delivery of structural landscape features (such as hedgerows) and many ecosystem services (such as improved water quality and erosion control) require spatial targeting at the landscape and farm / field scale (BTO, 2005; LUC & GHK Consulting, 2008; Diebel et al., 2008; Moreno-Mateos & Comin, 2010; and Cole et al., 2013). These aspects along with the time taken for detectable results from biodiversity and ecosystem creation, restoration and enhancement works have been taken into account in the development of the CSFF logic model and theory of change and scope of the CSFF evaluation design to test whether the right features are being created/ managed in the right places.



Table 2: Examples of literature on the timescales for habitat and ecosystem creation and restoration

Habitat / ecosystem creation / restoration

Target Timescale Evidence Reference

Ponds, saltmarshes and reedbeds

To create habitat of conservation quality

1-2 years Some newly created habitats have been colonised relatively quickly and by species of conservation value.

Morris et al., 2006

Creation of intertidal habitats

To support assemblages of waterbirds

At least 5 years

Intertidal habitats created to support assemblages of waterbirds can take at least five years to determine whether the restored site is of conservation value.

Atkinson et al., 2001

Habitat creation on arable land

Not specified Not stated Habitats have been established easily on arable land (through agri-environment schemes (AES)) with predictable outcomes (based on the literature), especially when seeded, although species composition tended to be dominated by species of arable and disturbed habitats.

Critchley et al., 2004

Creation of neutral and calcareous grasslands

To create habitat of conservation quality

>100 years Creation of floristically rich grasslands (neutral and calcareous) on former arable land may take more than 100 years to develop.

Morris et al., 2006

Restoration of species rich ancient calcareous grasslands

To create habitats of similar conservation value to historic sites.

Decades to centuries

The succession of plant communities from disturbed land to species-rich ancient calcareous grassland communities took from decades to centuries to stabilise. The course of the succession was found to depend on site conditions and the availability of adjacent colonisation sources.

Gibson & Brown, 1991

Restoration of neutral grasslands

To create habitats of similar conservation value to historic sites.

>100 years While young restored grasslands can be rich in species and include indicators of unimproved grassland, the full range of (MG5c) species continues to develop for well over a century.

Gibson, 1998

Restoration of calcareous grasslands

To restore calcareous grassland to resemble ancient grassland communities

> 60 years A comparison of 40 restoration sites with 40 paired reference sites, showed that there was little overlap between restored and ancient grassland communities even after 60 years. A high phosphorus level in the soil was found to be a constraining factor on restoration. Proximity to good quality grassland had a positive effect on naturally regenerated sites.

Fagan et al., 2008

Restoration of species-rich lowland grassland

To create species-rich grassland of conservation value

Not specified

The restoration of botanically species-rich grassland of conservation value on previously intensively managed agricultural land was found to be technically feasible within a relatively short time frame. However, the development of invertebrate assemblages appeared to be a slower process.

Walker et al., 2003

Creation of heathland To create habitat of Not The success of heathland creation appears to be influenced by factors such as soil pH and Morris et al.,





on arable land conservation quality specified the presence of species of disturbed land. The study concluded that valuable lowland heathland may be difficult to create from arable land.

2006

Heathland restoration Not specified Not specified

The use of palaeoecological evidence (e.g. via pollen taxa, macrofossil and sediment records) can provide a perspective on ecological processes operating, the habitat and species presence and their response to climate and human-induced disturbances and the changes to ecological networks.

Groves et al., 2012

Ancient woodland Not possible

Woodland creation can also result in a rich assemblage of species (plants and animals), although the long life-cycle of trees (50 to 500 years) and the slow development of woodland soils, along with the centuries of management that have shaped the distinctive features of ancient woods, means that it is not possible to re-create woodlands that resemble ancient woods.

Morris et al., 2006

Maintenance of Ancient and Ornamental Woodlands

To develop a flexible management regime for Ancient and Ornamental Woodlands

Not specified

Pollen sequences have shown that the species composition of the New Forest has changed over the past 600 years, reflecting the response to natural and human-induced disturbances. Climate change and continued human activity on the future trajectory of the New Forest woodlands leads to the need for a flexible approach to management of the habitat.

Grant and Edwards, 2008

Restoration of ecosystems in general

Increasing biodiversity and ecosystem services

<5 to 300 years

An analysis of 89 studies from across the globe found that the timescales for restoration of degraded ecosystems in relation to increasing biodiversity and ecosystem services ranged from less than 5 to 300 years. However, restoration did not necessarily achieve the quality of biodiversity and ecosystem services of intact ecosystems. The results also showed that biodiversity and ecosystem service restoration were positively correlated and therefore that restoration of biodiversity should support increased provision of ecosystem services.

Rey Benayas et al., 2009

General ecosystem recovery

To identify landscape scale ecosystem trends

Not specified

Palaeoevidence sources can help to describe change and variation within the landscape and at local levels and link human activities with ecosystem trends.

Shaw and Whyte, in press

General ecosystem recovery

Recovery of ecosystems from a major perturbation to the pre-perturbation state

10 to 50 years

Peer-reviewed studies of the recovery of ecosystems (from across the world) following cessation of a major perturbation found that most ecosystems recovered on timescales of 10 to 50 years. In general, aquatic ecosystems recovered more quickly compared to terrestrial ecosystems; this was thought to reflect the shorter turnover times for the longest living species and nutrient pools in aquatic systems.

Jones & Schmitz, 2009





General ecological restoration

To restore resilient self-sustaining ecosystems

Variable The timescales for ecological restoration (recovery of resilient, self-sustaining ecosystems) varies widely. Full ecological restoration, and therefore the full ecosystem service benefits from restoration, takes a long time. However, partial recovery with consequent ecosystem service benefits can occur quite rapidly.

TEEB, 2009

Table 3: Examples of literature on the knowledge of historical ecosystems as a basis for restoration targets

Ecological restoration Target Timescale Evidence Reference


To restore ecosystems to the state before disruption

Not specified

Knowledge of historical ecosystems provides a basis for identifying restoration targets. Ecosystems have been altered to varying extents by human activity and climate change and many historical ecosystem types are too remote in time to provide meaningful restoration targets. Despite inevitable environmental change it is helpful to use past ecological states as reference points for restoration. The paper suggests that instead of looking to natural states of ecosystems as targets for restoration, the emphasis should be on ecosystem function, goods and services.

Jackson & Hobbs, 2011


To restore ecosystems Not specified

Due to the implications of climate change in relation to ecological restoration, the paper suggests that a balance is required between rebuilding past ecosystems and attempting to develop resilient ecosystems for the future.

Harris et al., 2006


To take an ‘open-ended’ approach

Not specified

A suggested approach for defining restoration outcomes is to take an ‘open-ended’ approach, particularly in situations where no reference system exists for defining restoration outcomes (such as Wicken Fen National Nature Reserve) or where restoration is planned on a large spatial scale. In such cases the goal for restoration could be framed in terms of promoting natural processes, mobile landscape mosaics and improved ecosystem services, as well as the actions needed to address the long-term outcomes.

Hughes et al., 2011



5. The Logic Model and Theory of Change

5.1 Overall approach

Logic modelling is a tool that can help organisations plan / design and evaluate interventions – logic models provide a systematic and visual way of presenting the key steps that need to be followed to turn a set of inputs into activities that are designed to contribute towards the delivery of a specific set of changes or impacts13. The key components of a logic model are:

Inputs: the resources (financial, time, people, skills etc) being invested.

Activities: the activities and processes undertaken to deliver the intervention’s objectives and wider objectives where relevant.

Outputs: the initial outputs produced through the delivery of planned activities.

Outcomes: the short and medium term results of the activities and outputs delivered.

Impacts: the longer term results achieved through the delivery of durable outcomes.

A logic model can help to define the metrics and / or questions to measure and evaluate the success of the intervention – i.e. the degree to which it has achieved its objectives. Metrics and questions may be defined for all elements of the logic model (e.g. inputs, activities, outcomes). Obtaining data on these metrics and questions can then provide evidence to assess the overall success of the intervention – e.g. which objectives were met and to what degree, were some aspects delivered more successfully than others, did the outputs contribute to outcomes / impacts as expected etc.

A range of different approaches to logic modelling are possible. Of these, the most widely used in evaluation projects are ‘theory based’ approaches. These approaches define assumed connections (theory) between a proposed intervention (its inputs and activities) and its anticipated impacts and then test this theory through the systematic collection of data, using a variety of research methods14. A strength of theory based approaches is their ability to work in situations where a quantitative experimental research design is not possible – e.g. where it is not possible to isolate variables other than those related to the intervention in order to look at what happened with the intervention, compared to what would have happened anyway. The corollary of this is a weakness of theory based approaches – i.e. the counterfactual is not always clearly described (and certainly less clearly than in an experimental evaluation design).

As outlined in Chapter 3, three broad options were identified for the CSFF evaluation design, including a ‘theory based’ approach. A ‘theory of change’ approach is considered as one option. Theory of change logic modelling can help the evaluation / evaluator to work out how and where progress is being / has been made along an intervention’s anticipated path of inputs, activities, outputs and outcomes towards desired final impacts. This type of approach can provide a basis for robust evaluations even where it is not possible to fully evidence the achievement of an intervention’s anticipated impacts15, as may be the case with CSFF given the inherent challenges in demonstrating causality between CSFF and environmental outcomes that could be delivered by a range of other policies and grant mechanisms. Using monitoring and evaluation to demonstrate the achievement of steps along the path can provide part of an evidence base for the achievement of

13 Logic mapping – hints and tips for better transport evaluations (Tavistock Institute, 2010a): https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/3817/logicmapping.pdf [accessed 08/05/15] 14 Guidance for transport impact evaluations – choosing an evaluation approach to achieve better attribution (Tavistock Institute, 2010b): http://www.tavinstitute.org/wp-content/uploads/2013/01/Tavistock_Report_Guidance_for_Transport_Evaluations_2010.pdf [accessed 08/05/15] 15 Logic mapping – hints and tips for better transport evaluations (Tavistock Institute, 2010a): https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/3817/logicmapping.pdf [accessed 08/05/15]


http://www.tavinstitute.org/wp-content/uploads/2013/01/Tavistock_Report_Guidance_for_Transport_Evaluations_2010.pdf




impacts, especially where the theoretical links or causal processes between the various components of the logic model are supported by literature and / or evidence from elsewhere.

A ‘theory of change’ based approach was used to model theoretical causal processes between CSFF inputs at one ‘end’ and anticipated outcome and impacts at the other. For the most part this involves ‘if…then’ reasoning16 – i.e. “if x happens then y should happen also”, and so on. A focused literature review was undertaken (where possible) to provide supporting evidence for the theoretical causal processes that could underpin the delivery of anticipated CSFF outcomes and impacts. Within a ‘theory of change’ based approach to CSFF M&E, metrics and questions could be used to track the delivery of activities and outputs ‘on the ground’ in facilitated areas. These are elements that could be readily monitored and attributed to CSFF. The modelled theoretical causal processes – supported by literature where relevant – would then provide the evidence base and rationale for the delivery of outcomes and impacts. The theory of change model developed for CSFF incorporates prioritised elements of the logic model only (see section 5.3). Further information on the ‘theory of change’ and the literature reviewed is provided in Figure 4 and Appendix 3.

5.2 Development of the logic model

The development of the logic model was informed by relevant good-practice guidance17 as well as examples of logic models from other relevant monitoring and evaluation frameworks18. In summary the main steps were:

Step 1: collate literature (policy, guidance, existing M&E policy etc) on CSFF, Countryside Stewardship and integrated / landscape scale approaches to land management more generally (see Table 4).

Step 2: review literature from Step 1 and identify possible logic model elements – inputs, activities, outputs, outcomes and impacts.

Step 3: collate and analyse possible logic model elements from Step 2 – group elements, identify possible key stone / critical elements, identify possible causal processes and links between elements.

Step 4: write-up logic model work to date in Interim Report and gather feedback from CSFF M&E scoping Steering Group.

Step 5: map out possible ‘theory of change’ between logic model elements and undertake literature review to evidence theoretical causal processes identified (where possible).

Step 6: development finalised CSFF logic model and ‘theory of change’ taking account of CSFF M&E scoping Steering Group comments at Step 4. Use the finalised logic model to scope CSFF evaluation approaches and possible data requirements and questions.

16 Ibid 17 Ibid 18 Rural Tourism Package: Monitoring and Evaluation Framework (Defra, 2014a): https://www.gov.uk/government/statistics/rural-tourism-feature-report-december-2011 [accessed 11/05/15]

https://www.gov.uk/government/statistics/rural-tourism-feature-report-december-2011




Table 4: Documents reviewed in the development of the CSFF logic model

Document Reference Common Agricultural Policy Pillar 2 Rural Development Programme (RDP) Environmental Land Management Schemes (ELMS) Monitoring and Evaluation Plan 2015-2020

Defra et al. (2014)

The new Common Agricultural Policy schemes in England: December 2014 update

Defra (2014b)

Note on scope and nature of the Countryside Stewardship Facilitation Fund CEP (2015) (Appendix 1)

Guide to Countryside Stewardship: facilitation fund Defra (2015a)

Biodiversity 2020 Outcome 1C – Ecosystem Approach Self-Assessment Criteria and Method

Terrestrial Biodiversity Group (undated)

Countryside stewardship facilitation fund – Application Form Natural England (undateda)

Countryside stewardship facilitation fund – Assessment and Scoring Process Natural England (undatedb)

5.3 Logic model for CSFF

The proposed logic model for CSFF is shown at Table 5. The elements within the logic model have been drawn from a review of key documents relating to the CSFF (Table 4). It should be noted that not all elements within the logic model are likely to be of equal importance, but some elements are likely to be of critical importance for the success of the intervention – these are referred to as ‘key stone elements’. For example, the group agreement and facilitation agreement outputs are likely to be instrumental for the delivery of all outcomes and impacts – e.g. they may set out partnership working arrangements for participating land managers (group agreement) as well as the location, scope and timing of land management activity to be undertaken (facilitation agreement).

This approach was supported by the Steering Group at the Interim Report stage and has therefore been taken forward and developed in this Final Report. The Steering Group suggested that categorisation by ‘keystone’ and / or ‘primary and secondary’ elements could be a useful way of prioritising M&E effort for CSFF. This thinking has been developed further through the categorisation of logic model elements – in particular, biodiversity and the water environment are key priorities for CSFF (see Chapter 1). Logic model elements at Table 5 have been categorised on the following basis:

1) Keystone elements – of critical importance to intervention success;

2) Biodiversity related elements; and

3) Water environment related elements.

These three categories could be considered as primary elements with all other elements (e.g. certain of the desk based activities, partnership working outcomes etc) as secondary elements only. Where relevant, this categorisation could then be used to prioritise M&E effort for CSFF.



Table 5: Countryside Stewardship Facilitation Fund FINAL DRAFT Logic Model Note: Keystone elements (of critical importance to the success of CSFF) are highlighted in bold red text. Stakeholders should note that all biodiversity and water environment related elements are considered to be ‘primary elements’ and could be used to prioritise CSFF M&E effort where required (see Section 5.3.

Inputs Activities Outputs Outcomes Impacts The resources (financial, time, people, skills etc) being invested in the facilitated management area

The activities and processes being undertaken by the facilitator and the participating land managers within the facilitated management area to deliver their objectives, and wider policy objectives

The initial outputs produced by the facilitator and the participating land managers through the delivery of planned activities

The short and medium term (2 – 3 years) results of the activities and outputs delivered in the facilitated management area

The longer term (3 years plus) results achieved through the delivery of durable outcomes within the facilitated management area

Such as… Such as… Such as… Such as… Such as… Defra funding: 1) for the CSFF; and

2) for spatially targeted and concentrated uptake of relevant CS options as prioritised through the CSFF project

At least 2,000ha of land per CSFF project19

At least 4 adjoining (or mainly adjoining) land holdings

Advice, support and training from specialist facilitators with expertise in at least one of the following: agriculture, forestry, water management or ecology

Advice, support and training from specialist external providers

Existing knowledge, skills and resources held by the participating land managers

Shared information and learning from others

Activities undertaken as part of the

CSFF application process:

Scope out CS priorities that could be addressed within the facilitated management area

Meetings, workshops etc with participating land managers to develop cooperation, working towards the production of a group agreement

Meetings, workshops etc with participating land managers to agree the CS priorities from the relevant National Character Area (NCA) statement of priorities that will be delivered within the facilitated management area

SWOT analysis (or similar) to identify land manager training needs

Meetings, workshops etc with participating land managers to identify the actions required to deliver the agreed CS priorities

Activities undertaken as part of the CSFF project and submission of subsequent CS applications:

Support for individual land

Desk based outputs:

Agreed list of CS priorities to take forward across the land holdings in the facilitated management area

Group agreement setting out how the group will operate, who will have what roles and how disputes will be handled

Land manager training and knowledge transfer plan

Facilitation agreement – a detailed plan setting out the CS outcomes that are to be delivered within the facilitated management area (should also explain why the anticipated outcomes are expected to be better through CSFF than if the land managers were acting independently)

Preparation and submission of individual but complementary and spatially coherent, targeted and concentrated CS applications from individual land managers across the facilitated management area

Links and synergies between facilitated land management activity

Biodiversity outcomes:

Improved condition and increased extent of priority habitats

Structural and functional habitat connectivity improvements at the landscape scale

Improvements to overall health and functioning of ecosystems

Site and landscape scale response of key species populations (farmland birds etc) to landscape scale management (e.g. population increases, enhanced biodiversity)

Landscape scale enhancement of ancient semi-natural woodland habitat networks from restoration of PAWS sites

Water environment outcomes:

Reduction in agricultural pollution loading (e.g. N, P, sediment etc)

Catchment scale enhancements to hydraulic properties of river channel, riparian zone and the wider floodplain

Improvements to overall health and functioning of ecosystems

Partnership working and land manager

Biodiversity impacts:

Site and landscape scale response of key species populations (farmland birds etc) to management (e.g. population increases, enhanced biodiversity)

Condition and extent of priority habitats is enhanced

Landscape scale habitat connectivity improvements contributing to enhanced resilience of wild species populations to climate change

Enhanced populations and diversity of key species groups (e.g. farmland birds, farmland butterflies etc)

Enhanced ecosystem service delivery

Water environment impacts:

Water quality improvements / compliance with WFD targets

Increased residence time and peak flow reduction (or delay) contributing to reduction in downstream flood risk

Biodiversity, water quality and flood risk management synergies delivered at the landscape scale

19 Smaller areas of land may be acceptable for CSFF projects where it is possible to demonstrate an obvious smaller environmental boundary (e.g. a small sub-catchment, contiguous area of woodland etc). The 2,000ha figure relates to the combined area of holdings within the facilitated management area and not the area of land being managed to deliver CS priorities



Inputs Activities Outputs Outcomes Impacts managers to check and re-position existing land management activity to support enhanced delivery at the landscape scale

Facilitator support for individual land managers in the facilitated management area to interpret the CS priority statements and associated guidance to enable submission of individual but complementary CS applications

Activities undertaken to support the delivery of on the ground land management:

Scoping and delivery of training and knowledge transfer to provide participating land managers with the skills required to deliver CS priorities

Facilitator works to maintain links between the land holdings / land managers within the facilitated management area and related local partnerships and initiatives

On the ground land management activity to deliver agreed CS priorities at the landscape scale

and wider land management partnerships and initiatives identified

Facilitator required to provide quarterly reports

Biodiversity / water environment related practical land management and land manager training outputs delivered on the ground:

Delivery of training and knowledge transfer to participating land managers

Priority habitats are placed under appropriate CS management options

Landscape scale delivery of spatially coherent priority habitat restoration and creation projects

Landscape scale delivery of spatially coherent land and water management designed to restore degraded ecosystems and provide climate change mitigation and adaptation

Landscape scale delivery of spatially coherent AES and woodland measures designed to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species

Landscape scale delivery of spatially coherent plantations on ancient woodland sites (PAWS) restoration projects

Landscape scale, spatially coherent delivery of CS options benefitting water quality

Provision of water quality advice to land managers within areas identified as a priority for water quality interventions / synergies

Landscape scale, spatially coherent delivery of CS options designed for flood risk management

capacity outcomes:

Land managers within the facilitated management area have the skills required to deliver CS priorities at the landscape scale

Land management activities across individual holdings within the facilitated management area are aligned to deliver at the landscape scale (rather than just the holding scale)

Work undertaken by the group of land managers within the facilitated management area supports and complements related partnerships and initiatives operating within the local landscape

Land management activity within the facilitated management area supports and complements related partnerships and initiatives operating within the local landscape

Changes in land manager attitudes and awareness (e.g. improved trust of government bodies, self-efficacy carrying out environmental land management techniques etc)

Partnership working and land manager capacity impacts:

Landscape scale sustainable land management practice embedded within general operations

Land managers have the confidence and awareness to engage with wider land management stakeholders and initiatives (e.g. NIAs, AONB partnerships etc) to deliver landscape scale action

Land managers have the confidence and awareness to pursue other sources of funding for landscape scale land management action (e.g. HLF)



Figure 4: Countryside Stewardship Facilitation Fund – example theory of change Note: Further information on the suggested keystone elements within the logic model is provided in the text.

Theory of change evidence (see Appendix 3 for further information) 1 LUC et al (2013) addresses concerns that previous monitoring of agri-environment schemes (AES) had not fully captured the landscape character and quality effects of these schemes. The

study assessed effects at two scales through the collection of new primary data: 1) local level – assessment of field survey data; and 2) strategic / National Character Area (NCA) level – assessment of digital data. The study found that AES has met its objective of conserving and enhancing landscape character and quality. The study does not differentiate between AES option delivery at the holding and landscape scale though it does discuss the key structural elements of landscape that AES has helped to deliver (e.g. hedgerows, small farm woodlands, semi-natural habitats etc). These structural elements of landscape can support biodiversity outcomes by facilitating landscape scale ecosystem processes (e.g. species migration, colonisation, interbreeding etc) through the provision of habitat networks (Watts et al, 2005; Smith et al, 2008; Briers, 2011; SNH, 2011). These structural landscape elements and the habitat networks they form can be more effectively planned and delivered at the landscape scale (ibid) e.g. by identifying strategic functional or structural ‘gaps’ in the network and appropriate land use / management intervention. 2 Moreno-Mateos and Comin (2010) discuss the need to integrate scales and objectives for the effective planning and delivery of wetland restoration and creation projects in agricultural

landscapes. They suggest that some wetland management objectives (i.e. ecosystem services) are best planned for at the catchment or sub-catchment (i.e. landscape) scale. Key examples are wild species populations / biodiversity, ecological networks / landscape heterogeneity, hydrological cycle function and water quality. The paper argues that some key land / water management objectives and ecosystem services can be best delivered through a more strategic planning approach – i.e. where the management objectives / ecosystem services are spatially explicit. 3 BTO (2005) investigated the potential of AES to contribute to the enhancement of farmland bird populations through the provision of better winter feeding conditions for seed eating

(granivorous) birds – e.g. over winter stubble retention. The study focussed on the spatial configuration of relevant AES options and the possible benefits for key farmland bird species. The



study found that enhancing winter food sources through the use of relevant AES options (e.g. payments for over winter stubble retention) can help to stem and reverse population declines. The study also found that results were most effective when enhanced food sources were provided at least 500m-1000m apart and that this sort of spatial configuration could be planned and delivered most effectively at the landscape scale. This provides an example of how the planning and delivery of land management for the conservation and enhancement of certain wild species populations can be delivered most effectively at the landscape scale. 4 Diebel et al (2008) considered optimal approaches for the planning and delivery of management interventions to reduce diffuse agricultural pollution. The study highlights the need for

planning at the field and landscape (catchment) scale for the most effective results – i.e. a strategic, landscape scale approach is required to ensure that the activities of individual land managers (at the field scale) is coordinated, however a field scale assessment is also required to identify nutrient loading and management opportunities to then feed into the prioritisation of intervention across the catchment. Diebel et al (ibid) propose the adoption of an ‘aggregated / targeted’ approach that would require close coordination of land managers across the catchment (e.g. to capture field scale data on existing conditions / land management, to ensure that land managers deliver the land management recommendations and to explain why management is being spatially targeted to some fields and not others).



Figure 4 contains several keystone elements that could be critical to the overall success of CSFF projects, as explained:

Support for land managers to check / reposition land management activity to support enhanced landscape scale delivery: would ensure that land management delivered at the holding and field scale is aligned to management objectives identified at the landscape scale. This is a critical activity to ensure that existing and new actions of individual land managers are supportive and complementary, as part of an overall strategy.

Group agreement: provides for a degree of formality to partnership working between participating land managers. I t will help ensure that the actions of individual land managers are delivered in line with an agreed plan (the facilitation agreement), supporting the management objectives identified at the landscape scale.

Facilitation agreement: formal agreement between Natural England and the applicant setting out detailed planning arrangements for the delivery of agreed Countryside Stewardship outcomes within the facilitated management area. The facilitation agreement is a landscape scale management plan setting out the management objectives and the range of activities to be delivered. It is a critical output guiding the activity of participating land managers in the facilitated area.

Preparation and submission of spatially coherent / targeted / concentrated CS applications: a key premise of CSFF is its ability to promote the greater uptake of relevant Countryside Stewardship options in a more spatially coherent and targeted manner, at the landscape scale. CSFF should result in the more concentrated uptake of environmental land management across distinct areas of countryside to support the delivery of relevant CS outcomes. Facilitator support for CS applications will be instrumental in securing funding for the delivery of environmental land management on the ground.

Landscape scale delivery of spatially coherent priority habitat restoration and creation projects: practical land management output underpinning the delivery of most (if not all) of the anticipated biodiversity and water environment related CSFF outcomes.

Landscape scale, spatially coherent delivery of CS options benefitting water quality: key practical land management output underpinning the delivery of many of the anticipated biodiversity and water environment related CSFF outcomes.

Provision of water quality advice to land managers within areas identified as a priority for water quality interventions / synergies: practical land / water management training and advice output underpinning the delivery of most (if not all) of the anticipated water environment related CSFF outcomes.

Improved condition and increased extent of priority habitats: outcome supporting the delivery of other outcomes (e.g. habitat connectivity improvements, improvements to overall health and functioning of ecosystems, wild species population enhancements etc) and underpinning many of the anticipated impacts of CSFF (e.g. enhanced resilience of wild species populations to climate change).

Reduction in agricultural pollution loading: outcome underpinning the delivery of CSFF water environment impacts – i.e. water quality improvements / compliance with WFD targets.



5.4 Additional logic model issues

There are some important logic model issues that will need to be considered as part of any eventual CSFF evaluation:

Incorporation of key Biodiversity 2020 Outcome 1C – Ecosystems Approach Self-Assessment Criteria and Method elements with the CSFF logic model: the Steering Group indicated the importance of this document which is focussed on integrated / ecosystems approach based land management at the landscape scale. It includes a range of useful criteria that could potentially be incorporated as activities and associated outputs within the CSFF logic model. Some of these would be non-essential ‘nice to dos’ and potentially beyond the core scope of the CSFF. These activities along with notes on the rationale for their inclusion (or not) within the CSFF logic model are given in Appendix 4.

Consideration of ecosystem services within the logic model: at present, the logic model identifies ecosystem services as outcomes and impacts implicitly and with a distinct focus on biodiversity / habitat networks (i.e. ecosystem processes / intermediate services), water quality and flood control. This reflects the overall objectives and focus of CSFF on biodiversity and the water environment (see Chapter 1). The Steering Group suggested some additional services that could be added (e.g. pollination, carbon storage) and there remains a decision as to whether these are included within the logic model. This will have implications for evaluation design beyond the scope of this commission e.g. further focused literature review work to evidence additional theory of change elements, identification of additional data sources for the counterfactual etc.

The importance of context / use of a ‘realist’ evaluation approach: the logic model as presented at Table 5 has been developed from the perspective of a ‘theory of change’ based approach to evaluation design. It may be that a ‘realist’ based design20 is as (or more) appropriate for the CSFF M&E given the potential importance of context determining the successful delivery of anticipated outcomes and impacts – e.g. patterns of land ownership within the facilitated area, history of previous attempts at partnership working, existing levels of land manager training and awareness etc. Depending on various issues (e.g. available budget for the CSFF M&E etc) there may be benefits in considering a more ‘realist’ evaluation design. This could, for example, involve an assessment of context for each CSFF project considered in the M&E and the development of a more specific logic model and M&E plan to account for this context. The Steering Group suggested that a combined ‘theory of change’ / ‘realist’ evaluation design could be appropriate, given the importance of context.

As the logic model is of central importance to CSFF M&E design, deliberation on these issues will inform the final design adopted. Options for CSFF M&E design are presented in Chapter 3 to help facilitate this deliberation.

20 Logic mapping – hints and tips for better transport evaluations (Tavistock Institute, 2010a): https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/3817/logicmapping.pdf [accessed 08/05/15]




6. Data and information

6.1 Discussion with Defra, Natural England and the Environment Agency

Discussions with Defra, Natural England and the Environment Agency drew out the requirements for monitoring and evaluation and potential data sources (baseline and annual figures), tools and modelled approaches.

Requirements of monitoring and evaluation of CSFF:

To know the added value that facilitation brings.

Measures to demonstrate outputs between facilitated and non-facilitated areas.

To understand the spatial coordination of agreements, proximity and management of options within agreements.

To cater for small scale facilitation activity within, for example, whole catchments.

To assess costs and benefits and whether there are more benefits for some costs or few benefits for more costs.

To measure and monitor the inputs, activities and outputs of the facilitation itself.

Opportunities for learning and sharing good practice from NIAs.

Data sources and tools:

Scheme option uptake data is available for quantitative and spatial analysis

While the expectation is that new agreements will be mapped in a similar way to the mapping of HLS agreements (subject to the IT platform functioning), other existing agreements might not be mapped.

There is potential for additional sampling / data gathering, including data associated with facilitation activities.

Modelling of activities for improved water quality can be undertaken using FARMSCOPER21 or the Environment Agency’s Catchment Change Matrix22, which have been used in monitoring and evaluation of Catchment Sensitive Farming.

Factors influencing the baseline conditions for comparing CS agreements / facilitated and non-facilitated areas:

No previous facilitation / coordination of land management activity

All or part of holding / area previously included within a facilitated landscape scale initiative

Previous and / or continuing AES agreements

6.2 Review of data used in monitoring and evaluation of NIAs

A range of data sources has been used by NIAs in collating information for M&E against a set of defined indicators (Appendix 2). Several national datasets used include SSSI condition data, priority habitat inventory, national forest inventory, AES option uptake and BARS. National monitoring and

21 FARMSCOPER, developed by ADAS, is used to model activities undertaken to reduce diffuse pollution and improve water quality within catchments. 22 Modelling approach developed by the Catchment Sensitive Farming Team.



national modelling approaches were also used but often the lack of annual or end of project or post-project repeat modelling limits the value of these approaches for evaluation – unless undertaken as a specific, thematic study. In addition, NIAs undertook their own local monitoring of delivery such as, species groups, contributions to connectivity, landscape character and assessment of partnership working.

Locally determined approaches to reporting on contributions to connectivity reflected works to create less fragmented habitats, such as habitat creation and management activities, and which NIAs considered to contribute to connectivity in their local context. M&E of partnership working covered use of resources (finances and time), identification of skill needs and assessment of partnership working. Data were collated through the Natural England monitoring of NIA progress and also via local surveys undertaken by NIAs. A key benefit of the partnership approach was considered to be bringing different organisations and people together under shared project objectives23.

Table 6 illustrates the potential for applying the national datasets (as used by NIAs in biodiversity and ecosystem services M&E) for monitoring and evaluation of CSFF. A number of national datasets and modelling approaches proposed within the indicators were not utilised by the NIAs but may still have potential – e.g. PSYCHIC / catchment sediment release models.

23 NIA M&E Year 2 Progress Report (Collingwood Environmental Planning, 2014a).



Table 6: National information / data sources relating to biodiversity and ecosystem services used in M&E of NIAs and their possible use for M&E of CSFF

Information / data source

Use of data source by NIAs

Use of data source by NIAs Issues with use of data for CSFF M&E Potential for use in M&E of CSFF

BARS Theme: Biodiversity Sub-themes: habitat and species

Extent of priority habitat management activity: extent of priority habitat managed to maintain & / or improve its condition; and extent of areas managed to restore / create habitat.

Extent of specific habitat managed for species needs.

NIA-specific addition of data and attribution of actions to the NIA from other projects contributing to the NIA objectives.

It is understood that there will be a periodic translation of CS options into biodiversity delivery input to BARS by NE nationally (as part of B2020 monitoring requirement).

The BARS tools allow for the collection of actions by third parties, therefore not all the activities within the facilitated area recorded on BARS may be due to CS options (however, as the facilitated area is likely to be small and holdings are likely to be in private ownership, the amount of third party activity could also be small).

Recording information on BARS (to monitor activities) would be an additional activity and not necessarily useful as scheme option uptake is to be recorded as part of CS M&E.

Useful in setting the CS in the context of other activities.

Important for the counterfactual in identifying past actions, baseline and third party actions.

SSSI condition monitoring

Theme: Biodiversity Sub-theme: habitat

Proportion of SSSIs in favourable or recovering condition

SSSI condition monitoring needs to be timed to enable changes in condition resulting from actions to be picked up. Alternate integrated site assessments (ISA) records combining SSSI and land management may provide more valuable assessments of interventions.

Timescale of supported actions likely to be insufficient to draw strong conclusions

SSSI condition monitoring is planned for use in M&E of CS

24.

Condition monitoring a sample of facilitated agreements (at the start and end of the facilitated period) could be undertaken for comparison with a similar non-facilitated area to help determine outcomes of facilitation.

Priority Habitat Inventory (PHI)

Theme: Biodiversity Sub-theme: habitat

Extent of priority habitats. The update schedule for the national PHI is not sufficient to monitor change within a facilitated area or for comparisons between facilitated areas.

CS actions may not be targeting priority habitat areas so may miss contribution to biodiversity and other multi-objective outputs.

Dataset (to monitor outputs) not useful over the time frame of CSFF. Local updates to the dataset could be incorporated for the schemes.

National Forest Theme: Ecosystem Percentage of woodland in active National Forest Inventory is updated As data on woodland management is

24

Defra, Natural England, Forestry Commission, Environment Agency and English Heritage (2014) CAP Pillar 2 Rural Development Programme ELMS M&E Plan 2015-2020



Information / data source

Use of data source by NIAs

Use of data source by NIAs Issues with use of data for CSFF M&E Potential for use in M&E of CSFF

Inventory and Woodland in Management Indicator

Services Sub-theme: Provisioning services

management annually and provides context for assessing woodland in active management.

The woodland in active management indicator is updated quarterly and relies on information supplied on woodlands in management.

available from option uptake through CS agreements (EWGS) it is not necessary to rely on the National Woodland Inventory or Woodland in Management Indicator for M&E in facilitated areas.

National Biodiversity Climate Change Vulnerability Model (NBCCVM)– Natural England

Theme: connectivity Used component calculations of the vulnerability model to report on functional connectivity. Initial runs of the model were supported by NE but tool available for repeat runs.

Might act as a suitable baseline as model grid is high resolution and model is available for re-run (e.g. on new priority habitats data layers. Can be run for smaller areas and as repeat models

May not be sensitive to the scale of the modifications being made by the CSFF actions and needs baseline and repeat data and runs to indicate change

Alternate local calculation methods/tools available and promoted within NIAs (e.g. ARCH

25, BEETLE

26 )

Data on baseline and CS actions will be recorded and need to be entered and calculated.

25 Tool developed under the Interreg project led by Kent County Council: Assessing Regional Habitat Change (ARCH). 26 Watts (2005)

Draft Report July 2015


6.3 Information and data requirements based on the CSFF logic model

The focus of this section on inputs, activities and outputs results from the strong steer from Natural England that these aspects formed a current gap in the CS M&E framework. Monitoring of CS outcomes and impacts are covered in the framework, but there is a gap in the sampling approach to facilitation agreements. This aspect is taken up in Chapter 3 on the scope of the CSFF evaluation.

As part of the CS M&E, data will be collected via:

Application data:

o Applicants will apply online and data held in the database that sits behind the online system.

o GIS maps will be produced via the mapping interface (once current problems have been resolved).

Payments data.

CS monitoring (to be determined).

For CSFF M&E, in addition to the information available through the conventional CS M&E, facilitators are expected to submit quarterly reports. The CS M&E and the facilitator quarterly reports are unlikely to provide information on the social dimension of facilitation, i.e. how facilitation has influenced the outputs, outcomes and impacts of CS delivery at the landscape scale. This information may best be provided by targeted questions to facilitators and also through interview surveys with land managers in facilitated areas and with a comparative sample in non-facilitated areas.

Potential approaches to CSFF M&E of the inputs, activities and outputs identified in the logic model have been considered (Appendix 5). For each input, activity and output, an indicator has been proposed and the main source of data for the M&E listed. The priority / keystone elements are summarised in Tables 7, 8 and 9 and linked to the evaluation approaches proposed in Chapter 3 (indicators in bold red text link to keystone elements identified in the logic model). The approaches listed in Tables 7, 8 and 9 contribute to evaluation of costs, the spatial coherence of delivery, partnership working and the facilitation itself.

As well as additional data, additional resources will be required for the suggested analysis of the CSFF M&E. Resources required are likely to be more intensive for some of the suggested approaches particularly where modelling is proposed. However, at this scoping stage it is not possible to provide any detail of the exact resources required. The full CSFF evaluation study would need to elaborate the appropriate indicators according to which evaluation option is selected.



Input indicators and data sources

Table 7: Potential indicators, approaches and data sources for CSFF M&E of facilitation inputs Indicator Purpose of data capture Data requirements / information to

record Data source Use of data Evaluation

option Potential data for use in assessing

CSFF Main source / point of data capture Potential use of data in evaluating

CSFF Overall option

Cost of facilitation.

To determine overall costs of facilitation and to use in analysis of benefits against costs.

Grant offer

Additional funding obtained

Payments for options prioritised through CSFF

CSFF grant offer

CSFF application form (Questions 24, 26, 27 & 28)

Quarterly Progress Reports

CS application data Payments data

Comparison of benefits against costs in facilitated areas with paired non-facilitated areas (Option 1a).

Comparison between different facilitated areas in relation to funding received and projected delivery or option uptake (outputs) (Option 2).

Analysis of the amount of CSFF funding per facilitated area in relation to outputs (Option 2): o No. of land managers o No. of options per unit area o Area of land covered by

individual options o Cumulative area of land

covered by all options (an area of land may be covered by more than one option)

o Number of contiguous options

3 (Evaluates cost input)

Facilitator expertise.

To help determine whether facilitator expertise has any influence on outputs within the facilitated area.

List of expertise of each facilitator categorised by: o Agriculture o Forestry o Water management o Ecology

o Other

CSFF application form (Questions 6, 7 & 8).

Additional data capture (via the facilitator) may be required to determine the baseline (see Table 11).

Preparation of matrix of expertise of facilitators for all facilitated areas for use in assessing influence of expertise on activities and outputs (Option 2).

2 (Evaluates facilitation

input)

External trainer expertise.

To help determine whether the expertise of external trainers has influenced the activities and outputs within the facilitated

Proposed external training providers.

CSFF application form.

Additional data capture (via the facilitator) may be required to determine the baseline (see Table

Preparation of matrix of expertise of external training providers in facilitated areas for use in assessing influence of external


input)



Indicator Purpose of data capture Data requirements / information to record

Data source Use of data Evaluation option

area. 11). trainer expertise on activities outputs (Option 2).

Type of information sharing.

To determine the knowledge input of land managers to the partnership and the influence of knowledge sharing on outputs.

Existing information and knowledge held by land managers.

Additional surveys with land managers to capture required data.

Assessment of baseline land manager knowledge for use in assessing change (Option 2).


input / partnership

working)

Activities indicators and data sources

Table 8: Potential indicators, approaches and data sources for CSFF M&E of facilitation activities Indicator Purpose of data capture Data requirements / information to


option

Potential data for use in assessing CSFF

Main source / point of data capture Potential use of data in evaluating CSFF

Overall option

Process of scoping CS priorities to be addressed within the facilitated management area.

To determine whether the approach to scoping priorities has influenced the selection of types and locations of options and hence whether this has affected the outputs.

Approach to scoping the priorities

The list of CS priorities to be covered by the facilitation agreement and area (ha) covered

Information should be available from the CSFF application form (Question 16)

Comparison of CS priorities identified in CSFF applications with options included in CS agreements for holdings within facilitated areas. This should indicate follow through from CSFF application to holding agreements (Option 2).

If possible, a comparison could be made between different scoping approaches to assess effectiveness in identifying priorities (Option 2).

2 (Evaluates facilitation activities)

Process of developing the group agreement and group working.

Information to be used in determining the effectiveness of group working and group cohesiveness.

Content of the group agreement

Activities undertaken to reach agreement.

Group working activities undertaken

CSFF application form (Question 12)

Quarterly Progress Reports.

Assessment of the effectiveness of group working (Option 2).

2 (Evaluates facilitation activities

and partnership

working)





Facilitator support to land managers for enhanced delivery at the landscape scale.

To determine whether the support provided to land managers has influenced the outputs.

Type of support provided

Number of hours support provided

Successful & unsuccessful applications within CSFF area.

Classification of all CS applications according to: o Within / outside a facilitated

area o Previously included within a

facilitated landscape scale initiative or not

o Support with preparation of CS application (from any source) received / not received.



Activity Evaluation Forms.

Quantitative data on facilitator input could be used to calculate input hours per ha (Option 2) for: o Comparison with other

facilitated areas. o Assessment of the level of

activity to achieve outputs.

Qualitative assessment of the effect of current and previous facilitation (Option 2).


Process of delivering / receiving training and knowledge transfer activities.

To determine whether the training received by land managers results in any changes in knowledge, skills, and attitudes.

Type, content and length of training and knowledge transfer activities, for example: o Land management o Machinery / equipment use o Fertilizer, pesticide use o Water quality related o Ecology

Purpose of the training

Planned learning outcomes

CSFF application form (Questions 20, 21 & 22)


Activity Evaluation Forms

Further information on changes in land manager knowledge, skills, and attitudes to be obtained through semi-structured interviews with a sample of land managers from all CSFF projects (to include questions on learning gained and application of learning)

Correlation of the types of training that land managers engage in with any changes in knowledge, skills, and attitudes (Option 2).


Process of maintaining links between land managers and with related local partnerships / initiatives.

To determine whether the types of activities that land managers engage in results in any changes in knowledge, skills, and attitudes identified through an interview survey. Questions required in survey on learning gained and application.

Activities undertaken to maintain the links between landowners and local partnerships and initiatives.

CSFF application form (Questions 23 & 25)


Activity Evaluation forms.

Further information on changes in land manager knowledge, skills, and attitudes to be obtained through semi-structured interviews with a sample of land managers from all CSFF projects

Correlation of the types of activities that land managers engage in with any changes in knowledge, skills, and attitudes (Option 2).

2 (Evaluates facilitation activities /

partnership working)





(to include questions on the usefulness of engaging with related local partnerships and initiatives and any resulting changes in the management of the holding).

Land management activities to deliver priorities at landscape scale.

To help determine whether the land management activities undertaken have resulted in improved delivery or priorities at the landscape scale.

Report of activities

Map of spatial distribution of CS priorities.

Activities that land managers are doing differently.



Additional survey information may be required, through semi-structured interviews with a sample of land managers from all CSFF projects, on the process of delivering CS priorities at the landscape scale.

Analysis of effectiveness of facilitated activities at delivering CS priorities at the landscape scale (Option 2).

2 (Evaluates

partnership working)

Outputs indicators and data sources

Table 9: Potential indicators, approaches and data sources for CSFF M&E of facilitation outputs Indicator Purpose of data capture Data requirements / information to


option Potential data for use in assessing

CSFF Main source / point of data capture Potential use of data in evaluating

CSFF Overall option

Approach to group operation identified in the group agreement.

To provide information to determine the effectiveness of the group agreement and group operation.

Date of the agreement


Identified roles

How the group operates and works together


Group Agreement


Analysis of links between group agreement and effectiveness of group working (Option 2).

2 (Evaluates facilitation outputs /

partnership outputs)

Content of the facilitation agreement.

To provide information on the agreed CS priorities and benefits to be delivered by the CSFF project.

Benefits (outcomes) to be delivered

Reasons why outcomes are expected to be better through CSFF than if land managers were acting independently

CSFF agreement.

CSFF application form (particularly Question 22)

Quarterly Reports.

Comparison of the agreed list of CS priorities in the facilitated area with a paired non-facilitated area (Option 1a).

Assessment of whether the benefits identified in the CSFF application form will be realised

3 (Evaluates facilitation outputs /

partnership outputs)





(Option 2).

Complementary and spatially coherent CS applications within the facilitated management area

To determine the difference made by facilitation on the achievement of complementary and spatially coherent CS applications.

Map showing spatial density and coherence of options (type, location and number of options).

CS application data (including GIS mapping).

Comparison of the spatial coherence of applications within facilitated areas with paired non-facilitated areas (Option 1a & 2).

Comparison of the spatial density of options within facilitated areas with paired non-facilitated areas (Option 1 & 2).

1 & 2 (Evaluates facilitation outputs /

spatial coherence)

Training delivered and knowledge transferred.

To provide information to determine the influence of training and knowledge transfer on achievement of outcomes.

Type of training or other form of knowledge transfer delivered.

Number of events held.

Names and numbers of participating land managers.



Further information on the usefulness and application of learning to be obtained through surveys with land managers.

Assessment of the use and value of training received by land managers (Option 2)


outputs / up skilling

outputs)

Priority habitats placed under appropriate CS management options.

To determine whether priority habitats have been placed under appropriate CS options.

Location and area of priority habitats under difference CS management options.

CS application data.

Ground-truthing of CS options in relation to priority habitats.

Comparison between facilitated areas and paired non-facilitated areas of the appropriateness of CS options applied to priority habitats (Option 1a & 2).

1 & 2 (Evaluates

facilitation)

Spatially coherent priority habitat restoration and creation projects.

To determine the difference made by facilitation on the achievement of spatially coherent priority habitat restoration and creation projects.

Map showing spatial delivery of options.


BARS.

Use of NBCCVM to test for delivery of spatially coherent priority habitat restoration and creation projects in facilitated and paired non-facilitated areas (Option 1a & 2).

Proportion of land covered by measures for priority habitat restoration and creation projects in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

1 & 2 (Evaluates

spatial coherence)

Spatially coherent land and water

To determine the difference made by facilitation on the achievement of spatially coherent land and water



Use of NBCCVM to test for delivery of spatially coherent land and water management to

1 & 2 (Evaluates

spatial





management measures.

management measures. restore degraded ecosystems and provide climate change mitigation and adaptation (Options 1a & 2).

Proportion of land covered by measures for land and water management to restore degraded ecosystems and provide climate change mitigation and adaptation in facilitated areas compared to paired non-facilitated areas (Options 1a & 2).

coherence)

Spatially coherent AES and woodland measures.

To determine the difference made by facilitation on the achievement of spatially coherent AES and woodland measures.



Use of NBCCVM or another model to test for spatial coherence of AES and woodland measures designed to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species (Option 1a & 2).

Proportion of land covered by measures designed to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

1 & 2 (Evaluates

spatial coherence)

Spatially coherent plantations on ancient woodland sites (PAWS).

To determine the difference made by facilitation on the achievement of spatially coherent plantations on ancient woodland sites (PAWS).



Use of NBCCVM or another model to test for delivery of spatially coherent plantations on ancient woodland sites (PAWS) restoration projects (Option 1a & 2).

Proportion of land covered by plantations on ancient woodland sites (PAWS) restoration projects (need to take account of potential overlap of measures on the

1 & 2 (Evaluates

spatial coherence)





ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

Spatially coherent delivery of CS options benefiting water quality.

To determine the difference made by facilitation on the achievement of spatially coherent delivery of options benefiting water quality.



Use of NBCCVM or another model to test for delivery of spatially coherent delivery of CS options benefitting water quality (Option 1a & 2).

Proportion of land covered by of CS options benefitting water quality (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

1 & 2 (Evaluates

spatial coherence)

Water quality advice provided.

To determine the effectiveness of water quality advice.

Number and types of option uptake.

Quarterly Progress Reports. Analysis using FARMSCOPER or the Catchment Change Matrix (Option 2).

2 (Evaluates

spatial coherence)

Spatially coherent CS delivery for flood risk management.

To determine the difference made by facilitation on the achievement of spatially coherent CS delivery for flood risk management.



Use of NBCCVM or another model to test for spatially coherent delivery of CS options designed for flood risk management (Option 1a & 2).

Proportion of land covered by CS options designed for flood risk management (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

1 & 2 (Evaluates

spatial coherence)



7. Conclusions

This study has drawn on the experiences of the NIA M&E in seeking to scope out the possible options for evaluating the CSFF consistent with the wider CS M&E. In particular, it has developed a CSFF-specific logic model and theory of change, supported by a focused literature review, to facilitate the development of possible options, along with the implications these would have in terms of data requirements and broad costs (e.g. based on the requirement for additional ecological survey work).

The study has sought to identify where data sources already exist and how these could be used to inform lower cost options for the CSFF M&E, as well as identify what other data sources could be used to furnish more elaborate and therefore costly options for evaluation, i.e. where additional new data collection, such as ecological surveys or interview surveys, would be required.

Significant challenges have also been identified for undertaking the CSFF M&E, including:

Context factors, e.g. for comparing facilitated and non-facilitated areas (such as risk of selection bias; the need to eliminate all other variables for a robust counterfactual).

Sampling strategy and sample size (depending on the evaluation option selected and the number of CSFF agreements made and holdings covered).

Generalisability of M&E findings – the tension between what may be regarded as a more robust (quantitative) counterfactual approach to M&E and the costs of delivering this approach for a representative (generalisable) sample size.

Three theoretical options are suggested:

1) A low cost ‘theory of change’ option that draws on existing data, supplemented with qualitative data as required (‘broad and shallow’)

2) A relatively high cost ‘experimental’ option which is likely to require significant new quantitative (ecological) data collection at the local/landscape scale (‘narrow and deep’)

3) A combined option, where some additional quantitative data collection is used to support and further evidence a theory of change approach.

The pros and cons of each option are listed in Table 10.



Table 10: Pros and cons of the three suggested evaluation options

Option Pros Cons Option 1 – Experimental / counterfactual approach

Provides a clear demonstration of what happened with the intervention (CSFF), compared to what would have happened anyway (no CSFF).

Costs anticipated to be high given the requirement for fresh ecological survey work (at a scale unlikely to be already available) to evidence the delivery of outcomes and impacts in facilitated and non-facilitated areas.

A focus on outcome and impact elements of the CSFF logic model would make good use of existing outcome and impact metrics from the CS M&E regime.

May involve the collation of some data and information on CSFF activity and output metrics / questions, in particular, where an assessment of costs and benefits of facilitation is an evaluation objective.

Identifying paired / suitably similar groups of holdings for Option 1a may be extremely difficult given the need to eliminate all variables except for facilitation.

Sample size is less likely to be representative due to the resource implications of carrying out fresh ecological survey work across paired groups of holdings.


Approach useful where the links between activities and outcomes are not straightforward and / or where providing comprehensive evidence on final impacts may not be possible.

A degree of primary data collation will be required (this is likely to be much less onerous than in Option 1).

Approach would focus on activities and outputs which are easier to attribute to facilitation.

To increase the robustness of the approach, further validation of the model would be required.

Likely to be the least cost option, because many of the necessary data will already be collected.

Consideration could be given to the inclusion of the additional activities identified through the review of the Biodiversity 2020 Outcome 1C self-assessment guide, which would add weight to evidencing the delivery of relevant ecosystem / landscape scale outcomes and impacts.

Sample size may be more representative (compared to Option 1) due to the less onerous data collection and reliance, to a degree, on secondary data

Qualitative data gathering (e.g. through interviews) could be tailored to available resources.


Brings the key strengths of Options 1 and 2 together in a combined approach.

Likely to be medium cost option between Options 1 and 2.



Whichever option (or combination of option components) is adopted, it will need to be proportionate to the scale of the intervention being evaluated, i.e. the available CS facilitation fund, the uptake of that fund in practice and the relative spatial extent of holdings under facilitated management, compared to non-facilitated.

From these Options, three Sampling Strategies are suggested (as discussed in Section 3.5) that combine elements of these Options. All the strategies include the need for a theory of change model (and therefore Option 2) as the anchor for understanding the mechanisms by which change is brought about by the activities undertaken. Relative and indicative costs for each sampling strategy are summarised in Table 11. The indicative costs of these sampling strategies range from 1.5% to 8.5% of the total budget for CSFF of £7.2 million. The CSFF represents 0.75% of the total budget available for Countryside Stewardship of £925 million.

Table 11: Summary of indicative costs for suggested sampling strategies

Elements Range of indicative costs

Strategy 1 Strategy 2 Strategy 3 Option 2 (Theory of Change) £60,000-

£75,000 £60,000-£75,000

£60,000-£75,000

Project management, reporting etc (fixed sum for all strategies)

£50,000-£70,000

£50,000-£70,000

£50,000-£70,000


£90,000-£450,000


£7,500- £17,500


£10,000-£15,000


£25,000-£30,000


£135,000

Total indicative costs £207,500-£612,500

£110,000-£145,000

£280,000-£325,000



8. References and bibliography

Atkinson, P.W., Cook, S., Grant, A. & Rehfisch, M.M. (2001) The success of creation and restoration schemes in producing intertidal habitat suitable for waterbirds. English Nature Research Report No. 425.

Briers, R. (2011). Habitat Networks – Reviewing the Evidence Base: Final Report. Report to Scottish Natural Heritage: Ref 29752. Edinburgh: Edinburgh Napier University.

British Trust for Ornithology (2005). The consequences of spatial scale for agri-environment schemes designed to provide winter food resources for birds. Report to Defra. Thetford: British Trust for Ornithology. [Defra Project Code BD1616]

CCRI (2012) Economics of co-ordination in Environmental Stewardship. Defra & Natural England.

Cole, L., Deane, R., Parker, S., Cooper, H., Manson, D., Cookson, W., Martin, J., Porter, J., Jagota, L., Haines-Young, R., and Swanwick, C. (2013). Monitoring the effects of Environmental Stewardship on Landscape Character and Quality. Report to Defra. London: Land Use Consultants. [Defra Project Code BD503]

Collingwood Environmental Planning (2013) Monitoring and Evaluation of Nature Improvement Areas: Year 1 (2012-2013) Progress Report. Report to Defra and Natural England. [Defra project code WC1061]

Collingwood Environmental Planning (2014a) Monitoring and Evaluation of Nature Improvement Areas: Year 2 (2013-2014) Progress Report. Report to Defra and Natural England. [Defra project code WC1061]

Collingwood Environmental Planning (2014b) Evaluating the NIAs Against the Counterfactual – Method Statement and Work Plan, 19 December 2014. [Defra project code WC1061]

Corbett, A., Hyslop, M., and Smith, M. (2009). Landscape scale approach to conservation management: Glasgow and Clyde Valley case study. Ecological Networks: Science and Practice: Proceedings of the 16th Annual IALE (UK) Conference. Edinburgh University, Edinburgh 1-3 September 2009. Garstang: IALE (UK), pp.161-168.

Critchley, C.N.R., Allen, D.S., Fowbert, J.A., Mole, A.C. & Gundrey, A.L. (2004) Habitat establishment on arable land: assessment of an agri-environment scheme in England, UK. Biological Conservation, 119, 429-442.

Defra (2011) Biodiversity 2020: A strategy for England’s wildlife and ecosystem services.

Defra (2012) Biodiversity Offsetting Pilots Technical Paper: the metric for the biodiversity offsetting pilot in England

Defra (2014a) Rural Tourism Package: Monitoring and Evaluation Framework. Available at: https://www.gov.uk/government/statistics/rural-tourism-feature-report-december-2011 [accessed 11/05/15]

Defra (2014b) The new Common Agricultural Policy schemes in England: December 2014 update. Available at: https://www.gov.uk/government/publications/cap-reform-december-2014-update [accessed 19/3/15]

Defra (2014c) National Pollinator Strategy: for bees and other pollinators in England

Defra (2015a) Guide to Countryside Stewardship: facilitation fund. Available at: https://www.gov.uk/government/publications/guide-to-countryside-stewardship-facilitation-fund/guide-to-countryside-stewardship-facilitation-fund [accessed 19/3/15]


https://www.gov.uk/government/publications/cap-reform-december-2014-update

https://www.gov.uk/government/publications/guide-to-countryside-stewardship-facilitation-fund/guide-to-countryside-stewardship-facilitation-fund




Defra (2015b) The new Common Agricultural Policy schemes in England: February 2015 update. Available at: https://www.gov.uk/government/publications/cap-reform-february-2015-update [accessed 19/3/15]

Defra Press Release (2015c) Available at: https://www.gov.uk/government/news/new-fund-to-support-joined-up-action-on-the-environment [accessed 19/3/15]

Defra, Natural England, Forestry Commission, Environment Agency and English Heritage (2014) CAP Pillar 2 Rural Development Programme ELMS M&E Plan 2015-2020.

Diebel, M.W., Maxted, J.T., Nowak, P.J., and Vander Zanden, M.J. (2008). Landscape planning for nonpoint source pollution reduction 1: A geographical allocation framework. Environmental Management, 42, pp.789-802.

Fagan, K.C., Pywell, R.F., Bullock, J.M. & Marrs, R.H. (2008). Do restored calcareous grasslands on former arable fields resemble ancient targets? Journal of Applied Ecology, 45: 1293-1303.

Gibson, C.W.D. (1998) South Somerset SSSIs: a study of neutral grassland succession. English Nature Research Report 266.

Gibson, C.W.D. & Brown, V.K. (1991) The nature and rate of development of calcareous grassland in Southern Britain. Biological Conservation, 58 (Issue 3): 297-316

Grant, M.J. and Edwards, M.E., 2008. Conserving Idealized Landscapes: Past History, Public Perception and Future Management, Vegetation History and Archaeobotany 17, 551-562

Groves, J.A., Waller, M.P., Grant, M.J. and Schofield, J.E., 2012. Long-term development of a cultural landscape: the origins and dynamics of lowland heathland in southern England, Vegetation History and Archaeobotany 21, 453-470

Hardiman, N., and Cathcart, R. (2013). The Synergies Project – Identifying opportunities for the integrated delivery of outcomes across the Biodiversity 2020, Water Framework Directive and Flood and Coastal Risk Management Programmes [not available online].

Harris, J.A., Hobbs, R.J., Higgs, E. & Aronson, J. (2006) Ecological Restoration and Global Climate Change. Restoration Ecology, 14(2): 170-176.

H.M. Government (2011a) The natural choice: securing the value of nature. The Stationary Office Ltd. Available for download at: www.official-documents.gov.uk

H.M. Government (2011b) The Magenta Book: Guidance for evaluation. London, HM Treasury.

Hughes, F.M.R., Stroh, P.A., Adams, W.M., Kirby, K.J., Mountford, J.O. & Warrington, S. (2011) Monitoring and evaluating large-scale, ‘open-ended’ habitat creation projects: A journey rather than a destination. Journal for Nature Conservation, 19:245-253.

Jackson, S.T. & Hobbs, R.J. (2011) Ecological restoration in the light of ecological history. Science, 325: 567-569.

Jones, H.P. & Schmitz, O.J. (2009) Rapid recovery of damaged ecosystems. PLoS ONE 4(5): e5653. Doi: 10.1371/journal.pone.0005653.

Land Use Consultants and GHK Consulting Ltd (2008). Provision of ecosystem services through the Environmental Stewardship scheme. Report to Defra. London: Land Use Consultants. [Defra Project Code NRO 121]

Lawton, J.H., Brotherton, P.N.M., Brown, V.K., Elphick, C., Fitter, A.H., Forshaw, J., Haddow, R.W., Hilborne, S., Leafe, R.N., Mace, G.M., Southgate, M.P., Sutherland, W.A., Tew, T.E., Varley, J. & Wynne, G.R. 2010. Making space for nature: a review of England’s wildlife sites and ecological network. Report to Defra. http://archive.defra.gov.uk/environment/biodiversity/documents/201009space-for-nature.pdf

https://www.gov.uk/government/publications/cap-reform-february-2015-update

https://www.gov.uk/government/news/new-fund-to-support-joined-up-action-on-the-environment


http://archive.defra.gov.uk/environment/biodiversity/documents/201009space-for-nature.pdf



Moreno-Mateos, D., and Comin, F.A. (2010). Integrating objectives and scales for planning and implementing wetland restoration and creation in agricultural landscapes. Journal of Environmental Management, 91, pp.2087-2095.

Morris, R.K.A., Alonso, I., Jefferson, R.G. & Kirby, K.J. (2006) The creation of compensatory habitat – can it secure sustainable development? Journal for Nature Conservation, 14: 106-116.

Natural England (2014) NELMs national targeting framework.

Natural England & Defra (2015) Countryside Stewardship: Statements of Priorities. Available at: https://www.gov.uk/government/collections/countryside-stewardship-statements-of-priorities [accessed 19/3/15]

Natural England (undateda) CSFF Application Form. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413510/facilitation-fund-application-form.pdf [accessed 19/3/15]

Natural England (undatedb) CSFF Assessment and Scoring Process. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413777/CSFF-scoring-process.pdf [accessed 19/3/15]

Rey Benayas, J.M., Newton, A.C., Diaz, A. & Bullock, J.M. (2009). Enhancement of biodiversity and ecosystem services by ecological restoration: a meta-analysis. Science, Vol. 325: 1121-1124.

Smith, M., Moseley, D., Chetcuti, J., and de Loanni, M. (2008). Glasgow and Clyde Valley Integrated Habitat Networks. Roslin: Forest Research.

Tavistock Institute. (2010a) Logic mapping – hints and tips for better transport evaluations. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/3817/logicmapping.pdf [accessed 08/05/15]

Tavistock Institute. (2010b) Guidance for transport impact evaluations – choosing an evaluation approach to achieve better attribution. Available at: http://www.tavinstitute.org/wp-content/uploads/2013/01/Tavistock_Report_Guidance_for_Transport_Evaluations_2010.pdf [accessed 08/05/15]

TEEB. (2009). The Economics of Ecosystems and Biodiversity for National and International Policy Makers - Chapter 9. www.teebweb.org

The Terrestrial Biodiversity Group. (undated). Biodiversity 2020 Outcome 1C – Ecosystem Approach Self-Assessment Criteria and Method [not available online].

Shaw, H. and Whyte, I., in press. Land management and biodiversity through time in upper Ribblesdale, North Yorkshire, UK: understanding the impact of traditional management. In Agnoletti, M. and Rotherham, I.D. (Eds.) Cultural Severance and the Environment - The Ending of Traditional and Customary Practice on Commons and Landscapes Managed in Common. Springer.

Walker, K.J., Stevens, P.A., Stevens, D.P., Mountford, J.O., Manchester, S.J. and Pywell, R.F. (2003) The restoration and re-creation of species-rich lowland grassland on land formerly managed for intensive agriculture in the UK. Biological Conservation, 119: 1-18.

Watts, K., Humphrey, J.W., Griffiths, M., Quine, C., and Ray, D. (2005). Evaluating Biodiversity in Fragmented Landscapes: Principles. Edinburgh: Forestry Commission.

Webster S. (undated) Powerpoint presentation: Facilitation Funding

https://www.gov.uk/government/collections/countryside-stewardship-statements-of-priorities

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413510/facilitation-fund-application-form.pdf


https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413777/CSFF-scoring-process.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413777/CSFF-scoring-process.pdf





http://www.teebweb.org/



Appendix 1: Note on scope and nature of the Countryside Stewardship Facilitation Fund

Introduction

The purpose of this note is to provide an understanding of the objectives, scope and nature of the Countryside Stewardship Facilitation Fund (CSFF).

Information has been extracted from relevant documents available on Countryside Stewardship (CS) and CSFF, as well as the inception teleconference discussion.

The note provides a summary of CS, which is required in order to understand the requirements for the CSFF.

Background on Countryside Stewardship

Countryside Stewardship (CS) is the name of the new environmental land management scheme, which is part of the 2014 to 2020 Rural Development Programme for England.

The RDPE is made up of four elements27:

1. Countryside Stewardship

2. Countryside Productivity

3. Growth Programme

4. LEADER

CS replaces Environmental Stewardship, the English Woodland Grant Scheme and the capital grants from the Catchment Sensitive Farming programme28.

Countryside Stewardship has 3 main elements29:

Higher Tier (similar to the current Higher Level Stewardship)

o Aimed at the most environmentally important sites and woodlands where complex management is needed (e.g. habitat restoration, woodland creation or tailored measures for priority species)

o Full range of options available and a wide range of the capital grants

o One-to-one advice and support available from Natural England and Forestry Commission advisers

Mid Tier (which will replace Entry Level Stewardship)

o Aims to address widespread environmental issues, such as reducing diffuse water pollution or improving the farmed environment for farmland birds and pollinators

o Not all options available in this tier

o Scheme targeting and scoring aims to encourage applicants to choose options that help achieve the environmental priorities that are important in their wider area. This

27 Defra (2015b) The new Common Agricultural Policy schemes in England: February 2015 update. Available at: https://www.gov.uk/government/publications/cap-reform-february-2015-update 28 Defra (2014b) The new Common Agricultural Policy schemes in England: December 2014 update. Available at: https://www.gov.uk/government/publications/cap-reform-december-2014-update. 29 Ibid

https://www.gov.uk/government/publications/cap-reform-february-2015-update

https://www.gov.uk/government/publications/cap-reform-december-2014-update



means that environmental benefits will not just be on individual holdings but more widespread.

A lower tier of capital grants, including the Hedgerows and Boundaries Capital Grants

The overall priority for CS is biodiversity. Another important priority is the contribution to meeting water quality commitments. The list of priorities for CS covers2:

Biodiversity (main priority)

Water quality (important priority)

Flood management

The historic environment

Landscape character

Genetic conservation

Educational access

Climate change adaptation and mitigation

The scheme also seeks to maximise opportunities to achieve benefits for biodiversity, water quality and flooding together (synergies)30. Integrated action to achieve the objectives of the Biodiversity 2020, Water Framework Directive and Flood and Coastal Risk Management programmes has been identified through the synergies project31. The structure and delivery of CS takes account of the recommendations made by the project.

A targeted approach is being used for Higher and Mid Tier CS agreements to ‘encourage applicants to enter into agreements which deliver the right environmental management in the correct combinations and in the right places’32.

Countryside Stewardship is open to all. It is a competitive process based on scoring of applications. Most multi-year agreements will be for 5 years. Longer agreements may be available in some exceptional circumstances. Applications for CS can be submitted from July 2015 and payments will begin in 2016.

Targeting framework and targeting statements

Targeting framework

Evaluation of previous schemes has shown that targeting secures better outcomes.

The purpose of targeting33 is to:

Deliver the right agreements and options in the right places

Focus delivery on priority environmental outcomes

Reflect the multi-objective nature of the scheme and enable multiple benefits to be delivered

The Countryside Stewardship targeting framework consists of: ‘a large data matrix (around 400 mostly national datasets; drawn from delivery bodies such as NE, EA and FC and other partners) which is related to the Rural Land Register (RLR) at a parcel level. Each of these datasets is weighted

30 Natural England (2014) NELMs national targeting framework. 31 Hardiman, N., and Cathcart, R. (2013). The Synergies Project – Identifying opportunities for the integrated delivery of outcomes across the Biodiversity 2020, Water Framework Directive and Flood and Coastal Risk Management Programmes [not available online]. 32 Defra (2014b) 33 Natural England (2014)



on a logarithmic scale basis (either 1, 10, 100, 1000). This method provides a good degree of separation and therefore allows the highest priorities to be readily identified. The weightings that have been applied to the datasets reflect Ministerial steers around prioritisation... for delivery for Biodiversity 2020 and Water Framework Directive outcomes. A full data catalogue has been produced that lists the data included in the database and the weighting applied to each dataset alongside additional metadata associated with this’34.

The framework has been used to prepare targeting statements which set out the priorities for each of the National Character Areas (NCA) of England. Bespoke targeting information, including key scheme options for delivering the priorities, for each holding is available via the CAP Information System35

Priority (targeting) statements

The priority statements set out the priorities for achieving the CS objectives (which reflect, for example, the need to achieve targets and actions under Biodiversity 2020 and the Water Framework Directive). Targeting statements have been produced for each National Character Area36, in a standard format using the following headings:

Choosing priorities – this lists the top priorities and other priorities

Biodiversity – top priorities

Water – top priorities

Historic environment – top priorities

Woodland – top priorities

Landscape – top priorities

Multiple environmental benefits

Other priorities

Applicants are required to choose at least one top priority.

Climate change adaptation and mitigation is a cross-cutting objective of CS set out under ‘Other priorities’ in the targeting statements. Resilience of CS priorities (e.g. biodiversity & water) to the impacts of climate change will be achieved by applicants selecting land management options and capital works which support the management of vulnerable features and habitats, including where vulnerabilities are increased by climate change37.

Outcomes for soil structure were raised in the inception teleconference. Soil is mentioned in the targeting framework, although not listed as a priority. It is generally referred to in the targeting statements in relation to soil erosion.

Genetic conservation and educational access do not appear to be covered in the targeting statements38. However, genetic conservation would be covered through the biodiversity priority.

34 Natural England (2014) 35 See: https://capreform.blog.gov.uk/tag/cap-information-service/ 36 See: https://www.gov.uk/government/collections/countryside-stewardship-statements-of-priorities 37 Climate change is covered in the last paragraph of each NCA targeting statement. See, for example: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/411126/NCA001-North-Northumberland-Coastal-Plain.pdf 38 From 4 statements searched (NAC001, NCA008, NAC119 and NAC151)

https://capreform.blog.gov.uk/tag/cap-information-service/

https://www.gov.uk/government/collections/countryside-stewardship-statements-of-priorities

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/411126/NCA001-North-Northumberland-Coastal-Plain.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/411126/NCA001-North-Northumberland-Coastal-Plain.pdf



The Wild Pollinator and Farm Wildlife Package

The Wild Pollinator and Farm Wildlife Package supports the National Pollinator Strategy39. One of the key delivery areas of the strategy is supporting pollinators on farmland.

The Wild Pollinator and Farm Wildlife Package is a specific group of management options that provides benefits for wild pollinators, farmland birds and other farm wildlife. These options will deliver essential resources (such as food, shelter and nesting sites) for wild pollinators and farm wildlife throughout the year2.

The package covers options for:

Pollen / nectar resources for pollinators and chick-food for birds

Nesting sites for birds and pollinators

Winter food for seed-eating birds

Meaningful changes and benefits for pollinators have been observed from Environmental Stewardship where options have been applied to over 3-5% of eligible land.

Under CS, Mid Tier applicants are expected to apply options over 3-5% of land eligible for the scheme and Higher Tier applicants on 5-10% of eligible land.

CS Capital grants

Capital grants are intended to support farming and forestry that benefit the environment. They are separate to the capital grants offered through the Higher or Mid Tier40. The grants41 include:

Hedgerows and Boundaries capital grant

Woodland capital grants covering:

o Woodland creation and maintenance

o Woodland management plans

o Woodland tree health

Water capital grants – for infrastructure works which help reduce water pollution from agriculture

Countryside Stewardship Facilitation Fund

The Countryside Stewardship Facilitation Fund (CSFF) provides funding for people or organisations (e.g. a lead farmer or NGO) that bring farmers, foresters and other land managers together to work cooperatively for environmental improvements at the landscape scale. The priority for this partnership and collective approach is to deliver shared local environmental outcomes that go beyond what could be delivered by individual holdings acting in isolation42.

Research into the economics of facilitated approaches in agri-environment schemes indicated that where complex or multiple environmental landscape scale objectives were required then a facilitated approach would be cost effective43.

39 Defra (2014c) National Pollinator Strategy: for bees and other pollinators in England. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/409431/pb14221-national-pollinators-strategy.pdf 40 Defra (2014b) 41 Defra (2015b) 42 See: https://www.gov.uk/government/publications/guide-to-countryside-stewardship-facilitation-fund/guide-to-countryside-stewardship-facilitation-fund 43 CCRI (2012) Economics of co-ordination in Environmental Stewardship. Defra & Natural England.

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/409431/pb14221-national-pollinators-strategy.pdf





The overall objective of the fund is: to support greater cooperation amongst farmers and others to deliver effectively the Countryside Stewardship Priorities at a landscape scale. This is to be achieved by: funding a facilitator to nurture the cooperation and to make available training and advice to the members of their group so that land is managed in a spatially coherent way44.

The fund of £7.2 million45 is intended to build on the success of existing landscape scale projects (including the GWCT farm clusters and NIAs) that have demonstrated the benefits of partnership working to achieve local priorities.

The area of land involved must:

Cover at least 2,000 hectares, unless there is an obvious smaller environmental boundary

Spread across at least 4 adjoining (or mainly adjoining) holdings.

Facilitators

Facilitators are expected to have expertise in at least one of the following:

Agriculture

Forestry

Water management

Ecology

The work undertaken by facilitators will cover46:

Working with a group of new or existing land managers to develop cooperation

Agreeing the CS priorities (from the priority statements) to take forward across the holdings

Helping group members to interpret the CS priority statements and associated guidance to enable submission of individual but complementary CS applications

Endorsing applications on Rural Payments47 to demonstrate that they are consistent with the group’s agreed objectives

Providing group members with the relevant skills and expertise required to deliver CS priorities either by providing the training and other knowledge transfer activities themselves or procuring it from others

Maintaining links with local partnerships and initiatives (as well as Defra) to ensure that work undertaken by the group is complementary

Providing information to Natural England to show what is being done differently as a result of the cooperation and the difference this makes to the delivery of CS priorities

To qualify for funding, the group will have to undertake new activities that result from cooperation. Activities might include aligning the management activities across different parts of the holdings to deliver at a landscape rather than single-farm scale.

The selection criteria for the facilitation fund are provided in Annex 1 of the Guide to Countryside Stewardship: facilitation fund.

44 Sarah Webster powerpoint presentation: Facilitation Funding 45 See press release (Defra, 2015c) at: https://www.gov.uk/government/news/new-fund-to-support-joined-up-action-on-the-environment 46 Section 2 of the Guide to Countryside Stewardship: facilitation fund 47 https://www.gov.uk/claim-rural-payments


https://www.gov.uk/claim-rural-payments



The window for applications is from 17 March 2015 until 13 May 2015. The intention is to have subsequent application windows in 2016 and 2017. Applications forms48 are submitted by email to Natural England and information is available on the assessment and scoring process49.

48 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413510/facilitation-fund-application-form.pdf 49 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413777/CSFF-scoring-process.pdf


https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/413777/csff-scoring-process.pdf



Appendix 2: Summary of data sources and analysis used for the M&E indicators

The table provides an overview of the main data sources and methods of analysis used by NIA partnership in relation to each of the M&E framework indicators50.

Data sources and methods of analysis used by the NIA indicator protocols

Theme and sub-theme

Ind

icat

or

cod

e

Indicator title

Ind

icat

or

cate

gory

Data sources Methods of analysis

NIA

dat

a th

rou

gh

BA

RS

Nat

ion

al d

ata

thro

ugh

BA

RS

Dir

ect

NIA

mo

nit

ori

ng

MEN

E

Surv

ey

(vis

ito

r e

tc.)

Loca

l & /

or

nat

ion

al

dat

ase

ts

GIS

an

alys

is

BA

RS

rep

ort

s

NE

sup

plie

d

Co

llati

on

of

valu

es

(e.g

. in

Exc

el)

Qu

alit

ativ

e

asse

ssm

ents

Biodiversity

Habitat

B01_H Extent of existing priority habitat managed to maintain / improve its condition

Core

B02_H Extent of areas managed to restore/create habitat

Core

B03_H Proportion of SSSIs in favourable or recovering condition

Optional

B04_H Total extent of priority habitat Core

Species

B05_S Extent of habitat managed to secure species-specific needs

Optional

B06_S Status of widespread species Optional

B07_S Status of focal species Optional

Invasive species B08_S Control of invasive non-native species Optional

Habitat connectivity B09_C Optional indicator of habitat connectivity Optional

B10_C Comparative indicator of habitat connectivity Core

Ecosystem Services

Cultural services ES01_C Measure of extent of land managed to enhance

landscape character Optional

50

Source: M&E of NIAs Phase 2: Evaluating the NIAs against the counterfactual – method statement and work plan (27/1/15).



Theme and sub-theme

Ind

icat

or

cod

e

Indicator title

Ind

icat

or

cate

gory


NIA

dat

a th

rou

gh

BA

RS

Nat

ion

al d

ata

thro

ugh

BA

RS

Dir

ect

NIA

mo

nit

ori

ng

MEN

E

Surv

ey

(vis

ito

r e

tc.)

Loca

l & /

or

nat

ion

al

dat

ase

ts

GIS

an

alys

is

BA

RS

rep

ort

s

NE

sup

plie

d

Co

llati

on

of

valu

es

(e.g

. in

Exc

el)

Qu

alit

ativ

e

asse

ssm

ents

ES02_C Length of accessible PROW and permissive paths created and/or improved

Optional

ES03_C Condition of historic environment features Optional

ES04_C Access to natural greenspace and/or woodland Optional

Supporting services ES05_S Area of habitat supporting pollinators Optional

Regulating services

ES06_R Contribution to water quality Optional

ES07_R Contribution to carbon storage & sequestration Optional

Provisioning services

ES08_P Area of more-sustainable agricultural production Optional

ES09_P Percentage of woodland in active management Optional

Social and Economic

Social impacts & well-being

S&E01_S Attitudes of local community to the natural environment and environmental behaviours

Optional

S&E02_S Number of educational visits Optional

S&E03_S Number and social mix of visitors to NIA sites Optional

S&E04_S Number and social mix of people involved in NIA activities and events

Optional

S&E05_S Level of outdoor recreation by NIA residents Optional

S&E06_S Number of volunteer hours on NIA activities Core

Economic values & impacts

S&E07_E Estimated value of visitor expenditure to local economy

Optional

S&E08_E Number of people employed in NIA activities Optional

Partnership working

Mobilisation of resources

PW01_R Project income and expenditure Core

PW02_R Financial value of help-in-kind Core



Theme and sub-theme

Ind

icat

or

cod

e

Indicator title

Ind

icat

or

cate

gory


NIA

dat

a th

rou

gh

BA

RS

Nat

ion

al d

ata

thro

ugh

BA

RS

Dir

ect

NIA

mo

nit

ori

ng

MEN

E

Surv

ey

(vis

ito

r e

tc.)

Loca

l & /

or

nat

ion

al

dat

ase

ts

GIS

an

alys

is

BA

RS

rep

ort

s

NE

sup

plie

d

Co

llati

on

of

valu

es

(e.g

. in

Exc

el)

Qu

alit

ativ

e

asse

ssm

ents

Efficient & effective delivery

PW04_E Fulfilment of identified skills needs Optional

PW05_E Attitudes of local community to NIA Optional

PW06_E Assessment of partnership working Optional

Leadership & influence

PW07_L Audience reach Optional

PW08_L Level of awareness of NIA in local community Optional

PW09_L Number of enquiries Optional



Appendix 3: Theory of change literature review

Approach to the review

As discussed in Chapter 5, a focused literature review was undertaken to evidence the theory of change model developed for CSFF where relevant / possible – i.e. using relevant literature to evidence the theoretical causal processes identified, especially when moving between activities / outputs and outcomes and impacts. In essence, the literature review has provided initial answers to the question: to what degree might action (activities and outputs) at the landscape scale contribute to the enhanced delivery of biodiversity and water environment outcomes and impacts? The focus on biodiversity and water environment reflects the CSFF’s focus on these issues (see Chapter 5). It should be noted that the literature review was narrow in scope reflecting the resources available for this project and also the scoping nature of CSFF M&E design at this stage.

The criteria for the literature review were focused on the anticipated outcomes and impacts of CSFF – i.e. the review was designed to explore the how and why landscape scale action may have the potential to support the delivery of relevant outcomes and impacts at the landscape scale also. The criteria / sub-questions were as follows – does action at the landscape scale have the potential to support the enhanced delivery of:

Condition of priority habitats [habitat condition improvements]

Structural and functional habitat connectivity

Ecosystem health and function

Wild species diversity and sustained populations

Ancient semi-natural woodland habitat networks

Reduced agricultural pollution loading

Catchment scale restoration of hydraulic properties or river channel, riparian zone and floodplain

Documents for the review were identified as follows:

Defra reports: the Defra Science and Research Projects pages51 were searched using the key word ‘agri environment’. The abstracts / executive summaries of the research reports identified were then reviewed to identify their potential relevance to the literature review question above. Relevant projects were then taken forward for more detailed review.

Academic journal articles: two key online databases were searched (Ecology Abstracts and Geobase) using the following search strategies: 1) Landscape scale management AND [literature review criteria e.g. condition of priority habitats]; and 2) Benefits of landscape scale management AND [literature review criteria e.g. condition of priority habitats]. Relevant abstracts were then reviewed and taken forward for further consideration where relevant as per the above.

51

Defra Science and Research Projects pages: http://randd.defra.gov.uk/ [accessed 12/05/15]

http://randd.defra.gov.uk/



Theory of change evidence review – summary

Reference Relevant literature review criteria

Notes on how the reference supports or contradicts theory of change assertions that landscape scale management can contribute to enhanced outcomes

Cole, L., Deane, R., Parker, S., Cooper, H., Manson, D., Cookson, W., Martin, J., Porter, J., Jagota, L., Haines-Young, R., and Swanwick, C. (2013). Monitoring the effects of Environmental Stewardship on Landscape Character and Quality. Report to Defra. London: Land Use Consultants. [Defra Project Code BD503]

Condition of priority habitats


Ecosystem health


Ancient semi-natural woodland habitat networks

This study was undertaken to address concerns that previous monitoring of agri-environment schemes (AES) had not fully captured the landscape effects of these schemes (i.e. the impacts of AES on key structural elements of the landscape).

The study considered effects at x2 different scales: 1) local level through the assessment of new field survey data; and 2) strategic / national character area (NCA) level through the assessment of digital data (remote sensing data etc).

The overarching finding of the study is confirmation that AES is meeting the objective of conserving and enhancing landscape character and quality, particularly in upland landscapes.

The study did not differentiate between AES options delivered at the holding or landscape scale. However it does discuss the structural elements of landscape (e.g. hedgerows, small farm woodlands, semi-natural habitats) and wider landscape benefits (e.g. conserving and reintroducing traditional land use patterns, colours and textures, helping to reinforce local distinctiveness etc) that AES has been helping to deliver.

Specific reference is made to the findings of this study informing monitoring and management planning within National Parks, AONBs and other integrated management projects such as NIAs.

Potential relevance to CSFF: this study has confirmed that AES contributes to landscape character and quality outcomes as anticipated. Given that key structural elements of landscape considered in the study (e.g. hedgerows, semi-natural habitats etc) can support biodiversity outcomes by facilitating landscape scale ecosystem processes

52,53,54,55 (e.g. species migration, colonisation,

interbreeding etc) through the provision of habitat networks, one could infer that the more coordinated, spatially coherent, joined-up delivery of these structural elements at the landscape scale, through CSFF for example, would deliver these outcomes potentially in a more effective manner.

Note: this study is supported by a related piece of work that delivered the new sampling methodologies to collect the data on the ground (Defra project code LM0425)

56

British Trust for Ornithology Wild species diversity and This study investigated the potential for AES to contribute to the enhancement of farmland bird

52 Evaluating biodiversity in fragmented landscapes: principles (Watts et al, 2005): http://www.forestry.gov.uk/pdf/fcin073.pdf/$FILE/fcin073.pdf [accessed 13/05/15] 53 Habitat networks – reviewing the evidence base (Briers, 2011): http://www.snh.gov.uk/docs/B831805.pdf [accessed 13/05/15] 54 Landscape scale approach to conservation management – Glasgow and Clyde Valley case study (Corbett et al, 2009): [not available online] 55 Glasgow and Clyde Valley Integrated Habitat Networks (Smith et al., 2008) 56 Assessing the effect of policy interventions on agricultural landscapes: http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&Completed=1&ProjectID=18836 [accessed 14/04/15]

http://www.forestry.gov.uk/pdf/fcin073.pdf/$FILE/fcin073.pdf

http://www.snh.gov.uk/docs/B831805.pdf

http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&Completed=1&ProjectID=18836





(2005). The consequences of spatial scale for agri-environment schemes designed to provide winter food resources for birds. Report to Defra. Thetford: British Trust for Ornithology. [Defra Project Code BD1616]

sustained populations populations through the provision of better winter feeding conditions for seed-eating (granivorous) birds e.g. over-winter stubble retention. The study considered, in particular, the relationship between spatial configuration of these type of AES options and the benefits for populations of key farmland bird species.

The study found that enhancing winter food sources through relevant AES options can help to stem and ultimately reverse population declines in granivorous bird species.

The study highlighted that these types of AES option should be targeted on areas with relatively low or recently declined local bird abundance (i.e. areas where populations are under pressure) though targeting should also ensure that there is a relatively plentiful supply of good breeding habitat that recovering bird populations can expand into.

Crucially, the study highlighted that the spatial configuration of relevant AES options (referred to as ‘winter food resource enhancement’ – e.g. payments for stubble retention) at the landscape scale, provides the most cost effective approach for delivery of winter food to different local populations of birds when the seed-rich habitats (e.g. stubble fields) are at least 500m-1000m apart.

Potential relevance to CSFF: taking into account the findings above, the spatially coherent delivery of relevant Countryside Stewardship options at the landscape scale through CSFF could support the effective delivery of outcomes for farmland bird populations – e.g. more effectively / strategically planning the spatial configuration of ‘winter food resource enhancement’ areas to benefit local bird populations.

Land Use Consultants and GHK Consulting Ltd (2008). Provision of ecosystem services through the Environmental Stewardship scheme. Report to Defra. London: Land Use Consultants. [Defra Project Code NRO 121]

All criteria This study investigated the type of ecosystem services provided by the Environmental Stewardship (ES) scheme including: 1) which services are potentially provided by ES; and 2) the degree to which service provision can be monitored.

The study highlights those ecosystem services that ES seems best able to deliver, including biodiversity, protection of soil organic matter, genetic conservation, water infiltration, flood alleviation, erosion control, water quality and pollination. Potential relevance to CSFF: the study highlights the key ecosystem services that can be provided by ES (the forerunner to Countryside Stewardship) and therefore provides evidence for the assertion in the CSFF logic model and theory of change analysis that appropriate land management can contribute to the delivery of CSFF priority services.

Crucially, the study highlights how effective delivery of many of the cultural and regulating (e.g. flood alleviation, erosion control) services is dependent on spatial targeting at the landscape and farm / field scale. The example provided is water related regulating services whereby spatial targeting at the landscape scale needs to focus on floodplains and at the farm / field scale, beside





watercourses. Potential relevance to CSFF: this supports the assertion that key regulating ecosystem services prioritised in CSFF could potentially be delivered more effectively through landscape scale planning and management.

The study also raises the issue of needing to maximise service delivery through highly focused targeting and ‘density of option uptake’. Potential relevance to CSFF: given that some services require landscape scale planning (see above), ‘density of option uptake’ could potentially be delivered most effectively through spatially coherent targeting across multiple holdings at the landscape scale – e.g. for erosion control or flood alleviation which may require low level “broad multi-objective” interventions but delivered across all fields in a catchment.

The study identifies different approaches for monitoring ecosystem service delivery from ES that could be applied in the CSFF M&E. All four of the monitoring approaches identified are relevant but the most relevant approaches are potentially: 1) using existing outcome indicators that can be mapped spatially and linked to option uptake through ES (or CSFF in this case) – e.g. modelled flood risk and recorded flooding impacts, both of which could be reduced through the use of appropriate options (riparian management, floodplain woodland etc); and 2) services where option uptake data (i.e. output elements of the LM) provides a reliable indicator of service delivery – e.g. genetic resources, pollination.

Diebel, M.W., Maxted, J.T., Nowak, P.J., and Vander Zanden, M.J. (2008). Landscape planning for nonpoint source pollution reduction 1: A geographical allocation framework. Environmental Management, 42, pp.789-802.

Reduced agricultural pollution loading

The paper recommends an ‘aggregated / targeted’ approach (see figure opposite) to the allocation of landscape scale nonpoint source pollution control effort, for the best water environment outcomes. The approach is delivered through four-steps that include a field scale assessment of nutrient loading and management opportunity to prioritise the location of intervention across the catchment (see p.799). Potential relevance to CSFF: the delivery of such an ‘aggregated / targeted’ approach would require a degree of landscape scale / strategic coordination – i.e. to capture field scale data on management practices, existing conditions etc and to ensure that land managers deliver the management recommendations and to explain to land managers why some areas are being prioritised over others etc.

The paper also acknowledges that the cost of land management and therefore the uptake of a desired land





management strategy is dependent on institutional arrangements such as grants policy (e.g. CSFF / CS priorities). Potential relevance to CSFF: a spatially targeted / aggregated approach that is planned at the landscape scale could deliver value for money by ensuring that intervention is targeted to the locations that would deliver the best water environment outcomes.

The paper argues for spatial targeting of management intervention at multiple scales: 1) landscape scale – selecting catchments where restoration potential is the greatest; and 2) field scale – selecting fields for specific management intervention that will deliver the greatest benefit (see figure above).

Moreno-Mateos, D., and Comin, F.A. (2010). Integrating objectives and scales for planning and implementing wetland restoration and creation in agricultural landscapes. Journal of Environmental Management, 91, pp.2087-2095.

Condition of priority habitats


Ecosystem health


Catchment scale restoration of hydraulic properties of river channel, riparian zone and floodplain

The paper highlights how every wetland restoration objective (e.g. water quality, biodiversity, soil improvement, flood management etc) has an optimal implementation scale and that the success of a restoration project will depend on the combination of efforts towards achieving objectives at different scales (p.2091 and 2092). Potential relevance to CSFF: effective wetland restoration may require a two-pronged / multi-scale approach: 1) strategic planning at the catchment scale to identify objectives and prioritise effort; and 2) site based planning for specific wetlands. Planning and delivery at both scales (and integration between scales) would benefit from a coordinated approach.

The paper specifies that planning for the delivery of multi-purpose wetland restoration / creation projects should be undertaken at the landscape scale (p.2094 and figure opposite). Potential relevance to CSFF: one key reason for this is that some ecosystem services / wetland management objectives will always be context specific i.e. where the value of the services delivered is influenced by causal variables – e.g. flood control services are only beneficial where there is a flood risk to address in the catchment and where the location of service delivery / management is such that there is a mechanism for the intervention to cause the desired hydrological / hydraulic impact.




Notes on how the reference supports or contradicts theory of change assertions that landscape scale management can contribute to enhanced outcomes The paper specifies the key wetland management objectives / ecosystem services that are best

planned for and delivered at broader scales – key examples are support for wild species populations (birds), landscape heterogeneity / ecological connections, water cycle function, water quality and soil improvements (p.2091 and figure below).



Appendix 4: Review of Biodiversity 2020 Outcome 1C Self-Assessment Criteria

The Biodiversity 2020 Outcome 1C Self-Assessment Criteria and Method form (TBG, undated) has been reviewed to identify potential additions to the CSFF logic model, as outlined in the table below. In essence, the criteria in the self-assessment form constitute the types of planning activities that a partnership of land managers within a facilitated management area would need to undertake if the management plan / actions were to truly embrace an integrated, ecosystems based approach. One could argue though that some of the activities listed below are perhaps less relevant for a smaller group of land managers working together, noting that the 2020 Outcome 1C self-assessment form is designed for use by much larger (spatial extent) partnerships e.g. NIAs, National Parks, AONBs etc. As such, it may be the case that some of the additional activities listed below are ‘nice to dos’ as opposed to essential activities that would need to be delivered by facilitated land managers to meet the CSFF eligibility criteria. This distinction is made in the table below. Where relevant, activities in the table below could be linked to additional outputs in the finalised logic model.



Review of Biodiversity 2020 Outcome 1C – Ecosystems Approach Self-Assessment Criteria and Method for possible additions to CSFF logic model

Possible additional activities to add to CSFF logic model Relevance to CSFF logic model / M&E

Notes / rationale

1. Undertake an audit of environmental assets (e.g. waterbodies, priority habitats, rights of way, designated sites etc) within the facilitated management area e.g. in terms of main problems and opportunities, management priorities etc. Note: this activity could perhaps be quite qualitative and based on land manager knowledge – e.g. participating land managers sat round a table with a map. Equally a lot of this information should be mapped and available already – e.g. waterbodies (EA), designated sites (NE / Magic), habitats and species (Local Records Centres), rights of way (County Councils) etc. A CSFF facilitator could collate al this information and then discuss with the participating land managers – e.g. to verify the data, discuss problems, opportunities and management priorities etc.

Potentially relevant – consider inclusion

This type of activity could be undertaken to support compliance with several of the CSFF eligibility criteria (CSFF online guide Annex 1) and several of the required facilitator activities (CSFF online guide section 2).

2. Undertake an assessment of ecosystem health, function and integrity – in relation to ecosystem processes / supporting services e.g. water cycling, soil formation, ecological interactions (landscape scale habitat connectivity) etc. Note: this activity is more technical and likely to be dependent on expert knowledge and maybe new data.

Not relevant – suggest that this is not included

This type of activity is likely to be too technical, onerous and data intensive for consideration within smaller scale partnership based management such as CSFF.

3. Scoping meeting to identify priority ecosystem services – develop an understanding of which ecosystem services are already being delivered within the facilitated management area and assess / map ecosystem services where possible (could be a simple exercise involving participating land managers sat round a table with a map), use direct or proxy data to assess and quantify ecosystem services where possible (may be too technical and / or unnecessary for the purposes of CSFF), identify priority ecosystem services for delivery within the facilitated management area (noting that these may help to address priorities outwith the management area – e.g. flood storage, strategic habitat networks etc).


Priority ecosystem services are already set for CSFF (habitat networks / ecological connectivity, flood risk management, soil erosion etc). There may be opportunities for landscape scale management through CSFF to incorporate consideration of wider ecosystem services so suggest that this activity could be included. It would need to be caveated to ensure that CSFF priority services are addressed first and foremost.

4. Develop stakeholder and community engagement plan to support Potentially relevant A simplified version of this could be covered by addressing CSFF



Possible additional activities to add to CSFF logic model Relevance to CSFF logic model / M&E

Notes / rationale

the development and (where relevant) delivery of activities within the facilitated management area. Note: stakeholder and community engagement is a key principle of the ecosystems approach though this activity may well not be relevant for CSFF.

– consider inclusion mandatory eligibility criteria on ‘engaging with other initiatives / partnerships to deliver CS priorities through a joined-up, coherent approach’. This could also address Q8 in the 2020 self-assessment form on understanding which ecosystem services people value – i.e. engaging with other relevant initiatives to understand what their ecosystem service management priorities are and how CSFF management could be aligned to support this, where relevant (see activity No.3 also)

5. Develop a management plan – the plan should set out objectives for the delivery of multiple benefits for biodiversity and ecosystem services through landscape scale joint action. The plan should identify priority ecosystem services, be transparent about ecosystem service tensions / conflicts and seek to protect and enhance ecosystem health function and integrity – the plan should closely link with the audit / assessment type activities above. Note: this sort of management plan could be delivered through the mandatory CSFF facilitation agreement.

Potentially relevant – consider inclusion and / or tweak the existing facilitation agreement activity / output to cover the issues here

Consider the possibility of the management plan / facilitation agreement including long term (10yrs+) objectives (recognising that facilitation agreements are only for up to 5yrs). This would help to address the issue below regarding Question 6 in the 2020 Outcome 1C self-assessment form – the need to include long-term management objectives long-term ecosystem restoration goals.

6. Undertake a forward looking risk and vulnerabilities assessment to ensure that the management plan addresses current and future risks, pressures and changes affecting environmental assets in the facilitated management area. Note: this activity would be linked to activity No.1 which would audit current environmental assets – the focus of No.6 however would be to ensure that planned management activity is designed to reflect future risks and uncertainty (e.g. climate change, diseases and pests etc).


The 2020 self-assessment form also includes a question (Q6) on long-term (10yrs+) management recognising that ecosystem restoration and management for the delivery of high value ecosystem services can take time (specific issues mentioned are ecological connectivity and climate change adaptation). Maybe this activity could include a time element to capture this, especially as the management plan / facilitation agreement is only ever going to be for 5yrs (the max agreement length). This could perhaps be covered through the CSFF mandatory eligibility criteria on facilitated land managers undertaking activities that are new to them as a result of cooperation – i.e. undertaking a forward looking risk assessment etc may help to identify long-term management objectives that might not otherwise have been considered.



Appendix 5: Potential approaches to CSFF M&E based on the logic model

Potential approaches to CSFF M&E based on the logic model inputs, activities and outputs are listed in Tables 11, 12 and 13. Priorities for further consideration and included in Tables 7, 8 and 9 of Section 6.3 are highlighted in pink. As with the logic model table, keystone elements are highlighted in bold red. All biodiversity and water environment related elements are considered to be ‘primary elements’ that could be a focus for CSFF M&E where prioritisation of effort is required.

Inputs data

Table 11: Potential approaches to CSFF M&E of facilitation inputs

Inputs Indicator Purpose of data

capture Baseline Data requirements /

information to record

Use of data Data source Assumptions

The resources (financial, time, people, skills etc) being invested in the facilitated management area.

The conditions prior to proposed facilitation, taking account of what has happened previously in the facilitated area.

Potential data sources for use assessing the benefits of facilitation.

Potential options / procedures for comparing the benefits of facilitation to scenarios without facilitation.

Main source / point of data capture

Assumptions made about planned data collection, the potential for further analysis and opportunity for additional data collection.

Such as… Such as… Data requirements: Options:

Defra funding: 1) for the CSFF; and 2) for spatially targeted and concentrated uptake of relevant CS options as prioritised through the CSFF project

Cost of facilitation. To determine overall costs of facilitation and to use in analysis of benefits against costs.

Record of whether the area covered by the CSFF has received funding for a previous landscape scale initiative (via the landscape scale initiatives database).

Record of proportion of facilitated area previously included within a landscape scale

Grant offer

Additional funding obtained

Payments for options prioritised through CSFF

Comparison of benefits against costs in facilitated areas with paired non-facilitated areas (Option 1a).

Comparison between different facilitated areas in relation to funding received and projected delivery or option uptake (outputs) (Option 2).

Analysis of the amount of CSFF

CSFF grant offer

CSFF application form (Questions 24, 26, 27 & 28)


CS application data Payments data

Data is available from the CSFF application forms and / or agreements.







initiative (if applicable).

funding per facilitated area in relation to outputs (Option 2): o No. of land

managers o No. of options

per unit area o Area of land

covered by individual options

o Cumulative area of land covered by all options (an area of land may be covered by more than one option)

Number of contiguous options

At least 2,000ha of land per CSFF project

57

Ha of land within facilitated areas.

To provide a record of size of facilitation area.

Area of land covered by the CSFF application (note that the area within the application is not the same as the benefiting area).

Record of area of land covered by CSFF.


57 Smaller areas of land may be acceptable for CSFF projects where it is possible to demonstrate an obvious smaller environmental boundary (e.g. a small sub-catchment, contiguous area of woodland etc). The 2,000ha figure relates to the combined area of holdings within the facilitated management area and not the area of land being managed to deliver CS priorities







At least 4 adjoining (or mainly adjoining) land holdings

Spatially coherent agreement areas.

To check continuity of holdings.

Number and location of holdings

Number of adjoining land.

Number of adjoining holdings per facilitated area.

Compare distance to nearest neighbour with scheme in place between facilitated and non-facilitated areas.


Mapping of facilitated areas undertaken.

Advice, support and training from specialist facilitators with expertise in at least one of the following: agriculture, forestry, water management or ecology

Facilitator expertise. To help determine whether facilitator expertise has any influence on outputs within the facilitated area.

Record of any previous facilitation through a landscape scale initiative and the expertise of that / those facilitator(s).

List of expertise of each facilitator categorised by: o Agriculture o Forestry o Water

management o Ecology

Other

Matrix of expertise of facilitators for all facilitated areas. Identify main area of expertise for each and see whether there is a correlation between expertise and types of options uptake within the facilitated area.

CSFF application form (Questions 6, 7 & 8).


Baseline information can be obtained from the facilitators or other sources.

Expertise of facilitators recorded from application forms.

Resources are available for the additional comparative analysis of the data.

Advice, support and training from specialist external providers

External trainer expertise.

To help determine whether the expertise of external trainers has influenced the activities and outputs within the facilitated area

Previous advice, support and training delivered to land managers within the facilitated group.

Proposed external training and providers.

Preparation of matrix of expertise of external training providers in facilitated areas for use in assessing influence of external trainer



Baseline data is available and can be collected from appropriate sources e.g. previous landscape scale initiatives (if relevant), or interview surveys







expertise on activities outputs (Option 2).

with land managers.

Existing knowledge, skills and resources held by the participating land managers

Skills held by land managers

To identify existing skills held by land managers as a baseline for assessing change resulting from training provision.

Existing knowledge, skills and resources held by participating land managers.

A record of existing knowledge, skills and resources of participating land managers.

Before and after survey interviews with a sample of landowners to identify expertise and expectations at the start of the scheme and then again at the end of scheme (and perhaps midway in the scheme) to see whether landowner knowledge, skills, attitudes and perceptions had changed as a result of the facilitation and participative working.

Additional data capture: surveys / interviews with land managers.

Additional information will be gathered (from facilitators or interview with land managers) to obtain baseline data.

Shared information and learning from others

Type and amount of information sharing.

To determine the knowledge input of land managers to the partnership and the influence of knowledge sharing on outputs.

Existing information and knowledge held by land managers.

A record of existing information and knowledge held by land managers.

Before and after survey interviews with a sample of landowners to identify expertise at the start of the scheme and then again at the end of scheme (and perhaps midway in the scheme) to see whether

Additional surveys with land managers to capture required data.

Additional information will be gathered (from facilitators or interview with land managers) to obtain baseline data.







landowner knowledge and perceptions had changed as a result of participative working and sharing information and learning from others.

Activities data

Table 12: Potential approaches to CSFF M&E of facilitation activities

Activities Indicator Purpose of data




The activities and processes being undertaken by the facilitator and the participating land managers within the facilitated management area to deliver their objectives, and wider policy objectives



Potential options for comparing the benefits of facilitation to scenarios without facilitation.

Main source / point of data capture



Activities

undertaken as part

of the CSFF

application process:







Scope out CS priorities that could be addressed within the facilitated management area

Process of scoping CS priorities to be addressed within the facilitated management area.

To determine whether the approach to scoping priorities has influenced the selection of types and locations of options and hence whether this has affected the outputs.

Previous experience in delivery for biodiversity / ecosystem services / CS priority type activities in the facilitated area, for example through: o Previous and

existing ES agreements.

o Landscape scale initiative objectives / activities.

o Activities undertaken by land managers of their own accord.

Approach to scoping the CS priorities.

List CS priorities per facilitated management area and area (ha) covered.

Comparison of CS priorities identified in CSFF applications with options included in CS agreements for holdings within facilitated areas. This should indicate follow through from CSFF application to holding agreements (Option 2).

If possible, a comparison could be made between different scoping approaches to assess effectiveness in identifying priorities (Option 2).

Information should be available from the CSFF application form (Question 16)

Baseline data on ES agreements can be obtained from existing AES datasets.

Information available on previous / continuing landscape scale initiatives.

Activities by land managers can be picked up through additional interview surveys or from facilitators.

The scoping processes undertaken by different facilitators can be categorised.

Meetings, workshops etc with participating land managers to develop cooperation, working towards the production of a group agreement

Process of developing the group agreement and group working.

Information to be used in determining the effectiveness of group working and group cohesiveness.

Previous collaborative working between land managers, e.g. o None. o Landscape

scale initiatives.

o Other cooperative


Activities undertaken to reach agreement. o Group working

activities undertaken

Assessment of the effectiveness of group working (Option 2).



Baseline data can be collected from facilitators or interviews with land managers.

Record of planned activities can be discerned from CSFF application forms or, more likely,







working. obtained from facilitators.

The processes undertaken by different facilitators for developing the group agreement can be categorised.

Meetings, workshops etc with participating land managers to agree the CS priorities from the relevant National Character Area (NCA) statement of priorities that will be delivered within the facilitated management area

Process of agreeing the CS priorities.

To help understand the way in which the group works together.

Previous collaborative working between land managers to identify priorities similar to those in CS, e.g. o None. o Through

landscape scale initiatives.

o Through other cooperative working.

Categorise and record planned activities, for example: o Number /

frequency of meetings / workshops.

o Agreed list of priorities.

o Map of facilitated area with spatial distribution of priorities showing connectivity of planned activities.

o Production of a management plan and broad content.

Number of meetings / workshops and the time taken to reach agreement may indicate or be a result of: o The degree of

commitment / cooperation / cohesiveness within the group.

o Whether the group has worked together previously.

o Ease of coming to agreement on CS priorities.

Use the cumulative time spent on meetings etc for different

Additional data capture: request for data from facilitator.


Record of planned activities can be discerned from CSFF application forms or obtained from facilitators.

The processes undertaken by different facilitators for agreeing the CS priorities can be categorised.







purposes (activities) as a cost when considering cumulative delivery of options (outputs).

SWOT analysis (or similar) to identify land manager training needs

Process for identifying training needs.

To identify training needs from the start of the scheme.

Previous training received by land managers.

List of land manager training needs.

Use to test whether needs have been addressed (as outputs).


Baseline data available from facilitators.

Land manager training needs specified in CSFF application.

The processes undertaken by different facilitators for identifying training needs can be categorised.

Meetings,

workshops etc

with participating

land managers to

identify the

actions required

to deliver the

agreed CS

priorities

Process for identifying actions required to deliver the agreed CS priorities.

To help understand the way in which the group works together.

Previous collaborative working between land managers to identify actions similar to those required for CS priorities, e.g. o None. o Through

landscape scale initiatives.

o Through other cooperative

Categorise and record planned activities, for example: o Number /

frequency of meetings / workshops.

o List of SMART actions.

Number of meetings / workshops and the time taken to reach agreement may indicate or be a result of: o The degree of

commitment / cooperation / cohesiveness within the group.

o Whether the group has



Record of planned activities can be discerned from CSFF application forms or obtained from facilitators.

The processes undertaken by different







working. worked together previously.

o Ease of coming to agreement on actions.

Use the cumulative time spent on meetings etc for different purposes (activities) as a cost when considering cumulative delivery of options (outputs).

facilitators for identifying actions required to deliver the agreed CS priorities can be categorised.

Activities undertaken as part of the CSFF project and submission of subsequent CS applications:

Support for individual land managers to check and re-position existing land management activity to support enhanced delivery at the landscape scale

Facilitator support to land managers for enhanced delivery at the landscape scale.

To determine whether the support provided to land managers has influenced the outputs.

Familiarity of land managers with working with a land management advisor.

Type of support provided



Classification of all CS applications according to: o Within /

outside a

Quantitative data on facilitator input could be used to calculate input hours per ha (Option 2) for: o Comparison

with other facilitated areas.

o Assessment of the level of activity to




Baseline information can be collected from facilitators, land managers or records of previous AES agreements.

Record of number of hours spent supporting individual land managers is







facilitated area o Previously

included within a facilitated landscape scale initiative or not

o Support with preparation of CS application (from any source) received / not received.

achieve outputs.


recorded by facilitators.

Facilitator support for individual land managers in the facilitated management area to interpret the CS priority statements and associated guidance to enable submission of individual but complementary CS applications

Facilitator support to land managers with CS applications.

To measure the amount of facilitator time provided in supporting landowners.

Familiarity of land managers with requirements for AES (land managers have submitted applications under previous schemes).

Number of hours spent supporting each individual land manager within the group.

Total number of hours support to all land managers within the group.

Quantitative data on facilitator input can be used to calculate input hours per ha for: o Comparison


o Assessment of the level of activity to achieve outputs.


Baseline information can be collected from facilitators, land managers or records of previous AES agreements.

Record of number of hours spent supporting individual land managers is recorded by facilitators.

Activities undertaken to support the delivery of on the ground land management:

Scoping and Process of To determine Previous advice, Type of support Quantitative data CSFF application Baseline data can







delivery of training and knowledge transfer to provide participating land managers with the skills required to deliver CS priorities

delivering training and knowledge transfer activities.

whether the training received by land managers results in any changes in knowledge, skills, and attitudes.

support and training received by land managers within the facilitated group.

provided



Classification of all CS applications according to: o Within /

outside a facilitated area

o Previously included within a facilitated landscape scale initiative or not

Support with preparation of CS application (from any source) received / not received.

on facilitator input could be used to calculate input hours per ha (Option 2) for: o Comparison


o Assessment of the level of activity to achieve outputs.


form (Question 20)



be collected from CSFF application forms, facilitator reporting or interview surveys with land managers.

Report of training and knowledge transfer activities provided through facilitator reporting.

Facilitator works to maintain links between the land holdings / land managers within the facilitated management area and related local partnerships and initiatives

Process of maintaining links between land managers and with related local partnerships / initiatives.

To determine whether the training received by land managers results in any changes in knowledge, skills, and attitudes.

Previous networking experience of by land managers with biodiversity / landscape scale initiatives.

Type, content and length of training and knowledge transfer activities, for example: o Land

management o Machinery /

equipment use o Fertilizer,

pesticide use

Correlation of the types of training that land managers engage in with any changes in knowledge, skills, and attitudes (Option 2).



Activity Evaluation Forms

Further information on changes in land manager

Baseline data can be collected from CSFF application forms, facilitator reporting or interview surveys with land managers.

Report of activities to maintain links







o Water quality related

o Ecology

Purpose of the training

Planned learning outcomes

knowledge, skills, and attitudes to be obtained through semi-structured interviews with a sample of land managers from all CSFF projects (to include questions on learning gained and application of learning)

between landowners and local partnerships and initiatives can be gained through facilitator reporting.

On the ground land management activity to deliver agreed CS priorities at the landscape scale

Land management activities to deliver priorities at landscape scale.

To help determine whether the land management activities undertaken have resulted in improved delivery or priorities at the landscape scale.

Previous experience of land managers in delivering CS priority type activity at the landscape scale.

Report of activities

Map of spatial distribution of CS priorities.

Activities that land managers are doing differently.

Analysis of effectiveness of facilitated activities at delivering CS priorities at the landscape scale (Option 2).



Additional survey information may be required, through semi-structured interviews with a sample of land managers from all CSFF projects, on the process of delivering CS priorities at the landscape scale.

Maps prepared as part of the application process.

Baseline information available from records of previous AES agreements or interview surveys with land managers.

Facilitators report activities undertaken.

Spatial distribution of CS activities mapped by facilitators or NE.



Outputs data

Table 13: Potential approaches to CSFF M&E of facilitation outputs Output Indicator Purpose of data




The initial outputs produced by the facilitator and the participating land managers through the delivery of planned activities



Potential options for comparing the benefits of facilitation to scenarios without facilitation.

Source / point of data capture



Desk based outputs:

Agreed list of CS priorities to take forward across the land holdings in the facilitated management area

List of CS priorities to take forward across the land holdings in the facilitated management area

To record the CS priorities to take forward across the land holdings in the facilitated management area

Previous delivery for biodiversity / ecosystem services / CS priority type activities in the facilitated area, for example through: o Previous and

existing ES agreements.

o Landscape scale initiative objectives / activities.

o Activities undertaken by land managers of their own accord.

List of CS priorities.

Check that agreed CS priorities have followed through to the individual application of land managers in the facilitated group.


Baseline data on ES agreements can be obtained from existing AES datasets.

Information available on previous / continuing landscape scale initiatives.

Activities by land managers can be picked up through interview surveys.

Group agreement setting out how the group will

Approach to group operation identified in the

To provide information to determine the

Land manager experience of group

Date of the agreement

Content of the

Analysis of links between group agreement and


Baseline information available from



Output Indicator Purpose of data capture

Baseline Data requirements / information to record


operate, who will have what roles and how disputes will be handled

group agreement.

effectiveness of the group agreement and group operation.

agreements. group agreement

Identified roles

How the group operates and works together

effectiveness of group working (Option 2).

Group Agreement


facilitators or interview surveys with land managers.

Date of agreement and date at which facilitation started available from facilitator reporting.

Land manager training and knowledge transfer plan

Training and knowledge transfer undertaken by land managers

To provide information for use in assessing learning gained by land managers.

Previous advice, support and training received by land managers within the facilitated group.

Information from training and knowledge transfer plan on number and type of different training events planned.

Information to provide background for the interview surveys with land managers on knowledge, skills and attitudes.

Facilitator reporting: the training and knowledge transfer plan.

Facilitator quarterly reports.

Baseline data can be collected from facilitator reporting or interview surveys with land managers.

List and number of training events included within ‘land manager training and knowledge transfer plan’ and available through facilitator reporting.

Facilitation agreement – a detailed plan setting out the CS outcomes that are to be delivered within the facilitated management

Content of the facilitation agreement.

To provide information to determine whether the benefits of facilitation identified in the application form have been realised.

Experience of previous facilitated delivery in the CSFF area.

Benefits (outcomes) to be delivered

Reasons why outcomes are expected to be better through CSFF than if land managers were

Comparison of the agreed list of CS priorities in the facilitated area with a paired non-facilitated area (Option 1a).

Assessment of whether the

CSFF agreement.

CSFF application form (particularly Question 22)

Quarterly Reports.

Baseline information available from CSFF application form or facilitator reporting.

Information on CS outcomes to be delivered and






area (should also explain why the anticipated outcomes are expected to be better through CSFF than if the land managers were acting independently)

acting independently

benefits identified in the CSFF application form will be realised (Option 2).

reasons why outcomes will be better through facilitation available through CSFF application forms.

Preparation and submission of individual but complementary and spatially coherent, targeted and concentrated CS applications from individual land managers across the facilitated management area

Complementary and spatially coherent CS applications within the facilitated management area

To determine the difference made by facilitation on the achievement of complementary and spatially coherent CS applications.

Map showing spatial density and coherence of options (type, location and number of options).

Comparison of the spatial coherence of applications within facilitated areas with paired non-facilitated areas (Option 1a & 2).

Comparison of the spatial density of options within facilitated areas with paired non-facilitated areas (Option 1 & 2).


Relevant information is available and there is a database of CS applications that: o Records both

successful and unsuccessful applicants.

o Flags applications within a facilitated area and part of a facilitated group.

o Flags applications within a facilitated area but not part of a facilitated group.

o Indicates whether a holding was






previously within a facilitated landscape scale initiative.

o Whether support received in preparing application.

Links and synergies between facilitated land management activity and wider land management partnerships and initiatives identified

Links and synergies between facilitated areas and local land management partnerships / initiatives.

To determine the links and synergies between facilitated areas and local land management partnerships / initiatives.

Extract list of links and synergies from the CSFF application form.

Use as background information to the survey with land managers on the question of the benefits of linking with local partnerships and initiatives.



Additional data capture: surveys / interviews with land managers.

Links and synergies available from CSFF application form.

Information on benefits of linking with local partnerships and initiatives gained through interview surveys with land managers or from facilitators.

Facilitator required to provide quarterly reports

Content of facilitator quarterly reports.

To check that relevant information is provided in order to undertake evaluation.

Facilitator quarterly reporting.

Information contained in quarterly reports is used to monitor progress with activities and outputs.

Biodiversity / water environment practical land management and land manager training outputs






delivered on the ground:

Delivery of training and knowledge transfer to participating land managers

Training delivered and knowledge transferred.

To provide information to determine the influence of training and knowledge transfer on achievement of outcomes.

Type of training or other form of knowledge transfer delivered.

Number of events held.

Names and numbers of participating land managers.

Assessment of the use and value of training received by land managers (Option 2)



Further information on the usefulness and application of learning to be obtained through surveys with land managers.

Interview surveys undertaken with land managers to gain information on knowledge gained.

Priority habitats are placed under appropriate CS management options

Priority habitats placed under appropriate CS management options.



Comparison between facilitated areas and paired non-facilitated areas of the appropriateness of CS options applied to priority habitats (Option 1a & 2).

CS application data.

Ground-truthing of CS options in relation to priority habitats.

CS priority habitat delivery translated via BARS (as part of B2020 reporting obligation, in a similar way to the current HLS translation).

Landscape scale delivery of spatially coherent priority habitat restoration and creation projects

Spatially coherent priority habitat restoration and creation projects.

To determine the difference made by facilitation on the achievement of spatially coherent priority habitat restoration and creation projects.


Use of NBCCVM to test for delivery of spatially coherent priority habitat restoration and creation projects in facilitated and paired non-facilitated areas (Option 1a & 2). o Proportion of


BARS.


NBCCVM will demonstrate delivery of spatially coherent priority habitat restoration and creation projects.

Information on






land covered by measures for priority habitat restoration and creation projects in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

areas of land covered by measures for priority habitat restoration and creation projects available through BARS.

Additional resources available for data analysis.

Landscape scale delivery of spatially coherent land and water management designed to restore degraded ecosystems and provide climate change mitigation and adaptation

Spatially coherent land and water management measures.

To determine the difference made by facilitation on the achievement of spatially coherent land and water management measures.


Use of NBCCVM to test for delivery of spatially coherent land and water management to restore degraded ecosystems and provide climate change mitigation and adaptation (Options 1a & 2).

Proportion of land covered by measures for land and water management to restore degraded ecosystems and provide climate change mitigation and adaptation in facilitated areas



NBCCVM will demonstrate delivery of spatially coherent land and water management to restore degraded ecosystems and provide climate change mitigation and adaptation.







compared to paired non-facilitated areas (Options 1a & 2).

Landscape scale delivery of spatially coherent AES and woodland measures designed to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species

Spatially coherent AES and woodland measures.

To determine the difference made by facilitation on the achievement of spatially coherent AES and woodland measures.


Use of NBCCVM or another model to test for spatial coherence of AES and woodland measures designed to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species (Option 1a & 2).

Proportion of land covered by measures designed to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).



NBCCVM or another model will demonstrate delivery of spatially coherent AES and woodland measures to benefit key species groups (e.g. farmland and woodland birds) and specific S41 species.







Landscape scale delivery of spatially coherent plantations on ancient woodland sites (PAWS) restoration projects

Spatially coherent plantations on ancient woodland sites (PAWS).

To determine the difference made by facilitation on the achievement of spatially coherent plantations on ancient woodland sites (PAWS).


Use of NBCCVM or another model to test for delivery of spatially coherent plantations on ancient woodland sites (PAWS) restoration projects (Option 1a & 2).

Proportion of land covered by plantations on ancient woodland sites (PAWS) restoration projects (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).



NCCBVM or another model will demonstrate delivery of spatially coherent plantations on ancient woodland sites (PAWS) restoration projects.


Landscape scale, spatially coherent delivery of CS options benefitting water quality

Spatially coherent delivery of CS options benefiting water quality.

To determine the difference made by facilitation on the achievement of spatially coherent delivery of options benefiting water quality.


Use of NBCCVM or another model to test for delivery of spatially coherent delivery of CS options benefitting water quality (Option 1a & 2).



NBCCVM or another model will demonstrate delivery of spatially coherent delivery of CS






Proportion of land covered by of CS options benefitting water quality (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

options benefitting water quality.


Provision of water quality advice to land managers within areas identified as a priority for water quality interventions / synergies

Water quality advice provided.

To determine the effectiveness of water quality advice.

Number and types of option uptake.

Analysis using FARMSCOPER or the Catchment Change Matrix (Option 2).


Appropriate information gathered for use in FARMSCOPER or the Catchment Change Matrix.


Landscape scale, spatially coherent delivery of CS options designed for flood risk management

Spatially coherent CS delivery for flood risk management.

To determine the difference made by facilitation on the achievement of spatially coherent CS delivery for flood risk management.


Use of NBCCVM or another model to test for spatially coherent delivery of CS options designed for flood risk management (Option 1a & 2).

Proportion of land covered by CS options designed for flood



NBCCVM or another model will demonstrate spatially coherent delivery of CS options designed for flood risk management.

Additional






risk management (need to take account of potential overlap of measures on the ground) in facilitated areas compared to paired non-facilitated areas (Option 1a & 2).

resources available for data analysis.