Implementing the Implementing the NATIONAL EVALUATION NATIONAL EVALUATION

STRATEGYSTRATEGYNiall McCann, Technical Assistance Project Niall McCann, Technical Assistance Project

on Programming, Monitoring and on Programming, Monitoring and EvaluationEvaluation

MINISTRY OF PUBLIC FINANCEMINISTRY OF PUBLIC FINANCEMANAGING AUTHORITY FORMANAGING AUTHORITY FOR

COMMUNITY SUPPORT FRAMEWORKCOMMUNITY SUPPORT FRAMEWORK

Parliament Building, Bucharest, 2 November Parliament Building, Bucharest, 2 November 20062006

The Strategy’s fields of The Strategy’s fields of intervention – the intervention – the

activities, the activities, the indicators, and next indicators, and next

stepssteps Fields of interventionFields of intervention1.1. TrainingTraining2.2. Network and resource supportNetwork and resource support3.3. Evaluations! (including evaluations of EU Evaluations! (including evaluations of EU

funded programmes, and nationally-funded funded programmes, and nationally-funded public interventions, initially through the public interventions, initially through the “Evaluation Facility.”“Evaluation Facility.”

4.4. Promoting the gathering of data relevant for Promoting the gathering of data relevant for evaluationevaluation

5.5. Embedding of evaluation practice in the design Embedding of evaluation practice in the design and management of public interventions and management of public interventions through a normative actthrough a normative act

1. Training1. Training

Training will encompass two specific Training will encompass two specific variants:variants:

1.1. Training in how to Training in how to dodo evaluations evaluations Target group – external evaluators in the Target group – external evaluators in the

academic, civil society and private sectors academic, civil society and private sectors (consultancy companies, etc.) – funding (consultancy companies, etc.) – funding source – Phare 2005 source – Phare 2005

2.2. Training in how to Training in how to managemanage evaluations evaluations Target group – civil servants and other Target group – civil servants and other

public managers – funding source – OP ACDpublic managers – funding source – OP ACD

2. Network and resource 2. Network and resource support support

The MACSF ECU will be supported to:The MACSF ECU will be supported to: Build up its role as a source of evaluation expertise Build up its role as a source of evaluation expertise

and a knowledge resource for other public bodies and a knowledge resource for other public bodies on all matters related to evaluation – funding on all matters related to evaluation – funding source – OP ACDsource – OP ACD

Liaise with an independent network of evaluation Liaise with an independent network of evaluation practitioners – funding source – Phare 2005 practitioners – funding source – Phare 2005

An independent network of evaluation An independent network of evaluation practitioners will also be supported to nurture practitioners will also be supported to nurture the development of a strong, independent the development of a strong, independent evaluation community – funding source – evaluation community – funding source – Phare 2005Phare 2005

3. 3. Evaluations!Evaluations!

A number of distinct evaluation A number of distinct evaluation programmes will be supported:programmes will be supported:

1.1. Evaluation Facility 1Evaluation Facility 1

2.2. Evaluation Facility 2Evaluation Facility 2

3.3. Phare Interim Evaluation SchemePhare Interim Evaluation Scheme

4.4. Interim evaluations of the NSRF Interim evaluations of the NSRF Operational ProgrammesOperational Programmes

5.5. Interim evaluations of the Fisheries OP Interim evaluations of the Fisheries OP and the National Plan for Rural and the National Plan for Rural DevelopmentDevelopment

3. Evaluations! (continued)3. Evaluations! (continued)

1.1. Evaluation Facility 1Evaluation Facility 1 – funding source – – funding source – Phare 2005 – Phare 2005 – Public institutions, Public institutions, primarily those that may not have had primarily those that may not have had any engagement with evaluation up to any engagement with evaluation up to now, will be encouraged to have their now, will be encouraged to have their public interventions evaluated by external public interventions evaluated by external evaluators.evaluators.

2.2. Evaluation Facility 2Evaluation Facility 2 – funding source – funding source OP ACD – this will, in effect, be an OP ACD – this will, in effect, be an expanded continuation of the pilot expanded continuation of the pilot programme (above). Evaluations will programme (above). Evaluations will cover all types of public interventions cover all types of public interventions (projects, programmes, policies).(projects, programmes, policies).

3. Evaluations!3. Evaluations!

3. 3. Phare Interim EvaluationPhare Interim Evaluation – funding source – Phare – funding source – Phare 2005 – each of the 10 Phare Monitoring Sectors will 2005 – each of the 10 Phare Monitoring Sectors will be evaluated on an annual basis. The successful be evaluated on an annual basis. The successful consultants will also conduct ad hoc and thematic consultants will also conduct ad hoc and thematic evaluations, as well as an annual country summary evaluations, as well as an annual country summary report.report.

4. 4. Interim evaluations of the NSRF OPsInterim evaluations of the NSRF OPs – funding – funding source – NSRF OP Technical Assistance Priorities, source – NSRF OP Technical Assistance Priorities, and OP Technical Assistance – the NSRF OPs will be and OP Technical Assistance – the NSRF OPs will be subject to interim evaluations in both 2009 and 2012. subject to interim evaluations in both 2009 and 2012. Ad hocAd hoc and and thematicthematic evaluations will also be evaluations will also be commissioned at either NSRF, OP, Priority Axis or commissioned at either NSRF, OP, Priority Axis or Key Area of Intervention level, that will examine Key Area of Intervention level, that will examine implementation or “horizontal” issues (equal implementation or “horizontal” issues (equal opportunities, etc.)opportunities, etc.)

5. 5. Interim evaluations of the Fisheries Op and the Interim evaluations of the Fisheries Op and the NPRDNPRD

4. 4. Promoting the Promoting the gathering of data gathering of data

relevant for evaluationrelevant for evaluation Funding source – OP ACD – aims Funding source – OP ACD – aims

to encourage development of to encourage development of monitoring and statistical monitoring and statistical systems that allow for easier systems that allow for easier access to relevant data for access to relevant data for evaluation throughout the evaluation throughout the entire public administration.entire public administration.

5. 5. Embedding evaluation Embedding evaluation practice in public practice in public

interventions through a interventions through a normative actnormative act

A two-staged approach:A two-staged approach:1.1. Researching and drafting of normative Researching and drafting of normative

acts necessary to embed evaluation acts necessary to embed evaluation practice into the design and practice into the design and management of public interventions – management of public interventions – funding source – Phare 2005funding source – Phare 2005

2.2. Assisting the public administration to Assisting the public administration to implement changes necessary to implement changes necessary to comply with the legal requirements, comply with the legal requirements, following Parliament’s adoption of the following Parliament’s adoption of the normative acts.normative acts.

Indicators to verify Indicators to verify implementation of implementation of

activitiesactivities Indicators are identified for each Field of Indicators are identified for each Field of

Intervention (FoI)Intervention (FoI) Indicators allow the NES Implementation Committee Indicators allow the NES Implementation Committee

to determine whether the identified activities are to determine whether the identified activities are being implemented, and should be updated annually.being implemented, and should be updated annually.

Some of the indicators are:Some of the indicators are: 60 students to graduate from evaluation training courses 60 students to graduate from evaluation training courses

for external evaluators (Training for external evaluators (Training FoI) FoI) At least 2 persons from middle-level management of county At least 2 persons from middle-level management of county

and city councils trained in the commissioning and and city councils trained in the commissioning and management of evaluation (Training FoI)management of evaluation (Training FoI)

A central database (and website) of evaluations is managed A central database (and website) of evaluations is managed by MACSF ECU, and updated on an ongoing basis by MACSF ECU, and updated on an ongoing basis (Resource and networking FoI)(Resource and networking FoI)

Evaluations of local service delivery commissioned by at Evaluations of local service delivery commissioned by at least 5% of local public authorities each yearleast 5% of local public authorities each year (Evaluation (Evaluation Facility FoI) Facility FoI)

Next steps…Next steps… The National Evaluation Strategy Implementation The National Evaluation Strategy Implementation

Committee should convene and agree a mandate – February Committee should convene and agree a mandate – February 20072007

Within each Field of Intervention the first steps should be…Within each Field of Intervention the first steps should be…1.1. TrainingTraining

TheThe “Development of a Professional Evaluation Community” Phare “Development of a Professional Evaluation Community” Phare project should be contracted – project should be contracted – July 2007July 2007

The MACSF ECU, the OP ACD MA should agree a 2-year action plan The MACSF ECU, the OP ACD MA should agree a 2-year action plan on a framework for evaluation training courses within the public on a framework for evaluation training courses within the public sector –sector – May 2007 May 2007

2.2. Resource and network supportResource and network support The MACSF ECU should agree an annual Work PlanThe MACSF ECU should agree an annual Work Plan - - August 2007August 2007 The MACSF ECU, the “Development of a Professional Evaluation The MACSF ECU, the “Development of a Professional Evaluation

Community” consultants and members of the Romanian evaluation Community” consultants and members of the Romanian evaluation community interested to develop an independent network of community interested to develop an independent network of evaluators should agree a plan of activitiesevaluators should agree a plan of activities – September 2007 – September 2007

3.3. Evaluations!Evaluations!

4.4. Promoting the gathering of data relevant for evaluationPromoting the gathering of data relevant for evaluation asdfasdf

5.5. Embedding evaluation practice in public interventions through Embedding evaluation practice in public interventions through a normative acta normative act

asdfasdf

Next steps (continued)Next steps (continued)……3. Evaluations!3. Evaluations!

Phare 2005 consultants will agree the programme of Phare 2005 consultants will agree the programme of evaluations with public administration bodies and the MACSF evaluations with public administration bodies and the MACSF ECU – ECU – January 2007January 2007

The Phare Interim Evaluation Scheme 2007-2009 should be The Phare Interim Evaluation Scheme 2007-2009 should be awarded – awarded – January 2007January 2007

OP Managing Authority Evaluation should complete their OP OP Managing Authority Evaluation should complete their OP MAEP – MAEP – January 2007January 2007

F4H.2 The MAC SF ECU should complete the NSRF MAEP – F4H.2 The MAC SF ECU should complete the NSRF MAEP – March 2007March 2007

The Managing Authorities for the NPRD and the Fisheries OP The Managing Authorities for the NPRD and the Fisheries OP should complete MA EP with the assistance of the MACSF should complete MA EP with the assistance of the MACSF ECU – ECU – June 2007June 2007

4. Promoting the gathering of data relevant for evaluation4. Promoting the gathering of data relevant for evaluation The MACSF shall oversee a programme of work to install The MACSF shall oversee a programme of work to install

proper, evaluation-friendly monitoring systems at all levels of proper, evaluation-friendly monitoring systems at all levels of the public administration – the public administration – ongoingongoing

5. Embedding evaluation practice in public interventions through 5. Embedding evaluation practice in public interventions through a normative acta normative act

Phare 2005 consultants will employ legal expertise to draft Phare 2005 consultants will employ legal expertise to draft whatever normative acts are necessary in order to embed whatever normative acts are necessary in order to embed evaluation practice into the management of public evaluation practice into the management of public interventions throughout the public administration – interventions throughout the public administration – December 2007December 2007