Walking on two legs:LEARNING EVALUATION

1

• Göran Brulin, Senior Analyst and professor, Swedish Agency for Economic and Regional Growth

• Sven Jansson, National Coordinator for Evaluation and Monitoring, Swedish ESF Council.

2

Walking on two legs:LEARNING EVALUATIONAccording to the strategy “Europe 2020” an economy based on knowledge and innovation should be developed. EU should promote a resource efficient, greener and more competitive economy that leads to growth. It should foster a high-employment economy that help deliver social and territorial cohesion!

3

Smarter, inclusive growth calls for a change in how programs and projects are conducted and funds carried out!

• There is a lack of learning approaches in evaluations!

• Evaluations are conducted just for the sake of evaluation!

• Evaluations are ritual and symbolic activities rather than processes for critical and constructive knowledge formation!

4

Reflections after the last programming period regarding learning and evaluation:- The Swedish National Audit Office: too much detailed control and

monitoring – too small investments in learning to support regional development!

- Swedish Institute for Growth Studies: big investment in regional development projects from the EU funds, almost no effects on regional growth can be identified!

- Swedish Agency for Public Management: too many small projects; too scattered project portfolio; hardly any cooperation between the Regional and Social funds!

- The Guidelines from the Commission: the traditional mid-term evaluations were often conducted too late and had little impact on the implementation of the programmes; should be replaced by ongoing evaluation in projects and of programmes this programming period!

5

Joint evaluation of the implementation organisation

”Ongoing evaluation” of programmes and in projects.

”Evaluation capacity building”

”Public debate”for sustainable regional growth and jobs!

Learning spiral:

6

The main task in the new generation of evaluation is to organise reflective learning processes for continuous improvement!

• Mid term evaluation came too late and was too expensive; minor impact in managing of the programmes.

• Quantitative rather than qualitative focus.• Indicatororiented evalutions means heavy focus on activities in

the projects (at the expense of the overall objectives). How convincing are the core indicators?

• Limited learning in the projects, in the program context and between stakeholders in national framework (labour market policy and regional growth policy actors).

7

Actions taken in Sweden during the programming period 2007 – 2013:• Ongoing evaluation of the eight ERDF programs,

and in major strategic projects.

• Ongoing evaluation in ESF (OP) and in major of project.

• Joint evaluation between the ESF and ERDF of the implementation organisation.

• Joint university course and reader in Learning Through Ongoing Evaluation at five universities

Actions taken in Sweden during the programming period 2007 – 2013• Learning network between evaluator creates

learning and capacity building

• Joint learning conferences and seminars in each of the eight regions/structural programmes

• Support project on Learning and Strategic Impact

8

9

Learning evaluation in the organisation of implementation:

• The ongoing evaluation has initially focused on the effectiveness processes of preparation of applications, case handling and prioritisation.

During the second half of the programming period, the ongoing evaluation:

• focus more on learning and strategic orientation.• and on whether the implementation organisation has managed to

assimilate documented experience and steer towards the programme goals better.

A main result of this effort has meant that customer focus has risen in both funds and initial conflicts in the implementation organisation have been solved!

10

Learning evaluation at the programme level:• An ongoing evaluation and process support structure of the eight

Swedish regional fund programmes (ERDF:s) are carried out.

• Teams of researchers are continuously studying the progress of the programmes, analysing the evaluation of the projects, and evaluating how the work is progressing at regional level. The main task is to ensure that the project portfolio is well balanced in terms of the programme’s goals. Results and impacts are in focus.

• Learning processes between the projects and in the programmes have been initiated and enhanced.

11

Learning evaluation at the project level:• Larger projects are requested to carry out ongoing

evaluation. They are supposed to set up a plan for ongoing evaluation, call for tender and learn from the ongoing evaluation to support continuous improvements.

• This evaluation approach supports project leader as well as managing authority and other stakeholders with knowledge.

• In general ongoing and learning evaluation at the project level has meant a clearer focus on job creation, regional growth and innovation.

12

The Social Fund support learning throughongoing evaluation by different means:• The Process Support Projects which aims to increase

opportunities for different actors in the system to set up new projects that are innovative and have an additional edge compared to traditional labour market policy.

• To shape learning to a wider circle of stakeholders and system changes five Thematic Project, which are designed to identify, refine and disseminate lessons learned from the projects and programme, have been started.

13

Structural fund programs could do better to support innovation, job creation and growth:

• if they work as ”venture capitalist” with ongoing evaluation of projects and learning processes within and between the projects.

Thereby, both experiences from processes and knowledge about ”products” and methods can be gathered.

By participation in public debate insights are spread and regional development are energized.