monitoring and evaluation of cohesion policy austrian experience and perspectives bratislava,...

Download Monitoring and evaluation of Cohesion Policy Austrian experience and perspectives Bratislava, November 30th 2010 R. Hummelbrunner, Graz Austria Slovak

Post on 18-Dec-2015




0 download

Embed Size (px)


  • Slide 1
  • Monitoring and evaluation of Cohesion Policy Austrian experience and perspectives Bratislava, November 30th 2010 R. Hummelbrunner, Graz Austria Slovak Society for Evaluation
  • Slide 2
  • Key features of regional policy in Austria Regional policy is competence of Lnder (regions - NUTS II) National level: coordination, orientation (Federal Chancellery) Mechanism for aligning regional policies at national level (ROK) Cohesion Policy Structural Funds in Austria 8 OPs for Objective 2 at NUTS II level, 1 OP for Objective 1 Phasing Out (Burgenland), 7 OPs for Objective 3 (ETC) - CBC strand Programmes at national level for Employment and Rural Development No other thematic / sectoral OPs at national level Central Monitoring System for all ERDF Programmes Austrian Conference for Spatial Planning (ROK) functions as network of MAs for ERDF Programmes Regional and Cohesion Policy in Austria
  • Slide 3
  • Financial Resources 2007-13 Objective SF in Mio Current prices Convergence / Phasing Out of which ERDF of which ESF 177,1 125,0 52,1 Regional Competitiveness and Employment of which ERDF of which ESF 1.027,3 555,0 472,3 Territorial Co-operation256,6 of which cross-border Co-operation223,9 of which trans-national Co-operation32,7 Total:1.461,1 Source: ROK
  • Slide 4
  • Co-ordination / exchange of evaluations at national level Operational Programmes Evaluation Team Platform Evaluation (ROK) Contracts Cross-programme exchange, horizontal themes Evaluation Team Evaluation Team Evaluation Team Evaluation Team
  • Slide 5
  • Key innovation during 2000 2006: On-going evaluation Voluntary activity of various ERDF Programmes Objective 1 Programme Burgenland, five Objective 2 Programmes, four INTERREG IIIA Programmes, URBAN II Graz Period between MTE and MTE Up-date (2003 2005) Implementation of on-going evaluations Integral element of evaluation contracts by MAs Specification and steering mostly by Working Group at programme level Focus: Result oriented (impact assessment) and/or process oriented (understanding of implementation and framework conditions) Documentation: Internal papers, minutes of workshops/meetings, contributions for Annual Reports Exchange / reflection on process and results of on-going evaluations at national level (ROK)
  • Slide 6
  • Evaluation components in Austria Programme period 2000-2006 MTEOn-going PurposeExternal needsInternal needs RequirementsReg.1260; EC-WP Managing Authority TimingSet deadlinescontinuous Focusprimarily result-orientedResult & Processoriented 2000 - 2001 - 2002 - 2003 - 2004 - 2005 - 2006 On-going MTE Up-Date. Mid-Term Evaluation & Up-date embedded in on-going evaluation process
  • Slide 7
  • Mid-Term Evaluation and Up-date External requirement, timing and topics pre-defined Relevant for: MAs (e.g. use of resources, n+2, achievement of targets) Monitoring Committee (e.g. Programme changes, reallocations) On-going Evaluation (between MTE and Up-date) Utilization-focused: Information needs/questions of programme actors Quick results for programme modifications, quality improvement Relevant for: Implementing Bodies at Measure level (e.g. funding conditions, project selection and development, publicity and visibility of measures) MAs (e.g. implementing mechanisms, results and impacts) Relevance of evaluation for steering (actors, functions)
  • Slide 8
  • Experience with On-going Evaluations 2000 2006 Value-added of On-going Evaluation External view on programme implementation, External mirror for MAs New role of evaluators: Coaching of programme actors in view of professionalising the entire programme system In-depth treatment of selected topics / areas Accumulation of insights into programme realities Processing of information in small portions, easier to handle Quick and flexible reaction on questions / problems of MAs Rapid implementation of findings / recommendations Timely reaction and adjustments at programme or measure level
  • Slide 9
  • Governance process for NSRF ( and nine Objective 2 OPs, to Promote learning and exchange of experience on OPs Provide ideas and practical knowledge for various stakeholders Generate contributions for future regional policy (EU, national ) Based on established networks and structures Network of Managing Authorities (within ROK) Steering Group (on-going definition of thematic priorities, work plan) Main elements of governance process Work commissioned by ROK (evidence, expertise) Workshops to discuss/validate expert inputs Events for wider public (dissemination, information) Innovations 2007-13: Strategic governance process
  • Slide 10
  • Outline of strategic governance process in Austria External influence Outputs (Projekts) Results Community Strategic Guidelines Community Strategic Guidelines NSRF STRAT.AT NSRF STRAT.AT Objectives and Strategy (Sub)objectives per Priority Strategy fields GOVERNANCE PROCESS NSRF (= M&E, interactive exchange, Reflection > institutional learning) Territorial framework conditions (physical, socio-economical, institutional) Analysis Use of outputs Contributions to objectives Regional Impacts PROGRAMME GOVERNANCE (= Achievement of objectives, regional impacts) Operational Programmes - Priorities - Actions - Mechanisms Inputs (Resources, Rules)
  • Slide 11
  • Analysis of common themes / challenges Example: Assessment of Structural Funds implementation system in Austria (Governance Check) Joint reflection and learning Example: 15 Years of INTERREG / ETC programmes in Austria - Looking back - looking ahead Comparative implementation analysis Example: Cross-programme evaluation of national funding scheme for environment / climate change Participation in larger scale cross-programme evaluations Example: Cross-programme evaluation of ETC programmes in Central and Southeast Europe, Operational and Thematic aspects Innovations 2007-13: Cross-programme activities
  • Slide 12
  • Innovations 2007-13: Process Monitoring of Impacts What is Process Monitoring of Impacts? Tool for impact-led monitoring of an intervention Focus on processes, which should lead to impacts Use of Logic Models (impact diagrammes) What is it for? Illustrates causal relations between programme outputs and effects in a plausible manner Enables a concise, systemic overview of impact patterns of complex interventions, e.g. programmes Provides early information for programme management (MAs, IBs) required for effective steering
  • Slide 13
  • Monitoring of change processes Impact chains as basic scheme Inputs are used to produce outputs (= projects) Outputs are used (by someone, in a specific manner) to reach results Results will lead to impacts (intended / unexpected) in a plausible manner InputsOutputsResultsImpacts Financial TableTypology of projects (output) to be supported Objectives at the level of Areas of intevention / Activities Objectives at the level of Priority or Program Result- indicators Output- Indicators Commitments Disbursements
  • Slide 14
  • Impact diagramme AoI A 4.1 Softmeasures to strengthen innovations in entreprises Enterprises (especially SME) carry out product finding processes Enterprises (especially SME) conceive innovation-/ investment projects New / improved products, services and production processes Enterprises (especially SME) introduce new technologies Enterprises (especially SME) collaborate in networks (also with large enterprises) Increased linkages between enterprises at regional level / scale Enterprises (especially SME) gain new markets Adaptation to international competition Increase in employment / new jobs Services to sensitize for innovations AF6 External expertise, process consulting AF6 Advise for co-operation AF6 Enterprises (especially SME carry out reorganization processes Sustainable stabilization of enterprises New contacts with clients/ new orders
  • Slide 15
  • Pilot applications with programmes Reg. Competitiveness 2007 - 2013 Programme Styria Based on impact diagrams established during programme preparation Application with 5 Areas of Intervention (of totally 11) Project sample (used for up-dating of impact diagrams): 70 Projects approved (used for weighting of impact chains): 840 Duration of pilot application: July - November 2009 Programme Carinthia Internal assessement of projects approved so far (ca. 40) based on impact diagrams (dating from ex-ante evaluation) Assessment accompanied externally Finalization (Report) October 2009
  • Slide 16
  • Application with programme Styria 2007 - 2013 Procedure / steps 1.Up-dating impact diagrams from the programming process (based on current version of internal Program Complement ) 2.Attributing all (840) projects in line with their primary impact chain (basis: short descriptions contained in ERDF Monitoring system, information provided by IBs) 3.Weighting of impact paths according to number of projects and financial volume, calculation and graphic representation 4.Integrating indicators (from ERDF Monitoring database) to provide information on final achievements of processes 5.Validating and complementing impact hypothesis through interviews with (selected) project owners 6.Discussing findings / conclusions with IBs (Funding Authorities)
  • Slide 17
  • Impact diagramme AoI 4.1 Softmeasures to strengthen innovations in entreprises Enterprises (especially SME) carry out product finding processes Enterprises (especially SME) conceive innovation-/ investment projects New / improved prod


View more >