united we stand. evaluating, cooperating, and other unlikely stories of evaluation capacity building...
TRANSCRIPT
![Page 1: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/1.jpg)
United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity
Building in Italy
Martina Bolli, Silvia Ciampi, Francesco Giordano & Laura Tagle
Italy’s National Evaluation System
![Page 2: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/2.jpg)
Content
1.Cohesion policy in Italy: long-term
background & contrasting
directions
2.The Italian evaluation policy
3.Efforts & (unlikely) achievements…
plus some drawbacks
![Page 3: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/3.jpg)
COHESION POLICY IN ITALY: LONG TERM BACKGROUND & CONTRASTING DIRECTIONS
![Page 4: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/4.jpg)
2007-2013 - Evaluation as one of the pillars of a (much needed) renewal of Italy’s regional policy
• Citizen-centered policies
• Strong focus on essential services—rather than on procedures & expenditure
• Emphasis on local knowledge
• Responsibility to regional governments
• Links with national policies
• Greater importance to administrative capacity
![Page 5: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/5.jpg)
The regional (= cohesion) & rural development policy in Italy
• Intentionality of territorial goals
• Additionality vis-à-vis ordinary policy
Resources come from EU & National funds
![Page 6: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/6.jpg)
Levels of programming: a complex framework
• National “umbrella” strategies for regional & rural development policies
• 7 Ministries develop their own programs• Each of the 21 Regions develops
– its multi-sector strategy– 5 Programs (each program is funded by one source)
“Naturally”, each of the 5 programs is usually managed by a separate Directorate
“Programs” end up being bureaucratic constructs rather than guides for action
![Page 7: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/7.jpg)
Over time, silos have been created
Sources of money
• EU Regional Fund• National funds • EU Social Fund• EU Rural Development
Coordination organisms
• Ministry for economic development
• Ministry for agriculture• Ministry for labour
Other departments
![Page 8: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/8.jpg)
….Beyond silos
2007-2013: intentional efforts to bridge gaps & introduce policy dialogue among and within
authorities in a new perspective
![Page 9: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/9.jpg)
At the same time, EU regulations ask for separate
programmes by EU source of money (& within Italy the
momentum for integration dwindles)
The National Strategy for 2007-2013 tried to make the various sources of funding work together through common:
• Programming
• Implementation
• Monitoring
• Surveillance
• Evaluation
![Page 10: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/10.jpg)
21 Regions (70%) 6 Ministries (30%)
Centre-North: 15% South: 85%
3bln€ to be given to Southern Regions on the basis of performance in waste, water, education and social services
Regional development strategyBetween 2007 & 2010: lower national resources (other than national cofinancing)•different distribution North-South•the instruments needed to implement nationally-relevant innovations (among which greater integration) dwindle
![Page 11: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/11.jpg)
THE ITALIAN EVALUATION POLICY
![Page 12: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/12.jpg)
Evaluation capacity building/1
From EU evaluation requirements towards a national policy
• Institutional building:– Creation of Evaluation Units within executive branch of
Regional authorities – Network of Evaluation Units
• Specialized: – Regional and Rural Development policies only
• Both demand & supply side– Improve regional authorities’ ability to Demand for & Use
evaluations
– Improve Evaluation Units ability to Manage and Conduct evaluations
![Page 13: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/13.jpg)
Evaluation capacity building/2
• Organized guidance & support (National Evaluation System)
• Enabling approach:– No rigid prescriptions– Reputational mechanisms
• Legitimize & support innovators at regional level
![Page 14: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/14.jpg)
Evaluation policy /1
• Nationally-funded interventions must be evaluated--not only those funded by EU
• Objects: – effects of a well-defined (however complex) intervention or group
of interventions on a problem, a social group, or an area—rather than a bureaucratic construct (e.g., a multisectoral “program”—tipically operating on a large area, affecting millionsof people)
– across bureaucratic borders: the evaluation object may have been financed by more than one source
– selectivity: choose to answer defined questions about “crucial & controversial issues”
![Page 15: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/15.jpg)
Evaluation policy/2
• Each Region drafts an Evaluation Plan:– What is evaluated (and what is not)
– Who (internally or externally) performs each evaluation
– When
– Organization of evaluation function
– Resources (human and financial) devoted to evaluations
![Page 16: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/16.jpg)
The National Evaluation System• UVAL – Ministry of Economic Development
(Coordinates)
• INEA – Ministry for Agricultural, Food and Forestry Policies
• ISFOL – Ministry of Labour
• Department for Equal Opportunities
• Evaluation Units of Regional Authorities
Its composition shows a clear intention to work together across silos
& involving Regions in a federal function
![Page 17: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/17.jpg)
The National Evaluation System Each component differs from the others
• Central units:– Have differing rules, interpretations, goals, tasks &
status
• Regional evaluation units:– Conduct or manage evaluations…– … but also provide project selection, planning,
knowledge management to the Region– Have background and skills closer to regional fund
policies (or national funds) rather than rural development or active labour policies
![Page 18: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/18.jpg)
EFFORTS & (UNLIKELY) ACHIEVEMENTS… PLUS SOME DRAWBACKS
![Page 19: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/19.jpg)
Common guidance - include & adjust for differing rules, traditions, goals
Personalized support to Regions (all managing authorities together)
Ensure quality & evaluation of evaluations (....)
Observe processes & disseminate information (jointly)
Facilitate dissemination (conference-like meetings, workshops, create & maintain eval database www.dps.tesoro/valutazione.it)
A “federal” function performed across silos & by central & regional units
![Page 20: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/20.jpg)
Some results
• All Regions developed an Evaluation Plan http://www.dps.tesoro.it/uval_linee_valutazione.asp#10
• Evaluations have been launched:– and even completed: 50! (in addition to the mandatory ex
post evaluations of rural development)
– and a lofty 60% available on the internet
• Evaluation units (established between 2000 & 2002) have started conducting internal evaluations
Still far from perfection but …
![Page 21: United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy Martina Bolli, Silvia Ciampi, Francesco Giordano](https://reader033.vdocuments.mx/reader033/viewer/2022061305/5514605d550346414e8b58be/html5/thumbnails/21.jpg)
Why we are happy anyway…
• Activities started (and 50 concluded) even though – (or just because?) there are no enforceable
requirements (no sanctions) – there is still strong inertia in the system
• old evaluations fulfilled needs of managing authorities, national coordinators, and Commission rapporteurs
• restricted market with “marriages” between managing authorities & evaluators
– still little pressure from social partners & general public
• Innovations go together: evaluations seem to be used as an instrument by Regions willing to improve
• Innovations implemented where there has been more support from central state (not only on evaluation)