esfri workshop monitoring of ris, periodic update of ... · esfri workshop – monitoring of ris,...
TRANSCRIPT
ESFRI WORKSHOP MONITORING OF RIs, PERIODIC UPDATE OF
LANDMARKS, USE OF KPIs
19-20 NOVEMBER 2018 Palazzo delle Stelline - Milano
Giorgio Rossi
We are convened at the former Orfanotrofio delle Stelline “Orphanage of the Starlets”
The “Spedale dei Poveri Mendicanti e Vergognosi” (Hospice for the Poor Beggars and Shameful) was created in 1578 on input by Carlo Borromeo (one of the leaders of the Catholic Counter-Reform of XVI century, with Ignazio da Loyola and Filippo Neri) within the premises of the monastery “Santa Maria della Stella”. The Hospice was open to all who where forced to beg for lack of subsistence. In 1534 the order of Padri Somaschi had built a dedicated orphanage for women. In 1583 Borromeo gave a strict rule to both the Hospice and the Orphanage.
In 1752 Maria Teresa d’Austria established the Spedale as Royal Female Orphanage, with new rules issued directly from Vienna.
The education of the girls was basic end less intense than for the corresponding male institution “I Martinitt”. Aim of the “Stelline” was to train the girls to be good mothers and house-wives. The more gifted ones were trained as house managers in noble family homes and were educated in art, music, dance, French or German language, history and galateo. In 1807 the Congregazione di Carità Napoleonica coordinated the administration of the Stelline and Martinitt with the Pio Albergo Trivulzio (old people hospital) and later, after Unità d’Italia, they became a unique administration. At the beginning of XX century the Stelline started to join public school and training outside the orphanage. As of 1970 the system has transformed in family-style housing communities the Palazzo delle Stelline was sold to the Comune di Milano that transformed it into a conference center.
Just Across the Street you see the church Santa Maria delle Grazie is a BASILICA and SANTUARIO belonging to the ORDINE DOMENICANO, built between 1492 and 1493 by the architect GUIINIFORTE SOLARI (construction master of DUOMO, OSPEDALE MAGGIONE –now University – and CERTOSA DI PAVIA) and BRUNELLESCHI upon patronage of the Duke of Milano LUDOVICO IL MORO to become a mausoleum for his family.
1943 bombing
Leonardo da Vinci, 1494-1498 fat tempera on plaster 460×880 cm In the REFETTORIO of the CONVENTO owned by COMUNE DI MILANO
THIS WORKSHOP ROADMAP 2016-2018 + new mandate of COUNCIL May 2018
• THE FIRST THREE ESFRI ROADMAPS (yellow period 2006-2010) led to a portfolio of 40 Projects
• In December 2012 the Competitiveness Council: “...EMPHASISES the need for renewing and adapting the mandate of ESFRI to adequately address the existing challenges and also to ensure the follow up of implementation of already ongoing ESFRI projects afte a comprehensive assessment, as well as the prioritisation of the infrastructure projects listed in the ESFRI Roadmap
• In 2012 ESFRI created an ad-hoc WG on “INDICATORS OF PAN-EUROPEAN RELEVANCE OF RIs” that reported to Forum in 2013
• In 2013 ESFRI and the EC run an ASSESSMENT of the ESFRI Projects in view of the INNOVTION EUROPE objective of IMPLEMENTING 60% of the Roadmap by 2016. It was carried out by the ad-hoc HIGH LEVEL EXPERT ASSESSMENT GROUP and led to conclusions on RIs being at different stages of implementation.
• IN 2014 ESFRI formulated the PRIORITIZATION OF SUPPORT TO ESFRI PROJECTS FOR IMPLEMENTATION that led to the recommendation of SPECIAL FUNDING to 12 RIs that needed extra support to meet the 2016 implementation deadline
• IN 2016 the NEW ESFRI ROADMAP (inaugurating the blu period) introduced the “eligibility maturity threshold” and the concept of 10-year cycle for PROJECTS and defined the LANDMARKS (29 assessed)
• The ROADMAP 2016 accordingly introduced the regular MONITORING of the ESFRI RIs, in praticular in view of 10-year expiration, and addressed the need of regular review of LANDMARKS
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
THIS WORKSHOP ROADMAP 2016-2018 + new mandate of COUNCIL May 2018
• In 2017-2018 ESFRI monitored the Projects and started addressing the development of the PERIODIC REVIEW of Landmarks
• In 2017 ESFRI engaged 4 Landmarks into a PILOT EXERCISE in view of formulating the REVIEW
• In 2018 the ESFRI FORUM discussed at length the concept of PERIODIC REVIEW, with different views expressed and a final
synthesis that involved a long non-session in Corfu when the formula of PERIODIC UPDATE of LANDMARKS was agreed
• The ROADMAP 2018 assessed for 8 Projects the new status of LANDMARKS
• In May 2018 the Competitiveness Council: “…INVITES Member States and the Commission within the framework of ESFRI to
develop a common approach for monitoring of their performance and… INVITES the Pan-European Research Infrastructures, on
a voluntary basis, to include it in their governance and explore options to support this through the use of Key Performance
Indicators”.
• ESFRI consequently decided to create and ad-hoc WG on MONITORING and approved its ToR in September 2018
• THIS WORKSHOP is therefore meant to address: LANDMARK PERIODIC UPDATE method and the MANDATE ON MONITORING
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
ABOUT ESFRI LANDMARKS ROADMAP 2016
1. FIRST INTRODUCED IN ROADMAP 2016 AS FOLLOWS
2. THE ROADMAP 2016 COMPRISED 29 LANDMARKS that have reached the
implementation phase are pan-European hubs of scientific excellence, and will require
continuous support to fulfil their mission and ensure their long-term sustainability
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs
19-20 November 2018, Milano
GIORGIO ROSSI
ABOUT ESFRI LANDMARKS ROADMAP 2016
3. LANDMARKS AS COMPLEMENTARY TO ESFRI PROJECTS
4. TEN-YEAR TIME WINDOW for completing their incubation and reaching
implementation
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
ABOUT ESFRI LANDMARKS ROADMAP 2018
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
1. THE DEFINITION HAS BEEN REFINED AS FOLLOWS
2. INTRODUCTION OF THE LYFECYLE APPROACH AND FURTHER REFINEMENTS
1. THE EVOLUTION IN TIME OF ESFRI PROJECTS AND
ESFRI LANDMARK
2. THE NEEDS AND TARGETS OF THE IMPLEMENTATION
PHASE ARE UNDERSTOOD AS A SEQUENCE OF PHASES
FROM THE CONCEPT TO OPERATION AND TO
TERMINATION
THE LYFECYCLE APPROACH FROM PROJECTS TO LANDMARKS
THE ROADMAP DYNAMICS MONITORING OF PROJECTS
GIORGIO ROSSI
1. PROJECTS HAVE TEN YEARS OF RESIDENCY ON THE ROADMAP TO REACH IMPLEMENTATION: THOSE THAT DO NOT MEET THIS GOAL ARE REMOVED FROM THE ROADMAP
2. CHECK OF THE OVERALL PROGRESS TOWARDS IMPLEMENTATION ACCORDING TO THE FULFILMENT OF MINIMAL KEY REQUIREMENTS AS DEFINED IN THE ESFRI ROADMAP GUIDE
3. ADVICES TO THE PROJECTS AND RECOMMENDATIONS TO THE FORUM, INCLUDING THE POSSIBLE PROMOTION OF PROJECTS TO THE STATUS OF LANDMARK
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
THE PILOT PERIODIC REVIEW OF ESFRI LANDMARKS has been carried out to test the methodology for assessing the state of play
of the ESFRI Landmarks. Four Landmarks volunteered to be part of the pilot exercise, upon invitation. They were:
• ICOS ERIC (ENV) AS A RI WITH GLOBAL OUTREACH
• ELIXIR (H&F) AS A DISTRIBUTED RI EXAMPLE DEALING WITH HIGH DATA-INTENSIVENESS AND E-NEEDS
• SPIRAL2 (PSE) AS A SINGLE SITE RI WITH SPECIAL IMPLEMENTATION CONDITIONS
• ESS ERIC (SCI) AS A DISTRIBUTED RI WITH ISSUES CONNECTED TO THE MANAGEMENT OF SOCIAL SCIENCES DATA AND E-NEEDS
The lessons learnt from the pilot exercise were presented to the EB and to the Forum.
The discussion at all FORUM meetings in 2018 addressed the very NATURE and SCOPE of the MONITORING in the case of
LANDMARKS
THE ROADMAP DYNAMICS PILOT PERIODIC REVIEW
THE LANDMARK DYNAMICS UPDATING OF LANDMARKS
GIORGIO ROSSI
1. LANDMARKS HAVE LONG TERM RESIDENCY ON THE ROADMAP TO PERFORM OPERATION AT THE TOP LEVEL IN THEIR DOMAIN AND AS REFERENCE RESEARCH INFRASTRUCTURES THAT NEED CONTINUOUS SUPPORT
2. PERIODICALLY THE EVIDENCE OF THE LANDMARK STATE OF PLAY (PERFORMANCE) MUST BE UPDATED IN A NON-INVASIVE, BUT FULLY TRANSPARENT WAY
3. A SUSTAINABLE CYCLE (FOR RIS AND FOR ESFRI) OF UPDATE CYCLE (5-7 YEARS, 5-6 LANDMARKS/YEAR) MUST BE ESTABLISHED, INDEPENDENT OF THE PERIODICITY OF THE ROADMAP UPDATES
MONITORING OF RIs NEW MANDATE
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
EU COUNCIL CONCLUSIONS OF 29 MAY 2018 ON ACCELERATING KNOWLEDGE CIRCULATION IN THE EU
“Stresses the importance of human resources and training skills as key factors in the success for Research
Infrastructures and ACKNOWLEDGES the need for Research Infrastructures to strengthen a service-driven
approach; INVITES Member States and the Commission within the framework of ESFRI to develop a common
approach for monitoring of their performance and INVITES the Pan-European Research Infrastructures, on a
voluntary basis, to include it in their governance and explore options to support this through the use of Key
Performance Indicators”.
THESE RECOMMENDATIONS INSPIRE THE FORMULATION OF THE
PERIODIC UPDATES OF LANDMARKS, and the work TOWARDS A FEW GENERIC KEY PERFORMANCE
INDICATORS.
PERIODIC UPDATES OF LANDMARKS THE WAY FORWARD
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
The concept of ESFRI Landmarks implies PERIODIC UPDATING of their state of play throughout ALL THE
LIFECYCLE PHASES.
ESFRI is aware that THE INCLUSIONS OF LANDMARKS in its Roadmap often ARE CONDITIONAL ON (ELIGIBILITY
FOR) REGIONAL, NATIONAL AND EUROPEAN FUNDING and underlines that these inclusions are a result of
the robust and transparent methodology.
ESFRI will be responsive to the possible requests for guidance indicated by the Research
Infrastructures, and will TAKE NOTE OF ALL THE INTERNAL AND EXTERNAL EVALUATIONS performed by their
stakeholders and users communities, in order to be effectively updated on the state of play of the
Landmarks.
PERIODIC UPDATES OF LANDMARKS THE WAY FORWARD
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
The GOAL of Periodic Updates of ESFRI of ESFRI Landmarks is TO UNDERSTAND THE SCIENTIFIC STATUS OF
EACH LANDMARK, considering its specific position in its lifecycle, and enabling ESFRI to fully deploy
its strategic role and support the Landmarks.
Moreover, periodic updates on ESFRI Landmarks will facilitate meaningful interaction between
ESFRI and its Delegations to ADDRESS THE GAPS AND AREAS OF STRATEGIC IMPORTANCE highlighted in the
Landscape analyses.
The periodic update of ESFRI Landmarks differs fundamentally from the evaluation and
assessment of proposals and from the monitoring of Projects along the following lines.
PERIODIC UPDATES OF LANDMARKS THE WAY FORWARD
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
• It will be independent from the ESFRI Roadmap updates and each Landmark is addressed for update periodically, taking into
account the specific context of each RI. The updates will be “reflected” in the Annual Reports and Roadmaps.
• The periodic updates primarily take stock of external and internal evaluations provided by the Landmarks themselves and
carried out by their internal bodies, funders and stakeholders. Importantly, the periodic review is directed towards providing
a basis for dialogue between ESFRI, Landmarks and stakeholders, taking into account also national roadmap processes.
• ESFRI will identify a combination of generic KPIs and specific KPIs – suggested by the RI itself – for updating the state of play
of the Landmarks. It is important that each KPI is set against certain targets or metrics in terms of self-assessment and
quality assurance.
• The updates of the Landmarks are meant to establish a lasting dialogue between ESFRI and the ESFRI Landmarks – and their
research communities – and with the involved countries.
• A non-bureaucratic, evidence based, periodic update of Landmarks will reinforce the strategic role and position of ESFRI in
supporting these RIs when referring to the EU Council, as well as in allowing for the maintenance of the Landmarks’ list.
PERIODIC UPDATES OF LANDMARKS TOWARDS A FEW GENERIC KEY PERFORMANCE INDICATORS
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
To help address the unique aspects of each RI, Landmarks will develop specific KPIs and report on the few generic ones chosen
by ESFRI, therefore updating their profile and capturing their own peculiarities.
Avoiding one-size-fits-all shortcomings, indicative elements for developing flexible KPIs include:
• scientific excellence outcomes, output and delivery of talent;
• reference role in the disciplinary field (uniqueness in capabilities, in capacity) at European and international level;
• progress in achieving the RI´s milestones along its lifecycle;
• impact, innovation, entrepreneurship;
• establishment and development of the RIs’ user community;
• scientific data management policy, metadata catalogue interfacing
• EOSC and similar open science initiatives, advanced data services for scientific analysis and for innovation developments;
• enforcement of quality control of access (peer review), data (FAIR and reproducible), and services to research and
innovation.
MONITORING MANDATE METHODS AND USE OF KEY PERFORMANCE INDICATORS
ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano
GIORGIO ROSSI
• European RIs, not developed within ESFRI, have often a long record of excellent performance and constitute
the majority of operational RIs in Europe. They are usually assessed by the OWNERS, often through
international ad-hoc committees;
• National Roadmaps are often based on national assessment/monitoring of performance. Good practices must
be captured and understood in order to develop a model that integrates and generalize them
• KPIs must help the monitoring, but SHOULD NOT become a SHORTCUT METROLOGY;
• The WG-MONITORING will work on it and report to ESFRI next summer;
• ESFRI will prepare a “PAPER ON MONITORING” by the end of 2019
Let’s have a fruitful ESFRI WORKSHOP on MONITORING OF RIs,
PERIODIC UPDATE OF LANDMARKS, USE OF KPIs
19-20 NOVEMBER 2018 Palazzo delle Stelline - Milano