horizon 2020 support to smart, green and integrated...

Horizon 2020 support

to Smart, Green and

Integrated transport

Studies and reports

EUROPEAN COMMISSION

Directorate-General for Research and Innovation Directorate H — Transport Unit H.1 — Transport Strategy

Contact: Roberta Zobbi

E-mail: [email protected]

European Commission B-1049 Brussels

mailto:[email protected]

EUROPEAN COMMISSION

Horizon 2020 support to

Smart, Green and Integrated

transport

Studies and reports

Directorate-General for Research and Innovation

2017 Smart, Green and Integrated Transport

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the

authors, and the Commission cannot be held responsible for any use which may be made of the information

contained therein.

More information on the European Union is available on the internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2016.

PDF ISBN 978-92-79-71792-5 doi: 10.2777/274467 KI-01-17-922-EN-N

EPUB ISBN 978-92-79-71792-5 doi: 10.2777/274467 KI-01-17-922-EN-N

© European Union, 2017.

Reproduction is authorised provided the source is acknowledged.

Cover images: © Lonely, # 46246900, 2011. © ag visuell #16440826, 2011. © Sean Gladwell #6018533,

2011. © LwRedStorm, #3348265. 2011. © kras99, #43746830, 2012. Source: Fotolia.com

EUROPE DIRECT is a service to help you find answers to your questions about the European Union

Freephone number (*): 00 800 6 7 8 9 10 11

(*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you)

http://europa.eu/

Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 5

PREFACE Horizon 2020 is the European Union’s largest Research and Innovation programme for the period

2014-2020 and the Transport Challenge is allocated a budget of €6339 million.

The objective of the interim evaluation is to assess the overarching technical, political and

operational objectives of the programme by looking at the achievements of Horizon 2020 and the

Societal Challenged 4 (SC4) in terms of relevance; effectiveness; coherence (internal and external);

efficiency and EU added-value.

The approach followed takes into account the increased emphasis on outcomes (i.e. the impacts for

stakeholders, expected and unexpected) rather than on outputs from research projects and

programmes. It has also looked at both FP7 and Horizon 2020 as there is an overlap between the

project completion of the former and latter. At the time of this interim evaluation most of the

Horizon 2020 projects are in the initial phases of development and none have been finished.

The methodology for this evaluation combines quantitative and qualitative data, analysis and

evidence validated with relevant stakeholders. The methodology used consists of:

- Document and statistical analysis;

- Internet based surveys for different target groups, but mainly the respondents were project

coordinators;

- Questionnaire based individual and group interviews with selected interested parties;

- Informal exchanges and information gathering; and

- Analysis of available EC data (partly through data and text mining).

The Expert Group made an effort to verify all information received from individuals1 against different

sources and where possible and appropriate to draw its own conclusions as experts. Care was taken

to ensure that the sampling was representational (across modes, instruments and

players/stakeholders and also in terms of member states, types of projects and programmes). To

gain additional insights, the group performed individual and group interviews. Where indicated that

a view is ‘shared’, this means that this view was mentioned in the survey and by those that have

been interviewed. When ‘many’ are referred to it means that the comment has been supported by a

number of survey respondents, interviews and by the Expert Group. Where the comment or view has

only been put forward by one or two interviewees the comment or view has been triangulated

against other evidence or included because this view is shared by the majority of the Expert Group.

Single or unsupported views have not been included in this report. The representativeness of

opinions and conclusions was further assured by the fact that interviewees and participants in the

stakeholder hearing were invariably high level representatives of relevant organisations or

committees. They could be trusted to speak with care and with due regard for their affiliation

although they had the liberty to vent personal opinions as well. In most cases the interviewees

explicitly mentioned it if they were speaking in a personal capacity. The online survey among project

coordinators, though yielding less response than was hoped for, was considered by the Expert Group

1 Sources were promised confidentiality both individually and for their affiliated institutions.


to be representative in numbers. The response to another survey among Programme Committee

members was so low that it was not considered representative and was therefore discarded.

The Expert Group realises that a number of opinions and recommendations may give food for further

investigation. Due to time constraints the Group was not able to pursue in detail all potentially

interesting remarks, especially in those cases that interviewees came up with spontaneous

comments that could not be checked against other material. The Expert Group took on those

comments at its discretion only if they considered them plausible enough and worthwhile

mentioning.

The Expert Group also puts forward a set of recommendations that can be considered as lessons

learnt and suggestions for consideration or improvement in the new round of research activities for

transport (FP9).

The report has been prepared by the following independent experts Joris Al, Heather Allen, Griet De

Ceuster, Ivonne Herrera and Jesús Monclús as part of an interim evaluation of the Societal Challenge

4 (Smart, Green and Integrated Transport) for the European Commission and not for general

publication. The information comes from a number of sources including an Internet based

questionnaire, interviews with key stakeholders, a stakeholder hearing and the collecting of data

(some of which were provided by the Commission). Comments from these sources are confidential

and stakeholder views were only considered when they were also shared by the Expert Group and

validated by other evidence to support this opinion. It is indicated in the text when this is not the

case.


THE EXPERT GROUP The report has been prepared by the following independent experts Joris Al, Heather Allen, Griet De

Ceuster, Ivonne Herrera and Jesús Monclús.

Joris Al (NL) is chair of the Expert Group. He was a civil servant with several ministries in the

Netherlands and later became a consultant several years before he retired in 2014. During the last

10 years of his career, he was Director of Road and Navigation Research and subsequently Director

for International Affairs with Rijkswaterstaat. He was president of FEHRL from 2010 to 2014 and

chaired the Executive Board of CEDR in 2013.

Heather Allen (UK) is an international expert on sustainable urban transport, climate change and

gender. She developed the Sustainable Development Charter for UITP (the International Association

of Public Transport) and their diversity programme and also led the Sustainable Transport Team at

one of UK’s leading research centres, Transport Research Laboratory (TRL).

Griet De Ceuster (BE) is general manager of Transport & Mobility Leuven. She is involved in long term

forecasts, policy evaluations and indicators for sustainable mobility. She is leading transport- and

emission modelling studies across Europe. She was involved in several research programmes on road

congestion indicators, long term transport prognoses, pricing issues, external costs and transport

technologies. Griet De Ceuster holds a MSc in traffic engineering and studied economy at the KU

Leuven and the TU Delft.

Ivonne Herrera (NO) has degrees in Electrical Engineering, a Master in Aeronautical Maintenance

and Production and a PhD in Safety Management and Resilience Engineering. She has more than 20

years of experience in the industry and research on avionics engineering, maintenance, safety and

risk analysis for aviation and petroleum industries. She is currently Senior Scientist in the area of

Information and Communication Technology and Adjunct Associate Professor teaching Master

courses addressing risk governance, safety management and resilience.

Jesús Monclús (ES) holds a PhD in Mechanical Engineering and a MS in Transportation Safety. He has

experience as National Contact Point and National Delegate for the FP7 Transport Programme

Committee and for the Clean Sky 1 National Representatives Group. He is now professionally

involved in injury prevention programmes, including road safety, at international level.


TABLE OF CONTENTS PREFACE ................................................................................................................................................... 5

THE EXPERT GROUP ................................................................................................................................. 7

TABLE OF CONTENTS ............................................................................................................................... 8

GLOSSARY .............................................................................................................................................. 11

1 EXECUTIVE SUMMARY .................................................................................................................. 17

2 INTRODUCTION ............................................................................................................................. 22

2.1 Context ................................................................................................................................... 22

2.2 Objectives and intervention logic .......................................................................................... 23

2.2.1 Objectives ...................................................................................................................... 23

2.2.2 Interim Evaluation.......................................................................................................... 26

2.2.3 Intervention logic ........................................................................................................... 27

3 IMPLEMENTATION STATE OF PLAY [Key data provided centrally by Commission to Expert group]

32

4 RELEVANCE .................................................................................................................................... 33

4.1 Is Horizon 2020 Transport tackling the right issues? ............................................................. 33

4.1.1 Relevance to the situation in Europe when Horizon 2020 Transport was designed and

how it is addressing Europe’s challenges ...................................................................................... 34

4.1.2 Areas of relevance of specific European interest .......................................................... 40

4.1.3 Horizon 2020 Transport and its international outreach ................................................ 43

4.2 Addressing specific stakeholder needs, including emerging issues ...................................... 45

4.2.1 Suggestions for improvements from a stakeholder perspective ................................... 51

4.3 Other issues relevant to Relevance ....................................................................................... 52

4.4 Lessons learnt and areas of improvement ............................................................................ 53

5 EFFECTIVENESS .............................................................................................................................. 56

5.1 Short-term outputs from the programme and early success stories .................................... 56

5.1.1 Success stories ............................................................................................................... 56

5.1.2 TRL levels........................................................................................................................ 57

5.1.3 Human capital development ......................................................................................... 58

5.1.4 Knowledge transfer ........................................................................................................ 59

5.1.5 Building research, development and innovation capabilities ........................................ 59

5.1.6 Competitiveness & exploitation .................................................................................... 60

5.2 Expected longer-term results from the programme ............................................................. 61

5.3 Progress towards attaining the specific objectives of SC4 - Smart, green and integrated

transports .......................................................................................................................................... 62


5.4 Progress towards the overall Horizon 2020 objectives ......................................................... 64

5.5 Lessons learnt/ Areas for improvement ................................................................................ 64

6 EFFICIENCY .................................................................................................................................... 65

6.1 Introduction ........................................................................................................................... 65

6.2 Overview of funding allocations ............................................................................................ 65

6.2.1 Budget of the overall programme ................................................................................. 65

6.2.2 Budget per activity area ................................................................................................. 66

6.2.3 Balance between instruments and funding schemes .................................................... 67

6.2.4 Description of topics ...................................................................................................... 69

6.2.5 Projects have equal size ................................................................................................. 70

6.2.6 Additional remarks on topic definition .......................................................................... 71

6.3 Programme's attractiveness .................................................................................................. 73

6.3.1 Analysis of the Participants ............................................................................................ 73

6.3.2 Oversubscription on some topics .................................................................................. 74

6.3.3 Adequacy of funding rules ............................................................................................. 79

6.3.4 The cost of submitting a proposal ................................................................................. 80

6.3.5 Two-stage approach ...................................................................................................... 81

6.4 Cost-benefit analysis .............................................................................................................. 82

6.5 Other issues related to efficiency .......................................................................................... 83

6.5.1 Evaluation process & selection criteria .......................................................................... 83

6.5.2 Length of process ........................................................................................................... 84

6.5.3 Administrative burden ................................................................................................... 85

6.6 Lessons learnt/ Areas for improvement ................................................................................ 86

7 COHERENCE ................................................................................................................................... 87

7.1 Internal coherence ................................................................................................................. 87

7.1.1 Internal coherence of instruments to implement Horizon 2020 Transport .................. 87

7.1.2 Internal coherence with other Horizon 2020 intervention areas .................................. 90

7.1.3 Ensuring that every euro spent counts twice ................................................................ 95

7.2 External coherence ................................................................................................................ 97

7.2.1 Coherence with other EU funding programmes ............................................................ 97

7.2.2 Coherence with other public support initiatives at regional, national and international

level 102

7.3 Lessons learnt/Areas for improvement ............................................................................... 104

8 EU ADDED VALUE ........................................................................................................................ 106

8.1 Understanding added value ................................................................................................. 106


8.2 Horizon 2020 transport programme demonstrating EU Added Value ................................ 106

8.2.1 European integration ................................................................................................... 106

8.2.2 Increasing European competitiveness ......................................................................... 108

8.2.3 Europe’s role in standardisation and harmonisation .................................................. 109

8.2.4 Building stronger Europe-wide research communities and level of expertise ............ 110

8.2.5 Delivering research that feeds into European policy objectives ................................. 112

8.2.6 Linking calls and topics more directly with policy expectations .................................. 112

8.3 Success of different instruments to deliver EU Added Value .............................................. 113

8.3.1 ERA-NET ....................................................................................................................... 113

8.3.2 CEF ............................................................................................................................... 113

8.3.3 PPPs and JUs ................................................................................................................ 113

8.4 Other issues related to EU Added value .............................................................................. 115

8.5 Lessons learnt and areas for improvement ......................................................................... 117

8.5.1 Increased integration ................................................................................................... 117

8.5.2 Formal evaluation and follow-up of projects .............................................................. 118

8.5.3 Future research directions organizing for innovation ................................................. 119

8.5.4 Dissemination and exploitation aspects ...................................................................... 119

9 SUCCESS STORIES FROM PREVIOUS FRAMEWORK PROGRAMMES ........................................... 123

10 LESSONS LEARNT / CONCLUSIONS .............................................................................................. 124

10.1 Relevance ............................................................................................................................. 124

10.2 Effectiveness ........................................................................................................................ 128

10.3 Efficiency .............................................................................................................................. 129

10.4 Coherence ............................................................................................................................ 130

10.4.1 Internal coherence ....................................................................................................... 130

10.4.2 External Coherence ...................................................................................................... 132

10.5 Added value ......................................................................................................................... 134

10.6 Conclusions and recommendations ..................................................................................... 136

LITERATURE .......................................................................................................................................... 140

ANNEX: SURVEY QUESTIONS FOR PROJECT COORDINATORS ............................................................. 142

ANNEX: INTERVIEW QUESTIONS FOR STAKEHOLDERS ........................................................................ 157


GLOSSARY € euro

3D 3 dimensions

5G 5th generation of mobile communications

ACARE Advisory Council for Aviation Research and Innovation in Europe

ACEA Association des Constructeurs Européens d’ Automobiles

ACEM Association des Constructeurs Européens de Motocycles

ALICE Alliance for Logistics Innovation through Collaboration in Europe

ARF Access to Risk Finance

ART Automated Road Transport

ATM Air Traffic Management

BBI Bio-Based Industries, a Joint Undertaking

CARE Community database on road accidents

CEF Connecting Europe Facility

C-ITS Collaborative ITS (intelligent transport systems)

CIVITAS City, Vitality and Sustainability, a European Initiative launched in FP7 to

promote sustainable transport in cities

CO2 Carbon dioxide

COM Commission européenne / European Commission

COP 21 Climate Change conference 21st

CORDA data COmmon Research DAta Warehouse

CP Collaborative projects, a funding scheme in FP7

cPPP Contractual public-private partnership

CS Clean Sky, a Joint Undertaking

CS2 Clean Sky 2, a Joint Undertaking

CSA Coordination and Support Action, a funding scheme in Horizon 2020

CSJU Clean Sky Joint Undertaking

CSO Civil Society Organisation


DG Directorate General

DG MOVE Directorate General Mobility and Transport

DG RTD Directorate General Research and Innovation

EAEC European Atomic Energy Community

EBSF European Bus System of the Future

EC European Commission

eCALL Emergency Call

ECSEL Electronic Components and Systems for European Leadership, a Joint

Undertaking

ECTP European Construction Technology Platform

ECTRI European Conference of Transport Research Institutes

EE Energy efficiency

EFSI European Fund for Strategic Investment

EG Expert Group

EGVI European Green Vehicles Initiative

EGVIA European Green Vehicles Initiative Association

EIB European Investment Bank

EIP European Innovation Partnership

EIT European Institute of Innovation & Technology

ELENA European Local ENergy Assistance, a fund of the EIB

ERA European Research Area

ERA-NET European Research Area Network

ERC European Research Council

ERDF European Regional Development Fund

ERRAC European Rail Research Advisory Council

ERTICO European Road Traffic Information Company

ERTMS European Rail Traffic Management System

ERTRAC European Road Transport Research Advisory Council


ESFRI European Strategy Forum on Research Infrastructures

ESIF European Structural and Investment Funds, a fund of the EIB

ETP European Technology Platform

ETRA Spanish traffic engineering company www.etra.es

EU European Union

EU13 European Union, 13 countries joining the European Union in 2004, 2007 and

2013 (Bulgaria, Croatia, Cyprus, Czech Republic, Estonia, Hungary, Latvia,

Lithuania, Malta, Poland, Romania, Slovakia, Slovenia)

EU28 European Union, all 28 countries that are members of the European Union

Euratom European Atomic Energy Community

EURICS Energy Union Integrated Strategy on Research, Innovation and

Competitiveness

EV Electric vehicle

ExGr Expert Group

FCH2 Fuel Cells and Hydrogen 2, a Joint Undertaking

FEHRL Forum of European Highway Research Laboratories

FET Future and Emerging Technologies

FIA Fédération Internationale de l'Automobile

FIM Fédération Internationale de Motocyclisme

FP7 7th Framework Programme

FP9 9th Framework Programme

FTI Fast Track to Innovation

GDP Gross Domestic Product

GHG Greenhouse gas

GNSS Global Navigation Satellite System

GV Green Vehicles

IA Innovation Action, a funding scheme in Horizon 2020

ICT Information and Communications Technologies

IMG Industry Manufacturing Group

http://www.etra.es/


IMO International Maritime Organisation

INEA Innovation and Networks Executive Agency

Io(T)T Internet of (Transport) Things

IoT Internet of Things

ITS Intelligent Transport System

JTI Joint Technology Initiative

JU Joint Undertaking

KIC Knowledge and Innovation Community

KPI Key performance indicator

LEIT Leadership in Enabling and Industrial Technologies

LIFE EU’s funding instrument for the environment and climate action

LNG Liquefied natural gas

LRIT Long-Range Identification and Tracking

M€ Millions of Euros

Marco Polo Logistics efficiency funding programme during FP7

MG Mobility and Growth

MIT Massachusetts Institute of Technology

MoU Memorandum of understanding

MS Member State

MSCA Marie Sklodowska-Curie Actions

n Number

NCP National Contact Point

NMBP Nanotechnologies, Advanced Materials, Biotechnology and Production

NMP Nanotechnologies, Advanced Materials and Advanced Manufacturing and

Processing

P2P Peer to peer

PDA Project Development Assistance, part of the Energy programme

POLIS Network of European cities and regions working together to develop

innovative technologies and policies for local transport


PPI Public Procurement for Innovative Solutions

PPIS Public Procurement in Innovative Services

PPP Public-private partnership

R&D Research and development

RDI Research, development and innovation

RI Research Infrastructures

RIA Research and Innovation Action, a funding scheme in Horizon 2020

RIS River Information System

RIS3 Regional Research and Innovation Strategies for Smart Specialisation

RPO Research performing organisations

RRI Responsible Research and Innovation

S2R Shift2Rail

SC1 Horizon 2020’s Societal Challenge 1, Health, Demographic Change and

Wellbeing

SC2 Horizon 2020’s Societal Challenge 2, Food security, sustainable agriculture and

forestry, marine, maritime and inland water research and the bioeconomy

SC3 Horizon 2020’s Societal Challenge 3, Secure, clean and efficient energy

SC4 Horizon 2020’s Societal Challenge 4, Smart, green and integrated transport

SC5 Horizon 2020’s Societal Challenge 5, Climate action, environment, resource

efficiency and raw materials

SC6 Horizon 2020’s Societal Challenge 6, Inclusive Societies

SC7 Horizon 2020’s Societal Challenge 7, Secure Societies

SDGs Sustainable Development Goals

SES Single European Sky

SESAR Single European Sky ATM Research, a Joint Undertaking

SEWP Spreading Excellence and Widening Participation

SME Small and medium enterprise

SRIA Strategic Research and Innovation Agenda of ACARE

SSN Safe Sea Net, vessel traffic monitoring and information system


STEER Intelligent Energy Europe, a funding programme during FP7

STRIA Strategic Transport Research and Innovation Agenda

SUMP Sustainable urban mobility plan

SWAFS Science with and for Society

TAG Transport Advisory Group

TEB Transit Elevated Bus

TEN-T Trans-European transport network

TP Technology platform

TRA Transport Research Arena Conference

TRIP Transport Research Innovation Portal, formerly known as the Transport

Research Knowledge Centre (TRKC)

TRIVALUE Ex-post evaluation of Transport Research and Innovation in the FP7

‘Cooperation’ Programme

TRKC Transport Research Knowledge Centre (currently known as Transport Research

Innovation Portal , TRIP)

TRL Technology Readiness Level

TSI Technical standard

UAV Unmanned aerial vehicle

Waterborne TP Waterborne Technology Platform

WP Work Programme


1 EXECUTIVE SUMMARY Horizon 2020, the European Union’s largest Research and Innovation programme for the period

2014-2020, was adopted in 2013 to help address the challenges Europe faces. It is built on three

main pillars: excellent in science, societal challenges and industrial leadership. These remain the core

values of Horizon 2020 today, although it is also recognised that some of the challenges may have

changed since its inception.

The Transport Challenge is allocated a budget of €6339 million for the period 2014-2020 and the

programme is structured into four broad lines of activities.

Horizon 2020 addresses seven societal challenges. The specific objective of the Societal Challenge 4

(SC4) ‘Smart, green and integrated transport’ is to boost the competitiveness of the European

transport industries and ‘to achieve a European transport system that is resource-efficient, climate-

and environmentally-friendly, safe and seamless for the benefit of all citizens, the economy and

society’.

This interim evaluation of the Smart, Green and Integrated Transport Societal Challenge of Horizon

2020 is part of the overall Horizon 2020 interim evaluation that is to be completed by the European

Commission by mid-2017. This report has been prepared during the period June – October 2016 by

an Expert Group (consisting of five members) as input to the EC internal evaluation of transport

research that will be conducted by DG RTD and DG MOVE.

The objective of the interim evaluation is to assess the overarching technical, political and

operational objectives of the programme by looking at the achievements of Horizon 2020 and the

Societal Challenged 4 (SC4) in terms of:

- Relevance;

- Effectiveness;

- Coherence (internal and external);

- Efficiency;

- EU added-value.

The approach followed by this evaluation differs from previous ones (such as the ‘Commitment and

Coherence, Ex Post evaluation of FP7’) as it takes into account the increased emphasis on outcomes

(i.e. the impacts for stakeholders, expected and unexpected) rather than on outputs from research

projects and programmes. It has also looked at both FP7 and Horizon 2020 as there is an overlap

between the project completion of the former and latter. At the time of this interim evaluation most

of the Horizon 2020 projects are in the initial phases of development.

It should be noted that this interim evaluation covers a period in time when the Commission

introduced a greater focus on bringing research findings closer to the market and during which the

expectation of the role of industry in policy delivery through participation in research has also


changed. Examples of these trends include the creation of Public Private Partnerships (PPP)2 and

Joint Undertakings (JU), the more open character of research calls, and the combination of the ERA-

NET models, supporting the shift in policy objectives. As the PPPs and JUs are subject to a separate

evaluation, the present evaluation does not encompass those evaluations in any great depth.

However, it does include them where they intersect with the objectives of this research programme

and SC4.

The methodology for this evaluation combines quantitative and qualitative data, analysis and

evidence validated with relevant stakeholders. The methodology used consists of:

- Document and statistical analysis;

- Internet based surveys for different target groups, but mainly the respondents were project

coordinators;

- Questionnaire based individual and group interviews with selected interested parties;

- Informal exchanges and information gathering; and

- Analysis of available EC data (partly through data and text mining)

The Expert Group made an effort to check all information received from individuals3 against different

sources and where possible and appropriate to draw its own conclusions as experts.

Care was taken to ensure that the sampling was representational (across modes, instruments and

players/stakeholders and also in terms of member states, types of projects and programmes). To

gain additional insights, the group performed individual and group interviews.

It should be noted that there were certain constraints in the preparation of this report, such as the

timing of the interim evaluation and the access to certain information such as data on the balance of

gender in projects. Also the analysis of newcomers has not been made in any great depth due to the

lack of information available.

The Expert Group looked at the relevance of Horizon 2020 to establish if it is tackling the right issues,

and pursuing the right (relevant) objectives and they investigated to what extent the programme will

yield relevant research results in terms of impact, deployment (especially looking at innovations) and

how the findings can respond to the stated societal challenge SC4 on smart, green and inclusive

transport.

Transport has a high impact and relevance not only on the daily lives of citizens and society but also

on economic growth and jobs (interpreting competitiveness in its widest sense). There are numerous

issues that are relevant both to transport and the specific objectives of Horizon 2020. The Expert

Group also looked at the extent that feedback loops can positively influence policy development at

European level.

It is considered that the overarching research objectives of the programme remain relevant although

it may be questioned if the total of contracted research will meet the ambitions stated in the societal

2The use of the term PPP in the text refers to the many different structures of public, private partnerships

rather than a legal entity except where more specifically we have used other terms such as Joint Undertaking

(JU) or Joint Technology Initiative (JTI) 3 Sources were promised confidentiality both individually and for their affiliated institutions.


challenges. The programme also remains extremely (and possibly increasingly) relevant in terms of

its international outreach and relevance. The topics of climate change, road safety and

standardisation were highlighted.

Effectiveness has been interpreted as ‘measurable impacts in relation to the overarching objectives

of Horizon 2020, SC4 and specifically to the project objectives’. The Expert Group also included

investigating the wider levels of effectiveness in terms of human capital development, knowledge

transfer (especially in respect to communication and dissemination of results) and, to some extent,

the take up of tools and recommendations by those not directly involved with projects. It would have

been ideal to associate these results with the predetermined objectives such as competitiveness, job

creation and societal impact but this went beyond the scope of the Terms of Reference for this

group.

A number of projects as yet unfinished are likely to exceed their initial Technology Readiness Levels

(TRLs). This is in line with the outcome and closeness to the market ambitions of Horizon 2020

(compared to FP7) and its objective to help research make the transition towards deployment and/or

further research.

The vast majority of stakeholders and participants agree that Horizon 2020 is a unique programme

and its outcomes would have been very difficult to obtain should it not have existed. It is likely that

the quality and quantity of outputs is expected to be similar for Horizon 2020 as for FP7. Some early

success stories were identified to illustrate a good (equivalent to FP7 or higher) level of effectiveness.

However, despite this and based on the data available, the number of knowledge transfer activities

are expected to be lower for Horizon 2020 compared to FP7.

The impact on research, development and innovation capabilities is expected to be similar for

Horizon 2020 compared to FP7. Almost all projects have undertaken deployment activities, or are

planning to do so, and direct contact with stakeholders is the most popular activity, in both in FP7

and Horizon 2020.

However, project coordinators state that they plan to undertake less activities overall in Horizon

2020 to help deploy the results than in FP7. The reasons are unclear, as one would expect a higher

number given the focus on more market-oriented research, and the Expert Group found this a

surprising result.

A number of concerns were raised in respect to the focus on the higher end TRLs as this was stifling,

rather than engendering innovation. Cost effective solutions in early stages of TRL readiness are seen

as being desirable additions to the overall work programme especially in respect to the inherent

inertia in the transport sector for change and the slow progress seen in many areas, especially in

decarbonisation and alternative fuels.

Horizon 2020 is expected by respondents to yield more long-term outcomes than FP7 but as few

projects are finished, it is not possible to draw this conclusion yet. Nonetheless Horizon 2020 is seen

to contribute to strategic cooperation in research going beyond FP7. In relation to SC4 concerns were

raised on the bias towards technology over socio-economic and behaviour research.

Efficiency was considered as the relationship between the resources used by an intervention and the

changes generated by the intervention (which may be positive or negative). The benefits of the

outputs and outcomes from the research should be achieved at a reasonable cost. It was noted that


there have been several improvements to the Horizon 2020 process compared to FP7, which are

appreciated by the participants: the current balance between instruments and funding is considered

good. The feedback that was received from the majority of stakeholders was that the overall budget

was large but still insufficient to reach the desired objectives. Also, in some areas the balance of

allocation of funds could be better. It was also recognised that there are no other European or

International research programmes that are better funded than Horizon 2020.

The management of the projects has been improved with the outsourcing of these activities to INEA.

However, the most debated areas were in relation to the call texts, the two-stage process and

scientific and innovation management. More transparency associated with definition in the calls,

more focus in topics and increasing the level of success of proposals, seen today as frustratingly low,

were widely requested. The two-stage process is considered to bring extra cost burdens to those

preparing the proposals and has not delivered substantial gains in shortening the time from call to

contract. It is also not certain that the new system is allowing researchers from all member states to

participate, as there is a growing bias for those who are already leaders in some areas to build strong

consortiums that may exclude smaller players who may have innovative ideas.

Internal and external coherence was thoroughly studied. Internal coherence looked at the coherence

of the programme and its instruments and, in particular, with regard to Public Private Partnerships

(PPPs), with other Societal Challenges and with other areas of Horizon 2020, (such as ERC, EIT-KICS).

External coherence looked at the relationships with other funding mechanisms of the EU (structural

funds, European Back for Investments) and with other national and regional funding programmes for

innovation.

Overall the structure and goals of the differing programmes and instruments are seen as being

coherent with the ambition of Horizon 2020 and the stated objectives of the Union for transport

research. However in some areas improvement in the implementation and how the instruments

work both individually, and especially as a complement to each other are suggested.

Coherence with other EU funding programmes is considered to be extremely important and

necessary to close the innovation divide between member states and increase lagging areas. Given

the importance of transport for economic growth and equity, it is clear that research and innovation

should be well connected with education and training employment policies.

From the Expert Group’s outreach and analysis there is widespread and enthusiastic appreciation of

the EU added value to transport research under this programme. This is not unique to Horizon 2020

as it was also felt for the previous Framework Programmes. Based on interview material and survey

results (PPMI survey and experts survey) overall Horizon 2020 plays a useful and important role in EU

research, boosting the necessity and strategic coordination of the research process.

The collaborative nature of Horizon 2020 is also thought to deliver more than the sum of fragmented

national programmes. It delivers clear European added value contributing to the excellence of

science, enhancement of scientific reputation and improving scientific quality. Areas where Europe is

perceived to be in a leadership position, e.g. in aviation or automotive, should continue to receive

support to keep in front as competition from outside Europe e.g. China, Korea or USA is strong.

Demonstrator projects undertaken by multi-players and pilot projects with various countries

participating are seen as adding particularly high value as, in the majority of cases, it would not be

possible to conduct them at national level.


There are important areas that require attention such as ensuring that project results and European

research knowledge is maximised. The number of outputs in terms of deliverables from FP7 and

Horizon 2020 is large but it is not easy for outsiders to be able to use this information or for those

who have not worked on the projects to retrieve it – a case of quantity over quality.

Overall from the information gathered by the Expert group, there is enthusiastic and broad support

for Horizon 2020 and for EU research programmes in general. This comes from all stakeholders,

including industry (private sector), member state governments, civil society and the research

community. Recognising that transport is unique in terms of its societal value, economic and

environmental impacts and the need for coordination of research in this field, there is definite

support for a new Framework Programme post Horizon 2020.

Looking more closely at the 5 criteria set out by the Commission to frame our work, and considering

the Societal Challenge 4 as requested, the Expert Group also puts forward a set of recommendations

that can be considered as lessons learnt and suggestions for consideration or improvement in the

new round of research activities for transport.


2 INTRODUCTION

2.1 CONTEXT

Horizon 2020, the European Union’s largest Research and Innovation programme was adopted in

2013 to help address the challenges Europe faces. It is built on three main pillars: excellence in

science, societal challenges and industrial leadership. These remain the core values of Horizon 2020

today although it is also recognised that some of the challenges may have changed since its

inception.

The EU 2020 strategy presented innovation and research as a key element in order to kick-start

economic growth and stressed the positive relation at an aggregate level between investment in

research and development (R&D) and socio-economic growth. It also highlights the positive relation

between such investments and productivity. Transport research, therefore, is a fundamental

constituent of the EU’s efforts to create and sustain economic and social growth, and this has rightly

been reflected by its overall position and funding within the Horizon 2020 Programme.

Transport is one of the seven “societal challenges” that form the basis of European policymaking

since 2009 and have become an established theme for European Research. Transforming transport

from being 95% dependent on fossil fuel, reducing its contributing of some 25% of Europe’s

Greenhouse Gas emissions (GHG) and causing 100,000 premature deaths from local air pollution into

a ‘Smart, Green and Integrated’ system is seen as being the key objective for Horizon 2020 transport

research actions4. To meet this the Union has allocated significant budget to transport research

(€938 million for the period 2016-2017 and overall earmarking nearly 8% of the Horizon 2020 budget

over the 7 years of the programme up to 2020). Within these seven challenges there are four that

can be considered to be of transversal interest (cross-cutting) with transport. These include “Secure,

clean efficient Energy; Climate action, environment, resource efficiency and raw materials; Europe in

a changing world – inclusive, innovative and reflective societies and Secure societies – protecting

freedom and security of Europe and its citizens.

It is also recognised that Horizon 2020, as an essential part of Europe’s research strategy, should

contribute to the overarching objectives and European ambitions of open innovation, open science

and being open to the world. Funding for research and innovation also aims to make Europe’s

transport system sustainable, seamless, competitive and responsive.

The period of 2012-2016 saw a number of global developments that have impacted the policy and

research landscapes. These include a number of new, important international political agreements

such as texts of the Sustainable Development Goals and the landmark Paris Climate Change

Agreement as well as an increase in the threat of terrorism, with major incidents occurring in several

member states, an unforeseen influx of migrants from a number of conflict and disadvantaged

regions and huge step-change progress in some areas of technological developments (examples

include 3D printing, smart phone applications and bottom up citizen centred innovative solutions

such as Uber).

4 Halving road fatalities by 2020 is also seen as being a key transport objective.


2.2 OBJECTIVES AND INTERVENTION LOGIC

2.2.1 OBJECTIVES

Horizon 2020 is the European Union's Research and Innovation Framework Programme for the

period 2014-2020 that is designed to help tackle current challenges by providing excellence in

science and cutting edge innovation. Horizon 2020 has been designed to address a number of critical

weaknesses in its science and innovation system (especially those identified in the Ex-ante Impact

Assessment of Horizon 20205). Europe had a number of early and promising leads in many key

technology areas, but in the face of growing global competition this advantage has not translated

into the strong innovative and competitive leadership positioning in line with this potential.

The EU regulation No 1291/2013 of the European Parliament and Council of 11 December 2013

states that Horizon 2020 helps to deliver the Union's objectives:

- to strengthen its scientific and technological basis by achieving a European Research Area

(ERA) in which researchers, scientific knowledge and technology circulate freely;

- to encourage the Union to advance towards a knowledge society; and

- to become a more competitive and sustainable economy in respect of its industry and to

ensure that the conditions necessary for the competitiveness of Union industry exist.

As stated in the Note on the Horizon 2020 Monitoring Report 20146 when quoting the EC’s

Communication Europe 20207. “Investment in research and innovation is essential for Europe’s

future and it is at the heart of the Europe 2020 strategy for smart, sustainable and inclusive growth

and of the priorities of the Juncker Commission”. Horizon 2020 is meant to contribute to building a

society and an economy based on knowledge and innovation in all member states of the European

Union through cutting edge research and development and by leveraging additional research,

development and innovation funding and by contributing to attain research and development

targets, including the target of 3% of GDP for research and development across the Union by 2020.

Preface to Open Innovation, Open Science, Open to the World

The words of Jean-Claude Juncker, President of the European Commission, could not be clearer in his

preface to the “Open Innovation, Open Science, Open to the World – a vision for Europe” booklet8:

“Most of the political priorities set for my mandate as President of the European Commission depend

to a greater or lesser extent on research and innovation. Research and innovation create investment

opportunities for new and better products and services and therefore increase competitiveness and

employment”.

5 https://ec.europa.eu/research/industrial_technologies/pdf/Horizon-2020-ex-ante-impact-assessement-

nmp_en.pdf 6 http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108

7 Communication from the Commission, Europe 2020: A strategy for smart, sustainable and inclusive growth,

COM(2010) 2020 final. 8 https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world-vision-

europe

https://ec.europa.eu/research/industrial_technologies/pdf/Horizon-2020-ex-ante-impact-assessement-nmp_en.pdf

https://ec.europa.eu/research/industrial_technologies/pdf/Horizon-2020-ex-ante-impact-assessement-nmp_en.pdf

http://ec.europa.eu/newsroom/horizon2020/document.cfm?doc_id=15108

https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world-vision-europe



Horizon 2020 supports the implementation of the Europe 2020 strategy, which sets out the Union’s

objectives of smart, sustainable and inclusive growth, highlighting the role of research and

innovation as key drivers of social and economic prosperity, environmental sustainability and other

Union policies9. In addition it sets itself ‘the goal of increasing spending on research and development

in order to attract private investment of up to two thirds of total investments, thereby reaching an

accumulative total of 3 % of gross domestic product (GDP) by 2020 while developing an innovation

intensity indicator’.

Horizon 2020 is organised into three pillars and subdivided into seven major societal challenges to

address Europe’s main challenges. The Transport Challenge is allocated a budget of €6339 million for

the period 2014-2020 and the programme is structured in four broad lines of activities aiming at:

- Resource efficient transport that respects the environment;

- Better mobility, less congestion, more safety and security;

- Global leadership for the European transport industry; and

- Socio-economic and behavioural research and forward-looking activities for policy making.

The specific objective of the Societal Challenge 4 ‘Smart, green and integrated transport’, and the

objective of this interim evaluation, is to boost the competitiveness of the European transport

industries and ‘to achieve a European transport system that is resource-efficient, climate- and

environmentally-friendly, safe and seamless for the benefit of all citizens, the economy and society’.

Horizon 2020 is also expected to contribute to the Juncker Commission’s Agenda for Jobs, Growth,

Fairness and Democratic Change (2014-2019), focused on ten priorities of strengthening Europe’s

competitiveness and stimulating investment for the purpose of job creation. In his mission letter10 to

the Commissioner for Research, Science and Innovation he states that the Commission as a whole

needs to aim to be more than the sum of its parts, by working together in strong teams in new

collaborative ways, ‘cooperating across portfolios to produce integrated, well-grounded and well

explained initiatives that lead to clear results’ with a focus on ensuring effective implementation and

follow-up on the ground.

Commissioner Moedas’ three goals or strategic priorities for EU research and innovation policy11 are:

- Open Innovation: more actors, more investment (more venture capital and a European Fund

of Funds, real synergies with the Structural Funds via a “Seal of Excellence”), better

regulatory environment for innovation and a new European Innovation Council

- Open Science, including a European Research Integrity Initiative

9 Research and innovation are also key factors for other flagship initiatives of the Europe 2020 strategy, notably

'Innovation Union', 'Resource-efficient Europe', 'An industrial policy for the globalisation era', and 'Digital

Agenda for Europe', and other policy objectives, such as climate and energy policy. Moreover, the objectives of

the Europe 2020 strategy relating to research and innovation include cohesion policy, which has a key role to

play through capacity-building and providing a stairway to excellence. 10

http://ec.europa.eu/commission/sites/cwt/files/commissioner_mission_letters/moedas_en.pdf 2014 11

Open Innovation, Open Science, Open to the World – a vision for Europe. Directorate-General for Research

and Innovation. ISBN 978-92-79-57346-0 - doi:10.2777/061652 - KI-04-16-263-EN-N.


http://ec.europa.eu/commission/sites/cwt/files/commissioner_mission_letters/moedas_en.pdf%202014



- Open to the World, with more engagement in science diplomacy and global scientific

collaboration and leading a global research area

Commissioner Moedas12 also specifically mentions three major challenges for research, which are:

- EU research rarely succeeds in getting research results to market. Technologies developed in

Europe are most of the time commercialised elsewhere.

- In some areas Europe falls behind on the very best science, although it generates more

scientific output than any other region in the world. At the same time, every part of the

scientific method is becoming an open, collaborative and participative process.

- Europe punches below its weight in international science and science diplomacy.

In the recent Commission Staff Working Document from July 2016 on the implementation status of

the European Transport White Paper13, the Commission states that five of the ten President’s

priorities are particularly relevant for transport: (1) a new boost for jobs, growth, investment, (2) a

deeper and fairer internal market with a strengthened industrial base, (3) a resilient energy union

with a forward looking climate change policy, (4) a connected digital agenda and (5) the EU as

stronger global actor. The Commission also stresses that these strategies encompass various policy

areas and reflect a more cooperative and Horizontal approach of the Commission to addressing main

challenges, and that all five priorities serve as a new impetus for various transport policy initiatives.

To a large extent, it is evident that President Juncker’s priorities concentrate on taking Europe out of

its current economic crisis. Horizon 2020 serves as a strong anchoring point for research and

innovation efforts in many countries and as a source of inspiration for optimising national and

regional research and innovation systems.

In addition to its contribution to the European Commission objectives, Horizon 2020 also aims at

addressing other issues raised in the framework of the evaluation of previous framework

programmes, notably FP714 and take up key lessons learned from these programmes.

Comparing Horizon 2020 objectives to the five FP7 objectives indicates the continuity from FP7 to

Horizon 2020:

1. Promoting excellence in research,

2. Fostering competitiveness and economic growth,

3. Contributing to solving social challenges,

4. Strengthening the human potential and researchers’ mobility,

5. Fostering transnational research cooperation.

How Horizon 2020, and to some extent FP7, is lining up with these goals are dealt with in this report,

especially under the topics of relevance and coherence. The general objectives of Horizon 2020

12

European Commission - Speech - Open Innovation, Open Science, Open to the World. 22 June 2015. Carlos

Moedas – Commissioner for Research, Science and Innovation. Brussels, ‘A new start for Europe: Opening up to

an ERA of Innovation’ Conference. http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm 13

http://ec.europa.eu/transport/themes/strategies/doc/2011_white_paper/swd(2016)226.pdf 14

Ex-post Evaluation FP7 and Horizon 2020 Ex-ante Impact Assessment

http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm

http://ec.europa.eu/transport/themes/strategies/doc/2011_white_paper/swd(2016)226.pdf


transport can easily be mapped against President Juncker’s top priorities15, showing their relevance.

In addition there is an increasing relevance for some international topics such as climate change and

topics that could help the Union and member states to address aspects of the Sustainable

Development Goals, a number of which are relevant in particular to the ambitions of SC4 – Smart,

Green and Integrated Transport. The main difference between the programme lies with the

objective of bringing the research findings closer to the market and how they may be exploited more

readily than those from FP7.

2.2.2 INTERIM EVALUATION

This interim evaluation of the Smart, Green and Integrated Transport Societal Challenge of Horizon

2020 is part of the overall Horizon 2020 interim evaluation that is to be completed by the European

Commission by mid 2017. This report has been prepared by an Expert Group (consisting of five

members16 as input to the EC internal evaluation of transport research that will be conducted by DG

RTD and DG MOVE.

The scope of the evaluation is defined in the Terms of Reference for this Expert Group and it covers

Societal Challenge 4 of Horizon 2020 activities from the start of the programme (2014) until

September 2016. In addition to the evaluation, the Expert Group was asked to provide

recommendations for the remainder of the Horizon 2020 Programme and future Framework

Programmes. These are provided at the end of this report.

The Horizon 2020 Smart, Green and Integrated Transport Societal Challenge Interim Evaluation

includes as far as possible, an assessment of the longer term impacts of previous Framework

Programmes (in particular FP7) and their contribution to the Europe 2020 Strategy. Therefore, it

considers the results of an additional ca.100 finished FP7 projects, corresponding to an additional

€300 million investment since the previous evaluation. Moreover, the present evaluation

methodology17 entails that the projects still under way have also been subjected to a scrutiny of their

prospective results (ca. 100 more projects, corresponding with ca. €650 million).

The approach followed by this evaluation differs from previous ones (such as the ‘Commitment and

Coherence, Ex Post evaluation of FP7’) as it takes into account the increased emphasis on outcomes

(i.e. the impacts for stakeholders, expected and unexpected) rather than on outputs from research

projects and programmes. The shift of focus to outcome and impact is considered as a major

transition and the increased budget for Horizon 2020 compared to FP7 is also noted as another key

difference.

The primary reference points are the policy objectives that Horizon 2020 aims at supporting in the

field of transport. These are:

- resource efficient transport that respects the environment, minimising transport's systems'

impact on climate and the environment (including noise and air pollution) by improving its

efficiency in the use of natural resources, and by reducing its dependence on fossil fuels.

15

http://ec.europa.eu/priorities/index_en 16

Due to unforeseen circumstances a fifth expert was added to the group in September 2016 17

In the specific survey set up for his evaluation, project managers of both finished and running projects were

targeted

http://ec.europa.eu/priorities/index_en


- better mobility, less congestion, more safety and security. The aim is to reconcile the growing

mobility needs for seamless transport, through innovative solutions for integrated, inclusive,

affordable, safe, secure and robust /resilient transport systems.

- reinforcing the global leadership, competitiveness and performance of European transport

manufacturing industries and related services including logistic processes and retain areas of

European leadership (such as aeronautics).

- socio-economic and behavioural research and forward looking activities for policymaking.

The aim is to support improved policy making which is necessary to promote innovation and

meet not only today’s but also future transport challenges and the societal needs related to

it.

In previous evaluations of FP7 and from earlier surveys (ex post evaluation FP7, TRIVALUE evaluation

of FP7, TAG report June 2016) a number of recommendations were made that Horizon 2020 should

address. The most important changes or improvements include:

- strengthen the relation between research and policies;

- create more conditions for the uptake of research results;

- improve measurement of impacts, with more emphasis on breakthrough innovations, and

consider how to increase competitiveness through research, international cooperation in

networks; and

- increase the engagement of civil society.

This interim evaluation has taken these into account where appropriate.

2.2.3 INTERVENTION LOGIC

The objective of the evaluation is to assess in relation to the overarching political and operational

objectives of the programme by looking at its achievements in terms of:

- Relevance;

- Effectiveness;

- Coherence (internal and external);

- Efficiency; and

- EU added value.

The five assessment aspects are the main criteria for the evaluation, determining whether the

Horizon 2020 SC4 is on track to deliver its objectives following the intervention logic, as designed by

the European Commission.


Figure 1

It should also be noted that this evaluation has taken place during a period when there has been a

shift in focus within the Commission (as referred to in section 1) and during which the expectation of

the role of industry and the private sector in policy and research has changed.

Examples include the creation of Public Private Partnerships (PPP) and Joint Undertakings (JU), the

more open character of research calls, the combination of the ERA-NET models, etc. These changes

are meant to reflect and support the shift of accent in the policy objectives. As the PPP’s and JU’s are

subject of a separate evaluation, the present evaluation does not encompass those evaluations in

any great depth, although it is recognised that their existence may well influence the working of

Horizon 2020. Relevant PPPs such as European Green Vehicle Initiative are included in this report.

Unless stated otherwise in the text, the generic term of Public Private Partnership (PPP) is used to

describe the various types of official PPPs and other public-private contracts between the EC and

third parties18. This interim evaluation has not gone into any great depth in its evaluation of PPPs as

this was out of the scope of this work but it has considered them when they have intersected with

the Horizon 2020 programme and especially in relation to aspects of SC4.

For the sake of this evaluation the terminology used in the intervention logic is defined as follows:

- Inputs: funds from FP7 (last part) and Horizon 2020 (until 2016 and excluding PPPs and JUs),

and the estimated leverage effect for national contributions and industry

- Activities: the FP7 projects completed after the ex post evaluation of FP7 (in lieu of Horizon

2020 projects being completed), the still running FP7 projects, the Horizon 2020 calls and

their selected projects until 2016

18

The term PPP in the text refers to many different structures of public-private partnerships rather than a legal

entity except where more specifically we have used other terms such as Joint Undertaking (JU) or Joint

Technology Initiative (JTI)


- Outputs: research reports, conferences, demonstration sites and pilot projects, articles in

scientific papers, patents etc. It must be noted that Horizon 2020 has not yielded many

actual (final) outputs yet.

- Results: proven and applied knowledge that is recognised as output such as described in the

achievements report, implemented research findings, working demonstrators etc. Again it

must be noted that Horizon 2020 has not yielded any direct results yet as research is still

going on for the first calls. Any changes from the expectations of FP7 results in respect to

Horizon 2020 (and SC4) results were noted where possible.

- Impacts: the mostly qualitative estimates of the effect of results (see achievements report),

or results expected on the basis of ex ante impact assessment of project proposals have been

considered.

Evaluation methodology

The methodology for this evaluation combines quantitative and qualitative aspects, complemented

by expert judgement based as much as possible on evidence and data. The data has been prepared

by the European Commission and supplied to the Expert Group, but it has not been extensively

analysed.

Figure 2: Methodology logic

The structure of the report follows the template provided by the Commission and the evaluation

concept in the following steps:

1. Inventory of the five criteria to establish the main challenges and questions per criterion.

2. Refine to identify key data and statistics, analyses and expert input.

3. Information gathering (document and data searches, surveys, interviews, stakeholder hearing,

examples of success stories)

4. Aggregate information

5. Conclusions and recommendations per criterion

6. Lessons learned and recommendations


With this in mind the Expert Group decided on an approach that focussed on evidence-based

information from multiple sources rather than statistical analysis. The methodology used consists of:

- document and statistical analysis (see literature list in annex),

- Internet based surveys for different target groups (see Annex for further information on the

questionnaire), of which the most response was received from project coordinators

- a number of semi-structured interviews with selected parties chosen to be representative for

the different subsectors of transport and covering the various profiles of participants,

- information gathering and analysis of available EC data (partly through data and text mining),

a hearing with interested parties and stakeholders who represented business and industry,

the research community, and governments (but not the Commission), and two formal

meetings with the Commission (one in September 2016 and a final one in October).

- informal exchanges,

Triangulation was used as rationale for using multiple qualitative and quantitative source of evidence

(Yin, 200919). The use of multiple source of evidence far exceeds other approaches limited to one

type of measurement. The most important advantage of triangulation is the development of

converging lines of inquiry. Thus, findings and conclusions are likely to be more convincing and

accurate if they are based on several sources of information.

To gain additional insights to the survey in terms of qualitative inputs, the Expert Group performed

individual and group interviews. These interviews were conducted using a list of questions related to

the each of the evaluation criteria, but the interviews mostly took place as a dialogue where the

interviewees had an opportunity to offer their own input on central topics. The profile of those

interviewed consists of a strategic selection of representatives are from different nationalities from

EC Officials, Programme Committee Members, Technology Platforms, Private Public Partnerships,

Joint Undertakings, Project Coordinators and other relevant stakeholders.

Overall a total of 24 people were interviewed during the period August – September 2016. In

general, each interview lasted 1.5 hours. As the interviewees had backgrounds from different

positions representing different key areas of transport, and they spoke on the behalf of the subsector

or stakeholder group that they represented. The results were considered by the Expert Group to be

very rich and informative, but due to time and resource limitations, the essential elements of these

interviews are presented in a condensed form. There has not been an opportunity to conduct an in-

depth analysis or for this to be entirely triangulated against all other sources.

Useful input was also collected from the Internet survey(s) and the stakeholder hearing. It was

nonetheless noticed that the survey was possibly the least effective method for data collection as

people are suffering from being over subscribed to take part in surveys. Despite this the Expert

Group feel it worthwhile to mention the high level of interest to take part when requesting

participation and that people were overall willing to take part and generous with their time.

However, despite several attempts to reach out to the rail sector the Expert Group did not get

spontaneous or enthusiastic responses to participate in this review. It is not entirely clear why this

was the case but therefore to some extent the comments collected are based on comments from

19

Ref: Yin, R.K. (2009). Case study research: design and methods. (4th

ed.) Thousand Oaks, CA: Sage


those outside of the rail sector. It is the opinion of the Expert Group from their own observations

that the isolation of the rail sector from other transport modes might be symptomatic for that sector.

The key areas of evaluation and analysis include the following:

Commission requested criteria:

- Relevance

- Coherence

- Effectiveness (including simplification as indicated in FP7 evaluation)

- Efficiency

- Added Value

Inward looking aspects:

- New collaborative ways of working

- Cooperation across portfolios

- Clarity of objectives of initiatives

- Results based outcomes (with policy feedbacks)

- Integration of modes and sectors

Outward looking aspects:

- Climate resilience and low carbon economy

- Job creation and competiveness

- Inclusion and equity (including safe and seamless)

- Open and excellence in Science

- Open Innovation

- Open to the World (including relevance to internationally)

The Expert Group would like to note that there were certain constraints in the preparation of this

report that has resulted in some areas not being able to be addressed in depth. This includes the

timing of the interim evaluation, which meant that no Horizon 2020 projects have actually been

completed and the limitations on scope and time of the contract. In addition some key areas of

information have not been able to be accessed in any depth such as the balance of gender in projects

and analysis of newcomers, due to the lack of information available. Therefore it has only been

possible to compare the procedures of allocation of grants, the call timings and text (terminology

used) between the two programmes as no impact of results of Horizon 2020 can be considered.

Despite this, it was interesting and useful to compare the two programmes, as there have been a

number of structural improvements made under Horizon 2020 and the Expert Group’s approach has

tried to capture bottom up experience that can be used to make further improvements and input

into the preparations of FP9.


3 IMPLEMENTATION STATE OF PLAY [KEY DATA

PROVIDED CENTRALLY BY COMMISSION TO EXPERT GROUP] PLEASE NOTE THAT THE INFORMATION FOR THIS CHAPTER IS TO BE PROVIDED BY THE COMMISSION

AND IS CONSTANTLY BEING UPDATED.

THE EXPERT GROUP HAS RECEIVED A NUMBER OF UPDATES (THE LAST ONE 10.10.2016) AND HAS

TAKEN THE RELEVANT INFORMATION FROM THIS INFORMATION INTO CONSIDERATION IN THEIR

EVALUATION. THE ‘CHAPTER’ IS PROVIDED IN AN ANNEX FOR REFERENCE PURPOSES


4 RELEVANCE As stated in the note on the Horizon 2020 Monitoring Report 201420 when quoting the EC’s

Communication Europe 202021: “Investment in research and innovation is essential for Europe’s

future and it is at the heart of the Europe 2020 strategy for smart, sustainable and inclusive growth

and of the priorities of the Juncker Commission”. This certainly applies to Horizon 2020 transport

research.

4.1 IS HORIZON 2020 TRANSPORT TACKLING THE RIGHT ISSUES?

Transport is unique in terms of its societal value and therefore relevance to SC4, as it touches

everyone’s daily lives and is a unique combination of:

- Its economic importance, not only for the number of jobs it offers in industries, companies

and services directly working in this sector but also because its services are required by all

other sectors (except for, up to some extent, digital virtual goods) to transport people and

goods facilitating trade between countries and regions.

- Its impact on the environment in terms of local and air quality and global pollution, energy

consumption and security, noise and land take.

- Its impact on the quality of life of every European citizen every single day: with trips to

schools, to work, for leisure or health reasons and for social contact but we also need to

recognise the negative impacts on society from traffic accidents, air pollution and the

severance of communities.

The following table summarizes the specific objectives of Horizon 2020 Transport challenge as well as

the four broad lines of activities of Horizon 2020 that provide the backbone of the work

programmes22.

Table 1: Horizon 2020 Transport specific objectives and lines of activities

Horizon 2020 Transport specific objectives Horizon 2020 Transport - Lines of activities

To boost competitiveness of the European

Transport industries

Global leadership for the European transport

industry

To achieve a European transport system that is

resource efficient, climate- and environmentally-

friendly

Resource efficient transport that respects the

environment

To achieve a European transport system that is

safe and seamless for the benefit of all citizens,

the economy and society

Better mobility, less congestion, more safety and

security

Socio-economic and behavioural research and

forward looking activities for policy making

20

http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108 21

Europe 2020: A strategy for smart, sustainable and inclusive growth, COM(2010) 2020 final. 22

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1291&from=EN. REGULATION (EU)

No 1291/2013 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 December 2013 establishing

Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020)


http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1291&from=EN


As the European Regulation establishing Horizon 2020 states23“investing in research and innovation

for a greener, smarter and fully integrated reliable transport system will make an important

contribution to the objectives of the Europe 2020 strategy and of its flagship initiative 'Innovation

Union'. Activities will support the implementation of the White Paper ‘Roadmap to a Single European

Transport Area - Towards a competitive and resource efficient transport system’. They will also

contribute to the policy goals outlined in the flagship initiatives 'Resource Efficient Europe', 'An

Industrial Policy for the Globalisation Era' and 'Digital Agenda for Europe' and interface with the

relevant Joint Programming Initiatives”.

In general, Horizon 2020, and especially the Transport Societal Challenge, is more oriented towards

solving the most pressing needs of society than its predecessor FP724, which was more focused on

science and technology (European Commission’s “Transport research and innovation achievements

report, 31 March 2016”25).

The Expert Group looked to establish if Horizon 2020 and the research calls of SC4 were tackling the

right issues, and if Horizon 2020 pursues the right (relevant) objectives. In addition, will the

programme yield relevant research results that will have the desired impact and to what extent is

innovation – in terms of potential and real impact on society, jobs and growing the economy –

treated? In other words, how does it contribute to responding to the stated societal challenges and

how can it help achieve policy objectives (in line with the requirement for it to become outcome

rather than output focussed).

4.1.1 RELEVANCE TO THE SITUATION IN EUROPE WHEN HORIZON 2020 TRANSPORT

WAS DESIGNED AND HOW IT IS ADDRESSING EUROPE’S CHALLENGES

The rationale and Union added value that represented the basis for the Horizon 2020 Transport, as

described in the European Regulation establishing Horizon 2020 can be summarized as follows:

- “European transport industry was facing increasingly fierce competition from other parts of

the world. Breakthrough technologies would be required to secure Europe's future

competitive edge and to mitigate the drawbacks of our current transport system.

- It was essential to reduce the environmental impact of Transport through targeted

technological improvement.

- Congestion was an increasing problem; systems were not yet sufficiently smart; alternative

options for shifting towards more sustainable modes of transport were not always attractive;

road fatalities remained dramatically high; citizens and businesses expected a transport

system that were accessible to all, safe and secure. The urban context posed specific

23

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1291&from=EN. REGULATION (EU)

No 1291/2013 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 December 2013 establishing

Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No

1982/2006/EC. 24

www.transport-research.info/programme/transport-including-aeronautics-Horizontal-activities-

implementation-transport-programme 25

Internal report of the European Commission made available to the Expert Group.

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1291&from=EN

http://www.transport-research.info/programme/transport-including-aeronautics-Horizontal-activities-implementation-transport-programme

http://www.transport-research.info/programme/transport-including-aeronautics-Horizontal-activities-implementation-transport-programme


challenges and provides opportunities to the sustainability of transport and for a better

quality of life.

- Within a few decades the expected growth rates of transport would drive European traffic

into a gridlock and make its economic costs and societal impact unbearable, with adverse

economic and societal repercussions.

The problems of pollution, congestion, safety and security were common throughout the Union

and called for collaborative Europe-wide responses… objectives could not be achieved through

fragmented national efforts alone… Emphasis would be placed on priority areas that match

European policy objectives where a critical mass of effort would be necessary, where Europe-

wide, interoperable or multimodal integrated transport solutions could help remove bottlenecks

in the transport system, or where pooling efforts transnationally and making better use of and

effectively disseminating existing research evidence could reduce research investment risks,

pioneer common standards and shorten time to market of research results.

Research and innovation activities should include a wide range of initiatives, including relevant

public-private partnership, that cover the full innovation chain and follow an integrated approach

to innovative transport solutions. Several activities were specifically intended to help bring results

to the market: a programmatic approach to research and innovation, demonstration projects,

market take-up actions and support for standardisation, regulation and innovative procurement

strategies all serve this goal. In addition, using stakeholders' engagement and expertise would

help bridge the gap between research results and their deployment in the transport sector.

Investing in research and innovation for a greener, smarter and fully integrated reliable transport

system would make an important contribution to the objectives of the Europe 2020 strategy and

of its flagship initiative 'Innovation Union'. The activities would support the implementation of the

White Paper "Roadmap to a Single European Transport Area - Towards a competitive and

resource efficient transport system". They would also contribute to the policy goals outlined in the

flagship initiatives 'Resource-efficient Europe', 'An Industrial Policy for the Globalisation Era' and

'Digital Agenda for Europe'. They would also interface with the relevant Joint Programming

Initiatives”.

In spite of a slower increasing pace of transport activities due to the long-lasting economic crisis and

some achievements in the reductions in traffic fatalities, the situation in Europe has evolved in the

direction of exacerbating the need for action as presented above: even fiercer international

competition, a larger share of anthropogenic impacts on environment due to transport, additional

transport services demand due to internet shopping, increased awareness of the impacts of

transport on health and social inclusion. The wording of the SC4 objectives is broad enough to cover

the additional challenges since the inception of Horizon 2020 (see table 6 in paragraph 4.2.)


Horizon 2020 Transport also continues to be clearly relevant for the following flagships initiatives of

the Europe 2020 strategy presenting in 201026:

- The Smart Growth goal: Digital Agenda for Europe, Innovation Union and Youth on the move

- The Sustainable Growth goal: Resource Efficient Europe and An industrial policy for the

globalisation era

- The Inclusive Growth goal: An agenda for new skills and jobs and a European platform

against poverty.

Transport is clearly a major factor in the first two initiatives linked to Smart Growth (digital

transformation of the transport sector and ‘Innovation Union’). It also plays a strong role in the

initiatives fostering inclusive growth, connected markets, logistics, PPPs (public-private partnerships)

and external policy instruments27. However, the connections between Horizon 2020 transport calls

and these flagships initiatives are not clear. Both in the hearing and in several interviews the cross-

cutting nature of transport was mentioned. Comments were made that this was not being specifically

picked up and that the horizontal aspects of the European Union policy identified in these flagship

initiatives were not sufficiently being picked up either. This is also the opinion of the Expert Group

based on their analysis.

With regard to more recent developments in EU strategy, the general objectives of Horizon 2020

transport can easily be mapped against President Juncker’s top priorities28, showing that they are

highly relevant. The relevance of Horizon 2020 transport can be clearly observed in the following

table, showing the evolution of top-level European priorities between 2010 and 2014. Horizon 2020

Transport priorities fit perfectly with them.

Table 2: Horizon 2020 Transport specific objectives compared with Europe 2020 and Juncker’s

priorities

Year 2010

Europe 2020

Year 2013

Horizon 2020 Transport

Year 2014

Juncker’s priorities

Smart growth Competitiveness of the European

Transport industries

Jobs, Growth and Investment

Sustainable growth A European transport system that is

resource efficient, climate- and

environmentally-friendly

Energy Union and Climate

Inclusive growth A European transport system that is

safe and seamless for the benefit of all

citizens, the economy and society

Justice and Fundamental

Rights

It is clear that transport has a high impact and relevance on economic growth and jobs (interpreting

competitiveness in its widest sense) and this is reflected in the following facts and figures from the

Horizon 2020 Transport Work Programmes, among other documents:

26

http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/flagship-initiatives/index_en.htm 27

http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/eu-tools-for-growth-and-jobs/index_en.htm 28


http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/flagship-initiatives/index_en.htm

http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/eu-tools-for-growth-and-jobs/index_en.htm



- In 201329: the transport sector directly represented 6.95% of the Union’s Gross Domestic

Product (GDP). The whole industry accounted for 7.03% of total employment in the EU,

corresponding to more than 14 million people in absolute terms. It is estimated that around

12 million jobs are in the automotive sector (both directly and indirectly). The aviation sector

alone generates around 2% of EU’s GDP (or some €110 billion). It directly employs between

1.4 million and 2 million people and indirectly supports between 4.8 million and to 5.5

million jobs. Its combined positive impact, including tourism, is thought to be as much as

€510 billion through the multiplier effect30. Both sectors are considered to be high-tech

sectors, employing highly qualified people and leading technologies in line with Europe’s

ambition to be a global leader. In addition there are technology spillovers from the work in

these areas that can be successfully transferred to other industrial sectors31.

- Freight transport and logistics keeps much of Europe’s economy moving. In 2012 (EU28)

freight transport was responsible to close to 3.8 trillion tonne-kilometres. 45% of this was

transported by road, 40% by sea, 9% by rail and about 3% by inland waterways. However,

Eurostat surveys estimate that 24% of road freight vehicles in the EU are running empty.

Flow imbalances are seen as being part of the problem and improvements in efficiency in the

EU logistics sector are estimated to be able to provide savings of between €100 billion and

€300 billion per year32.

In the Commission Staff Working Document of July 2016 on the implementation status of the

European Transport White Paper33, the Commission states that five of the President ’s ten priorities

are particularly relevant for transport. These are namely: (1) a new boost for jobs, growth,

investment, (2) a deeper and fairer internal market with a strengthened industrial base, (3) a resilient

energy union with a forward looking climate change policy, (4) a connected digital agenda and (5)

ensuring that the EU is a stronger global actor. The Commission also stresses that these strategies

encompass various policy areas and reflect a more cooperative and horizontal approach of the

Commission to addressing main challenges, and that all five priorities serve as a new impetus for

various transport policy initiatives.

Of particular relevance is the link between transport research and climate. This is clearly seen in the

data provided by the European Commission as part of its cross-cutting reporting34. Based on the EC’s

data, the share of EC contribution to Climate Change (all projects) is 23.9% while for the entire

Horizon 2020 it is 60.1%35. It can be concluded from these two values, that transport is more relevant

for climate change than for Horizon 2020 in general. Only “SC3 Secure, clean and efficient energy” is

more relevant than transport with a percentage of 99.1%. Interestingly, “SC5 Climate action,

29


Brussels, 7.12.2015.COM(2015) 598 final. An Aviation Strategy for Europe.http://eur-lex.europa.eu/legal-

content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN 31

https://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/main/Horizon 2020-

wp1415-transport_en.pdf 32

10% to 30% improvement would yield these savings

http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2016_2017/main/Horizon 2020-



Based on a set of Key Performance Indicators KPIs 35

Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016.


http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN


https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/main/h2020-wp1415-transport_en.pdf


http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-wp1617-transport_en.pdf




environment, resource efficiency and raw materials” has a percentage of 47.1%, smaller than in the

case of transport.

A similar analysis based on the same information can be offered in the case of the share of EC

contribution to Sustainable Development (all projects). Here transport is 85.5%, while for the entire

Horizon 2020 it is 46.1%. In fact, transport is the fourth largest area of Horizon 2020, after “SC2 Food

security, sustainable agriculture and forestry, marine, maritime and inland water research and the

bioeconomy”, “SC3 Secure, clean and efficient energy” and “SC5 Climate action, environment,

resource efficiency and raw materials”. This relates also to section 8.2.5 concerning Horizon 2020

delivering research that provides feedback to policy (to be found in the section 8 on added value).

The 2011 White Paper36 is one of the initiatives foreseen to deliver on the resource-efficient Europe

flagship37 and the relevance of Horizon 2020 transport to the European transport policy presented in

it38 is undeniable.

The following table maps the links between the Transport White Paper goals and the specific

objectives of the Horizon 2020 transport challenge. Horizon 2020 transport research is especially

relevant by contributing to new technologies, developing new ways of implementation, pilots and

demonstrations, and for socio-economic and behavioural research.

Table 3: Horizon 2020 Transport specific objectives compared with Transport White Paper goals

Transport White Paper Horizon 2020 Transport specific objectives

To boost competitiveness of the European Transport industries

GENERAL GOAL: a 60 % reduction of CO2 by 2050 compared to 1990 levels and by around 20% by 2030 compared to emissions in 2008. Operation goal 1. Cars; aim to halve the use of 'conventionally-fuelled' cars in cities and have virtually CO2-free city logistics in major urban centres by 2030. Operation goal 2. Low-carbon fuels in aviation should reach 40 % by 2050 and CO2 emissions from maritime bunker fuels should be reduced by 40 % by 2050 compared to 2005 levels.

To achieve a European transport system that is resource efficient, climate- and environmentally-friendly

OPERATIONAL GOALS 3 TO 10. Operation goal 3. Freight; modal shift from road transport. Operation goal 4. EU-wide high-speed rail network. Operation goal 5. Multimodal TEN-T core network. Operation goal 6. Long-term comprehensive network. Operation goal 7. Traffic-management systems in all modes. Operation goal 8. Multimodal transport information. Operation goal 9. Close to zero fatalities in road transport. Operation goal 10. Towards ‘user pays’ and ‘polluter pays’

To achieve a European transport system that is safe and seamless for the benefit of all citizens, the economy and society

36

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52011DC0144&from=EN 37

http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf 38

http://ec.europa.eu/transport/themes/strategies/2011_white_paper_en.htm. Commission White Paper

"Roadmap to a Single European Transport Area – Towards a competitive and resource efficient transport

system" (COM(2011)0144).


http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf

http://ec.europa.eu/transport/themes/strategies/2011_white_paper_en.htm


Of the above operation goals, developing and deploying new and sustainable fuels and propulsion

systems, optimising the performance of multimodal logistic chains and increasing the efficiency of

transport and of infrastructure use with information systems are particularly related to Horizon 2020

Transport objectives.

In July 2016, the Commission presented its Implementation Assessment of the White Paper39. The

main conclusions are that there are no indications of substantial change in the main transport trends,

identified in 2011 White Paper, in recent years. ‘Transport activity continues to raise concerns about

its negative externalities, notably GHG emissions, air pollution, noise, congestion, safety. Road

remains a predominant mode of transport and source of emissions, while aging and urbanisation

remain major demographic trends for transport. Still some new societal developments and

technological advancements have gained on pace and are likely to shape the functioning of transport

services already in the near future’.

As a general conclusion, the Commission states that ‘the 2011 White Paper defines a long-term vision

until 2050 for a transport sector that continues to serve the needs of the economy and of the citizens

while meeting future constraints’.

For instance, increased action is needed at EU level in connection with road safety, a key societal

challenge showing stagnation and a deviation from the projected trend. This is an area where

research is playing40 a key role.

Figure 3: Road fatalities in the EU since 2001.

Source: DG MOVE, CARE database (provisional data for 2015)

39


The Transport Work Programme 2014-2015 introduced a specific topic titled “MG.3.4-2014. Traffic safety

analysis and integrated approach towards the safety of Vulnerable Road Users” and the Transport Work

Programme 2016-2017 includes several topics related to road safety such as the topic “MG-3.2-2017:

Protection of all road users in crashes”.



The future Strategic Transport Research and Innovation Agenda (STRIA), as part of the upcoming

communication on Energy Union Integrated Strategy on Research, Innovation and Competitiveness

(EURICS), will help to streamline the research and innovation efforts and focus them on the most

pressing challenges and the most beneficial actions for transport.

In general, and based on interviews, hearings and our own expert opinion, it can be stated that the

aim of Horizon 2020 Transport to address the broad number of important issues and deliver

excellence and impactful relevant research and development outcomes under one programme

constitutes a large challenge. Several stakeholders questioned whether the work programmes are

too broad and if they should rather focus or the more pressing needs or market failures instead of

trying to cover all aspects.

4.1.2 AREAS OF RELEVANCE OF SPECIFIC EUROPEAN INTEREST

Horizon 2020 transport research is certainly relevant for many important European initiatives,

policies and communications, but this varies depends on the topic as this section presents.

(A) DIGITAL SINGLE MARKET PRIORITY

It is more complex to find direct relevance with President Juncker’s priorities than to align Horizon

2020 Transport specific objectives with the new priorities set up by Commissioner Moedas41: Open

Innovation, Open Science and Open to the World. This is mainly due to the more general character of

these three priorities affecting, in one way or another, all research and innovation domains and

challenges.

For instance, within the Digital Single Market priority, the need for affordable parcel delivery costs is

specifically mentioned42 since more than 90% of e-shoppers consider low delivery prices and

convenient return options as important when buying online. In fact, 62% of companies that are

willing to sell online say that high delivery costs are a problem for them. The EC’s 2015

communication “A Digital Single Market Strategy for Europe” actually dedicates an entire section to

affordable high-quality cross-border parcel delivery43. Research and innovation in this area is clearly a

relevant area for Horizon 2020 Transport but it has not yet been addressed to the extent that a

digital single market requires.

The Horizon 2020 Transport Work Programme 2014-2015 makes an explicit reference to the Moedas

priority of Open Science: ‘a novelty in Horizon 2020 is the Open Research Data Pilot which aims to

improve and maximize access to and re-use of research data generated by projects’44.

With regard to the Open to the World priority, there is a clear and very important link with Horizon

2020 transport and this is presented later in a specific section on international outreach.

According to the European Commission’s data, cross-cutting KPIs reporting on the share of EC

contribution to the Digital Agenda (all projects) in the Societal Challenge SC4 Smart, green and

integrated transport come in at 17.6% while for the Horizon 2020 as a whole the percentage reaches

41

http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm 42

https://ec.europa.eu/priorities/sites/beta-political/files/dsm-factsheet_en.pdf 43

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52015DC0192&from=EN 44

https://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/main/Horizon 2020-

wp1415-transport_en.pdf

http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm

https://ec.europa.eu/priorities/sites/beta-political/files/dsm-factsheet_en.pdf





24.1%45. This can be considered to be below optimal levels to achieve the desired Digital Agenda

objectives.

(B) RESOURCE EFFICIENT EUROPE

In the case of the flagship initiative “Resource Efficient Europe”, the European strategy indicates that

the Commission will work46:

- To present proposals to modernise and decarbonise the transport sector thereby

contributing to increased competitiveness. This can be done through a mix of measures e.g.

infrastructure measures such as early deployment of grid infrastructures of electrical

mobility, intelligent traffic management, better logistics, reduction of CO2 emissions for road

vehicles as well as for the aviation and maritime sectors…

- To accelerate the implementation of strategic projects with high European added value to

address critical bottlenecks, in particular cross border sections and intermodal nodes (cities,

ports and logistic platforms)

The Resource Efficient Europe communication also highlights that47:

- Technological improvements, a significant transition in energy, industrial, agricultural and

transport systems, and changes in behaviour as producers and consumers is required to

achieve a resource-efficient Europe;

- Production processes (such as Just-in-time or the collection of waste and recycling) designed

to reduce the energy may conversely require more transport.

- The EU’s industrial policy for the globalisation era specifically indicates that the Commission48

should ensure that transport and logistics networks enable industry to have effective access

to the Single Market and the international market beyond. Trans-European transport

networks should be aligned with the requirements of a resource-efficient, low-carbon

economy.

- Policy measures to improve resource efficiency and overall economic competitiveness must

place greater emphasis on 'getting prices right' and making them transparent to consumers,

for instance in transport, energy. There is a need for new models focused on specific policy

areas and sectors such as energy and transport.

More specifically in the low carbon and climate change areas, Horizon 2020 Transport Research and

innovation also represents a necessary contribution towards the objectives set in several European

Commission communications:

a) COM(2011) 112 final. A Roadmap for moving to a competitive low carbon economy in 205049:

reducing CO2 emissions by at least 60 % in transport by 2050 compared to 1990 levels.

b) COM/2015/080 final. A Framework Strategy for a Resilient Energy Union with a Forward-

Looking Climate Change Policy50: by 2030, at least 27% share of renewable energy consumed

45

Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016. 46

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:EN:PDF 47

http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf 48

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:EN:PDF 49

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0112:FIN:en:PDF

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:EN:PDF

http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:EN:PDF



in the EU and at least 27% improvement in energy efficiency. At least 40% domestic

reduction of greenhouse gas emissions by 2030 and 60% by 2040. Reaching and exceeding

these intermediary objectives will allow the EU to pursue the goal of a 80-95% decrease in

greenhouse gas emissions by 2050. This Energy Union Communication includes the

announcement of a “Strategic Transport Research and Innovation Strategy” STRIA to be

presented in the next months.

The Expert Group, supported by comments in several interviews and the hearing, found it difficult to

establish clear links between these high-level numerical policy objectives and the specific

contribution expected from some topics, in particular those less prescriptive (for instance, what will

be the contribution to “an 80-95% decrease in greenhouse gas emissions by 2050” that will be

provided by a certain project concentrating of improving powertrain efficiency). Topics do

correspond to the general objectives but quantifying its impact towards them is more complex. The

future Strategic Transport Research and Innovation Agenda (STRIA) should fill this gap by producing

roadmaps and looking more precisely to what can be performed in research to achieve the

overarching objectives. STRIA is considered a key component designed to streamline the research

and innovation efforts and focus them on the most pressing challenges and the most beneficial

actions for transport.

Probably the most recent document that shows the relevance of Horizon 2020 Transport research

and innovation is the European Commission’s communication “Accelerating Europe’s transition to a

low-carbon economy”51. This communication stresses, among other important issues, the role of the

EU’s cities in the fight against climate change at local level and the key role of the energy sector but

also that of the industry, buildings, transport, waste, agriculture, land-use and forestry.

(C) GLOBAL INITIATIVES

In addition to all the previous European Union initiatives, there is a very relevant number of

international agreements that have been signed over the past 12 months, notably the Paris Climate

Change agreement of 201552 and the new Sustainable Development Goals (SDGs) adopted last year

201553.

In preparation of the International Climate Change conference (COP 21), held in Paris in December

2015, countries submitted national climate action plans54. The European Union announced 55 that the

50

http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2015:80:FIN 51

Brussels, 20.7.2016.COM(2016) 500 final. Communication from the Commission to the European Parliament,

the Council, the European Economic and Social Committee and the Committee of Regions. Accelerating

Europe's transition to a low-carbon economy. Communication accompanying measures under the Energy

Union Framework Strategy: legislative proposal on binding annual greenhouse gas emissions reductions by

Member States from 2021 to 2030, legislative proposal on the inclusion of greenhouse gas emissions and

removals from land use, land use change and forestry into the 2030 climate and energy framework and

communication on a European Strategy for low-emission mobility.

https://ec.europa.eu/transparency/regdoc/rep/1/2016/EN/1-2016-500-EN-F1-1.PDF 52

http://ec.europa.eu/clima/policies/international/negotiations/paris/index_en.htm 53

Transforming our World: The 2030 Agenda for Sustainable Development

United Nations, 2015.https://sustainabledevelopment.un.org/post2015/transformingourworld/publication 54

INDCs – (Intended) National Determined Contributions

http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2015:80:FIN

https://ec.europa.eu/transparency/regdoc/rep/1/2016/EN/1-2016-500-EN-F1-1.PDF

http://ec.europa.eu/clima/policies/international/negotiations/paris/index_en.htm

https://sustainabledevelopment.un.org/post2015/transformingourworld/publication


EU and its Member States are committed to a binding target of at least 40% domestic reduction in

greenhouse gas emissions by 2030 compared to 1990; at least a 27% share of renewable energy

consumption and at least 27% energy savings compared with the business-as-usual scenario56 with its

new 2030 Framework for climate and energy, an increase of targets stated in the White Paper. These

commitments are economy wide and transport is only one of the sectors that will contribute to these

reductions. Horizon 2020 Transport Research and innovation represents a necessary contribution

towards these low carbon objectives and those set out in several European Commission

communications.

This is supported with the data provided by the European Commission as part of its cross-cutting KPIs

reporting. The relevance of transport in the EC contribution to Climate Change and to Sustainable

Development was already mentioned in section 4.1.1, with a percentage of 85.5%. This is largely

coherent with the figures provided by the European Commission as part of its cross-cutting KPIs

reporting. Based on the EC’s data, the percentage of budget of topics in the Work Programme

mentioning at least one third country or region is 28.5% in the case of transport, while it is only

14.8% for the entire Horizon 202057. It can be concluded from these two values, that international

cooperation is much more important for transport than for Horizon 2020 in general.

It is also clear that these international agreements send a clear signal to all stakeholders, investors,

businesses, civil society and policy-makers that the global transition to clean energy and a shift away

from fossil fuels is here to stay58. The role of transport in achieving this is a major one.

By contributing to the economy, competitiveness and job creation, the transport sector is also

relevant to delivering other key SDGs such as poverty, social exclusion and social inequity. This

should be able to be measured rather than assumed. Working on new indicators that more clearly

show how sustainable transport improves equity, inclusion and uplifts people out of poverty would

fit with the evolution of challenges mentioned in the previous section, and is considered by the

Expert Group and others to be an interesting topic for developing research questions in the future.

Therefore it is considered that Horizon 2020 Transport has the potential to help Europe and its

member states to deliver on the Sustainable Development Goals and the International Climate

Change Agreement.

After having analysed the two Horizon 2020 Transport Work Programmes published so far, the

Expert Group agrees that there has been some progress, and the Horizon 2020 Transport Research

agenda is relevant towards achieving a Resource Efficient Europe in terms of technology, energy

transition and to some extent transformation of transport systems.

4.1.3 HORIZON 2020 TRANSPORT AND ITS INTERNATIONAL OUTREACH

One of the top Commissioner Moedas priorities is ‘Open to the World’, so it is worthwhile revisiting

how Horizon 2020 transport is directly linked to this priority.

55

www4.unfccc.int/Submissions/INDC/Published%20Documents/Latvia/1/LV-03-06-EU%20INDC.pdf 56

www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ec/145397.pdf 57


http://ec.europa.eu/clima/policies/international/negotiations/paris/docs/qa_paris_agreement_en.pdf

http://www4.unfccc.int/Submissions/INDC/Published%20Documents/Latvia/1/LV-03-06-EU%20INDC.pdf

http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ec/145397.pdf

http://ec.europa.eu/clima/policies/international/negotiations/paris/docs/qa_paris_agreement_en.pdf


Firstly, the international dimension in Horizon 2020 transport is seen as being more strategic, and

reinforced, as became clear from the interviews, for instance. The link of the Horizon 2020 transport

Societal Challenges and the Open to the World priority is clearly presented in the 2016-2017 Horizon

2020 Transport Work Programme59:

“International cooperation will have a key role to play. Global challenges such as CO2 and

polluting emissions, oil dependency, transport safety and security, noise pollution, and

standardisation of many services, products and procedures will benefit from global

solutions...”

As one specific example of these challenges, it is emphasized that transport safety will be a priority

for a Euro-African science diplomacy partnership. Since FP7 transport research has a long tradition of

international cooperation activities in aviation, renewable fuels, smart mobility and safety. The

openness to the world dimension of the Transport Work Programmes is also reflected in the

following sentence taken from the 2016-2017 Work Programme:

“All new priorities in the Work Programme… have a two-fold aim: addressing key challenges

that Europe faces, and making our industry more competitive and cooperative through

transferring these solutions and standards worldwide, as other regions are confronted with

similar challenges”.

The Horizon 2020 Monitoring Report 2014 also indicates60:

“Identifying suitable themes and partners for targeted international cooperation activities

was an important part of the preparation of the first Horizon 2020 work programmes. The

impact of this approach is reflected in the increase from 12% (FP7 baseline) to 22% (October

2015) in the indicator on the budget share of Work Programme topics mentioning

international cooperation or a specific third-country or region”.

Based on the specific analysis conducted for this report, it is estimated that around 24% of the

budget of the first two rounds of Horizon 2020 transport calls (without including PPPs) is related to

international cooperation, either because the topic directly addresses it or because international

cooperation is mentioned.

This percentage is therefore comparable to the average (22%) reported in the Horizon 2020

Monitoring Report 2014, and is also largely coherent with the figure provided by the European

Commission as part of its cross-cutting KPIs reporting. As mentioned and based on the EC’s data, the

share of EC contribution to International Cooperation topics is 28.5% and from this it can be

concluded, that international cooperation is much more important for Transport than for Horizon

2020 in general.

On the other hand, and based on the data available to as to 1st of August 2016, the share of EC

Contribution to Participations in Signed Grants represents a 0.1% for the entire Horizon 2020

59



http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108





Transport, a 0.2% for projects excluding the SME Instrument and the JUs, a 0.4% in the SME

Instrument and a 0% in JUs61.

4.2 ADDRESSING SPECIFIC STAKEHOLDER NEEDS, INCLUDING EMERGING

ISSUES

Horizon 2020 Transport offers comprehensive consultation mechanisms with stakeholders that are

designed to help the programme to adapt to new scientific and socio-economic developments

including:

- Consultations with Member States through the Transport Programme Committee

(Comitology procedures62)

- Transport Advisory Group

- Consultations with key stakeholders such as industry groups, research associations,

technology platforms and other networks such as POLIS, ERTICO…

- Open or public consultations

- Internal consultations…

This evaluation relies partly on an extensive consultation process, including contacts with technology

platforms (e.g. ERTRAC, ERRAC, Waterborne, ACARE, and ECTP), support actions (e.g. ALICE),

association (e.g. POLIS, ECTRI, FEHRL, EREA, ECVIA), key SMEs (e.g. ERTICO), JU’s and industry

associations (e.g. IMG, ACEA, ACEM), modal representative associations (e.g. FIM, FIA), large or key

participants in projects, and of course member states. The roadmaps produced by these groups are

of particular relevance for transport and many of the topics or areas for research are continuously

being considered in the calls.

In its Scoping Paper for the preparation of the next 2018-2020 Work Programme63, the Commission

reports that the Transport Advisory Group (TAG) pointed out the relevance of both the content of

the previous Work Programmes and the priority challenges identified in the first TAG report in 2014

and in its follow-up report submitted to the European Commission (May 2016). They stressed the

need to continue with a disruptive rather than incremental approach to research in order to respond

rapidly to shifting transport paradigms with new revolutionary technologies, business environment

and mobility patterns. In addition, the TAG identified a number of new issues, which have emerged

since 2014, for instance:

- The positive impact of the COP21 agreements providing the transport sector with a clear

agenda and mandate. An increasing impact of climate change and extreme weather

phenomena risks calls for new approaches towards preparedness and resilience, particularly

for major transportation nodes and mass transport systems.

61

Data provided by the EC for the purposes of this interim assessment in the document “Interim Evaluation -

Projects - EU.3.4 - Transport.xlsx” 62

http://eur-lex.europa.eu/summary/glossary/comitology.html 63

First Orientations towards the scoping paper for Horizon 2020 Societal Challenge 'Smart, Green and

Integrated Transport'. Internal document of the European Commission that has been made available to the

Expert Group.

http://eur-lex.europa.eu/summary/glossary/comitology.html


- A slow growth world full of uncertainties contributing to accumulated delays in building and

maintaining infrastructure and mass transit systems, with accompanying negative impacts on

accessibility, congestion, safety and the overall reduced efficiency of transport systems.

- Security threats targeting Europe and its logistic and strategic centres,

- Lower oil prices affecting many economic drivers (although it may have a positive impact on

air transport and the auto industry) it impacts on alternative fuel research and development,

and may lead to delays in Europe’s transition towards a carbon-free economy.

- Large-scale migration flows impact border management, hamper mobility, increase costs,

lower accessibility and accentuate regional differences.

- The explosion of new business models related to digitalisation, disruptive technologies and

new actors from other non-transport sectors.

Formal consultations with key stakeholders such as industry groups, research associations,

technology platforms and other networks such as POLIS, ERTICO took place in 2016 as part of the

preparatory work for the drafting of the Work Programme 2018-2020. Approximately 40

organisations from all transport modes and including research organisations, industry associations,

public bodies and users of transport as well as citizens associations participated in the stakeholder

consultation from 23/3/2016- 11/5/2016. A number also took the trouble to prepare position papers

(e.g. ECTRI and ETRA) to not only to set out their priority areas, but also to voice concerns on present

calls, processes and programmes. The priority areas of research underpinning the current strategic

objectives highlighted by the stakeholders were64:

- Fighting climate change and the slow progress to decarbonisation passenger and freight

transport

- The resilience of transport systems

- Electrified transport, including energy harvesting and storage systems

- Seamless and integrated transport (with particular emphasis of public transport),

multimodality, interoperability

- A single market for mobility as a service

- Digitalisation, including multi-model trip information, digital transport areas, big data and

transport, Internet of (Transport) Things Io(T)T

- Safety and security

- Environment and health

- Governance and regulation

New areas (inter alia) identified by the stakeholders that they would like to see included in the next

work programme were:

- Automated transport: not only road vehicles, but including drones, vessels, infrastructure for

automated transport and covering new terms such as big data, drones, etc.65

- Transport and health (obesity and active transport, mainly)

- Circular economy and its relation to transport

64

WP 2018-2020. Summary of replies to public consultation. Internal doc prepared by DG RTD, Transport. 65


content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN




- New nanotechnology materials with self-reporting and self-healing capabilities, for instance

for infrastructure

- Hyper-connected transport (including 5G systems for rail communications), physical Internet,

disruptive technologies

- Vulnerabilities and new security threats (including cyber-security and personal data

protection)66

Other more general proposals from the stakeholders for improving the programme include:

- To increase focus on bottom-up technology (lower TRLs) R&D in the Work Programme, since

JUs are already dealing with higher TRLs.

- Few innovative options can be integrated easily into today’s transport systems without major

step changes occurring either within or from outside the sector, highlighting increased effort

towards ‘low tech’ cost effective solutions that can bridge the gap between now and post

2020 transport systems.

- Basic research into desirable futures and transition pathways, social impacts and evaluation

or assessment of the effects of changes (such as emissions, climate change), technological

transformation (e.g. the sharing economy).

- Increase integrated approaches with other sectors and working arrangements with other

DGs such as energy (regarding climate change, for instance), ICT or Nanotechnologies,

Advanced Materials and Advanced Manufacturing and Processing (NMP).

- Request for more clearly defined and specific call topics and descriptions rather than the

current rather broad approach to calls (that can be widely interpreted). This is confirmed

with frustration expressed by stakeholders in the interviews especially in respect to the

evaluations of proposals, as it is felt that experts are not be able to evaluate effectively so

many perspectives.

- More continuity in the research topics (a topic is included only once in the whole plan

period) and also in the terminology used.

- “Emergency” calls in between (larger) programmed calls.

- Stronger emphasis on user perspectives, user education and behaviour, social inclusion,

reduction of inequities, and user outreach and implementation issues (governance, user

behaviour, stakeholder resistance to change, new funding models…) and less bias towards

technology.

- Include disruptive approaches, game changers, and the study of market failures in all modes.

- Reinforcement of international cooperation via bilateral and multilateral funding

mechanisms in support of Europe’s global role and the competitiveness and access to

markets of the European stakeholders (industries, research institutes, etc.)

- Increased clarity on how stakeholder and expert guidance offered to the Commission is

treated in respect to developing calls and responding to new challenges (see above game

changers / new concepts etc.).

Many of the proposals from the stakeholders represent a desire for increased attention to a medium

to long term vision, and are perfectly aligned with EU’s high level objectives. These include

transport’s impact on climate change, improving the energy efficiency and sustainability of transport,

66

Probably a cross-cutting issue with other Horizon 2020 societal challenges


exploiting innovative technologies, deploying more Intelligent Transport Systems, tackling safety and

security, and more generally fostering the competitiveness of the European transport industry in

global markets etc.

In line with this, the “Scoping Paper” from the TAG proposed the following strategic orientations for

2018-202067.

Table 4: Strategic orientations proposed by the Transport Advisory Group for the future Work

Programme 2018-2020

Technology oriented research and innovation

increasingly complemented with a strong focus

on user needs, societal issues and regulatory

framework.

Accounting for the people: demand, needs and

behaviours; inclusion and access.

Promotion of an innovation strategy aimed to

decarbonisation, digitisation and human-centred

transport system in an increasingly integrated

manner.

Technologies transforming the transport system

Towards an integrated, sustainable and robust

transport system.

Competitiveness, business models and markets.

This is somewhat validated by the project coordinators surveyed as part of this assessment, who

think that the Horizon 2020 Transport programme objectives take into account the latest scientific,

socio-economic, political developments in the field of transport research and innovation at the

national/European and international level68 but also expressed dissatisfaction on how the results

were the taken forward.

The responses to the survey are shown in the table below.

Table 5: Do the Horizon 2020 Transport work programme objectives take into account the latest

scientific, socio-economic, political developments in the field of transport research and innovation at

the national/European and international level? Answers to the project coordinator survey.

0 (not taken into account)

1 2 3 4 5 (fully taken into account)

I don’t know

Scientific 0% 0% 4% 9% 61% 22% 4%

Socio-economic

0% 0% 4% 13% 52% 17% 13%

Political 0% 0% 4% 9% 39% 13% 35%

Results show that the majority agreed (78%) that Horizon 2020 Transport takes the latest socio

economic trends into account from a scientific perspective, however the feedback loop to policy was

less well established or clear to them. Only 50% felt that this was somewhat satisfactory while 34%

67

First Orientations towards the scoping paper for Horizon 2020 Societal Challenge 'Smart, Green and

Integrated Transport'. Internal document of the European Commission that has been made available to the

Expert Group. 68

Survey to project coordinators design and distributed as part of the present assessment


did not know. The latter is of most interest as it is clearly not seen by those answering the survey as

being of critical importance and their focus of interest is clearly more technology rather than policy

development biased.

Finally, the appreciation of increased regional relevance on Horizon 2020 Transport over FP7 was

also mentioned in the stakeholder hearing (September 2016) but the participants further validated

that there was a not a strong link to policy development especially at regional levels.

As the Transport Work Programme 2014-2015 explains in the case of aviation, the proposed Topics

are in line with the Strategic Research and Innovation Agenda (SRIA) of the Advisory Council for

Aviation Research and Innovation in Europe (ACARE). They are also aligned with the goals of the

Report of the High Level Group on Aviation Research published in 2011 under the title “Flightpath

2050”. In fact, the aviation research and innovation activities within Horizon 2020 Transport are seen

as being perfectly aligned with the aviation strategy for Europe presented by the European

Commission in December 201569.

Another example is the following excerpt from the 2016-2017 Work Programme: “Waterborne topics

reflect an assessment of research needs which takes into account the current priorities of the

stakeholders…”.

From a transport policy perspective and the July 2016 Implementation Assessment of the Transport

White Paper, the current situation is seen as having changed from that of 2011 and new

socioeconomic challenges relevant to Horizon 2020 Transport include:

Demographic and urbanisation trends differing to those predicted in published literature as

guidance for development.

Increased complexity of business and new Circular Economy approaches

Deep changes in supply and value chains with the growing importance of software,

globalisation, ICT and 3D printing.

Fragmented and incomplete framework conditions for “smart transport” in areas such as

standardisation, interoperability and data exchange.

Growth of digitalisation and mobility as a service.

Automation and connected vehicles challenges beyond technology (human factors).

Unexpected interest and technology readiness levels (TRL) in new concepts such as drones.

Strong growth in the collaborative and sharing economy (cloud funding, bottom up solutions

and sharing rather than owning concepts).

Increasing importance of the role of active modes in the urban transport mix (especially in

response to growing concerns on transport related impacts on health).

Growing dissatisfaction of European citizens of contentious negative externalities of

transport (air quality, safety of vulnerable users, noise, congestion, land take etc).

Increasing security threats, including cyber-security.

Unresolved social issues such as the recent migration challenges.

69


content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN




As can be seen, many of these challenges listed in the revision of the Transport White Paper have

also been identified in interviews, by survey responses, informally and at the hearing (September

2016). The list above also reflects directly the proposals for new developments and emerging issues

proposed by stakeholders above (TAG report, Scoping Paper…). The following table shows how the

new challenges identified in the Interim Assessment of the Transport White Paper can be perfectly

tackled within the existing Horizon 2020 Transport specific objectives.

Table 6: Link between the Horizon 2020 Transport specific objectives and the new socioeconomic

challenges identified by the various stakeholders

New socioeconomic challenges identified in the Interim Assessment of the Transport White Paper

Horizon 2020 Transport specific objectives

• Increased complexity of business and new Circular Economy approaches

• Unexpected growth in the collaborative and sharing economy (cloud funding, bottom up solutions and sharing rather than owning concepts)

• Deep changes in supply and value chains with the growing importance of software, globalisation, ICT and 3D printing

• Growth of digitalisation and mobility as a service.

• Unexpected interest and technology readiness levels (TRL) in new concepts such as drones.

To boost competitiveness of the European Transport industries

• Increasing importance of role of active modes in the urban transport mix (especially in response to growing concerns on transport related impacts on health)

• Increasing security threats, including cyber-security.

• Fragmented and incomplete framework conditions for “smart transport” in areas such as standardisation, interoperability and data exchange.

To achieve a European transport system that is resource efficient, climate- and environmentally-friendly

• Demographic and urbanisation trends differing to those predicted in published literature as guidance for development.

• Automation and connected vehicles challenges beyond technology (human factors).

• Growing dissatisfaction of European citizens of contentious negative externalities of transport (air quality, safety of vulnerable users, noise, congestion, land take etc).

• Unresolved social issues such as the recent migration challenges.

To achieve a European transport system that is safe and seamless for the benefit of all citizens, the economy and society

The purpose of this table is to show that all the previous emerging issues can be accommodated in the existing

specific Transport objectives. An analysis of how they are being tacked in the Work Programmes or what is the

actual room for action from the Horizon 2020 Transport challenge would require a deeper analysis.

One important concluding remark at this point is that Horizon 2020 Transport objectives are quite

clearly fully capable of including all the above new trends and issues and are still fully relevant. In


other words, that Horizon 2020 Transport objectives do not need to be adapted to the changed

scenario and challenges but, instead, that new and emerging issues can be embraced by existing

objectives

4.2.1 SUGGESTIONS FOR IMPROVEMENTS FROM A STAKEHOLDER PERSPECTIVE

Horizon 2020 Transport has sound procedures in place for stakeholder consultation and flexibility of

the Work Programmes is shown, for instance, by the introduction of a large specific call on

Automated Road Transport in WP 2016-2017 with €114 million budget including full demonstration

activities. However, a number of participants in the interviews and the hearing mentioned the need

for added flexibility and quicker processes to analyse the relevance and effects of new, quickly

emerging needs (for example: the consequences of autonomous driving). At present, new and

“urgent” topics sometimes need to undergo a long process until they find their way into the text of

the calls. This is considered by the Expert Group to diminish some aspects of relevance.

From the survey, comments offered by project coordinators included a specific concern that there

was some inflexibility especially in respect to the technology based EU Road Maps. The speed of

innovation is traditionally difficult to anticipate over any length of time70. Few people are able to

correctly estimate the take up and success of these disruptive and counter-intuitive technologies and

business models. The recommendation in this case may be to insist on swift mechanisms for annual

updates of those technology maps.

A minority of stakeholders. mainly from technology platforms, also voiced their opinion on a lack of

transparency or information in connection with the preparation of the work programmes71. In

particular they were suggesting more information on how the final process of prioritizing proposed

topics in the Work Programme is implemented (in other words, what is included and what not). This

should be taken into consideration as a different issue to the generally well-accepted and

appreciated process of collecting information from the stakeholders and the consultation process.

These remarks are in connection with how the individual and collective inputs to this process are

treated which is unclear to stakeholders currently. The Expert Group suggests that a solution could

be found with increased efforts in communicating on how work programmes are prepared and how

inputs are treated (as well as more clarity on how actors (new and non-traditional) can get involved

into the process to ensure that new ideas are brought to the table). This increased effort could come

from the EC itself (for instance with a short section on the Work Programmes explaining what inputs

where received and how they were treated generally) and from the National Contact Points (NCPs)

when presenting the work programmes to the national constituencies.

An important issue regarding the involvement of stakeholders that recurrently arose during the

interviews for this interim assessment is the limited involvement of representatives from the softer

transport modes. It is recognised that this may partially be due to the fact that they are not

constituted in well-defined groups, as the majority of other more traditional transport modes are.

Stakeholders such as civil society organisations that represent citizens at large, pedestrians,

passengers of all transport modes, union representatives were mentioned.

70

Note the explosive success of AirBnB or Uber over the past five years 71

Impacting both relevance and coherence


As part of its cross-cutting KPI reporting, the European Commission monitors the percentage of

Responsible Research and Innovation (RRI) projects where citizens, Civil Society Organisations (CSOs)

and other societal actors are involved72. While for the entire Horizon 2020 the share of signed grants

flagged as RRI-relevant (all projects) is 6.9%, in the case of transport the share is only of 3.6%. This is

considered to be low and should be strengthened.

It is also felt by the Expert Group, based on the analysis of the work programmes and comments

from interviews and the hearings, that this has led to a dominance of some aspects of the road maps

from the larger actors (industries) rather than looking for the best solution to address the challenges

facing Europe and that Horizon 2020 Transport is designed to address.

4.3 OTHER ISSUES RELEVANT TO RELEVANCE

Overall the increased attention in Horizon 2020 and to the transport component in relation to

strengthening close-to-market activities is considered to be an important and positive change,

increasing the potential for European research to be cutting edge and world beating.

A preliminary analysis of the 2014-2015 and 2016-2017 Transport Work Programmes concludes that

particular reference to pilots, demonstrations are included in topics that represent a 17.5% of the

budget of the calls (a budget of €317.8 million out of a total of €1819.9 million for the first two

Transport calls). In this regard, putting more emphasis during the evaluation of proposals on the path

to market of innovations (as it is already the case in the SME Instrument) might provide a clear

message to innovators. This higher emphasis could have two dimensions: specific briefing to the

evaluators insisting on the importance of close-to-market project activities or even changing the

evaluation criteria in order to give larger weight to the impact criterion.

Results from the survey show that few patents have materialised directly from the last batch of FP7

projects but the number of prototypes and major demonstration projects is much more promising.

The fundamental role the Commission plays in funding major demonstration projects was confirmed

in the Stakeholder hearing (September 2016).

But, as the Commission presents in its Horizon 2020 Monitoring Report 2014, bridging from discovery

to market application is also achieved through innovation actions73. In this respect, the report

indicates that:

“Overall, 202 innovation action projects have been signed in 2014, with a requested EC

contribution of €1071 million. This represents 4.20% of the total number of successful projects

signed related to calls closed in 2014 (4809) and 12.65% of the total EC contribution allocated

to these successful projects (€8467.83 million)”.

In the case of transport, and based on the analysis of the budget per topic conducted for this report,

the budget allocated to Innovation Actions in the 2014-2015 and 2016-2017 Work Programmes

reaches the 25.9%, much higher than the indicated overall 12.65%.

72





New tools such as the Common Exploitation Booster74, supported through the Horizon 2020

Common Support Centre, and obviously the centre in itself, are valuable tools for project participants

in their path towards commercialization.

Particular attention is deserved here on the reflection of the balance between incremental research

and radical innovations and integrating them into policy (see also previous point on timeliness and

relevance). As this sentence from the Work Programme 2014-2015 summarizes, “crafting the right

rail innovation strategy will require a step well beyond just technology”. It is very often mentioned

that incremental research will not be enough to respond to the difficult challenges ahead and that

breakthrough innovations are indeed needed regardless of the transport mode at stake. The

inflexibility of the programme is brought into question here.

Aviation research, for instance, includes a small-scope topic on radical or ‘Breakthrough innovation

for European aviation’ which is intended to make proof of concepts and technologies in the TRL

range 1-2. Obviously longer-term research has a privileged place in the line Socio-economic Research

and Forward Looking Activities of the Transport call, but short-term interests should not be side lined

or marginalised.

On the other hand, high-risk research of a more interdisciplinary nature is addressed in the Future

and Emerging Technologies (FET) under the Excellent Science part of Horizon 2020. Despite this, the

links between FET and transport calls are seldom explained and, more importantly, utilized.

It is interesting to note that some aspects of the international trend in transport research and

innovation can be better defined as “radical innovations for the short to medium term”: Tesla over-

the-air software upgrades of its electric vehicles or new concepts such as Hyperloop75, Space

Tourism76, Uber and the use of drones for parcel delivery by Amazon or the Chinese Transit Elevated

Bus (TEB)77. It gives sometimes the impression that Europe is still deciding what the barriers for

innovations are, while real large-scale demonstrations are already taking place in other parts of the

world. A clearer view should be established when it comes to differentiate between research and

deployment: it is clear from the examples above that demonstrating radical innovations take less

time than deploying them, so it might be worthwhile supporting the former.

4.4 LESSONS LEARNT AND AREAS OF IMPROVEMENT

Generally Horizon 2020 Transport is seen as being enormously relevant for EU environmental

sustainability, society at large, competitiveness and boosting the European economy. From the

transport policy point of view, the transport actions under Horizon 2020 were largely based on the

Transport White Paper and as long as it is still pursued and relevant, so is Horizon 2020 transport. It

retains, and may increase, in relevance as additional objectives arise in fields such as security.

74

http://exploitation.meta-group.com/Pagine/About-Us.aspx 75

https://hyperloop-one.com 76

www.virgingalactic.com 77

https://en.wikipedia.org/wiki/Transit_Elevated_Bus

http://exploitation.meta-group.com/Pagine/About-Us.aspx

https://hyperloop-one.com/

http://www.virgingalactic.com/

https://en.wikipedia.org/wiki/Transit_Elevated_Bus


Its goals are clear for stakeholders: namely to improve competitiveness of the European transport

industry and to provide European citizens the transport system of the future that is green, efficient

and safe.

Horizon 2020 and Horizon 2020 transport has undergone reform when compared to FP7 (or other

previous Framework programmes) in terms of impact and implementation orientation, including

reorienting focus on innovation, demonstration and the introduction of new financial instruments.

The programme has also been highly relevant during the current economic crisis, not only in terms of

funding for research but it has also served to better identify priorities and to focus available

resources at national level. For example it was timely, in particular, for the automotive sector and

helped avoiding job losses in the field of research. A clear example of this alignment of priorities can

be found below in connection with Clean Sky, but it was also specifically mentioned by two of the

interviewees from the Transport Programme Committee. In one case, it was metioned that even if

there were not a national research programme specifically dealing with transport issues, the Horizon

2020 Transport priorities had been included in the general research programme.

Relevance of the work programmes is reassured by a sound work programme preparation and in the

call topics: such as the example of the Automated Road Transport (ART) call in Work Programme

2016-201778. Relevance can also include a national or regional dimension (several of the stakeholders

interviewed referred to “national relevance” of Horizon 2020 Transport) and, in this regard, the links

of Horizon 2020 Transport with national and other regional research programmes also helps to

underpin its relevance. The example from the Aviation Sector on signing Memorandum of

Understandings are thought to help underpin the potential of regional and European cooperation as

they bring together synergies between Horizon 2020 Transport and the regions’ interests. The

regions participate in defining Clean Sky calls and JUs give input to regional programmes to avoid

duplication and to perform complementary work. It is seen as being highly strategic and CS has

signed 11 MoUs and other are in the pipeline.

New research concepts such as “Open innovation” must be better explained and need to be better

understood by transport research participants. Openness seems to sometime conflict with

competitiveness and particular caution should be given, for instance, when transferring IPRs outside

the EU79. The transport sector is a very competitive sector, both inside Europe and outside (for

instance, new Asian companies entering the rail /road sectors) therefore too much openness is not

considered to universally wise in this sector. The concept of Open Science also collides frequently

with patents, the latter being a KPI for Horizon 2020. To some extent, those participating in Horizon

2020 transport sometimes question the relevance of investing in a programme that is too open.

The Expert Group found, based on interviews and hearings, that it is difficult to establish clear links

between high-level policy objectives and particular project objectives. In this regard, it is expected

that the future Strategic Transport Research and Innovation Agenda (STRIA) should fill this gap by

producing roadmaps and looking more precisely to what can be performed in research to achieve the

78

This “thematic” call included both an ICT Infrastructure topic directly introduced by the EC as a necessary

cornerstone of the ART system and also a specific topic on safety and end-user acceptance. 79

Art. 30.3 of the Model Grant Agreement already obliges beneficiaries in certain cases to notify the EC about

an intended IPR transfer outside the EU and gives the Commission the right to object such a transfer


overarching objectives. STRIA is considered as a key component designed to streamline the research

and innovation efforts and focus them on the most pressing challenges and the most beneficial

actions for transport.

Overall the transport research agenda of Horizon 2020 Transport enormously and clearly is relevant

to addressing the Societal Challenges outlined in SC4. It is seen as being fully relevant for achieving

key European goals such as competitiveness, environmental sustainability and social liveability.

However several areas of improvement have been identified in this interim evaluation.

One important remark at this point is that Horizon 2020 Transport objectives are quite clearly fully

capable of including all the above new trends and issues. In other words, that Horizon 2020

Transport objectives do not need to be adapted to the changed scenario and challenges but, instead,

that new and emerging issues can be embraced by existing objectives.


5 EFFECTIVENESS Effectiveness has been interpreted as measurable impacts in relation to the overarching objectives of

Horizon 2020 and more specifically to programme and project objectives. The Expert Group also

included the wider levels of effectiveness in terms of communication, dissemination of results (also

covered in greater length in the chapter on EU Added Value) and the take up of tools and

recommendations beyond the boundaries of the project especially by those not directly involved

with projects. Ultimately it would have been ideal to associate these results with the predetermined

objectives such as competitiveness, job creation and societal impact but this went beyond the scope

of the Terms of Reference for this group.

5.1 SHORT-TERM OUTPUTS FROM THE PROGRAMME AND EARLY

SUCCESS STORIES

The number of deliverables from FP7 and Horizon 2020 is impressive but extracting relevant and

useful knowledge from them is not easy. This aspect of knowledge ‘draining’ was mentioned in many

discussions and interviews with respondents involved in this evaluation process.

What happens after the end of any project is out of control of the programme, although in the Expert

Group’s opinion (and this point was also validated in several of the interviews) this might be one of

the most important outcomes of working towards achieving the goals. The impression from some

project participants80 is that the results are “put on the shelf”, and lack any practical or substantive

analysis of results or innovation, despite the original project goal to exploit the results. It is also

feared that the broad scope of the topics as presented in Horizon 2020 might lead to an increase of

such ‘valueless’ projects. In the paragraphs below, some topics were highlighted.

5.1.1 SUCCESS STORIES

As Horizon 2020 has just started and few projects have been finalised, examples of promising

projects for the Expert Group to investigate were few and far between. Many FP7 projects are now

finished, providing good examples of projects delivering results, and there is no evidence to date that

suggests outputs from Horizon 2020 will be any different (as the conditions are similar) from those of

FP7. Good examples can be found in the areas of the Green Vehicle Initiative and the urban mobility

calls (some are documented as in ERTRAC From Collaborative Research Projects to Market

Deployment with 12 success stories). Other programmes such as the CIVITAS programme is also

considered a good example (see later in Chapter on European Added Value). Such examples have

helped Europe to become a world reference in urban mobility (e.g. Sustainable Urban Mobility Plans

- SUMPs) and related ICT applications and provide a boost to exporting technology and knowledge to

third countries.

Other examples of commercialisation of results have also been found:

- TrafficFlow81 started with initial funding made available by the Regional Government of

Tuscany in the framework of the EraSME project ORUSSI (Era-Net - FP7)

80

In the survey and the interviews. 81

http://cordis.europa.eu/project/rcn/200334_en.html

http://cordis.europa.eu/project/rcn/200334_en.html


- CELSO82 on Automated Vehicle Monitoring systems

- Horizon 2020: Magenta Lab83,

More recently appreciation was expressed for the funding of the SETRIS Project, which brings

together five Transport European Technology Platforms (ETPs) – road, rail, air, water and logistics –

and a balanced representation of all transport modes with a variety of industrial/ commercial

players. Their aim is to deliver a cohesive and coordinated approach to research and innovation

strategies for all transport modes in Europe especially in line with the White Paper “Roadmap to a

Single European Transport Area”, and it is seen that this will help fill gaps and smooth any overlaps.

5.1.2 TRL LEVELS

The Horizon 2020 programme is set up to bring research, with higher technology readiness levels

(TRL84) closer to market making the transition towards deployment and/or further research more

likely than in FP7.

This is confirmed by the project coordinators, as can be seen in the figure below showing the TRL

levels of a sample set of FP7 and Horizon 2020 projects. It is clearly visible that Horizon 2020 aims at

a higher TRL levels than FP7 and this is also confirmed by the approach of PPPs to research.

Figure 4: Technological Readiness Level (TRL). Source: Survey among project coordinators, September

2016, n=21 for FP7, n=15 for Horizon 2020.

The higher TRL levels require a different approach in Horizon 2020 compared to FP7. There is a need

to improve the transition to deployment beyond traditional dissemination/exploitation activities and

this is seen by all stakeholders as one of the big challenges in Horizon 2020. Horizon 2020-funded

research should not only project transport futures, but also seek to influence them in various ways.

More effort should be put in market readiness through diffusion and involving targeted clients.

A stronger emphasis on the road to market, including combination of other funding sources outside

Horizon 2020 during the evaluation phase would send a clear message about the possibilities for

these external opportunities. In addition a more detailed section in the work programme for

additional funding opportunities and synergies, or something similar, would be helpful in this

82

http://cordis.europa.eu/project/rcn/204322_it.html 83

www.magentalab.it 84

https://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/annexes/Horizon 2020-

wp1415-annex-g-trl_en.pdf

http://cordis.europa.eu/project/rcn/204322_it.html

http://www.magentalab.it/

https://ec.europa.eu/research/participants/data/ref/Horizon%202020/wp/2014_2015/annexes/Horizon%202020-wp1415-annex-g-trl_en.pdf

https://ec.europa.eu/research/participants/data/ref/Horizon%202020/wp/2014_2015/annexes/Horizon%202020-wp1415-annex-g-trl_en.pdf


respect, as currently those involved in projects have little motivation to go searching for more

funding once a project is finished.

The converse of this is that a number of concerns were raised in respect to the focus on the higher

end TRLs and that this was stifling, rather than engendering, innovation. Cost effective solutions in

early stages of TRL readiness were seen as being desirable additions to the overall work programme

especially in respect to the inherent inertia in the transport sector for change and the slow progress

seen in many areas, especially in decarbonisation and alternative fuels. This seemed to be most

prevalent in the complex sectors of road and rail.

5.1.3 HUMAN CAPITAL DEVELOPMENT

The human capital development is expected to be similar for Horizon 2020 as for FP7. This can be

derived from a survey among project coordinators. The two graphs below show that the inputs, in

terms of numbers of researchers involved (both male and female) are expected to be slightly lower

while the outputs in terms of impact on mobility and gender equality are expected to be slightly

higher.

Figure 5: Average number of human resource related activities targeted or achieved at the end of the

project. Source: Survey among project coordinators, September 2016, n=30..32 for FP7, n=20.22 for

Horizon 2020.

Figure 6: Assessment of human capital impact targeted or achieved at the end of the project. Source:

Survey among project coordinators, September 2016, n=26..29 for FP7, n=18..19 for Horizon 2020.

An important point was brought up specifically by some participants in the stakeholder hearing,

notably from the research community, that there was little room for failure. It is not suggested that

poor projects should be supported – but the knowledge and human capital development value of a

project that has encountered major unexpected barriers is not valued under the current system. It is

thought that the experience of not being fully successful is thoroughly undervalued and it should find


a place in the system – after all Edison made 1,000 unsuccessful attempts at inventing the light bulb

and in his opinion ‘the light bulb was an invention with 1000 steps’.

5.1.4 KNOWLEDGE TRANSFER

The number of knowledge transfer activities is expected to be lower for Horizon 2020 than for FP7.

This can be derived from a survey among project coordinators, which is shown in the graph below. A

possible explanation could be the focus of Horizon 2020 on bringing the transport research closer to

the market. This has an influence on the areas of knowledge transfer as conferences and

publications, which are rather academically oriented.

Figure 7: Average number of knowledge transfer related activities targeted or achieved at the end of

the project. Source: Survey among project coordinators, September 2016, n=31..32 for FP7, n=21.22

for Horizon 2020.

5.1.5 BUILDING RESEARCH, DEVELOPMENT AND INNOVATION CAPABILITIES

The impact on research, development and innovation capabilities is expected to be similar for

Horizon 2020 compared to FP7. This can be derived from a survey among project coordinators,

which is shown in the graph below.

Figure 8: Assessment of impact on research, development and innovation capabilities targeted or

achieved at the end of the project. Source: Survey among project coordinators, September 2016,

n=31..32 for FP7, n=19..21 for Horizon 2020.


5.1.6 COMPETITIVENESS & EXPLOITATION

Almost all projects have undertaken exploitation activities, or are planning to do so. The graph

below shows that direct contact with stakeholder is the most popular activity, both in FP7 and

Horizon 2020.

In Horizon 2020 project coordinators plan to undertake less activities overall to help exploit the

results than in FP7, as can be seen in the graph below. The reasons are unclear, as one would expect

a higher number given the focus on more market-oriented research, and the Expert Group found this

a surprising result.

Figure 9: Average number of exploitation activities targeted or achieved at the end of the project.

Source: Survey among project coordinators, September 2016, n=31 for FP7, n=20 for Horizon 2020.

Despite the slightly lower numbers on activities (inputs), project coordinators expect a slightly higher

impact on competitiveness in Horizon 2020 than in FP7, as can be seen in the graph below.

Figure 10: Assessment of impact on competitiveness targeted or achieved at the end of the project.

Source: Survey among project coordinators, September 2016, n=30 for FP7, n=20..21 for Horizon

2020.


5.2 EXPECTED LONGER-TERM RESULTS FROM THE PROGRAMME

This section explains the expected long-term results based on the outputs presented in the previous

section. One of the main advantages of Horizon 2020 is that it is more focused on implementation of

research results than FP7. It is expected from this focus that more research will be put into the

market. The emphasis upstream (project preparation) is clearly on exploitation and deployment. This

should lead to outputs that matter and avoid the production of ‘vague’ scientific reports that no one

reads. Although FP7 put down the roots for later development, it has not yet yielded concrete

deliverables. Horizon 2020 is seen to be giving a clear push in stimulating innovation.

However in the responses from the stakeholders there are some concerns that the ambition for this

to be improved is clear but that the conditions for implementation may be lacking. Notably:

- A focus on project management rather than content from INEA (i.e. are the deliverables

submitted on time, do they follow the due process and are they in line with the contractual

arrangements?).

- There are many good tools and outputs from FP7 which have not been fully exploited but are

difficult to extract in the present system.

- Experienced participants sense a feeling of ‘déjà vu’ when some calls are published and feel

that these topics have already been successfully researched in previous FPs but the results

have been buried, so that newer staff members of the Commission are not aware of them

(and therefore it is unlikely that anyone else is either).

- The Horizon 2020 format is perceived as too rigid, too fragmented and too short term oriented for innovative research and development, going beyond the classical R&D.

Nonetheless, Horizon 2020 is contributing to strategic cooperation in research going beyond FP7. The

change of mind-set of the participants is perhaps one of the main achievements, by illustrating that

cooperation has added value. Private partners are brought together, and a market place is created

for tools and products that are useful for the European transport scene.

As few projects are finalised it is still unclear to what extent Horizon 2020 will solve big societal

challenges in transport such as air quality, congestion, or safety/security. Despite this at least project

coordinators are more optimistic about Horizon 2020 than on FP7 as can be seen in the results from

a survey in the graph below.

Figure 11: Percentage of projects that expect to contribute to one of the long term goals. Source:

Survey among project coordinators, September 2016, n=32 for FP7, n=22 for Horizon 2020.


5.3 PROGRESS TOWARDS ATTAINING THE SPECIFIC OBJECTIVES OF SC4 -

SMART, GREEN AND INTEGRATED TRANSPORTS

The specific objective of the SC4 Smart, green and integrated transport is formulated as follows:

“Achieving a European transport system that is resource-efficient, climate- and environmentally-

friendly, safe and seamless for the benefit of all citizens, the economy and society.”

The Specific Programme is structured in four broad lines of activities aiming at:

A. Resource efficient transport that respects the environment. The aim is to minimise transport

systems’ impact on climate and the environment (including noise and air pollution) by

improving its efficiency in the use of natural resources, and by reducing its dependence on

fossil fuels.

B. Better mobility, less congestion, more safety and security. The aim is to reconcile the

growing mobility needs with improved transport fluidity, through innovative solutions for

seamless, inclusive, affordable, safe, secure and robust transport systems.

C. Global leadership for the European transport industry. The aim is to reinforce the

competitiveness and performance of European transport manufacturing industries and

related services including logistic processes and to retain areas of European leadership (e.g.

such as aeronautics).

D. Socio-economic and behavioural research and forward looking activities for policy making.

The aim is to support improved policy making which is necessary to promote innovation and

meet the challenges raised by transport and the societal needs related to it.

These activities are addressed in the 2014-201585 Work Programme by three calls for proposals:

1. Mobility for Growth

2. Green Vehicles

3. Small Business Innovation for Transport

Looking to evaluate if the objectives are translated effectively into the calls, a certain technology bias

and imbalance in weighting against the socio-economic aspects of SC4’s objectives can be observed.

The table below shows that with respect to the translation of the objectives into the number of

topics (totalling 53), around half of them (28 topics) mention an expected impact that can be related

to “Resource efficient transport that respects the environment”. The other 3 objectives (or parts of

it) are less frequently mentioned, with “Socio-economic and behavioural research and forward

looking activities for policy making” as the lowest. This was validated in several interviews, (and was

a point picked up in the ECTRI position paper as input to the new WP sent to the Commission June

2016).

The same goes for the translation of the objectives into projects (outputs). 73 of the 128 projects are

in a topic that mentions the environmental issues while only 29 are in a topic that mentions an

expected impact in transport policies. This is also confirmed in interviews where several

interviewees, especially those near the policy making process have pointed out that the connection

with EC priorities in terms of policy implementation is lagging behind in Horizon 2020. Although this 85

The 2016-2017 Work Programme could not be analysed, as the calls are still open. No results are known to

date in terms of number of projects or budgets.


may not part of the Horizon 2020 programme, the process should place more emphasis on the

feedback loops to policy. This is also visible on the participant’s side: policy oriented topics are largely

oversubscribed in numbers of proposals (see further under the chapter on efficiency).

From this assessment of both the Mobility for Growth and Green Vehicles work programmes it can

be concluded that not all of the above objectives are translated evenly in the topics nor the projects.

Table 7: Number of projects in which a specific objective is mentioned as an expected impact, MG and

GV calls 2014 and 2015.

Objective Topics Projects

A Resource efficient transport that respects the environment 28 73 A1 Resource efficient 10 33

A2 Climate friendly 19 48

A3 Environmentally friendly (including noise and air pollution) 18 38

B Better mobility, less congestion, more safety and security 19 49 B1 Better mobility 11 29

B2 Less congestion 7 24

B3 More safety and security 7 19

C Global leadership for the European transport industry 22 53

D Socio-economic and behavioural research - improved policy making 16 29

TOTAL number of projects / topics

(projects or topics can be mentioned under more than 1 objective)

53 128

When distributing the budget (total grants amounting to €770 million for 128 projects distributed

across the four objectives), an even stronger emphasis on the Objective A is visible: 48.4% of the

budget goes to “Resource efficient transport that respects the environment” while only 5.9% can be

allocated to “Improved policy making”.

Budgets are split per topic into the four objectives with percentages based on the emphasis given on

each of the objectives in the topic expected results. This gives a weighted division of the budget.

Figure 12: Division of budgets by specific objectives, MG and GV calls 2014 and 2015.


5.4 PROGRESS TOWARDS THE OVERALL HORIZON 2020 OBJECTIVES

The overall objective of Horizon 2020 is “Contribute to building a society and an economy based on

knowledge and innovation across the Union by leveraging additional R&D&I funding and by

contributing to attaining R&D targets, including the target of 3 % of GDP for R&D across the Union by

2020. It shall thereby support the implementation of the Europe 2020 strategy and other Union

policies, as well as the achievement and functioning of the European Research Area (ERA).”

All stakeholders and participants of the programme agree that Horizon 2020 is a unique programme

and its outcomes would have been very difficult to obtain should it not have existed.

Main expected outcomes that are mentioned in the interviews and surveys are:

• Europe-wide collaboration that could not have happened without the programme. FP7 and

Horizon 2020 have been very effective in bringing researchers and research institutes

together and sharing knowledge. In some countries it might be easier to get national funds

but the added value of the programme is to forge links with other stakeholders that would

otherwise not be possible. This is considered to be of high value.

• European cooperation leads to European rather than national access to markets, and in some

cases global.

• Results would be very difficult to obtain should the programme not exist.

5.5 LESSONS LEARNT/ AREAS FOR IMPROVEMENT

There are numerous lessons but the main lessons learnt are:

- Short-term output activities are seen as being equal or slightly lower than FP7 in terms of

KPIs86.

- Higher TRL levels are now well covered but gaps in research interests of lower levels of TRLs

should now be given more focus.

- Long term objectives of socio-economic and behavioural research and forward looking

activities for policy-making is underfunded and not given enough exposure.

86

The Key Performance Indicators which are particularly relevant for the Societal Challenges are:

• Number of publications in peer-reviewed high impact journals

• Number of patent applications and patents awarded

• Number of prototypes and testing activities

• Number of joint public-private publications

• New products, processes, and methods launched into the market


6 EFFICIENCY

6.1 INTRODUCTION

Efficiency considers the relationship between the resources used by an intervention and the changes

generated by the intervention (which may be positive or negative).

The main question considered was to establish if the effects (benefits) were achieved at a reasonable

cost.

There have been several improvements to the Horizon 2020 process compared to FP7 and the

following have been investigated as part of this interim evaluation:

- The programme’s budgets, area balance, project size and topic description

- The programme’s attractiveness to the right participants

- The processes for the evaluation of proposals and the project management

6.2 OVERVIEW OF FUNDING ALLOCATIONS

This section describes the overall budget, and the split per activity area, instrument (funding scheme)

and topic.

6.2.1 BUDGET OF THE OVERALL PROGRAMME

Up to now (September 2016), the Horizon 2020 SC4 programme has resulted in 512 projects with a

total grant of €1400 million. This includes the MG (Mobility for Growth) and GV (Green Vehicles)

calls, the SME Transport calls, as well as the calls from the JUs (SESAR, Clean Sky and Shift2Rail).

The challenge SC4 in Horizon 2020 (including the JUs) is allocated a budget of €6339 million for the

full period 2014-2020.

Compared with the FP7 budget of €4160 million over the full 7 years (2007-2013), this is a significant

increase, though the two programmes are not fully comparable as some areas have been removed or

added.

In general terms, the majority of those responding to questions87 recognised that the overall budget

is large and that there are no other European or International research programmes that are better

funded than Horizon 2020. At the same time there is a clear wish for additional budget.

The high number of applications for funding transport project shows there is a not only a need but

also a clear interest in this sector. To some extent this is unsatisfactory for those applying for funds

as despite investing time and effort in developing good proposals that may achieve a score of 14/15

points in project evaluation, they still do not get funded. This leads to disappointment and

frustration. It is recommended to find a solution how to improve dealing with high scoring projects in

the next framework programme (see further under 6.3 – Programme’s attractiveness).

87

In the survey, during the interviews and during the stakeholder hearing.


6.2.2 BUDGET PER ACTIVITY AREA

Although the total budget may be considered adequate, and the general feeling is positive, a few

remarks were given during the interviews and in the survey by representatives of certain modes:

- For some areas, the Horizon 2020 budget is the same as in FP7, however elsewhere

in the RTD programmes budgets have been cancelled. For example, this is the case

for urban transport, as the STEER programme has been discontinued, meaning a

decrease in budget overall for urban topics.

- For some thematic areas, the budget is similar or even larger than in FP7, but the

share compared to the other areas has decreased.

- There is a more limited number of multimodal or crossmodal topics which is not in

line with the ambition of integration and new cross-cutting ways of working within

the Commission (a Juncker request). This is further expanded in relation to

objectives in the chapter on effectiveness.

From the data gathered, it can be observed that aviation is allocated the largest share of the budget

(34.1%). Although it received more under FP7 (23.4 Aviation + 18.7 Clean Sky + 9.3% SESAR), it

should be noted that Galileo (11.9%) was included in Transport, and the administration was also

added.

Figure 13: Budget share by activity area, including JUs.

Horizon 2020 project grants up to August

2016 (€1400 million)

Source: CORDA

FP7 Transport (including Aeronautics) budget

breakdown by activities (€4160 million)

Source: Transport Research in the 7th Framework

Programme (2007 – 2013): an overview of programme

implementation statistics


Figure 14: Grants for Horizon 2020 projects up to August 2016, per activity area, including JUs.

Source: CORDA

This also shows up in the number of projects below:

Figure 15: Number of Horizon 2020 projects up to August 2016, per activity area, including JUs.

Source: CORDA

6.2.3 BALANCE BETWEEN INSTRUMENTS AND FUNDING SCHEMES

The balance between RIA, IA, CSA and SME is generally considered good.

Some related remarks expressed by a few of the respondents during the interviews:

- Shared budgets between topics could lead to a topic having zero proposals funded

simply because the scoring of different topics is grouped and ranked into one

budget.

- The system does not encourage enough disruptive research, and favours incremental

research. This has too small a share in the programme and the impact of initiatives

like the European Institute of Innovation and Technology does not begin to compare

with output of universities like Massachusetts Institute of Technology (MIT). It is

considered that this sets European research at a disadvantage globally.


Figure 16: Number of Horizon 2020 projects up to August 2016, per funding scheme and activity area,

including JUs. Source: CORDA

Figure 17: Grants for Horizon 2020 projects up to August 2016, per funding scheme and activity area,



6.2.4 DESCRIPTION OF TOPICS

In general the Horizon 2020 topics are defined in a much broader way than in FP7. Participants in

the interviews, the survey and the stakeholder hearing have mixed views on this but there is

generally a negative bias. It is accepted that, on the positive side, wide topics favour the participation

of many applicants and allow multiple interpretations. Less prescriptive topics open the door to any

idea, which is can lead to innovations. But it is not confirmed that this actually happens. Concerns

about the evaluation of proposals are also expressed by most participants with respect to the broad

interpretation of topics.

In a survey project coordinators consider the less prescriptive topics having a minor positive impact

on the effectiveness of Horizon 2020, although the responses are mixed.

Figure 18: Assessment of the less prescriptive topics on the effectiveness of Horizon 2020 compared to

FP7 from 33 coordinators that participated both in FP7 and Horizon 2020.

Source: Survey among project coordinators, September 2016.

However, some participants in the survey and most interviewees consider this in a negative light for

the following expressed reasons:

- Topics could be more specific, with the requirements being a little bit more prescriptive and

more strategic. This has been quite noticeable compared to FP7 calls and people struggle

sometimes to understand what the Commission may be looking for – again leading to wasted

effort in project preparation. Applicants have the feeling to promise to rescue the world in

order to get their project granted.

- Although broader topics lead to a wide spectrum of proposals, it seems to be that this leads

to proposals that are too big and vague to assess the specific needs of the sectors (as

previously mentioned).

- Frustrations arise because of rejected proposals, as the evaluators do not seem to have been

able to properly assess the benefits of a project. This is a reflection on the need for a strict

number of evaluators and due to the broad and varied interpretation it is unlikely that the

selected experts can fulfil all these roles.

- Difficulty in the selection process. It is difficult to evaluate proposals that may be very

different but still responding to the same topic, thus giving room for subjectivity of the

evaluators. In addition, actually finding/sourcing experts who are able to evaluate a broad

number of interpretations of one topic is seen as being very demanding (reflected in

previous comment).

- Broader topics with less information lead to oversubscription, the success rate decreases.


6.2.5 PROJECTS HAVE EQUAL SIZE

The average size of projects up till now was €4.23 million for the combined IA and RIA projects, which

is slightly lower compared to €4.47 million for the collaborative projects FP7. This is somewhat

contradictory to the more broad topic definition and the expressions of the participants in the

section above.

For CSA this is €1.80 million in Horizon 2020 compared to €1.12 million for FP7, which indicates an

increase.

Details can be found in the figures below.

Figure 19: Average funding per project by funding scheme in Horizon 2020 up to August 2016, in €,


Figure 20: Average funding per project by activity area and funding scheme in Horizon 2020 up to

August 2016, in €, including JUs. Source: CORDA


Figure 21: Average funding per project by PPP and funding scheme in Horizon 2020 up to August

2016, in €. Source: CORDA

Figure 22: Average funding per project by research area and funding scheme, in million €. Source:

Transport Research in the 7th Framework Programme (2007 – 2013): an overview of programme

implementation statistics

6.2.6 ADDITIONAL REMARKS ON TOPIC DEFINITION

In the survey, the clarity of the goals and topics is described as ‘good’ (mark 3 out of 5), for both

Horizon 2020 as FP7 project coordinators. Respondents that participated in both programmes do

not see much difference between them, except for budget adequacy, where Horizon 2020 is

considered slightly worse.

From this In general, the quality/choice of the topics is good, and it is felt by participants that the Info

Days help to clarify detailed remarks and impressions.


Figure 23: Top graph: average rating (0 very poor .. 5 excellent) according to coordinators of 32 FP7

and 21 Horizon 2020 projects. Bottom graph: assessment of Horizon 2020 compared to FP7 from 33

coordinators that participated both in FP7 and Horizon 2020. Source: Survey among project

coordinators, September 2016.

Also, project coordinators consider the multiannual Work Programme having a large positive impact

on the effectiveness of Horizon 2020. It is appreciated that the topics are known ahead, though the

other side of this is that proposal preparation now starts much earlier, leading to higher costs.

Considering this, the helpful Info Days come too late in the process.

Figure 24: Assessment of the Multiannual Work Programmes on the effectiveness of Horizon 2020

compared to FP7 from 33 coordinators that participated both in FP7 and Horizon 2020. Source:

Survey among project coordinators, September 2016.

Though the general feeling is positive, participants made the following remarks during the interviews

and in the survey:

- Topics differ too much in style and type, from very specific to very general. Continuity of

terminology would be helpful.

- There is a certain amount of duplication in the programme.

- No room for urgent topics, nor sufficient flexibility allowed for new developments. It

takes up to 2 years for many projects to go from publication of the work programme to

signed contract, which is considered very long (although it is recognised that time of

project approval to signed contract has been shortened). This makes it difficult to have a

‘close to market’ approach. To obtain good results within a technical research, a project

must be launch in a very rapid manner: in some cases the technical ideas grow old very

quickly. The two-step process is not helping to shorten this.

- Fragmentation of topics and lack of multimodal interest. Compared to FP7 it is

considered that the modal silo approach has become stronger and not weaker in Horizon

2020.


6.3 PROGRAMME'S ATTRACTIVENESS

This section describes the attractiveness of the programme to the participants to the calls in terms

of:

- Participation of the right participants

- Oversubscription on some topics

- Adequacy of funding rules

- The cost of submitting a proposal

- Two-stage approach

6.3.1 ANALYSIS OF THE PARTICIPANTS

Both Horizon 2020 and FP7 have been very effective in bringing researchers and research institutes

together and sharing knowledge and this is widely accepted by the majority of the participants and

stakeholders (validated in the interviews and in the chapter in effectiveness).

Figure 25: Success rate by organisation type in FP7. Source: Transport Research in the 7th Framework

Programme (2007 – 2013): an overview of programme implementation statistics, tables 12 and 13,

page 11.


Figure 26: Success rates (as % of proposals submitted, and as % of budget available) by organisation

type in Horizon 2020 for Societal Challenge 4 'Smart, green and integrated transport' up to date. All

project types (CSA, IS, RIA). Source: CORDA data, 594 projects, 1 October 2016

It was mentioned by several of the participants that the submission deadlines of Horizon 2020 were

often inconvenient (notably end of Summer, early January). It is however not clear if valuable

candidates have not taken part in proposal preparations due to poor timing.

6.3.2 OVERSUBSCRIPTION ON SOME TOPICS

The calls for proposals in FP7 over the period 2007 – 2013 attracted a total of 2656 proposals leading

to 620 grants. This represents a success rate of 23,4% in participations and 25,4% in funding.

For Horizon 2020, 512 grants have been awarded to date, with a general success rate for proposals

to make it to being awarded a grant is around 16%. This includes the SME calls, while two-stage

proposals were only counted when they make it to the second stage. It is noted, that the SME calls

are far below the other calls in Horizon 2020, with an average success rate of only 10.1% of the

proposals that make it to a grant. Leaving out the SME calls, this figure is 23.8% (number of

proposals), which is equal to the FP7 rate.

As mentioned, a 20% average success rate is perceived by the stakeholders as being low for the

effort involved of project preparation. A lack of perceived success works in two ways – on the one

hand only those that can afford to prepare proposals will do so and, on the other, it can compromise

participation and those that may be the most innovative potential partners are not able or interested

in participating, as the gains versus effort ratio is not attractive.


Figure 27: Success rates for Horizon 2020 projects up to August 2016, per funding scheme and activity

area, including JUs. Note that the two-stage proposals are only counted when they make it to the

second stage. Source: CORDA


Figure 28: Success rate by research area in FP7. Source: Transport Research in the 7th Framework

Programme (2007 – 2013): an overview of programme implementation statistics

Although the figures do not really show a drop in overall success rate (when leaving out the SME

calls), all participants and stakeholders are worried about the low success rates of proposals in some

specific topics. Also there are clearly too many proposals leading to oversubscription in some topics,

which is considered a serious problem. A topic might be crucial to bring forward for European

transport research, reflected by the number of proposals submitted, but if only one project is funded

it is unlikely that this will provide the opportunity for excellence in this domain. In particular, this is

considered to be the case for topics that are more multimodal or cross-cutting, like the urban and

socio-economic calls. An extreme case mentioned is the MG-9.2-2014 topic, where 20 proposals lead

to 1 funded project. Even when it was known beforehand that the budget was limited, participants

considered this topic of great importance. Concerns were expressed that Horizon 2020 did not pick

up this message earlier.

To some extent it was felt that this is a result of the already mentioned (section 6.2.4 above)

relatively open formulation of the calls, leading to a wide spectrum of proposals. There is a

contradiction in having wide areas within topics giving the possibility to put forward a lot of different

ideas, and a high level of competition that is not connected to the amount that is available for

funding.

Secondly, the selection procedure is perceived as not being transparent enough by the project

coordinators that participated in the survey and interviews, outcomes being so uncertain; so

participants are less motivated to provide quality proposals. Unsuccessful proposals are given to

chance to ask for a redress. INEA statistics show that of the 416 received proposals, only 4 requests

for a redress were received, of which 2 were not upheld and 2 were upheld but did not lead to re-

evaluation. This shows that participants either acknowledge the result of the evaluation, or that they

do not believe the effort of going into a redress procedure would matter considering that the

available funds are probably exhausted. It is not clear to the Expert Group if one or the other is

predominant.


Considering the SME calls, the low success rate is caused more by the low quality of the proposals

than strict oversubscription. SME proposals are relatively easy to submit (no other partners needed,

call open every 3 months, very broad topics) which might lead to more ‘give it a try’ types of

proposals. This is also partly visible in the data on SME proposals, and a lot of the failed proposals

are resubmitted once or even twice.

When looking at the JUs, it is clear that proposals under SESAR and Shift2Rail topics have a much

higher success rate than those under the general Horizon 2020 framework. On the other hand, Clean

Sky has a low success rate, or formulated in a positive way: a high supply of research proposals

resulting in an oversubscription.

Figure 29: Success rates for Horizon 2020 projects up to August 2016, per funding scheme and PPP.

Note that the two-stage proposals are only counted when they make it to the second stage. Source:

CORDA

A drawback of the low success rate is that some very valuable projects do not get funding. Looking at

the Mobility for Growth and Green Vehicle calls, 301 out of 510 proposals made the threshold88 with

an evaluation score of at least 10/15, but only 167 (55,5%) of them got funded with a total budget of

€862 million. The 134 remaining proposals account for a €787 million budget. This can be

considered as a latent demand for valuable research that will be lost.

The graph below shows the percentage of projects that get funded by their evaluation score.

Obviously, the high scores have a better chance to get funded than the lower scores. However, there

were proposals with a very high mark of 14/15 that did not get funding. On the other hand, some of

the fairly weak proposals with a 10/15 got funded. This is due to the fact that some topics get

oversubscribed while others do not. Also, sometimes topics are grouped into one funding budget,

causing one topic to get more funds than the other if the quality of the proposals significantly differs

88

Overall threshold is not the only indicator. Proposals do not only need to meet the overall threshold of 10/15

but also 3/5 for each of the 3 subcriteria. Proposals may be above the overall threshold but fail on an

individual criterion threshold. The data on the subcriteria was not available.


between topics. This leads to certain topics getting less funds than expected and can cause another

push towards high or low success rates by topics. All this might explain the feeling a lot of

participants have when they describe the evaluation process as a ‘lottery’.

Figure 30: Success rates per evaluation score in Horizon 2020 up to August 2016 for M.G. and G.V.

calls, including JUs. Note that the two-stage proposals are only counted when they make it to the


The figures below show the two worst cases:

- Green Vehicles: 2 out of 10 proposals with 14/15, 2 out of 7 proposals with 13,5/15 and 5 out

of 13 proposals with 13/15 were not funded.

- M.G. socio-economic topics: 2 out of 2 proposals with 13,5/15 and 3 out of 5 proposals with

13/15 were not funded.

Figure 31: Success rates per evaluation score in Horizon 2020 up to August 2016 for the G.V. calls.

Note that the two-stage proposals are only counted when they make it to the second stage. Source:

CORDA


Figure 32: Success rates per evaluation score in Horizon 2020 up to August 2016 for the socio-

economic topics in the M.G. calls. Note that the two-stage proposals are only counted when they

make it to the second stage. Source: CORDA

6.3.3 ADEQUACY OF FUNDING RULES

The simplification of funding rules is overall appreciated as a good and clear step forward compared

to FP7. In particular the fixed overhead is a good idea in Horizon 2020: it leads to a lower

administrative burden and keeps out high overhead percentages some companies claimed in FP7.

However ‘simplification’ of the personnel allowances to a single rate was not appreciated, as there is

a need for different expert level involvements in projects in order to deliver excellence, which now is

difficult to include into projects.

A few other minor critical remarks were formulated by stakeholders and survey respondents:

- Equal funding is unfavourable for universities and SMEs and favourable for public bodies

(which are already funded by taxes via other means).

- Personnel cost rules (what is included and what not) are still too complicated.

- No flexibility left to increase or decrease the funding rates for some topics.

- Some large companies have relatively high overhead costs in reality, and cannot cover all

costs.

- A too large emphasis on IAs and too little focus on CSA and RIAs (or lower TRLs research).

In a survey project coordinators consider the new types of action to have a generally positive impact

on the effectiveness of Horizon 2020, although the response is quite mixed.

Figure 33: Assessment of the new types of actions (mainly IA and RIA instead of CP) on the

effectiveness of Horizon 2020 compared to FP7 from 33 coordinators that participated both in FP7

and Horizon 2020. Source: Survey among project coordinators, September 2016.


6.3.4 THE COST OF SUBMITTING A PROPOSAL

The cost of submitted a proposal is recognised as an investment and is considered to be too high by

many of the project coordinators that participated in the survey and in the interviews. In particular,

looking at topics with a low success rate (below 10%), the effort does not relate positively to the

benefit of getting the grant. If the effort to be put in a proposal averages 5% of the grant amount

(which seems to be fair), for the price of 20 proposals, one can fund a project.

In the survey, the selection process is described as ‘acceptable’ to ‘good’ (marked 2 to 3 out of 5), by

both Horizon 2020 and FP7 project coordinators. Respondents that participated in both programmes

have a mixed to slightly positive feeling about the improvement in Horizon 2020. It should be noted

that this was a survey among project coordinators, thus making it biased to the ‘winners’ point of

view.

Figure 34: Top graph: average rating (0 very poor and 5 excellent) according to coordinators of 32 FP7




Improvements can be made in making the system more clear to participants on the topic, leaving out

unnecessary information that is not relevant to them.

Attention should be paid to smaller companies and research institutes, who have to compete with

bigger companies who have the budgets to preparing a proposal, which is a very expensive and time

consuming activity and is not funded.

Due to the longer periods between the publication of the Work Programme and the call deadlines,

the proposal preparation phase takes longer. Partner networking and proposal writing is a

continuous process now. The drawback of this is that there is no resting period anymore and that it

increases the difficulty for newcomers to be integrated in a project.


6.3.5 TWO-STAGE APPROACH

The cost of submitting a proposal is particularly high for two-stage proposals. The two-stage

approach is designed to allow a lighter application process in the first stage. In principle, a two-stage

approach seems sound. However, almost all of the participants and stakeholders in the interviews

recommended to stop using the two-stage approach or to dramatically change it, and it is not seen as

an improvement of the FP7 process. It takes as much time to put together the first of the two stages

as is needed to cover the majority of tasks for delivering the whole project – but due to the time

frame all momentum is lost and the final proposal needs to be completely revisited for the

preparation of the second stage.

This is confirmed in the survey project coordinators, who consider the two-stage approach to have a

negative impact on the effectiveness of Horizon 2020 – see graph below.

Figure 35: Assessment of the two-stage evaluation on the effectiveness of Horizon 2020 compared to

FP7 from 33 coordinators that participated both in FP7 and Horizon 2020. Source: Survey among

project coordinators, September 2016.

The stakeholders stated the following arguments against the two-stage approach, which the Expert

Group agrees with from their own experience:

1. Double work for participants as in order to be successful in stage 1, it is necessary to invest

already a lot of money and effort in creating a good proposal and a solid consortium. This makes

stage 1 already a full proposal with a high investment and an extremely low success rate.

2. It encourages a lot of small (low quality) opportunistic competition in the first round.

3. Less oversubscription and no need to have a two-stage process if the scope of the topics were

more precise and better defined.

4. Different evaluation experts in the two stages cause a lack of coherence between evaluations

(inconsistent scores for the different stages). A proposal getting low marks in stage 2 should

already have been flagged in stage 1.

5. It lengthens the time from call to grant, while one of the continuous goals in the programme is to

shorten and ease the process.

6. The success rate in stage 2 (see graph below) does not differ much from a success rate for a

single stage proposal (it is slightly higher in 2014, equal in 2015). However, there were many

more proposals in stage 1 than in stage 2. There is a massive hidden oversubscription in stage 1,

leading to large administrative costs, both on INEA’s and on the applicants’ side. It is thought

that leaving out the second stage could lead to an increase in proposals that have to go through

the full evaluation, but looking at the numbers in the one-stage topics, this would be only a small

increase and there would be significant other gains.


Figure 36: Success rates for one and two stage topics in the 2014 and 2015 M.G. calls of Horizon

2020, including JUs. Note that the two-stage proposals are only counted when they make it to the


6.4 COST-BENEFIT ANALYSIS

Horizon 2020 is as a much ‘closer to the market’ programme than FP7, which is valued by the

participants and stakeholders. The take up and transfer of mature technologies is well appreciated.

As mentioned, it is too early to assess whether the resources are reasonable in light of the results

that have been or are likely to be generated. However, in terms of value for money some interesting

discussions came up around the interpretation of value.

Cost benefit looks at the ratio of cost to benefit in purely financial terms, yet many of the participants

and stakeholders clearly stated that from their perspective the main value of taking part in European

Research projects was in the collaborative exchanges, the learning and the opportunity to work on

major challenges from a multicultural and multidisciplinary perspective. This value should possibly be

more formally recognised and perhaps monetised. In addition it was felt that the cross-cutting and

independent nature of European research should also recognise the value of failure: not all projects

should be designed to succeed but to explore different routes and potential solutions. Under the

strict obligation to ‘succeed’ it is likely that innovation will be overlooked. This feedback came from

interviews and the stakeholder hearing.


6.5 OTHER ISSUES RELATED TO EFFICIENCY

The new management by INEA is considered neutral to positive compared to FP7.

Three issues emerge:

- Evaluation process & selection criteria

- Length of process

- Administrative burden

6.5.1 EVALUATION PROCESS & SELECTION CRITERIA

There are some complaints by the project coordinators about the evaluation procedure in Horizon

2020, though none of the stakeholders’ experiences are seen as being worse than FP7. In general, it

is considered a fair process though some improvements are needed.

The main concern is that experts doing the evaluation are not informed enough, or have insufficient

expertise to do the job. This is partly explained by the wide interpretation of the topics, which

obviously requires a broader set of expertise to evaluate. A proposal may require an evaluator to

have scientific knowledge, practitioner experience and/or state-of-the-art business or industry

understanding. Together with gender and nationality balance, it is not easy to find experts with no

conflict of interest. As mentioned, broad topics set out in the call make it also difficult to evaluate

proposals that may be very different.

In addition to other concerns about the two-stage proposals, there are different experts in stage 1

and stage 2. These sometimes have different opinions on the proposal leading to a lack of coherence

in the evaluation and thus to a loss of trust in the process by the applicants. Obviously, the

evaluation criteria are wider in stage 2 than in stage 1: e.g. implementation is only assessed at stage

2. Also, in case of a two-stage proposal there is almost no feedback to applicants after the first stage.

In the survey, the selection process is described as ‘acceptable’ to ‘good’ (marked 2 to 3 out of 5), for

both Horizon 2020 as FP7 project coordinators. Respondents that participated in both programmes

see a slight improvement in Horizon 2020. Note that this was a survey among project coordinators,

thus making it biased to the ‘winners’ view on the picture.






6.5.2 LENGTH OF PROCESS

When competing with research outside Europe, the shortest possible process is the best. Horizon

2020 is considered already significantly better than FP7. The time to contract has notably been

improved but the two-stage process is still seen by the participants89 as taking too long. For projects

with requested high TRLs the market opportunities might have already shifted after that long period

of time.

However, the Expert Group considers it beneficial for the applicants that the work programmes are

published two years ahead so applicants are able to prepare and know what is coming.

Areas for improvement that were stated by the stakeholders and agreed with by the Expert Group:

- Before publication of the call, the process should be speeded, up in particular when it

comes to work programme definition and publication. Information required as input in

the definition phase of the next work programme is not sufficiently available.

- The time from publication of the call text to grant agreement could be reduced with a

shorter evaluation process (which is now 5 months). Especially the time between the

delivery of evaluation expert results and the annotation of the results to the applicant.

- Either the two-stage process is abolished or changed as it now can take 24 months from

initial call to contract signing and starting research

- The non-negotiation clause, which is new in Horizon 2020, is seen to possibly save time

from the Commission’s perspective but it comes at a price: once a project is selected,

adaptation is only possible during the contract, with amendments and does not seem to

have substantially reduced the contracting time overall (from call to contract). Some

participants experience the grant preparation process as ‘hurried’ with a resulted project

start that was not optimal. Adapting the number of deliverables is seen as being

89

Both in the survey and the interviews.


especially necessary. In a sample of 125 projects, in total 45 such amendments were

made: small reorientations, demonstrations that cannot be implemented, issues

discovered during contract negotiation.

In the survey, the project management is described as ‘acceptable’ to ‘good’ (marked 2 to 3 out of 5)

by both Horizon 2020 and FP7 project coordinators. Respondents that participated in both

programmes see a slight improvement in Horizon 2020, mainly on time between submitting a final

proposal and signing the grant – which is logical as there is no negotiation phase anymore.





6.5.3 ADMINISTRATIVE BURDEN

The management of the programme has been outsourced to the new INEA agency since the

beginning of Horizon 2020. Besides Horizon 2020, INEA also manages CEF and Marco Polo.

There has been significant effort and investment to simplify the processes, which is recognised and

appreciated by both the participants and the EC. In some cases Horizon 2020 is now considered less

bureaucratic than many national research programmes. The new participants’ portal has been

reworked but it is not considered to be entirely successful. There are still some user interface issues

as it is still quite complex and not considered to be intuitive; it requires quite some experience to

navigate it. An example is that the grant agreement preparations require to copy paste all

information manually from the proposal, while one would expect this to be automatic. The portal

can be very slow, and has unnecessary pop-up windows. Especially SMEs and smaller research

entities lack this experience and find uploading technically challenging and time consuming.

A weakness, pointed out by several project coordinators, is that the split between the management

(at INEA) and the content and programme definition (at the DGs and JUs) leads to a longer feedback

loop from on-going and finished projects to the new work programmes and towards EC policies in

general. Similarly, a scientific or strategic follow up of the project by EC project officers during the

project, complementary to the management, might turn out to be necessary to keep the loop closed.


In the survey, the project management is described as ‘good’ to ‘very good’ (marked 3 to 4 out of 5),

by both Horizon 2020 as FP7 project coordinators. Respondents that participated in both

programmes see an improvement in Horizon 2020. Despite the criticism mentioned above, Horizon

2020 was considered to be better than FP7 with regard to the participant portal, the grant

preparation process and the monitor and reporting procedures.





6.6 LESSONS LEARNT/ AREAS FOR IMPROVEMENT

Approaches that could be considered to generate further efficiency gains:

- Topics are too broadly defined and should be made more precise.

- Change the two-stage proposal submission either by reverting to a single stage or

refining the calls so they are more precise and setting clear time frames for the

whole process

- Process from call to contract is still considered to be too long (although already

improved compared to FP7)


7 COHERENCE The Expert group has approached coherence of the Horizon 2020 Transport Societal Challenge

according to the following lines:

1. Internal coherence of Horizon 2020 Transport

- Coherence among instruments

- Coherence of actions implemented in transport under Horizon 2020 and, in

particular, with regard to PPPs

- Coherence with other Societal Challenges of Horizon 2020

- Coherence among other areas of Horizon 2020, from the ERC, EIT-KICS…

2. External coherence of Horizon 2020 Transport actions in relation to:

- Coherence with other funding mechanisms of the EU (structural funds, European

Investment Back, among others)

- Coherence with national and regional funding programmes for innovation.

7.1 INTERNAL COHERENCE

7.1.1 INTERNAL COHERENCE OF INSTRUMENTS TO IMPLEMENT HORIZON 2020

TRANSPORT

Currently, Horizon 2020 transport calls include the following list of main instruments.

Table 8: Horizon 2020 Transport instruments

General instruments Focused instruments Complementary instruments

Research and Innovation Action

(RIA)

Innovation Action (IA)

Coordination and Support

Action (CSA)

Joint Undertakings / PPP

SME Instrument(s)

Fast Track to Innovation (FTI)90

ERA-NET (Cofunded)

Inducement prize

Public Procurement for

Innovative Solutions (PPI)

Cofunded91

Provision technical services

Expert contracts

The instruments for funding for basic or general research, as well those that are more focused ones

such as JUs, PPPs, the SME Instrument or the ERA-NET Cofund92 and supporting tools related to

Public Procurement in Innovative Services (PPIS)93, tenders and expert involvement, show a cohesive

framework. None of the interviewees indicated that this was not the case. This is also the opinion of

the Expert Group. On the other side, a small number of participants in interviews and the hearing

90

https://ec.europa.eu/programmes/Horizon2020/en/Horizon 2020-section/fast-track-innovation-pilot 91

http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/annexes/Horizon 2020-

wp1415-annex-e-inproc_en.pdf 92

http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/annexes/Horizon 2020-

wp1415-annex-d-eranet-cofund_en.pdf 93

http://ec.europa.eu/research/participants/data/ref/Horizon 2020/other/wp/2016-2017/annexes/Horizon

2020-wp1617-annex-ga_en.pdf

https://ec.europa.eu/programmes/horizon2020/en/h2020-section/fast-track-innovation-pilot

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-e-inproc_en.pdf

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-e-inproc_en.pdf

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-d-eranet-cofund_en.pdf

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-d-eranet-cofund_en.pdf

http://ec.europa.eu/research/participants/data/ref/h2020/other/wp/2016-2017/annexes/h2020-wp1617-annex-ga_en.pdf

http://ec.europa.eu/research/participants/data/ref/h2020/other/wp/2016-2017/annexes/h2020-wp1617-annex-ga_en.pdf


referred that the reason for deciding whether an IA or a RIA instrument was used in a specific topic

was not always clear and this could therefore be improved.

With regard to the 55 project coordinators surveyed for this interim assessment, the outcome is not

so conclusive, although answers tend to be more positive than negative as the following tables show

(the high percentage of “I don’t know” responses should be noted).

Table 9: Complementarity of the different mix of available instruments (in the case of Horizon 2020,

JU/PPPs, IA, RIA, CSA, SME instrument, etc.)

0

(non-existing)

1 2 3 4 5

(at a 100% degree)

I don´t know

+ No answer

Complementarity

among instruments?

0% 4% 6% 13% 17% 0% 61%

Synergy among

instruments?

0% 7% 4% 9% 13% 2% 65%

Table 10: Overlaps and gaps among the different mix of available instruments

0

(non-existing)

1 2 3 4 5

(at a 100% degree)

I don´t know

+ No answer

Overlaps among instruments? 6% 0% 6% 0% 4% 0% 85%

Gaps among instruments? 0% 0% 2% 2% 7% 4% 85%

The following individual comments were indicated when a low scoring (between 3 and 5) was

indicated in the previous table:

- Topics in CS2 and Horizon 2020 often overlap (e.g. strong ecological impact in both areas)

- There is bias towards higher technologies with the research programmes, and less emphasis

on what may actually be needed to solve the challenges of transport especially within SC4.

This could be tackled in a low technology or policy way, but often the opportunities to do so

are missed.

- Gaps between available instruments between low TRL (1-3) to high TRL (5-6).

There is a general acknowledgement (from interviews and survey) of the advantages of the PPPs.

They are perceived by stakeholders during the interviews as part of a coherent set of instruments in

areas with clear focus and need for higher TRLs. The PPPs are in particular considered a useful

instrument for reaching higher TRLs in connection with specific technological challenges. However a

small number of stakeholders (mainly from outside the PPPs) suggested the need for greater

coordination across the various transport initiatives (i.e. how each PPP relates specifically to the

other and to the regular Horizon 2020 programme): stronger links between air and surface PPPs, for

example, and opportunities for best-practices exchange. A table with positive characteristics and

areas of improvements of JUs and PPPs is included in the European Added Value in section 8 of this

interim assessment report.


The following table shows the current Joint Undertakings and contractual Public-Private Partnerships

(cPPPs) supported by the Horizon 2020 Transport budget94 or with direct transport applications:

Table 11: Chart of Joint Undertakings and contractual Public Private Partnerships fully or partially

supported by the Horizon 2020 Transport Societal Challenge

Joint Undertakings and contractual Public-

Private Partnerships (cPPPs) funded through the

Horizon 2020 transport budget

EC Contribution

(€ million)

Total Costs

(€ million)

Crowding-in

Effect

Single European Sky ATM Research (SESAR) 2020 500 1500 300%

Clean Sky 2 (CS2) 1800 4000 222%

Shift2Rail (S2R) 450 920 204%

European Green Vehicles Initiative (EGVI) 750 1500 200%

Total 3500 7920 226%

Other Joint Undertakings with direct

applications to Horizon 2020 transport

EC Contribution

(€ million)

Total Costs

(€ million)

Crowding-in

Effect

Fuel Cells and Hydrogen 2 (FCH2) 665 1300 195%

Electronic Components and Systems for

European Leadership (ECSEL)

1180 5000 424%

Total 1845 6300 341%

Without considering the budget reserved for institutional PPPs (Clean Sky 2, SESAR, Shift2Rail and

Fuel Cells and Hydrogen 2), RIA and IA account for around 75% of the budget available for calls

(including topics with the European Green Vehicles Initiative)95. More specifically, IAs represent

25.9% of the total budget while RIAs amount to 50.0% of the total work programme budget (and an

additional 0.2% is allocated to topics proposed for both IA and RIA).

This percentage largely corresponds to the one reported by the European Commission as part of its

cross-cutting KPIs reporting. Based on the EC’s data, the share of EC contribution to Innovation

Actions (IA) in signed grants represents a 29.4% of the total (for the entire Horizon 2020, this

percentage is 16.8%)96.

According to this data, the percentage of EU financial contribution focussed on Demonstration and

Piloting activities within Innovation Actions (IA) reaches 50.3% in the case of Transport while for the

entire Horizon 2020 reaches 77.8%. Although the percentage for Transport is lower than the average,

it is clear that half of the budget focussing on Demonstration and Piloting activities is still a

remarkably high percentage to this type of activities.

94

http://ec.europa.eu/programmes/horizon2020/en/area/partnerships-industry-and-member-states 95

Own analysis of work programmes conducted for this interim assessment. See annex at the end of this

document. 96

Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016.

http://ec.europa.eu/programmes/horizon2020/en/area/partnerships-industry-and-member-states


ERA-NETs cover an important link between European Commission actions and national or regional

levels. ERA-NET actions are designed to be particularly suited for the involvement of small actors and

national, regional or local public administrations, helping to facilitate the adaptation and deployment

of innovations. However there is a gap between the objectives and ambition of ERA-NET and the

implementation that is reflected in its low take up level. This is clear when compared with other large

projects (for instance, the FP7 ERA-NET Electromobility+ pooled together some €20 million from the

participating countries and regions as well as from the EC97 and this budget for the entire ERA-NET is

comparable to the EC contribution to a single large innovation action).

It was mentioned on several occasions by stakeholders that the SME instruments including the Small

business innovation research for Transport call are seen as being successful. Respondents refer to the

SME instrument as complementary to the SME participation target in conventional Horizon 2020

calls: SMEs in the SME instrument playing the central role and considering closer to market activities

while participation in a consortium provides more opportunities for networking and becoming part

of larger value chains, for example.

As a general conclusion of this subsection, and based on document analysis and interactions with

stakeholders (interviews and hearings), the Expert Group considers that overall the structure and

goals of the instruments are seen as being coherent with the ambition of Horizon 2020 transport,

especially SC4 and the stated objectives of the Union for transport research.

7.1.2 INTERNAL COHERENCE WITH OTHER HORIZON 2020 INTERVENTION AREAS

Transport is a cross-cutting issue and there are numerous cross-references to transport research and

innovation in other Horizon 2020 intervention areas.

Furthermore, as expressed in the Energy Union paper, inter-sectorial issues requiring research and

innovation and contributing to the EU's competitiveness need to be jointly addressed in order to

identify the links, synergies and trade-offs. This is of particular importance for the production,

storage and use of alternative energies and fuels (including electricity and biofuels)98 used for

transport especially in respect to decarbonising transport and shifting it away from fossil fuel use.

The general calls of Horizon 2020 Transport include references to other calls and initiatives that are

relevant for the Transport Challenge, e.g. the calls ‘Blue Growth’, and ‘Smart Cities and

Communities’. This is intended to create synergies with other parts of Horizon 2020 such as Space,

the ‘Secure societies’ Challenge and generic research on materials and manufacturing techniques.

The latter are dealt with in the Leadership in Enabling and Industrial Technologies (LEIT) part of

Horizon 2020 under Nanotechnologies, Advanced Materials and Advanced Manufacturing and

Processing (NMP) and Factories of the Future. Recycling of materials and replacement of critical raw

materials are normally addressed by the ‘Climate action, resource efficiency and raw material’

societal challenge. A number of actions related to the production of alternative fuels for aviation or

advanced energy storage systems and interfaces between vehicles and energy recharging

infrastructures can also be found under the ‘Secure, clean and efficient energy’ challenge.

97

http://electromobility-plus.eu/wp-content/uploads/Electromobility+overview_Apr2014.pdf 98

Internal European Commission’s document that has been made available to the Expert Group.

http://electromobility-plus.eu/wp-content/uploads/Electromobility+overview_Apr2014.pdf


Clear indications are given in the Work Programmes of where these opportunities are and a good

example in the 2014-2015 Work Programme is the following reference: “In addition to the topics of

this call, a topic on post lithium ion batteries for electric automotive applications (NMP 17 – 2014) is

included in ‘Nanotechnologies, Advanced Materials and Advanced Manufacturing and Processing

(NMP)’ under “’Leadership in Enabling and Industrial Technologies’ (LEIT)”. The 2016-2017 Work

Programme explains, for instance, that “transport-related actions are also included in other parts of

Horizon 2020 Work Programme 2016-2017, particularly in the LEIT/NMBP call 'Nanotechnologies,

Advanced Materials, Biotechnology and Production'; LEIT/Space call 'Applications in Satellite

Navigation – Galileo'; and SC/Energy calls 'Competitive Low-Carbon Energy' and 'Smart Cities and

Communities'”. The development of batteries is a good example of collaboration within the EC

among different Societal Challenges of Horizon 2020: NMP, Energy and Transport. This internal

collaboration in the EC is performed via the Challenge Group consultation at programme level. Since

the information about this working group does not yet reach the research community, they refer to a

lack of coordination among challenges which may not entirely be the case but that there are gaps in

communication. It is therefore recommended that Horizon 2020 internal collaboration is better and

more widely communicated.

Another concrete example of opportunity for transport research and innovation is found is the

following EE-22-2016-2017: Project Development Assistance of the Energy programme99: This is

designed to help build the technical, economic and legal expertise needed for project development

with public and private actors that will lead to concrete investments.

The PDA focuses on the sectors of existing public and private buildings; street lighting;

retrofitting of existing district heating/ cooling; energy efficiency in urban transport (such as

transport fleets, the logistics chain, e-mobility, modal change and shift) in urban/suburban

agglomerations and other densely populated areas and energy efficiency in industry and

services.

In a recent survey conducted on behalf of the European Commission as part of the general Horizon

2020 Interim Assessment exercise, project coordinators were asked to assess whether their projects

were expected to have impact on the societal challenges in the next 10 years (with a ‘Yes/No’

answer). The original table included in that report is reproduced below100. Societal Challenge 4

Transport has been highlighted with a yellow background. As can be seen, respondents indicated

that transport projects will have the largest impact or contribution in the following Horizon 2020

areas: SC5 “Climate Action, Environment, Resource Efficiency and Raw Materials” (62.0%) and SC3

“Secure, clean and efficient energy” (36.5%). Around one fourth of project coordinators indicated

that their project will also impact SC1 “Health”, SC6 “Inclusive Societies” and SC7 “Secure Societies”.

Only 9.3 of Transport projects, according to this survey, would impact SC2 “Food”.

99


wp1617-energy_en.pdf 100

PPMI. FRAMEWORK CONTRACT FOR THE PROVISION OF SERVICES TO THE COMMISSION IN THE FIELD OF

EVALUATION OF RESEARCH AND INNOVATION PROGRAMMES AND POLICIES (2012/S 144-240132). OVERVIEW

OF Horizon 2020 SURVEY RESULTS. Internal European Commission document made available for the purposes

of this interim assessment.

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-wp1617-energy_en.pdf

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-wp1617-energy_en.pdf


It is also remarkable that according to this survey, transport is placed as the Societal Challenge with

smallest contribution from the Excellent Science part of Horizon 2020, with the exception of the

Space project coordinators, who mention the transport area most frequently as contributing to

excellence.

Table 12: Expected impact on societal challenges within the next 10 years

Question 15- Grid question with ‘Yes’/’No’ answer categories

Horizon 2020 section SC1 SC2 SC3 SC4 SC5 SC6 SC7

Excellent Science

Future and emerging technologies (n

= 16) 33.3 % 40 % 57.1 % 20 % 52.4 % 30 % 25 %

Research Infrastructures (n = 27) 52.2 % 52.4 % 23.8 % 18.2 % 43.5 % 40.9 % 36.4 %

Industrial leadership

NMPB (n = 96) 42.4% 29.0% 52.6% 23.2% 61.9% 18.0% 14.6%

Subtotal within NMPB: PPP projects

(n=32) 25.9% 7.0% 69.6% 19.8% 68.3% 23.4% 9.7%

Information and Communication

Technologies (n = 177) 52.0% 21.5% 32.2% 34.5% 30.0% 55.8% 38.5%

Space (n = 36) 28.2% 31.4% 33.1% 52.3% 44.0% 29.0% 50.6%

Innovation in SMEs (n = 30) 24.4% 24.3% 26.8% 19.9% 19.9% 26.0% 21.5%

Societal Challenges

SC1 (n = 106) 98.1% 9.8% 1.7% 2.1% 5.3% 35.6% 9.6%

SC2 (n = 43) 49.2% 98.6% 21.4% 4.6% 86.4% 25.8% 14.2%

SC3 (n = 124) 21.4% 19.0% 97.5% 34.2% 86.6% 29.4% 17.7%

SC4 (n = 96) 26.1% 9.3% 38.5% 96.1% 62.0% 28.9% 23.4%

SC5 (n = 71) 39.2% 57.9% 57.9% 28.5% 95.7% 34.5% 26.0%

SC6 (n = 32) 53.6% 16.5% 17.5% 20.7% 32.5% 90.2% 35.9%

SC7 (n = 31) 38.6% 33.3% 25.7% 36.2% 30.2% 53.1% 93.3%

Spreading Excellence and Widening participation + Science with and for Society + other programmes

Spreading Excellence and Widening

Participation (n = 24) 64 % 44 % 52 % 26.9 % 44 % 51.9 % 35.7 %

Science with and for Society (n = 10) 57.1 % 50 % 50 % 50 % 57.1 % 87.5 % 42.9 %

Fast Track to Innovation Pilot (n = 10) 66.7 % 33.3 % 50 % 33.3 % 66.7 % 33.3 % 0 %

Euratom (n = 3) 33.3 % 0 % 100 % 0 % 50 % 33.3 % 33.3 %

Total 46.9 % 29.4 % 41.6 % 32.8 % 50.9 % 38.6 % 27.2 %

Total number of valid responses 920 905 914 906 909 911 902

As part as the present Horizon 2020 Transport interim assessment another survey among project

coordinators was run by the EC and included the question of what other Horizon 2020 research areas

or programmes also represent an opportunity for transport. The following table shows the ranking of

areas associated with higher opportunity, complementarity or synergies (percentage indicates the

sum of percentages of respondents that indicated opportunity, complementarity or synergies) across

a number of sectors.


Table 13: Ranking of areas associated with higher opportunity, complementarity or synergies with

Horizon 2020 Transport project results. Analysis of the answers to the project coordinator survey

ICT Research & Innovation 29%

Innovation 27%

Energy 26%

Environment & Climate Action 26%

Research Infrastructure 25%

International Cooperation 23%

Partnerships with Industry and Member States 23%

Key Enabling Technologies (including NMBP) 22%

SMEs 22%

Security 20%

Funding Researchers 19%

Raw Materials 13%

Bio-based Industries 12%

Agriculture & Forestry 11%

Health 10%

Space 10%

Aquatic Resources 6%

Social Sciences & Humanities 6%

Society 6%

Biotechnology 4%

Food & Healthy Diet 2%

Project coordinators were asked whether there are enough information and coordination

mechanisms within those other areas of Horizon 2020 that are complementary or synergetic for

transport. Most of the respondents agreed that there are enough information and coordination

mechanisms, as the following table shows.

Table 14: Are there enough information and coordination mechanisms within those other areas of

Horizon 2020 that are complementary or synergetic for transport? Analysis of the answers to the

project coordinator survey.

Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don´t know

4% 30% 39% 9% 13% 4%

This rather positive picture obtained from the project coordinators survey does not fully coincide

with the opinions of some interviewees, who indicated concerns about the amount or accessibility of

information regarding other funding opportunities either within other areas of Horizon 2020 or

outside Horizon 2020. One reason for this apparent discrepancy may be the different nature of

respondents (project coordinators in the survey generally agreeing but technology platform

representatives acknowledging during the interviews that they seldom deal with other areas of

Horizon 2020) and the fact that during interviews, additional comments were able to surface and

become more relevant. Another explanation, probably more important, is that while the question to

project coordinators answered a question specific about internal coherence within other areas of


Horizon 2020, interviews often mixed comments with regard to external coherence or coordination,

Anyway, as almost a quarter of the project coordinators surveyed also indicated that they either

rather or fully agreed that there are not enough information and coordination mechanisms this

indicates in the opinion of the Expert Group that, even if project coordinators tend to have a positive

opinion on this matter, opportunities for better dissemination still exist. One way of increasing

internal coherence awareness could be the periodic publication of some sort of “transport across

Horizon 2020: from ERC to the Societal Challenges and to the EIT” guide with examples and

opportunities for the transport sector.

Project coordinators responding to the Expert Group survey also generally agreed that projects will

produce interdisciplinary solutions, which cut across multiple specific objectives of Horizon 2020 (for

instance, new batteries for electric vehicles or light-weight material applications for high-speed

trains), as the following table shows.

Table 15: Will the project produce interdisciplinary solutions, which cut across multiple specific

objectives of Horizon 2020? Analysis of the answers to the project coordinator survey


17% 26% 22% 13% 7% 15%

Some of the cross-cutting examples provided by the project coordinators are:

Lighter aircraft components and repair concepts that can be transferred to other transport

modes.

New antifouling coatings for marine transport, but also adoptable to static constructions (e.g.

renewable marine energy)

Expertise in propulsion and aerodynamics applicable for all other disciplines in high-speed

trains, automotive, aircraft

Application of robotics platforms to waterborne sector

Development of a technology that can be implemented in the design and production of

lightweight UAV machines and/or in the "health" monitoring of aerospace structures

Aero-servo-elastic research touching flight control, aircraft structures and aerodynamics,

which could be interesting for other industries like wind energy also.

Lightweight material applications due to innovative and composite processes for surface

transport and aerospace that are applicable to other sectors.

High efficient combustion engines that will be applied in different truck applications, but also

potential for off-road application.

New motor topology applicable to EV-s and industrial applications.

The activity will address both energy efficiency and air quality aspects.

Flexible low cost forming technology, machine learning algorithm applied to manufacturing

technologies.

When project coordinators were asked about the EC’s diversification of research and innovation

programmes (for example research infrastructures supported within the European Strategy Forum

on Research Infrastructures ESFRI, or the fundamental research supported by the European Research

Council ERC projects, Knowledge and innovation Communities launched by the European Institute for


Technology EIT…) they tended to indicate that there was little duplication and that the actions were

mostly complementary and synergetic. The results of the survey are shown in the table below.

Table 16: Results of survey on level of duplication of topics

0

(non-existing)

1 2 3 4 5

(at a 100% degree)

I don´t know

+ No answer

Duplication among actions 13% 13% 11% 7% 55%

Complementarity among

actions

11% 17% 15% 6% 52%

Synergy among actions 4% 7% 20% 15% 4% 50%

7.1.3 ENSURING THAT EVERY EURO SPENT COUNTS TWICE

The results of the analysis of the budget repartition per topic among the different lines of activities,

as shown in the following table (source: Expert Group analysis of topics) are of particular interest

here.

Table 17: Analysis of the budget repartition per topic Cross-cutting refers mostly to “global

leadership” + “resource efficiency”

Line of activity WP 2014-2015 WP 2016-2017 Total

Better mobility 231.2 26.2% 112.3 15.5% 343.5 18.9%

Cross-cutting 414.0 47.0% 514.2 46.5% 928.2 50.9%

Global leadership 55.3 6.3% 115.3 7.0% 170.7 9.4%

Resource efficiency 105.2 11.9% 124.0 12.7% 229.2 12.6%

Socio-econ. & forward looking 75.8 8.6% 72.5 18.3% 148.3 8.2%

Total general 881.5 100.0% 938.4 100.0% 1819.9 100.0%

As can be seen in the table above, half of the budget is directed into topics addressing several

general objectives, mainly global leadership and resource efficiency. In other words: leadership

through efficiency. As long as this combination is supported by strong legislation requiring products

and services to be efficient, it should support competitiveness and leadership of the European

industry (and not additional costs for guaranteeing efficiency), however the concerns over the links

between research outputs and policy that have been previously mentioned should be noted.

As the Horizon 2020 Transport Work Programme 2016-2017 explains, all new priorities were related

to one or more of the four broad lines of activities defined in the Specific Programme. One excellent

example of both competitiveness and societal objectives being coherently addressed by Transport

research and innovation in Horizon 2020 Transport is the Automated Road Transport call in Work

Programme 2016-2017:

“Automated Road Transport holds the promise to help address many of the major challenges

of today's transport system, such as user safety, energy efficiency, air quality and congestion,

and to enhance the drivers' individual comfort and convenience. At the same time, it

represents a critical testing ground for the ability of the European automotive industry to

preserve and consolidate its global leadership”.


Other examples of multiple objectives are the institutional PPPs themselves. With the PPPs, the

cross-cutting nature of the players involved provides a level of multidisciplinary exchanges that

would not otherwise occur. It is more difficult for the Expert Group to dig deeper into challenges

such as the interaction between environment and transport policy with competitiveness issues of the

European industries within the PPPs. For instance, the Clean Sky Joint Undertaking and its successor

programme in Horizon 2020 Transport target primarily the reduction of the impact on the

environment through integration and validation of existing technologies up to Technology Readiness

Level 6. For that purpose, it will also include the type of work, which was carried out in Integrated

Projects under the Seventh Framework Programme.

The growing number of PPPs demonstrates that they are seen as a successful instrument. As they

include the private sector and there is significant co-funding it is considered that they are also

helping to position European Research to be more competitive, closer to the market and to help

bridge the gap between applied research, commercialisation and new business models, i.e. in other

words to be coherent with the overarching ambitions and goals of the Union. They are also seen as

serving a useful purpose in avoiding isolated actions and linking of projects. PPPs/JUs help to

generate a stronger community of research actors around one area and they cluster resources and

talent around well-defined topics. Indeed the waterborne sector is keen to create such a partnership

for their sector as they see it being able to deliver more and better continuity of research.

In the case of SESAR, a combination of optimization or air capacity and the development of new

technologies are pursued. The SESAR Joint Undertaking develops solutions for a seamless, efficient

and cost effective management of air traffic, including services of European GNSS and covers the full

range of TRL from 1 to 6. SESAR represents a clear example where both companies benefit from the

opportunity of researching and developing new generation of world-class competitive air traffic

management products and services and, simultaneously, citizens see how their travel time is reduced

and the prize of their air tickets is diminished.

Similar dual benefits can also be attributed to other transport innovations such as eCall for

automotive or the ERTMS for rail whose development was significantly supported by the European

research and innovation framework programmes. Also the European Green Vehicles Initiative will

improve at the same time competitiveness of the automotive industry, among other industries, and

improve quality of life in cities (less noise, less pollution…).

As part of this interim assessment a survey among project coordinators found strong agreement

when they were asked whether it was possible to justify that the project was going to simultaneously

tackle societal challenges (i.e. supporting mobility of the elderly while, at the same times, creating

new business opportunities) and give rise to new competitive businesses and industries. Results are

shown in the table below.

Table 18: Is the project going to simultaneously tackle societal challenges (i.e. supporting mobility of

the elderly while, at the same times, creating new business opportunities)? Analysis of the answers to

the project coordinator survey.

Totally

agree

Agree Rather

agree

Rather

disagree

Disagree Totally

disagree

I don’t know + No

answer

17% 33% 35% 2% 6% 0% 7%


The following comments were offered by those who agree to some extent:

Lowering emission of GHG, and creating new opportunities for European businesses. Spin offs

are currently undertaken.

Contribution to decreased CO2 emissions, improved marine environment, while creating

increased competitiveness of European paint industry

The high-speed aircraft development would fundamentally change the mode of operation at

airport, for business and industries and increase the time efficiency now lost during travel.

Meanwhile the use of hydrogen will open new potentials in all other areas where (stored)

energy is needed e.g. automotive...

The project was to identify future road safety research needs, thus tackling one of the

greatest societal challenges with more than 20,000 fatalities on Europe’s roads each year. At

the same time, safety innovations which will result from the proposed research have the

potential to maintain and further improve the competitiveness of the European industry.

Environmental benefits of new technologies are to be demonstrated. The use of these new

technologies will certainly open new business opportunities and moreover reduce energy use

and emission production.

With an increased level of traffic safety for older road users they will be more mobile in future

and industry partners may take their advantage of that.

New motor topologies that can open new business opportunities while reducing EU

dependency on critical materials (environmental impact).

New business opportunities for reducing congestion/mobility issues in urban areas.

Making suppliers aware of cities' needs and requirements will contribute to extending the

market for ITS solutions.

7.2 EXTERNAL COHERENCE

7.2.1 COHERENCE WITH OTHER EU FUNDING PROGRAMMES

In addition to Horizon 2020 Transport, existing and new following sources of funding can be applied

to transport research and innovation outside Horizon 2020:

Connecting Europe Facility (CEF)

European Fund for Strategic Investment (EFSI)

European Structural and Investment Funds (ESIF)

European Local ENergyAssistance (ELENA)

Coherence with other EU funding programmes is considered to be extremely important and

necessary to close the innovation divide between member states and help lagging areas. In the

COUNCIL DECISION of 3 December 2013 establishing the specific programme implementing Horizon

2020 - the Framework Programme for Research and Innovation (2014-2020)101 it is described as:

101

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013D0743&from=EN

http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013D0743&from=EN


F. Widening participation

The research and innovation potential of the Member States, despite some recent

convergence, remains very different, with large gaps between "innovation leaders" and

"modest innovators". Activities shall help close the research and innovation divide in Europe

by promoting synergies with the European Structural and Investment Funds (ESI Funds) and

also by specific measures to unlock excellence in low performing research, development and

innovation (RDI) regions, thereby widening participation in Horizon 2020 and contributing to

the realisation of the ERA.

The 2012 EC’s Communication “Research and innovation for Europe’s future mobility - Developing a

European transport-technology strategy” also makes reference to other EU funding programmes,

such as funding from the Trans-European Transport Network programme (Connecting Europe

Facility), Cohesion Fund and European Regional Development Fund (ERDF)102:

“Funding from the Trans-European Transport Network programme (Connecting Europe

Facility), Cohesion Fund and European Regional Development Fund (ERDF) supports market

uptake and deployment. The Programme for the Competitiveness of Enterprises and SMEs

was also specifically mentioned. The EU also fulfils an important role of coordination through

a variety of instruments and partnerships, such as European Technology Platforms and Joint

Technology Initiatives, and is a forerunner in proposing broad political commitments and

regulatory requirements. So far, in FP7, the Marie Curie Actions have awarded €43.5 million

to transport-related research, providing attractive career development opportunities to

researchers”.

It is therefore important that the Horizon 2020 transport combines, in a proper way, excellence and

leadership and at the same time helps to reduce inequalities at EU level. As a recent analysis of the

participation of European universities in FP7 highlights103: “Overall, FP7 EU support compensated to

some extent for the wide disparity in total R&D but risk of diverging trajectories: need for more

inclusive Framework Programmes”.

An analysis of synergies between Horizon 2020 Transport and the Cohesion policy 2014-2020104 was

also prepared in 2013 by an expert group from the FP7 Transport Programme Committee. The report

identified synergies between the Cohesion policy thematic priorities and issued a series of

recommendations for optimally exploiting them.

As part as this interim assessment a survey among project coordinators was run and included the

question of whether complementarities and synergies are necessary with other public support

102

Brussels, 13.9.2012.COM(2012) 501 final. COMMUNICATION FROM THE COMMISSION TO THE COUNCIL

AND THE EUROPEAN PARLIAMENT. Research and innovation for Europe's future mobility - Developing a

European transport-technology strategy.

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0501:FIN:en:PDF 103

The role and engagement of universities in the Framework Programmes. DG RTD Midday Seminar

Brussels, 28 June 2016. ANDREA CIFFOLILLI. ISMERI EUROPA. Doc made available by the EC to the Expert

Group. 104

http://bookshop.europa.eu/en/synergies-between-the-transport-component-of-Horizon-2020-and-the-

cohesion-policy-2014-2020-pbKI0214072/


http://bookshop.europa.eu/en/synergies-between-the-transport-component-of-horizon-2020-and-the-cohesion-policy-2014-2020-pbKI0214072/

http://bookshop.europa.eu/en/synergies-between-the-transport-component-of-horizon-2020-and-the-cohesion-policy-2014-2020-pbKI0214072/


initiatives such as the Connecting Europe Facility (CEF), European Structural and Investment Funds

(ESIFs), Regional Research and Innovation Strategies for Smart Specialisation (RIS3), Smart Cities

European Innovation Partnership EIP, European Investment Bank’s European Fund for Strategic

Investments (EFSI), LIFE programme. The large majority of participants in the survey indicated that

these types of complementarities and synergies were necessary, as the following table shows.

Table 19: Are complementarities and synergies with other public support initiatives necessary (CEF,

ESIFs, RIS3, EIPs, EFSI, LIFE…)? Analysis of the answers to the project coordinator survey

0 (not necessary

at all)

1 2 3 4 5

(totally necessary)

I do not know

+ No answer

Connecting Europe

Facility (CEF)

0% 0% 2% 6% 19% 19% 55%

European Structural and

Investment Funds (ESIFs)

0% 0% 4% 7% 17% 22% 50%

Research and Innovation

Strategies for Smart

Specialisation (RIS3)

0% 2% 0% 4% 17% 19% 59%

Smart Cities EIP 0% 2% 0% 6% 11% 22% 59%

EFSI 0% 2% 0% 2% 13% 15% 69%

LIFE 0% 2% 0% 2% 20% 17% 58%

According to the EIB, the transport sector has benefited from around 6% of the EFSI investments105.

This represents, in opinion of the Expert Group, a small percentage of the total, considering the large

cost of some transport projects.

Generic references in this regard, such as the sentence in the Work Programme 2014-2015

“innovation activities linked with other EU funding mechanisms such as cohesion and regional funds

should be considered”, may not be enough. Much more specific is the 2016-2017 Work Programme

that reads in one of its sections:

“In the urban mobility domain, strong attention is paid to user-driven innovation, deployment

of innovations by a topic linking with the Connecting Europe Facility of sustainable urban

mobility solutions and technical advice (topic “MG-4.3-2017: Innovative approaches for

integrating urban nodes in the TEN-T core network corridors”), and by a topic that supports

capacity building of local authorities through the ELENA (“European Local ENergy

Assistance”) facility The ELENA Facility topic is in line with the European Council's support for

the Commission's and the EIB's intention to strengthen technical assistance to projects at the

European level”.

Additionally, the infrastructure topics section in this Work Programme states that projects aiming at

a fast implementation of results, should demonstrate their readiness for timely deployment. They

could then be considered for further support under the EU complementary schemes available at the

105

www.eib.org/efsi


moment of project conclusion (e.g. follow-up of CEF). The following is an excellent example of the

use of CEF funds in order to support deployment of transport solutions106:

“The EU's TEN-T Programme will co-fund with almost €5 million a study and a pilot

deployment of 200 charging points for electric vehicles on the main French highways. The

project will contribute to the development of charging infrastructure and enable a wider use

of electric transport in Europe.”

The majority of the Transport Programme Committee members and technology platform

representatives interviewed considered that the level of information on opportunities for the

continuation of Horizon 2020 Transport projects with external sources of funding (either for further

research or for exploitation of resources) that finally reaches potential proposers is still insufficient.

Several Transport Programme Committee members reported that their countries (either at national

or at regional level) have put in place coordination mechanisms with European level initiatives,

although it is not clear whether this is the case for all European countries. However, during the

interviews requests to provide examples of projects that had been successful in combining different

funding sources for different phases did not yield any clear results. Therefore, there seems to be a

clear opportunity to explain better the whole landscape of transport research and innovation

opportunities: ERC, EIT, KETs, FIT, Public Procurement, EIB’s funds (ESIF, ELENA), EFSI, Cohesion

Funds, CEF, venture capital and so on.

The lack of information referred to by the interviewees should be put under the light of the large

amount of existing information on this issue that, for some reason, does not yet fully reach the

research community. Examples of dissemination activities are:

- EC’s website about synergies between Horizon 2020 and the European Structural and

Investment Funds107

- National or regional seminars 108,109

- Presentations during the Horizon 2020 info days…

A clear example of the need to connect Horizon 2020 Transport activities and national funding, for

instance use CEF or EIB funds, is provided in one of the interviews conducted as part of this interim

assessment: “the states or regions must perform the right investments for implementing e.g. better

connections between transport modes, otherwise nothing will happen and intermodality will not

occur”.

Links between Clean Sky and national and regional activities supported by CEF (as reported by the

JU) provide an excellent example of fostering complementarities between EU funding programmes.

Synergies between Horizon 2020 Transport and the regions’ interests (including smart

specialization RIS3) are a key element for Clean Sky.

Each region has to consider their best forces and where they want to specialize. For example

106

https://ec.europa.eu/inea/en/news-events/newsroom/200-charging-points-electric-vehicles-open-france-

eu-support 107

https://ec.europa.eu/research/regions/index.cfm?pg=synergies 108

https://era.gv.at/object/document/1297/attach/WS_ESIF_HORIZON_2020_13_Feb_2014.pdf 109

http://wire2015.eu/en/side-events/workshop-on-horizon-2020-structural-funds-synergy

https://ec.europa.eu/inea/en/news-events/newsroom/200-charging-points-electric-vehicles-open-france-eu-support

https://ec.europa.eu/inea/en/news-events/newsroom/200-charging-points-electric-vehicles-open-france-eu-support

https://ec.europa.eu/research/regions/index.cfm?pg=synergies

https://era.gv.at/object/document/1297/attach/WS_ESIF_HORIZON_2020_13_Feb_2014.pdf

http://wire2015.eu/en/side-events/workshop-on-horizon-2020-structural-funds-synergy


there are about twenty regions that have identified aerospace as their specialization; others

are targeting transverse areas that are highly relevant for aviation such as material,

embedded software. Rather than having regions investing separately, and supporting their

individual regional initiatives, it is essential to connect to common programs, projects at

European level.

CSJU signed 11 MoUs and have another 10 in the pipeline mostly with regions but also,

according to the level where the structural funds are managed, states, (Romania, Czech

Republic and Portugal).

There are pilot projects already started with regions that are complementary to Clean Sky

and are consistent with the relevant region priorities.

Clean Sky may give input to the region for defining its programme or its call(s) to avoid

duplication of work and to perform complementary activities beyond CS

Clean Sky is also developing a kind of “seal of excellence”, like the SME instrument, for

second-ranked proposals provided that they have been positively scored and that they

represent a clear added value to the programme.

In all cases the purpose of the JU is to actively involve the beneficiary of the regional funding

with the Clean Sky network; it should not be limited to adhering to a nice “label” or “seal”. It

is a technical, practical, concrete approach.

Another very promising excellent example of action designed to promote synergies between Horizon

2020 transport and other financing instruments is the “Seal of Excellence”. As the Horizon 2020

Monitoring Report 2014 explains it110:

“The Horizon 2020 “SME instrument” has been selected for the introduction of the “Seal of

Excellence” because of its relevance to regional and national funders, as the project proposals

are mostly led by a single SME and address small-scale R&I actions close to the market with a

clear territorial impact. Regions/Member States interested in funding these types of

proposals could use European Structural and Investment Funds (ESIF) resources (in line with

ESIF priorities and in compliance with national and relevant EU rules) or their own

national/regional resources to grant funding without carrying out an additional qualitative

evaluation”.

On the other hand during the outreach held for this interim assessment, a SME highlighted that

despite having the “Seal of Excellence’ it was still very difficult to transfer this to any other funding

opportunities and there was extensive effort required again to apply for other ‘pots of money’ that in

the end meant that the SME was taking up higher and higher financial risks for less gains.

In total, a large number of stakeholders and the Expert Group think that it would be desirable that

research support instruments have the same rules so that projects that have had success, but not to

the level of funding, can more easily apply for funding from other instruments without having to start

from scratch. This concept was broadly supported.

110




7.2.2 COHERENCE WITH OTHER PUBLIC SUPPORT INITIATIVES AT REGIONAL,

NATIONAL AND INTERNATIONAL LEVEL

As part as the interim assessment survey, project coordinators were asked whether in the project

coordinator’s country, and based on his or her experience as researcher, Horizon 2020 transport

research programme was influencing their national research programmes. The majority of

participants in the survey indicated that there was a positive influence of Horizon 2020 transport in

the national research programmes.

Table 20: Influence of Horizon 2020 Transport on national programmes

Very

negatively

Negatively No

influence

Positively Very

positively

I don’t

know

Is Horizon 2020 Transport

influencing national

research activities?

0% 2% 24% 52% 7% 15%

Two of the four Transport Programme Committee members (including one former participant in the

committee) interviewed for this interim assessment also made reference to the positive influence of

the transport components of Horizon 2020 and their national programmes. The national

representative of a small country indicated that even if there was not a dedicated transport research

programme in that country, some of the Horizon 2020 Transport priorities made its way into the

general national research programme. A representative from a large to medium sized country

indicated that Horizon 2020 Transport had contributed to align and concentrate national priorities in

transport research with those of Horizon 2020 Transport. Finally, a national representative from a

large European country referred to a long-standing cooperation with others in transport research

that could be a good example for other countries, and was helped by the underpinning of Horizon

2020 and SC4. However, some Transport NCPs stated that sometimes they feel quite unable to help

advise participants about any calls beyond Horizon 2020 Transport that could provide cross-cutting

opportunities.

In a recent survey conducted on behalf of the European Commission as part of the general Horizon

2020 Interim Assessment exercise, project coordinators were asked whether they had sought

additional or follow-up funding when developing their project and/or before being granted Horizon

2020 funding111. The following table compares the general responses for the entire Horizon 2020 and

for the Transport challenge.

Table 21: Share of beneficiaries who sought additional or follow-up funding when developing their

project and before being granted Horizon 2020 funding (by funding source).

Own funding of

project partners

Public national

/regional schemes

Other EU

programmes

Private/industrial

sources

Transport (n=96) 55.3% 35.1% 16.7% 45.4%

Horizon 2020

(n=963)

51.6% 45.5% 26.4% 39.2%

111

PPMI. Framework contract for the provision of services to the commission in the field of evaluation of

research and innovation programmes and policies (2012/s 144-240132).Overview of horizon 2020 survey

results. Internal European Commission document made available for the purposes of this interim assessment.


For the whole of Horizon 2020, around half of survey respondents (52%) sought their own funding

and a substantial share (39%) also considered private/industrial sources as a potential source of

funding. In the case of transport, a slightly higher share of respondents sought their own funding

(55%) and other private and industrial sources (45%). From this information, transport is the Horizon

2020 area with highest positive responses in this regard, indicating the importance of Horizon 2020

to transport research. On the contrary, the proportion of transport respondents that sought other

public national and regional schemes or other EU programmes was considerably lower.

As the authors of this survey state, “in line with our expectations the absolute majority of the

beneficiaries expect to secure further funding for their projects in the future”. General Horizon 2020

figures and those of transport are rather similar in this case. This is shown in the next table.

Table 22: Share of beneficiaries who expect to secure additional R&D funding for their projects in the

future (by funding source)

Own funding of

project partners

Public national

/regional schemes

Other EU

programmes

Private/industrial

sources

Transport (n=96) 74.2% 74.5% 77.2% 72.4%

Horizon 2020

(n=963) 77.1% 78.1% 82.5% 72.4%

As part as this interim assessment a survey among project coordinators was run and included the

question of whether the project was using previous research from other EU-funded projects or from

other national or regional projects. The majority of participants in the survey indicated that there

was previous support to their research activities.

Table 23: Is the project using previous research from other EU-funded projects or from other national

or regional projects? Analysis of the answers to the project coordinator survey.

From other previously EU funded projects 28% From other previously EU funded projects + From previously funded national research 6% From other previously EU funded projects + From previously funded national research + From previously funded regional research + From research previously conducted at internal level by the participants

13%

From other previously EU funded projects + From previously funded national research +From research previously conducted at internal level by the participants

19%

From other previously EU funded projects + From research previously conducted at internal level by the participants

2%

From previously funded national research + From previously funded regional research + From research previously conducted at internal level by the participants

2%

From research previously conducted at internal level by the participants 9% From previously funded national research + From research previously conducted at internal level by the participants 6% From other previously EU funded projects + From previously funded regional research + From research previously conducted at internal level by the participants 2% Other 6% Not answered 9% Total general 100%


A recent analysis of the participation of the industries in FP7112 highlighted that “Framework

Programs (FPs) had had a relatively significant impact on European RPOs in several ways. In

particular, FPs have had a relevant impact on the national research policies, leading to the

harmonisation across EU MS both in terms of procedures and in terms of thematic areas of research”.

7.3 LESSONS LEARNT/AREAS FOR IMPROVEMENT

Overall the structure and goals of the instruments are seen as being coherent with the ambition of

Horizon 2020 Transport and the stated objectives of the Union for transport research. However in

some areas improvement in the implementation and how the instruments work both individually and

in complement to each other can be improved.

The Horizon 2020 Monitoring Report 2014 published in the spring of 2016 offers a general overview

of the first round of calls launched during 2014113. It covers all areas of Horizon 2020 and includes a

specific section about the Societal Challenge Transport in Annex 3. It is evident that there are

opportunities for transport researchers and innovators in many other areas of Horizon 2020 beyond

SC4 and transport. In order to obtain a general overview of those, it would be useful to assess what

role transport could play in areas or tools such as European Research Council (ERC), Future and

Emerging Technologies (FET), Marie Sklodowska-Curie Actions, European Research Infrastructures

(RI), Access to Risk Finance (ARF), Fast Track to Innovation (FTI), Space (Galileo applications), NMP,

other Societal Challenges, European Institute of Technology (EIT), SME instrument including the

Small business innovation research for Transport calls and the Seal of Excellence, Spreading

Excellence and Widening Participation (SEWP), Science with and for Society (SWAFS).

The majority of project coordinators that participated in the survey conducted for this interim

assessment indicated that generally there was a positive influence on the national research

programmes. This was corroborated by the Transport Programme Committee members and other

stakeholders who were interviewed for this interim evaluation. Based on the interviews and

exchanges with stakeholders, it is clear that while some countries or regions have put in place

coordination mechanisms with European level initiatives, this is not the case for all European

countries.

Improving the communication with NCPs on opportunities for transport so that they could provide

timely and pertinent advice to potential national participants especially in respect to cross-cutting

opportunities and funding was seen as an area for improvement.

In fact, questions have arisen in this evaluation on whether transport researchers feel they have easy

access to information on funding opportunities beyond Horizon 2020 Transport. These discussions

included some discussion on the extent to which they are interested (or able) to take these up,

especially for transport researchers applying to calls outside of those published specifically on

transport. From the stakeholder hearing, interviews and surveys it seems to be that generally it was

112

An analysis of the role and impact of industry participation in the framework programmes. Under the

"Framework Contract for the provision of services to the Commission in the field of evaluation of research and

innovation programmes and policies".DG RTD Midday Seminar. Brussels, 28 June 2016. Note: doc made

available by the EC to the Expert Group. 113




considered not interesting for transport researchers to move out of their usual field of activity. On

one side, participants found difficult to get familiar with the specificities, networks, research agendas

and previous research belonging, for instance, to other Societal Challenges. On the other side, when

considering other programmes outside Horizon 2020, such as the CEF programme, stakeholders

mentioned different processes, financial and reporting rules that they had to make an extra effort

but the gains of doing this were not large enough to be attractive.

In spite of the important efforts made by the Commission in communicating the opportunities for

funding, there is still a lot more to do. A clear opportunity exists to explain better the landscape of

transport research, innovation and funding opportunities that could help coherence across the many

aspects of transport and the various programmes.


8 EU ADDED VALUE

8.1 UNDERSTANDING ADDED VALUE

“Horizon 2020 shall maximise Union added value and impact, focusing on objectives and activities

that cannot be efficiently realised by Member States acting alone. Horizon 2020 shall play a central

role in the delivery of the Europe 2020 strategy for smart, sustainable and inclusive growth ("Europe

2020 strategy") by providing a common strategic framework for the Union's funding of excellent

research and innovation, thus acting as a vehicle for leveraging private and public investment,

creating new job opportunities and ensuring Europe's long-term sustainability, growth, economic

development, social inclusion and industrial competitiveness, as well as addressing societal

challenges across the Union.” (EU, 2013)

“The Union level funding of transport research and innovation will complement Member States'

activities by focusing on activities with a clear European added value. This means that emphasis will

be placed on priority areas that match European policy objectives where a critical mass of effort is

necessary, where Europe-wide, interoperable or multimodal integrated transport solutions can help

remove bottlenecks in the transport system, or where pooling efforts transnationally and making

better use of and effectively disseminating existing research evidence can reduce research investment

risks, pioneer common standards and shorten time to market of research results.” (EC SC, 2013)

8.2 HORIZON 2020 TRANSPORT PROGRAMME DEMONSTRATING EU

ADDED VALUE

From our outreach and analysis there is a common and enthusiastic appreciation of the EU added-

value to transport research under Horizon 2020 (although this is not unique to Horizon 2020 as it was

also the case for the previous Framework programmes). Based on interview material and survey

results among transport stakeholders (PPMI survey and experts survey), overall Horizon 2020 plays a

useful and important role in EU research, boosting the necessity and strategic coordination of

research across modes and is designed to addresses cross-cutting societal challenges (such as SC4).

The added value of the collaborative nature of Horizon 2020 is also thought to deliver more than the

sum of fragmented national programmes. The programme delivers clear contributions to the

excellence of science, the enhancement of scientific reputation and improving scientific quality

(referenced from Horizon 2020 monitoring report and mentioned by 57% PPMI survey respondents).

8.2.1 EUROPEAN INTEGRATION

As the integration of Europe is high on many agendas, it is seen as being correct and of value that

transport research should take place at European rather than only at national levels. This could not

happen with the same scope and involvement of the most relevant players without a programme

such as Horizon 2020. It is clear that one country is unlikely to be a leader in all the areas of research

required to address the major challenges Europe currently faces but, in particular, the key societal

transport challenges (such as SC4) and other cross-cutting issues. The European model works very

well in this respect.


Figure 40: Percentage of answers of interim evaluation survey on the uniqueness of Horizon 2020

It is clear that the added value of knowledge gained from the programmes and projects combined

with the coordination and interactions between SMEs, research organizations, industries and

universities from different member states working together, is high. This is confirmed by nearly 80%

of the survey respondents for this interim evaluation who considered the added value in these terms

as being either very important or extremely important. This is consistent with the research

performed within FP7 where the ex-ante evaluation also reveals strong added value described in

similar terminology (TRIVALUE, 2013).

In particular this value is associated with the research and innovation that could, or would, not have

been funded at national level. The PPMI survey for Transport Societal Challenge (SC4) shows that

about half respondents (49%) indicated that the project would not have gone ahead at all without it,

while 44% indicate that the project would have gone ahead but with significant modifications. For

the projects that would have gone ahead within SC4, the data shows that the scope would have been

narrower (68%), 77% state that the timeframe would have been longer and with fewer partners

(51%), underpinning a clear added value of the SC4 Programme. Without Horizon 2020, 73% of PPMI

responders (SC4) indicated that their international relationships and networks would also have been

affected.

By involving a wide variety of players there are gains in European cross-border cooperation,

knowledge sharing, and clear synergies and cross-fertilization of ideas brought together under the

common objective. This helps to achieve research excellence at European and international levels. It

is felt that the focus on commercialising outputs tends to undervalue this collaborative aspect of

European Research, which is valued very highly by the participants (from PPMI survey, survey for this

interim evaluation and was especially mentioned in the interviews with stakeholders and stakeholder

hearing).

In addition, from the same sources, there is perceived added value gained from projects where there

is collaboration between market players who are competitors under normal circumstances. It would

be extremely unlikely for them to work together if the project was not under a European

programme, and it is also felt that this is aspect is under acknowledged by the Commission. This

allows those that do not normally share perspectives, approaches and knowledge to do so. There are

multiple benefits from this as the research community gets a better understanding of commercial

imperatives and bottlenecks; and commercial players are able to understand requirements and


positions for non-commercial ‘pure’ research to be undertaken. The input from stakeholders to this

interim evaluation indicated that this aspect is much appreciated by those taking part, especially

when the research is ‘upstream’ of commercialisation (i.e. in the pre-competitive stages). Risks and

costs, as well as knowledge, can be shared. It is also recognised (by the same players) that there are

sensitive areas when it comes to research outputs that touch on competitive aspects and ownership,

which are not always entirely comfortable between players (from interviews with stakeholders).

8.2.2 INCREASING EUROPEAN COMPETITIVENESS

In terms of impact on competitiveness and European leadership, 80% of the respondents to the

survey indicated medium to much added value in terms of improving market positions or

competitiveness. A large number (93%) of SC4 respondents to the PPMI survey indicated the

programme made a significant contribution to leadership and enabling industrial technologies to be

brought to market. Areas where Europe is perceived to be in a leadership position e.g. in aviation or

automotive, should receive continued support to keep this position as there is strong competition

from out of Europe e.g. from China, Korea or USA. However the focus of this support should be

directed towards taking these industries away from fossil fuel and helping get them onto low carbon

and more inclusive pathways.

Demonstrator projects and pilot projects undertaken by multi-players and participation of various

countries are seen as adding particularly high value. In the majority of cases it would not be possible

to conduct such trials at national level as there are not enough different players at national level nor

the funds to do so. The Horizon 2020 survey (PPMI, 2016, p. 29) records large-scale demonstration,

prototypes and testing, for new or improved commercial products and developing business models

with the highest levels of added value (more than 55% of Horizon 2020 responders).

There are also clear synergies in terms of access to larger markets and new business opportunities.

Projects implemented through national funds, as seen to have reduced competitive positions

internationally (from the same survey (PPMI SC4) 79% responded that they expected a decrease in

this aspect) as well as a decrease in access to new markets (69%). These responses can be taken as

evidence of EU added value in respect to national or regional funded programmes/projects.

Another important positive aspect of the PPPs (JUs) is the integration of several research projects

under a single programme. This has helped to promote multi-stakeholder developments and

practices, an important cooperation towards the continuation of Europe’s cutting edge technologies.

It is considered that this helps to bring stability/continuity of research and the ability to build in

flexibility.

Horizon 2020 projects also help participants to share risks, which is seen to bring competitive gains.

72% of the responders to the interim evaluation survey (question 32) indicated between medium to

much added value in risk reduction, and this was confirmed in both stakeholder hearing and

interviews. Member States may be able to do this on a national level but the Horizon 2020 scale is

significantly larger.

In terms of impact, the following table shows the contribution of expected research outputs if

implemented).


Table 24: Expected research outputs from SC4 Horizon 2020 projects: answers from survey among

project coordinators.

Contribution to Number

of

references

References /

No of

projects

Decarbonising and "greening" the transport system 39 72%

Increasing efficiency of the whole transport system 37 69%

Improving safety & security of passengers, aircraft, vehicles and vessels,

and infrastructures

22 41%

Strengthening the competitiveness of European industry 42 78%

Pioneering the Transport of the future (long term perspective) 20 37%

Enhancing and strengthening the European Research Area (ERA) 26 48%

Globalisation, external dimensions and international cooperation 17 31%

Further development and implementation of EU transport policy 25 46%

As has been mentioned, Horizon 2020 transport has been timely for preserving the competitiveness

of some subsectors. It was particularly timely for the automotive sector as many research topics

were threatened with the economic downturn (preservation of jobs and recruitment in a context of

financial crisis, EGVI, 2016). The new structures and opportunities provided by Horizon 2020 have

helped to preserve highly qualified jobs and skills in the R&D automotive value chain e.g. with the

PPP European Green Cars Initiative (EGVI, 2016). This is a clear validation of the ambition to fulfil the

Junker plan and the competitive aspect of Horizon 2020.

In terms of addressing Europe’s Societal Challenges, especially SC4, it should not be forgotten that

jobs need to be created in non-fossil fuel based industries as well and especially jobs in the

automotive industry need to be sustainable, i.e. relevant to future transport helping shift the burden

of change that this subsector must make. The negative externalities caused by the automotive

industries, especially the loss of productivity from congestion estimated at around 2% of Europe’s

GDP, as well as the societal impacts of loss of life and injury from road traffic accidents and air quality

issues, need to be balanced by the positive aspects of job retention or creation within this sector.

Well-directed research can help Europe prepare for this shift.

8.2.3 EUROPE’S ROLE IN STANDARDISATION AND HARMONISATION

Playing a role in developing international standards is an important role of the EC and its research

programme in terms of leverage of transport research, although this aspect may not be always be

visible, it is seen to be of crucial importance to the industry and for the deployment of innovative

solutions.

Interoperability, which is still considered a challenge especially in the rail sector would be even worse

without Horizon 2020 (and previous FPs). This is being delivered by improving technical standards

(TSIs), which have been worked on since 2002. While it is recognised there is some progress, there

are still considerable bottlenecks and challenges to making this a reality for both passenger and

freight services. Research has helped deliver some solutions but much of the new research will be

delivered via Shift2Rail (S2R). From our interviews Shift2Rail was heavily criticised for having a


structure that, in real terms, almost excludes the research community from being able to take part in

substantive research unless they have support from inside S2R114. This community expressed this

view quite strongly in interviews. It was also commented on that it feels excluded from being able to

help guide research due to the structure of this PPP. Interoperability is also seen as being a key topic

of interest to deliver many of Europe’s policy objectives such as the introduction of alternative fuels,

e-mobility, integrated passenger ticketing between multiple modes, constraining conventionally

fuelled vehicles in urban areas, cross border freight to mention a few.

Standardisation is also a key issue in terms of materials, components and alternative fuel standards

including electromobility (plugs and charging infrastructure) and needs to also address many societal

aspects in respect to inclusiveness, such as levels of accessibility (e.g. for citizens who are less mobile,

older, etc.). Research needs to deliver the evidence base for policy development and this often

requires managing conflicting aspects and delivering recommendations based on validated

information and data. An example includes the dangers of silent electric vehicles, seen as being a

solution to noise levels and pollution, but this lack of noise is a danger to those who are

audio/visually impaired as they can neither see nor hear them. This work would transcend national

capacities and fulfils the role of research by a supranational body such as the Union. The ART

(Autonomous Driving) call is another positive example of where European led research adds value.

Indeed without the Union promoting a choice of solutions that sets a level playing field for the

industry to respond with new products and services that are interoperable across all Member States,

it will be difficult if not impossible to achieve the inclusive, low carbon and fossil fuel free transport

future that is desired and an objective of SC4 and Horizon 2020.

8.2.4 BUILDING STRONGER EUROPE-WIDE RESEARCH COMMUNITIES AND LEVEL OF

EXPERTISE

The role of the EC in reducing gaps in research competences and knowledge across Europe is also

very relevant. On the one hand, collaboration between the wide variety of players in the research

community is seen as being rather positive as it encourages knowledge sharing as previously

mentioned. On the other hand the current trend of focussing on developing research based on

excellence is not seen as being helpful to reduce disparities. Indeed it is sometimes considered to be

counterproductive. For instance, it becomes more difficult to enter consortium bidding for proposals

if you are not a strong player, creating vicious circles where strong players are oversubscribed and no

one wants to associate themselves with weaker players (who in turn are therefore not able to learn

and develop and become strong players). This view was shared by a number of interviewees.

Therefore the Expert Group would like to highlight this as a risk rather than validated evidence of the

present situation. Comments such as “we were surprised when xyz project was funded as we did not

think it was a strong consortium” were frequent enough for the Expert Group to take note. To some

extent this underpins the opinions of many that the project evaluations are not easy to understand

114

It should be noted that despite several attempts to reach out to the rails sector the Expert Group did not get

spontaneous or enthusiastic responses to participate in this review, so these comments are based on

comments from those outside of the rail sector. However it is also the opinion of the Expert Group from their

observations that the isolation of the rail sector from other transport modes is a symptom of many potential

problems within the rail sector.


(although the EG is also conscious of few requests for re-evaluation115). There are other instruments,

such as structural funds, that were mentioned as having stronger leverage in terms of reducing

disparities.

Half of the respondents in the survey said that if the research funding had been sourced from national

or regional funds, the following areas would have been affected:

Less mobility of researchers (qualitative data inputs from interviews and 53% percentage of

PPMI SC4 respondents),

Decreased ability to provide funds for PhD’s or the capability to provide training (qualitative

data interviews and 59% percentage of PPMI SC4),

Less career development of researchers (qualitative data inputs from interviews, 50%

percentage of PPMI SC4 respondents)

Diminished capacity to attract researchers and other staff (71% of PPMI SC4 respondents)

This, combined with the results from the figure below show that there is both perceived and real

added value to European funding for transport research and that this plays a strong role in creating

strong, multi-discipline Europe - wide research teams, the Expert Group is not aware of any other

region in the world that can boast this for 28 countries.

Figure 41: Percentage of answers indicating the numbers of research projects that would not have

implemented without EU funding (57% positive responses).

115

The Expert Group would like to highlight that the low figure for revaluation should not be automatically

interpreted as a high level of satisfaction with the process. It was mentioned on numerous occasions (especially

in interviews and at the stakeholder hearing) that it is not felt to be useful to complain as the budget has by

then usually been allocated and there would not be any likelihood for a project to be re-evaluated and funded.


8.2.5 DELIVERING RESEARCH THAT FEEDS INTO EUROPEAN POLICY OBJECTIVES

The PPMI Horizon 2020 survey provides evidence showing a strong relation between transport and

other societal challenges such as: SC1 Health and demographics change and wellbeing (26% of PPMI

SC4 respondents); SC3 Secure, clean and efficient energy (39% of PPMI SC4 respondents), SC5

Climate action, environment, resource efficiency and raw materials (62% of PPMI SC4 respondents),

SC6 Inclusive, innovative and reflective societies (29% of PPMI SC4 respondents). One of the most

noticeable evolutions within Horizon 2020 is the aspect of multimodality or intermodality.

An example includes electrification, which may be applied through different areas, cross mobility and

handling of big data, activities within Horizon 2020 but not addressed in the previous FP. However,

based on responses and in the opinion of the Expert Group there is a need to improve coordination

between the various societal challenges. From responses gathered during this process, it is

concluded that overall the calls associated with addressing them tend to focus on the industrial

needs of particular industries and not necessarily on the transport needs. This opinion was validated

by interviews and other exchanges articulated, in some cases, as the desire to see improved

cooperation and coordination across sectors and units within the Commission. It should be noted

that this was also requested in the EU President’s letter to Commissioner Moedas mentioned earlier

in this report.

8.2.6 LINKING CALLS AND TOPICS MORE DIRECTLY WITH POLICY EXPECTATIONS

While recognising that to some extent it is difficult for projects and project officers to address what is

expected from them in terms of policy impacts, the inputs from the various sources in this evaluation

suggest that there is insufficient feedback on results and policy feedback. This was mentioned in

various ways in a number of the interviews and also in informal feedback. It should be noted from

the low score on this point from respondents to the internet survey this was not seen as a priority for

outputs.

Substantiating policy impact was not easy for this evaluation as some projects are followed up by the

EC while other projects are monitored by INEA. This is seen as a parallel effort (interviews with

stakeholders). The Expert Group could not find any clear guidance or understanding on how policy

monitoring is happening, and harvesting policy relevant data also remained unsatisfactory. In FP7,

projects were clearly disconnected from each other, but now in Horizon 2020 links are made through

“collaboration agreements”, which is seen as a positive development but it is thought that the

process in place may not be robust enough.

Many interviewees mentioned that links between policy objectives and topics are not always clear, in

particular with less-prescriptive topics, and participants also see this as a ‘grey’ area. This was also

brought up in the stakeholder hearing. The perception of those respondents and the Expert Group is

that few projects are properly evaluated post project completion.

However, the Expert Group agreed that it is too early to know about the specific project outcomes of

Horizon 2020 and give a definitive answer concerning the coordination of policies in Europe, so the

Expert Group were not able to establish any clear policy feedback loops. Nonetheless, concerning

‘contribution’ to EU transport policy objectives, 56% of respondents to the survey consider the

European added value between important to very important but from the results of the Expert

Group’s survey, fewer make the explicit connection.


8.3 SUCCESS OF DIFFERENT INSTRUMENTS TO DELIVER EU ADDED VALUE

8.3.1 ERA-NET

The potential of ERA-NET and ERA-NET Plus to deliver added value mainly relies in the collaboration

and alignment of national and EU research and innovation activities (this observation is based on

interview material):

Both of them represent useful instruments to guide member state’s research priority areas

and are seen as being complementary to the rest of Horizon 2020 instruments. A good

example is the Infrastructure Innovation Programme116, with nine projects launched in 2015,

and initiative that includes pilots, and if successful, the industry will exploit results.

It would appear that the concept of these instruments is seen as being intrinsically good but

the implementation is not well thought out and overall they may not respond to market

needs as they are not easily and widely taken up. Many interviewees mentioned that in their

view they do not work well, and to date a limited amount of funds have been mobilized from

this instrument. Stakeholders find the evaluation and rules of participation barriers to

participation.

The low level of participation appears due, on the one hand, to a lack of interest from industries and

the small scale ERA-NET and, on the other, how the instrument relies mainly on richer countries co-

funding. From input to this process, ERA-NET Plus schemes are less attractive to smaller and less

wealthy member states. 92% (50 out of 54) respondents to the Expert Group survey have not

considered to apply to the ERA-NET programme and only one proposal was eligible and retained.

8.3.2 CEF

Connecting Europe Facility (CEF) delivers European added value by supporting the deployment of

promising solutions that have been validated within transport research funded activities e.g. pilot

demonstrators or prototypes. However in the Internet survey 55% of respondents answered that

they did not know if CEF was a good facility, and only 38% said that it was either very or necessary to

have. The following observations are based on interview material:

It is seen as a good instrument to ensure deployment of promising solutions identified by

research and innovation activities. SESAR connects to deployment through CEF funding and

Shift2Rail also plans to use it. Another example found concerned companies taking initiatives

based on successful research solutions (after demonstration and are able to build business

cases), such as the case of Heathrow airport. Here research was used to improve the airport

capacity in certain meteorological conditions and it was able to build a business case and

deployed the solutions from specific SESAR developments with good results (ACARE, SESAR,

2015 and two interviewees).

8.3.3 PPPS AND JUS

The role of the PPPs and JUs can boost European added value but this relies on the connection of

research and innovation to industrial strategies and access to the market, especially from the outputs

of demonstrators. These multiplayer projects usually involve Europe’s top players and best expertise,

116

Infravation, www.infravation.net

http://www.infravation.net/


thus the resulting quality could not be achieved within one country. This section reflects insights

from consolidated interview material.

Overall, the variety of instruments is acknowledged as being important and the responses are mostly

positive. The most frequently mentioned pros and cons and recommendations taken from a variety

of sources to the Expert Group are shown in the table below.

Positive Negative

- Stability of funds (even during budgetary

cuts) allowing achievement of goals and

tangible results (such as the implementation

of roadmaps)

- Alignment and coherence between industry

and society’s interest. SESAR is put forward

as an example of the combination of

industrial interests and benefits for the

citizens.

- The market impact of PPPs is faster and

larger due to the better focus on the direct

R&D needs of industry. Industry will not

invest in products that are not marketable.

- The rate of success in calls is higher in the

PPPs. That is both due to the choice of

themes (close to market) and the more

‘cooperative’ procedure.

- PPPs/JUs generate a strong community of

research actors around one area. They

cluster resources and talent around a well-

identified topic.

- All projects are related and they are few

isolated actions. Otherwise fragmented

research is integrated, so it ends up working

in a coherent fashion

- The process for being able to launch one of

these initiatives is very long and roadmap

objectives tend to be broad, influenced by

the input from all the actors and “need” to

satisfy them all.

- If the roadmaps are not updated this can

lead to wasted efforts

- Some PPPs or JUs should be more

transparent in terms of participation, and call

preparation.

- Information to Member States do not reach

the Transport Programme Committee.

- Coherence and linking across transport

initiatives is still insufficient.

Areas of improvement that might not apply to all PPPs and JUs

- The topic should be narrow enough so that the (successful) progress of the results is able to

justify the investment. (Note from Expert Group: topics are considered to be already narrow

within some PPPs/JUs but this could possibly be improved as some JUs are more mature than

others.)

- The results to be achieved need to be in line with the available funding and ambition needs to be

tailored to what is feasible rather than what is desired.

- PPPs should have a larger leverage effect on contributions from industry and this should always

be transparent.

- Stakeholders capable of responding to the challenge should be in the lead while, at the same

time, leaving the door open to new participants, (including those that are not actual members of


the JUs). All stakeholders must be on board in an active and inclusive way. (Note from Expert

Group: Results are a little patchy as in some JUs, but success stories showed good involvement of

SMEs and new comers.)

- PPPs and JUs should concentrate on higher TRLs and more fundamental (blue sky) research

should still be tackled in the normal calls. Full coordination should exist to avoid overlapping

topics both in the PPPs/JUs and in the normal calls.

- Simultaneously, they should provide room for bottom-up proposals and approaches as well as for

longer-term visions.

- DG RTD and DG MOVE should cooperate more intensively in all the PPP and JUs and the

development of research agendas as input on research gaps is given to them rather than directly

to JUs.

- The Member States Representative Group in all PPPs should be more active in connecting

national research activities with those in Horizon 2020 (it is noted that Clean Sky is making

notable progresses in this direction).

- It is suggested to deepen synergies between PPPs and the SME Instrument.

The number of PPP/JUs initiatives is already very large and caution should be taken when considering

new ones. This should only be done after a clear gap analysis of needs. SESAR represents a particular

case where policy, benefits for the citizens and competitiveness come together. It represents the link

between industry and ATM sector and its process is seen as being excellent. It operates under a

Master Plan adopted by Member States that includes reviews every two years (content and future

deployments). SESAR also connects research results to deployment (including feedback on how

deployment is progressing). The JU supports the European Commission in their international contacts

on aviation and ATM issues and this close working is seen as delivering added value.

8.4 OTHER ISSUES RELATED TO EU ADDED VALUE

Concrete evidence that research and innovation outcomes will be exploited or built upon is part of

adding value to research investments. In Horizon 2020, the emphasis on exploitation and

deployment was more than in FP7. This is appreciated by the majority of those consulted most of the

interviewees, although some find this link difficult to determine at proposal stage, and it is also

expected that Horizon 2020 projects will lead to increased impacts, although there is no evidence of

this yet.

There are a number of examples from previous framework programmes that have been

implemented such as can the next generation of aircraft as shown in figure below. It is also clear that

these technologies and developments achieved at European level within Clean Sky, would not have

taken place without a Europe wide programme.


Figure 42: Success stories – Aviation Research and Innovation (ACARE, 2016)

Under FP5, FP6 and FP7 research addressing the safe reduction of wake turbulence separation

minima has resulted in aircraft separation schemes have been initiated at two European mayor

airports (London Heathrow and Paris Charles de Gaulle). This has contributes to reduction of flight

delays (up to 50% of which are generated by adverse weather) and an increase of runway throughput

in the order of 5% without compromising safety levels for these operations. In the road transport,

examples include: Crash Test Dummies, results from this project are used by car industry to design

safer cars; Personal Rapid Transport enabled by the CityMobil project implementing 20 pods that run

automatically as illustrated in figure below and others noted in the ERTRAC publication (2016). The

European Bus System of the Future (EBSF) aimed to develop a new generation of urban bus systems

adapted to the needs of European Cities and this project made a major step forward in bringing all

the relevant actors together from industry and operations, who were at that time not used to

working together. A positive outcome has been the take up of the outcomes of this project in the

design of bus drivers’ compartments, which is now widely replicated in the buses built today.

Figure 43: Success stories from road transport (ERTRAC, 2016): THORAX – Thoracic Injury assessment

for improved vehicle safety and CityMobil PRT system was implemented by BAA (British Airport

Authorities).

The “Oceans of Tomorrow” initiative was mentioned in some interviews as it helped to coordinate

maritime research (EU, 2014) but it has since been discontinued. This was noted with

disappointment by some. Other good examples from this sector include the development of

biosensor technologies for environmental monitoring (FP7 EnviGuard ongoing) and examination of

aspects of the wind farm life cycle (LEANWIND, FP7)


Figure 44: EnviGuard and Leanwind – Oceans of Tomorrow (EU, 2014)

8.5 LESSONS LEARNT AND AREAS FOR IMPROVEMENT

There is evidence in the data collected that most of the research projects implemented could not

have been performed outside the programme. Working at European level has provided greater scope

and scale especially to many projects, making them more relevant to tackling the SC4 challenge and

create more impact, than would otherwise have occurred.

However, there is a need to improve the understanding of linking outputs to policy development and

feedback, as well as improve learning exchanges and the transfer of knowledge from project outputs

to increase impact. Current work concerning the STRIA process (under development) is appreciated

and is seen as a positive development, addressing the current perceived silo approach in some areas

of the transport programme. It is expected to address these needs (as expressed by stakeholders) for

a coordinated and strategic long term, holistic vision for transport.

Despite the generally high level of satisfaction in terms of European added value, several areas

remain for which improvements were suggested by those interviewed or surveyed.

8.5.1 INCREASED INTEGRATION

- There is little or no incentive to deliver integrated solutions beyond Horizon 2020 projects or

for PPPs to work with other transport modes to develop integrated approaches and

solutions. There is a need to rethink how this could be achieved, as the cross-fertilisation of

ideas across sectors is not sufficient and there is high potential for this to increase added

value.

- Strengthening research areas that are considered important, but that are currently less

developed, is seen as an area of attention. Comments made during interviews imply that this

may be because these topics are linked to certain interest groups or due to historical

reasons. Despite divergent views from the interviews on how to include certain topics, which

are seen as being possibly sensitive, the Expert Group would like to encourage the

Commission to reinforce taking a system wide approach, looking especially at how transport

modes can and should work together, including (inter alia) logistics, integrating with ITS

research (e.g. under C-ITS), sustainable infrastructure, and including more activities to

promote the benefit of the softer modes of transport (walking and cycling) to frame its

research agenda. In particular this refers to socio economic and behaviour areas of research.


This opinion is substantiated based on responses especially from the interviews and the

discussions at the stakeholder hearing. It also refers to the comment on the vertical

approach to addressing the societal challenges mentioned earlier117.

- There appear to be few incentives to address the need for interdisciplinary research. The EC

itself is not seen to be well organised to support or enhance interdisciplinary research –

leading to the question of how integration can be achieved without this.

- It is seen that Europe needs to play a very strong role in facing global challenges such as

increased automation and standardisation (examples include electric mobility, alternative

fuels, emissions testing standards and ITS). The added value of Europe acting as one entity in

such areas carry weight in the global market and applying research outputs should put

Europe in the lead, but presently this pathway is not secure.

8.5.2 FORMAL EVALUATION AND FOLLOW-UP OF PROJECTS

- From the feedback, the Expert Group concludes that the evaluation of finalised projects

needs further attention, especially since there is little incentive to take any outputs beyond

the final project presentation. Several interviewees expressed frustration that the

Commission/Project Officer did not attend the final conference or presentation of final

results. In addition, there were numerous comments about the duplication of research

topics that had been treated in previous framework programmes. The proper value of these

projects was not seen to be fully harvested (diminished added value), as project results are

being recorded in ways that do not allow anyone who had not worked on the project to fully

understand the potential of the output and the present (listed as a deliverable in databases

with no reference to outputs).

- A loss of domain expertise and a focus on project management was cited frequently as being

a real threat. There is a significant reduction in the content review from project officers and

the Commission is seen as being resource stretched. This results in a focus on process rather

than content, examples of this include prioritising administrative aspects rather than project

R&D content.

- It is felt that the TRIP118 portal is focussed rather on quantity than quality, despite certain

improvements and which to some extent is understandable as those managing the portal are

not responsible for the quality of the documentation but rather that deliverables are

uploaded. However it is difficult to access meaningful information on a project via this site

once it has finished. If one knows the name of the project, searches can work well, but if for

instance a local authority might be interested to know what is available on urban road pricing

there are not enough parameters that would allow to extract meaningful research that could

be used for policy or programme development. In other words too many ‘hits’ are presented

and it would be very time consuming to wade through this information to find relevant

examples.

117

See also section on internal coherence. 118

The TRIP portal is the Transport Research Innovation Portal formerly known as the Transport Research

Knowledge Centre (TRKC) - to collect, structure, analyses and disseminate the results of EU supported transport

research and research financed nationally I the European Research Area (ERA) as well as selected global

research programmes (www.transport-research.info)

http://www.transport-research.info/


8.5.3 FUTURE RESEARCH DIRECTIONS ORGANIZING FOR INNOVATION

- Gaps are also noted in the follow-up of on implementation of research and innovations

(Interview material from stakeholders representatives from aviation, road and maritime).

These are most apparent in the follow-up from demonstrator and pilot projects to full-scale

applications. It is felt by the Expert Group that this is due (partly) to the lack of support for

project coordinators and participants to take successes to market. Many good products are

not able to become mainstream, sometimes expressed as ‘overcoming the valley of death’,

as they cannot access the necessary funding levels easily for furthering research. The effort

to try to get further funding is beyond available resources. This is seen as a bottleneck for

innovation, as it is thought (and confirmed in interviews) that there are many small projects

and demonstrators that could be taken further or that might be relevant to other sectors (an

example is an environmental friendly seat based on cane sugar fibre). Solutions to take

forward high potential outputs could be better considered within this programme.

- Some subsectors feel that they are lagging behind in investment and attention (based on

interview responses). For example, it was mentioned that the maritime sector the EC

currently has a limited role, less than desirable. This could (and should) be much larger and a

better cooperation with River Committees and IMO could engender this. The relation

between different functions of maritime are not sufficiently considered e.g. fishery, seismic

activities, LNG as new fuel. Based on interview material and Expert Group analysis: a PPP in

the maritime sector could possibly present an opportunity to integrate policy research in a

broad programme, overcoming the silos that naturally exist in the sector – the willingness

and interest of the sector for this was reinforced at the stakeholder hearing.

- Overall remarks were made by enough interviewees to be noted by the Expert Group that

current research appears to have a too narrow a focus. There is a definite bias towards

encouraging technological easily quantified solutions to the detriment of socio economic and

behaviour research (linked to the KPIs of patents and registered outputs). It is felt that this

misses opportunities and creates research gaps in areas of multimodality, linking modes

together and activity areas within a transport journey (stations were mentioned in this

respect but it is not thought that this is the only example), in other words having a more

focussed systems approach.

8.5.4 DISSEMINATION AND EXPLOITATION ASPECTS

The growing attention for dissemination is overall seen as positive. The survey of Horizon 2020

projects indicated that without Horizon 2020 funding, participation in scientific conferences,

seminars or workshops and other outreach activities would have been significantly reduced (63% of

participants to PPMI SC4 Survey). Specific activities targeting policy makers (58% of participants of

PPMI SC4 Survey) would have been less and fewer publications in referred journals and books (56%

of participants to PPMI SC4 Survey).

At the end of 2015 the EC conducted a survey among FP7 Transport project participants with the aim

of analysing the exploitation of transport R&I outcomes119. In the introductory section it is explained

119

Exploitation of transport R&I results: report on a survey of FP7 participants. June 2015. EUROPEAN

COMMISSION, DIRECTORATE-GENERAL FOR RESEARCH & INNOVATION. Directorate H – Transport H.1 –

Strategy. Note: internal EC doc.


that an overall strategy for the dissemination and exploitation of project results is currently

developed for Horizon 2020 with the aim to:

- Increasing the availability of project outputs (results/data)

- Describing a process for the feed-back of results into policy making

- Supporting the exploitation of Horizon 2020 research results at project level

The aim of the FP7 Transport Results Survey was to identify project results at (individual) participant

level and related needs for their dissemination exploitation. In this document the results of the FP7

transport survey are presented and some conclusions drawn, some of which can be considered to be

of interest to the dissemination of Horizon 2020 projects (noting that the majority of them are not

completed at this time).

Dissemination should be understood differently from communication. While the first is directly

towards peers and researches, communication (a two way process) should address more actively the

policy makers and the general public. Most projects still rely on traditional methods of dissemination

– newsletter, conferences, and a website. These tools do not always reach the desired target

audience today as people (especially decision makers and those in charge of policy development or

implementation at national and local levels) are inundated with such requests by email. The lack of

knowledge about good projects beyond those involved directly or indirectly with the projects is seen

as being a result of this. Therefore, in terms of added value, dissemination actions with impact (for

both FP7 and Horizon 2020 so far) are seen as being insufficient even within the research

community.

There are some good examples of dissemination such as the Clean Sky Forum, the Green Aviation

Conference, projects represented in aeronautics conferences, and most especially the Transport

Research Arena (TRA), which is used as a key opportunity for dissemination (www.traconference.eu).

SESAR and ETRAC have also showcased results from projects together with a catalogue of solutions.

Such catalogues open a possibility to the European industry and service providers in providing

solutions outside Europe and for examples to be easily sourced and referenced.

There have also been recent publications Research Theme Analysis Reports (e.g. on Urban Mobility)

based on input from the TRIP portal. These are looked at positively as extracting information from

this portal is not easy. The search fields are quite limited and adding more layers of search or having

a more performing search engine use could help people find the information that they want more

quickly. Although the cost of printing is always high people still have a mild preference for paper

copies of reference documentation but electronic versions should also be easy to find on the

Internet. As these are spread across many websites, this is not always the case.

It is suggested that the EC should ensure that proper dissemination partners are included in the

projects, as this is not seen to be a priority of partners (besides academic papers and publications). In

addition researchers are not communication specialists and they are comfortable using channels of

communication for research purposes, which may not be the best ones for dissemination for

exploitation purposes. Events where prototypes of already available or if research outcomes can be

demonstrated that they might benefit from a more formal EU communication service where experts

are available, even if this facility only provided guidance. The EC has an effective communications

machine that could be very useful to provide such support to researchers. .

http://www.traconference.eu/


From the Internet-based survey conducted by the expert group, suitable actions to support the

dissemination and exploitation of transport results include:

- Brokerage events including the EU information days and those hosted at major transport

events, such as the TRA conference120, commercial conferences and other transport relevant

events offer FP7 and future Horizon 2020 project participants (and owners of IPRs/results)

the opportunity to establish business contacts with potential users, buyers, and funders of

downstream development.

- Ad-hoc initiatives address specific needs, in particular those of SMEs. The 'Transport SMEs

Innovation Day' provide SMEs the opportunity to be briefed on financing possibilities e.g.

EASME, RSSF etc., and to have possible meetings with venture capitalists/risk financers and

this is welcomed.

- Continuous monitoring and analysis of transport research projects and related results (FP7

& Horizon 2020 projects) via surveys on research outcomes at individual project level and

also the analysis at aggregated level collected into achievements reports also help to get the

word out.

- Clustering activities (as they occur at TRA and AERODAYS) were mentioned on several

occasions as being underutilized. These events are seen as being ideal forums for the

dissemination of Horizon 2020 results as they cluster several research and innovation

initiatives as well as allowing researchers to meet and exchange ideas (often leading to

innovative solutions being developed). The TRA is seen especially as being a very useful

event for the many areas of transport research and it is useful for dissemination beyond the

research community that the proceedings are made publically available via Elsevier and

Science Direct121. However the audience is often the research communities.

Suggestions for improvement include:

- Level of dissemination is still artificial, limited and not creative. It is not easy for researchers

to become communication specialists or for them to fully grasp expectations and the EC

could provide help in this area from its own communication specialists.

- Dissemination should focus on a broader community of researchers, consultants, city

authorities, transport operators and other decision makers (peer-to-peer). However,

dissemination towards the general public is not seen as being very relevant for Horizon 2020

projects.

- Too many projects use traditional means such as websites, conferences and newsletters,

considered innovative ten years ago. Today people are inundated with requests to attend

meetings/workshops and conferences and without budget to travel it is difficult to attract

the target audiences mentioned above. The use of social media is not widely used and could

be an efficient and effective channel for dissemination. New ways and channels can be

encouraged.

120

www.traconference.eu 121

www.sciencedirect.com/science/journal/23521465/14

http://www.traconference.eu/

http://www.sciencedirect.com/science/journal/23521465/14


- The use of questionnaires and Internet surveys is also not seen to be as effective as it was.

This was confirmed by the experience of the Expert Group who would have liked to have

more responses to their survey but stakeholders are frequently requested to take part in

surveys and their time for such activities is limited.

- For effective dissemination of outputs, there is a need to balance a holistic vision on the one

hand, emphasising integral approaches to knowledge and, on the other, translating this into

accessible, transparent and comprehensible language for wider audiences. It is not felt that

this is optimal yet.

Overall Horizon 2020 is seen as adding a lot of value to both transport research and to achieving

European Union objectives. The collaborative approach to investigating transport and societal

challenges and the funding opportunities available with Horizon 2020 are seen as being both ‘first in

class’ in Europe but also globally, as the Expert Group is not aware of any other research programme

of this size and scope that can be compared to Horizon 2020.


9 SUCCESS STORIES FROM PREVIOUS FRAMEWORK

PROGRAMMES This has been covered in the text with a number of examples that can be developed further

Please provide three examples of success stories based on an assessment of FP7 projects whose

outcomes/results have had a significant added value and/or impact on the society and/or the

economy and which have outstanding EU added value using the same criteria as in the previous

section: effectiveness, efficiency and synergy.


10 LESSONS LEARNT / CONCLUSIONS Overall from the information gathered by the Expert group there is quite broad support for Horizon

2020 from all stakeholders, including industry (private sector), member state governments, civil

society and the research community. Recognising that transport is unique in terms of its societal

value, economic and environmental impacts, and the need for coordination of research in this field,

there is definite support for a new Framework Programme post Horizon 2020.

Looking more closely at the five criteria set out by the Commission to frame this work, and

considering the Societal Challenge 4 as requested, the Expert Group puts forward the following that

can be considered as lessons learnt and suggestions for improvements in a new round of research

activities for transport.

The Expert Group has structured this chapter to show what in their opinion is working relatively well;

what is not and how this might be addressed. These observations are validated by the various

interactions with the stakeholders (interviews, survey results, data provided, the stakeholder hearing

and the opinions of the Expert Group based on their experience and the information reviewed as

part of this exercise). The recommendations and conclusions set out at the end of the chapter refer

to Horizon 2020 and the SC4 challenge in general unless specified otherwise.

10.1 RELEVANCE

Generally transport research under Horizon 2020 is seen as being extremely relevant for European

policy, environmental sustainability, society at large, competitiveness and boosting the European

economy. It becomes more and more relevant to international challenges such as security, reducing

carbon emissions and the fight against climate change. Its relevance during the current economic

crisis is also clear and this was especially noted for the automotive sector. It is also important in

terms of making funding for research available and in the way it has helped to identify research

priorities and to focus available resources at national level.

Horizon 2020 was mostly based from the policy perspective on the Transport White Paper and as

long as that is still pursued and relevant, so is Horizon 2020. From the technology point of view,

Horizon 2020 Transport is largely based on the contributions of the European Technology Platforms,

among other actors. There are clear links between Horizon 2020 and President Junker’s priorities and

the general objectives of Horizon 2020 transport can be clearly mapped against Juncker’s top

priorities122.

The Expert Group agrees that there has been progress compared to other framework programmes

and the Horizon 2020 research agenda is helping to achieve a Resource Efficient Europe in terms of

technology, energy transition and to some extent the transformation of transport systems. There is

definite support for the EC’s efforts to direct research towards concrete results, deployment and thus

to increasing the impact on solving the transport societal challenge(s) and there is full support for

how the programme is promoting excellence in science.

122




The process of defining research themes and work programmes is sound and well developed.

Relevance of the work programmes is notably assured by reflecting new developments in the

definition of new calls: a relevant example being the ART call in WP 2016-2017 which put vehicle

automation on the research agenda in parallel to innovative developments in the industry.

Some stakeholders are more involved in Horizon 2020 compared to FP7.Examples include end users

and cities. This is seen as being very positive because it facilitates innovation. Large players are

already fully involved and smaller players (especially those who have experience of FP7 or Horizon

2020) have opportunities to take part. Direct contact between the private sector and end users is

also of seen as being of vital importance to the success of the programme, and there has been

notable progress to involve them in Horizon 2020 compared to FP7.

PPPs and ETPs also fulfil positive roles in achieving the Horizon 2020 goals by bringing together

different interests through their multi-actor composition (industry, research providers, governments,

urban interests, etc.). Their role in defining and agreeing medium and long-term visions, strategies

and roadmaps is also seen as being highly relevant.

The general goals of Horizon 2020 are clear for stakeholders and there is not much doubt that

Horizon 2020 as a whole is pursuing the right objectives overall.

Areas for improvement

In general, and based on interviews, hearings and our own expert opinion, it can be stated that the

challenge of Horizon 2020 Transport to address the broad number of important issues and to deliver

excellence and impactful research and development outcomes under one programme is large.

The Expert Group identified a number of areas where improvements can be considered. These focus

primarily on improving communication between the European Commission (high level) objectives

and the concrete project expectations.

1. Reconciling different objectives

While Horizon 2020 is considered to be on the right track to realise its objectives, there is discussion

if Horizon 2020’s accent on impact and deployment (direct economic objectives) sometimes

overshadows the objectives of fundamental research. New topics need to be able to be brought into

calls quicker and innovation should be valued higher.

Although the overarching goals of the programme are clear, it is more difficult for stakeholders to

establish clear links between the high-level policy objectives and particular project objectives.

International cooperation in Horizon 2020 is seen as being more restricted but more strategic, when

compared to FP7, yet the relevance of SC4 in particular is seen as being highly relevant

internationally. ‘Open to the World’ and international cooperation activities are already included in

the work programmes but some countries are more important strategically for transport than others

(e.g. USA). International cooperation, on the other side, is a strategy issue with always increasingly

importance.

It is recognised that the free flow of information is essential within projects, but this is not yet always

seen as being optimal. Too much openness reinforces doubts about conflict of interests. This was

especially noted in relation to international exchanges, as EU and national industrial interests and


competitiveness on technology-oriented research in transport has a direct relevance on competition

within the European market and outside (for instance, new Asian companies entering the rail sector).

Perhaps the main instrument for realising open innovation is through networks of participants both

in general calls and in PPPs while, at the same time, protecting confidentiality.

A number of interviewees and other stakeholders questioned whether the societal challenges in

broad transport work programmes cover too many aspects to be properly treated under the current

programme. They suggested that it might be better to concentrate (or at least focus extra efforts) on

where the biggest gains can be obtained in terms of achieving the general objectives and added

value. Some respondents referred to the option to focus on market failures (those areas that actors

are not eager to tackle but that are vital for the society and the achievement of the policy goals).

2 Improve and clarify the links between overarching objectives and research calls

The process to develop the work programmes is mature and efforts are continuously being made to

strike the right balance between competitiveness and policy (although this is not yet fully

convergent. Stakeholders do not always find it easy to link the high level objectives to the specific

calls. It is felt that although this may be clear within the EC, but the intended participants are not

aware of these internally agreed objectives and the links upstream could be made clearer. The new

STRIA could fill this gap by producing roadmaps and to more precisely make the links between

research and design pathways to achieve the overarching objectives.

For example, researchers do not always get a clear message of what is expected from them in some

specific topics, in particular those that are less prescriptive, in respect to policy linkages. How the

objectives regarding Open Science, Openness to the world etc. are linked back to the decision making

process is not clear. It is suggested that progress towards objectives should be measured at

programme level and not only at project level. In other words, to explain how project X reduction in

pedestrian fatalities in the elderly age group adds up to the reduction of project Y in the risk of

severe head injuries of cyclists and to the improvement in frontal crash avoidance in rural roads of

project Z in order to reach the general goal of reduction fatalities in the EU by 50% in the next

decade. A simple but effective system could be developed with inputs from stakeholders on how this

might be achieved.

3 Complementarity and continuity of research calls

The overall opinion of the Expert Group is that individually projects yield relevant results: “as

generally there are no bad projects”. Several experienced participants (who may have been involved

with earlier framework programmes) are concerned that they see calls coming up that resemble

work done in previous projects reinforcing incremental rather than innovative research outputs.

More emphasis should be put on robust and simple structures and tools to maximise the use of

previous research work (a catalogue of research results from projects and a how they are been used

and evolved by later projects, for example).

Stakeholders also felt that the sum of the projects (per call and in total) does not yet yield a

sufficiently integrated approach to provide the answer to the particular societal challenges of SC4. It

is suggested that the project selection process is not focussed enough to form consistent packages of

projects that are complementary and this could be improved. The Expert Group supports this line of

thinking.


New emerging issues (as already mentioned by the TAG, in the Scoping Paper and brought up during

the public consultation) need to be introduced earlier in the preparation of the next round of calls.

This was also mentioned in the White Transport Review 2016. In addition some issues identified by

the above sources are still receiving limited attention in the work programmes (softer modes of

transport were mentioned).

4 Preparation of work programmes and call texts

The majority of stakeholders understand (and appreciate) the process for consultation and input into

the work programmes but they are not always satisfied with the traceability of the contributions in

the final proposal texts. This point is also argued by some members of the Transport Programme

Committee. The impression is that the advice given by the different advisory bodies and stakeholders

are put in a “black box” from which emerges a work programme and calls, and it is only when this is

published that they may see (or not) if their suggestions have been taken up. As the text is published

it is too late to modify this in any way. More explanation on how and why certain choices were made

would be highly welcomed and would improve relationships.

5 Enhanced work programme content flexibility

Many stakeholders keep insisting on the rapid emergence of new research areas and that currently it

may take several years until these topics find their way into the calls. This was mentioned in the TAG

report in 2014 and in its follow-up report submitted to the European Commission (May 2016). This

group also stressed the need to continue with a disruptive rather than incremental approach to

research in order to respond rapidly to shifting transport paradigms with new revolutionary

technologies, business environment and mobility patterns. These views were widely supported by

other stakeholders taking part in this interim evaluation and the Expert Group thinks that this directly

influences relevance as results may arrive too late for market developments. There is a need to find a

better balance to address research topics, which were not initially foreseen.

This could be guaranteed by the introduction of “quick-reaction” small or focused calls via

amendments of WPs or open topics. Transport Programme Committee members should be open to

reserve a small budget for this sort of calls, utilizing express written procedures (a mechanism

frequently used in between scheduled Transport Programme Committee meetings) to expedite the

content approval or by directly delegating to the EC the possibility to define calls of limited scope

without the need of them being included in the biennial work programmes.

6 International cooperation

Caution not to open up research results too much, in view of competition, was frequently mentioned

as an issue. Precompetitive research in coordination with international partners in areas such as

automation is welcomed, but there is a fine line between early precompetitive research and when

such actions enter the competitive zone.

However, there is room for further improvement in:

• Increasing the focus of international cooperation on removing international regulatory obstacles

in large markets for instance with US or Asia.

• Enhancing the deployment of European solutions internationally in countries like China and USA

as currently dissemination seems to be mostly limited to the EU area.


Of particular international relevance is the link between transport research and climate where

European transport research has excellent credentials and also in connection with road safety.

10.2 EFFECTIVENESS

The effectiveness chapter looked into the intended and unintended effects on the programme and

the extent that the Societal Challenge SC4 caused the observed changes or effects. As previously

mentioned these observations are based on the premise of similarities between FP7 and Horizon

2020 project outputs.

When measuring effectiveness in terms of market readiness of research results, there is a positive

development as the output of research shows higher TRL levels especially in relation to the outputs

of research from PPPs.

There are well-defined ambition levels for increasing the effectiveness of results, which are

appreciated by participants. It is also felt that this is an important area to keep in focus especially if

research is to help keep Europe competitive and to protect, maintain and create jobs. It is therefore

clear that the expected impact of the research (proposals) is high and there is nothing to indicate

that this will not be the case.

The Expert Group also found that Horizon 2020 is effective in helping many countries and industries

in the transport sector to overcome the economic down turn. Three areas were mentioned

consistently:

• Ensuring that transport research as such was being addressed by funding important research

topics

• Helping to maintain and create highly skilled job opportunities where Europe is in a leading

position

• Helping countries to increase the effectiveness of member states national and regional research

systems (for instance, through being able to set priorities more effectively against European

research opportunities). This is of particular interest with lower income member states.


1 Transition from FP7 to Horizon 2020

Effectiveness is partly defined by communication about the research process. The transition from FP7

to HORIZON 2020 may have been underestimated. The changes are much bigger than they appeared

and the estimated 18 months may have been too little to adapt to the new way of working.

Communication on these changes as well as on the broader relationships between the EC objectives

and the work programmes and project selection should have taken a higher priority.

2 Technology Readiness Levels

Although Horizon 2020 is definitely on track to deliver higher end TRLs, there is some concern about

the relatively smaller attention to research with lower TRL outcomes. Effectiveness should not only

to be measured in terms of quantitative results (e.g. % reduction of CO2). Creating and maintaining

an innovative research ‘ecosystem’ should also be acknowledged as an effectiveness objective of

Horizon 2020 (especially as that is a generally recognised added value of EU intervention). The accent

on quantitative targets can lead to pseudo accurateness. The value of learning (as mentioned in the


chapter as ‘the right to fail’) as such seems to be underrated compared to FP7. This presents a

concern that elements intrinsic to delivering excellence in research are either being, or are at risk of

being lost in the implementation of this new system.

There is a need to improve the transition to deployment beyond traditional dissemination and

exploitation activities and this is seen as one of the big challenges in Horizon 2020. Horizon 2020

funded research should not only project transport futures, but also seek to influence them in various

ways. More effort should be put in market readiness through diffusion and involving targeted clients.

From this investigation there is some concern about the relative attention to research with lower TRL

outcomes, which may provide valuable transitions in the short and medium term.

3 Links between socio-economic and behavioural research

The fourth objective of the programme “Socio-economic and behavioural research and forward

looking activities for policy making” appears to be under developed. Long term objectives on socio-

economic and behavioural research and forward looking activities for policy making is underfunded

and not given enough exposure.

10.3 EFFICIENCY

Out of the five criteria, efficiency evoked much discussion among stakeholders and there are

diverging views. Overall it is agreed that Horizon 2020 is clearly being managed and run more

efficiently than its predecessor. There are a number of improvements that have been implemented

and which are generally appreciated by stakeholders, they are seen to have increased the efficient

delivery of Horizon 2020 projects. For example, INEA is seen to be doing its job in quite an efficient

and effective manner. Time between project approval and grant has decreased and the management

of the projects has been standardised.

Although this evaluation did not evaluate the PPPs, they were considered where they touched on the

delivery of the research objectives of SC4. Overall the PPPs are considered by the participants to be a

very efficient (and effective) way to allocate funds to projects and they increase focus on a variety of

otherwise fragmented research topics.


1. Rules and procedures

Although the improvements compared to FP7 are recognized it is still felt that the rules governing

calls are too many and complicated. This is passable for specialists and large institutes but for those

only occasionally participating or for small companies, it constitutes a major barrier.

2. Expectations and terminology of calls

The openness (or broadness as it is often called) of the call topics is not generally considered an

improvement. Although this approach attracts a larger number of interested parties and invites more

responses than a more focussed text, often the smaller, more specific research areas are lost in final

proposals. It is widely felt that as Horizon 2020 call texts are so broad, encompassing many different

possible research proposals, it is also excessively difficult to evaluate and compare proposals

properly.


This also leads to oversubscription on some topics. Here, there is a risk that excellent researchers

become discouraged by high oversubscription rates and low possibility of achieving project funding.

Confidence in the evaluators is certainly not unanimous. Many experienced and knowledgeable

evaluators are not allowed because of the conflict of interest clause which is handled very

stringently, leaving the evaluation in the hands of what are perceived as less competent evaluators.

Feedback from INEA into the policy directorates is not yet standardised. There are few possibilities

for INEA to challenge unclear topic descriptions or make suggestions in respect to how they might be

evaluated in the calls pre-publication and the feedback from the selection process should be

improved so participants understand more clearly what will be evaluated.

3. Two stage process

There are definitely negative feelings about the two-stage approach in its present form. Although the

intention of simplification is appreciated, in order to develop a good proposal the effort required for

these types of proposals is seen as being almost the double. Some would therefore want to go back

to single stage, others would prefer the first stage to be simplified considerably, e.g. by reducing the

number of proposing consortia to a minimum at a much earlier stage.

The Expert Group suggests that some improvements can be put in place in the near future and there

is no need to wait for the next framework programme.

4. Non-negotiation

The role of negotiations about project proposals has dramatically changed since FP7. Some consider

this an improvement as this diminishes the influence of the EC policy officials in the proposal phase.

Others deem this a clear deterioration because of diminished flexibility when submitting what is

really needed. It would be helpful to revisit the whole decision making chain from objective to

project to identify opportunities to improve the project and to make necessary changes or

modification identified during proposal evaluation or negotiation phase.

10.4 COHERENCE

10.4.1 INTERNAL COHERENCE

The coordination and mix of different actions of Horizon 2020 Transport (CSA, RIA, IA, conferences,

etc.) should provide a coherent approach to address the objectives of SC4 and Horizon 2020, and

especially create synergies between research fields and topics. The internal coherence of the

programme in terms of available instruments has noticeably improved and there is a general support

of the current mix of instruments. In particular, the new SME instrument gets very positive feedback

and the variety of PPPs are acknowledged to be important.

Several other challenges faced by the Union are relevant for transport and represent an opportunity

for the sector. In particular the link with the Energy Societal Challenge is evident. There are also clear

links between Transport and several KET.

On the whole, the structure, goals and instruments of Horizon 2020 are seen as being internally

coherent and coherent with the ambition and objectives of the Union.



1. Coherence of instruments and types of actions

Although smaller than in FP7, the number of instruments is still quite large, and caution should be

taken when considering new ones. This should continuously be kept in mind, with a view to

simplification. Several actors refer to difficulties to understand the difference between the different

instruments, such as a RIA and an IA. Information and definitions should be more precise and

clearer. New instruments should only be acceptable after a clear gap analysis and under the principle

of “one new comes in, two old go out”.

Industry participation in some CSAs is still seen as being insufficient, particularly when they are used

to monitor progress, prepare new technologies or roadmaps.

The public procurement of innovative solutions is still underexploited, in particular the results of

large-scale pilots. Some instruments are either underused or unanswered within Horizon 2020

transport. For example the take up of ERA-NET is not yet widespread and while researchers are

rather interested in CSAs, it seems that the level of interest of (national or regional) public

administrations is not yet equivalent.

2. Relationships between PPPs and other SC4 instruments

Despite the general appreciation of PPPs, it is considered that some are working better than others.

It is suggested to increase coordination among PPPs and JUs by integrating their work programmes

more closely, and learning from those that are now quite mature. Care to ensure that there is the

opportunity for the research community to take part in research with the JTUs is also key (rail and

the Shift2Rail PPP was mentioned here in particular as the research community considers itself

excluded from influencing research topics, calls and to some extend even responding to the limited

number of calls that are made outside of the PPP).

The Expert Group would like to recommend to increase efforts to ensure that modal ‘research silos’

are not created and that the value of an integrated approach, in respect to responding to the

objectives of SC4, and to making Horizon 2020 coherent with transport being smart, green and

integrated, is promoted more forcefully.

3. Combined funding opportunities

Existing links between transport and other societal challenges are not always translated into

combined funding opportunities from the different challenges and instruments associated with

them. The need for more joint calls has been proposed on several occasions (for instance in the

maritime domain combining fishing and transport issues) but this is not yet widespread enough. It is

also thought that deepening combined funding opportunities would also help respond more fully to

the encouragement for ‘new ways of working’ suggested by the Juncker Commission (Open

Innovation, Open Science and the Presidents letter to the Commissioner).

In spite the existence of internal coordination mechanisms within the EC, such as the Challenge Task

Forces, more coordination inside the EC is frequently mentioned. In particular, it is recognized that

efforts have been made within the EC services to institute cross-cutting/transversal themes and

related coordination structures which are successful in a number of fields, notably the relation

between SC4 and SC5, it is felt that this could go further.


Increased transversal coordination is suggested to be encouraged within advisory groups and ETPs,

since feedback from stakeholders suggests that they are also thinking “vertically” within this

transport challenge.

It is suggested that FP9 increasingly gathers efforts from all DGs into key societal challenges. Once

these overarching objectives are defined, contributions from the various DGs could be combined into

a really coordinated research and innovation programme.

4. Role of advisory groups

It is felt that some advisory groups are oriented towards direct interests and specific research areas,

and less towards the usefulness of the sum of the total of research outputs, supporting a silo

approach. Technology platforms are an example of this in some cases. It is suggested that they could

produce roadmaps that address more than one societal challenge and this could reinforce coherence

between the various research topics affecting transport (for instance, it is difficult for ACARE SRIA to

influence other work programmes outside transport).

The Transport Programme Committee could provide support to participants in terms of facilitating

information on national and regional opportunities in the transport research and innovation arena.

5. Communication as an instrument for coherence

Although the Participant’s Portal represents a notable improvement, it is still difficult for many

participants to identify all opportunity for transport innovation. Despite efforts to simplify access it

was still considered by many stakeholders to be too complicated – differing terminologies and

generally there is still progress to be made in simplifying and clarifying processes and procedures.

Researchers still mention that they find it difficult to identify and take advantage of all opportunities

(outside Pillar III). Improving the clarity of information on opportunities for transport in the ERC, FET,

KETs etc. would be welcomed by many. More information was also requested regarding new

instruments such as the Open Research Data Pilot of the Fast Track to Innovation (FTI) calls.

There are difficulties in properly communicating the number and scope of programmes to the

researchers. Those that are used to participating in European projects are seen as having an

advantage, over new comers. It is suggested that it would be useful to have a ‘map’ of programmes

to help interested parties “find their way” or navigate to those programmes that respond best to

their interests. It would also be welcomed if the work programmes would be more specific about

opportunities in other areas of Horizon 2020. A dedicated section in the work programme is

suggested to centralize information on other internal opportunities for funding within Horizon 2020.

Some NCPs stated that sometimes they feel unprepared for changes to systems and that they do not

receive updated and regular communications from the Commission to help advise them about any

calls that could provide cross-cutting opportunities. The Expert Group felt that this could easily be

addressed by the Commission.

10.4.2 EXTERNAL COHERENCE

The landscape of related programmes to SC4 is clearly very large and having more opportunities is

seen as being positive, as long as there is coordination and overlapping activities are avoided.

Structural Funds play a key role to help more actors and regions become part of the Horizon 2020

top-league (particularly through areas such as research infrastructures and research


capabilities).Cohesion Funds are also seen as being important for European regions and the same can

be said for RIS3.A Spanish region for instance, has successfully utilized Cohesion Funds in connection

with one of the Clean Sky calls and SESAR represents a good example when it comes to using CEF

funds for the deployment of new technologies.

Horizon 2020 is clearly guiding national priorities, in particular in smaller countries and even in those

where there is not a specific transport research and innovation programme. Clean Sky and Shift2Rail

are good examples of this. New cooperative arrangements such as MOUs between PPPs and member

states are emerging and expectations are high that they will also accelerate research take up and fill

any gaps that are not presently covered by Horizon 2020.The MoU between Clean Sky and several

European countries or regions in the Czech Republic, Spain is one example.

Member States also use European Technology Platforms roadmaps and agendas (such as, for

instance, the aeronautics SRIA) as guidance for their programmes. In this regard, there is certain level

of coordination and also coherence in objectives.


1 Coherence with other EU funding instruments

Structural Funds and Cohesion Funds help to bring in more actors and regions, however, the

coordination of Horizon 2020 research with such funding sources in the design stage (and later) is

perceived to be complicated because of the different rules. In fact, it was not possible to identify

projects that explicitly present at the beginning how they plan to combine different funding sources

until they reach the market or the solutions are deployed or reach the society. There are still legal

barriers among different DGs or EC initiatives that hinder the combination of funds (for instance

funds from DG Regio and funds from CEF).

Although sometimes awareness and knowledge of external partners is limited, respondents show a

keen interest in the involvement of external parties, especially in finding other financing sources to

assure continuity of implementation of solutions after the completion of research. External partners

mentioned were financial institutions, industrial partners as well as governments. On the other hand,

research partners are slightly wary of too large such involvement at an early stage as this may

influence the independence of research and introduce too many political elements into the research

proposals and compromise the research itself.

Generally it is extremely difficult for participants (in particular for industry participants) to be experts

in the requirements of all opportunities in all related areas: Horizon 2020, Structural Funds, EIB,

especially as there are different financial rules in the different funding programmes. Organizations

and companies currently need to have dedicated experts in all the instruments in order to take

advantage of the opportunities.

2 Coherence with regional or national programmes

There is still a need for better communication and better coordination concerning national and

Horizon 2020 research agendas and outcomes. The relation with regional funds is still considered

weak. The respondents to the Expert Survey were generally not aware of any structural effort to

involve regional funds in the transport programme and they frequently referred to the complexity

implied in different programmes having different financing rules.


Especially in view of the supported and increasing coordination with national programmes, there is

still a need for better communication and better coordination. The Transport Programme Committee

might provide support to participants in terms of facilitating information on national and regional

opportunities in the transport research and innovation arena

It is thought that more seconded national experts working in the EC, could enrich both sides (EC and

nations and regions).

10.5 ADDED VALUE

This criterion addresses the question of added-value resulting from the EU intervention(s), compared

to what could be achieved by member states at national and/or regional level.

Out of the five criteria, EU added value consistently scored highly and creating and maintaining an

innovative research “ecosystem” is seen as being important. While recognising that the respondents

to interviews, surveys, hearing etc. are all more or less involved professionally with EU research,

which could create a bias in favour of the existence of an EU research programme, it was also noted

that these respondents were sufficiently critical of the programme to provide valuable input. It is

therefore safe to say that this bias does not stand in the way of the conclusions that the added value

of EU funded research is broadly recognised as being very good.

The most frequently mentioned positive reflections in terms of added value are:

• Without EU funding many projects would not have gone ahead (or not in the same scope);

• Increased competitiveness, access to larger markets and new business opportunities;

• Contributions to excellence of science, enhancement of scientific reputation and improving

scientific quality

Horizon 2020 (as well as FP7) is seen as being highly instrumental in creating a strong and connected

European research community, based on open exchange of knowledge and researchers. This is

essential in view of the increasing complexity of the research landscape, new research methods and

the requirement to involve interdisciplinary top-researchers to address Europe’s societal challenges,

especially SC4.

PPPs and JUs are acknowledge as positive instruments in terms of market impact as they address

industry needs and help align research to them. In particular the sharing of costs and risks in the

development of new technologies, as well as, testing and validating multi-player demonstrator and

pilot exercises.

Setting the basis for standardisation (as a follow up of research) is seen as being an important

outcome of European added value. There is also a deep appreciation on the dissemination of results,

the open science approach and opportunities that the EC offers notably conferences like Transport

Research Arena and AERODAYS.


The following issues require more attention to improve on the (perceived) added value of the

programme.


1. Increased emphasis on implementation, deployment and cross fertilization of existing research

There is still not enough research and innovation being implemented from FP7 or (to date) Horizon

2020. This includes follow-up from pilots to full scale deployment. Participants expect more support

from the Commission to help take results to the market and to feed results of (small) projects and

demonstrators into other sectors. Even from the Commission side, interest sometimes seems to

wane after project completion.

The cross fertilisation through sharing research findings between transport modes and even other

societal challenges would also add more value. The programme relies too much on participants

themselves looking for external opportunities to exploit project results. Some useful lessons can be

learned from SESAR, on the use of complementary instruments like CEF to support deployment of

successful outcomes.

Possible solutions identified from interview material and Expert Group knowledge include:

• Making more resources available from the Commission to assist following up research and

innovation content and increase exploitation.

• Organising earlier commitment from clients.

2. Increase coverage and equality in research

The programme should encourage participation of new players and/or improve coordination with

other instruments as means to enhancing the impact of the Horizon 2020 Programme and reduce

disparities. Addressing the disparities that exist in transport research has to be in balance with

scientific excellence, gender and skills.

All modes should feel that they are equally covered – it has already been mentioned that softer

modes need more attention. Currently the EC plays a limited role in the maritime sector and it is felt

that this could and should be much larger, with a focus on improving cooperation key players and

bringing multimodality into play more strongly.

3. Future research directions organizing for innovation

Stakeholders identified some potentially fruitful areas where they would like to see action taken,

such as more cooperation and where possible system integration of transport modes (a systems

approach), logistics, ITS, infrastructures, the softer modes of transport (that are much less

organized). In spite of the broader call topics, there seems to be little incentive for interdisciplinary

research. Areas where this is needed include the new forms of digitalisation and automation,

interconnectivity, cyber-security, drones, cloud applications, the interface between technology and

social acceptability, demographic changes such as the recent explosion of migrants into Europe,

changes in mobility behaviours, the sharing economy and unanticipated consequences of new forms

of transport were all mentioned by stakeholders.

4. Delivering research and innovation results that feed European policy

There is a need to improve information to projects concerning expectations in terms of impact.

Better guidance and monitoring mechanisms are required to ensure harvesting projects relevant

data into policy development.


5. Increased involvement with transport relevant players

Increasing the involvement with civil society was also remarked on in previous evaluations and to

some extent this has improved but not yet to the extent that is desirable. There are still many

(possibly too many) traditional players, few representatives of the ‘softer modes’, especially walking,

passenger bodies and civil society organisations that may be only indirectly associated with transport

but are key players in societal issues including SC4.

6. Dissemination

The efforts for increased dissemination via the Horizon 2020 Portal and dissemination through

conferences such as bi-annual Transport Research Arena are well appreciated.

However, there seems to be a need for a clear definition of what dissemination means for projects,

what the programme expectations are and what the criteria are to evaluate the effectiveness of

dissemination.

Improving the added value of project results by better dissemination (especially beyond those

directly involved in the project) beyond the research community and also outside of Europe is seen

as being useful. To some extent this is also associated with exploitation mentioned earlier.

7. Improving the added value of some instruments

Some of the instruments are not easy to access. For example, many respondents were aware of the

ERA-NET facility as a means of cooperating on a European scale and although it is believed to have

many advantages in terms of added value, as it is clear that the motivation to use it lies with the

member states. The drawback of ERA-NET is that it is necessary to have initial funding from a number

of partners before the EC can top up the budget (and then lays down the rules). INFRAVATION was

mentioned frequently as a good example of how ERA-NET may work, with a specific accent on

innovation in the transport infrastructure field. At the same time it is felt that the rules governing

ERA-NET might be revisited to widen the possibility of participation.

10.6 CONCLUSIONS AND RECOMMENDATIONS

This report is clearly structured around the five main criteria that the Commission requested the

Expert Group to consider – namely: Relevance; Coherence; Effectiveness; Efficiency and Added

Value. The Expert Group also chose to look at how Horizon 2020 was responding to certain inward

looking aspects chosen from key high level Commission objectives such as to what extent Horizon

2020 was helping to encourage new collaborative ways of working and cooperation across portfolios

(and linking to the sections looking at coherence). In addition, it touched on considering if the

objectives of the initiatives/ programmes and research topics were clear, to what extent it could

show an increase in results based outcomes (with policy feedbacks) and how modes and sectors

were increasingly integrated in respect to the SC4 challenge.

It was also necessary to make links with a number of key outward facing aspects that would help to

underpin the relevance of the leadership positioning of European research in terms of Europe’s

responses to job creation and competiveness; climate resilience and low carbon economy; the

societal challenges of inclusion and equity (including safe and seamless transport); and the ambition

of the Commission for Open Science; Open Innovation and being Open to the World.


Overall the Expert Group can wholeheartedly say that there is enthusiastic and broad support for EU

research and in particular Horizon 2020, among all stakeholders, from industry to member state

governments and the research world. Its role in advancing cutting edge transport research is

undeniable. There is also definitely support for a new FP.

Horizon 2020 is seen as being of great relevance for EU policy, environmental sustainability, society

and help keep Europe’s economy competitive. It is also seen to have growing relevance in emerging

areas of societal interest such as security, the fight against climate change

The following recommendations are based on the perceptions of stakeholders involved in the

Horizon 2020 programme, the Expert Group findings and expert judgement.

1. Shorten timelines to be able to adapt to change

The programme’s flexibility should be increased to accommodate the sudden emergence of

important topics, at the same time increasing opportunities to link portfolios and funding

opportunities. It is recommended to consider the feasibility of focused “emergency calls” in between

the publication of biennial Work Programmes or, as an alternative, the introduction of a permanent

topic on “emerging issues” in all calls.

2. To consider the focus of the programme

The Commission should consider if focussing the programme on a smaller number of really pressing

transport challenges that can only be tackled on a European level and have the greatest EU added

value has more advantages than disadvantages. At the moment funds are too widely spread over too

many issues for this to take place. Although it is recognised that it is necessary to achieve a balance

between the many different smaller and larger stakeholders, the different modes, industry, policy

and other interests, the current balance of players tends to yield a compromise that is suboptimal for

all parties and is difficult to explain to a critical general public. Increased attention to this within and

between EU institutions and stakeholders would be welcomed.

3. Overcome current bias on technology within the Societal Challenge 4

There is a perceived technology bias to the allocation of funding and social-economic aspects of

transport are seen as being underfunded. There is a focus on higher levels of technology overlooking

possible low cost and accessible solutions that could help bridge a number of gaps and help in the

transition phase especially towards low carbon transport and changing behaviour. The Expert Group

encourages the Commission to further close this gap.

4. Improve the balance between short term and long-term objectives

Especially in respect to socio-economic and behavioural research and forward looking activities for

policy making are seen as being underfunded and this should be addressed in upcoming calls and

programmes. The EC could look 5-10 years ahead and propose research themes that support the

necessary developments (back-casting), rather than the present focus of addressing incremental

research.


5. Increase the opportunities for innovation

The Commission should reconsider the balance between incremental research and visionary or

radical evolution. In respect to the choice of topics for work programmes and calls the advisory and

stakeholder bodies by their input seem to consolidate the traditional choice of topics rather than to

be champions of breakthrough innovation. The current decision making process does not specifically

encourage innovations.

6. Review the progress indicators and what is valued.

Progress towards objectives should be measured not only at project level, or at transport model

level, but also at programme level. For instance, the impact of aviation depends also from the surface

transport part of the trip. In addition consider the value of exchange and the creation of robust

multinational research networks and the knowledge gained from cooperation as a progress indicator,

as this is presently undervalued.

7. Improve the clarity and precision of calls

Improving the transparency of the decision making process and giving clearer indications of how calls

will be evaluated would be welcomed and could also address the low rate of success and over

subscription problems currently encountered. New research policies as intended with Horizon 2020

or priorities such as “Open innovation” must also be better explained and understood by Transport

research participants and there must be enough time to adapt to these changes. It is still worth the

investment in communication for the rest of the Horizon 2020 work programme.

8. Revise and simplify some structures and processes

The following areas would benefit from streamlining and simplification of processes:

• Simplify (access for participants to) existing instruments and actions (rather than creating new

ones) such as ERA-NET

• Revisit the two-stage proposal process taking into account the key problematic areas

• European Technology Platforms should produce roadmaps that include specific targets to be

achieved and ways to control the progress towards them

• Consider turning TPs into Innovation rather than Technology Platforms.

• Consider putting more emphasis in the evaluation criteria on activities that would support the

market introduction and deployment of research findings, and especially to reinforce the links

with policy. This could be achieved by the increased weighting of impact evaluation criteria:

impact on policy, impact on competitiveness, impact of sustainability, impact on economy and

society at large.

9. Improve knowledge sharing especially in the following areas:

• Knowledge of the results of previous projects

• Knowledge about funding opportunities from other sources and improve links with other funding

instruments

• Improve dissemination of results (including tools) outside of the research community


10. More attention to a system-wide innovative approach

In order to achieve SC4, transport needs to be addressed as a system rather than as a sum of modal

solutions, with a stronger accent on innovation. Possible areas of action include:

• Consider transforming the “European transport corridors” financed by the CEF into “European

transport and innovation corridors” where the development, testing, demonstration and

deployment of transport innovation would acquire a more important role.

• Reinforce the links between EU level Transport research and innovation and national and

regional action.

• Support regional transport research cooperation efforts in particular in areas with less tradition

such as the South of Europe or Eastern Europe and using the inspiration of other regions or areas

such as the France-Germany transport research joint programme, Nordic cooperation…

• Develop clearer links between transport research and innovation and skills for jobs (for instance,

in the new market of electric vehicles and battery manufacturing and recycling).

• Consider preparing a report on “transport research and innovation opportunities across all

Horizon 2020 areas”, as a learning exercise and base for further dissemination actions.

11. Retain high levels of knowledge and professionalism

It is recommended that note is taken of the risks of under resourcing the management of the

programme on the Commission side in terms of expertise (including the outsourced executive role of

INEA) and taking care that there is a good balance of project management (necessary for the efficient

running of the programme) with practical knowledge and policy experience in place to provide

content support to projects.

CONCLUSION

In conclusion the Expert Group can see a high level of potential within this challenge of Horizon 2020.

There are a number of noticeable improvements in Horizon 2020 compared to FP7 and efforts to

follow through on previous evaluation recommendations by the Commission are clear.

Despite this there are still a number of key areas especially in terms of linking research to job

creation, policy development and taking a more holistic approach to find solutions to this Societal

Challenge that can further improve both Horizon 2020 and future Framework Programmes. The

important economic contribution the transport sector brings in terms of direct and indirect European

employment opportunities are widely recognised. However the negative externalities such as loss of

productivity (in the region of 2% of GDP) due to congestion, and the social burden of poor air quality

and traffic accidents cannot be overlooked as these need to decline more swiftly than current trends

indicate.

Europe must not only face major societal challenges as outlined in SC4 but use its research

programmes to help to create a new paradigm for transport, based on evidence of what works. This

is critical for its transport systems and will help it shift away from the current reliance on fossil fuel,

build new skills and ensure that it is ready for major socio-demographic changes.


LITERATURE Analysis of the Role and Impact of Industry Participation in the Framework Programmes, ppt, EC 2016

Analysis of the Role and Impact of Research Performing Organisations in the Framework

Programmes, ppt, EC 2016

Aviation 15 years of Research and Innovation – Success stories; Advisory Council for Aviation

Research and Innovation in Europe (ACARE), 2016

Clean Sky 1st Interim Evaluation, December 2010

Clean Sky 2 Impact Assessment, final report Expert group, EC September 2012

Commitment and Coherence, ex post evaluation transport research FP7, EC November 2015

CORDA Horizon 2020 Dataset xls, EC 28-6-2016

Data Dictionary, Horizon 2020 CORDA standard definitions, EC 22-April 2016

Dataset on impact criterion for projects Horizon 2020, EC August 2016-08-23

Diverse documents relating to Horizon 2020, Corda etc.

ERTRAC, 12 Success Stories, ERTRAC April 2016

European Green Vehicles Initiative Impact Assessment, EC 2016

Executive Summary Of The Impact Assessment Accompanying the Communication from the

Commission 'Horizon 2020 - The Framework Programme for Research and Innovation', EC, 2011

First Horizon 2020 Monitoring Report 2014, EC 2016

First Orientations towards Scoping Paper Transport Horizon 2020 WP 2018-2020, EC 2016

From collaborative research projects to market deployment, 12 success stories; European Road

Transport Research Advisory Council (ERTRAC), 2016

Horizon 2020 Indicators, EC 2015

Horizon 2020 Work Programme 2014 – 2015 Smart, green and integrated transport

Horizon 2020 Work Programme 2016 – 2017 Smart, green and integrated transport

Impact Assessment Accompanying the Communication from the Commission 'Horizon 2020 - The

Framework Programme for Research and Innovation' (EC, Commission staff working paper, Brussels

30 November 2011)

Impact Assessment of the European Green Cars Initiative, EGVI, 2016

Interim Evaluation Horizon 2020, Template for the thematic contributions, EC 2016

Intervention logic Horizon 2020 evaluation, and detailed version, EC July 2016


Interview guidelines, EC internal doc

Kick off Meeting, ppt, EC 26-6-2016

Open Innovation, Science and World, EC 2016

PPMI Survey results on Added Value, EC September 2016

Preliminary Data Horizon 2020 Transport, xls, EC June 2016

Regulation (EU) No 1291/2013 of the European Parliament and of the council of 11 December 2013

establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and

repealing Decision No European Union (2013), 1982/2006/EC

Report Support Group Horizon 2020 evaluation, EC internal July 2016

Second Interim Evaluation of the Clean Sky Joint Undertaking, EC, November 2013

Strategic Research and Innovation Agenda, Introductory paragraph, EC 2016

Survey model for Horizon 2020 NMBP projects, EC internal doc

Terms of reference interim Evaluation Transport Horizon 2020, EC June 2016

The Contribution of Framework Programmes to Major Innovations, ppt, JIIP June 2016

The Ocean of Tomorrow Projects (2010-2013) Joint Research Forces to meet Challenges in Ocean

Management; European Union (EU), 2014

The Role and Engagement of Universities in the Framework Programmes, ppt, ISMERI EUROPA, June

2016

Transport advisory group reports: Response to 7 questions, EC June 2014; Summary advice on WP

2016-17; European Transport decarbonisation, EC December 2015, Final Transport Advisory Group

report, EC May 2016

Transport in FP7, Facts and Figures, EC, version 26-2-2014

Transport Research and Innovation Achievements Report , draft, EC March 2016

TriValue Cooperation Programme, Deliverable 4.2

TriValue, Ex post evaluation of Transport Research and Innovation in the FP7 “Cooperation

Programme”

White Paper, Roadmap to a Single European Transport Area – Towards a competitive and resource

efficient transport system /* COM/2011/0144 final

WP 2018-2020, Summary of replies to public consultation, EC


ANNEX: SURVEY QUESTIONS FOR PROJECT

COORDINATORS Project data

Although, as explained, results will be presented in an aggregated manner (not at project level), we

ask you to provide the following data so we can get a better understanding of your answers when we

process the information.

What is your project acronym?

Project number:

Start date (month, year):

End date (month, year):

Transport sector addressed:

- Aviation

- Rail

- Road

- Waterborne

- Infrastructure

- Intelligent Transport Systems

- Logistics

- Urban transport

- Intermodal or multimodal

- Other, please specify

Programme objective addressed:

- For Horizon 2020 projects (ref. https://ec.europa.eu/programmes/horizon2020/en/h2020-

section/smart-green-and-integrated-transport):

o Resource efficient transport that respects the environment

o Better mobility, less congestion, more safety and security

o Global leadership for the European transport industry

o Socio-economic and behavioural research and forward looking activities for policy

making

o Other, please specify

- For FP7 projects:

o Aeronautics projects

The greening of air transport

Increasing time efficiency

Ensuring customer satisfaction and safety

Improving cost efficiency

Protection of aircraft and passengers

Pioneering the air transport of the future

https://ec.europa.eu/programmes/horizon2020/en/h2020-section/smart-green-and-integrated-transport

https://ec.europa.eu/programmes/horizon2020/en/h2020-section/smart-green-and-integrated-transport


Other, please specify

o Surface projects

The greening of surface transport

Encouraging modal shift and decongesting transport corridors (co-modality)

Ensuring sustainable urban mobility

Improving safety and security

Strengthening competitiveness

Other, please specify

In your opinion is your project more policy oriented or more technology oriented?

- Technology oriented

- Policy oriented

- Both

- I don’t know

Type of action:

- For Horizon 2020 projects:

o Innovation Action (IA)

o Research and Innovation Action (RIA)

o Coordination and Support Action (CSA)

o Other, please specify (Fast Track to Innovation -FTI, SME Instrument, Inducement

Prize, ERA-NET Cofund…)

o Other, please specify

- For FP7 projects

o Collaboration Project (CP) – Level 0 (aeronautics only)

o Collaboration Project (CP)

o Collaborative Project – Focused Project (CP-FP)

o Collaboration Project – Integrated Project (CP-IP)

o Coordination and Support Action – Coordination Action (CSA-CA)

o Coordination and Support Action – Support Action (CSA-SA)

o Other, please specify (ERA-NET, ERA-NET+…)

Project website (if already available):

Relevance of the Horizon 2020 Transport activities (Note: if you are an FP7 coordinator, please skip

this entire section on “Relevance”)

1. Would you say that the overarching Horizon 2020 Transport programme objectives, beyond

those of your topic, are clear enough? Note: if you are an FP7 coordinator, please skip this

question

Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don’t know

2. Do you think that the overarching Horizon 2020 Transport programme objectives take into

account the latest scientific, socio-economic, political or any other nature developments in


the field of transport research and innovation at the national/European and international

level? Note: if you are an FP7 coordinator, please skip this question

0 (not taken into

account)

1 2 3 4 5 (fully taken into

account)

I don’t

know

Scientific

Socio-economic

Political

Other emerging

needs

Other emerging needs (please comment briefly):

If you have selected a value between 0 and 3 in any of the items, please briefly explain:

3. To what extent does the Horizon 2020 Transport programme address stakeholders’ needs in

terms of tools used? Note: if you are an FP7 coordinator, please skip this question

0 (not

covered)

1 2 3 4 5 (fully

covered

I don’t

know

Degree to which the stakeholders’ needs

are covered


4. To what extent does the Horizon 2020 Transport programme address stakeholders’ needs in

terms of research areas covered? Note: if you are an FP7 coordinator, please skip this

question

0 (not

covered)

1 2 3 4 5 (fully

covered

I don’t

know

Degree to which the stakeholders’ needs

are covered


Internal coherence (among Horizon 2020 activities)

5. To your knowledge, what other Horizon 2020 areas or programmes also represent an

opportunity for transport research, innovation and implementation? If you are not an FP7

coordinator, please skip this question

Opportunity Complementary

area

Synergetic

area

Overlapping

area

Duplicated

area

Agriculture &

Forestry

Aquatic Resources

Bio-based Industries

Biotechnology

Energy

Environment &


Climate Action

Food & Healthy Diet

Funding Researchers

Health

ICT Research &

Innovation

Innovation

International

Cooperation

Key Enabling

Technologies

Partnerships with

Industry and Member

States

Raw Materials

Research

Infrastructures

Security

SMEs

Social Sciences &

Humanities

Society

Space

Other: please specify

6. Would you say that there are enough information and coordination mechanisms within

those other areas of Horizon 2020 that are complementary or synergetic for transport? If you

are an FP7 coordinator, please skip this question


7. Please comment on the coherence of the different mix of available instruments (in the case

of Horizon 2020, JU/PPPs, IA, RIA, CSA, SME instrument, etc.; in the case of FP7, JU/PP, CP,

CSA, Competitiveness and Innovation Programme CIP for SMEs, etc.)?

0 (non-

existing)

1 2 3 4 5 (at a 100%

degree)

I don’t

know

Complementarity among

instruments?

Synergy among instruments?


If you have indicated above a low mark (0 to 3), please briefly explain the reasons:

0 (non-existing) 1 2 3 4 5 (at a 100% degree) I don’t know

Overlaps among instruments?

Gaps among instruments?

If you have indicated above a high mark (3 to 5), please briefly explain the reasons:

8. Is your project going to produce interdisciplinary solutions which cut across multiple specific

objectives of Horizon 2020 (for instance, new batteries for electric vehicles, or lightweight

material applications for high-speed trains)?

Totally agree Agree Rather agree Rather disagree Disagree Totally disagree

Please briefly explain:

9. Would you be able to justify that your project is going to simultaneously tackle societal

challenges (i.e. supporting mobility of the elderly while, at the same times, creating new

business opportunities) and give rise to new competitive businesses and industries?


Please briefly explain:

10. From a broader perspective, what is your opinion about the EC’s diversification of research

and innovation programmes (for example research infrastructures supported within

European Strategy Forum on Research Infrastructures ESFRI, or the fundamental research

supported by the European Research Council ERC projects, Knowledge and innovation

Communities launched by the European Institute for Technology EIT…)?

0 (non-existing) 1 2 3 4 5 (at a 100% degree)

Duplication among actions

Complementarity among actions

Synergy among actions

If you have indicated that there are duplications or you have given a low mark (0 to 3) for

complementarity and synergy, please briefly explain the reasons:

External coherence (still within the EU but beyond Horizon 2020 activities)

11. In your country, and based on your experience as researcher, would you say that the Horizon

2020 Transport research programme is influencing your national research programmes?

Very

negatively

Negatively No

influence

Positively Very

positively

I don’t

know

Is Horizon 2020

influencing national

research activities?


If you have indicated that Horizon 2020 is having a negative influence on your national programme,

please briefly explain

12. Do you think that complementarities and synergies are necessary with other public support

initiatives such as the Connecting Europe Facility (CEF), European Structural and

Investment Funds (ESIFs), Regional Research and Innovation Strategies for Smart

Specialisation (RIS3), Smart Cities European Innovation Partnership EIP, European

Investment Bank’s European Fund for Strategic Investments (EFSI), LIFE programme?

0 (not

necessary at

all)

1 2 3 4 5 (totally

necessary)

I do not

know

Connecting Europe Facility (CEF)

European Structural and Investment

Funds (ESIFs)

Research and Innovation Strategies for

Smart Specialisation (RIS3)

Smart Cities EIP

EFSI

LIFE

If you have selected a low mark (0 to 3) in any of the items, please briefly explain:

13. Is your project using previous research from other EU-funded projects or from other national

or regional projects:

- From other previously EU funded projects

- From previously funded national research

- From previously funded regional research

- From research previously conducted at internal level by the participants

- Other, please comment

14. If you project is based on previously funded research (see previous question), do you think

that the previous funding research programmes (either EU, national, regional or internal)

represent a coherent scenario for research and innovation?

0 (not coherent at all) 1 2 3 4 5 (fully coherent)

Coherence among research programmes

Effectiveness

15. How many of the following human resource related activities were undertaken during or

after the project?

Number already

achieved to date

Number targeted or

projected at the end of the

project

I don´t

know

Number of training events organised

Average number of participants to 1


training event

Total number of PhD Thesis and Master

Thesis supported by the project

Total number of personnel involved in

the project implementation - males

Total number of personnel involved in

the project implementation - females

16. Please indicate what impact your FP7 or Horizon 2020 participation has had (or is expected

to have) for you or your partners in the area of human capital.

Strong

increase

Medium

increase

Low

increase

No effect I don´t

know

Gender equality

Mobility of own researchers

Ability to hire researchers

across the EU

17. How many of the following outcomes were achieved or will be achieved?

Number

already

achieved to

date

Number targeted or

projected at the end

of the project

I don´t

know

Publications in peer reviewed journals or books

Publications at conferences

Patent applications

Prototypes (i.e. new materials, physical parts,

machinery, facility infrastructure)

Pilot scale applications / demonstration

activities

Testing activities -- validations & verifications

Software, tools, methods, databases, models,

system architecture applications that are not

marketed (i.e. internal tools, etc.)

New norms and standards

New products introduced to the market: new

materials, physical parts, machinery, facility

infrastructure

New services introduced to the market

(includes ICT and non-ICT services)

New technologies

PhD Thesis or MSc Thesis within the project

Assessments (impact assessments, consultancy


studies)

Handbooks and guidelines

Policy recommendations

18. Please identify the Technological Readiness Level (TRL) of the outcome / results of your

project.

TRL level I don´t know

Before the project

Already achieved

Targeted at the end of the project

Not applicable

Note: Horizon 2020 TRL definition can be found in

https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-

wp1415-annex-g-trl_en.pdf

19. Please indicate what level of impact your Horizon 2020 participation has had (or is expected

to have at the end of the project) for you or your partners on research, development and

innovation capabilities.

Strong

increase

Medium

increase

Low

increase

No

effect

I don´t

know

Understanding / knowledge in existing areas

Understanding / knowledge in new areas

Access to physical R&D infrastructure (labs,

technology, …)

Access to non-physical R&D infrastructure

(organisation, coordination, …)

Scientific or technological collaboration with

other institutes/businesses/universities

Access to international networks and

collaborations

20. Please indicate what level of impact your Horizon 2020 participation has had (or is expected

to have) for you or your partners on competitiveness.

Strong

increase

Medium

increase

Low

increase

No

effect

I don´t

know

Access to known markets and

business opportunities

Opening up of new markets and

business opportunities

Competitive position nationally

Competitive position internationally

https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-g-trl_en.pdf

https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-g-trl_en.pdf


21. Were any of the following exploitation activities undertaken during the project? Please tick

the corresponding boxes.

Targeted at the end

of the project

To

date

I don´t

know

Development of business plan and/or market analysis

Direct contacts with stakeholders

Contact with potential clients

Contacts with private investors (Venture Capitalists,

Business Angels...)

Agreements with private investors

Technology transfers

Creation of companies (spin off/start ups) as result of the

project

Creation of association / non-commercial legal entity(ies)

to maintain and further develop project network

Other, please briefly explain

22. Did the project help you to obtain or leverage additional investment or capital from other

sources or from other financial institutions for activities not covered by the European

Commission’s grant (for instance, continuation of the project, additional market activities)?

- Yes

o If yes, please indicate source of additional funding: ______

o What covered that additional funding: ____________________

- No

- I don´t know

23. Assuming that the project research outputs are implemented, do you expect your research

outputs to contribute to(please select as many answer as you wish):

- Decarbonising and "greening" the transport system

- Increasing efficiency of the whole transport system

- Improving safety & security of passengers, aircraft, vehicles and vessels, and

infrastructures

- Strengthening the competitiveness of European industry

- Pioneering the Transport of the future (long term perspective)

- Enhancing and strengthening the European Research Area (ERA)

- Globalisation, external dimensions and international cooperation

- Further development and implementation of EU transport policy

- I don´t know


Efficiency

24. Please provide ratings as regards the following Horizon 2020 processes for your project:

0

(very

poor)

1

(poor)

2

(acceptable)

3

(good)

4 (very

good)

5

(excellent)

I do

not

know

Effort required to find new

calls

Effort required to prepare

and submit a proposal

Effort required to

participate as a newcomer

Clarity of general goals of

the Transport Programme

Clarity of the rules for

participation (funding,

administrative,

reporting…)

Clarity of the calls for

proposals

Adequacy of the topic

budget to the topic

description

Clarity of the evaluation

criteria and processes

The time taken to evaluate

and select proposals

The transparency of the

selection process

The quality of the

selection process

The feedback given in

relation to the final

rejection of proposals

The time between opening

of the call and deadline for

proposal submission

The time from proposal

submission to signature of

contracts and launch of

projects

The grant preparation

process

The procedures for project

monitoring and reporting


The role of the experts

taking part in the periodic

and final evaluations

The time taken to issue

payments to grant holders

Support by the project

officer (technical content

of the project)

Support by the project

officer (administrative,

flexibility…)

The Participants Portal

Networking events

supported or organised by

the EC

If you have indicated any low rating (0, 1, 2 or 3) in any of the items above, please briefly explain:

25. Only if you have participated in both FP7 and Horizon 2020: which of the following

procedures considering the whole project life cycle did you experience as an improvement?

If you are just an FP7 coordinator or a Horizon 2020 coordinator, please skip this question

FP7 was better

than Horizon

2020

No change from

FP7 to Horizon

2020

Horizon 2020

is better than

FP7

I do not

know

Effort required to find new calls

Effort required to prepare and

submit a proposal

Effort required to participate as a

newcomer

Clarity of general goals of the

Transport Programme

Clarity of the rules for participation

(funding, administrative,

reporting…)

Clarity of the calls for proposals

Adequacy of the topic budget to the

topic description

Clarity of the evaluation criteria and

processes

Effort required to participate as a

newcomer

The time taken to evaluate and

select proposals


The transparency of the selection

process

The quality of the selection process

The feedback given in relation to

the final rejection of proposals

The time between opening of the

call and deadline for proposal

submission

The time from proposal submission

to signature of contracts and launch

of projects

The grant preparation process

The procedures for project

monitoring and reporting

The role of the experts taking part

in the periodic and final evaluations

The time taken to issue payments

to grant holders

Support by the project officer

(technical content of the project)

Support by the project officer

(administrative, flexibility…)

The Participants Portal

Networking events supported or

organised by the EC

Please add any remarks or explanations we might wish:

26. Only if you have participated in both FP7 and Horizon 2020, please indicate whether the

following changes from FP7 to Horizon 2020 have had any impact in the effectiveness of the

Horizon 2020 Transport research programme:

Major

negative

effect

Minor

negative

effect

No

effect

Minor

positive

effect

Major

positive

effect

I don´t

know

Multiannual Work

Programmes

Less prescriptive topics

New types of actions

(mainly IA and RIA

instead of CP)

Two-stage evaluation

If you have indicated “negative effect” in any of the items above, please briefly explain:


EU added value

27. What were the main reasons to apply for EC funding for your project?

Not

important

at all

Slightly

important

Important Very

important

Extremely

important

I

don´t

know

Contribution to EU

transport policy

objectives

Funding opportunity

Cooperation with other

countries/institutes

Access to complementary

expertise

Maximise exposure of

results

Understanding the needs

and characteristics of a

wider market

Overcoming specific

barriers to the market

uptake of research results

Capitalising on previous

EU projects experience

and networks

Other

If other, please briefly explain:

28. In your opinion or based on your experience, would a similar project have taken place at all

without the EC funding?

- Yes

- No

- I do not know

29. In particular, does the Horizon 2020 Transport programme offer participants unique

opportunities to do work that could not be done through other programmes (for instance,

because of the inter-regional collaboration)?

- Yes

- No

- I do not know

30. In your opinion, would you say that the Horizon 2020 Transport programme addresses policy

objectives (such as resources efficient, safe, environmentally friendly transport…) which are

not covered by other European or international initiatives?


If you have disagreed, please briefly explain:


31. To what extent does the fact that this is an EU research project influence the impact of the

project?

0 (no

impact)

1 2 3 4 5 (maximum

impact)

I do not

know

Influence on the impact of the

project

Degree of synergies

32. Attributable to the EC funding, what kind of additionality effects do you expect in your

project, e.g.:

Added value/ size Little

added

value

Medium

added value

Much

added

value

I don´t

know

Funding being leveraged in addition to the EC

contribution(input added value)

Faster progress (added value by acceleration)

Work done on a larger scale(scale

additionality)

Risk reduction

Improved market position or

competitiveness

Improving scientific quality

Other

If you indicated “little added value”, please explain your answers briefly:

33. Attributable to EC funding, are there any conditions that hamper creation of impact of your

project, e.g.:

Hampering factor Please provide brief explanations Not applicable

Compromises on scope and/or method

Less proficient partners from other countries

Other factors, please explain briefly

34. Did you consider applying for funding under the ERA-NET programme?

Please explain briefly

This is an ERA-NET project - what are the advantages?

Yes, but this was rejected

No, not considered


Lessons learned

35. Please include any final comments or suggestions for improvement of the Horizon

2020Transportresearch programme(2,000 characters maximum)

Contact info (optional)

Do you agree that we follow-up / contact you?

- Yes

- No

If so, please indicate your contact name:

- Contact e-mail:

- Contact phone number:

THANK YOU VERY MUCH FOR YOUR PARTICIPATION!!!


ANNEX: INTERVIEW QUESTIONS FOR STAKEHOLDERS Introduction

As part of the general interim evaluation of the Horizon 2020 research programme, the European

Commission has asked a group of external experts to evaluate the transport paragraph of the

Programme. This expert group of four has decided to hold ca. 25 interviews among stakeholders,

policy makers, national representatives and project coordinators. The findings from these interviews

will be aggregated and not traceable to the individual interviewees, thus assuring confidentiality.

Together with results from surveys, document and data search they will form the basis of this

evaluation.

The results of this evaluation will be published by the European Commission as part of the overall

Horizon 2020 interim evaluation.

General background

1. Date of interview …../…../2016

2. Name of interviewee (NB.: interview answers will be rendered anonymously) ……………

3. Affiliation (please specify name)

0 Research Provider ……………..

0 PPP ……………..

0 Joint Undertaking ……………..

0 Advisory Committee ………….

0 Technology Platform …………

0 Industry …………..

4. What is the specific responsibility of your organisation in relation to Horizon 2020 and/or

FP7: ………………

5. What is your own specific responsibility with regard to FP7 and/or Horizon 2020

Relevance

6. Please describe in your view the overall objective of the Horizon 2020 SC4 programme. What

needs and priorities is it designed to address? To what extent have (FP7 and) Horizon 2020

work programmes contributed to EC priorities (notably: New Commission (Juncker) priorities;

Open Innovation, Open Science, Open to the World; White paper Transport policy objectives;

European Research Area and Innovation Union flagship initiative)? Do you think today that

the current transport programme objectives are still relevant: have you run some kind of gap

analysis?

7. To what extent did objectives evolve from FP7 to Horizon 2020? How was that

communicated? To what extent do the original (FP7 and) Horizon 2020 objectives still

correspond to the needs and developments within the EU (e.g. socio-economic, political ,

scientific, technological needs and developments)?

8. Does the Horizon 2020 SC4 programme take into account the latest developments in the

field of transport research and innovation at the national/European and international level?

Did circumstances change since the outset of Horizon 2020? How do/did Horizon 2020 work

programmes address these changes?

https://ec.europa.eu/priorities/index_en

https://ec.europa.eu/priorities/index_en


9. How have the research themes been initially defined in FP7 and Horizon 2020? Are you

aware of how are these themes translated into Work Programmes, research calls, project

selection?

10. Looking at Horizon 2020 and FP7, are there any gaps or overlaps that influence the relevance

of the programmes? What elements in particular ensure the relevance of the programme?

Synergies?

11. Which mechanisms are in place to keep ensuring relevance of the transport programme? To

what extent does the programme address stakeholder needs in terms of tools and research

areas covered?

12. In case of PPP or Joint Undertaking:

Please describe to what extent your PPP or TP proposed dissemination and outreach

activities that will effectively reach citizens

Please explain why your PPP or research agenda is relevant to European Objectives: (list

different objectives)

Please briefly explain what is the added value of PPP or JU as an instrument.

Coherence (internal)

13. Please comment on the coherence of the different mix of available actions (CSA, RIA, IA,

conferences, etc)? Does this mix provide a coherent , coordinated approach to address the

objectives (overlap; complementary; synergy; gaps)? How is coordination ensured between

the different initiatives?

14. Please provide illustrations and highlight the explicit actions undertaken to foster coherence

and complementarity.

15. What other work programmes (and their funding mechanisms) exist within Horizon 2020 and

how far do these funds support transport research and implementation? (e.g. Energy,

Climate action, security)

16. What is your opinion about the EC’s diversification of research and innovation programmes

(for example research infrastructures within ESFRI, ERC projects, Knowledge and innovation

Communities launched by the EIT…)?

17. Can you provide examples of projects or programmes that score positively in terms of

coherence (interdisciplinary solutions, supporting cross-cutting objectives or purposes (e.g.

societal challenges and business support))

18. In the case of PPPs or JU’s, how would you justify this particular instrument as part of the mix

of instruments?

Coherence (external)

19. In your opinion does the Horizon 2020 SC4 programme fill a policy gap that is not covered by

other European or international initiatives in the field of transport?

20. How are your research agenda’s influencing the EU FP7 and Horizon 2020 programmes.

21. In particular does the Horizon 2020 SC4 programme offer participants unique opportunities

to do work that could not be done through other regional/national/EU programmes. Was

there a marked difference between FP7 and Horizon 2020?

22. What are the links between these programmes and the Horizon 2020 SC4 programme? Do

you see a need for ensuring coherence and synergies between the Horizon 2020 SC4


programme and European/international programmes serving similar objectives? How do you

think this can be achieved.

23. Are you aware of possible complementarities and synergies with other public support

initiatives at regional, national and international level in the Transport arena? E.g.

Connecting Europe Facility (CEF), Regional Development Fund, Structural and Investment

Fund, Regional Research and Innovation Strategies for Smart Specialisation (RIS3), European

Fund for Strategic Investments (EFSI), LIFE programme? To what extent are they

complementary? Do they act in synergy? Are there any gaps or overlaps? How is it ensured

that these programmes are coherent and complementary with EC actions (explicit actions

undertaken to ensure them)? Please indicate areas where further synergies and

complementarity should be sought.

Effectiveness

24. Overall, what are the main expectations from the projects i.e., the outcomes that the

projects should guarantee to deliver at the end of their lifetime?

25. In your opinion, will/has the delivery of the results together lead to the achievement of the

programme’s objectives (resource efficiency respecting environment; better mobility, more

safety/security; global leadership for Eur. transport industry; socio-economic and policy

supporting research)? Is this measurable and verifiable? In particular, and in your opinion, do

the current and expected achievements of the programme allow to deliver on the SC4

objectives (resource efficient/respect environment; better mobility; global leadership T-

industry; socio-economic policy)?

26. Do you think the project application and selection processes are effective? Are they more

effective than in FP7? In your opinion how could they be improved to maximise the benefits

of the programme? Can you assess the potential benefits of the Horizon 2020 SC4

programme participation for the participants (public and private). Are the participants the

ones that the programme targeted?

27. What do you think will be the main long-term result of the programme? (top 3) If the

programme would not have been implemented, how could this result have been achieved

otherwise?

28. To what extent does the programme communication/valorisation strategy allow identifying,

capitalising upon and (possibly) transferring good practices/results to stakeholders?

29. Are there any elements of the programme that are more or less effective than others, and, if

so what lessons can be drawn from this?

Efficiency

30. How is the Horizon 2020 SC4 programme managed in terms of procedures, administration,

contractual issues etc.? How does Horizon 2020 compare to FP7 in this respect. Do you see

any possible improvements in the way the Horizon 2020 SC4 programme is run?

31. Do you consider that the benefits achieved through the Horizon 2020 SC4 programme have

been/will be gained at a reasonable cost? Is there a difference between the different types of

activities of the programme? Is it possible to compare cost and benefits of the Horizon 2020

SC4 programme with FP7 or with other still existing similar programmes?

32. Do you think the foreseen budgets in the programme are sufficient? What is your opinion on

the allocation of budgets to the different objectives and topics?


Added Value

33. What role does the EC play in coordinating transport research policies in Europe (e.g. reduce

disparities, raise standards, create synergies). Please give examples of added value in this

respect? Has that role evolved from FP7 to Horizon 2020?

34. What is the added value of research outcomes in FP7 and Horizon 2020 that could not have

been achieved at National, level (e.g. publications in high impact journals; patents;

prototypes; public-private publications; cleaner products with lower environmental impact;

more efficient process (e.g. including minimize waste, reuse or recycle); new/alternative

technologies, services, products, knowledge; widening participation)? Please specify and give

examples.

35. What is the added value of the programme in terms of:

- Multiplier effect

- Larger scale of research

- Complementarity in researcher expertise

- Less risk

- Better quality through more expertise

- Trusted and objective process

- Mobility of researchers

- Global actor objective

- Other

36. To what extent does ERA-NET(plus) under FP7 or Horizon 2020 stimulate programmes with

EU added value: please indicate what added value under which programmes, give examples.

Does the EC stimulate ERA-NET? What criteria relevant for added value apply for EU to

participate in ERA-NET(+). How is the combination of ERA-NET and ERA-NET+ in Horizon 2020

working out?

37. What is your opinion about the way projects are taking up their responsibility for EU wide

dissemination? Good examples? How will other EU countries or institutes profit of that

dissemination?

38. Do you have any evidence so far that the research and innovation outcomes will be exploited

or built upon, e.g. with funding from other sources, after the end of funding period of the

projects (sustainability), (added value, valley of death)?

39. Is there any evidence yet that FP7 has had a leverage effect in terms of impact (e.g. speeding

up innovation, market introduction of new technology, additional financing)? In how far is

Horizon 2020 fitted with conditions to achieve a (larger?) leverage effect? Please give

examples. In your opinion, what further actions are needed to maximise the impact of the

Horizon 2020 SC4 programme?

40. Do you have concrete examples of Horizon 2020 Transport contributing to Open Innovation,

Open Science and Open to the world (e.g. improving R&I capacities of participants, increased

transparency, better communication)?

41. To your opinion, what are the current strengths and weaknesses of the Horizon 2020 SC4

programme? What would you modify in the way the Horizon 2020 SC4 programme is

organised and implemented to attain a higher EU added value? Any disadvantages of

participating in EU projects (having to compromise on scope and/or method; bureaucracy,

slow process; IPR issues; less proficient partners from other countries; language issues)?


42. Lessons learned

Please provide three examples of success stories based on an assessment of FP7 projects

whose outcomes/results have had a significant added value and/or impact on the society

and/or the economy and which have outstanding EU added value based on their

effectiveness, efficiency and synergy. Are the instruments IA and RIA better than previous

ones?

43. Do you have ONE success story regarding Horizon 2020 Transport contribution to each of

the following areas:

- Area 1 Resource Efficient transport that respects the environment

- Area 2 Better mobility, less cohesion, more safety (and security)

- Area 3 Global leadership for the European Transport Industry

- Area 4 Socio-Economic and Behavioural Research and Forward Looking activities for

policy making.

44. Final question: do you have any comments, observations, opinions, etc., whether on content

or process of this interview or the Horizon 2020 interim evaluation in general that we have

not touched upon and that you would like to share with us?


This report has been prepared as part of an interim evaluation of the Societal Challenge 4

(Smart, Green and Integrated Transport) for the European Commission and not for

general publication. The information comes from a number of sources including an

Internet based questionnaire, interviews with key stakeholders, a stakeholder hearing

and the collecting of data (some of which were provided by the Commission). Comments

from these sources are confidential and should not be able to be traced back to their

owners or their organisations. Stakeholder views were only considered when they were

also shared by the Expert Group and validated by other evidence to support this opinion.

It is indicated in the text when this is not the case.

Studies and reports