ex-post evaluation of fp6 nmp at project level

275
Ex Post Evaluation of FP6 (NMP) Project Level Research & Innovation Industrial technologies EUROPEAN COMMISSION EUR 24935 EN Studies and reports Research & Innovation Industrial technologies EUROPEAN COMMISSION

Upload: phungtruc

Post on 08-Dec-2016

223 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Ex-post Evaluation of FP6 NMP at project level

Ex Post Evaluation of FP6 (NMP)Project Level

Research &Innovation

Industrial technologies

E U R O P E A NCOMMISSION

EUR 24935 ENStudies and reports

Research &Innovation

Industrial technologies

E U R O P E A NCOMMISSION

Page 2: Ex-post Evaluation of FP6 NMP at project level

EUROPEAN COMMISSION

Directorate-General for Research and InnovationDirectorate G - Industrial TechnologiesUnit G.1 - Horizontal Aspects

Contact: Jesús Alquézar Sabadie

European CommissionOffice CDMA 06/112B-1049 Brussels

Tel. (32-2) 29-51167Fax (32-2) 29-67023

E-mail: [email protected]

Page 3: Ex-post Evaluation of FP6 NMP at project level

EUROPEAN COMMISSION

Ex Post Evaluation of FP6 (NMP)

Project Level

2011 EUR 24935 ENDirectorate-General for Research and Innovation

Cooperation/ Industrial Technologies

Authors:Inno AG and Atlantis Research S.A.

Edited bythe European Commission

Directorate-General for Research and InnovationDirectorate Industrial Technologies

Page 4: Ex-post Evaluation of FP6 NMP at project level

EUROPE DIRECT is a service to help you find answers to your questions about the European Union

Freephone number (*):

00 800 6 7 8 9 10 11(*) Certain mobile telephone operators do not allow access to 00 800 numbers

or these calls may be billed

LEGAL NOTICE:

Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of the following information.

The views expressed in this publication are the sole responsibility of the authors and do not necessarily reflect the views of the European Commission.

More information on the European Union is available on the Internet (http://europa.eu).

Cataloguing data can be found at the end of this publication.

Luxembourg: Publications Office of the European Union, 2011

ISBN 978-92-79-21152-2doi 10.2777/74830

© European Union, 2011Reproduction is authorised provided the source is acknowledged.

Pictures © Fotolia, 2011 (unless otherwise specified)

Page 5: Ex-post Evaluation of FP6 NMP at project level

EX-POST EVALUATION OF FP6-NMP

LOT 1: PROJECT LEVEL

Study N°2008/S 209-277161

Final report – June 2011

This report constitutes the ex-post evaluation of FP6-NMP, conducted at

project level from June 2009 to June 2011. On the basis of the outputs and

outcomes of the funded projects, it evaluates the effectiveness of the

thematic area "Nano-technologies and nano-sciences, knowledge based

multifunctional materials, and new production processes and devices"

within the specific programme for research, technological development

and demonstration "Integrating and strengthening the European

Research Area (2002-2006)"

Page 6: Ex-post Evaluation of FP6 NMP at project level

DISCLAIMER

The opinions expressed in this

document represent the

authors’ points of view which

are not necessarily shared by

the European Commission.

Page 7: Ex-post Evaluation of FP6 NMP at project level

ACKNOWLEDGEMENTS

This analysis is commissioned by the European Commission, Directorate General for research

and Innovation and carried out by inno AG and Atlantis Research, with the support of their

subcontractor PNO. The team is led by Dr Franck Le Gall (inno group) with the assistance of Ms

Foteini Psarra (Atlantis Research S.A.).

The analysis has been managed by unit G1 (Industrial technologies - Horizontal aspects).

Contact person are Mr Michel POIREAU and Mr Jesus ALQUEZAR-SABADIE.

inno AG and Atlantis Research would like to express their appreciation to all of the people,

experts, and organisations that have participated in the study during the various meetings,

surveys and interviews. Without those contributions, the findings of the study would not have

been as complete as they are.

Special thanks are addressed to the Expert Advisory Board created for the study: Dr Peter M.

Nagy, Prof. Benoit Gailly, Dr. David Zaruk, Ms. Neelina Hermina (Ineke) Malsch, Mr. Ken Guy,

Mrs Effie Amanatidou, Dr Marco Falzetti, Dr Jean-Claude Charbonnier, Dr Markus Dickerhof

and Prof. Andreas Gerdes.

Page 8: Ex-post Evaluation of FP6 NMP at project level

FOREWORD

Europe’s sustainable competitiveness and thus its welfare depend on the intensity,

effectiveness and efficiency of its innovation processes. Innovation demands strategic

investment. Investors in general and investors of tax payers’ money in particular need to

ensure that they invest into the most relevant fields and ensure that the return on investment

is maximised. Evaluations are highly instrumental to serving this responsibility.

Consequently, DG Research and Innovation mandated inno group and Atlantis Research to

evaluate the FP6-NMP Programme and to prepare a sound basis for proposing alterations,

adjustments and new mechanisms for the design and elaboration of the FP7 NMP successor.

This report documents the findings from an Ex-post evaluation of the FP6 NMP programme

focussing on the impact generated at the micro-project level. The FP6 NMP Programme has

proven to be a powerful tool to stimulate and enable novel RTD activities that would not have

been performed otherwise at all or only substantially later and less comprehensively. In

parallel to this study, a second evaluation was performed at the macro-strategic level [8].

The main strength of the NMP programme has been its unique capability to bring together the

best European, and in many cases also international, research groups jointly generating

innovation success concretised in many cases by direct commercial returns. The NMP

programme has promoted excellence in industrial technologies research and value generation

throughout all industrial sectors, including nanosciences and nanotechnologies, materials and

new production technologies.

On behalf of the full evaluation team, we want to express our gratitude to the more than three

thousand representatives from business, research and policy areas who spent substantial time

in sharing their experiences and views in personal discussions and in writing with us. We

warmly thank the ten leading experts in NMP areas as well as the specialists in evaluation and

impact assessment of research policy who formed our Expert Advisory Board. We are grateful

to the team of DG Research and Innovation which entrusted us with the mandate of the

evaluation, provided highly valuable input to the study and ensured access to crucial

information.

We hope that this report may inspire and be beneficial in decision processes to boost Europe’s

innovation success and wish you an enjoyable reading.

Peter Heydebreck

Managing Director, inno group

Angelos Manglis

Managing Director, Atlantis

Research

Page 9: Ex-post Evaluation of FP6 NMP at project level

9

CONTENTS

List of figures ................................................................................................................................................................. 13

Part A - Executive summary ............................................................................................................................... 17

1 Introduction .......................................................................................................................................................... 19

2 characteristics of participating organisations and projects .............................................................. 20

3 motivation for participation and outputs, outcomes and results achieved ................................ 21

4 impacts and achievements of the FP6 NMP Programme ................................................................... 22

5 Programme implementation and monitoring ......................................................................................... 23

6 Recommendations for policy and programme management ........................................................... 24

6.1 Recommendations at NMP Policy Level ............................................................................................. 24

6.2 Recommendations at Projects Selection and Implementation Level ..................................... 25

Part B - Detailed report ......................................................................................................................................... 27

1 The NMP Environment In FP6 ....................................................................................................................... 29

1.1 Introduction ................................................................................................................................................... 29

1.2 FP6 NMP Objectives .................................................................................................................................... 31

1.2.1 FP6 NMP Objectives hierarchy .................................................................................................... 31

1.2.2 NMP and international cooperation .......................................................................................... 34

1.2.3 NMP and SMEs .................................................................................................................................... 35

1.2.4 European Technology Platforms ................................................................................................ 35

1.3 FP6 NMP implementation context ........................................................................................................ 36

1.3.1 Instruments ......................................................................................................................................... 36

1.3.2 FP6 NMP Calls ..................................................................................................................................... 37

1.4 Evaluation Issues ......................................................................................................................................... 38

1.5 Objectives of the study ............................................................................................................................... 39

2 Methodology ......................................................................................................................................................... 40

2.1 Introduction ................................................................................................................................................... 40

2.2 Evaluation Framework .............................................................................................................................. 41

2.3 Data Sources................................................................................................................................................... 41

2.3.1 EC Database ......................................................................................................................................... 41

2.3.2 NMP FP6 Participants survey ....................................................................................................... 41

2.3.3 Online Survey for the non-selected applicant proposals .................................................. 44

2.3.4 Selection of Case Studies ................................................................................................................ 44

Page 10: Ex-post Evaluation of FP6 NMP at project level

10

2.3.5 Additional Sources............................................................................................................................. 45

2.4 Statistical Analysis ....................................................................................................................................... 45

2.4.1 Preparation of databases ................................................................................................................ 45

2.4.2 First level analysis ............................................................................................................................. 45

2.4.3 Second level analysis ........................................................................................................................ 46

2.4.4 Selection of criteria and indicators ............................................................................................. 46

2.5 Limitations of the study ............................................................................................................................. 47

2.5.1 Attribution ............................................................................................................................................ 47

2.5.2 Subjectivity of results ....................................................................................................................... 47

2.5.3 Modelling limitations ....................................................................................................................... 47

2.5.4 Classification of institutions .......................................................................................................... 48

3 FP6 NMP Overview ............................................................................................................................................. 49

3.1 Overall participation to FP6 NMP .......................................................................................................... 49

3.1.1 Time to contract ................................................................................................................................. 49

3.1.2 Overview of participating organisations .................................................................................. 50

3.1.3 Geographical coverage ..................................................................................................................... 57

3.2 Project overview ........................................................................................................................................... 63

4 Motivation and Outputs .................................................................................................................................... 65

4.1 The FP6 NMP Project Portfolio ............................................................................................................... 65

4.2 Assessment of NMP Participant Motivation ...................................................................................... 72

4.2.1 Industrial organisation perspective ........................................................................................... 76

4.2.2 SME and Large Enterprise perspective ..................................................................................... 77

4.2.3 Higher education and research institute perspective ......................................................... 79

4.3 Assessment of Achieved Outputs, Outcomes and Results ........................................................... 82

4.3.1 Scientific and Technological Outputs......................................................................................... 82

4.3.2 Commercial Return on Investment ............................................................................................ 83

4.3.3 Assessment of Dissemination & Communication Channels ............................................. 85

4.3.4 Strengths and weaknesses of FP6 NMP .................................................................................... 89

5.1 Costs vs. Achievements .............................................................................................................................. 91

5.2 European Additionality .............................................................................................................................. 93

5.3 Impacts on RTD Capabilities .................................................................................................................... 95

5.3.1 Impacts on the Participating RTD Teams ................................................................................ 95

5.3.2 Impacts on the Participating Organisations ........................................................................... 97

5.4 Impacts in the Social Sphere and the Broader Environment ..................................................... 98

5.5 Impacts on the European Research Area......................................................................................... 102

Page 11: Ex-post Evaluation of FP6 NMP at project level

11

5.6 Transience of Impacts .............................................................................................................................. 104

5.7 Success Factors and Obstacles.............................................................................................................. 107

6 Conclusions and recommendations .......................................................................................................... 111

6.1 Introduction ................................................................................................................................................. 111

6.2 Conclusions on the appropriateness of the project mix ............................................................ 112

6.3 Conclusions on outputs, outcomes and results achieved .......................................................... 113

6.4 Conclusions on the efficacy of implementation and monitoring procedures ................... 114

6.5 Recommendations ..................................................................................................................................... 119

6.5.1 Recommendations at NMP Policy level .................................................................................. 119

6.5.2 Recommendations at the implementation level ................................................................. 123

7 References ............................................................................................................................................................ 127

Part C - Annexes..................................................................................................................................................... 129

Annex I - Criteria and indicators ......................................................................................................................... 131

Annex II - FP6 NMP Calls ........................................................................................................................................ 140

Annex III - Projects analysed as case studies ................................................................................................. 142

Annex IV - Case study template ........................................................................................................................... 150

Annex V - Survey aggregated level analysis .................................................................................................... 153

V.1 - General Info ............................................................................................................................................. 153

V.2 - Your Experience with Project .......................................................................................................... 158

V.3 - Projects Outputs .................................................................................................................................... 161

V.4 - Project Goals and Added Value ....................................................................................................... 166

V.5 - Project Impacts ...................................................................................................................................... 169

V.6 - Commercialization of Results .......................................................................................................... 174

V.7 - Additionality ........................................................................................................................................... 175

V.8 - Success Factors and Obstacles ........................................................................................................ 179

Annex VI - Survey: industry level analysis ...................................................................................................... 184

VI.1 - General Information ............................................................................................................................ 184

VI.2 - Your Experience with Project .......................................................................................................... 193

VI.3 - Projects Outputs .................................................................................................................................... 200

VI.4 - Project Impacts ...................................................................................................................................... 219

VI.5 - Commercialization of Results .......................................................................................................... 232

VI.6 - Additionality ........................................................................................................................................... 234

VI.7 - Success Factors and Obstacles ........................................................................................................ 243

Annex VII - Detailed analysis of project outputs........................................................................................... 254

VII.1 - Organisation type= Higher education (HE) .......................................................................... 254

Page 12: Ex-post Evaluation of FP6 NMP at project level

12

VII.2 - Organisation type= Research (RES) ........................................................................................ 254

VII.3 - Organisation type= Industry (IND) ......................................................................................... 255

VII.4 - Organisation type= Large enterprises (LE) ......................................................................... 255

VII.5 - Organisation type= SMEs (SME) ............................................................................................... 255

Annex VIII - Survey factorial analysis ............................................................................................................... 257

VIII.1 - Results of factorial analysis of impacts on participant organisations ....................... 261

VIII.2 - Results of factorial analysis of impacts on the broader environment ...................... 265

VIII.3 - Results of factorial analysis of impacts on success factors and obstacles ............... 271

Page 13: Ex-post Evaluation of FP6 NMP at project level

13

LIST OF FIGURES

Figure 1: Budget growth of the Framework Programmes. ........................................................................ 29

Figure 2: EU research: evolution of funded thematic areas over Framework Programmes [6]. 30

Figure 3: Structure of FP6-NMP Programme ................................................................................................... 34

Figure 4: Repartition of allocated EC contribution over FP6 NMP pillars (Source: EC database)

............................................................................................................................................................................................. 37

Figure 5: Repartition of funded projects over instruments and pillars (Source: EC database) .. 38

Figure 6: Comparison of country representation in EC database and survey population ............ 42

Figure 7: Comparison of organisations representation in EC database and survey population 43

Figure 8: Comparison of project instruments representation in EC database and survey

population ....................................................................................................................................................................... 43

Figure 9: Time to contract of FP6 NMP projects (average=455 days, min=237 days, max=918

days) .................................................................................................................................................................................. 50

Figure 10: Profile of participating organisations, (Source: EC database, n= 2798 Organisations)

............................................................................................................................................................................................. 50

Figure 11: Gender repartition, (Source: EC Database, n= 5497 Participants).................................... 51

Figure 12: Participants profiles (Source: EC Database, n= 2798 Organisations) .............................. 52

Figure 13: Repartition of the participants regarding their adhesion to the European Union,

(Source: EC database, n =5497 Participants) ................................................................................................... 52

Figure 14: Comparison of coordinators organisation type with participants organisation type

(Source: EC database, n=389 Coordinators, n= 5497 Participants) ....................................................... 53

Figure 15: Participation of SMEs in FP6 NMP projects as partners and coordinators per call,

(Source: Corda database) ......................................................................................................................................... 54

Figure 16: EC contribution to EU27 SMEs per call, (Source: Corda database) ................................... 54

Figure 17: Number of participants per EU27 country, (Source: EC Database: n=5497

participants) .................................................................................................................................................................. 58

Figure 18: Participation rate and population per country, with regression line (R²=0.898),

(Source: EC database) ................................................................................................................................................ 58

Figure 19: correlation between GDP and EC contribution, with regression line (R²=0.936),

(Source: EC database) ................................................................................................................................................ 59

Figure 20: Correlation between EC contribution and EU27 GERD, with regression line

(R²=0.9198), (Source: EC database) .................................................................................................................... 59

Figure 21: EC contribution return on EU27 GDP share and GERD comparison, (Source: OECD)

............................................................................................................................................................................................. 60

Figure 22: Evolution of participants, (Source: Corda database) .............................................................. 61

Page 14: Ex-post Evaluation of FP6 NMP at project level

14

Figure 23: Time line of EC contribution to Member States, (Source: Corda database) ................... 61

Figure 24: FP6 Collaborative links for NMP (Source: European Commission, extracted from

Corda)................................................................................................................................................................................ 62

Figure 25: Start and end date of the 389 FP6 NMP projects, (Source: EC database; n=389

projects) ........................................................................................................................................................................... 64

Figure 26: NMP average Project duration, (Source: EC database; n=389 projects) ......................... 64

Figure 27: Sectorial repartition of FP6 NMP participants, (Source: FP6 NMP Participants

survey, n=1181) ............................................................................................................................................................ 66

Figure 28: Dimensions of research activities conducted in FP6NMP projects as perceived by

survey respondents (Percentage of total respondents, source: FP6 NMP Participants survey,

n=1181) ............................................................................................................................................................................ 70

Figure 29: Continuation of R&D in the area, (Percentage of total respondents, Source: FP6 NMP

Participants survey, n=1181) .................................................................................................................................. 70

Figure 30: Importance of Goals for Participants in RTD projects, (Source: FP6 NMP Participants

survey, n=1.181) ........................................................................................................................................................... 73

Figure 31: FP6 NMP on the innovation pathway ............................................................................................ 74

Figure 32: Importance of Goals for Industrial Participants in RTD projects, (Source: FP6 NMP

Participants survey – selection of Very Important and Important responses, n= 453) ................. 76

Figure 33: Important Outputs for Industrial Participants in RTD projects, (Source: FP6 NMP

Participants – selection of Very Important and Important responses, n=453) ................................. 77

Figure 34: Importance of Outputs for SME Participants in RTD projects, (Source: FP6 NMP

Participants survey- selection of Very Important and Important responses, n=106) .................... 78

Figure 35: Importance of Outputs for large enterprises in RTD projects, (Source: FP6 NMP

Participants survey- selection of Very Important and Important responses, n=145) .................... 79

Figure 36: Importance of goals for HE and RES participants of RTD projects, (Source: FP6 NMP

Participants survey – selection of Very Important and Important responses, n=657)................... 79

Figure 37: Importance of Outputs for higher education institutions in RTD projects, (Source:

FP6 NMP Participants survey – selection of Very Important and Important responses, n= 353)

.............................................................................................................................................................................................. 81

Figure 38: Importance of Outputs for research institutions in RTD projects, (Source: FP6 NMP

Participants survey – selection of Very Important and Important responses, n= 353) ................. 81

Figure 39: Possibility of Commercial Returns (source: FP6 NMP Participants survey, n=1181)

.............................................................................................................................................................................................. 83

Figure 40: Levels of expected Commercial Returns (source: FP6 NMP Participants survey,

n=202) .............................................................................................................................................................................. 83

Figure 41: Visibility of RTD NMP projects in different media channels, (Source: FP6 NMP

Participants survey, n=1181) .................................................................................................................................. 86

Figure 42: Importance of communication of project results to society, (Source: FP6 NMP

Participants survey, n=1181) .................................................................................................................................. 89

Figure 43: The BioMine wiki main page

(http://wiki.biomine.skelleftea.se/wiki/index.php/Main_Page) ............................................................ 89

Figure 44: Benefits vs. Costs of participation, (Percentage of respondents, source: FP6 NMP

Participants survey, n=1181) .................................................................................................................................. 92

Figure 45: Achievement of goals - Repartition of total number of respondents per percentage of

Page 15: Ex-post Evaluation of FP6 NMP at project level

15

responses on the different goals (Source: FP6 NMP Participants survey, n=1181) ........................ 92

Figure 46: Project characteristics of those FP6 NMP participants who claimed that would

continue performing their NMP project in case of absence of EU funding, (Percentage of total

respondents, source: FP6 NMP Participants survey, n=395) .................................................................... 93

Figure 47: Repartition of non-selected applicants per type of organisation, (Source: FP6 NMP

non-successful participants survey, n= 1336) ................................................................................................. 94

Figure 48: Conditions under which projects have been resubmitted, (Source: FP6 NMP non-

successful participants survey, n= 1336) .......................................................................................................... 94

Figure 49: Impacts on R&D team capabilities, Percentage of Respondents, (Source: FP6 NMP

Participants survey, n=1181) ................................................................................................................................. 96

Figure 50: Impacts on participating organisations, (Source: FP6 NMP Participants survey,

n=1181) ........................................................................................................................................................................... 97

Figure 51: Impacts of FP6 NMP projects on broader environment, (Source: FP6 NMP

Participants survey, n = 1181) ............................................................................................................................... 99

Figure 52: FP6 contribution to societal and sustainability challenges, (Source: Ex-post

evaluation of FP6 NMP: Strategic level report [8]) ...................................................................................... 102

Figure 53 : Success factors and obstacles (Source: FP6 NMP Participants survey, n=1181)..... 109

Page 16: Ex-post Evaluation of FP6 NMP at project level

Part A - EXECUTIVE SUMMARY

Page 17: Ex-post Evaluation of FP6 NMP at project level

Executive Summary

19

1 INTRODUCTION Over the last decade the European Union has been paving the way for the transformation of

Europe into the most competitive and dynamic knowledge economy in the world. But these

efforts have been impeded by the current economic crisis that has affected all European

countries. Stronger efforts are needed to ensure that the EU becomes an Innovation Union,

with European industry producing innovative products and services that will boost national

economies and ensure sustainable development across the entire continent. But how can the

EU ensure that it supports appropriate innovative areas, i.e. those with the potential to become

economic engines that allow it to compete successfully at a global level? What is the

appropriate policy mix at an EU level that needs to be designed and implemented? Which

policies would complement national policies, minimise overlaps and maximise outputs and

related benefits?

European Commission programmes have helped consolidate the knowledge base in industrial

technologies for many years. At the start of FP6, however, further actions were needed to both

strengthen and translate this knowledge base into viable and sustainable products and

services. The reformation of European industry from resource-based to knowledge-intensive

was one of the major objectives of the 6th Framework Programme. The creation of a European

Research Area (ERA) for industrial technologies was considered to be the most effective and

efficient way to help all Member States meet the interlinked challenges of competitiveness,

environmental sustainability and employment. In this context, a complete and novel thematic

area that focused upon nanotechnologies, nanosciences, knowledge-based multifunctional

materials and new production processes and devices was pursued within FP6: the NMP

programme. This programme was the main research and innovation-related instrument of the

EU in an area that could have significant scientific and technological outputs and economic

impacts on global scale in the long run. A total of €1,429 billion was allocated for the funding

of projects in the area over a period of 4 years (2002-2006), constituting 8% of the total FP6

budget.

The current report presents the main findings of a wide-ranging, two-year ex-post evaluation

exercise based on surveys of both FP6 NMP participants and unsuccessful applicants to the

programme. In addition, 100 FP6 NMP projects were selected as case studies and closely

reviewed via personal interviews and the examination of other relevant information in order

to get further in-depth information and insight into:

- the factors that affected the extent and range of impacts of NMP projects at individual,

organisational, societal, economic and environmental levels across both Europe and more

globally;

- the programme aspects and policy angles that should be retained (or altered) in future

NMP programmes.

The analysis took into consideration two unique factors that affected the course of the

programme:

1) The first one was the initiation of NMP as a separate, broad-based RTD funding programme.

Although disparate NMP activities had been long supported by the EC via various programmes

Page 18: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

20

(such as Quality of Life and Brite/Euram in FP5), in FP6 it was decided that all NMP-related

activities would be pursued within their own programme.

2) The second important aspect of the FP6 NMP programme was the historic enlargement of

the European Union from 15 to 25 Member States in 2004 – right in the middle of the NMP

programming period.

The results, conclusions and recommendations produced during the study – and most

importantly during the elaboration of the case studies – were discussed thoroughly with an

Expert Advisory Board, composed of prominent NMP experts and evaluation and research

policy specialists, as well as with the European Commission in order to distil lessons learned

from the successes and failures of the programme. In addition, a workshop with prominent

experts was organised in Brussels by the EC, in order to discuss the findings of this study. This

highly consultative and participatory process ensured transparency and facilitated the

integration of a wide variety of views into the current report, which will hopefully help the

Commission shape future NMP programmes. Overall, we have tried to produce a report that

will allow all readers – irrespective of their degree of familiarity with the NMP area – to gain

insight into the accomplishments of this highly innovative research support programme.

2 CHARACTERISTICS OF PARTICIPATING ORGANISATIONS AND

PROJECTS In total, 5.497 participations from 2.798 organisations were registered in 449 NMP projects

during FP6. The programme was characterised by the strong participation of

business/industry organisations complemented by an almost equal share of education

and research organisations (35%, 32% and 26% respectively), similar to the overall 6th

Framework Programme. The strong presence of commercial (45%) and private (61%)

organisations was coherent with the importance attached in current innovation policy theory

to public-private partnerships and the crucial role played by ‘intermediary organisations’.

SMEs, the backbone of the European economy, received significant support from the

Programme – with higher than average SME participation than for FP6 as a whole – although

in absolute figures their participation was slightly lower than the foreseen objective1.

In emerging areas, such as nano-biotechnologies, Integrated Projects and Networks of

Excellence were built at European level and many of them are still active today. The specific

calls for SMEs allowed many small and medium sized enterprises from all over Europe to

collaborate with highly reputable research establishments, to gain new knowledge, to expand

their skills and knowledge bases and to provide real-life perspectives on problems to their

more academic and theoretical counterparts. This helped counteract the ‘European paradox’,

which claims that while the EU is very good in producing knowledge at the universities and

research institutes, it is extremely poor in connecting this knowledge with value creation.

The enlargement of the European Union from 15 to 25 Member States also significantly

impacted the Programme. Prior to enlargement in 2004, a large number of organisations from

candidate countries participated in the programme but with low EC funding in comparison to

1 The FP6 NMP objective was 15% while in total the SMEs that participated mounted to 13% of the overall population

Page 19: Ex-post Evaluation of FP6 NMP at project level

Executive Summary

21

the other participants. After enlargement, the proportion of organisations from new

Member States decreased, but at the same time the share of the EC contribution they

received increased to a level comparable to that of other EU countries.

Overall, the NMP Programme attracted most of the top EU research institutions and

companies (i.e. R&D performing institutions and companies with headquarters in the EU).

Without the participation of these organisations, the NMP programme could not have hoped to

make a significant impact on either the quality of research undertaken or the subsequent

diffusion, adoption and commercial exploitation of research outputs and results. The

involvement of leading non-EU research organisations (i.e. those perhaps performing research

within the EU as well as elsewhere, but with headquarters located outside of the EU) was less

marked, largely due to issues such as lack of conformity between EU and third country grant

agreements, but the NMP Programme still managed to allocate 100 million Euros, or 35%

of the overall FP6 target (285 million in total), to international cooperation between

European and non-EU organisations.

3 MOTIVATION FOR PARTICIPATION AND OUTPUTS, OUTCOMES AND

RESULTS ACHIEVED The vast majority of participants underlined that the FP6 NMP Programme was a strategic

tool to elaborate novel RTD activities that they would not have been able to perform

otherwise. The main players in the NMP Programme participated mostly for strategic reasons

and the strong long-term market orientation of research organisations complemented the

shorter-term business perspective of industry and SMEs in particular, with many of these

associations identified as very productive. In general, the main strength of the programme was

its unique ability to bring together the best European, and in many cases, international

research groups, raising the visibility of individual participants and improving trans-European

networking via the provision – for the most part – of adequate funding levels.

As specified in its programming documents, the underlying objective of the FP6 NMP

Programme was to move towards a knowledge-based and more environmentally friendly

industry through an integrated approach combining materials science, nanotechnology,

production technologies, information technologies, biotechnologies, and so forth so as to effect

an efficient transformation from resource-based to knowledge-based European industries. In

these terms, the goals of the participating organisations were in line with the objectives of

the programme since scientific and technological goals were considered the most

important reason for participating in NMP projects followed by economic and

health/environmental goals. The participants mainly aimed to explore whether their

innovative ideas – from a scientific and technological point of view – could be conceptually

proved; or whether they could make an innovative scientific or technological breakthrough at a

later point, usually after the end of the project, that could be commercially exploited. This

finding is in line with the pre-competitive nature of FP6, of which NMP was a part, and also

with the fact that NMP participants stated that in the future they would attempt to access

funding from national/regional programmes, as such programmes usually allow a focus on the

‘practical aspects’ of innovation and the market exploitation of ideas.

The FP6 NMP programme performed well in that the immediate outputs generated by

Page 20: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

22

projects were in line with the strategic objectives of the programme, i.e. to foster

‘innovation’ and to promote the ‘transformation of industry’. From a scientific point of view,

the programme improved the EU science base. Although few, if any, projects produced

results that had visibly radical impacts on the global course or direction of scientific

theory or practice, the programme as a whole nevertheless produced strong scientific and

technological outputs and enhanced the knowledge bases and skills of participants. The

Programme also improved the operational processes of many organisations and led to product

and process innovations that are likely to lead to economic outputs for a significant proportion

of the total population. In more detail, a quarter of the FP6 NMP population surveyed stated

that they had realised or expect to realise in the near future commercial returns through

the exploitation of their FP6 NMP project results, with around half of these expecting

commercial returns of more than €100,000 on an annual basis.

4 IMPACTS AND ACHIEVEMENTS OF THE FP6 NMP PROGRAMME The FP6 NMP programme produced significant scientific spill-over effects across Europe

that would not have been attainable in the absence of EU funding even with national or

regional funding programmes. The high additional value of the collaborative and multi-

disciplinary approach of EC-funded research as well as the critical importance of research co-

operation at the European level was highly appreciated by all NMP project participants,

particularly SMEs. Knowledge cross-fertilisation and competence integration at the European

level became the cornerstone for significant impacts in the scientific and technological sphere,

to the benefit of participating organisations and the scientific and technological community as

a whole. Co-operation at European level was also critical for the improved alignment of RTD

outputs with the needs and requirements of users in the European Internal Market.

The programme contributed to the research efforts needed to tackle the so-called ‘Grand

Challenges’ ("global warming, tightening supplies of energy, water and food, ageing societies,

public health, pandemics and security", according to the Lund Declaration). In spite of the

absence of many highly visible or concrete impacts of the NMP programme on society and

environment, some participants rated their work as highly relevant to the ‘Grand Challenges’,

with the impacts produced considered significant by around one fifth of the total population.

But as pointed by members of the Expert Advisory Group, discoveries occurring during EU

funded projects may take a while before being able to be provided as innovations to society or

applied to the environment. Overall, opinions collected during the interviews showed that EU

research policy in these spheres has facilitated trans-border initiatives and led to the

harmonisation of strategic objectives, helping to create a real focus within the European

Research Area on the research needed to tackle ‘Global Challenges’.

Individual researchers in FP6 NMP projects gained important personal benefits such as

enhancement of their knowledge and skills, new R&D partnerships, improved access to

complementary expertise and joint international research actions. At an organisational

level, the NMP programme improved the economic sustainability of the participating

organisations while allowing them to consolidate their competitive position, form new

partnerships and networks and enhance their reputation and image. A significant outcome

of the NMP programme was also its contribution to the future financial sustainability of many

SMEs. In addition to gains in prestige and visibility, almost one quarter of SMEs considered

Page 21: Ex-post Evaluation of FP6 NMP at project level

Executive Summary

23

that their NMP projects had increased their turnover significantly, while one fifth felt that

it had improved their market share and productivity.

Overall, the success of the FP6 NMP in terms of building capacity and creating the basis for

further research and innovation in Europe is appreciated by all participants. But there is still a

potential problem in terms of the transformation from research to innovation. Projects with

exploitable results frequently continue their research through other research-oriented projects

rather than taking the risk of going to the market. Often this is perfectly justifiable in terms of

the stage of maturity that projects have reached along development trajectories, with the

outputs of any single research project often needing to be combined with the results of other

projects – past and future – before further progress can be made in terms of realising

innovation potential. But on occasion there was the suspicion that some NMP participants

were reluctant to emerge from the “comfort zone” that public funding provides, preferring to

seek additional funding for further research rather than pushing ahead along the innovative

trail. Additional incentives may thus be needed to reduce levels of risk-averseness in the

EU.

Factors that have positively impacted FP6 NMP projects should be preserved, e.g. the

flexibility that allowed projects to change direction in response to external events and the

high level of funding allocated per project - compared to national schemes that usually do not

support large-scale, international, collaborative research projects. This latter aspect is

considered of primary importance in the current economic environment, where Member States

significantly affected by the economic recession in recent years may not be able to maintain or

expand funding levels for research projects. Finally, the multidisciplinary approach pursued

within the FP6 NMP projects proved an excellent vehicle for getting to know and work with

SMEs.

Nevertheless, policy issues have impeded the course of NMP projects that need to be

addressed in the next programming period. First of all, more transparent consultation with

stakeholder communities and a more explicit ‘programme logic’ can produce a programme

that is more robust as well as adjusted to market and business needs and requirements.

Perhaps such a transformation would allow the programme to exploit its added value in a

more strategic way, setting a European agenda that would attract a rich set of regional and

national actors as well as reputable international institutions. In addition, external factors

such as the lack of European-wide intellectual property right regimes and regulations are

barriers to research exploitation and should be further addressed by subsequent NMP

programmes. Overall, R&D policies around Europe are still fragmented, despite the catalytic

role played by EU initiatives, and further coordination amongst policy areas is necessary.

5 PROGRAMME IMPLEMENTATION AND MONITORING In many cases, the supportive role played by the EC was much appreciated, particularly the

demanding role played by Project Officers. However, a number of issues were recorded where

improvements are needed in future.

One of the commonest topics identified as an issue during personal interviews with project

participants involved the appointment of new Project Officers during projects. Such changes

affected the course of the projects and caused unnecessary delays. Another issue, discussed

Page 22: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

24

quite often during the meetings of the evaluation team with the Expert Advisory Board, was

the time lag between idea formulation and contract signature date (i.e. time to contract).

NMP is an area with significant and continuous changes and evolutions on a global scale. So

within the observed average time to contract of 1.25 years (455 days), the external context of a

project evolves and may have significant implications for project expectations. Project

objectives, for example, need to become much more adventurous when developments

elsewhere shift technological frontiers rapidly forward). Changes need to be made to improve

flexibility, e.g. via the adoption of open, continuous call project selection processes, or via

procedures that allow projects to change direction more easily, to adjust levels of funding, to

make adjustments to expected time horizons etc. Changes are also needed to improve the time

to contract and to enhance the attractiveness of the programme to non-EU based organisations

that may be more accustomed to fast-track selection processes.

Another important issue that requires further attention is the administrative burden imposed

on participants, regarded as ‘heavy’ in FP6 and still ponderous in FP7, though some

progress had been made. This parameter was also identified by a number of SMEs as an

important obstacle to their participation in the FP6 NMP programme. Overly bureaucratic

procedures constitute an obstacle to innovation and prevent potential innovators from

presenting proposals.

6 RECOMMENDATIONS FOR POLICY AND PROGRAMME MANAGEMENT The recommendations of this report have emerged from the review of all collected primary

and secondary information, complemented by discussions held between the evaluation team,

the Expert Advisory Board and the European Commission, and informed by ideas and

proposals generated during the conduct of the case studies. They have been classified into

three distinct and interlinked areas, according to the wider policy or implementation level they

address. In some cases they address issues outside the boundaries of the Programme, but they

are included because of their relevance to the future development of the Programme.

6.1 Recommendations at NMP Policy Level

1. Support for SMEs: The EC should retain instruments specifically involving calls for

SMEs in the forthcoming Horizon 2020 Framework, but it should encourage smaller

consortia (with a maximum of 15 partners) in line with the developments in FP7.

2. End-User Participation: Project consortia should encourage end-user participation in

project planning in order to enhance short to medium-term market impacts. In projects

with longer market potential, mechanisms providing participants with access to

comprehensive market outlook assessments would help them assess potential

exploitation pathways.

3. Standards and Regulations: Coordination of the standardisation activities conducted

within NMP, possibly via Support Actions, could help maintain close link between

projects and standards development even beyond the normal lifetime of projects. This

should be attempted at an international level and in collaboration with ETPs.

4. Visibility through Dissemination: Apart from international peer-reviewed publications,

the focus of dissemination activities should include reports and other material aimed at

the ‘first circle’ of support for SMEs (development agencies, local customers and

investors). The production of such material should not then be neglected.

Page 23: Ex-post Evaluation of FP6 NMP at project level

Executive Summary

25

5. Improvement of Public Dialogue: Projects should aim not only to produce material for

publications, conferences, etc., but also to plan activities that communicate research

results to the public, thus informing local/regional schools, universities, communities

etc. about developments of interest.

6. Market Exploitation Seminars: In order to ensure the timely consideration of market

opportunities, Market Exploitation seminars should be held early in the lifetime of

projects and organised according to the needs of different project consortia. In general,

projects should include marketing plans within their proposals to ensure that some

thought is given to market needs in the medium-term.

6.2 Recommendations at Projects Selection and Implementation

Level

1. Project Selection Process: The EC should tie project evaluation (project monitoring and

review) closer to the proposal selection process. For example, the Proposal Evaluation

Report (ESR) could be made available to project reviewers.

2. Time to Contract: A usual plea of FP participants is the minimisation of time to contract.

Especially in the case of NMP this is of imperative importance. A new mechanism could

be developed which would allow novel ideas twith clear market outputs to be funded

as projects though a more flexible and short evaluation process.

3. Scientific and Technical Monitoring: Having regular inspections of projects (even small

ones) by external experts, perhaps with a periodicity of one year between reviews,

would facilitate rapid reactions to problems arising during the course of a project or to

deviations from project plans.

4. Output Monitoring: Recording project outputs in a centralised system should be an

obligation for beneficiaries, with the contents of this system feeding into the technical

review of projects. Although the recording of outputs has already been implemented

within FP7, the registration of outputs is currently independent of the review process.

5. Impact Monitoring: The impact section of project proposals could identify expected

short-term impacts (e.g. six months after project termination). This would then be

revised at the final evaluation of projects (conducted within the first two months after

project conclusion). A proportion of project budget could be reserved for dissemination

and exploitation to support the realisation of these short-term impacts, with payment

occurring after a simple review of dissemination and exploitation results by the project

officer.

6. Market Analysis: A milestone-based market analysis should be performed at project

start, estimating not only market growth but also looking at competing technologies

and positioning industry partners vis-à-vis their competition. This would lead to the

better alignment of projects with market requirements.

Page 24: Ex-post Evaluation of FP6 NMP at project level

Part B - DETAILED REPORT

Page 25: Ex-post Evaluation of FP6 NMP at project level

Detailed report

29

1 THE NMP ENVIRONMENT IN FP6

1.1 Introduction

Since 1984, the European Union has been supporting research and development activities with

an instrument called the ‘Framework Programme’ (FP). It is a multi-annual plan proposed by

the European Commission and adopted by the Council and the European Parliament following

a co-decision procedure. The FP of interest within this report is FP6 which ran from 2002 to

2006. Framework programme activities are organised in thematic areas covering almost all

scientific disciplines. The major share of available funding is allocated to collaborative research

and technological development projects through open ‘Calls for proposals following

transparent procedures.

Since FP1, the overall proportional funds provided by the Framework Programmes, as

compared to national R&D funding, have become more and more important (Figure 1).

Figure 1: Budget growth of the Framework Programmes.

FP6 stood in a row of previous Framework Programmes and built upon the achievements

reached and the lessons learned. Within the course of FP5, three major political objectives and

strategies were launched and served as a background for the definition of FP6:

The Lisbon economic objective: the Lisbon European Council Summit held in March

2000 set an objective to ‘make the European Union the most competitive and dynamic

knowledge-based economy in the world by 2010 [1].

The European Research Area: the European Research Area (ERA) was proposed by

the Commission in January 2000. It has since been endorsed by the Heads of State and

Government, and is now the major research policy nexus in Europe [2]. It is described

as “an area for the coherent and coordinated pursuit of research activities and policies,

and an area in which researchers and knowledge move freely”

The Barcelona Summit: EU Heads of State and Government, when they met in

Barcelona in March 2002, agreed that ”overall spending on R&D and innovation in the

EU should be increased with the aim of approaching 3% of GDP by 2010, where two-

0

10

20

30

40

50

60

1984 1989 1994 1999 2004 2009 2014

Bu

dg

et

(bil

lio

ns €

)

FP1FP2 FP3

FP4 FP5 FP6

FP7

Page 26: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

30

thirds of this new investment should come from the private sector”.

The FP6 programme was structured around three headings:

- focusing and integrating Community research on seven thematic areas;

- structuring the European Research Area considering research & innovation, human

resources & mobility, research infrastructures and science & society;

- strengthening the foundations of the European Research Area, through coherent

development of research and innovation-stimulation policies and activities in Europe.

The intention of the European Union during the 6th Programme period has then been to create

the appropriate foundations so as to establish a European Research Area that would transform

Europe into the most competitive and dynamic knowledge economy in the world. As a

founding part of the competitiveness objective, the manufacturing capability of Europe has

attracted much attention with the requirements to maintain both its competitiveness and

sustainable development. The manufacturing industry makes use of a wide variety of

technologies which need to be mastered at a pace at least equal to the one of the evolution of

the products being manufactured. This increasing complexity implies first mastering lower

scales, down to the nanometre size but also the development of new materials demonstrating

more and more extreme and even “smart” properties which are able to adapt to a changing

environment.

These considerations led to the creation by the European Commission, under the Industrial

Technologies umbrella, of the NMP (Nanotechnology and nanosciences, knowledge-based

multifunctional Materials and new Production processes and devices) thematic area within the

6th Framework Programme (FP6). As stated in the Expert Advisory Group on NMP report [3]

“the importance of the NMP industrial programme as a major engine for Europe’s growth,

sustaining employment and creating new jobs and providing solutions to its grand challenges has

been confirmed and is likely to grow as the major global trading blocs increasingly focus on the

concepts of the physical economy”. Actions towards such investments are further pursued

during the present Programme period (2007-2013) [4].

Figure 2: EU research: evolution of funded thematic areas over Framework Programmes [6].

Other

Basic research

Coordination and development

Space

Science and Society

Training of researchers

Disseminationand exploitation

International Cooperation

Socio-economic

Transport

Energy

Life sciences

Industrial and materials technology

Environment

IT and Communications

FP

1

FP

2

FP

3

FP

4

FP

5

FP

6

FP

7

100

90

80

70

60

50

40

30

20

10

0

FP

budget

repart

itio

n(%

)

Page 27: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

31

Industrial and material technologies appeared in the course of FP5 (Figure 2) and NMP

appeared for the first time as an independent and concrete axis for EU research in FP6. The

programme intended to promote "a structuring effect on research and technological

development in Europe, including the Member States, associated candidate countries and other

associated countries and make a significant contribution to the establishment of the European

Research Area and to innovation" [5]. In other words, the transformation of European industry

from a resource-based into a knowledge-intensive one that produces high value products and

the relevance for European competitiveness has been one of the major objectives in FP6. This

implies a closer link between research and industry and enhances the role of innovation.

The budget allocated to the NMP thematic area was €1 429 M, representing 1O% of the FP6

budget earmarked to focus and strengthen Community research (Table 1).

1. Focusing and integrating Community research 14 682 Thematic priorities2 12 438

• 1. Life sciences, genomics and biotechnology for health 2514 • 2. Information society technologies 3 984 • 3. Nanotechnologies and nanosciences, knowledge-based

multifunctional materials and new production processes and devices 1 429 • 4. Aeronautics and space 1 182 • 5. Food quality and safety 753 • 6. Sustainable development, global change and ecosystems 2 329 • 7. Citizens and governance in a knowledge-based society 247

Specific activities covering a wider field of research 1 409 • Policy support and anticipating scientific and technological needs 590 • Horizontal research activities involving SMEs 473 • Specific measures in support of international cooperation3 346

Non-nuclear activities of the Joint Research Centre 835

2. Structuring the European Research Area 2 854

3. Strengthening the foundations of the European Research Area 347

TOTAL 17 883 Table 1: FP6 budget repartition (EUR million) (source Cordis)

1.2 FP6 NMP Objectives

1.2.1 FP6 NMP Objectives hierarchy

As stated in the FP6 NMP Work Programme, the underlying objective of the NMP thematic area

is to move from a resource based industry towards a knowledge-based and more

environmentally friendly one through an integrated approach combining materials sciences,

nanotechnology, production technologies, information technologies, biotechnologies, etc.

2 Of which at least 15 % for SMEs. 3 The amount of EUR 346 million aimed at funding specific measures in support of international cooperation involving developing countries, Mediterranean countries (including the Western Balkans), and Russia and the New Independent States (NIS). Another EUR 312 million was earmarked to finance the participation of third-country organisations in the Thematic Priorities and in the

specific activities covering a wider field of research, thus bringing the total amount devoted to international cooperation to EUR 658 million. Additional resources were made available under section 2.2 “Human resources and mobility” to fund research training for third-country researchers in Europe.

Page 28: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

32

This means having the capacity to move towards knowledge-based products and processes:

- breakthrough in new applicable knowledge and long-term RTD [(Research and

Technological Development)]

- wider scope for industrial research (environment, health, energy, employment, education

and training, legal and financial aspects, science and society)

- ensuring multi-disciplinarity, cross-sectoral and life-cycle approaches

- integration of actors, sectors, expertise, disciplines, technologies, activities, funds.

The overall mission of the programme for the period 2002-2006 has been to translate the

advantages of European excellence into a real competitive advantage for industry. To

contribute to this process, the FP6 NMP was organised into three research pillars and a fourth

one dealing with integrating actions4:

Pillar 1: Nanotechnologies and nanosciences

Nanotechnologies and nanosciences represent a new approach to materials science and

engineering, as well as for design of new devices and processes. Europe enjoys a strong position in

the nanosciences that needs to be translated into a real competitive advantage for

European industry. The objective is twofold:

1. to promote the creation of an RTD-intensive European nanotechnology related industry,

2. to promote the uptake of nanotechnologies in existing industrial sectors.

Research may be long-term and high risk, but will be oriented towards industrial

application and/or co-ordination of efforts at EU level. An active policy of encouraging industrial

companies and SMEs, including start-ups, will be pursued through the promotion of strong

industry/research interactions in consortia undertaking projects with substantial critical mass.

Research and development activities should also promote development of new professional skills.

Whenever appropriate, societal, health, ethical and regulatory issues, and in particular metrology

aspects, should be addressed.

Pillar 2: Knowledge-based multifunctional materials

New, high knowledge-content materials, providing new functionalities and improved

performance, will be critical drivers of innovation in technologies, devices and systems, benefiting

sustainable development and competitiveness through multi-sectoral applications. Since these

applications have a strong impact on individuals and on society as a whole, a new research

culture will be required. RTD activities are expected to be high risk, inter and

multidisciplinary, long term and generic, with potential benefits in material, maintenance and

energy savings as well as on health, safety and the environment. Breakthroughs will come not

only from the new materials developed but also from new processing and from the new

approaches taken for example using renewable raw materials. To assure Europe's strong position

in emerging technology markets the various actors need to be mobilised through leading edge

RTD partnerships and high-risk research.

Pillar 3: New Production Processes and Devices

New production concepts need to be designed, based on breakthrough organisational, quality and

technological developments, supporting new products, processes and services. The goal is to

support the transformation of European industry towards more knowledge-based and value-

4 http://cordis.europa.eu/nmp/whatis.htm

Page 29: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

33

added industries and improved competitiveness and sustainability. To this end it is vital to

provide the industrial systems of the future with the necessary tools for efficient life-cycle

design, production, use and recovery, decreasing at the same time internal and external

costs and reducing major accident hazards. Appropriate organisational models and improved

knowledge management should support technological developments and innovation routes.

Flagship research projects need to be carried out, highlighting the importance of collaboration

between research and industry, the major outcome of which would be a framework for

"manufacturing in 2010" based on improved co-ordination and integration of research efforts at

the European level.

Pillar 4: Integration of nanotechnologies, new materials, and new production

technologies for improved security and quality of life

This area was added to the first three areas, as defined in the specific programme, due to the

"integrating" challenge of the expected output and due also to the number of expressions of

interest received on the subject. A specific target should indeed be to put materials science and

advanced industrial technologies at the service of health. In this context, integration of

technological developments, and in particular of the new generation of smart and hybrid

materials interacting with their surrounding and related manufacturing equipment, is bringing

huge potential for the development of sensors, actuators and devices, leading to a greater

security and safety of people and the environment.

Pillar 5: Cross-cutting activities

A reference to this 5th pillar appears in the 2nd edition of the FP6 NMP Work Programme5.

Coordinated actions ran under this pillar were intended to improve synergies between

Member States (current and new) and Associated States’ research activities as well as with

other research actions.

Figure 3 shows the graphic representation of the structure of the NMP programme and its

respective objectives hierarchy.

The general principles and objectives of the NMP Programme, outlined in the introduction,

have been translated by the Lot 2 Study [8] into a number of Strategic Objectives identified as

follows:

o Transformation of industry,

o Strong presence and interaction of innovative enterprises, universities and

research organisations in research actions,

o Integration of education and skills development with research activities,

o Creation of Europe wide networks and projects providing access to new

technologies,

o New approaches implemented in particularly in SME intensive sectors,

o Sustainability assured in RTD activities,

o Enthusiasm for science assured

5 http://cordis.europa.eu/fp6/sp1_wp.htm#nmp

Page 30: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

34

Figure 3: Structure of FP6-NMP Programme

1.2.2 NMP and international cooperation

International cooperation embraces participation of organisations from non-EU countries

(third countries6 as well as with countries with which the EU has bilateral agreements7) in

standard projects and specific activities stimulated by the EC and by the authorities of third

countries in the framework of the bilateral political dialogues. A budget of 285 million Euros

was foreseen for funding third country participation in RTD actions across all thematic

priorities [9].

In NMP, the basic premises of international cooperation are mutual and public benefit for all

stakeholders, a balance between cooperation and competition and a prior consent from

industry to start cooperation by guaranteeing an appropriate management of IPR issues.

6 Third countries : “INCO" countries - developing countries; Mediterranean partner countries; Western Balkan countries, Russia and other newly independent states 7 Countries with which the EU has bilateral agreements for research and technology development: Argentina, Australia, Canada, Chile, China, India, Russia, South Africa and USA

Page 31: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

35

Another instrument for international cooperation in the field of NMP is the Intelligent

Manufacturing Systems (IMS), an industry-led, international business innovation and R&D

programme established to develop the next generation of manufacturing and processing

technologies. It includes companies and research institutions from the European Union,

Mexico, Korea, Switzerland, and the United States. Its renewal in 2007 aimed at providing a

new impetus to stimulate cooperation among the main industrial regions of the world, but

Japan abandoned the scheme in April 2010. Nevertheless, in parallel, the new membership of

Mexico has been welcomed.

1.2.3 NMP and SMEs

SMEs represent more than 99% of European industry and employ about two-thirds of the

European workforce. Because of this recognised role in the European economy, supporting the

innovation potential of SMEs is one of the few quantified objectives of FP6 which states that at

least 15% of the FP6 budget should be allocated to SMEs participating in FP6 research projects.

This target was set at the NMP thematic area level as well and new opportunities were opened

up for SMEs, which could either participate in the specific activities to be carried out by and for

the SMEs within Integrated Projects and Networks of Excellence or that could play a leading

role in Integrated Projects specifically dedicated for SMEs. As stated in the FP6 NMP Work

Programme 2002-2003, “In order to reach this objective, special actions are foreseen such as

SME specific calls for proposals in the context of the new instruments, reinforcement of National

Contact Points, and specific training and take-up measures. In addition, the involvement of SMEs

is taken into account in the evaluation criteria particularly for the new instruments. Also the fact

that enterprise groupings which represent large communities of SMEs may play an active role in

the new instruments will contribute to reaching the above-mentioned objective.”

1.2.4 European Technology Platforms

European Technology Platforms (ETP) first appeared in the 2002 Communication on Industrial

Policy in an Enlarged Europe with the proposed objective to work out a long term strategic plan

for R&D for specific technologies involving major economic or societal challenges by ensuring

synergy among public authorities, users, regulators, industry, consumers and pole of excellence.

ETPs then emerged as industry led fora working on the development and maintenance of

agendas of research priorities for their particular sector. ETPs provide valuable input to the

European Commission to increase effectiveness and efficiency of its research funding

programs. In June 2011, 36 ETPs were in existence and nine of them were related to

production and processes following the classification provided by the European Commission

(see Table 2).

While not being in their strict sense FP6 instruments, the development of European

Technology Platforms was done in symbiosis with the FP6 programme: FP6 projects provided

resources and data to the Platforms, the Platforms fed back into the FP6 with long term

strategic research agenda based on stakeholder consultation.

Page 32: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

36

Table 2: Individual ETPs as of June 2011 (source; http://cordis.europa.eu/technology-platforms/individual_en.html)

1.3 FP6 NMP implementation context

1.3.1 Instruments

The NMP programme utilised a combination of traditional and new instruments for its

elaboration.

With respect to new instruments, Integrated Projects (IPs) were intended to promote

breakthrough-driven research and thus were foreseen to contain a coherent set of activities

including RTD, innovation-related activities, training, knowledge management and last but not

least, dialogue with the public. Networks of Excellence (NoEs) aimed at strengthening

European S&T excellence by means of the progressive and long-lasting integration of a critical

mass of research capacities in selected areas. They should have long-term objectives and

contribute to advancing knowledge for sustainability, competitiveness and dynamism of EU

industry.

The traditional instruments utilised, providing continuity with the past, were: Specific

Targeted Research Projects (STREPs), Coordination actions (CAs) and Specific Support Actions

(SSAs).

STREPs in FP6 evolved from the shared-cost RTD and demonstration projects of FP5,

addressing ambitious and highly creative research and targeted at exploring frontiers of

knowledge or innovation-driven projects for SMEs. CAs were a continuation of the Concerted

Actions/Thematic Networks used in FP5 and intended to strengthen the links between

different research initiatives, such as Eureka, COST, ESF, national and Commission RTD

activities. Finally, SSAs aimed at supporting the implementation of NMP priorities, by funding

activities such as studies, benchmarks, foresight, elaboration of technology roadmaps and

promotion and dissemination of knowledge and good practices.

Page 33: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

37

As derived from the information provided by the EC at the beginning of the study, the

distribution of the 389 NMP FP6 projects considered in the ex-post evaluation, across the

aforementioned axes is shown in Table 3. It should be noted that at the launch of the study, the

FP6 NMP was not completely finished so not all projects are included in the analysis. An extract

made from the Corda database on December, 2nd 2010 shows that a total of 444 projects were

funded under FP6 NMP [11].

NoEs 22 projects 5,66%

IPs 95 projects 24,42%

CAs 16 projects 4,11%

SSAs 36 projects 9,25%

STREPs 220 projects 56,56%

TOTAL 389 projects 100%

Table 3: Repartition of the FP6 NMP projects screened during the ex-post evaluation of FP6 NMP (Source: European

Commission database)

1.3.2 FP6 NMP Calls

The FP6 NMP calls were described in the FP6 NMP Work Programme. As far as visible within

the work programme, no budget pre-allocation was made within the pillars.

Figure 4 and Figure 5 show that the highest part of the EC funds (56%) went to pillar 2, with a

high proportion of STREPs and to the pillar 3 with a high proportion of Integrated Projects.

Figure 4: Repartition of allocated EC contribution over FP6 NMP pillars (Source: EC database)

Nano-technologies

and

nanosciences17%

Knowledge-based

multifunctional

materials29%

New Production Processes and

Devices

27%

Integration23%

Cross-cutting activities

4%

Page 34: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

38

Figure 5: Repartition of funded projects over instruments and pillars (Source: EC database)

1.4 Evaluation Issues

Despite Europe’s research excellence, the translation of research outputs into commercial

products has not yet fully been attained. An example is provided by the Nanotechnologies and

Nanoscience pillar of NMP. EU nanotech patenting lags significantly behind the US: 26% of

nanotech related patents are filed by the EU25 while 40% by the US8. This is due to a number

of reasons, such as the fragmentation of EU market and the lack of critical mass which

consequently reduces the effectiveness of the commercialisation of nanotechnology.

Environmental, health and safety concerns, standardisation and public opinion issues also

hinder market acceptance and the effective deployment of nanotechnology9 while lack of

engineering expertise seems to be holding back adoption in EU Member States [16].

It is expected in the years to come that further developments in nanotechnology, materials

technology, biotechnology and information technology will continue to incorporate

innovations from multiple scientific disciplines in a “convergence” that is expected to have

profound effects on society [17]. In order for Europe to benefit from Nanosciences,

Nanotechnologies, Materials and new Production Technologies (NMP), industrial

developments need to be based on a combination of traditional manufacturing processes and

technologies with innovative processes that will allow the creation, manipulation and

8 OECD Patents database, 2007

9 The European Commission adopted in February 2008 the Code of Conduct for Responsible Nanosciences and

Nanotechnologies research.

Nano-technologies and

nanosciences

Knowledge-based

multifunctional

materials

New Production Processes and

DevicesIntegration

Cross-cutting activities

STP 65 86 22 37 10

SSA 10 6 11 9

NOE 6 11 3 2

IP 6 14 39 32 4

CA 1 3 11 1

0

20

40

60

80

100

120

140Fu

nd

ed

pro

ject

s

Page 35: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

39

integration of new ingredients, platforms, products, etc., so as to achieve an increase of

revenues and at the same time significant reduction of costs.

A thorough understanding of the trajectory of industrial developments so far and the

identification of key drivers and trends as well as ‘wild cards’ that can affect the course of

tomorrow’s industrial (r)evolution should be the thorough input for the development of future

scenarios that will reflect the possible developmental trajectories of industry and

nanotechnologies in particular. Such scenarios are considered important for all involved

stakeholders, at European and national levels, and more specifically for all sectors of the

industry likely to benefit from the pervasive characteristics of the NMP technologies. Such a

forward looking perspective is indispensable if Europe wishes to lead the changes in the

market benefiting from NMP at the global level.

These are some of the key challenges of the FP6-NMP programme being evaluated at project

level within this study.

1.5 Objectives of the study

This study is complementing the one conducted at the strategic level [8] and aims at

evaluating, on the basis of the outputs and outcomes of the funded projects, the effectiveness of

the thematic area "Nano-technologies and nano-sciences, knowledge based multifunctional

materials, and new production processes and devices" within the specific programme for

research, technological development and demonstration "Integrating and strengthening the

European Research Area (2002-2006)".

Building upon the analysis of the information on individual projects projects collected via

online surveys, personal interviews, project reports, etc., the study then draws conclusions on

the effectiveness and efficiency of the NMP activity in achieving its stated objectives, and

makes recommendations on ways to enhance the effectiveness of current and future similar

RTD funding activities. Overall, the study addresses the following questions:

o Did the programme attract and select the right sort of projects to achieve its objectives?

o Have the projects been monitored, reviewed, and steered in such a manner as to

contribute optimally to the objectives of the programme?

o Can measures be taken in order to enhance the performance, success, and impact of the

projects?

Page 36: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

40

2 METHODOLOGY

2.1 Introduction

The Lot 1 study of the project “Ex-post evaluation of the NMP programme (FP6)” focuses on the

micro-project level and thus its main aim is to acquire information from the participating and

non-participating (applicant) organisations about the added value of the programme, its

impacts on the researchers and organisations that took part in the programme as well as its

impacts on the broader EU socio-economic level. Moreover, this study is aiming to identify the

factors that are related to effectiveness and efficiency of the NMP programme. The collected

information has allowed the identification of the thresholds of the programme, its main

competencies and obstacles and thus, this study, has developed and proposed appropriate

policy recommendations. The results of this study, when combined with the results of the Lot 2

study (macro-strategic level) [19], will provide DG RTD with a sound basis for proposing

alterations, adjustments and new mechanisms for the design and elaboration of NMP projects

in the FP7 successor.

The Ex-post Evaluation of FP6 NMP at project level, was a two-year evaluation based on a

defined methodology composed of several steps.

- A preliminary source of information was taken from the European Commission

databases, which were collected, cleaned, and analysed so to build the basis for the

overall evaluation. The provided EC databases contained a total of 389 projects

having been co-financed under the FP6 NMP and taken into consideration in this

evaluation. While these databases were complete regarding administrative and

financial information of the projects, the database related to project outputs and review

results were largely incomplete and incoherent. It thus has not been exploited in this

study. It appears that other projects were later on added in the database. In an extract

made from Corda in April 2011, 444 projects were mentioned as funded under

NMP10. Due to the late addition of these extra projects, and if not otherwise stated,

these additional projects have not been taken into account in the analysis.

- Moreover, a list of relevant evaluation criteria and their associated indicators was built.

These criteria and indicators were estimated during the evaluation procedure so as to

ensure that they were the most appropriate ones.

- An Experts group composed of ten experts was also created specifically for the

evaluation and was consulted at each step of the evaluation. Experts were selected for

their competencies in NMP related technologies, socio-economic impact studies and

innovation policies.

To reach the final objective of the evaluation and to cover the whole process, several steps

were followed:

- After having selected the relevant criteria and indicators (section 2) and after having

analysed the European Commission databases (section 3), a web survey dedicated to

the FP6 NMP participants enabled a large collection of significant data.

10 Including projects funded under joint-programme actions.

Page 37: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

41

- A methodological approach was developed to select 100 of the 389 co-financed projects

in order to create case studies for each of the projects selected (sections 4and 5).

- All of these steps were developed whilst taking into consideration the making of the

correct recommendations for future NMP framework programmes (section 6).

2.2 Evaluation Framework

The impact evaluation used both qualitative and quantitative methods. Data on calls, proposals,

thematic, participants and projects, as well as quantitative data from samples surveyed were

collected, treated and crossed with qualitative data. In addition, factual data related to projects

and partners as well as to the innovation support measures were gathered. Moreover, in order

to gather relevant and reliable data and information, individual evaluation of 100 completed

projects was conducted answering evaluation questions regarding the goals and objectives,

the outputs and achievements, the impacts, and the success factors and obstacles of the

projects (see Annex IV).

The information gathered from the evaluation of the 100 individual projects provided sound

and evidence based answers to strategic questions which were important for the evaluation of

the strategic impact of the NMP programme as a whole.

Using these complementary methods in this evaluation framework, the working team managed

to reduce the uncertainties that arose during the interpretation phase and to build upon the

analysis and the synthesis of the programme allowing to draw conclusions on the effectiveness

of the NMP activity in achieving its stated objectives, and to make recommendations about the

ways in which to enhance the effectiveness of current and future similar RTD funding activities.

In the following sections, the methods used to perform the assessment of the FP6 NMP

programme during this evaluation are further analysed.

2.3 Data Sources

2.3.1 EC Database

The EC Database allows access to detailed information on concluded FP6 calls for proposals,

including statistics on participants, aggregated statistics on proposals by applicants ID, country

and thematic and a number of other criteria.

2.3.2 NMP FP6 Participants survey

As part of the data collection step, a web-survey was carried out to the 5.497 participants of

the 389 FP6 NMP projects.

The survey was divided in 8 distinct sections, dealing with basic details about the

respondents, the experience of the respondent within the project, the project outputs, the

project goals and the added value given by the NMP project, the analyses of project impacts,

the potential commercialisation of results, the addition of the NMP programme and the success

factors and obstacles met during the project.

A campaign with the aim of receiving responses to this survey was organised, invitations were

sent, and a return rate of 33% was obtained.

Page 38: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

42

To ensure a true representation of the collected responses against the original NMP

population, several calculations and comparisons were made: an initial outlook and finalisation

of the responses database, representativeness, comparison to the countries’ repartition,

comparison on the organisations’ types, comparison to the project instrument repartition and

statistical validity (²). These analyses demonstrated a slight under-representation of industry

across the respondents. To compensate for this, weighting factors have been applied for

analysis pursued at an aggregated level.

279 (72%) of the overall 389 NMP projects were represented in the results of the web survey.

For the purpose of this evaluation, only the projects finished before the 31st August 2009

(when the web survey was completed) were taken into consideration. Thus the total response

rate on project level was 94% as 279 of the 294 NMP projects finished before the 31st August

2009 were represented in the results database of this survey. From the 279 projects, 150 NMP

project coordinators participated in the survey, therefore the coordinators’ response rate was

53%.

Below (Figure 6) is the “Top 15 Participant countries” comparison between the EC database

data and the survey results. As indicated, the survey results population reflects the initial

overall NMP programme population.

Figure 6: Comparison of country representation in EC database and survey population

Figure 7 shows the comparison between the EC database and the survey results in terms of

organisation type. It can be seen that the responses collected from the web survey also reflect

the repartition of the overall population.

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

Pe

rce

nta

ge in

th

e p

op

ula

tio

n

EC database

Survey results

Page 39: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

43

Figure 7: Comparison of organisations representation in EC database and survey population

Finally, the comparison of project instrument repartition between the participants of the

survey (279 projects) and all participants to the FP6 NMP programme (389 projects), as

indicated in Figure 8 below, shows once more that the responses collected from the web

survey reflect the repartition of the overall population. The slight discrepancy on STREP and IP

representation is mostly linked to the under representation of industry representatives, as

stated earlier.

Figure 8: Comparison of project instruments representation in EC database and survey population

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Pe

rce

nta

ge o

f an

swe

rs

Organisation Type

Survey Results

EC Database

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

STREP IP NOE CA SSA

Pe

rce

nta

ge

Project Instrument

EC Database

Survey results

Page 40: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

44

2.3.3 Online Survey for the non-selected applicant proposals

In order to examine the whole participation in the FP6 NMP in detail, another web-survey was

conducted and launched, dedicated to all of the organisations who made a proposal under this

programme but who were not selected for co-funding. This aimed to evaluate the reasons of

proposal failure and the impact that not being selected had on the organisations that submitted

a project.

The invitations to participate in the survey were sent to 20.135 persons and 13.246 invitations

reached their addressees. The return rate was 10%, having received 1.386 responses. This

return rate is much lower than the one obtained but this is explained by the facts that:

- there is no contractual link between the organisation and the European Commission for

non selected proposals,

- there are more errors in the database as it has not been checked during the project

negotiation stage,

- and finally, no reminder was launched for this survey, the 10% rate was obtained with

only one invitation to fill out the survey.

The questionnaire was made up of about 40 questions related to basic details of the

respondents, to their application for the project (the evaluation they received), the envisaged

outputs and impacts, and a section on the potential recommendations they may have.

2.3.4 Selection of Case Studies

In order to select 100 of the 389 projects co-financed under FP6 NMP, a methodological

approach was followed based on the three following main points:

A stratified sampling approach was preferred to limit bias,

The stratification process started during the definition of criteria and revised regularly

during the profiling process. Stratum was chosen based on reliable data so no further

bias was brought into the results due to inaccurate information. The stratification

variables were project instrument, coordinator profile and project thematic area.

The sample size within strata was defined making use of proportionate stratification.

The first selection step aimed at selecting the projects taking into account the project

instruments, the coordinator profiles and the project thematic area. The selected population

followed closely the initial repartition of the population on these three dimensions. This step

enabled the selection of 36 projects.

A second selection step was used to pursue the selection process for including three

additional selection methods:

Use of results obtained in the web survey about project results and successes (The

results of 5 questions were used related to project outputs, benefits and impacts)

Use of known facts related to projects, leading to an interesting viewpoint

Use of random allocation

This allowed the further selection of 43 projects. Then the 21 remaining case studies were

selected by the evaluation teams following recommendations made by other project

Page 41: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

45

participants and European Commission officers as well as project news available on the web

which showed interesting success stories.

The selection of the projects was also refined with regards to unpredictable facts linked to the

projects. Thanks to the results of the selection, 100 projects were selected (see Annex III) and

100 case studies were realised.

The process was built upon three to five interviews per project, following an interview

guideline deeply detailed and based on the criteria and indicators previously defined. The

analysis of project documents received both from the European Commission and the project

coordinators was a precious source of information.

2.3.5 Additional Sources

Other sources used for this evaluation were Eurostat and OECD databases, providing reliable

macroeconomic data on countries. Furthermore, communications and publications from the

European Commission as well as expert group reports and analysis of the nanotechnology field

have been used for this study.

2.4 Statistical Analysis

2.4.1 Preparation of databases

As part of the study, the European Commission made their databases available which contain

the details of projects and participants of FP6 NMP projects. The databases included project

details, such as organisation type, organisation contacts, overall budget and EC contribution

information, starting and finishing dates, etc.

All the databases received by the EC have been cleaned and data has been aggregated into one

unique database.

Once the “cleaning” was done, the new database on which the whole analysis has been built

represented a total of 389 FP6 NMP projects.

Project profiles (based on project instrument characteristics and time-scale dimension) and

partner profiles (Geographical dimension, single participants, coordinators) were extracted

and detailed.

On the basis of 5.497 participations in the 389 NMP projects, the total cost of the project was

€2.348.775.880,47. The total EC contribution was of €1.442.682.656,38, representing 61,42%

of the total cost of the 389 projects.

2.4.2 First level analysis

Firstly, there was an initial statistical analysis (averages, percentage, etc.) that allowed the

working team to extract all possible information from the data collected and present the

findings of the study. Specifically, information was obtained and charts were presented to

demonstrate the data gathered on an aggregated level as well as on organisation size (in

particular SMEs and large enterprises) level, sectorial level, NMP programme pillar level and

type of organisation (higher education, research institute, industry or other) level.

Additionally, cross-tabulations between certain questions in order for further information to

Page 42: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

46

be extracted on specific aspects of the surveys conducted were also performed (e.g., how many

SMEs that had high R&D capacity produced economically related outputs).

2.4.3 Second level analysis

The hypothesis underlying the methodology framework was that relationships exist between

“input variables” characterising the structure, organisation, scale and content of projects and

“output variables” (or “performance variables”) characterising project outputs, outcomes and

impacts. The second level of statistical analysis focused on the analysis of the frequency

distributions in relation to the ‘performance variables’ as well as on the cross-correlations

between these variables and the various ‘input variables’. Data analysis techniques, including

multivariate methods capable of identifying groups of results and types of

stakeholders/projects (cluster analysis) and searching for correlations between them, as well

as variable reduction techniques such as the Principal Component Factor analysis were utilised

so as to identify meaningful clusters and groups of projects or relationships among the

different variables. In particular:

- Principal Component Factor analyses were utilised so as to group similar types of

variables together under e.g., specific categories describing the nature of projects, the

obstacles and success factors, the impacts and the importance of goals/ achievements;

- Correlation analyses were executed in order to examine the associations of these sets

of variables among them.

2.4.4 Selection of criteria and indicators

The selection and listing of evaluation indicators was a key step in our evaluation procedure, as

one of their main attributes is to ensure that during the data gathering stage all the information

required to answer the evaluation questions will be collected providing the necessary

information for the statistical analysis. For this reason and prior to selecting any indicator, our

study team, based on the relevant policy documents built the objectives hierarchy of the

programme (see section 1.2.1). The objectives hierarchy identified and defined the categories

of objectives targeted by the NMP thematic area (e.g. scientific, economic, social, etc.). This step

allowed our team to comprehend the quintessence of the NMP programme, its purpose and

objectives and also to statistically extrapolate the interlinkages between the different parts and

components of the programme by developing the appropriate indicators. (The final list of

criteria and indicators of the study is available in Annex I).

Overall, the indicators identified were measured having as reference the following four axes:

relevance of the Objectives set by the project with the Programme objectives

relevance of the Results achieved with the overall objectives of the NMP Programme

Impacts on the researchers that elaborated the projects, the organisations that

participated in the projects and the third outer circle referring to project partners and

the broader socio-economic environment.

Finally, it should be mentioned that in order to allow for a comparison with other

programmes, some of the criteria and indicators used were similar to the criteria and

indicators used in other evaluations.

Page 43: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

47

2.5 Limitations of the study

During the elaboration of the study a number of issues had been encountered that affected the

collection and interpretation of information as well as the elaboration of relevant conclusions

and recommendations. This section summarises the key issues that need to be taken into

account in understanding the results presented in the report.

2.5.1 Attribution

Analyses have been performed to identify statistical relationships. However, a statistical

relationship does not always imply causality. From what it has been possible to gather, the

evaluators cannot always claim that the reported changes, e.g., numbers of jobs created, can be

entirely attributed to the NMP project itself rather than to external factors, e.g., other projects

carried out internally, past experience of partners, etc. Moreover, ambiguity regarding

outcomes arose for some projects as it was not always possible to identify the drivers of, for

example, an increased number of jobs, or to state whether they were the result of the project or

broader contextual changes. Such information has been obtained through the conduct of the

Case Studies, however generalisations or aggregations were not always feasible to perform.

2.5.2 Subjectivity of results

The main input for this report has been the perceptions of participants or applicants of the

NMP programme on its added value, its success factors and obstacles and the impacts that have

been achieved or not due to its funding. Whether collected online or personal interviews, in

essence the same target group has been used: the persons (researchers, managers, directors,

etc.) that have been involved directly or indirectly in the projects. Although the collection of so

many judgemental perceptions can be considered - which deliver a potential hazard for the

study - it should also be noted that the evaluators approached a high number of participants

and applicants during the conducting of the study. The collection of a large number of samples

allowed them to consider the aggregation of judgemental values as a lead to an objective

conclusion.

2.5.3 Modelling limitations

Causal modelling has been limited by the absence of appropriate baseline measures (i.e.,

similar projects without FP5 funding or FP5 NMP funded projects). It is possible that any

changes over the course of the project may reflect other external variables, such as specific

market changes, rather than be attributable specifically to the project success or its funding. It

is assumed here that measures of success represent success due to the project rather than

other drivers such as market forces.

Furthermore, both the survey and the case studies were focused on RTD related activities

which imply that answers from coordination and support actions (CSA) are of less relevance:

Most of the respondents from CSA recognised the inappropriateness of the survey and

commented on that point in the free text area available at the end of the survey

CSAs were removed from first round of project selection of case studies as the focus of

case studies was on RTD activities. Only one coordination action (µSAPIENT) and 2

Support Actions (NANOFOREST and SMART) were selected as case studies to evaluate

Page 44: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

48

the extent to which these instruments are understood by the participants and their

contribution to the creation of a global and coherent framework within FP6 NMP.

2.5.4 Classification of institutions

In the report, many references are made to industry. These organisations are classified as IND

within the EC database. This classification does not give any indication on the size of the

companies (it can be either a large multinational, or a 10 person organisation, etc.), the field of

activity (it can be a manufacturer, a consultancy company, etc.) or the sector activity. Some

further indication is given in other fields of the database: is the company an SME (Yes/No) and

NACE code but these fields are not always filled in which can induce inaccuracy in the

calculations. Whenever feasible, data has been completed with further information from the

survey so as to increase the accuracy of the results.

Page 45: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

49

3 FP6 NMP OVERVIEW

3.1 Overall participation to FP6 NMP

The FP6 NMP programme included projects that started between 2002 and 2006. Table 4

below gives the overview of the total number of participations (partners), of projects and of

organisations within the NMP FP6. As it can be seen, 5.497 participations from 2.798

Organisations11 were registered in selected projects under FP6 NMP. A total number of 449

NMP projects were retained for negotiation with a success rate of 16%.

Total number of

proposals

Total number of

projects retained

for negotiation

Total number of

partners

Total number of

organisations

2343 44912 5497

2798

Success rate of project awarded

16 % (21 %)13

Table 4: Overview of FP6 NMP participation, (Source: EC database, basis: 389 projects)

Also, these 389 NMP projects had a total cost of around €2,35 billion. The total EC contribution

was approximately €1,44 billion, representing 61% of the 389 projects total cost.

3.1.1 Time to contract

The ‘Time to Contract’ (delay between the project submission and the contract signature date)

is an important indicator as it provides information on the ability of the administration to

provide timely support to research and innovation. The time to contract of FP6 NMP projects

has been calculated by comparing the call deadlines, indicated within the FP6-NMP annual

work programmes14 with the effective dates of contract signatures (Figure 9) as available in

the EC database for the 389 projects. The calculated average time to contract is 455 days with a

minimum of 237 days and a maximum of 918 days. This average is higher than the one

reported in the final evaluation of FP6 (384 days) [20]. A time to contract of more than one

year questions the ability of the programme to provide projects at the technological state of the

art and to answer the market needs. Moreover, adjustment of project objectives in the

negotiation phase, to align with the evolution of external contexts, is rarely observed. Reducing

the time to contract should be one of the primary objectives of administrative burden

reduction.

11 Some organisations participated in several projects, as discussed in Table 5. 12 Including the one retained under the FP6 IST thematic, for joint call programmes. 13 Lowest percentage including proposals submitted in the 1st stage. Highest percentage calculated taking into account only the 2nd stage 14 http://cordis.europa.eu/fp6/dc/index.cfm?fuseaction=UserSite.NMPCallsPage

Page 46: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

50

Figure 9: Time to contract of FP6 NMP projects (average=455 days, min=237 days, max=918 days)

3.1.2 Overview of participating organisations

As indicated in Figure 10 below, out of the 2.798 participating organisations identified, 18%

were higher education institutes, 55% were industrial organisations, 17% research institutes

and 10% other types of organisations.

Figure 10: Profile of participating organisations, (Source: EC database, n= 2798 Organisations)

Also, as it can be seen in Table 5 below, the majority (75%) of the 2.798 organisations

participated only once in the NMP projects within the FP6, while one organisation participated

in 79 NMP projects! According to the statistics performed, a significant number of

organisations participated in two NMP projects on average, throughout FP6.

0

5

10

15

20

25

30

35

0 200 400 600 800 1000 1200

Nu

mb

er

of

pro

ject

s

Time to contract (Day)

Page 47: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

51

Number of participations 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Organisations 2082 342 121 57 36 31 30 13 16 11 9 8 3 7 3 1

Number of participations

17 18 20 21 23 25 27 31 32 35 45 66 67 79

Organisations 8 2 2 1 2 3 1 1 2 1 2 1 1 1

Table 5: Number of participations to FP6 NMP projects, (Source: EC database, n= 2798 Organisations)

With regards to the gender of the participants, 81% of the participants were men, whilst 17%

of them were women. No data was in the EC database for 2% of the participants. This ratio is

equivalent to the one observed at the whole FP6 level but lower than the overall percentage of

female researchers recorded in Europe in 2003 (29%) [20].

Figure 11: Gender repartition, (Source: EC Database, n= 5497 Participants)

Moreover, it is worth noticing in Figure 12 that although SMEs represent 65% of the EU’s gross

domestic product, the “backbone” of the economy is weakly represented in the FP6 NMP

projects. Nevertheless, the strong presence of commercial (45%) and private (61%)

organisations was coherent with the importance attached in current innovation policy theory

to public-private partnerships and the crucial role played by ‘intermediary organisations’. In

further detail, among the 5.497 participants, 13% were SMEs, 38% national / governmental

and 15% public organisations.

Male81%

Female16%

No answer

3%

Page 48: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

52

Figure 12: Participants profiles (Source: EC Database, n= 2798 Organisations)

Figure 13 below shows the repartition of the 5.497 participants with regards to their country

of location: 90% of the participants are located in the European Union region, 6% were from

countries associated with the European Union, 2% were from EU candidate countries, and 2%

were non European participants.

Figure 13: Repartition of the participants regarding their adhesion to the European Union, (Source: EC database, n =5497 Participants)

3.1.2.1 Overview of Coordinator profiles

As it is noticed in Figure 10, the repartition of the coordinators is analogous to the total

number of participants per organisation type.

0%

10%

20%

30%

40%

50%

60%

70%

EU (90,07%)

Associated countries to EU

(5,62%)

Associated candidate

countries (2,04%)

Countries with

coop agreement (0,25%)

Under agreement

with the EU

(0,02%)

Non European

Union (2%)

Page 49: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

53

Figure 14: Comparison of coordinators organisation type with participants organisation type (Source: EC database, n=389 Coordinators, n= 5497 Participants)

However, there is a significant difference between the percentage of coordinators (7,5% ) who

came from SMEs compared to the 13% of the total number of partners that came from this type

of organisation. Also, there is a significant difference between the percentage of coordinators

(32%) and the percentage of total number of partners (45%) who came from commercial

organisations. Moreover, it is noticeable that 46% of the coordinators were from governmental

organisations, a higher percentage compared to the 38% of the total number of participants

coming from this type of organisation and that 23% of the coordinators were from public

organisations, when only 15% of the total number of participants were from this type of

organisation as well.

Finally it is worth mentioning that the majority of the coordinators came from private

organisations as well as the vast number of participants.

3.1.2.2 Overview of participating SMEs

The FP6 NMP Work Programme quantifies very few objectives but it is at least clear on its

target for the funding of SMEs: Particularly, it is stated that “at least 15% of the funding

allocated to the Priority Thematic Areas of research is foreseen for SMEs”.

On the basis of the 389 projects, the FP6 NMP Work Programme did not reach this target as the

only 9% of total EC contribution was distributed to SMEs.

With regards to the participants from SMEs in FP6 NMP, 13% of the total number of

participants were from SMEs (see Section 3.1.2 above).

Nevertheless as it is indicated in Table 6 below , the SMEs participate in 11% of the total

project expenditures of FP6 NMP and receive 9% of the EC funding, which is above the 6.1%

estimated for FP6 [21].

13%

45%38%

61%

38%

15%8%

32%

46%

59%

40%

23%

All participants Coordinators

Page 50: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

54

Participant Total Cost

Participant EC Contribution

Total 2 348 775 880 € 1 442 682 656 €

SMEs 262 082 296 € 135 072 415 €

Ratios 11% 9%

Table 6: SMEs participation ratios, (Source: EC Database, n= 389 projects)

Compared to other thematic areas, the NMP one launched specific calls targeting SMEs (see list

of the FP6 NMP calls in Annex II). These calls indicated as an eligibility criterion that the

project should be clearly led by SMEs. The charts below (Figure 15) show that on three SME-

oriented calls, two have supported the SMEs participation as well as the project coordination

by SMEs.

Figure 15: Participation of SMEs in FP6 NMP projects as partners and coordinators per call, (Source: Corda database)

Similarly, as illustrated in Figure 12, the EC contribution to SMEs on these specific calls has

significantly increased as it followed the SME participation rate.

Figure 16: EC contribution to EU27 SMEs per call, (Source: Corda database)

Among the 100 case studies analysed, 10 were selected from these calls (IPs for SMEs) and are

presented below:

14%

8%

35%

9%

13%

29%

2%

9% 7%10%

16%

N/A

31%

10%

33%

17%

27%

0%

5%

10%

15%

20%

25%

30%

35%

40%

FP6-NMP calls (in chronological order from left to right)

% of participating SMEs

% of SME coordinatorFP6-2002-NMP-2

FP6-2003-NMP-SME3 FP6-2004-NMP-SME4

11%

4%

25%

13%11%

23%

2%

5%4%

6%

11%

0%

25%

7%

0%

5%

10%

15%

20%

25%

30%

FP6-NMP calls (in chronological order from left to right)

EC contribution to SMEs

FP6-2002-NMP-2FP6-2003-NMP-SME3

FP6-2004-NMP-SME4

Page 51: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

55

Case 1: (26 partners) Objectives were

changed during the project, and some

skipped. The objectives were only

partially realised. A request for project

lifetime extension had been requested and refused.

Reasons of failure were the too large number of

partners, too ambitious objectives, lack of

commitment of some partners and a lack of coherence

within the project plan. Nevertheless, the partners

individually still think that participating in this

project has been a positive experience as it increased

their technological knowledge and allowed for

building a network that is still active today.

Case 2: The project consortium is made of

28 participants across Europe, including

7 universities, 3 research centres and 18

SMEs. The consortium was said to be too

large for efficient management but at the

same time, its multidisciplinarity was described as an

asset. The project had no commercial outcome but

managed to produce two patents within the consortium

and two license agreements which have been signed with

a company external to the consortium. The SMEs

highlighted the scientific and prestige benefits gained

within the project.

Case 3: (16 partners) The scientific

target and project objectives were not

achieved as the whole project was

based on the optimistic technological

assumptions which were not realised. The long term

impacts of the project will be insignificant as it stands

today. Still many of the partner organisations seem

rather satisfied with the project and have gained a lot

of knowledge which they have successfully integrated

into their own processes and indirectly developed new

products and markets. Regarding difficulties, some

SMEs were following their own agenda as they were

restricted to their daily business. This created some

difficulties for the collaboration between SMEs and

research organisations.

Integrating scientific breakthrough and prototyping

in the same project fails if scientific breakthroughs do

not happen. This should have been tackled in the risk

management part of the project. Nevertheless, a

prototype has received praise in Austria even though

its characteristics were not the one aimed at. It later

got nominated for a “Stadt Preiss”.

Case 4: (22 partners linked with a

consortium of 6 Chinese organisations) The

first objective of technology development

and prototyping was achieved but the

second objective dealing with the constitution of a

Network of SME’s that could provide and customize the

tools developed for a wide range of manufacturing

sectors was not achieved. Participation of end-user SMEs

has not been up to the expectations and hindered in a

great extent the potential benefits the partners could

have from the commercialisation of project outputs. SMEs

had the chance to be transformed from less RTD intensive

into more capable and RTD aware ones. From an

economic perspective, some of the partners did mention

economic benefits from the exploitation of results that

ranged from 10% to 30% of their companies’ turnover.

The major mentioned obstacle to project success has been

the limited productivity and devotion of some members of

the consortium and the utilisation of less experienced

researchers in the project. Also, from the interviews, the

collaboration with China did not happen.

Case 5: (25 partners) The project was

able to achieve its objectives and

technical goals. The benefits of the

project can be felt both commercially

and technically by the partners. Long term impacts

are now foreseen from the project: improved and

predictable product life, reduced material content

without downside, innovative materials and processes

and original & cost-effective design solutions. The

project benefited from successful dissemination

activities carried out with a centralised approach

through which three partners coordinated the

dissemination and training activities. 78 documents

were prepared and two different training courses

(level 1 and 2). Contribution to standardisation

activities was made (Organisation for

Standardisation (ISO)). Many of the partners are still

collaborating closely today. The European

Commission has noted that the consortium was very

effective and has encouraged it to present new

proposals.

Case 6: (53 partners) Several products have

been commercialised and are today on the

market. New characterisation methods,

methodologies and tools were developed.

Overall, the project made a significant change in the

textile industry; the economic growth is more stable, the

general know how has been increased, and the services

offered have been improved. Partners are still

collaborating today. 13 innovative products were

commercialised by the partners and eight technologies

transferred from research to industrial partners for

medium term exploitation. The general commercial

outcome is difficult to estimate. As an example, for one of

the organisations, the estimate turnover linked to the

project is about €50 000, with a potential in 2010

estimated of 500 000 growing € 2 million per year in the

coming years. The effective project management system

used in the project and multi-disciplinary and multi-

sector consortium was mentioned as key success factors.

Page 52: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

56

Case 7: (33 partners) The consortium

lacked efficiency due to the creation of

clans and communications difficulties

among such a number of partners. The

project built five pilot lines and allowed the developed

technology to be adopted by end user SMEs in the

consortium. Despite providing higher quality results,

the real market adoption will face a number of

difficulties related to replacement of running

technologies, acceptance of higher cost (process

compatibility with mass production being questioned)

and the dangerousness of the process.

Case 8: (43 partners) The project had the

ambitious objective of reengineering a

whole production chain. Strengths have

been the large creation of networks with

specialists and identification of many

potential spin-offs. Weaknesses have been the number of

partners involved and the disparity in commitment. One

of the SMEs created a new R&D department. The amount

of patents filed by SMEs and universities has been limited

by the cost of patenting. Only a few partners succeeded in

turning results into commercialisation, Three partners

estimated their commercial return per year to be

respectively: € 120 000, € 50 000 and € 100 000. To take

full advantage of the project, the standardisation and

regulatory framework is now being updated to include

more stringent specifications compatible with the

developed technologies

Case 9: (24 partners) The project

developed advanced biocomposite

materials which have been applied in

20 prototypes targeting different

sectors, in-line with the project

objective. The commercialisation of these products is

difficult, as the biocomposite materials are expensive,

while they need further improvement in order to

overcome the technical limitations identified during

the project. Anyway, it was mentioned by the

interviewees that many of the participating SMEs,

managed to get new clients on the field of bio-based

materials thanks to the knowledge and reputation

earned in the project. Size of the consortium has been

an obstacle which has been overcome by a highly

skilled coordinator.

Case 10: (32 partners) Project objectives

have been achieved and involve not only

scientific, prestige and networking

aspects but most importantly economic

aspects as well as more than 20 exploitable services (nine

patents had been filed and gained based on the results of

the project, nine copyrights/ trademarks/ registered

designs have been gained also, two spin-off companies

were created and software packages have been

developed) that have been produced by the organisations

that participated. Some of them are currently ready to be

launched as products in the medical and actuators fields

but for proper commercial exploitation of the products

elaborated, the regulatory framework obstacles need to

be updated.

Table 7: Overview of 10 case studies related to projects from calls IP for SMEs

The main issue appearing in all the case studies from the calls IP

for SMEs is the size of the consortia, ranging from 16 to 53

partners! In all cases, the issues associated with the

management of large consortia have been mentioned as an

obstacle to project success. This has had a critical effect when

objectives of the project appeared to be unclear or too

ambitious. In that case, commitment from organisations, mainly

SMEs, to the projects rapidly decreased.

Regarding the disadvantages, some delegates of SMEs spoke very little or no English. This was

a major limitation to their participation. Moreover, the administrative barrier was hard to

climb for SMEs which had a lot of production to manage in parallel. Additionally, some SMEs

did not have any R&D service identified. Furthermore, in order for an SME to become part of a

consortium participating in a European project, it should have at least two years of experience

and positive balance sheets; this may have posed as a barrier to participation, even if the

elements required were not so important. Moreover, SMEs had to spend resources, (which

were difficult to find), on managing their Intellectual Property Rights, so as to be sure that they

would stay competitive after the project.

Call for SMEs is a key tool to

get SMEs benefiting from the

programme but they should

be limited to consortia of

manageable size

Page 53: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

57

On the other hand, these negative aspects did not darken the picture too much, which is

overall positive, as some of the participating SMEs managed to obtain appreciable benefits,

even if they did not participate in satisfying projects. Specifically, all the aforementioned

disadvantages were counterbalanced by several positive points. SMEs were incontestably more

reactive than big companies. They rapidly carried out the research results in order to exploit

them while they were also trying to get more finance. Moreover, by participating in NMP

projects, SMEs managed to get access to research and market networks, accessible previously

only by big companies, improving their competitive position, as well as their position towards

clients, suppliers, etc. They became actors on the international market, and their participation

in the projects was a starting point towards potential exportation of their products on an

international level. SMEs also gained knowledge on becoming innovative and on generating

ideas for new products.

3.1.2.3 International cooperation

Another NMP FP6 objective was to transform Europe into a more attractive working place for

researchers from outside Europe (Lisbon Agenda), compared with FP5.

As stated in the objectives of FP6 NMP, a budget of 285 million Euros was foreseen for funding

third country participation in RTD actions across all thematic priorities.

Participants Total Cost

Participants EC Contribution

Average co-financing rate

European Union 2 169 232 331 € 1 341 927 624 € 62%

Associated candidate countries 18 798 277 € 12 277 133 €

65%

Associated countries to EU 140 353 221 € 75 337 029 €

54%

Countries with cooperation agreement 1 653 211 € 82 500 €

5%

Other 18 738 838 € 13 058 369 € 70%

TOTAL NON EU 179 543 549 € 100 755 031 € 56%

Table 8: Repartition of FP6 NMP budget over non European Countries, (Source: EC Database: 389 projects)

As shown in the table, the funding of third countries within FP6 NMP has been to the order of

100 million Euros, contributing to 35% of the FP6 overall objective.

During the interviews, the feedback received on international cooperation presented a mixed

opinion. On one hand, there are examples of projects having successful results thanks to the

added valued gained by the participation of non European organisations, while in other

projects, difficulties to cooperate were recorded, mostly due to language and cultural

differences.

3.1.3 Geographical coverage

The graph below (Figure 17) shows the number of participants coming from the EU27

countries. As it can be seen in Figure 18 where it illustrates that the number of the participants

is strongly related to the population of each country

Page 54: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

58

Figure 17: Number of participants per EU27 country, (Source: EC Database: n=5497 participants)

Figure 18: Participation rate and population per country, with regression line (R²=0.898), (Source:

EC database)

In general, the EC contribution to countries is correlated to the countries’ participation in the

EU27 GDP, as well as with their expenditure in R&D. Thus the Member States with the highest

GDP in the EU27, including Spain, France, United Kingdom, Italy and Germany have received

the biggest part of the EC contribution

However, as it is illustrated in Figure 20 not all countries have exploited their full potential and

they have not received satisfactory benefits in return for their R&D efforts (measured in GERD)

or in relation to their importance in the EU27 economy. The ratio of EC contribution to share

of EU27 GDP shows clearly that some countries received less EC contribution in the NMP

programme compared to what they contribute to the EU27 GDP and their national effort in

R&D. For example, the Nordic countries (Finland, Sweden and Denmark) that have the highest

GERD in the EU, did not get an analogous part from the EC contribution (Figure 21). Also,

France, Italy and the UK did not fully exploit their R&D potential.

962

584 578 570

401

263 250191 165 150 146 132 118 105 73 65 54 54 50 35 28 16 12 10 7 5 3

0

200

400

600

800

1000

1200

DE IT UK FR ES NL BE SE PL FI AT EL PT DK CZ IE RO SI HU SK BG LV EE LT LU CY MT

DE

ES

FRITUK

R² = 0,899

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

0% 5% 10% 15% 20% 25%

% of EU27 population

participation rate

Page 55: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

59

Figure 19: correlation between GDP and EC contribution, with regression line (R²=0.936), (Source:

EC database)

Figure 20: Correlation between EC contribution and EU27 GERD, with regression line (R²=0.9198),

(Source: EC database)

DE

ES

IT

FR

UK

R² = 0,936

0%

5%

10%

15%

20%

25%

30%

0% 5% 10% 15% 20% 25%

Share of EU27 GDP

% of total EC contribution to EU27 members states

AT BE

DE

DK

ES

FI

FRIT

NLSE

UK

R² = 0,9198

0%

5%

10%

15%

20%

25%

0,0% 5,0% 10,0% 15,0% 20,0% 25,0% 30,0% 35,0%

% of total National GERD from EU27

% of total EC contribution to EU27Members states

Page 56: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

60

Figure 21: EC contribution return on EU27 GDP share and GERD comparison, (Source: OECD)

With regards to the new Member States that have entered the EU community during the FP6

(ten countries joined the EU in 2004 and 2 countries in 2007), it can be observed in the two

graphs below (Figure 22 and Figure 23) that surprisingly, the amount of participation from

organisations from these countries decreased over the FP6 but at the same time, their

respective share of EC contribution increased. A possible explanation from this is the emphasis

put in place in the first FP6 NMP call to integrate organisations from the candidate countries:

“Proposers based in Associated States may take part in this programme on the same footing and

with the same rights and obligations as those based in Member States. In addition, this work

programme underlines the importance of involving associated candidate countries in the

Community's research policy and in the European Research Area. Specific support actions will

also be implemented to stimulate, encourage and facilitate the participation of organisations

from the candidate countries in the activities of the priority thematic areas. These will comprise

information, awareness and training activities, promotion of candidate country competencies,

support to researchers from these countries to participate in conferences and to prepare

proposals, establishment and reinforcement of networks or centres of excellence between Member

States and candidate countries, and between centres of excellence of candidate countries and

within candidate countries, measures in support of SMEs in candidate countries to better

participate, evaluation of RTD systems and policies in a particular field, the screening of research

establishments active in a particular field, and prospective studies aimed at defining research

policies and organisation of research systems in a particular field.”

2.2 1.880.50

0.35

1.25

2.522.58

0.72

0.51.05

3.44

2.17

0.93

1.17

1.11

0.67 1.65

0.38

0.93

1.760.54

0.71 0.393.85

1.27

0.57

1.75

0%

50%

100%

150%

200%

250%

AT BE BG CY CZ DE DK EE EL ES FI FR HU IE IT LT LU LV MT NL PL PT RO SE SI SK UK

Ratio %of EC contribution to share of EU27 GDP

GERD in % of National GDP

Page 57: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

61

Figure 22: Evolution of participants, (Source: Corda database)

Figure 23: Time line of EC contribution to Member States, (Source: Corda database)

78%93% 94% 96% 94%

22%7% 6% 4% 6%

2003 2004 2005 2006 2007

Participants from new entrant countries(since 2004)

EU15 participants

84%

63%73%

59%

72%

37%27%

41%

28%

2003 2004 2005 2006 2007

New entrants

EU15

% of EC contribution to EU27 participants % of EC contribution to EU27 participants % of EC contribution to EU27 participants % of EC contribution to EU27 participants % of EC contribution to EU27 participants % of EC contribution to EU27 participants % of EC contribution to EU27 participants % of EC contribution to EU27 participants

16%

Page 58: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

62

Figure 24: FP6 Collaborative links for NMP (Source: European Commission, extracted from Corda)

AT BE DE DK EL ES FI FR IE IT LU NL PT SE UK BG CY CZ EE HU LT LV MT PL RO SI SK EU Total

 Member States  AT 142 110 615 47 68 292 63 274 25 218 9 138 50 102 258 25 1 42 1 30 1 5 2 92 32 26 42 14 2724

(MEMBER) BE 110 189 855 63 145 345 102 674 56 619 13 267 177 173 525 12 2 34 4 37 13 5 0 155 53 53 16 19 4716

DE 615 855 1914 296 464 1625 472 1980 207 2186 31 928 476 768 1846 94 15 267 13 152 29 18 7 552 166 177 107 59 16319

DK 47 63 296 55 56 115 56 224 20 168 4 94 33 97 244 8 0 32 2 10 10 3 0 40 23 28 11 10 1749

EL 68 145 464 56 78 197 70 268 37 387 2 116 74 122 270 34 6 27 4 13 4 4 2 102 61 28 18 10 2667

ES 292 345 1625 115 197 575 213 765 107 984 6 491 306 320 862 42 4 114 13 73 5 10 3 316 106 78 38 15 8020

FI 63 102 472 56 70 213 132 319 28 268 2 142 54 204 365 14 0 39 15 33 1 4 0 105 20 21 32 7 2781

FR 274 674 1980 224 268 765 319 892 161 1305 27 439 324 414 1222 35 2 119 9 85 14 15 1 361 99 99 63 41 10231

IE 25 56 207 20 37 107 28 161 33 139 1 48 27 49 188 7 0 23 2 15 8 7 1 36 18 9 8 6 1266

IT 218 619 2186 168 387 984 268 1305 139 1108 11 512 378 440 1142 58 9 135 13 118 9 17 3 422 179 83 86 42 11039

LU 9 13 31 4 2 6 2 27 1 11 0 7 3 7 11 0 0 2 0 1 0 1 0 2 0 1 0 1 142

NL 138 267 928 94 116 491 142 439 48 512 7 266 123 213 559 16 7 51 7 51 10 9 1 196 61 47 22 16 4837

PT 50 177 476 33 74 306 54 324 27 378 3 123 118 94 192 13 0 16 3 17 3 3 1 83 30 26 5 5 2634

SE 102 173 768 97 122 320 204 414 49 440 7 213 94 154 414 9 0 30 6 46 4 3 1 111 52 38 23 11 3905

UK 258 525 1846 244 270 862 365 1222 188 1142 11 559 192 414 859 67 7 132 18 119 19 29 6 420 109 119 69 25 10096

BG 25 12 94 8 34 42 14 35 7 58 0 16 13 9 67 11 1 10 2 8 5 4 2 35 17 8 9 2 548

CY 1 2 15 0 6 4 0 2 0 9 0 7 0 0 7 1 0 1 1 1 1 1 0 3 1 1 1 0 65

CZ 42 34 267 32 27 114 39 119 23 135 2 51 16 30 132 10 1 16 4 10 2 2 0 42 9 13 17 7 1196

EE 1 4 13 2 4 13 15 9 2 13 0 7 3 6 18 2 1 4 1 3 1 3 0 4 3 4 3 1 140

HU 30 37 152 10 13 73 33 85 15 118 1 51 17 46 119 8 1 10 3 28 3 11 1 33 11 14 14 2 939

LT 1 13 29 10 4 5 1 14 8 9 0 10 3 4 19 5 1 2 1 3 16 2 0 10 5 8 3 0 186

LV 5 5 18 3 4 10 4 15 7 17 1 9 3 3 29 4 1 2 3 11 2 3 1 8 6 6 3 0 183

MT 2 0 7 0 2 3 0 1 1 3 0 1 1 1 6 2 0 0 0 1 0 1 0 2 2 1 1 0 38

PL 92 155 552 40 102 316 105 361 36 422 2 196 83 111 420 35 3 42 4 33 10 8 2 181 51 42 48 6 3458

RO 32 53 166 23 61 106 20 99 18 179 0 61 30 52 109 17 1 9 3 11 5 6 2 51 60 20 14 5 1213

SI 26 53 177 28 28 78 21 99 9 83 1 47 26 38 119 8 1 13 4 14 8 6 1 42 20 10 8 2 970

SK 42 16 107 11 18 38 32 63 8 86 0 22 5 23 69 9 1 17 3 14 3 3 1 48 14 8 23 4 688

EU 14 19 59 10 10 15 7 41 6 42 1 16 5 11 25 2 0 7 1 2 0 0 0 6 5 2 4 0 310

Total 2724 4716 16319 1749 2667 8020 2781 10231 1266 11039 142 4837 2634 3905 10096 548 65 1196 140 939 186 183 38 3458 1213 970 688 310 93060

 Candidate Countries  HR 1 1 7 1 1 2 1 1 0 1 0 2 0 1 3 1 0 0 1 1 0 0 0 1 1 1 1 1 30

(CANDIDATE) TR 13 36 79 11 14 32 21 61 4 66 0 17 22 19 80 8 0 4 3 13 2 2 1 25 11 15 5 2 566

Total 14 37 86 12 15 34 22 62 4 67 0 19 22 20 83 9 0 4 4 14 2 2 1 26 12 16 6 3 596

 Associated Countries  CH 127 146 738 62 76 281 99 392 48 415 6 195 91 127 333 20 3 43 2 23 9 6 2 87 55 38 13 10 3447

(ASSOCIATE) IL 38 44 225 17 28 73 9 87 14 156 2 68 23 31 111 13 0 14 0 22 3 6 2 27 18 11 8 2 1052

IS 1 0 2 1 1 1 0 1 2 2 0 1 1 2 1 2 0 0 0 1 0 1 1 1 1 1 1 0 25

LI 0 0 3 0 0 0 0 2 0 5 0 1 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 13

NO 19 43 182 28 40 71 24 101 9 136 4 79 23 58 96 5 2 12 1 7 1 1 1 31 11 6 8 8 1007

Total 185 233 1150 108 145 426 132 583 73 714 12 344 138 218 542 40 5 70 3 53 13 14 6 146 85 56 30 20 5544

 Member States 

(MEMBER)

Page 59: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

63

While the share of new entrants increased over time, Figure 24 shows that

collaborations between countries remained mostly located within the EU15.

3.2 Project overview

The 389 NMP FP6 projects in the supplied database were examined having as basis the

project instruments, the EC contribution attributed per project instrument and the

number of participants per project instrument.

As it can be seen in Table 9below, in terms of number of projects, more than half of the

funded projects were STREPs, while in terms of EC contribution the largest share of the

NMP programme budget was obtained by IPs.

Project Instrument

Total number of

projects

% of project instruments

EC contribution % of EC

contribution

Number of

partners

% of partners

Average number of

partners per project

STREP 220 56,56 % 442 641 327,26 € 30,68% 1952 35,51% 8,9

IP 95 24,42 % 812 556 543,14 € 56,32% 2485 45,21% 26,2

NOE 22 5,66 % 157 221 742,96 € 10,90% 441 8,02 % 20

CA 16 4,11 % 15 550 544,90 € 1,08% 372 6,77 % 23,3

SSA 36 9,25 % 14 712 498,12 € 1,02% 247 4,49 % 6,9

Total 389 100,00 % 1 442 682 656,38 € 100,00% 5497 100,00 % 14,1

Table 9: Analysis of FP6 NMP participation over project instruments, (Source: EC Database; n=389 projects)

The graph below (Figure 25) shows the repartition of the projects taking into

consideration the start and end dates. 36% of the NMP projects within the FP6 (139)

were launched in 2004. 33% of them started in 2005. 26% in 2006 and 5% started in

2007. Although FP6 started in 2002, almost all projects were launched between 2004

and 2006 and had an average duration of 3,5 years.

Page 60: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

64

Figure 25: Start and end date of the 389 FP6 NMP projects, (Source: EC database; n=389 projects)

Specifically, as shown in the Figure 26 below, almost half of the total NMP projects

lasted 36 months, 25% lasted 48 months, 31 projects lasted 24 months or less, and 24

lasted more than 48 months.

Figure 26: NMP average Project duration, (Source: EC database; n=389 projects)

Since the scope of this study is to explore the impacts of the NMP projects, it was

decided that only projects finished six months before the interview process would be

taken into consideration. 294 projects were finished before September 2009 and these

were taken into consideration for the survey analysis. 341 of the 389 projects were

finished before 31 December 2009.

0

20

40

60

80

100

120

140

2003 2004 2005 2006 2007 2008 2009 2010 2011

2

139

130

100

18

0 0 0 003

8

26

69

130

105

42

6

Num

ber o

f pro

ject

s

Year

Start Date

End date

Page 61: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

65

4 MOTIVATION AND OUTPUTS From this section onwards we discuss the results of the NMP programme across the

main axes of analysis that involve the nature of the research conducted and the impacts

achieved and expected to be achieved. These axes, already described in Section 1.2.1

move away from the administrative nomenclature of RTD programmes towards

concepts and categories that are more relevant to contemporary readers. The

categories, which have been developed based on the quantitative and qualitative

analysis of the information obtained via this study, the results of Lot 2: Strategic Impact

Study [8], the FP8 Ex-ante impact assessment study [10] as well as other relevant

studies in the field, are considered essential tools for the analysis of the impacts of the

NMP programme and central to its understanding.

4.1 The FP6 NMP Project Portfolio

A notable difference between the FP6 programme and the preceding FP5 has been the

shift in emphasis from short to longer-term activities and from incremental to

breakthrough innovation strategies; a change also depicted in the NMP programme. As

stated in the NMP programming documents, the key intention of the EC with the NMP

programme has been the stimulation of major advances in industrial materials and

processes so as to facilitate the shift of the EU to a sustainable, knowledge-based society

[13].

As mentioned in earlier parts of this report, the NMP Programme was characterised by

a strong participation of business / industry organisations (38%) with an almost equal

share of education and research organisations (30% and 26% respectively) following

the participation pattern of the overall FP6 [14]. As depicted in Figure 27, providers of

professional, scientific and technical services (34%), educational institutes (27%) and

manufacturing companies (22%) composed the majority of NMP participants. SMEs

were 14% of the total population, participating mainly in STREPs and IPs and were

typically R&D-intensive organisations, mostly manufacturers or providers of

professional, scientific and technical activities. Large enterprises counted for 12% of

the total population, participating mainly in STREPS, IPs and CAs and belonging mostly

to the same sectors as SMEs.

Page 62: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

66

Figure 27: Sectorial repartition of FP6 NMP participants, (Source: FP6 NMP Participants

survey, n=1181)

The information concerning the organisations that participated in FP6 NMP projects

was cross-referenced with information presented in the EU Industrial Scoreboard of

2010 [10], so as to identify whether the top R&D investing companies of the world

participated in the programme. As depicted in Table 10, a significant number of

European top R&D investment companies have also participated in FP6 NMP

projects, confirming that the programme has attracted the best research

institutions and the most innovative companies of the EU. The importance in the

participation of these organisations is rather considerable as without their

involvement, the NMP Programme could not have hoped for a significant impact in

either the quality of research undertaken nor in the subsequent diffusion, adoption and

commercial exploitation of the research as such organisations are considered ‘hubs’ of

Europe’s national and industrial research networks. Furthermore, the collaboration

with such ‘elite’ organisations allows a strong transfer of knowledge and know-how

and a significant spill-over effect over the whole R&D community.

% 10% 20% 30% 40%

Information and Communication

Real estate activities

Water supply/Sewerage/waste mgmt/etc

Electricity, gas steam and air conditioning supply

Public administration/defence/social security

Wholesale and retail trade, etc.

Agriculture, forestry and fishing

Transportation and storage

Administrative and support service activities

Human health and social work activities

Mining and quarrying

Construction

Other service activities

Other

Manufacturing

Education

Professional, scientific & technical activities

%

%

%

%

%

%

%

%

1%

1%

1%

2%

2%

8%

22%

27%

34%

Page 63: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

67

MNIBS

A succesful joint EU-US project

The MNIBS project (Multiscale Modelling of Nanostructured

Interfaces for Biological Sensors) was a co-funded project between

the EU and the US, bringing together top researchers15 from both

sides of the Atlantic Ocean, that aimed at developing a fundamental

understanding of the segregation and self-assembly of phospholipids

and polypeptide amphiphiles at interfaces and their interaction with

aqueous phases and liquid crystalline materials.

The joint EU NMP – US NSF Call scheme has proved to be a very

flexible tool for enabling collaboration beyond the borders of the EC.

Although the objectives of the project were considered as highly

ambitious by the project partners, since the project mainly involved

the production of novel knowledge at the international level, the

outcomes of the project could be commercially exploited. As of today,

an American company that was indirectly involved in the project via a

US partner, is commercially exploiting results of the project.

Additionally, a number of EU companies have already expressed their

commercial interest to further utilise outcomes of the project, and a

multinational company is going to collaborate with members of the

consortium in order to further research aspects of MNIBS that could

be commercially exploited in the near future.

Case study 1: MNIBS: a successful EU-US cooperation project.

From the Table 10 we can also identify that the top

international R&D investment companies are not

evident in FP6 overall or in NMP. The CSF Ex-Ante IA

report for NMP [11] has identied a number of issues

that may have resulted in this outcome, such as: the

legal issues that prevent organisations from third

countries participating; the EU grant agreement general conditions that do not fit with

US regulations for the most relevant universities; intellectual property issues, financial

and management concerns and even customs issues seem to hinder the collaboration

between EU and non-EU organisations. As an example, many FP6-NMP projects had

collaboration with Russian organisations, due to their geographical proximity and

strong positioning in nanotechnologies16 but several issues were mentioned such as

difficulties in transfering equipment or payments through the customs. In contrast with

these conclusions, a carefully planned cooperation set-up through a joint call, can prove

to be very succesful (Case study 1).

15 MNIBS project had as partners Northwestern University, University Of Wisconsin-Madison and Purdue University respectively ranked #12, #45 and #56 in the National University Rankings, which consider 262 national universities. 16 Project Istok-Soyuz, “Encouraging EU-RU cooperation in the field of ICT”, http://www.istok-soyuz.eu/index.php/target-eeca-countries/66-russia

The NMP Programme

attracted the best research

institutions and the most

innovative companies

Page 64: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

68

Case studies demonstate that the main players engage in strategic projects. The

available funding is not the main driver and their strong long term market orientation

complement the shorter business perspective of SMEs and such an association has

proved to be very successful (see Case study 2). So top industrial R&D players can be

attracted by demonstraing to them the high level of research conducted and the

technological potential. As shown in section 4.3.3, dissemination activities are a useful

tool to capture this attention (see Case study 6).

Rank Company

(nationality)

Participation

in FP6 NMP

Rank Company

(nationality)

Participation

in FP6 NMP

1 Toyota (Japan) No 15 GlaxoSmithKline

(UK)

Yes

2 Roche (CH) No 16 Merck (US) No

3 Microsoft (US) No 17 Intel (US) Yes

4 Volkswagen (DE) No 18 Panasonic (Japan) No

5 Pfizer (US) No 19 Sony (Japan) Yes

6 Novartis (CH) No 20 Cisco Systems (US) No

7 Nokia (FI) No 21 Robert Bosch (DE) Yes

8 Johnson & Johnson

(US)

No 22 IBM (US) No

9 Sanofi-Aventis (FR) No 23 Ford (US) No

10 Samsung (South

Korea)

No 24 Nissan (Japan) No

11 Siemens (DE) Yes 25 Takeda (Japan) No

12 General Motors (US) No 27 AstraZeneca (UK) No

13 Honda (Japan) No 29 Bayer (DE) Yes

14 Daimler (DE) Yes 30 EADS (NL) Yes

Table 10: Participation of top R&D companies of 2010 in FP6 NMP, (Source: EU industrial R&D scoreboard 2010 and FP participation data, EC and survey databases)

Page 65: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

69

NEXT

Engaging big players

NEXT was a €21 M IP project gathering 23 partners including large

industry players who had a driving role for market requirements.

The main outcomes of the NEXT project have been six demonstrators

accompanied by guidebooks, methodologies and software tools for

developing new business models. Some of the prototypes developed

were already commercially exploited (e.g., the 2 dof robot) before the

end of the project, while others were on the verge of

commercialisation. Three patents were already registered before the

end of the project, while others were still pending.

Additionally, half of the survey respondents mentioned that their

organisation realised or expects to realise commercial returns as a

result of the exploitation of project outcomes. As stated in the

interviews with selected partners of the project, the NEXT project

created new business opportunities for all partners, from which direct

and commercial benefits are gained. Furthermore, scientific,

networking and prestige benefits were also other important benefits

for the NEXT partnership.

Case study 2: NEXT: Engaging big players

Figure 28 reports the overall views of respondents in response to twelve RTD

dimensions, and these answers are represented as a percentage of the total number of

respondents. In summary “The majority of respondents saw most research

conducted in FP6 NMP programme as highly scientifically and technically complex,

of high strategic significance for their institutions but not involving too high a

commercial or technical risk and certainly not being of high cost for the research

involved”. Around half of the respondents participated in applied research projects and

one fifth in fundamental research projects. The majority of them built on past R&D

conducted in house and funded from national or regional programmes (partially or to a

larger extent). As stated in the Lot 2 study [8] “...many of the national NMP-related

programme,[....] were addressing the issue of national actors’ participation in FP projects.

National funding was often designed in such a way, that it was supplementary to

European financing. In some countries (especially new MS and associated states),

participation in EU projects was made a strategic priority, especially in cases of countries

that struggle with lack of own resources”.

Page 66: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

70

Figure 28: Dimensions of research activities conducted in FP6NMP projects as perceived by survey respondents (Percentage of total respondents, source: FP6 NMP Participants survey,

n=1181)

The majority of the survey participants stated

that they will continue performing R&D in the

area through national (65%) and EU (56%)

funding programmes (Figure 29). According to a

more detailed statistical analysis performed, it was identified that around half of the

participating research institutions (46%) will attempt to retrieve international funds

for the continuation of their future NMP research activities. It would be interesting in

future studies to identify the impact of international partners in opening European

institutions to international projects and sources for research funding.

Figure 29: Continuation of R&D in the area, (Percentage of total respondents, Source: FP6 NMP Participants survey, n=1181)

0%20%

40%60%

80%100%

High Cost

Commercial Risks

Applied Research

Involvement of top researchers (int)

S&T Risks

Core technology area

Links with in-house projects

High Strategic Importance for org

Long-term nature of R&D

Not feasible without external collabs

Involvement of top researchers (EU)

S&T Complexity

33%

42%

52%

52%

55%

61%

61%

67%

68%

70%

79%

83%

Almost 90% of participating

organisations will continue

pursuing research in the area

Page 67: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

71

Nano2Life

Structuring a new EU research

community

The Nano2Life project was one of the very first European nanobiotech

Networks of Excellence, dedicated to establishing and structuring a

Nanobiotech framework in Europe. The consortium consisted of 23

partners from higher education & research institutes. The project

emerged as it was recognised by the EU that there was much

fragmentation in this area and that to make Europe an international

leader in nanobiotechnology, the establishment of a nanobiotech

network was an essential step. The consortium was created by

getting as many multidisciplinary and nanobiotech institutions

involved as possible, noted by the number of German partners who

already benefited from a national network, as well as making sure that

there was strong and adequate geographic coverage.

While the project failed in setting-up a sustainable structure, it

created a strong community which had a structuring effect in Europe

and among its achievements gave birth to 75 complementary

research projects directly or indirectly linked to the project, 51

patents were filed for during and after the project. Interviewees

mentioned that without EC funding, the NoE would not have been

built

Case study 3: Nano2Life: Structuring a new EU research community

The five main European Technology Platforms (ETPs) that attracted most of NMP

participants were EuMaT (35%), ManuFuture (24%), MINAM (17%), NanoMedicine

(14%) and SusChem (10%). According to the interviews performed it was evident that

many of these participations were not only active but also that the FP6 NMP strongly

contributed to the establishment and consolidation of related ETPs such as EuMaT or

ManuFuture. An example, the RI-MACS project built the operational tool of ManuFuture

while the VRL-KCIP project had 18 of it experts joining the 40 participant prospective

group of ManuFuture. Interviewees mentioned that contributing to the ETP’s Strategic

Research Agenda was an excellent opportunity to get involved in the definition of work

programmes and thus getting further projects to fund the area of research.

NMP is a rapidly evolving field and thus an external

environment was expected to affect the course of some

projects: around two thirds of the participants stated

that changes in their projects external environment did

occur but they were either insignificant or even in the

cases where they were significant they did not cause

major changes to the projects. Among those participants

The NMP programme

demonstrated enough

flexibility to accommodate

internal and external

evolutions of context

Page 68: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

72

where changes in the external environment did require a change of their projects, this

usually involved an alteration of the composition of the consortium, its timing and

nature of the deliverables and more rarely a redefinition of the scientific objectives of

the project. As observed also from the conducted interviews, changes did occur in

consortium composition, project objectives (either due to technological issue or

changes in partners’ plans) and external context but FP6 NMP allowed flexibility that

allowed projects to encompass and integrate alterations and fluctuations of their

external environment within their auspices. The only very limited cases where issues

have been reported were in projects for which a change would have been necessary but

was not implemented due to a coordinator or consortium being afraid to modify the

initial plan.

4.2 Assessment of NMP Participant Motivation

Figure 30 below shows the importance of the various

goals to the different categories of organisations

involved in the FP6 NMP programme. The differences in

the goals are noteworthy, even if somewhat expected.

S&T goals were considered the most important reason

for participating in NMP projects for the majority of the participants to the survey

(88%) followed by economic and health/ environmental goals (considered important

by 46% and 41% of the population respectively). The case studies show that in most of

the cases, the core members of the consortium had collaborations and personal

relations prior to the project set-up and this is recognised as a success factor for the

project execution. The project ideas come mostly from 3 sources:

A coordinator facing a particular challenge that it is not able to solve on its own.

Research conducted within a national project that is needed to be taken up to

the EU level to get the ability to cover the whole lifecycle of a technology or a

product (projects HOLIWOOD or TUNCONSTRUCT are examples of such

situations).

A previous FP (mostly FP5 in the case of FP6 NMP) project.

Scientific & technological

advancements have been

the main target of FP6 NMP

participants

Page 69: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

73

Figure 30: Importance of Goals for Participants in RTD projects, (Source: FP6 NMP Participants survey, n=1.181)

As stated in Section 1.2, the underlying objective of the FP6 NMP programme was to

move towards a knowledge-based and more environmentally friendly industry through

an integrated approach combining materials science, nanotechnology, production

technologies, information technologies, biotechnologies, and so forth so as to opt for an

efficient transformation from resource-based to knowledge-based European industries. In

this view, the goals of the participating organisations were in line with the objectives of

the programme as the majority the participants stated during the online survey that

they did not have high economic expectations from their projects. However they mainly

wanted to explore whether their innovative ideas – from the scientific and

technological point of view – could be conceptually proved or if they could accomplish

an innovative scientific or technological breakthrough that at a later point, usually after

the end of the project, could be commercially exploited. This is also in line with the pre-

competitive nature of FP6. Perhaps this also justifies why organisations stated in the

survey, and also during the interviews, that in the future they shall address

national/regional programmes for their new NMP projects, as such programmes

usually allow a focus on “practical aspects” of innovation and the market exploration of

their ideas.

An interesting situation shown in some projects is the case where a patent is filled prior

the project and serves as an input to the project (Figure 31). When IPR issues are

clearly and transparently solved within a consortium agreement prior to the project, it

can prove to be very successful to transform an idea into a market opportunity as

shown by project AUTOBONE (see Case study 4).

2

8

18

19

16

1

6

12

13

9

2

14

20

19

15

8

27

27

26

20

30

27

16

16

24

58

19

7

6

16

-100,00% -50,00% 0,00% 50,00% 100,00%

Scientific and technological goals

Commercial and economic goals

Policy-oriented goals

Social goals

Health and environment related goals

Not applicable Very Low importance Low importance Moderate importance High importance Very high importance

Page 70: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

74

Figure 31: FP6 NMP on the innovation pathway

AUTOBONE

Building proof of concepts

through FP6 projects

The aim of the AUTOBONE STREP project was to combine the

developments in the areas of tissue engineering based on materials

and 3D scaffolds development with large laboratories and a few

commercial activities in the field of bioreactors. It would transpose

the concepts of tissue engineering and regeneration of the frontiers of

new developments: the automated production of autologous bone

replacement materials. One of the goals of this project was to develop

a bioreactor that would be set-up within the confines of a hospital and

which would be used to homogenously activate specially designed

porous matrices with stem cells from harvested bone marrow

aspirates of the patient to produce antilogous hybrid bone graft

materials.

A comparison between the project’s objectives and outcomes

indicates that what was initially planned was actually accomplished to

a high degree. It is very important to state the fact that the concept of

implementing bone reengineering assembling with the

development of biomaterials was a concept that was patented by

an academic partner of the project prior to its elaboration. Since

AUTOBONE was funded, a proof of principle has been

accomplished which was the development of an automotive

bioreactor device. In this sense and as a result, the university started

up a new spin-off company for this device which will be involved,

among others, in the exploitation of the concept following

AUTOBONE’s results. The idea was thus written taken into account

this patent. IPR issues were already discussed prior to its submission

so no problems were encountered in this respect whatsoever.

Case study 4: AUTOBONE: Building proof of concepts through FP6 projects

IPR issues clarifiedwith the consortium

prior project building

Patent

•Filed by university

Proof of concept

•Done in FP6 NMP project

Spin-off

•Created by the university

Page 71: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

75

The NMP programme responded well to its strategic objectives, i.e., to foster

‘innovation’ on the one hand and promote the ‘transformation of industry’ on the other,

in terms of the immediate outputs generated by the projects. Table 11 below shows

that the programme produced strong S&T outputs, aimed to enhance the knowledge

and skills of participants and to improve the operational processes of the organisations,

as well as significant economic outputs, through product and process innovations.

Very high importance

High importance

Moderate importance

Low importance

Very low importance

Non applicable

Knowledge and Skills Group of Outputs

Publications 28,1% 25,7% 24,6% 11,6% 6,3% 3,8%

PhDs 19,5% 20,4% 16,2% 17,9% 13,7% 12,2%

Training Programmes 13,6% 27,4% 27,7% 17,3% 6,6% 7,4%

Good practices 25,1% 40,7% 22,2% 6,5% 1,8% 3,6%

Product Innovations Group of Outputs

New/improved materials 30,4% 29,0% 15,7% 7,0% 7,0% 10,8%

New/improved services/products/components

29,8% 29,3% 14,9% 8,6% 5,5% 11,9%

New/improved software/simulation models

18,9% 19,2% 17,6% 15,7% 12,6% 16,0%

New/improved manufacturing systems

26,0% 23,5% 18,2% 10,5% 7,8% 14,0%

Prototypes, demonstrators & pilots

27,1% 27,3% 14,9% 10,4% 6,6% 13,6%

Patent applications 14,8% 15,6% 20,5% 16,5% 15,4% 17,2%

Copyrights, trademarks, etc. 7,1% 12,9% 17,5% 18,9% 21,1% 22,4%

Organisational Processes Group of Outputs

New/improved processes 31,7% 30,0% 15,1% 8,4% 4,6% 10,2%

New production concepts 20,7% 26,4% 16,3% 10,7% 9,5% 16,4%

New tools for efficient life-cycle design, etc.

11,3% 19,7% 21,0% 15,3% 13,6% 19,1%

New organisational models 6,6% 10,7% 18,4% 20,0% 20,0% 24,4%

Sustainable processing of multifunctional materials

21,4% 20,5% 16,6% 12,7% 10,4% 18,4%

Quality and safety assurance Group of Outputs

New/improved quality standards

20,7% 23,9% 22,3% 13,7% 7,2% 12,2%

New/improved control devices 15,5% 19,6% 21,1% 16,9% 9,5% 17,4%

New/improved characterisation tools

18,4% 25,0% 22,4% 12,6% 7,4% 14,1%

Safer products 13,4% 17,7% 17,0% 16,5% 14,7% 20,6%

New/improved knowledge management tools

10,6% 15,5% 18,4% 17,4% 17,3% 20,8%

Table 11: FP6 NMP Projects Outputs, (Source: NMP Participants survey, n=1.181))

As observed from the table above, the most important outputs elaborated were

knowledge and skills related (primarily publications and good practices) as well as

product innovations, such as new/improved materials, services/ products/

Page 72: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

76

components and processes and prototypes, demonstrators and pilots. However, since a

large part of the accomplished outputs did involve economic benefits it was anticipated

that the “registration” of obtained knowledge and created outputs through patenting,

registering, trade-marking procedures, etc., would be considered as important by a

higher portion of the total population than actually observed. From the case studies,

two answers can be provided:

- For one part of the organisations, knowledge registration is not a priority and

procedures to do so are even unknown. Several interviewees underlined the

need to get support on these procedures and the role of ESIC seminars in

getting a more comprehensive vision of registration issues was acknowledged.

- For another part of the organisations, FP6 NMP projects aim at providing an

incremental knowledge, feeding the overall innovation process of the

organisation. Registration and exploitation of knowledge is then not a targeted

output of the project and is generally done, with less visibility, within the

organisation.

4.2.1 Industrial organisation perspective

As observed in Figure 32 the main objectives of the industrial organisations that

participated in the FP6 NMP programme, according to the views collected during the

online survey and respective personal interviews, were similar to the overall

population: the industrial organisations were primarily interested in gaining further

scientific and technological insight in the NMP area, secondarily in economic benefits

and thirdly in environmental and health issues.

Figure 32: Importance of Goals for Industrial Participants in RTD projects, (Source: FP6 NMP Participants survey – selection of Very Important and Important responses, n= 453)

According to Figure 33, the most important outputs produced by the majority of

industrial organisations were a combination of knowledge and innovation outcomes

that would assist them in enhancing their knowledge basis (good practices) on the one

hand as well as obtaining economic benefits via the commercial exploitation of

innovations produced (new/improved services/ products/ component/processes,

Page 73: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

77

prototypes, demonstrators and pilots) on the other. Whilst patenting and copyrighting

of the obtained knowledge, outputs, results, etc., was of significant importance only to a

small part of the total respective population. As seen from the case study, this trend for

industry is mostly explained by the second explanation given above. FP6 NMP projects

only cover a small part of the industry innovation cost and registration of knowledge is

not an expected output as it is tackled within the company processes. Extracting the

real contribution of one project in the overall innovation process of a company is

indeed difficult, with the exception of small companies for which the ratio between

project knowledge creation and company overall knowledge creation is higher.

Figure 33: Important Outputs for Industrial Participants in RTD projects, (Source: FP6 NMP Participants – selection of Very Important and Important responses, n=453)

4.2.2 SME and Large Enterprise perspective

Around one third of the total population of NMP was composed of SMEs and large

enterprises (LEs) that aimed, as depicted in Figure 32 above, primarily for S&T goals,

similarly to all other participants of the programme. But interestingly SMEs have been

by far more interested in economic benefits and achievements than large enterprises.

This is in line with the vision from the case studies stated previously. The relative

impact of a project is much higher for a small company than for a large one. Large

companies’ main goal of participation was gaining access to scientific and technological

‘windows of opportunity’, sharing technical risks and using the NMP programme as an

extension and consolidation of research collaborations, especially with research

providers.

SMEs as well as large enterprises mainly aimed for innovations close to the market

Page 74: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

78

through the development/improvement of services/products/components, processes

and materials as well as the elaboration and exchange of good practices. According to

the advanced statistical analyses elaborated on the two populations, the opinions of SMEs

and LEs differed significantly (95% confidence interval) only on the importance of

development/improvement of quality standards: half of the participating SMEs stated

that this was an important output of their project while significantly less - around 1/3 - of

the large enterprises had a similar view.

Figure 34: Importance of Outputs for SME Participants in RTD projects, (Source: FP6 NMP

Participants survey- selection of Very Important and Important responses, n=106)

Page 75: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

79

Figure 35: Importance of Outputs for large enterprises in RTD projects, (Source: FP6 NMP

Participants survey- selection of Very Important and Important responses, n=145)

4.2.3 Higher education and research institute perspective

Higher education establishments and research organisations have been primarily

interested in S&T goals and relatively less on other aspects (economic, policy, health,

environmental and social ones), as shown in Figure 36 below. An interesting

observation concerns the economic ambitions of the two types of organisations: one in

every three research organisations were interested in such goals in comparison to one

in every four higher education institutes. The difference is statistically significant (95%

confidence interval), as are the differences in policy and health and environmental

goals that have been rated higher by research organisations than academia.

Figure 36: Importance of goals for HE and RES participants of RTD projects, (Source: FP6 NMP Participants survey – selection of Very Important and Important responses, n=657)

Page 76: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

80

Further differences were observed on the importance of achieved outputs between the

higher education institutes and the research organisations as depicted in Figure 37 &

Figure 38. As anticipated the majority of academics considered as primarily important

outputs of their NMP projects the enhancement of scientific knowledge and skills, via

publications, training programmes and the development and exchange of good

practices, as well as the improvement of materials and processes. As well as the

enhancement of their scientific knowledge and skills, the majority of research

organisations considered as important outputs the enhancement of production

elements via the development/ improvement of materials, processes,

products/components/services. Between the two populations advanced statistical

analyses were conducted so as to identify statistically significant differences on the

importance of their outputs. According to these analyses, the responses of higher

education institutes and research organisations differed significantly about the

importance of knowledge-related outputs (publications, PhDs and other formal

qualifications and training programmes), quality and safety assurance outputs

(development of new or the improvement of processes, characterization tools and

quality standards) as well as of product enhancement outputs (development and or

improvement of components/ products/ services and the creation of prototypes and

demonstrators). Although the difference in the knowledge related outputs was

anticipated, the differences concerning the other outputs are considered captivating.

No clear explanation was obtained from the case studies on this issue despite one

element of explanation could be proposed: as shown earlier, the focus of the industry is

on the development and improvement of news services, products and components. To

gain research contracts from the industry, the higher education institutes and the

research organisations need to provide different and complementary expertise,

necessary for the industry without raising IPR issues. The higher education institutes

and the research organisations provide the underlying knowledge on top of which the

industry is able to develop and improve its products and services.

Page 77: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

81

Figure 37: Importance of Outputs for higher education institutions in RTD projects, (Source: FP6 NMP Participants survey – selection of Very Important and Important responses, n=

353)

Figure 38: Importance of Outputs for research institutions in RTD projects, (Source: FP6

NMP Participants survey – selection of Very Important and Important responses, n= 353)

Page 78: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

82

4.3 Assessment of Achieved Outputs, Outcomes and Results

4.3.1 Scientific and Technological Outputs

It has to be underlined at this point that getting a detailed and precise view of project

outcomes is a difficult task when it comes to making an aggregated analysis. Even the

interviews performed during the construction of the case studies did not allow the

evaluation team to get precise information on the selected projects. When looking at

the interviewed partners, retrieval of exhaustive and descriptive lists of outcomes was

nearly impossible: there were always uncertainties such as the number, type and status

of publications or patents produced by the organisations. A general issue encountered

here is that projects contribute to the general RTD activity of the organisations and so

beyond the deliverables due within the project, it is often difficult to estimate the extent

to which the project contribute to an achievement (publication, patent, commercial

project).

The Scientific and Technological outputs produced per partner increased linearly with

the size of the organisation, as anticipated and depicted in Table 12 below: SMEs

produced on average 2 publications and participated in 7 conferences while higher

education, research institutes and large enterprises produced on average at least 6

publications and participated in more than 11 conferences. Detailed statistics related to

project outputs as expressed within the survey are given in Annex VII

Average number of articles

Average number of participation in

conferences

Higher education 12 14 Research institutes 11 15 SMEs 3 7 Large enterprises 6 11 AGGREGATED RESULTS 9 11

Table 12: Average number of articles and participations in conferences per organisation type, (Source: FP6 NMP Participants survey, n=1181)

With respect to S&T achievements with short-to-medium term economic perspectives,

i.e., patents, software packages, spin-off companies, etc., we observe that larger

organisations (i.e., higher education institutes, research organisations, large

enterprises) tended to produce patents more often than SMEs while SMEs tended to

register more often their knowledge via copyrights, trademarks, etc. With respect to

spin-offs around one in every ten higher education and research institutes as well as

SMEs tended to establish one while one in every twenty large enterprises produced

such an output. Almost twice as many SMEs and large enterprises contributed to

standards/normalisation bodies than higher education and research institutes. This is

in line with expectations as standardisation and normalisation issues are of greater

concern for the industry. Nevertheless, it may be advisable to search for an increased

participation of higher education and research institutes in pre-standardisation

activities. Indeed, definition of standardisation mandates require early information

Page 79: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

83

which can be partly provided by the research activity.

% of project with at

least one patent

% of project with at least

one copyright

% of project with at

least one spin-off

% of project with at least

one developed software

% of project that contributed

to standards

Higher education

21% 4% 8% 24% 9%

Research institutes

26% 7% 9% 23% 11%

SMEs 18% 10% 9% 29% 23% Large enterprises

26% 5% 5% 29% 20%

AGGREGATED RESULTS

20% 8% 7% 23% 12%

Table 13: Economic and quality assurance outputs of FP6 NMP projects, (Source: FP6 NMP Participants survey, n=1181)

4.3.2 Commercial Return on Investment

24% of the organisations that participated in the survey stated that they have realised

or expect to realise commercial returns through the exploitation of their project results,

while 28% considered such exploitation highly unlikely. Among those that expect

commercial results, around half expect commercial returns of more than €100.000

on an annual basis.

Figure 39: Possibility of Commercial Returns (source: FP6 NMP Participants survey, n=1181)

Figure 40: Levels of expected Commercial Returns (source: FP6 NMP Participants survey, n=202)

Around one fourth of industrial participants, 17% of research institutes and only 6% of

higher education organisations expect commercial benefits to be gained from their

participation in NMP projects. Amongst them, almost half of the industrial

organisations, 35% of the research organisations and 19% of the higher education

institutes expect revenues of at least € 100.000 per annum.

One third of the SMEs and 15% of the large enterprises expect to gain commercial

benefits from the exploitation of their NMP results and among them 28% of SMEs and

almost half of the large enterprises expect revenues of more than € 100.000 per

Page 80: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

84

annum. The case studies show that in most of the cases, the commercial returns are yet

to be realised but are anticipated to be confirmed as expected. A few cases of revenues

gained from patent licensing have been announced.

From a pillar perspective, and according to the case studies, projects under the NMP3

pillars reveal that the technologies developed in the projects allowed for process

improvement and that this, while improving the company efficiency, does not create

commercial returns as such (see Case study 5 and Table 14).

CEC-MADE-SHOE

Increasing EU competitiveness

through process innovation

CEC-MADE-SHOE is an IP gathering 56 partners. It was proposed

under the 3rd NMP pillar in a joint IST-NMP call. To maintain the

European competitiveness on the shoe manufacturing market, the

project aimed at developing radical new manufacturing processes,

new materials with a focus on environmental friendliness, an

actualised relationship between the supply and demand side and the

whole electronic integration of the value chain permitting the optimal

use of nanotechnologies and intelligent materials.

While the project produced limited direct commercial returns and did

not lead to an extensive amount of knowledge registration, it had a

number of outputs contributing to EU competitiveness (see Table 14).

From the interviews it appeared that while not all the research was

ready for the market at the end of the project, 70% of the results were

useful to take it further. This project was an initiator of improvement

and the first results are slowly showing off. All commercial results are

foreseen to become visible within 1 to 5 years after finalisation of the

project: 20% in the first and second year and 10% in the third, fourth

and fifth year. Also, as an economic impact, the one-step production

process developed by the project improved productivity more than

15% for small series production, which is the trend for European

companies

Case study 5: CEC-MADE-SHOE: Increasing EU competitiveness through process innovation

Page 81: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

85

Output Description

New / improved materials New bio and eco-friendly leathers (more than18 materials in 48 different finishings) and glues (3 typologies) for footwear products.

New lining s and soles exploiting advanced formulations/properties New/ improved manufacturing systems

A new last milling process An advanced one step production line, integrating shoe manufacturing

phases Pervasive quality controls as knowledge generators (online monitoring,

process setup definition, knowledge extraction) New/ improved services/ products/ components/ appliances / devices

3 new typologies of advanced shoe product: Bioshoe – totally bio product Active shoe – shoe incorporating new functionalities/properties Seamless-Snap shoe – shoe conceived though new manufacturing

approaches (no sewing, direct manufacturing) Magic Mirror – a tool for virtual try–on of the to-be-manufactured shoe 3D integrated CAD environment

Sustainable processing of multifunctional materials

New bio and eco-friendly leathers, and PU/TPU formulations

New/improved processes Direct manufacturing of shoe products by sintering

New/improved control devices and instruments

IEC 61499 standard based control system design tool

New/improved characterisation tools

CEC co-design concept and tool Virtual shoe test bed – a tool for virtual simulation of physical properties of the to-be-manufactured shoe

New production concepts Instant mould - a faster and quicker alternative to metal mould

New tools for efficient life-cycle design, production, use and recovery of systems

Definition of standards for business interoperability (product codification and business documents like order, delivery note, etc.

Process integration and coordination (PIC-Shoe) – tool supporting integrated B2B functionalities along the production chain, from suppliers to producers to third party manufacturers

Integrated European Retail Platform (IERP) – tool supporting electronic communication between producers and retailers

New organisational models New business models for three shoe typologies Multipurpose process simulation for production evaluation and

organisation New / improved knowledge management tools

Sales forecasting tool

Prototypes, demonstrators and pilots

Several shoe samples produced by exploiting new advanced materials and shoe solutions

One integrated technological demonstrator at CNR-ITIA Integrated pilot plant

Networked demonstrator of ICT solutions for production management (PIC-shoe and IERP), involving different companies - as nodes – across Europe

Demonstrator of the one-step production process, still in real operation in a Portuguese leading company.

Patent applications Patent application on Magic Mirror

Copyrights, trademarks, registered designs, licences etc.

STEP For Green Label

Other outputs Fashion and style guidelines for new shoe concepts

Table 14: Example of project outputs (Source: CEC-MADE-SHOE IP project)

4.3.3 Assessment of Dissemination & Communication

Channels

As depicted in Figure 41 below, the media channels that were mostly utilised by the

NMP projects participants were addressing the scientific community (i.e., S&T journals,

conference papers, etc.) as well as specific target groups mostly through the

participants’ own websites. Around 40% of the respondents stated that they promoted

information about their projects through the Commission’s websites while with respect

to the utilisation of more conventional and broad audience media such as local or

national printed and online newspapers as well as TV and radio programmes, between

14% and 24% of participants used such means. According to further analyses

Page 82: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

86

performed, it was identified that around one third of respondents considered the mass

media, such as TV, radio programmes and newspapers (online and offline) as non-

applicable means of project dissemination.

Figure 41: Visibility of RTD NMP projects in different media channels, (Source: FP6 NMP Participants survey, n=1181)

As anticipated a higher percentage of higher education and research institutes

(“academia”) utilised media channels for promoting their research results rather than

SMEs and large enterprises (“Industrial partners”). This result is in line with the finding

of the Lot 2 study, in which it was stated “...the dissemination in NMP FP6 (...was...)

addressed towards academia only and not enough towards the industry and the broad

public” [8]. On the one hand, this result is understandable under the light of the

purposes for dissemination among the two populations: academia mainly aims at

producing publications and articles so as to enhance its academic and departmental

prospects, while industrial partners on the other hand wish to publish information

about their achievements so as to attract new customers and improve their reputation

and visibility. However, one should also take into consideration that NMP project

participants were required to create dissemination strategies (recorded in the Plan for

Using and Disseminating Knowledge (PUDK)) so as to ensure maximisation of

dissemination efforts and provision of information to a large audience. Based on the

qualitative analysis of almost 100 PUDK reports (collected during the assessment of the

case studies) it was evident that these strategies mostly involved the presentation of

information through conferences, project meetings, workshops, publications, etc. and

mainly aimed at reaching the scientific community.

Page 83: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

87

MULTIPROTECT

Dissemination to increase

exploitation

The MULTIPROTECT brought together 31 academic and industrial

researchers from 13 countries. In a period of 51 months, it developed

several nano-structured coating materials, and the most promising

were up-scaled and transferred to the industrial project partners for

application and testing on demonstrator parts comparable to parts

taken from production. From these developments, new market

opportunities are expected in the future for the aerospace and

automotive industries as well as for coatings producers and coatings

end-users.

The technology developed during MULTIPROTECT will eventually

become commercially exploitable as the follow up projects based on

the research results and developments of MULTIPROTECT have

already tackled the problems that occurred regarding the cost and the

technical limitations. For example the MUST project, which is based

on MULTIPROTECT, will release nanocontainers that will be

developed and incorporated in commercial paints, lacquers and

adhesive systems to prepare new products exhibiting self-healing

properties. This will have great economic impact on the members of

the consortium as well as on European industrial competitiveness.

According to the interviewees all dissemination activities

contribute to the project success and put the basis for further

collaboration with big industries in other projects, like MUST.

Case study 6: MULTIPROTECT: dissemination to attract big players..

The case studies show some successful initiatives. As an example, the Nano2Life NoE

wanted to attract industrial interest toward the NoE and developed the following

strategy: no industrial group was present in the consortium so as to leave room for

academics and researchers to create their own community in an emerging field of

research (nanobiotechnologies). But efforts were put on presenting information in a

condensed and professional way so to attract industry attention. Finally, the project

coordinator underlined that the organisation of industry workshops (see Case study 7)

had been very successful: “Companies have successfully participated in knowledge

transfer, information flow, and networking programmes of Nano2Life. Access and

participation to these activities has allowed companies to expand their technology

portfolio and overall researchers to establish connections and acquire know-how which

will hopefully be sustained beyond the life of this NoE”.

Page 84: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

88

Nano2Life

Industry workshops

“Last 6th May Nano2Life organised an INDUSTRY WORKSHOP. This

workshop, being one of the last events of Nano2Life, was an

international event offering a forum for the exchange of innovative

business solutions with the participation of Nano2Life top research

institutions.

28 participants from 17 companies from 5 different European

countries attended this event.

In this event, Nano2Life

1. presented to Industrial nano2Life partners the conclusions of

four scientific research programmes of the network, of key

interest for the audience, and focusing on the state of the art

developments and future trends,

2. showed the key conclusions of the 8 prospective workshops of

Nano2Life carried out at WP5,

3. created a forum for the exchange of opportunities and

cooperation between industry partners and N2L scientific

leaders, specifically focused on the on the forthcoming calls of

FP7 and on how IP should be managed by companies at a

coordinated project,

4. successfully generated a roundtable discussion on how industry

and academia in the nanobio sector should interact in the

future.

Conclusions of this WS are the number of scientific, technical and

strategic exchanges among participants and the feedback of industry

on their relationship with Nano2Life during its life-span.”

Case study 7: Nano2Life Industry Workshops (Source: http://www.nano2life.org/news.php?nid=59)

Interestingly, the importance of communication of project results to the wider society

was rated rather high by all participants as shown in Figure 42 below. Half of the large

enterprises and 55% of the SMEs considered as important the communication of their

project results to society while 72% of higher education institutes and 64% of research

organisations considered the same. However, as seen previously, the use of media

channels targeting the wider public is not seen as a priority which may reduce the

impact on the wider public. This is confirmed by the Lot 2 study previously mentioned:

the broader public barely benefited, according to the views of the coordinators, but this

view was considered rather biased due to the unawareness of the perception that these

user groups developed about the benefits of these NMP projects.

Page 85: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

89

Figure 42: Importance of communication of project results to society, (Source: FP6 NMP Participants survey, n=1181)

In addition, some projects

mentioned the use of Web2.0 (the

participative web) technologies

which may appear usual as seen

today but were mostly emerging

trends at the time of FP6.

As an example, one can mention the

BioMine wiki which was developed

by the project to offer a free, web-

based, cross-disciplinary

information source on metal

extraction, removal of metal from

effluents, microbes and elemental

cycles and how microbes affect

minerals and metals. Nevertheless,

this wiki has not been maintained

since the end of the project and alternatives have been suggested by other case studies

to have a sustainable information repository such as making use of existing repository

with higher visibility (Wikipedia, etc.) or developing transverse support actions

maintaining such repositories for the project community.

4.3.4 Strengths and weaknesses of FP6 NMP

During case studies, interviewees were questioned about the main strengths and

weaknesses of the FP6 NMP to support the development of industrial technologies in

Europe.

One of the most mentioned strengths is that the programme leaves the flexibility of the

research projects: to change, possibility to be more pertinent than what is proposed.

The programme also aims at validating the existing know-how of all partners. The

presence of demanding and supportive project officers was also mentioned as an asset

for the project to evolve toward success.

Figure 43: The BioMine wiki main page

(http://wiki.biomine.skelleftea.se/wiki/index.php/Main_Page)

Page 86: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

90

The high level of funding allocated per project compared to national schemes is well

appreciated and its added value is often mentioned as national schemes do not allow

propelling large scale international collaboration projects.

Finally the multidisciplinary approach pursued within these projects imply that these

types of FP6 projects are clearly ideal to come in contact with many interesting

organisations, as well for education, research as industrial organisations. In particular

it is an excellent manner to get to know and work with SMEs.

Regarding the weaknesses of the FP6-NMP programme, the administrative and

financial aspects being hard to manage always comes first. Another weakness raised is

the time that passes between the end of the project and the effective payment by the EC

for the project. In some cases this caused major difficulties for several SMEs

involved in the project. Still on the financial issue, the funding level being limited to

50% for SMEs (raised to 75% in FP7) is mentioned as a limiting factor to foster SME

participation.

Finally, the exploitation part of the programme could be further enhanced. There is no

follow-up on the promised exploitation and impact achievements once the project is

finalised. Several interviewees mentioned the need to have a part of the project budget

be reserved for an exploitation period at the end of the project, so to enforce the

“impacts section” of the project proposals.

Page 87: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

91

5 IMPACTS AND ACHIEVEMENTS OF THE FP6 NMP PROGRAMME There are many ways to assess the achievements and impacts accruing to participants

in collaborative R&D programmes, such as NMP. Primarily, it is vital to identify and

comprehend whether the organisations’ participation was beneficial from an overall

perspective, i.e., whether the benefits resulting from their participation outweighed the

costs of involvement and to what extent the resulting success of participants was a

consequence of participation (the issue of attribution). Secondly, it is important to

explore whether or not participation in the FP6 NMP programme allowed participants

to do something that would not have been possible in the absence of programme

funding (i.e., the issue of ‘pure additionality’), or allowed them to do something more

efficiently and effectively than before (the issue of ‘behavioural additionality’).

Moreover, goal attainment needs to be assessed, particularly the attainment of those

goals considered as primarily important by participants. The impacts associated with

these achievements should be examined, for the attainment of important goals does not

always lead to large impacts and, conversely, modest attainments in areas of lesser

importance sometimes turn out to have significant impacts on organisations. While,

finally, the factors that affected the occurrence (or not) of all project achievements and

impacts has to be investigated so as to ensure that future R&D supportive programmes

will further enhance supporting factors and eliminate all obstacles.

5.1 Costs vs. Achievements

As seen in Figure 44, there is little doubt that the benefits of participation in NMP

projects outweighed the costs, as stated by over half of the respondents (53%), while

only 14% of the surveyed population stated otherwise. Interestingly, this repartition

has remained almost the same since the FP3 Impact Assessment17 with the main

difference being a negative shift, as fewer NMP R&D participants – both academics and

industrialists – perceive the benefits as outweighing the costs (53% compared with

69% of FP3 and FP4 participants) and more stating that the benefits equalled the risks

(35% compared with 19% in 2000). This shift is modest but disturbing if it reflects a

genuine lessening of the perceived benefits of participation in European research

programmes. Moreover, benefits exceeded costs for a greater number of academics

(66%) and research institutes (59%) than industrial partners (39%), whereas costs

exceeded benefits for 21% of the industrial respondents compared to less than 10% of

academics and partners from research institutes. From SMEs point of view, one third of

them stated that the benefits of their participation outweighed the respective costs in

comparison to the statements of almost half of large enterprises partners (46%). As

arising from the case study, SMEs can be more affected by the administrative overhead

of European project participation which negatively affects their perception on the

return on investment. Indeed, while large companies have most of the time dedicated

services for the management of administrative issues, SMEs need first to learn the

process of FP6 participation and then provide the reporting without necessarily having

the adequate tools to do it.

17 European Commission, “FP5 Impact Assessment: Survey conducted as part of the Five Year Assessment of EU Research Activities (1999-2003)”, February 2005

Page 88: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

92

Figure 44: Benefits vs. Costs of participation, (Percentage of respondents, source: FP6 NMP Participants survey, n=1181)

With respect to achievements in relation to participants’ goals (Figure 45 below),

almost half of the respondents (47%) stated that their S&T achievements exceeded

their initial expectations but less than 15% stated something relevant for any other

goal (economic, social, policy, health/environment).

Figure 45: Achievement of goals - Repartition of total number of respondents per percentage of responses on the different goals (Source: FP6 NMP Participants survey,

n=1181)

A similar repartition was observed across higher education, research institutes and

2

17

37

39

32

2

5

3

2

2

12

20

8

7

8

36

46

41

42

43

34

11

8

8

11

13

2

2

2

2

-100,00% -50,00% 0,00% 50,00% 100,00%

Scientific and technological goals

Commercial and economic goals

Policy-oriented goals

Social goals

Health and environment related goals

Not applicable Achievementsmuch lessthan

expectations

Achievementsless thanexpectations

Achievementsequal toexpectations

Achievementsgreater thanexpectations

Achievementsmuch greaterthan

expectations

Page 89: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

93

industrial participants. Moreover, 43% of SMEs and 33% of large enterprises stated

that they achieved significant scientific and technological achievements with only a few

stating significant achievements in any of the other categories, similar to the overall

population.

5.2 European Additionality

The additionality of European funding relates to the probability of NMP programme

participating organisations having attained the same impacts in the absence of

Commission funding. According to the views of the NMP participating organisations,

two in every three (66%) stated that they would not have been able to perform their

NMP R&D project in case of absence of EU funding. Among those who would have

continued in the absence of EU funding (1/3 of participant population), the majority

would elaborate projects that would involve top European researchers, be of high

scientific and technical complexity, apply to a core technology area of their

organisation, have high linkages with other in-house projects, would be of long-term

R&D nature and of high strategic importance for their organisation. These projects

were seen to be important research projects and thus funding from other sources

would have been claimed. The conditions under which these participants would have

continued involved replacement of EU funds with external (54% of participants) or

internal (33%) funds, a smaller consortium (63%), on a similar (52%) to longer (30%)

time scale, with a similar set of objectives (52%) and expectations (56%) and involve

an analogous range of applications (52%).

Figure 46: Project characteristics of those FP6 NMP participants who claimed that would continue performing their NMP project in case of absence of EU funding, (Percentage of

total respondents, source: FP6 NMP Participants survey, n=395)

A separate survey was conducted amongst those organisations and individuals who

had not attained EU funding for their NMP project in order to identify the course of

Page 90: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

94

these projects. To get a more in-depth view of the respective population, 18% of them

were SMEs, 42% higher education institutes, 18% industrial organisations, 36%

research institutes and 5% other types of institutes (Figure 47). Moreover, 43% of

them submitted a proposal for a STREP project, 31% for an IP, 19% for a NoE and 7%

for SSAs and CAs.

Figure 47: Repartition of non-selected applicants per type of organisation, (Source: FP6 NMP non-successful participants survey, n= 1336)

According to the responses of the non-selected participants, 14% were encouraged by

EC officials to resubmit their proposal. Amongst the ones that resubmitted a proposal

40% got funded from EU programmes. The resubmitted proposals were very similar

to the original ones in terms of consortium composition (academics, industrial

organisations, national and foreign partners), level of expectations and timing (Figure

48).

Figure 48: Conditions under which projects have been resubmitted, (Source: FP6 NMP non-successful participants survey, n= 1336)

Larger range of applications

Greater expectations

More ambitious objectives

Longer time-scale

More international partners

More national partners

More industrial partners

More academic/research …

More partners

-100,00% -50,00% 0,00% 50,00% 100,00%

Less partners

Less academic/research partners

Less industrial partners

Less national partners

Less international partners

Shorter time-scale

Less ambitious objectives

Lower expectations

Smaller range of applications

Page 91: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

95

Among the proposals not resubmitted, the main reasons were: the lack of a new call in

line with the project idea (31%), the lack of interest from the partner’s side (26%) and

in 20% of the cases the rejection of the proposals deprived the majority of the

applicants the opportunity to achieve scientific and technical achievements (83%), to

gain commercial benefits (63%), to improve their networking with research teams

abroad (62%) and their knowledge of technological trends (54%).

Finally, the five main reasons for which these NMP proposals were not accepted,

according to the opinions of the interviewed applicants were:

1. The high competition (41% of responses)

2. The technical understanding of EU officials and reviewers (35% of responses)

3. The lack of available funding (31% of responses)

4. The clarity of project objectives (30% of responses)

5. The ambition of project goals (26% of responses)

5.3 Impacts on RTD Capabilities

5.3.1 Impacts on the Participating RTD Teams

The impacts of the NMP RTD programme projects upon the different participants were

statistically analysed18 so as to explore whether meaningful clusters of impacts

prevailed in certain categories. According to these analyses, two significant clusters

occurred: one involved impacts concerning the internal development of the R&D

participating teams involving enhancement of knowledge bases, skills, competencies

and critical mass of R&D of the individual researchers participating in the NMP projects,

and the other cluster involved the improvement of extroversion of the researchers,

including impacts upon access to complementary expertise and the formation of new

R&D partnerships and linkages with universities, research and business organisations. In

more detail, the most important impacts on the majority of participating teams were:

the enhancement of knowledge bases (80%) and skills (79%), the formation of new R&D

partnerships (77%), the improvement of access to complementary expertise (74%) and

the enhancement of joint international research actions (61%) highly confirmed also by

the case studies.

18 For further information on the statistical analysis deployed, see section 2.4

Page 92: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

96

Figure 49: Impacts on R&D team capabilities, Percentage of Respondents, (Source: FP6 NMP Participants survey, n=1181)

A majority of participants from industrial

organisations, SMEs as well as large enterprises

considered that the most significant impacts on their

teams’ R&D capabilities involved the enhancement of

their knowledge bases, skills and competences, the

improvement of R&D linkages and partnerships with

universities and research institutes and consequently

the improved access to complementary expertise.

Researchers from higher education institutes and research organisations responded

similarly to the overall population, while also considered the international research

activities they performed to be important while in the case of “enhanced career” the

responses of higher education researchers differed significantly from researchers from

research institutes: 58% of researchers from higher education institutes considered it

as a significant impact of their participation in NMP in comparison to 39% from

research institutes.

Table 15: Important and Very Important impacts on participants by type of participant, (Source: FP6 NMP Participants survey)

n=106 n=145 n=356 n=301 n=453

SMEs Large Enterprises HE RES IND

Enhanced knowledge bases 70% 74% 88% 84% 72%

Enhanced skills 70% 72% 88% 82% 71%

Enhanced career 25% 18% 58% 39% 23%

Establishment of critical mass of R&D 32% 30% 55% 48% 29%

Enhanced ability to manage R&D 37% 29% 46% 44% 36%

Improved access to complementary expertise 68% 71% 78% 78% 69%

Formation of new R&D partnerships 75% 71% 83% 84% 70%

Improved linkages with Unis 72% 81% 87% 84% 71%

Improved linkages with business 53% 43% 46% 51% 46%

Enhanced joint int research actions 53% 50% 74% 68% 49%

Extending knowledge bases

and skills of participants

together with improvement of

networking have been the

major impacts on

participating teams’ R&D

capabilities

Page 93: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

97

5.3.2 Impacts on the Participating Organisations

A similar statistical analysis was conducted on the responses of participants concerning

the impacts of the NMP projects on the organisations (as entities) that participated in

the programme. It was concluded that two main clusters of impacts were formulated:

the first cluster involved impacts on the economic sustainability of the participants,

involving the enhancement of turnover and profitability, productivity, market share,

commercial linkages and access to new markets, while the second cluster involved the

consolidation of participants through improvement of competitive position,

formation of new partnerships and networks and the enhancement of their reputation

and image.

According to the participants’ responses in the web survey, the most important impacts for at least half of the participating organisations were: the enhancement of reputation (63%), the formation of new partnerships and networks (57%) and the improved competitive position (50%) as depicted in

Figure 50 below.

Figure 50: Impacts on participating organisations, (Source: FP6 NMP Participants survey, n=1181)

Based on the more detailed analyses performed and depicted in Table 16 below, it was

identified that the majority of higher education and research institute participants

enhanced their knowledge bases and their extroversion through the conduct of

international research actions, while similar responses were provided by around half of

the industrial organisation participants.

Page 94: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

98

Table 16: Important and Very Important impacts on participating organisations by type of participant, (Source: FP6 NMP Participants survey)

More than half of the SMEs that participated in the

online survey stated the most important impacts of

their NMP projects upon their organisations were in

decreasing order of significance the enhancement of

skills (65%) and knowledge (58%), the improvement

of linkages with universities (53%) as well as the

improvement of career prospects (53%). Significantly

fewer representatives of large enterprises identified impacts gained by their

institution: 47% of the respective population stated that their NMP projects enhanced

their international research actions and 44% stated that they improved their knowledge

bases. An interesting and significant difference – from a qualitative as well as statistical

point of view – concerns the importance of increased turnover as a direct impact of

NMP projects upon the participating organisations: 12% of large enterprises

consider that their projects increased their turnover significantly and 9% that it

improved their market share and productivity, while these percentages climb up

to 23%, 20% and 18% respectively for SMEs.

Combining the information obtained from the previous analysis with the results

of this one and the interviews conducted when drafting the case studies, it is

evident that individual researchers tend to participate in FP6 NMP projects in

order to improve their knowledge and skills as well as their networking while

organisations tend to participate in NMP projects so as to improve their

competitive position (long-term goal) and also their economic and networking

aspects (short to long-term goals).

5.4 Impacts in the Social Sphere and the Broader

Environment

As recorded in the NMP web survey addressed to all NMP participating organisations,

at least half of the respondents identified as most significant impacts the production of

S&T results beyond the state-of-the-art (68%), the achievement of innovative

breakthroughs (57%) and the improved coordination between research and industry

(56%). A small fraction of the total respective population identified as significant

impacts of their NMP projects the improved confidence of consumers in N&N products,

the development of new and existing standards and policies or the increase of

n=106 n=145 n=356 n=301 n=453

SMEs Large Enterprises HE RES IND

Enhanced knowledge bases 58% 44% 69% 68% 55%

Enhanced skills 65% 37% 37% 50% 53%

Enhanced career 53% 34% 54% 56% 43%

Establishment of critical mass of R&D 23% 12% 9% 15% 19%

Enhanced ability to manage R&D 18% 9% 14% 16% 17%

Improved access to complementary expertise 22% 9% 4% 10% 18%

Formation of new R&D partnerships 35% 26% 7% 21% 32%

Improved linkages with Unis 42% 24% 15% 23% 36%

Improved linkages with business 16% 6% 8% 10% 12%

Enhanced joint int research actions 50% 47% 67% 65% 42%

It is evident from both survey and

case studies that the NMP

programme did make a significant

difference to a considerable part of

SMEs in terms of future financial

sustainability

Page 95: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

99

employment levels as depicted in Figure 51 below.

Figure 51: Impacts of FP6 NMP projects on broader environment, (Source: FP6 NMP Participants survey, n = 1181)

The majority of participants from higher education institutes have considered as

positive impacts at the socio-economic level, in accordance to anticipations, the

production of S&T results beyond the state of the art (80%), the achievement of

innovative breakthroughs (68%) and the development of the education environment

(65%). On the other hand, more than half of Industrial participants considered as

positive impacts the production of S&T results beyond the state of the art (55%) and the

coordination between research and industry (54%) while most of the representatives of

research institutes aimed at a combination of the aforementioned impacts, i.e. the

production of S&T results beyond the state of the art (75%), the achievement of

innovative breakthroughs (62%) and the improved coordination between research and

industry (58%).

Page 96: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

100

Table 17: Important and Very Important impacts on the broader environment by type of participant, (Source: FP6 NMP participants survey)

Around half of the SMES and large enterprises reported as significant impacts of their

NMP projects for the broader environment the production of S&T results beyond the

state of the art (62% and 57% respectively); the improved coordination between

research and industry (55% and 47% respectively); and the achievement of innovative

breakthroughs (53% and 46% respectively).

The impacts that were considered of low significance for the majority of NMP RTD

participants involved

policy level impacts :

o development of new and existing policies: 66%;

o development of new and existing standards: 60%;

o improved public confidence: 64%;

improved working and living conditions

o improved employment levels: 62%;

o improved preservation of the natural heritage: 76%;

o improved living conditions at large: 64%.

Interestingly, although environmental goals were ranked as third most important

among all NMP RTD participants, yet the impacts on improvement of environmental

conditions were considered not so significant by most respondents (Improved

treatment of emissions: 66%; Reduction or prevention of emissions: 58%; Improved

savings of natural resources or reduction of energy consumption: 57%).

The impacts of the NMP RTD programme projects upon the broader environment – as

recorded by the participating organisations in the respective online survey - were

statistically analysed so as to explore whether meaningful clusters of impacts prevailed

in certain categories. According to these analyses, four significant clusters occurred: the

first one involved the enhancement of the scientific excellence of the EU, by

promoting S&T results beyond the state-of-the-art and the achievement of innovative

breakthroughs; the second involved the opening of new horizons through the

n=106 n=145 n=453 n=356 n=301

SMEs Large Entr IND HE RES

Improved educational development 26% 31% 24% 65% 44%

Production of S&T results beyond state-of-the-art 62% 57% 55% 81% 75%

Achievement of innovative breakthroughs 53% 46% 46% 68% 62%

Development of new and existing markets 41% 29% 34% 16% 30%

Development of new and existing standards 24% 15% 19% 13% 11%

Development of new and existing policies 14% 12% 11% 9% 14%

Improved access to information and knowledge in the community at large 37% 33% 35% 45% 45%

Improved public understanding of N&N 24% 20% 23% 40% 34%

Improved confidence of consumers in N&N /products 16% 13% 16% 21% 19%

Improved living conditions at large 11% 9% 13% 15% 17%

Improved employment levels 16% 11% 12% 12% 10%

Reduction or prevention of emissions 21% 24% 24% 22% 23%

Improved treatment of emissions 19% 14% 19% 14% 17%

Improved savings of natural resources or reduction of energy consumption 28% 22% 26% 25% 27%

Improved preservation of the natural heritage 18% 10% 16% 9% 11%

Improved coordination between research and industry 55% 57% 54% 55% 58%

Improved non-EU cooperation 25% 15% 18% 35% 24%

Page 97: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

101

development of new and existing markets, standards and policies; the third involved

the improvement of visibility of nanotechnologies and nanoproducts by the broader

society; and the final one involved the improvement of the environment through the

reduction / prevention / treatment of emissions, improvement of preservation of

natural resources, etc.

Climate change, ageing society, sustainability, food security, energy efficiency, etc. are

already affecting economies and societies in the world; their impact is expected to

further increase within the coming decades, as discussed in the Lund declaration of July

2009, which underlined the role that European research should focus on identifying

new ways and measures to tackle them [22]. These so called Grand Challenges

represent real risks, but they can also provide opportunities for new activities, goods

and services and for moving towards the smart, green and inclusive economy.

The rationale of the FP6 NMP programme itself is aimed at addressing some of these

challenges: “The activities carried out in this area are intended to help Europe achieve a

critical mass of capacities needed to develop and exploit, especially for greater eco-

efficiency and reduction of discharges of hazardous substances to the environment,

leading-edge technologies for the knowledge-based products, services and manufacturing

processes of the years to come” [5].

As depicted in Lot 2 study that focused on the views of the NMP RTD project

coordinators, most of them did not consider (Figure 52) that NMP FP6 had a positive

impact on environmental sustainability. This position is comprehensible if one takes

into account that CO2-neutrality in materials and processes was not high on the agenda

at the beginning of the programme. Advances such as lighter materials and increased

energy efficiency of the industrial production were on the other hand estimated to be

positively affected by NMP FP6 in the long-run. These differences in appreciation may

be related to the fact that it is difficult, for a researcher working in a specialised field to

evaluate the impact of his research on grand challenges such as the global warming.

Page 98: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

102

Figure 52: FP6 contribution to societal and sustainability challenges, (Source: Ex-post evaluation of FP6 NMP: Strategic level report [8])

5.5 Impacts on the European Research Area

The aim of the European Research Area (ERA) has been to "...enable researchers,

research institutions and businesses to increasingly circulate, compete and co-operate

across borders. The aim is to give them access to a Europe-wide open space for knowledge

and technologies in which transnational synergies and complementarities are fully

exploited"19. How did NMP contribute to the formulation of ERA? As explored by the

NMP Lot 2 Study, the main reasons for which project coordinators decided to submit a

proposal under the programme were:

The possibility to co-operate with international partners was the main

motivation to apply to FP6 (by 81% of project coordinators).

The technological ambition is the second principal reason to apply, a factor

linked with the need for funds not available at national level (third block).

According to the conducted interviews with project coordinators [8], the main benefits

of their participation in the programme were: the community/network building (80%),

the improved access to international knowledge (73%), the creation of sustainable

relationships for research (67%) and the possibility to work in big consortia (65%)

that enjoy a high scientific level (58%).

As identified via the relative questions of the participants online survey conducted in the

19 http://ec.europa.eu/research/era/understanding/what/what_is_era_en.htm

Please assess the contribution of the project you coordinate(d) within NMP

FP6 to the following overall objectives of the EU

with regard to societal and sustainability aspects of European R&D

activities:

5

8

8

10

10

13

13

17

19

10

12

11

13

19

19

33

33

37

26

30

28

24

16

28

29

40

26

25

63

50

52

53

56

43

25

14

19

31

9

22

0% 25% 50% 75% 100%

Increased sustainable transport

Improved conservation and management of

natural resources

Increased sustainable consumption

Containment of climate change / increased

usage of renew able energy sources

Improved handling of threats to public health

Increased aw areness of ethical issues of

NMP related research

Improved (conditions for) gender equality

Increased dialogue w ith the public

Increased aw areness of issues of

sustainability in NMP related research

Increased sustainable production

P ercentage (n=204-209)

major

contribution

medium

contribution

minor

contribution

no contribution

Source: Austrian Institute for SME

Page 99: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

103

Lot 1 study, the programme produced strong knowledge and skills outputs apart from

significant economic outputs, through product and process innovations. Furthermore,

the participating R&D teams also enjoyed significant long-term benefits as participation

in the NMP programme allowed the participating teams to improve the extroversion of

their researchers as well as their knowledge bases and skills, and on an organisational

level they improved their competitiveness and visibility as well as their sustainable

economic progress. Finally, it should not be forgotten that these outputs and impacts

would not have been attainable in the absence of EU funding for two thirds of the NMP

population that participated in the online survey conducted during the Lot 1 study. This

was confirmed during the case studies. Most of the interviewees recognised that they

were already involved in their project field of research through other funds but that the

project would not have happened without EU funding (see FAME example - Case study

8).

The above confirms that the EU funding has produced significant scientific spill-

over effects across Europe that would not have been attainable in the absence of

EU funding and even with national or regional funding programmes.

Page 100: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

104

FAME

Structuring the ERA

The FAME project was the very first worldwide Network of

Excellence dedicated to establishing, structuring and defragmenting

the hybrids and ceramics industry in Europe. The consortium

consisted of 15 partners from higher education & research institutes.

The project achieved most importantly its principal objective of

setting up a sustainable financial and legal institute to take over the

legacy of the project. This institute known as EMMI (European

Multifunctional Materials Institute) was set up as a not-for-profit

organisation in Belgium in January 2008. It acts as a reference and

communication for researchers and industry in the field of hybrids

and ceramics and helps to roadmap European research in objectives

and direction in this field. Two patents were filed during the course of

the FAME project, 144 collaborative publications were drafted, 167

exchange visits spanning over 774 weeks were financed and further

EU projects came about as a direct result of FAME but not the amount

that was hoped for by the FAME project members. They thus hope

that through the additional influence of EMMI, they will have a

greater chance of successfully proposing new projects. Notably one of

the project proposals that the consortium did succeed was IDS-

FunMat, an international joint Doctoral School in Materials Science,

which opened in October 2009. 22 PhDs will be supported under the

programme and each one will be supported by two different

institutions from two different countries of the consortium

12 of the 15 partners are still collaborating today through EMMI and

recognise that without EC funding, the NoE would not have been

built.

Case study 8: FAME project: Structuring the ERA.

5.6 Transience of Impacts

As pointed out during the interviews conducted with participants of selected NMP

projects in the context of the Lot 1 NMP study, the particular added value of the

programme related not simply to the international dimension per se, but that the

bringing together of a variety of interested parties from different backgrounds was

important in achieving many of the impacts reported. Thus an extremely important

source of European added value was the NMP Programme’s general insistence on the

involvement of both researchers and users from across Europe within projects. As an

example, the TUNCONSTRUCT IP project focused on increasing efficiency in tunnel

construction work and was positively evaluated during its final review for its users

involvement: “the consortium includes key stakeholders and a proper involvement of end-

users in evaluating the key results which actually allows the effective translation of the

Page 101: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

105

project achievements in practical applications”. Another example, in the case of a STREP

project, is ISAMCO (Case study 9).

On the other hand, some experts highlighted that the research agenda of most of the

RTD projects was largely set by the developers. As a consequence, major conceptual

and sometimes also prototyping work had already been undertaken before the users

could have their say and the application could therefore not be considered fully

oriented to user needs. In other words, the involvement of end users was good mainly

in quantitative terms, while from a qualitative perspective it did cause a few issues.

Reasons indicated were that in many cases, the end users were too focused on the

market application and not open to accommodate technological uncertainties. This

created tensions in some of the evaluated consortia and nullified the advantage of

having end-users on board.

Page 102: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

106

ISAMCO

Users in the loop

The ISAMCO STREP project was characterised by the study of the

state-of-the-art of Ionic Polymer Metal Composites (IPMCs) and its

fabrication process, followed by the realisation of some IPMC samples

and later by the definition of their electromechanical properties. The

main objective of the work carried out was to create the specimens

which should be used first for the modelling activity and then for the

realisation of the expected devices.

The main outputs of the project involved the definition of state-of-

the-art in the area of IPMCs and the contribution - at an international

level - to this scientific area. Although direct commercial outputs

were not created as part of the ISAMCO project, one patent was

awarded and an application has been made for another one, a spin-off

company was created that aims to further explore the commercial

opportunities of ISAMCO developments and six software packages

were developed. As the partners stated, such cutting-edge

breakthrough research does not often meet the maturity of markets

and thus immediate commercial exploitations are quite hard to

accomplish.

The main success factors of the project involved the scientific

capabilities of the consortium members, their ability to cooperate

efficiently and effectively, the ambitiousness and clarity of project

objectives and the integration of end users in the partnership. The

NMP programme overall was considered as an appropriate

programme for funding such research. Furthermore, the fact that the

programme allowed for SMEs as well as a range of European partners

to be involved in the project gave a more open view while

simultaneously requiring real life applications.

Case study 9: ISAMCO: Users in the loop

Overall, stakeholders interviewed in the framework of both Lot 1 and Lot 2 studies

indicated the high additional value of the collaborative and multi-disciplinary

approach of EC-funded research as well as the critical importance of research co-

operation at the European level was highly appreciated by all NMP project

participants, particularly SMEs. Knowledge cross-fertilisation and competence

integration at the European level became the cornerstone for the significant impacts in

the scientific and technological sphere, to the benefit of the participating organisations

and the scientific and technological community at large. Frequently stressed also, was

the high value of gaining access to expertise and key actors in other countries and

contexts. Co-operation at European level was therefore considered critical (also) for an

improved alignment of the RTD outputs with the needs and requirements of users in

the European Internal Market.

Page 103: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

107

It would be of concern if the impacts of the projects and the associated European added

value were ephemeral – lasting no longer than the duration of the project. Participants’

estimates of the duration of project impacts show (depicted in Table 18 below) that

60% of the NMP survey respondents felt that the impacts of their projects on their RTD

capabilities would have a lifetime greater than three years. Approximately half of the

respondents also thought that impacts on their organisations, on project partners and

immediate users, and on the broader environment would also last for at least three

years. According to more in-depth analyses conducted on participant type level,

although the lifetime of impacts is estimated to be long (>3 years) for the majority of

researchers of higher education institutes and research organisations, the lifetime of

impacts for the majority of researchers working in industrial firms, SMEs and large

enterprises is considered moderate to long (2-3 years or more).

Very

long

lifetime

(greater

than 5

years)

Long

lifetime

(3-5

years)

Modera

te

lifetime

(2-3

years)

Short

lifetime

(1-2

years)

Very

short

lifetime

(less

than 1

year)

Not

applica

ble

No

respons

es

Impacts on

participating teams 25% 34% 24% 10% 3% 4% 6

Impacts on

participating

organisation

17% 28% 30% 13% 5% 7% 7

Impacts on the broader

environment 24% 22% 17% 8% 5% 23% 10

Table 18: The Transience of NMP RTD Project Impacts, (Source: FP6 NMP Participants survey, n=1181)

5.7 Success Factors and Obstacles

A critical aspect of the work elaborated under the Lot 1 study was the identification of

factors and variables that affected the course of the projects, according to the

participants’ own views. According to the relevant literature, a project’s progress and

overall performance can be affected by a range of factors, both endogenous and

exogenous to the project. Internal project parameters involve issues such as the quality

of a project’s leadership, the technical capabilities of the partners, the clarity of project

goals and the level of communication between partners. Changes in the external

environment can affect progress or commercial success of a project can, for example,

drastically affect the relevance and utility of project goals and outcomes and undermine

eventual success. These issues were examined in the context of the Lot 1 study, during

both the online survey as well as the personal interviews with selected projects, and

the main results are presented hereunder.

Although the majority of participants in FP6 NMP attained their overall goals and felt

that the impacts and benefits from their participation outweighed the respective costs,

as described in earlier sections of this report, there were significant variations in the

factors that were related to the performance of the projects.

Page 104: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

108

In further detail, the conduct of the Case Studies demonstrated that success of projects

relied primarily on the consortium capabilities, with unsurprisingly the scientific

and technological capability of the individual partners being the most vital criterion.

But this is not enough to move on the road to success. To be turned into a successful

consortium, it was equally underlined that there was a need of complementarity of

goals amongst the consortium. But the complementarity of project goals had to be in

line with the strategic goals of the participating organisations. Indeed several projects

mentioned that overlapping goals did lead to IPR issues, which at minimum slowed

down the project but in some cases even led to project failure in achieving their goals,

especially when it came to commercial exploitation. Then to put the consortium in

efficient motion, the managerial capacity of the coordinators and the capability of

the consortium to set-up reactive and trustworthy communication channels were

highly ranked as success criteria. The “friendly atmosphere” wording was often used to

describe the one prevailing in successful projects. This “friendly atmosphere” concept

indeed refers to the trust and confidence cultivated within these consortia and their

capability to work as a team even when facing difficulties. A parameter that often

influenced the establishment and proper operation of communication channels has

been the prior acquaintance of the partners themselves and also the dedication of a

team of people on the project management of the project, a factor linearly related to the

size of the consortium (the larger the consortium the more people required on the

management of the project). Another important contributor to a project’s success has

been the previous experience of partners with EU projects. This allowed projects to

start faster, limiting the loss of time often occurring at project start.

Having ambitious and clear project objectives was also said to influence the project’s

success. Ambition of objectives is a motivating factor for partners but it also attracts

attention from stakeholders external to the project, thus easing dissemination and

valorisation of project results during and most importantly after the project’s lifecycle.

Clarity of project objectives encourages partners to focus on the key aspects of the

project, not diverting onto unnecessary activities. An additional factor more rarely

mentioned as contributing to the adequate focus of the project was the presence of end

users as they allow better specification of the needs of the target group(s) to be

achieved. Nevertheless, these latest arguments were not always seen as positive

contributors to project success. In more research-oriented projects, presence of end

users is underlined as not adding anything to the R&D process. This holds especially

true when end users were SMEs: they are much more focused on their business issues

and do not manage to align their own objectives with the project’s ones. This can

contribute to partial failure of the project. Also, too ambitious project objectives may

rapidly appear to be (partially) unachievable which in turn may also lead to project

failure.

Finally, a success factor mentioned during the interviews for the Case Studies of the Lot

1 study has been the ESIC seminars organised by the NMP officers. Despite the

issues that were observed in their performance, timing in relation to the project

lifecycles, etc., it was acknowledged by many project participants that it was a helpful

tool and assisted them in considering the possibilities and opportunities and means

they could utilise for the promotion and valorisation of their project results.

Page 105: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

109

All the above are also represented in the results of the web-survey where the most

positive influential factors of the NMP projects, according to the views of the majority

of the participants, were: the scientific and technical competencies of consortium

members (93%); the availability of prior knowledge, technical inputs and state-of-the-art

technological equipment (86%) as well as resources (78%); the ambitiousness (85%),

clarity (84%) and complementarity (79%) of project goals; the project management

skills of the coordinator (79%) and last but not least the effective communication

between partners (78%). According to a more in-depth and advanced statistical analysis

conducted on the obtained responses from the web-survey, six main groups of success

factors were identified:

1. Partners scientific and managerial competencies

2. The proper and adequate formulation of project idea and proposal (setting the

overall framework)

3. The proper assistance and support by the EC

4. The prior knowledge of EU project procedures

5. The exploitation prospects and

6. The integration of market aspects into the RTD projects.

Figure 53 : Success factors and obstacles (Source: FP6 NMP Participants survey, n=1181)

According to the in-depth analysis conducted on the data, the most important success

factors for the vast majority of the participants from higher education and research

institutes were the same as the overall ones; however, among them different weights

were attributed to the significance of certain factors. In more detail, 80% of

representatives from research institutes considered the previous experience of FPs as a

success factor of their project in comparison to 66% of higher education institutes

1

3

11

9

1

1

2

1

13

8

11

8

1

1

1

2

19

5

13

4

4

4

26

0

0

0

1

0

2

0

1

1

1

0

0

1

0

1

0

1

0

1

2

1

2

2

0

2

1

3

1

4

4

5

6

5

2

2

5

3

7

2

4

2

5

6

5

8

6

5

20

20

33

6

13

15

15

47

35

22

17

14

12

13

12

34

18

26

35

29

46

49

41

49

41

40

44

35

44

44

25

39

40

45

52

51

49

51

29

47

42

38

40

30

14

53

25

26

14

48

45

35

34

8

12

26

27

27

32

29

34

13

28

13

16

21

10

3

-100,00% -50,00% 0,00% 50,00% 100,00%

Internal S&T competencies of your team

Managerial Capabilities of team

Previous experience with FPs

Exploitation capabilities of your organisation

S&T competencies of partners

PM skills of coordinator

Complementarity of partner goals

The effective communication between partners

The Intellectual Property Right arrangements between partners

The exploitation capabilities of partners

The integration of end users in the partnership

The knowledge of industrial processes (source, design, plan, …

The clarity of project objectives, management of the scope …

The ambition of project goals

The availability of adequate resources (manpower, budget and …

The availability of prior knowledge or technical inputs

The availability of additional funding from partners or external …

The availability of state-of-the-art technological equipment

The knowledge of markets (identified existing needs, …

The administrative capabilities of EU officials

The technical understanding of EU officials and EU reviewers

The contractual obligations with the European Commission

The changes in regulatory environment / legislation policies / …

Not applicable Very negative influence Negative influence No or neutral influence Positive influence Very positive influence

Page 106: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

110

while the integration of end users in partnerships was ranked as a significant success

factor by more research institutes than higher education establishments (73% and

56% respectively).

With respect to SMEs and large enterprises, both populations mentioned the same

success factors with the exception of the importance of the project management skills

of the coordinator: 81% of SMEs considered them of vital importance for the successful

elaboration of their project in comparison to 51% of representatives of large

enterprises.

From an obstacles perspective, information was mainly derived from the Case Study

interviews rather than the web-survey conducted amongst FP6 NMP participants.

According to the conducted interviews, beside the management costs and the

contractual obligations of the project still mentioned has a high obstacle to

participation, two major obstacles are mentioned: firstly the size of the consortium. A

total of 15 to 20 partners appearing to be a maximum for RTD projects. In networking

projects, the maximum manageable size would rather be in the order of 30 partners in

the presence of a capable coordinator. The second major obstacle identified has been

the inadequate management of IPR. This issue can be illustrated by two case studies,

the first one related to a badly managed IPR and the second one showing the same case

of a patent brought into a project but with IPR managed a different way, supporting the

project success:

Case 1: a partner brings a key

patented knowledge in the project,

as a basis of further development in

the project. This partner does not

notify the consortium that a licence to exploit this

patent has also been granted to a third company

prior the project. At the end of the project, it

appeared that the consortium did not manage to

achieve the expected goals while they were

achieved and exploited in a lower amount of time

by the third party.

Case 2: a university brings a patented technology to the project which realises

proofs of concept based on this patent. At the end of the project, a spin-off is created from the university with privileged access to the patent and the ability to exploit it thanks to the work achieved in the project.

Table 19: Examples of failure and success due to IPR management.

It should be noted that in comparison with FP6, FP7 now requests IPR arrangements to

be agreed and signed before the official launch of the project. Whether this alteration

has improved the issues identified and faced in FP6 needs to be further reviewed.

Page 107: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

111

6 CONCLUSIONS AND RECOMMENDATIONS

6.1 Introduction

As mentioned in Section 1.4, the current study aims to draw conclusions on the

effectiveness, efficiency and efficacy of the NMP programme in achieving its objectives

and accordingly to make recommendations on ways to enhance the added value and

spill-over effects of future NMP research activities. In particular, the study addressed

the following questions:

Did the programme attract and select the right sort of projects to achieve its

objectives?

Have the projects been monitored, reviewed, and steered in such a manner as to

contribute optimally to the objectives of the programme?

Can measures be taken in order to enhance the performance, success, and impact

of the projects?

The high level objectives of FP6 have been to “...have a structuring effect on research and

technological development in Europe, including the Member States, associated candidate

countries and other associated countries and make a significant contribution to the

establishment of the Europe Research Area and to innovation” [23] and contribute to the

formulation and establishment of the European Research Area. The FP6 NMP

programme has been a real success and a structuring programme for Industrial

Technologies in Europe as it funded a mix of projects implemented by distinct

organisations from Europe as well as all over the world that promoted the scientific

knowledge base of Europe and paved the way for innovations to occur not only during

the programming period (2002-2006) but also afterwards.

Nevertheless, the programme did face several issues that affected the course of the

projects and the programme overall. These involved not only implementation

processes and procedures but also policy level issues that affected the diffusion of NMP

impacts on socio-economic levels. The confrontation of these issues is particularly

important currently in light of the current economic environment that demands rapid,

flexible and targeted research activities aiming at producing innovations with social

and economic benefits on a global scale. The recommendations formulated aim thus to

improve not only the implementation aspects of the programme but also its projects

selection processes so as to reflect the new impetuses put forward by the proposed

Innovation Union [24].

The analysis performed on the outputs, outcomes, results and impacts of FP6 NMP

programme on its participants, their organisations and the overall socio-economic level

has taken into consideration two unique factors that affected various aspects of the

programme:

1) The first one was the initiation of NMP as an individual RTD funding programme.

Although NMP activities have been long supported by the EC via various programmes

(such as Quality of Life in FP5), yet in FP6 it was decided for the first that NMP activities

would be pursued individually with their own programme.

Page 108: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

112

2) The second important aspect of FP6 NMP programme has been the historic moment

that affected the course of the European Union as a whole. In further detail, in 2004 the

largest enlargement of the European Union – up to date – was performed growing the

EU from 15 to 25 Member States, right in the middle of the programming period.

6.2 Conclusions on the appropriateness of the project mix

As depicted in all the analyses performed during the evaluation of the FP6 NMP

programme, the programme achieved the attraction of the best European research and

academic institutions to participate, some of the world’s top R&D spending private

companies and a number of innovative SMEs. A mix of projects was pursued, via its five

pillars of the programme, allowing for applied as well as basic research to take place all

over Europe. The majority of projects would not have happened and would not have

produced their outputs and impacts in the absence of EU funding, although the figures

have shown a relevant decrease since the launch of FP funded research activities.

Although at this phase this issue is not yet alarming, it should be taken into

consideration during the design of future NMP supporting programme.

In emerging areas, such as the nano-biotechnologies, Integrated Projects (IPs) and

Networks of Excellence (NoEs) were built at the European level; some of the NoEs are

still vibrant today as those who were students at the project time and participated in

the project events, are now reaching researcher positions in universities and are still

using the networks they built during their projects. The specific calls for SMEs allowed

many small and medium sized enterprises from all over Europe to collaborate with

highly reputable research establishments, to receive new knowledge and expand their

skills and knowledge basis while providing real-life perspectives and problems to

research theoreticians. This has contributed to the confrontation of the ‘European

paradox’. a term first introduced in the ‘Green Paper on Innovation’ of 1995 [12], which

claims that while the EU is very good in producing knowledge at the universities and

research institutes, it is extremely poor in connecting this knowledge with value

creation. The NMP programme also enhanced international cooperation as it

managed to achieve about 1/3 of the respective overall FP6 objectives.

Nevertheless, although participation from third countries – on total level – was rather

high, the programme still did not manage to attract the best international research

organisations while among the top international R&D investing organisations, NMP

attracted only a small percentage of them. The necessity of international collaborations

with prestigious international institutions and also enterprises is understandable in

light of the rapidness of evolution of the area and the progression of other countries,

such as the US and India, in capitalising the knowledge obtained from research projects.

Thus, further improvement is required so as to ensure that the top European and

international companies collaborate in the coming NMP projects and produce results

that (eventually) will have commercial applications.

As aforementioned, one of the parameters that have affected the course of FP6 NMP

programme has been the historical enlargement the EU from 15 to 25 Member States,

right in the middle of the programming period in 2004. The analysis made on the

participants of the FP6 NMP programme clearly shows two phases in their

participation: prior to the enlargement, a high number of organisations from candidate

Page 109: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

113

countries participated in the programme but with a low EC funding in comparison to

the other participants. After enlargement in 2004, the proportion of organisations

from new Member States decreased (from 22% in 2003, the percentage fell to 6%

in 2007) but at the same time, their share of the EC contribution increased to a

level comparable to other EU countries (from 16% in 2003 to 28% in 2007).

Overall, the NMP Programme attracted most of the top EU research institutions and

companies (i.e. R&D performing institutions and companies with headquarters in

the EU). Without the participation of these organisations, the NMP programme could

not have hoped to make a significant impact on either the quality of research

undertaken or the subsequent diffusion, adoption and commercial exploitation of

research outputs and results. The involvement of leading non-EU research

organisations (i.e. those perhaps performing research within the EU as well as

elsewhere, but with headquarters located outside of the EU) was less marked, largely

due to issues such as lack of conformity between EU and third country grant

agreements, but the NMP Programme still managed to allocate 100 million Euros, or

35% of the overall FP6 target (285 million in total), to international cooperation

between European and non-EU organisations.

6.3 Conclusions on outputs, outcomes and results achieved

NMP in the dawn of the new programme period will have to face an exquisite challenge:

to significantly and effectively contribute to the transformation of Europe into an

Innovation Union [24]. According to the analyses conducted and presented during this

study, it has been identified that the NMP Programme during FP6 achieved the

creation of a sound scientific basis around the EU and promoted industry’s

competitiveness through the support of innovation. The Programme produced

strong S&T outputs, enhanced the knowledge bases and skills of participants and

improved the operational processes of the organisations, while also producing

significant economic outputs, through product and process innovations. In particular

from an innovation / economic exploitation point of view, the participants of the

programme did achieve important benefits as ¼ of the population stated that they have

realised or expect to realise in the near future commercial returns through the

exploitation of their FP6 NMP project results and around half of them expect

commercial returns of more than €100.000 on an annual basis.

Factors that have positively impacted FP6 NMP projects should be preserved, such as

the flexibility that allowed projects to change direction in response to internal or

external events and to be more pertinent than what were proposed to be at the time of

proposal preparation. Another significant positive policy angle was the high level of

funding allocated per project, compared to national schemes that usually do not allow

propelling large scale international collaboration research projects. This aspect is

considered of primary importance in the current economic environment where

Member States, significantly affected by the economic recession in recent years may not

be able to maintain or expand funding levels for research projects. On the other hand,

although the level of funding is considered a significant positive factor, the efficiency of

funding and the returns of investment (cost-benefit of expenditure) are rather difficult

to estimate especially in comparison with national programmes. Overall, the

Page 110: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

114

multidisciplinary approach pursued within the FP6 NMP projects imply that these

types of projects are promoting the exchange of views and opinions among a range of

interesting organisations and in particular has been an excellent manner to get to know

and work with SMEs.

Nevertheless, several issues have impeded the course of NMP projects which should be

addressed in the next programming period. These issues were mainly related to the

implementation of the programme while at policy level a number of improvements can

also be accomplished. First of all, more transparent consultation with stakeholder

communities and a more explicit ‘programme logic’ can produce a programme that is

more robust as well as adjusted to market and business needs and requirements. Such

an ameliorated, more transparent and, probably also, more detailed (including from a

quantitative point of view) design can enable the programme to act more effectively as

a ‘focusing device’, giving signals and incentives to link RTD activities and outcomes to

needs and opportunities. Such a transformation can allow the programme to use its

added value in a more strategic way, setting European agenda that will attract the

different regional and national actors as well as international reputable institutions.

Also, the design of the programme could also look not only at the

accomplishment and progressions of other leading countries, such as the US and

India, but also identify the means they use for the promotion and exploitation of

this research field’s outcomes and outputs. As stated in the Innovation Union

document [24], low private investments in Europe in the field of industrial

technologies, full of economic potential, as well as a weak presence of European

companies in areas like biotechnology, semiconductors and software and hardware, in

which R&D is a critical competitiveness factor, are significant issues that need to be

dealt with. European companies tend to concentrate their investments in medium-high

R&D intensity sectors (e.g. aerospace and defence, chemicals, automobiles), while the

US puts most efforts in high R&D intensity sectors and a large part of the R&D

investment gap between the EU and the US is explained by the smaller number of

young innovative companies in high R&D-intensity sectors.

On the societal and environmental policy levels, the impacts of FP6 NMP were not

as significant as expected. FP6 NMP contribution to sustainability, health, ethical and

gender issues or management of natural resources was quite or very reduced. This is

an area that needs attention during the design of future NMP programmes, since "...[Key

Enabling Technologies] will be at the forefront of managing the shift to a low carbon,

knowledge-based economy" [25].

6.4 Conclusions on the efficacy of implementation and

monitoring procedures

At the implementation level, as depicted from the analyses conducted in the context of

this study, in many cases the EC played a crucially supportive role and the

demanding Project Officers were appreciated. However, a number of issues were

recorded that should be further improved in the forthcoming years.

One of the commonest topics identified as an issue during the personal interviews with

project participants involved changes in Project Officer. Such changes affected the

Page 111: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

115

course of the projects and caused unnecessary delays. Another issue, discussed quite

often during the meetings of the evaluation team with the Expert Advisory Board, was

the time lag between idea formulation and contract signature date (i.e. “time to

contract”). NMP is an area with significant and continuous changes and evolutions on a

global scale. So within the observed average time to contract of 1,25 years (455 days),

the external context of a project evolves and may have significant implications for

project expectations. Project objectives, for example, need to become much more

adventurous when developments elsewhere shift technological frontiers rapidly

forward). Changes need to be made to improve flexibility, e.g. via the adoption of open,

continuous call project selection processes, or via procedures that allow projects to

change direction more easily, to adjust levels of funding, to make adjustments to

expected time horizons etc. Changes are also needed to improve the time to contract

and to enhance the attractiveness of the programme to non-EU based organisations

that may be more accustomed to fast-track selection processes.

Another important issue that requires further attention is the administrative burden

imposed on participants, regarded as ‘heavy’ in FP6 and still ponderous in FP7, though

some progress had been made. This parameter was also identified by a number of

SMEs as an important obstacle to their participation in the FP6 NMP programme.

Overly bureaucratic procedures constitute an obstacle to innovation and prevent

potential innovators from presenting proposals. R&D policies around Europe are still

fragmented, despite the catalytic role played by EU initiatives, and further coordination

amongst different policy areas is also necessary.

Overall, the success of the FP6 NMP in terms of building capacity and creating the

basis for further research and innovation in Europe is appreciated by all

participants. But there is still a potential problem in terms of the transformation

from research to innovation. Projects with exploitable results frequently continue

their research through other research-oriented projects rather than taking the risk of

going to the market. Often this is perfectly justifiable in terms of the stage of maturity

that projects have reached along development trajectories, with the outputs of any

single research project often needing to be combined with the results of other projects

– past and future – before further progress can be made in terms of realising innovation

potential. But on occasion there was the suspicion that some NMP participants were

reluctant to emerge from the comfort zone that public funding provides, preferring to

seek additional funding for further research rather than pushing ahead along the

innovative trail. This phenomenon can be attributed to some of the factors identified in

the present study: often researchers do not look for other sources of finance than

public ones as they do not always have the necessary (entrepreneurial) skills to do so.

This is demonstrated by the data on NMP participation: in many cases, research

projects were built on previous experiences funded by the public sector and the

intention is to go further, with public funding again, usually via national or regional

programmes. Private funding sources appear to be a (very) secondary option. This

should be further investigated in view of the renewed Europe 2020 Innovation Union

flagship objective of 3% of EU GDP to be devoted to Research and Innovation, with a

large proportion to come from private investments.

Another point that should be further reviewed is the impact of the creation of

Page 112: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

116

Technology Transfer offices across many European universities on the level of

registered patents. According to statements of participants recorded during the

conducted interviews, these offices assist researchers in obtaining patents, trademarks,

copyrights, etc. and thus release the burden from researchers who usually are less

willing to invest time on the registration of their obtained knowledge. To this end, the

recent Communication of the EC on “Improving knowledge transfer between research

institutions and industry across Europe: embracing open innovation” [26] is reinforcing

this objective through provision of assistance for the enhancement of coordination

between the Member States and the Community to act together in a mutually

reinforcing way in order to overcome some of the existing obstacles in promoting the

trans-national dimension of knowledge transfer between public research institutions

and third parties, including industry and civil society organisations.

Additional incentives may thus be needed to reduce levels of risk-averseness in

the EU.

Page 113: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

117

Overall, and as identified from all the analyses conducted, the vast majority of

participants underlined that the FP6 NMP programme has been a strategic tool to

elaborate novel RTD activities that would not have been able to have been

performed otherwise. The main strength of the programme has been its unique

capability to bring together the best European, and in many cases international,

research groups producing impacts on participant’s visibility and trans-European

networking through adequate – in most cases although with some issues - funding.

Among the weaknesses of the NMP programme, in accordance to the whole of FP6

programme, the administrative complexity and the delays in payments were frequently

mentioned. The low success rate in project proposal selection although mentioned as a

weakness yet it was stated that this highly selective process creates prestige for the

successful proposers. In the particular case of SMEs, being selected in an EU project can

even be a passport to access further national funding or opening to new international

markets.

The programme also contributed to research efforts needed to tackle the so-called

Grand Challenges ("global warming, tightening supplies of energy, water and food,

ageing societies, public health, pandemics and security", according to the Lund

Declaration). In spite of the absence of a number of significant or concrete impacts of

the NMP programme on society and environment (although highly rated as important

by a share of participants, yet the impacts produced were considered significant by

around one fifth of the total population), the EU research policy has facilitated trans-

border initiatives and pushed for a certain harmonisation of strategic objectives,

moving towards a real European Research Area. Furthermore, as pointed by members

of the Expert Advisory Group, discoveries occurring during EU funded projects may

take a while before being able to be provided as innovations to society or applied to the

environment.

Topics like the reinforcement of intellectual property rights at the European level or

standardisation, necessary to the internal market, have a strong research component

that goes beyond national borders, requiring European actions. NMP has contributed to

their evolution and has been affected also by the national frameworks. Coordination at

the EU level of regulations and standardisations is expected to have a significant impact

on the course of the area and an appropriate ‘policy mix’ will be required.

Finally, the NMP programme needs to address directly the ‘European paradox’

phenomenon. As identified in the FP8 Ex-ante impact assessment study [10], the

European gap between research and innovation partly is linked with deficient "soft" or

"generic" skills (e.g. entrepreneurship) of people involved in research. European

researchers often lack a clear vision about market needs and mechanisms. NMP in FP6

played a relevant role to make researchers aware about the necessary

multidisciplinarity, networking, and other generic skills but further actions need to be

established so as to reinforce Europe’s intention to foster entrepreneurship, especially

in the light of the current environment that requires from all Member States and all

people to participate efficiently and willingly in the economic prosperity of their areas.

Page 114: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

118

Page 115: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

119

6.5 Recommendations

The recommendations of this report have emerged from the review of all collected

primary and secondary information, complemented by discussions held between the

evaluation team, the Expert Advisory Board and the European Commission, and

informed by ideas and proposals generated during the conduct of the case studies. They

have been classified into three distinct and interlinked areas, according to the wider

policy or implementation level they address. In some cases they address issues outside

the boundaries of the Programme, but they are included because of their relevance to

the future development of the Programme.

6.5.1 Recommendations at NMP Policy level

6.5.1.1 Support for SMEs

Rationale

While performing better than the overall FP6 average, the FP6 NMP still

underperformed in respect with the SMEs integration objectives. The relative success

of FP6 NMP may be explained by the launch of specific calls such as “IP for SMEs”

favouring the SME leadership in the projects. These projects were appreciated by the

SMEs but at the same time the project performances were reduced by the very large

size of consortia.

Proposition

The EC should retain calls for Integrated Projects specifically involving SMEs (like in

FP6 ‘IP for SMEs’ but it should encourage smaller consortia (with a maximum of 15

partners), in-line with the developments in FP7. Particular attention should be paid

also to the “Research for the benefit of SMEs” scheme of FP7, which focuses on smaller

scale projects dealing with the outsourcing of research activities to a reduced number

of SMEs (approximately 2 to 3). These schemes may be complementary as the IP for

SMEs intends to be more representative of an activity sector.

Possible drawbacks

Reducing the size of the consortia may also reduce the representativeness of the

project in a given sector which could in-turn reduce capabilities to impose solutions on

the market or influence consensus based processes such as standardisation.

6.5.1.2 End-user participation

Rationale

The role of end-users has proved valuable in projects where commercial impacts are

expected but not significantly in other types of projects. As shown in the current

evaluation report, this appeared to be the case mostly for Pillars 3 (New Production

Processes and Devices) and 4 (Integration of nanotechnologies, new materials, and new

production technologies for improved security and quality of life) while in pillars

having a higher focus on research (i.e., pillars 1 and 2), end-users present in the

Page 116: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

120

consortium felt frustrated when the research did not go on the way initially anticipated.

Proposition

It is recommended to encourage end-user participation in project planning to have a

short to medium term market impact. In these projects, it is advisable to have

exploitation experts that would rather look transversally at potential exploitation

paths. In projects with longer market potential, mechanisms providing participants

with access to comprehensive market outlook assessments would help them assess

potential exploitation pathways. In any case, it is encouraged to associate end-users in

the prototyping and demonstration phases of the consortium, do they belong to the

consortium or not and end-users/partners of the consortium participating throughout

the course of the project so as to ensure market orientation throughout the lifecycle of

the project.

For projects being more research oriented and in which commercial opportunities are

less identifiable at the project start, a continuous outlook of exploitation paths together

with technology transfer activities could be run as identified tasks within the projects.

Such activities could be closely monitored within the ESIC activity (see

Recommendation 6.5.1.4)

Possible drawbacks

As also discussed in the Conclusions section, European research needs a shift towards

identification and contribution to real problems which are usually identified and

discussed during a project’s lifetime by the end-users that are mostly SMEs or private

companies. A possible drawback would be the marginalisation of these types of

participants to such “partner positions” within consortia and their contribution to be

more minimal and only when required. Additionally, such a marginalisation will hinder

the open communication and cooperation between different types of participants,

significantly impacting upon multi-disciplinarity, a crucial aspect of NMP.

6.5.1.3 Standards and regulation

Rationale

The NMP technologies usage requires close links to the standards and regulation area.

The standards and regulation area prepares a market for favouring a specific

technology. As an example, defining more stringent or more complete

specification on one standard can help increasing the quality of a product while

favouring the companies possessing the technology to reach such specifications.

Some projects related to human beings failed to achieve the expected impacts

due to the lack of anticipation of the necessity of length and complex clinical

trials. Other projects, having considered the issue of clinical trials from the start

and having planned how they would be run after project end, from the proposal

stage, appeared to be more successful. Presence of such plans in the proposal

could be an evaluation criterion for projects requiring clinical trials before

market deployment. It has to be noted that clinical trial time-frames and

complexity are not compatible with existing project instruments and would

required new instruments if they would be supported in the next programme.

Page 117: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

121

The potential adverse effects of NMP technologies on health have to be

addressed. While a number of projects appeared to manage this aspect

successfully, some others were rather simply blocked by the difficult adoption

of nanotechnologies due to the fears inspired by this technology.

Proposition

Contribution to standardisation activities should be encouraged and performed in a

coordinated way: RTD project development and standards creation and maintenance

(which require high industry support) are two parallel tracks with different timeframes

and objectives. It should not be expected that projects would create standards as such

but rather that they will feed into the standardisation process with their results. A

coordination of the standardisation activities conducted within NMP, possibly via

Support Actions could help maintain close link between projects and standards

development, even beyond the normal lifetime of project's. This should be attempted at

an international level and in collaboration with ETPs.

Such coordinated actions could not only look at contribution to standards but could

also support projects in their use of standards, enlarged to the compliance with the

regulatory framework.

Possible drawbacks

The issue of standards and regulations goes beyond the auspices of NMP programme

or even FP. It requires coordination and cooperation of Member States so as to ensure

agreements are reach on standardisation mandates for European Standardisation

Organisations.

6.5.1.4 Visibility

DISSEMINATION

Rationale

Visibility and prestige gained during participation in EU funded projects are highly

appreciated by the participants. It appears to be a key outcome of FP6 NMP in

supporting organisations competitiveness, especially for SMEs that make use of this

international reference to raise further funds.

Proposition

Apart from international peer-reviewed publications, the focus of dissemination efforts

should include activities and other material aimed at the ‘first circle’ of support for

SMEs (development agencies, local customers and investors). The foreseen visibility

actions should be as clearly as possible described in the proposal documentation and

justified by relevant market analysis. As mentioned also earlier in the report, a

drawback of the EU research community is the lack of thorough market research prior

to the conduct of close-to-market research projects.

Possible drawbacks

Distortion of competition: although it is important for organisations participating in EU

funded projects to promote their research results and diffuse the knowledge generated

Page 118: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

122

so as to accomplish spill-over effects, funding of promotional activities to organisations

with commercial applications could lead to distortion of competition.

Another drawback could be the general description of the promotional activities at the

proposal phase or even the description of more activities that during the project

elaboration – with necessary amendments – will be transformed into research

activities. Perhaps, it would be also appropriate to fund promotional activities near or

after the project end to only those projects with significant innovations.

Finally, a drawback concerns the need for confidentiality especially when marketable

innovations are concerned: industrial organisations wish to register their innovations

prior to their communication to the public while academics and researchers wish to

promote their research work.

PUBLIC DIALOGUE

Rationale

Increase of public understanding of nanotechnologies, nanosciences and nanoproducts

is essential for the furthering of the area at the European level.

Proposition

The projects should not only aim for publications, conferences, etc. but also to plan

activities that open up the research results to the local, regional, national or even

international communities, thus informing local/regional schools, universities,

communities etc. about developments of interest. The teams should organise events so

as to inform local/regional schools, universities, communities, etc., and also use the

public media (radio, television, or even social media such as Facebook, Twitter, etc.)

about the outcomes of their projects. Two-way discussions should be encouraged so to

capture and take into account any counter argument that may be expressed. A number

of spill-over effects are expected to be gained from such activities namely increase of

interest of young people especially towards science and research, improvement of

public understanding, etc.

Possible drawbacks

Again the issue of need for discretion posed by industrial partners wishing to

commercially exploit the results of their NMP project could pose obstacles. Another

potential drawback can be the ethical concerns of people posed by the integration of

new technologies and innovations in our everyday lives. Such concerns will need to be

addressed by the projects with the assistance of relevant experts.

MARKET EXPLOITATION SEMINARS

Rationale

The majority of FP6 participants that participated in Market Exploitation Seminars

(ESIC seminars in FP7) considered them as a positive asset provided by the EC. The

recommendations being made mainly involved the timing of these seminars (too early

or too late in projects lifecycle).

Proposition

Page 119: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

123

In order to ensure the timely consideration of market opportunities, Market

Exploitation Seminars should be held from project start and organised according to the

needs of different project consortia. In general, projects should include marketing

plans within their proposals to ensure that some thought is given to market needs in

the medium-term.

Possible drawbacks

Market Exploitation Seminars, as organised until now, have been quite generic and

were mainly meaningful for organisations not aware or accustomed to the promotion

of their products. Perhaps different seminars should be organised for different types of

participants (industry, academics, etc.) but such a differentiation will impose additional

burden upon Project Officers.

6.5.2 Recommendations at the implementation level

6.5.2.1 Project selection

LINKING PROJECT SELECTION AND EVALUATION

Rationale

Currently, a complete separation between the proposal evaluation and the project

review exists whereas knowing the specific reasons for which a project has been

selected could help in putting the focus on the specific achievements to be looked at

during its execution.

Proposition

The EC could tie project evaluation (project monitoring and review) closer to the

proposal selection process. For example:

Whenever feasible, proposal evaluators could be selected as external project

reviewers

The Proposal Evaluation Report (ESR) could be made available to the project

reviewers so as to allow reviewers to anticipate a list of the expectations and

issues to be looked at during the project execution.

Possible drawbacks

This proposition could involve some additional workload for EC Project Officers as they

should organise and monitor this process of transmission of information between

proposal evaluators and project reviewers.

TIME TO CONTRACT

Rationale

A continuous plea from FP participants over the years is the minimisation of the time to

contract. Especially, in the case of NMP it is of vital importance to choose for funding

and implement projects in the shortest possible time as during partners interviews

there were cases recorded where partners developed a novel output only a little later

than rivals in the US or another competing country.

Page 120: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

124

Proposition

In parallel to the usual evaluation process, a new mechanism can be designed and

implemented which will focus on close to the market novel project ideas. This

mechanism will allow project ideas t o be evaluated whenever they are received by

external independent experts.

Possible drawbacks

Such changes require a holistic FP approach that extends NMP authorisation.

6.5.2.2 SCIENTIFIC AND TECHNICAL MONITORING

Rationale

A number of researchers felt frustrated of the low level of inspection of the produced

technical deliverables. They have been asking for technical analysis of produced work

with an outlook on the achieved progress beyond the state of the art by peer

scientific/technological experts.

Proposition

Having regular inspections by external experts (a periodicity of one year between

reviews, seems to be a good compromise) even for small projects would permit to react

more rapidly on problems occurring on the project plan. Also, such inspections will

allow Project Officers that take over projects during their lifetime to identify quickly

issues with these projects, their overall scientific and technical progress and thus

minimise adjustment time.

Possible drawbacks

Such a proposition may require further efforts from the Project Officer’s side in

organising these mid-term reviews.

6.5.2.3 Monitoring the outputs

Rationale

Projects outputs related data have been poorly recorded within FP6. Information is not

consistently presented within database and project reports are not always archived as

they should be. Searching for output data at the ex-post evaluation stage is not the

optimum solution as such an ex-post evaluation should rather focus on longer term

impacts. Furthermore, such data will allow the proper and timely monitoring.

Proposition

Recording project outputs in a centralised system should be an obligation for

beneficiaries and provided content could be reviewed during the technical review of

the project. Provided data should contain enough information to evaluate its quality

level and clearly identify the project role.

As an example, recording of publications should include the type of publication, a

reference number such as the DOI, the list of partners who contributed, etc.

This proposition has been implemented within FP7 but registration of outputs is done

independently from the reviewing process.

Page 121: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

125

Possible drawbacks

Increase of administrative burdens upon participants during reporting periods.

6.5.2.4 Reviewing the impacts

Rationale

In project proposals, the section on impacts has a non negligible weight in the proposal

evaluation but is later on not really taken into account during the project’s lifetime as

the impacts are announced to happen after the projects final evaluation is elaborated

when the project is officially terminated.

Proposition

The impact section of project proposals could identify expected short-term

impacts (e.g. six months after project termination). This would then be revised

at the final evaluation of projects (conducted within the first two months after

project conclusion). A proportion of project budget would be reserved for

dissemination and exploitation to support the realisation of these short-term

impacts and corresponding budget payment would occur after a simple review

of dissemination and exploitation results by the project officer

Elements of the project proposal which cannot be evaluated during project

execution should not be part of the project selection criteria as this tends to

lead to false promises in the project proposal.

A part of the budget could be reserved for dissemination and exploitation

during a period of approximately six months after the project end and efforts

done during this period should demonstrate a progress in the impact

achievement plan.

Possible drawbacks

The project evaluation and payment cannot be delayed until these impacts as these

would create cash flow issues for a number of participants, especially SMEs. However, a

part of the final payment could be made after the evaluation of accomplishment of

short term impacts, but again this would put additional burden upon Project Officers.

6.5.2.5 Market analysis

Rationale

A number of projects failed to reach their market impact either because the market

evolution was insufficiently evaluated or because the new technology supposed to

replace an existing one failed in doing so, either because of the changes required to

adopt the new technology (too high cost/benefit ratio) or more often, because the

technology already in place on the market also got improved while the project was

developing so the planned technological advantage was not obtained at project end.

Proposition

A milestone-based market analysis should be performed at project start, estimating not

only market growth but also looking at competing technologies and positioning

Page 122: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

126

industry partners’ vis-à-vis their competition. This would better alignment to market

requirements.

Possible drawbacks

Although such a requirement seems like additional administrative work for projects, it

is estimated to bring significant positive effects upon project courses. Perhaps, it would

be advisable to have such regular reviews for projects with close to the market

innovations and products and projects with blue-sky long-term results to perform such

reviews in less frequent intervals.

Having a potential project termination at the 6th month could lead to a prolonged

starting time as participants may be reluctant to engage resources in a project that may

be stopped.

Page 123: Ex-post Evaluation of FP6 NMP at project level

Detailed Report

127

7 REFERENCES [1] ‘Lisbon European Council 23 and 24 March 2000 – Presidency conclusions’,

http://www.europarl.europa.eu/summits/lis1_en.htm

[2] ‘Towards a European Research Area ‘, Commission of the European Communities, COM(2000) 6 final

[3] Position paper on future RTD activities of NMP for the period 2010 – 2015, NMP expert advisory group (EAG), November 2009.

[4] ‘Europe 2020, a strategy for smart, sustainable and inclusive growth’, Communication from the Commission, COM(2010) 2010.

[5] Decision No 1513/2002/EC of the European Parliament and of the Council of 27 June 2002 concerning the Sixth Framework Programme of the European Community for research, technological development and demonstration activities, contributing to the creation of the European Research Area and to innovation (2002 to 2006)

[6] ‘EU contribution to R & D in textiles; Technical textiles as innovative segment of the industry: Perspectives for technical textiles in FP7’, John CLEUREN, European Commission DG research, Directorate for Industrial Technologies, EUROMED conference, 2010, http://ec.europa.eu/enterprise/newsroom/cf/_getdocument.cfm?doc_id=5680

[7] Position paper : Mid-term assessment of FP6 TP3, Expert Advisory Group, version 31 January 2005

[8] “Ex-post evaluation of FP6 NMP, Strategic level”, Final report, June 2011

[9] “International Scientific Co-operation (INCO) in the Sixth Framework Programme (2002-2006)”, European Commission, ftp://ftp.cordis.europa.eu/pub/fp6/docs/marketing_inco_en.pdf.

[10] European Commission, Monitoring Industrial Research: the 2010 EU Industrial R&D Investment Scoreboard

[11] Alquézar J., “CSF Ex-Ante Impact Assessment report for NMP” – draft version, May 2011

[12] ‘Green Paper on Innovation’, 1995, http://europa.eu/documents/comm/green_papers/pdf/com95_688_en.pdf.

[13] European Commission, FP6 Focuses on breakthrough technologies, European Industrial Research Journal, Issue 1, July 2003

[14] European Commission – DG Budget, Participation survey and assessment of the impact of the actions completed under the 6th Framework Programme, Final Report prepared in the context of DG BUDG Framework Service Contracts on Evaluation and Evaluation-related services-Ref. No BUDG 06/PO/01/Lot3, Brussels, April 2009

[15] Commission Communication COM(2002) 714 final of 11.12.2002 - "Industrial Policy in an Enlarged Europe”

[16] Federal Ministry of Education and Research “The High Tech Strategy for Germany”.

[17] NMP Expert Advisory Group (EAG), Position paper on future RTD activities of NMP for the period 2010 – 2015, EUR 24179 EN, November 2009

[18] “RTD Evaluation Toolbox”, IPTS, Joanneum Research, 2002.

[19] Ex-post evaluation of NMP (FP6) at Strategic level (Lot 2), June 2010 http://ec.europa.eu/research/industrial_technologies/pdf/nmp-fp6-ex-post-evaluation_en.pdf

[20] "Evaluation of the Sixth Framework Programme for Research and Technological Development" 2002-2006, Report of the Expert Group, February 2009.

[21] "FP6 participation", Vonortas, 2008

[22] The Lund Declaration, “Europe must focus on the grand challenges of our time”, July 2009.

[23] ‘Europe 2020, a strategy for smart, sustainable and inclusive growth’, Communication from the Commission, COM(2010) 2020.

[24] ‘Europe 2020 Flagship Initiative - Innovation Union’, Communication from the Commission, COM(2010) 546 final.

[25] ‘Preparing for our future: Developing a common strategy for key enabling technologies in the EU’, COM(2009) 512/3

[26] “Improving knowledge transfer between research institutions and industry across Europe: embracing open innovation”, COM(2007) 182 final

Page 124: Ex-post Evaluation of FP6 NMP at project level

129

Part C - ANNEXES

Page 125: Ex-post Evaluation of FP6 NMP at project level

Annexes

131

Annex I - Criteria and indicators

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

Factual data related to project and partners

Project data NMP project characteristics

Average project duration

Instrument (% of responses)

Topic/thematic area (% of responses)

Average total project cost

Average total EC contribution

Average size of consortium (Number of partners)

Average project funded by EC (combination of all aforementioned indicators)

X

X

X

X

X

X

X

Partner data NMP participants characteristics

Legal status (% of responses)

Size of organisation (% of responses)

Industrial sector(s) in which the company operates (% of responses)

Average number of participation(s) into FP project(s) prior to the considered one (% of responses)

International cooperation (% of responses concerning non-EU

X

X

X

X

X

X

Page 126: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

132

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

participants)

Nature of project coordinator: Industrial (large/small) enterprise vs. academic institution (% of responses)

R&D capabilities of organisation (% of responses)

Average participant and average coordinator (combination of all aforementioned indicators)

X

X

X

Project characteristics

Nature of research conducted

Time horizon of project implementation, commercial/technical risk level, inter/multi-disciplinarity of research, etc. (%)

Strategic importance of project (%)

Networking with other organisations (%)

Orientation : application vs. enabling technology (%)

Future continuation of R&D in area (%)

Type of past R&D on which NMP project was built on

Average type of NMP project per instrument

X

X

X

X

X

X

X

Page 127: Ex-post Evaluation of FP6 NMP at project level

Annexes

133

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

Project Objectives – Relevance of Programme Objectives

Importance of goals

Relevance of programme objectives

Ranking of importance of scientific, commercial/economic, policy, social and health and environment goals promoted by NMP project participants (%)

X

X X

Project Achievements – Effectiveness of Programme

On research team level

S&T advancements

Sustainability of research

Importance of publications / PhDs/ Training programmes/ Exchange of good practices (%)

Average number of participations (presentations or posters) in conferences

Average number of publications in peer review journals

X

X

X

X

(X)

(X)

X

X

X

On organisation level

Enhanced reputation and image

Ability to carry out new activities or enter new markets

Importance of development of new / improvement of materials, manufacturing systems, services/products/components, processing of multifunctional materials, processes, prototypes, patent applications, copyrights (%)

Average number of patents gained

Average number of copyrights/ trademarks/ registered designs

X

X

X

X

X

X

Page 128: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

134

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

Average number of spin-off companies created

Average number of software packages developed

X

X

X

X

On broader environment

Improvement of quality of life

Improvement of energy consumption and utilisation of primary resources

Importance of development of new / improvement of quality standards and measurement techniques, control devices and instruments, software/ simulation models, characterisation tools, production concepts, tools for efficient life-cycle design, organisational models, improved knowledge management tools (% of responses)

Importance of development of safer products (% of responses)

Average number of standards/normalisation bodies projects contributed to

X

X

X

X

X

X

Success factors and obstacles – Efficiency

Organisation internal factors (S&T competencies, managerial competences, previous experience, exploitation capabilities)

Organisation external factors (S&T competencies, managerial competences, previous experience,

X

X

X

X

X

X

Page 129: Ex-post Evaluation of FP6 NMP at project level

Annexes

135

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

exploitation capabilities of partners, communication, complementarity of goals, IPR arrangements, integration of end users, knowledge of markets,

Involvement of end users and target groups in project

Changes in external environment requiring transformation of objectives / context of project)

Project related factors (knowledge of industrial processes within consortium, clarity of project objectives, ambition of project goals, adequate resources, prior knowledge/technical inputs, additional funding, equipment, changes in regulatory environment)

EU support (administrative capabilities of EU officials, technical understanding of EU officials, contractual obligations)

X

X

X

X

X

X

Achievements in relation to expectations – Efficiency

Ranking of achievements: much greater than expectations to achievements; much less than expectations (%)

X (X)

Commercialisation % of respondents stating high X

Page 130: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

136

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

of results likelihood of commercial returns from NMP project results

Average estimated commercial return

Average increase of yearly commercial returns

X

X

Benefits vs. Costs Comparison between benefits obtained via project participation vs. costs involved

% of responses stating higher benefits than costs

X

Communication of results

Importance of communicating project results to society

X X

Impacts

Impacts on research team

S&T advancements

Networking

Enhanced knowledge base and skills (%)

Enhanced career prospects (%)

Improvement of access to expertise and establishment of critical mass of R&D (%)

Formation of new partnerships and R&D linkages (%)

Enhanced joint international research actions (%)

X

X

X

X

X

X

X

X

X

X

Impacts on own organisation

Economic

Networking

Ability to carry out new activities or enter new areas (%)

X

X

X

Page 131: Ex-post Evaluation of FP6 NMP at project level

Annexes

137

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

Enhanced reputation and image (%)

Improved competitive position (%)

Enhanced ability to produce, deliver or exploit new products, processes or services (%)

Improved commercial performance (Increased turnover, profitability, productivity, market share) (%)

Formation of new partnerships (%)

Formation of new businesses (%)

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Impacts on the broader environment

S&T advancements

Economic level

Improvement of public awareness and understanding

Policies

Environment and Health

Networking

Production of S&T results beyond state of the art, achievement of innovative breakthroughs (%)

Improved educational developments (%)

Development of markets (%)

Improvement of employment levels (%)

Development of standards and policies (%)

Improvement of coordination of research (%)

Improved public access to

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Page 132: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

138

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

information, public awareness and understanding and improvement of consumer confidence (%)

Improvement of living conditions and environment (%)

Improved international cooperation (%)

X

X

X

X

X

X

Media visibility impacts

Influence of media visibility to project impacts

X X X

Sustainability

Transience of impacts On R&D capabilities (%)

On overall organisation (%)

On the broader environment (%)

X

X

X

X

X

X

Continuation of research

Continuation of research in area (%)

X X

Additionality

EC additionality EC funding additionality

What is the added value of the execution of the project?

Percentage of projects that would not be further pursued in the absence of EU funding although being in the core strategic area of their organisation

Percentage of projects that did not receive funding from the EC, were in the core business area of their institution and were not further

X

X

X

X

X

X

Page 133: Ex-post Evaluation of FP6 NMP at project level

Annexes

139

Topic/Criteria Related evaluation

question Indicators

Main source

EC database

Web survey

Project reports

Interviews/ Case

studies

Analysis

pursued

Non-participants survey: amount of projects that although not selected by EC for funding have been elaborated with other funds

Characteristics the projects would have in case not selected by the EC for funding

X

Behavioural Additionality

What was the impact from the participation, specifically in the way research is conducted by the organisation and the teams involved.

X

Page 134: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

140

Annex II - FP6 NMP Calls

The table hereunder provides a list of the call for proposals launched under the NMP

thematic within the FP6.

Identifier Description

FP6-2002-NMP-1

Thematic call in the area of “Nano-technologies and nano-sciences,

knowledge-based multifunctional materials, and new production

processes and devices”

FP6-2002-IST-NMP-1 Thematic call in the area of “manufacturing, products and services

engineering in 2010”

FP6-2002-NMP-2

Dedicated call for SMEs in support to the development of new

knowledge based added value products and services in traditional

less RTD intensive industries

FP6-2003-ACC-SSA-

General Specific Support Actions (SSA) for Associated Candidate Countries

FP6-2003-ACC-SSA-

NMP

Targeted Specific Support Actions (SSA) for Associated Candidate

Countries

FP6-2003-NMP-NI-3

Thematic call in the area of 'Nanotechnologies and nanosciences,

knowledge-based multifunctional materials and new production

processes and devices - NI '

FP6-2003-NMP-SME-3

Dedicated call for IPs for SMEs in the area of 'Nanotechnologies and

nanosciences, knowledge-based multifunctional materials and new

production processes and devices'

FP6-2003-NMP-

STEEL-3

Dedicated call in the area of 'Very low CO2 Steel Processes', launched

in co-ordination with the 2003 and 2004 calls of the research Fund for

Coal and Steel, as referred to in the Council Decision 2003/78/EC (O.J.

L29/28 of 05.02.03)

FP6-2003-NMP-TI-3-

main

Thematic call in the area of 'Nanotechnologies and nanosciences,

knowledge-based multifunctional materials and new production

processes and devices'

FP6-2003-NMP-TI-3-

ncp

Thematic call in the area of 'Nanotechnologies and nanosciences,

knowledge-based multifunctional materials and new production

processes and devices'

FP6-2004-NMP-NSF-1

Thematic call in the area of 'Nanotechnologies and nanosciences,

knowledge-based multifunctional materials and new production

processes and devices '

FP6-2004-IST-NMP-2 Second Joint Call between thematic priorities 2 and 3

FP6-2004-NMP-NI-4

Thematic call in the area of 'Nanotechnologies and nanosciences,

knowledge-based multifunctional materials and new production

processes and devices'

FP6-2004-NMP-TI-4

Thematic call in the area of 'Nanotechnologies and nanosciences,

knowledge-based multifunctional materials and new production

processes and devices '

FP6-2004-NMP-SME-4 Dedicated call for IPs for SMEs in the area of 'Nanotechnologies and

nanoscieces, knowledge-based multifunctional materials and new

Page 135: Ex-post Evaluation of FP6 NMP at project level

Annexes

141

production processes and devices'

FP6-2006-TTC-TU-

Priority-3

Specific call to promote the participation of partners from Targeted

Third Countries in projects for which contracts are already signed or

under negotiation in priority thematic areas of research.

Page 136: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

142

Annex III - Projects analysed as case studies

N° ID Acronym Call

Identifier Instrument Title Pillar Pillar description

1 11734 ECOBINDERS FP6-2003-NMP-SME-

3 IP

Furan and lignin based resins as eco-friendly and durable solutions for wood preservation, panel, board and design products

NMP-3 New Production Processes and Devices

2 11783 MULTIPROTEC

T FP6-2003-NMP-NI-3

IP Advanced environmentally friendly multifunctional corrosion protection by nanotechnology

NMP-2 Knowledge-based Multifunctional Materials

3 11799 HOLIWOOD FP6-2003-NMP-SME-

3 IP

Holistic Implementation of European thermal treated hard wood in the sector of construction industry and noise protection by sustainable, knowledge-based and value added products.

NMP-3 New Production Processes and Devices

4 11815 NEXT FP6-2003-NMP-NI-3

IP NEXT GENERATION PRODUCTION SYSTEMS NMP-3 New Production Processes and Devices

5 11816 IMPULSE FP6-2003-NMP-NI-3

IP Integrated Multiscale Process Units with Locally Structured Elements

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

6 11817 TUNCONSTRU

CT FP6-2003-NMP-NI-3

IP Technology innovation in underground construction NMP-4 Integrating NMP-1, NMP-2 and NMP-3

7 11844 NAPOLEON FP6-2003-NMP-NI-3

IP Nanostructured waterborne polymer films with outstanding properties

NMP-2 Knowledge-based Multifunctional Materials

8 13545 NANOSPIN FP6-2003-NMP-TI-3-

MAIN STREP

Self-Organised Complex-Spin Magnetic Nanostructures

NMP-3 New Production Processes and Devices

9 13644 MULTIMATDE

SIGN

FP6-2003-NMP-TI-3-

MAIN STREP

Computer aided molecular design of multifunctional materials with controlled permeability properties

NMP-2 Knowledge-based Multifunctional Materials

10 13683 ALUHEAT FP6-2003-NMP-TI-3-

MAIN STREP High Efficiency Aluminium Billet Induction Heater NMP-3 New Production Processes and Devices

11 13698 SA-NANO FP6-2003-NMP-TI-3-

MAIN STREP

Self Assembly of Shape Controlled Colloidal Nanocrystals

NMP-1 Nanotechnologies and Nanosciences

Page 137: Ex-post Evaluation of FP6 NMP at project level

Annexes

143

12 13883 FLEXONICS FP6-2003-NMP-TI-3-

MAIN STREP

Ultra-high barrier films for R2R encapsulation of flexible electronics

NMP-2 Knowledge-based Multifunctional Materials

13 13912 SMARTCAP FP6-2003-NMP-TI-3-

MAIN STREP

Injectable macroporous biomaterial based on Calcium Phosphate cement for bone regeneration

NMP-2 Knowledge-based Multifunctional Materials

14 13989 ECOFIT FP6-2003-NMP-TI-3-

MAIN STREP

Eco-efficient machine-tools by means of radical mass and energy needs reduction

NMP-3 New Production Processes and Devices

15 13998 PREDFIRE-

NANO

FP6-2003-NMP-TI-3-

MAIN STREP

Predicting Fire Behaviour of Nanocomposites from Intrinsic Properties

NMP-2 Knowledge-based Multifunctional Materials

16 16375 MNIBS FP6-2004-

NMP-NSF-1 STREP

Multiscale Modeling of Nanostructured Interfaces for Biological Sensors

NMP-5 Cross-cutting activities

17 16433 SINPHONIA FP6-2004-IST-NMP-2

STREP Single-photon nanostructured detectors for advanced optical applications

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

18 16547 CEGRIS II FP6-2004-IST-NMP-2

STREP Centreless grinding simulation part II NMP-4 Integrating NMP-1, NMP-2 and NMP-3

19 16626 CELLFORCE FP6-2004-IST-NMP-2

STREP Development of a single cell based biosensor for subcellular on-line monitoring of cell performance for diagnosis and healthcare

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

20 16710 NANOMESO FP6-2004-

NMP-NSF-1 STREP Size Effects in Mechanical Properties NMP-5 Cross-cutting activities

21 16726 SMMART FP6-2004-IST-NMP-2

IP System for Mobile Maintenance Accessible in Real Time

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

22 16882 MODULINSPEX FP6-2004-IST-NMP-2

STREP Modular High Speed X-ray Detection and Sorting Systems for Production Process Control and Packaging Quality Control

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

23 16938 RI-MACS FP6-2004-IST-NMP-2

STREP Radically Innovative Mechatronics and Advanced Control Systems

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

24 17002 BIODIAGNOSTI

CS FP6-2004-IST-NMP-2

STREP Biological diagnostic tools using microsystems and supersensitive magnetic detection

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

25 17350 BIO-MEDNANO FP6-2004- STREP Integrating enzymes, mediators and nanostructures NMP-4 Integrating NMP-1, NMP-2 and NMP-3

Page 138: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

144

IST-NMP-2 to provide bio-powered bio-electrochemical sensing systems

26 17498 DYNAMITE FP6-2004-IST-NMP-2

IP Dynamic Decisions in Maintenance NMP-4 Integrating NMP-1, NMP-2 and NMP-3

27 505724 HIPERMAG FP6-2002-

NMP-1 STREP

Nano- and micro-scale engineering of higher-performance MgB2 composite superconductors for macro-scale applications

NMP-2 Knowledge-based Multifunctional Materials

28 32202 EMDPA FP6-2004-NMP-TI-4

STREP

New Elemental and Molecular Depth Profiling Analysis of Advanced Materials by Modulated Radio Frequency Glow Discharge Time of Flight Mass Spectrometry

NMP-2 Knowledge-based Multifunctional Materials

29 32378 ECLIPS FP6-2004-NMP-TI-4

STREP Extended Collaborative integrated Life cycle supply chain Planning System

NMP-3 New Production Processes and Devices

30 33191 NANOXIDE FP6-2004-NMP-TI-4

STREP Novel Nanoscale Devices based on functional Oxide Interfaces

NMP-2 Knowledge-based Multifunctional Materials

31 33227 µSAPIENT FP6-2004-NMP-TI-4

CA Synergetic Process Integration for Efficient Micro and Nano Manufacture

NMP-3 New Production Processes and Devices

32 33254 SANTS FP6-2004-NMP-TI-4

STREP Synthesis and nanotechnologial application of tethered silicates

NMP-1 Nanotechnologies and Nanosciences

33 33256 BENATURAL FP6-2004-NMP-TI-4

STREP Bioengineered Nanomaterials for research and Applications

NMP-1 Nanotechnologies and Nanosciences

34 500057 NANO2LIFE FP6-2002-

NMP-1 NOE A network for bringing NANOtechnologies TO LIFE NMP-1 Nanotechnologies and Nanosciences

35 500095 MASMICRO FP6-2002-

NMP-1 IP

INTEGRATION OF MANUFACTURING SYSTEMS FOR MASS-MANUFACTURE OF MINIATURE/MICRO-PRODUCTS

NMP-3 New Production Processes and Devices

36 500096 CANAPE FP6-2002-

NMP-1 IP

Carbon Nanotubes for Applications in Electronics, Catalysis, Composites and Nano-Biology

NMP-1 Nanotechnologies and Nanosciences

37 500120 NAPA FP6-2002-

NMP-1 IP Emerging Nanopatterning Methods NMP-1 Nanotechnologies and Nanosciences

38 500159 FAME FP6-2002-

NMP-1 NOE

FUNCTIONALISED ADVANCED MATERIALS ENGINEERING OF HYBRIDS AND CERAMICS

NMP-2 Knowledge-based Multifunctional Materials

39 500160 AIMS FP6-2002- IP Advanced Interactive Materials by Design NMP-2 Knowledge-based Multifunctional

Page 139: Ex-post Evaluation of FP6 NMP at project level

Annexes

145

NMP-1 Materials

40 500198 NANOQUANTA FP6-2002-

NMP-1 NOE

Nanoscale Quantum Simulations for Nanostructures and Advanced Materials

NMP-1 Nanotechnologies and Nanosciences

41 500252 METAMORPHO

SE FP6-2002-

NMP-1 NOE

MetaMaterials ORganized for radio, millimeter wave, and PHOtonic Superlattice Engineering

NMP-2 Knowledge-based Multifunctional Materials

42 500273 I*PROMS FP6-2002-

NMP-1 NOE

Innovative Production Machines and Systems (I* PROMS)

NMP-3 New Production Processes and Devices

43 500274 4M FP6-2002-

NMP-1 NOE

Multi-Material Micro Manufacture: Technologies and Applications

NMP-3 New Production Processes and Devices

44 500283 EXPERTISSUES FP6-2002-

NMP-1 NOE

Novel Therapeutic Strategies for Tissue Engineering of Bone and Cartilage Using Second Generation Biomimetic Scaffolds

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

45 500328 FRONTIERS FP6-2002-IST-NMP-1

NOE

NoE research and facilities directed at instrumentation for manufacturing and analysis of single molecules and individual nanoclusters, targeted at life sciences

NMP-1 Nanotechnologies and Nanosciences

46 500329 BIOMINE FP6-2002-

NMP-1 IP Biotechnology for Metal bearing materials in Europe NMP-3 New Production Processes and Devices

47 500345 ECOTARGET FP6-2002-

NMP-1 IP

New and innovative processes for radical changes of the European pulp and paper industry

NMP-3 New Production Processes and Devices

48 500355 NAIMO FP6-2002-

NMP-1 IP

NAnoscale Integrated processing of self-organizing Multifunctional Organic Materials

NMP-2 Knowledge-based Multifunctional Materials

49 500465 STEPS FP6-2002-

NMP-1 IP

A Systems Approach to Tissue Engineering Processes and Products

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

50 500895 INSIDE_PORES FP6-2002-

NMP-1 NOE

IN Sltu study and DEvelopment of processes involving nanoPORous Solids

NMP-1 Nanotechnologies and Nanosciences

51 501084 INMAR FP6-2002-

NMP-1 IP Intelligent Materials for Active Noise Reduction NMP-4 Integrating NMP-1, NMP-2 and NMP-3

52 504017 CORNEA

ENGINEERING FP6-2002-

NMP-1 STREP

THREE-DIMENSIONAL RECONSTRUCTION OF HUMAN CORNEAS BY TISSUE ENGINEERING

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

53 505206 HYMM FP6-2002-

NMP-1 STREP

Advanced Hybrid Mechatronic Materials for ultra precise and high performance machining systems design

NMP-2 Knowledge-based Multifunctional Materials

Page 140: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

146

54 505247 NANOFOREST FP6-2002-

NMP-1 SSA

A nanotechnology roadmap for the forest products industry

NMP-1 Nanotechnologies and Nanosciences

55 505275 ISAMCO FP6-2002-

NMP-1 STREP

IONIC POLYMER-METAL COMPOSITE AS SENSOR AND ACTUATOR: APPLICATION IN MOTION CONTROL

NMP-2 Knowledge-based Multifunctional Materials

56 505285 SEMINANO FP6-2002-

NMP-1 STREP

PHYSICS AND TECHNOLOGY OF ELEMENTAL, ALLOY AND COMPOUND SEMICONDUCTOR NANOCRYSTALS: MATERIALS AND DEVICES

NMP-1 Nanotechnologies and Nanosciences

57 505339 KOBAS FP6-2002-

NMP-2 IP

Knowledge Based Customized Services for Traditional Manufacturing Sectors Provided by a Network of High Tech SMEs

NMP-3 New Production Processes and Devices

58 505463 MACE FP6-2002-

NMP-1 STREP

Multifunctional Advanced Carbon Aluminium Composite for Electricity Transport

NMP-2 Knowledge-based Multifunctional Materials

59 505466 PROHIPP FP6-2002-

NMP-2 IP

NEW DESIGN AND MANUFACTURING PROCESSES FOR HIGH PRESSURE FLUID POWER PRODUCTS

NMP-3 New Production Processes and Devices

60 505567 BIOCELSOL FP6-2002-

NMP-1 STREP

Biotechnological Process for Manufacturing Cellulosic Products with Added Value

NMP-3 New Production Processes and Devices

61 505580 DT-CRYS FP6-2002-

NMP-1 STREP

Double Tungstate Crystals: synthesis, characterization and applications

NMP-2 Knowledge-based Multifunctional Materials

62 505626 DESYGN-IT FP6-2002-

NMP-1 STREP

DEsign, SYnthesis and Growth of Nanotubes for Industrial Technology

NMP-1 Nanotechnologies and Nanosciences

63 505630 MUSTWIN FP6-2002-

NMP-1 STREP

Micromachined Ultrasound transducers for wide range application in Medical imaging and Non Destructive Testing

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

64 505634 X-TIP FP6-2002-

NMP-1 STREP

Nano-scale chemical mapping and surface structural modification by joined use of X-ray microbeams and tip assisted local detection.

NMP-1 Nanotechnologies and Nanosciences

65 505641 GANANO FP6-2002-

NMP-1 STREP

New Generation of GaN-based sensor arrays for nano- and pico-fluidic systems for fast and reliable biomedical testing

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

66 505642 ANSWER FP6-2002-

NMP-1 STREP

Artificial Nanomaterials for Short Wavelength Emission in the infraRed

NMP-2 Knowledge-based Multifunctional Materials

67 505654 PROFORM FP6-2002- STREP Transforming nano-particles into sustainable NMP-3 New Production Processes and Devices

Page 141: Ex-post Evaluation of FP6 NMP at project level

Annexes

147

NMP-1 consumer products through advanced product and process formulation

68 505657 MINUET FP6-2002-

NMP-1 STREP

Miniaturised Ultrasonic, Engineered-Structure and LTCC-Based Devices for Acoustics, Fluidies, Optics and Robotics

NMP-2 Knowledge-based Multifunctional Materials

69 505669 CIDNA FP6-2002-

NMP-1 STREP

Control of assembly and charge transport properties of immobilized DNA

NMP-1 Nanotechnologies and Nanosciences

70 505711 AUTOBONE FP6-2002-

NMP-1 STREP

PRODUCTION UNIT FOR THE DECENTRALISED ENGINEERING OF AUTOLOGOUS CELL-BASED OSTEOINDUCTIVE BONE SUBSTITUTES

NMP-4 Integrating NMP-1, NMP-2 and NMP-3

71 505862 INSERT FP6-2002-

NMP-1 STREP Integrating Separation and Reaction Technologies NMP-3 New Production Processes and Devices

72 505864 FLEXIFUNBAR FP6-2002-

NMP-2 IP

MULTIFUNCTIONAL BARRIERS FOR FLEXIBLE STRUCTURES (PAPER, LEATHER, PAPER)

NMP

nanotechnologies and nanosciences, knowledge based multifunctional materials and new production processes and devices

73 505895 NANOCHEMSE

NS FP6-2002-

NMP-1 STREP Nanostructures for Chemical Sensors NMP-1 Nanotechnologies and Nanosciences

74 505899 SOPHIED FP6-2002-

NMP-2 IP

Novel sustainable bioprocesses for the European colour industries

NMP-3 New Production Processes and Devices

75 505928 MATECO FP6-2002-

NMP-1 STREP

New Coatings Deposited by PACVD for Corrosion Protection

NMP-2 Knowledge-based Multifunctional Materials

76 505954 RAMATI FP6-2002-

NMP-1 STREP Rapid manufacturing of titanium implants NMP-3 New Production Processes and Devices

77 507378 CEC-MADE-

SHOE FP6-2002-IST-NMP-1

IP Custom, Environment, and Comfort made shoe NMP-3 New Production Processes and Devices

78 507437 CUSTOM-FIT FP6-2002-IST-NMP-1

IP

A knowledge-based manufacturing system, established by integrating Rapid Manufacturing, IST and Material Science to improve the Quality of Life of European Citizens through Custom fit Products.

NMP-3 New Production Processes and Devices

79 507487 VRL KCIP FP6-2002-IST-NMP-1

NOE Virtual research Lab for a Knowledge Community in Production

NMP-3 New Production Processes and Devices

80 507978 EUPASS FP6-2002- IP Evolvable Ultra-Precision Assembly Systems IST-1 Applied IST research addressing major

Page 142: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

148

IST-NMP-1 societal and economic challenges

81 515743 IONMET FP6-2003-NMP-SME-

3 IP

New Ionic Liquid Solvent Technology to Transform Metal Finishing Products and Processes

NMP-3 New Production Processes and Devices

82 515757 MIND FP6-2003-NMP-NI-3

NOE Integrated Piezoelectric Devices NMP-2 Knowledge-based Multifunctional Materials

83 515762 I-STONE FP6-2003-NMP-SME-

3 IP

Re-engineering of natural stone production chain through knowledge based processes, eco-innovation and new organisational paradigms

NMP-3 New Production Processes and Devices

84 515767 MAGMANET FP6-2003-NMP-NI-3

NOE Molecular Approach to Nanomagnets and Multifunctional Materials

NMP-2 Knowledge-based Multifunctional Materials

85 515769 BIOCOMP FP6-2003-NMP-SME-

3 IP

New Classes of Engineering Composite Materials from Renewable Resources

NMP-3 New Production Processes and Devices

86 515803 CHARPAN FP6-2003-NMP-NI-3

IP CHARGED PARTICLE NANOTECH NMP-3 New Production Processes and Devices

87 515810 LEAPFROG FP6-2003-NMP-NI-3

IP LEADERSHIP FOR EUROPEAN APPAREL PRODUCTION FROM RESEARCH ALONG ORIGINAL GUIDELINES

NMP-3 New Production Processes and Devices

88 515813 AVALON FP6-2003-NMP-SME-

3 IP

Multifunctional textile structure driving new production and organizational paradigms by textile SME Interoperation Across high-added-VALue SectOrs for knowledge-based product/ service creatioN

NMP-3 New Production Processes and Devices

89 515825 MANUBUILD FP6-2003-NMP-NI-3

IP Open Building Manufacturing NMP-4 Integrating NMP-1, NMP-2 and NMP-3

90 515831 VIRTHUALIS FP6-2003-NMP-NI-3

IP Virtual Reality and Human Factors Applications for Improving Safety

NMP-3 New Production Processes and Devices

91 515844 INNOVATIAL FP6-2003-NMP-NI-3

IP Innovative processes and materials to synthesise knowledge-based ultra-performance nanostructured PVD thin films on gamma titanium aluminides

NMP-2 Knowledge-based Multifunctional Materials

92 515846 NAPOLYDE FP6-2003-NMP-NI-3

IP Control and smart devices NMP-3 New Production Processes and Devices

Page 143: Ex-post Evaluation of FP6 NMP at project level

Annexes

149

93 515848 SOILCY FP6-2003-NMP-SME-

3 IP

NEW SUSTAINABLE COMPRESSOR OIL PRODUCTION AND USE. TOWARDS A LONG ECO-EFFICIENT LIFE CYCLE

NMP-3 New Production Processes and Devices

94 515960 ULCOS FP6-2003-

NMP-STEEL-3

IP Ultra-Low CO2 Steelmaking NMP-5 Cross-cutting activities

95 516865 TASNANO FP6-2003-NMP-TI-3-

MAIN STREP

Tools and Technologies for the Analysis and Synthesis of Nanostructures

NMP-1 Nanotechnologies and Nanosciences

96 516943 NANOBIOCOM FP6-2003-NMP-TI-3-

MAIN STREP

INTELLIGENT NANOCOMPOSITE FOR BONE TISSUE REPAIR AND REGENERATION

NMP-2 Knowledge-based Multifunctional Materials

97 516961 BASE FP6-2003-NMP-TI-3-

MAIN STREP

Bio-based Functional materials from Engineered Self-Assembling Peptides

NMP-2 Knowledge-based Multifunctional Materials

98 517002 IOLISURF FP6-2003-NMP-TI-3-

MAIN STREP

Ionic Liquids as a novel electrochemical medium for radically innovative, cost effective, and environmentally friendly surface treatment technologies

NMP-2 Knowledge-based Multifunctional Materials

99 517036 PECTICOAT FP6-2003-NMP-TI-3-

MAIN STREP

Nanobiotechnology for the coating of medical devices

NMP-1 Nanotechnologies and Nanosciences

100

517045 SMART FP6-2003-NMP-TI-3-

MAIN SSA

Foresight Action for Knowledge Based Multifunctional Materials Technology

NMP-2 Knowledge-based Multifunctional Materials

Page 144: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

150

Annex IV - Case study template

A template was defined at the project start, together with interview guidelines. This template has been used by the evaluation team for

the drafting of the 100 case studies.

Project case study

Replace by project logo REPLACE BY PROJECT ACRONYM

Project full title: XXXX

Project instrument: STREP/IP/NoE/CA/SSA

Start date: DD/MM/YY Duration: XX M

Project action: NMP-1/ NMP-2/ NMP-3/ NMP-4/ NMP-5

Project reference:

Project total budget:

Project website: http://www.xxx

Keywords:

Project short abstract (300-600 characters)

Page 145: Ex-post Evaluation of FP6 NMP at project level

Annexes

151

Consortium:

List of partners with their profile

Case study executive summary

Case study analysis

Project emergence and objectives How did the project emerge? Would that project have existed without the EC funding?

Describe here the project objectives at the beginning. Were they ambitious? Realistic? Did they evolve during project course?

Are they still seen as valid objectives?

To what extent have they been achieved?

Project outcomes and impacts Did the activities run smoothly? Were necessary competencies present?

What have been the noticeable benefits gained by the partners?

Did the projects generated or will generate commercial outcomes? Have any patents been filed?

Project long term impacts and sustainability Expand on long-term scientific/socio/economic/environmental impacts of the project

Are these impacts going to be sustainable?

What are/have been the barriers to further exploitation?

Are the individual partners still pursuing RTD in the field?

Is the consortium still cooperating (at least partly) or intend to?

Did the project contribute to ETPs or collaborated with them?

Success factors and obstacles What have been the main success factors and obstacle for the project?

Strengths and weaknesses

What have been the strengths and weaknesses of theFP6-NMP programme for that project?

Communication and dissemination What have been the main dissemination and communication activities of the project?

Did they contribute to the project success?

Monitoring activity Each project evaluation report shall also include an indication of how to check if the project's exploitation and impact develop

as expected (what to look for, when, and where).

Page 146: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

152

Conclusion

Points of deviation What are the key positive and negative highlights of the case study, that make this project different from other ones?

Interesting points to exploit further What have been the success/failure factors affecting the project?

Are there expected outcomes/impacts in the near future, related to that project, which should be looked at?

Key lessons learnt What are the lessons learnt in that case study which may later on be looked at when drafting the recommendations?

Page 147: Ex-post Evaluation of FP6 NMP at project level

Annexes

153

Annex V - Survey aggregated level analysis

This annex presents the aggregated analysis to all answers of the survey for selected projects.

V.1 - General Info

i) Question: Project Instrument

Participation in Project Instruments – Overall Results

Total Number of respondents: n=1.181

Page 148: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

154

ii) Question: How many employees are there overall in your organization?

Total Number of Employees in the Organization - Overall Results

Total Number of respondents: n=1162

Page 149: Ex-post Evaluation of FP6 NMP at project level

Annexes

155

iii) Question: In which Industry/business sector does your organization operate?

Industry / Business Sectors - Overall Results

Total Number of respondents: n=1152

Page 150: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

156

iv) Question: How do you rate the R&D capabilities (e.g. expertise of personnel, share of researchers in personnel,

existence of separate R&D department, volume of investment in R&D activities, research infrastructures

available, etc.) of your organization?

R&D Capabilities of Respondents - Overall Results

Total Number of respondents: n= 1166

v) Question: Please choose only one of the following: Research organization, Higher or secondary education

establishment, Industry, Other.

Type of Organizations - Overall Results

Total Number of respondents: n= 1181

Page 151: Ex-post Evaluation of FP6 NMP at project level

Annexes

157

vi) Question: SME Status

SMEs Status – Overall Results

Total Number of respondents: n= 1157

vii) Question: In which Pillar of NMP have you participated?

NMP Pillars - Overall Results

Total Number of respondents: n= 1160

Page 152: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

158

V.2 - Your Experience with Project

i) Question: How would you describe the project along the following dimensions?

Description of projects – Overall Results

Total Number of respondents: n≈ 1175

Page 153: Ex-post Evaluation of FP6 NMP at project level

Annexes

159

ii) Question: Did your NMP project build on R&D that your organization did before?

Percentage of Projects Built-on Past R&D – Overall Results

Total Number of respondents: n=1173

iii) Question: On what type of past R&D did your NMP project built?

Type of R&D the NMP project Built On – Overall Results

Total Number of respondents: n≈ 900

Page 154: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

160

vi) Question: Is R&D in this area continuing or likely to continue?

Continuation of R&D - Overall Results

Total Number of respondents: n=1155

Page 155: Ex-post Evaluation of FP6 NMP at project level

Annexes

161

V.3 - Projects Outputs

vi) Question: From your organization perspective, which of the following were important outputs of the project?

Important Outputs of the Projects – Overall Results

Total Number of respondents: n≈1175

Page 156: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

162

Quantification of project Outputs

vii) If known, please quantify…

a) …the number of peer reviewed journal article/conference papers to which your organization/research tem

contributed

Number of Peer Reviewed Journal Articles/Conference Papers – Overall Results

Total Number of respondents: n=962

b)….the number of participations in conferences by members of your organization

Number of Participations in Conferences – Overall Results

Page 157: Ex-post Evaluation of FP6 NMP at project level

Annexes

163

Total Number of respondents: n=996

c)…the number of patents gained due to research conducted

Number of Patents Gained- Overall Results

Total Number of respondents: n=906

d)…the number of Copyrights/ Trademarks/ Registers Designs applied for

Number of Copyrights/ Trademarks- Overall Results

Page 158: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

164

Total Number of respondents: n=866

e)…the number of Spin-offs companies created

Number of Spin-Offs Companies – Overall Results

Total Number of respondents: n=891

f)… the number of software packages developed

Number of Software Packages – Overall Results

Page 159: Ex-post Evaluation of FP6 NMP at project level

Annexes

165

Total Number of respondents: n=894

e)… the number of standards/normalization bodies you contributed to

Number of Standards/Normalization Bodies the NMP Projects Contributed to - Overall Results

Total Number of respondents: n=866

Page 160: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

166

V.4 - Project Goals and Added Value

i) Question: How Important were the following goals for your Organization?

Importance of Goals – Overall Results

Total Number of respondents: n≈1178

Page 161: Ex-post Evaluation of FP6 NMP at project level

Annexes

167

ii) Question: How did the achievements live up to your organization’s expectations?

Expectations and Achievements – Overall Results

Total Number of respondents: n≈1178

Page 162: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

168

iii) Question: Overall, were the benefits of participation in the project greater or less than the costs involved?

Costs VS Benefits – Overall Results

Total Number of respondents: n=1181

Page 163: Ex-post Evaluation of FP6 NMP at project level

Annexes

169

V.5 - Project Impacts

i) Question: How significant were the impacts of the project on your R&D team capabilities?

Impacts of the Projects on R&D Capabilities – Overall Results

Total Number of respondents: n≈1178

Page 164: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

170

ii) Question: How significant were the impacts of the project on your overall organization?

Impacts of the Projects on Participating Organizations – Overall Results

Total Number of respondents: n≈1178

Page 165: Ex-post Evaluation of FP6 NMP at project level

Annexes

171

iii) Question: How significant were the impacts of your NMP project on the broader environment?

Projects Impacts on Broader Environment – Overall Results

Total Number of respondents: n≈1178

Page 166: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

172

iv) Question : To what extent are the following impacts likely to be transient or long-lasting?

Transience of impacts – Overall Results

Total Number of respondents: n≈1175

Page 167: Ex-post Evaluation of FP6 NMP at project level

Annexes

173

iv) Question : To which of the following European Technology Platforms did your project contributed to?

Projects Contribution to European Technology Platforms

Page 168: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

174

V.6 - Commercialization of Results

i: Question : Has your organization realized, or expects to realize, commercial returns as a result of exploitation

of the project results?

Commercialization of Results

Total Number of respondents: n=1181

ii: Question :What is your estimated actual commercial return (in euro per year) from the project?

Estimated actual commercial return (in Euro per year) from the project

Total Number of respondents: n=232

Page 169: Ex-post Evaluation of FP6 NMP at project level

Annexes

175

V.7 - Additionality

i) Question: In case you had not received EU funding, would you have continued with your project?

Percentage of Respondents that would have (not) continued with their projects in case they had not

received EU Funding – Overall Results

Total Number of respondents: n=1181

ii) Question: Source of Funds – In case you had not received EU funding and would have continued with your

project.

Source of Funds – Overall Results

Total Number of respondents: n=391

Page 170: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

176

iii) Question : Synthesis of Consortium - In case you had not received EU funding and would have continued with

your project

Synthesis of Consortium –Overall Results

Total Number of respondents: n=390

Page 171: Ex-post Evaluation of FP6 NMP at project level

Annexes

177

iv) Question : Time Scale of Project - In case you had not received EU funding and would have continued with

your project

Time Scale of Project – Overall Results

Total Number of respondents: n= 388

v) Question : Project Objectives - In case you had not received EU funding and would have continued with your

project

Project Objectives – Overall Results

Total Number of respondents: n= 393

Page 172: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

178

vi) Question: Project Expectations in terms of net benefits - In case you had not received EU funding and would

have continued with your project

Project Expectations – Overall Results

Total Number of respondents: n= 392

vii)Question: Project Applications in terms of net benefits -In case you had not received EU funding and would

have continued with your project

Range of Project Applications – Overall Results

Total Number of respondents: n= 390

Page 173: Ex-post Evaluation of FP6 NMP at project level

Annexes

179

V.8 - Success Factors and Obstacles

i) Question: Which of the following factors had an important influence on the project achievements?

Success Factors and Obstacles – Overall Results

Total Number of respondents: n= 1180

Page 174: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

180

ii) Question: To what extent did visibility in any of the media channels listed below affect overall project impacts?

The visibility of Media Channels and their affect in the Projects Impacts – Overall Results

Total Number of respondents: n≈1177

Page 175: Ex-post Evaluation of FP6 NMP at project level

Annexes

181

iii) Question: How would you rate the Importance of communicating your project results to the society?

Importance of communicating your project results – Overall Results

Total Number of respondents: n=1175

iv) Question: How would you rate the involvement of end users and target groups in the project?

Significance of the involvement of end users and target groups in the project – Overall Results

Total Number of respondents: n=1177

Page 176: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

182

v) Question: During project course, did changes in the external environment required transformation of the

objectives and/ or context of your project?

Number of Transformations Occurred – Overall Results

Total Number of respondents: n=1177

vi) Question: Indicate all the changes that occurred (which were significant and did cause transformation of the

project objectives and /or context)?

Changes Occurred – Overall Results

Total Number of respondents: n=131

Page 177: Ex-post Evaluation of FP6 NMP at project level

Annexes

183

vii) Question: How did the project adapt to the changes?

Projects adaptation to changes – Overall Results

Total Number of respondents: n=131

Page 178: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

184

Annex VI - Survey: industry level analysis

This annex presents the survey results obtained when looking only at answers from the industry.

VI.1 - General Information

i) Question: Project Instrument

Participation in Project Instruments – Large Enterprises Results

Total Number of respondents: n=145

Page 179: Ex-post Evaluation of FP6 NMP at project level

Annexes

185

Participation in Project Instruments – SMEs Results

Total Number of respondents: n=106

ii) Question: How many employees are there overall in your organization?

Total Number of Employees in the Organization – Large Enterprises Results

Total Number of respondents: n=145

Page 180: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

186

Total Number of Employees in the Organization – SMEs Results

Total Number of respondents: n=106

Page 181: Ex-post Evaluation of FP6 NMP at project level

Annexes

187

iii) Question: In which Industry/business sector does your organization operate?

Industry / Business Sectors – Large Enterprises Results

Total Number of respondents: n=143

Page 182: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

188

Industry / Business Sectors – SMEs Results

Total Number of respondents: n=158

Page 183: Ex-post Evaluation of FP6 NMP at project level

Annexes

189

iv) Question: How do you rate the R&D capabilities (e.g. expertise of personnel, share of researchers in personnel,

existence of separate R&D department, volume of investment in R&D activities, research infrastructures

available, etc.) of your organization?

R&D Capabilities of Respondents – Large Enterprises Results

Total Number of respondents: n= 145

R&D Capabilities of Respondents – SMEs Results

Total Number of respondents: n=106

Page 184: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

190

v) Question: Please choose only one of the following: Research organization, Higher or secondary education

establishment, Industry, Other.

Type of Organizations – Large Enterprises Results

Total Number of respondents: n=145

Type of Organizations – SMEs Results

Total Number of respondents: n= 106

Page 185: Ex-post Evaluation of FP6 NMP at project level

Annexes

191

vi) Question: In which Pillar of NMP have you participated?

NMP Pillars – Large Enterprises Results

Total Number of respondents: n= 145

Page 186: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

192

NMP Pillars – SMEs Enterprises Results

Total Number of respondents: n= 106

Page 187: Ex-post Evaluation of FP6 NMP at project level

Annexes

193

VI.2 - Your Experience with Project

i) Question: How would you describe the project along the following dimensions?

Description of projects – Large Enterprises Results

Total Number of respondents: n≈ 145

Page 188: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

194

Description of projects – SMEs Results

Total Number of respondents: n≈ 106

Page 189: Ex-post Evaluation of FP6 NMP at project level

Annexes

195

ii) Question: Did your NMP project build on R&D that your organization did before?

Percentage of Projects Built-on Past R&D – Large Enterprises Results

Total Number of respondents: n=145

Percentage of Projects Built-on Past R&D – SMEs Results

Total Number of respondents: n=105

Page 190: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

196

iii) Question: On what type of past R&D did your NMP project built?

Type of R&D the NMP project Built On – Large Enterprises Results

Total Number of respondents: n≈ 105

Type of R&D the NMP project Built On – SMEs Results

Page 191: Ex-post Evaluation of FP6 NMP at project level

Annexes

197

Total Number of respondents: n≈72

vi) Question: Is R&D in this area continuing or likely to continue?

Continuation of R&D – Large Enterprises Results

Total Number of respondents: n≈ 145

Continuation of R&D – SMEs Results

Page 192: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

198

Total Number of respondents: n≈ 102

Comments and significant Statistical differences on “Part 2 - Projects Description”

Both Participants from SMEs and Large Enterprises had the same experience participating in the NMP funded

projects.

Specifically, both consider that the projects were of medium cost but involved high Commercial risk as well as

Technical risk and complexity. Moreover, both participants from SMEs and large Enterprises considered that

the projects involved applied research which had high linkages with other in-house projects as well as with core

technology areas of their organizations. However, it should be mentioned that the 73,1% of respondents from

Long Enterprises consider that their projects included long-term nature of R&D when this percentage is only

59,9% for the participants from SMEs.

Also, the majority of the persons involved in the projects from both SMEs and Large enterprises believe that the

projects were only feasible with external collaborators and required the high involvement of both top European

and international researchers and organizations.

Also, it is worth mentioning, that the majority of the participants from both SMEs and Large enterprises consider

that the NMP project they had participated was of high strategic importance for their organization and that it was

build on R&D that their organization did before. Also, the vast majority of the participants stated that they shall

continue performing R&D in the area

Page 193: Ex-post Evaluation of FP6 NMP at project level

Annexes

199

Large Enterprises

High Cost

Commercial

Risk S&T Risks

S&T

Complexity

Long-term

nature of R&D

Applied

Research

Core

technology

area

High

Strategic

Importance

for org

Not

feasible

with

external

collabs.

Links with

in-house

projects

Involvement

of top

researchers

(EU)

Involvement

of top

researchers

(intl)

Very low 8,3% 13,2% 4,9% 2,8% 3,4% 2,8% 20,7% 6,2% 3,4% 5,5% 2,8% 15,4%

Low 12,5% 20,1% 11,8% ,7% 2,8% 15,9% 31,7% 5,5% 10,3% 15,2% 5,5% 16,1%

Medium 43,8% 23,6% 23,6% 14,5% 20,7% 29,7% 22,1% 13,8% 29,0% 27,6% 15,2% 21,7%

High 27,1% 27,8% 41,7% 46,2% 49,0% 36,6% 17,2% 27,6% 37,9% 35,9% 40,7% 30,8%

Very High 8,3% 15,3% 18,1% 35,9% 24,1% 15,2% 8,3% 46,9% 19,3% 15,9% 35,9% 16,1%

35,4% 43,1% 59,7% 82,1% 73,1% 51,7% 25,5% 74,5% 57,2% 51,7% 76,6% 46,9%

SMEs

High Cost

Commercial

Risk S&T Risks

S&T

Complexity

Long-term

nature of R&D

Applied

Research

Core

technology

area

High

Strategic

Importance

for org

Not

feasible

with

external

collabs.

Links with

in-house

projects

Involvement

of top

researchers

(EU)

Involvement

of top

researchers

(intl)

Very low 3,8% 6,7% 3,8% 1,9% 1,0% 31,1% 4,8% 1,9% 3,8% 2,9% 13,2%

Low 13,5% 20,0% 12,4% 4,7% 4,8% 13,3% 30,2% 10,5% 9,4% 11,3% 4,8% 17,9%

Medium 45,2% 27,6% 27,6% 8,5% 34,3% 28,6% 14,2% 10,5% 22,6% 32,1% 15,2% 23,6%

High 31,7% 32,4% 44,8% 54,7% 37,1% 45,7% 17,0% 29,5% 39,6% 35,8% 39,0% 29,2%

Very High 5,8% 13,3% 11,4% 32,1% 21,9% 11,4% 7,5% 44,8% 26,4% 17,0% 38,1% 16,0%

37,5% 45,7% 56,2% 86,8% 59,0% 57,1% 24,5% 74,3% 66,0% 52,8% 77,1% 45,3%

Statistical

Analysis 0,338 0,419 0,560 - 1,032 2,331 - 0,853 0,179 - 0,035 - 1,426 0,173 0,110 0,247 -

Page 194: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

200

VI.3 - Projects Outputs

vi) Question: From your organization perspective, which of the following were important outputs of the project?

Important Outputs of the Projects – Large Enterprises Results

Total Number of respondents: n≈145

Page 195: Ex-post Evaluation of FP6 NMP at project level

Annexes

201

Important Outputs of the Projects – SMEs Results

Total Number of respondents: n≈145

Page 196: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

202

Quantification of project Outputs

vii) If known, please quantify…

a) …the number of peer reviewed journal article/conference papers to which your organization/research tem

contributed

Number of Peer Reviewed Journal Articles/Conference Papers – Large Enterprises Results

Total Number of respondents: n= 113

Number of Peer Reviewed Journal Articles/Conference Papers – SMEs Results

Total Number of respondents: n=74

Page 197: Ex-post Evaluation of FP6 NMP at project level

Annexes

203

b)….the number of participations in conferences by members of your organization

Number of Participations in Conferences – Large Enterprises Results

Total Number of respondents: n=115

Number of Participations in Conferences – SMEs Results

Total Number of respondents: n=79

Page 198: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

204

c)…the number of patents gained due to research conducted

Number of Patents Gained- Large Enterprises Results

Total Number of respondents: n=110

Number of Patents Gained- SMEs Results

Total Number of respondents: n=78

Page 199: Ex-post Evaluation of FP6 NMP at project level

Annexes

205

d)…the number of Copyrights/ Trademarks/ Registers Designs applied for

Number of Copyrights / Trademarks- Large Enterprises Results

Total Number of respondents: n=103

Number of Copyrights/ Trademarks- SMEs Results

Total Number of respondents: n=74

Page 200: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

206

e)…the number of Spin-offs companies created

Number of Spin-Offs Companies – Large Enterprises Results

Total Number of respondents: n=107

Number of Spin-Offs Companies – SMEs Results

Total Number of respondents: n= 76

Page 201: Ex-post Evaluation of FP6 NMP at project level

Annexes

207

f)… the number of software packages developed

Number of Software Packages – Large Enterprises Results

Total Number of respondents: n=109

Number of Software Packages – SMEs Results

Total Number of respondents: n=77

Page 202: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

208

e)… the number of standards/normalization bodies you contributed to

Number of Standards/Normalization Bodies the NMP Projects Contributed to – Large Enterprises

Results

Total Number of respondents: n=107

Number of Standards/Normalization Bodies the NMP Projects Contributed to – SMEs Results

Total Number of respondents: n= 71

Page 203: Ex-post Evaluation of FP6 NMP at project level

Annexes

209

Comments and significant Statistical differences on “Part 3 – Project Outputs”

Both participants from SMEs and large Enterprises have also the same views on the importance of the their

projects outputs

The majority of the participants stated that most significant outputs were: the development of new/improved

services, products and components; the development of new / improved processes; the development of good

practices, the development of new improved materials and the development of prototypes, demonstrators and

pilots.

Participants from SMEs and Large Enterprises have also the same views on which were the less important

outputs of the projects they have participated in. Among the low importance outputs were the development of

new organizational models and knowledge management tools; PhD programmes and the development of

copyrights, trademarks, licenses etc.

Moreover, it should be mentioned that almost half (49,5%) of the respondents from SMEs have stated that the

New/Improved Quality Standards and measurement techniques were important outputs of the projects. On the

other hand, the same opinion on the importance of this output was expressed by the 36,1% of the respondents

from Large Enterprises.

Page 204: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

210

Large

Enterprises

Publications

Phds and

other

Qualifications

Training

programmes

Good

Practises

New/Improved

Materials

New/Improved

Manufactoring

Systems

New/Improve

d Sevices/

Products

Sustainable

Processing of

Multifuctonal

Materials

New/Improved

Processes

New/Improved

Quality

Standads and

meaurement

techniques

New/

Improved

control

devices and

instruments

New/Improved

Software

/Simulation

Models

New

Improved

characterisat

ion tools

New

productio

n concepts

New tools for

efficient life-cycle

design, production,

use and recovery of

systems

New

organizati

onal

Models

New/

Improved

knowledge

Manageme

nt tools

Safer

Products

Prototypes,

Demonstra

tors and

Pilots

Patent

Applications

Copyrights,

Trademarks,

registered

Designs,

Licences etc…

Other

Outputs

Not

Applicable

5,5% 15,3% 11,7% 6,2% 13,8% 13,1% 9,0% 19,3% 9,7% 13,2% 15,9% 13,2% 17,9% 14,5% 15,2% 24,1% 22,9% 19,6% 12,5% 18,3% 20,8% 57,0%

Very low 11,7% 22,2% 6,2% 1,4% 9,0% 9,0% 4,1% 10,3% 2,8% 8,3% 11,7% 14,6% 8,3% 11,0% 14,5% 31,0% 25,7% 19,6% 6,9% 14,1% 26,4% 15,6%

Low 20,0% 24,3% 26,2% 11,7% 5,5% 11,7% 11,0% 16,6% 13,1% 16,7% 15,2% 13,9% 14,5% 10,3% 17,2% 17,2% 18,1% 13,3% 9,0% 16,2% 18,8% 3,1%

Moderate 29,7% 20,1% 29,0% 20,0% 14,5% 15,9% 11,0% 17,9% 12,4% 25,7% 25,5% 18,1% 22,1% 18,6% 25,5% 15,9% 13,2% 16,8% 16,0% 21,8% 16,0% 8,6%

High 20,0% 11,8% 19,3% 38,6% 28,3% 20,0% 31,0% 18,6% 33,8% 20,1% 18,6% 22,2% 25,5% 31,0% 17,9% 6,2% 11,8% 14,7% 31,9% 12,7% 11,8% 10,2%

Very High 13,1% 6,3% 7,6% 22,1% 29,0% 30,3% 33,8% 17,2% 28,3% 16,0% 13,1% 18,1% 11,7% 14,5% 9,7% 5,5% 8,3% 16,1% 23,6% 16,9% 6,3% 5,5%

33,1% 18,1% 26,9% 60,7% 57,2% 50,3% 64,8% 35,9% 62,1% 36,1% 31,7% 40,3% 37,2% 45,5% 27,6% 11,7% 20,1% 30,8% 55,6% 29,6% 18,1% 15,6%

SMEs

Publications

Phds and

other

Qualifications

Training

programmes

Good

Practises

New/Improved

Materials

New/Improved

Manufactoring

Systems

New/Improve

d Sevices/

Products

Sustainable

Processing of

Multifuctonal

Materials

New/Improved

Processes

New/Improved

Quality

Standads and

meaurement

techniques

New/

Improved

control

devices and

instruments

New/Improved

Software

/Simulation

Models

New

Improved

characterisat

ion tools

New

productio

n concepts

New tools for

efficient life-cycle

design, production,

use and recovery of

systems

New

organizati

onal

Models

New/

Improved

knowledge

Manageme

nt tools

Safer

Products

Prototypes,

Demonstra

tors and

Pilots

Patent

Applications

Copyrights,

Trademarks,

registered

Designs,

Licences etc…

Other

Outputs

Not

Applicable

2,8% 21,0% 5,7% 2,8% 9,4% 10,4% 8,5% 17,9% 9,5% 10,5% 17,1% 17,1% 18,3% 15,2% 13,3% 24,0% 20,0% 11,4% 9,5% 13,3% 17,1% 61,8%

Very low 13,2% 23,8% 12,3% ,9% 6,6% 4,7% ,9% 13,2% 4,8% 6,7% 9,5% 14,3% 13,5% 7,6% 14,3% 17,3% 18,1% 16,2% 6,7% 13,3% 24,8% 11,8%

Low 21,7% 23,8% 24,5% 9,4% 8,5% 11,3% 5,7% 10,4% 7,6% 11,4% 16,2% 17,1% 16,3% 15,2% 16,2% 25,0% 18,1% 23,8% 15,2% 21,0% 22,9% 4,9%

Moderate 29,2% 19,0% 25,5% 22,6% 19,8% 22,6% 10,4% 17,9% 12,4% 21,9% 19,0% 21,9% 25,0% 19,0% 20,0% 17,3% 21,0% 19,0% 14,3% 20,0% 19,0% 12,7%

High 20,8% 10,5% 25,5% 44,3% 23,6% 26,4% 34,9% 13,2% 29,5% 29,5% 20,0% 17,1% 12,5% 24,8% 23,8% 8,7% 13,3% 19,0% 20,0% 10,5% 9,5% 2,9%

Very High 12,3% 1,9% 6,6% 19,8% 32,1% 24,5% 39,6% 27,4% 36,2% 20,0% 18,1% 12,4% 14,4% 18,1% 12,4% 7,7% 11,4% 10,5% 34,3% 16,2% 6,7% 5,9%

33,0% 12,4% 32,1% 64,2% 55,7% 50,9% 74,5% 40,6% 65,7% 49,5% 38,1% 29,5% 26,9% 42,9% 36,2% 16,3% 24,8% 29,5% 54,3% 26,7% 16,2% 8,8%

Statistical

Analysis -0,014 -1,255 0,887 0,560 -0,249 0,094 1,673 0,757 0,595 2,134 1,045 -1,787 -1,752 -0,420 1,443 1,033 0,863 -0,213 -0,200 -0,508 -0,389 -1,665

Page 205: Ex-post Evaluation of FP6 NMP at project level

Annexes

211

Fourth Part -Project Goals and Added Value

i) Question: How Important were the following goals for your Organization?

Importance of Goals – Large Enterprises Results

Total Number of respondents: n≈144

Page 206: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

212

Importance of Goals – SMEs Results

Total Number of respondents: n≈106

Page 207: Ex-post Evaluation of FP6 NMP at project level

Annexes

213

ii) Question: How did the achievements live up to your organization’s expectations?

Expectations and Achievements – Large Enterprises Results

Total Number of respondents: n≈146

Page 208: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

214

Expectations and Achievements – SMEs Results

Total Number of respondents: n≈105

Page 209: Ex-post Evaluation of FP6 NMP at project level

Annexes

215

iii) Question: Overall, were the benefits of participation in the project greater or less than the costs involved?

Costs VS Benefits – Large Enterprises Results

Total Number of respondents: n=145

Costs VS Benefits – Large SMEs Results

Total Number of respondents: n=106

Page 210: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

216

Comments and significant Statistical differences on “Part 4 - Project Goals”

The vast majority of both participants from SMEs and Large Enterprises stated that the most important goals of

their NMP project were the Scientific and Technological goals. Also, of great importance for the SMEs were the

Commercial and Economic goals of the projects (72,6%), while for Large Enterprises only 51% of the

respondents considered these goals as very important. Moreover, about 40% of the respondents from both

SMEs and Large enterprises consider that the health and environmental goals were of high importance. On the

other hand, the social goals as well as the policy goals were considered to be of low importance for both

participants from SMEs and Large Enterprises.

Page 211: Ex-post Evaluation of FP6 NMP at project level

Annexes

217

Part 4 - Project Achievements

In terms of achievement of goals, a great percentage of both participants from SMEs and Large Enterprises

consider that the scientific and technological achievements, the Commercial & Economic Achievements, the

Policy oriented Achievements, the Social Achievements and the Health and environmental Achievements of

their projects were equal to their expectations.

Large Enterprises

Other Goals S&T goals

Commercial

and Economic

goals

Policy-

Orineted

Goals Social goals

Health and

environmental

goals

Non applicable

78,7% 4,2% 5,6% 16,7% 18,1% 16,7%

Very low importance

5,7% ,7% 3,5% 12,5% 14,6% 6,9%

Low importance

6,6% 2,1% 11,9% 19,4% 28,5% 18,1%

Moderate

importance

1,6% 11,1% 28,0% 23,6% 21,5% 16,0%

High importance

2,5% 41,7% 31,5% 19,4% 12,5% 25,0%

Very high importance

4,9% 40,3% 19,6% 8,3% 4,9% 17,4%

7,4% 81,9% 51,0% 27,8% 17,4% 42,4%

SMEs

Other Goals S&T goals

Commercial

and Economic

goals

Policy-

Orineted

Goals Social goals

Health and

environmental

goals

Non applicable

78,5% ,9% ,9% 16,0% 17,0% 11,3%

Very low importance

7,5% 1,9% 3,8% 11,3% 15,1% 9,4%

Low importance

3,2% 6,6% 10,4% 21,7% 26,4% 21,7%

Moderate

importance

6,5% 11,3% 12,3% 31,1% 21,7% 19,8%

High importance

1,1% 33,0% 33,0% 12,3% 13,2% 20,8%

Very high importance

3,2% 46,2% 39,6% 7,5% 6,6% 17,0%

4,3% 79,2% 72,6% 19,8% 19,8% 37,7%

Statistical Analysis 1,049 - 0,532 - 3,600 1,484 - 0,491 0,741 -

Page 212: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

218

Part 4 - Benefits Vs Costs

43,4% of SMEs and 36,6% of Large Enterprises considered that the costs equalled benefits. However, 33% of

SMEs considered that benefits exceeded costs while this percentage climbs up to 46,2% for Large Enterprises.

On the other hand, it is worth mentioning that 19,8% of participants from SMEs consider that the costs

exceeded benefits while 15,2% of participants from Large enterprises shared the same opinion.

Large

Enterprises SMEs

Statistical

Analysis

46,2% 33,0% -2,139

No opinion 2,1% 3,8%

Benefits greatly

exceeded costs

11,7% 9,4%

Benefits

exceeded costs

34,5% 23,6%

Costs equalled

benefits

36,6% 43,4%

1,094

Costs exceeded

benefits

9,7% 18,9%

Costs greatly

exceeded

benefits

5,5% ,9%

15,2% 19,8% 0,950

Page 213: Ex-post Evaluation of FP6 NMP at project level

Annexes

219

VI.4 - Project Impacts

i) Question: How significant were the impacts of the project on your R&D team capabilities?

Impacts of the Projects on R&D Capabilities – Large Enterprises Results

Total Number of respondents: n≈145

Page 214: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

220

ii) Question: How significant were the impacts of the project on your overall organization?

Impacts of the Projects on R&D Capabilities – SMEs Results

Page 215: Ex-post Evaluation of FP6 NMP at project level

Annexes

221

Total Number of respondents: n≈106

Impacts of the Projects on Participating Organizations – Large Enterprises Results

Page 216: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

222

Total Number of respondents: n≈145

Impacts of the Projects on Participating Organizations – SMEs Results

Page 217: Ex-post Evaluation of FP6 NMP at project level

Annexes

223

Total Number of respondents: n≈105

iii) Question: How significant were the impacts of your NMP project on the broader environment?

Projects Impacts on Broader Environment – Large Enterprises Results

Page 218: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

224

Total Number of respondents: n≈145

Projects Impacts on Broader Environment – SMEs Results

Page 219: Ex-post Evaluation of FP6 NMP at project level

Annexes

225

Total Number of respondents: n≈105

iv) Question : To what extent are the following impacts likely to be transient or long-lasting?

Transience of impacts – Large Enterprises Results

Total Number of respondents: n=145

Transience of impacts – SMEs Results

Page 220: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

226

Total Number of respondents: n≈105

iv) Question : To which of the following European Technology Platforms did your project contributed to?

Projects Contribution to European Technology Platforms – Large Enterprises Results

Page 221: Ex-post Evaluation of FP6 NMP at project level

Annexes

227

Projects Contribution to European Technology Platforms – SMEs Results

Page 222: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

228

Comments and significant Statistical differences on “Part 5 – Projects Impacts”

Part 5 – Impacts on R&D capabilities

Concerning impacts on R&D capabilities the most significant impacts achieved for both the participants from

SMEs and Large Enterprises involve the enhancement of knowledge bases; the enhancement of skills and

competences; the improvement of R&D linkages with Universities and Research Institutes; the formation of

new R&D partnerships; and the improved access to complementary expertise.

Part 5 – Impacts on overall Organizations

According to the responses it can be seen that in general, the NMP projects had a greater impact on the

participating SMEs, than on the participating Large Enterprises respectively.

Specifically, the 58,5% of SMEs consider that their participation in the project significantly enhanced the

reputation and image of their organization, while 44,4% of the participants coming from Large enterprises

think the same about their projects. Furthermore, 65% of SMEs consider that their participation in the project

has significantly enhanced their ability to produce or deliver new products while this was stated only by 37% of

participants from Large Enterprises. Furthermore, 53% of SMEs consider that they have significantly improved

their competitive position, while around one third of participants from large enterprises perceived a similar

impact.

On the other hand, only 12% of large enterprises consider that the projects increased turnover while this

percentage climbs up to 23% for SMEs. Also only 9% of large enterprises consider that their participation in the

projects has enhanced productivity while this percentage is double (18%) for SMEs. Moreover, only 9% of large

enterprises consider that their participation in the projects improved market share while this percentage is

more than 20% for SMEs. In addition, only 24% of large enterprises considered that their organizations has

managed to improve commercial linkages thanks to their participation in the projects while this percentage

reaches 41,5% for SMEs. Moreover, only 5,5% of Large enterprises consider that the projects they have

participated had as an impact the formation of new business entities while this percentage is almost three

times more (16%) for SMEs.

Also, it is worth mentioning that a great percentage of participants from both SMEs and Large Enterprises

consider that the projects had a great impact on the formation of new partnerships and networks.

In general, the least significant impacts on the overall organizations for both SMEs and Large enterprises

involved the formation of new business entities; the access to new markets; the improvement of market share;

the increase of turnover and profitability; and the enhanced productivity.

Page 223: Ex-post Evaluation of FP6 NMP at project level

Annexes

229

Large Enterprises

Enchanced

reputation

and image

Enchanced

ability to

produce or

deliver new

products

Improved

competitive

position

Increased

turnover and

profitability

Enhanced

productivity

Improved

market share

Access to

new markets

Improved

commercial

linkages with

other org.

Formation of

new business

entities

Formation

of new

partnershi

ps and

networks

Other

Impacts

21,5% 24,8% 31,0% 44,1% 45,8% 43,8% 38,6% 36,1% 49,0% 21,5% 10,7%

Not Applicable 2,1% 9,7% 6,2% 16,8% 25,0% 23,6% 20,0% 14,6% 35,2% 9,0% 76,2%

Very low Significance 9,7% 11,0% 15,9% 24,5% 24,3% 25,0% 17,9% 18,8% 29,7% 9,7% 6,6%

Low Significance 11,8% 13,8% 15,2% 19,6% 21,5% 18,8% 20,7% 17,4% 19,3% 11,8% 4,1%

Medium Significance 31,9% 28,3% 29,0% 27,3% 20,1% 23,6% 15,9% 25,0% 10,3% 22,9% 8,2%

High Significance 31,9% 26,2% 24,1% 9,1% 6,9% 6,3% 17,2% 17,4% 3,4% 29,9% 1,6%

Very High Significance 12,5% 11,0% 9,7% 2,8% 2,1% 2,8% 8,3% 6,9% 2,1% 16,7% 3,3%

44,4% 37,2% 33,8% 11,9% 9,0% 9,0% 25,5% 24,3% 5,5% 46,5% 4,9%

SMEs

Enchanced

reputation

and image

Enchanced

ability to

produce or

deliver new

products

Improved

competitive

position

Increased

turnover and

profitability

Enhanced

productivity

Improved

market share

Access to

new markets

Improved

commercial

linkages with

other org.

Formation of

new business

entities

Formation

of new

partnershi

ps and

networks

Other

Impacts

14,2% 13,2% 17,9% 41,0% 45,7% 44,3% 30,2% 25,5% 34,9% 18,9% 5,4%

Not Applicable 0,0% 1,9% 2,8% 4,8% 17,1% 16,0% 10,4% 7,5% 41,5% 7,5% 79,3%

Very low Significance 2,8% 3,8% 7,5% 17,1% 25,7% 21,7% 6,6% 6,6% 24,5% 6,6% 2,2%

Low Significance 11,3% 9,4% 10,4% 23,8% 20,0% 22,6% 23,6% 18,9% 10,4% 12,3% 3,3%

Medium Significance 27,4% 19,8% 26,4% 31,4% 19,0% 17,9% 24,5% 25,5% 7,5% 23,6% 6,5%

High Significance 40,6% 43,4% 35,8% 17,1% 13,3% 17,0% 24,5% 27,4% 10,4% 23,6% 4,3%

Very High Significance 17,9% 21,7% 17,0% 5,7% 4,8% 4,7% 10,4% 14,2% 5,7% 26,4% 4,3%

58,5% 65,1% 52,8% 22,9% 18,1% 21,7% 34,9% 41,5% 16,0% 50,0% 8,7%Statistical Analysis 2,223 4,545 3,051 2,246 2,046 2,720 1,597 2,884 2,606 0,544 1,154

Page 224: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

230

Part 5 –Impacts on the Broader Environment

Concerning impacts on the broader environment, the most significant for both participants from

SMES and Large Enterprises were reported to be: the production of S&T results beyond the state of

the art; the achievement of innovative breakthroughs; and the improved coordination between

research and industry.

The least significant impacts involve: the development of new and existing policies; the improved

public understanding on N/T; the improved confidence of consumers in N&N/products; the

formation of new entities; and the improvement of living conditions.

Part 5 –Transience of Impacts

In general it can be seen that a great percentage of both participants from SMEs and Large

enterprises consider that impacts on R&D capabilities as well on the overall organizations and the

broader environment are expected to have a long lifetime (3 or more years). However, it worth

mentioning that the percentage of respondents from SMEs (47,6%) who consider that the lifetime of

the impacts on the overall organizations is going to be long, is greater from the percentage of

respondents from Large Enterprises (33,8%).

Page 225: Ex-post Evaluation of FP6 NMP at project level

Annexes

231

Large Enterprises

Impacts on on R&D

capabilities of Staff

Impacts on Overall

Organization

Impacts on the

Broader

Environment

Not Applicable 7,6% 15,2% 29,7%

Very Short Life time 5,5% 6,9% 4,1%

Short Life time 13,1% 11,7% 9,0%

Moderate life time 26,9% 32,4% 19,3%

Long Lifetime 28,3% 20,0% 18,6%

Very Long Life Time 18,6% 13,8% 19,3%

46,9% 33,8% 37,9%

SMEs

Impacts on on R&D

capabilities of Staff

Impacts on Overall

Organization

Impacts on the

Broader

Environment

Not Applicable 5,7% 2,9% 22,9%

Very Short Life time 2,8% 4,8% 5,7%

Short Life time 12,3% 11,4% 6,7%

Moderate life time 25,5% 33,3% 15,2%

Long Lifetime 31,1% 30,5% 18,1%

Very Long Life Time 22,6% 17,1% 31,4%

53,8% 47,6% 49,5%

Statistical Analysis 1,079 2,215 1,837

Page 226: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

232

VI.5 - Commercialization of Results

i: Question : Has your organization realized, or expects to realize, commercial returns as a result of

exploitation of the project results?

Commercialization of Results - Large Enterprises Results

Total Number of respondents: n=145

Commercialization of Results - SMEs Results

Total Number of respondents: n=106

Page 227: Ex-post Evaluation of FP6 NMP at project level

Annexes

233

ii) Question: What is your estimated actual commercial return (in euro per year) from the project?

Estimated actual commercial return (in Euro per year) from the project – Large Enterprises

Results

Total Number of respondents: n=28

Estimated actual commercial return (in Euro per year) from the project – SMEs Results

Total Number of respondents: n=39

Page 228: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

234

Comments and significant Statistical differences on “Part 6 – Commercialization of Results”

25,5% of respondents from Large enterprises participated in the survey consider that it is likely for

their organization to realize commercial returns as a result of exploitation of the project results while

the percentage of respondents from SMEs climbs up to 40,6%.

VI.6 - Additionality

i) Question: In case you had not received EU funding, would you have continued with your project?

Percentage of Respondents that would have (not) continued with their projects in case they

had not received EU Funding – Large Enterprises Results

Large

Enterprises SMEs Statistical Analysis

20,0% 22,6% 0,503

Not

applicable

14,5% 3,8%

Very

unlikely

11,0% 9,4%

Unlikely 9,0% 13,2%

Neutral 40,0% 33,0%

Likely 18,6% 22,6%

Very likely 6,9% 17,9%

25,5% 40,6% 2,513

Page 229: Ex-post Evaluation of FP6 NMP at project level

Annexes

235

Total Number of respondents: n=145

Percentage of Respondents that would have (not) continued with their projects in case they

had not received EU Funding – SMEs Results

Total Number of respondents: n=106

ii) Question: Source of Funds – In case you had not received EU funding and would have continued

with your project.

Source of Funds – Large Enterprises Results

Total Number of respondents: n=42

Page 230: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

236

Source of Funds – SMEs Results

Total Number of respondents: n=21

iii) Question : Synthesis of Consortium - In case you had not received EU funding and would have

continued with your project

Synthesis of Consortium – Large Enterprises Results

Total Number of respondents: n≈42

Page 231: Ex-post Evaluation of FP6 NMP at project level

Annexes

237

Synthesis of Consortium – SMEs Results

Total Number of respondents: n≈23

iv) Question : Time Scale of Project - In case you had not received EU funding and would have

continued with your project

Time Scale of Project – Large Enterprises Results

Total Number of respondents: n= 41

Page 232: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

238

Time Scale of Project – SMEs Results

Total Number of respondents: n= 23

v) Question : Project Objectives - In case you had not received EU funding and would have continued

with your project

Project Objectives – Large Enterprises Results

Total Number of respondents: n= 42

Page 233: Ex-post Evaluation of FP6 NMP at project level

Annexes

239

Project Objectives – SMEs Results

Total Number of respondents: n=2 3

vi) Question: Project Expectations in terms of net benefits - In case you had not received EU funding

and would have continued with your project

Project Expectations – Large Enterprises Results

Total Number of respondents: n= 42

Page 234: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

240

Project Expectations – SMEs Results

Total Number of respondents: n=2 3

vii) Question: Project Applications in terms of net benefits -In case you had not received EU funding

and would have continued with your project

Range of Project Applications – Large Enterprises Results

Total Number of respondents: n= 42

Page 235: Ex-post Evaluation of FP6 NMP at project level

Annexes

241

Range of Project Applications – SMEs Results

Total Number of respondents: n=2 3

Comments and significant Statistical differences on “Part 7 - Additionality

The majority of the respondents from SMEs (78,3%) stated that they would not proceed with their

NMP project in case of absence of EU funding.

Among the remaining 21,7% of the respondents from SMES that stated that they would proceed

with their project in case of absence of funding,

around half (52,2%) stated that they would continue with internal funds, 39,1% with external

funds and 8,7% with no replacement of EU funds;

around 78,4% with fewer partners in total;

around 56,5% with the same number of academic partners (the other 43,5% with the fewer

number of academic partners);

52,2% with fewer industrial partners, 26,1% with the same number of partners and 21,7%

with more industrial partners;

52,2% with the same number of national partners(but 43,5 with less national partners);

59,1% with the fewer number of international partners

Around half (52,2%) of the respective respondents from SMEs have also stated that they

would elaborate their project in the same time-scale, and 34,8% that they would elaborate it

over a shorter timescale.

43,5% of the respondents from SMEs stated that they would perform it with the same

objectives, 34,8% with less ambitious objectives and 21,7% with more ambitious objectives.

Page 236: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

242

43,5% of the respondents from SMEs stated that they would have similar expectations,

26,1% that they would have greater expectations and 30,4% lower expectations.

Finally, 52,2 % stated that they would expect similar applications from their NMP project and

30,4% that they would expect smaller range of applications.

Similarly, the majority of the respondents from Large Enterprises (69,7%), stated that they would not

proceed with their NMP project in case of absence of EU funding.

Among the remaining 31,3% of the respondents from Large Enterprises that stated that they would

proceed with their project in case of absence of funding,

45,5% stated that they would continue with internal funds, 35,7% with external funds and

23,8% with no replacement of EU funds;

Around 66,7% with less partners in total;

Around 56,1% with less number of academic partners (the other 43,5% with the same

number of academic partners);

41,5% with the same number of industrial partners, 39,1% with less number of partners and

19,5% with more industrial partners;

65,9% with the same number of national partners and 29,3 with less national partners;

51,2% with the same number of international partners and 43,9 with less international

partners.

58,5% of the respective respondents from Large Enterprises have also stated that they

would elaborate their project in the same time-scale, and 22% that they would elaborate it

over a longer timescale and 19,5% that it would elaborate it over a shorter time scale.

45,2% of the respondents stated that they would perform it with the same objectives, 33,3%

with less ambitious objectives and 21,4% with more ambitious objectives.

52,4% of the respondents stated that they would have similar expectations, 19% that they

would have greater expectations and 28,6% lower expectations.

Finally, 54,8% stated that they would expect similar applications from their NMP project and

31% that they would expect smaller range of applications.

Page 237: Ex-post Evaluation of FP6 NMP at project level

Annexes

243

VI.7 - Success Factors and Obstacles

i) Question: Which of the following factors had an important influence on the project achievements?

Success Factors and Obstacles – Large Enterprises Results

Total Number of respondents: n≈145

Page 238: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

244

Success Factors and Obstacles – SMEs Results

Total Number of respondents: n≈106

Page 239: Ex-post Evaluation of FP6 NMP at project level

Annexes

245

ii) Question: To what extent did visibility in any of the media channels listed below affect overall

project impacts?

The visibility of Media Channels and their affect in the Projects Impacts – Large Enterprises

Results

Page 240: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

246

Total Number of respondents: n≈145

Page 241: Ex-post Evaluation of FP6 NMP at project level

Annexes

247

The visibility of Media Channels and their affect in the Projects Impacts – SMEs Results

Total Number of respondents: n≈106

Page 242: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

248

iii) Question: How would you rate the Importance of communicating your project results to the

society?

Importance of communicating your project results – Large Enterprises Results

Total Number of respondents: n=144

Importance of communicating your project results – SMEs Results

Total Number of respondents: n=106

Page 243: Ex-post Evaluation of FP6 NMP at project level

Annexes

249

iv) Question: How would you rate the involvement of end users and target groups in the project?

Significance of the involvement of end users and target groups in the project – Large

Enterprises Results

Total Number of respondents: n=145

Significance of the involvement of end users and target groups in the project – SMEs Results

Total Number of respondents: n=105

Page 244: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

250

v) Question: During project course, did changes in the external environment required

transformation of the objectives and/ or context of your project?

Number of Transformations Occurred – Large Enterprises Results

Total Number of respondents: n=145

Number of Transformations Occurred –SMEs Results

Total Number of respondents: n=106

Page 245: Ex-post Evaluation of FP6 NMP at project level

Annexes

251

vi) Question: Indicate all the changes that occurred (which were significant and did cause

transformation of the project objectives and /or context)?

Changes Occurred – Large Enterprises Results

Total Number of respondents: n=145

Changes Occurred – SMEs Results

Total Number of respondents: n=106

Page 246: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

252

vii) Question: How did the project adapt to the changes?

Projects adaptation to changes – Large Enterprises Results

Total Number of respondents: n≈ 7

Projects adaptation to changes – Large Enterprises Results

Total Number of respondents: n≈ 4

Page 247: Ex-post Evaluation of FP6 NMP at project level

Annexes

253

Comments and significant Statistical differences on “Part 8 - Success Factors and Obstacles”

The vast majority of the respondents coming from SMEs and Large Enterprises identified the same

factors as significant for the successful completion of their projects, with just some small differences

in the order of significance.

SMEs Success Factors:

1. The internal Scientific and Technological Competences of the team

2. The S&T competences of Partners

3. The Project Management skills of the Coordinator

4. The availability of Prior Knowledge or technical inputs

5. The complementary of Partners Goals

6. The ambition of Project Goals

7. The clarity of Project objectives, management of the scope and expectations

8. The knowledge of industrial processes (source, design, plan, make) within partnership

9. The effective communication between partners

10. The availability of adequate resources (manpower, budget and equipment)

Large Enterprises Success Factors:

1. The S&T competences of partners

2. The internal S&T competences of the team

3. The availability of Prior Knowledge or technical inputs

4. The knowledge of industrial processes (source, design, plan, make) within partnership

5. The clarity of Project objectives, management of the scope and expectations

6. The complementary of Partners Goals

7. The Project Management skills of the Coordinator

8. The ambition of project goals

9. The effective communication between partners

10. The availability of adequate resources (manpower, budget and equipment)

No significant statistical differences on the success factors and no significant obstacles were

identified by the web survey respondents.

Page 248: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

254

Annex VII - Detailed analysis of project outputs

This annex presents the statistics related to project outputs as declared by the participants

within the survey.

VII.1 - Organisation type= Higher education (HE)

Descriptive Statistics

N Minimum Maximum Mean

Std. Deviation

Variation Coefficient

Articles 328 ,00 350,00 11,6394 20,916 1,797

Conferences 326 ,00 500,00 13,6420 28,736 2,106

Patents 296 ,00 10,00 ,4074 ,943 2,316

Copyrights, Trademarks 285 ,00 20,00 ,1133 1,016 8,967

Spin-offs 292 ,00 4,00 ,1083 ,374 3,453

Software 293 ,00 18,00 ,6084 1,507 2,477

Standards 285 ,00 5,00 ,1521 ,505 3,317

Valid N (listwise) 269

VII.2 - Organisation type= Research (RES)

Descriptive Statistics

N Minimum Maximum Mean

Std. Deviation

Variation Coefficient

Articles 263 ,00 400,00 10,9447 27,825 2,542

Conferences 269 ,00 600,00 14,7822 44,142 2,986

Patents 242 ,00 10,00 ,6695 1,362 2,035

Copyrights, Trademarks 229 ,00 8,00 ,2141 ,879 4,105

Spin-offs 235 ,00 5,00 ,1535 ,515 3,355

Software 234 ,00 40,00 ,8524 3,216 3,773

Standards 230 ,00 25,00 ,2981 1,481 4,968

Valid N (listwise) 210

Page 249: Ex-post Evaluation of FP6 NMP at project level

Annexes

255

VII.3 - Organisation type= Industry (IND)

Descriptive Statistics

N Minimum Maximum Mean

Std. Deviation

Variation Coefficient

Articles 324 ,00 100,00 3,7782 8,575 2,270

Conferences 345 ,00 100,00 4,1671 8,240 1,977

Patents 327 ,00 25,00 ,5747 2,157 3,753

Copyrights, Trademarks 311 ,00 5,00 ,1739 ,705 4,058

Spin-offs 321 ,00 5,00 ,1221 ,528 4,329

Software 324 ,00 10,00 ,6144 1,270 2,068

Standards 312 ,00 19,00 ,3585 1,305 3,639

Valid N (listwise) 286

VII.4 - Organisation type= Large enterprises (LE)

Descriptive Statistics

N Minimum Maximum Mean

Std. Deviation

Variation Coefficient

Articles 113 0 100 5,46 12,467 2,283

Conferences 115 0 600 10,56 56,548 5,357

Patents 110 0 6 ,55 1,186 2,174

Copyrights, Trademarks 103 0 5 ,13 ,710 5,622

Spin-offs 107 0 2 ,06 ,269 4,795

Software 109 0 40 ,93 4,015 4,334

Standards 107 0 19 ,47 1,959 4,191

Valid N (listwise) 97

VII.5 - Organisation type= SMEs (SME)

Descriptive Statistics

N Minimum Maximum Mean

Std. Deviation

Variation Coefficient

Articles 112 ,00 32,00 3,3807 5,062 1,497

Conferences 121 ,00 200,00 6,9115 24,876 3,599

Patents 117 ,00 25,00 ,6829 3,306 4,841

Copyrights, Trademarks 109 ,00 3,00 ,1601 ,553 3,451

Spin-offs 113 ,00 5,00 ,2234 ,782 3,497

Software 114 ,00 6,00 ,5579 1,172 2,101

Standards 105 ,00 3,00 ,3481 ,699 2,007

Valid N (listwise) 96

Page 250: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

256

Page 251: Ex-post Evaluation of FP6 NMP at project level

Annexes

257

Annex VIII - Survey factorial analysis

The following annex presents the results of factorial analysis of impacts on participants R&D capabilities

Correlation Matrixa

Q_5_1a Q_5_1b Q_5_1c Q_5_1d Q_5_1e Q_5_1f Q_5_1g Q_5_1h Q_5_1i Q_5_1j Q_5_1k

Correlation Q_5_1a 1,000 ,759 ,442 ,463 ,345 ,475 ,440 ,471 ,279 ,428 ,026

Q_5_1b ,759 1,000 ,454 ,503 ,411 ,471 ,418 ,453 ,319 ,387 ,063

Q_5_1c ,442 ,454 1,000 ,579 ,521 ,362 ,351 ,342 ,285 ,350 ,059

Q_5_1d ,463 ,503 ,579 1,000 ,643 ,449 ,412 ,421 ,370 ,413 ,173

Q_5_1e ,345 ,411 ,521 ,643 1,000 ,404 ,364 ,375 ,419 ,383 ,185

Q_5_1f ,475 ,471 ,362 ,449 ,404 1,000 ,537 ,573 ,406 ,439 ,084

Q_5_1g ,440 ,418 ,351 ,412 ,364 ,537 1,000 ,727 ,450 ,548 ,044

Q_5_1h ,471 ,453 ,342 ,421 ,375 ,573 ,727 1,000 ,448 ,554 ,071

Q_5_1i ,279 ,319 ,285 ,370 ,419 ,406 ,450 ,448 1,000 ,436 ,107

Q_5_1j ,428 ,387 ,350 ,413 ,383 ,439 ,548 ,554 ,436 1,000 ,145

Q_5_1k ,026 ,063 ,059 ,173 ,185 ,084 ,044 ,071 ,107 ,145 1,000

Sig. (1-tailed) Q_5_1a ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,205

Q_5_1b ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,025

Q_5_1c ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,033

Q_5_1d ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_1e ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_1f ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,004

Q_5_1g ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,086

Q_5_1h ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,013

Q_5_1i ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_1j ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_1k ,205 ,025 ,033 ,000 ,000 ,004 ,086 ,013 ,000 ,000

a. Determinant = ,007

Page 252: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

258

KMO and Bartlett's Test

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. ,884

Bartlett's Test of Sphericity Approx. Chi-Square 4748,715

df 55

Sig. ,000

Communalities

Initial Extraction

Q_5_1a 1,000 ,715

Q_5_1b 1,000 ,712

Q_5_1c 1,000 ,631

Q_5_1d 1,000 ,717

Q_5_1e 1,000 ,666

Q_5_1f 1,000 ,563

Q_5_1g 1,000 ,744

Q_5_1h 1,000 ,758

Q_5_1i 1,000 ,512

Q_5_1j 1,000 ,584

Q_5_1k 1,000 ,675

Extraction Method: Principal Component Analysis.

Total Variance Explained

Component

Initial Eigenvalues Extraction Sums of Squared

Loadings Rotation Sums of Squared

Loadings

Total % of

Variance Cumulative

% Total % of

Variance Cumulative

% Total % of

Variance Cumulative

%

1 5,056 45,960 45,960 5,056 45,960 45,960 3,155 28,678 28,678

2 1,146 10,414 56,374 1,146 10,414 56,374 2,956 26,870 55,548

3 1,076 9,783 66,157 1,076 9,783 66,157 1,167 10,609 66,157

4 ,836 7,602 73,759

5 ,603 5,482 79,241

6 ,558 5,076 84,317

7 ,449 4,081 88,399

8 ,439 3,991 92,390

9 ,337 3,061 95,450

10 ,269 2,448 97,899

11 ,231 2,101 100,000

Extraction Method: Principal Component Analysis.

Page 253: Ex-post Evaluation of FP6 NMP at project level

Annexes

259

Component Matrixa

Component

1 2 3

Q_5_1a ,723 -,045 -,437

Q_5_1b ,733 ,052 -,414

Q_5_1c ,654 ,372 -,255

Q_5_1d ,744 ,399 -,066

Q_5_1e ,684 ,439 ,080

Q_5_1f ,725 -,190 ,040

Q_5_1g ,744 -,400 ,175

Q_5_1h ,763 -,386 ,167

Q_5_1i ,611 -,112 ,356

Q_5_1j ,698 -,189 ,248

Q_5_1k ,166 ,508 ,625

Extraction Method: Principal Component

Analysis; a. 3 components extracted

Rotated Component Matrixa

Component

1 2 3

Q_5_1a ,379 ,707 -,268

Q_5_1b ,339 ,750 -,186

Q_5_1c ,156 ,768 ,130

Q_5_1d ,277 ,741 ,301

Q_5_1e ,267 ,641 ,429

Q_5_1f ,647 ,381 -,002

Q_5_1g ,834 ,218 -,034

Q_5_1h ,836 ,241 -,029

Q_5_1i ,640 ,171 ,271

Q_5_1j ,706 ,251 ,151

Q_5_1k ,062 ,028 ,819

Extraction Method: Principal Component Analysis; Rotation Method: Varimax with Kaiser Normalization; a. Rotation converged in 5 iterations

Page 254: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

260

Component Transformation Matrix

Component 1 2 3

1 ,719 ,684 ,125

2 -,582 ,493 ,647

3 ,381 -,537 ,752

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.

Page 255: Ex-post Evaluation of FP6 NMP at project level

Annexes

261

VIII.1 - Results of factorial analysis of impacts on participant organisations Correlation Matrix

a

Q_5_2a Q_5_2b Q_5_2c Q_5_2d Q_5_2e Q_5_2f Q_5_2g Q_5_2h Q_5_2i Q_5_2j Q_5_2other

Correlation Q_5_2a 1,000 ,458 ,590 ,303 ,247 ,166 ,216 ,230 ,188 ,484 ,024

Q_5_2b ,458 1,000 ,618 ,573 ,494 ,472 ,495 ,460 ,281 ,313 ,035

Q_5_2c ,590 ,618 1,000 ,531 ,447 ,397 ,380 ,342 ,269 ,405 ,075

Q_5_2d ,303 ,573 ,531 1,000 ,713 ,726 ,687 ,579 ,422 ,215 ,081

Q_5_2e ,247 ,494 ,447 ,713 1,000 ,703 ,582 ,533 ,423 ,174 ,117

Q_5_2f ,166 ,472 ,397 ,726 ,703 1,000 ,781 ,654 ,484 ,121 ,128

Q_5_2g ,216 ,495 ,380 ,687 ,582 ,781 1,000 ,719 ,433 ,191 ,091

Q_5_2h ,230 ,460 ,342 ,579 ,533 ,654 ,719 1,000 ,436 ,272 ,147

Q_5_2i ,188 ,281 ,269 ,422 ,423 ,484 ,433 ,436 1,000 ,282 ,163

Q_5_2j ,484 ,313 ,405 ,215 ,174 ,121 ,191 ,272 ,282 1,000 ,083

Q_5_2other ,024 ,035 ,075 ,081 ,117 ,128 ,091 ,147 ,163 ,083 1,000

Sig. (1-tailed) Q_5_2a ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,232

Q_5_2b ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,139

Q_5_2c ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,010

Q_5_2d ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,006

Q_5_2e ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_2f ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_2g ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,002

Q_5_2h ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_2i ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_2j ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,005

Q_5_2other ,232 ,139 ,010 ,006 ,000 ,000 ,002 ,000 ,000 ,005

a. Determinant = ,003

Page 256: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

262

KMO and Bartlett's Test

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. ,886

Bartlett's Test of Sphericity Approx. Chi-Square 5421,832

df 55

Sig. ,000

Communalities

Initial Extraction

Q_5_2a 1,000 ,742

Q_5_2b 1,000 ,642

Q_5_2c 1,000 ,707

Q_5_2d 1,000 ,762

Q_5_2e 1,000 ,668

Q_5_2f 1,000 ,833

Q_5_2g 1,000 ,763

Q_5_2h 1,000 ,643

Q_5_2i 1,000 ,474

Q_5_2j 1,000 ,626

Q_5_2other 1,000 ,806

Extraction Method: Principal Component Analysis.

Total Variance Explained

Component

Initial Eigenvalues Extraction Sums of Squared

Loadings Rotation Sums of Squared

Loadings

Total % of

Variance Cumulative

% Total % of

Variance Cumulative

% Total % of

Variance Cumulative

%

1 5,068 46,069 46,069 5,068 46,069 46,069 4,204 38,220 38,220

2 1,556 14,143 60,211 1,556 14,143 60,211 2,362 21,472 59,692

3 1,042 9,475 69,687 1,042 9,475 69,687 1,099 9,995 69,687

4 ,790 7,182 76,869

5 ,594 5,403 82,271

6 ,458 4,160 86,431

7 ,426 3,875 90,306

8 ,341 3,104 93,411

9 ,297 2,703 96,114

10 ,248 2,256 98,370

11 ,179 1,630 100,000

Extraction Method: Principal Component Analysis.

Page 257: Ex-post Evaluation of FP6 NMP at project level

Annexes

263

Component Matrixa

Component

1 2 3

Q_5_2a ,491 ,707 -,014

Q_5_2b ,731 ,264 -,197

Q_5_2c ,684 ,479 -,091

Q_5_2d ,848 -,147 -,147

Q_5_2e ,786 -,208 -,080

Q_5_2f ,827 -,383 -,058

Q_5_2g ,815 -,305 -,074

Q_5_2h ,766 -,227 ,071

Q_5_2i ,591 -,135 ,326

Q_5_2j ,420 ,609 ,279

Q_5_2other ,166 -,101 ,877

Extraction Method: Principal Component Analysis.

a. 3 components extracted.

Rotated Component Matrixa

Component

1 2 3

Q_5_2a ,096 ,856 -,022

Q_5_2b ,540 ,576 -,135

Q_5_2c ,383 ,746 -,056

Q_5_2d ,828 ,273 -,034

Q_5_2e ,794 ,192 ,030

Q_5_2f ,908 ,059 ,072

Q_5_2g ,864 ,121 ,048

Q_5_2h ,763 ,170 ,178

Q_5_2i ,531 ,173 ,403

Q_5_2j ,038 ,743 ,269

Q_5_2other ,065 ,013 ,895

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

a. Rotation converged in 4 iterations.

Page 258: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

264

Component Transformation Matrix

Component 1 2 3

1 ,870 ,479 ,115

2 -,471 ,877 -,091

3 -,144 ,025 ,989

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Page 259: Ex-post Evaluation of FP6 NMP at project level

Annexes

265

VIII.2 - Results of factorial analysis of impacts on the broader environment

Correlation Matrixa

Q_5_3a

Q_5_3b

Q_5_3c

Q_5_3d

Q_5_3e

Q_5_3f

Q_5_3g

Q_5_3h

Q_5_3i

Q_5_3j

Q_5_3k

Q_5_3l

Q_5_3m

Q_5_3n

Q_5_3o

Q_5_3p

Q_5_3q

Correlation

Q_5_3a 1,000 ,467 ,407 ,092 ,239 ,266 ,336 ,368 ,304 ,274 ,262 ,201 ,227 ,220 ,206 ,298 ,333

Q_5_3b ,467 1,000 ,719 ,204 ,202 ,095 ,286 ,284 ,239 ,220 ,269 ,204 ,148 ,243 ,143 ,380 ,242

Q_5_3c ,407 ,719 1,000 ,320 ,252 ,150 ,274 ,284 ,274 ,235 ,313 ,173 ,138 ,238 ,198 ,362 ,245

Q_5_3d ,092 ,204 ,320 1,000 ,557 ,429 ,259 ,197 ,335 ,349 ,454 ,344 ,314 ,371 ,360 ,339 ,207

Q_5_3e ,239 ,202 ,252 ,557 1,000 ,614 ,357 ,241 ,335 ,422 ,402 ,367 ,384 ,339 ,436 ,318 ,323

Q_5_3f ,266 ,095 ,150 ,429 ,614 1,000 ,441 ,224 ,353 ,421 ,412 ,402 ,446 ,361 ,465 ,303 ,342

Q_5_3g ,336 ,286 ,274 ,259 ,357 ,441 1,000 ,377 ,358 ,343 ,364 ,283 ,309 ,302 ,321 ,401 ,333

Q_5_3h ,368 ,284 ,284 ,197 ,241 ,224 ,377 1,000 ,722 ,331 ,289 ,165 ,205 ,162 ,255 ,250 ,329

Q_5_3i ,304 ,239 ,274 ,335 ,335 ,353 ,358 ,722 1,000 ,476 ,401 ,272 ,321 ,257 ,342 ,334 ,326

Q_5_3j ,274 ,220 ,235 ,349 ,422 ,421 ,343 ,331 ,476 1,000 ,540 ,477 ,470 ,438 ,466 ,364 ,347

Q_5_3k ,262 ,269 ,313 ,454 ,402 ,412 ,364 ,289 ,401 ,540 1,000 ,470 ,446 ,508 ,444 ,412 ,352

Q_5_3l ,201 ,204 ,173 ,344 ,367 ,402 ,283 ,165 ,272 ,477 ,470 1,000 ,755 ,720 ,588 ,399 ,274

Q_5_3m

,227 ,148 ,138 ,314 ,384 ,446 ,309 ,205 ,321 ,470 ,446 ,755 1,000 ,713 ,678 ,349 ,330

Q_5_3n ,220 ,243 ,238 ,371 ,339 ,361 ,302 ,162 ,257 ,438 ,508 ,720 ,713 1,000 ,630 ,410 ,296

Q_5_3o ,206 ,143 ,198 ,360 ,436 ,465 ,321 ,255 ,342 ,466 ,444 ,588 ,678 ,630 1,000 ,305 ,305

Q_5_3p ,298 ,380 ,362 ,339 ,318 ,303 ,401 ,250 ,334 ,364 ,412 ,399 ,349 ,410 ,305 1,000 ,335

Q_5_3q ,333 ,242 ,245 ,207 ,323 ,342 ,333 ,329 ,326 ,347 ,352 ,274 ,330 ,296 ,305 ,335 1,000

Sig. (1-tailed)

Q_5_3a ,000 ,000 ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3b ,000 ,000 ,000 ,000 ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3c ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3d ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3e ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Page 260: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

266

Q_5_3f ,000 ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3g ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3h ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3i ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3j ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3k ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3l ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3m

,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

,000 ,000 ,000 ,000

Q_5_3n ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3o ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3p ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_5_3q ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

a. Determinant = ,000

Page 261: Ex-post Evaluation of FP6 NMP at project level

Annexes

267

KMO and Bartlett's Test

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. ,887

Bartlett's Test of Sphericity Approx. Chi-Square 9246,679

df 136

Sig. ,000

Communalities

Initial Extraction

Q_5_3a 1,000 ,510

Q_5_3b 1,000 ,818

Q_5_3c 1,000 ,778

Q_5_3d 1,000 ,685

Q_5_3e 1,000 ,727

Q_5_3f 1,000 ,645

Q_5_3g 1,000 ,417

Q_5_3h 1,000 ,787

Q_5_3i 1,000 ,742

Q_5_3j 1,000 ,519

Q_5_3k 1,000 ,514

Q_5_3l 1,000 ,781

Q_5_3m 1,000 ,808

Q_5_3n 1,000 ,787

Q_5_3o 1,000 ,651

Q_5_3p 1,000 ,446

Q_5_3q 1,000 ,357

Extraction Method: Principal Component Analysis.

Page 262: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

268

Total Variance Explained

Component

Initial Eigenvalues Extraction Sums of Squared

Loadings Rotation Sums of Squared Loadings

Total % of

Variance Cumulative

% Total % of

Variance Cumulative

% Total % of

Variance Cumulative

%

1 6,587 38,745 38,745 6,587 38,745 38,745 3,618 21,283 21,283

2 1,968 11,576 50,321 1,968 11,576 50,321 2,542 14,955 36,238

3 1,310 7,706 58,027 1,310 7,706 58,027 2,463 14,491 50,729

4 1,109 6,522 64,549 1,109 6,522 64,549 2,349 13,820 64,549

5 ,889 5,232 69,781

6 ,731 4,298 74,079

7 ,686 4,033 78,112

8 ,610 3,587 81,699

9 ,538 3,166 84,866

10 ,477 2,808 87,674

11 ,398 2,343 90,017

12 ,384 2,257 92,274

13 ,345 2,031 94,306

14 ,265 1,560 95,866

15 ,252 1,482 97,348

16 ,240 1,412 98,760

17 ,211 1,240 100,000

Extraction Method: Principal Component Analysis.

Component Matrixa

Component

1 2 3 4

Q_5_3a ,489 ,472 ,114 -,189

Q_5_3b ,464 ,606 ,479 ,069

Q_5_3c ,490 ,575 ,408 ,205

Q_5_3d ,587 -,087 -,082 ,572

Q_5_3e ,652 -,096 -,224 ,492

Q_5_3f ,653 -,189 -,301 ,304

Q_5_3g ,593 ,191 -,169 ,021

Q_5_3h ,514 ,436 -,457 -,352

Q_5_3i ,621 ,269 -,476 -,239

Q_5_3j ,697 -,083 -,142 -,085

Q_5_3k ,711 -,062 ,011 ,074

Q_5_3l ,705 -,419 ,269 -,190

Q_5_3m ,720 -,443 ,164 -,258

Q_5_3n ,713 -,369 ,337 -,172

Q_5_3o ,706 -,365 ,048 -,134

Q_5_3p ,617 ,152 ,197 ,058

Q_5_3q ,550 ,149 -,123 -,129

Extraction Method: Principal Component Analysis.

Page 263: Ex-post Evaluation of FP6 NMP at project level

Annexes

269

Total Variance Explained

Component

Initial Eigenvalues Extraction Sums of Squared

Loadings Rotation Sums of Squared Loadings

Total % of

Variance Cumulative

% Total % of

Variance Cumulative

% Total % of

Variance Cumulative

%

1 6,587 38,745 38,745 6,587 38,745 38,745 3,618 21,283 21,283

2 1,968 11,576 50,321 1,968 11,576 50,321 2,542 14,955 36,238

3 1,310 7,706 58,027 1,310 7,706 58,027 2,463 14,491 50,729

4 1,109 6,522 64,549 1,109 6,522 64,549 2,349 13,820 64,549

5 ,889 5,232 69,781

6 ,731 4,298 74,079

7 ,686 4,033 78,112

8 ,610 3,587 81,699

9 ,538 3,166 84,866

10 ,477 2,808 87,674

11 ,398 2,343 90,017

12 ,384 2,257 92,274

13 ,345 2,031 94,306

14 ,265 1,560 95,866

15 ,252 1,482 97,348

16 ,240 1,412 98,760

17 ,211 1,240 100,000

a. 4 components extracted.

Page 264: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

270

Rotated Component Matrixa

Component

1 2 3 4

Q_5_3a ,139 ,435 -,018 ,548

Q_5_3b ,080 ,128 ,043 ,890

Q_5_3c ,041 ,110 ,191 ,853

Q_5_3d ,187 ,031 ,784 ,188

Q_5_3e ,212 ,185 ,798 ,108

Q_5_3f ,308 ,282 ,685 -,028

Q_5_3g ,195 ,454 ,331 ,251

Q_5_3h ,035 ,871 ,046 ,161

Q_5_3i ,153 ,814 ,220 ,091

Q_5_3j ,470 ,423 ,330 ,105

Q_5_3k ,463 ,268 ,413 ,240

Q_5_3l ,855 ,077 ,173 ,116

Q_5_3m ,866 ,173 ,162 ,031

Q_5_3n ,848 ,050 ,163 ,197

Q_5_3o ,726 ,215 ,279 ,026

Q_5_3p ,348 ,200 ,265 ,463

Q_5_3q ,263 ,456 ,182 ,214

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

a. Rotation converged in 5 iterations.

Component Transformation Matrix

Component 1 2 3 4

1 ,632 ,470 ,490 ,374

2 -,589 ,406 -,151 ,683

3 ,352 -,636 -,310 ,613

4 -,361 -,458 ,801 ,137

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Page 265: Ex-post Evaluation of FP6 NMP at project level

Annexes

271

VIII.3 - Results of factorial analysis of impacts on success factors and obstacles

Q_8_1a Q_8_1b Q_8_1c Q_8_1d Q_8_1e Q_8_1f Q_8_1g Q_8_1h Q_8_1i Q_8_1j Q_8_1k Q_8_1l

Correlation Q_8_1a 1,000 ,292 ,104 ,189 ,349 ,211 ,239 ,261 ,137 ,146 ,085 ,141

Q_8_1b ,292 1,000 ,260 ,327 ,222 ,289 ,231 ,255 ,198 ,216 ,170 ,156

Q_8_1c ,104 ,260 1,000 ,176 ,097 ,088 ,127 ,173 ,100 ,082 ,141 ,071

Q_8_1d ,189 ,327 ,176 1,000 ,193 ,134 ,119 ,157 ,281 ,389 ,211 ,186

Q_8_1e ,349 ,222 ,097 ,193 1,000 ,372 ,524 ,497 ,217 ,275 ,202 ,194

Q_8_1f ,211 ,289 ,088 ,134 ,372 1,000 ,434 ,442 ,264 ,268 ,166 ,161

Q_8_1g ,239 ,231 ,127 ,119 ,524 ,434 1,000 ,572 ,277 ,358 ,249 ,223

Q_8_1h ,261 ,255 ,173 ,157 ,497 ,442 ,572 1,000 ,286 ,328 ,177 ,218

Q_8_1i ,137 ,198 ,100 ,281 ,217 ,264 ,277 ,286 1,000 ,517 ,280 ,245

Q_8_1j ,146 ,216 ,082 ,389 ,275 ,268 ,358 ,328 ,517 1,000 ,341 ,283

Q_8_1k ,085 ,170 ,141 ,211 ,202 ,166 ,249 ,177 ,280 ,341 1,000 ,496

Q_8_1l ,141 ,156 ,071 ,186 ,194 ,161 ,223 ,218 ,245 ,283 ,496 1,000

Q_8_1m ,291 ,294 ,121 ,168 ,394 ,470 ,400 ,437 ,252 ,291 ,296 ,316

Q_8_1n ,346 ,284 ,089 ,189 ,429 ,370 ,415 ,421 ,250 ,282 ,220 ,260

Q_8_1o ,300 ,230 ,063 ,131 ,394 ,361 ,396 ,393 ,253 ,263 ,225 ,215

Q_8_1p ,308 ,207 ,098 ,133 ,408 ,297 ,379 ,326 ,150 ,228 ,158 ,223

Q_8_1q ,205 ,155 ,167 ,111 ,187 ,101 ,189 ,210 ,188 ,209 ,073 ,056

Q_8_1r ,243 ,149 ,034 ,176 ,310 ,211 ,287 ,247 ,274 ,266 ,126 ,133

Q_8_1s ,090 ,264 ,104 ,320 ,159 ,215 ,266 ,220 ,326 ,389 ,361 ,460

Q_8_1t ,174 ,290 ,180 ,232 ,290 ,348 ,331 ,286 ,261 ,257 ,215 ,187

Q_8_1u ,195 ,302 ,208 ,229 ,322 ,364 ,319 ,290 ,252 ,272 ,236 ,222

Q_8_1v ,118 ,233 ,166 ,237 ,233 ,260 ,292 ,256 ,236 ,254 ,230 ,215

Q_8_1w ,009 ,128 ,123 ,251 ,069 ,097 ,115 ,103 ,234 ,238 ,249 ,242

Sig. (1-tailed) Q_8_1a ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,002 ,000

Q_8_1b ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1c ,000 ,000 ,000 ,001 ,002 ,000 ,000 ,000 ,003 ,000 ,009

Q_8_1d ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1e ,000 ,000 ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Page 266: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

272

Q_8_1f ,000 ,000 ,002 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1g ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1h ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1i ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1j ,000 ,000 ,003 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1k ,002 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1l ,000 ,000 ,009 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1m ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1n ,000 ,000 ,002 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1o ,000 ,000 ,018 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1p ,000 ,000 ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1q ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,008 ,031

Q_8_1r ,000 ,000 ,125 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1s ,001 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1t ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1u ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1v ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1w ,384 ,000 ,000 ,000 ,011 ,001 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1m Q_8_1n Q_8_1o Q_8_1p Q_8_1q Q_8_1r Q_8_1s Q_8_1t Q_8_1u Q_8_1v Q_8_1w

Correlation Q_8_1a ,291 ,346 ,300 ,308 ,205 ,243 ,090 ,174 ,195 ,118 ,009

Q_8_1b ,294 ,284 ,230 ,207 ,155 ,149 ,264 ,290 ,302 ,233 ,128

Q_8_1c ,121 ,089 ,063 ,098 ,167 ,034 ,104 ,180 ,208 ,166 ,123

Q_8_1d ,168 ,189 ,131 ,133 ,111 ,176 ,320 ,232 ,229 ,237 ,251

Q_8_1e ,394 ,429 ,394 ,408 ,187 ,310 ,159 ,290 ,322 ,233 ,069

Q_8_1f ,470 ,370 ,361 ,297 ,101 ,211 ,215 ,348 ,364 ,260 ,097

Q_8_1g ,400 ,415 ,396 ,379 ,189 ,287 ,266 ,331 ,319 ,292 ,115

Q_8_1h ,437 ,421 ,393 ,326 ,210 ,247 ,220 ,286 ,290 ,256 ,103

Q_8_1i ,252 ,250 ,253 ,150 ,188 ,274 ,326 ,261 ,252 ,236 ,234

Q_8_1j ,291 ,282 ,263 ,228 ,209 ,266 ,389 ,257 ,272 ,254 ,238

Q_8_1k ,296 ,220 ,225 ,158 ,073 ,126 ,361 ,215 ,236 ,230 ,249

Q_8_1l ,316 ,260 ,215 ,223 ,056 ,133 ,460 ,187 ,222 ,215 ,242

Page 267: Ex-post Evaluation of FP6 NMP at project level

Annexes

273

Q_8_1m 1,000 ,641 ,485 ,406 ,172 ,207 ,252 ,342 ,351 ,308 ,142

Q_8_1n ,641 1,000 ,538 ,411 ,228 ,269 ,228 ,343 ,349 ,275 ,124

Q_8_1o ,485 ,538 1,000 ,481 ,268 ,318 ,219 ,293 ,316 ,287 ,158

Q_8_1p ,406 ,411 ,481 1,000 ,202 ,344 ,200 ,278 ,289 ,261 ,070

Q_8_1q ,172 ,228 ,268 ,202 1,000 ,267 ,153 ,184 ,148 ,113 ,278

Q_8_1r ,207 ,269 ,318 ,344 ,267 1,000 ,228 ,198 ,213 ,180 ,088

Q_8_1s ,252 ,228 ,219 ,200 ,153 ,228 1,000 ,338 ,334 ,305 ,356

Q_8_1t ,342 ,343 ,293 ,278 ,184 ,198 ,338 1,000 ,776 ,624 ,221

Q_8_1u ,351 ,349 ,316 ,289 ,148 ,213 ,334 ,776 1,000 ,575 ,198

Q_8_1v ,308 ,275 ,287 ,261 ,113 ,180 ,305 ,624 ,575 1,000 ,264

Q_8_1w ,142 ,124 ,158 ,070 ,278 ,088 ,356 ,221 ,198 ,264 1,000

Sig. (1-tailed)

Q_8_1a ,000 ,000 ,000 ,000 ,000 ,000 ,001 ,000 ,000 ,000 ,384

Q_8_1b ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1c ,000 ,002 ,018 ,001 ,000 ,125 ,000 ,000 ,000 ,000 ,000

Q_8_1d ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1e ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,011

Q_8_1f ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,001

Q_8_1g ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1h ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1i ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1j ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1k ,000 ,000 ,000 ,000 ,008 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1l ,000 ,000 ,000 ,000 ,031 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1m ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1n ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1o ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1p ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,010

Q_8_1q ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1r ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,002

Page 268: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

274

Q_8_1s ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1t ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1u ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1v ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000 ,000

Q_8_1w ,000 ,000 ,000 ,010 ,000 ,002 ,000 ,000 ,000 ,000

Page 269: Ex-post Evaluation of FP6 NMP at project level

Annexes

275

KMO and Bartlett's Test

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. ,895

Bartlett's Test of Sphericity Approx. Chi-Square 8690,068

df 253

Sig. ,000

Communalities

Initial Extraction

Q_8_1a 1,000 ,480

Q_8_1b 1,000 ,585

Q_8_1c 1,000 ,501

Q_8_1d 1,000 ,528

Q_8_1e 1,000 ,534

Q_8_1f 1,000 ,511

Q_8_1g 1,000 ,576

Q_8_1h 1,000 ,568

Q_8_1i 1,000 ,591

Q_8_1j 1,000 ,663

Q_8_1k 1,000 ,603

Q_8_1l 1,000 ,700

Q_8_1m 1,000 ,616

Q_8_1n 1,000 ,598

Q_8_1o 1,000 ,587

Q_8_1p 1,000 ,506

Q_8_1q 1,000 ,653

Q_8_1r 1,000 ,486

Q_8_1s 1,000 ,555

Q_8_1t 1,000 ,821

Q_8_1u 1,000 ,779

Q_8_1v 1,000 ,673

Q_8_1w 1,000 ,517

Extraction Method: Principal Component Analysis.

Total Variance Explained

Component

Initial Eigenvalues Extraction Sums of Squared

Loadings Rotation Sums of Squared

Loadings

Total % of

Variance Cumulative

% Total % of

Variance Cumulative

% Total % of

Variance Cumulative

%

1 6,799 29,562 29,562 6,799 29,562 29,562 4,184 18,193 18,193

2 1,938 8,427 37,989 1,938 8,427 37,989 2,425 10,543 28,736

3 1,480 6,435 44,425 1,480 6,435 44,425 2,092 9,096 37,833

4 1,257 5,465 49,890 1,257 5,465 49,890 1,947 8,465 46,297

5 1,105 4,806 54,695 1,105 4,806 54,695 1,554 6,756 53,054

6 1,053 4,577 59,272 1,053 4,577 59,272 1,430 6,219 59,272

7 ,966 4,199 63,471

Page 270: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

276

8 ,854 3,712 67,183

9 ,741 3,221 70,404

10 ,709 3,082 73,487

11 ,694 3,019 76,506

12 ,590 2,565 79,071

13 ,578 2,512 81,583

14 ,571 2,482 84,065

15 ,541 2,351 86,417

16 ,479 2,081 88,497

17 ,473 2,056 90,553

18 ,466 2,025 92,578

19 ,403 1,754 94,332

20 ,394 1,715 96,046

21 ,383 1,667 97,714

22 ,314 1,364 99,078

23 ,212 ,922 100,000

Extraction Method: Principal Component Analysis.

Page 271: Ex-post Evaluation of FP6 NMP at project level

Annexes

277

Component Matrix

a

Component

1 2 3 4 5 6

Q_8_1a ,433 -,323 ,070 ,309 ,245 ,168

Q_8_1b ,484 ,063 -,134 ,320 ,474 -,039

Q_8_1c ,259 ,154 -,222 ,401 ,443 ,062

Q_8_1d ,418 ,371 ,093 ,350 ,220 -,191

Q_8_1e ,618 -,357 ,068 -,010 ,020 -,139

Q_8_1f ,587 -,222 -,099 -,163 ,070 -,277

Q_8_1g ,655 -,251 ,075 -,126 -,042 -,248

Q_8_1h ,636 -,283 ,081 -,036 ,064 -,268

Q_8_1i ,516 ,265 ,274 ,089 -,176 -,376

Q_8_1j ,575 ,272 ,328 ,063 -,117 -,364

Q_8_1k ,462 ,377 ,279 -,319 ,203 ,163

Q_8_1l ,466 ,330 ,322 -,384 ,203 ,286

Q_8_1m ,680 -,228 ,030 -,216 ,168 ,160

Q_8_1n ,675 -,298 ,047 -,089 ,067 ,198

Q_8_1o ,638 -,298 ,084 -,043 -,138 ,253

Q_8_1p ,572 -,324 ,037 -,007 -,088 ,254

Q_8_1q ,362 -,027 ,126 ,544 -,339 ,309

Q_8_1r ,461 -,141 ,203 ,252 -,385 -,011

Q_8_1s ,536 ,478 ,156 -,101 -,024 ,069

Q_8_1t ,637 ,188 -,590 -,066 -,167 -,016

Q_8_1u ,646 ,167 -,562 -,083 -,106 -,013

Q_8_1v ,570 ,247 -,494 -,110 -,173 ,026

Q_8_1w ,345 ,495 ,090 ,147 -,215 ,278

Extraction Method: Principal Component Analysis.

a. 6 components extracted.

Rotated Component Matrix

a

Component

1 2 3 4 5 6

Q_8_1a ,500 -,089 -,016 -,053 ,409 ,228

Q_8_1b ,257 ,147 ,091 ,120 ,689 -,005

Q_8_1c ,016 ,143 ,033 -,021 ,690 ,050

Q_8_1d -,008 ,098 ,194 ,452 ,513 ,119

Q_8_1e ,678 ,099 -,013 ,232 ,093 ,045

Q_8_1f ,572 ,272 ,006 ,271 ,079 -,176

Q_8_1g ,642 ,178 ,052 ,359 ,001 -,031

Q_8_1h ,645 ,113 ,005 ,350 ,121 -,050

Q_8_1i ,158 ,119 ,188 ,704 ,044 ,139

Q_8_1j ,210 ,091 ,262 ,722 ,078 ,120

Q_8_1k ,164 ,074 ,729 ,172 ,083 -,052

Q_8_1l ,219 ,036 ,803 ,072 ,031 -,023

Q_8_1m ,684 ,180 ,317 -,007 ,123 ,001

Q_8_1n ,701 ,150 ,221 -,012 ,112 ,150

Q_8_1o ,655 ,154 ,182 ,000 -,023 ,318

Q_8_1p ,621 ,136 ,123 -,056 ,022 ,287

Q_8_1q ,165 ,046 -,025 ,094 ,151 ,769

Page 272: Ex-post Evaluation of FP6 NMP at project level

Ex-post evaluation of NMP (FP6) - Project level

278

Q_8_1r ,358 ,057 -,043 ,327 -,069 ,491

Q_8_1s ,070 ,263 ,576 ,338 ,106 ,154

Q_8_1t ,238 ,850 ,078 ,108 ,137 ,078

Q_8_1u ,270 ,811 ,103 ,097 ,164 ,042

Q_8_1v ,170 ,771 ,161 ,101 ,082 ,082

Q_8_1w -,133 ,250 ,433 ,174 ,087 ,460

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

a. Rotation converged in 10 iterations.

Component Transformation Matrix

Component 1 2 3 4 5 6

1 ,685 ,420 ,339 ,359 ,252 ,217

2 -,693 ,294 ,553 ,316 ,159 ,044

3 ,110 -,796 ,386 ,369 -,181 ,190

4 -,175 -,160 -,443 ,155 ,600 ,602

5 ,087 -,278 ,218 -,217 ,720 -,550

6 ,029 -,019 ,431 -,751 ,014 ,499

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Page 273: Ex-post Evaluation of FP6 NMP at project level

European Commission

EUR 24935 EN - Ex Post Evaluation of FP6 (NMP). Project Level

Luxembourg: Publications Office of the European Union

2011 — 272 pp. — 21,0 x 29,7 cm ISBN 978-92-79-21152-2doi 10.2777/74830

Page 274: Ex-post Evaluation of FP6 NMP at project level

HOw tO ObtAIN EU PUblICAtIONS

Free publications:

• via EU Bookshop (http://bookshop.europa.eu);

• at the European Union’s representations or delegations. You can obtain their contact details on the Internet (http://ec.europa.eu) or by sending a fax to +352 2929-42758.

Priced publications:

• via EU Bookshop (http://bookshop.europa.eu).

Priced subscriptions (e.g. annual series of the Official Journal of the European Union and reports of cases before the Court of Justice of the European Union):

• via one of the sales agents of the Publications Office of the European Union (http://publications.europa.eu/others/agents/index_en.htm).

Page 275: Ex-post Evaluation of FP6 NMP at project level

Over the last decade the European Union has been paving the way for the transformation of Europe into the most competitive and dynamic knowledge economy in the world. In that sense, the European Union’s Framework Programmes on Research (FP) have helped to consolidate the knowledge base in industrial technologies for many years. The reformation of European industry from resource-based to knowledge-intensive was indeed one of the major objectives of the 6th Framework Programme. This report presents the main findings of the Ex-Post Evaluation of FP6 for the ‘nanotechnologies, nanosciences, knowledge-based multifunctional materials and new production processes and devices’ (NMP) theme. It is based on a detailed analysis of FP6-funded projects in the area of NMP, aiming at understanding the factors that affected the extent and range their impacts at individual, organisational, societal, economic and environmental levels, as well as the programme aspects and policy angles that should be retained (or altered) in future NMP programmes.

KI-N

A-24935-EN

-N