fp7 interim evaluation - choose your language · europe and beyond; • increasing the quantity and...

1

Interim Evaluation of the 7th Framework Programme

STAKEHOLDER CONSULTATION

This is a summary report on the results of the Stakeholder Consultation on the 7th Framework Programme which was undertaken during July and August 2010. The consultation forms part of the evidence used by the Expert Group which is charged with carrying out the FP7 Interim Evaluation.

The Stakeholder Consultation covered major FP7 implementation issues such as impact, effectiveness, simplification, key achievements and strengths, as well as addressing possible shortcomings, including a question on priorities for any future Framework Programmes.

Under the guidance of the Expert Group an online survey was prepared, and around 150 key organisations including research organisations, firms and other types of stakeholders in the implementation of the FP were invited to contribute. In addition, the survey was extensively advertised through online and other media to allow any interested party to contribute, both those with direct experience of the FP as well all groups or individuals who wished to be heard.

Overall the aim of the exercise was to engage the relevant communities in the evaluation of FP7. Whilst the responses could not be claimed to be a statistically valid sample, nonetheless they have provided a very broad range of opinion from a wide cross section of the research community and European citizens in general. 561 responses were received with 41% of the respondents having been involved in an FP project. Respondents came from all EU counties, as well as countries associated with the Framework Programme and other countries outside Europe.

This document contains several components: a statistical analysis on the closed questions; a synthesis of the key points which emerged from the comments; and, using a special IT tool, an analysis of the free text to produce “word clouds” based on the words which appear more frequently in the source text. Contact: European Commission Research Directorate General Unit A.3: 'Evaluation and Monitoring of Programmes' Eszter Batta, Isabelle Dupont-Cavé, Peter Fisch, Gerburg Larsen, Neville Reeve

2

1. FUNDING

Is the current level of FP7 funding sufficient to achieve its objectives?

3

Analysis of responses

Comments of the respondents tended to highlight the following:

• funding is sufficient at the projects level, but insufficient to fully achieve the overall objectives of FP7;

• increased funding will be necessary if Europe is not to fall behind

competitors in the US and emerging countries such as China, India or Brazil;.

• funding might be sufficient if it were more effectively distributed

and if administrative burdens were reduced - the distribution of funding does not sufficiently benefit “New Member States”, small countries, industry/SMEs – impression of a “closed club”;

• the issue of low success rates in some areas should be linked

with question about the overall level and availability of funding.

Is the current level of FP7 funding sufficient to achieve its objectives?

3%

47%43%

7%

More than sufficientSufficientNot sufficientDon't know

4

2. IMPACT

To what extent are FP7 research activities likely to produce enduring impacts?

5



• lack of clarity on the role of the Commission regarding the Joint Programming initiative - without a coordinated approach there is a risk of reinforcing the gap between the bigger and smaller Member States as well as between the richer and financially weaker;

• a lack of TRL (technology readiness level) research, which can

bridge between near market and more fundamental research; • certain FP7 activities in need of higher funding: People Specific

Programme, Ideas Specific Programme, JTIs, Research Infrastructures, Research for SMEs, SSH, KBBE;

In a general senses, some of the more frequent comments included statements to the effect that it is still too early to say what are the long-term impacts of the FP and FP7 has already or is likely to produce enduring impacts, although these will vary across FP7 programmes, mechanisms, activities, and projects. More specifically, a wide range of positive impacts were identified including:

To what extent are FP7 research activities likely to produce enduring impacts?

17%

66%

13%4%

Highly likelyLikelyNot likelyDon't know

6

• Networking and collaboration effects such as transnational

cooperation, better integration, and research excellence; • leverage effects towards building scientific communities in

Europe and beyond; • increasing the quantity and the quality of Europe's research

knowledge base; • investment in people such as supporting the next generation of

researchers, and strengthening research career development; • forging stronger links between academia and industry; • structural impact on national funding agencies.

The responses also drew attention to a wider set of issues in relation to impacts. For instance, several respondents drew attention to the fact that enduring impacts are often not accounted for (People, Ideas), in particular the longer-term impacts of the ERC on innovation. Other respondents noted that there were unintended and/or undesirable impacts such as the “monopolisation" of certain research topics. Respondents also referred to issues which were seen as limiting the capacity to develop enduring impacts, such as a lack of continuity in funding and weak systems for dissemination and exploitation of scientific results.

7

3. EFFECTIVENESS

Has the implementation of FP7 been effective?

8


In relation to the core question, there was a generally positive answer to the effect that FP7 implementation is seen as effective. Respondents highlighted the difficulties encountered with JTI procedures which were described as heavy and not consistent with the procedures used for the rest of the FP. “…..parties have to cope with a vast and diverse variety of hardly comparable and little transparent sets of rules." In a number of comments the above-mentioned point was broadened further in criticism of the wide range of research-related implementation schemes and also the different rules governing these, “… the increasing use of pilot “programmatic approaches” which apply different rules in each programme (JTIs, art. 185, KICs…)” and “….the largest barrier to effectiveness has been the inconsistent interpretation of the rules and late publication of guidance documents….”

Has the implementation of FP7 been effective?

18%

67%

8% 2% 5% Yes

Generally yes, but with somemajor problemsGenerally no, although withsome successesNo

Don’t know

9

4. SIMPLIFICATION

Are you aware of the measures to simplify the Framework Programmes and in your view have they been successful?

10


The majority of respondents gave an informed and detailed set of comments on the achievements so far of simplification and the future agenda. There was general support for the Commission's approach, including the Simplification Communication, with the perception that simplification was producing some beneficial results so far, but also with the claim there is much more to do. At the same time, there was also considerable criticism of the lack of consistency in the interpretation of rules, with the suggestion that a 'clearing house' could be used to reduce this i.e. ensure more common approach. A common remark was the need to distinguish between simplification for those managing the FP and the reality of these changes for the participants (some comments suggested that the creation of REA had derived more from the Commissions internal politics than it had from the needs of the research community).

Are you aware of the measures to simplify the Framework Programmes and in your view have they

been successful?

12%

55%

15%

15% 3% Aware and believe they aremostly successfulAware and believe they arepartially successfulAware and believe they arelargely unsuccessfulNot aware

Don't know

11

5. ACHIEVEMENTS

What are the key achievements / strengths of the Framework Programmes in general and FP7 in particular?

12


• the Framework Programmes have been acting as a catalyst for cooperation, collaboration and networking – this has encouraged researchers to "think European" with long-term positive impacts on the establishment of scientific communities – also important has been the building of cooperation between researchers and enterprises through e.g. JTIs and ETPs;

• the ERC is seen as a major breakthrough, stimulating ground-

breaking research;

• the Framework Programmes are seen as a reliable source of funding, planned on a multiannual basis, with a budget big enough to fund ambitious R&D projects or topics that would not be otherwise financed on national level;

• the Framework Programmes have paved the way to EU R&D

integration and have prepared a solid foundation for the ERA, with particular emphasis on mobility and career development and the role of ESFRI for support infrastructures;

• the strategic approach of the Framework Programme's has improved including the definition of urgent research needs and the connection to the policy context – there should be a bigger focus on shaping future markets.

• the Framework Programmes play an important role in harmonising standards, terminologies, methodologies, practices in science

13

6. SHORTCOMINGS

Are there shortcomings in the Framework Programmes in general and the FP7 in particular that you think should be corrected?

14


• inconsistent interpretation of rules between the various functions (project officers, auditors) and the different bodies involved (executive agencies, DGs and units);

• oversubscription and problematic participation of small or new

applicants – to tackle the problem suggestions include more flexibility in the choice of the consortia composition or size, more 2-stage evaluation procedures and longer call application periods;

• time-to-grant is too long and there is a general complexity of

procedures, partly due to the rage of implementation models (JTIs, art 185…);

• there is a need for more consistent feedback from the evaluation

process;

• FP7 successes and results are not sufficiently visible, notably through a lack of publicity and dissemination;

• the need for more flexibility (both in the selection of research

topics and processes) – this to include more space for bottom-up schemes, and open, cutting-edge blue-sky research as well as flexibility in the choice of the consortia composition or size;

• the lack of adequate mechanisms for transforming knowledge

into money and a need for more focus on exploitation and valorisation in the design of instruments;

• lack of coherence and coordination between instruments such as

the CIP, EIT, and Structural Funds;

• lack of transparency in the governance process including the process of design and selection of research topics – particular reference to JTIs in this context.

15

7. PRIORITIES

What would be the priorities in your own area (funding, research, and implementation) for any future Framework Programmes?

16

Issues being raised tended to fall under three headings. Regarding implementation, comments drew attention to the need for:

• simplification of administration, less administrative burdens,

simplified documentation and a reduction to the minimum of reporting and auditing procedures;

• clarification of definitions used in the legal basis including more

consistent terminology; • addressing perceived shortcomings of evaluation procedure; • clarification of implementation structures prior to the launch of

FP8 calls; • avoiding monopolisation of topics by big projects and

overspecialisation; • ensuring consistent cooperation between the EU and Member

States to enhance effectiveness of research activities in ERA.

Regarding distribution and the level of funding, comments drew attention to the need for:

• more resources, with current funding levels deemed insufficient

to meet expectations; • more funding for Marie Curie Actions, with the emphasis on

attracting and educating talented young researchers; • more funding for the ERC; • more funding for expanding links between academia and

industry; • limiting/focusing topics and increasing the budget for remaining

calls; • develop programme-funding (such as COFUND or Joint

Programming), rather than project funding.

17

Regarding areas of research comments drew attention to the need for:

• an interdisciplinary rather than thematic approach to tackle global

challenges; • a specific budget for urgent political needs; • more 'public good' research in which industry under invests in.

Specific research topics mentioned included:

• health, public health, environment-related health effects, clinical

trials; • environment, climate change, energy; • food, food safety, nutrition, biotechnology; • international cooperation; • aeronautics, road infrastructure, air transport;

On the question of providing more support to fundamental/frontier research or applications oriented research, the replies were contradictory.

18

8. SATISFACTION

Overall are you satisfied with the Framework Programme and FP7 in particular?


Overall are you satisfied with the Framework Programme and FP7 in particular?

6%

67%

23%

3% 1%

Very satisfiedSatisfiedModerately dissatisfiedVery dissatisfiedDon't know

fp7 interim evaluation - choose your language · europe and beyond; • increasing the quantity and...

Documents