expert group on indicators on entrepreneurial learning and

Expert Group on Indicators on Entrepreneurial Learning and Competence: Final Report DG Education and Culture Framework Contract 02/10 – Lot 1

29 April 2014

A report submitted by ICF GHK on behalf of the European Commission Stelina CHATZICHRISTOU

GHK Consulting Ltd

146 Rue Royale

B-1000 Brussels

[email protected]

www.ghkint.com

mailto:[email protected]

Document Control

Document Title Final Report on Indicators on Entrepreneurial Learning and Competence

Prepared by Nick Henry, Stelina Chatzichristou

Checked by Stelina Chatzichristou, Nick Henry

Date 29 April 2014

Contents

Executive summary .......................................................................................................... iii

1 Introduction .........................................................................................................1 1.1 Why a focus on entrepreneurship education? ......................................................................... 1 1.2 Why a focus on monitoring? .................................................................................................... 2 1.3 Developing the monitoring of entrepreneurship education at the EU level .............................. 2 1.4 The structure of this Report ..................................................................................................... 3

2 Monitoring Entrepreneurship Education in Europe ................................................5 2.1 Creating a monitoring framework ............................................................................................. 5 2.2 The starting point: a European definition of entrepreneurship education ................................ 5 2.3 What to measure – identifying the scope and key elements of the monitoring approach ....... 6 2.4 An assessment of existing data sources for monitoring entrepreneurship education in

Europe ...................................................................................................................................... 9

3 A Framework for Indicators of Entrepreneurship Education in Europe ................. 16 3.1 Introduction ............................................................................................................................ 16 3.2 Currently available indicators for monitoring entrepreneurship education at EU Level ......... 16 3.3 A set of Priority Main and Sub-Indicators for entrepreneurship education for EU-level

monitoring .............................................................................................................................. 20 3.4 Summary and recommendations for improved measures ..................................................... 25

4 Monitoring Entrepreneurship Education at Member State Level ......................... 27 4.1 The state of play: a short review ............................................................................................ 27 4.2 Challenges for Member States in developing and running a monitoring framework ............. 28 4.3 Inspiring examples: Case Studies from Member States ........................................................ 28 4.4 Lessons and Recommendations to Member States .............................................................. 33 4.5 Key messages to Member States .......................................................................................... 37

Annex 1 List of Expert Group Members ....................................................................... 39

Annex 2 Statistical Annex............................................................................................ 40

Annex 3 Assessing Entrepreneurship Education: Draft survey module ......................... 41

Annex 4 Monitoring of Entrepreneurship Education in Member States- Case Studies .. 44

Annex 5 References .................................................................................................... 60

iii

Executive summary

Entrepreneurship education and Europe

The Europe 2020 strategy1 recognises that if Europe is to meet the economic and social challenges it

faces there is a critical requirement for its citizens to become more entrepreneurial across all walks of

life - for example, in economic and social innovation, new business creation, employability and active

citizenship2. The European Commission has been a long term supporter of entrepreneurship

education as it sits at the heart of an ’entrepreneurial ecosystem for Europe’.

In 2006 the Recommendation of the European Parliament and the Council on Key Competences for

Lifelong Learning recognised ‘sense of initiative and entrepreneurship’ as one of eight key

competences for every European citizen3. Subsequently, entrepreneurship education has been

incorporated in the common objectives for the education and training systems of the EU (ET2010 and

ET 2020), highlighted as a key action within the Rethinking Education4 communication and remains

one of the three main pillars of the Entrepreneurship 2020 Action Plan “Reigniting the Entrepreneurial

Spirit in Europe”5 adopted in January 2013. The recent Annual Growth Survey 2013

6 has underlined

further the importance of developing entrepreneurial skill sets amongst European youth.

Addressing the need for improved monitoring of entrepreneurship education in Europe

Defining a framework of monitoring indicators will allow for an assessment across Europe of the extent

of current entrepreneurship education activity. Monitoring is a key process that supports the ‘open

method of coordination’ (OMC) whereby governments learn from each other by sharing information,

peer learning and a common perspective of policy exchange. The availability of harmonised data for

all EU Member States allows for cross-country assessments and peer learning – of development

stage, progress and effectiveness.

In December 2012, the European Commission created an Expert Group on Indicators of

Entrepreneurial Learning and Competence (“Group”). This Report presents the outputs, findings and

conclusions of the Group. In particular, it presents:

■ Priority Indicators for the monitoring of entrepreneurial learning and competences at EU

level, based upon robust and rigorous international data sets, including identification of any

issues of gap filling, improvement and data development; and,

■ Case Studies of current examples of monitoring and evaluation of entrepreneurship

education, learning and outcomes at Member State level. These Case studies support a set

of lessons and recommendations to Member States regarding the development and/or

implementation of evaluation and monitoring frameworks for entrepreneurship education.

Existing indicators of entrepreneurship education at EU-level (and their gaps)

This Report outlines the parameters of a best practice monitoring framework for entrepreneurship

education before undertaking a process of assessment of existing indicators and related data sources.

The following international data sources that can support indicators of entrepreneurship education at

the European level are identified:

1 http://ec.europa.eu/europe2020/index_en.htm

2 See, for example, the strong entrepreneurship agenda expressed recently in the 2013 Country Specific

Recommendations, http://ec.europa.eu/europe2020/making-it-happen/country-specific-recommendations/index_en.htm 3 Recommendation of the European Parliament and of the Council (18.12.06) on key competencies for lifelong

learning, 2006/962/EC, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF 4 Rethinking Education Communication COM(2012)669 http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0669:FIN:EN:PDF 5 Entrepreneurship 2020 Action Plan, http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0795:FIN:en:PDF 6 Available at http://ec.europa.eu/europe2020/making-it-happen/annual-growth-surveys/index_en.htm

http://ec.europa.eu/europe2020/index_en.htm

http://ec.europa.eu/europe2020/making-it-happen/country-specific-recommendations/index_en.htm


http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0669:FIN:EN:PDF


http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0795:FIN:en:PDF


http://ec.europa.eu/europe2020/making-it-happen/annual-growth-surveys/index_en.htm

iv

Existing sources

■ Eurydice Q&A on Entrepreneurship Education at School in Europe;

■ Flash Eurobarometer;

■ Global Entrepreneurship Monitor (GEM): Annual Population Survey;

■ Global Entrepreneurship Monitor (GEM): Special Topic, 2008.

Possible sources

■ OECD Teaching and Learning International Survey (TALIS);

■ OECD Programme for International Student Assessment (PISA).

Key data gaps and monitoring challenges exist, including:

■ Lack of coverage of the full set of entrepreneurship education activities taking place – at all

educational levels and in all learning environments;

■ The current inability to report against a single individual or cohort of individuals through the

progression model of entrepreneurship education i.e. from education through learning outcomes

to entrepreneurial activity;

■ A key weakness that learning outcomes are self-reported (introducing the issue of self-efficacy);

and

■ That no robust and credible impact indicator exists for entrepreneurship education at the level of

the population (although it does at project level).

A set of Priority Indicators for entrepreneurship education at EU-level

Aligning with the DG EAC Joint Assessment Framework methodology for regular monitoring of

indicators and benchmarks, the Report identifies a set of priority Main and Sub-Indicators for EU-level

entrepreneurship education (see summary Table ES1 below).

This methodology concentrates monitoring on a small set of robust indicators – normally a quantitative

main indicator (with a small number of sub-groups/breakdowns) and a few associated quantitative and

qualitative sub-indicators to shed further light on the phenomenon in question.

It should be noted that in line with the monitoring of existing benchmarks in other areas of key

competences, the main indicator for entrepreneurship education and competences should ideally be

one based on a direct assessment of learning outcomes. However, currently available indicators are

based on self-reporting rather than direct assessment of learning outcomes. Given this current

position, Table ES1 below reports a Main Indicator based upon entrepreneurship education activity.

v

Table ES1: Priority Indicators for Entrepreneurship Education for EU-level Monitoring

Main Indicator

% OF POPULATION AGE 18-29 WHO HAVE TAKEN PART IN A PRACTICAL ENTREPRENEURIAL LEARNING ACTIVITY ( DEFINED AS TURNING IDEAS INTO ACTION AND/OR DEVELOPING YOUR OWN PROJECT TO ACHIEVE A GOAL) AS PART OF THEIR PRIMARY/SECONDARY/TERTIARY EDUCATION

Indicator breakdowns: gender; education level of the learning activity (primary/secondary (ISCED2011, 1-3), post-secondary/tertiary (ISCED2011, 4-8))

Data Source: New survey module that builds on existing international survey activity (results expected 2015)

Qualitative Sub-Indicator Quantitative Sub-Indicator

There is a specific national (regional) strategy for the implementation of entrepreneurship education in general school education and/or objectives related to entrepreneurship education as part of a broader education strategy

Data Source: Eurydice analysis of Entrepreneurship Education in Europe (2014/15)

% of population age 18-29 who have taken part in a practical entrepreneurial learning activity (defined as turning ideas into action and/or developing your own project to achieve a goal) outside of their primary/secondary/tertiary education


Learning outcomes (attitudes, skills, knowledge) related to entrepreneurship education are explicitly stated in the national (regional) curriculum


Educators’ training in entrepreneurship education (definition of indicator to be elaborated depending on data developments)

Data Source: To be explored if inclusion can be made in OECD Teaching and Learning International Survey (TALIS, 2018)

There is a policy and/or framework for educator education in entrepreneurship education


% of 15 year-olds below intermediate proficiency level in collaborative problem solving

Data Source: OECD Programme for International Student Assessment (PISA) assessment of “collaborative problem solving” (2015)

Gain from entrepreneurial learning on entrepreneurial activity of the population age 18-29 years in the last 12 months


vi

To achieve the Priority Indicators requires the following developments in data sources:

■ New indicators on entrepreneurial learning activity and gain from entrepreneurial learning using

data collected through a small survey module for inclusion in an existing international survey

vehicle (a draft survey instrument is provided in this Report);

■ An indicator on collaborative problem solving using results of OECD Programme for

International Student Assessment (PISA, 2015) to be published in 2016;

■ A small expansion in the range of qualitative policy input indicators collected in Eurydice (2012)

and which is due to be repeated in 2014/15; and

■ A new indicator on educator development – to be explored through discussion with OECD

Teaching and Learning International Survey (TALIS) about the potential to insert questions to

teachers on entrepreneurship education training for 2018.

Future indicator Recommendation on learning outcomes

A suitable Main Indicator based upon learning outcomes of entrepreneurship education is currently not

available. To address this, the OECD could potentially build on its existing PISA activity to achieve the

assessment and measurement of the breadth of competences and learning outcomes of

entrepreneurship education.

Monitoring Entrepreneurship Education at Member State Level: Case Studies, Lessons and Recommendations

Alongside the proposed monitoring framework, the Report uses a series of Member State Case

Studies to provide further illustration of the types of data collection and monitoring activity of

entrepreneurship education that are being undertaken at national level across Europe.

Each Case Study can be mapped by its position within the proposed monitoring framework providing

illustration of both depth and breadth of monitoring activity (see Figure ES1 below).

For example, the work of the European Training Foundation (ETF) is focussed on developing a strong

input framework for entrepreneurship education whereas the example of Finland is focused on

‘outcomes for individuals’ and, in this illustration, those outcomes for teachers. Young Enterprise

Denmark is seeking to monitor from inputs to impacts but is especially innovative in its assessment of

the key (and difficult) relationship between education and learning outcomes. Most recently, Wales

has put in place a complete monitoring framework based on youth entrepreneurship.

vii

Figure ES1 Monitoring Entrepreneurship Education Case Studies by Position in the Monitoring Framework

Lessons and Recommendations to Member States

Notwithstanding the challenges, monitoring entrepreneurship education is beneficial both for Member

States and for Europe overall. Earlier sections outline a potential framework for monitoring

entrepreneurship education at EU level, highlighting the range and coverage of metrics required and

the indicators that exist or will need to be developed to support such a framework.

The monitoring framework and Case Study material has been used to generate a set of such lessons

and recommendations to support Member States in their development of monitoring frameworks for

entrepreneurship education. The recommendations also draw on examples of good practices in

Member States beyond the case studies presented and discussed in the meetings of the Group.

Although each Member State should adjust its monitoring framework to national/regional specificities,

recommendation can be provided to support Member States as to what a framework for monitoring

entrepreneurship education could entail and what should be considered when creating such a

framework and its constituent indicators.

■ Lesson 1 : Agree on a definition of entrepreneurship education

Recommendation: A coherent monitoring system is based on a single and widely agreed definition of

entrepreneurship education that will allow for transparency and comparability of results in the long

term. So, competent authorities should agree with all relevant stakeholders and involved actors on a

definition of entrepreneurship education that will underline the broad basis of the concept.

■ Lesson 2: An action plan or strategy for entrepreneurship education should have a monitoring

framework built in

Recommendation: Especially relevant for countries that have not yet developed an entrepreneurship

education strategy, the monitoring framework should seek from the start to be an integral part of the

entrepreneurship education strategy and/or policies. This means that the strategies and policies

should incorporate the use of indicators and monitoring logics in line with an overall monitoring

framework, instead of a fragmented approach with a proliferation of definitions and ad-hoc indicators.

■ Lesson 3: Monitoring systems should be integral to and recognised in the education system

Recommendation: If monitoring is not integral to the education system itself, then it will not be

recognised or sustainable. Indicators and measures should be built into existing educational

monitoring to ensure it is an integral aspect of organisational processes.

viii

■ Lesson 4: The importance of involvement of Ministries of Education and cross-ministerial

cooperation

Recommendation: Concerning entrepreneurship education, engagement of education policy makers is

critical. Monitoring frameworks can have strengthened visibility, legitimacy and impact through a close

cooperation with other relevant ministries (e.g. Ministries of economy and employment).

Member States should engage the education authorities at all stages of the development and

implementation of the monitoring system - hopefully as part of cross-governmental collaboration.

It is important to ensure that the education agenda and objectives are reflected in the monitoring

system, so education authorities should have a leading role. The impact interests of different policy

areas/ministries need to be reflected, however, with a balance between focus on education and focus

on economic outputs (such as, for example, business start-up rates).

■ Lesson 5: Stakeholder engagement is key

Recommendation: Monitoring systems should be a joint venture designed, owned and adopted by all

relevant stakeholders – such as public authorities, teachers, school administrations, learners, training

providers, local businesses and the community. The relevant stakeholders may differ in each Member

State and between education levels and types of education.

These stakeholders should be engaged at all stages of the design and implementation of the

monitoring system. At the same time, entrepreneurial learning activities run by some of these

stakeholders should be included in the monitoring system. In that way, monitoring tools can also

function as motives for improved performance and professional development of stakeholder groups.

■ Lesson 6: Link the monitoring framework to wider policy areas, such as education, economic

development and innovation strategies.

Recommendation: Monitoring systems should consider entrepreneurial education as one element

within wider policy monitoring, to develop and contribute to a broader understanding of the education

and entrepreneurial ecosystem. In that way, the monitoring system can potentially support greater

understanding of national / regional entrepreneurial performance and the longer term impact of

entrepreneurship education can be studied for its contribution to wider impacts.

■ Lesson 7: Monitor across the policy development process

Recommendation: It is possible to monitor at any stage in the policy development process – from

creation of inputs to global impacts – and monitoring frameworks act as a tool of policy development,

as is evident from the Case Studies (Annex 4).

When developing a monitoring system Member States should aim to cover all the different stages of

entrepreneurship education from policy development, through activity, outcomes and impact, although

some stages can be given particular attention according to national priorities and the current state of


Good measures of the impact of entrepreneurship education cannot be developed in isolation from

indicators of entrepreneurship activity and outcomes.

■ Lesson 8: The monitoring system should seek to include all types of entrepreneurship education.

Recommendation: Given the ‘practical’ nature of entrepreneurship education, substantial activities

take place in various settings across education and broader stakeholder systems (for example, work

placements). Monitoring frameworks should be designed to grasp where activities take place and who

is delivering what, both within and beyond the education system.

■ Lesson 9: Use of multiple monitoring methods and tools to grasp a broader and in-depth

understanding

Recommendation: Entrepreneurship education can take many forms and can be delivered by multiple

actors in various sectors. In order to evaluate what works across all types and levels, multiple

monitoring methods can be used alongside indicator collection (research, targeted studies, self-

assessment tools etc.). Research and targeted studies may supplement the monitoring system and

support greater understanding of differences between groups, types of entrepreneurship education

ix

etc., as well as trends that are identified through monitoring indicators. Furthermore there are

limitations to what a set of indicators can capture.

However, any monitoring system should avoid becoming too complex and extensive, as this can make

it difficult to communicate results and assure sustainability over time.

Supplemental studies should not replace the monitoring system, since they:

– Will usually focus on one education level or programme; and

– May use different definitions of entrepreneurship and its education, hindering comparability

and conclusions formulation;

Ideally, in order to address limitations and conflicts due to different views/definitions, all involved

stakeholders should align on key definitions and goals before the initiation of the study.

■ Lesson 10 : There is a need for a long-term view when developing a monitoring system

Recommendation: Monitoring systems should follow a long-term strategic approach agreed among

key stakeholders. The long-term nature of a monitoring system also requires data collection to be

developed with the view to providing comparable data over time.

■ Lesson 11: Benchmarking enhances quality

Recommendation: Shared multi-country frameworks allow for benchmarking and enhanced exchange,

co-operation and learning. Member State monitoring and indicator systems should consider their

ability to support robust international benchmarking. It is therefore advisable that countries also make

use of indicators and data sources that allow for international comparisons (for example, indicators in

the proposed framework for monitoring of entrepreneurship education at EU level).

■ Lesson 12: A monitoring system should capture learning outcomes

Recommendation: Robust assessment of learning outcomes and linking these with entrepreneurial

learning activity and entrepreneurial actions requires careful design of monitoring systems – including

over time and allowing for self-efficacy. Member States should ensure common agreement and

understanding of clearly defined learning outcomes is in place to support credible monitoring activity.

■ Lesson 13: Recognise the value and limitations of self-assessment tools

Recommendation: Such tools can offer significant benefits, as they can:

– Support the target group (e.g. teachers) in professional development: self-assessment tools

can be perceived as development tools, rather than monitoring;

– Provide immediate and interactive way of data collection regarding what is implemented

(practices/methods); how it is perceived; etc.

– Support systematic data collection on a decentralised level: schools, region, municipalities and

also country/national level.

At the same time, it should be recognised that the results of such tools can be biased due to self-

selection; for example, the voluntary nature of the tool and reporting fatigue from participants (i.e.

teachers).

For the introduction and implementation of such tools, a bottom-up approach is recommended as it

will underline the use of the tool as a mean of personal development.

1

1 Introduction

This Report is the output of the DG EAC Expert Group on Indicators on Entrepreneurial Learning and Competence. The key objective of the Group has been to agree on a limited set of Main and Sub-Indicators of entrepreneurship education that would be suitable for regular monitoring at EU-level, given an assessment of the current state of play in monitoring at EU and Member State level and proposed data developments.

1.1 Why a focus on entrepreneurship education?

The Europe 2020 strategy7 recognises that if Europe is to meet the economic and social

challenges it faces there is a critical requirement for its citizens to become more

entrepreneurial across all walks of life - for example, in economic and social innovation, new

business creation, employability and active citizenship8 (especially amongst young people).

In particular, any dynamic economy and society requires people who have the motivation,

knowledge and skills to become entrepreneurs.

Entrepreneurship education sits at the heart of any ’entrepreneurial ecosystem for Europe’,

because education is key to shaping young people’s mindset, attitudes and skills.

The European Commission remains a long term supporter of entrepreneurship education. As

early as 2000, the Lisbon Agenda identified the need to build entrepreneurship into the

curriculum from an early age to allow children to develop a set of skills which would promote

entrepreneurial activity9. By 2006, the Recommendation of the European Parliament and the

Council on Key Competences for Lifelong Learning had recognised ‘sense of initiative and

entrepreneurship’ as one of eight key competences for every European citizen and which

was expected thereafter to be instilled at all stages of education and training10

.

Subsequently, entrepreneurship education was incorporated in the common objectives for

the education and training systems of the EU, as articulated in the Education and Training

2010 Work Programme (ET2010), and enhancing entrepreneurship at all levels of education

and training continues as one of the four Strategic Objectives that the European Council has

set for ET202011

.

Recently, the Commission Communication ‘Rethinking Education: Investing in skills for

better socio-economic outcomes’12

has re-emphasised the message that, in order to build

21st century skills, efforts should be concentrated on developing entrepreneurial skills such

as critical thinking, initiative, problem-solving and working collaboratively. In 2013,

entrepreneurial education and training was adopted as one of the three main pillars of the

Entrepreneurship 2020 Action Plan “Reigniting the Entrepreneurial Spirit in Europe”13

and

7 http://ec.europa.eu/europe2020/index_en.htm

8 See, for example, the strong entrepreneurship agenda expressed recently in the 2013 Country Specific

Recommendations, http://ec.europa.eu/europe2020/making-it-happen/country-specific-recommendations/index_en.htm 9 European Commission Report (2001) “The Concrete Future Objectives of Education Systems”, http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2001:0059:FIN:EN:PDF 10

Recommendation of the European Parliament and of the Council (18.12.06) on key competencies for lifelong learning, 2006/962/EC, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF 11

Council conclusions of 12 May 2009 on a Strategic Framework for European Cooperation in Education and Training (‘ET 2020’) 2009/C 119/02, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2009:119:0002:0010:EN:PDF See, also, Entrepreneurship 2020 Action Plan (EAP) - Communication from the European Commission to the Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions 12

Rethinking Education Communication COM(2012)669 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0669:FIN:EN:PDF 13

Entrepreneurship 2020 Action Plan, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0795:FIN:en:PDF







http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2009:119:0002:0010:EN:PDF






2

the recent Annual Growth Survey 201314

has underlined further the importance of developing

entrepreneurial skill sets amongst European youth, linking such skills to employability and

business creation.

1.2 Why a focus on monitoring?

Monitoring should be recognised as a key component of both the policy development

process and the learning process for practitioners more widely. It comprises part of the good

practice process of evaluation in public policy15

. Most particularly, in relation to the activities

of the European Commission, monitoring is a key process that supports the ‘open method of

coordination’ (OMC) whereby governments learn from each other by sharing information,

peer learning and a common perspective of policy exchange.

On a Member-State level, monitoring supports policy, programme and project development.

Monitoring can support understanding of ‘what is (or is not) working’, guide implementation

improvements and provide clarity on the efficiency of an intervention. By determining

impacts, monitoring supports understanding of what is achievable, what progress is being

made and provides a potential evidence base to engage stakeholders, support policy

development and facilitate the promotion of the object of intervention (for example,

entrepreneurship education).

When monitoring is systematic and repeated it can allow comparisons between different

programmes and initiatives, identification of trends, and (ex ante) forecasting of the expected

results of (entrepreneurship education) programmes.

Common monitoring on a European level provides additional benefits. The availability of

harmonised data for all EU Member States allows for cross-country assessments and peer

learning – of development stage, progress and effectiveness. Moreover, monitoring

supports the European Union in understanding the progress and effectiveness of its

programmes and initiatives. A monitoring framework and relevant indicators on an EU level

also allows the EU to assess progress against strategic objectives and targets (in this

context, progress towards realisation of the ET2020 objectives). Based on the information

regarding progress and effectiveness, monitoring indicators can support evidence-based

decisions on policy changes.

1.3 Developing the monitoring of entrepreneurship education at the EU level

Given strategic and policy recognition of the importance of entrepreneurship education, there

exists a key requirement to understand and develop the state of play of entrepreneurship

education across Europe. In support of this, there is the need to develop an agreed

framework of European measurements of entrepreneurship education. Defining such a

framework of indicators will allow for an assessment across Europe of the extent of current

entrepreneurship education activity, its economic and social outcomes and impacts and,

thus, the potential for further European learning, development and progress towards ET2020

objectives.

In 2011, GHK Consulting undertook a study on the state of play in Europe concerning

measurement (and indicators) of entrepreneurship education16

. The Study underlined that

whilst there existed substantial interest in the value of entrepreneurship education across

Europe, monitoring and evaluation frameworks and associated indicators were, at best, “in

14

Available at http://ec.europa.eu/europe2020/making-it-happen/annual-growth-surveys/index_en.htm 15

See, for example, Evalsed (2012) The Resource for the Evaluation of Socio-Economic Development; http://ec.europa.eu/regional_policy/sources/docgener/evaluation/guide/guide2012_evalsed.pdf. 16

GHK (2011) Final Report: Order 121 - Study on Support to Indicators on Entrepreneurship Education; Framework Contract No. EAC 19/06; http://ec.europa.eu/education/library/study/2011/entrepreneurship_en.pdf


http://ec.europa.eu/regional_policy/sources/docgener/evaluation/guide/guide2012_evalsed.pdf

http://ec.europa.eu/education/library/study/2011/entrepreneurship_en.pdf

3

their infancy”17

. Similarly, in 2012, Eurostat stressed the need also for further data and

indicator development on entrepreneurship education18

.

In December 2012, to address this need to develop and agree on measurements of

entrepreneurship education, the European Commission created an Expert Group on

Indicators of Entrepreneurial Learning and Competence (Group) (see Annex 1 for a List of

Group Members). The Group aims were:

■ The promotion of the development and improvement of monitoring of entrepreneurial

education on an EU level by DG EAC; and

■ The provision of insights and recommendations on monitoring of entrepreneurship

education on a Member State level.

Specifically, Group objectives were to:

■ Identify the key questions/elements considered relevant for entrepreneurship education

policy monitoring;

■ Identify and discuss the quality/relevance of existing data sources and indicators;

■ Agree on a limited set of indicators that would be suitable for regular monitoring at EU

level;

■ Recommend priorities and ways to address existing data gaps/need for data refinement

in the medium term, also seeking inspiration from experiences/best practices at

national/regional level; and

■ Advise policy makers on monitoring at national level, drawing on existing lessons/best

practice.

This Report presents the outputs, findings and conclusions of the Group.

Taking into consideration the existing indicators on entrepreneurship education and their

limitations, as well as the need to monitor entrepreneurship education more broadly, the

Group suggests a set of indicators, consisting of one Main indicator and seven Sub-

indicators. Where needed, the sub-indicators are broken down for gender, age group and/or

education levels.

The work of the Group has contributed to the policy guidance work of the Thematic Working

Group on Entrepreneurship Education, especially regarding measurement and impact of


The recommendations on a set of entrepreneurship education indicators contained in this

Report support the update of the current EU framework of indicators and benchmarks on

education and training19

.

The recommendations for priority indicators and data developments should also allow for a

strengthened analysis of entrepreneurship education in forthcoming editions of the annual

DG EAC Education and Training Monitor.

1.4 The structure of this Report

This Report is structured as follows:

17

Ibid. 18

Eurostat (2012) Entrepreneurship Determinants: Culture and Capability, p.24; http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-31-12-758/EN/KS-31-12-758-EN.PDF 19

As proposed in: Commission Staff Working Paper accompanying the COM (2011) 902. Analysis of the implementation of the Strategic Framework for European cooperation in education and training (ET2020) at the European and national levels. (http://register.consilium.europa.eu/pdf/en/11/st18/st18577-ad01.en11.pdf); Communication to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions- Draft 2012 Joint Report of the Council and the Commission on the implementation of the Strategic Framework for European cooperation in education and training (ET2020). Education and Training in a smart, sustainable and inclusive Europe (http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0902:FIN:en:PDF)

http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-31-12-758/EN/KS-31-12-758-EN.PDF

http://register.consilium.europa.eu/pdf/en/11/st18/st18577-ad01.en11.pdf



4

■ Section 2: provides an overview of the state of play in monitoring entrepreneurship

education in the EU. To do so it outlines the basic foundations of a monitoring framework

that need to be in place - for example, an agreed definition of the object of study

(entrepreneurship and entrepreneurship education) and its aims and objectives (the

intervention logic), the existence of agreed metrics and indicators across stakeholder

groups, and the data sources potentially available to support the monitoring process;

■ Section 3: provides an assessment of the current range of indicators available for

measuring entrepreneurship education at EU level. On this basis, a set of Priority

Indicators (one Main and some Sub-Indicators) for Entrepreneurship Education for EU

Level Monitoring are proposed, including Recommendations for indicator and data

development in the short to long term; and

■ Section 4: provides an overview of the state of play in monitoring and evaluation of

entrepreneurial education across Member States. Based on case studies analysed in the

Group meetings and other inspiring examples, this Section provides lessons and

recommendations to Member States regarding the development and/or implementation

of evaluation and monitoring frameworks for entrepreneurship education.

A series of Annexes provide supporting material:

■ A list of the Group members;

■ A statistical mapping of the ‘existing’ indicators of entrepreneurship education for

Member States;

■ A draft survey module to be used in monitoring on an EU level; and

■ Greater detail on the Case Studies of monitoring and evaluation of entrepreneurial

education, learning and outcomes at Member State level.

5

2 Monitoring Entrepreneurship Education in Europe

2.1 Creating a monitoring framework

Section 1 introduced the sustained policy interest at European level in creating strong and

widespread systems of entrepreneurial education as the basis for an entrepreneurial Europe.

In 2006, the OECD-Eurostat Entrepreneurship Indicators Programme (EIP) pioneered the

development of statistical concepts and methods to support entrepreneurship policy making.

EIP does, however, define entrepreneurship in relation to economic activity only and, in

terms of entrepreneurship education, the EIP framework is focused on outcomes and

impacts. Entrepreneurship training and education is, then, only partially covered as one

element of the ‘entrepreneurship ecosystem’ that determines entrepreneurship impacts20

.

The key foundation for European policy development on entrepreneurship education has

been development of an agreed European definition of entrepreneurship (as a key

competence) and agreement on the distinctive pedagogy of entrepreneurship education (‘a

practical chance to turn ideas into action’). Subsequently, in seeking to understand progress

and good practice, the interest of policy makers and practitioners has moved to the

construction of monitoring and evaluation frameworks.

To define a monitoring framework requires a number of building blocks including common

agreement on the object of monitoring (“entrepreneurship education”) and its components –

what it is, what it comprises, and what results occur through its activities.

Once these components are agreed, the process can move to agreeing the metrics,

indicators and datasets which exist (or can be created) that are best felt to be able to

describe the object over time and space and its process of change – for example, how the

comprehensive and sustained delivery of an entrepreneurial learning activity to a set of

students may lead to sustained growth in entrepreneurial actions which impact on the

economy (e.g. a new firm) or society (e.g. community role models).

2.2 The starting point: a European definition of entrepreneurship education

A key moment in the ambition to foster an entrepreneurial culture and mindset across

European Member States remains the 2006 Recommendation of the European Parliament

and the Council on Key Competences for Lifelong Learning. This document identified ‘sense

of initiative and entrepreneurship’ as one of eight key competences for every European

citizen to be instilled at all stages of education and training21

.

It is at this point that the European Commission adopted the following definition of

entrepreneurship as a key competence22

:

“Sense of initiative and entrepreneurship refers to an individual's ability to turn

ideas into action. It includes creativity, innovation and risk-taking, as well as the

ability to plan and manage projects in order to achieve objectives. This supports

individuals, not only in their everyday lives at home and in society, but also in the

workplace in being aware of the context of their work and being able to seize

opportunities, and is a foundation for more specific skills and knowledge needed by

20

EIP has established a framework that contains categories for indicators. The framework looks not only at the expression of entrepreneurship, but also at the factors that influence entrepreneurship. EIP aims at collecting data regarding determinants of entrepreneurship, specifically: Regulatory Framework; Market Conditions; Access to Finance; Creation and Diffusion of Knowledge; Entrepreneurial Capabilities and Entrepreneurial Culture. The relevant data is drawn from various international and national sources; see OECD (2012) Entrepreneurship at a Glance 21

Recommendation of the European Parliament and of the Council (18.12.06) on key competencies for lifelong learning, 2006/962/EC, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF 22

Recommendation of the European Parliament and of the Council (18.12.06) on key competencies for lifelong learning, 2006/962/EC p.8


6

those establishing or contributing to social or commercial activity. This should

include awareness of ethical values and promote good governance.”

Importantly, the definition set a number of parameters for any potential monitoring

framework:

■ Entrepreneurship as a competence refers not just to acquisition of attitudes, skills and

knowledge but also implies the ability of an individual with that competence to take action

and turn ideas into reality. Entrepreneurship education must therefore work towards

equipping and enabling the individual to take such actions;

■ Entrepreneurial activity is not narrowly related to economic life, such as the world of work

and commerce, but extends more widely to the other areas of human activity such as

society, government and the home. Entrepreneurship education in turn needs to reflect

this; such as by including considerations of social enterprise, social innovation and

creativity more widely;

■ Similarly, entrepreneurial activity does not just relate to external activities, such as

creating a new business, but also to taking innovative and creative actions within an

existing organisation. The term “intrapreneurial activity” is frequently used to describe

such activity within existing organisations – and needs to be considered and included in


Alongside this broad definition of entrepreneurship has been that of a broad definition of

entrepreneurship education. Following UNESCO and the European Training Foundation

(ETF)23

, entrepreneurship education is taken to be:

“All forms of learning, education and training which contribute to entrepreneurial spirit,

competence and behaviour; with or without a commercial objective”

When thinking of the parameters for any monitoring framework, this definition implies that

entrepreneurial education can encompass all types of learning. It also encompasses all

levels of education and training, from early years through to continuing professional

development (CPD).

In total, these considerations imply that the (entrepreneurial) individual sits at the heart of an

ecosystem for entrepreneurship education which includes the individual, educators,

institutions, policymakers and the educational and business environment. Thus, any

monitoring frameworks developed should recognise the role of each participant in the

ecosystem and be designed with awareness of the abilities and limitations of each actor to

influence the system, its outcomes and impacts.

2.3 What to measure – identifying the scope and key elements of the monitoring approach

Alongside European agreement of entrepreneurship as a key competence, in 2010 a report

from DG Enterprise24

detailed that a set of common views were discernible across Europe

on the aims and objectives of entrepreneurship education – and which could be elaborated

as an intervention logic. In other words, a common understanding could be written of the

goals and rationale for entrepreneurship education, the types of activities it might entail and

the expected economic and social outcomes and impacts that might be expected to result.

Within monitoring and evaluation, development of intervention logics forms the basis for

monitoring frameworks. As agreed statements by stakeholders on activities, expected results

and impacts, intervention logics define what it is that needs to be measured and monitored.

23

See, for example, European Training Foundation (2013), Inform – Entrepreneurial Learning Issue, Issue 16/June 2013, http://www.etf.europa.eu/eventsmgmt.nsf/%28getAttachment%29/30148C231FA87366C1257B95004780EF/$File/INFORM_16_Entrepreneurial%20learning.pdf 24

European Commission DG Enterprise (2010) Towards Greater Cooperation and Coherence in Entrepreneurship Education: Report and Evaluation of the Pilot Action High Level Reflection Panels on Entrepreneurship Education initiated by DG Enterprise and Industry and DG Education and Culture,

http://www.etf.europa.eu/eventsmgmt.nsf/%28getAttachment%29/30148C231FA87366C1257B95004780EF/$File/INFORM_16_Entrepreneurial%20learning.pdf


7

Once an intervention logic is in place, the issue then becomes identifying suitable metrics,

indicators and data sources for the monitoring framework that are robust enough for

stakeholders to agree that the process of activity and change is being adequately described

and tracked ‘in reality/on the ground’.

GHK (2011)25

developed an ‘entrepreneurship education intervention logic for Europe’

comprised of six stages (‘from input to global impacts’). The intervention logic is based on

the journey of the individual through the system – from a systematic infrastructure being put

in place to provide an entrepreneurship education for (target) population cohorts to the

competence gained through that education by the individual and subsequently the use of

that competence in an entrepreneurial activity. Using this existing body of knowledge and the

expertise of the Group, the following parameters then framed the process of definition of a

monitoring framework and indicators:

■ That the monitoring framework use and build on the agreed ‘entrepreneurship education

intervention logic for Europe’ but that it be simplified from a six stage to a four stage

model (inputs, activities/process, outcomes and impacts, see Figure 2.1);

■ That the intervention logic model adopted:

– Recognise the importance of and need to report on policy inputs given the current

state of play in Member States;

– Understand the entrepreneurial learning process as a progression of the individual -

one of an individual receiving entrepreneurship education, gaining key competences

and demonstrating those through entrepreneurial action that generates impacts in

economy and society; and

– Seek highly robust reporting of impact at the level of the population.

■ To avoid indicator proliferation and be in line with the Joint Assessment Framework

methodology applied by DG EAC in its regular monitoring of indicators and benchmarks,

that at most only a couple of main indicators be defined together with a limited set of

sub-indicators – whilst retaining credibility and robustness as a monitoring framework;

■ The potential to include qualitative input/system indicators and quantitative indicators;

■ That the proposed set of indicators be viewed as feasible in terms of potential for use at

a) EU level and b) Member State level; and

■ That Member State coverage of chosen indicators should be high and include the ability

for indicator ‘breakdowns’ (geography, socio-economic groups, educational level, etc.).

The logic chain outline in Figure 2.1 overleaf is comprised of four stages. The logic chain

provides the framework for what should be measured – allowing subsequent work on

technical definition of suitable indicators and data sources:

■ Input: a strategic policy action or structure that embeds an agreed common

understanding of entrepreneurship education, such as the existence of an

entrepreneurship education national/regional strategy or action plan;

■ Activity/Process: Measures of delivery activity; for example, the number of educators

receiving initial training in entrepreneurial education or the percentage of the

population that has received entrepreneurship education through taking part in a

practical entrepreneurial learning activity;

■ Outcomes; refer to results from entrepreneurship education programmes that can be

identified immediately (attitude, skills, knowledge); for example, perceptions of learners

regarding their skills and knowledge to start a business or the percentage of 15-year-

olds with a good proficiency level in collaborative problem solving;

■ Impact: the impacts on society (cohesion, social innovation, etc.) and economy

(employability, business creation, etc.) of the range of entrepreneurial actions

undertaken, due (at least in part) to entrepreneurship education.

25

GHK (2011) Final Report: Order 121 - Study on Support to Indicators on Entrepreneurship Education; Framework Contract No. EAC 19/06; the intervention logic was built on earlier work of DG Enterprise,

8

Figure 2.1 Entrepreneurship Education Logic Model (adapted from GHK (2011) p.13)

Impact

(across all

population

cohorts)

Greater levels of active

citizenship and social

innovation

(including social

entrepreneurship)

More creative, adaptable

and employable

workforce

More productive and

innovative businesses

(‘intrapreneurship’)

Increased rates of business start-up

and survival

Increase rates of start-up and

survival of social

businesses/enterprises

Outcomes

Population have the experience, competence and ability to create a new project, venture or initiative either for themselves, their community or an employer

(that involves turning an idea into action)

Activity

Proceess

Individuals experience teaching and learning of core competences (attitude, skills, knowledge)

through practical entrepreneurial learning embedded throughout education and training

Input: National/

regional

framework

A common understanding by all stakeholders of the definitions, objectives and targets of

“entrepreneurship education”, embedded in the national (or regional) context

9

2.3.1 What makes a good indicator for monitoring entrepreneurship education?

The development of monitoring frameworks requires the identification of the characteristics

of “good indicators”. A key issue for indicators concerned with entrepreneurship education is

the directness of their link to educational activity:

■ Figure 2.1 above highlights how competence and learning outputs of entrepreneurship

education - such as mindset, skills and knowledge – can easily be generated through

non-educational activity. This is further accentuated when seeking to determine if

subsequent actual entrepreneurial action (economic or social) is due to educational

activity or other influences of the entrepreneurial ecosystem (for example, employment

legislation, funding and finance opportunities, etc.);

■ In contrast, this underlines also the spectrum of activities and environments that should

be understood to comprise entrepreneurship education; individual experiences of

learning can all be shown to contribute to entrepreneurial mind-sets, competences and

behaviour;

■ The link to educational activity needs also to be understood in terms of the issue of

timescale. For example, most education occurs, of course, as children or young people;

so indicators and outcomes concerning (entrepreneurial) educational experiences of the

population may reflect education systems that may even no longer be in place and

where there have been many other influences in the intervening years; and

■ There is a need, then, for open and transparent acknowledgement of the issues of

attribution; in other words, given the range of determinants of entrepreneurial action and

the timescales over which such actions may occur, can an entrepreneurial action or

impact be linked back or attributed to an entrepreneurship education experience

(‘causality’). This issue is common within the field of entrepreneurialism and education

more broadly – is openly acknowledged - and is illustrated by the strong position held by

the concept of self-efficacy in the literature.

Self-efficacy is a measure of the belief in one's own ability to complete tasks and reach

goals and holds strong resonance in the field of entrepreneurialism. In other words, certain

individuals and societies may have a high self-efficacy in terms of their perception of their

ability to successfully perform the roles and tasks of entrepreneurship, notwithstanding the

role of entrepreneurship education or other similar interventions. Any truly accurate

measurement of the outcomes of entrepreneurship education needs to take account of this

difficult issue.

See Bandura, A. (1997) Self-efficacy: The exercise of control

2.4 An assessment of existing data sources for monitoring entrepreneurship education in Europe

Good indicators are determined, ultimately, by the feasibility of collecting meaningful and

credible data for them.

Drawing from previous studies as detailed above, and alongside review of recent

developments, the following provides an assessment of data sources identified at the

European level. These sources are split between those that currently cover

entrepreneurship education and those that have the potential to be developed to provide

coverage in the future. The section ends on key gaps and challenges for the provision of

data to support indicators of entrepreneurship education given the current availability of data.

2.4.1 Current data sources

The following data sources are reviewed:

■ Eurydice Q&A on Entrepreneurship Education at School in Europe (2012);

■ Flash Eurobarometer (2012);

10

■ Global Entrepreneurship Monitor (GEM): Annual Population Survey; and,

■ Global Entrepreneurship Monitor (GEM): Special Topic, 2008.

The data sources are assessed regarding their content, coverage and breakdowns.

2.4.1.1 Eurydice Q&A on Entrepreneurship Education at School in Europe (2012)

Description, Coverage and Breakdowns

Eurydice undertook a questionnaire and data collection of policy documents across 31

European countries and regions at primary (ISCED Level 1) and general secondary

education (ISCED 2 and 3) for the school year 2011-2012.

Coverage included:

■ National strategies and action plans to encourage the integration of entrepreneurship

education;

■ Entrepreneurship education approaches and subject curricula in national documentation;

■ Areas of learning outcomes defined for entrepreneurship education and any practical

guidelines to support teachers; and

■ Initiatives to promote entrepreneurship education and current educational reform

programmes.

Strengths and Weaknesses

The data source provides a comprehensive and robust qualitative statement based upon the

application of a common methodology by a team of national units across Member States.

The state of play in regions of specific Member States (for example, the Flemish, French and

German communities in Belgium) are also examined, due to the central role those regions

play in education.

Information and data provided by Member States was not independently reviewed to provide

comparability. The study did not include vocational, technical or commercial schools. Higher

education was also not included.

2.4.1.2 EU Flash Eurobarometer (2012)


Flash Eurobarometer No 354 (2012) “Entrepreneurship in the EU and beyond” covered all

EU Member States (then 27 countries) and 13 countries from outside the EU. Over 42,000

respondents from different social and demographic groups were interviewed via telephone –

including at least 1,000 in each Member State.

Weighting of responses against population demographics was undertaken in reporting.

Flash Eurobarometer on entrepreneurship is expected to be repeated every 3 years.


At the outset, it needs to be recognised that Eurobarometer is designed first and foremost as

a public opinion tool rather than a research survey intended to collate detailed material on

individual topics such as entrepreneurship education.

Particular identified advantages and weaknesses include:

■ Allows for comparability:

– Provides an EU aggregate for all indicators examined, along with data on the specific

EU Member States;

– Covers all EU Member States (bar Croatia which will be included in the next survey);

– Allows for the comparison of EU with other developed countries, underlying the value

of international benchmarking;

■ Focus on the youth: The survey addresses people aged 15 years and over – although

the sample size for youth is limited (especially at Member State level;

11

Definition of entrepreneurship: The survey questions are based on the broader

concept of entrepreneurship (turning ideas into action, developing your own projects)

rather than just business start-up, matching the agenda of DG EAC.

However:

■ Sample size: sample size per Member State poses challenges regarding the statistical

soundness of detailed analyses regarding breakdowns - for example, results by gender

or age group;

■ Target groups: Although it includes people aged 15 and over, the Eurobarometer

targets the general population. So, indicators assessing impact of entrepreneurship

education may be providing intelligence on past and obsolete education systems;

■ Phrasing of questions: Flash Eurobarometer survey is conducted through telephone

interviews, which means that there is limited time for the respondent to process a

question and to seek clarification. This highlights also that as with all surveys based on

subjective responses, response bias can be a challenge regarding data quality.

2.4.1.3 Global Entrepreneurship Monitor: Annual Population Survey (APS)


The GEM APS runs on a yearly basis and monitors the developments in entrepreneurship

and entrepreneurial attitudes and aspirations across more than 60 countries (69 countries in

2012).

The minimum sample consists of at least 2,000 respondents, age 18 and over. To date,

reporting has tended to focus on the age cohort of 18 – 64 year olds. Samples are selected

randomly, but according to rules and principles set centrally by GEM Global.

In the 2012 survey, 22 EU countries participated: Estonia, Hungary, Latvia, Lithuania,

Poland, Romania and Austria, Belgium, Denmark, Finland, France, Germany, Greece,

Ireland, Italy, Netherlands, Portugal, Slovakia, Slovenia, Spain, Sweden and the United

Kingdom.

GEM APS provides data for many breakdowns: all questions/indicators are disaggregated by

age, education level, gender and income level (broken down in three tiles). Results can be

calculated for all stages of entrepreneurial activity, namely nascent and/or new

entrepreneurs, established entrepreneurs or individuals that have no entrepreneurial activity.

Results and breakdowns are available by country, country category (e.g. innovation-driven),

and total GEM.


■ The survey allows for comparison with all the developed countries outside the EU. It

should be noted, however, that such comparisons should take into consideration the

different economic and political structures that can affect comparability;

■ The survey is well-known and established in all participating countries; so, its indicators

are well understood in literature and participating countries, while they have been

embedded in evaluation/monitoring frameworks of national strategies on

entrepreneurship education, such as in Wales;

– One of the main indicators, percentage of the total population engaged in early-stage

entrepreneurial activity (TEA) can be used for benchmarking and monitoring the

development of start-ups; TEA also includes “starting a business for your employer”,

which is also significant to measure.

■ A range of detailed breakdowns are available and allow for cross- comparisons between

countries and/or sub-groups of participants, leading to rich insights.

Weaknesses include:

■ The Annual Survey does not include questions specifically on entrepreneurship

education (although see GEM Special Report below);

12

■ GEM does not include people below 18 years old, currently in secondary education;

■ Not all EU countries participate in the GEM survey (although Member State participation

is rising);

■ The same countries do not participate in the survey every year, creating challenges for

consistent benchmarking;

■ Although the TEA indicator is well-known in countries and is readily available to use it is

not sufficient for measuring the impact of entrepreneurship education, since:

– it discusses only the business aspect of entrepreneurship;

– the impact of entrepreneurship education cannot be directly identified through the

specific indicator;

– TEA does not capture social entrepreneurship (although a GEM Special Topic on

Social Entrepreneurship has taken place);

■ GEM includes questions that are based on participants’ subjective understanding/

personal opinion (e.g. % of the population that believe they have the necessary skills or

knowledge to start a business);

■ Whilst the sample size is relatively large per country, there are limits to statistical validity

for certain breakdown analyses – and which vary by country.

2.4.1.4 2008 Global Entrepreneurship Monitor Special Report: A Global Perspective on Entrepreneurship Education and Training


Besides the annual survey, GEM runs a survey on a specific topic each year. In 2008, the

topic was on entrepreneurship education and training. In total, 38 countries participated in

the special study.

As with GEM APS, only the adult population (18-64) participated. The survey examined the

level and the source of entrepreneurship education and training received by participants.

The Special Report provides data on type of training (in-school vs. non-school); source of in-

school training (school vs. tertiary); providers of non-school entrepreneurship training

(universities, employers, public agencies, online, etc.); and on whether the participants

attended the trainings voluntarily or if it was compulsory.

Results can be broken down by age groups, gender, involved in early-stage entrepreneurial

activity (TEA), and not involved (non-TEA). Average age of participants and/or specific sub-

groups can also be calculated from detailed results.


The 2008 report focused on entrepreneurship education and training, providing data that had

not been collected before.

The Report also developed a sophisticated methodology to assess the gain from training as

“the increased odds of engaging in entrepreneurial behaviour due to training, rather than a

consequence of some prior desire to behave entrepreneurially”.

Gain from Training

Gain from training (GFT) is a measurement methodology developed by the Global Entrepreneurship Monitor to estimate the gain from undertaking training in entrepreneurship.

GFT seeks to account of the range of other factors which might determine an individual (or groups) propensity to be entrepreneurial. In other words, it seeks to control for these aspects to truly get at ‘the difference that training makes’ in terms of subsequent entrepreneurial orientation after training.

Prior orientation to be entrepreneurial can often be seen in voluntary participation in

13

entrepreneurship education/training. This is generally seen as self-selection in that individuals with an interest in entrepreneurship are more likely to choose to train, and so an understanding of voluntary or compulsory engagement in training by the individual is important.

This prior orientation might also be prevalent in certain population cohorts (demographic background such as age). In this instance, a key process of control is large and randomised samples of respondents within country – thus also controlling for national cultures or contexts.

Ultimately, GFT measures the increase in odds that individuals with a given set of demographic characteristics will have a particular entrepreneurial orientation if they have ever taken compulsory training versus individuals with identical demographic characteristics but without such training.

Source: Adapted from GEM (2008) A Global Perspective on Entrepreneurship Education and Training, A Special Report.

More specifically, results are provided for the effect of gain from training in Entrepreneurial

Awareness, Attitudes, Intention and Activity (fear of failure; have the skills and knowledge to

start a business; know personally someone who recently started a business; identify

business opportunities in the area); and in relation to type of entrepreneurial activity

undertaken by the individual (new or nascent entrepreneur; or expects to start a business in

3 years).

Weaknesses are that the survey was ad hoc and provides information on a specific year.

Also, results do not cover all EU countries and miss out individuals below 18 years old.

Entrepreneurship outcomes are focused on economic pathways (business start-up).

2.4.2 Potential data sources

The following data sources are reviewed:

■ OECD Teaching and Learning International Survey (TALIS); and

■ Programme for International Student Assessment (PISA).

The data sources are assessed regarding their potential content, coverage and breakdowns.

2.4.2.1 OECD Teaching and Learning International Survey (TALIS)26


TALIS is a survey run by the OECD, addressing teachers and school principals. The first

survey ran in 2008 and a further survey was carried out in 2013 with results due in mid-2014.

As in 2008, TALIS 2013 surveyed teachers and principals of lower secondary education

only. In total, 42 countries/regions (33 OECD and 9 non-OECD) participated, including18 EU

Member States/regions: Belgium (Flanders), Bulgaria, Croatia, Latvia, Romania, Czech

Republic, Denmark, Estonia, Finland, France, Italy, Netherlands, Poland, Portugal, Slovak

Republic, Spain, Sweden, United Kingdom (England).

Depending on the size of the country, the average sample size was about 200 schools and

20 teachers within each of these schools. Schools and teachers within those schools were

arbitrarily selected.

TALIS 2013 focused on:

■ Teacher training and professional development;

■ Teachers’ appraisal and feedback;

■ School climate;

■ School leadership;

■ Teachers’ instructional beliefs; and

■ Teachers’ pedagogical practices.

26

http://www.oecd.org/education/school/oecdteachingandlearninginternationalsurveytalishome.htm


14

Although TALIS does not focus on entrepreneurship education, this is the only identified data

source that engages with teachers across Europe through a common methodology.

Given the importance of (initial and continuous) teacher education on entrepreneurship

education and teachers’ perceptions and skills regarding entrepreneurship education, TALIS

could facilitate the collection of relevant data.

2.4.2.2 Programme for International Student Assessment (PISA)


The OECD Program for International Student Assessment (PISA) is an international

assessment that measures 15-year-old students' reading, mathematics, and science literacy.

PISA also measures functional skills including financial literacy in 2012 and collaborative

problem solving to be assessed in 2015.

The tests are designed to assess to what extent students at the end of compulsory education

can apply their knowledge to real-life situations and be equipped for full participation in

society.

Through the combination of a number of QA measures PISA produces high quality

instruments and outcomes with superior levels of validity and reliability to improve the

understanding of education systems as well as students’ knowledge, skills and attitudes.

Results are broken down by country, gender, race/ethnicity and socio-economic context. The

statistical analysis includes the use of control groups, reducing biases.

Although it did not touch upon entrepreneurship education, PISA (2012)27

included an

assessment of students' financial literacy and the results for over 60 countries are to be

published in June 201428

.

The working definition of financial literacy for PISA 2012 is as follows:

Financial literacy is knowledge and understanding of financial concepts and risks,

and the skills, motivation and confidence to apply such knowledge and

understanding in order to make effective decisions across a range of financial

contexts, to improve the financial well-being of individuals and society, and to

enable participation in economic life.

The working definition of collaborative problem solving for PISA 2015 (results to be

published in 2016) is:

Collaborative problem-solving competency is the capacity of an individual to

effectively engage in a process whereby two or more agents attempt to solve a

problem by sharing the understanding and effort required to come to a solution and

pooling their knowledge, skills and efforts to reach that solution.


Core strengths of the PISA approach are direct assessment of learning outcomes,

undertaken through a rigorous testing process.

In relation to entrepreneurship education, a small number of elements of learning outcomes

(financial literacy, problem solving) are (or will be) tested. These tests, however, only take

place at age 15 and at the beginning of a key educational period when entrepreneurship

competences are known to expand and accelerate, partly due to enhanced provision of

entrepreneurship education. The theoretically based nature of PISA assessment is also

considered to not fully reflect the inherent ‘ideas into action’ ethos of entrepreneurship

learning outcomes.

27

http://www.oecd.org/pisa/pisaproducts/PISA%202012%20framework%20e-book_final.pdf 28

http://www.oecd.org/finance/financial-education/oecdpisafinancialliteracyassessment.htm, cited 25/3/2014

http://www.oecd.org/pisa/pisaproducts/PISA%202012%20framework%20e-book_final.pdf

15

2.4.3 Planned developments in data sources

A number of developments are planned for identified data sources:

■ Eurydice Q&A on Entrepreneurship Education at School in Europe: To date, the study

has only been undertaken on one occasion (2012). Plans exist for the Q&A process to

be developed as a full study to be undertaken in 2014/15. It is expected that a more

extensive process of survey (breadth and depth) will be undertaken and the potential

exists for extension of coverage to VET and higher education.

This more extensive process has the potential to expand the detail and breakdowns on

policy inputs and to investigate entrepreneurial learning outcomes in greater detail29

.

Also, DG EAC will further collaborate with Eurydice in its next study, so as to get data to feed

the indicators set. In the next Eurydice study, entrepreneurial learning outcomes will be

examined in detail.

■ Flash Eurobarometer (2012): Last undertaken in 2012, the entrepreneurship survey is

expected to be repeated in 2015 providing the opportunity for further refinement of

questions (although not of number of questions);

■ Global Entrepreneurship Monitor (GEM): Annual Population Survey: Undertaken on an

annual basis, continued expansion of participating countries is expected; and

■ Programme for International Student Assessment (PISA): In 2015, it will include

assessment of collaborative problem solving for the first time.

2.4.4 Summary key gaps and challenges

While a number of robust international datasets exist in support of monitoring

entrepreneurship education and its impacts, substantial key gaps and challenges exist,

including:

■ Lack of coverage of the full set of entrepreneurship education activities taking place – at

all educational levels and in all learning environments;

■ The current inability to report against a single individual or cohort of individuals through

the progression model of entrepreneurship education i.e. from education through

learning outcomes to entrepreneurial activity;

■ The weakness that self-efficacy is not accounted for in current repeat survey activity; and

■ That no robust and credible impact indicator exists for entrepreneurship education at the

level of the population (although it does at project level).

29

Such an expansion would provide the potential to link the study results on learning outcomes to wider developments in the European Qualifications Framework.

16

3 A Framework for Indicators of Entrepreneurship Education in Europe

3.1 Introduction

The following Section provides key outputs of the DG EAC Expert Group on Indicators of

Entrepreneurial Learning and Competence, including the selection of a set of ‘priority

indicators’ for monitoring entrepreneurship education in Europe.

Taking account of validity, robustness, coverage and data availability, this Section provides:

■ A discussion and assessment of the current range of indicators available for measuring

entrepreneurship education at EU level;

■ A proposed set of Priority Main and Sub-Indicators for Entrepreneurship Education for

EU Level Monitoring; and

■ Recommendations on improved indicator and data development in the medium and long

term.

3.2 Currently available indicators for monitoring entrepreneurship education at EU Level

Given the sources and approach to monitoring entrepreneurship education outlined in

Section 2, Table 3.1 overleaf provides an overview of available indicators of

entrepreneurship education at EU level that can be generated from current robust data sets

and that cover most (if not all) of the Member States. The analysis below reviews each

indicator in detail.

3.2.1 Policy infrastructure (or inputs) indicators

Generically, given the early stages of development of systems of entrepreneurship education

across many Member States and substantial levels of educational reform, it is useful and

important to have ‘early stage’ indicators indicative of the emerging prioritisation and

implementation of entrepreneurship education.

Review of current data reveals the substantial limitations on data availability and highlights

that only indicators that are qualitative in nature (‘do key policy documents exist’) rather than

quantitative (for example, Member State’s ‘% of expenditure on entrepreneurship education’)

are available at the present time.

Furthermore, in any qualitative process of review of central steering documents it needs to

be recognised also that country specific governance styles (for example centralised or

localised) need to be accommodated.

EXISTING INDICATOR: There is a specific national (regional) strategy for the

implementation of entrepreneurship education into general school education and / or

objectives related to entrepreneurship education are included as a part of a broader

education strategy.

GHK (2011) and Eurydice (2012) note the importance of policy statements (of one form or

another) as the basic building block for the systemic delivery of entrepreneurship education.

Put another way, it is generally the need to report on such strategies, action plans or national

initiatives that generate the requirements for monitoring indicators.

This qualitative indicator recognises that statements of policy may be found in a diversity of

types of central steering document and focuses specifically on the incorporation of

entrepreneurship in to education strategy and objectives.

Data Source: The indicator can be generated from Eurydice Q & A on Entrepreneurship

Education at School in Europe (2012).

The Eurydice study is due to be repeated in 2014/2015 utilising a stronger research

methodology and greater breadth and depth of coverage of educational levels and types

(see Section 2.4.3 above).

17

Table 3.1 Assessment of current indicators for monitoring entrepreneurship education at EU level

Stage Indicator Description Data Source and Coverage Limitations

Policy Infrastructure (qualitative indicator)

There is a specific national (regional) strategy for the implementation of entrepreneurship education and/or objectives related to entrepreneurship education as part of a broader education strategy

Eurydice Q&A on Entrepreneurship Education at School in Europe (2012): covered 31 European countries at primary (ISCED 1) and general secondary education (ISCED 2 /3)

Limited information on ‘traction and extent’ of the strategy, including assessment of the position of entrepreneurship education in broader employment and/or economic growth strategies i.e. has entrepreneurship education achieved cross-departmental status in policy documents?

Policy Infrastructure (qualitative indicator)


Eurydice Q&A on Entrepreneurship Education at School in Europe (2012): covered 31 European countries at primary (ISCED 1) and general secondary education (ISCED 2 /3)

One of the strongest indicators of embedment is explicit inclusion within curricula – but a fully common European understanding and approach to the learning outcomes of entrepreneurship education is still in development.

Activity % of the population age 15 and over who have taken part, at school or university, in any course or activity about entrepreneurship (defined as turning ideas into action, developing your own projects)

EU Flash Barometer (2012) Entrepreneurship in the EU and Beyond: covered 40 countries including all Member States

Only learning experiences of the population obtained in school/university are covered despite an agreed and much broader definition of entrepreneurship education – and it is not known when activity took place at the level of the individual.

Outcome % of the population 18-64 who believe they have the necessary skills or knowledge to start a business

Global Entrepreneurship Monitor: Annual Population Survey (2012): covered 69 countries including the large majority of Member States

The entrepreneurial outcome measured is only a narrow business start-up concept of entrepreneurship. No direct relationship exists between this outcome and the individuals’ previous experience of entrepreneurship education The issue of self-assessment and ‘self-efficacy’.

Outcome % of the population age 15 and over who “strongly/totally agree” that their school education helped to develop a sense of initiative/sort of entrepreneurial attitude

EU Flash Barometer (2012) Entrepreneurship in the EU and Beyond: covered 40 countries including all Member States

A direct connection to experience of (only) school education is made but the issue of self-assessment and self-efficacy remains, including that education could have occurred many years in the past.

Impacts No robust indicator exists Robust assessment of impact is deemed to be the ability to directly link entrepreneurial learning activities with outcomes and entrepreneurial action – whether for your employer, in economic or social venture creation or in society

No suitable data sources exist Case study evidence exists for such impacts for the individual but no robust indicator is available at the level of the population

Impact data currently do not hold suitable levels of validity and robustness at the level of the European population in terms of linking impacts directly to the activities of entrepreneurship education.

18

EXISTING INDICATOR: Learning outcomes (attitudes, skills, knowledge) related to

entrepreneurship education are explicitly stated in the national (regional) curricula.

Within education, one of the strongest indicators of embedment is explicit inclusion within

curricula.

Whilst a fully common European understanding and approach to learning outcomes of

entrepreneurship education is still in development, the indicator draws from Eurydice (2012)

which has utilised the common EU approach of key competences and attitude, skills and

knowledge learning outcomes to assess their inclusion within curriculum/central steering

documents across Member States.

Data Source: The indicator can be generated from Eurydice Q & A on Entrepreneurship

Education at School in Europe (2012).

The Eurydice study is due to be repeated in 2014/2015 utilising a stronger research methodology

and greater breadth and depth of coverage of educational levels and types (see Section 2.4.3

above). It is expected also that further developments in common European measurement of

learning outcomes of entrepreneurship education will have taken place.

3.2.2 Activity indicator

In general, activity indicators are seeking to capture the experience of entrepreneurship

education by European citizens including, if possible, across different population segments (such

as young people) and through different delivery channels (such as school or higher education).

EXISTING INDICATOR: % of population age 15 and over who have taken part, at school or

university, in any course or activity about entrepreneurship (defined as turning ideas into

action, developing your own projects).

This indicator provides an understanding of how many European citizens have experienced

entrepreneurship education – although it is not known when this took place.

In particular, the indicator asks about an individual’s experience of best practice educational

entrepreneurship activity, namely that based on ‘doing’ (‘ideas into action’) – although only

learning experiences of the population obtained in school/university are covered despite an

agreed and much broader definition of entrepreneurship education (see Section 2.2 above).

Data Source: The indicator relates directly to a question contained within EU Flash

Eurobarometer (2012) and allows further understanding of which citizens – for example, by

gender, age, residence, employment status – across the EU27 Member States have had the

experience, although such breakdowns are not possible consistently at Member State level.

3.2.3 Outcomes indicators

Outcomes indicators are powerful in their ability to assess if the experience of entrepreneurship

education has had the desired effects. Within entrepreneurship education, for example, do EU

citizens who receive entrepreneurship education subsequently possess the learning outcomes of

attitude, knowledge and skills of entrepreneurship - and which will form the foundation for

subsequent entrepreneurial behaviour?

EXISTING INDICATOR: % of the population 18-64 who believe they have the necessary

skills or knowledge to start a business

Drawn from the highly robust and annual Global Entrepreneurship Monitor carried out within and

beyond Member States, this outcome indicator provides an understanding of the adult population

(including by age, gender, etc.) who believe they could undertake an entrepreneurial action in the

field of the economy; namely, start a business.

Two major limitations should be noted, however:

■ The entrepreneurial outcome measured is limited to a narrow business start-up concept of

entrepreneurship – other economic and, of course, social outcomes are generated by

entrepreneurship education; and

19

■ No direct relationship exists between this outcome and the individuals’ previous experience of

entrepreneurship education. In other words, they may hold this belief of competence but have

had no previous entrepreneurship education experience and/or not view their belief in their

skills as related to any previous educational experience.

This raises the issue also of ‘self-efficacy’ in any individual’s perception of their skills and

capabilities. As an outcome indicator of ‘belief’ ideally the indicator should seek to ‘control’ for

those individuals who have a stronger innate sense of their efficacy to achieve tasks

notwithstanding any educational or training experience.

Data Source: The indicator relates directly to a question contained within Global

Entrepreneurship Monitor: Annual Population Survey.

EXISTING INDICATOR: % of population age 15 and over who “strongly/totally agree” that

their school education helped to develop a sense of initiative/ sort of entrepreneurial

attitude

This outcome indicator provides assessment of the entrepreneurship education learning outcome

of ‘attitude’.

The indicator makes a direct connection to actual experience of entrepreneurship education –

although only school education, and only in a general sense - and also has the same issue of

‘self-efficacy’ as noted for the previous indicator above.

The issue of time exists also as, for the population as a whole, school education could have

occurred many years in the past.

Data Source: The indicator relates directly to a question contained within EU Flash

Eurobarometer (2012) and allows further understanding of which citizens – for example, by

gender, age, residence, employment status – across the EU27 Member States have had the

experience, although such breakdowns are not possible consistently at Member State level.

3.2.4 Impacts

The expert opinion of the Expert Group is that current indicators that have at times been used as

indicators of the impact of entrepreneurship education – such as new firm formation or levels of

social volunteering – currently do not hold suitable levels of validity and robustness at the level of

the European population.

These impacts can be valid at, for example, case study, project and programme level - where an

individual’s or cohorts’ journey from entrepreneurship education activity through achievement of

competences to their use in entrepreneurial actions, which generate impacts, has the potential to

be directly traced (or strongly inferred).

At the current time, at European level, any such similar impact indicators cannot be linked directly

to the activities of entrepreneurship education in a credible manner.

3.2.5 Mapping Current Indicators for EU Entrepreneurship Education

Notwithstanding the limitations of current indicators, a Statistical Annex in this Report provides a

set of spread-sheets that detail the indicator values for currently available indicators by Member

State contained within Table 3.1. In combination, these provide a mapping of the current

European state of play for entrepreneurship education.

20

3.3 A set of Priority Main and Sub-Indicators for entrepreneurship education for EU-level monitoring

Section 3.2 has assessed the current range of indicators available to monitor entrepreneurship

education at EU level. Drawing from this assessment, and the knowledge of potential data

developments as set out in Section 2.4, the following Section 3.3 utilises the DG EAC Joint

Assessment Framework approach to indicators and benchmarks to identify a set of priority

indicators (one Main and a few Sub-Indicators) suitable for EU-level monitoring of


Specifically, this DG EAC methodology concentrates monitoring on a small set of robust

indicators – normally a quantitative main indicator (with a small number of sub-

groups/breakdowns) and a few associated quantitative and qualitative sub-indicators to shed

further light on the phenomenon in question.

The set of Priority Indicators and their data sources are summarised in Table 3.2 overleaf and the

analysis below reviews each indicator in detail.

It should be noted that the set of Priority Indicators takes account of a new potential data

source – an additional survey module to existing international survey activity on

entrepreneurship education - developed as part of the work of the Expert Group and to be

piloted in Summer 2014 (see Annex 3 for a draft survey instrument).

3.3.1 Identifying the Main Indicator

The difficulty of a Main Indicator based on learning outcomes…

From the outset, this Report and the work of the Expert Group has stressed that

entrepreneurship is a key competence. In line with the monitoring of existing benchmarks in other

areas of key competences, notably the benchmark on basic skills of 15 year-olds in literacy,

numeracy and science, the main indicator for entrepreneurship education and competences

should ideally be one based on a direct assessment of learning outcomes.

Currently available indicators are not based on a direct assessment of entrepreneurship learning

outcomes and as self-reported measures they include the problem of addressing self-efficacy.

The OECD Programme for International Student Assessment (PISA) is based upon direct

assessments of competences and earlier sections have identified two competences under the

programme that have a relation to entrepreneurship; the assessment of “financial literacy” in

PISA 2012, and “collaborative problem solving” from PISA 2015.

These upcoming measures from PISA capture some elements of entrepreneurship key

competences, but it is the view of the Expert Group that they do not cover entrepreneurship key

competences fully enough to be used as a main indicator for learning outcomes in this area. This,

in turn, generates a Recommendation by the Group.

Recommendation

Common and widespread agreement on the identification and assessment of the competences

and learning outcomes of entrepreneurship education remains a key requirement – both in terms

of an agreed spectrum of competences and the measurement of such competences through

methods of direct assessment.

It is recommended that the OECD takes this activity forward through further developments in

PISA (building on on-going work such as the assessment of “collaborative problem solving”) and

other suitable tools as a key support to the widespread adoption of entrepreneurship education in

and beyond Europe. This work can also benefit from the foreseen European Commission

initiative on developing a framework for entrepreneurship competences.

It should be underlined that, ideally, future steps should include assessments targeted to different

age groups, as entrepreneurial competences may progress over time.

21

…leads to the selection of an activity based Main Indicator

In the absence of a Main Indicator based on learning outcomes, it is appropriate to search for a

Main Indicator based on entrepreneurship education activity. Such activity-based main indicators

are found for existing benchmarks; for example, for early childhood education and care and for

adults’ participation in lifelong learning.

MAIN INDICATOR: % of population aged 18-29 who have taken part in practical

entrepreneurial learning activity (defined as turning ideas into action and/or developing

your own project to achieve a goal) as part of their primary/secondary/tertiary education

The indicator is a slight expansion of that which already exists, with a continued focus on the

experience of best practice entrepreneurship education - based on the opportunity of ‘turning

ideas into action and developing your own project’ – and for a key cohort of the population.

The potential exists for this indicator to be generated in the relatively short term through a small

additional survey module to existing international survey activity such as Global Entrepreneurship

Monitor Annual Population Survey. A pilot module is to be tested in 2014 for potential main data

collection in 2015. Annex 3 provides a draft survey instrument.

3.3.1.1 Sub-groups/breakdowns of the Main Indicator

For the main indicator the following disaggregation will be applied at EU level:

■ Gender;

■ Education level of the entrepreneurial activity:

– School level (ISCED2011, 1-3);

– Post-secondary and tertiary education (ISCED2011, 4-8).

Once the new data collection is put in place there may be scope for further refining the education

level breakdown, by identifying participation in entrepreneurship education as part of vocational

education and training.

The use of sub-groups/breakdown at Member State level will depend on sample size and

reliability analysis (these will be tested on the data generated by the new survey module).

22

Table 3.2 Priority Main and Sub-Indicators for Entrepreneurship Education for EU-level Monitoring

Priority Indicators for Entrepreneurship Education for EU-level Monitoring

Main Indicator

% OF POPULATION AGE 18-29 WHO HAVE TAKEN PART IN A PRACTICAL ENTREPRENEURIAL LEARNING ACTIVITY ( DEFINED AS TURNING IDEAS INTO ACTION AND/OR DEVELOPING YOUR OWN PROJECT TO ACHIEVE A GOAL) AS PART OF THEIR PRIMARY/SECONDARY/TERTIARY EDUCATION

Indicator breakdowns: gender; education level of the learning activity (primary/secondary (ISCED2011, 1-3), post-secondary/tertiary (ISCED2011, 4-8))


Qualitative Sub-Indicator Quantitative Sub-Indicator

There is a specific national (regional) strategy for the implementation of entrepreneurship education in general school education and/or objectives related to entrepreneurship education as part of a broader education strategy


% of population age 18-29 who have taken part in a practical entrepreneurial learning activity (defined as turning ideas into action and/or developing your own project to achieve a goal) outside of their primary/secondary/tertiary education




Educators’ training in entrepreneurship education (definition of indicator to be elaborated depending on data developments)

Data Source: To be explored if inclusion can be made in OECD Teaching and Learning International Survey (TALIS, 2018)

There is a policy and/or framework for educator education in entrepreneurship education


% of 15 year-olds below intermediate proficiency level in collaborative problem solving

Data Source: OECD Programme for International Student Assessment (PISA) assessment of “collaborative problem solving” (2015)

Gain from entrepreneurial learning on entrepreneurial activity of the population age 18-29 years in the last 12 months


23

3.3.2 The Quantitative Sub-Indicators

The selection of the quantitative sub-indicators aims to complement the Main Indicator and shed

further light on entrepreneurship education activity and competences.

1st

QUANTITATIVE SUB-INDICATOR: % of population age 18-29 who have taken part in a

practical entrepreneurial learning activity (defined as turning ideas into action and/or

developing your own project to achieve a goal) outside of their primary/secondary/tertiary

education.

The first sub-indicator acknowledges the importance of all forms of entrepreneurial learning and,

together with the Main Indicator, will provide full coverage of entrepreneurship education activity.

The indicator is expected to become available in 2015 (in line with the Main Indicator data source

developments discussed in 3.3.1 above).

2nd

QUANTITATIVE SUB-INDICATOR: Educators’ training in entrepreneurship education

(to be specified based on future data developments).

The second sub-indicator acknowledges the importance of educators in fostering high quality

entrepreneurial learning activity.

The indicator would measure the extent to which educators in Europe are receiving training in

entrepreneurship education at their initial entry point in to the profession.

It is important to note that this sub-indicator is dependent upon future developments in the OECD

Teaching and Learning International Survey (TALIS) and that the potential expansion of the

TALIS questionnaire is a suggestion of the Group and its realisation depends on the OECD.

Data from TALIS will only be available at the earliest in 2019.

3rd

QUANTITATIVE SUB-INDICATOR: % of 15 year-olds below intermediate proficiency

level in collaborative problem solving

The third sub-indicator is based on the foreseen 2015 OECD Programme for International

Student Assessment (PISA) assessment of “collaborative problem solving”.

Although not the same as entrepreneurship competence, “collaborative problem solving” is a

competence which supports entrepreneurial action. It can also be seen as a possible first step

towards a more comprehensive assessment of entrepreneurship competences within PISA. This

indicator will be an outcome of PISA 2015, with data becoming available in 2016/17.

The indicator is more useful than the other soon-to-be-available competence under PISA of

“financial literacy”: “collaborative problem-solving’” is a more transversal skill with greater

applicability across (especially younger) ages.

4th

QUANTITATIVE SUB-INDICATOR: Gain from entrepreneurial learning on

entrepreneurial activity of the population age 18-29 years in the last 12 months.

A key concern in the field of entrepreneurship education is that any impacts (and impact

indicators) can be directly linked to the activities of entrepreneurship education. The fourth sub-

indicator links entrepreneurial learning with entrepreneurial activity.

This indicator provides a robust measure of additional entrepreneurial impacts that have occurred

in Europe in the previous 12 months due to entrepreneurial learning. Put another way, this

indicator can measure the difference that entrepreneurship education has made on

entrepreneurial activity by the target population.

This indicator is based on the aforementioned new survey-module (first results are expected in

2015). The measure is robust due to two key aspects:

24

■ By sharing the same data collection activity as that for assessment of activities and

outcomes, a direct relationship for individuals can be traced from education to outcomes to

entrepreneurial activity; and

■ The indicator is generated through the data in a manner which allows issues beyond

entrepreneurship education which influence impact to be controlled for (for example, self-

efficacy and demographic characteristics)30

.

A further aspect to this sub-indicator would be to allow for the development of questions for the

three main types of entrepreneurship: start-up activity, intrapreneurship and social

entrepreneurial action.

3.3.3 The Qualitative Sub-Indicators

Qualitative sub-indicators on the policy infrastructure (inputs) are included to support the

understanding of patterns in observed countries.

The qualitative indicators will be based on the Eurydice study foreseen to be carried out next in

2014/2015. The study is an improvement of a Q&A data collection carried out by Eurydice in

2011 (published 2012) utilising a stronger research methodology and seeking greater breadth

and depth of coverage of educational levels and types (see section 2.4).

The qualitative sub-indicators will cover primary and secondary school, and it is also

recommended to seek to cover tertiary education. Future developments in cooperation with

Cedefop and ReferNet might also allow for coverage of VET policies in this area.

1st

QUALITATIVE SUB-INDICATOR: There is a specific national (regional) strategy for the

implementation of entrepreneurship education into general school education and / or

objectives related to entrepreneurship education are included as a part of a broader

education strategy.

Strategies are important building blocks for the systemic delivery of entrepreneurship education.

Strategies are official policy documents developed from competent authorities that discuss a

significant area of policy making and draw a general and detailed outline of steps to be taken for

a specific (usually meso/macro) period of time. Depending on the governmental structure, they

can be national, regional or even local. Strategies can be comprised of a vision, identify

objectives and goals (qualitative and/or quantitative), describe processes, authorities and people

in charge, funding sources, make recommendations etc. A strategy also determines the

partnerships that need to be constructed: between authorities, authorities and stakeholders,

between stakeholders themselves, etc. Given the broadness of a strategy, partnerships should

be considered as one of its integral parts that formulate the way the strategy will be developed

and its goals achieved31

. So, a strategy on entrepreneurship education is a relevant policy

document that aims at promoting this type of education in the country/region32

.

Entrepreneurship education strategies can regard all or some of the education levels.

The issue of strategies in higher education is different than in other education levels, given the

level of autonomy that higher education institutions enjoy across European countries. In order to

address policies in higher education, an additional sub-indicator could be included under the

1st Qualitative Sub-indicator, relating to the existing funding regimes in higher education that

support the promotion of entrepreneurship education.

30

This analytical methodology has been applied previously in GEM (2008) A Global Perspective on Entrepreneurship Education and Training, GEM Special Report 31

The inclusion of partnerships in the definition allows for the proposed improved measure of an indicator on strategic partnerships in entrepreneurship education to be considered as part of the final indicator set. 32

This definition of an entrepreneurship education strategy should not be considered “final”: in its forthcoming study, Eurydice is expected to provide an official and more elaborate definition of an entrepreneurship education strategy.

25

It should be noted that a strategy is different than an entrepreneurship education

programme/initiative in several ways: a programme/initiative is shorter in length; it usually covers

a smaller geographical area or one level of entrepreneurship education.

The basis for the indicator will be developed further within the forthcoming Eurydice study

methodology process but should:

■ Clearly distinguish strategy from programme and project activity more generally;

■ In the general description of the components of strategy as related to entrepreneurship

education include an expectation of strategic partnerships between public, private and non-

governmental sectors; and

■ Recognise that the existence of ‘national/regional’ strategy towards entrepreneurship

education in higher education may take a different form given greater levels of autonomy.

2nd

QUALITATIVE SUB-INDICATOR: Learning outcomes (attitudes, skills, knowledge)

related to entrepreneurship education are explicitly stated in the national (regional)

curricula.

Within education, one of the strongest indicators of embedment is explicit inclusion within

curricula.

Whilst a fully common European understanding and approach to learning outcomes of

entrepreneurship education does not exist, further developments in common European

measurement of learning outcomes of entrepreneurship education are taking place.

3rd

QUALITATIVE SUB-INDICATOR: There is a policy and/or framework for educator

education in entrepreneurship education

The delivery of a comprehensive, high quality system of entrepreneurship education is highly

dependent on the awareness, engagement and professional skills of educators in relation to


This indicator would assess the extent to which policies and plans are in place to train educators

as a key component to any comprehensive good practice delivery system for entrepreneurship

education.

3.4 Summary and recommendations for improved measures

3.4.1 Summary

In seeking a robust, valid and credible framework for monitoring entrepreneurship education

across Europe it has been identified that solid cross-national data sets exist but they are limited

in their ability to ‘feed and populate’ an indicator set for EU-level monitoring at the current time.

A set of current indicators can be put forward which provide assessment of the development of

entrepreneurship education in terms of both policy and delivery structures across Europe,

although they are partial in enumerating all entrepreneurship activity.

More fundamentally, whilst current indicators can track to some extent entrepreneurial

competence across Europe they cannot adequately assess what has generated such

competence – including specifically the role of entrepreneurship education – and cannot yet

assess subsequent impacts in terms of entrepreneurial activities (breadth, depth and cause).

Given this conclusion, this section has identified a set of suggestions for improvements that the

Expert Group believes would provide a robust, valid and credible statement of entrepreneurship

education across Europe, its influence on competences and entrepreneurial activity and,

ultimately, clear measurement of ‘the difference that entrepreneurship education makes’.

As part of this process, the short and medium-term process of data development to achieve this

improved basis for monitoring has been considered, given both timescales and resource

26

constraints. This has allowed the Group to put forward a proposal for a rigorous set of Priority

Main- and Sub-Indicators under the DG EAC Joint Assessment Framework methodology.

3.4.2 Recommendations

To achieve the Priority set of Main- and Sub-Indicators would require the following developments:

■ New indicators on entrepreneurial learning activity and gain from entrepreneurial learning

using data collected through a small survey module for inclusion in an existing international

survey vehicle (a draft survey instrument is provided in this Report);

■ An initial indicator on collaborative problem solving using results of OECD Programme for

International Student Assessment (PISA, 2015) to be published in 2016;

■ A small expansion in the range of qualitative policy input indicators collected in Eurydice

(2012) and which is due to be repeated in 2014/15; and

■ A new indicator on educator development – to be explored through discussion with OECD

Teaching and Learning International Survey (TALIS) about the potential to insert questions to

teachers on entrepreneurship education training for 201833

.

Recommendation for longer-term improvement of monitoring of learning outcomes

Common and widespread agreement on the identification and assessment of the competences

and learning outcomes of entrepreneurship education remains a key requirement – both in terms

of an agreed spectrum of competences and the measurement of such competences through

assessment rather than self-reporting.

It is recommended that the OECD takes this activity forward within PISA and other suitable tools

as a key support to the widespread adoption of entrepreneurship education in and beyond

Europe.

33

It should be noted that the potential expansion of the TALIS questionnaire is a suggestion of the Group and its realisation depends on the OECD.

27

4 Monitoring Entrepreneurship Education at Member State Level

4.1 The state of play: a short review

Although interest in entrepreneurship education has grown significantly within the policy agendas

of Member States, there is room for improvement regarding measuring progress, performance

and the impact of relevant education policies, projects and initiatives: for example, only about half

of EU Member States monitor entrepreneurship education at all. In countries that do, monitoring

takes place either through an established monitoring framework that promotes systematic

evaluation or through the collation of indicators from national or regional statistics offices (e.g.

Regional Statistical Bureaus in Germany), national/regional surveys, and international surveys

(such as GEM and Flash Eurobarometer). Overall, however, monitoring and evaluation

frameworks and associated indicators in Member States are, at best, “in their infancy”.34

Monitoring frameworks can be found in countries/regions that have a targeted strategy on

entrepreneurship education. More specifically:35

■ From the 11 EU countries that have such a strategy in place, about one third ((Flemish

community of Belgium, Croatia, Denmark and Wales) monitor entrepreneurship education in

a systematic way. The way these frameworks function, their focus and breadth does vary

considerably;

■ Another third of those countries collect relevant indicators on a systematic (annual) basis.

This is the case, for example, in Finland and Sweden;

■ The rest of the countries that have a strategy in place do not monitor entrepreneurship

education through any tool or framework.

Approximately half of the EU countries that do not have an entrepreneurship strategy in place

collect indicators and/or run ad hoc evaluations and surveys on specific education programmes

and initiatives (for example, in Austria, England, Poland and Slovenia).

4.1.1 Indicators collected in Member States

Member States choose to collect a wide range of indicators whether for systematic or ad hoc

evaluations. GHK (2011) underlined that:

■ Around 30 indicators are collected across Member States, while 20 of them are the most

common;

■ Indicators are collected for all stages of the intervention logic for entrepreneurship education

(as analysed in Section 2.3). Most indicators focus on the potential impact of

entrepreneurship education:

– Impact indicators are mainly drawn from the annual GEM survey and examine

developments in start-up rates. Indicators that try to explicitly link entrepreneurship

education with firm creation have been found from other sources, too; in Germany, a

singular, annual survey includes the indicator “% of students who start up a business

after 3/4 years of entrepreneurship education completion”.

■ Some Member States (for example Belgium (Flanders), Germany, and Sweden) also share

‘outcomes’ indicators, assessing the development of entrepreneurship competences; and

■ A number of Member States also share a measure regarding the percentage of the working

population who has received entrepreneurship education.

34

GHK (2011) 35

The analysis for existing frameworks and indicators collected is mainly based on GHK (2011). The study did not examine thoroughly all Member States, but rather focused on 10 Member States/regions: Belgium (Flanders), Denmark, Finland, Ireland, the Netherlands, Germany (Baden-Württemberg), Poland, Spain (regional), Sweden, UK (Devolved Administrations and national programmes).

28

4.2 Challenges for Member States in developing and running a monitoring framework

Entrepreneurship is a multi-faceted education area that can take multiple forms, involve a great

variety of stakeholders and touch upon other policy areas. Bearing in mind national/regional/local

goals and specificities, Member States may a face a number of challenges while developing,

running and ensuring the sustainability of a monitoring framework for entrepreneurship education.

For example:

■ The sustainability of the monitoring system is highly dependent on the commitment of the

policy leaders that can drive change and allocate sufficient resources. When engaged policy

leaders are out of office, monitoring may not be continued;

■ An effective monitoring system should provide information and overview of all types and

forms of entrepreneurship education. The identification of entrepreneurship education across

all levels may ask for mapping of the state of play across the country/region. Otherwise,

monitoring will probably include only the most visible types/forms of entrepreneurship

education;

■ Monitoring systems may only focus on specific geographic areas and/or learners’ groups and

miss out information across all relevant cohorts;

■ Monitoring frameworks should provide valuable data regarding all outcomes of

entrepreneurship not only business creation;

■ The impact of education in general and entrepreneurship education in particular on the

actions of individuals (and subsequently economy and society) may not be evident

immediately. Measuring of such impact asks for a long-term approach in monitoring;

■ Especially in countries with little experience of monitoring/evaluation systems, stakeholders

may be resistant to the introduction and implementation of a new monitoring framework;

■ The monitoring framework may be focused only on national/regional systems, and not

facilitate cross-country/international benchmarking;

■ A monitoring framework may be collecting information relevant only to education goals and

not include indicators that touch upon other policy agendas (for example, employment and

economic growth).

4.3 Inspiring examples: Case Studies from Member States

Despite the challenges, valuable insights can be drawn from existing frameworks and monitoring

tools that are in place in some Member States. The following section presents some inspiring

case studies that have been presented and discussed in the Expert Group meetings36

.

Each Case Study can be mapped by its position within the Group’s proposed monitoring

framework providing illustration of depth and breadth of monitoring activity and responses to the

challenges of monitoring (see Figure 4.1).

It can be seen, for example, that the work of the European Training Foundation (ETF) is focussed

on developing a strong input framework for entrepreneurship education whereas the example of

Finland is focused on ‘outcomes for individuals’ and, in this illustration, outcomes for teachers.

Young Enterprise Denmark is seeking to monitor from inputs to impacts but is especially

innovative in its assessment of the key (and difficult) relationship between education and learning

outcomes. Most recently, Wales has put in place a complete monitoring framework for youth

entrepreneurship.

36

The case studies are presented in detail in Annex 5.

29

Figure 4.1 Monitoring Entrepreneurship Education Case Studies by Position in the Monitoring Framework

Source: ICF GHK

30

4.3.2 European Training Foundation: Monitoring the policy inputs of entrepreneurship education

European Training Foundation (ETF): Monitoring the policy inputs of entrepreneurship education

As part of the Policy Index for the Small Business Act, ETF has created a monitoring framework for

the development of policy, structures and delivery frameworks for entrepreneurial education and

learning. This framework is known as the Human Capital Indicators for the Pre-Accession

Region in relation to the Small Business Act for Europe37

.

The framework comprises 23 indicators each of which is measured on a 5-level cumulative scale

over a maximum of 3-4 years.

Indicators are cumulative allowing assessment of progress in the development of the policy

framework (inputs) in any one country and multi-country benchmarking as an impetus is for

exchange and cooperation, structured networks and sharing of good practice and lessons.

37

ETF/SEECEL (2011) Small Business Act for Europe. Human Capital Indicators for the Pre-accession Region 2011. Available at: http://www.seecel.hr/UserDocsImages/Documents/Documents%20Section/SBA%202011%20Indicators.PDF [Accessed 12 June 2013].

http://www.seecel.hr/UserDocsImages/Documents/Documents%20Section/SBA%202011%20Indicators.PDF

31

4.3.3 Monitoring entrepreneurship education ecosystem inputs: supporting Finnish teachers

Monitoring entrepreneurship education ecosystem inputs: supporting Finnish teachers through a self-evaluation tool38

Teachers are key to the delivery of entrepreneurship education but, often, are themselves

uncertain of their professional development concerning entrepreneurship education. With the

Measurement Tool for Entrepreneurship Education, teachers can evaluate their performance, receive

immediate systematic feedback and further develop their teaching practices related to


The measurement tool is an easy-to-use, Web-based self-evaluation tool, which helps teachers

develop their know-how in entrepreneurship and enterprise education.

Additionally, the macro-level results of the Measurement Tool provide monitoring data to support

national and regional policy making on entrepreneurship education.

The tool was deliberately developed jointly by researchers, national authorities, teachers and

practitioners to support adoption.

Use of the Finnish version of the tool is already widespread and the tool is also available in Swedish,

with an English version currently in development.

4.3.4 The Netherlands Actionprogramme Education and Enterprise (AEE): creating an entrepreneurial pipeline

The Netherlands Actionprogramme Education and Enterprise (AEE): creating and tracking an entrepreneurial pipeline

The Ministry of Economic Affairs and Ministry of Education have joined together in a ‘pipeline’

approach to increase entrepreneurship education in the Netherlands and generate more enterprising

pupils and students who will start their own businesses within 5 years of finishing education.

AEE has involved a substantial range of entrepreneurial education activities across education

systems and stakeholders – from primary to university, teacher training and incubation. Over 75,000

students had been engaged directly in projects financed through the action programme, 4,400

teachers and over 500 schools;

Longitudinal monitoring and evaluation is a key component of the programme as it has sought to

understand how students in Netherlands experience entrepreneurial education and move through a

ladder of entrepreneurial learning outcomes (awareness, knowledge, skills, behaviour) and,

ultimately, potential action in the economy.

Repeat surveying of pupils/student recipients of entrepreneurship education is taking place on a bi-

annual basis. The research methods have included internet surveys among 706 schools (which

include universities) and 1,241 students in middle and higher vocational education; 35 in-depth

interviews with school managers; and a selection and description of 16 best practices.

38

See http://developmentcentre.lut.fi/english.asp?show=yrittajyyskasvatus

http://developmentcentre.lut.fi/english.asp?show=yrittajyyskasvatus

32

4.3.5 Young Enterprise Denmark: monitoring learning outcomes

Young Enterprise Denmark (FFE-YE): monitoring learning outcomes and impacts over time

The Danish Foundation for Entrepreneurship – Young Enterprise (FFE-YE) is responsible for

implementation of the Danish national strategy for entrepreneurship education.

Since establishment in 2010, through annual longitudinal impact surveys, FFE-YE has been mapping

the national spread of entrepreneurship education including seeking to measure the impact of

entrepreneurship education on outcomes with regard to skills, competences, and motivation for

entrepreneurship.

FFE-YE measures the effects of entrepreneurship education in terms of behaviour through

longitudinal surveys at three levels:

■ 9th grade - lower secondary/basic school (initially 2,000 individuals surveyed and to be

expanded and continually repeated);

■ Company Programme - higher secondary

■ Universities - mainly graduate level (550 students including a control group)

FFE-YE is seeking to understand the longevity of learning outcomes and if they lead to

entrepreneurial actions and impacts (‘the difference that entrepreneurship education makes’. It is

using highly robust research methods to both assess the impacts of different teaching methods, take

account of self-efficacy, and seek to create a ‘policy off’ (or counterfactual) baseline amongst a

control group of students who have not received entrepreneurship education.

4.3.6 The Welsh Youth Entrepreneurship Strategy: monitoring across all stages

The Welsh Youth Entrepreneurship Strategy: monitoring a strategy

The cross-governmental Youth Entrepreneurship Strategy (YES) outlines an Action Plan, spanning

from 2010 to 201539

, which delivers initiatives and programmes across three delivery themes.

Delivery Theme Initiative

Engage: the value of entrepreneurship

Big Ideas Wales (website and portal)

The Dynamo Programme (mobile entrepreneurship

role model programme)

Empower: entrepreneurial learning

opportunities

National Enterprise Primary School Competition

The ACRO model (entrepreneurship skills

development)

Linking Education and Economy: Entrepreneurial Hubs

and National CPD

Equip: support to create and grow

businesses

Promoting start-ups (boot camps with existing

businesses)

The three themes of YES (Engage-Empower-Equip) have been used to develop an evaluation

framework for the Strategy. Two sets of measures are reported against:

■ The YES Action Plan Measures – which outlines inputs, outputs and impacts; and

■ The YES Strategic Impact Measures – quantifying the impacts, and tracking progress.

Annual measurement data is drawn from sources including national surveys, on-line tools,

questionnaires, GEM surveys, and the Carnegie Trust Enterprising Survey.

39

http://wales.gov.uk/topics/businessandeconomy/publications/yesactionplan1015/?lang=en


33

4.4 Lessons and Recommendations to Member States

Notwithstanding the challenges, monitoring entrepreneurship education is beneficial both for

Member States and for Europe overall. Earlier sections outline potential frameworks for

monitoring entrepreneurship education at EU level, highlighting the range and coverage of

metrics required and the indicators that exist or will need to be developed to support such a

framework.

These frameworks have been developed from current data collection and monitoring activity at

Member State level. Although each Member State should adjust its monitoring framework to

national/regional specificities, recommendation can be provided to support Member States as to

what a framework for monitoring entrepreneurship education could entail and what should be

considered when creating such a framework and its constituent indicators.

The following provides such lessons and recommendations that have been drawn from analysing

practices from Member States40

.

■ Lesson 1 : Agree on a definition of entrepreneurship education

Recommendation: A coherent monitoring system is based on a single and widely agreed

definition of entrepreneurship education that will allow for transparency and comparability of

results in the long term. So, competent authorities should agree with all relevant stakeholders

and involved actors on a definition of entrepreneurship education that will underline its broad

concept.

In Spain, autonomous communities are responsible for implementing education policies. Despite the highly devolved governance system of the country, the main education law (Organic Law 2/2006) that refers to all education levels and describes national curricula introduces the concept of entrepreneurship. The Regional Ministries of Education put in practice the law in the different autonomous communities and base entrepreneurship education initiatives/programmes on the common definition

41.

■ Lesson 2: An action plan or strategy for entrepreneurship education should have a

monitoring framework built in

Recommendation: Especially relevant for countries that have not yet developed an

entrepreneurship education strategy, the monitoring framework should seek from the start to be

an integral part of the entrepreneurship education strategy and/or policies. This means that the

strategies and policies should incorporate the use of indicators and monitoring logics in line with

an overall monitoring framework, instead of a fragmented approach with a proliferation of

definitions and ad-hoc indicators.

■ Lesson 3: Monitoring systems should be integral to and recognised in the education

system

Recommendation: If monitoring is not integral to the education system itself, then it will not be

recognised or sustainable. Indicators and measures should be built into existing educational

monitoring to ensure it is an integral aspect of organisational processes.

This is the case in the Netherlands: although the monitoring system in place touches upon different aspects and levels of entrepreneurship education, it is not a part of the education system per se. This is also evident from the fact that the impact measures collected focus only on business creation and do not include purely educational goals.

■ Lesson 4: The importance of involvement of Ministries of Education and cross-

ministerial cooperation

40

A table with the key lessons and recommendations is available in Annex 5. 41

GHK (2011)

34

Recommendation: Concerning entrepreneurship education, engagement of education policy

makers is critical. Monitoring frameworks can have strengthened visibility, legitimacy and impact

through a close cooperation with other relevant ministries (e.g. Ministries of economy and

employment).

Member States should engage the education authorities at all stages of the development and

implementation of the monitoring system - hopefully as part of cross-governmental collaboration.

It is important to ensure that the education agenda and objectives are reflected in the monitoring

system, so the education authorities should have a leading role. The impact interests of different

policy areas/ministries need to be reflected, however with a balance between focus on education

and focus on economic outputs (such as, for example, business start-up rates).

Collaboration between various Ministries and/or governmental agencies can be found in Croatia, Denmark, Finland, the Netherlands, Sweden and Wales.

A broad range of policy authorities, besides the Ministry of Education, are involved in monitoring practices in Member States. For example in Croatia

42, entrepreneurship

education strategy and its monitoring is under the responsibility of various Ministries (Entrepreneurship and Crafts, Science, Education and Sports, Agriculture etc.) and other bodies, such as Chambers of Economy, Commerce, Crafts etc.

■ Lesson 5: Stakeholder engagement is key

Recommendation: Monitoring systems should be a joint venture designed, owned and adopted

by all relevant stakeholders – i such as public authorities, teachers, school administrations,

learners, training providers, local businesses and the community. The relevant stakeholders may

differ in each Member State and between education levels and types of education.

These stakeholders should be engaged at all stages of the design and implementation of the

monitoring system.

At the same time, entrepreneurial learning activities run by some of these stakeholders should be

included in the monitoring system. In that way, monitoring tools can also function as motives for

improved performance and professional development of stakeholder groups.

Wales provides an inspiring example of engaging policy-makers.

The Youth Entrepreneurship Strategy (YES) and Action was a joint initiative of the Welsh Government Department of Business, Enterprise, Technology and Science and the Department of Education and Skills. All programmes and initiatives of YES involve local communities and entrepreneurs as key players.

YES is evaluated by two sets of measures (see Annex 4). The one includes inputs, outputs and impact indicators and the second monitors the progress of YES against quantified impact indicators. The impact indicators underline that the Strategy aims at promoting employment goals, as well, through monitoring start-up rates.

■ Lesson 6: Link the monitoring framework to wider policy areas, such as education,

economic development and innovation strategies.

Recommendation: Monitoring systems should consider entrepreneurial education as one

element within wider policy monitoring, to develop and contribute to a broader understanding of

the education and entrepreneurial ecosystem. In that way, the monitoring system can potentially

support greater understanding of national / regional entrepreneurial performance and the longer

term impact of entrepreneurship education can be studied for its contribution to wider

impacts.

42

http://www.seecel.hr/print.aspx?id=1215; http://www.e4e.com.hr/?lng=en

35

Impact measures of the Welsh YES Strategy and the Actionprogramme Education and Enterprise in the Netherlands focus on business creation indicators, which is one of the goals of the entrepreneurship education strategy, but also relates to the employment agenda.

■ Lesson 7: Monitor across the policy development process

Recommendation: It is possible to monitor at any stage in the policy development process – from

creation of inputs to global impacts – and monitoring frameworks act as a tool of policy

development, as is evident from the Case Studies (Annex 4).

When developing a monitoring system Member States should aim to cover all the different stages

of entrepreneurship education from policy development, through activity, outcomes and impact,

although some stages can be given particular attention according to national priorities and the

current state of entrepreneurship education.

Good measures of the impact of entrepreneurship education cannot be developed in isolation

from indicators of entrepreneurship activity and outcomes.

■ Lesson 8: The monitoring system should seek to include all types of entrepreneurship

education.

Recommendation: Given the ‘practical’ nature of entrepreneurship education, substantial

activities take place in various settings across education and broader stakeholder systems (for

example, work placements). Monitoring frameworks should be designed to grasp where activities

take place and who is delivering what, both within and beyond the education system.

In Flanders Belgium43

, the entrepreneurship education Action Plan is monitored through various ways that try to grasp different forms of entrepreneurship education. Monitoring takes place through:

Collection of indicators concerning entrepreneurship education activities, work placements and “performance” of relevant websites (Competento);

Studies that evaluate specific educational projects and/or entrepreneurship attitudes of students, such the Effecto report; and

Self-assessment tools, such as the ENTRE-mirror.

■ Lesson 9: Use of multiple monitoring methods and tools to grasp a broader and in-

depth understanding

Recommendation: Entrepreneurship education can take many forms and can be delivered by

multiple actors in various sectors. In order to evaluate what works across all types and levels,

multiple monitoring methods can be used alongside indicators collection (research, targeted

studies, self-assessment tools etc.). Research and targeted studies may supplement the

monitoring system and support greater understanding of differences between groups, types of

entrepreneurship education etc., as well as trends that are identified through monitoring

indicators. Furthermore there are limitations to what a set of indicators can capture.

However, any monitoring system should avoid becoming too complex and extensive, as this can

make it difficult to communicate results and assure sustainability over time.

Supplemental studies should not replace the monitoring system, since they:

– will usually focus on one education level or programme; and

– may use different definitions of entrepreneurship and its education, hindering

comparability and conclusions formulation;

43

GHK (2011)

36

Ideally, in order to address limitations and conflicts due to different views/definitions, all involved

stakeholders should align on key definitions and goals before the initiation of the study.

Flanders, Belgium provides an inspiring example of the use of different monitoring methods and tools.

■ Lesson 10 : There is a need for a long-term view when developing a monitoring system

Recommendation: Monitoring systems should follow a long-term strategic approach agreed

among key stakeholders. The long-term nature of a monitoring system also requires data

collection to be developed with the view to providing comparable data over time.

In Denmark, the effect of entrepreneurship education is measured by the Foundation for Entrepreneurship –Young Enterprise (FFE-YE)

44 through longitudinal surveys. Selected

samples of students from different levels of education are followed for a number of years. This allows for the collection of solid data that can be used for identification of trends, and facilitate forecasting of training needs. Most of all, the Danish approach allows for the evaluation of the long-term effect of learning outcomes of entrepreneurship education- although learning outcomes are not per se embedded in the national curricula

45.

A long-term view is also recognisable in the monitoring approaches in Finland, Norway, Sweden and Wales.

■ Lesson 11: Benchmarking enhances quality

Recommendation: Shared multi-country frameworks allow for benchmarking and enhanced

exchange, co-operation and learning. Member State monitoring and indicator systems should

consider their ability to support robust international benchmarking. It is therefore advisable that

countries also make use of indicators and data sources that allow for international comparisons

(for example, indicators in the proposed framework for monitoring of entrepreneurship education

at EU level).

European Training Foundation (ETF) / South East Europe Centre for Entrepreneurial Learning (SEECEL):

The ETF/SEECEL project (Human Capital Indicators for the pre-accession region in relation to the Small Business Act for Europe)

46 provides an inspiring relevant example.

The tool has dual implementation and function: it provides evidence and insights for policy development on a national level. At the same time, benchmarking across countries allows for networking, collaboration and exchange of best practices, lessons and concerns.

■ Lesson 12: A monitoring system should capture learning outcomes

Recommendation: Robust assessment of learning outcomes and linking these with

entrepreneurial learning activity and entrepreneurial actions requires careful design of monitoring

systems – including over time and allowing for self-efficacy. Member States should ensure

common agreement and understanding of clearly defined learning outcomes is in place to

support credible monitoring activity.

This is the case, for example, in the Netherlands and Denmark, as described earlier.

■ Lesson 13: Recognise the value and limitations of self-assessment tools

Recommendation: Such tools can offer significant benefits, as they can:

44

http://eng.ffe-ye.dk/about-ffe-ye.aspx 45

More information to be found in Annex 5 46

See Annex 5 for more details

37

– Support the target group (e.g. teachers) in professional development: self-assessment

tools can be perceived as development tools, rather than monitoring;

– Provide immediate and interactive way of data collection regarding what is implemented

(practices/methods); how it is perceived; etc.

– Support systematic data collection on a decentralised level: schools, region,

municipalities and also country/national level.

At the same time, it should be recognised that the results of such tools can be biased due to self-

selection; for example, the voluntary nature of the tool and reporting fatigue from participants (i.e.

teachers).

For the introduction and implementation of such tools, a bottom-up approach is recommended

as it will underline the use of the tool as a mean of personal development.

The Teachers’ Self-Evaluation Tool for Entrepreneurship Education in Finland (see section 4.3.2

above) is a relevant example.

4.5 Key messages to Member States

Monitoring activity should be understood as a continuous development process embedded in the

education system. In this sense, the lessons and recommendations put forward above are

believed to hold relevance across the variety of monitoring positions held by countries and

regions. It is the case, nevertheless, that a greater number are likely to be pertinent to those

countries and regions where development of monitoring frameworks for entrepreneurship

education remain in their infancy.

Countries/regions that do not have a monitoring framework in place should primarily gather

all relevant public authorities and stakeholders and agree on what will be monitored: the definition

of entrepreneurship education. The definition will allow for the subsequent selection of

meaningful indicators and measuring tools; the involvement of all levels and types of

entrepreneurship education; and can highlight links to policy areas beyond education.

Monitoring means continuous improvement. So, even countries/regions that already have a

monitoring framework in place may be minded to go back to these “early steps” and re-affirm any

definition of entrepreneurship education as the basis for measurement and monitoring.

The entrepreneurship education strategy/action plan, where in place, should be used as a

compass for the monitoring framework: both should be based on the same definition and address

all stakeholders. Since the monitoring framework should be embedded in the strategy,

developing or improving the framework can support improvements in the strategy and vice versa.

Countries that are about to launch an entrepreneurship education strategy have the opportunity

to simultaneously develop a monitoring framework. Existing monitoring practices, even on an ad

hoc basis, can help to shape a broader framework. Thus, a qualitative audit of how

entrepreneurship education is already monitored (through studies, evaluation of isolated

programmes etc.) can identify useful tools, which are the stakeholders at what education levels,

what capacity and expertise for monitoring exists, etc.

Ideally, a monitoring framework should be forward-looking and recognise the long term nature of

impacts within entrepreneurship education. This implies a long term developmental goal for

countries/regions to include indicators/measures that cover all stages of the logic model and a

monitoring framework that can support a culture of long-term improvement and increased

efficiency and effectiveness of entrepreneurship education policy and activity.

This links again to where monitoring frameworks are in place reviewing the lessons and

recommendations to support consolidation and expansion within better developed understanding

of “what works”. From this more confident and informed position (including of impacts):

a) The links of entrepreneurship education with other strategies and goals, regarding innovation,

competitiveness, employment and economic and regional development might be strengthened;

38

b) A mature monitoring position will facilitate the inclusion of multiple tools and measures that

have been proved efficient in delivering additional intelligence of various types of

entrepreneurship education and

c) Countries and regions can further use the monitoring framework as a performance

management system of stakeholders.

Alternatively, a strong monitoring framework at the beginning that includes performance

indicators and development tools provides a strong signal of intent.

Most importantly, countries/regions at any stage of developing a monitoring framework should

ensure that their framework is not inward-looking: including indicators from already existing

sources (national, European and international), will allow for benchmarking between regions

(where relevant), with other EU Members and global competitors. In this way, the monitoring

framework can function as a bridge between evidence-based policy on entrepreneurship

education and improved outcomes and impacts.

In summary, the above lessons provide useful guidance to Member States that seek to develop

or improve the way entrepreneurship education is evaluated. The lessons and recommendations

highlight key elements that countries and regions should keep in mind. Certain lessons may be

more important during the initiation phase of a framework, but even these can be seen to support

countries/regions using the feedback loops of the monitoring process to ensure development and

enhanced policy performance.

It should be noted that there is no “best approach or monitoring model” put forward here:

the framework and the tools used should fit the relevant national/regional structure, needs and

culture. To give one example, in Finland, it is reported that the implementation of

entrepreneurship education strategy is mostly based on mutual trust- a cultural element that

probably would not apply in a number of other countries. Nevertheless, the development of

lessons and recommendations undertaken here does assume that the benefits of benchmarking,

cross-country collaboration and an outward-looking perspective are part and parcel of defined

national/regional needs.

39

Annex 1 List of Expert Group Members

Group members are listed in alphabetical order

Expert name Affiliation

Ania Bourgeois EACEA-Eurydice

Efka Heder SEECEL

Nick Henry

ICF GHK (and Coventry University since September 2013)

Caroline Jenner JA-YE Europe

Gavril Lasku/ Anthony Gribben ETF

Jonathan Levie GEM

Maja Ljubić SEECEL

MariaRosa Lunati OECD Statistics

Martino Rubal Maseda ETF

Jim Metcalfe Carnegie Trust

Guillermo Montt* OECD Education

Kjartan Steffensen DG EAC

Elin McCallum DG EAC

Georgi Dimitrov DG EAC

Simone Baldassarri DG Enterprise

*Mr Montt from OECD, Directorate for Education, attended the 1st meeting of the Expert Group.

40

Annex 2 Statistical Annex

The Statistical Annex is attached as an Excel file.

41

Annex 3 Assessing Entrepreneurship Education: Draft survey module

The questionnaire will be further developed and adjusted as part of the piloting foreseen in 2014.

Questions for an Activity indicator:

The set below would allow for an activity indicator broken down by:

- Voluntary/compulsory;

This distinction is also important to avoid self-selection bias in a gain from training indicator.

- "primary or secondary education" (ISCED2011 level 1-3), "post-secondary or tertiary education" (ISCED2011 level 4-8).

Normally, when operationalizing questions about education levels in the national questionnaires, the levels should be translated into national terminology known to respondents.

Entrepreneurial learning as part of primary or secondary education

AFED1a. I would like to ask some questions about education and training. Have you ever taken part in any practical entrepreneurial learning activity as part of your education at primary or secondary level? This should include some activity in which you learned how to turn ideas into action and/or developed your own project to achieve a goal.

Yes ..................................................................... 1

No ....................................................................... 2

Don't know .......................................................... 8

Refused .............................................................. 9

Filter: IF YES (1) at QAFED1a. OTHERS GO TO AFED2a

AFED1b.

Voluntary…………………………………… .......... 1

Compulsory……………………………….. ........... 2

Don’t know...………………………….................. .8

Refused……….………………… ................ …….9

Filter: It should be considered to add a filter so as to only ask AFED2a to individuals with an educational attainment higher than ISCED 3 (i.e. higher than upper secondary education).

42

Entrepreneurial learning as part of post-secondary or tertiary education

AFED2a. Have you ever taken part in any practical entrepreneurial learning activity as part of your education at post-secondary or tertiary level? This should include some activity in which you learned how to turn ideas into action and/or developed your own project to achieve a goal.

Yes ..................................................................... 1

No ....................................................................... 2

Don't know .......................................................... 8

Refused .............................................................. 9

Filter: IF YES (1) at QAFED2a. OTHERS GO TO ANFE1a

AFED2b.

Voluntary…………………………………… .......... 1

Compulsory……………………………….. ........... 2

Don’t know...…………………………. ............ .8

Refused……….………………… ................ …….9

Entrepreneurial learning beyond and after primary/secondary/tertiaryeducation

A2NFE1a. Have you taken part in any practical entrepreneurial learning activity which was not part of your primary/secondary/tertiary education? This should include some activity in which you learned how to turn ideas into action or developed your own project to achieve a goal. (add examples)

Yes ..................................................................... 1

No ....................................................................... 2

Don't know .......................................................... 8

Refused .............................................................. 9

Filter: IF YES (1) at QA2NFE1a. OTHERS GO TO O1a

A2NFE1b.

Voluntary…………………………………… .......... 1

Compulsory……………………………….. ........... 2

Don’t know...………………………….................. .8

Refused……….………………… ................ …….9

Question for an Outcome indicator (improved self-efficacy measure):

O1a. Do you have the skills, knowledge and experience to create a new project, venture or initiative either for yourself, your community or an employer that involves turning an idea into action. This could be for profit or for the common good.

43

Yes ..................................................................... 1

No ....................................................................... 2

Don't know .......................................................... 8

Refused .............................................................. 9

Question for impact indicator. Need to link entrepreneurial learning with entrepreneurial

activity through Gain from Training measure. For this we need questions on entrepreneurial

activity.

I1a. In the last twelve months, you have played a leading role in starting a new project, venture or initiative either for yourself, your community or your employer (for example, developing or launching new goods or services, or setting up a new business unit, a new establishment or subsidiary) that involved turning an idea into action. This could be for profit or for the common good.

Yes ..................................................................... 1

No ....................................................................... 2

Don't know .......................................................... 8

Refused .............................................................. 9

Filter: IF YES (1) at QI1a. OTHERS END

I1b. Did you participate in this new project, venture or initiative as an employee? In other words, as an employee, were you in the last 12 months, actively involved in the development of new activities for your main employer, such as developing or launching new goods or services, or setting up a new business unit, or a new establishment or subsidiary?

47

Yes ..................................................................... 1

No ....................................................................... 2

Don't know .......................................................... 8

Refused .............................................................. 9

Filter: IF YES (1) at QI1a. OTHERS END

I1c. Was this new project, venture or initiative for profit or non-profit?

Profit ................................................................... 1

Non-profit ............................................................ 2

Don't know .......................................................... 8

Refused .............................................................. 9

47

This is the definition of Entrepreneurial Employee Activity used by GEM, as provided by the GEM member of the Group.

44

Annex 4 Monitoring of Entrepreneurship Education in Member States- Case Studies

The Case Studies

Each Case Study can be mapped by its position within the proposed monitoring framework

providing illustration of both depth and breadth of monitoring activity (see Figure A4.1).

It can be seen, for example, that the work of the European Training Foundation (ETF) is

focussed on developing a strong input framework for entrepreneurship education whereas

the example of Finland is focused on ‘outcomes for individuals’ and, in this illustration, those

for teachers. Young Enterprise Denmark is seeking to monitor from inputs to impacts but is

especially innovative in its assessment of the key (and difficult) relationship between

education and learning outcomes. Most recently, Wales has put in place a complete

monitoring framework based on youth entrepreneurship.

Figure A4.1 Monitoring Entrepreneurship Education Case Studies by Position in the Monitoring Framework

Source: ICF GHK

European Training Foundation: Monitoring the policy inputs of entrepreneurship education

A Policy Index for the Small Business Act

The European Training Foundation (ETF) “is an EU agency that helps transition and

developing countries to harness the potential of their human capital through the reform of

45

education, training and labour market systems in the context of the EU's external relations

policy”48

.

ETF works to improve vocational training systems in non-EU countries, mostly in

neighbouring regions, such as the countries preparing for EU accession, North Africa, the

Middle East, the Balkans and the former Soviet Union.

In unison with global and European realisation, these countries have expressed their desire

to increase opportunities for lifelong entrepreneurial learning, including as part of their

commitment to EU enterprise charters. Prominent amongst these charters is the Small

Business Act for Europe which includes two Principles (Principles I and VIII) which refer

directly to entrepreneurship:

■ “Create an environment in which entrepreneurs and family businesses can thrive and

entrepreneurship is rewarded”; and

■ “Promote the upgrading of skills and all forms of innovation”.

Building on ETF’s expertise in monitoring, they have created a Policy Index for the Small

Business Act and, in turn, nested within Index, is a monitoring framework for the

development of policy, structures and delivery frameworks for entrepreneurial education and

learning (see Figure A4.2).

This nested monitoring framework is known as the Human Capital Indicators for the Pre-

Accession Region in relation to the Small Business Act for Europe49

.

Figure A4.2 A Monitoring Framework for Entrepreneurship Education Policy Inputs sits inside the Policy Index for the Small Business Act

Source: Gribben, A. (2012) Presentation to the Expert Group on Data and Indicators on

Entrepreneurial Learning, Brussels, December 11.

48

http://etf.europa.eu/web.nsf/pages/home 49

ETF /SEECEL(2011) Small Business Act for Europe. Human Capital Indicators for the Pre-accession Region 2011. Available at: http://www.seecel.hr/UserDocsImages/Documents/Documents%20Section/SBA%202011%20Indicators.PDF [Accessed 12 June 2013].


46

The entrepreneurship education indicators

There are 23 indicators based on the relevant Policy Index Dimensions of the Small

Business Act – 18 based on “Entrepreneurship Education and Training” and 5 on “Skills &

Innovation” (see Table A4.1).

Table A4.1 ETF Small Business Act entrepreneurship education indicators

SBA Principle Thematic area Indicators

Principle I:

Create an

environment in

which

entrepreneurs and

family businesses

can thrive and

entrepreneurship

is rewarded

Policy framework for

entrepreneurial learning

■ Policy partnership;

■ Policy elaboration process;

■ Policy support resources;

■ Monitoring and Evaluation;

■ Good practice exchange;

■ Non-formal learning

Lower Secondary Education

(ISCED 2)

■ Organisation;

■ Entrepreneurship key competence;

■ Learning environment

Upper Secondary Education

(ISCED 3)

■ Organisation;

■ Entrepreneurship key competence;

■ Learning environment

Tertiary Education (ISCED

5&6)

■ National Higher Education Policy on

Entrepreneurial Learning;

■ Good practice in higher education

■ Higher education cooperation with the

world of business

Female Entrepreneurship

■ Policy support framework for promotion

of female entrepreneurship;

■ Training;

■ Financing;

■ National Network of Women

entrepreneurs

Principle VIII:

Promote the

upgrading of skills

and all forms of

innovation

Enterprise skills

■ Training Needs Analysis (TNA);

■ Access to training;

■ Quality Assurance;

■ Start-ups50

;

■ Enterprise growth

Each indicator is measured on a 5-level cumulative scale over a maximum of 3-4 years. An

example of an indicator definition is provided below.

An SBA Entrepreneurship Education Indicator: Policy partnership

Indicator name: Policy partnership

Rationale: Efficiency and effectiveness in lifelong entrepreneurial learning requires policy coherence between the ranges of stakeholders involved.

Objective: Government, private sector and civic-interest groups work in partnership to develop lifelong entrepreneurial learning.

Level 1: No structured cooperation between public, private and non-governmental sectors on entrepreneurial learning.

Level 2: An on-going national dialogue with view to structure cooperation between public,

50

“Start-up training comprises management, basic finance and basic marketing skills supported by public and/or private funds”.

47

private and non-governmental sectors on entrepreneurial learning.

Level 3: A national entrepreneurial learning partnership has been established between public, private and non-governmental sectors to promote entrepreneurial learning.

Level 4: State funds ensure sustainable contribution of entrepreneurial learning partnership to national developments (e.g. administrative support, work plan, capacity development).

Level 5: An entrepreneurial learning partnership advises on a range of national strategies (education, employment, SME, R&D) and action plans.

Monitoring indicators supporting policy development

The indicators are designed to support the strategic and policy making capacity of the

partner countries, including signposting the actions required to create a framework for the

delivery of entrepreneurship education.

Furthermore, the indicators are cumulative – allowing assessment of progress in the

development of the policy framework in any one country (see Figure A4.3 below) – and as

the common framework has been adopted across numerous countries progress can be

assessed for country groups (for example, West Balkans, etc.).

Figure A4.3 The Cumulative Approach of SBA Entrepreneurship Education Indicators

Source: Adapted from Gribben, A. (2012) Presentation to the Expert Group on Data and

Indicators on Entrepreneurial Learning, Brussels, December 11.

Learning and Insights on Monitoring Structures and Practices

The focus of this ETF Case Study is inputs to an entrepreneurial education and learning

system.

A carefully defined input framework – which has gained consensus across client countries –

has been utilised to support the introduction and development of a policy framework for

entrepreneurial learning.

Within countries the framework acts as a national policy development instrument: identifying

policy interest and ownership across stakeholders and through its developmental approach

providing impetus for continued reform.

48

As a multi-country benchmarking tool, this impetus is reinforced – but as much through

exchange and cooperation, structured networks and sharing of good practice and lessons

than through any sense of ranking.

Monitoring entrepreneurship education ecosystem inputs: supporting Finnish teachers

A Teachers Self Evaluation Tool for Entrepreneurship Education

In Finland, entrepreneurship education has been included in the national core curriculum for

basic and upper secondary level education since 1994. It is embedded as a cross-curricula

theme in all subjects at every level.

The “Measurement Tool for Entrepreneurship Education” is the result of a four-year

development project (2008-2012) funded by the European Social Fund and the Finnish

National Board of Education (see

http://developmentcentre.lut.fi/english.asp?show=yrittajyyskasvatus). The project was led by

Lappeenranta University of Technology, in cooperation with the Ministry of Education,

National Board of Education, local authorities, teachers and third sector organisations.

The tool focuses on the entrepreneurial educational input of teachers.

What is it?

The product contains two elements: the measurement tool and the steering mechanism.

■ The measurement tool is an easy-to-use, Web-based self-evaluation tool, which helps teachers develop their know-how in entrepreneurship and enterprise education.

■ The steering mechanism provides decision makers with the possibility to analyse, set priorities and implement national and / or regional policies on entrepreneurship education based on the macro-level results of the Measurement Tool.

Who is it for?

■ For teachers in secondary education who wish to develop as entrepreneurship and enterprise educators.

■ For work communities that want to raise entrepreneurship education expertise to a new level.

■ For decision-makers who are interested in the regional and / or national situation of entrepreneurship education.

Why?

A teacher’s work requires constant development; self-evaluation is a necessary prerequisite for this development. As teachers often work alone in the classroom, they must be able to examine their own work in order to identify targets for improvement. With the Measurement Tool for Entrepreneurship Education, teachers can evaluate their performance, receive systematic feedback and further develop their teaching practices related to entrepreneurship education.

The project involved inputs from multiple researchers synthesising a range of theories and

data sources. In addition to the development of the Measurement Tool, 38 articles were

published during the project. Use of the Finnish version of the tool is already widespread and

the tool is also available in Swedish, with an English version currently in development.

The development of an effective measurement for entrepreneurship education in Finland

required a ‘triple helix’ of cooperation between three different groups of specialists involved:

■ Researchers of entrepreneurship education and their insights and frameworks, so that

the measurement tools would be reliable and valid. Research cooperation included

building and forming the measurement tools and testing of the models;


49

■ National authorities taking part and providing the institutional support and leverage for

the introduction of the tools into the national framework. National and local authorities’

participation ensures that the tools developed would fit into the national or local context

and objectives and that they would be included in the national programmes (the

practitioners are thus expected to follow the guidelines) and,

■ The local practitioners and their specialist understanding of their work and fresh insights

into the tool. They were also central participants as testing partners of the new tools.

Teachers

1. Teach: The teacher utilises various work methods, techniques, learning environments and networks in order to teach entrepreneurship education to their students.

2. Measure: They set aside 15 minutes, log in at the website and answer a series of questions.

3. Develop: Feedback on teachers’ performance is sent immediately to his or her email address. The email message includes numeric results, comparisons and tips for development, as well as links to additional materials. The feedback and tips enable the teacher to develop their teaching methods accordingly.

The teachers’ responses are made anonymous and saved to a database. Feedback is

based on 5 main themes (and 12 subthemes):

■ entrepreneurial pedagogy: entrepreneurship education contents used in teaching;

■ collaboration in networks;

■ operating culture and learning environment; and,

■ planning and evaluation of entrepreneurship education.

Decision Makers

1. Analyses: Provides statistical analysis on the current status of entrepreneurship education in the country or region. Comparisons can then be made between regions and between educational levels.

2. Select goals for improvement: For example: to evaluate the match between national or regional policies and entrepreneurship education practices; to target specific entrepreneurship education goals; and to operationalise entrepreneurship education as measurable activities.

3. Implementation: For example: policy issues in the region, new issues in the questionnaire; resourcing new initiatives in the region; introducing best practice; and awarding good performance.

In essence, the Tool supports teachers in their professional development of

entrepreneurship education in an immediate and interactive fashion supports systematic

(research) understanding of the on-going practices and methods of entrepreneurship

education in a school, region, or country and provides a route to introducing new policies,

strategies and activities.


The Tool focuses on a key determinant of the quality of entrepreneurship education –

teachers – and delivers a real time and interactive monitoring system which simultaneously

supports the development of capacity and expertise.

This tool was generated through bringing key stakeholder groups together – for credibility

and subsequent adoption.

50

The Netherlands Actionprogramme Education and Enterprise (AEE): creating an entrepreneurial pipeline

Developing a pipeline of economic entrepreneurs

In previous years, the Netherlands government had seen the Ministry of Economic Affairs

and Ministry of Education joining together to promote entrepreneurship education. This

culminated in the 2008 launch of Actionprogramme Education & Enterprise (AEE).

Actionprogramme Education & Enterprise (AEE) aims to:

■ increase the number of educational institutions where entrepreneurship education is

integrated into policy, organisational structures and the curriculum; and

■ increase the number of pupils/students who are more enterprising, are positive about

entrepreneurship and who start their own business within five years of having finished

their education.

It is seeking to do this through using entrepreneurship education to stimulate entrepreneurial

attitudes and create entrepreneurial knowledge within an entrepreneurial pipeline (see

Figure A4.4).

Figure A4.4 The Entrepreneurial Pipeline in Netherlands

Source: Beemsterboer, E. (2013) Presentation to the Expert Group on Data and Indicators

on Entrepreneurial Learning, Brussels, February 13.

Given this pipeline approach, the programme includes the whole chain from primary

education to universities. It is based on the professional development of teachers and

management in schools delivering entrepreneurial education - and with the schools sitting at

the centre of regional networks including companies and entrepreneurs.

AEE has involved a substantial range of entrepreneurial education activities across

education systems and their stakeholders (see Figure A4.5 below), including:

■ Education Entrepreneurship Networks: networks of primary, secondary schools,

middle vocational education and businesses/ local government;

■ Education and Enterprise: a total of 28 enterprising projects were launched in 2008.

Six are Centres of Entrepreneurship in higher education. The other 22 projects are

spread over primary education up to middle vocational education;

■ Entrepreneurship Training Programmes: in collaboration with the Netherlands

Institute for Curriculum Development, a train-the-trainer programme has been developed

for teaching staff;

■ Global Scholarship Programme: in cooperation with Kauffman Foundation (USA) for

excellent students;

■ National Certificates for Enterprise Education for students in middle vocational

education;

■ Best practice development: knowledge sharing between projects and unrolling

practices;

51

■ Effects of entrepreneurship education: monitoring, evaluation and research; including,

■ Programme valorisation: the programme links activities promoting enterprise and

entrepreneurship in higher education with encouraging and facilitating start-up

businesses;

■ Other EE activities: network events, collaborations, PR and communication, etc.

Figure A4.5 Entrepreneurship education leads to entrepreneurial learning outcomes and impacts

Source: Beemsterboer, E. (2013) Presentation to the Expert Group on Data and Indicators

on Entrepreneurial Learning, Brussels, February 13.

Defining, assessing and tracking entrepreneurial learning outcomes in students

Notably, monitoring and evaluation is a key component of the programme as it has sought to

develop how students in Netherlands experience entrepreneurial education and move

through a ladder of entrepreneurial learning outcomes (awareness, knowledge, skills,

behaviour) and, ultimately, potential action in the economy (see Figure 4.5 above).

Learning outcomes have been defined at each level and a programme of repeat surveying of

pupils/student recipients of entrepreneurship education is taking place on a bi-annual basis.

The research methods include: internet surveys among 706 schools (which include

universities) and 1,241 students in middle and higher vocational education; 35 in-depth

interviews with school managers; and a selection and description of 16 best practices.

In 2012:

■ Over 75,000 students had been engaged directly in projects financed through the action

programme, 4,400 teachers and over 500 schools;

■ In vocational education and universities, over 80% now have specific activities aimed at

entrepreneurship;

■ A total of 32% of education-institutions had entrepreneurship in their mission and 22%

had anchored entrepreneurship education in their curriculum;

■ Over 60% of all students in vocational education and universities said they were aware

that their school/university is encouraging their entrepreneurial ambitions;

■ The percentage of young people who see entrepreneurship as a serious career option

had more than doubled (>60%) since 2007; and

■ A total of 20% of the students were certain they wanted to become an entrepreneur after

study (up from 13% in 2007).

Educational Entrepreneurship Networks

Education and Enterprise

Entrepreneurship Training Programmes

Primary

- Awareness

- Attitudes

- Kmowledge

Secondary

- Attitudes

- Knowledge

- Skills

Middle Vocational

- Knowledge

- Skills

- Behaviour

- Start a business

Higher

- Knowledge

- Skills

- Behaviour

- Start a business

Valorisation

Start

- Start a business

- Behaviour

- Growth

Global Scholarship Programme

52


AEE involves a substantial and sustained entrepreneurship education programme

throughout the Dutch education system. Targeted across all aspects of the entrepreneurial

education ecosystem, monitoring is directed at the activities and capacity of the ecosystem

and the entrepreneurial learning outcomes achieved by pupils/students, especially at middle

and higher levels.

The aim is for longitudinal study of learning outcomes including, ultimately, the potential for

economic impacts (start-ups) to materialise. It is noted, however, that a range of other

factors are highly likely to influence such an impact, including not least the broader

entrepreneurship ecosystem activities of AEE (through its Valorisation theme).

Young Enterprise Denmark: monitoring learning outcomes

Providing the evidence base for a national strategy

The Danish Foundation for Entrepreneurship – Young Enterprise (FFE-YE) is responsible for

implementation of the Danish national strategy for entrepreneurship education.

Development of the national strategy for entrepreneurship education involved the

cooperation of four government ministries (Business and Growth; Science, Innovation and

Higher Education; Children and Education; and Culture), supported by some of the largest

companies in Denmark, major banks and trade unions.

Since establishment in 2010, FFE-YE has been mapping the national spread of

entrepreneurship education through annual longitudinal impact surveys.

The ultimate aim of the monitoring programme is to measure the impact of entrepreneurship

education on outcomes with regard to skills, competences, and motivation for

entrepreneurship.

What is it?

FFE-YE measures the effects of entrepreneurship education through longitudinal surveys at three levels :

■ 9th grade - lower secondary/basic school

■ Company Programme - higher secondary

■ Universities - mainly graduate level

Entrepreneurship is measured in terms of behaviour.

The organisation also engages teachers in order to develop entrepreneurship teaching which will help embed the theme into study programmes, courses and teaching methods for students.

Why?

No systematic research has been conducted on what impact entrepreneurship education has on children and young people with regard to their future education, career and, ultimately, economic growth in Denmark. The purpose of the studies of FFE-YE is to provide evidence for the effect of entrepreneurship education in the Danish educational system.

The process of longitudinal monitoring of learning outcomes

Of note is that, essentially, such longitudinal monitoring entails direct evidence for the

translation of outputs (students receiving entrepreneurship education) to learning outcomes

(attitude, skills, knowledge) to potential impacts such as business start-ups.

53

Lower secondary / 9th grade

The aim of this research strand is to follow a group of pupils from lower secondary school

throughout their education as well as their future work life. Studies suggest that students are

most likely to start their own business five years or more after graduation. In addition, the

intention is to expand the database annually with a significant number of individuals with the

purpose of continually growing the number of participants. The initial database includes

2,000 individuals.

In an initial survey, all 2,000 individuals in the database were mailed a copy of the survey,

along with an explanation of the study, and asked to fill in and return in the enclosed

postage-paid return envelope or electronically via a link. To ensure a reasonable response

rate, participants were promised a cinema ticket as appreciation for their participation. In

total, 856 responses to the survey were received, 680 of which were useable.

The survey contained 54 questions covering the following topics:

■ Entrepreneurship Education - Has the pupil received entrepreneurship training?

■ Entrepreneurial behaviour and ambitions - How enterprising are young people?

■ Connectedness and aspirations for the future - How do young people feel about

themselves, school, their peers, and their future?

■ Attitudes and intentions - How do young people feel about entrepreneurship and what

are their intentions with regard to starting their own business?

Impact of entrepreneurship education: initial findings for lower secondary 9th grade study

Analysis of the survey returns highlighted that, pupils who have received entrepreneurship

education:

■ Are happier about school;

■ More confident of the future;

■ Hold higher ambitions regarding education and future career; and

■ Are more aware of their value to society.

Most young people (95%) were found to be positive towards entrepreneurship, a small

percentage (2.4%) were in the process of starting a company, and more than half (53%)

would like to become entrepreneurs.

University level

This research strand will follow students at the graduate level over a period of seven years.

Their performance in entrepreneurial indicators will be measured at several stages: before

they begin their graduate studies, after the first year, immediately after graduation, then one,

three and five years after graduation. The intention again is to gather data from a new cohort

of students every year, so a large database will be built up that will allow for advanced

statistical measurements.

Students from seven entrepreneurial courses at four different universities were chosen for

the initial study, along with eight suitable control groups consisting of students from three

universities. These students have very similar characteristics and backgrounds and study

similar courses as the students in the experiment group; the main difference is that they are

not exposed to entrepreneurship teaching. In total, there were 556 students in the first round

of data collection, divided equally in the experiment group and control group.

The survey contained 65 questions covering the following topics:

■ Creativity

■ Planning

■ Marshalling

■ Management of uncertainty

■ Financial knowledge

■ Attitudes and intentions

54

Impact of entrepreneurship education: initial findings for university level study

This study goes much deeper into the measurement of actual teaching than in the study of

the ninth graders, and focuses in particular on the impact on the respondents’

entrepreneurial self-efficacy (i.e. the students’ confidence that they are able to perform

entrepreneurial activities). This is because the project is keen to develop a credible research

design that can be used to measure the effect of different types of entrepreneurship

education.

The longitudinal nature of the study design means that it is only possible to present concrete

results after the second round of data collection. Some initial findings from the first round of

data collection are:

■ Entrepreneurial self-efficacy has been judged to be a suitable measure of

entrepreneurial behaviour;

■ The percentage of nascent entrepreneurs is higher in the entrepreneurial courses than in

the control groups – 31%, compared to 11%; and

■ If we do not consider the nascent entrepreneurs, then there is no significant difference in

entrepreneurial self-efficacy between the two groups.


The FFE-YE provides an example of a systematic attempt to monitor the later stages of an

entrepreneurship education monitoring framework. In other words, it is seeking to

understand the longevity of learning outcomes and if they lead to entrepreneurial actions and

impacts. In later educational years when this is most likely to happen (i.e. university

education), FFE-YE is using robust research methods to both assess the impacts of different

teaching methods and seek to create a ‘policy off’ (or counterfactual) baseline amongst a

control group of students who have not received entrepreneurship education.

The Welsh Youth Entrepreneurship Strategy: monitoring across all stages

The Youth Entrepreneurship Strategy (YES) was launched in 2004, aiming to boost the

entrepreneurial confidence of youth to enable them to positively contribute to the economic

and social success of the community. It is a joint initiative between the Welsh Government

Department of Business, Enterprise, Technology and Science and the Department of

Education and Skills.

The YES Strategy and the latest Action Plan (2010-2015)51

aim at “developing and nurturing

self-sufficient, entrepreneurial young people in all communities in Wales, who will contribute

positively to economic and social success”. YES has four key audiences: young people,

education, business and the community. The Strategy mainly focuses on young people.

What is it?

The Youth Entrepreneurship Strategy outlines an Action Plan, spanning from 2010 to 2015, which delivers initiatives and programmes across three delivery themes:

■ Engaging – to promote the value of entrepreneurship to create opportunities and develop young people;

■ Empowering – to provide young people with entrepreneurial learning opportunities; and

■ Equipping – to support young people to create and grow businesses.

Who does it target?

The strategy and action plan targets the youth. It also requires coordination and involvement from:

51


http://wales.gov.uk/topics/businessandeconomy/publications/yesactionplan1015/?lang=en%20

55

■ Education providers;

■ Entrepreneurs and the business community; and

■ The wider community.

Why?

There are approximately 210,700 businesses in Wales, with 94% of them employing less than 10 people. It has the lowest GVA

52 per head in the UK with youth/ graduate

unemployment at 23.6%. The combination of these spurred the creation of the Youth Entrepreneurship Strategy and Action Plan, to engage, empower and equip the youth to create self-employment and entrepreneurial opportunities.

The YES Action Plan: Policy Context

The programmes and initiatives operating under each delivery strand actively seek to make

connections across policy areas in Wales, providing a structure and focus for

entrepreneurship education (as can be seen in Figure A4.6).

Figure A4.6 The Policy Context

Source: YES Action Plan 2010-2015

The Relevant Initiatives and Programmes

A number of initiatives and programmes operate under the 2010-2015 Action Plan across

three delivery themes.

Delivery Theme

Initiative Description

Engage

Big Ideas Wales

A website hosting all the relevant information on the Action Plan

and its initiatives. The most notable features include:

■ The Enterprise Catalyst – a self-assessment tool which

evaluates the entrepreneurial skills of the user. This is

included in curricula. The results of this feed into the

evaluation framework, collating information on the profiles of

users.

■ Information on success stories, guides for exploring

business ideas, information on business start-up services

offered by the Welsh Government and its partners,

workshops and competitions to development business

ideas.

The Dynamo

Programme

An entrepreneurship role model programme, facilitating visits

between entrepreneurs and schools, colleges, universities and

52

Gross value added

56

Delivery Theme

Initiative Description

community groups / organisations. The entrepreneurs provide

inspiration and a realistic perspective on entrepreneurship.

These entrepreneurs come from the local community and all

volunteer their time.

Empower

National Enterprise

Primary School

Competition

This skills competition in primary education, called “Enterprise

Troopers”, was launched in May 2013. The competition

demonstrates the links between education providers (specifically

teachers) and the Government, allowing teachers to show what

they do in class and help them develop to the next level.

The ACRO model

This model is used for entrepreneurship skills regarding four

areas: Attitude; Creativity; Relationships; and, Organisation. The

Welsh Baccalaureate is based on essential skills corresponding

to this model. The development of these skills will be

compulsory in the Baccalaureate pre and post-16. The

Baccalaureate is currently delivered to 90% of schools, with a

goal to reach 100%.

Linking Education and

Economy:

Entrepreneurial Hubs

and National CPD

To adopt a more regional approach for strategic leadership, six

Youth Entrepreneurship Hubs and a national Continuous

Professional Development (CPD) Hub have been developed.

The hubs are led by universities and colleges, with six regional

start-up providers also working with the Hubs.

The national hub is led by the University of South Wales,

drawing expertise from other universities, such as the University

of Swansea. This hub coordinates teacher training taking place

by each hub, ensuring higher quality knowledge sharing. They

also collaborate with teacher colleges. The regional hubs are

obliged to collaborate with the national one.

Equip Promoting start-ups

In order to bridge the gap between a business idea and initiating

a start-up, this programme links young entrepreneurs with

existing businesses. It is due to be launched in September, and

aims to assist all young entrepreneurs who apply to the

programme through a website in order to participate in a boot

camp. After the completion of the boot camp, they can apply for

one of the pledges available. These young entrepreneurs attend

monthly meetings with businesses, where they present their

progress and get feedback. Businesses involved in the boot

camp identify three levels of start-ups: pre start-up, post start-up

and high growth start-ups allowing tailored support programmes.

Start-ups can be reassessed.

Inputs, Outputs and Impacts

The three themes of YES (that is, Engage-Empower-Equip) are used as the basis for the

development of the evaluation framework of the Strategy. There are two sets of measures

for YES:

■ The YES Action Plan Measures – which outlines inputs, outputs and impacts; and

■ The YES Strategic Impact Measures – quantifying the impacts, and tracking progress

against these.

Table 4.2 overleaf breaks down the inputs, outputs, impacts and the quantification of impacts

of the YES action plan.

As it can be seen the impacts are quantifiable strategic impact measures. The data to

measure these is drawn from various sources e.g. national surveys, questionnaires, on-line

tools, GEM surveys, the Carnegie Trust Enterprising Survey. This is all independently

collected to avoid any bias. Such data is collected at the end of every academic year, and

collection began three years ago.

57

This framework for monitoring and evaluating progress is important to inform the future

direction of YES.

Results

To date, significant progress has been made against these quantifiable impact targets.

Trends emerging from the data indicate positive progress.

■ Attitudes regarding entrepreneurship are changing:

– the trend among young people under 30 who wish to be their own boss is rising

consistently (from 41% in 2004 to 55% in 2012, based on the Wales Omnibus Survey

2012);

■ Early-stage entrepreneurship (GEM) is rising:

– the rate has trebled from 2002 to 2011;

– the rate for Wales for those age 18-24 is higher than the UK average (10.2% vs.

6.2%); and

■ Graduate start-up rates are encouraging: universities in Wales who make up 5% of UK’s

Higher Education population, but:

– generate 9.6% of all UK graduate business start-ups; and

– 10.26% of active firms lasting 3 years or more.

Additional benefits have also been achieved, including:

■ The creation of a national strategic framework with buy-in from key partners;

■ A simple model for entrepreneurship that the education sector can relate to and

implement;

■ A focus on skills that can be mapped directly into the curriculum; and

■ Engagement of entrepreneurs in delivery.


The evaluation framework of the YES Strategy includes both quantitative and qualitative

reporting. All stages of the logic model are covered, from inputs to impacts. Impact

measurements are fed within schools, colleges and universities.

The case of Wales underlines the importance of engagement of policy makers and of

cross-governmental collaboration in developing a more constructive and long-term

approach towards entrepreneurship education. The Strategy depends highly on the

participation of stakeholders in various activities. Stakeholders, especially from the local

business community are an integral part of the Strategy’s programmes and initiatives.

Moreover, the Strategy is very closely linked to the employability agenda, underlying the

importance of bridging entrepreneurship education with other policy areas.

Further steps could be taken to improve progress tracking, especially assessing the causal

impact; for example, how much has the YES Action Plan impacted the changes monitored,

especially regarding entrepreneurship actions that take place beyond the Strategy’s realm.

58

Table A4.2 YES Action Plan Measures

Delivery Theme

Inputs Outputs Impacts Impact Measurement Impact Target

Engaging

On line

signposting &

communications

■ Number of

sessions on Big

Ideas Wales.com

■ Number of users

of online tools to

test

entrepreneurial

skills

■ Number of

businesses

engaged with

Higher Education

Raise awareness

of

entrepreneurship

On line Enterprise

Catalyst assessment

Awareness of

Enterprise

5% increase

on baseline

Children’s Omnibus

Survey (7-18)

Awareness of self-

employment as a

career

75%

Awareness

raising activity

■ Number of young

people involved in

awareness raising

activities

Increase in

entrepreneurial

mindset

Remain above UK

average

Biennial

survey

Entrepreneurial

engagement

■ Number of young

people receiving

role model

presentations

■ Number of

partner

organisations

registered on

BigIdeasWales.co

m

More young people

wanting to be self-

employed

Children’s Omnibus

Survey (7-18)

Desire to run a

business

50%

Annual Wales

Omnibus Survey

(18+)

Desire to run a

business

Increase

from 50% to

55%

Dynamo Role Model

Project Post 16

Questionnaires

Intent to start up in

next 1-3 years

Increase by

5%

Empowering

Developing &

supporting

entrepreneurial

educators

■ Number of staff

engaged in

entrepreneurial

development

■ Number of

educators on

IEEP, EULP &

Masters

■ Number of

entrepreneurship

qualifications

developed Increase in the

entrepreneurial

capacity of young

people

Online Enterprise

Catalyst assessment

Enterprise “fuel”

Remain

above UK

average

Entrepreneurial

practical

experiences

■ Number of young

people involved in

entrepreneurial

activity as part of

the curriculum

■ Number of

schools

participating in

National Primary

Competition

■ Number of young

people involved in

Young people

developing

entrepreneurial skills

Welsh Baccalaureate

Monitoring

Under

development

59

entrepreneurial

practical

experiences in

colleges &

Universities

Regional,

national &

international

Good Practice

■ Number of

regional, national

& international

conferences

■ Number of links

with international

partners

NCEE Higher

Education Survey

Student engagement

in entrepreneurship

To remain

above UK

average

Equipping

Preparing for

start-up

■ Number of young

people attending

pre-start up

events

■ Number of young

people attending

taster sessions

More

entrepreneurially

active young

people

Global

Entrepreneurship

Monitor (GEM) –

Profile of

entrepreneurially

active young people

aged 18-24

To remain

above UK

average

Start up support

■ Number of young

people receiving

start up support

■ Number of

graduates

receiving start up

support

■ Number of

bursaries

awarded

■ Number of young

people being

mentored

Increase in the

number of

business start-ups

& survival rates by

graduates & young

people

Higher Education

Business and

community

interaction survey

(HEBCIS) – Profile of

graduate start-up

companies & those

surviving 3 years as

% of UK population

To remain

above UK

average

60

Annex 5 References

■ Bandura, A. (1997) Self-efficacy: The exercise of control. Worth Publishers.

■ Council (2007) Council conclusions of 25 May 2007 on a coherent framework of indicators and

benchmarks for monitoring progress towards the Lisbon objectives in education and training C 311

p.13-15 of 21.12.2007; http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2007:311:0013:0015:EN:PDF

■ Council (2009) Council conclusions of 12 May 2009 on a strategic framework for European

cooperation in education and training (‘ET 2020’) 2009/C 119/02; http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2009:119:0002:0010:EN:PDF

■ Croatia monitoring of entrepreneurship education, information available at

http://www.seecel.hr/print.aspx?id=1215; http://www.e4e.com.hr/?lng=en

■ European Commission (2001) Report from the commission: The concrete future objectives of

education systems; http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2001:0059:FIN:EN:PDF

■ European Commission (2001). Communication from the Commission, Making a European Area of

Lifelong Learning a Reality, available at http://www.bologna-berlin2003.de/pdf/MitteilungEng.pdf

■ European Commission, DG Enterprise (2010) Towards Greater Cooperation and Coherence in

Entrepreneurship Education: Report and Evaluation of the Pilot Action High Level Reflection Panels on

Entrepreneurship Education initiated by DG Enterprise and Industry and DG Education and Culture,

http://ec.europa.eu/enterprise/policies/sme/promoting-entrepreneurship/education-training-

entrepreneurship/reflection-panels/files/entr_education_panel_en.pdf

■ European Commission (2011) Commission Staff Working Paper accompanying the COM (2011) 902.

Analysis of the implementation of the Strategic Framework for European cooperation in education and

training (ET2020) at the European and national levels;


■ European Commission (2011) Communication to the European Parliament, the Council, the European

Economic and Social Committee and the Committee of the Regions- Draft 2012 Joint Report of the

Council and the Commission on the implementation of the Strategic Framework for European

cooperation in education and training (ET2020). Education and Training in a smart, sustainable and

inclusive Europe; http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0902:FIN:en:PDF

■ European Commission (2012) Rethinking Education: Investing in skills for better socio-economic

outcomes. COM (2012)669; http://eur-


■ European Commission (2013) Entrepreneurship 2020 Action Plan (EAP): Reigniting the

entrepreneurial spirit in Europe. COM (2012) 795 final; http://eur-


■ European Commission (2013) 2013 Country Specific Recommendations,


■ European Commission (2013) Annual Growth Surveys; available at


■ Europe 2020 Strategy http://ec.europa.eu/europe2020/index_en.htm

■ European Parliament and the Council (2006) Recommendation on key competencies for lifelong

learning, 2006/962/EC; http://eur-

lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF

■ European Training Foundation (2011) Small Business Act for Europe. Human Capital Indicators for the

Pre-accession Region 2011;





http://www.seecel.hr/print.aspx?id=1215

http://www.e4e.com.hr/?lng=en



http://www.bologna-berlin2003.de/pdf/MitteilungEng.pdf

http://ec.europa.eu/enterprise/policies/sme/promoting-entrepreneurship/education-training-entrepreneurship/reflection-panels/files/entr_education_panel_en.pdf

http://ec.europa.eu/enterprise/policies/sme/promoting-entrepreneurship/education-training-entrepreneurship/reflection-panels/files/entr_education_panel_en.pdf












61

http://www.seecel.hr/UserDocsImages/Documents/Documents%20Section/SBA%202011%20Indicator

s.PDF

■ European Training Foundation (2013), Inform – Entrepreneurial Learning Issue, Issue 16/June 2013;

http://www.etf.europa.eu/eventsmgmt.nsf/%28getAttachment%29/30148C231FA87366C1257B950047

80EF/$File/INFORM_16_Entrepreneurial%20learning.pdf

■ European Training Foundation (2013) ETF key indicators 2012: overview and analysis. Torino:

European Training Foundation;

http://www.etf.europa.eu/webatt.nsf/0/4091C214695FACD2C1257B640062A3E5/$file/Key%20indicato

rs%202012.pdf

■ European Training Foundation (2013), Inform – Entrepreneurial Learning Issue, Issue 16/June 2013,

http://www.etf.europa.eu/webatt.nsf/0/4373A704755A8C13C1257B940038C3B9/$file/INFORM_16_Ent

repreneurial%20learning.pdf

■ European Training Foundation; http://etf.europa.eu/web.nsf/pages/home

■ Eurostat (2012) Entrepreneurship determinants: culture and capabilities;


■ Eurydice (2012) Entrepreneurship Education at School in Europe National Strategies, Curricula and

Learning Outcomes;

http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/135EN.pdf

■ Evalsed (2012) The Resource for the Evaluation of Socio-Economic Development; available at

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/guide/guide2012_evalsed.pdf.

■ Foundation for Entrepreneurship –Young Enterprise (FFE-YE) in Denmark http://eng.ffe-ye.dk/about-

ffe-ye.aspx

■ GEM (2008) A Global Perspective on Entrepreneurship Education and Training, A Special Report

http://www.gemconsortium.org/docs/download/276

■ GESIS - Leibniz-Institute for the Social Sciences;

http://info1.gesis.org/dbksearch19/sdesc2.asp?no=4350&tab=3&ll=10&notabs=&af=&nf=1&search=&s

earch2=&db=E

■ GHK (2011) Final Report: Order 121 - Study on Support to Indicators on Entrepreneurship Education;

Framework Contract No. EAC 19/06;


■ International Social Survey Programme (ISSP); http://www.issp.org/index.php

■ ISSP Research Group, International Social Survey Programme (ISSP): Work Conditions III 2005:

Questionnaire;

http://info1.gesis.org/dbksearch19/sdesc2.asp?no=4350&tab=3&ll=10&notabs=&af=&nf=1&search=&s

earch2=&db=E

■ Kelly, D.J.; Singer, S.; Herrington, M. (2011) Global Entrepreneurship Monitor: 2011 Global Report;


■ Measurement Tool for Entrepreneurship Education (Finland), information available at


■ OECD: The OECD-Eurostat Entrepreneurship Indicators Programme (EIP): background information;

http://www.oecd.org/industry/business-

stats/theentrepreneurshipindicatorsprogrammeeipbackgroundinformation.htm

■ OECD (2009) Measuring Entrepreneurship A Collection of Indicators;

http://www.oecd.org/industry/business-stats/44069965.pdf

■ OECD (2010) Ministerial report on the OECD Innovation Strategy. Innovation to strengthen growth and

address global and social challenges: Key Findings;

http://www.oecd.org/dataoecd/51/28/45326349.pdf





http://www.etf.europa.eu/webatt.nsf/0/4091C214695FACD2C1257B640062A3E5/$file/Key%20indicators%202012.pdf

http://www.etf.europa.eu/webatt.nsf/0/4091C214695FACD2C1257B640062A3E5/$file/Key%20indicators%202012.pdf

http://www.etf.europa.eu/webatt.nsf/0/4373A704755A8C13C1257B940038C3B9/$file/INFORM_16_Entrepreneurial%20learning.pdf

http://www.etf.europa.eu/webatt.nsf/0/4373A704755A8C13C1257B940038C3B9/$file/INFORM_16_Entrepreneurial%20learning.pdf

http://etf.europa.eu/web.nsf/pages/home


http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/135EN.pdf

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/guide/guide2012_evalsed.pdf

http://eng.ffe-ye.dk/about-ffe-ye.aspx

http://eng.ffe-ye.dk/about-ffe-ye.aspx

http://info1.gesis.org/dbksearch19/sdesc2.asp?no=4350&tab=3&ll=10&notabs=&af=&nf=1&search=&search2=&db=E



http://www.issp.org/index.php





http://www.oecd.org/industry/business-stats/theentrepreneurshipindicatorsprogrammeeipbackgroundinformation.htm

http://www.oecd.org/industry/business-stats/theentrepreneurshipindicatorsprogrammeeipbackgroundinformation.htm

http://www.oecd.org/industry/business-stats/44069965.pdf

http://www.oecd.org/dataoecd/51/28/45326349.pdf

62

■ OECD (2012) Entrepreneurship at a glance 2012; http://www.oecd-ilibrary.org/industry-and-

services/entrepreneurship-at-a-glance-2012_entrepreneur_aag-2012-en

■ OECD (2012) Entrepreneurship at a glance 2012: Annex A. List of Indicators of Entrepreneurial

Determinants; http://www.oecd-ilibrary.org/sites/entrepreneur_aag-2012-

en/03/01/index.html?contentType=/ns/StatisticalPublication,/ns/Chapter&itemId=/content/chapter/entre

preneur_aag-2012-27-

en&containerItemId=/content/serial/22266941&accessItemIds=&mimeType=text/html

■ OECD Programme for International Student Assessment (PISA), 2012


■ OECD Teaching and Learning International Survey (TALIS), 2013


■ Welsh Youth Entrepreneurship Strategy (YES) Action Plan information available at


■ World Economic Forum: Global Confidence Index; http://www.weforum.org/content/pages/global-

confidence-index

■ World Economic Forum (2009) Executive Summary: A Report of the Global Education Initiative;

http://www.weforum.org/pdf/GEI/2009/EE_ExecutiveSummary.pdf

■ World Economic Forum (2011) Final Report on the Entrepreneurship Education Workstream, Summer

2011; http://www.weforum.org/reports/final-report-entrepreneurship-education-workstream-summer-

2011

http://www.oecd-ilibrary.org/industry-and-services/entrepreneurship-at-a-glance-2012_entrepreneur_aag-2012-en

http://www.oecd-ilibrary.org/industry-and-services/entrepreneurship-at-a-glance-2012_entrepreneur_aag-2012-en

http://www.oecd-ilibrary.org/sites/entrepreneur_aag-2012-en/03/01/index.html?contentType=/ns/StatisticalPublication,/ns/Chapter&itemId=/content/chapter/entrepreneur_aag-2012-27-en&containerItemId=/content/serial/22266941&accessItemIds=&mimeType=text/html%20






http://wales.gov.uk/topics/businessandeconomy/publications/yesactionplan1015/?lang=en%20%20%20%20

http://www.weforum.org/content/pages/global-confidence-index

http://www.weforum.org/content/pages/global-confidence-index

http://www.weforum.org/pdf/GEI/2009/EE_ExecutiveSummary.pdf

http://www.weforum.org/reports/final-report-entrepreneurship-education-workstream-summer-2011

http://www.weforum.org/reports/final-report-entrepreneurship-education-workstream-summer-2011