1

Assuring the quality of VET systems through the definition of the expected outcomes: A cross

country analysis in seven countries

Giorgio Allulli, Kim Faurschou, Tina Bertzeletou

2

Reasons for a study on outcome

standards

Standards are technical specifications, measurable,

drawn up by consensus and approved by a

recognised organisation. In VET, they serve to

optimise the inputs and/or outputs of learning.

Output standards set the level of performance to be

attained by a VET system, a provider or an individual.

3

Reasons for a study on outcome

standards Working with the standards allows for:• clearly stated objectives;• inclusive processes involving (all or most of) the

stakeholders;• motivation of actors to achieve concrete results;• evaluations (since they are stated in unambiguous terms);• comparisons;• general acceptance (since they are generally

acknowledged and shared);• confidence that quality requirements have been fulfilled;• responding to the general tendency observed to move

from an input logic to an output approach to VET systems and funding.

4

Reasons for a study on outcome

standards Quality assurance includes also the accreditation of a VET provider/training programme or service, by the relevant legislative and professional authorities, of having met predetermined standards.

Outcome standards will be used to help define (and verify) the learning outcomes of the national qualification frameworks to be developed on a voluntary basis by the Member States within the context of the European Qualifications Framework (EQF) once the Recommendation adopted (Spring 2007).

5

• “In the past, the debate around the public services centred just on how much governments were investing – both in terms of money, and in terms of other resource ‘inputs’, including the number of doctors, nurses, teachers and police officers. Since the introduction of PSAs, the debate has shifted. Now we can measure how effectively resources are being used and whether services are delivering the outcomes that will really make a difference to people’s lives.”

(Foreword to the Public Service Agreements white paper 2005/ 2008)

Reasons for a study on outcome

standards

7

Field of application Seven Member States have been investigated by the two experts contracted by Cedefop (Giorgio Allulli and Kim Faurschou) with the active participation of:• Anne Marie Charroud, Sylvère Chirache, Pierre Le Douaron (France)

• Philip Pedersen, Ken Thomassen (Denmark)

• Stephen Haxton, Steve Hunter, Grace Proudfoot, Anne Greaves, Janet Ryland (United Kingdom)

• Marie Gould, Barbara Kelly, Margaret Kelly (Ireland)

• Andreas Krewerth, Heike Schwarzbauer, Bent Paulsen, Beate Scheffler (Germany)

• Siep Jurna, Hugo Hopstaken, Sjoerd Roodenburg, Roy Tjoa, Thea van den Boom (The Netherlands), and

• Italy has been covered by Giorgio Allulli himself.N.B.: In italics the names of the ENQA-VET members.

8

Targets at system level: main examples

• Budget Law (LOLF, FR) • Public Service Agreements (UK)• Basic performance levels (IT)• Specific targets are connected to:

– Pupils and adults achievements (IE, NL, UK)– Dropout rate (NL,UK,FR)– Participation rate (NL,IE, DE,UK,FR)– Diploma or qualification rate (IE,DK,FR,UK)– Satisfaction rate (NL)– Employability rate (DK)– Special groups (IE)

9

Targets for VET providers

• Floor targets connected to accreditation (IT) • Floor targets (England)• Criteria and parameters connected to self evaluation

and external evaluation (IE)• Voluntary targets (Success rate and grades, DE, NL)• Setting of indicators to monitor outcomes (DK, FR)

10

Outcome standards for learners

• National basic standards and regional specific standards (IT)

• Referentiels de certification (FR)• National Vocational Qualifications, SVQ (UK)• National Qualification framework (IR)• The Ministry set the objectives for general subjects,

the Trade committees for the vocational subjects (DK)

• Basic structure at federal level. Specific reordering prepared by Länder experts (DE)

11

Actors of the process

• Parliament and ministries (national targets, output standards)

• National Technical authorities (output standards)• Regional political authorities (regional targets,

regional standards)• Social partners (sectoral standards)

Sometimes organized in permanent committees

12

Rewards and penalties

• Providers who do not reach floor targets can loose accreditation (IT)

• Providers who do not reach floor targets can loose their grant, or have their managers removed (UK)

• The Minister can impose two types of sanctions on institutions (NL):– deprivation of privileges – withholding or suspension of funds

• The Ministry offers the providers additional funding if they attain a number of goals (DK)

13

Reasons for private companies to offer training

– Trainees meet company requirements 94 %– Skilled employees not available on the job market 90 %– Prevention of personnel fluctuation 80%– Opportunity to «pick and choose » the best trainees 74 %– Avoidance of wrong hiring decisions 73

%– Cut costs for «breaking in » new employees 58 %– Enhances the company' s reputation 57

%– Trainees develop into productive employees 42 %– Cuts personnel- recruiting costs 35 %

(«Why training pays», 2000, Germany)

Still difficult in many countries to find enough enterprises.

14

Process of decentralization and autonomy

• The State set the “basic performance levels” to maintain coherence to the system (IT)

• Contract agreements between the Ministry of Education and the local academies to set the objectives to reach (FR)

• Regional Skills Partnerships have been given a greater role in the regional application of national targets (UK)

• VET providers have a great autonomy for adapting VET to local needs (DK)

15

Trends

• Further use of output standards instead of input- and process standards (IT)

• New form of financing based on results instead of activities (FR)• Sharper, simpler accountabilities and incentives for delivering

improvements in public services (UK)• Common awarding structure (IE)• The new Vocational Training Act encourages the stakeholders

to develop new forms of training and examination (DE) • More focus on linking economic grants to output indicators (DK)• Not only focus on output and transparency (NL)

16

Challenges

• Process of decentralization and autonomy• Complexity of the social development and the

influence of the context • Long-term impact of targets • Challenge between quantity and quality • Statistical burden • Sharpness and flexibility • Stakeholders’ involvement • Standards updating• Rewards and penalties

17

Possible European initiatives

• Two main objectives:– to monitor the developments of the EC policies – to confront and to discuss the experiences realized

• Specific activities:– organize a thematic group on the definition of targets at

system and providers level – organize Peer learnings to discuss the experience of targets

setting • Topics

– which targets and standards exist– how are they measured– what links between inputs, processes and outputs