cedefop – employer survey on skill needs in europecedefop – employer survey on skill needs in...

CEDEFOP –Employer Survey on Skill Needs in Europe

Project overview

Arnold Riedmann, TNS Infratest SozialforschungMiriam Gensicke, TNS Infratest SozialforschungBernd Dworschak, Fraunhofer IAOFrancis Green, Institute of Education, University of London

Cedefop Expert workshopPrague, 29/30 April 2010

Chart 2Cedefop expert workshop, Prague, 29/30 April 2010

Agenda

Aims and challenges of the project

The research group, its tasks and work organisation

Work packages of Phase 1

Methodological outline of the project


1. Aims and challenges


The challenges

The aim of the project

is to develop and pilot a survey instrument to reliably identify future needs of occupations, skills, competences and qualifications in enterprises in Europe, covering the whole economy.

As a result,

a detailed, robust and validated methodology should be made available for future regular use at EU-27 level that ensures data according to agreed international quality standards.

The tasks

range from agreeing the topics to be investigated, via developing adequate questionnaires and sampling frames to data collection and reporting.


Major aspects to be considered

Which type of questions related to current and future skill needs can be answered by employers?

…of different size-classes (firms with a specialised Human Resources department vs. firms without such a department) ?

…of different sectors (highly innovative sectors vs. “normal” sectors) ?

Which type and depth of information is needed in order to get an “added value” from an European employer survey?

Additional information must be complementary to other types of information (incl. existing Cedefop forecast tools), filling a real “gap”

Additional information must be useful for the policy making process

(How) Is it possible to bring the practical possibilities/limitations and the information needs in line with each other?


Different approaches

Critical view on approaches discussed in the feasibility study:

Vacancies (“normal” and “hard to fill” vacancies)

Occupational structure and its evolution over time (stock and flow)

Vocational training

These approaches have their merits, but have limitations especially regarding their usefulness for forecasting

Alternative proposal: “task-oriented” approach


Future of the project

Prospects for the future of the survey:

New survey coordinated by Eurostat?

New survey managed by Cedefop (similar to Eurofound or EU-OSHA surveys)?

(“Only”) toolkit for national skill need surveys?

Survey with compulsory or voluntary participation?

The future of the survey is still open

The current project thus has to cover different options/scenarios (e.g.: national/international, small/large samples, ad hoc/regular, compulsory/voluntary…)


2. The research group, its tasksand work organisation


The research group: institutes and persons

“Core research group”:

TNS Infratest Sozialforschung (Arnold Riedmann, Miriam Gensicke)

Fraunhofer IAO, Stuttgart (Bernd Dworschak, Hartmut Buck, Alexander Karapidis, Helmut Zaiser)

Prof. Francis Green, LLAKES, University of London

“Extended expert group”:

Jos Sanders/Karolus Kraan (TNO Work and Employment, NL)

Lina Vaitkute/Beleckiene Giedre (Qualifications and VET Development Centre, LT)

Olympia Kaminioti (Employment Observatory Research Informatics S.A., EL)

Vera Czesana/Hana Zackova (National Training Fund NVF, CZ)

Magdolna Benke (National Institute for Vocational and Adult Education NIVE, HU)

John McGrath (Irish Employment and Training Authority FAS, IE)

Cristina Mocanu (National Research Institute for Labour and Social Protection INCSMPS, RO)

Ferran Mané (URV Foundation, ES)


Tasks and time schedule – All PhasesYearMonth 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12

PHASE 1: Development of the survey instrument

WP 1 Specification of required survey outcomesDef. of information needs and required survey outcomes

Analysis of different approaches for operationalisation

WP 2 Statistical methodology and survey guidelinesChoice of approach(es);development of survey concept

WP3 Development of the master questionnaireDevelopment of draft master questionnaire versions

PHASE 2: Improvement of the questionnaire and pilot survey preparation

WP 4:Development of national quest.; pre-test

Preparation of national language versions, pre-testing and analysis of pre-test results

Final questionnaire revisions (master & national)

WP 5: Preparation of the pilot survey

Writing of pilot survey manual

PHASE 3: Implementation of the pilot survey

WP 6 Fieldwork

Preparation of fieldwork and fieldwork phase

WP 7: Data processing and quality assessment

Weighting, data checks & data validation

WP 8 Data analysis, summary pilot survey report and manual for final survey instrument

Data analysis & report

Writing of a handbook for employer surveys on skill needs

20112010


Task distribution I: TNS Infratest Sozialforschung

Background/ relevant experiences:Survey institute with a European-wide network of fieldwork institutes (TNS group)

Broad experiences in the conceptualisation and implementation of surveys among employers: IAB Establishment Panel, Eurofound establishment surveys (ESWT 2004/05 and ECS 2009), Establishment survey of the European Agency for Safety and Health at the Workplace (EU-OSHA) etc.

Experiences in the field of skill research: ADeBar studies (in cooperation with Fraunhofer IAO), PIAAC Germany, BiBB/BAUA Employee survey (question bloques on qualifications and training) etc.

Tasks in the project:Formal lead

Main responsibility for statistical concept

Involvement in questionnaire development

Main responsibility for phases 2 and 3 (translations, pre-testing, fieldwork, data-cleaning, reporting)


Task distribution II: Fraunhofer IAO

Background/ relevant experiences:Founding member and coordinator of the German FreQueNZ network on early identification of skill needs (started in 1999)

Member of CEDEFOP’s Skillsnet

Experience in the field of early identification of skill needs for many years: Various sectoralstudies (e.g. ADeBar, in cooperation with TNS Infratest, on renewable energies, pharma, facility management and many others)

Tasks in the project:Lead in Phase 1 (conceptual work, questionnaire development)

Participation in phases 2 and 3 (pre-testing, fieldwork, reporting)


Task distribution III: Prof. Francis Green

Background/ relevant experiences:UK skills survey

Major role in the development of the JRA module to be used in the PIAAC surveys

Participation in the MEADOWS project (development and testing of employer survey questions related to organisational and technological changes)

Tasks in the project:Participation in all work packages of Phase 1

Leading role in the questionnaire development

Participation in phases 2 and 3 (pre-testing, fieldwork, reporting)


Task distribution IV: Extended expert group

Background/ relevant experiences:Direct involvement in the conceptualisation of various skill need surveys, both among employers and employees (e.g. in Spain, Romania, Ireland)

Experiences with European wide skill surveys as users ( usefulness for the political process at the national level)

Participation in Cedefop’s Skillsnet

Tasks in the project:Support in the definition of information needs

Support in the choice of approaches and design of the survey concept

Advice on the practicability of the preferred approach in the own country

Support in the design and revision of questionnaires


3. Work packages of Phase 1


Work Package 1: Specification of required survey outcomes

Critical review of feasibility study

Identification of key political questions with respect to skill need forecasting

Assessment of (national or cross-national) employer surveys dedicated to future skill needs (approaches, outcome, data-collection methodology)

Definition of information gaps to be filled with the survey and of expectations on part of Cedefop and its stakeholders

Which information gaps do exist in the field of skill need forecasting?

Which of these could be filled with an employer survey?

Which approach(es) would be most suitable to this end?

Results will be documented in the interim report (June 2010)


Work Package 2: Statistical methodology and survey guidelines

Development of a concept for a detailed and complete employer sample survey:

Choice of an approach and description of the approach

Choice of data collection methodology and description of methodological implementation:

Data-collection method

Sampling strategy

Translation procedure

Etc….

Selection of countries for the pilot interviews

Results will be documented in the final report (November 2010)


Work Package 3: Development of the master questionnaire

Procedure:

Joint development of 1st draft questionnaire within the core group

Discussion of the draft within the research group and with Cedefop

Revised 2nd and 3rd draft (modified on base of further comments)

Important criteria for all draft versions:

Easily understandable questions

Questions that can reasonably be answered in an interview situation (level of knowledge on part of respondent)

Cross-national comparability! (differences in definitions/concepts to be taken into account)

End product: English master questionnaire in a version (almost) ready for being pre-tested


4. Methodological outline of theproject


Data collection methodology – general options I4 basic options:

Face-to-face (PAPI or CAPI)

+ High response rates, possibility to apply visual means (e.g. lists of occupations), complex filtering possible

- Most expensive method (in most countries); in PAPI version no ad hoc data plausibility control

Telephone (CATI)

+ Reasonably high response rates and low response bias (interviewer-based), complex filtering possible

- No visual means applicable, limited interview duration, no possibility to look up statistical information in company files

Mail

+ Application of visual means, respondent can take time to reflect on questions, to discuss issues with colleagues or to look up information in the company files

- In some countries extremely low response rates (if voluntary survey); enhanced danger of a response bias; no ad hoc data quality control (call-backs)

Online (CAWI)

+ similar advantages as mail, complex filtering and ad hoc data plausibility control possible

- Similarly low or even lower response rates than mail; danger of response bias (incomplete questionnaires), no full internet coverage of businesses


Data collection methodology – general options II

There is no generally best method; method has to suit the chosen approach towards the mapping of skill needs

+ CATI/CAPI: higher response rates & low self-selection bias due to interviewer intervention; complex filtering possible

+ Mail/(Online): More time to reflect & time to discuss issues with colleagues; particularly useful for the questions on FUTURE skill needs; but: response rates for a voluntary survey unacceptably low in some countries, in mail no complex filtering possible

Method mix as a possible further option

Unit of enquiry: Company or establishment


Data collection methodology I – Proposed pilot design

Preliminary recommendations for the pilot survey:

CATI survey or maybe 2-3 countries by CATI and 2-3 countries by mail

1.000 interviews per country

20-25 minute interview

Unit of enquiry: Preference for establishment level for reasons related to contents (but: feasibility & later usage!)

Universe: Establishments/companies from 1 dependent employee onwards, all relevant sectors

But: Changes of the pilot survey design are still possible

Main criterion: Which method renders more/higher quality information? The method has to suit best the defined information needs and the approach finally chosen


Thank you for your attention!

cedefop – employer survey on skill needs in europecedefop – employer survey on skill needs in...

Documents