systems approach for better education results...

SABER – SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS

6WORKFORCE DEVELOPMENT

What Matters in Workforce

Development: An Analytical

Framework for the Pilot Phase

Jee‐Peng Tan

Kiong Hock Lee

Alexandria Valerio

Robert McGough

© 2012 The International Bank for Reconstruction and Development / The World Bank

1818 H Street NW

Washington DC 20433

Telephone: 202‐473‐1000

Internet: www.worldbank.org

1 2 3 4 15 14 13 12

This work is a product of the staff of The World Bank with external contributions. The findings,

interpretations, and conclusions expressed in this work do not necessarily reflect the views of

The World Bank, its Board of Executive Directors, or the governments they represent.

The World Bank does not guarantee the accuracy of the data included in this work. The

boundaries, colors, denominations, and other information shown on any map in this work do

not imply any judgment on the part of The World Bank concerning the legal status of any

territory or the endorsement or acceptance of such boundaries.

Rights and Permissions

The material in this work is subject to copyright. Because The World Bank encourages

dissemination of its knowledge, this work may be reproduced, in whole or in part, for

noncommercial purposes as long as full attribution to this work is given.

Any queries on rights and licenses, including subsidiary rights, should be addressed to the

Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax:

202‐522‐2422; e‐mail: [email protected].

Cover design: Patricia Hord.Graphik Design, Alexandria, VA

What Matters in Workforce Development: An Analytical Framework for the Pilot Phase iii

Contents

About the Series ............................................................................................................... v

About the Authors ........................................................................................................ vii

Acknowledgements ........................................................................................................ ix

Executive Summary ........................................................................................................ xi

Introduction ...................................................................................................................... 1

Purpose, Audience, and Relation to Other Skills‐Related Work ............................... 2

A Note on Terminology .................................................................................................. 3

Conceptual Framework ................................................................................................... 6

Linking Polices to System Performance ...................................................................... 12

Data for Benchmarking WfD Policies and Institutions............................................. 15

Piloting the SABER‐WfD Methodology and Expected Outputs ............................. 38

Conclusion ...................................................................................................................... 40

Appendix 1: Functional Dimensions, Policy Goals, and Actions in SABER‐

WfD............................................................................................................................ 41

Appendix 2: Benchmarking Rubrics ........................................................................... 42

References ....................................................................................................................... 51

What Matters in Workforce Development: An Analytical Framework for the Pilot Phase v

About the Series

Building strong education and training systems that promote learning and foster

skills acquisition is fundamental to development and economic growth. Over the

past few years, as developing countries have succeeded in building more

classrooms, getting millions more children into school, and expanding

educational opportunities at higher levels, the education community has begun

to embrace the vision of ensuring that graduates not only exit the system with

employable skills but are also equipped to learn new skills as job opportunities

and labor market conditions evolve over course of their working lives. The

responsiveness and relevance of the education and training system depend not

only on the amount of resources invested, but also on the influence of policies

and institutions on how resources are allocated and how well they are used to

achieve results on the ground.

In May 2011, the World Bank Group launched Education Sector Strategy

2020: Learning for All, which outlines an agenda for achieving “Learning for All”

in the developing world over the next decade. To support implementation of the

strategy, the World Bank commenced a multi‐year program to support countries

in systematically examining and strengthening the performance of their

education systems. This evidence‐based initiative, called SABER (Systems

Approach for Better Education Results), is building a toolkit of diagnostics for

examining education and training systems and their component policy domains

against global standards, best practices, and in comparison with the policies and

practices of countries around the world. By leveraging this global knowledge,

SABER fills a gap in the availability of data and evidence on what matters to

achieve better education and training results.

SABER‐Workforce Development, one of the domains examined within the

SABER program, focuses on documenting and assessing workforce development

policies around the world. Its goal is to shed light on how WfD systems might

bring into closer alignment skills supply and demand, by enabling individuals to

acquire job‐relevant skills while also giving employers access to workers with the

required skills to fill staffing gaps and improve business productivity. To help

explore the state of knowledge in the area, the SABER‐Workforce Development

team invited leading researchers, analysts, and practitioners from developing

and industrialized countries to come together to discuss workforce development

issues. The papers and case studies on workforce development in the pilot phase

are the result of those conversations and the underlying research. Prior to

publication, all of the papers benefited from a rigorous review process, which

included comments from World Bank staff, academics, development

practitioners, and national workforce development experts.

vi Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough

All SABER‐Workforce Development papers in this series that are developed

in the pilot phase were made possible by support from the government of the

United Kingdom under its Partnership for Education Development (PFED) with

the World Bank, and from the government of the Republic of Korea under the

Korean Trust Fund.

What Matters in Workforce Development: An Analytical Framework for the Pilot Phase vii

About the Authors

Jee Peng Tan is currently Education Advisor in the Human Development

Network of the World Bank where she leads the Network’s program on

workforce development. Between 2005 and 2009 she was Education Advisor in

the Africa Region’s Human Development Department where she led the

Department’s analytical and policy work on education in Africa. A staff member

of the World Bank since 1981, Ms. Tan has worked as an economist in several

Regions of the institution, including South Asia, East Asia, and the Middle East

and North Africa. She has written widely on education‐related policy matters

and taught courses on policy analysis. She pioneered the use of Quality

Enhancement Reviews to improve the quality of World Bank operations in

education; and created the EdStats Website as a knowledge‐sharing platform.

She was also a core member of the team for the 1993 World Development Report on

Investing in Health. Ms. Tan has a BSc in economics from the London School of

Economics, and a PhD in economics from Princeton University.

Kiong Hock Lee is currently a consultant in the Human Development Network

of the World Bank where he works as a member of the SABER‐Workforce

Development team. Mr. Lee worked briefly in the banking sector, then as an

economics lecturer and later professor in the Faculty of Economics and

Administration, University of Malaya, Malaysia. He has written on education

and labor‐related issues and has taught courses in the economics of education,

labor, and personnel economics. He is a graduate of the School of Economics,

University of New South Wales, has a PhD in economics from the London School

of Economics, and an MA in religion from the Reformed Theological Seminary.

Robert McGough is a specialist in technical and vocational education and

training (TVET) with more than 30 years’ experience in the field. He has taught

at the secondary and post‐secondary levels and has held leadership positions in

government and corporate training initiatives in the United States, including the

Dunwoody Industrial Institute in Minnesota, Texas Instruments Corporation,

Westinghouse Corporation, and Minnesota Diversified Industries. He was on the

staff of the World Bank between 1983 and 2003 where he was involved in

training‐related operations around the world. Mr. McGough has published two

books on the organization and administration of TVET, more than 60 refereed

articles and numerous World Bank publications. He has an EdM in industrial

education from the University of Missouri and a PhD in vocational‐technical

education from the University of Missouri.

viii Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough

Alexandria Valerio is currently a Senior Economist in the Education Department

of the World Bank’s Human Development Network where her work focuses on

workforce development policies, skills measurement, and entrepreneurship

education and training. Prior to her current position, Ms. Valerio was responsible

for leading the World Bank’s policy dialogue and lending programs in education

and training in Argentina, Brazil, Chile, Nicaragua, Paraguay, Panama, Angola,

and Mozambique. She has extensive experience in the areas of education policy,

planning, and finance; workforce development; technical and vocational

education and training; and early childhood development. Her publications

include country and cross‐country studies on skills development, assessments of

the social impact of school fee removal schemes, cost and finance of early

childhood development programs and expansion of secondary education, and

school‐based HIV/AIDS prevention programs. Ms. Valerio holds an MA in

public administration from Syracuse University and a PhD in comparative

education and economics of education from Columbia University.

What Matters in Workforce Development: An Analytical Framework for the Pilot Phase ix

Acknowledgements

This paper has benefited from the support of the SABER‐Workforce

Development team, whose past and current members include Richard Arden,

Rita D. Costa, Angela Demas, Ryan Flynn, Manorama Gotur, Lan Joo, Lorelei

Lacdao, Joy Yoo‐Jeung Nam, Marie Madeliene Ndaw, Trinh Nguyen, Sungmin

Park, Brent Parton, John Sloan, Bernardo Vasconcellos, and Cecilia Zanetta. The

authors wish to thank numerous colleagues at the World Bank and members of

two advisory panels for helpful suggestions on the ideas discussed in the paper.

They are grateful for financial support from the government of the United

Kingdom through the Partnership for Education Development (PFED) between

the World Bank and the UK Department for International Development (DfID);

and from the government of the Republic of Korea through the Korean Trust Fund.

None of the above individuals or parties is responsible for the views expressed in

the paper; the authors alone are. Readers are welcome to send comments on the

paper to the authors: Jee‐Peng Tan ([email protected]), Kiong Hock Lee

([email protected]), Alexandria Valerio ([email protected]), and Robert

McGough ([email protected]).

What Matters in Workforce Development: An Analytical Framework for the Pilot Phase xi

Executive Summary

Equipping the workforce with job‐relevant skills is a continuing challenge

around the world. In developing and emerging economies, the pool of educated

labor has been rising over time as a result of past investments in education and

training. At the same time, there is growing evidence of persistent mismatches in

skills demand and supply, with many school leavers and graduates unable to

find jobs commensurate with their education and training, while employers

complain of skills shortages and mismatches. Employers also bemoan the

scarcity of “soft skills” (such as problem‐solving, communication, and

effectiveness in team work), skills that are considered essential for business

productivity in today’s global economy. More broadly, skills constraints are

making it difficult for companies to innovate and move into more lucrative areas

of economic activity. The inability to integrate into and ascend the value chain in

global production networks keeps countries in a low‐skills trap and impedes

their efforts to accelerate economic growth and poverty reduction. The

persistence of these problems suggests that workforce development in most low‐

and middle‐income countries is not functioning as well as might be desired.

As a follow‐up to its Education Sector Strategy 2020, which was launched in

May 2011, the World Bank started the Systems Approach for Better Education

Results (SABER) initiative, whose aim is to create diagnostic tools for assessing

the policy and institutional factors that influence the performance of education

and training systems. Workforce development (WfD) is one of several domains

included in the SABER initiative. The focus is on how well the education and

training system is equipping individuals with skills to meet the demand for skills

in the labor market. The tool is expected to help identify promising actions to

achieve better results in light of global good practices in enhancing the impact of

workforce development on economic progress. It places explicit focus on three

functional dimensions of WfD policies and institutions: (a) strategic framework;

(b) system oversight; and (c) service delivery.

In the current phase of the project, the SABER‐WfD tool was applied to a

group of five pilot countries—Chile, the Republic of Korea, Ireland, Singapore,

and Uganda. Because the main purpose is to test the methodology, these

countries were chosen somewhat opportunistically to span a reasonably broad

range of experience. The inclusion of Ireland, Korea, and Singapore allowed the

tool to be tested for its ability to capture key policy and institutional changes in

settings where skill development was an explicit part of the countries’ strategies

for rapid economic growth. The inclusion of Chile and Uganda took advantage

of the availability of significant knowledge available to the research team in the

context recent World Bank operations and analytical work in these countries.

xii Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough

Insights from the pilot phase are being used in two ways: to sharpen the

SABER‐WfD tool, and to create the beginnings of a new knowledge asset on

promising policies to advance workforce development in the World Bank’s

partner countries.

What Matters in Workforce Development: An Analytical Framework for the Pilot Phase 1

What Matters for Workforce

Development: An Analytical

Framework for the Pilot Phase

Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough

Introduction

Many developing and emerging economies have invested in education and

training and have made good progress in expanding coverage at all levels. They

are also paying more attention to learning outcomes, to the broader array of

skills beyond academic achievement, and to technical and vocational education

and training.1 Yet, a growing pool of better‐educated labor has not always

produced the expected results. Indeed, many countries find themselves in a

paradox where large numbers of graduates are unable to find jobs commensurate

with their education and training while employers complain of skills shortages

and mismatches. Viewed from a longer‐term perspective, an added

disappointment is that these countries, especially the poorest ones, continue to

find it difficult to diversify their economies and acquire the business and

technological capabilities required to produce and sell more sophisticated goods

and services. Skills constraints are part of the problem, suggesting that the

system for workforce development in most countries is not functioning as well as

might be desired.2

This paper proposes a new diagnostic tool to enhance the World Bank’s

assistance to its partner countries with regard to the challenges of workforce

development.3 The broader concept is the World Bank’s Skills toward

Employability and Productivity (STEP)4 framework, which sets forth a holistic

model encompassing five components for human development to support

economic and social progress: (a) starting right in early childhood; (b) laying a

strong foundation in basic and secondary education; (c) building and upgrading

job‐relevant skills; (d) fostering innovation and entrepreneurship; and (e)

matching skills demand and supply. Workforce development focuses on—but is

1 One reflection of the increasing focus on learning outcomes is the prevalence of international projects to

measure learning outcomes. Examples include Trends in International Mathematics and Science Study

(TIMSS), Program for International Student Assessment (PISA), and Programme for the International

Assessment of Adult Competencies (PIAAC). 2 A recent review of the problems associated with workforce development through technical and

vocational education and training can be found in Asian Development Bank (2009). 3 The World Bank’s partner countries refer to low‐ and middle‐income countries as well as fragile states. 4 For more details on the STEP framework, see World Bank 2010a.

2 Jee‐Peng Tan, Kiong Hock Lee, Alexandria Valerio, and Robert McGough

not strictly limited to—the third of these five components. The project is being

implemented under a new World Bank initiative known as Systems Approach

for Better Education Results (SABER),5 which was launched following the May

2011 launch of the institution’s Education Strategy 2020.6 SABER‐Workforce

Development (SABER‐WfD) shares the overall initiative’s goal of creating a tool

for systematic documentation and assessment of a country’s policies and

institutions in education and training in light of global good practice.

The rest of the paper is organized into six sections. The next section below

elaborates on the purpose and intended audience of the project; and situates it in

relation to other skills‐related work at the World Bank. The section that follows

clarifies two terms used throughout this paper: workforce development (WfD)

and benchmarking. The next four sections present the paper’s substantive core:

the conceptual framework, key propositions on the link between policies and

outcomes in workforce development, the analytical strategy for data collection,

and the design of the data collection instrument. The penultimate section

explains how the tool is being tested in the pilot cohort of countries—Chile,

Ireland, Korea, Singapore, and Uganda— and the outcomes to date. The last

section concludes the paper.

Purpose, Audience, and Relation to Other Skills‐

Related Work

The World Bank engages as a member of the development community in

assisting partner countries to advance social and economic progress. Its

involvement takes many forms, which for simplicity may be categorized under

three broad spheres of activity: knowledge generation, policy dialogue, and

operational support. SABER‐WfD targets the piece pertaining to policy dialogue.

Its purpose is to create a systematic approach for documenting and assessing

countries’ policies and institutions relating to WfD. The tool is thus intended to

shed light on an important part of the “black box” that separates tangible inputs

from outcomes. Its application in multiple countries can culminate in a

systematic comparative database to foster cross‐country learning and support

policy dialogue on promising approaches for WfD. In the current experimental

phase of the SABER‐WfD initiative, the main purpose of this paper is to explain

the tool’s design and implementation in the five pilot countries. Lessons from

this phase are expected to inform efforts to refine the tool, clarify the protocols

for data collection and analysis, explore formats for reporting the results, and

examine how best to use the results in policy dialogue.

5 See World Bank (2012) and the resources at http://go.worldbank.org/NK2EK7MKV0 (accessed on

November 9, 2011). 6 See World Bank (2011) and http://go.worldbank.org/DTQZ9EKJW0 (accessed on November 9, 2011).


The SABER‐WfD tool complements other skills‐related work at the World

Bank. Two areas of work are worth highlighting. One pertains to the

measurement of skills gaps and mismatches. Recent studies have pointed to the

inadequacy of common indicators such as educational attainment and related

details on fields of study, type of training, and so forth (see, for example,

Hanushek and Wößmann 2006; Sondergaard and Murthi 2012)7 These indicators

also neglect the “soft” skills valued by employers (see, for example, di Gropello

et al. 2011a and 2011b) and rewarded by market forces (see, for example,

Heckman et al. 2006). To narrow the measurement gap, the World Bank is

currently implementing a multi‐country project known as the Skills for

Employment and Productivity (STEP) Skills Measurement Study.8

The second broad area of related work pertains to the impact of policies and

investments in skills development. Promoting the welfare of individuals through

skills development is the focus of many initiatives by governments, international

donors, and private entities. But definitive knowledge about what does and does

not work in different settings is limited (see, for example, Betcherman et al. 2004

and 2007). To fill this knowledge gap, the World Bank has been expanding its

portfolio of impact evaluation with the support of partners in order to narrow

the evidence gap.9 On a more limited and selective basis, the World Bank also

fosters South‐South exchange, including through study visits that enable policy

makers in partner countries to gain first‐hand exposure to and engage in

dialogue and exchange with their peers in other countries.

A Note on Terminology

Workforce development and benchmarking are two terms that are used

frequently in this paper. Because they mean different things to different people,

it is useful to clarify at the outset how we use them in the content of the SABER‐

WfD project.

Workforce development has attracted growing interest in recent years from a

wide range of researchers and policy makers, many of whom are concerned

about the implications for skills development associated with five major factors:

globalization, technology, the role of knowledge and innovation in the “new

economy,” political change, and demographic shifts (Jacobs and Hawley 2008).

Although the term raises issues at the level of individuals, organizations, and

societies, we use it in this paper to refer to the challenges faced by a national,

regional, provincial, or sector‐based system in responding to two expectations:

7 While useful, these indicators are insufficient for policy development, given their generally weak link to

individuals’ labor market outcomes and to countries’ rate of economic growth. 8 Results from the study are expected in 2013. 9 For more information on ongoing evaluations of skills‐related initiatives, see www.worldbank.org/sief

(accessed on November 9, 2011).


Enabling individuals to acquire the knowledge, practical skills, and

attitudes for gainful employment (including self‐employment) or

improved work performance in a particular trade or occupation; and

Providing employers with an effective means to communicate and meet

their demand for skills.10

With regard to the first expectation, it is clear that job‐relevant skills are

acquired through diverse pathways: some are established in early childhood

education (for example, language skills) and reinforced through subsequent

general schooling at the primary and secondary levels (for example, literacy and

numeracy); others rely on pre‐employment vocational training at the secondary

and post‐secondary levels; and still others through apprenticeships, on‐the‐job

training via formal and nonformal arrangements, and informal learning. An

evaluation of workforce development thus requires a comprehensive assessment

of the entire education and training system and its effectiveness with regard to

the skills of both the future pipeline of workers and that of incumbent workers.

Such a comprehensive assessment is beyond the scope of SABER‐WfD. Instead

we narrow our focus to those parts of the system with an explicit mission of

enabling individuals to acquire job‐relevant skills, either through initial

preparation or continuing education and training. The tool thus puts little focus

on early childhood education, primary education, and general secondary

schooling, even though these are all clearly essential in laying a strong

foundation for skills acquisition. To avoid overlap with the SABER tools

addressing higher education, SABER‐WfD focuses on the segment of the

workforce filling mid‐level positions (for example, skilled craftsmen, technicians,

production supervisors, and so forth). Formal training in the skills associated

with such positions typically begins at the secondary or post‐secondary non‐

university levels and can progress up to the tertiary level in some fields.11

The second expectation of workforce development puts the spotlight on

employers as users of skills. As such, their involvement in WfD is essential to

ensure a responsive system for skills supply. In a well‐functioning WfD system,

employers are able to meet their demand for skills and compete successfully in

product and services markets while individuals are able to acquire the

knowledge and skills to prosper in the labor market. Such a system can help

boost economic productivity and growth; creating and maintaining its

10 This dual expectation is shared by the U.S. National Governors Association, which notes that to

develop the skilled and knowledgeable workforce required for economic competitiveness in the global

economy, “state workforce and education systems are designed to provide the skilled workers employers

need to thrive and the education and training individuals need to prosper in todayʹs labor market.” See

http://www.nga.org/cms/home/nga‐center‐for‐best‐practices/center‐issues/page‐ehsw‐issues/page‐workforce‐

development.html (accessed on August 12, 2012). 11 University‐level training typically supplies workers with advanced technical and research skills that

are essential for advancing the development of knowledge‐intensive economic sectors. It is the focus of a

separate domain in the SABER project and is therefore treated more fully there.


performance is thus a key concern of policy makers pursuing skills‐intensive

strategies for economic development.

Benchmarking is another term used in this paper. The concept gained

prominence in 1979 in light of what Xerox Corporation did to stem the loss of its

photocopier business to Japanese competitors (Camp 1989). The company

undertook a systematic assessment of its business practices, in the process

identifying ways to improve manufacturing design, boost production efficiency,

and lower costs, all of which enabled it to regain its competitive edge. The

concept and practice of benchmarking have since become widespread as a well‐

regarded and versatile tool for evaluating organizational and institutional

performance (Dattakumar and Jagadeesh 2003).

Often used as a continuous process for enhancing outcomes, benchmarking

now finds wide application in the public and private sectors throughout the

developed world.12 An organization that embarks on benchmarking assesses

various aspects of its processes in relation to those of leading organizations,

usually from a peer group defined for the purposes at hand. The exercise poses

such questions as the following for self‐examination: (a) how do we compare to

others in the same business; (b) how good do we want to be; (c) who is doing it

the best; (d) how do they do it; (e) how can we learn from and adapt what they

do to suit our circumstances and needs; and (f) how can we excel and become a

standard of performance for others to aspire to?13 The process allows an

organization to develop plans for adapting specific best practices to improve its

own performance.

For our purpose, national or subnational systems of WfD rather than

companies are the relevant units of analysis.14 Conceptually, the foregoing

questions relating to the benchmarking of companies remain relevant when

considering systems of WfD. Many countries are interested in knowing how well

their policies and institutions compare to those in nations with more effective

systems for workforce development. To systematize the comparison, we develop

performance metrics for key aspects of WfD policies and institutions and define

the corresponding stages of development based, whenever possible, on the

experiences of nations with successful WfD systems. Where data are scarce, the

benchmarks are based on a review of the relevant literature and on the logic of

practical progression as a system improves. Finally, it is important to emphasize

that the purpose of benchmarking is not to rank countries but to create a

12 See, for example, http://www.well.com/user/benchmar/germany/Germany.html (accessed on

November 9, 2011). 13 Adapted from Alstete (1995). 14 See O’Lawrence (2007) for an application of the benchmarking approach to a system for skills

development in the state of California in the United States. Although the approach may be used to

evaluate specific programs for skills development (for example, adult worker retraining and re‐

employment schemes, formal industry‐based apprenticeship schemes, or disadvantaged youth

employment/training schemes), the SABER‐WfD project will focus on national systems or subnational

systems.


common frame of reference for policy discussion and to facilitate self‐diagnosis

as a basis for improvement.

Conceptual Framework

Key elements of the conceptual framework for the SABER‐WfD project appear in

figure 1.15 The framework recognizes that achieving coherence in WfD policies

requires a simultaneous consideration of the demand for skills as well as the

supply. A well‐functioning WfD system with a high degree of coherence

between skills demand and supply achieves faster economic growth by: (a)

improving trainees’ employability; (b) supplying the workers needed by firms to

improve productivity, product quality, and competitiveness; and (c) enabling

firms and entrepreneurs to expand their technological capability and move up

the value chain where profit margins are often more attractive. Where this match

is poor, slower growth and the associated problems of joblessness and

underemployment, brain drain and technological stagnation or laggardness are

an ever‐present prospect.

Figure 1: Conceptual Framework for Workforce Development

Source: Authors’ construction.

15 The proposed benchmarking approach complements the World Bank’s MILES framework, which is

used by some to evaluate employment issues through five lenses: Macroeconomic conditions, Investment

climate and infrastructure; Labor market regulation and institutions; Education and skills development;

and Social protection, where the capitalized letters are used to form the acronym MILES (see Banerji et al.

2008).


In developed countries, the economic crisis of 2008 and its aftermath have

increased the urgency of addressing WfD issues (see, for example, OECD 2010a).

In less‐developed countries, skills to increase the employability of youth and to

help accelerate overall growth also feature regularly in both national and

international dialogue (see, for example, World Bank 2010b, 2010c; World Bank

2012). Below we elaborate on the main components of the framework relating to

skills demand and supply and to the linkages between these two parts of a WfD

system.

The demand for skills. The government’s economic policies set the overall

context for skills demand in multiple ways. Wage policies and related legislation,

for example, influence the flexibility of the labor market, the hiring practices of

firms, and the aggregate demand for labor and its distribution between formal

and informal sectors.16 Tax policies affect firm and worker behavior, including

their incentives to train. The spending policies of governments often have a

significant impact on skills demand. Large infrastructure investments, for

example, create a demand for various construction, engineering, and specialized

skills. In some countries, governments also make strategic investments to nurture

or grow selected sectors and these may call for new skills beyond what is

available in the current labor pool (see, for example, UKCES 2009). In countries

where the government takes a proactive approach to economic management, its

policies may also affect firms’ business strategies and the incentives for

innovation and adoption of new technologies. These forces in turn shape the

demand for skills that may be required for technology absorption and

advancement.

Forward‐looking policies must obviously take into account global trends

and practices and evaluate their implications for the local economy.

Demographic changes, globalization and technological advances, urbanization,

and climate change are some of the key influences on the dynamics of skills

demand. The world’s population is aging as a result of declining fertility and

rising life expectancy.17 In developed economies, the trend is already translating

into a growing demand for health and personal care services and for related

skilled workers, often imported from abroad, required to provide these

16 In Mexico, for example, Levy (2008) notes that the government requires firms to provide health

insurance, pensions, and other social benefits directly to salaried workers, whereas non‐salaried workers

get these services through government pension and insurance programs. The difference in policy

effectively puts a tax on salaried workers but offers a subsidy to their peers in the informal arrangements.

Levy estimates that salaried benefits in Mexico are 30–35 percent less than their actual value to workers;

and that the gap is sufficiently large to keep workers in low productivity work in the informal sector,

thus slowing overall economic growth. 17 In developed economies, 33 percent of the population in 2050 is projected to be at least 60 years old,

compared with just 22 percent in 2010. In developing economies, the corresponding percentages are 20

percent and 9 percent. Africa has the youngest population, but even there, the share of the population at

least 60 years of age is projected to rise from 5 percent in 2010 to 11 percent by 2050 (UN 2009).


services.18 In less‐developed economies, the share of younger workers—many of

them new entrants to the labor force and often in the midst of starting their

families—is increasing rapidly. They represent both a pool of potential

employees and a market for consumer goods and services and the associated

demand for skilled workers to produce and sell these products. In most

countries, however, the potential has yet to translate into tangible results in the

form of jobs and sustainable income growth.

Globalization and technology, particularly in information and

communication, are an even more obvious influence on the demand for skills.

Firms everywhere face increasing pressures to innovate and diversify into newer

and more lucrative markets, as profit margins in traditional industries are

squeezed by intense global competition and the shortening of product shelf life.

Among the more dynamic firms that compete in export markets, skills

bottlenecks are a chronic complaint, even in countries with highly educated

workforces.19 Rapid urbanization, a trend that some countries actively promote

in order to foster dynamic clusters of economic activity, is also ratcheting up the

demand for talent and skilled workers, including service workers to build and

maintain lively and livable urban spaces. In some settings, the lure of overseas

jobs has fueled the demand for skills, encouraging potential migrants to obtain

training to improve their prospects abroad. Climate change is yet another global

trend with important implications for skills demand. As countries scale up

mitigation or coping responses, they will require workers skilled in a wide range

of “green” technologies.

These global trends combine with conditions in a particular sector or a

locality to shape the demand for specific skills. In the United States, for example,

a 2009 survey of manufacturing businesses found emerging skills shortages in

three sectors (aerospace and defense, life sciences and medical devices, and

energy and resources) and highlighted bottlenecks for skilled production

workers and for scientists and engineers (DMIO 2009). In Europe, WfD polices

are being informed by detailed analyses by the European Center for the

Development of Vocational Training for the 27 countries of the European

Union.20 In India, the boom in information technology (IT) and IT‐enabled

services is increasing the demand for technicians—estimated at 500,000 a year—

as well as for English language skills (Sudan et al. 2010). One careful study

18 At the same time, older workers in developed economies are finding it necessary to postpone their

retirement and seek new skills in an effort to remain employable and productive. As a result the concept

of lifelong learning has gained growing relevance in public policy debates. 19 See, for example, data from the World Bank’s Enterprise Surveys posted at

http://enterprisesurveys.org. 20 According to CEDEFOP (2010), projections for 2020 suggest a net loss of some 2.5 million jobs in the

primary sector (mainly agriculture) and 2.0 million in traditional manufacturing, balanced against gains

of some 7.0 million jobs in business and other services, 3.4 million in distribution, transport, hotels and

catering, and 1.0 million in education, health, and other nonmarketed services. Demand is expected to

rise for highly‐ or medium‐skilled workers, while it will shrink for workers in clerical services, crafts, and

related trades and agriculture.


shows that each new call center raises enrollments in English‐language schools in

the center’s vicinity by nearly six percent (Oster and Millett 2010).

The demand for quality skills. A large number of surveys confirm that

employers value not just a worker’s ability to perform specific technical tasks,

but also his or her possession of “soft” skills or behaviors (Ferrier et al. 2003a;

UKCES 2009). In the United States, manufacturers have consistently ranked

highly skilled and flexible workers as one of two top drivers of performance (the

other is new product innovation).21 In the Philippines and Indonesia, employer

surveys conducted in 2008 revealed a high demand for three “soft” skills for

managers and production staff alike: problem‐solving, leadership, and

communication skills. In Vietnam, the top behavior prioritized by employers in a

2009 survey was punctuality. In the same survey, employers also valued

workers’ practical and technical knowledge more highly than theoretical

knowledge. A recent survey of Indian employers highlighted deficiencies in the

“soft” skills of the graduates from engineering colleges (Blom and Saeki 2011).

The importance placed on “soft” skills is not surprising given the nature of work

in today’s economy.

An increasingly integrated global economy with intense competition, a

growing body of scientific and technological knowledge, and widening access to

knowledge via digital media are reinforcing the demand for skills that go beyond

simple book learning and/or the competence to execute only specific tasks. The

ability to solve problems, to learn new skills, and to adapt to new conditions, all

of which require a solid foundation in literacy and numeracy, are especially

valuable in contexts of rapidly evolving business conditions (Wang 2012).22 These

profound changes are transforming today’s jobs, increasing the premium on

workers that can handle nonroutine tasks as a normal part of their jobs (Levy and

Murnane 2005). In settings where most school leavers must find their livelihood

in the informal sector, having the flexibility, entrepreneurship, and initiative to

find work must also be added to the list of core skills that young people will

require for a successful transition from school to work (for example, Adams

2007).

The supply of skills. The technical and vocational education and training

(TVET) system is often perceived as the main source of workforce skills

development.23 This is understandable as public TVET institutions often host

21 See DMIO 2009. 22 Because of the importance of these cognitive skills, the World Bank recently embarked on the STEP

Skills Measurement project, which involves administering tests to adults ages 15–64 as part of the effort

to document the levels and distribution of such skills in the workforce and to relate the possession of

these skills to individuals’ earnings and employability (among other outcomes). 23 It is useful to note that the acronym TVET represent the first letter of four words with specific

meanings: “technical” and “vocational” typically refer, respectively, to technician‐level and craftsmen‐

level programs; “education” refers to exposure to the world of work and preparation for further studies

in technical and vocational fields; and “training” refers to preparation for entry into, or upgrading in,

specific occupations or clusters of occupations.


major government‐financed programs for formal skills training. However, in

many countries, other sources of training exist, including private TVET

institutions and employer‐based training programs. In middle‐ to high‐income

nations, and in particular in large industrialized nations such as the United

States, Germany, and Japan, the main suppliers of middle‐ to higher‐level skills

training are the employers themselves. Because firm‐based training activities and

programs are fragmented across the sector, often unpublicized outside the firms,

and typically paid for by the firms themselves, their national contribution to WfD

is not always well‐understood or recognized.

A wide range of programs in fact offers opportunities for skills training in

most countries. They include formal, nonformal, and informal programs, each of

which may require different levels of prior educational attainment and may

operate under different institutional homes, typically though not exclusively, the

ministries of education and of labor. While both formal and nonformal TVET

refer to structured programs, they often fall under the mandate of different

parent ministries. Formal programs usually take place in the education system and

end with formal certification or accreditation of qualifications; nonformal programs

often occur outside the education system (hosted by such ministries as labor,

social welfare, industry, and so forth), in or outside the workplace, and may or

may not lead to an accredited qualification. Where granted, the qualification may

not relate to the certification system hosted by the education ministry. Informal

TVET comprises unorganized and unstructured learning that is seldom the

responsibility of any single agency; it often occurs in the workplace, for example,

when a junior worker learns the ropes simply by working alongside or watching

more experienced colleagues.24 Despite their informality, these arrangements can

be a significant source of learning for tacit skills; their prevalence often depends

on firms’ workplace practices and organizational culture.

To add to the complexity, the system for skills provision may also include

such programs as: (a) formal, industry‐based apprenticeships, usually for

training in advanced skills; (b) training partnerships involving organized labor

and firms, often with formal curricula taught by training providers; (c) technical

training for and within the military; and (d) special needs training (that is, skills

training for select populations, such as prisoners, persons with disabilities, and

the disadvantaged).

In all TVET systems, government policies influence to varying degrees the

way the system is organized, governed, and financed and how skills attainment

is certified and recognized. They therefore affect the efficiency of service

provision, the effectiveness of the system in meeting training objectives, the

quality and relevance of skills that it produces, and the accessibility of training

opportunities, particularly among disadvantaged population groups. They also

24 Nonformal TVET should not be confused with TVET programs that are intended to equip trainees for

jobs in the informal sector of the economy.


shape the incentives for both training providers and learners and influence the

aggregate supply of skills and its diversity by level and specialization.

The impact of policies on the role of employers in WfD planning and

implementation—at both the strategic and operational levels—warrants close

attention. In well‐performing systems, policies are in place to improve the

responsiveness of training providers by encouraging them to view employers,

both public and private, as critical partners whose input and feedback help

ensure the continued relevance of training services as labor market conditions

evolve. Such systems are demand‐led. It is important to clarify that being

demand‐led does not always mean that the system is driven by employers alone.

For countries trying to move into higher value‐added production, politicians’

policies regarding the economy’s strategic direction also influence the demand

for skills. A demand‐led system would be sensitive to such demand as well. By

contrast, in systems where the relation between training providers and

employers is weak, the risk of mismatches in both the quantity and quality skills

is ever present. In such supply‐driven systems, employers that face critical skills

gaps may resort to addressing the problem themselves, as in India in such sectors

as information technology and communication and pharmaceuticals (as

documented, for example, in Tan and Nam 2012).

Alignment of skills supply and demand. A key challenge in all WfD systems

is to ensure a good match between skills demand and supply. Where the match

is good, significant benefits can accrue in the form of a more productive

workforce, higher rates of employment and labor utilization, progress in the fight

against poverty, and, for the economy as a whole, tangible movement up the

value chain of economic activity (see, for example, Ferrier et al. 2003b). The

alternative scenario is that of a weak system delivering a poor match between

skills demand and supply. In such systems, the risks include high

unemployment and underemployment, often coexisting with chronic skills gaps

felt by employers, emigration of skilled workers, and an economy showing few

signs of diversifying and upgrading of its technological capabilities.

In reality, skills misalignment is a common problem in most countries.

Numerous surveys of employers consistently reveal concerns about skills

constraints on business performance and about the lack of certain types of skills

among employees (for example, de Gropello et al. 2011a and 2011b). These

problems stem to a large extent from what economists describe as market and

government failures. Below we elaborate on an analytical strategy to benchmark

the policies that governments might use to overcome such flaws and enhance

outcomes in workforce development.


Linking Polices to System Performance

We shall focus on actions by government to promote a closer connection between

skills demand and supply. This emphasis does not mean that the government is

the most important actor among those with a role in WfD. Rather, it is intended

to facilitate dialogue on how the government might be more effective, whether

acting directly or through the intermediation of other actors, in advancing its

goals for workforce development. Below we begin by elaborating on three broad

areas where weaknesses in policies and related institutions and processes might

arise; we then state three hypotheses to guide our strategy for data collection and

analysis.

Impediments to Better Outcomes

Economic theory about market and government failures identifies several

conceptual obstacles to a well‐functioning WfD system. In their comprehensive

review, Almeida et al. (2012), for example, discuss failures stemming from: (a)

imperfections in capital markets which prevent individuals and firms from

borrowing for training investments; (b) limited information, uncertainty, and

myopia that distort the actions of individuals and firms with regard to training

investments; (c) coordination failures arising from weak linkages among key

stakeholders (for example, employers, and training providers); and (d) labor

market imperfections (for example, firms demanding suboptimal levels of

training for their workers for fear of losing their trained workers to competitors).

Theoretical considerations alone are insufficient, however, for designing

policies to improve the system’s performance. Also needed is a deeper

understanding of the specific policies that matter and the circumstances under

which they matter. The evidence on this score is, unfortunately, sparse at present,

particularly as it pertains to the experience of developing and emerging

economies. We can nonetheless begin to contribute to the dialogue by creating

and testing a new methodology that documents existing policies and institutions

systematically and benchmarks them against practices in systems that have made

significant progress over time. This approach is characteristic of the overall

SABER initiative. Its application in multiple countries creates a comparative

database that can be analyzed to narrow gaps in our knowledge about effective

policies and institutions.

Our data collection effort focuses on three areas where we believe market

and government flaws are particularly relevant as impediments to better

outcomes in workforce development: governance, finance, and information. We

briefly elaborate on them below to set the stage for discussing the design of our

data collection instrument.

Governance. Workforce development is a complex policy arena. It involves

multiple stakeholders at different levels of decision‐making whose goals, roles,

and responsibilities may overlap or be in conflict (for example, ADB 2009 and


2010). At least four sets of stakeholders can be discerned: (a) public officials

representing different ministries and levels of authority; (b) employers in their

role (individually and through their trade associations) as a source of jobs,

market intelligence, expertise, and advice to inform the training curricula, and

possibly also of gifts and advocacy to support training programs; (c) training

providers (again individually and corporately) who offer pre‐employment, on‐

the‐job and targeted training and career development services, networking

among trainees, and a pool of potential recruits for employers; and (d)

individuals who have dual roles, as trainees and as incumbent or future workers

(who may be represented through unions), whether working for others or in self‐

employment. In this complex environment, appropriate governance

arrangements can help clarify roles, responsibilities, and accountabilities among

the stakeholders and enhance the quality of key relationships among them.

Finance. All WfD systems depend on adequate and well‐deployed

resources, financial and non‐financial, to function effectively. Funding

arrangements—how money is mobilized, allocated, and channeled—shaped

incentives and therefore the decisions made by individuals, training providers,

as well as employers and firms in their capacity as users of skills. Key design

choices include: (a) the aggregate level and pattern of distribution in budget

allocations; (b) the criteria for deciding on funding allocations, and on recipients’

access to and continuation of funding; and (c) diversity in the sources mobilized

to support investments in WfD. Funding arrangements can exert an influence

through channels on both the demand side and supply side and thus have the

potential to help narrow gaps in skills supply and demand. In‐kind resources,

such as employers’ contribution of machinery or staff time for training purposes,

are often difficult to document but their presence is an important indicator of

engagement with a key group of stakeholders.

Information. Reliable and timely information provide signals that condition

the behaviors of key actors on both the demand and supply sides in the market

for skills. Useful information include the following: the skills gaps faced by

employers; the menu and quality of training options and their costs; the

employment and earnings of the graduates of training programs; and the

trustworthiness of the formal certification of skills attained. The availability of

such information can help influence the decisions of both individuals and

training providers and align them to the demand for skills in the labor market.

However, if an important economic goal of the country is to grow and upgrade

into new and emerging areas of economic activity that require new technological,

managerial, and operational capabilities, then passive information flows alone

may not suffice to minimize skills gaps and mismatches. In such settings

proactive coordination to bring the behaviors of the relevant parties into closer

alignment with the areas of emerging skills demand may be appropriate.


Hypotheses on Improving WfD Outcomes

Improving the performance of the WfD system is akin to moving up the rungs of

a ladder. For example, four rungs on the ladder corresponding to poor, modest,

good, or great outcomes may be characterized as follows:

Poor outcomes:

– Few employers value the WfD system; and

– Interest in hiring graduates from the system is low.

Modest outcomes:

– System produces some useful outputs;

– Trainees and employers are generally dissatisfied; and

– Access to training is limited and quality of skills imparted is poor.

Good outcomes:

– System trains large numbers;

– Most trainees and employers are satisfied with the system;

– Some scope exists for continuous learning and skills upgrading; and

– System lags behind in skills for innovation and technology upgrading.

Great outcomes:

– System makes a difference and enjoys employers’ and trainees’

confidence;

– Employers participate actively to provide feedback and support;

– Graduates secure gainful employment;

– System encourages continuous learning and skills upgrading; and

– System adapts quickly to new economic conditions and opportunities.

The challenge for policy makers is to situate their WfD system in the

continuum of outcomes and to determine a context‐appropriate strategy for

progressing to the next rung of performance. Their task is arduous, given our

current incomplete knowledge about effective policies. An important objective of

the SABER‐WfD initiative is therefore to help narrow this knowledge gap by

accumulating and analyzing the evidence in this regard. To guide our data

collection effort and analytical strategy we proceed by proposing three

hypotheses on the links between polices and institutions in WfD and the

system’s performance.

Hypothesis 1. We contend that well‐functioning WfD systems require

mutually reinforcing policies that simultaneously address weaknesses in

governance, finance, and information. Policy gaps in any of these areas yield

subpar outcomes across countries in a consistent and similar way. Accordingly,

examining WfD policies in countries that have improved the performance of

their WfD systems will provide insight into priorities for reform and suggest

strategies for plugging the most damaging gaps at a given level of system

development.


Hypothesis 2: We suggest that good policies on their own are insufficient to

improve WfD outcomes. Effective institutions for implementation and efficient

feedback mechanisms are also critical in a dynamic reform agenda that delivers

results. These features combine to create a virtuous cycle of informed policy

design, follow‐up action, and self‐correcting adjustments to minimize skills gaps

and mismatches in a timely fashion. Suboptimal outcomes stem, on the other

hand, from any or all of the following: weak or incomplete policy design, lack of

attention to implementation, or disjointed feedback mechanisms.

Hypothesis 3. We assert that a “learn‐to‐do and do‐to‐learn” approach is

essential in the search for better outcomes in workforce development. It

minimizes what some writers have referred to as the risk of “borrowing” policies

that may have worked in one context and applying them in another with little

adaptation or consideration of local conditions (for example, Allais 2010). The

approach reinforces the notion that, in a field of still evolving knowledge,

successful reforms will require intentional learning by national policy makers

through systematic efforts to build domestic capacity for policy design and

implementation.

The three hypotheses influence our approach to data collection and analysis.

The first explains the collection of comprehensive yet specific data on policies

and institutions addressing market or government failures in the three areas

identified earlier: governance, finance, and information. The second motivates

the collection of evidence on policy implementation, not just the existence of

policies on paper. The third hypothesis leads us to document the use of feedback

mechanisms and evidence of intentional learning in the process of policy reform

and implementation. Through carefully crafted questions and answer options,

the SABER‐WfD instrument generates data for scoring a country’s WfD policies

and institutions against predefined rubrics, as explained in a later section and in

Appendix 2. Suffice it here to say that the rubrics describe a progression in scores

corresponding to increasing evidence of institutionalized processes, of policy

implementation informed by robust analysis, and of continuous learning through

robust feedback mechanisms.

Data for Benchmarking WfD Policies and Institutions

The data are generated through a survey‐like data collection instrument (DCI).

Because the respondent is not a person but a country (or subnational entities), an

immediate question concerns the perspective from which the questions would be

answered. Keeping in mind that the SABER‐WfD initiative is intended mainly to

support policy dialogue among interlocutors in the human development sector,

the most appropriate perspective is that of the line ministries responsible for

WfD. These would typically be the ministries of education and of labor, although

other special agencies may be relevant. Below we elaborate on the overall


structure of the dataset to be generated, elaborate on its content, and summarize

its key features.

Overall Structure of the Dataset

The dataset has a cascading structure: it starts with three broad functional

dimensions of policies and institutional processes; each dimension is associated

with a set of three policy goals; in turn each policy goal is associated with a set of

three actions. The structure is intentionally comprehensive and sufficiently

detailed to endow the contents of the dataset with specific and tangible meaning.

Three functional dimensions of policy choices. The three dimensions are: (a)

the strategic framework, which sets the direction and authorizing environment for

WfD; (b) the oversight arrangements, which guide the functioning of the system;

and (c) the service delivery system, through which outcomes on the ground are

achieved (figure 2).25 From the perspective of the line ministries, strategy is about

sensing, influencing, and responding to the external environment for WfD;

oversight is about governing the activities of all stakeholders with a direct

interest in WfD activities (that is, setting “rules of the game”); and delivery is

about managing the activities of those responsible for training provision.

Figure 2: Functional Dimensions of WfD Policies


25 The functional dimensions of policies in the figure 2 bears some resemblance to the threefold challenge

that the UKCES (2009) believes the United Kingdom faces in its attempt to create a better, more

responsive, demand‐led, and effective WfD system. The three challenges are: (a) “a ‘policy gap’, where a

stronger emphasis on the ‘demand’ side is required to bring about better balance to the policy agenda”;

(b) “a ‘measurement gap’, where developing a more appropriate suite of ‘success measures’ for the

system would help better align policy with the 2020 Ambition, and policy with delivery”; and (c) “a

‘policy to practice gap’, where delivery and arrangements on the ground do not always fulfill the

ambition of the policy promise.”

Strategy:Aligning to national

economic goals

Oversight:Governing to achieve

desired goals

Delivery:Managing for tangible

results


The three functional dimensions are interconnected, as the figure suggests.

The quality of the strategic framework depends on the role that WfD is expected

and enabled to play in fostering the economic success of individuals, businesses,

and the country as a whole. The resulting authorizing environment sets the stage

for oversight institutions and mechanisms that govern the efforts of the parties

involved—policy makers, employers, training providers, trainees—so as to foster

alignment with desired WfD goals. In turn, the oversight arrangements define

the operational context for training provision by public as well as private

providers. The performance of these service providers in equipping trainees with

skills for employment and productivity provide the feedback that closes the

policy‐making loop.

In essence, the tool is about characterizing a holistic system for WfD in terms

of the workings of two markets: “the [labor] market in which employers and

individuals trade work for wages [or earnings]; and the training market in which

individuals obtain training [and therefore acquire skills] from training

providers” (Karmel 2011).

Drilling down to the policy goals and actions. Each functional dimension is

given a more specific meaning by associating them with three policy goals

(figure 3). In total, nine policy goals are identified that are likely to be relevant

across a wide range of developing and emerging economies. Each policy goal is

disaggregated into three actions to clarify is practical meaning. The grand total of

27 actions in the entire dataset appears in Appendix 1.

Figure 3: Policy Goals under Each Dimension


Note: The figure represents the policy goals through the use of single‐word labels, with each row corresponding

to the concepts of governance, finance, and information respectively. The next section of the paper explains

more fully what lies behind these labels.

Strategy

Direction

Priorities

Coordination

Oversight

Pathways

Resources

Standards

Delivery

Content

Incentives

Outcomes


The foregoing structure provides a snapshot of the type of policy data

collected by the DCI. While the information may be organized differently and

while the policy goals and action may overlap because of linkages among them,

the approach satisfies the desire for a well‐structured and parsimonious

instrument that suits our purpose in the SABER‐WfD initiative. Below we

develop the ideas in greater detail and elaborate on their links to the questions

that are included in our data collection instrument.

Dimension 1: Strategic Framework

WfD is not an end in itself but an input toward broader goals—of boosting

employability and productivity; of relieving skills constraints on business

development and growth; and of advancing overall economic growth and social

well‐being. Its multidimensional nature poses complex challenges and tradeoffs.

For example, when policy makers refer to skills for growth, they are typically

concerned about skills that can help the economy diversify, innovate, and

upgrade its technological capability. However, because these skills pertain, by

and large, to formal sector jobs they may be irrelevant to the majority of workers

in developing countries who earn their living in the informal sector.

A second source of complexity is that all jobs call for a combination of skills.

Bloom’s (1976) classic taxonomy makes a distinction among cognitive, affective,

and psychomotor skills.26 Development of these skills typically involves different

periods of gestation; the sequencing and timing of investments also depends on

the specific skills required in different jobs. Some skills (for example, basic

literacy and numeracy) entail sustained effort from relatively young ages (for

example, through early childhood education and general basic education); others

call for specific programs that meet employers’ more immediate needs. On a

fundamental level, WfD strategies entail trade‐offs between short‐ and longer‐

term objectives, often because resources are limited and because of differences in

the pattern of costs and benefits across investments in skills development.

A third reason for the complexity of WfD is that outcomes depend on the

actions of multiple stakeholders, including individuals, employers, training

providers, and government officials (often from several ministries or agencies).

Asymmetric access to credit, imperfect or incomplete information, misalignment

26 Cognitive skills refer to a person’s grasp of academic disciplines such as English, mathematics, as well

as various pure and applied sciences, history, and so forth. Affective refer to his or her perceptions about

work, concept of self and others, and attitude toward timeliness, accuracy, quality, and performance.

Psychomotor skills refer to possession of skills required to perform the tasks or duties involved in an

occupation, job, or business (for example, operating a lathe, preparing architectural plans, installing

equipment, and so forth). The Dictionary of Occupational Titles (DOT) in the United States makes a

distinction among jobs according to the degree of involvement with Data, People, and Things (see

http://www.oalj.dol.gov/PUBLIC/DOT/REFERENCES/DOTAPPB.HTM (accessed on October 10, 2011).

The O*NET’s Content Model, which replaced the DOT in 1998, continues to emphasize the multifaceted

skills requirements of jobs (for more information, see http://www.onetcenter.org/content.html (accessed

on October 10, 2011).


of incentives and weak cooperation among these actors are some of the obstacles

that can lead to suboptimal outcomes in WfD.27

These complex objectives and challenges of WfD lead us to emphasize the

following policy goals under the functional dimension relating to strategy: (a)

alignment of the direction for WfD to the country’s goals for economic growth

and poverty reduction, (b) prioritization of a demand‐led approach to WfD, and

(c) facilitation of critical coordination among the actors at the leadership level of

decision‐making. Bringing together these elements requires visionary leadership

based on an understanding of and conviction about the role and nature of WfD

in national development. It necessitates sound judgment on how to solve short‐

run challenges without losing sight of longer‐run ambitions for WfD. And it calls

for clarity in removing practical roadblocks to cross‐sector cooperation among

diverse stakeholders where such cooperation is vital for achieving critical

breakthroughs.

Policy Goal 1: Clarifying the direction for WfD. Advocacy for WfD by

national leaders provides the public support to legitimize it as a priority for

economic development and to ensure that it receives due attention in policy

dialogue and plans for implementation. To be credible, advocacy needs to be

based on realistic and accurate assessment of the priorities for WfD in light of the

country’s economic prospects. To achieve impact, the assessment must be

followed by policies and practices that promote and sustain coherence between

the demand for and the supply of skills. Thus, to flesh out the goal of bringing

clarity to the strategic direction for WfD we identify the following policy actions:

Advocate for WfD as a priority for economic development;

Evaluate economic prospects and its implications for skills; and

Develop strategic polices to align skills demand and supply.

In reviewing the pattern of economic growth over the past few decades, the

Commission for Growth and Development (2008) has emphasized the

importance of “leaders who are committed to achieving growth and who can

take advantage of opportunities from the global economy.”28 In the area of

workforce development to support growth, East Asia’s fastest growing

economies over the past 50 years—the Republic of Korea; Taiwan, China; and

Singapore—provide the clearest examples of the role of leaders in shaping the

skills agenda. These nations have built a strong foundation in basic skills through

general education for all and now consistently appear among the top‐performing

nations in international tests of student learning (OECD 2010b; Mourshed et al.

2010). At the same time, they have also developed training systems that

27 See Almeida et al. (forthcoming 2012) for an elaboration of various market and government failures

that can lea to suboptimal outcomes in workforce development 28 See

http://www.growthcommission.org/index.php?Itemid=169&id=96&option=com_content&task=view



effectively equip workers with job‐relevant skills to expand their own

technological capabilities (see, for example, Ashton et al. 2002; Green et al. 1999;

Kuruvilla et al. 2002). The increasing sophistication of the exports from these

countries is one sign of success. In Korea, for example, electronic, electrical, and

other high‐technology products made up nearly 40 percent of the country’s

exports in 2006, up from less than 5 percent in 1980 (Yusuf and Nabeshima 2010).

Policy Goal 2: Prioritizing a demand‐led approach to WfD. A key principle

is to recognize the role and influence of employers, firms, and individuals in

determining the demand for skills.29 Because these stakeholders are the ultimate

end‐users of skills, a demand‐led approach to WfD at all levels of decision

making is essential to minimize potential mismatches between skills supply and

demand. For the same reason, translating the ambition of skills‐led economic

development will typically require policies to intensify employers’ and firms’

demand for more and better skills to help raise productivity. Because skill sets

take time to develop, policies often produce results only after a lag. The

discrepancy in timeframe suggests that monitoring the pipeline of skills against

emerging and changing trends in demand and taking steps to avert serious skills

imbalances and mismatches are highly critical in setting strategic priorities in

WfD. Taking the foregoing considerations into account, we associate the

following policy actions with the goal of prioritizing a demand‐led approach to

WfD:

Ensure active engagement of employers and industry in developing WfD

policies;

Strengthen firmsʹ demand for skills to improve productivity; and

Address critical challenges in the future supply of skills.

Country conditions influence the specific ways in which governments set

strategic priorities for WfD. To illustrate, in 1996 Costa Rica prioritized the

introduction of new courses in technical training and new curricula as part of its

eventually successful bid. Led by the country’s president, the country succeeded

in persuading Intel, a global computer chip manufacturer, to set up the firm’s

newest plant in the country (Spar 1998; World Bank/MIGA 2006).30 In subsequent

years, the country continued to align its education and training programs to meet

the needs of employers in the information technology industry.31 The IT industry

29 The role of individuals matters not only as potential employees but also as own‐account workers. The

latter are especially important in settings where most job opportunities are in self‐employment or in

family businesses. 30 The Technological Institute of Costa Rica (ITCR) introduced a one‐year certificate program (for high

school graduates), and a one‐year associate degree and English language training programs. The

technical courses focused on new fields such as semiconductor manufacturing and microelectronics, and

later also included materials science (Spar 1998; World Bank/MIGA 2006). 31 These demand‐responsive efforts included, for example, the following: (a) new technical programs and

enhanced curricula at the country’s leading educational institutions— ITCR, the University of Costa Rica

(UCR), and the National Training Institute (INA), especially during 1999–2003; (b) an English

reinforcement program at ITCR; (c) links between UCR’s School of Physics and technical and vocational


is today a major part the economy, accounting for nearly 19 percent of exports in

2009, compared with just 4 percent in 1997.32 Mexico’s Integral Quality and

Modernization (CIMO) program exemplifies an effort by the government to

foster skills upgrading among firms.33 In Korea, the introduction in 2010 of a

high‐profile government initiative involving “meister” high schools exemplifies

an effort to address emerging shortages of technicians for the country’s priority

sectors (for example, electricity, media, logistics, shipbuilding, IT, and

automotive).34

Policy Goal 3: Strengthening critical coordination to realize WfD

objectives. A key challenge in WfD is that it is by nature complex in terms of the

activities and their timing, involving an intricate web of stakeholders with

diverse interests, roles, and responsibilities. Ensuring that the combined effort of

these partners is consistent with the country’s key priorities for WfD is therefore

an important goal of strategic coordination. Such coordination typically requires

leadership at a sufficiently high level to overcome barriers to cross‐sector or

cross‐ministerial cooperation that are often impossible to resolve among peers

with similar levels of decision‐making authority. Fundamentally, the core issues

pertain to the “interactions among institutions, processes and traditions that

determine how power is exercised, how decisions are taken on issues of public

and often private concern, and how citizens and other stakeholders have their

say” (Abrams et al. 2003). An arrangement that clarifies roles and responsibilities

and fosters communication among the relevant WfD stakeholders sets the stage

for coherent and well‐coordinated actions by these players (see, for example,

ADB 2009). Thus, for the goal of strengthening critical coordination, we identify

the following actions:

Ensure coherence of key strategic WfD priorities;

Institutionalize the structure of WfD roles and responsibilities; and

Facilitate communication and interaction among all WfD stakeholders.

Strategic coordination is particularly relevant for major initiatives in WfD

that are breaking new ground, that involve new or nontraditional partners, and

that possibly also introduces new operational procedures. Such coordination at

schools for electronics; and (d) outreach programs in technology and science for teachers and students in

primary, middle, and high schools. 32 Data from World Development Indicators (see http://data.worldbank.org/data‐catalog/world‐

development‐indicators/wdi‐2011, accessed November 27, 2011). 33 For more information about CIMO, see Tan et al. 2004. 34 These shortages stem from the preference of young Koreans and their families for studies at the

universities. To increase the attraction of a non‐university route to the labor market, the “meister” high

schools follow new curricula that explicitly cultivate “21st century skills.” A personal visit in May 2011 by

two of this paper’s authors to Sudo Electric High School in Seoul, one of the first “meister” schools that

the country’s president inaugurated in March 2010, found the following practices: (a) using a student‐

centered pedagogy; (b) including training on personality development; (c) offering students

opportunities for study abroad; (d) using English for teaching whenever the teachers are able to do so;

and (e) ensuring strong linkages with specific employers.


both the strategic and operational levels can help avoid duplication—or worse,

misalignment or conflict—of effort and ensure that the most critical initiatives

receive the necessary moral and material support for experimentation,

consolidation, and maturation. Ireland’s experience with the establishment of

new regional technical colleges (RTCs) beginning in the late 1960s is an example

of coordinated effort to supply mid‐level technicians for the country’s emerging

industries (O’Hare 2008).35 The Indian government’s decision to create several

new bodies in a three‐tiered governance arrangement for WfD—the National

Council for Skills Development, chaired by the Prime Minister; the National Skill

Development Coordination Board, chaired by the deputy chairman of the

Planning Commission; and the National Skill Development Corporation, headed

by an eminent private sector industrialist—exemplifies a more recent effort at

strategic coordination at the highest level of government.36

Dimension 2: System Oversight

The second functional dimension of WfD policy choices is primarily concerned

with the design of systemic “rules of the game” to help realize the ambition of

growth‐promoting WfD envisioned in the strategic framework. To make our task

manageable, we focus on components of the WfD system that relate to skills

development for mid‐level rather than highly specialized professionals and

managers. Skills acquisition at this level occurs mainly through pre‐employment

training at the secondary and post‐secondary levels and on‐the‐job training,

whether in formal, nonformal, or informal arrangements, including

apprenticeships. The main objective of oversight is to facilitate efficient and

effective skills acquisition by individuals to improve their employability and

productivity so that employers are able to meet their demand for skilled workers

in a timely manner. The expected result is the minimization of mismatches in

skills supply and demand, thereby enhancing the contribution of WfD to

economic growth and social progress. In countries with large informal sectors

where most people are self‐employed, the oversight function also entails

ensuring that individuals are able to access training services that are effective in

enhancing their earning power.

If the markets for skills functioned perfectly there would be a limited role for

oversight. Individuals would have access at no cost to accurate and timely

information about the types of skills in demand; they would know where and

how to get trained; they could easily mobilize the funds needed to pay for the

35 O’Hare (2008) elaborates that the Ireland Development Authority (renamed since 1994 as the Industrial

Development Agency (Ireland)), in its effort to convince foreign businessmen to invest in the country,

used a portfolio of incentives to encourage a greater output of skilled technicians throughout Ireland,

while simultaneously involving the leaders and staff of the training institutions to participate actively in

its promotional efforts (for example, by attending functions where foreign businessmen were gathered,

and travelling to events in potential investor countries). 36 For more details of the arrangements in India, see

http://labour.nic.in/policy/NationalSkillDevelopmentPolicyMar09.pdf (accessed July 19, 2011).


training; and once trained, they would be able to send accurate and credible

signals to potential employers regarding the skills they possess. Likewise,

employers would have reliable and timely information about the skills of

prospective and incumbent employees; they would be able to make accurate

choices among possible training options for their workers; and they would face

no constraints in organizing or purchasing training. Training providers, on their

part, would be fully aware of the skills that are in demand and could tailor their

programs accordingly; and they would have the information to create efficient

pathways for skills acquisition for their clients, for example, through articulation

agreements with providers that offer complementary services.

This scenario is unrealistic and highly unlikely in practice, however.

Information is costly to generate and to acquire, and it may not be complete,

relevant, accurate, or timely (see, for example, Woods and O’Leary 2007).

Individual firms often cannot spare the time to codify their skills requirements,

particularly if they are small or operate in a fast‐changing business climate. Other

distortions may arise from individuals succumbing to the temptation to overstate

their knowledge and skills in order to land a desirable job or a promotion.

Likewise, training providers may be motivated to mask the quality of their

program offerings in the interest of attracting and retaining their clientele.

Inadequate information aside, financial constraints may also prevent those who

can benefit from training from making the investment, thus leading to

suboptimal investments in WfD.

The foregoing considerations motivate us to specific the following three

policy goals for the oversight dimension of our analytical construct: (a)

diversifying the pathways for skills acquisition so as to encourage lifelong

learning as a basis for adapting to evolving labor market conditions and keeping

up with technology in the workplace; (b) ensuring efficiency and equity in

funding WfD investments, thus fostering the system’s long‐term financial

sustainability; and (c) assuring relevant and reliable standards for quality in

service provision and skills acquisition, thus creating more efficient signals in the

market for skills.

Policy Goal 4: Diversifying the pathways for skills acquisition. The

underlying aim here is to keep the skills of the workforce up to date and adapted

to changing economic conditions and opportunities. Some of the main reasons

that make lifelong learning an imperative today include the emergence of new

occupations and skills, and the rapid transformation of existing ones, the

explosion in knowledge and technology, the shift to an information society, the

introduction of new ways to organize the workplace in an increasingly

integrated economy, and an aging society (see, for example, Kiley and Cannon

2000; Wang 2012).

However, lifelong learning, even for the motivated, may be haphazard or

even impractical if few pathways for skills acquisition exist beyond the initial

level of education and areas of training that a young person started out with. The


absence of a qualifications framework and of routine procedures for the

recognition of prior learning presents an added impediment. Individuals without

the formal credentials for admission into their chosen training programs are

assumed to have no relevant skills and must thus start their training at a more

basic level than necessary. The result is to prolong training time and increase

costs for some prospective trainees, which may in turn discourage them from

embarking on plans to learn new skills or build on existing ones. Finally, given

that labor market conditions are always evolving, it is important that all available

options for gaining more and better skills, particularly those funded publicly,

remain responsive and sensitive to emerging trends and opportunities for

employment. Thus, the ability to adjust or terminate programs that no longer fit

the bill or to add new ones is a hallmark of effective oversight. Taking the

foregoing considerations into account, we associate with the goal of diversifying

the pathways for skills acquisition the following actions by the government:

Foster articulation across levels of instruction and types of programs;

Promote lifelong learning with recognition of prior learning; and

Set policies and procedures for the renewal of publicly funded programs.

In a well‐articulated system for WfD, students can transfer from course to

course, progress to higher levels of training or gain access to programs in other

fields (MacKenzie and Plovere 2009). Good examples of articulation

arrangements can be found in the United States, where many community

colleges adopt such practices as: (a) give students credit for work experience in

industry; (b) collaborate with high schools to offer preparatory courses so that

students may fulfill the prerequisites for full‐time enrollment in a community

college; and (c) enter into agreements with universities and other colleges, most

often within their respective states, to provide students with the option to

continue on to college‐ or university‐level programs once they have proven

themselves in a community college program.37 In some countries, a national

qualifications framework provides the scaffolding for defining pathways for

lifelong learning and skills acquisition. As Hanf and Hippach‐Schneider (2005)

explain, the framework essentially “takes all of a countryʹs formally recognized

qualifications and arranges them in a clearly defined structure…. [Qualifications

are] understood as sets of certified or documented skills, with no regard given to

the respective learning path.”

Interest in national qualifications frameworks has grown in recent years and

more than 100 countries worldwide are now reportedly at various stages of

building their own.38 A review by Allais (2010) of the experiences in 16 countries

37 For example, the community colleges in the states of Virginia and Maryland have such agreements

with their respective state colleges and universities. 38 Part of the interest stems from employers in countries that employ foreign workers. For these

employers, the qualifications framework in the sending countries is a tool for gauging the quality of

potential recruits from these countries. Another source of interest stems from the growing demand for

intercollegiate credit transfers and articulations across programs and institutions.


cautions against overly ambitious expectations of such frameworks, however.

Success requires significant resources and patient work by skilled personnel over

many years—conditions that are often absent in developing and emerging

economies. In addition to fostering articulation and the mapping of

qualifications, countries have also found it necessary to exercise oversight of

program offerings as well, particularly in publicly funded institutions, to

minimize what ADB (2009) notes is a ubiquitous tendency to repeat courses year

after year without regard to labor market conditions. Insisting on industry

involvement in decisions about program offerings is an increasingly popular

method for providing oversight of the content of publicly funded training

programs.

Policy Goal 5: Ensuring efficiency and equity in funding for WfD. The

underlying design challenge here is to influence WfD catalytically through the

government’s role in funding WfD, ensuring efficient and effective use of the

available funds, and in fostering partnerships that can multiply the resources,

pecuniary and otherwise, that are available to encourage and support investment

in WfD by individuals and employers. Governments fund WfD in part to

counteract possible suboptimal levels of investment arising from such problems

as credit constraints (for example, those faced by students from poor families),

myopia on the part of prospective trainees and firms, incomplete information, or

other imperfections in the market for training. Because funding for WfD is

required on a continuing basis while public budgets are often limited and

subjected to intense competition from other competing projects, a clear strategy

for sustaining support for WfD, one that benefits from continuous assessment

and adjustment, can help ensure that the funding is stable, sustainable, and

sufficient to achieve the most important goals at reasonable cost. Such a strategy

would recognize that besides public budget allocations, resources for WfD may

also come from fees paid by trainees or their sponsors, and contributions by

private entities, nongovernmental organizations, local communities, and/or

external donors. Some of the contribution may materialize in the form of in‐kind

donations that constitute essential inputs for effective training programs,

including, for example, equipment and the services of experienced skilled

workers. In light of the foregoing considerations, we choose the following actions

by the relevant government authorities to flesh out the goal of ensuring

efficiency and equity in funding WfD investments:

Articulate a strategy for funding WfD;

Allocate public funds for WfD to achieve results with efficiency; and

Foster partnership between WfD authority(s) and stakeholders.

In practice, funding is a ubiquitous concern in WfD systems (see, for

example, Wolf and Erdle 2009). In poorly funded systems, providers are hard

pressed to satisfy the expectations of employers who in turn distance themselves

from the providers; the result is to reinforce providers’ tendency toward supply


driven program offerings, which further undermines their ability to respond to

employers’ needs. These conditions reduce the potential for productive

partnerships in WfD. Many systems in developing and emerging economies fall

victim to this vicious circle because governments simply do not have enough

resources to prioritize TVET when other parts of the education system, such as

primary and secondary schooling, are also poorly funded (see, for example,

Atchoarena 2002; Dunbar 2011). Support for TVET is moreover often weakened

by the fact that most students come from poorer families that typically lack the

political clout to influence budget allocations, as well as by the general

perception of TVET as a “second‐class” route to the labor market with doubtful

impact in equipping students with job‐relevant skills.

Some countries have nonetheless successfully tackled some of these

problems. In Switzerland, for example, TVET tracks at the secondary level offer

excellent programs and pathways to tertiary level courses, which attract a

sufficiently meaningful share of the country’s top students to overcome public

bias against such programs (OECD 2010a; FOPET 2012). In Singapore, sizable

investments in a high‐quality TVET system over the years, coupled with

sustained attention to the employability of graduates, have lowered social

resistance to the TVET programs that the less academically inclined students

enter after 10 years of general schooling (Law 2008).

With regard to funding for the training of incumbent workers, a common

arrangement involves collecting a tax or levy from firms, often in relation to the

size of the payroll, and using the proceeds to reimburse firms that train

(Johanson 2009). Some observers cite Malaysia and Korea as examples of

successful schemes (for example, Lee 2009; Tan 2001). As for allocating funding

to achieve results, having transparent and explicit allocation criteria and

ensuring that the criteria themselves are reviewed and adjusted in light of

experience, are a sine qua non of good oversight of WfD resources. Accordingly,

some recent World Bank operations that support TVET, for example in Pakistan

and India, seek to foster these practices.39 Regarding partnerships, the

relationship between Costa Rica’s training institutions and Intel mentioned

earlier exemplifies arrangements that have been sustained over time because

they generate benefits for all parties: the firm, the participating training

institutions, and the trainees.

Policy Goal 6: Assuring relevant and reliable standards for quality in WfD.

The underlying issue is trust in the services offered by training providers and in

the skills acquired by individuals. Providers are keen to assure potential clients

of the quality of their programs and to market their services, particularly when

39 For example, the Sindh Skills Development Project and the India Vocational Training Improvement

Project; more information can be found on these projects at, respectively, the following sites:

http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSitePK=40941&me

nuPK=228424&Projectid=P118177; and

http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSitePK=40941&me

nuPK=228424&Projectid=P099047 (accessed on October 13, 2011).


these are offered on a fee‐for‐service basis. Obtaining accreditation from a

recognized authority is an efficient way to do so. The process involves an

authorized accrediting body certifying, typically on a term‐limited and

renewable basis, that the training provider has met externally established

standards for excellence or quality in service delivery (Vlăsceanu et al. 2007).

Standards for skills acquisition are a related but separate aspect of quality

assurance. Often codified in the form of a credential or qualification, they

provide benchmarks for testing and certifying the knowledge and competencies

of the graduates of training programs. Established standards help training

providers clarify expectations about learning outcomes and competencies; they

benefit trainees by providing an easy way to advertise their skills to potential

employers; and, if reliable, the standards also assist employers in expediting the

early stages of recruitment. In light of the foregoing ideas, we associate with the

goal of assuring standards the following actions:

Specify accreditation standards for training providers;

Strengthen skills testing and certification; and

Assure the credibility of accreditation and of skills certification.

Countries face a variety of challenges with regard to quality assurance and

standards in WfD. In recent years, the rapid expansion of enrollments at all levels

of education and training in many developing and emerging economies has

overburdened their public institutions and stimulated an explosive growth of

nonstate providers (see, for example, King and Palmer 2010). The diversity in

options for WfD investments is a positive development, but in the absence of

adequate standards or regulation it could also mean a proliferation of

substandard programs (see, for example, Atchoarena 2002; Dunbar 2011).

Specifying minimum standards is one way to weed out the worst operators

while allowing others to reach for a higher bar, if desired—for example, by

obtaining accreditation with internationally recognized bodies such as the

International Standards Organization (ISO).40

Experience with standards for skills certification is highly diverse as well.

Ganzglass et al. (2011) stress the importance of several factors, including

stakeholder involvement in setting standards, benchmarking of the standards to

employers’ requirements, and reliable processes for validating the standards and

for assessing learning. The system adopted by the National Institute for

Metalworking Skills in the United States is an example of the application of these

principles.41 In less developed nations, where the informal sector dominates the

economy, the testing and certifying of skills present special challenges, in part

because many skills are acquired through traditional apprenticeships and

40 Singh and Sareen (2006) note the rising popularity of ISO 9000 certification among Indian educational

institutions. 41 National Institute for Metalworking Skills. Undated. “Skills Standards,” online: https://www.nims‐

skills.org/web/nims/7 (accessed June 17, 2011).


standards are often tacit and probably uneven across practitioners (Atchoarena

2003; de Largentaye 2009; Janjua and Naveed 2009). Nonetheless, in the Indian

government’s nationwide Modular Employable Scheme, launched in 2010, a key

feature is to codify the standards for skills acquisition and certification for a wide

spectrum of jobs, many of them in the informal sector.42

Dimension 3: Service Delivery

This functional dimension of policy choices shifts our focus from the strategic

and systemic to the operational. It addresses implementation challenges in WfD

that stand between a country’s ambition for WfD to support economic

development and what materializes in reality. Training providers, both

government and nonstate, are the main channels through which the country’s

strategic and systemic policies are translated into operational applications. Thus,

we disaggregate this policy dimension into the following three policy goals: (a)

fostering relevant curricula content in training programs; (b) designing

incentives to promote excellence in training provision; and (c) enhancing

accountability for results in WfD, notably through appropriate monitoring and

evaluation arrangements.

Policy Goal 7: Fostering relevant content in training programs. Because a

key objective in WfD is to equip graduates with job‐relevant skills, the content of

training programs warrants close attention. In order to offer market‐relevant

programs, training providers require access to reliable sources of information

about current and emerging skills demand. Employers and industry associations

are one source of such information. However, in the absence of systematic

arrangements it may be difficult and probably too costly for individual training

providers to gather the desired information. Another source of information,

particularly in relation to future skills needs, is research institutions. Because the

mission of such institutions places them at the forefront of developing, adapting,

and introducing products and services that are new to the country or region,

their activities can offer useful insights into the future demand for skills. Besides

the linkages at the group level, each training provider can also benefit from close

ties with the specific employers or industry that they serve. Such ties are an

important source of inputs and guidance on ways that an individual provider

can improve the market relevance of its program offerings.

Important as the curriculum is, it is insufficient by itself to ensure effective

training provision; needed too are skilled instructors and administrators as well

as appropriate facilities, equipment, and materials. Each provider must be able to

attract the number, types, and quality of staff required to deliver the expected

services within its budget constraints. Because good TVET teachers are highly

sought after by industry for their technical skills, training providers are often

hard pressed to satisfy employers’ demand for skills unless they are able to find

42 See http://dget.nic.in/mes/sectors.htm (accessed on July 30. 2011).


creative solutions to their staffing constraints. Training providers must also have

access to the facilities, equipment, and instructional materials required for

implementing their training programs. These items in the WfD system are the

next most costly after teachers; their costs per instructional hour per trainee are

often several times as high as those in academic programs at comparable levels

of instruction. In modern WfD systems, the challenge of providing trainees with

experience in handling actual production equipment is sometimes met through

the use of competency‐based modular curricula that include on‐the‐job training

as part of the program. While attractive, this option is often difficult to organize

in low‐income settings.

Taking into account the foregoing considerations, we choose the following

policy actions to define the goal of fostering relevant content in training

programs:

Strengthen linkages among training institutions, industry and research

institutions;

Integrate industry inputs into the design of training programs; and

Enhance competence of WfD administrators and instructors.

The relationship between the Korea University of Technology and Education

(KUT) and Samsung exemplifies all three facets of the delivery of highly relevant

training content in a high‐tech industry.43 KUT hosts the Employee Vocational

Education Program (EVEP), which offers short courses to Samsung’s own

workers as well as those from the firm’s subcontractors. KUT provides the

premises, and the teaching and administrative staff. Samsung, for its part, leads

in defining the course curricula, provides the equipment, and lends in‐house

experts to co‐teach the courses. The collaboration benefits all parties. Samsung

gains from having subcontractors that can meet its product specification and

quality standards; its subcontractors gain from being able to retain or expand

their business with the company. Most importantly, the university gains from the

opportunity to involve its faculty in organizing and teaching the courses. This

involvement enhances the competence of its administrators and instructors in the

design and delivery of industry‐relevant training programs.

Linkages of this nature are not confined to the industrial sector or to new

and emerging industries. In China, the Beijing Agricultural School (BAS), for

instance, has established linkages with “industry” in the form of farmers and

village communities to disseminate the fruits of agro‐technology research in the

agricultural sector and upgrade the technological competencies of farmers—the

incumbent workers—through formal and nonformal training programs

(UNESCO 2002). The collaboration has resulted in a diversification of

agricultural and animal husbandry products, higher yields, and higher incomes

for the farmers and villages. BAS’s faculty benefit from their hands‐on

interactions with the farmers and villagers, for instance, through problem‐

43 This information is based on a personal visit to KUT in May 2011 by two of the authors of this paper.


solving and developing new managerial practices to meet the needs of their

trainees. The school itself has benefitted through the broadening of its

curriculum, which has become more responsive to the needs of the rural

population. BAS’s infrastructure has been strengthened through the

establishment of experimental facilities in the form of test fields, animal

management technique testing grounds, and rural mechanics services.

Policy Goal 8: Incentivizing excellence in training provision. Training

providers play a key role in equipping individuals with skills for employment

and productivity. The incentives they face are therefore an essential link in the

results chain in workforce development. Where providers must compete for a

share of the training market they are more likely to customize program offerings

to meet individuals’ and firms’ demand for training. This responsiveness in

training provision is critical, given the nature of employers’ demand for skills in

today’s highly competitive economic environment. Globalization and

technological advances, in particular, have produced conditions for substantial

churning in the labor market as new jobs replace existing ones and create a

continuous demand for fresh skills. Because this demand is impossible to predict

with a high degree of precision, the responsiveness of the WfD system depends

not on top‐down planning but on having a diversity of providers that are

appropriately motivated to meet the demand for skills (see, for example, Booth

and Snower 1996).

Financial viability is a powerful incentive that conditions the behavior of

many nonstate training providers. Providers that depend largely on the fees paid

by individuals or firms (typically on behalf of their employees) have strong

incentives to explore the specific skills required by employers and to tailor their

training programs accordingly. By working closely with employers and

developing a reputation for responsive and effective training, these providers

can improve the attractiveness of their services, thereby assuring the commercial

viability of their business. A downside of the profit motive, however, is that it

may drive providers to cut corners as a cost‐saving measure or hide serious flaws

in their operations that are difficult for a poorly informed public to detect. In

some countries, weak regulation of standards in a context of rapid expansion has

led to a proliferation of private sector providers, many of them offering

substandard services (see, for example, Atchoarena 2002).

Relying on state providers is no panacea, however, for the problems of poor

quality services. In many developing and emerging economies, constraints in

public funding typically limit the scope of publicly provided training programs.

Moreover, state provision suffers from a catalogue of common deficiencies that

include excessive centralization of management, limited scope for initiative and

flexibility at the level of individual institutions, outdated curricula, inadequate

infrastructure, poorly trained staff, and tenuous or nonexistent links with

employers (see, for example, Dunbar 2011; ADB 2009). The combination of weak

incentives and low capacity imply that providers will, at best, be slow in


addressing employers’ demand for skills and, at worst, be completely

unresponsive.

In light of the foregoing considerations, we propose the following actions for

the policy goal of providing incentives to foster excellence in training provision:

Promote diversity in training provision;

Incentivize private providers to meet WfD standards; and

Motivate public training institutions to respond to the demand for skills.

An example of action to diversify training provision includes Kenya’s large‐

scale voucher program for workers in the informal sector (also known as the “Jua

Kali” sector). The intervention increased competition and stimulated the

provision of training tailored to the unique needs of informal sector workers

(Johanson and Adams 2004). More recently, in 2008, another experiment with

vouchers in Kenya offers suggestive evidence that institutions that enrolled

voucher winners were more likely to expand their course offerings compared to

institutions that did not enroll any voucher recipients (Hicks et al. 2011). In

Australia, the government of the state of Victoria introduced a user‐driven

funding model that it argues would “drive greater competition (thereby

achieving increased accountability, accessibility and quality) and respond more

effectively to changing labour market needs” (Skills Victoria 2011).

To reinforce the benefits of competition in training provision, it is important

to ensure that private providers satisfy some minimum standards for quality.

The Singapore Accreditation Council (2008), for example, encourages providers

to seek accreditation in order to assure potential clients of the quality of the

services being offered. Linking access to public funding to accreditation status is

one way to minimize the proliferation of low‐quality programs while promoting

diversity in the training market. For providers in the public sector, the more

important challenge is to improve their responsiveness to employers’ demand

for skills. A common approach is to stipulate the inclusion of industry or

employer representatives on the boards of training institutions. In the United

States, for example, Lindburg (2009) notes that most of California’s community

colleges have institutionalized advisory boards to which they turn for guidance

and inputs to ensure that course offerings are relevant to industry needs. Among

developing countries, the practice is spreading, although evidence on its impact

is still unclear; Mozambique and Pakistan are two examples where boards are

being set up under ongoing World Bank–financed operations.

Policy Goal 9: Enhancing accountability for results in WfD. Systematic

monitoring and evaluation of skills development policies and programs and their

impact on outcomes are a hallmark of strong WfD systems. This function gives

policy makers and other interested parties valuable information about the

current status of WfD and its alignment with the country’s economic

development strategy. It also enables overseers of the system to keep track of the

activities and progress of training providers. Appropriately designed, it can also


help to separate promising options from dead ends for improving WfD, thereby

enriching the evidence that informs policy dialogue and reform. Monitoring and

evaluation are thus essential tools for ensuring that the country’s current WfD

policies and institutions are in fact delivering the expected outcomes.

Three aspects of monitoring and evaluation warrant special attention. The

first is the pervasiveness of the culture and practice of collecting, sharing, and

analyzing data to discern trends in skills demand and supply, and emerging

challenges in gaps and mismatches. In developing and emerging economies, data

on WfD and related issues are often limited in quality, scope, timeliness, and

accessibility. Educational attainment and self‐reported literacy are frequently the

only measures of the supply of “skills” that are available to inform policy

analysis and dialogue. For a few countries, scores on international cognitive tests

provide additional insights. These data are useful but insufficient in light of

recent research and feedback from employers about skills mismatches and the

importance of cognitive skills (as opposed to educational attainment alone) as

well as noncognitive skills for workplace success in the twenty‐first century (for

example, di Gropello et al. 2011a and 2011b; Sondergaard and Murthi 2012).

A second aspect of monitoring and evaluation that is relevant from the

perspective of the overseers of the WfD system is the reporting required of

training providers. Trainees’ employment, earnings, and retention by their

employers are the key measures of the success of training programs. Once this

principle is understood, the focus of monitoring and evaluation can be more

clearly defined and targeted. Although training programs may have several

monitoring requirements, the evaluator will be most interested in how cost‐

effective they are in equipping trainees to satisfy the needs of the employers. By

requiring providers to report on their operations and outcomes, the overseers of

the WfD system would be able to create a centralized database for tracking

activities in the system as a whole. This can also provide the comparative data

for individual providers to evaluate their own performance.

Finally, monitoring and evaluation also play a critical role in helping to

identify promising innovations in WfD. This function is essential for keeping up

with changes in skills demand and new ideas in training methodology. While the

innovations may relate to any aspect of the system (for example, finance,

governance, and administration), their common goal is to boost the system’s

performance, either at the systemic or institutional levels. Adapting from the

model in Mulgan and Albury (2003), the process of innovation typically passes

through four phases: (a) generating possibilities through the stimulation and

support of new ideas; (b) incubating and prototyping to develop and test the

most promising of the new ideas; (c) replicating and scaling up the best (that is,

most cost‐effective and most relevant to the local context) among the tested ideas

in a timely and sustainable manner; and (d) distilling and codifying lessons to

institutionalize knowledge and foster a culture of continuous learning and

improvement.


The foregoing considerations suggest that the goal of enhancing

accountability may be advanced through actions that:

Strengthen WfD monitoring and evaluation;

Specify reporting requirements by training institutions; and

Increase the focus on outcomes, efficiency, and innovation in service

delivery.

Efforts to strengthen the monitoring and evaluation function in WfD systems

are being made in many countries. A recent example is the collection of new,

internationally comparable data on workforce skills by the member countries of

the Organization for Economic Co‐Operation and Development (OECD) under

the Programme for the International Assessment of Adult Competencies

(PIAAC), which aims to provide a new source of information about the quality of

skills in the adult population and the link between skills and employment and

productivity (Schleicher 2008). A similar effort by the World Bank known as the

Skills toward Employment and Productivity (STEP) Measurement Study is

underway to support the collection of similar data by national teams in eight

low‐ and middle‐income countries. Korea is an example of a country where

policy makers have access to a compilation of comprehensive and up‐to‐date

data from surveys and other sources to inform policy design (see, for example,

KRIVET 2007). In terms of reporting requirements, WfD systems such as

Singapore’s follow the good practice of requiring public sector training providers

to issue annual reports as an arrangement for accountability.44 With regard to

strengthening the focus on outcomes, efficiency, and innovation, a welcome

trend is the growing practice of rigorous impact evaluation in the field of skills

development, including in the setting of developing and emerging economies.

Recent examples include evaluations of youth training programs in several Latin

American countries (for example, Ibarraran and Shady 2009) and of programs

targeted at adolescent girls in countries around the world.45

Summary of the Functional Dimensions and Related Policy Goals

In summary, the SABER‐WfD dataset has an analytical structure that recognizes

three broad functional dimensions of policy choices and institutional processes

for workforce development: strategy, oversight, and service delivery. For each

functional dimension, it highlights policy goals and related actions that address

market and government failures in governance, finance, and information. Figure

4 recapitulates the policy goals explained above while Appendix 1 provides

additional details.

44 For examples, see the annual reports of the Singapore Workforce Development Authority at

http://www.wda.gov.sg/content/wdawebsite/L209‐001About‐Us/L209D‐AnnualReports.html (accessed

August 12, 2012). and of one of Nanyang Polytechnic, at http://www.nyp.edu.sg/about‐nyp/media‐

centre/publications/annual‐report (accessed August 12, 2012). 45 For more details on the Adolescent Girls Initiative at the World Bank see http://go.worldbank.org/

5PYHEZS360 (accessed on November 9, 2011).


Figure 4: Summary of the Functional Dimensions and Policy Goals in the SABER‐WfD Framework

Dimension Policy Goal Action

9 Actions

9 Actions

9 Actions


Note: See Appendix 1 for details on the policy actions.

Design of the Data Collection Instrument

For each policy action the SABER‐WfD data collection instrument (DCI) poses a

set of questions to which answers are supplied from the perspective of the line

ministries responsible for various aspects of WfD. In general, the answers are

designed to document the existence of policies and their scope, as well as

evidence on policy implementation and feedback loops to inform policy

adjustments and improvements.

The instrument is similar to other surveys in some ways but also differs from

them in important aspects. Like other surveys, the SABER‐WfD DCI has an

intentionally parsimonious design in order to manage the costs and timeliness of

data collection. Because the data are meant for benchmarking, the DCI

emphasizes the collection of diagnostic rather than comprehensive data about

WfD policies and institutions. Further, the fact that the respondent is not a

person but a country and its system of WfD implies that the data must be

gathered indirectly, typically from documents and credible informants. Most of

the data are, moreover, qualitative rather than quantitative in nature, which

presents challenges of precision and objectivity in data collection. We discuss

below the approach taken to address these multiple concerns.

Data for benchmarking. Following the practice in the World Bank’s overall

SABER initiative, WfD policies are benchmarked to four stages of development:

latent, emerging, established, and advanced. The options for answering

questions in the data collection instrument are thus designed to help locate

policies in relation to these four stages. Given the qualitative nature of the data

• 1. Clarifying the direction for WfD

• 2. Prioritizing a demand‐led approach

• 3. Strengthening critical coordination Strategy

• 4. Diversifying pathways for skills acquisition

• 5. Ensuring efficiency and equity in funding

• 6. Assuring relevant and reliable standardsOversight

• 7. Fostering relevance in training programs

• 8. Incentivizing excellence in training provision

• 9. Enhancing accountability for results Delivery


being collected, we begin with a generic rubric (see figure 5) and adapt the scale

to the data collected by the instrument for each of the 27 actions identified in our

framework.

Figure 5: Generic Rubrics for Benchmarking WfD Policies


To illustrate, consider the functional dimension relating to strategy. One of

the three policy goals focuses on clarifying the direction for workforce

development. This goal is itself disaggregated into three actions, one of which

relates to advocacy for workforce development in line with national goals for

economic growth and poverty reduction. The DCI poses two questions on this

action: (a) the type of leadership that advocates for WfD; and (b) the

prioritization of WfD in the country’s agenda for economic development. The

answers are used to benchmark the policy action according to the following

rubric:

Latent: WfD is not prioritized; it does not enjoy the advocacy of visible

champions;

Emerging: Political and other leaders recognize the importance of WfD

for economic development; steps are being taken to develop relevant

policies for skills development and to build institutional capacity for

policy implementation;

Established: Political and other key leaders in industry and the administration provide sustained support for WfD; the institutional

arrangements have been established and WfD priorities are being

implemented; and

Latent•Limited engagement

Emerging

•Some instances of good practice

Established

•Systemic good practice

Advanced

•Systemic good practice meeting global standards


Advanced: WfD is fully integrated into national policies and strategies,

reflecting a holistic approach to WfD; 46 the institutional arrangements for

implementation are fully functional and capable of continuous self‐

evaluation and improvement.

The rubrics for the other 26 policy actions in our tool appear in Appendix 2.

Where the questionnaire relating to a particular action contains several

questions, the answers are aggregated by putting more weight on praxis than on

concepts or plans that exist on paper.47 By assigning a value of 1 for a latent level

of policy development, 2 for an emerging level, 3 for an established level, and 4

for an advanced level, we can create scores for all 27 policy actions for which the

SABER‐WfD data collection instrument gathers data. Taking advantage of the

nested structure of our analytical approach, we compute the scores for the nine

policy goals from the scores for the corresponding policy actions; the calculation

simply involves averaging the scores for the three policy actions associated with

each policy goal. Likewise, the scores for the functional dimensions are a simple

average of the scores for the relevant policy goals.

Balance between diagnostic and comprehensive data. The idiosyncrasy and

complexity of WfD systems present challenges for the design of the DCI. Because

the data are intended for benchmarking WfD systems across countries, all the

questions in the instrument offer the data collector a fixed menu of answers from

which to choose as well as the option to provide additional information that may

be taken into account, as appropriate, in scoring the choices made from the fixed

menu. This feature helps to avoid the loss of potentially important country‐

specific information while minimizing the difficulty of processing data from a

fully open‐ended questionnaire design.

A more difficult challenge relates to the complexity of WfD systems. This

complexity comes to the fore in the context of questions in the DCI relating to the

operational aspects of WfD system. WfD systems typically involve multiple

authorities, often with complementary as well as overlapping responsibilities.

The training programs that are available may target clients at different levels of

instruction, and training providers are likely to serve a diversity of employers

from different sectors of the economy. In large countries, the WfD system may be

highly differentiated across regions. In light of this complexity, the temptation is

to collect comprehensive data on all aspects of the system. However, such a

strategy requires substantial time, effort, and resources, and may in fact exceed

the data required for our needs.

46 A holistic approach is one that ensures systemic coherence by addressing multiple dimensions of skills

development, including: (a) aligning skills training to employers’ needs and national investment

priorities; (b) building strong foundational skills through early childhood development and general

education; and (c) meeting the needs of vulnerable populations (for example, informal sector, traditional

agriculture, and so forth). It also recognizes that interventions often require different time frames to

achieve their impact. 47 Details on our weighting procedure are available from the authors upon request.


As a practical strategy, we proceed on the assumption that the WfD system

may be benchmarked based on the functioning of its most salient or dominant

components. It puts all systems in the best possible light, thus removing some of

the ambiguity inherent in the benchmarking exercise. Unless a single WfD

authority oversees both pre‐employment and in‐service training, the DCI

requests information about the ministries most likely to be involved in oversight

and service delivery functions; in most cases, the ministry of education and the

ministry of labor are the key WfD authorities. To take into account country

specificities, the DCI includes an option for the data collector to include other

ministries should these play an important role in the system. The DCI also

requests information separately for two levels of instruction: secondary and post‐

secondary (non‐university). For a few questions in the DCI, meaningful answers

are possible only in relation to particular economic sectors (for example,

regarding the linkages between training provision and industry) or occupations

(for example, the organization of skills testing). In such cases, the DCI requests

information for the two most dominant or fastest‐growing sectors or

occupations. For large countries, data collection through the DCI is more feasible

and meaningful at the subnational level rather than for the country as a whole.

Data on the de jure and the de facto situation. In line with its intended

purpose, the DCI contains many questions about WFD policies, processes, and

institutions. However, the responses to the questions may reflect the de jure

situation rather than the reality on the ground, thus compromising the validity of

the benchmarking results. For example, many low‐income countries have well‐

written plans to make WfD a key asset for economic development and some even

report plans for sophisticated funding systems for WfD (for example, vouchers

for individuals to pay for skills development). Yet, for a variety of reasons,

including weak capacity and lack of resources, the plans are often not

implemented. In such situations, it would be inappropriate to benchmark the

system based entirely on the intended policies rather than on actual practice. The

difficulty, however, is that a full‐fledged data collection effort to verify

implementation of the stated plans and policies would cost too much and take

too long.

To strike a balance, we adopt the strategy of adding a few more questions

under each topic of inquiry in the DCI to find out if the most important plans,

policies, or programs among those listed, typically no more than the top two,

have been evaluated for impact, and if the recommendations from the

evaluations have led to follow‐up actions. The information clearly does not fully

capture the de facto situation and it is certainly no substitute for a more

comprehensive documentation of the status of policy implementation. The

approach nonetheless provides sufficient information to make it possible to

situate the policy action being benchmarked in relation to the four stages of

development used in our benchmarking exercise.


Piloting the SABER‐WfD Methodology and Expected

Outputs

The SABER‐WfD benchmarking project is best viewed as an exploratory effort by

the World Bank and its development partners to systematize and share cross‐

country knowledge about effective policies and practices. Thus, a spirit of

caution and openness to learning accompanies the implementation of the data

collection instrument described above. Our hope is that this learning by doing

approach would provide a basis for refining both the survey instrument and the

process of data collection. As more data are accumulated, the project should help

deepen our knowledge about the links between policies and approaches and their

impact on outcomes in WfD. This knowledge would in turn inform dialogue about

reform priorities to advance WfD in low‐ and middle‐income countries.

An implementation plan with four phases. The four chronologically

sequenced phases are: (a) creation of a data collection instrument; (b) testing of

the data collection instrument in a first cohort of countries; (c) data analysis and

production of benchmarking results for use in preparing a SABER‐WfD

barometer report for each country; and (d) distillation of lessons to improve the

analytical strategy, the data collection instrument, and the protocol for data

collection and analysis. These resources would facilitate efficient application of

the SABER‐WfD approach in other countries.

As of this writing, the first phase has been completed following

consultations with key experts from around the world.48 The second and third

phases have been completed or are nearing completion in the following

countries: Chile, Ireland, Korea, Singapore, and Uganda. These countries were

selected opportunistically to test the methodology and the working hypotheses.

Ireland, Korea, and Singapore exemplify countries that have made skills

development a critical part of the development strategies and have achieved

undisputed progress over several decades. The inclusion of Chile and Uganda in

the pilot cohort took advantage of the availability of significant knowledge

available to the research team in the context recent World Bank operations and

analytical work. They also allow us to test the relevance of the methodology in

the context of low‐ and middle‐income countries.

As of this writing, a process of review is underway to collect feedback in

order to improve and update the data collection instrument, to enhance the

48 The formal consultations include: (a) a July 2010 meeting with World Bank staff involved in skills‐

related operations and analytical work; (b) an October 2010 consultation with a 16‐person external

advisory group comprising of seasoned TVET leaders from around the world; and (c) a January 2011

workshop with World Bank retirees with significant experience and expertise in the field of skills

development and policy dialogue in developing countries. The interlocutors represent the perspectives

of academics, practitioners, and development specialists and bring together knowledge and expertise

covering a wide range of country perspectives. Feedback from the group has helped to refine all

dimensions of the data collection instrument.


presentation of the findings and to reflect on effective ways to apply the findings

in policy dialogue.49 In response to requests from various quarters, data

collection has started in a number of countries in East Asia (for example,

Malaysia and Vietnam), the Middle East (for example, Turkey), and Europe and

Central Asia (for example, Bulgaria and Romania).

Key processes. Work on the first cohort of countries has proceeded smoothly.

The experience is already informing the approach for countries interested in a

SABER‐WfD benchmarking study.50 Once the decision has been taken to prepare

such a study, the first step is to decide on the assignment of responsibility for

data collection and report writing. Typically, a national team comprising a

seasoned principal investigator and other researchers would perform these tasks.

As the data are gathered, the team may organize work‐in‐progress discussions

with the relevant government counterparts as well as World Bank staff and

others for the purpose of enhancing the completeness and reliability of the data.

Following completion of data collection the data will be processed based on

standard scoring rules in order to generate benchmarking scores that place the

policy actions, policy goals and functional dimensions on the four‐point rubric

developed for the SABER‐WfD study. Upon validation of the benchmarking

results through consultation with national counterparts and other interested and

knowledgeable parties, the national team would prepare a SABER‐WfD Country

Report to summarize the findings. The report would serve as a resource for

ongoing dialogue among the national team and its development partners

regarding WfD policy priorities and reform.

As the SABER‐WfD benchmarking study expands to include more countries,

a systematic cross‐country database on WfD policies and institutions is expected

to emerge. This resource could provide insights for testing and refining the three

hypotheses presented earlier in the paper. At the same time, the accumulation of

experience would shed light on the relevance and usefulness of the

benchmarking approach for supporting policy dialogue on WfD challenges. The

experience is expected to generate ideas for continuous improvements in the

technical resources associated with the SABER‐WfD initiative, including the

conceptual framework, the data collection instrument and protocols, and the

approach for data scoring and report writing.

Expected outputs. Consistent with the experimental nature of the SABER‐

WfD project, the expected outputs are best seen as vehicles for learning. Specific

outputs from this phase of the work includes: (a) a first version of the SABER‐

WfD data collection instrument, which will be updated based on the experience

in applying it in the pilot cohort; (b) rubrics for benchmarking policies for WfD

(see Appendix 2); (c) codified procedures and templates to guide data collection

and processing; (d) SABER‐WfD Country Reports for the first cohort of countries;

49 See, for example, http://blogs.worldbank.org/education/voices‐from‐former‐african‐ministers‐of‐

education‐on‐challenges‐of‐workforce‐development. 50 This interest would typically be conveyed via the World Bank’s task team leader for the country.


and (e) the beginnings of a network of WfD practitioners within the World Bank

and in partner organizations and countries that are equipped to implement the

benchmarking tool and process.

Conclusion

In the quest for economic growth, many countries see workforce development,

often in the form of technical and vocational education and training, as an

essential investment. However, TVET has continued to pose daunting challenges

because it is expensive and has often failed the test of meeting employers’ skills

requirements. Although the mismatch between what the system supplies and

what employers demand has many causes, some of them beyond the control of

training programs, policy makers are keen to learn from the experiences of

countries that have made progress in equipping trainees with job‐relevant and

productivity‐enhancing skills. Some of them have turned to the World Bank,

among others, for assistance in mobilizing this knowledge to inform efforts to

improve the effectiveness of their workforce development strategies. This paper

is a response to this growing interest among the Bank’s partner countries. The

work is admittedly experimental at this stage. It has involved the articulation of a

conceptual framework and an analytical strategy as well as the creation of a new

data collection instrument and its application in a first cohort of countries. It is

hoped that the accumulation of experience from this phase marks the beginning

of a process that would in time add a new tool to the toolkit that policy makers

and their development partners can use in the search for better policies and

practices to improve outcomes in workforce development.


Appendix 1: Functional Dimensions, Policy Goals,

and Actions in SABER‐WfD

Dimension Policy goal Action

Strategic Framework

Articulating a Strategic Direction

Advocate for WfD as priority for economic development

Evaluate economic prospects for skills

Develop policies to align skills demand and supply

Prioritizing a Demand-Led Approach

Promote a demand-driven approach to WfD

Strengthen firms' demand for skills to improve productivity

Address critical challenges in the future supply of skills

Strengthening Critical Coordination

Ensure coherence of key strategic WfD priorities

Institutionalize the structure of WfD roles and responsibilities

Facilitate communication and interaction among all WfD stakeholders

System Oversight

Diversifying Pathways for Skills Acquisition

Foster articulation across levels and programs

Promote lifelong learning

Set policies and procedures to renew programs

Ensuring Efficiency and Equity in Funding

Articulate funding strategy

Allocate funds to achieve efficient results

Foster partnerships

Assuring Relevant and Reliable Standards

Specify accreditation standards

Strengthen skills testing and certification

Assure credibility of accreditation and of skills testing

Service Delivery

Fostering Relevance in Training Programs

Link training providers, industry and research institutions

Integrate industry inputs into the design of training programs

Enhance competence of WfD administrators and instructors

Incentivizing Excellence in Training Provision

Promote diversity in training provision

Incentivize private providers to meet WfD standards

Motivate public training institutions to respond to the demand for skills

Enhancing Accountability for Results

Strengthen the WfD monitoring and evaluation system

Specify reporting requirements by training institutions

Increase focus on outcomes, efficiency, and innovation in service delivery



Appendix 2: Benchmarking Rubrics


Goal Action Level of Development

Latent Emerging Established Advanced

Art

icul

atin

g a

Str

ateg

ic D

irec

tion

for

Wor

kfor

ce D

eve

lopm

ent

Advocate for WfD as a priority for economic development

WfD is not prioritized in national economic development.

Political and other leaders recognize the importance of WfD for economic development; economic development plans have identified a few WfD priorities.

Political and other key leaders in industry provide sustained support for WfD; economic development plans assess and specify several WfD priorities that are being implemented.

WfD is fully integrated into national policies and strategies, reflecting a holistic approach51 to WfD; economic development plans formally assess and specify a wide range of WfD priorities that are supported by implementation plans and budgets; these are subject to continuous evaluation and improvements.

Evaluate economic prospects and its implications for skills

Policies are being developed but are not based on formal analyses of skills demand.

A few policies have been developed on the basis of occasional assessments to address imbalances between skills demand and supply; these policies and interventions are subject to in-house reviews.

A range of policies based on occasional and routine assessments by government and independent WfD stakeholders have been implemented to address skills imbalances; these are subject to routine in-house reviews and independent external evaluations.

Policies are formulated on the basis of well-informed analyses, including assessments by independent organizations, and they are routinely reviewed and updated with inputs from relevant stakeholders to ensure program offerings fit the economic climate and demands for new skills.

Develop polices to align skills demand and supply

The concept of a demand-driven approach52 to WfD has yet to emerge.

A demand-driven WfD strategy is beginning to take shape but policy reforms are often impeded by various difficulties.

A demand-driven WfD strategy informed by appropriate analyses is accompanied by some policy reforms that have been implemented.

A well-informed demand-driven WfD strategy with continuous evaluation and improvements has been internalized as a standard practice.

51 A holistic approach is one that addresses multiple dimensions of skills development, including: (a)

aligning skills training to employers’ needs and national goals for productivity, growth, and poverty

reduction; (b) governing the system to achieved the desired national goals, and (c) ensuring tangible

results on the ground. 52 In a demand‐driven strategy, the demand for skills drives the supply of training services.

Arrangements to achieve this relationship between skills supply and demand include: the involvement

of employers in shaping training policies and provision, financing tied to employment outcomes, and so

forth.





Prio

ritiz

ing

a D

eman

d-le

d A

ppro

ach

Promote a demand-driven approach

There is limited or no attempt to incorporate business and industry inputs in establishing and implementing WfD priorities.

Business and industry play an advisory role in establishing and implementing WfD priorities based on occasional studies and assessments.

A demand-driven approach to WfD is in place with business and industry providing inputs for setting WfD priorities based on routine assessments provided by government agencies, employers, trade associations, and labor unions.

A demand-driven approach to WfD has been fully established with business and industry playing both advisory and executive roles supported by routine assessments from government agencies, other key WfD stakeholders, and independent organizations.

Strengthen firms' demand for skills to improve productivity

Few incentives and services exist to support skills development for technology upgrading by firms.

Incentives and services are in place to provide selective support for technology-related skills upgrading; incentive programs are subject to occasional reviews but often without adequate follow-up of recommendations.

Incentives and services enable firms to expand the skills sets of workers to facilitate technology adaptation and adoption for greater productivity; these measures are supported by routine reviews followed by implementation of some review recommendations.

Incentives and services enabling firms to address skills constraints impeding their ability to upgrade technologies and productivity are well established; these are routinely reviewed and adjusted for impact; all key review recommendations are implemented.

Address critical challenges in the future supply of skills

There is limited or no formal assessment of the future supply of skills.

Future supply of skills is assessed on an occasional basis; recommendations from assessments are implemented with some delay, often without adequate funding and assignment of responsibility for implementation.

Assessments of future skills supply are routinely conducted for key sectors at the regional and national levels; recommendations are implemented with little delay; responsibilities for implementation of recommendations are made explicit but without explicit attention to monitorable goals.

Future skills supply is routinely assessed for multiple industries and sectors at the national and international levels; recommendations are implemented promptly; responsibilities for implementation are clearly spelled out and attention is given to the realization of monitorable goals.





Str

engt

heni

ng C

ritic

al C

oord

inat

ion

Ensure coherence of key strategic WfD priorities

There is no mechanism in place to ensure coherence of key strategic WfD priorities among WfD leaders.

Coherence of key strategic WfD priorities at the leadership level is achieved through informal processes that yield limited WfD outcomes.

Coherence of key strategic WfD priorities at the apex leadership level is achieved through formal and informal mechanisms that yield positive WfD outcomes.

Formal mechanisms overseeing coordination and implementation of WfD strategies are in place and they support strengthening structures of WfD policy development, budget allocations, and assessments of future skills demand and supply.

Institutionalize the structure of WfD roles and responsibilities

Roles and responsibilities for WfD are not formally defined, leaving the WfD authority without a clear mandate.

Roles and responsibilities of WfD stakeholders are poorly defined, leaving the WfD authority with a limited mandate and limited resources to discharge its responsibilities effectively.

Roles and responsibilities are well-defined and supported by legislation and resources that enable the WfD authority and key stakeholders to discharge their respective functions effectively.

Clear WfD roles and responsibilities have been institutionalized through legislation and the WfD authority has the mandate to formulate and request resources that are needed for the relevant authorities to discharge their responsibilities in a transparent and effective manner.

Facilitate communication and interaction among all WfD stakeholders

No formal process exists for engaging all stakeholders.

Informal structures exist that facilitate communication and interaction among key stakeholders.

Formal structures exist in key economic sectors that support extensive communication and interaction among the relevant stakeholders.

Formal structures fostering extensive interactions among WfD stakeholders that culminate in consensuses on WfD priorities and policies are in place in most sectors.


Functional Dimension 2: System Oversight



Div

ersi

fyin

g P

ath

way

s fo

r S

kills

Acq

uisi

tion

Foster articulation across levels and programs

No functioning articulation arrangements.

Ad hoc articulation arrangements exist within secondary schools and post-secondary institutions; only ad hoc incentives are in place to foster articulation across levels of instruction.

Ad hoc articulation arrangements exist across institutions at the secondary and post-secondary levels; a program of incentives is in place to foster articulation arrangements.

Standardized articulation arrangements exist across secondary and post-secondary programs as well as between TVET and higher education; a system of incentives is in place to foster articulation across programs and levels of education and training.

Promote lifelong learning

No arrangements or public resources are in place to support lifelong learning, recognition of prior learning, and disadvantaged groups.

Ad hoc private resources and arrangements support lifelong learning and recognition of prior learning; publicly funded training programs exist with for disadvantaged groups subject to some restrictions.

School- and community-based resources and arrangements support lifelong learning and recognition of prior learning; publicly funded training programs with minimal restrictions are available for most disadvantaged groups.

Integrated regional or national system with one-stop online resources and standardized arrangements support lifelong learning and recognition of prior learning; publicly funded training programs provide open access to all disadvantaged groups.

Set policies and procedures to renew programs

There are no set policies to manage program offerings; training providers may introduce, adjust, or close publicly funded programs at will.

Introduction, adjustment, and closure of publicly funded programs are made through ad hoc, nonstandardized processes; applications for these changes must be done personally and are vetted by ad hoc committees.

Introduction, adjustment, and closure of publicly-funded programs are based on a few explicit and standardized requirements; applications can be made online and they are vetted by formal committees with some representation from other WfD stakeholders.

Management of publicly funded training programs are made on the basis of comprehensive and explicit requirements that include labor market analyses; applications can be made online and they are vetted by formal committees with representation from other WfD stakeholders and they operate with a commitment to act in a timely manner.





Ens

urin

g E

ffici

ency

an

d E

quity

in F

undi

ng

Articulate funding strategy

Ad hoc funding of WfD by multiple stakeholders; no evaluation of funding allocation and strategy.

Systematic funding of WfD is determined by government agencies with annual budget appropriations and line-item allocations; only occasional evaluations of funding allocation and strategy.

Systematic funding of WfD is determined by government agencies with advice from key stakeholders; annual budget appropriations are supported by detailed spending plans; there are routine evaluations of funding allocation and strategy.

Systematic funding of WfD is determined through consensus building among government agencies and key stakeholders; annual budget appropriations are supported by detailed spending plans to foster improved performance; routine evaluations of funding allocation and strategy are accompanied by appropriate reforms as needed.

Allocate funds to achieve efficient results

No formal process for allocating public funds for WfD.

A formal process without explicit criteria is in place; there are no reviews of allocation criteria.

A formal process for allocating public funds based on explicit criteria exists; there are periodic reviews of the criteria but recommended changes face relatively long implemented lags.

Allocation of WfD funds is based on explicit criteria aligned with WfD priorities, including efficiency in resource utilization; there are frequent reviews of the criteria and recommendations are implemented in a timely manner.

Foster partnerships

Limited or no partnership between WfD authority and stakeholders in business and industry; key stakeholders provide few, if any, resources toward meeting WfD priorities.

Limited partnership with business and industry is in place; partners have access to some public resources; key stakeholders contribute a small range of resources toward WfD priorities.

Extensive partnership between WfD authority and key stakeholders in business and industry; partners have access to some public resources; key stakeholders contribute a broad range of resources for WfD.

An institutionalized partnership network with open membership for all WfD stakeholders is in place; partners have access to wide range of public resources; key stakeholders contribute an extensive range of resources to meet WfD priorities.





Ass

urin

g R

elev

ant a

nd R

elia

ble

Sta

ndar

ds

Specify accreditation standards

No accreditation standards have been established; training providers are free to offer any program.

Some accreditation standards have been established; standards are infrequently reviewed; accreditation applies to public institutions only.

An accreditation agency has been established with standards developed jointly with relevant stakeholders; standards are reviewed internally on a regular or as needed basis; accreditation applies to public institutions and nonstate providers receiving public funding; renewal applies to the latter only.

An accreditation agency with standards reflecting international best practices is in place; accreditation standards are reviewed frequently both internally and by independent parties; accreditation and renewal of accreditation is compulsory for all public institutions and nonstate training providers, regardless of their sources of funding.

Strengthen skills testing and certification

Competency-based testing has yet to be introduced; testing is largely based on theoretical knowledge and administered by training providers themselves.

Competency-based testing applies to critical occupations in key sectors; testing may focus on a mix of theory and practice and is administered and certified by independent third parties.

A standardized competency-based testing system is in place and applies to most occupations; testing may focus on a mix of theory and practice and is administered and certified by independent third parties.

A standardized competency-based testing system has been established for most occupations; IT-based testing focuses on theory and practice and is administered and certified by independent third parties.

Assure credibility of accreditation and of skills certification

There is limited attention to accreditation standards.

Accreditation standards are established through ad hoc arrangements; some support is provided to encourage nonstate providers to seek accreditation; credibility of skills testing is ensured through explicit standardized testing protocols.

Accreditation standards established with inputs from WfD stakeholders apply to all institutions and providers receiving public funding; credibility of skills testing is ensured through explicit standardized testing protocols and accreditation of testing centers.

A license to operate is issued only to institutions and providers meeting accreditation standards determined by key WfD stakeholders; credibility of skills testing is ensured through standardized testing protocols, accreditation of testing centers, and random audits.


Dimension 3: Service Delivery



Fos

terin

g R

ele

vanc

e in

Tra

inin

g P

rogr

ams

Link training industry and research institutions

Weak or no links between training institutions and industry and research institutions.

Informal links exist between some training institutions and industry and research institutions to improve training relevance and quality.

Formal links exist between some training institutions and industry and research institutions, leading to significant collaboration in several activities.

Formal links exist between most training institutions and industry and research institutions, leading to significant collaboration in a wide range of activities such as the provision of industry internships and training, and the introduction and redesign of training programs.

Design training with industry inputs

Industry has limited or no role in identifying, prioritizing and designing publicly funded programs.

Industry has an advisory role in identifying, prioritizing, and designing publicly funded programs in some training institutions, usually through informal contacts.

Industry has both an advisory and a decision-making role in identifying, prioritizing, and designing publicly funded programs in some training institutions.

Industry has a widespread advisory and decision-making role in identifying, prioritizing, and designing publicly funded programs in most training institutions.

Improve competence of WfD administrators and instructors

Few or no measures are in place to enhance the competence of WfD administrators and instructors.

Recruitment of administrators and instructors is based on minimum academic qualification(s), with provisions for some in-service training and performance-based recruitment and retention measures based on occasional evaluations.

Recruitment of administrators and instructors is based on minimum academic qualification(s), with provisions for in-service training and performance-based recruitment and retention measures that are based on routine evaluations.

Recruitment of administrators and instructors occurs through a competitive process based on both academic qualification(s) and industry experience, with a wide range of in-service training programs and performance-based recruitment and retention measures based on routine evaluations.




Ince

ntiv

izin

g E

xcel

lenc

e in

Tra

inin

g P

rovi

sio

n

Promote diversity in training provision

Training occurs through state provision only, with no incentives to promote nonstate provision of training.

Training policies allow some private providers to operate; training is provided mainly by nonprofit providers with few incentives in place to foster nonstate provision of training.

Training policies facilitate participation of nonstate training providers; training is offered mainly by NGOs, with a system of incentives that are evaluated routinely are in place to foster nonstate provision.

A highly diverse mix of nonstate training providers offer training within a comprehensive system with a wide range of incentives is in place to foster nonstate provision; incentives are subject to evaluations and the recommendations are implemented.

Incentivize private providers to meet WfD standards

No incentives are in place to encourage nonstate providers to meet WfD standards.

At least one incentive that is subject to occasional evaluation is in place to encourage nonstate providers to meet WfD standards, but no review system exists to ensure continued adherence to WfD standards.

A system of financial and nonfinancial incentives that are subject to occasional and routine evaluations is in place to encourage nonstate providers to meet WfD standards; periodic audits are conducted to ensure continued adherence to WfD standards.

A comprehensive system of incentives that are subject to both occasional and routine evaluations and adjustments is in place to encourage nonstate providers to comply with WfD standards; periodic audits with penalties for noncompliance are conducted and enforced to ensure continued adherence to WfD standards.

Motivate public training institutions to respond to demand for skills

No mechanism or process is in place to ensure training institutions are demand-driven.

Training institutions are expected to meet target repetition and graduation rates but few incentives are in place ensure they are demand-driven.

Training institutions are expected to meet a wider range of WfD outcomes; some incentives and penalties that are subject to both occasional and routine evaluations are in place to ensure these institutions respond to the demand for skills.

Training institutions are expected to meet a wide range of WfD outcomes; a robust system of incentives and penalties that is subject to both occasional and routine evaluations and adjustments is in place to ensure that the training institutions are driven by employers’ demands for skills.




Enh

anc

ing

Acc

ount

abi

lity

for

Res

ults

Strengthen WfD monitoring and evaluation

Limited attention is placed on the monitoring and evaluation of skills demand and supply; an overview of WfD data is available through informal channels only.

Occasional monitoring and evaluation of skills demand and supply is in place; an overview of WfD data is available only in government agencies.

Routine monitoring and evaluation of skills demand and supply is in place; an overview of WfD data is available in published reports and Websites.

Skills demand and supply are monitored and evaluated through routine surveys and specially commissioned studies; WfD data are available from a consolidated Website.

Specify reporting requirements by training institutions

No specific data collection and reporting are required; training providers maintain their own databases.

Public institutions and nonstate training providers are required to collect and maintain administrative and graduation statistics; data reporting is voluntary for nonstate providers but they may be notified of noncompliance.

Public institutions and nonstate training providers are required to collect, maintain and submit a comprehensive list of data through an integrated management information system to the WfD authority; timely submission is fostered through incentives for compliance and penalties for noncompliance.

Both public institutions and nonstate training providers are required to collect, maintain, and submit a comprehensive list of data, including client feedback, to the WfD authority using an integrated management information system; incentives, penalties, and data quality audits are performed to ensure timely reporting of reliable data.

Increase focus on outcomes, efficiency and innovation

No system of evaluation and monitoring is in place to ensure efficiency in delivery of training services.

Occasional evaluation and monitoring of limited aspects of training services is in place with results used to provide feedback to the training institutions; information on labor market outcomes of graduates is publicly available for some institutions only.

Routine evaluation and monitoring of several key aspects of training services is in place with results used to provide feedback to training institutions, to prioritize funding allocations, and identify good practices in service delivery; information on labor market outcomes of graduates is publicly available for all institutions.

Institutionalized routine evaluation and monitoring of all key aspects of the delivery of training services with results used to provide feedback to institutions, to prioritize funding allocations, indentify good practices, and to identify options for system-level improvements; online dissemination of labor market outcomes of graduates is available to all users.


References

Abrams, Peter, Grazia Borrini‐Feyerabend, Julia Gardner, and Pippa Heylings.

2003. “Evaluating Governance: A Handbook to Accompany a Participatory

Process for a Protected Area.” Unpublished manuscript. Gatineau, Quebec:

Parks Canada and TILCEPA. http://cmsdata.iucn.org/downloads/

evaluating_governance_handbook.doc (accessed on October 27, 2011).

Adams Arvil V. 2007. “The Role of Youth Skills Development in the Transition to

Work: A Global Review.” HDNCY 5, Human Development Network

Children and Youth Department, World Bank, Washington, DC.

http://siteresources.worldbank.org/INTCY/Resources/395766‐

1187899515414/RoleofYouthSkills.pdf (accessed on October 27, 2011).

Allais, Stephanie. 2010. The Implementation and Impact of National Qualifications

Frameworks: Report of a Study in 16 countries. Geneva: Skills and

Employability Department, International Labour Office (ILO).

http://www.ilo.org/wcmsp5/groups/public/‐‐‐ed_emp/‐‐‐ifp_skills/

documents/meetingdocument/wcms_126589.pdf (accessed on October 27,

2011).

Almeida, Rita, Jere Behrman, and David Robalino. 2012. “The Economic

Rationale for Skills Development Policies.” In The Right Skills for the Job?

Rethinking Training Policies for Workers, ed. Rita Almeida, Jere Behrman, and

David Robalino. Washington, DC: Human Development Perspectives,

World Bank. http://elibrary.worldbank.org/content/book/9780821387146

(accessed on August 13, 2012).

Alstete, Jeffrey W. 1995. “Benchmarking in Higher Education.” ASHE‐ERIC:

Higher Education Report 5. The George Washington University Graduate

School of Education and Human Development, Washington DC.

Ashton, David, Francis Green, Johnny Sung, and Donna James. 2002. “The

Evolution of Education and Training Strategies in Singapore, Taiwan and S.

Korea: A Development Model of Skill Formation.” Journal of Education and

Work 15(1): 5–30.

———. 2009. Good Practice in Technical and Vocational Education and Training.

Manila: Asian Development Bank. http://www.adb.org/

Documents/Guidelines/Good‐Practice‐Education‐Training/Good‐Practice‐

Education‐Training.pdf (accessed on October 27, 2011).

Asian Development Bank (ADB). 2010. Education and Skills: Strategies for

Accelerated Development in Asia and the Pacific. Mandaluyong City,

Philippines: Asian Development Bank.


Atchoarena, David. 2002. “Private Training.” In World Bank Study on: Vocational

Skills Development in Sub‐Saharan Africa—a Working Group Review, ed.

Martha Caddell and Keith Holmes. Edinburgh: Working Group for

International Cooperation in Skills Development. http://www.norrag.org/

medias/documents/pdf/publications/papers/Paper7.pdf (accessed on

October 27, 2011).

Banerji, Arup, Robert Holzmann, Pierella Paci, Carmen Pages, Stefano Scarpetta,

and Milan Vodopivec. 2008. “MILES to Go: A Quest for an Operational

Labor Market Paradigm for Developing Countries.” Unpublished

manuscript. Social Protection and Labor Sector, World Bank, Washington,

DC. http://siteresources.worldbank.org/INTLM/Resources/390041‐

1212776476091/5078455‐1267646113835/MILESThequestoperational

LMparadigm_Jan212008.pdf (accessed on October 27, 2011).

Betcherman, Gordon, Karina Olivas, and Amit Dar. 2004. “Impacts of Active

Labor Market Programs: New Evidence from Evaluations with Particular

Attention to Developing and Transition Countries.” Social Protection

Discussion Paper Series 0402, World Bank, Washington, DC.

http://info.worldbank.org/etools/docs/library/251019/day6DiscussionPaper

Series0402April6Se1.pdf (accessed on October 27, 2011).

Betcherman, Gordon, Martin Godfrey, Olga Susana Puerto, Friederike Rother,

and Antoneta Stavreska. 2007. “Global Inventory of Interventions to

Support Young Workers: Synthesis Report.” Social Protection Discussion

Paper 0715, World Bank, Washington, DC. http://siteresources.

worldbank.org/EXTECAREGTOPYOUTH/Resources/YEI_Synthesis_Paper

_Final_July_13_2007.pdf (accessed on October 27, 2011).

Blom, Andreas, and Hiroshi Saeki. 2011. “Employability and Skill Set of Newly

Graduated Engineers in India.” World Bank Policy Research Working

Paper No. 5640, World Bank, Washington, DC.

Bloom, Benjamin.S. 1976. Human Characteristics and School Learning. New York:

McGraw‐Hill.

Booth, Alison L., and Dennis J. Snower. 1996. “Introduction: Does the Free

Market Produce Enough Skills?” In Acquiring Skills: Market Failure, Their

Symptoms and Policy Responses. Alison L. Booth and Dennis J. Snower, eds.

Cambridge: Press Syndicate of the University of Cambridge.

Camp, Robert C. 1989. Benchmarking: The Search for Industry Best Practices that Lead

to Superior Performance. Milwaukee, Wisconsin: ASQC Quality Press.


Center for the Development of Vocational Training (CEDEFOP). 2010. Skills

Supply and Demand in Europe. Medium Term Forecast Up To 2020.

Luxembourg: Publications Office of the European Union.

http://www.cedefop.europa.eu/download‐manager.aspx?id=15535&lang=

en&type=publication (accessed on October 27, 2011).

Commission for Growth and Development. 2008. The Growth Report: Strategies for

Sustained Growth and Inclusive Development. Washington, DC: World Bank.

http://cgd.s3.amazonaws.com/GrowthReportComplete.pdf (accessed on

October 27, 2011).

Dattakumar, R., and R. Jagadeesh. 2003. “A Review of Literature on

Benchmarking.” Benchmarking: An International Journal 10(3): 176–209.

de Largentaye, Armand Rioust. 2009. “Vocational Training and the Informal

Economy.” In Promoting Pro‐Poor Growth Employment. Paris: OECD.

http://www.oecd.org/dataoecd/63/11/43514554.pdf (accessed on October 27,

2011).

Deloitte, The Manufacturing Institute, and Oracle (DMIO). 2009. “People and

Profitability: A Time for Change: A 2009 People Management Practices

Survey of the Manufacturing Industry.” http://www.deloitte.com/

assets/Dcom‐UnitedStates/Local%20Assets/Documents/us_pip_people

managementreport_100509.pdf (accessed on October 27, 2011).

di Gropello, Emanuela, with Aurelien Kruse and Prateek Tandon. 2011a. Skills for

the Labor Market in Indonesia: Trends in Demand, Gaps, and Supply.

Washington, DC: World Bank. http://ddp‐ext.worldbank.org/EdStats/

IDNpub11.pdf (accessed on October 27, 2011).

di Gropello, Emanuela, with Hong Tan and Prateek Tandon. 2011b. Skills for the

Labor Market in the Philippines. Washington, DC: World Bank.

Dunbar, Muriel. 2011. “Engaging the Private Sector in Skills Development.”

Guidance Notes: A DFID Practice Paper. London: Department for

International Development, Government of the United Kingdom.

Federal Office for Professional Education and Technology (FOPET). 2012.

Vocational and Professional Education and Training in Switzerland: Facts and

Figures. Basel: Federal Office for Professional Education and Technology.

Ferrier, Fran, Clifford Trood, and Karen Whittingham. 2003a. Going Boldly into the

Future: A VET Journey into the National Innovation System, Adelaide: National

Centre for Vocational Education Research (NCVER), Australian National

Training Authority. http://www.ncver.edu.au/research/proj/nr9036_

vol1.pdf (accessed on October 27, 2011).


Ferrier, Fran, Clifford Trood, and Karen Whittingham. 2003b. Going Boldly into

the Future: A Series of Case Studies of Co‐operative Research Centres and their

Relationships with the VET Sector. Adelaide: National Centre for Vocational

Education Research (NCVER), Australian National Training Authority.

http://www.ncver.edu.au/research/proj/nr9036cs.pdf (accessed on October

27, 2011).

Ganzglass, Evelyn, Keith Bird, and Heath Prince. 2011. Giving Credit Where Credit

is Due: Creating a Competency‐Based Qualifications Framework for Postsecondary

Education and Training. Washington, DC: Center for Postsecondary and

Economic Success. http://www.clasp.org/admin/site/publications/files/

Giving‐Credit.pdf (accessed on October 27, 2011).

Green, Francis, David Ashton, Donna James, and Johnny Sung. 1999. “The Role

of the State in Skill Formation: Evidence from the Republic of Korea,

Singapore, and Taiwan.” Oxford Review of Economic Policy 15(1): 82–96.

Hanf, Georg, and Ute Hippach‐Schneider. 2005. “What Purpose do National

Qualifications Frameworks Serve? A look at Other Countries.” Vocational

Training in Research and Practice (Bundesinstitut für Berufsbildung) Special

Edition H 20155 (Promoting Innovations in Vocational Education and

Training: An Exchange of German Experiences): 9–14. http://www.bibb.de/

veroeffentlichungen/en/publication/download/id/1692 (accessed on

October 27, 2011).

Hanushek. Eric A., and Ludger Wößmann. 2006. “Does Educational Tracking

Affect Performance and Inequality? Differences‐in‐Differences Evidence

Across Countries.” Economic Journal 116(510): C63–C76.

Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of

Cognitive and Noncognitive Abilities on Labor Market Outcomes and

Social Behavior.” NBER Working Paper 12006, National Bureau of

Economic Research, Cambridge, MA.

Hicks, Joan Hamory, Michael Kremer, Isaac Mbiti, and Edward Miguel. 2011.

“Vocational Education Voucher Delivery and Labor Market Returns: A

Randomized Evaluation among Kenyan Youth.” Report for Spanish Impact

Evaluation Fund (SIEF) Phase II, World Bank, Washington, DC.

Ibarraran, Pablo, and David Rosas Shady. 2009. “Evaluating the Impact of Job

Training Programs in Latin America: Evidence from IDB Funded

Operations.” Journal of Development Effectiveness 2(1): 195–216.

Jacobs, Ronald L., and Joshua Hawley. 2008. “Emergence of Workforce

Development: Definition, Conceptual Boundaries, and Implications.” In

International Handbook of Education for the Changing World of Work (Vol. 6), ed.

R. MacLean and D. Wilson. Dordrecht, Netherlands: Springer Science +

Business Media B.V.


Janjua, Shehryar, and Arif Naveed. 2009. “Skills Acquisition and the Significance

of Informal Training System in Pakistan—Some Policy Implications.”

RECOUP Policy Brief No. 7, Research Consortium on Educational

Outcomes and Poverty, Cambridge. http://www.dfid.gov.uk/r4d/PDF/

Outputs/ImpOutcomes_RPC/pb7.pdf (accessed on October 29, 2011).

Johanson, Richard. 2009. “A Review of National Training Funds.” Social

Protection Discussion Paper 0922, World Bank, Washington, DC.

http://siteresources.worldbank.org/SOCIALPROTECTION/Resources/SP‐

Discussion‐papers/Labor‐Market‐DP/0922.pdf (accessed on October 29,

2011).

Johanson, Richard, and Arvil Van Adams. 2004. Skills Development in Sub‐Saharan

Africa. World Bank Regional and Sectoral Studies. Washington, DC: World

Bank.

Karmel, Tom. 2011. The Challenge of Measurement: Statistics for Planning Human

Resource Development. Adelaide: National Centre for Vocational Education

Research. http://www.ncver.edu.au/research/core/2418.pdf (accessed on

October 29, 2011).

Kiley, Margaret, and Robert Cannon. 2000. Leap into Lifelong Learning. Adelaide:

Centre for Learning and Professional Development, University of Adelaide.

King, Kenneth, and Robert Palmer. 2010. “Planning for Technical and Vocational

Skills Development.” Fundamentals of Educational Planning No. 94.

UNESCO: International Institute for Educational Planning.

Korea Research Institute for Vocational Education and Training (KRIVET). 2007.

Human Resources Development Indicators in Korea, 2007. Seoul: KRIVET.

Kuruvilla, S., C. L. Erickson, and A. Hwang. 2002. “An Assessment of the

Singapore Development System: Does It Constitute a Viable Model for

Other Developing Countries?” World Development 30(8): 1461–1476.

Law, Song Seng. 2008. “Vocational Technical Education and Economic

Development: The Singapore Experience.” In Toward a Better Future.

Education and Training for Economic Development in Singapore since 1965, eds.

Lee Sing Kong, Goh Chor Boon, Birger Fredriksen, and Tan Jee Peng.

Washington, DC: World Bank and National Institute of Education,

Singapore.

Lee, Young‐Hyun. 2009. “Vocational Education and Training in the Process of

Industrialization.” Understanding Korean Educational Policy, Vol. 5.

Educational Development Institute, Seoul. Korean. http://eng.kedi.re.kr/

khome/eng/education/educationSeries.do# (accessed on October 30, 2011).

Levy, Santiago. 2008. Good Intentions, Bad Outcomes: Social Policy, Informality, and

Economic Growth in Mexico. Washington, DC: Brookings Institution Press.


Levy, Frank, and Richard J. Murnane. 2005. The New Division of Labor: How

Computers Are Creating the Next Job Market. Princeton, NJ: Princeton

University Press.

Lindburg, Lori. 2009. Replicating Success: Innovative Collaborations between the

Biotech Industry and Education. San Francisco: Bay Bio Institute, City College

of San Francisco. http://www.bio‐link.org/home/sites/files/replicating

success_finalscreen.pdf (accessed on October 30, 2011).

MacKenzie, Jeanne, and Rose‐Anne Polvere. 2009. “TVET Glossary: Some Key

Terms.” In International Handbook of Education for the Changing World of

Work, ed. R. Maclean and D. Wilson. Dordrecht, Netherlands: Springer

Science + Business Media B.V.

Mourshed, Mona, Chinezi Chijioke, and Michael Barber. 2010. How the World’s

Most Improved School Systems Keep Getting Better. McKinsey and Company.

http://mckinseyonsociety.com/downloads/reports/Education/How‐the‐

Worlds‐Most‐Improved‐School‐Systems‐Keep‐Getting‐Better_Download‐

version_Final.pdf (accessed on October 30, 2011).

Mulgan, Geoff, and David Albury. 2003. Innovation in the Public Sector. London:

Strategy Unit, Cabinet Office.

O’Hare, Daniel. 2008. “Education in Ireland: Evolution of Economic and

Education Policies since the Early 1990s.” In An African Exploration of the

East Asian Education Experience, ed. Birger Fredriksen and Tan Jee Peng.

Washington, DC: World Bank. http://www.adeanet.org/adeaPortal/

adea/Biennale%202008/Documentation/Papers%20for%20presentation/01.%

20Session%201/Final%20PDF%20documents/Session%201%20Doc%204%20

World%20Bank%20ENG.pdf (accessed on October 30, 2011).

OʹLawrence, Henry. 2007. “A Case Study Report of a Comprehensive

Benchmarking Review of the Status of Vocational Education Students in

California: Building a New Relationship between Community Colleges

and California State University System.” Journal of Career and Technical

Education 23(1): 85–96. http://scholar.lib.vt.edu/ejournals/JCTE/v23n1/pdf/

olawrence.pdf (accessed on October 30, 2011).

Organization for Economic Cooperation and Development (OECD). 2010a.

Learning for Jobs: Synthesis Report of the OECD Reviews of Vocational Education

and Training. Paris: OECD http://www.mskills.com.au/DownloadManager/

downloads/Learning%20for%20jobs%20report.pdf (accessed on October 30,

2011).

———. 2010b. PISA 2009 Results: What Students Know and Can Do: Student

Performance in Reading, Mathematics and Science. Vol. 1. Paris: OECD.

http://www.oecd.org/dataoecd/10/61/48852548.pdf (accessed on October 30,

2011).


Oster, Emily, and M. Bryce Millett. 2010. “Do Call Centers Promote School

Enrollment? Evidence from India.” NBER Working Paper 15922, National

Bureau of Economic Research, Washington, DC.

Schleicher, Andreas. 2008. “PIAAC: A New Strategy for Assessing Adult

Competencies.” International Review of Education 54(5‐6): 627‐650.

Singapore Accreditation Council. 2008. “Frequently Asked Questions: What Are

the Benefits of Being Accredited?” http://www.sac‐accreditation.gov.sg/

accreditation.asp#qn1 (accessed on June 17, 2011).

Singh, Chandandeep, and Kuldeep Sareen. 2006. “Effectiveness of ISO 9000

Standards in Indian Educational Institutions: A Survey.” International

Journal of Services Technology and Management 7(4): 403–415.

Skills Victoria. 2011. “Victorian Government Response to Skills Australia

Consultation: Creating a Future Direction for Australian Vocational

Education and Training.” Mimeo. Skills Victoria, Department of Education

and Early Childhood Development, Melbourne. http://www.eduweb.

vic.gov.au/edulibrary/public/commrel/about/skills/victoriangovernmentres

ponsetoskillsaustraliaconsultation.doc (accessed on October 30, 2011).

Sondergaard, Lars, and Mamta Murthi. 2012. Skills, Not Just Diplomas: Managing

Education for Results in Eastern Europe and Central Asia. Directions in

Development Series. Washington, DC: World Bank.

Spar, Debora. 1998. “Attracting High Technology Investment: Intel’s Costa Rican

Plant.” Foreign Investment Advisory Service. Occasional Paper 11,

International Finance Corporation and World Bank.

Sudan, Randeep, Seth Ayers, Philippe Dongier, Arturo Muente‐Kunigami, and

Christine Zhen‐Wei Qiang. 2010. The Global Opportunity in IT‐Based Services.

Assessing and Enhancing Country Competitiveness. Washington, DC: World

Bank. http://www.infodev.org/en/Document.875.html (accessed on October

30, 2011).

Tan, Hong. 2001. “Malaysia’s Human Resource Development Fund: An

Evaluation of Its Effects on Training and Productivity.” Mimeo.

Washington, DC: World Bank. http://info.worldbank.org/etools/docs/

library/206512/Malaysia%20Human%20Resources%20Development%20Fu

nd%20English.pdf (accessed on October 30, 2011).

Tan, Hong, Gladys Lopez‐Acevedo, M. Sanchez, and M. Tinajero. 2004.

Evaluating SME Programs in Mexico Using Panel Firm Data. Washington, DC:

World Bank Institute, World Bank.


Tan, Jee‐Peng and Joy Yoo‐Jeung Nam. 2012. “Pre‐Employment Technical and

Vocational Education and Training: Fostering Relevance, Effectiveness, and

Efficiency.“ In The Right Skills for the Job? Rethinking Training Policies for

Workers, ed. Rita Almeida, Jere Behrman, and David Robalino. Washington,

DC: Human Development Perspectives, World Bank

http://elibrary.worldbank.org/content/book/9780821387146 (accessed on

August 13, 2012).

United Kingdom Commission for Employment and Skills (UKCES). 2009.

Ambition 2020: World Class Skills and Jobs for the UK. 2009 Report.

London: UKCES. http://www.ukces.org.uk/assets/bispartners/ukces/docs/

publications/ambition‐2020‐the‐2009‐report.pdf (accessed on November 1,

2011).

United Nations Educational, Scientific and Cultural Organization (UNESCO).

2002. “Best Practices” in Technical and Vocational Education: China, the People’s

Democratic Republic of Korea and Mongolia. Beijing: UNESCO.

http://unesdoc.unesco.org/images/0012/001257/125705eo.pdf (accessed on

November 1, 2011).

United Nations Population Division of the Department of Economic and Social

Affairs (UN). 2009. World Population Prospects: the 2008 Revision. New York:

United Nations.

Vlăsceanu, Lazăr, Laura Grünberg, and Dan Pârlea. 2007. Quality Assurance and

Accreditation: A Glossary of Basic Terms and Definitions. Revised and updated

edition. Bucharest: UNESCO‐CEPES. http://www.cepes.ro/publications/

pdf/Glossary_2nd.pdf (accessed on November 1, 2011).

Wang, Yidan. 2012. Education in a Changing World: Flexibility, Skills and

Employability. Washington, DC: Human Development Network, World

Bank.

Wolf, Artjom, and Andrea Erdle. 2009. Key Aspects of Economics of Technical and

Vocational Education and Training (TEVET): Lessons Learned and Gaps to be Filled.

Eschborn: Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ).

http://www2.gtz.de/dokumente/bib‐2009/gtz2009‐0390en‐economics‐tvet.pdf


Woods, James F., and Christopher J. O’Leary. 2007. “Principles of Labor Market

Information.” Employment Research Newsletter 14(2). W.E. Upjohn Institute

for Employment Research, Kalamazoo, MI.

World Bank. 2010a. Stepping Up Skills for More Jobs and Higher Productivity.

Washington, DC: Human Development Network, World Bank.

http://siteresources.worldbank.org/EXTHDOFFICE/Resources/5485726‐

1281723119684/Stepping_up_skills.pdf (accessed on November 1, 2011).


———. 2010b. “Sustaining Educational and Economic Momentum in Africa.”

Africa Human Development Series, World Bank Working Paper Series 195,

World Bank, Washington, DC. http://siteresources.worldbank.org/

EDUCATION/Resources/278200‐1099079877269/Sustaining_edu_econ_

momentum_Africa.pdf (accessed on November 1, 2011).

——. 2011. Learning for All. Investing in People’s Knowledge and Skills to Promote

Development. World Bank Group Education Strategy 2020. Washington, DC:

World Bank. http://siteresources.worldbank.org/EDUCATION/Resources/

ESSU/Education_Strategy_4_12_2011.pdf (accessed on November 9, 2011).

——. 2012. “SABER Overview Paper: The What, Why, and How of the Systems

Approach for Better Education Results.” Review draft. Education

Department, Human Development Network, World Bank, Washington,

DC.

World Bank/Multilateral Investment Guarantee Agency (MIGA). 2006. The Impact

of Intel in Costa Rica. Nine Years After the Decision to Invest. Investing in

Development Series. Washington, DC: MIGA. http://siteresources.

worldbank.org/EXTEXPCOMNET/Resources/2463593‐1213887855468/44_

The_impact_of_Intel_in_Costa_Rica.pdf (accessed on November 1, 2011).

Yusuf, Shahid, and Kaoru Nabeshima. 2010. Changing the Industrial Geography in

Asia: The Impact of China and India. Washington, DC: World Bank.