performance evaluation report job creation and skills

IN.496-20 16 December 2020

Performance Evaluation Report Job Creation and Skills Development in the Philippines (TA 8164, 8334, 8335, and 8984) The attached report is circulated at the request of the Director General, Independent Evaluation Department. This report is also being made publicly available. For Inquiries: Walter Kolkma, Independent Evaluation Department (Ext. 4189) Eungji Kim, Independent Evaluation Department (Ext. 1297)

Performance Evaluation

Report

EvaluationIndependent

Raising development impact through evaluation

Job Creation and Skills Development in the Philippines

Technical Assistance Performance Evaluation Report December 2020

Job Creation and Skills Development in the Philippines This document is being disclosed to the public in accordance with ADB’s Access to Information Policy.

Reference Number: Technical Assistance Numbers: 8164, 8334, 8335, and 8984 Independent Evaluation: TE-070

NOTES

(i) The fiscal year of the Government of the Philippines ends on 31 December. (ii) In this report, “$” refers to United States dollars. (iii) For an explanation of rating descriptions used in Asian Development Bank evaluation

reports, see Asian Development Bank. 2016. Guidelines for the Evaluation of Public Sector Operations. Manila.

The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of the management of IED, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the IED does not intend to make any judgment as to the legal or other status of any territory or area.

Director General

Marvin Taylor-Dormond, Independent Evaluation Department (IED) Deputy Director General Veronique N. Salze-Lozac'h, IED Director Walter Kolkma, Thematic and Country Division, IED

Team leader Eungji Kim, Senior Evaluation Specialist, IED

Team member Michelle Angieline Dantayana, Evaluation Assistant, IED

Abbreviations ADB – Asian Development Bank AIM – Asian Institute of Management ARTA – Anti-Red Tape Authority ASEAN – Association of Southeast Asian Nations BLE – Bureau of Local Employment CPS – country partnership strategy DMF – design and monitoring framework DOF – Department of Finance DOLE – Department of Labor and Employment DOT – Department of Tourism

eSMP – electronic service management program FGD – focus group discussion HEI – higher education institution IBPAP – IT & Business Process Association of the Philippines ICT – information and communication technology ILS – Institute for Labor Studies ISGS – industry skills grant scheme IT – information technology IT-BPO – information technology and business process outsourcing JSP – JobStart Philippines LGU – local government unit LST – life skills training PDP – Philippine Development Plan PEIS – Public Employment Information System

PESO – Public Employment Service Office RIA – regulatory impact assessment RIS – regulatory impact statement SMEs – small and medium-sized enterprises TA – technical assistance TCR – technical assistance completion report TISP – Tourism Industry Skills Program TPER – technical assistance performance evaluation report UPOU – University of the Philippine Open University

Contents Acknowledgments ix

Basic Data xi Executive Summary xiii Map ix

Chapter 1: Introduction 1

A. Evaluation Purpose and Process 1

B. Technical Assistance Objectives 2

C. Technical Assistance Completion Reports 3

Chapter 2: Design and Implementation 5

A. Rationale 5

D. Cost and Financing 8

E. Executing Arrangements 8

F. Consultants 9

G. Scheduling 9

H. Design Changes 10

Chapter 3: Performance Assessment 11

A. Relevance 11

B. Effectiveness 17

C. Efficiency 21

D. Sustainability 23

Chapter 4: Other Assessments 27

A. Impact 27

B. ADB and Executing Agency Performance 30

Chapter 5: Overall Assessment, Issues, Lessons, and Follow-Up Actions 31

A. Overall Assessment 31

B. Issues 32

C. Lessons 33

D. Follow-Up Actions 34

Appendixes 36

1. Focus Group Discussions 37

2. Design and Monitoring Frameworks of Technical Assistance Projects 38

3. Summary of Accredited Hotels and Resorts 45

4. List of Regulatory Impact Assessment Statements 46

5. DOLE RIA Capacity Building Activities 48

6. Jobstart Summary Performance 49

7. List of Participating PESOs in Jobstart 50

8. Tourism Industry Employment and Share in Gross Domestic Product 51

9. Number of Inbound Visitors Arrivals, 2015–2019 52

10. Statistical Analysis Result from Deparment Of Labor and Employment’s JobStart Philippines Data 53

11. Lessons from the Job Creation and Skills Development Technical Assistance Projects 54

Acknowledgments This evaluation was led by a team from the Independent Evaluation Department (IED) of the Asian Development Bank. The final report was prepared by team leader Eungji Kim (Senior Evaluation Specialist); supported by Michelle Angieline Dantayana (Evaluation Assistant); and consultants Clark Dingcong and Imelda Baleta. Eungji Kim led the evaluation mission in the Philippines during February 2020. The team received technical feedback from Pablo Acosta, World Bank human development expert; and comments from Ari Perdana, Evaluation Specialist, IED. The report was prepared under the guidance of Marvin Taylor-Dormond, Director General; and Walter Kolkma, Director, Thematic and Country Division. The team would like to thank stakeholders, beneficiaries, government officials of the Department of Tourism and Department of Labor and Employment, development partners from Canada and the Republic of Korea who cofinanced some of the technical assistance, and staff from the Asian Development Bank (ADB) Philippine Country Office and ADB headquarters for their time and sharing of their opinions. The team would also like to thank the local government officials met in Bohol, Cavite, Cebu, Davao, Palawan, Pampanga, and National Capital Region, who made invaluable contributions to this evaluation through their participation in focus group discussions and surveys. IED retains full responsibility for this report.

Basic Data

Key Information TA 8164 TA 8334 TA 8335 TA 8984

Title

Strengthening Knowledge-Based Economic and Social Development

Improving Competitiveness in Tourism

Employment Facilitation for Inclusive Growth

Support for the Nationwide Rollout of JobStart Philippines

Implementation

2012–2016 2013–2017 2013–2017 2015–2020

Budget approved ($) 500,000 7,116,900 5,661,900 8,798,500 ADB 0 0 70,000 1,300,000 Cofinancing ($)

500,000 7,116,900 5,591,900 7,498,500

Budget disbursed ($)

385,012 5,915,142 4,789,220 5,200,909 (ongoing)

Cofinancing agency (Country)

e-Asia and Knowledge Partnership Fund (Republic of Korea)

Canadian International Development Agency (Canada)



Executing agency

Department of Finance

Department of Tourism

Department of Labor and Employment

Department of Labor and Employment

Milestones

President’s or Board approval

14 Sep 2012 5 Mar 2013 5 Mar 2013 30 Oct 2015

Signing of TA agreement

18 Oct 2012 15 Apr 2013 16 May 2013 21 Jan 2016

Fielding of consultants

19 Sep 2013 07 Nov 2013 16 May 2013 ...

TA completion Expected 15 Oct 2015 28 Feb 2017 28 Feb 2017 31 Dec 2020 Actual

15 Oct 2016 31 Aug 2017 31 Aug 2017 ...

Account closing Expected 15 Oct 2015 28 Feb 2017 28 Feb 2017 31 Dec 2020 Actual

9 Jan 2017 16 Aug 2017 18 Dec 2017 ...

Circulation of TA completion report

July 2018 May 2018 April 2018 ...

ADB = Asian Development Bank, TA = technical assistance. Source: Asian Development Bank.

Executive Summary

This evaluation examined support from Asian Development Bank (ADB) to the Government of the Philippines for interventions in skills development and job creation. It assessed four technical assistance (TA) projects implemented from 2013 onwards: (i) Strengthening Knowledge-Based Economic and Social Development, (ii) Improving Competitiveness in Tourism, (iii) Employment Facilitation for Inclusive Growth, and (iv) Support for Nationwide Rollout of Jobstart Philippines. These projects were selected to examine how ADB used TA projects to support skills development and employment in response to the need for better opportunities for Filipinos, particularly the youth, as part of an effort to reduce poverty and promote inclusive growth.

The evaluation assessed the first TA project less than successful, while ADB’s TA projects on tourism and employment facilitation were found to be successful. The last TA project, on Jobstart Rollout, was not rated as it is still ongoing. The preliminary development impacts and performance of both ADB and the borrower were deemed satisfactory, except that the first TA was less than satisfactory. Despite inconsistent performance and outcome of the TA projects, they all contributed to demonstrating the impact of giving employers a greater role in the planning stage of the skills training. Overall, this evaluation shows that ADB’s work on skills development is significant and could play a vital role in shaping the government’s sectoral policy, regardless of the amounts spent. Despite varied levels of performance and outcome, the four TA projects demonstrated the benefits of improving the capacity of government agencies mandated to do skills development. Expanding Public Private Partnerships in skills development is crucial—employers need to play a clear role in helping with the skills training of their employees or trainees. Successful innovations need to be pursued and refined. Continuity in ADB’s effort in developing skills across different fields of intervention is important, in order to be contribute to government policy in this area.

The evaluation cites three issues to consider with respect to improving performance of skills development and job creation TA projects in the Philippines: (i) procedural inefficiencies persist in ADB’s TA implementation, as not enough has been done to streamline lending to local government units; (ii) appropriate certification of training courses offered by the government was not incorporated into the project design of the Tourism TA and the At-Risk Youth TA, affecting the TA projects’ effectiveness and sustainability; and (iii) inadequate monitoring of key indicators during implementation made it difficult for performance measurement, thereby undermining the process of learning and improving the programs while undertaking them. In following up, ADB should ensure that (i) lessons from evaluations feed into scale-up efforts in similar sectors or regions, and (ii) institutionalization of the life skills training module is promoted to maximize the impact and sustainability of the JobStart TA projects.

Background

The Philippines is among the fastest growing economies in Southeast Asia and was on track to achieving its target of becoming an upper middle-income country by 2022 before the coronavirus disease (COVID-19) pandemic struck in 2020. While national poverty incidence fell from 25.2% of the population in 2012 to 21.6% in 2015, the poverty rate nevertheless translated to almost 22 million poor people and is considered high for a middle-income country. Furthermore, household income inequality in the Philippines is among the

most extreme in Southeast Asia. The prevalent and wide income disparities across the regions indicate that the benefits of economic growth have been distributed unevenly. To reduce this inequality, the Asian Development Bank (ADB) utilized capacity development technical assistance (TA) projects to support job creation and sustainable economic growth through education. The following TA projects were assessed: (i) Strengthening Knowledge-Based Economic and Social Development

xiv Job Creation and Skills Development in the Philippines

(information technology and business process outsourcing [IT-BPO] TA), (ii) Improving Competitiveness in Tourism (Tourism TA), (iii) Employment Facilitation for Inclusive Growth (At-Risk Youth TA), and (iv) Support for Nationwide Rollout of JobStart Philippines (JobStart Rollout TA). These TA projects were selected to examine how ADB has recently used TA to support skills development and employment in response to the development need for providing better employment for Filipinos, particularly the youth.

Evaluation Purpose and Process This TA performance evaluation assessed the TA projects for relevance, effectiveness, efficiency, and sustainability to arrive at overall success ratings. The evaluation also assessed development impacts and the performance of ADB, the borrower, and the executing agency. The evaluation was based on desk review of project documents, consultations with the project teams at ADB headquarters and in the Philippine Country Office, development partners, and government executing agencies. Interviews, field visits, and focus group discussions were conducted in 13 locations in the country from December 2019 to

February 2020 to gain insights from implementing agencies and project beneficiaries. Also, a brief survey was conducted in parallel to gather additional assessments.

Expected Impacts, Outcomes, and Outputs The Strengthening Knowledge-Based Economic and Social Development TA was formulated to address skills shortages in the high-growth industry of IT-BPO. It aimed to develop a replicable and sustainable system of information and

communication technology knowledge hubs that would improve teaching and learning of IT-BPO industry-designed curricula in higher education institutions. The Tourism TA was intended to raise the level of competitiveness in tourism by supporting the implementation of a regulatory impact assessment (RIA) program, a hotel and resort quality assurance and accreditation system, and a tourism industry skills program. The At-Risk Youth TA was designed to boost job creation by building capacity in labor market policy

development, improving the delivery of employment services for at-risk youths, and supporting a pilot for JobStart (providing employment assistance to young job-seekers). The JobStart Rollout TA sought to build on the results and lessons learned from the JobStart pilot and expand it nationwide.

Assessment of Performance IT-BPO TA. Despite being well-aligned with priorities of both the government and ADB, the TA’s lack of clear management and reporting structure aggravated the loosely integrated

implementing agencies and resulted in the mid-implementation departure of one of these, the Asian Institute of Management, thus contributing to 12 months of delay. The ambitious project outcomes could not be achieved within the given time frame. A lack of information technology (IT) infrastructure in rural areas was not taken into consideration, and this hampered deployment of IT-BPO industry-designed curricula in higher educational institutions outside the National Capital Region. Though the TA’s ambitious goals included forming linkages between industry and academia to provide relevant and up-to-date training, no single executing or implementing

agency had the capacity or mandate to link e-learning and job integration. The innovation required by the TA design simply outpaced the realities of what was possible at the time in the Philippines. The IT-BPO TA was assessed less than successful, as the prior design shortcomings made it less than relevant and the TA’s underperformance in terms of both output and outcome made it less than effective. The project’s completion delay and abrupt departure of an implementing agency are among the factors making the TA less than efficient. It also was less than sustainable due to

lack of sustained outcome from the e-learning modules and near total loss of institutional knowledge within both executing and implementing agencies. Tourism TA. The project’s innovation was dependent upon public–private partnership (PPP) between the Department of Tourism (DOT), employers, and tourism sector industry groups. The project successfully updated the hotel and resort quality assurance and accreditation system

Executive Summary xv

and embedded the RIA component within DOT’s policy framework. The recently formed Anti-Red Tape Authority has borrowed heavily from that framework to develop and incorporate RIAs at a national level. Regarding skills development, the TA laid the foundation for the Office of Industry Manpower Development within the DOT and was instrumental in DOT’s allocation of PHP100 million per year to support national rollout of a follow-up training program. The Tourism TA was assessed successful, despite poor selection of key performance indicators and uneven monitoring. The TA and its innovations were found to be relevant to the policies and needs of the tourism sector. Outputs and outcomes were mostly achieved in a timely manner and within budget, making the TA both effective and efficient. In view of the sustained policy and institutional framework within DOT and its continuing support for tourism sector capacity development through its own follow-up program, the evaluation assessed the TA most likely sustainable. At-Risk Youth TA. Besides supporting the capacity development of both Department of Labor and Employment (DOLE) and Public Employment

Service Office (PESO) officers in employment services and RIA, the At-Risk Youth TA’s most successful and visible component was the JobStart Philippines (JSP) pilot in 4 PESOs, which was expanded to 10 additional sites by project completion. In 2016, the central government fully committed to JSP by enacting the JSP Bill and allocating yearly funding for a nationwide JSP rollout. The At-Risk Youth TA was assessed successful, despite poor selection of key performance indicators and uneven monitoring. The TA project was relevant to the policies and needs of the labor

sector. Outputs and outcomes were mostly achieved, making the TA effective. The implementation did not suffer significant delays and produced most of the deliverables within budget, making the TA efficient. Considering the impact of the pilot JSP, the ongoing follow-up TA, and enactment of the JSP Bill, the TA was deemed likely sustainable. JobStart Rollout TA. Based on the successful pilot in the At-Risk Youth TA, the JobStart Rollout TA

introduced an additional innovation: offering different pathways for JobStart trainees and JobStarters, as it had been observed during the pilot that not all youths needed to go through the full employment facilitation cycle. This innovation made the design more flexible and responsive to the varying capacity and readiness of JobStarters. Although the TA design was deemed relevant, it was noted that both DOLE and PESOs were sharing a greater burden of funding the technical training and on-the-job training for the JobStarters. Although it had been meant to foster local ownership and long-term sustainability, it resulted in the TA being vulnerable to unpredictable and inconsistent funding, which delayed implementation and even curtailed program participation in some local government units (LGUs). So far the TA was deemed effective, but the evaluation noted that more effort is needed to streamline administrative procedures while easing requirements for beneficiaries and implementing partners to incentivize their participation in JSP.

Overall Assessment Despite showing varied levels of performance and

outcome, each of the four TA projects contributed to demonstrating (i) the need to identify or improve the capacity of government agencies in the developing member countries (DMCs), especially those that are mandated to do skills development; (ii) the importance and impact of PPP in skills development—specifically by giving employers a greater role in planning and coordinating the skills training of their employees or trainees; and (iii) the need for continued refinement of a successful innovation in order to maximize impact while minimizing risk. This evaluation also showed the significance of ADB’s work on skills development in a country, how

continuity in the effort across different fields of intervention is important, and how the interventions can play a vital role in shaping the government’s policy in this area.

Issues, Lessons, and Follow-up Actions Issues Issue 1. Procedural inefficiencies persist in ADB’s TA implementation, as not enough has been done

xvi Job Creation and Skills Development in the Philippines

to streamline lending to LGUs. The Philippines Country Partnership Strategy Final Review Validation, 2011−2016 noted that ADB should explore new ways to resolve the existing challenges in effectively delivering traditional project loans, especially when these are channeled through multiple government agencies or LGUs. It should be noted that most beneficiaries of the related TA attributed high dropout rates among both trainees and employers to delays in training caused by slow disbursement of funds from the central government agencies. ADB needs to think beyond simply providing support for nationally managed TA projects and start preparing the groundwork for streamlined, direct financing mechanisms for LGUs. Issue 2. Appropriate certification of training courses was not incorporated into the project design of the Tourism TA and At-Risk Youth TA, affecting the TA projects’ effectiveness and sustainability. Despite efforts to ease the burden for employers in coming up with the training plan for both JSP and the industry skills grant scheme, the learning curve for ADB’s TA projects proved to be too steep for small and medium-sized enterprises in rural areas without standard templates or technical support to fall back on.

Also, without standardized training courses, JobStarters or employees ran the risk of limiting themselves to a very narrow slice of the job market. By cooperating with the Technical Education and Skills Development Authority, the government agency with the mandate to provide technical training, the TA project could have maximized the effectiveness and sustainability of the technical skills training support. Issue 3. Inadequate monitoring of key indicators during implementation made it difficult for performance measurement, thereby undermining the processes of learning and improving the

programs while undertaking them. TA completion reports (TCRs) across the various projects were deficient in reporting the performance indicators as identified in the design and monitoring frameworks, and some even failed to report detailed ratings for the TA projects. This could reflect an actual lack of monitoring of key performance indicators during implementation. In preparation of future TCRs, reporting of these indicators and ratings should be required to enable objective assessment of project

performance. Furthermore, the selection of performance indicators for future TA projects should consider feasibility of the target as well as practicality of monitoring to maximize the accuracy of performance measurement and the chance for learning through evaluation. Lessons Lesson 1. The innovativeness of TA cannot make up for lack of preparedness or capacity of the DMC agencies involved. In ADB’s Country Partnership Strategy: Philippines, 2018–2023, ADB planned to pilot test an enterprise-led learning network project, which is exactly what the less-than-successful IT-BPO TA had set out to do. ADB should avoid repeating the same mistakes by taking the following precautions: (i) verify soundness of the government’s policy framework and long-term strategy in any pertinent nascent sector, (ii) carefully examine the feasibility of PPP arrangements in education and labor market programs, and (iii) make sure that both private and public sectors are ready to cooperate in delivering concrete and verifiable outcomes. Lesson 2. Clear implementation arrangements

with executing and implementing agencies are essential for the success of skills development projects. Implementation arrangements should have clear institutional roles and responsibilities, and the executing agency should have a clear and strong mandate for monitoring and enforcing these roles. Strong leadership by the executing agency, supported by technically capable implementing agencies, is vital for a project’s successful implementation. This is illustrated by the less than successful implementation of the IT-BPO TA project under the leadership of the Department of Finance.

Lesson 3. Poor identification of relevant stakeholders can limit ADB’s engagement with them, critically affecting the project’s effectiveness, participation and support of local businesses, and access to the domestic job market. For the IT-BPO TA, there was no government agency with a clear mandate for the sector and too many implementing agencies were brought in to fill the vacuum with their various goals. On the other hand, the other three TA projects did not identify sufficient partners

Executive Summary xvii

outside DOLE, DOT, and LGUs. Thus, they did not avail themselves of networks that could have helped raise public awareness on the innovations or elevated the quality of technical training.

Lesson 4. Overly ambitious project design can cause the implementation of interventions at inappropriate levels, resulting in suboptimal outcome for the project. Designing the TA projects to initiate and embed RIA within the existing DOT and DOLE frameworks during each project’s timespan was quite an ambitious undertaking, especially considering that RIAs are administratively and technically challenging. Also, most of the RIAs formulated by DOLE and DOT were not fully implemented, because implementation of the RIAs’ recommendations often was beyond the mandate of each department. By contrast, the recommendations from RIAs at LGU level were mostly implemented because they required passage of local ordinances falling within the purview of the LGUs.

Lesson 5. Processes need to be simplified to minimize inefficiencies in scaling up pilots. Documentation and reporting requirements for training participation were viewed as burdensome and presented a barrier to entering ADB’s TA projects. This discouraged potential employers and caused them to withdraw their interest from the program. This was especially true for small and medium-sized enterprises employers who lacked dedicated personnel in their finance and human resource departments. Although executing agencies and ADB might have been able to support and push a few dozen beneficiaries through for the pilot, one cannot expect large-scale rollout to succeed without first streamlining procedures and requirements for the beneficiaries and implementing partners.

Lesson 6. Concerted effort is needed to raise public awareness concerning key innovations in skills development projects. Advocacy and information dissemination campaigns should have been included in the TA project design.

Project officers should support the local PESOs in (i) diversifying promotion channels, (ii) seeking out and proactively communicating with unemployed, at-risk youths, (iii) actively networking with local employers and promoting participation in JSP, and (iv) sharing and cross-referencing potential candidates for the various skills development and employment support programs within the PESO. Follow-Up Actions

Follow-Up Action 1. Lessons from previous evaluations should feed into scale-up efforts in similar sectors or regions. In large skill development TA projects that aim to scale up on certain innovations, the findings from earlier pilots of said innovation should be fed into the design of the scale-up. The JobStart Rollout TA was already in mid-implementation when results of the impact evaluation from the At-Risk Youth TA were released. Consequently, these were not effectively fed back into the scale-up. Follow-Up Action 2. Institutionalization of the life skills training (LST) module should be promoted to maximize the impact and sustainability of the JobStart TA projects. LST was repeatedly mentioned by various stakeholders as being the most effective and innovative component of JSP. Although technical training and on-the-job training are not without merit, JSP is one of many programs in PESOs that offer such technical training. Inasmuch as LST is a unique innovation introduced within the JSP, however, it runs the highest risk of being discontinued beyond the TA project cycle. The risk can be mitigated if the LST module is adopted by government agencies, such as DOLE, and carried on by developing a localized LST module, spearheading the regular update and revision of LST training manuals, hosting and implementing the training of trainers programs,

ensuring quality control of LST programs, and acquiring funding to roll out the LST at a truly nationwide scale.

CHAPTER 1

Introduction

1. Since 2012, the Asian Development Bank (ADB) has approved four technical assistance (TA) projects in job creation and skills development with the common goal of providing better employment opportunities for Filipino youth as part of an overall effort to reduce poverty and economic vulnerability of the non-poor to external shock. This Chapter outlines the purpose and process of the independent evaluation and summarizes each TA project’s expected impact, outcomes, and outputs.

A. Evaluation Purpose and Process

1. Purpose. This TA performance evaluation report (TPER) examines ADB’s support to the Philippine government’s intervention for skills development and job creation. It is intended to contribute to learning and accountability, as well as to improving project design and effectiveness in country and sector operations within ADB. The evaluation assessed the projects (i) Strengthening Knowledge-Based Economic and Social Development, 1 (ii) Improving Competitiveness in Tourism, 2 (iii) Employment Facilitation for Inclusive Growth, 3 and (iv) Support for Nationwide Rollout of JobStart Philippines (ongoing).4 These TA projects were selected to examine how ADB recently has used such projects to support skills development and employment in responding to the development need of providing better employment for Filipinos, particularly the youth.

2. Objective. The overall objective of the evaluation was to assess how ADB’s support contributed to enhancing the country’s economic capacity at national and provincial levels through skills development and job creation projects. Specifically, it examined progress on related issues, including the extent and sustainability of outcomes, ownership, and institutional capacity development for accrued impact and outputs. The evaluation was based on the overarching question identified in the evaluation approach paper: What are the limits and potential of a series of capacity development TA projects in developing the necessary skills and creating jobs in the Philippines? 3. Scope. The TPER assesses the TA formulation, design, implementation, achievements, sustainability, and development impacts. Specifically, it (i) assesses the TA relevance, effectiveness, and efficiency in achieving its objectives and outputs, sustainability of its outcomes and outputs, institutional development, and socioeconomic impacts; (ii) assesses the performance of ADB, executing and implementing agencies, and other participating institutions; and (iii) identifies issues and lessons for future ADB operations in this area. 4. Methodology. The TPER evaluates the TA in accordance with guidelines for evaluating public sector operations using the evaluation criteria of relevance, effectiveness, efficiency, and sustainability.5 The TPER made use of primary and secondary data and included (i) a desk review of relevant information

1 ADB. 2012. Technical Assistance to the Republic of the Philippines for Strengthening Knowledge-Based Economic and Social

Development. Manila. 2 ADB. 2013. Technical Assistance to the Republic of the Philippines for Improving Competitiveness in Tourism. Manila. 3 ADB. 2013. Technical Assistance to the Republic of the Philippines for Employment Facilitation for Inclusive Growth. Manila. 4 ADB. 2015. Technical Assistance to the Republic of the Philippines for Support for the Nationwide Rollout of JobStart Philippines.

Manila. 5 Independent Evaluation Department (IED). 2016. Guidelines for the Evaluation of Public Sector Operations. Manila: ADB.

2 Job Creation and Skills Development in the Philippines

concerning the TA; (ii) consultations with the project team at ADB headquarters and in the Philippines Country Office, development partners, and other stakeholders; and (iii) consultations with government and implementing agencies and other beneficiaries. As part of the independent evaluation mission, field visits and focus group discussions (FGDs) were conducted in 13 locations in the Philippines from December 2019 to February 2020 to gather the views of implementing agencies and beneficiaries in the National Capital Region, Cavite, Pampanga, and as far as Bohol, Cebu, Davao, and Palawan. Forty-eight FGDs were conducted, with 214 participants providing their views and information on the TA projects (Appendix 1).

B. Technical Assistance Objectives 5. The four capacity development TA projects were formulated to support the government’s efforts toward achieving inclusive growth at a sustained high rate while contributing to mass employment and reducing poverty. Of the four TA projects, three were designed to support policy-based loans and one was stand-alone. 6. Strengthening Knowledge-Based Economic and Social Development (information technology and business process outsourcing [IT-BPO] TA). This TA was a stand-alone project developed to address skills shortages within high-growth industries in the Philippines. Even though IT-BPO was the fastest growing employment sector, the supply of high-quality labor had not kept up with demand. To help in addressing this gap, the TA sought to achieve a replicable and sustainable system of information and communication technology (ICT) knowledge hubs developed for improved teaching and learning of IT-BPO industry-designed curricula in higher education institutions (HEIs). It aimed to align higher education curricula with international standards and labor market demand while building on public and private initiatives. The TA was focused on helping Filipino youths while strengthening partnerships between industry and academic institutions. 7. Improving Competitiveness in Tourism (Tourism TA). This TA was attached to the policy loan on Increasing Competitiveness for Inclusive Growth Program. 6 Philippine tourism had lagged against neighbors such as Indonesia, Malaysia, and Thailand in terms of tourist arrivals and revenues. One reason was deemed to be that the skills of youth and graduates entering the industry did not match the skills required as a result of the industry’s standards being outdated and poorly implemented. In view of these needs, the TA was designed to improve competitiveness in tourism by increasing employment in the industry as one critical channel for inclusive, broad-based growth in the Philippines. To do so, the TA supported implementation of a regulatory impact assessment (RIA) program, a hotel and resort quality assurance and accreditation system, and a tourism industry skills program. 8. Employment Facilitation for Inclusive Growth (At-Risk Youth TA). This TA was also attached to the policy loan on Increasing Competitiveness for Inclusive Growth Program (footnote 6). The TA was envisaged to contribute to the employability of at-risk youths by improving the employment facilitation systems to mitigate the difficulty high school graduates and dropouts were experiencing when entering the labor market. It was designed to boost job creation by building capacity in labor market policy development; improving delivery of employment services for at-risk youths; 7 and supporting implementation of the pilot phase of JobStart, which assisted high school graduates seeking employment.

6 ADB. 2012. Report and Recommendation of the President to the Board of Directors: Policy-Based Loan to the Republic of the

Philippines for Subprogram 1: Increasing Competitiveness for Inclusive Growth Program. Manila; and ADB. 2014. Report and Recommendation of the President to the Board of Directors: Policy-Based Loan to the Republic of the Philippines for Subprogram 2: Increasing Competitiveness for Inclusive Growth Program. Manila.

7 At-risk youth are defined as those who are (i) Filipino citizens; (ii) 18–24 years of age at the time of registration; (iii) at least high school graduates; (iv) not employed, studying, or undergoing training at the time of registration; and (v) with less than 1 year of work experience. JobStart was initially called MyFirstJob during the pilot phase.

Introduction 3

9. Support for the Nationwide Rollout of JobStart Philippines (JobStart Rollout TA). This TA was designed to continue building on the results and lessons learned from the preceding pilot included in the At-Risk Youth TA. It supported the nationwide rollout of JobStart Philippines (JSP), which aimed to implement the program in 64 local government units (LGUs) and targeted more than 64,000 out-of-school youths. Twenty-four of these LGUs and 15,000 beneficiaries would be supported by the TA. The TA supported the policy-based loan for Facilitating Youth School-to-Work Transition Program and is currently ongoing.8 10. Table 1 summarizes the intended impact, outcome, and outputs of the ADB TA projects supporting job creation and skills development in the Philippines. Appendix 2 provides details of the design and monitoring framework (DMF) for each of the TA projects.

Table 1: Intended Impact, Outcomes, and Outputs of Technical Assistance Projects

Strengthening Knowledge-Based

Economic and Social Development

(TA 8164)

Improving Competitiveness in

Tourism (TA 8334)


(TA 8335)

Support for the Nationwide Rollout of JobStart Philippines

(TA 8984)

Imp

act

Improved alignment of higher education programs with needs of the economy (specifically the IT-BPO industry)

Increased employment in the tourism industry

Enhanced employability of at-risk youths

Enhanced employability of at-risk youths

Outc

om

e Establishment of replicable

and sustainable knowledge hubs for the instruction of IT-BPO industry-designed curricula in HEIs

Improved competitiveness of Philippine tourism

Improved capacity of PESOs to provide employment services to at-risk youths

Effective JSP rollout by DOLE and PESOs

Outp

ut

• Access to knowledge resources for teaching and learning IT-BPO industry-designed courses increased

• Higher education curricula and instruction enhanced

• Partnerships between industry and academic institutions strengthened

• Regulatory impact assessment program implemented

• Hotel and resort quality assurance and accreditation system implemented in four regions

• Tourism industry skills development program developed and piloted in four regions

• Capacity for labor policy analysis and formulation strengthened

• MyFirstJob (JSP) pilot project implemented

• Capacity development of JSP units at DOLE headquarters and regional DOLE offices strengthened

• Life skills training curriculum rolled out nationally

• Capacity of 24 PESOs in JSP implementation developed

DOLE = Department of Labor and Employment, HEI = higher education institution, IT-BPO = information technology and business process outsourcing, JSP = JobStart Philippines, PESO = Public Employment Service Office, TA = technical assistance. Source: Asian Development Bank.

C. Technical Assistance Completion Reports

11. Of the four TA projects, two were self-rated successful, one was self-rated less than successful, and the fourth is ongoing. Key findings from TA completion reports (TCRs) for the three completed projects are as follows:

(i) IT-BPO TA. The TCR rated the TA less than successful due to missed performance targets. During the independent evaluation, a lack of mid-implementation assessment on the relevance of DMF indicators, along with inadequate documentation on whether and how

8 ADB. 2017. Report and Recommendation of the President to the Board of Directors: Proposed Programmatic Approach and

Policy Loan for Subprogram 1 to the Republic of the Philippines on Facilitating Youth School-to-Work Transition Program. Manila.


these were achieved, made it difficult to verify the TA’s actual performance. The TCR stated that the TA was challenged by limited human resources and low student demand for business process outsourcing (BPO)-related courses. TA oversight and monitoring were evidently weak, as the consultants relied heavily on their understanding of activities and ADB relied heavily on the capacity of the consultants. Despite this, the TCR claimed that the TA contributed to online IT-BPO industry-developed course offerings and opened opportunities for strengthening partnerships between HEIs and industry. As such, the TCR assessed the TA’s outputs as likely sustainable because strong frameworks had been put in place to allow regular updating of courses and to solicit continued input from local industry partners.

(ii) Tourism TA. The TCR rated the TA successful overall, relevant due to its alignment with the priorities outlined in the National Tourism Development Plan, and also effective as results achieved under the respective outputs areas contributed to the outcome of improved competitiveness in the tourism sector. The TA’s support to the Department of Tourism (DOT) on its RIA program assisted in improving regulations that reduced regulatory compliance costs for tourism businesses. The implementation of a tourism enterprise skills survey helped DOT identify key skills deficiencies and provided it with a model to prioritize skills shortages through detailed labor market analysis in the sector. The DOT national accommodation standards were piloted in 141 establishments. The post-pilot review confirmed DOT standards to be robust and practical. It was rated efficient, as the TA adhered to timely implementation and achievement of intended outcomes. The TA funding was not fully utilized however, due to ring-fencing of funds to cover for foreign exchange risk. The TA was rated most likely sustainable, as the government of the Philippines took over funding of certain components of the program and thereby demonstrated its long-term commitment in the sector.9

(iii) At-Risk Youth TA. This TA was also rated successful by the TCR. It was relevant based on the TA’s alignment with the Department of Labor and Employment’s (DOLE) priorities, country development priorities, and ADB strategies and policies at time of formulation and completion, and effective as it achieved its performance indicators on support to DOLE on developing RIA, labor policy review, and increased use of labor market information. The pilot phase of JSP was successfully completed and expanded to cover 10 additional Public Employment Service Offices (PESOs). The significant progress achieved under the JSP pilot led to a DOLE agreement for a national rollout of the JSP. It was rated efficient, as changes made to program design and implementation were minor and in response to additional resources required for the planned expansion of JobStart. The TA was responsive to the needs of the government and the TA implementation unit played an active role in sustaining momentum for implementation. Nevertheless, the TA funding was not fully utilized due to ring-fencing of funds to cover for foreign exchange risk. The TA was rated most likely sustainable as evidenced by the government of the Philippines’ commitment to institutionalize the JSP in 2016 through enactment of the JSP Bill and securing government funding.

9 DOT’s campaign resulted in $2 million being allocated by Department of Budget Management for national rollout and was

critical in ensuring sustained investments in skills development within the tourism sector.

CHAPTER 2

Design and Implementation

A. Rationale 12. Background and context. The four TA projects were aimed at developing systems and capacities that would facilitate job placement in the IT-BPO industry, in the tourism industry, and specifically for “at-risk” youth. These projects were formulated at a time when the Philippine economy needed to achieve more inclusive growth, as it was experiencing average annual growth of 6.3%,10 and poverty incidence was declining (from 26.6% in 2006 to 21.6% in 2015).11 This growth had contributed to substantial decline in the unemployment rate from 11.8% in 2014 to 5.3% in 2018. Despite these improvements, the poverty rate translated into 22 million poor people, which was still considered quite high for a middle-income country,12 and wide income disparities were prevalent across the country. Some regions of the Visayas and Mindanao were registering poverty rates of more than 33%. 13 Furthermore, income inequality in the Philippines was greater than in its Southeast Asian counterparts, implying that the benefits of economic growth were unevenly distributed.14 The government was pursuing a policy of inclusive growth to address the failure of economic growth to benefit much of the country’s population.15

13. Project rationale. As part of The Philippine Development Plan (PDP), 2011−2016, the government identified three broad strategies aimed at achieving inclusive growth: (i) attaining high, sustained economic growth through a stable macroeconomic environment, rapid growth of industry, investments in infrastructure, and curbing corruption and enforcing the rule of law; (ii) providing equal access to development opportunities by investing in human capital, especially in education, health, and other basic social services, and by leveling the playing field through improving access to infrastructure, credit, land, technology, and other productive inputs; and (iii) formulating effective social safety nets to ensure both the protection and promotion of extremely vulnerable groups (Figure 1). Meanwhile, ADB’s partnership with the Philippines was based on the intersection of PDP priorities with Strategy 2020 and its key objective of helping the Philippines achieve high, inclusive, and sustainable growth. While focusing on infrastructure, environment, and education, ADB continued its approach of combining policy-based lending with capacity development to support broader governance reforms while gradually expanding sector-level policy and investment lending. The four TA projects aimed at supporting the PDP’s goal of “increased productivity and efficiency” by intensifying the culture of competitiveness and focusing interventions in key industry areas (such as IT-BPO and tourism) while being a part of ADB’s push to increase testing of innovative and knowledge solutions and drawing on ADB’s position to identify and

10 During 2010–2018, economic growth averaged 6.3% per annum, a substantial increase from the 4.5% average annual growth

recorded during 2000–2009. 11 Government of the Philippines. Philippine Statistics Authority. 2016. 2015 Official Poverty Statistics of the Philippines. Quezon

City. 12 ADB. 2018. Country Partnership Strategy: Philippines, 2018–2023 – High and Inclusive Growth. Manila. 13 ADB. 2009. Poverty in the Philippines: Causes, Constraints, and Opportunities. Manila. 14 According to World Bank Development Indicators, the Philippine Gini coefficient declined from 0.46 in 2012 to 0.44 in 2015.

Gini coefficients were nevertheless lower for Indonesia (0.39), Viet Nam (0.38), Cambodia (0.31), and the Lao People’s Democratic Republic (0.38).

15 Key outputs of the said policy are the Philippine Development Plan (PDP), 2011–2016 and PDP, 2017–2022. The former emphasized high growth that is sustained, massively creates jobs, and reduces poverty, while aiming to achieve these through massive investment in physical infrastructure, transparent and responsive governance, human resource development, and employment generation. In the succeeding PDP, 2017–2022, the government sought to lay a stronger foundation for more inclusive growth, a high-trust and resilient society, and a globally competitive knowledge economy. It intended to make growth more inclusive by reducing poverty incidence in the rural areas, creating more jobs, making individuals and communities more resilient, driving innovation, and building greater trust in government and society.


influence trends in Asia and the Pacific region, its capacity for interdisciplinary and integrated approaches, its ability to blend knowledge with development finance, and its capacity to facilitate knowledge exchange and cooperation among developing countries in the region.

Figure 1. Philippine Development Plan Strategic Framework for Industry and Services

Source: Philippine Development Plan, 2011–2016.

Performance Assessment 7

(i) IT-BPO TA: Shortage of high-quality labor in the sector. In the mid-2000s, IT-BPO was the fastest-growing employment sector and ranked high for generating revenue,16 as IT-BPO accounted for 34.5% of annual employment growth and 37.9% of revenues growth from 2004 to 2009.17 Revenues reached $8.9 billion in 2010, and 638,000 workers were employed in the IT-BPO sector by end of 2011. By 2016, the industry was expected to generate revenues of more than $20 billion while providing 1.3 million jobs. Such outlook for the IT-BPO sector increased the demand for high-quality workforce, which exceeded the supply despite a steady influx of tertiary graduates into the IT-BPO sector. The country’s educational institutions were unable to provide sufficient numbers of suitable graduates to meet the rapidly growing labor demand of the IT-BPO industry. The IT-BPO TA was formulated to counteract the widening talent-supply gap that was being exacerbated by weak university–industry linkages and higher education programs. By introducing an innovative, higher education program to ensure that appropriate skills were sustainably developed, the TA sought to strengthen the country’s educational system and make it relevant to the country’s comparative advantage in the IT-BPO sector.

(ii) Tourism TA: Skills mismatch and outdated standards in tourism industry. More than 3.7 million people were working in the Philippines tourism industry as of 2011, most of whom were living outside the main urban centers.18 Nevertheless, the number of foreign tourists visiting the Philippines in 2010 (3.5 million) was dwarfed by the numbers for Malaysia (20 million), Thailand (16 million), and Indonesia (5 million). The 2011 World Economic Travel and Tourism competitiveness index had ranked the Philippines 94th of 139 countries due to its poor regulatory framework, business environment, tourism infrastructure, and service standards. The industry was beset by problems of low service standards, skills mismatch of new employees, and an outdated hotel and resorts assurance rating system that had been poorly implemented. Vocational training curricula in the tourism industry were not regularly updated to keep pace with changing international standards and practices. The Tourism TA was formulated to improve competitiveness of the Philippines’ tourism sector, develop the skills of the industry’s labor force, and increase employment.

(iii) At-Risk Youth TA and JobStart Rollout TA: Slow school-to-work transition among youth. In 2008, unemployment among the youth (15 to 24 years old) was estimated at 21% and accounted for 50% of all estimated unemployed persons in the Philippines.19 A 2012 study conducted by ADB on the youth labor market found that high school graduates and dropouts had difficulty integrating into the labor market and that it took an average of 4 years for a high school graduate to find a permanent wage job compared to 2 years for college graduates.20 Frequent and extended periods outside the labor market without training further damaged the future career and employment prospects of young people, and this could perpetuate a lifetime of poverty for those from poor households. In this context, the At-Risk Youth TA was formulated to help the DOLE improve the government’s employment facilitation systems by improving the capacity of PESOs and enhancing the employability of at-risk youths. The TA helped achieve policy actions on implementing a regulatory review of the labor code, capacity development for the institutionalized PESOs, and pilot implementation of JobStart in four LGUs through the PESOs. The JobStart Rollout TA followed to build upon the gains from the pilot-testing

16 IT-BPO refers to the whole outsourcing industry in the Philippines and includes IT and engineering services outsourcing. BPO

refers to customer service (voice and non-voice), finance and accounting, human resources, transcription, legal processes, health care information management, other knowledge process outsourcing, and other services not under IT or engineering services outsourcing.

17 Government of the Philippines. Commission on Information and Communications Technology. 2010. The Philippine Digital Strategy-Transformation 2.0: Digitally Empowered Nation. Manila.

18 Government of the Philippines. Department of Tourism. 2011. Philippine National Tourism Development Plan 2011–2016. Manila.

19 Philippines Household Labor Force Survey, 2003–2008. 20 K. Bird. 2012. Are Filipino Youth Off to a Good Start? Youth Labor Market Experience in the Philippines. ADB. Manila.


of JobStart in the At-Risk Youth TA and to set up the institutional structure for the national rollout of the JSP.

D. Cost and Financing

14. Table 2 provides details on the approved and disbursed funding amounts and the sources for financing the four TA projects. Of the four, the IT-BPO TA was co-funded by the e-Asia Knowledge Partnership Fund, and the other three by the Canadian International Development Agency.

Table 2: Amount and Funding Source of the Technical Assistance Projects

Technical Assistance

Approved Amount

($)

Budget Disbursed

($)

ADB Funding Amount

($)

Cofinancing Amount

($)

Cofinancing Agency

(country)


500,000 385,012 0 500,000 EAKPF (Republic of Korea)


7,116,900 5,915,142 0 7,116,900 CIDA (Canada)


5,661,900 4,789,220 70,000 5,591,900 CIDA (Canada)

Support for the Nationwide Rollout of JobStart Philippines (ongoing)

8,800,000 5,200,909 1,300.00 7,500,000 CIDA (Canada)

ADB = Asian Development Bank, CIDA = Canadian International Development Agency, EAKPF = e-Asia Knowledge Partnership Fund. Source: Asian Development Bank.

E. Executing Arrangements 15. IT-BPO TA. The Department of Finance (DOF) was executing agency for the TA. Chaired by DOF, the steering committee was comprised of representatives from the Commission on Higher Education, National Economic and Development Authority, and representatives from industry and academia. The implementing agencies were the IT & Business Process Association of the Philippines (IBPAP), the University of the Philippines Open University (UPOU), and the Asian Institute of Management (AIM). IBPAP was given the task of managing overall implementation of the TA, UPOU designed and deployed e-learning products for industry-designed courses in HEIs, and AIM was tasked with leading the TA knowledge components. 16. Tourism TA. The DOT was executing agency for the TA, and, as such, the project implementation unit was established at DOT and funded by the TA. The implementation unit was responsible for coordinating TA activities, overall administrative tasks, project monitoring, and providing support to the project steering committee and the national and four regional tourism skills project committees. The steering committee, chaired by DOT, provided oversight functions. There were subcommittees reporting to the steering committee as follows: (i) DOT’s RIA steering committee, responsible for implementing the regulatory review and reform program; (ii) DOT’s Division of Accreditation and Regulation, responsible for implementation of the hotel and resort accreditation pilot program; and (iii) the national tourism skills project committee, responsible for implementing the skills development program and skills grant scheme. 17. At-Risk Youth TA and JobStart Rollout TA. DOLE was executing agency for the TA. A project implementation unit was established within DOLE’s Institute for Labor Studies (ILS) to oversee implementation of the output on strengthening capacity for labor policy analysis and formulation, while another project implementation unit in DOLE’s Bureau of Local Employment (BLE) oversaw implementation of the JobStart pilot. The LGUs through their PESOs in the pilot areas were the implementing agencies. In the follow-up TA, DOLE-BLE and the PESOs maintained their respective roles as executing and implementing agencies.


F. Consultants 18. IT-BPO TA. The TA required 5 person-months of international consulting services and 56 person-months of national consulting services. The IBPAP, UPOU, and AIM were engaged as consultants using single-source selection in accordance with the ADB Guidelines on the Use of Consultants (2013, as amended from time to time). ADB approved minor changes to the consulting service requirements of these agencies to allow for more realistic implementation arrangements, such as adjusted person-months and different expert specialization. Of the 14 expected person-months allocated to AIM (7 person-months international and 7 person-months national), only 2 international person-months and 3 national person-months were completed. An additional 28 person-months of national input was added to IBPAP from the original 12 person-months, mainly due to responsibilities that were transferred to it from AIM. UPOU was able to complete its consultations within the 25 person-months allocation of national inputs. 19. Tourism TA. The University of Queensland was contracted to implement the project, with a team of 21 persons comprised of both international and national consultants. The TA planned for 114 person-months of international inputs and 269 person-months of national inputs. The consulting team worked well together and coordinately closely with DOT. It mostly conformed to the planned consultation, as the team utilized 116 person-months of international inputs and 232 person-months of national inputs.

20. At-Risk Youth TA. To support DOLE-ILS in strengthening the capacity for labor policy analysis and formulation, 8 international consultants (40 person-months) and 3 national consultants (42 person-months) were planned for engagement. Meanwhile, 5 international consultants (53 person-months) and 7 national consultants (92 person-months) were planned to support DOLE-BLE in implementing the JobStart pilot. Of the total 227 planned person-months for both outputs, 185 were utilized (91 international and 94 national). Only one international labor policy expert and two national consultants were engaged to support DOLE-ILS, while Scope Global Pty Ltd. was engaged to work with DOLE-BLE.

21. JobStart Rollout TA. Scope Global Pty Ltd. was recruited to ensure continuity in policy advice and capacity in transition from the pilot in the At-Risk Youth TA to the nationwide rollout of JSP. It was envisaged to engage 2 international experts (15 person-months each) and 19 national specialists (total of 265 person-months). The plan was to have five project coordinators based in the National Capital Region and another nine project coordinators stationed in the rest of the country.

G. Scheduling

22. The first three TA projects took around 4 years to complete (Table 3), while the JobStart Rollout TA is expected to take more than 5 years. The IT-BPO TA was extended by 1 year to accommodate additional activities needed to complete full e-service and management program courses and to ensure sustainability of activities. The Tourism TA was extended by 6 months to help DOT draft a concept paper to support nationwide rollout of the skills grant program that had been piloted under the TA. The At-Risk Youth TA was extended by 6 months to finalize preparations for the follow-up JobStart Rollout TA.

Table 3: Summary Schedule of Technical Assistance Projects

Technical Assistance Approval

Date Effective

Date

Original Completion

Date

Revised Completion

Date

Length of Completion Extension

Strengthening Knowledge-Based Economic and Social Development 14 Sep 2012 18 Oct 2012 15 Oct 2015 15 Oct 2016 +12 months

Improving Competitiveness in Tourism 5 Mar 2013 15 Apr 2013 28 Feb 2017 31 Aug 2017 +6 months

Employment Facilitation for Inclusive Growth 5 Mar 2013 15 May 2013 28 Feb 2017 31 Aug 2017 +6 months


Technical Assistance Approval

Date Effective

Date

Original Completion

Date

Revised Completion

Date

Length of Completion Extension

Support for the Nationwide Rollout of JobStart Philippines 30 Oct 2015 21 Jan 2016 31 Dec 2020 ongoing ongoing

Source: Asian Development Bank.

H. Design Changes 23. The three completed TA projects had minor changes in design during implementation. For the IT-BPO TA, changes were in the consulting service requirements and an extension of 12 months to complete the remaining activities. For the Tourism TA, changes included the addition of an international labor market and skills forecaster, a human resource strategist, and an evaluation expert, as well as engagement of a public relations firm and a national firm to conduct a visitors’ satisfaction survey. For the At-Risk Youth TA, changes included an increase in inputs of the international employment expert and of project coordinators to advise on the rollout process, shortening of the technical training offered by employers, and redevelopment of the life skills training (LST) program in response to disparities in the quality of content developed by providers.

CHAPTER 3

Performance Assessment 24. This Chapter assesses each TA project’s performance based on four core criteria: (i) relevance to the development strategies of the government and ADB, and relevance to achieving the envisaged objectives; (ii) effectiveness of outcomes and outputs; (iii) efficiency of the utilization of resources; and (iv) sustainability of outcomes and outputs.

A. Relevance 25. The three TCRs rated the TA projects relevant except for the IT-BPO TA which was found to be less than relevant. Although that TA was strategically aligned with the government’s development priorities and constituted a timely response to the key development issues confronting the IT-BPO industry, the design was not conducive to achieving the objective. That is because there were significant deficiencies that affected the delivery of targeted outputs and the intended outcome. The other TA projects were strategically aligned with the country’s development priorities and ADB’s strategies, appropriately addressed the key issues and constraints identified, were well-designed to achieve the expected outcomes, incorporated innovative features that were responsive to the identified development issues, and eventually were scaled up. Based on assessments in the following paragraphs, the TPER assessed the TA projects’ relevance as shown in Table 4.

Table 4: Relevance Ratings of the Job Creation and Skills Development TA Projects


Relevance Evaluation Subcriteria

Relevance Rating for each TA

Strategic Alignment

Support for Policy Reforms Project Design

Innovations in TA Design


Fully aligned

Did not support

Not appropriate, given the lack of IT-BPO leadership

E-learning was too ambitious in both

scope and execution Less than relevant


Fully aligned

Strongly supported Appropriate

Empowering employers aided

sustainability Relevant


Fully aligned

Strongly supported

Appropriate, but better integration needed (i.e., RIA)

Packaging of LST and career guidance

raised employability Relevant Support for the Nationwide Rollout of JobStart Philippines

Fully aligned

Strongly supported

Appropriate, but stronger M&E

needed for rollout

Various pathways within JSP ensured flexibility for youths Relevant

IT-BPO = information technology and business process outsourcing, JSP = JobStart Philippines, LST = life skills training, M&E = monitoring and evaluation, RIA = regulatory impact assessment, TA = technical assistance. Source: Independent Evaluation Department.

26. Strategic Alignment. All four TA projects were strategically aligned with the government’s development priorities identified in the PDP, 2011−2016, which emphasized sustained high and inclusive growth, massive jobs creation, and poverty reduction. Based on the priorities identified in the PDP, ADB’s assistance was well positioned in the IT-BPO sector, tourism industry, and youth employment subsector as follows:


(i) IT-BPO TA. The TA aligned well with the government’s priorities, as IT-BPO had been identified in the PDP as among the 10 priority development sectors with the highest potential for promoting economic growth and job creation. It was also fully aligned with the government’s knowledge strategy and program and ADB’s knowledge management plan.21 The TA’s push to match higher education programs to labor market demand reflected the strategic priorities of ADB’s Education Sector Operations Plan for strengthening postsecondary education. 22 The TA also aligned with ADB’s Country Partnership Strategy (CPS) for the Philippines, which sought high, inclusive, and sustainable growth for the country.23

(ii) Tourism TA. The TA was in line with PDP, 2011–2106, which pinpointed the tourism sector for long-term development based on sector performance, job creation possibilities, and potential positive spillover effects to other sectors. The TA was aligned with the priorities outlined in the Philippine government’s National Tourism Development Plan, 2011−2016 that included adopting a framework for hotel and resort quality assurance, tourism industry skills development, and regulatory review of the industry.24 The TA was also included in ADB’s Country Operations Business Plan (COBP,) 2013−2015 for the Philippines.25

(iii) At-Risk Youth TA. The TA was aligned with PDP, 2011−2016, which promoted job creation as a central theme of the country’s economic program and had set an ambitious goal to create 1 million jobs per year in order to maintain the unemployment rate within the range of 6.8% to 7.2%. The TA was included in ADB’s COBP, 2013–2105 for the Philippines, which specified support to the government’s thrust for inclusive growth, while also being aligned with ADB’s CPS, 2011−2016.

(iv) JobStart Rollout TA. The TA was consistent with both PDP, 2011−2016 and its successor, PDP, 2017−2022. The latter emphasized accelerated human capital development as critical for making growth inclusive and reducing poverty and income inequality. The government’s blueprint for job creation prioritizes entrepreneurship, reducing youth unemployment, education, and training, while aiming to create 7.5 million jobs in key job-generating sectors. The TA was aligned with ADB’s CPS, 2018–2023 for the Philippines that supported policy reform, institutional capacity development, and financing investments that promote high and inclusive growth.26 It was consistent with ADB’s Strategy 2030 that seeks to address remaining poverty and reduce inequality as one of the strategy’s seven priorities.27

27. Support for policy reforms. Except for the IT-BPO TA, all TA projects supported policy reforms to improve competitiveness for inclusive growth and facilitating youth employment. Specifically, the Tourism TA and At-Risk Youth TA were associated with ADB’s programmatic approach and policy-based loan to the Philippine government on Increasing Competitiveness for Inclusive Growth, 28 while the JobStart Rollout TA was also linked to another programmatic approach and policy-based loan, the

21 ADB. 2012. Country Operations Business Plan: Philippines 2013−2015. Manila (Supplementary Document: Knowledge Strategy

and Program, 2013−2015). 22 ADB. 2010. Education by 2020: A Sector Operations Plan. Manila. 23 ADB. 2011. Country Partnership Strategy: Philippines 2011−2016. Manila. 24 Government of the Philippines. Department of Tourism. 2011. Philippine National Tourism Development Plan, 2011−2016.

Manila. 25 ADB. 2012. Country Operations Business Plan: Philippines, 2013−2015. Manila. 26 ADB. 2018. Country Partnership Strategy Philippines, 2018−2023 – High and Inclusive Growth. Manila. 27 ADB. 2018. Strategy 2030: Achieving a Prosperous, Inclusive, Resilient and Sustainable Asia and the Pacific. Manila. 28 ADB. 2012. Report and Recommendation of the President to the Board of Directors: Proposed Programmatic Approach and

Policy-Based Loan for Subprogram 1 to the Republic of the Philippines for Increasing Competitiveness for Inclusive Growth Program. Manila; and ADB. 2014. Report and Recommendation of the President to the Board of Directors: Proposed Policy-Based Loan for Subprogram 2 to the Republic of the Philippines for Increasing Competitiveness for Inclusive Growth Program. Manila.


Facilitating Youth School-to-Work Transition Program. 29 Overall, the three TA projects contributed significantly to the government’s broader policy reforms to improve competitiveness in tourism and facilitate youth employment—mainly through the regulatory frameworks established by the RIAs, the quality assurance system adopted for the tourism industry, and the skills developed to facilitate employment.

28. Project design. The TA designs were generally appropriate for achieving the intended outputs and outcomes, except in the case of the IT-BPO TA.

(i) IT-BPO TA. The TA addressed the skills shortage in the IT-BPO sector by helping align higher education curricula with international standards and labor market demand while building on public and private initiatives. The TA sought to increase access to industry-designed courses on IT-BPO outsourcing and strengthen partnerships with industry and academic institutions. The TA design, aimed at strengthening the higher education system by introducing an innovative and creative program, was relevant to the country’s competitive advantage in the IT-BPO industry but encountered shortcomings during actual implementation. First, as there were no existing government agencies that could lead the capacity development of IT-BPO during TA formulation, DOF was selected by default as executing agency, despite lacking the public mandate and technical expertise to oversee the sector. As DOF and the project steering committee that it chaired did little to provide guidance and coordination needed for the three implementing agencies to work in synergy, this left a vacuum wherein IBPAP had to step in, in an ad-hoc manner.

29 ADB. 2017. Report and Recommendation of the President to the Board of Directors: Proposed Programmatic Approach and

Policy-Based Loan for Subprogram 1 Republic of the Philippines: Facilitating Youth School-to-Work Transition Program. Manila; and ADB. 2019. Report and Recommendation of the President to the Board of Directors: Proposed Programmatic Approach and Policy-Based Loan for Subprogram 2 Republic of the Philippines: Facilitating Youth School-to-Work Transition Program. Manila.

Box 1. International good practices on anticipating future skills needs and stimulating growth

• Ireland: The Expert Group on Future Skill Needs (EGFSN) analyses future skill needs and develops proposals for how to meet them through a broad membership that includes business representatives, educationalists, trade unionists, and policymakers. The breadth of participation enables EGFSN to identify changing occupational profiles within sectors and the demand for different occupations. EGFSN identified the key elements to be included in a generic skills portfolio for the future: basic or fundamental skills (literacy, numeracy, information and communication technology); people-related skills (communication, team-working); and conceptual and thinking skills (collecting and organizing information, problem-solving, planning and organizing, learning to learn, innovation and creative skills). It provides advice on how to improve job-seekers’ awareness of sectors where skills are in demand and the qualifications required.

• Brazil: The wide replication of Brazil’s National Service for Industrial Training (SENAI) is a good measure of success. SENAI is run by an association of industries, funded by a levy on the industrial payroll, and has sister institutions serving various sectors (agriculture, small enterprise, service sector, etc.). SENAI’s “Prospecting Model” adjusts training provision based on analysis of take-up rates for emerging technologies and new forms of work organization. The model generates estimates of job requirements over a 5-year period based on studies of technological and organization prospecting, tracking of emerging occupations, and monitoring demand trends for vocational training. However, the proportion of young people able to take advantage of training opportunities is limited by the quality of basic education.

• Republic of Korea: At the core of the country’s sustained growth pattern lies a government-led skills development strategy. The rapid progress in closing the productivity gap reflected an economic development strategy based on investment and on research and development. Investment in a well-educated and highly skilled workforce was an integral part of encouraging the adoption of new technologies. A current challenge is to avert shortages in the higher-end vocational occupations by increasing the attractiveness of nonacademic skills-development paths.

Source: International Labour Organization. 2010. A Skilled Workforce for Strong, Sustainable and Balanced Growth: A G20 Training Strategy. Geneva.

https://www.oecd.org/g20/summits/toronto/G20-Skills-Strategy.pdf



Second, the TA design decision to contract multiple institutions as implementing agencies brought to the table differing philosophies and approaches, which often clashed and created unnecessary challenges during TA implementation. Aggravated by the lack of clear management and reporting structure, loose integration among the implementing agencies unraveled and resulted in the mid-implementation departure of AIM, with IBPAP picking up where AIM had left off. Third, the timeline was too ambitious given the sheer scope of the intervention and the expected outputs and outcomes. Steps necessary to ensure sustainability and development impact—such as course accreditation and acceptance by schools, twinning arrangements between foreign and domestic HEIs, and piloting of a replicable and sustainable system of ICT-enabled knowledge hubs—could not be undertaken without significant additional time. Fourth, the lack of IT infrastructure in rural areas (i.e., availability of high-speed internet connection or e-learning labs) had not been taken into consideration in the design. This hampered deployment of ICT-enabled BPO industry-designed curricula in HEIs outside the National Capital Region.

(ii) Tourism TA. The TA supported operationalization of key interventions from the National Tourism Development Plan addressing issues of low quality standards and skills mismatch, specifically through capacity building for regulatory review and reform, raising tourism and hospitality service standards, and developing a skills base to support growth in the tourism sector. To do so, the TA design contained the following three main components: (i) implementation of the RIA program to promote policy reforms, (ii) implementation of a hotel and resort quality assurance and accreditation system to improve service quality standards, and (iii) development of a tourism industry skills development program and piloting it in four regions to improve capacity. Although the design was relevant, such key performance indicators as reduction in the cost of compliance with regulations in the tourism industry and tourism professionals achieving mutual recognition under the Association of Southeast Asian Nations (ASEAN) agreement were not monitored or reported. Moreover, the indicator on increased employment in the tourism industry as a metric for impact was too ambitious as employment was influenced by too many external factors outside the TA’s scope.

(iii) At-Risk Youth TA. During TA implementation, it was noted that some of the DMF indicators, such as comparative indicators on the job placement rate of similar groups that were not receiving JSP services and the number of employers that utilized PESOs not offering JSP services, had not been monitored. In the TCR, progress was reported for only 7 out of 15 indicators identified in the DMF, thus indicating weak monitoring of performance indicators. The TA could have benefitted from an assessment at the outset as to the appropriateness and feasibility of generating and monitoring of the indicators on a regular basis. Also, the project design could have better integrated the RIA intervention into the rest of the TA.

(iv) JobStart Rollout TA. The TA aimed at scaling up on the JSP pilot model in the At-Risk Youth TA by rolling out the LST curriculum nationally, building capacity of JSP units at DOLE headquarters and its regional offices, and building capacity of PESOs participating in JSP. The JobStart Rollout TA brought about its own set of challenges, however, as both DOLE and PESOs were now sharing a greater burden of funding the technical training and on-the-job training for the JobStarters. Although this was theoretically ideal, because it was meant to foster local ownership and long-term sustainability, in actual practice unpredictable and inconsistent funding from DOLE either delayed or curtailed implementation in many LGUs, making it much more challenging to keep the LGUs engaged and interested in the project. Also, documentation and reporting requirements for training participation that were viewed as burdensome during the pilot have largely remained, discouraging potential employers, and causing them to lose interest and withdraw from the program. Although DOLE might have been able to support and push a few PESOs across the finish line during the pilot implementation, doing so for a large-scale rollout would severely strain DOLE’s administrative capacity. As such, the project


design could have benefitted had more efforts been made to streamline administrative procedures and ease requirements for the beneficiaries and implementing partners, thereby incentivizing more to buy into the JSP Rollout TA rather than to make it riskier and more cumbersome.

29. Innovations in TA Design. Each TA introduced its own set of innovations into its designs to improve or reform existing policies and processes and that would increase effectiveness, impact, or sustainability of the TA in its respective sector.

(i) IT-BPO TA. The TA introduced innovations meant to strengthen the access of potential employees from remote or rural regions to IT-BPO industry training and education. These consisted of developing and implementing web-based programs for teaching industry-based courses in IT-BPO and establishing linkages between industry and academia to provide relevant and up-to-date training. The pipeline to help facilitate job integration into the IT-BPO industry through e-learning, however, was implemented separately by one of the implementing agencies, as participation of other agencies in the ecosystem, including the employers and community of practice, was missing due to their lack of institutional capacity to scale up the training. Meanwhile, HEIs lacked both teachers and capacity to upgrade the level of their service management program.30 The innovation required by the TA simply outpaced the realities of what was possible at the time in the Philippines.

(ii) Tourism TA. The TA was an innovative approach to skills development that involved public–private partnership between DOT, employers, and industry groups. The program was an employer-led initiative targeting four fast-growing tourist destinations in the Philippines, namely Bohol, Cebu, Davao, and Palawan. The TA piloted the industry skills grant scheme (ISGS), which provided grants to the tourism industry to help increase efficiency, improve revenue, strengthen employee performance and confidence, and boost customer satisfaction. A key innovation in the ISGS was the unprecedented level of control and ownership given to owners in the local tourism sector, as they could define

30 The service management program was approved by the Commission on Higher Education in 2012 and was a 21-unit minor

degree course that could be taken by university or high school students intending to go into IT-BPO. It was designed by industry practitioners to provide students with the required skills set.

Box 2. Lessons from Brazil’s SENAI in Skills Development Training

A review of Brazil’s National Service for Industrial Training (SENAI) experience over its most recent 6 decades provided the following lessons independently of the country-specific context: • Need for inclusive institutional framework: Training should not be provided within an institutional framework

that excludes the informal sector and is potentially subject to political uncertainty. To achieve this goal, policymakers have to ensure that any intervention to create or modify the vocational training system is oriented around offering each ethnic group the same development chances and that there is a focus on finding the proper financing balance between flexible and fixed sources (sale of services and levy) that avoids fluctuations and uncertainties. The set of interventions has to support (i) the reduction of occupational segmentation, thereby encouraging women to move from labor-intensive towards capital-intensive sectors, and (ii) promotion of opportunities of internal migration.

• Independent and decentralized governance structure: Training institutions should not be under exclusive control of government officials. An independent and decentralized governance structure would encourage the emphasis to be on improving labor market outcomes of all workers (including shop floor workers) while ensuring training courses meet the demands of the private sector across the territory, independently of whether they are in the informal or formal economy.

Source: C. Barría and S. Klasen. 2016. The impact of SENAI’s vocational training program on employment, wages, and mobility in Brazil: Lessons for Sub Saharan Africa? The Quarterly Review of Economics and Finance. 62. pp. 74–96.


the group to be trained, identify and request the type of training needed, and identify the pool of candidate trainers to provide the training. DOT’s rollout program that followed up the TA, the Tourism Industry Skills Program, retained this innovation, albeit with a slight twist: it shifted the control away from individual owners to associations of resort and business owners in order to improve coordination in selecting training programs.

(iii) At-Risk Youth TA. The TA piloted MyFirstJob, which was renamed as JSP. JSP was an innovative job facilitation scheme for at-risk young Filipinos and was conceived as a demand-driven system in that the employers led the process of determining the skills they needed, the technical training required, and the recruitment of interns. It adopted an integrated approach that offered at-risk youth LST, career guidance, and opportunities for technical training and internship with private sector employers.

(iv) JobStart Rollout TA. The successful piloting of MyFirstJob through the At-Risk Youth TA led to a rebranding and the follow-up Support for the Nationwide Rollout of JSP. An additional innovation introduced in the design was to offer different pathways for JobStart trainees, or JobStarters, as it was observed during the pilot that not all youths needed to go through the full employment facilitation cycle, and specifically there were those who were relatively more job-ready and just need to be matched with or referred to an employer. The innovation in the rollout by offering different pathways made the design more flexible and responsive to the varying capacity and readiness of each JobStarter. Furthermore, to facilitate transition to employment, the career guidance component was merged with LST and the internship period was reduced from 6 to 3 months. Figure 2 shows the revised JSP cycle.

30. Outreach. There was a lack of concerted effort to raise public awareness for JobStart. Despite the TA project’s being meant to alleviate poverty for those most vulnerable, insufficient public relations activities or campaigns were targeted to reach out to them. Instead, the LGUs and departments’ regional offices used existing or familiar human networks in order to elicit participation from those less needy and thus, less qualified, simply because they had greater access to information concerning the TA programs. It was observed during the FGDs, for example, that a noticeable number of Jobstarters were college graduates and as such had greater probability of becoming employed. In Davao, the use of such classic media as television and radio advertisements was lauded but most JobStarters credited social media, chat groups, and personal references as channels through which they first learned about JSP. Rural communities with impoverished youths were reached through the barangay leaders’ personal networks. In addition, social media should be used as a supplementary network to reach youths with greater access to smartphones.

Wage Employment

Technical Training (3 months)

Life Skills Training & Career Guidance (10 days)

Registration

Referral to Other Programs

Job Matching

Internship (3 months)

Figure 2. JobStart Philippines Training Cycle

Source: Department of Labor and Employment. JobStart.


31. Across all four TA projects, outreach to women was nevertheless significant. In JobStart, 58% of the beneficiaries were women. In the IT-BPO TA, 60% of teachers participating in the online teacher training program and 49% of students that completed the online training modules were women. In the Tourism TA, 45% of those trained in industry skills were women. These results indicate relatively successful gender mainstreaming of TA projects. This was evidenced during the survey of FGD participants, where there were no significant variations along gender lines, thus signifying that the TA projects made sure no gender bias was “perceived” by the beneficiaries during planning and implementation.

B. Effectiveness 32. Assessment of the TA projects’ effectiveness looked at whether the intended outcomes were achieved when evaluated against the targets specified in the DMF and the extent to which these outcomes were attributable to each project’s intervention. Appendix 2 states targets and accomplishments of each TA in relation to its DMF. Based on assessments in the following paragraphs, the TPER assessed the TA projects’ relevance as shown in Table 5. Table 5: Effectiveness Ratings for the Job Creation and Skills Development Technical Assistance Projects


Effectiveness Evaluation Subcriteria Effectiveness Rating for each Technical

Assistance Output Outcome

Strengthening Knowledge-Based Economic and Social Development Not fully achieved Not fully achieved Less than effective Improving Competitiveness in Tourism Mostly achieved Achieved Effective Employment Facilitation for Inclusive Growth Achieved Achieved Effective Support for the Nationwide Rollout of JobStart Philippines

Mostly achieved (ongoing)

Mostly achieved (ongoing) Effective

Source: Independent Evaluation Department.

33. Outputs of IT-BPO TA. The TA’s expected outputs were all not fully achieved at completion.

(i) The output on increased access to knowledge resources for teaching and learning IT-BPO industry courses was partly achieved. The training materials for teachers and five electronic service management program (eSMP) courses were digitized and offered online, but the target of establishing at least three knowledge hubs in HEIs was not fully realized. Only one online knowledge hub was established at UPOU and only one physical hub was completed at IBPAP. Meanwhile, the business model for operating and sustaining the knowledge hubs was successfully conceptualized, but its delivery was significantly delayed until the end of project implementation due to IBPAP’s having to take over the task for AIM. As such, the proposed business model could not be piloted due to lack of time remaining within the TA’s timeline.

(ii) The output on enhancement of higher education curricula and instruction was partly achieved, as only 379 (42%) of the targeted 900 faculty members from the participating HEIs completed the online teacher training on IT-BPO industry-designed courses and 2,276 (76%) of the targeted 3,000 students completed the online learning modules. The shortfall for teachers could be attributed to lack of advocacy for the program and incentives for teachers to get certified, as the trainings were neither formally recognized nor were sufficient stipends available. The students suffered from the same lack of advocacy, worsened by the lack of familiarity with e-learning during TA implementation.

(iii) The output on strengthening partnerships with industry and academic institutions was likewise not fully realized. Although the TA did see the development of a partnership framework between HEIs and industry and signing of memoranda of agreement between IT-BPO companies and schools, the intended twinning arrangements between foreign


and domestic HEIs were not achieved due to the unrealistic timeline set under the TA that did not provide enough time for that process to bear fruit.

34. Outcomes of IT-BPO TA. The intended outcome of a replicable and sustainable system of ICT-enabled knowledge hubs developed for improved teaching and learning of IT-BPO industry-designed curricula in HEIs was not fully achieved. UPOU conducted both in-person and online training for teachers, but the extent to which HEIs adopted the industry courses was unclear due to poor monitoring. The number of HEIs adopting the industry-designed online courses was not closely monitored or reported by the project, and neither were there any indications that the industry-developed courses had been adopted by the HEIs after TA completion. Furthermore, the global competitiveness assessment tool was not adopted, as the HEIS lacked (i) funds for its administration, (ii) necessary e-learning infrastructure, and (iii) a policy framework within which to assess the teachers’ capacities and to match them to the industry needs. Due to shortcomings in the design and lack of infrastructure that resulted in the TA underperforming in terms of both outputs and outcomes, the TPER assessed it less than effective. 35. Outputs of Tourism TA. The TA had three key outputs, namely implementation of the RIA, implementation of a hotel and resort quality assurance and accreditation system in four regions, and development of a tourism industry skills development program and piloting thereof. The outputs were mostly achieved as follows:

(i) For implementation of the RIA, three out of the four indicators were achieved and one was partly achieved. Although 10 regulatory impact statements (RISs) were completed during implementation as planned,31 only 4 had been implemented because the 3-year time frame was insufficient for recommendations in the remaining 6 RISs to be fully manifested. Also, implementing the recommendations required enforcement powers that did not rest with DOT but fell within the purview of the LGUs or other agencies.

(ii) For implementation of the hotel and resort quality assurance and accreditation system, the National Accommodation Standards incorporating service, hospitality, efficiency, friendliness, and food were revised and published, as were the Mabuhay standards developed for small accommodation providers. 32 A quality assurance manual was developed governing implementation of the new standards, and the performance targets for training, assessment, and accreditation were all met or exceeded (Appendix 3).

(iii) For development of a tourism skills development program, development and piloting within the four pilot regions were partly achieved, as two of the three targets were reached. Development of a human resource strategy was achieved, with DOT completing the first Tourism Human Resource Development Strategy and Action Plan 2015–2020. The number of tourism workers under the program reached 7,500, which exceeded the target of 5,000, but the ratio of female trainees was 47.3%, which was below the target of 55%. Also, there were no monitoring data confirming whether the target of at least 500 tourism professionals being mutually recognized under ASEAN agreement was achieved. It is highly unlikely that this was achieved.

36. Outcomes of Tourism TA. The outcomes of the TA were achieved, as described below. Because both outputs and outcomes were mostly achieved, the TA was assessed effective.

31 A RIS is a public consultation document that is completed for all proposals on regulations considered to have a potential

significant impact on stakeholders. It is the second stage of impact assessment in the RIA process. The third and final stage is the regulatory assessment summary (RAS), which is a summary document of the development process and outcomes of the RIS, including compliance with regulatory best practice principles. It is prepared and attached to each proposed regulation for review by the secretaries of concerned departments and other relevant decision makers (e.g., Congress, the President, Office of the President) when approval is sought for a regulation.

32 Mabuhay is a general term used for a tourist inn, pension house, motorist hotel, bed and breakfast, homestay, or vacation home. Mabuhay standards cover the minimum requirements in the operations of this accommodation category.


(i) The ranking of the Philippines’ tourism industry by the World Economic Forum jumped by 15 places from 94 out of 139 countries in 2011 to 79 out of 136 countries in 2017. In 2019, the industry’s ranking improved further to 75 out of 140 countries. While these improvements could not be attributed solely to the TA project, its support in aligning accreditation and accommodation systems with international standards while providing capacity development for DOT personnel and training skills for employees matching the industry’s needs likely contributed to increasing the industry’s competitiveness.

(ii) The targeted 15% reduction in the business costs of compliance with regulations in the tourism industry was likely achieved, as implementation of the four RISs had a favorable effect on reducing the cost of compliance in areas where they were implemented. For example, the online option for travel tax processing improved efficiency and reduced costs to tourists overall, while at Puerto Princesa, the number of days to secure a Mayor’s Permit for Travel and Tour Agencies was reduced by 70%.

Box 3. Philippines: Puerto Princesa’s Regulatory Impact Assessment

In January 2016, the City Tourism Department of Puerto Princesa applied for the local government unit (LGU) regulatory impact assessment (RIA) Grant Scheme supported by Asian Development Bank (ADB) through the Department of Tourism (DOT). The purpose of the grant was to review and update the Tourism Code of 2011 particularly those provisions related to travel and tour agencies, tourist transport operators, and tour guides. The RIA intended to promote ease of doing business for the private sector by reducing documentary requirements; encouraging unlicensed tour agencies, operators, and guides to secure licenses, and promoting a tourist-friendly Tourism Code. The RIA process involved extensive consultations with stakeholders. Review of the Tourism Code and consultations yielded a recommendation to simplify the processes and requirements for obtaining a Mayor’s Permit. The review concluded that cutting red tape would greatly support the tourism stakeholders by making it easier to do business and reducing costs. The RIA was implemented, resulting in a shortening of licensing time for travel and tour agencies from 10 to 3 days while reducing the number of documents required from 17 to 8. For tourist transport operators, the number of days was decreased from 4 to 2 days and the number of documents was cut from 14 to 10. Shrinking the number of processing days and documentary requirements diminished costs and resulted in savings both to operators and the LGU. After ADB’s technical assistance (TA) project implementation, the RIA process was mainstreamed in the operations of the City Tourism Department, and the LGU has continually improved its internal business processes by establishing a “Business One-Stop Shop” aimed at reducing the paperwork and time needed for local business registration and renewal. The key factors for this success were (i) strong ownership by the LGU, (ii) an institutionalized City Tourism Department that contributed to greater stability of TA implementation, (iii) extensive consultation with stakeholders, and (iv) support of the local chief executive. Sources: Independent Evaluation Mission; Focus group discussions; and Local Government Unit of Puerto Princess City. 2016.

Regulatory Impact Statement: Review of Local Ordinance No. 495 “Tourism Code” of Puerto Princesa. Puerto Princesa.

37. Outputs of At-Risk Youth TA. The project was expected to: (i) strengthen the capacity for labor and policy analysis; and (ii) implement the pilot of MyFirstJob, later renamed JobStart. Based on the results, both outputs were delivered. Nevertheless, the indicators identified to measure these outputs were not well monitored. Of the 15 indicators specified in the DMF, 8 were not reported in the TCR. This reflects poor monitoring of performance indicators. Actual data relating to the DMF indicators gathered by the independent evaluation mission are presented in Appendix 2, Table A2.3.

(i) Of the six indicators for the output on capacity for labor policy analysis and formulation strengthened, four were achieved, one partly achieved, and one indicator not reported on in the TCR. The indicator not reported on was the protocols for early employment recovery program, which had not been formulated for monitoring. The target for RIA


work plans was achieved, but the target on the number of completed RIAs was not. Of the 10 RISs expected to be delivered under the TA, 9 RISs were completed.33 Of these, only two were applied to enhance existing labor policies while eight remained as recommendations that had not been implemented as of December 2019. Although DOLE’s ILS did the research work for the RIAs, their implementation was often beyond the mandate of DOLE-ILS and political support from other agencies was needed for effective follow-up on the RIA recommendations. In terms of RIA capacity building, an implementation team was formed within DOLE comprising of the DOLE RIA Committee, DOLE RIA Working Group, Agency RIA teams, and RIA Administrative Support Group.34 It provided the structure for RIA implementation and building capacity for the RIA processes and tools. The TA successfully conducted 10 training and capacity building activities on the RIA methodology (Appendix 5).

(ii) The TPER assessed that all indicators for outputs on the JobStart pilot were achieved, even though the performance regarding institutionalization of PESOs with standard labor market information and employment systems was not directly verifiable. Of the four pilot PESOs, two improved their labor market information and employment mediation by at least 75%, and one improved by 60%.35 The evaluation team directly observed that standard labor market information system and reports were generally in place in all the DOLE and PESO offices. The target for an impact evaluation was achieved with a randomized impact evaluation completed on the pilot program. Though the number of youth participants completing LST was 1,413, lower than the target of 1,600, during the pilot phase of 2014 to 2015, that figure jumped to 3,398 in 2016 due to the expansion to 10 LGUs (Appendix 6). Such success in piloting of the JobStart program resulted in an agreement with DOLE for a national rollout and passage of a law in 2016 to institutionalize it.36

38. Outcomes of At-Risk Youth TA. The outcomes of the TA were aimed at improving the capacity of PESOs to provide employment as described below. Because both outputs and outcomes were mostly achieved, the evaluation assessed the TA effective.

(i) The first indicator was the number of persons finding jobs through PESOs per year, and this was overachieved. The number of persons finding jobs through institutionalized PESOs increased by 15% annually from 1,936 per PESO during 2005–2010 to 3,844 per PESO from 2010–2015 (target: 10% increase per year). However, the second indicator (increase in participating PESOs’ number of employers using their services in MyFirstJob over PESOs not offering the MyFirstJob project) was not verifiable, because monitoring was not done on PESOs not participating in the TA.

(ii) Aside from the original indicators, there were other signs indicating achievement of the envisaged outcome of PESOs’ improved capacity to provide employment. The successful completion of the pilot phase of MyFirstJob led to an agreement with DOLE for a national rollout of JSP that has covered 35 LGUs, and labor market information has been made readily available in an online platform through the Public Employment Information System (PEIS),37 which now facilitates intermediation of employment by the PESOs across the country.

39. Outputs of JobStart Rollout TA. The ongoing TA is expected to deliver three outputs: (i) capacity development of JSP units at DOLE headquarters and regional offices strengthened, (ii) life skills for

33 The RIAs undertaken under the TA are shown in Appendix 4, Table A4.2. 34 Department of Labor and Employment. 2012. Administrative Order No. 29-A Series of 2012. Manila. 35 Based on the final report prepared for the pilot phase of JobStart: DOLE. 2016. JobStart Philippines Program Pilot Phase Final

Report 2014−2016. Manila. 36 Republic Act No. 10869. 2016. An Act Institutionalizing the Nationwide Implementation of JobStart Philippines.. Manila. 37 Formerly Skills Registry System (SRS).


employability training curriculum rolled out nationally, and (iii) capacity of 24 PESOs in JSP implementation developed. Most of the targets of these expected outputs had been achieved as of December 2019.38

(i) As of December 2018, the output on capacity development of JSP units in DOLE headquarters and regional offices had been delivered. The JSP unit was established in DOLE-BLE and in the regional offices, formalizing the institutional structure for JSP within DOLE for the long term. Regarding the output on life skills for employability, 13,158 youth had been trained and completed the LST, exceeding the target of 5,000. On development of the implementation capacity of the PESOs, JSPs had been established in 35 PESOs, exceeding the target of 24. The only indicator that had not been achieved was the training of PESO staff on delivering LST, for which 74 PESO staff had been trained (target: 400).

(ii) The PEIS had been implemented and was operational in all 35 PESOs, exceeding the target of 24. A total of 440 employers had participated in JSP in these PESOs, substantially greater than the target of at least 10. Meanwhile, training of PESO staff on JSP business operations, gender sensitivity, career guidance, and coaching are ongoing.

40. Outcomes of JobStart Rollout TA. The intended outcome was for the JSP rollout by DOLE and PESOs to commence, which already had been achieved as of December 2019. By December 2018, a total of 35 LGUs had signed memoranda of agreement with DOLE for JSP implementation, which exceeded the target of 24. A total of 21,063 at-risk youth (58% female) across 35 PESOs had accessed or registered for JSP services, substantially greater than the target of 10,000. Based on the observations that both outputs and outcomes had been achieved except for a few interventions that were still ongoing, the evaluation assessed the TA effective.

C. Efficiency 41. Based on the assessments below, the TPER assessed the TA projects’ efficiency as in Table 6.

Table 6: Efficiency Ratings of the Job Creation and Skills Development Technical Assistance Projects


Efficiency Evaluation Subcriteria Efficiency Rating for each Technical

Assistance Time

Utilization Fund

Utilization Process

Efficiency


12-months delay

Less than 80%

Less than Efficient Less than efficient


6-months delay

Greater than 80% Efficient Efficient


6-months delay

Greater than 80% Efficient Efficient

Support for the Nationwide Rollout of JobStart Philippines Ongoing

Not rated (ongoing)

Not rated (ongoing) Not rated

Source: Independent Evaluation Department.

42. Efficiency in Time Utilization. According to the TCRs, the completed TA projects were deemed efficient with the exception of the IT-BPO TA. The resources used in that TA did not fully deliver the expected outcome, while implementation was delayed by 12 months. The arrangements for implementation and coordination among agencies were not the most efficient. On the other hand, the other two completed TA projects did not suffer from such significant delays or inefficiencies. In the ongoing JSP Rollout TA, three issues have been causing implementation delays. First, delays in downloading of funds to LGUs from DOLE caused JobStart registrants to lose interest and drop out from the program due to the long waiting time. Second, there were instances when the memoranda of

38 For output indicators and targets for TA 8984, refer to Appendix 2, Table A2.4.


agreement with employers were not ready in time for deployment of LST finishers to the technical skills training and internship phase. Third, service providers were often not contracted in time for the LST due to the government’s financial management processes. These issues have significantly delayed implementation of the TA and could critically reduce its efficiency unless addressed before completion. 43. Efficiency in Fund Utilization

(i) IT-BPO TA. Of the $500,000 approved for this TA, only 77% was utilized. This was due in part to outputs and outcomes not being fully realized. The funds were underutilized despite a 12-month extension to ensure completion of the expected outputs on the partnerships between industry and academic institutions, these being increased access to knowledge resources for teaching and learning IT-BPO courses and enhancement of higher education curricula. This indicates less than efficient use of resources in delivering the intended outputs and outcome.

(ii) Tourism TA. Of the $7,116,900 approved for this TA, 83% was utilized to achieve the intended outcome. The remaining 17% was unutilized due to ring-fencing of funds as buffer for foreign exchange risk. During pilot testing of the ISGS, 7,500 industry workers were trained at an average cost of $130 per trainee. That would be considered cost-effective compared to other skills trainings.39 The TA successfully exploited economies of scale in the delivery of training by providing matching grants to networks of employers in order to address the constraint of small-scale operations’ inability to provide quality training due to lack of resources. Furthermore, the RIAs from the TA helped the LGUs introduce regulations that reduced regulatory compliance costs for tourism businesses. As such, the envisaged outcome of improved competitiveness of the tourism industry was realized with no cost overruns, indicating efficient utilization of resources.

(iii) At-Risk Youth TA. Of the $5,661,900 amount approved for this TA, 85% was utilized. The remaining 15% was unutilized due to ring-fencing of funds for foreign exchange risk. The amount utilized delivered the envisaged outcome of improved capacity of PESOs to provide employment services while successfully piloting the JSP, thus indicating efficient utilization of resources.

44. Efficiency in Process

(i) IT-BPO TA. This TA was extended by 12-months to accommodate additional activities needed to complete the e-service and management program courses in their entirety. IBPAP’s lack of project management experience and unfamiliarity with ADB processes and procedures contributed to the delay, as evidenced by the 1 year it took to field the first consultant from date of signing. The arrangement for implementation was not optimal, as there was no single entity tasked with managing and coordinating the TA activities. This was aggravated by lack of clear reporting structure for the three implementing agencies involved. This deficiency of structure negatively affected the working dynamics among implementing agencies and later resulted in AIM being unable to execute its deliverables. This forced IBPAP to complete the development of a viable business plan for the knowledge hubs in cooperation with the Asia Pacific College.

(ii) Tourism TA. This TA adhered to timely implementation and achievement of intended outcomes. The first consultant was fielded after about 6 months and thereafter the activities were on track. The TA was extended by 6 months to help DOT draft a concept paper to support national rollout of a skills grant program similar to ISGS. The extension

39 Based on the 2015 State of Industry Report of the Association Talent Development (ATD), organizations spent an average of

$1,252 per employee on training and development initiatives. ATD is the world’s largest professional membership organization supporting those who develop knowledge and skills, improve performance, and help achieve results for the organizations they serve. Its members are drawn from more than 120 countries and work in public and private organizations in every industry sector.


was reasonable to maximize the use of resources and benefits of the project while improving sustainability of the intervention. The tourism industry was a well-organized and coordinated industry, and that also helped in facilitating project implementation. The process is assessed efficient.

(iii) At-Risk Youth TA. This TA suffered no significant delays in implementation. Consultants were immediately fielded upon signing of the TA. By mid-implementation (November 2015), the TA had completed the scheduled registrations, career guidance, LST, and job placements in all 4 pilot PESOs and expanded to 10 other PESOs. The TA was extended by 6 months to finalize the follow-on TA for the national rollout of JSP,40 which is a reasonable delay. In view of the efficient use of resources in achieving the intended outcome and with no major setback in implementation, the TA is assessed efficient.

(iv) JobStart Rollout TA. In this TA, several administrative issues that could negatively impact efficiency of implementation were noted. First, delays in disbursement of funds sourced from government coffers led to service providers not being contracted in time, which in turn led to significant number of youth registrants dropping out from the program.41 Second, there were instances when the memoranda of agreement with employers were not ready in time for deployment to the technical skills training and internship phase. Third, many of the JobStarters did not complete the full cycle because they found the PHP200 allowance for LST and 75% of the minimum wage allowance not sufficiently attractive. This caused between 45% and 68% of LST finishers to drop out during the technical training and internship between 2017 and 2019.

D. Sustainability 45. IT-BPO TA. The TCR assessed the TA’s output likely sustainable, as strong frameworks were supposedly in place to allow regular updating of courses and solicit continued input from local industry partners. The TCR also posited that university and industry partnerships should continue to be relevant in providing increased opportunities to students and a more appropriately skilled workforce for the companies. After TA completion, UPOU was able to establish a knowledge hub. It continues to offer courses but requires additional support to update and maintain the modules to meet the rapidly changing needs of the industry. On the demand side, there was low demand for the e-learning modules as the IT-BPO industry’s capacity has grown considerably to the point where firms have started directly hiring undergraduate candidates and then providing them on-the-job training to fill in the talent gap. 46. The evaluation deems that the results of institutional capacity development are unlikely to be sustained without further support from ADB. The TA helped strengthen IBPAP’s institutional capacity in project management during implementation, but the team in charge of the TA was disbanded after its completion and knowledge was not effectively transferred to those who remained. Furthermore, the TA design did not include support for institutionalization of monitoring and evaluation system within IBPAP or DOF. This can be attributed to a lack of leadership and mandate on the part of DOF. To sustain the TA envisaged outcomes, it would have been vital to embed the skills development initiatives within an institutional framework. Neither IBPAP nor DOF could provide the necessary framework, however, and that contributes to the TA’s outcome being less than sustainable. 47. Tourism TA. Since completion of the TA, the updated hotel and resort quality assurance and accreditation system has been set in place as well as the institutional arrangements to implement the system. This is evidenced by the revised national accommodation standards that were aligned with international standards. The RIA component has also become embedded within DOT’s existing policy

40 ADB. 2015. Technical Assistance to the Philippines for Nationwide Rollout of JobStart Philippines. Manila. 41 The LSTs could not be implemented immediately because the LGUs had not received funds yet, causing youth registrants to lose

interest in the program because of waiting time they could have spent searching for jobs. During implementation of the rollout from 2017 to 2019, the initial drop-out (those that registered for JSP and eventually enrolled in LST) ranged from 15% to 27% (See Appendix 6).


framework through the Office of Tourism Development Planning, Research and Information Management. This office continues to conduct consultations and workshops on the use of RIAs with various stakeholders for better quality of regulations in the industry. In August 2014, the DOT Secretary issued an order enjoining the department and attached agencies to adopt the RIA manual developed under the TA. Following the training conducted on the RIA, the National Economic and Development Authority began the process of developing RIA guidelines for the whole government. This role was later taken over by the Anti-Red Tape Authority (ARTA) when it was established in 2018.42 Since its inception, ARTA has been drawing heavily on DOT’s RIA manual and experience in developing and incorporating RIAs within the government policy framework.

48. With regards to institutional capacity for skills development, the TA helped form the foundation for the Office of Industry Manpower Development within DOT, which had not existed prior to the TA. This office enables DOT to have stronger focus on systematic development of human resource within the industry. It periodically updates and implements the human resource development strategy and action plan as well. As for the tourism sector’s overall capacity, DOT’s campaign to develop tourism-specific skills resulted in the allocation of PHP100 million ($2 million) per year by the Department of Budget and Management to support national rollout of a follow-up program from 2017 to 2019.43 The ISGS piloted under the TA was renamed the Tourism Industry Skills Program (TISP), which brought about certain changes in compliance with the shift in the overall policy direction from the new 2016 administration. The changes introduced in the TISP were threefold: (i) allocation of an equal amount of budget per region, regardless of each region’s capacity, need, or performance; (ii) removal of cost sharing by the employers as a prerequisite for the grant application; and (iii) main focus on number of trainings completed and participating trainees. The resulting TISP saw the innovative aspect of the TA’s ISGS significantly undermined as it incentivized regional DOT offices to implement more low-cost, low-tech basic trainings rather than offering customized, cutting-edge trainings based on the needs of the employers.

49. In summary, the institutional and implementation frameworks for quality assurance and accreditation as well as for regulatory reforms using the RIA have been set in place. The Office of Industry Manpower Development was established within DOT to support continued skills development of the industry. These institutional frameworks introduced by the TA are likely sustainable. Although certain elements of innovation on skills development introduced under the TA have been undermined by the changes introduced in the TISP, these policy shifts can still be taken as evidence of flexibility within the DOT to keep up with ever-changing government policies and to sustain its efforts at developing tourism skills. In view of the sustained policy and institutional framework along with the continuing efforts at capacity development in the tourism sector, the evaluation assessed the TA most likely sustainable.

Box 4. International Good Practices on Upgrading Skills of Workers

• France: National and particularly regional governments provided generous funding to help enterprises train or retrain workers, often in combination with reduced working hours but without loss of salary, as an alternative to retrenchment. A Social Investment Fund financed by the state, the European Social Fund (EUR 5 billion), and social partners (EUR 500 million) was set up to finance measures promoting employment of young people, enabling workers made redundant to reenter the labor market, and facilitating access to vocational training.

• Germany: If the employer devotes down time to staff training, the government reimbursed up to 100% of the employers’ social security contributions.

• Canada: Individual training plans range from upgrading skills in current jobs, preparing for promotions, and even training for jobs outside the company. Workers remain employed—helping retain aggregate demand in hard-hit communities—and acquire new skills, while employers are able to retain staff and avoid having to train new workers when markets pick up.

Source: International Labour Organization. 2010. A Skilled Workforce for Strong, Sustainable and Balanced Growth: A G20 Training Strategy. Geneva.

42 Republic Act 11032. An Act Promoting Ease of Doing Business and Efficient Delivery of Government Services, by Amending

Republic Act No. 9485, Otherwise Known as the Anti-Red Tape Act of 2007, and for Other Purposes. Manila. 43 In 2020, the allocation was reduced to PHP80 million due to the government’s other priorities.




50. At-Risk Youth TA. Toward building capacity for labor policy analysis and formulation, DOLE-ILS staff was trained in the use of RIAs and applied the methodology to successfully formulate nine RIAs through this TA. Though there is still a portion of the original group of core staff within ILS that have been trained and is able to do the RIAs, there was no clear structure established within DOLE to sustain the knowledge. Also, there was no champion within DOLE to provide leadership and help RIA processes become mainstreamed during policy and regulatory formulation. Due to changes in administration and priorities, RIA has not been sustained in DOLE, making the intervention less than likely sustainable.

51. The pilot phase of JobStart was successfully completed, then expanded to 10 additional PESOs. It also led to an agreement with DOLE for the national rollout. The follow-up TA, Support for the Nationwide Rollout of JSP, was formulated to build upon the gains of the pilot and reach more beneficiaries. In 2016, the government committed to institutionalizing JSP through enactment of the JSP Bill and allotted PHP106 million for the nationwide rollout.44 To date, JSP has been rolled out to 35 of more than 1,500 LGUs in the country (Appendix 7). These developments contributed to sustainability of the project outcomes. Considering the relative weight of the pilot Jobstart, the ongoing follow-up TA, and enactment of the JSP Bill, the TPER assessed the TA likely sustainable.

52. JobStart Rollout TA. In line with the national rollout and passage of the JobStart law, JobStart was institutionalized within DOLE’s institutional framework at national and regional levels, with focal persons assigned at field level. JSP was mainstreamed in the employment program of DOLE and the institutional structure established provided the channel for long-term implementation. The evaluation nevertheless noted issues that could negatively affect sustainability of the program. First is the political risk posed by sudden shifts in priorities and human resource complement that often can be seen in LGUs following an election. Such events have resulted in significant loss of institutional memory concerning ADB TA and related innovations. This could be attributed to the lack of an enforcement system for effective handover of knowledge among LGU personnel.45 Second, it was observed that despite efforts of the project management office, many of the PESO LGUs still lack capacity in implementing follow-up programs and are heavily dependent on external service providers. This has been further aggravated by the LGUs’ dependence on temporary, casual, or “job order” workers; lack of institutionalized PESOs; and lack of local budgetary support for JobStart, all of which could diminish sustainability of the nationwide rollout. 53. Based on overall feedback received during the FGDs at various PESOs and analysis of the official performance data from DOLE (Table 7), it was evident that LST was the single most effective intervention within the JSP at helping at-risk youths get hired. As such, ADB should consider concentrating on developing the LST instruction capacity within DOLE and PESOs to ensure maximum impact and sustainability after TA completion. Without additional efforts to institutionalize both PESOs and JSP while streamlining the intervention, the TA would likely be less than sustainable.

Table 7: Ratings of the Job Creation and Skills Development Technical Assistance Projects

Variables

% Employed out of LST % Employed after TTI Statistical Significance Obs. Mean Std. Err. Obs. Mean Std. Err

Employment rate for all JSP 81 36.4% 2.1% 81 21.7% 2.3% ***

Employment rate for pilot 12 47.0% 5.1% 12 12.6% 4.0% ***

Employment rate for rollout 69 34.5% 2.3% 69 23.2% 2.6% **

44 Republic Act No. 10869. 2016. An Act Institutionalizing the Nationwide Implementation of JobStart Philippines. Manila. 45 The Department of the Interior and Local Government circulated Memorandum No. 2019-39 (13 March 2019) requiring local

governance transition teams to ensure the smooth transition by gathering, securing, and preserving all official documents and/or records of LGU official transactions. This memo has not been diligently complied with, however, and this has contributed to extensive loss of knowledge in LGUs that have not been institutionalized.


JSP = JobStart Philippines, LST = life skills training, Obs. = number of observations, Std. Err. = standard error, TTI = technical training and/or internship. *, **, *** = difference is statistically significant at the 90%, 95%, or 99% level, respectively. Source: Department of Labor and Employment, JobStart.

CHAPTER 4

Other Assessments

A. Impact 54. The impact of the three completed TA projects is assessed in this section. The impact of the JobStart Rollout TA was not assessed or rated because it is ongoing. 55. Impact of IT-BPO TA. The envisaged impact was for the Philippine education system to score at least 4.50 on the Institute for Management Development World Competitiveness Online Database, up from 4.03 in 2009, by 2018. This was achieved with the score of 5.24 in 2018. This evaluation notes, however, that the target was a higher order indicator that can be influenced by many external factors. The shortfall in establishing viable knowledge hubs and the low demand for online courses reduced the development impact of the TA. Also, the accreditation program developed for the eSMPs had minimal impact on the industry. Of the 900 teachers targeted, only 379 (42%) faculty members received certification using online teacher training programs. Additional training and employment opportunities for students were opened through partnerships between HEIs and industry, but the extent of job placement was not clear due to lack of monitoring by either DOF or IBPAP.

56. Nonetheless, the need to fill the talent gap in the industry was picked up by other HEIs. For example, the Asia Pacific College took some of the lessons learned from the TA and is currently developing plans to roll out eSMP courses to be taught in malls across the country. Another example is the Carlos Hilado Memorial State College in Negros Occidental, which has continued to offer the service management program courses since 2016. During 2016 to 2019, the college produced 1,795 graduates of the program. Despite these unintended benefits of the TA to IT-BPO, the evaluation assessed the TA’s impact to be less than satisfactory.

57. Impact of Tourism TA. The expected impact of the project was increased employment in the tourism industry above the long-term average annual growth of 3.5% by 2020. During 2013 to 2018, employment in tourism grew at an annual average of 2.8%, which was lower than the target. It should be noted, however, that change in employment growth within the industry cannot be attributed to the project alone, as there are several factors that affect the number of jobs. In terms of its contribution to total employment, the share of tourism steadily increased from 12.1% in 2012 to 13.0% in 2018 (Appendix 8). During 2016 to 2019, inbound visitors grew by an average of 11.4% (Appendix 9. In terms of its contribution to the economy, tourism as a proportion of gross domestic product grew from 7.9% in 2012 to 12.7% in 2018 (Appendix 8).

58. In terms of institutional development, the TA helped develop the capacity of the DOT by building the foundation for the Office of Industry Manpower Development, which institutionalized human resource development within DOT. The support provided to DOT in developing RIAs assisted in improving regulations that reduced regulatory compliance costs for those RIAs that were implemented. Despite the difficulty in implementation, the RIAs have proven invaluable for the government at large, especially as this has become a valuable tool for evidence-based policymaking at all governmental levels. At the LGU level, in Puerto Princesa, the number of days for processing business licenses for travel and tour agencies was reduced by 50%, while in Samal Island Garden City, officials have used RIA to improve an existing environmental user’s fee scheme that had not been updated for more than a decade. At national level,


ARTA has been drawing heavily on the DOT’s experience to incorporate RIA into the government’s policy on anti-red tape and ease of doing business. 59. The TA had unintended positive impacts on the industry. In Davao, various tourism associations were formed to facilitate TA implementation and managed to remain active in their role well into the implementation of DOT’s TISP. The networks formed among the employers in the associations brought about sustained impacts that include peer learning through knowledge sharing, sharing of able trainers through the peer networks, and reduced costs and risks of skills training through economies of scale. At the national level, DOT, by coordinating closely with the tourism associations, managed to squeeze more effective skills training programs from TISP’s comparatively limited budget and streamlined documentation requirements. This effectively alleviated one of the major sources of employers’ discontent during ADB’s ISGS implementation. Considering these benefits and the impact that has been sustained beyond the TA, the evaluation assessed the TA’s impact satisfactory.

Box 5. Philippines: Transforming South Palms Resort Panglao South Palms Resort Panglao is a 90-room beach resort within the 5-kilometer stretch of the white-sand Alona Beach. Prior to Asian Development Bank’s (ADB) technical assistance, the resort was saddled with negative online reviews and ranked significantly lower in TripAdvisor compared to its comparable competitors. In 2014, occupancy was below 50%, which negatively affected the resort’s revenue stream. Under the ADB-supported Industry Skills Grant Scheme (ISGS), South Palms completed two batches of skills training, significantly improving its food offering and the quality of service. To ensure that the lessons learned were at par with international standards, both training sets used materials developed specifically for Tourism Professional Certification Boards and National Tourism Professional Board of ASEAN Member States. After the trainings South Palms saw immediate rise in its occupancy rate to 58% in 2015, 78% in 2016, steadily increasing to 84% and 85% in 2017 and 2018, respectively, and peaking at 90% in 2019. Success in training, however, also contributed to high attrition, as 60% of those trained left for better paying jobs in other cities and countries. Those that remained were recognized for their developed skills and were promoted, as they demonstrated high levels of loyalty, satisfaction, expertise, and eagerness to share their knowledge with younger, less experienced colleagues. As such, most of them went on to take supervisory positions and even assisted in developing in-house training manuals for the resort’s human resources department. Source: ADB Independent Evaluation Mission; South Palms Resort; and University of Queensland. 2017. Final Report on Improving Competitiveness in Tourism. Manila.

60. Impact of At-Risk Youth TA. The intended impact of increasing youth employment was not achieved. During 2010−2015 youth full-time wage employment increased by 1.3% per annum,46 which was less than the target of 5.0%. This evaluation is of the view that the target was too ambitious given the size and scope of the TA. There are too many external factors beyond the TA scope that could affect youth employment.

61. Another impact indicator is the compared job placement rates of youths that received JobStart services with a “control” group that did not receive the same services. The job placement rates for the control group were not monitored, making it impossible to accurately measure. An impact evaluation using a randomized control trial was performed after the TA implementation. Its findings indicated that the JobStart pilot program significantly increased income levels but did not appear to have influenced employment levels significantly.47 Furthermore, the pilot program’s impact was complemented by other aspects of training in that beneficiaries were likely to have taken additional training other than from the

46 Based on data from the Philippine Statistics Authority, Labor Force Survey. 47 D. Jayasuriya. 2016. Impact Evaluation Study: PHI-Employment Facilitation for Inclusive Growth. Manila.

Other Assessments 29

program, believed that training was relevant to their jobs, and had greater satisfaction out of their job choice.48 62. Without JobStart, it took an average 3 years for high school graduates to find a job while this required only a year for college graduates. After going through JobStart, however, it took the beneficiaries an average of 10 days to a maximum of 6.3 months to find their first jobs.49 This indicates a significant impact of the program on facilitating employment of at-risk youth and in building their confidence when entering the job market. On the other hand, on the technical training provided to JobStarters by employers, both DOLE and PESOs were overwhelmed by the basic logistics of JSP and could not effectively control the quality or standardize the training. Despite efforts by DOLE to ease the burden on employers in coming up with training plans, the process of preparing these was often too difficult for small and medium-sized enterprises (SMEs), while neither DOLE nor PESOs had the technical expertise to review or assess the submitted plans. In hindsight, including the Technical Education and Skills Development Authority into the initial project design would have enhanced the quality of technical training and eased technical certification of employees, thereby further maximizing the impact on skills development through JSP.

Box 6. International Good Practices on Certification and Quality of Training

• European Qualification Framework (EQF): The EQF is a “translation grid” between European Member States’

qualification systems to help employers and workers better understand European Union (EU) citizens’ qualifications. Intended to support mobility and lifelong learning, the EQF establishes equivalences between qualifications and certificates obtained in different countries. It is a reference tool for both employers and workers when comparing the qualification levels of different countries and various education and training systems. The EQF includes eight broad categories of skills or “reference levels,” ranging from basic to the most advanced. Each category includes descriptions of what workers should know and be able to do, regardless of where their diplomas, qualifications, or certificates were obtained. In part, this responds to the particular circumstance of opening labor markets at the Europe-wide level, where individual institutions’ reputations for quality of training are not known across the broader EU.

• Strategic Plan for Continuous Education, Innovation, and Employment, Argentina 2020: Argentina’s plan

focuses on improving training at the enterprise level through uses of its Tax Credit Regime to incentivize small and medium-sized enterprises (SMEs) into investing in training workers. SMEs can finance training projects up to the equivalent of 8% of the sum of total remuneration. They can also be reimbursed for costs incurred for skills assessment and certification in addition to actual training, creating an incentive to boost recognition of skills learned informally or on the job. This feature helps make the program (begun in 2007) attractive to SMEs, which comprise 70% of the beneficiaries.

Source: International Labour Organization. 2008. Skills for Improved Productivity, Employment Growth and Development, International Labor Conference, 97th Session, Report V, Geneva.

63. In terms of institutional impact, the project helped strengthen the capacity for labor policy analysis and formulation of DOLE with nine RISs produced and staff trained on RIA processes and tools. Institutionalized PESOs were able to set in place labor market information and employment intermediation systems with the development of the PEIS. That system made labor market information available to the LGUs in an online platform. Nevertheless, PESOs’ capability to carry out the complete cycle of JobStart employment facilitation independently remains a challenge. Many of them lack adequate capacity to implement JSP without support from the project. Although the TA had significant institutional development impact on DOLE and positive effects on Jobstarters and employers, most of the original impact targets were not achieved. As such, the TPER assesses the TA impact less than satisfactory.

48 The study was limited by low statistical power of less than 0.80 due to the low proportion of control group participants in the

sample. Hence, the probability of concluding that there is no effect, when in fact there is, increases. 49 Based on official employment data from Department of Labor and Employment, JobStart.


B. ADB and Executing Agency Performance

64. Asian Development Bank. In the case of the IT-BPO TA, ADB should have done a more rigorous feasibility study to better identify the risks in implementation and perhaps formulate a better design. Oversight and monitoring of the TA was weak, as consultants relied heavily on their understanding of the TA even as ADB relied heavily on the capacity of consultants who were experts only within their silos. ADB should not have committed to the TA without identifying a lead executing agency that could provide expertise and guidance for the IT-BPO sector. This would have been the Department of Information and Communication Technology, which was established in 2016.50 Lastly, ADB should have more thoroughly reviewed the DMF to determine if the targets and expected outcome were realistic and achievable given the resources, timeline, risks, and constraints. As such, the evaluation deems ADB’s performance for the TA less than satisfactory.

65. In the other three TA projects, ADB’s performance was assessed satisfactory. ADB engaged well with the executing agencies and was responsive to issues during implementation. It established strong relationships with the counterparts in DOLE and DOT while engaging closely with the tourism industry. Supervision during implementation was adequate and periodic review missions were carried out to monitor progress and issues. Analytic and diagnostic work during formulation was adequate. In the case of the JobStart Rollout TA, ADB strove to incorporate most of the lessons learned from the pilot.

66. Executing Agencies. The performance of DOF as executing agency for the IT-BPO TA was assessed less than satisfactory, as DOF’s performance was found to be inadequate in monitoring TA implementation, encouraging cooperation among agencies, and delivering the expected outcome and outputs. In the other TA projects, the executing agencies’ performances were assessed satisfactory. The DOT demonstrated strong leadership and ownership of the project, as it had coordinated effectively with stakeholders to ensure cooperation within the tourism industry while its officers and staff assigned to the project demonstrated commitment to the aspirations of the project. DOLE also demonstrated strong ownership of the two JobStart TA projects that it executed. DOLE engaged well with stakeholders, including the PESOs that were the implementing agencies of JobStart.

50 Republic Act No. 10844. An Act Creating the Department of Information and Communications Technology. 20 May 2016. Manila.

CHAPTER 5

Overall Assessment, Issues, Lessons, and Follow-Up Actions

A. Overall Assessment

67. The summary ratings for the four TA projects are presented in Table 8. Of the three completed TA projects, only the IT-BPO TA was assessed less than successful, while the others were rated successful. The JobStart Rollout TA was assessed only for relevance and effectiveness as it is ongoing.

Table 8: Ratings of the Job Creation and Skills Development Technical Assistance Projects

Evaluation Criteria

Strengthening Knowledge-Based Economic

and Social Development

Improving Competitiveness

in Tourism

Employment Facilitation for

Inclusive Growth

Support for the Nationwide Rollout

of JobStart Philippines

TCR TPER TCR TPER TCR TPER TPER

Relevance Not rated Less than relevant Relevant Relevant Relevant Relevant Relevant

Effectiveness Not rated Less than effective Effective Effective Effective Effective Effective

Efficiency Not rated Less than efficient Efficient Efficient Efficient Efficient

Sustainability Likely

sustainable Less than likely

sustainable Most likely sustainable

Most likely sustainable

Most likely sustainable

Likely sustainable

Overall Rating Less than successful

Less than successful Successful Successful Successful Successful

Development Impact Not rated

Less than satisfactory Satisfactory Satisfactory Satisfactory

Less than satisfactory

Performance of recipient and executing agency Satisfactory

Less than satisfactory Satisfactory Satisfactory Satisfactory Satisfactory

Performance of ADB

Partly satisfactory

Less than satisfactory Satisfactory Satisfactory Satisfactory Satisfactory

ADB = Asian Development Bank, TCR = technical assistance completion report, TPER = technical assistance performance evaluation report. Source: Independent Evaluation Department.

68. Potential and limitation of TA projects. On the overarching question of the TPER, the evaluation observed that potential for TA success was very much dependent on external factors such as sustained public and private investment in the sector, established technical and administrative capacity in the local institution, and strong political leadership within the government counterpart agencies. On the other hand, TA projects should be much more grounded in solving specific issues. Some of the goals in these cases were overambitious, making it difficult to attribute many of the higher-level outcomes directly to the TA projects as described in their DMFs. A TA project should be designed based on a clear theory of change with realistic outcomes and with the aim that it serves as a testbed for piloting innovations and raising local institutional capacity to ensure that any successful innovations could be expanded upon afterwards, either through local governments’ own initiatives or supported by new loans.


69. Shared innovation among the TA projects. Despite the inconsistent performance and outcomes of the TA projects, they should be commended for demonstrating the impact of a shared innovation: giving the employers a greater role in the planning stage of skills training. It had been generally noted that most teachers and trainers face the same problem: they tend to teach what they know instead of what their students need to learn. To counteract this, all four TA projects were designed to involve the employers in the planning and management of the training programs. That helped keep the trainers abreast of the changing technologies and equipment in use at the workplace, as well as to track what occupations and skills are declining or rising in demand.

B. Issues 70. Procedural inefficiencies persist in ADB’s TA implementation, as not enough has been done to streamline lending to LGUs. The Country Partnership Strategy Final Review Validation, 2011−2016 for the Philippines noted that ADB should explore new ways to resolve the existing challenges in effectively delivering traditional project loans, and especially when these are channeled through multiple government agencies or LGUs,51 as were the cases for most of the TA projects reviewed. The evaluation noted that, except for the case of the IT-BPO TA, most beneficiaries of the related TA projects attributed high dropout rates among both trainees and employers to delays in training, usually caused by slow disbursement of the TA project funds by the central government agencies. ADB needs to think beyond simply providing support for nationally managed TA projects and instead begin preparing the groundwork for streamlined, direct financing mechanisms for LGUs. 71. Appropriate certification of training courses was not incorporated into the project design of the Tourism TA and the At-Risk Youth TA, affecting the TA projects’ effectiveness and sustainability. Despite efforts to ease the burden for employers in coming up with training plans for both JSP and ISGS, without standard templates or professional support, the learning curve proved to be too steep for SMEs, especially those in rural areas. Also, without standardized training courses, JobStarters or employees ran the risk of wasting time and effort undertaking non-standard training that would prove useful to only a very narrow set of industry employers. Moreover, without standardized training courses, JobStarters or employees ran the risk of limiting themselves to a very narrow slice of the job market. The Technical Education and Skills Development Authority, as the government agency most qualified and best equipped to support both the LGUs and the employers in ensuring quality technical training, should have been given a greater role in supporting and defining the standards for the technical trainings in the TA projects. That would have maximized the effectiveness and sustainability of the technical skills training support provided by the TA. Closer cooperation with the Technical Education and Skills Development Authority during the training planning phase of the TA projects could ensure (i) more consistent, higher-quality technical training; (ii) greater likelihood of employees acquiring national certification upon completion of training; and (iii) less administrative burden for DOLE, DOT, and employers of the trainees. 72. Inadequate monitoring of key indicators during implementation made it difficult for performance measurement, thereby undermining the processes of learning and improving the programs while undertaking them. TCRs were deficient in reporting the performance indicators as identified in the DMFs, and some even failed to report detailed ratings for the TA projects. This could be attributed to a lack of actual monitoring of the DMF indicators during TA implementation cycle. In preparing future TCRs, reporting of these indicators and ratings thereof should be required to enable objective assessment of project performance. Furthermore, the formulation and selection of the performance indicators for the DMF should take into consideration the feasibility of the target as well as practicality of monitoring in order to maximize the accuracy of performance measurement as well and the chances for learning through evaluation.

51 IED. 2017. Philippines: Country Partnership Strategy Final Review Validation, 2011–2016. Manila: ADB.

Overall Assessment, Issues, Lessons, and Follow-Up Actions 33

C. Lessons

73. Table 9 summarizes the lessons for job creation and skills development as noted for each TAs, which are included in Appendix 1. The paragraphs following the table contain universally applicable lessons for the sector.

Table 9: Summary of Lessons from the Job Creation and Skills Development TA Projects

TA Project Lessons

IT-BPO TA

a. Clear governance and leadership are essential for successful TA implementation.

b. Feasibility survey can reduce the risk of an innovative intervention’s failing. c. TA project planning should be realistic when it comes to resources and time.

Tourism TA

d. Outcomes of capacity development for LGUs can be most sustainable. e. Spillover from capacity development could bring about significant long-term impact.

f. Provision of career path is essential for retaining trained personnel.

g. Innovation needs to be flexible to stay pertinent. h. Shift in the government’s policy could undermine the TA’s innovations. i. Institutionalization of innovation is needed for resilience.

j. Support for independent business owners needs to be incorporated into the TA. k. Networking and empowering of local communities are key to sustainable intervention.

At-Risk Youth TA and JobStart Rollout TA

l. Institutionalized PESOs are needed as focal points for intervention. m. Coordinated public relations campaigns need to be included in future interventions.

n. Consistent monitoring of the beneficiary performance is needed to promote accountability.

o. Greater incentivization is needed for small and medium-sized enterprise employers. p. Improvement is needed in the process for identifying potential JobStarters.

q. LST-centric evolution of the JSP should be actively explored.

r. Strong ownership and quality control are needed for LST. IT-BPO = information technology and business process outsourcing, JSP = JobStart Philippines, LGU = local government unit, LST = life skills training, PESO = Public Employment Service Office, TA = technical assistance. Source: Independent Evaluation Department. Appendix 11.

74. The innovativeness of TA cannot make up for a lack of preparedness or capacity of the developing member country agencies involved. According to ADB’s Country Partnership Strategy: Philippines, 2018−2023, ADB will “pilot test an enterprise-led learning network project—an innovative financing model for training workers in sectors with skill shortages through enterprise-led models.”52 Noting that this was exactly what the less than successful IT-BPO TA had set out to do in 2012, ADB should take extreme care not to repeat the same mistakes just for the sake of being innovative. As the initiative in the partnership strategy aims to “contribute to programs dealing with the fourth industrial revolution,” ADB should verify the soundness of the government’s policy framework and long-term strategy in any pertinent nascent sector, carefully examine the feasibility of public–private partnership arrangements in education and labor market programs, and especially make sure that both private and public sectors are ready and willing to cooperate for better delivery of concrete and verifiable outputs and outcomes. 75. Clear implementation arrangements with executing and implementing agencies are essential for the success of skills development projects. Implementation arrangements should have clear institutional roles and responsibilities, and the executing agency should have a clear and strong mandate for monitoring and enforcing these roles. Strong leadership by the executing agency that is supported by technically capable implementing agencies is vital for successful implementation of a project. This is illustrated by the less than successful implementation of the IT-BPO TA under the leadership of DOF. 76. Poor identification of relevant stakeholders can limit ADB’s engagement with them, critically affecting the project’s effectiveness, participation and support of local businesses, and access to the domestic job market. For the IT-BPO TA, there was no government agency with a clear mandate for the

52 ADB. 2018. Country Partnership Strategy: Philippines, 2018−2023 – High and Inclusive Growth. Manila (p. 11).


sector and too many implementing agencies were brought in to fill the vacuum with their various goals, which led to conflicting agendas, adding to overall inefficiencies. On the other hand, the other three TA projects did not identify sufficient partners outside DOLE, DOT, and LGUs. That meant the projects failed to take advantage of networks that could have helped raise public awareness on the innovations or ensure quality technical training. 77. Overly ambitious project design can cause the implementation of interventions at inappropriate levels, resulting in suboptimal outcome for the project. Initiating and embedding RIA within the existing DOLE and DOT frameworks and within the timespan of the TA projects was quite an ambitious undertaking. Considering that RIAs are administratively and technically challenging, the project timeline might have been too short. Also, most of the RIAs formulated by DOLE and DOT were not fully implemented, because implementing the recommendations of the RIAs was often beyond the mandate of each department. In contrast, the recommendations from RIAs at LGU level, such as in Puerto Princesa and Tagbilaran, were mostly implemented because they required passage of local ordinances within the purview of the LGUs. Reflecting upon how ARTA has been benchmarking RIA methodology from DOLE and DOT, it might have been overly ambitious to include RIA into the TA projects. 78. Processes need to be simplified to minimize inefficiencies in scaling up pilots. Documentation and reporting requirements for training participation were viewed as burdensome and presented an entry barrier to ADB TA projects, discouraging potential employers and causing some to withdraw from the program. This was especially true for SME employers who participated in ISGS or JSP and which had limited personnel in their finance and human resource departments. Although executing agencies and ADB might have been able to support and push a few dozen beneficiaries across the finish line during the pilot implementation, one cannot expect large-scale rollout to be successful without first streamlining and simplifying administrative procedures and requirements for the beneficiaries and implementing partners. 79. Concerted effort is needed to raise public awareness about key innovations in skills development projects. Advocacy and information dissemination campaigns should have been built into the designs of the TA project to deepen its outreach. For example, many of the JobStarters were contacted not because they were the most in need but because they had greater access to information such as social media, chat groups, and personal networks. Project officers should support the PESOs in (i) diversifying channels of promotion to include social media plus various networks and associations; (ii) seeking out and proactively communicating with unemployed, at-risk youths; (iii) actively networking with local employers and promoting participation in JSP; and (iv) sharing and cross-referencing potential candidates for the various skills development and employment support programs within the PESO.

D. Follow-Up Actions 80. Lessons from previous evaluations should feed into scale-up efforts in similar sectors or regions. In large TA projects incorporating innovations to be scaled up or rolled out at a nationwide scale, design formulation should incorporate a rigorous impact evaluation component to determine the actual impact of the program on beneficiaries. More importantly, key findings from TA projects should feed into the design of the follow-up interventions, especially for large scale-ups like the JobStart Rollout TA. Although an impact evaluation using randomized control trial was included in the At-Risk Youth TA and concluded that “JSP significantly increased income levels but didn’t significantly improve employment levels,” the JobStart Rollout TA was already in mid-implementation and could not incorporate appropriate design revisions to improve employment levels for the youths as originally envisaged. Even though timely feedback was impossible due to the JobStart Rollout TA’s project timeline, future implementations, or expansion of JobStart should actively consider significant revisions of the JSP model in light of both the impact evaluation and this TPER.

Overall Assessment, Issues, Lessons, and Follow-Up Actions 35

81. Institutionalization of the LST module should be promoted to maximize the impact and sustainability of the JobStart TA projects. Among the several components within JSP, LST was repeatedly mentioned by many stakeholders as being the most effective and innovative. There were PESOs that succeeded in getting their JobStarters employed without the technical skills training and internship. This low-cost variation of “LST-only training” for JSP has been successfully adopted in Taguig and General Trias, both of which are highly developed urban centers with ready pools of well-developed employers. In these areas, not only did the employers prefer to hire LST graduates as soon as possible but they also had the capacity to provide in-house technical training without further support from the PESOs. It was also noted that JSP is just one of many programs in a PESO’s portfolio supporting skills development and job training. As LST is a unique innovation introduced within the JSP, however, it runs the highest risk of being discontinued in PESOs at the end of the TA project cycle or JSP. This risk could be mitigated if the LST module will be adopted by government agencies such as DOLE. The government can further expand upon it by developing a localized LST module, spearheading the regular update and revision of LST training manuals, hosting and implementing the training of trainers programs, ensuring the quality control of LST programs, and acquiring funding to roll out the LST at a truly nationwide scale.

Appendixes

APPENDIX 1: FOCUS GROUP DISCUSSIONS

Date Location Group Type Number of Participants

10 Feb 2020 Panglao, Bohol Tourism-LGU 2 Tourism (Employers) 1 Tourism (Employees) 9 11 Feb 2020 Tagbilaran, Bohol JSP-PESO LGU 5 JSP (Employees) 9 JSP (Employers) 6

12 Feb 2020 Cebu City, Cebu Tourism-DOT 3 JSP-DOLE 3 JSP-PESO LGU 6 JSP (Employers) 2 JSP (Employees) 1

13 Feb 2020 Lapu-Lapu, Cebu Tourism-LGU 3 Tourism (Employers) 8 Tourism (Employees) 1

17 Feb 2020 Davao City Tourism-DOT 4 Tourism-LGU 4 Tourism (Employers) 6 Tourism (Employees) 7

18 Feb 2020 Tagum City, Davao del Norte JSP-PESO LGU 5 JSP (Employers) 4 JSP (Employees) 4 Panabo City, Davao del Norte JSP-PESO LGU 4 JSP (Employers) 5 JSP (Employees) 8

19 Feb 2020 Davao City, Davao JSP-DOLE 8 JSP-PESO LGU 10 JSP (Employees) 5 JSP (Employers) 5 20 Feb 2020 Puerto Princesa, Palawan JSP-PESO LGU & DOLE 4 JSP (Employees) 5 JSP (Employers) 5

21 Feb 2020 Puerto Princesa, Palawan Tourism-LGU 3 Tourism (Employees) 2 Tourism (Employers) 3

24 Feb 2020 Manila, National Capital Region JSP-DOLE 3

Quezon City, National Capital Region

JSP-PESO LGU 4

JSP (Employees) 2 JSP (Employers) 1

26 Feb 2020 Taguig City, National Capital Region

JSP-PESO LGU 4

IBPAP 3

27 Feb 2020 General Trias, Cavite JSP-DOLE 2 JSP-PESO LGU 5 JSP (Employees) 2 JSP (Employers) 3 28 Feb 2020 San Fernando, Pampanga JSP-DOLE 5 JSP-PESO LGU 6 JSP (Employees) 6 JSP (Employers) 8 Total 48 Focus Groups 214

DOLE = Department of Labor and Employment, DOT = Department of Tourism, IBPAP = IT & Business Process Association of the Philippines, JSP = JobStart Philippines, LGU = local government unit, LST = life skills training, PESO = Public Employment Service Office. Source: Independent Evaluation Department.

APPENDIX 2. DESIGN AND MONITORING FRAMEWORKS OF TECHNICAL ASSISTANCE PROJECTS

Table A2.1 Design and Monitoring Framework for TA 8164—Strengthening Knowledge-Based Economic and Social Development

Design Targets

Technical Assistance Completion Report

(July 2018) Independent Evaluation

(March 2020) Impact Improved alignment of higher education programs with needs of the economy

Philippine education system scoring at least 4.50 on the Institute for Management Development World Competitiveness Online Database by 2018, up from 4.03 in 2009

Not reported. Achieved. Philippine education score by the Institute for Management World Development was 5.24 (2018).a However, impact indicator cannot be solely attributed to the project.

Outcome

A replicable and sustainable system of ICT-enabled knowledge hubs developed for the improved teaching and learning of IT-BPO industry-designed curricula in HEIs.

IT-BPO industry-designed courses adopted in at least 3 HEIs by Q3 2015.

Partly achieved. Number of designed courses adopted in HEIs not reported. UPOU has developed a strong repository of digitized business communication and service management courses, and the knowledge hub provided the framework for sustainability of these courses and continued alignment with the needs of the IT-BPO industry.

Partly achieved. Indicator not closely monitored, but there are indications that industry-designed courses were adopted in HEIs. UPOU conducted both face-to-face and online models with teachers from 25 schools. It was not clear, however, the extent to which the HEIs adopted the industry-designed courses. Sustainable and replicable knowledge hubs for IT-BPO were not fully realized.

Adoption of GCAT in selected HEIs by Q3 2015

Not achieved. Adoption was no longer relevant due to preference for other assessment tools.

Not achieved. Schools were not ready to adopt the tool due to cost, as well as lack of infrastructure and policy.

Outputs 1. Access to knowledge

resources for teaching and learning IT-BPI industry-designed courses increased

Business model for operating and sustaining the knowledge hubs developed by Q1 2013

Partly achieved. Business model was developed for operating and sustaining knowledge hubs. Initial target was to establish at least three knowledge hubs but only one online knowledge hub was established at UP Open University and a physical hub at BPAP’s offices.

Partly achieved. Business model was conceptualized but not fully implemented. One online knowledge hub at UPOU and a physical hub in IBPAP were completed. The target was to establish at least three knowledge hubs in selected HEIs.

Digitized teacher training materials and study modules incorporated in the knowledge hubs by Q3 2014

Achieved. Digitized teacher training materials and five study modules incorporated into these hubs and offered online.

Achieved. UPOU digitized all five eSMP courses, which were offered online.

Appendix 2: Design and Monitoring Frameworks of Technical Assistance Projects 39

Design Targets


(July 2018) Independent Evaluation

(March 2020) 2. Higher education

curricula and instruction enhanced

900 faculty members from participating HEIs, of which at least 50% are women, completing online teacher training programs on IT-BPO industry-designed courses by Q2 2015

Partly achieved. Teacher certification conducted for 379 faculty members (59.9% female) using online teacher training programs.

Partly achieved. Blended teaching was conducted for certification of 379 faculty members against a target of 900. Of these, 59.9% were female against the target of 50% female.

3,000 students, of which at least 50% are females, complete online learning modules and study guides on IT-BPO industry-designed courses by Q2 2015

Partly achieved. 2,276 (48.9% women) students completed online learning modules and study guides on IT-BPO-designed courses.

Partly achieved. 2,276 students completed online learning modules, of which 48.9% were women (against target of 50%).

3. Partnerships between industry and academic institutions strengthened

At least 3 partnership agreements involving HEIs and industries signed by Q4 2012

Achieved. Partnership framework developed. Partnerships between HEIs and industry were successful in securing additional training and employment opportunities for students.

Achieved. Several memoranda of agreement were signed between IT-BPO companies and schools.

At least 2 twinning arrangements between foreign and domestic HEIs established by Q2 2014

Not achieved. Twinning partnership not established.

Not achieved.

eSMP = electronic service management program, GCAT = Global Competitiveness Assessment Tool (a test for basic competency of entry-level employees.), HEI = higher education institution, IBPAP = IT & Business Process Association of the Philippines, ICT = information and communication technology, IT-BPO = information technology and business process outsourcing, UPOU = University of the Philippines Open University. a Institute for Management Development, World Competitiveness Ranking, Philippines, 2019 (www.imd.org/wcc). Sources: ADB. 2018. Technical Assistance to the Republic of the Philippines for Strengthening Knowledge-Based Economic and Social Development. Manila; IT & Business Process Association of the Philippines. 2016. Technical Assistance Consultant’s Report: Philippines Strengthening Knowledge-Based Economic and Social Development. Manila; University of the Philippines Open University. 2016. Technical Assistance Consultant’s Report: Philippines Strengthening Knowledge-Based Economic and Social Development. Manila.


Table A2.2 Design and Monitoring Framework for TA 8334—Improving Competitiveness in Tourism

Design Targets


(May 2018) Independent Evaluation

(March 2020) Impact Increased employment in the tourism industry

(By 2020) Employment increased above the long-term average growth rate (2000–2010 baseline: average employment growth of 3.5%)

Not reported. Partly achieved. During 2012–2018, employment in tourism industry grew by an average of 2.7% (Philippine Statistics Authority).

Outcome Improved competitiveness of Philippine tourism

By February 2017 Ranking in the World Economic Forum travel and tourism competitiveness index improved by at least 15 places (2011 baseline: 94th of 139 countries)

Achieved. Ranking in the World Economic Forum ranking increased from 94 out of 139 countries to 82 among 140 countries.

Achieved. In 2019, ranking in travel and tourism competitiveness index increased to 74 out of 140 countries.

Business cost of compliance with regulations in the tourism industry reduced by 15% (baseline 2013 business regulation survey)

Not reported. Achieved. Of the 10 RIAs formulated at the national level, 4 were implemented. The 6 RIAs not implemented were beyond the purview of DOT. In Puerto Princesa, the updated Tourism Code was implemented. Those RIAs implemented helped reduce business cost of compliance in tourism.

Outputs

1. Regulatory impact assessment program implemented

By February 2017 Annual work plans for 2012–2015 approved (2011 baseline: no data available)*

Achieved. Achieved.

10 RIAs completed and published on DOT website (2011: no data available)*

Partly achieved. 10 RIS and 4 RIAs completed

Achieved. 10 RIS completed but only 4 were implemented.

At least 50 staff at DOT trained in RIA process and tools (2011 baseline: no data available)*

Achieved. 400 persons trained in RIA including 120 LGU staff

Achieved.

Business licensing reform action plans endorsed by four pilot LGUs

Not reported. Completion report did not report that reform actions plans were endorsed. 600 businesses were surveyed as part of the Business Licensing and Permits Survey to inform selection of grantees and implement RIA scheme in LGUs.

Partly achieved. Only 2 LGUs implemented RIAs. Actions were taken in Puerto Princesa to pilot implementation of the red tape reduction strategies and RIS through issuance of local ordinance. Panglao incorporated RIA in the municipal provincial policymaking process.


Design Targets


(May 2018) Independent Evaluation

(March 2020)

2. Hotel and resort quality assurance and accreditation system implemented in four regions

By February 2017 Accreditation system established providing for certification of hotel and resort standards (2012 baseline: no data available)*

Achieved. National accommodation standards revised and published. Mabuhay standards developed for small accommodation providers. Quality assurance manual developed.

Achieved. Revised and published National Accommodation Standards. Mabuhay standards developed for small accommodation providers. Quality assurance manual developed.

100 hotels accredited under the new national hotel quality assurance system pilot (2012 baseline: no data available)*

Not reported. Completion report did not report on the number of hotels accredited. 25 small accommodation providers assessed using Mabuhay standards.

Achieved. 2,146 hotels accredited under the new quality assurance system during 2012–2018.

At least 60 assessors certified (2011 baseline: no data available)

Not reported. Only the 35 DOT assessors trained using Mabuhay standards were reported. DOT accreditation governance reviewed including recommendations provided for further improvement.

Achieved. 100 DOT and industry third party assessors trained in the revised National Accommodation Standards. 35 DOT assessors trained in Mabuhay standards.

3. Tourism industry skills development program developed and piloted in four regions

(By February 2017) At least 5,000 tourism workers accessed skills development through the tourism industry skills development program pilot, with at least 55% women workers (2011 baseline: no data available)*

Achieved. 7,550 program participants trained, of which 47.3% were women and 52.7% men. Tourism Industry Skills Development Program launched. Tourism enterprise skills survey implemented. Implementation and pilot testing of ISGS, which awarded 49 grants.

Achieved. 7,500 personnel trained, 48 grantees.

At least 500 tourism professionals achieved mutual recognition under ASEAN agreement (2011 baseline=no data available)*

No data presented. No data.

Human resource strategy developed (2011 baseline: no data available)*

Not reported in completion report.

Achieved. DOT completed the first tourism human resource development strategy and action plan, 2015–2020.

ASEAN = Association of Southeast Asian Nations, DOT = Department of Tourism, HEI = higher education institution, ISGS = industry skills grant scheme, LGU = local government unit, RIA = regulatory impact assessment, RIS = regulatory impact statement. * Work commenced in 2011 or 2012 where relevant. Sources: ADB. 2018. Technical Assistance Completion Report: Improving Competitiveness in Tourism. Manila; University of Queensland. Final Report: Improving Competitiveness in Tourism Project Philippines. Queensland; Independent Evaluation Department.


Table A2.3 Design and Monitoring Framework for TA 8335—Employment Facilitation for Inclusive Growth

Design Targets

Technical Assistance Completion Report *

(April 2018) Independent Evaluation

(March 2020)

Impact Employability of at-risk youths enhanced

By 2019 Youth wage employment increased by 5.0% per annum (1991–2008) baseline: 3.4% per annum)

Not achieved. Youth full-time wage employment increased by 1.29% per annum during 2010–2015.

Job placement rate for youths received MyFirstJob services 20% higher than for similar group not receiving MyFirstJob services

Not reported. Impact evaluation indicated that JobsStart pilot program significantly increased income levels but did not appear to influence employment levels in a significant manner.

Job placement rate for youths receiving MyFirstJob services in jobs with social protection benefits 20% higher than for similar group not receiving MyFirstJob services

Not reported. No data.

Outcome Capacity of PESOs to provide employment improved

By 2017 Number of persons finding jobs through PESOs increased by 10.0% per annum (2010 baseline: 1,000 per PESO)

Overachieved. The number of persons finding jobs through institutionalized PESOs increased by 15% from 1,936 per PESO during 2005–2010 to 3,844 per PESO from 2010–2015.

Overachieved. The number of persons finding jobs through institutionalized PESOs grew by 15% during 2005–2010 to 2010–2015.

Participating PESOs increased the number of employers using their services in the MyFirstJob by 20% over PESOs not offering the MyFirstJob project

Not reported. Unable to verify. No data for PESOs not offering MyFirstJob.

Outputs

1. Capacity for labor policy analysis and formulation strengthened

By February 2017 Annual regulatory impact assessment (RIA) work plans for 2013–2015 approved (2011 baseline: no data available)

Achieved. Support to DOLE in regulatory review of its labor code following the department’s RIA template.

Achieved.

Ten regulatory impact assessments on the Labor Code of the Philippines completed and published on DOLE website (2011 baseline: no data available)

Partly achieved. DOLE produced 9 RIAs on selected labor regulations in the Philippine labor code.

Achieved.

At least 60 DOLE staff trained on RIA processes and tools, monitoring and evaluation impact evaluation methodologies, and policy development (2011 baseline: no data available)

Number of trained staff not reported in completion report. DOLE staff trained in RIA processes and tools.

Achieved. 10 training activities conducted must have covered DOLE staff from the national to regional level.


Design Targets

Technical Assistance Completion Report *

(April 2018) Independent Evaluation

(March 2020)

At least five labor policy research studies completed and published (2011 baseline: no data available)

Not reported. Partly achieved. Nine policy research studies were undertaken for the RIAs. Outputs were not published because no RIAs were implemented.

Impact evaluation assessment reports for at least one labor or employment program completed and published (2011 baseline: no data available)

Not reported. Achieved. Impact evaluation of JobStart pilot program conducted.

Protocols for early employment recovery program prepared (2011 baseline: no data available)

Not reported. Not achieved. No information available.

2. My FirstJob pilot project implemented

Number of institutionalized PESOs with standard labor market information and employment intermediation systems in place increased to 100 (2011 baseline: 89)

Achieved. There was an increase in number of PESOs with labor market information and employment intermediation systems in place with the development of electronic skills registry system (eSRS) that automated manual process and made labor market information available on an online platform.

Achieved. Of the four pilot PESOs, two improved their LMI and employment mediation by at least 75% and one improved by 16%. Standard labor market information system and reports were generally in place.

At least 1,600 youth participants complete one or more of the following services: career counselling, vocational training, and workplace experience (2011 baseline: no data available)

Partly Achieved. 1,413 participants completed life skills training. 37.5% (or 537) of 1,432 candidates available for placement completed technical training. 688 JobStarters completed technical training through 50 memoranda of agreement with employers.

Achieved. 1,819 beneficiaries completed full cycle during 2014–2015.

Gender responsive manual for employment services produced

Not reported. Achieved. Gender action plan developed.

Impact evaluation reports for MyFirstJob pilot project completed

Not reported. Achieved. Impact evaluation completed.

DOLE = Department of Labor and Employment, PESO = Public Employment Service Office, RIA = regulatory impact assessment. a Data not presented in the technical assistance completion report were taken from the project completion report on Increasing

Competitiveness for Inclusive Growth, which the TA supported. Source: ADB. 2018. Technical Assistance Completion Report: Philippines Employment Facilitation for Inclusive Growth. Manila.


Table A2.4 Design and Monitoring Framework for TA 8984—Support for the Nationwide Rollout of JSP (As of 31 December 2018)

Impact: The TA is aligned with the Nationwide Rollout of JSP based on the Philippine Development Plan 2011–2016 and will contribute to the government’s efforts to increase youth employment. In particular, the project will support the government’s efforts to establish the nationwide institutional setup for the JSP rollout (2015–2020).

Project Results Chain Performance Indicators with Targets

and Baselines Status (DOLE)

Outcome JSP rollout by DOLE and PESOs commenced

By December 2021: a. 24 LGUs signed MOUs with

DOLE to implement JSP Program in their PESOs

b. At least 15,000 at-risk youth access JSP services through 24 PESOs and graduate from JSP

a. Achieved. 36 LGUs signed MOA with DOLE for JSP since implementation. b. Achieved. 21,063 at-risk youth across 36 PESOs accessed JSP services, of which 58% are female (cumulative).

Outputs

1. Capacity development of JSP units at DOLE headquarters and RDOs strengthened

By December 2020: 1.a. JSP unit established in DOLE

Achieved. JSP unit established at DOLE-BLE.

1.b. JSP established in 4 DOLE RDOs Achieved. JSP unit established in DOLE-ROs (with ongoing request for permanent plantilla positions).

1.c. 100 DOLE staff trained in JSP business operations (SOPs, SRS, employer MOAs and training plans, financial management, and the monitoring system), at least 55% of staff of which are female

Achieved. 461 trained JSP unit staff at DOLE-Head Office and Regional Office/Field Offices delivering services.

2. Life skills for employability training curriculum rolled out nationally

By December 2020 2.a. 300 staff from DOLE, PESO and training institutes trained as trainers in life skills delivery, at least 55% of which are female

Ongoing. 74 trained PESO staff delivering LST.

2b. At least 15,000 youth access life skills training course, at least 55% of which are female

Achieved. 13,158 youth trained and completed LST.

3. Capacity of 24 PESOs in JSP implementation developed

By December 2020 3a. JSP established in 24 PESOs

Achieved. JSP established in 35 PESOs.

3b. At least 240 PESO staff trained in JSP business operations (SOPs, SRS, employer MOAs and training plans, financial management and monitoring systems), at least 55% of which are female

Ongoing.

3c. At least 80 PESO staff trained in career guidance and coaching techniques, at least 55% of which are female

Ongoing. 46 PESO trained in career guidance and coaching techniques.

3d. SRS (Public Employment Information System) implemented and operational in all 24 PESOs

Achieved. Public employment information system implemented and operational in all 35 PESOs.

4e. At least 10 employers prequalified to participate in JSP in each of the 24 PESOs

Achieved. 440 total employers participated in JSP in 35 PESOs.

BLE = Bureau of Local Employment, DOLE = Department of Labor and Employment, JSP = JobStart Philippines, LGU = local government unit, LST = life skills training, MOA = memorandum of agreement, MOU = memorandum of understanding, PESO = Public Employment Service Office, RDO = regional DOLE office, SOP = standard operating procedure, SRS = Skills Registry System. Sources: ADB. 2015. Technical Assistance to the Republic of the Philippines for the Support for the Nationwide Rollout of JobStart Philippines. Manila; and Department of Labor and Employment.

APPENDIX 3: SUMMARY OF ACCREDITED HOTELS AND RESORTS

Number of Accredited Hotels and Resorts under the New Quality Assurance System (2012–2018)

Year Hotels Resorts 2012 202 176 2013 236 189 2014 345 273 2015 295 289 2016 477 376 2017 179 112 2018 412 302 Total 2,146 1,717

Source: Department of Tourism.

APPENDIX 4. LIST OF REGULATORY IMPACT ASSESSMENT STATEMENTS

Table A4.1 Department of Tourism: Regulatory Impact Statements

Title Objective

1. Rules and Regulations Governing the Business of Sports in SCUBA Diving in the Philippines

To ensure that the health and safety of the sports SCUBA divers in the Philippines are appropriately managed and that persons operating SCUBA businesses have the right skills and are ethical practitioners.

2. Cost Recovery for Department of Tourism Hotel Accreditation

Identify mechanism and evaluate appropriate fee structures on accreditation to recover government costs in providing accreditation services to primary tourism enterprises.

3. Rules and procedures governing the conduct of marine wildlife tourism interactions in the Philippines

To protect marine wildlife and its habitats from tourism activities and to ensure the safety of tourists.

4. Travel Tax – Processing and Collection Reform the processing and collection of travel tax to (i) minimize cost to the government and taxpayers, (ii) simplify and streamline travel tax processing, and (iii) ensure sufficient funds collection to be used for the purposes of the Tourism Act 2009.

5. Special Resident Retiree’s Visa Application Fee

Evaluate the proposal of the Philippine Retirement Authority (PRA) to adjust the Special Resident Retirees’ Visa Application Fee.

6. Duty Free Philippines – Improvement on Shopping Privileges

The objectives were to (i) augment the service facilities provided to tourists, (ii) improve the duty free shopping experience, (iii) increasing foreign exchange earnings and revenue, and (iv) increase the attractiveness of the country as a tourist destination.

7. Investment Options – Philippine Retirement Authority

Provide guidance on how to evaluate Philippine Retirement Authority’s proposal of providing bed and breakfast facilities for current and prospective retirees.

8. Review of Ecotourism Standards Identify an option that would best achieve the following: (i) support government policy to make the country a leading ecotourism destination; (ii) increase resources from government, development partners, and private companies; and (iii) provide efficiency in administration and compliance of standards for ecolodges and ecoguides.

9. Foreign Retiree Tracking System – Philippine Retirement Authority

Review the Philippine Retirement Authority’s tracking system of foreign retirees.

10. Small Accommodation Standards (Mabuhay Standards)

Identify an option that would best achieve the following: (i) support government policy on accreditation of small accommodation, (ii) provide accommodation standards and criteria relevant to market conditions, and (iii) provide efficiency and administration and compliance with accommodation standards.

Source: Department of Tourism.


Table A4.2 Department of Labor and Employment: Regulatory Impact Statements

Title Objective

1. Apprenticeship (Technical Vocational and Skills Development Authority)

Address the labor market mismatch, high and increasing number of people not in employment, education, or training, and low participation rate in existing apprenticeship program.

2. Security of Tenure (Bureau of Local Employment)

Identify best option for rigidities in firing: costly and lengthy litigation; proliferation of short-term employment; weakened security of tenure; and weak Philippine competitiveness.

3. Employment Insurance (Institute for Labor Studies)

Address the problem of no unemployment related benefit scheme that facilitates re-employment and provides income support to involuntarily unemployed workers. Present social security laws do not cover unemployment as one of the contingencies.

4. Magna Carta for Seafarer (Bureau of Workers with Special Concerns)

To ensure protection and decent work for Filipino seafarers and to promote compliance with maritime regulations while minimizing burden on the industry so they can remain competitive and productive.

5. Public Employment Service Office (Bureau of Local Employment)

To ensure that the employment programs of the Department reach the target beneficiaries through the help of the Public Employment Service Office (PESO). Strengthen the PESOs in performing their mandated functions.

6. Productivity (National Wages and Productivity Commission)

Identify legislative impediments to workers and enterprises productivity.

7. Special Program for the Employment of Students (Bureau of Local Employment)

Address the following problems: (i) lack of mechanisms for facilitating absorption of trainees into the labor force, (ii) delay in process-cycle time, (iii) difficulties encountered by potential beneficiaries in complying with the requirements, (iv) high government subsidy instead of maximizing private sector participation, and (iv) lack of participation from small enterprises.

8. Overseas Employment (Philippine Overseas Employment Administration)

Address the problem of reported cases of abuses, exploitation, and violation of rights of migrant workers and overseas foreign workers.

9. Article 40 of Labor Code (Bureau of Local Employment)

Address the issues on shortage of highly specialized skills, anticipated increased mobility within an integrated Association of Southeast Asian Nations (ASEAN) agreement, restrictions in complying with the commitments on movement of natural persons, prolonged vacancies for occupations requiring highly specialized skills, and restrictions on employment of foreign nationals.

Source: Department of Labor and Employment.

APPENDIX 5. DOLE RIA CAPACITY BUILDING ACTIVITIES

Training Activity Date

1. RIA Training Session 30 April 2012 2. RIA Training Session 21 May 2012

3. DOLE Regional Seminar–Workshop in Mindanao 30 July 2012 4. DOLE Regional Seminar–Workshop in Visayas 31 July 2012

5. DOLE Regional Seminar–Workshop in Luzon 12 August 2012

6. DOLE RIA Technical Learning Sessions on Guidelines and Forms 22 November 2012 7. Convention of the RIA Committee and Constitutions of RIA Task Teams 20 February 2014

8. DOLE Seminar–Workshop on RIA 12–14 March 2014 9. DOLE Seminar–Workshop on RIA 21–23 May 2014

10. NEDA–DOT–DOLE–ADB Retreat Stock–take and Future Directions of Regulatory Impact Analysis in the Philippines 29–30 April 2015

ADB = Asian Development Bank, DOLE = Department of Labor and Employment, DOT = Department of Tourism, NEDA = National Economic and Development Authority, RIA = regulatory impact assessment. Sources: Department of Labor and Employment; and Institute for Labor Studies.

APPENDIX 6. JOBSTART SUMMARY PERFORMANCE

Year

Participating Employers* (number)

Registered for JSP

(number)

Enrolled in LST

(number)

Finished LST

(number) Finished LST (%)

Registered that

Enrolled in LST (%)

Women Finished

LST (number)

TTI Graduates (number)

LST Finishers

that Finished TTI (%)

Job Placement Rate** (%)

With National

Certificate (number)

2014–2015 50 5,424 1,819 1,413 78 34 na 539 38 75 0 2016 137 4,989 3,398 2,989 88 68 56 598 20 70 49 2017 174 9,221 7,803 7,077 91 85 56 2,580 36 68 410 2018 48 2,493 1,810 1,770 98 73 61 972 55 73 266 2019 72 2,156 1,673 1,570 94 78 58 770 49 67 na Total 481 24,283 16,503 14,819 90 68 58 4,689 32 71 725 JSP = JobStart Philippines, LST = life skills training, na = data not available, TTI = technical training and/or internship. * Employers with Memorandum of Agreement. ** Refers to beneficiaries that completed LST and found jobs, absorbed by partner employer, found jobs at other employers, and found jobs after referral to other programs. Sources: JobStart Philippines, Bureau of Local Employment, Department of Labor and Employment.

APPENDIX 7: LIST OF PARTICIPATING PESOS IN JOBSTART

Region LGU-PESO

National Capital Region Caloocan City

Mandaluyong City

Muntinlupa City

Pasig City

Pasay City

Paranaque City

Makati City

Taguig City

Quezon City Cordillera Autonomous Region Baguio City Region 1 Rosales City

Dagupan City Region 2 Santiago City

Tuguegarao City Region 3 San Fernando City

Municipality of Marilao Region 4 Binan City

General Trias City

Puerto Princesa City Region 5 Legazpi City Region 6 Iloilo Province

San Carlos City Region 7 Cebu City

Dumaguete City

Bohol Province Region 8 Tacloban City Region 9 Pagadian City Region 10 Cagayan de Oro City

Municipality of Manolo Fortich Region 11 Davao City

Panabo City

Tagum City Region 12 General Santos City

Kidapawan City Region 13 Agusan del Sur Province LGU = local government unit, PESO = Public Employment Service Office Source: Department of Labor and Employment.

APPENDIX 8. TOURISM INDUSTRY EMPLOYMENT AND SHARE IN GROSS DOMESTIC PRODUCT

Indicator 2012 2013 2014 2015 2016 2017 2018 Average

Employment in tourism industry (‘000) 4,561 4,709 4,819 4,971 5,224 5,268 5,365 4,988 Share of tourism in total employment (%) 12.1 12.4 12.7 12.8 12.8 13.1 13.0 12.7 Growth of employment in tourism industry (%) - 3.2 2.3 3.2 5.1 0.8 1.8 2.8 Share of tourism to gross domestic product (%) 7.9 8.2 9.0 10.1 10.7 12.2 12.7 10.1

Sources: Labor Force Survey, Annual Survey of Philippine Business and Industry, and Census of Philippine Business and Industry of the Philippine Statistics Authority.

APPENDIX 9. NUMBER OF INBOUND VISITORS ARRIVALS, 2015–2019

Year Number Growth (%) over previous year

2015 5,360,682 2016 5,967,005 11.31 2017 6,620,908 10.96 2018 7,168,467 8.27 2019 8,260,913 15.24

Average 6,675,595 11.44 Source: Department of Tourism.

APPENDIX 10. STATISTICAL ANALYSIS RESULT FROM DEPARMENT OF LABOR AND EMPLOYMENT’S JOBSTART PHILIPPINES DATA

Variables Group 1 Group 2 Statistical

Significance Obs. Mean Std. Err. Obs. Mean Std. Err

01. Male Employment Rate (Post-LST vs. Later) 81 34.9% 2.2% 81 22.8% 2.4% *** 02. Female Employment Rate (Post-LST vs. Later) 81 37.7% 2.2% 81 21.2% 2.4% *** 03. Employment Rate (Post-LST vs. Later) 81 36.4% 2.1% 81 21.7% 2.3% *** 04. Male Employment Rate for Pilot (Post-LST vs. Later) 12 46.4% 5.3% 12 13.8% 4.5% *** 05. Female Employment Rate for Pilot (Post-LST vs. Later) 12 47.9% 5.2% 12 12.3% 3.9% *** 06. Employment Rate for Pilot (Post-LST vs. Later) 12 47.0% 5.1% 12 12.6% 4.0% ***

07. Male Employment Rate for Rollout (Post-LST vs. Later) 69 32.9% 2.4% 69 24.4% 2.7% * 08. Female Employment Rate for Rollout (Post-LST vs. Later) 69 35.9% 2.3% 69 22.8% 2.7% *** 09. Employment Rate for Rollout (Post-LST vs. Later) 69 34.5% 2.3% 69 23.2% 2.6% ** 10. Post-LST Male Employment Rate (Pilot vs. Rollout) 12 46.4% 5.3% 69 32.9% 2.4% * 11. Post-LST Female Employment Rate (Pilot vs. Rollout) 12 47.9% 5.2% 69 35.9% 2.3% - 12. Post-LST Employment Rate (Pilot vs. Rollout) 16 41.5% 4.9% 69 34.5% 2.3% -

13. Non-Post-LST Male Employment Rate (Pilot vs. Rollout) 12 13.8% 4.5% 69 24.4% 2.7% - 14. Non-Post-LST Female Employment Rate (Pilot vs. Rollout) 12 12.3% 3.9% 69 22.8% 2.7% - 15. Non-Post-LST Employment Rate (Pilot vs. Rollout) 12 12.6% 4.0% 69 23.2% 2.6% - 16. Untraced Male Jobstarter Ratio (Pilot vs. Rollout) 12 39.9% 4.6% 69 42.7% 2.9% - 17. Untraced Female Jobstarter Ratio (Pilot vs. Rollout) 12 39.7% 4.4% 69 41.3% 2.9% - 18. Untraced Jobstarter Ratio (Pilot vs. Rollout) 12 40.3% 4.2% 69 42.2% 2.8% -

19. All Employed Male Ratio (Pilot vs. Rollout) 12 60.1% 4.6% 69 57.3% 2.9% - 20. All Employed Female Ratio (Pilot vs. Rollout) 12 60.3% 4.4% 69 58.7% 2.9% - 21. All Employed Ratio (Pilot vs. Rollout) 12 59.7% 4.2% 69 57.8% 2.8% - 22. Post-LST Employment Rate (Male vs. Female) 81 34.9% 2.2% 81 37.7% 2.2% - 23. Non-Post-LST Employment Rate (Male vs. Female) 81 22.8% 2.4% 81 21.2% 2.4% - 24. Untraced Jobstarter Ratio (Male vs. Female) 81 42.3% 2.6% 81 41.1% 2.5% - 25. All Employed Ratio (Male vs. Female) 81 57.7% 2.6% 81 58.9% 2.5% -

LST = life skills training, Obs. = observations, Std. Err. = standard error. *, **, *** = difference is statistically significant at the 90%, 95%, or 99% level, respectively. Source: Department of Labor and Employment, JobStart.

APPENDIX 11: LESSONS FROM THE JOB CREATION AND SKILLS DEVELOPMENT TECHNICAL ASSISTANCE PROJECTS

TA Project Lessons

IT-BPO TA

a. Clear governance and leadership are essential for successful TA implementation. The TA lacked a governmental agency with a clear mandate to lead the IT-BPO sector at the time of TA implementation. The project’s implementation progress was overseen by DOF, which provided little guidance or coordination for the three implementing agencies. This resulted in the departure of one implementing agency and failure to deliver the majority of outcomes.

b. Feasibility survey can reduce the risk of an innovative intervention’s failing. Despite the efforts by implementing agencies to develop and roll out online courses within the TA project cycle, provision of training opportunities to rural regions through both online and physical knowledge hubs proved impractical due to limited Internet bandwidth in the region. The situation has not improved significantly years after TA completion, which begs the question as to whether the Philippines was ready for such innovation in the first place.

c. TA project planning should be realistic when it comes to resources and time. Capacity development in the IT-BPO sector required closely coordinated improvements in policy, training program, and infrastructure. Assuming or expecting that policy and infrastructural prerequisites would fall into place once ADB provided the training program was overly ambitious, especially with just a $0.5 million, 4-year TA project. More practically, this TA should have been a simple policy intervention with a capacity development component for government counterparts and concluded with the delivery of a long-term policy framework and preliminary plans for a pilot.

Tourism TA

d. Outcomes of capacity development for LGUs can be most sustainable. The Panglao municipal officers mentioned how the RIA framework was successfully incorporated into the municipal policymaking process, and, as a result, more RIAs were being undertaken leading to innovation in tourism operations, such as an online reservation and quota management system for scuba divers. Tourism service quality and accreditation standards have been continually updated with environmental, governance, and customer handling requirements while being enforced through adoption by the LGUs of a “no accreditation, no permit” policy.

e. Spillover from capacity development could bring about significant long-term impact. In Puerto Princesa, RIA implementation was successful when all the LGUs got involved in the pilot TA and witnessed firsthand how RIA helped reduce inherent systematic inefficiencies and bureaucratic processes. Years after completion of the TA, the LGU has continued improving its business processes, establishing a “Business One-Stop Shop” to reduce paperwork and time needed by local small and medium-sized enterprises for business registration and license renewal.

f. Provision of career path is essential for retaining trained personnel. The most resilient of trainees were those given enough recognition for their developed skills and who also were fortunate enough to be promoted by their employers. Despite seeing more than half of their training cohorts leave for better paying jobs in other, larger cities or countries, those who remained expressed high levels of loyalty, satisfaction, expertise, and eagerness to share their knowledge and skills with younger, less experienced colleagues as trainers of trainees.

g. Innovation needs to be flexible to stay pertinent. The TA’s innovative approach to selection of training programs gave an unprecedented level of control and ownership to the owners of the local tourism sector operators. However, ongoing support from DOT for a follow-up to ADB’s innovative program has shifted the control to associations of resort owners in order to improve efficiency in coordinating selection of the training programs and to ensure the participation and access to the support for the smaller resorts and establishments.

Appendix 11: Lessons from the Job Creation and Skills Development Technical Assistance Projects 55

TA Project Lessons

h. Shift in the government’s policy could undermine the TA’s innovations. DOT had intended to sustain the innovations introduced by ADB’s TA through the TISP, but, due to changes in administration, the overall policy direction had shifted. As such, the following variations to the implementation guidelines were introduced into the TISP: (i) allocation of equal amount of budget per region, regardless of each region’s capacity or need; (ii) cost sharing by the employers was no longer required in applying for the TISP grant; and (iii) a performance indicator based on number of trainings completed and number of participating trainees. The resulting iteration of DOT’s TISP has seen the innovative aspect of the TA significantly undermined, as TISP incentivized regional DOT offices to implement more low-cost, low-tech basic trainings that cater to a larger base rather than offering customized, cutting-edge trainings to meet the needs of a specific group of employers.

i. Institutionalization of innovation is needed for resilience. It was noted that many LGUs had complete personnel turnover following the results of the May 2019 local elections, resulting in significant loss of institutional memory concerning ADB’s TA and related innovations. This could be attributed to the lack of a system for enforcing effective handover of knowledge among LGU personnel. Also, lacking career officers specializing in certain technical fields, some LGUs had shed most, if not all, of their previous personnel that had supported ADB’s TA project implementation. Designing future TA projects for resilience through institutional adoption of the key innovations should be given foremost consideration.

j. Support for independent business owners needs to be incorporated into the TA. The evaluation team encountered trainees who were self-employed in the tourism sector (e.g., tour guides, small boat operators, a women’s group) who did not have access to the logistical support needed to apply for ADB’s skill development program because support was only offered to employers. As such, they had to do all the paperwork required of large employers while also working and being trained as employees. In the future, ADB should lower the bar of entry for the self-employed, as these are people who need the capacity development most but can least afford it.

k. Networking and empowering of local communities are key to sustainable intervention. In Davao, various tourism sector associations had been formed to facilitate the implementation of ADB TA and then managed to stay strong and active in their role well into implementation of DOT’s TISP. The sense of community and camaraderie that exist among the employers in the said associations brought about sustained impacts, including peer learning through sharing knowledge across the private sector network, sharing a pool of able trainers, and reducing the cost and risk of skills training through economies of scale. Larger, well-established resorts were incentivized to open their ADB-supported training programs to smaller resorts in an arrangement that was referred to as “big brother, small brother,” resulting in what can be termed a cascading of learning and sharing of knowledge that has been perpetuated within Davao’s private sector tourism network.

At-Risk Youth TA and JobStart Rollout TA

l. Institutionalized PESOs are needed as focal points for intervention. Despite efforts by the project management offices, some LGUs were not sufficiently equipped or trained to implement the follow-up programs but relied heavily on external service providers or project coordinators. LGUs lacked personnel solely in charge of JSP or allocated casual or job order workers to the program. This situation posed high risk for loss of ownership and disruption of knowledge transfer. Also, the PESOs did not allocate budget for JSP, which diminished sustainability of the intervention.

m. Coordinated public relations campaigns need to be included in future interventions. Despite that the TA projects were aimed at alleviating poverty for the most vulnerable, not enough of those targeted were aware of them. This was because the PESOs used existing or familiar human networks to elicit participation from those with greater access to information from the PESOs. Future TA projects need PR campaigns tailored to the realities of the target area, as rural communities with the most impoverished families were being reached through local community leaders’ personal networks and social media was useful in reaching out to youths.

n. Consistent monitoring of the beneficiary performance is needed to promote accountability. As PESO officials were required by the TA guidelines to periodically monitor the performance of the JobStarters in person, they had to spend upwards of 60% of their working hours calling the employers, visiting the JobStarters at their workplaces to ensure compliance by all parties, and providing administrative support for them. Both employers and JobStarters who received these check-ups expressed significant trust and confidence as a result of these personal interactions.


TA Project Lessons

o. Greater incentivization is needed for small and medium-sized enterprise employers. Although employers cited improved attitudes and work ethics among JobStarters as one of the biggest benefits of the JSP, they also noted that the actual level of skills did not differ significantly between the JobStarters and those who were hired without the training and that the prerequisites and reporting required after LST dramatically heightened the cost of entry for the smaller business owners. That was all the more the case if they only ended up with less than a dozen JobStarters. Increasing fiscal support and simplifying logistics would incentivize greater participation and more equitable impact on small and medium-sized enterprises in rural areas.

p. Improvement is needed in the process for identifying potential JobStarters. Instead of depending on word-of-mouth, one-time events, and personal networks within the LGU, PESO officials should diversify channels of promotion, seek out and proactively communicate with unemployed, at-risk youths, and actively weed out overqualified, unqualified, or mismatched candidates. They also should share and cross-reference potential candidates for PESOs’ various training and employment support programs instead of having them compete against one another for the same candidates.

q. LST-centric evolution of the JSP should be actively explored. It should be noted that the acceptance of “different pathways” after LST has somewhat diluted the original innovation of the JSP by allowing JobStarters to be hired directly after only completing LST. This has reduced emphasis on the holistic approach to skills development. Although a significant departure from the original model, this variation could become a low-cost, high-impact, and sustainable program that is more compatible with the situations in highly developed urban centers like Taguig and General Trias.

r. Strong ownership and quality control are needed for LST. The concern about achieving and maintaining a minimal level of quality for LST training remains, and this would be impossible at present without the support of external service providers. DOLE-BLE should establish itself as the authority in LST by taking ownership of the training module, spearheading the regular updating and revision of training manuals, hosting and implementing the training of trainers programs, and ensuring quality control of the LST programs.

ADB = Asian Development Bank, BLE = Bureau of Local Employment, DOF = Department of Finance, DOLE = Department of Labor and Employment, DOT = Department of Tourism, IT-BPO = information technology and business process outsourcing, JSP = JobStart Philippines, LGU = local government unit, LST = life skills training, PESO = Public Employment Service Office, RIA = regulatory impact assessment, TA = technical assistance, TISP = Tourism Industry Skills Program. Source: Independent Evaluation Department.

performance evaluation report job creation and skills

Documents