developing rehabilitation assistance to schools and

D-RASATI 2

Developing Rehabilitation Assistance to Schools and Teacher Improvement October 2013 – December 2016

Final Report

January 31, 2017

Submitted by World Learning

Overall Task Order Manager

Contract No.: AID-OAA-I-12-00008 Task Order No.: AID-268-TO-13-00004, AID-269-TO-13-0004, and

REQM‐268‐14‐000032

This report is made possible with the support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of World Learning and do not necessarily reflect the views of USAID or the United States Government.

D-RASATI 2 (October 2013 – December 2016) Final report, December 30, 2016 World Learning

ii

Table of Content Abbreviations .................................................................................................... iii Executive Summary............................................................................................ 1

Introduction ....................................................................................................... 5

Project Overview and Rationale ......................................................................................................... 5

The Project Components ................................................................................................................ 7 Roles and responsibilities of World Learning and its implementing partners ................................ 8

Project Accomplishments .................................................................................. 9

Development Objective 1: Improved Capacity of the Public Sector in Providing Transparent and Quality Services Across Lebanon ........................................................................................................ 9

ICT integration in education............................................................................................................ 9 Expand extracurricular activities at intermediate and secondary schools ................................... 11 Support to host schools ................................................................................................................ 12

IR1.1 Improved public basic education available to all school-aged citizens ................................... 12

Sub-IR 1.1.1 School learning environments improved ................................................................. 13

Procurement of Tier Model ICT equipment .............................................................................. 13 Implementing MEHE’s Effective School Standards in relation to ECA, LDP and SIP ................. 14 Provision of stipends and PSS stationary kits to 640 primary host schools .............................. 17

Sub-IR 1.1.2 School learning opportunities increased .................................................................. 17

Information and Communications Technologies (ICT) ............................................................. 18 English language proficiency and teaching methodology training ........................................... 21 Extra-Curricular Activities (ECA) ................................................................................................ 22 Psychosocial Support (PSS) ....................................................................................................... 23 Leadership Development Program (LDP) .................................................................................. 24 School Improvement Program (SIP) .......................................................................................... 25

Project Monitoring and Evaluation .................................................................. 27

Project Monitoring ............................................................................................................................ 27 Accomplishments Relative to Targets ............................................................................................... 29

Impact and Sustainability ................................................................................. 35

Impact ............................................................................................................................................... 35 Sustainability ..................................................................................................................................... 37

Challenges and Responses ............................................................................... 39

Delays to requests from D-RASATI 2 to give feedback or approvals due to limited resources at MEHE and CERD ................................................................................................................................ 39 Outdated/incomplete data at MEHE on schools and school staff .................................................... 42 Lower attendance to some trainings than anticipated..................................................................... 43


iii

Abbreviations

AP Wireless access point AMIDEAST America-Mideast Educational and Training Services AUB American University of Beirut ALLC American Lebanese Language Center-International House Beirut CAE Cambridge Advanced English CBM Computer Business Machine consortium CEFR Common European Framework Reference for Languages CERD Center for Educational Research and Development DOPS Guidance and Counselling Department (within MEHE) DOT Digital Opportunity Trust DQA Data Quality Assessment D-RASATI Developing Rehabilitation Assistance to Schools and Teacher Improvement D-RASATI 1 First phase of the D-RASATI program D-RASATI 2 Second phase of the D-RASATI program ECA Extracurricular Activities ELTM English Language Teaching on Methodology for primary schools ESS Effective Schools Standards FCE Cambridge English: First FED Faculty of Education at the Lebanese University IASC Inter-Agency Standing Committee ICT Information and Communication Technology LDP Leadership Development Program LU Lebanese University MEHE Ministry of Education and Higher Education MOU Memorandum of Understanding PCELT Professional Certificate in English Language Teaching PSS Psychosocial support RNA Rapid Needs Assessment RTWS Rapid Teacher Workforce Study SIP School Improvement Program/Plan TOT Training of Trainers UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children's Fund USAID United States Agency for International Development USAID/L USAID/Lebanon WL World Learning


1

Executive Summary

Project Overview MEHE has been leading efforts for national education reform through the adoption and implementation of a 5‐year Education Sector Development Plan (2010-2015) which aimed to improve the quality of the public education services. The D-RASATI program, approved by USAID in 2010, was designed to address the underlying reasons that hinder Lebanon’s public school system from providing high-quality education, in par with the private school system. In 2012, the first phase of D-RASATI (D-RASATI 1) resulted in the production of a -Project Master Plan – aligned with MEHE’s Education Sector Development Plan – to guide the program’s efforts to improve the learning environment in public schools and ultimately student achievement. The second phase of the program, D-RASATI 2, was implemented from 2013 to 2016, building on the achievements of D-RASATI 1. D-RASATI 2 is a capacity building project, and fits within the USAID/Lebanon Country Development Cooperation Strategy (CDCS). It responds to the CDCS Development Objective 1 “Improved capacity of the public sector in providing transparent and quality services across Lebanon” and IR 1.1 “Improved public basic education available to all school-aged citizens”, specifically sub-IR 1.1.1 “School learning environments improved” and sub-IR 1.1.2 “School learning opportunities increased”. Based on the Project Master Plan, D-RASATI 2 produced several documents to guide MEHE’s policy formulation and implementation to modernize the public schools’ services with regards to the integration of ICT, extracurricular activities (ECA), and psychosocial support (PSS); equipped schools with ICT Tier Model technology; financed school improvement projects; and upgraded the knowledge and skills of practitioners on ICT, English language proficiency and teaching methodology, ECA, school leadership, and school improvement processes. D-RASATI 2’s design was based on the theory that operational and technical expertise acquired by government officials and school staff will help them make lasting change to their practice, enhance learning environments and improve instruction in public schools. Ultimately, these changes will result in better learning outcomes for students, and raise public trust and confidence in Lebanon’s public school system. D-RASATI 2’s scope of work also included specific activities to help MEHE cope with the burden placed by the Syrian crisis on the public education sector. These activities were designed based on a rapid needs assessment (RNA) of a sample of primary public schools most affected by the Syrian crisis (host schools) which determined the immediate school needs in terms of supplies and trainings.

Project Accomplishments D-RASATI 2 achieved, and often overachieved, its objectives and set targets, despite its short life and the implementation challenges it faced. As per D-RASATI’s Master Plan, and in contribution towards USAID’s Development Objective 1, the project developed strategies and guidelines for MEHE to guide decision making and action in the following areas:

. Integrate ICT in education. D-RASATI 2 produced the “Lebanon National Educational Technology Action Plan”, the roadmap for the integration of technology in the Lebanese public school system. The Action Plan provided a comprehensive and collective approach for ICT integration in public school education with milestones and budgetary expectations,


2

covering Access to Technology, Curriculum and Instruction, Assessment, Professional Development, and Leadership. The Action Plan was validated by representatives of MEHE and CERD during several workshop meetings. The project also developed pioneer ICT training material for the secondary school levels for Math, Sciences and English to provide CERD trainers with sample lessons integrating ICT and equipping them to develop similar material in the future for other subjects and other grades.

. Expand extracurricular activities (ECA) at intermediate and secondary schools. The project developed an ECA strategy in line with MEHE’s Effective School Standards (ESS) where definition, understanding, and vision for ECA was unified among all stakeholders. The strategy highlighted the establishment of policies and procedures to provide ECAs for all students in support of the academic educational process. D-RASATI 2 also developed 7 ECA modules to practically support and encourage ECA planning and implementation of ECA activities at the school level.

. Integrate psychosocial support within the public school system. D-RASATI 2 produced 3 main documents in support of MEHE’s efforts to alleviate the burden caused by the Syrian crisis on the public primary host school sector. The first document was a recommendation report based on a rapid needs assessment (RNA) of a sample of primary public schools located in communities with high concentrations of Syrian refugees. The RNA highlighted the immediate need for training teachers in general, and for training teachers on the integration of PSS activities in the classroom in particular. The second product was the development of pioneer training material on the integration of PSS activities in the classroom; the material was customized to the needs of the formal education sector and adapted to the Lebanese context. The third document was a PSS Policy Recommendation Paper which provided MEHE with an overall vision and guidelines for the institutionalization of PSS in the public school system.

In fulfillment of its objectives under IR 1.1 – sub-IR 1.1.1 “School learning environments improved”, D-RASATI 2’s activities were in line with D-RASATI Master Plan regarding:

. ICT Integration in education. The project equipped 176 schools, 6 CERD centers and DOPS’ central office with ICT Tier Model technology, 40% beyond its target of 126 schools;

. Implementation of MEHE’s Effective School Standards (ESS) in relation to ECA, LDP and SIP. Under the project, 732 public schools were thoroughly introduced to MEHE’s ESSs and implemented projects that reflect some aspects of these standards. The original target of 583 schools was exceeded by 26%;

. Support to primary host schools most affected by the Syrian crisis. The project distributed stipends in the form of school supplies to alleviate the burden caused by overcrowding of students and lack of school fund, and PSS activity kits to some of the most affected schools.

In fulfillment of its objectives under IR 1.1 – sub-IR 1.1.2 “School learning opportunities increased”, D-RASATI 2 delivered 7,175 training units in 30 separate training programs – 1,026 units were delivered to officials and administrators and 6,149 to teachers and educators. This was 14% above the target of 6,280 training units.

. Delivery of Trainings to teachers, teacher trainers, DOPS and MEHE officials. The trainings aimed to upgrade the knowledge and expertise of the public school workforce and to embed effective educational methodologies in the public school system for the provision of higher


3

quality educational services. The trainings covered ICT integration in the classroom; English language proficiency and teaching methodology; ECA integration at the school level; school leadership; and school improvement plans. In 4,589 of the training units for which successful training is measured, the participants achieved the training goals based on the criteria of the respective trainings – 641 officials and administrators and 3,948 teachers and educators, hence, exceeding the original target of 3,860 by 19%.

Impact Based on the 4-level Kirkpatrick Model – the worldwide standard for evaluating the effectiveness of trainings –, the project conducted trainee surveys to measure impact at the first two levels of the Model, i.e. trainee response to the training, and assimilation of the new knowledge and skills and increases in efficacy to perform their work. Due to the short implementation period for each of D-RASATI 2’s trainings, the impact of the project activities on the quality of the public education services and the performance of the students could not be measured at the end of the project. Overall, 96% of the 4,795 trainees who completed a satisfaction-with-training survey responded that the trainings were relevant to their professional development needs and that they were satisfied with the training modus operandi – 61% highly satisfied, and 35% satisfied. As to the assimilation of modern knowledge and methodologies taught, the analysis of pre and post-training tests showed that 87% of the 3,326 trainees who completed both tests showed increased proficiency after receiving the trainings. On average, proficiency increased by 40%. In addition, the Generalized Self-Efficacy survey (GSE) which measures the increase in capacity to perform at work showed that 54% of the trainees reported an increased sense of self-efficacy – 54% for women and 56% for men. These findings confirm the significant positive impact of D-RASATI 2 trainings on the trainees. It can be safely inferred that this positive impact will be also reflected on the quality of the public school education services, and ultimately, on the performance of students. The premise is that if the training is relevant to the trainees, they are satisfied with what they learned, and their proficiency has increased, the training is most likely to result in lasting changes that boost performance and impact on the ultimate beneficiaries, the students, given an enabling environment. In addition, two activity evaluation studies were conducted; the internal impact study of the PSS trainings conducted by World Learning; and the independent evaluation of the English language trainings conducted by the Cambridge English Language Assessment Center. The PSS impact study concluded that the PSS training increased teachers’ implementation of practices that support students’ psychosocial wellbeing, and that there were measurable improvements in the classrooms environment and in the general psychosocial wellbeing of the children. However, given the short 2-months timeframe between the baseline and endline assessments, the levels of reported friction between students did not change following the training. As for the evaluation of the English language trainings, it found that teachers felt more confident teaching in English; teachers also reported the adoption of the pedagogical practices of their English language trainers, which were based on a communicative methodology, and were both motivated and inspired to try new methods of teaching as a result. Moreover, an independent performance evaluation of the project was conducted by the USAID-funded Performance Management and Support Program for Lebanon (PMSPL). Over a three-week period in August-September 2016, the PMPL assessed the effectiveness of the project in achieving its planned results, and the sustainability of the benefits accrued from its activities. The evaluation concluded that “Overall, D-RASATI 2 was able to achieve—and [in] some cases overachieved—its intended results during a short project timeframe to meet its objectives. The project’s strong leadership and management, adherence to the Master Plan, and approach to working with MEHE


4

and CERD resulted in successfully implemented activities. [However,] Given delays inherent [to a] in close collaboration with the MEHE, the short project life—initially two years—was insufficient for a project of this scope and scale.”

Sustainability The likelihood for sustaining and expanding the benefits accrued from project activities is high as a result of good practice in activity design and implementation: The project constantly worked towards securing buy-in and ownership of MEHE, CERD, and DOPS through close coordination and continuous consultations with, and the provision of technical advice to, MEHE and CERD. This was complemented with assessments and gap analyses during project implementation when possible. Accordingly, the project refined the vision for training themes to respond to the needs of the public school staff. All these efforts contributed to embedding technical capacity in the public sector, and to the enhancement of the sustainability of the D-RASATI 2’s interventions. Project activities were designed to target all strata of the public school workforce. MEHE staff who play a role in policy making and implementation; CERD trainers and DOPS counselors who have a vital role in knowledge transfer to teachers during and beyond the life of the project; school principals who manage the public schools; and school teachers and staff who are in direct contact with students were all targeted within the activities being implemented. Project training materials were based on MEHE’s priorities and were adapted to the Lebanese curriculum and to the context that CERD can use to train school principals and teachers in the future. The independent project performance evaluation confirms the above as it states “MEHE and CERD had buy-in for D-RASATI 2’s plan and generally found it relevant to their priorities. This was especially true for the ICT Action Plan, English, LDP, and SIP. They also accepted the relevance of ECA and PSS, but were less fully invested at the start”. It also states “The evidence collected shows that CERD plans and has the capacity, build through TOTs, to take D-RASATI 2 capacity development material forward as part of its official teacher trainings and likewise for MEHE. Key informant interviews also indicated that DOPS and CERD operate as separate bodies through different channels of reporting, and that D-RASATI 2 was exceptional at targeting both bodies”. The evaluation concludes that “Broadly, most D-RASATI 2 activities that involved TOT are likely be sustained through institutionalization and adoption by MEHE and CERD. Engaging with CERD was essential to this sustainability”.


5

Introduction

Project Overview and Rationale

On September 13, 2013, USAID awarded World Learning the 2nd phase of the Lebanon Developing Rehabilitation Assistance to Schools and Teacher Improvement program – D-RASATI 2 – under the FORECAST 2 IDIQ mechanism. D-RASATI 2 was a 2-year Cost-Plus-Fixed Fee (Completion) task order that was originally valued at $24.5 million. In 2014, given the gravity and the sensitivity of the situation produced by the Syrian crisis and its impact on the hosting communities and the public education system, D-RASATI 2 received $4.5 million supplementary funds to expand existing activities to target host schools1. Accordingly, the project completion date was extended till December 2016. D-RASATI 2 is the 2nd phase of the D-RASATI program (2010 – 2016) which aimed to address the underlying reasons that hinder Lebanon’s public school system from providing high-quality education, in par with the private school system. D-RASATI 2 fits within the USAID/Lebanon Country Development Cooperation Strategy (CDCS) and responds to the Development Objective 1 “Improved capacity of the public sector in providing transparent and quality services across Lebanon” and IR 1.1 “Improved public basic education available to all school-aged citizens”, specifically sub-IR 1.1.1 “School learning environments improved” and sub-IR 1.1.2 “School learning opportunities increased”. D-RASATI 2 was implemented in all 6 Lebanese Governorates (Box 1). The general perception in Lebanon is that private schools are able to provide better education services because of better resources. However, private education is more expensive than public education. Unlike public school education, private education gives students access to innovative

1 Host public schools count at least 20% non-Lebanese among their student population

Box 1 – D-RASATI 2 areas of intervention


6

educational institutions that provide Lebanon’s youth with the 21st century skills needed to succeed in today’s social and economic environment. Those students who graduate from public schools have fewer opportunities than their counterparts who receive private education to enter universities or obtain productive jobs. As a result, graduates from the private sector have higher chances to obtain more productive and profitable jobs. In essence, the low quality of public education has resulted in a polarized education market where divisions between the rich and poor are stark. The Lebanese Government did not sufficiently invest in the development and modernization of the public school education sector over the past years in order to provide better public education services. Shortages in investments have resulted in a number of challenges which the public education sector continues to face2:

. An outdated national curriculum, which was last updated 18 years ago, in 1998;

. Limited quality training and continued-education opportunities for school administrators and teachers which led to shortages in a skilled public education body that uses modern education methodologies and tools. In particular, there is a severe shortage of qualified English and French language teachers who can teach math and science in these languages, especially in the intermediate and secondary stages;

. Inexistent national standards for systematically measuring student learning, aside from at the level of the official high-stake “Brevet” and “Baccalaureate” certification grades;

. Few or inexistent basic educational and technological equipment at schools. Most schools are not equipped with computer and science labs; and

. A poorly maintained infrastructure3, whereby the majority of schools face challenges of inadequate bathrooms, heating, electricity, and physical structure (broken windows, leaking roofs).

All of these factors contributed to poor school test scores as well as an increase in the number of youth who drop out of school earlier than desired. The large Syrian refugee influx, since the onset of the Syrian crisis in 2011, has exacerbated this long-standing situation. Today, Lebanon has the highest number of refugees per capita in the world4. As this crisis continues, the number of Syrians seeking refuge and education in Lebanon stays on the rise. The related emerging issues magnified and accentuated the Lebanese public educational system deficiencies in terms of both infrastructure and teacher capacity. The poor maintenance and limited space of the school’s infrastructure have made it more difficult to absorb and retain the high number of refugee students. Teachers and school staff are not trained to deal with, and instruct, refugee children suffering from psychological distress, which directly affect teachers and the quality of education delivered in the public schools5. Additionally, most subjects in the Lebanese curriculum are taught in foreign languages which was a serious obstacle for Syrian students who were previously enrolled in the all-Arabic Syrian curriculum schools. As a consequence, the Lebanese public education system was not equipped to integrate a continuously increasing flow of vulnerable students. This is evidenced by a 70% dropout of Syrian refugee students from public schools in 2011-20126.

2 UNESCO. Various dates, last of which is May 2016. Institute for Statistics: http://data.uis.unesco.org/ 3 USAID. 2016. https://www.usaid.gov/lebanon/education 4 “Comparing the number of refugees to the national population of a host country shows that Lebanon topped this list with 206 refugees per 1,000 inhabitants”. http://www.unhcr.org/56655f4e0.pdf 5 World Learning. 2014. “Education Rapid Needs Assessment”. D-RASATI 2 project. 6 Hana A. El-Ghali, Nadine Ghalayini, Ghida Ismail. (2016, March). Responding to Crisis: Syrian refugee education in Lebanon, Policy Brief, 7, AUB policy institute, Beirut.

http://data.uis.unesco.org/

https://www.usaid.gov/lebanon/education

http://www.unhcr.org/56655f4e0.pdf


7

MEHE has been leading efforts for national education reform and has adopted a 5‐year National Education Strategy which highlighted its priorities in providing quality education. In 2012, the first phase of D-RASATI (D-RASATI 1) resulted in the production of a Five Year Master Plan for MEHE –aligned with MEHE’s 5‐year National Education Strategy – to guide the program’s efforts to improve student and teacher performance. D-RASATI 1 also refurbished 183 public schools and equipped prioritized schools with science laboratory equipment. D-RASATI 2’s design is based on the theory that operational and technical expertise acquired by government officials and school employees through the project will help them make lasting change to their practice; and that use-focused standards, better ICT resources, English language proficiency of teachers, school leadership, community engagement, and extra-curricular activities will enhance learning environments and instruction in Lebanon’s public schools, ultimately resulting in better learning outcomes for students. All this will raise public trust and confidence in Lebanon’s public school system. D-RASATI 2 built on the achievements of D-RASATI 1, and implemented specific activities outlined in the five‐year master plan aiming to:

1. Improve the learning environment in public schools through the procurement of Information Communication Technology (ICT) equipment and ICT training;

2. Increase learning opportunities through English teacher training and extra‐curricular activities (ECA);

3. Increase stakeholder engagement in the public school through leadership development with parents and community involvement;

4. Increase student access to higher quality instruction and learning environments by providing teacher training on topics and methods for managing new pressures on classrooms in schools hosting communities;

5. Improve social cohesion among Lebanese students, families, and Syrian refugees in the classroom and community, and establish a mechanism to closely monitor these activities and measure a reduction in tensions between students and the school level.

The Project Components

D-RASATI 2 consisted of 3 components and 6 tasks, as follows: Component 1: Provision of Equipment and Computer Labs to Public Schools

Task 1: a. Develop an ICT action plan based on MEHE’s National ICT in Learning Strategy – to guide the deployment of ICT to the entire public school system; b. Develop ICT TOT training modules and pilot them in an ICT TOT to CERD trainers, and revise in light of feedback received; c. Implement follow-on steps; d. Provide extended online TOT coaching and support to CERD trainers

Task 2: Procure ICT Equipment for public schools and CERD Training Centers to implement the 1st phase of MEHE’s 5-year Master Plan in 126 schools; and procure 51 additional Tier 2 Model ICT equipment to 50 additional secondary schools and the DOPS central office

Component 2: Provision of In‐Service Public School Teacher Training

Task 3 – Provision of trainings on English language proficiency and teaching methodology: a. Administer the BULATS English language placement test; b. Train public school teachers in the English language after their English language level is determined in the BULATS test; c. Deliver PCELT course and certification to CERD trainers; d. Deliver TOT training on English language instruction and methodology (ELTM) to PCELT-certified CERD trainers; e. Train


8

teachers with advanced language skills on ELTM; f. Certify advanced-level teachers on FCE and CAE

Task 4 – Provision of trainings to school principals and teachers to encourage the integration of ICT in teaching and learning: a. Train and prepare school administrators to integrate technology in their schools; b. Provide in-school ICT coaching to teachers; and Train CERD and DOPS representatives, and provide refresher training to school teachers, on ICT equipment use – to integrate ICT in classrooms teaching

Task 5 – aiming to institutionalize ECA in the public school system, and to provide support to host schools including the integration of psychosocial support concepts at the classroom level: a. Expand extra‐curricular activities (ECA) at intermediate and secondary schools – including the development of a comprehensive national strategy to promote ECA within the education system, and the development of 7 ECA modules to guide schools in designing and implementing ECAs; the training of school representatives on implementing ECA; the delivery of ECA workshops to MEHE officials and ECA TOT to CERD trainers; and the distribution of the ECA modules to all public intermediate and secondary schools; and b. Provide support to targeted host schools to alleviate pressure on public schools hosting high numbers of Syrian refugee students. Activities include the implementation of a Rapid Needs Assessment (RNA) and a Rapid Teacher Workforce Study (RTWS); the provision of trainings on psychosocial support (PSS) to CERD trainers, DOPS counselors, and teachers from primary host schools; the sensitization of school principals on PSS in emergency; and an assessment of the trainings’ impact on reducing tensions at schools.

Component 3: School Leadership Development with Parents and Community Involvement

Task 6a – Enhance the leadership skills and practices of school principals: Implement the school Leadership Development with parents and community involvement activity (LDP) through the delivery of LDP training to school principals;

Task 6b – Enhance the ownership and practices of school teams to lead the efforts to improve their school learning environment: a. Deliver SIP training and coaching to SIP school teams; b. Provide the SIP schools with up to $4,000 per school to implement SIP projects; c. Develop an awareness campaign; d. Provide a $1,100 stipend to finance the urgent teaching and learning materials needs of targeted host schools located in communities with a high concentration of Syrian refugees.

Roles and responsibilities of World Learning and its implementing partners

To ensure an efficient, high-quality, results-oriented implementation of D-RASATI 2, World Learning assembled a project team with strong local ties to the community, theoretical and practical technical expertise, and nation-wide education reform management experience. World Learning partnered with the American University of Beirut (AUB), the American-Mideast Educational and Training Services, Inc. (AMIDEAST), the American Lebanese Language Center (ALLC), and the Digital Opportunities Trust (DOT). The REED Company was contracted to provide security and risk management services for the project. The roles and responsibilities of World Learning and its implementation partners were as follows:

Organization Primary Role(s)

World Learning (WL)

Ensures oversight of all D-RASATI 2 activities to make sure that D-RASATI 2’s goals are met Leads the Provision of Equipment and Computer Labs to Public Schools Leads the Training and Certification of Public School Teachers in English Language (PCELT);

Delivery of TOT training on English language instruction and methodology (ELTM) to 36 PCELT-certified CERD trainers; Training up to 300 primary school teachers with advanced language skills on ELTM

Leads Training and preparing school administrators to integrate technology in their schools Leads the implementation of activities in support of targeted Host Schools


9

Leads on the provision of a $1,100 stipend to procure items requested by the primary host schools most in need

American-Mideast Educational and Training Services, Inc. (AMIDEAST)

Leads on Expanding extra-curricular activities (ECA) at intermediate and secondary schools Leads on Delivering LDP training to school principals Leads on the Delivery of SIP training and coaching to SIP school teams; Provision of the SIP

schools with up to $4,000 grant per school to implement SIP projects; and on Developing an awareness campaign

American University of Beirut (AUB)

Leads on Developing an ICT action plan based on MEHE’s National ICT in Learning Strategy – to guide the deployment of ICT to the entire public school system; Developing ICT TOT training modules and pilot them in an ICT TOT to CERD trainers, and revise in light of feedback received; and Implementing follow-on steps

American Lebanese Language Center (ALLC)

Delivers BULATS tests and FCE and CAE certification, and trains public school teachers in the English language

Digital Opportunities Trust (DOT)

Provides ICT in-school coaching for teachers; and Train CERD and DOPS representatives, and provide refresher training to school teachers, on ICT equipment use

REED Provides security and risk management services for the project

D-RASATI 2 has directly contributed to upgrade and modernize the Lebanese public school system and enhance learning environments and instruction. This is expected to lead to better learning outcomes for students and increased public trust in Lebanon’s public school system. The project elaborated strategies and guidelines for MEHE, equipped public schools with modern ICT, financed school development projects, and procured school supplies and books to meet the urgent needs of schools most affected by the Syrian crisis. It also developed training materials and transferred operational and technical expertise to the administrators and teachers of public schools on ICT integration in education, English language proficiency and methodology, the modern concepts of school leadership and improvement, and the use of extra-curricular activities and psycho-social support at the schools. At the same time, it embedded expertise and knowledge within the public education institutions (MEHE and the Center for Educational Research and Development (CERD)) to foster the sustainability and continuity of benefits accrued from the project.

Project Accomplishments

Development Objective 1: Improved Capacity of the Public Sector in Providing Transparent and Quality Services Across Lebanon

In line with its contractual deliverables, D-RASATI 2 developed strategies and guidelines for MEHE to guide decision making and action to integrate ICT in education, expand extracurricular activities at intermediate and secondary schools, and support MEHE in alleviating the burden placed by the Syrian crisis on the public education system.

ICT integration in education

In 2015, and in close collaboration with CERD, DOPS, the Educational Inspectorate, and the Faculty of Education at the Lebanese University (LU-FED), D-RASATI 2 produced the “Lebanon’s National Educational Technology Action Plan”, which represented the roadmap for the integration of technology in the Lebanese public school system. The Action Plan used technology integration standards and indicators set by the International Society for Technology in Education (ISTE)7, and was validated by representatives of the stakeholders during several workshop meetings.

7 ISTE standards are used worldwide to create student-centered learning environments in which students are creators, innovators, researchers, critical thinkers and ethical digital citizens.


10

The Action Plan is divided into 5 components that address the strategic goals and objectives of the NETSP. The document recommended that the implementation process encompass all components simultaneously, since they are intertwined and provide a comprehensive approach for the integration of ICT in Lebanon’s public education schools. In brief, the components are:

1. Component 1 – Access to Technology in terms of physical infrastructure, connectivity, tools, and capacities of human resources. This component recommends the use of the Education Enterprise Architecture (EEA) tool to host the interdependent components of ICT integration into a cohesive cost-efficient structure.

2. Component 2 – Technology and Curriculum and Instruction. This section recommends

revising the national curriculum to identify the needs for technology integrated lesson plans and e-content. This is to enhance student learning by allowing interactive and authentic learning experiences. The first step would be to develop technology integrated lesson plans, and standards for e-content for selected grade levels and selected subject matters. Piloting the e-content would provide the needed feedback and recommendations to tailor the e-content and lesson plans and rollout this process to all grade levels and subject matters.

3. Component 3 – Technology and Assessment. This component addresses the need for a

modern assessment infrastructure that includes assessment rubrics, technology standards and performance indicators. These tools would be piloted to collect feedback which will be used to refine the assessment process.

4. Component 4 – Technology and Professional Development. This section recommends

conducting a comprehensive needs assessment of teachers and coaches’ technical abilities and ICT integration skills to serve as a baseline for the development of relevant face-to-face and online training modules for the teaching workforce. Related recommendations include utilizing web-based tools to connect trainers, experts, teachers and coaches; and provide an online resource for digital content.

5. Component 5 – Technology and Leadership. This section addresses the critical role of

leadership in integrating technology and thus the need for educational leaders to possess the necessary set of skills to drive strategic ICT planning. It recommends programs to empower and equip the strategic leadership to facilitate the development and implementation of technology integration plans at the school, regional, and central levels.

D-RASATI 2 was the vehicle for implementing several activities recommended by the ICT Action Plan. The first is the development of ICT training material for CERD in the form of 15 training manuals for the integration of ICT into the curriculum of the secondary school levels (grades 10, 11 and 12) for the Math, Sciences and English subject matters. The ultimate purpose of the training manuals was to provide trainers with sample lessons integrating ICT into their subjects, as well as, to equip them with the needed skills to develop similar training manuals on their own when they roll out the training to teachers. The training manuals would enable CERD to build the capacity of public school teachers to integrate technology into classroom content – thus bringing the ultimate vision of the National Educational Technology Strategic Plan and the ICT Action Plan to fruition. The design of the training material covered 6 days – divided into 2 sets of training content, the 1st for a 3-day instruction on generic ICT tools, and the 2nd for a 3-day instruction on subject-specific ICT tools. The generic ICT tools included software such as wikis, blogs and Google sites, and the subject-specific tools included Science probe ware and subject-specific software used for conducting simulations. To enable a wider use of the training material for Math and the sciences the manuals were translated


11

into French; this was to accommodate for the large number of trainers and teachers in the public school system who are not proficient in English and who are to be trained in French. The experts who developed the ICT TOT material also developed protocols and checklists to be used by the CERD trainers in the preparation and the implementation of future ICT trainings. The protocols included steps for the preparation, the implementation and the assessment of ICT trainings. The other activities that are included within the ICT Action Plan and which D-RASATI 2 implemented include the development of an online platform to connect ICT trainers and experts; assessment rubrics for teachers, e-content standards and performance indicators; the training of CERD trainers, DOPS teacher mentors, and teachers on the integration of ICT in the classroom and on the use of the ICT Tier Model adopted by MEHE; and the procurement of ICT equipment to secondary public schools designated by MEHE. These activities and their achievements are discussed in the next sections.

Expand extracurricular activities at intermediate and secondary schools

The project team worked closely with the ECA strategy development committee from MEHE to design a comprehensive National ECA Strategy with the model to institutionalize ECAs within the public school system. The committee consisted of members representing DOPS, the Cultural Affairs and Fine Arts Service, the Sports and Scouts Unit, the Secondary School Directorate, the Regional Educational Directorates and a selected group of school principals. The ECA strategy was developed in line with MEHE’s Effective School Standards published by MEHE in 20148. Specifically, it responded to indicator 1 under the “School life and the development of students on the personal and social levels” field which states, “The provision of extracurricular activities for all students that supports the learning and teaching and educational process.” The final ECA strategy document unified the definition, understanding, and vision for extracurricular activities among all stakeholders. Extracurricular activities were defined as “purposefully designed spaces where all students can develop knowledge, practice skills, discover interests, cultivate attitudes, and actively apply their learning in an environment that encourages them to express themselves freely beyond classroom restrictions”. The strategy envisioned that extracurricular activities be embedded in the educational experience of each student, and it set the goal of offering ECA programs in 90% of public schools in Lebanon by the year 2020. The strategy highlighted the support and actions needed to achieve the overarching goal, a major component of which is the establishment of policies and procedures that focus on defining the roles and ensuring the commitment and cooperation of all participants as shown in Figure 1. It recommended to invest in ECA-related capacity-building opportunities and resources to enhance the current professional development programs and embed skills for the design and implementation of ECA at schools. It also highlighted the fact that providing schools with ready-made ECA modules, schools will be more prepared to offer extracurricular activities for their students. Adding a practical foundation, D-RASATI 2 developed 7 extracurricular programs in the form of booklet guides for the implementation of 25 ECA activities. The ECA thematic areas covered culture and language; citizenship and community service; arts; ICT; science; and project-based activities.

8 MEHE’s Effective School Standards (ESS) was completed in March 2014 under the World Bank-financed Education Development Project 2


12

The booklets were distributed to all intermediate and secondary public schools along with an ECA stationary and arts kit to facilitate the immediate implementation of ECAs at the level of the school.

Support to host schools

In order to provide MEHE with the most targeted and effective support in responding to the overwhelming needs of the Syrian refugee students, D-RASATI 2 conducted 2 assessments of the teachers’ training needs in the primary public schools located in communities with high concentrations of Syrian refugees. Two months after MEHE opened afternoon second shifts in 79 of its primary public schools, with the assistance of the UNHCR9, the project conducted the 1st rapid needs assessment (RNA) from February to March 2016. The RNA collected and analyzed current data from the field on the day-to-day challenges facing students, teachers, and administrators, and resulted in a series of recommendations to guide MEHE’s and D-RASATI 2’s consultations and decisions regarding the project’s interventions at the level of the host schools. In 2014 and 2015, a rapid increase was registered in the number of school-aged Syrian refugee students which had serious implications on the public schools located in the host communities. This necessitated an update of the situation on the ground to inform the design of effective action plans that could efficiently respond to the needs of public schools hosting Syrian refugee students. To this end, D-RASATI 2 planned a Rapid Teacher Workforce Study (RTWS) in 2015 to complement the RNA and inform decisions on how to provide adequate education services to the high numbers of Syrian students that come from not only diverse academic background, but also from diverse social and political backgrounds. However, MEHE did not consider the RTWS as a priority and this created a major limitation for the project to complete this deliverable. The project team was not able to collect additional data from the field and resorted to tapping into relevant data available in 2016 from other studies and activities conducted by the project. The project compiled the RTWS by using the metasynthesis methodology10 of all relevant findings and data collected and analyzed by the project between 2014 and 2016. The data sources were the RNA; feedback collected from PSS trainees through several surveys at different stages of the PSS trainings; and the PSS impact study. PSS Policy Recommendation Paper: A PSS Policy Recommendation Paper was developed by the project to provide MEHE with an overall vision and a pathway map for PSS institutionalization in public schools.

IR1.1 Improved public basic education available to all school-aged citizens

D-RASATI 2 worked with 1,252 (98%) of the Lebanese public schools through either making a direct intervention at the school level to benefit the learning environment, or through training the school administrative and teaching body to transfer knowledge and expertise. D-RASATI 2’s activities indirectly benefitted a total of 310,409 public school students.

9 The second shifts aimed to absorb 27,550 Syrian refugee children and offer them a lighter version of the regular Lebanese curriculum. 10 Metasynthesis is an intentional and coherent approach to analyze data across studies. It is a process that enables researchers to identify a specific research question and then search for, select, appraise, summarize, and combine evidence to address the research question. Elizabeth J. Erwin. Journal of Early Intervention 2016. Department of Early Childhood, Elementary and Literacy Education, College of Education and Human Services, Montclair State University, Montclair, NJ. http://jei.sagepub.com/content/33/3/186.abstract

http://jei.sagepub.com/content/33/3/186.abstract


13

Sub-IR 1.1.1 School learning environments improved

Several of D-RASATI 2’s activities had a direct impact on the learning environment at the school. These consist of:

. the procurement of Tier Model ICT equipment to 176 secondary schools, 6 CERD centers, and the central DOPS office;

. the design and implementation of LDP and SIP projects at 679 schools which met MEHE’s Effective School Standards;

. the provision of grants to improve the school learning environments at 191 schools, and

. the provision of teaching and learning material to increase access to quality education and meet the urgent needs for school supplies and books of 64 primary public schools located in communities most affected by the Syrian crisis.

Procurement of Tier Model ICT equipment

Procurement of ICT equipment is a necessary first step to support the Lebanese public school system to achieve MEHE’s vision for technology and learning. ICT procurement under D-RASATI 2 was informed by MEHE’s National Educational Technology Strategy, the results of a school survey conducted under D-RASATI 1, and the equipment standards and specifications elaborated in close consultation with MEHE under D-RASATI 1 as well. The project aimed to complete MEHE’s Phase 1 of technology provision as presented in the D-RASATI Master Plan, i.e. to provide ICT equipment for 126 secondary public schools. In close consultation with MEHE, the project elaborated a statistical model using the school-level data collected under D-RASATI 1 to select 126 secondary schools based on their readiness to receive and operate ICT equipment and on an equitable geographic distribution among the 6 Lebanese Governorates. An ICT readiness assessment was conducted for the 126 schools to update the data collected under D-RASATI 1 in terms of availability of network connectivity, electricity, space to place and use the equipment, and the overall ICT culture at the school. This was used to further refine the ICT needs of the schools in line with MEHE’s criteria for technology placement in schools. D-RASATI 2 worked closely with MEHE to refine the technical specifications of the ICT equipment and the quantities assigned for each technical tier factoring in the current technology updates available on the market. In parallel, D-RASATI 2 completed a market survey to estimate the budget needed for each type of equipment, and conducted a gap analysis to determine individual school needs against the technical specifications and the refined tiers. The project then developed a list of items to be procured for each school based on the tiered technology configurations outlined in the National Educational Technology Strategy. The ICT equipment included laptops and student tablets, LCD overhead projectors, networking equipment, and software such as the LANschool to connect teachers and students in the classroom. Early on, MEHE requested D-RASATI 2 to include the 6 CERD regional training centers among the beneficiaries of the ICT equipment procurement. This was not foreseen in the D-RASATI 2 budget and scope of work, but USAID and D-RASATI 2 accommodated MEHE’s request to increase the sustainability and continuity of the ICT procurement benefits. Additionally, MEHE requested D-RASATI 2 to finance internet and network connectivity within the beneficiary schools. Based on budget availability, this request was also accommodated but only for a selected number of schools assigned to the highest technological tier, i.e. Tier 4 schools. To optimize the use of the D-RASATI 2 funds, the project negotiated with MEHE on the possibility to leverage existing resources at MEHE to benefit the targeted schools. MEHE was able to extend its current agreement with Microsoft – to purchase software at a special reduced price – to cover software procured by D-RASATI 2, allowing for substantial savings on software expenditure.


14

Subsequently, MEHE organized the schools per technological tier and the ICT procurement proceeded accordingly. By May 2015, all ICT equipment was delivered, installed and on-site tested at the 126 schools and CERD centers. The 3-year maintenance plan was activated to ensure the smooth operation of the equipment during the 2015-2016 academic year, and MEHE took over full ownership and responsibility for the ICT equipment delivered. At the beginning of the 2015-2016 academic year, network disruptions and software/hardware malfunctions were reported by the schools, at which point the root causes were investigated by D-RASATI 2 and the vendor. This resulted in updating the different software and the replacement of the malfunctioning CISCO wireless access points (APs) with the ASUS APs free of charge. Field visits to the schools and CERD centers confirmed that the ICT equipment received through D-RASATI 2 is well functioning and being used by the teachers at the schools. In August and September 2016, the project used savings from various activities to procure supplementary ICT equipment for the 126 schools and 6 CERD centers; such as server monitors, projector screens and mobile carts, and to equip 50 additional secondary schools and the central DOPS office with the Tier 2 ICT Model, at the request of MEHE. By December 28, 2016, all the equipment was delivered, installed and on-site tested. On December 22, 2016, D-RASATI 2 officially handed over to MEHE the responsibility for overseeing the continued use and maintenance of the equipment.

Implementing MEHE’s Effective School Standards in relation to ECA, LDP and SIP

The Effective School Standards (ESS) document was finalized in March 2014 under the World Bank-financed Education Development 2 project which ran in parallel with D-RASATI 2. The ESS document provided detailed indicators under 10 themes that include the school’s vision and mission, school leadership and management, teaching and learning, resources and infrastructure, professional development, building partnerships with parents and the local community, and school life and student development. D-RASATI 2 referenced the ESS in all activities related to ECA, LDP, and SIP; this was to ensure that targeted schools work towards achieving the requirement for the effective school profile. Extra-Curricular Activities (ECA): The “School life and the development of students on the personal and social levels” theme under the ESS highlights the role of the school in addressing each student’s non-academic needs. The related indicator 1 “The provision of extracurricular activities for all students that supports the learning and teaching and educational process” highlights the importance of organizing regular ECAs at public schools. D-RASATI 2’s ECA-related activities set the stage to institutionalize ECAs in the public school system and empowered 125 intermediate and secondary schools to implement ECAs in accordance with MEHE standards; under this task the following was achieved:

1. The National ECA strategy developed under the project set a unified vision for ECA among all stakeholders. The first objective stated in the strategy aims at ensuring that the provision of ECAs for students is a fundamental responsibility of each school. The strategy recommends that MEHE create an ECA-conducive environment and support for schools to be able to organize ECA effectively.

2. Seven ECA modules booklets were produced by D-RASATI 2 and were distributed to the

1,03111 intermediate and secondary public schools. The modules detailed the step-by-step instructions for organizing and implementing ECA activities at the school level. To further

11 10 of the 1,041 intermediate and secondary public schools had been closed by MEHE by the time the ECA modules and kits were distributed to schools


15

facilitate the implementation of ECAs, D-RASATI 2 also procured and distributed ECA stationary and arts kits to the same 1,031 schools.

3. The ECA capacity building activities were as follows:

a. MEHE officials were equipped with the expertise and tools needed to support schools in organizing ECAs for students and facilitate ECA implementation, as ECA coordinators.

b. Representatives from schools were engaged to expand their knowledge, build their skills and develop their attitudes to effectively lead the organization and facilitation of extracurricular activities in their respective schools. These school representatives were equipped to create an enabling environment to play the role of ECA facilitators at their schools and implement extracurricular activities in line with MEHE’s ESS.

It should be noted that 39 schools participated in both the ECA and LDP programs. The schools’ ECA representatives who received the ECA training used the newly gained ECA knowledge and skills to support their school principal in implementing the elective module under the LDP. To facilitate the implementation of ECAs at the school level, the project distributed 7 copies of the 25 ECA activities booklets sets to each of the 1,031 intermediate and secondary public schools along with ECA stationary and arts kits. The booklets presented step-by-step instructions on how to organize and implement ECAs at the school level; and the ECA kits contained stationary and arts supplies to use for implementing ECAs. The project also provided 30 ECA kits to MEHE to use as seen fit. Leadership Development (LDP): D-RASATI 2’s LDP program resulted in several outputs which enabled 488 schools to achieve progress towards meeting MEHE’s Effective School Standards:

1. A consensus, facilitated by D-RASATI 2, was reached between MEHE and LU mandating LU-Faculty of Education (FED) to provide training to in-service school principals12. The related Memoranda of Understanding (MOU) in 2014 and 2015 gave the LU-Faculty of Education (FED) the authority to deliver certificates of successful course completion to school principals who were appointed to their position after April 23, 2009 and who fulfill the course success criteria. Having this certificate enabled the incumbent to receive the 15% salary increase mandated by law to school principals.

2. The LDP training delivered by D-RASATI 2 included 3 phases, the last of which was a practical

project-based (practicum) phase where principals put learning into action. With the help of the LDP trainers and coaches, school principals chose a topic for their practicum aiming at developing the educational and/or administrative processes in their respective schools. LDP projects designed and implemented by principals at 488 schools were fully streamlined within several of the categories under MEHE’s ESS:

a. Supervision: projects which fit under this ESS category aimed at adopting active

learning methods of teaching, and supporting underperforming students to improve their academic achievements especially related to Arabic, English or French language proficiency. Schools which implemented these projects achieved progress towards meeting 2 MEHE standards subcategories – “improved teaching and learning” and “school-life, personal and social development of the learners”;

12 An MOU already existed between MEHE and LU covering the inter-agency cooperation to provide pre-service training to school principals


16

b. Extracurricular Activities: LDP participants’ projects responding to this ESS category tackled different topics: health education, library activities, ecology, sports services, arts activities, school clubs, community service, school journal, and the integration of Syrian refugees. Schools which implemented these projects achieved progress towards meeting MEHE’s standards related to “school-life, personal and social development of the learners”;

c. ICT integration: Relevant projects fostered the computerization of the school

management system/Information system or library, and the use of available ICT resources in the teaching-learning process. Schools which implemented these projects achieved progress towards meeting MEHE’s standards subcategories – “improved learning and teaching resources” and “infrastructures and utilities”;

d. School management: projects under this category fostered improved school

management processes through improving the school collaboration with parents and the community; responding to community needs by creating preschool classes; developing a counseling system; elaborating behavioral rules; providing library/learning center services to the students; and enhancing the physical environment inside and outside the school. Schools which implemented these projects achieved progress towards meeting MEHE’s standards related to “Leadership and School Management”, “Human Resources” and “Infrastructures and utilities”;

e. Special needs services: projects under this category targeted children with learning

difficulties and emotional/physical special needs through training the parents and the teachers on the most effective ways to deal with children with special needs. Schools which implemented these projects achieved progress towards meeting MEHE’s standards related to “teaching and learning” and “achieving equity among the learners and supporting children with special needs”.

School Improvement (SIP): D-RASATI 2’s SIP Program aimed to empower school teams to lead their own school reform and improvement process. Besides providing training to school teams on the modern methods for defining school improvement goals and designing and implementing school improvement projects, the project provided schools with small grants to finance the winning project proposals. Achieving progress towards at least one of the MEHE standards was an inherent guiding principle in the design of the SIP projects, and the grant proposal documents included a section where schools identified the MEHE standard that each project would address and described how the project would help in improving that standard. All 191 schools which participated in and successfully completed the SIP program demonstrated “positive change” and improvement in relation to at least one of the MEHE standards. The SIP program addressed 6 main priority areas, all of which are in line with MEHE’s ESS:

1. Teacher training which addressed standards related to “improved teaching and learning” 2. Minor improvements in school environment which addressed standards related to “School

Infra-structure and other related services” 1. Development of remedial programs which addressed standards related to “Teaching and

Learning” and “School Life and student services” 2. Launching ECAs which addressed standards related to improve “School Life and student

services” 3. Increased parental and community involvement which addressed standards related to

“increased community engagement”


17

4. Acquisition of materials, equipment and resources which addressed standards related to enhanced “Teaching and Learning Resources”.

Furthermore, all 191 SIP schools registered progress towards achieving standards related to School Leadership and constant School Improvement:

1. School Leadership: One of the standards in this category is that the school has an environment that supports school improvement. By forming a committee and completing the school improvement program, it is inferred that all 191 schools showed improvement in this standard;

2. Constant School Improvement: This category outlines the school improvement process

where the standards that are used to describe it include gathering evidence and analyzing them to diagnose the school needs, conducting a school-study, designing and implementing a project, as well as monitoring and evaluating the implementation of the plan. The SIP training embedded this modern approach to school improvement in the school teams which participated in the program. All 191 schools received SIP grants on the basis that their proposed projects were the result of this participatory approach to planning, and thus, all schools have demonstrated improvement under this ESS standard.

In conclusion, the SIP program equipped schools with the capacity to align with MEHE standards. The program allowed the schools to thoroughly work with the MEHE standards and gave them the space to elaborate on some of them.

Provision of stipends and PSS stationary kits to 640 primary host schools

As part of its activities, D-RASATI 2 provided some of the urgent needs of 64 primary schools most

affected by the Syrian crisis. The project provided school supplies that can alleviate the burden caused by the overcrowding of students and lack of school funds, and the ensuing growing tensions among students. Based on the RNA findings and the requests of the school principals and teachers surveyed during the RNA, a list of the requested items, with their costs, was drawn and it included Sciences, language, and Math educational kits; electronic equipment such as projectors and projector screens, TVs, DVD players, portable CD players, and video cameras; white boards; and playground equipment. The list was circulated to schools and each school chose the items they were in most need of. The total value of the items received per school is $1,100. In total, 303 items were procured by the project and distributed to the 64 schools by May 2015. Following the training of teachers in primary host schools on the PSS 5-Day Essentials of PSS Course (detailed under the section “Sub-IR 1.1.2 School learning opportunities increased” of this report), PSS stationary kits were distributed to the 506 teachers who attended at least 50% of the training. The kits included stationary items that are needed to implement PSS activities in the classroom. Items include flipchart board and paper, construction paper, and other stationary items. At the request of MEHE, in November 2016, the project used savings from various activities to procure and distribute additional 464 PSS kits. These schools, designated by MEHE, and are expected to be put the kits to good use once the schools’ teachers receive PSS training through CERD’s in-service professional development program. The project also distributed 12 PSS kits to the regional CERD teacher training centers, at the rate of 2 kits per center, to facilitate the roll out of trainings to additional teachers from primary schools.

Sub-IR 1.1.2 School learning opportunities increased

In fulfillment of its objective to increase learning opportunities for the staff of the public school system, the project delivered 7,175 training units in 30 separate training programs – 2,128 (30%) to


18

men and 5,047 (70%) to women. 1,026 units were delivered to officials and administrators and 6,149 to teachers and educators. The project’s LOP target was 6,280 training units, and thus, the project over delivered against this indicator by 14%. The trainings aimed to upgrade the knowledge and expertise of the public school workforce and to embed modern educational methodologies in the public school system, hence provide higher quality educational services to public school students. The trainings covered ICT integration in the classroom; English language proficiency and teaching methodology; ECA integration at the school level; school leadership; and school improvement. The targeted training participants were diverse:

. MEHE staff who play a role in policy making and implementation;

. CERD teacher trainers and DOPS counselors and teacher mentors who have a vital role in knowledge transfer to teachers during and beyond the life of the project.

. School principals who manage the public schools and provide support to their teachers; and

. School teachers and staff who are in direct contact with students and who put in action the knowledge and skills gained through D-RASATI 2 training programs.

In 4,589 of the units of trainings for which successful training is measured, the training participants achieved the training goals based on the criteria of the respective trainings – 1,206 men and 3,383 women (641 officials and administrators and 3,948 teachers and educators). The 3,860 target was exceeded by 19%.

Information and Communications Technologies (ICT): Training on ICT aimed at simultaneously upgrading the ICT knowledge and skills of teachers, CERD trainers, and DOPS counselors and teacher mentors to integrate ICT into the classroom, and make use of the ICT Tier Model technology equipment procured by the project. ICT training also targeted the principals of the 176 schools which received the ICT equipment to prepare them to receive and maintain the equipment. The following sections provide more details on ICT trainings conducted under the project: ICT TOT for CERD trainers: The ICT teacher training modules were piloted in a 6-day ICT TOT in September 2014 which was attended by 54 CERD trainers, 5 DOPS teacher mentors, and 2 MEHE staff from the IT department. The TOT guided trainers in learning how to integrate cutting edge ICT tools and practices in training, teaching, and learning. It provided step-by-step instruction to participants on how to integrate the latest ICT tools in teaching while following the ISTE standards in lesson planning. It helped trainers perform the hands-on activities included in the training manuals and learn the contextual application of a wide range of technology tools; such as desktop applications, Web 2.0 tools, and the use of probes and sensors in science classes. At a follow-up workshop in December 2014, the participants prepared and presented lessons integrating ICT and received feedback from their trainers and peers. This allowed for an open exchange of experiences and perspectives on problems of using technology in a real classroom setting. From February to May 2015, 22 of the CERD trainers rolled out the ICT training to teachers in the CERD teacher training centers in different regions around the country. In these sessions the CERD trainers were able to apply their acquired knowledge and skills in authentic settings. The D-RASATI 2 ICT TOT trainers observed and supported the CERD trainers and provided them with feedback to improve their delivery of the training. Between September and December 2015, and during the last phase of the ICT TOT activity, the CERD and DOPS participants were introduced to the free open source Learning Management System (LMS), Moodle (Modular Object-Oriented Dynamic Learning Environment), with the objective of


19

creating an online learning community for the ICT teacher trainers. The participants attended a face-to-face workshop at the beginning of this phase, whereby they engaged with step-by-step hands-on activities on the online platform to familiarize themselves with the use of Moodle both on the computer and on a mobile device. The training continued online whereby the ICT TOT trainers and the participants shared e-resources and contributed to discussion forums based on the TPACK (Technology Pedagogical Content Knowledge) framework. The project ICT TOT trainers also developed a weighted assessment rubric which included e-content quality standards. The rubric graded participants on the frequency of their online interaction and participation in online fora, the quality of their contributions to thematic fora, and the quality of their final project. Regarding participation in online fora, quality parameters revolved around relevance of content to the topic discussed, use of technology and appropriate knowledge transfer methodologies, and showing constructive integration of technology into the topic under discussion. As to the final project, quality was measured according to whether effective and appropriate technology-rich outcomes were created, a balanced use of the needed resources is factored in, and an assessment rubric is included. The participants filled a self-assessment survey about the knowledge and skills gained from the training. The analysis of the responses showed that the participants agreed that the online training conducted under the ICT online coaching phase has impacted their training practices. In parallel, the project ICT TOT trainers reported that the participants became self-regulated learners who acquired teamwork and communication skills, and developed creative thinking and problem solving skills. ICT-in school coaching: The Digital Opportunities Trust’s (DOT) TeachUp! (TU!) model formed the foundation of the in-school teacher coaching on using ICT in their classrooms. The program aimed to rapidly and deeply build the capacity of school staff to immediately start using the ICT equipment procured under the project and to effectively integrate technology into teaching and learning. TU! is a comprehensive school-based program that operates within institutions to facilitate educational reform in accordance with the UNESCO ICT Competency Framework for Teachers. Under D-RASATI 2, the TeachUp! training material was translated into Arabic and adapted to the Lebanese context. It included modules such as 21st Century teaching methods; integrating ICT in Learning; digital citizenship; considering gender in TU!; developing new communications skills; modern pedagogical approaches to ICT learning; Integrating ICT in the classroom; and monitoring and evaluating TU! projects. From March 2014 to May 2015, D-RASATI 2 offered concentrated in-school ICT coaching to 1,257 teachers – 968 officially registered in the program and 289 receiving ad-hoc ICT support. These coaching activities took place in the 126 schools which received ICT equipment under the project. In total, 13,706 individual coaching sessions13, 2,717 group workshops, and 2,086 quick support sessions14 were offered. 427 ICT tools were introduced to teachers, the top 25 of which included the use of the interactive board software (ActiveInspire), digital presentation tools (Prezi), Collaboration and brainstorming tools (Edraw mind map), administration and management systems (Microsoft Office, Jing, Google Drive, and Gmail), Media Creation and Authoring (Movie Maker, Paint, Format Factory), Math & Science visualization tools (Scidot, Sine Qua Non), and video and podcasting resources (YouTube, Web Sites Resources). Having ICT coaches 3 days per week at the school, provided support for teachers to integrate ICT effectively in their classrooms. The project ICT coaches also instructed the teachers and the

13 An individual coaching session is 50 minutes and is tailored to the teacher’s professional development plan 14 Quick support sessions may take up to 15 minutes. Beneficiaries are teachers or administrative staff who drop by the ICT coach’s office to request assistance on any ICT issue


20

administrative staff at the schools on how to properly operate and maintain the ICT equipment procured by D-RASATI 2. As a result of the program, 723 teachers integrated technology in real time in 6,284 classroom sessions across all subject matters including French language, Arabic language, History, and Philosophy. Teachers were also introduced to the educational DOT platform for D-RASATI 2, an online community for the teachers registered in the ICT in-school coaching program. Teachers were encouraged to use the platform to create and share digital educational resources (REDs), and to access the digital library which includes media, documents and studies per subject matter. In total, 670 teachers registered and used the DOT platform for the D-RASATI2 program; 127 teachers (19%) contributed to the Digital Education Resources (RED) library content, uploading and sharing 201 REDs. Teachers also contributed to the platform’s social content, posting blogs and participating in its online fora. By the end of the program, 635 (82%) of the 771 teachers who took both entry and exit self-assessment tests about their ICT knowledge showed significant progress in learning. ICT Readiness training was delivered to 607 representatives from the 176 targeted ICT schools and CERD teacher training centers. These trainings were to prepare schools and CERD centers to receive the ICT equipment procured through D-RASATI 2. The training presented the ICT equipment procurement activity and the classification of public schools into tiers according to MEHE’s National Technology in Education strategy. It also explained the ICT equipment delivery process. In addition, information sessions were delivered, the ICT equipment vendor, CBM, on the use and operation of the ICT equipment received through D-RASATI 2. Extended ICT coaching on the use and operation of the Tier Model ICT equipment: High demand from teachers and school principals to continue with the ICT-in school coaching program15 prompted D-RASATI 2 to offer additional ICT trainings on the use and maintenance of ICT Tier Model equipment. From April to September 2016, the trainings targeted teachers from the schools which received ICT equipment through D-RASATI 2 under the 1st and 2nd rounds of the procurement process. To enhance the sustainability of the training benefits and utilization of the ICT equipment, the project also offered intensive TOT on the use and maintenance of ICT equipment to CERD teacher trainers who will in their turn rollout the trainings to other teachers in the future. DOPS counselors who will provide sustained support to teachers at the classroom level were likewise trained. This was based on DOPS request to provide the DOPS representatives, who demonstrated advanced digital and pedagogical skills, with intensive follow up sessions to strengthen their digital classroom skills and prepare them for the 2016/2017 year. The training material was developed by D-RASATI 2 and placed on Google Drive for all trainees to access and keep as reference. It covered Classroom ICT setup operation and maintenance (server, cabinet, router, tablets, laptops); tablet Troubleshooting (touch screens, keyboard, Wi-Fi disconnections, etc.); and software applications, such as the classroom management software LANschool. In total, 491 trainings units were delivered – 38 to CERD trainers, 141 to DOPS counselors, and 312 to teachers.

15 Most schools expressed a keen interest for additional ICT in-school coaching and support, including guidance on the use and troubleshooting of tablets and the LANschool software in the classroom. The major reason for the need for continued ICT support is that the schools received the ICT equipment late in the school year in 2015, and the teachers did not have sufficient time to practice working with the tablets and the LANschool software before the in-school coaching program ended


21

English language proficiency and teaching methodology training: D-RASATI’s Master Plan set the English language proficiency benchmarks for public school teachers at the CEFR16 C117 level for English language teachers and the B218 level for Science and Mathematics teachers. To meet MEHE’s standards, the D-RASATI program, in both its phases19, administered the BULATS test to 4,063 public school teachers to inform their placement into the different class levels of English language training offered under D-RASATI 2. 112 teachers were tested under D-RASATI 2 in 2013. From January 2014 to February 2016, D-RASATI 2 delivered 2,509 training units on English language proficiency at the elementary, primary, pre-intermediate, intermediate, pre-FCE, advanced, and pre-CAE levels. 975 of the training units were delivered to primary host school teachers. At the end of each class, the progress towards achieving a higher CEFR level was measured for each participant. By

February 2016, the results showed that 70 teachers – including 50 English language and 23 Math and Science teachers – had reached the C1 CEFR level; and 583 teachers – including 326 English language and 222 Math and Science teachers – had reached the B2 CEFR level. In parallel, D-RASATI 2 delivered the accredited and internationally recognized PCELT20 course to 33 CERD and DOPS representatives (24 CERD trainers and 9 DOPS counselors). All participants successfully completed the course and were awarded the internationally recognized PCELT certificate. The certificate qualified the participants to deliver highly interactive, integrated skills English courses focused on proficiency development. In

16 CEFR refers to the Common European Framework Reference for Languages which consists of English language proficiency benchmark levels. It was put together by the Council of Europe as a way of standardizing the levels of language exams in different regions. It is very widely used internationally and all important exams are mapped to the CEFR 17 The C1 level means that the incumbent can: Understand a wide range of demanding, longer texts, and recognize implicit meaning; express ideas fluently and spontaneously without much obvious searching for expressions; use language flexibly and effectively for social, academic and professional purposes; and produce clear, well-structured, detailed text on complex subjects, showing controlled use of organizational patterns, connectors and cohesive devices 18 The B2 level means that the incumbent can: Understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialization; interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party; and produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options 19 D-RASATI 1 administered the BULATS test to 4,063 teachers in 2012, and D-RASATI 2 to 112 teachers in 2013 20 The Professional Certificate in English Language Teaching (PCELT) is an accredited 120-hour teacher training course designed to meet the growing demand for quality training in English language teaching in the Arab world

English language training – December 2014

English language training – March 2014


22

addition, the project facilitated the certification of 220 teachers at the FCE21 level, and 22 teachers at the CAE22 level. This adds to CERD’s capital in terms of a pool of trainers who have the capacity to roll out English language trainings to teachers and to help them achieve MEHE’s English language proficiency benchmarks.

To complement training on language proficiency, the project delivered English Language Teaching Methodology (ELTM) TOT to 33 PCELT-certified CERD and DOPS representatives – 23 from CERD

and 10 from DOPS. The CERD and DOPS staff rolled out the ELTM to 183 primary school English

Language coordinators and teachers with advanced language skills. Thus, D-RASATI 2 upgraded the teaching methodology skills of the English language coordinators and empowered teachers to become English language coordinators and mentors for other teachers in their schools.

Extra-Curricular Activities (ECA): D-RASATI 2 built the capacity of MEHE, CERD, and public schools to institutionalize ECAs in the public school system. It conducted ECA workshops for 21 representatives from CERD and MEHE to build their capacity to provide sustained support, mentoring and coaching to schools as they implement ECAs. The training included discussions on the definition of ECA and its role in the educational process; and the roles of MEHE representatives in supporting schools in providing ECA for students. By the end of the workshop, the participants gained a deeper understanding of the challenges and opportunities for schools to integrate ECA, and acquire the necessary skills needed to facilitate the implementation of ECA in schools.

21 This international certificate proves that the incumbent has the language skills to live and work independently in an English-speaking country or study on courses taught in English 22 This international certificate proves that the incumbent has achieved a high level of English ability which can be used to teach in academic and professional contexts. The CAE certificate is accepted globally by over 4,000 institutions

ECA workshops for MEHE – January 2015

ELTM TOT – July 2015

PCELT – August 2014


23

The project also empowered representatives from 125 intermediate and secondary public schools to plan and organize ECAs in their schools. The ECA training introduced them to the ECA strategy adopted by MEHE and taught them the skills necessary to analyze their school’s environment, develop a plan for creating an enabling environment for ECA, and design ECA activities, such as recreational activities, school clubs and project-based activities. By the end of the training, the school representatives (ECA facilitators) had gained the skills to play the role of ECA facilitators and to develop, implement and evaluate a yearly school plan for ECA based on student needs and interests. Pre and post training ECA tests were administered to the training participants to measure the progress in ECA proficiency due to the training. Analysis of the tests showed a significant increase in ECA proficiency in different areas, with the highest increase corresponding to “designing and facilitating extracurricular activities”. To spread the benefits of the project’s ECA activities beyond the 125 schools and to facilitate the implementation of ECAs, the ECA modules booklets which describe the step-by-step implementation of ECA activities in 7 thematic areas were distributed to all intermediate and secondary public schools along with ECA stationary kits. In parallel, D-RASATI 2 provided TOT to 33 CERD trainers on the use of the ECA modules which enabled them to rollout the training to all public schools beyond the project life.

Psychosocial Support (PSS): The PSS learning program delivered by D-RASATI 2 aimed to equip teachers with the knowledge, skills, and tools to provide psychosocial support that all children need, in particular vulnerable children. It transferred know-how to teachers on developing positive classroom relationships and environment, and fostering socio-emotional competence in children. Through its PSS-related activities, D-RASATI 2 provided MEHE the foundation upon which it could

build its PSS strategy, and consolidate and integrate a coherent PSS program into all public schools. The project developed the 1st PSS training material for the formal public school system in Arabic and adapted it to the Lebanese context. In addition, the project developed material for coaching teachers on PSS on which DOPS counselors received an additional training day. The material provided a classroom observation protocol to DOPS counselors to guide them in supporting and coaching teachers on PSS.

ECA for school representatives – March 2015

PSS TOT – November 2015


24

D-RASATI 2 delivered PSS TOT to 132 representatives from MEHE and CERD in 2 cohorts, of whom 57 CERD teacher trainers and 48 DOPS counselors are considered successfully trained. The training built the teacher trainers’ capacity to rollout the training to teachers during and beyond the project life, and the capacity of DOPS counselors to provide sustained support to teachers through mentoring and coaching. Under D-RASATI 2, 22 CERD trainers rolled out the training to 523 teachers from 191 primary public host schools, and 508 teachers successfully completed the training and started to implement PSS in their classrooms to reduce tensions among students victim of trauma. The DOPS counsellors who received the training visited the teachers in their classrooms and provided them with support as they implemented PSS activities. A knowledge questionnaire administered to the teachers before and after the training revealed that the collective progress in learning increased by 46% at the end of the training. In March 2016, the project organized workshops for principals of primary public schools most affected by the Syrian crisis to sensitize them on the concepts and benefits of PSS in emergency. This was necessary to elicit their support for the teachers who are implementing PSS in their schools. 59 school principals attended these workshops. At the end of the PSS trainings, lessons learned workshops were organized to collect feedback from the CERD trainers who rolled out the PSS training and the DOPS school counsellors and teacher mentors who conducted coaching visits in schools. A study to measure the impact of PSS training on the level of tensions inside the classroom in 50 schools was completed in August 2016. The study concluded that the PSS training increased teachers’ implementation of practices that support students’ psychosocial wellbeing. Empirical evidence suggested that classroom environments improved, especially with regard to levels of trust among students, their sense of safety and their acceptance of diversity. The report also concluded that the general psychosocial wellbeing of children improved after the training.

Leadership Development Program (LDP): Under this activity, 13 Master trainers who are part of the Faculty of Education at the Lebanese University (LU-FED) faculty and 53 trainers recruited by an LU-FED committee received LDP TOT over 2 days to prepare them to roll out the training to school principals based on the training material developed by LU-FED. The LDP training enhanced the leadership capacity and skills of 453 school principals for effective school management, school improvement, and community involvement. The successful completion of the training built the leadership qualifications of the school principals and enabled them to secure official tenure and receive the 15% salary raise indemnity mandated by Lebanese Law #73. The training was delivered in 3 phases:

PSS rollout to teachers – March 2016


25

1. Phase 1 was the 12-week, 144-hour classroom-based interactive training phase, delivered over two 6-hour training days per week. This part of the training consisted of 6 modules and equipped school principals with modern knowledge and skills to improve their practices for educational leadership and supervision, and school management and educational planning. It introduced them to the concepts and benefits of active learning methodologies and the use of technology in management, and presented and explained the public employee bylaws.

2. Phase 2 was the self-study phase, complemented by online meetings with LDP trainers via an LDP e-portal, whereby school principals worked independently on 3 mandatory and 1 elective modules. The mandatory modules included advanced practices for the use of information for decision making and for management and educational planning. During this phase the principals also selected and completed one of the 2 elective modules – enabling school clubs and implementation of ECAs, or developing school services to meet the needs of special needs students.

3. Phase 3 was the practical project-based phase (Practicum Phase). Here the principals were to put learning into action and to develop and implement the educational and/or administrative projects in their schools. Coaches visited schools twice during this phase to provide support to the school principals – the first visit to support the principal in launching their LDP project, and the second towards the end of the phase to evaluate the project performance and achievements. This is described in detail under the section of this report “Implementing MEHE’s Effective School Standards in relation to ECA, LDP and SIP”.

Pre and post training tests were administered for each of the 6 modules in phase 1 to measure the progress in the learning of the trainees. The analysis of the tests revealed that 97% of the principals had gained extra knowledge and proficiency in school leadership due to the training.

School Improvement Program (SIP): This training aimed to upgrade the knowledge and skills of school principals and their teams to identify the improvement priorities of their schools, develop action plans to address the underlying challenges, and implement and evaluate the improvement plans. The program employed a dynamic and participatory approach to decision-making by school principals while engaging key stakeholders – teachers, students, and the local community – in the process. Specifically, the training aimed to embed soft skills in the school teams such as reflection and collaboration, and to enhance the team’s data collection, documentation, planning and organizing skills. Six senior coaches and 26 coaches were recruited to support the SIP school teams in their learning and implementation of SIPs. Before launching the SIP program, orientation meetings were held for the coaches to brief them on the D-RASATI 2 project and the SIP program. They were introduced to their planned roles and responsibilities throughout the duration of the program. A Coaching Reference document, along with a glossary of SIP terms, were developed and given to the coaches

LDP Cohort 1 Phase 1 – March 2014


26

to guide their interactions with the school teams. Thereafter, regular monthly meetings were organized for the coaches for different purposes including:

. Training on the ongoing SIP processes so that the coaches could provide focused support to the school teams;

. Receiving feedback on the progress of the school teams in their SIP learning and implementation process, and on the challenges faced;

. Disseminating information to the school teams, e.g., the grant disbursement process and timeframe; and

. Discussing lessons learned and evaluating the SIP experience and the learning that took place both at the coach’s level and the schools’ level.

The school teams, composed of the school principal and 2 school representatives, received face-to-face training on the skills needed to assess the improvement needs of their schools, and on the design and implement a school improvement plan (SIP). School teams also benefited from regular coaching sessions on a bi-weekly basis in-between training sessions. These were dedicated to support schools in applying the SIP concepts taught and in implementing their school improvement plans and projects. The SIP training and coaching was delivered over 3 phases:

a. Phase 1 - School Review and needs assessment:

This is the basic stage in any school improvement process that guides the planning committees to accurately assess their school’s situation based on evidence and precise data collected from different sources. Teams from 205 schools participated in this phase and acquired skills on evidence-based inquiry, reflective practices, teamwork skills, data collection, data analysis and reporting skills.

b. Phase 2 - SIP Design: Upon completion of the school review, and based on prioritized needs that

are diagnosed during the first phase, school teams proceeded with developing their School Improvement Plans (SIPs). This phase embeds skills for a systematic methodology to define goals and objectives, design an action plan and allocate the budget needed.

c. Phase 3 - SIP Implementation: In this final stage in the school improvement process, the school

teams were prepared to implement their SIPs, factor in implementation challenges and mitigation measures, monitor progress, and evaluate the impact of their school plans.

191 school teams succeeded in completing the program which culminated in receiving grants, valued at $4,200 per school, and which went towards implementing their school improvement plans.

Orientation of SIP coaches – February 2014

School review workshop – March 2014


27

Acquiring the SIP soft skills ensured the continuation of benefits accrued from this program as the skills can be applied in any other school improvement project in the future. In the purpose of evaluating the school teams’ acquisition of the soft skills and overall performance, an evaluation rubric, with its corresponding key and scoring sheet, was developed. Based on their scores, schools were categorized into 1 of 4 SIP proficiency categories:

Stellar (star) if they score between 23-28 points

Achieving if they score between 16-22 points

Challenging if they score between 11-15 points

Lagging if they score below 10 points

Coaches scored the performance of each school team they supported twice, 6 months into the SIP training, and 6 months later. The analysis of the scores showed that the majority of the schools – 153 schools– exhibited a satisfactory level of SIP skill acquisition, of which 44% achieved the stellar status. The performance of the school teams also increased with time and support from coaches, whereby, 10 of the 39 schools which scored unsatisfactory at the midterm evaluation improved their performance and joined the “satisfactory performance” group (Table 1):

Table 1 – Scoring the performance of the SIP schools’ teams

Category # of schools in

midterm evaluation

# of schools in

final evaluation

Average score at

midterm evaluation

Average score at

final evaluation

Stellar 67 (34.9%) 71 (37.1%)

19.7

20.02 Achieving 86 (44.79%) 91 (47.6%)

Challenging 32 (16.67%) 23 (12%)

Lagging 7 (3.65%) 6 (3%)

In May 2015, towards the end of the program, the performance of the program was evaluated by the team of SIP experts and coaches who worked closely with the SIP school teams for around 18 months. It was concluded that as a result of the training and coaching, the school teams gained sufficient awareness on how to plan and implement a development project, and they acquired modern management skills that can be used in other projects in the future. These skills included:

Increased leadership capacities in committee members and perception of themselves as partners in the school improvement process

Linking school needs to student learning goals, utilizing reflection and brainstorming for diagnosis and planning purposes, utilizing the ESP as a main starting point to any school improvement initiative, as opposed to randomly choosing school improvement projects

planning and decision-making based on evidence derived from collecting and analyzing data

Documenting and report writing

Implementing, monitoring, and evaluating projects

Project Monitoring and Evaluation

Project Monitoring

D-RASATI 2’s M&E Plan was developed in conformity with ADS 203.3.5. It identified 13 performance indicators to track and monitor D-RASATI 2’s progress towards achieving its targets. 5 of the indicators are “Standard Foreign Assistance” (F) which USAID uses to consolidate key results across the globe and report to Congress; 1 is gender-sensitive in compliance with USAID’s new requirements; and the remaining 7 are custom indicators specific to the activities and purpose of D-RASATI 2. The indicators are a mix of output and outcome indicators, to track both the


28

implementation of project activities against the work plan, and gage the difference these activities made on the stakeholders and intended beneficiaries. The M&E plan specified the data quality assessment (DQA) procedures that the M&E staff used to ensure the data collected and reported conforms with USAID’s data quality standards (Validity, Reliability, Precision, Integrity, Timeliness). Accordingly, an M&E system was established to which all implementing partners subscribed through signing an M&E protocol. The M&E protocol was used as a management tool to ensure the timely and reliable collection and reporting of quality performance data.


29

Accomplishments Relative to Targets

The baseline value for all D-RASATI 2 indicators are set at 0 because all the activities that report to the D-RASATI 2 indicators were implemented for the first time under the D-RASATI program. Table 2 – D-RASATI 2’s LOP indicators, targets and achievements

Indicator LOP Target Project Achievement Performance Explanation for under or over achievement above 10%

# Type Level Name

DO 1 Improved capacity of the public sector in providing transparent and quality services across Lebanon

1 F Output

Number of laws, policies, regulations, or guidelines developed or modified to improve primary grade reading programs or increase equitable access (D-RASATI) (3.2.1-38)

26 27

103%

Rapid Needs Assessment report 1 1

15 ICT TOT manuals 15 15

ECA strategy 1 1

ICT Action Plan 1 1

Rapid Teacher Workforce Study (RTWS) report 1 1

ECA modules 7 7

PSS Policy Paper - 1

IR1.1 Improved public basic education available to all school-aged citizens

2 C Outcome Number of schools demonstrating improvement relative to the ESP (with reference to ECA, LDP, and SIP) Up to 583

732 without duplication, or 804 with duplication

126% The cumulative target is exceeded by 26%. Both the LDP and SIP programs involved the design and implementation of projects to improve the learning environments at the schools, in line with MEHE’s Effective School Standards (ESS). In both the LDP and the SIP

ECA schools 104 125 120%

LDP schools 315 488 155%


30


# Type Level Name

SIP schools 164 191 116%

programs a larger number of quality school projects streamlined within the ESS were designed and implemented by the school principals and their teams than originally anticipated. The targets were exceeded by 55% and 16%, respectively. On the other hand, the ECA program did not involve the implementation of projects, it rather equipped and enabled the school representatives who met the successful training criteria to design and implement ECA programs in their schools. The training on ECA was streamlined with the national ECA strategy that was being concurrently developed. The ECA strategy is in full alignment with MEHE’s ESS. Thus, it can be deduced that those who were successfully trained on ECA have been enabled to implement ECAs in line with MEHE’s ESS. The number of 125 ECA schools whose representatives were successfully trained exceeded the 104 target.

Sub-IR 1.1.1 School learning environments improved

3 F Output Number of schools using Information and Communication Technology due to USG support (3.2.1-36)

Up to 113 126 112%

The target is exceeded by 12%. By May 2015, 126 schools had received ICT equipment and their teachers received ICT in-school coaching. By June 2015, 91 (72%) of these schools had operated the equipment, mainly because it was the end of the school year. At the beginning of the 2015-2016 school year, the D-RASATI 2 team contacted all 126 schools and polled them about the use of the ICT equipment procured under D-RASATI 2. The phone survey revealed that all 126 schools were using the ICT equipment, mainly the laptops and LCD projectors.

4 C Outcome Number of LDP school improvement projects completed Up to 315 488 155%

The target is exceeded by 55%. This is mainly the result of: . A larger number of school principals participated in the LDP

training program than originally planned (by 21%); and . A larger number of school principals than anticipated completed

the last of the 3 phases of the LDP training. The last phase consists of designing and implementing an LDP project. This is mainly due to the fact that they could receive the salary increase that goes with their new duties as school principals only if they successfully complete all the 3 phases of the LDP training. Another important factor was the persistent support they received from the LDP trainers and coaches.


31


# Type Level Name

5 C Outcome Number of schools implementing school improvement plans with grants

Up to 164 191 116%

The target is exceeded by 16%, despite the fact that only 205 (84%) of 244 schools anticipated to participate in the SIP program actually participated. Of the 205 schools, only 192 (94%) submitted SIP proposal projects to receive a SIP grant to implement their project. At the proposal evaluation, 191 proposals won grants, which still exceeded the 164 target by 16%. This is mainly due to the continuous support and coaching provided by the SIP coaches to help the school teams design projects that met the SIP grants criteria.

6 C Output Number of national awareness campaigns developed 1 1 100%

7 F Output Number of textbooks and other teaching and learning materials (TLM) provided with USG assistance (3.2.1-33)

16,176 16,552 123%

The target is exceeded by 23% because TLM were distributed to the additional participants who joined trainings above the expected number. This is the case for ICT TOT, Extended ICT coaching, PSS TOT, and LDP

Sub-IR 1.1.2 School learning opportunities increased

8 C Output Number of administrators and officials trained with USG support (i.e. number of training units delivered to administrators and officials with USG support)

Up to 944 Total: 917 unique participants (529 men and 388 women) through the delivery of 1,026 units of training to 589 men and 437 women

109% The target was exceeded by 9%, due to: . Delivering ICT readiness training to the principals of the 50

additional schools which received ICT equipment under D-RASATI 2; and

. A larger number of school principals participated in the LDP training program than originally planned (by 21%)

ICT TOT - 2 men

ICT Readiness for school principals 126

Total: 163 (93 men and 70 women) - ICT Readiness 1 for 126 schools: 128 (77 men and 51 women) - ICT Readiness 2 for 24 new schools: 15 (4 men and 11 women) - ICT Readiness 3 for 26 new schools: 20 (12 men and 8 women)

ECA workshops for MEHE 24 16 (6 men and 10 women)

PSS TOT - 1 woman

Sensitization on PSS in emergency for school principals 65 59 (25 men and 34 women)

LDP 485 587 (318 men and 269 women)

SIP 244 198 (145 men and 53 women)

9 F Output Number of administrators and officials successfully trained

with USG support (3.2.1-3)

Up to 495 Total: 640 unique participants (368 men and 272 women) through the delivery of 641 training units to 368 men and 273 women

129%

The target is exceeded by 29%. This is mainly the result of: . A larger number of school principals participated in the LDP

training program than originally planned (by 21%); and . A higher success rate than the foreseen 65% of participants (77%

for SIP and 86% for SIP). ICT TOT - 2 men

ECA workshops for MEHE 16 15 (6 men and 9 women)


32


# Type Level Name

LDP 315 453 (233 men, 220 women)

SIP 164 171 (127 men and 44 women)

10 C Output Number of teachers/educators/ teaching assistants

trained/coached with USG support (i.e. number of training

units delivered to teachers/educators/ teaching assistants with

USG support)

Up to 5,336 Total: 4,319 unique participants (1,040 men and 3,279 women) through the delivery of 6,149 units of training (1,537 men and 4,612 women)

115% The target is exceeded by 15% mainly due to high demand on ICT and PSS-related trainings. ICT-related trainings: At the request of MEHE, the project procured ICT equipment for 176 schools, 6 CERD centers and the DOPS central office (when it was originally planned to procure equipment to only 126 schools). Accordingly, the ICT trainings were offered to additional teachers, school administrators, and CERD and DOPS representatives. PSS TOT: PSS is a novel intervention that is in the process of being adopted at schools in Lebanon. The only PSS training material suitable for the formal education context was developed, translated, and adapted to the Lebanese context under D-RASATI 2 for the benefit of MEHE and CERD. Both CERD and DOPS showed high interest in acquiring PSS knowledge and skills to equip their trainers and counselors to provide quality PSS training and in-school support to teachers. Accordingly, 98 CERD and DOPS representatives registered for the PSS TOT when 70 were invited to attend. Additionally, CERD requested a repeat of the training to be ready to offer the training to all public school teachers starting the 2016-2017 school year.

ICT TOT 50 59 (25 men and 34 women) 18%

ICT TOT online coaching 50 39 (15 men and 24 women)

PCELT 36 33 (2 men and 31 women)

ELTM TOT 36 33 (3 men and 30 women)

ELTM rollout 300 224 (11 men and 213 women)

English language training 2,500 2,509 (669 men and 1,840 women)

ICT Readiness 126

Total: 444 (140 men and 304 women) - For 126 schools: 430 (132 men and 298 women) - For 50 additional schools: 14 (8 men and 6 women)

52%

ICT in-school coaching 630

Total: 968 (265 men and 703 women) - Y1: 123 (35 men and 88 women) - Y2: 845 (230 men and 615 women)

54%

Extended ICT coaching

Total: 357

(36 CERD, 119 DOPS,

202 teachers from 126 schools)

Total: 491 (166 men and 325 women) - CERD: 38 (19 men and 19 women) - DOPS 1: 119 (48 men and 71 women) - DOPS 2: 22 (12 men and 10 women) - Teachers from 126 ICT schools: 222 (58 men and 164 women) - Teachers from 24 additional ICT schools: 90 (29 men and 61 women)

38%

ECA workshops for MEHE - 5 (1 man and 4 women)

ECA for school representatives 160 142 (24 men and 118 women) 89%

ECA TOT for CERD trainers 33 33 (11 men and 22 women) 100%

PSS TOT 70

Total: 133 (19 men and 114 women) - PSS TOT: 98 (14 men and 84 women) - PSS TOT 2 for CERD: 35 (5 men and 30 women)

90%


33


# Type Level Name

PSS rollout 500 523 (28 men and 495 women) 105%

SIP 488 513 (158 men and 355 women) 105%

11 F Output Number of teachers/educators/ teaching assistants who

successfully completed in-service training or received intensive

coaching or mentoring with USG support (3.2.1-31)

Up to 3,365 Total: 3,141 unique participants (664 men and 2,477 women) through 3,948 training units for 838 men and 3,110 women

93% in terms of unique

individuals or 117% in terms

of training units

In terms of training units, the target has been exceeded by 17%. The target of 3,365 was set in terms of training units. It has been computed in function of the number of training units under indicator 10 that are at least 2 days in duration (as stipulated by USAID regulations). The coefficient was set at 65% at the beginning of the project because of time constraints that participants may have, and which could prevent them from attending the full training. However, many teachers attended more than one training under D-RASATI 2 which makes the number of unique individuals less than the number of training units.

ICT TOT 33 59 (25 men and 34 women) 179%

ICT TOT online coaching 33 25 (9 men and 16 women) 76%

PCELT 24 33 (2 men and 31 women) 138%

ELTM TOT 23 33 (3 men and 30 women) 143%

ELTM rollout 195 183 (8 men and 175 women) 94%

English Language training 1,625 1,936 (465 men and 1,471 women) 119%

ICT in-school coaching 630 614 (151 men and 463 women) 97%

ECA workshops for MEHE - 3 women

ECA for school representatives 104 123 (23 men and 100 women) 118%

PSS TOT 46

Total: 105 (17 men and 89 women) PSS TOT: 74 (13 men and 61 women) PSS TOT 2 for CERD: 31 (4 men and 27 women)

228%

PSS Rollout 325 479 (26 men and 453 women) 147%

SIP 327 355 (109 men and 246 women) 109%

12

GN

DR

-3 Outcome Proportion of females who report increased self-efficacy at the

conclusion of USG supported training/programming

25%

Overall: 54% More than 2-

fold

The target was consistently exceeded. The project team does not

have hard evidence to explain this result. However, one possible

explanation could be that the trainings responded directly to the

professional development needs of the trainees. In addition, the

instruction methodology adopted by all project trainings – the

experiential learning approach – resulted in that the trainees felt

comfortable and engaged in the trainings. This not only increased

their proficiency in the subject of the training, but also their feeling of

self-efficacy in their ability to apply what they learned at work.

ICT TOT 38%

PCELT cohort 1 83%

PCELT cohorts 2 & 3 47%

ELTM TOT 40%

ELTM rollout 56%

English language training 52%

ICT in-school coaching 60%

Extended ICT coaching for CERD and DOPS CERD 44%; DOPS 39%


34


# Type Level Name

Extended ICT coaching for teachers Tier 2&3 54%

Extend ICT coaching teachers Tier 4 43%

Extend ICT coaching teachers from 24 new ICT schools 40%

Extend ICT coaching DOPS 2 30%

ECA for school representatives 84%

PSS TOT 41%

PSS TOT 2 for CERD 62%

PSS Rollout 52%

LDP 78%

SIP 79%

13 C Outcome Number of schools implementing ECA activities under new

system

104 125 120%

Exceeded target by 20%. As mentioned under Indicator 2, the ECA program did not involve the implementation of projects, it rather equipped and enabled the school representatives who met the successful training criteria to design and implement ECA programs in their schools. The training on ECA was streamlined with the national ECA strategy that was being concurrently developed. The ECA strategy is in full alignment with MEHE’s ESS. Thus, it can be deduced that those who were successfully trained on ECA will be implementing ECA projects in line with MEHE’s ESS. The number of 125 ECA schools whose representatives were successfully trained exceeded the 104 target.


35

Impact and Sustainability

Impact

According to the Kirkpatrick Model23 (box 3), the worldwide standard for evaluating the effectiveness of training, the value of any type of training, formal or informal, can be assessed across four levels. Level 1 – Reaction, evaluates how participants respond to the training; Level 2 – Learning, measures if they actually learned the material; Level 3 – Behavior, considers if they are using what they learned on the job; and Level 4 – Results, evaluates if the training had a positive impact on the targeted beneficiaries. In the context of D-RASATI 2, the impact of the training programs could only be measured at the first 2 levels of the Kirkpatrick’s model, i.e. at the level of the trainees – CERD and DOPS representatives, and the school staff. The short implementation period of the trainings did not allow the measurement of impact on the quality of the public school services, nor on the performance of the students. D-RASATI 2’s M&E system factored in the collection and analysis of data to enable gaging the reaction and learning of trainees. The analysis of the data collected showed that the impact of the trainings on the trainees was significantly positive. It can be safely inferred that this positive impact will be also reflected on the quality of the public school education services, and ultimately, on the performance of students. The premise is that if the training is relevant to the trainees, they are satisfied with what they learned, and their proficiency has increased, the training is most likely to result in lasting changes that boost performance and impact on the ultimate beneficiaries – the students –, given an enabling environment. The section below describes the impact of trainings on the reaction and learning of trainees:

1. Trainee satisfaction with the training (Reaction24): A satisfaction/feedback survey was administered at the end of each D-RASATI 2 training in which trainees were asked whether the trainings were relevant to their professional development needs and whether they were satisfied with the training. Overall, 96% of the 4,795 trainees who filled the survey responded in the affirmative to both questions – 61% highly satisfied, and 35% satisfied.

23 The evaluation model was developed by Donald Kirkpatrick in the 1950s. He is the past president of the American Society for Training and Development (ASTD). Source: http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel 24 According to the Kirkpatrick evaluation model

Box 3 - The Kirkpatrick Model

http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel


36

2. Increased proficiency of trainees (Learning25): D-RASATI 2 employed 2 surveys to assess learning and its impact on the trainee’s sense of self-efficacy in performing at work:

a. Pre and post training tests or self-assessments were administered to measure the

increased proficiency in the subject which is directly attributable to the training. 87% of the 3,326 trainees who filled both entry and exit surveys showed increased proficiency after receiving the trainings. On average, proficiency increased by 40%. In the particular case of SIP, the SIP coaches scored the performance of each school team they supported twice, 6 months into the SIP training, and 6 months later. The analysis of the scores showed that the majority of the schools – 153 (80%) of the 192 schools which submitted SIP proposals – exhibited a satisfactory level of skill acquisition, of which 44% achieved the stellar status. With time and support from coaches, 10 (26%) of the 39 schools which scored unsatisfactory at the midterm evaluation improved their performance and joined the “satisfactory performance” group;

b. The Generalized Self-Efficacy survey (GSE) was administered at the beginning and

end of each of D-RASATI 2 trainings that were at least 1 day in duration. The end score was compared to the starting score to measure the increase in self-reported sense of self-efficacy (or capacity to perform at work) that was gained as a result of receiving the training. On average, 54% of the trainees reported an increased sense of self-efficacy – 54% for women and 56% for men.

In addition, 3 evaluation studies were conducted within the life of the project: the impact study of the PSS trainings conducted by D-RASATI 2 as part of its contractual deliverables; and 2 independent evaluations of the English language trainings and of the overall project performance that were conducted by the Cambridge English Language Assessment Center and the PMPSL project (on behalf of USAID), respectively. The project used the following as proxies to measure impact of teacher training:

Assessment of the English language trainings: The 2015 qualitative impact study evaluated the English language trainings delivered by D-RASATI 2 from January 2014 to June 2015. The study found that the courses had a positive impact on the teachers’ language development, the highest impact being on the vocabulary and grammar skills as self-reported by teachers, and least impact being on listening skills. The end-of-training exam data suggests that listening is the area in need of further attention as the majority of teachers who took the FCE test did not achieve B2 in listening. Teachers indicated that they felt more confident teaching in English as a result of taking the course and a majority of teachers expressed their desire to continue studying English. One of the most significant positive impacts of the course was teachers reporting the adoption of the pedagogical practices of their English language trainers, which were based on a communicative methodology, and were both motivated and inspired to try new methods of teaching as a result.

Impact of the PSS trainings on reduced tensions inside the classroom: The 2016 PSS impact study was conducted by D-RASATI 2 to measure the impact of the PSS trainings at the classroom level, in terms of reduced tension and aggression among students, and improved classroom environments. The report concluded that the PSS training increased teachers’ implementation of practices that support students’ psychosocial wellbeing. Empirical evidence suggested that classroom environments improved, especially with regard to levels of trust among students, their sense of safety and their acceptance of diversity; and that the general psychosocial wellbeing of children improved after the training. However, given the short timeframe between the baseline and end line

25 According to the Kirkpatrick evaluation model


37

assessment - about 2 months- the levels of reported friction between students did not change following the training. It is possible that the improvements in classroom environment and the psychosocial wellbeing of the children had yet to affect levels of friction between students, which were low to begin with. Overall performance evaluation of D-RASATI 2: The independent performance evaluation of the project was conducted by the USAID-funded Performance Management and Support Program for Lebanon (PMSPL) on behalf of USAID over a three-week period that extended between August and September 2016. The evaluation assessed the effectiveness of the project in achieving its planned results, and the sustainability of the benefits accrued from its activities. The evaluation concluded that “Overall, D-RASATI 2 was able to achieve—and [in] some cases overachieved—its intended results during a short project timeframe to meet its objectives. The project’s strong leadership and management, adherence to the Master Plan, and approach to working with MEHE and CERD resulted in successfully implemented activities. [However,] Given delays inherent [to a] in close collaboration with the MEHE, the short project life—initially two years—was insufficient for a project of this scope and scale.” Based on the above, and in line with the Kirkpatrick’s Model, it can be deduced that D-RASATI 2 had positive impact on the quality of the public school education services.

Sustainability

Elements of sustainability were built into the design of D-RASATI 2 from the start. D-RASATI 2 is essentially a capacity building project whose direct beneficiaries are mainly the public schools’ administrative and teaching bodies. D-RASATI 2 directly imparted knowledge and skills to school principals and teachers to raise proficiency in effective education practices. Embedding knowledge and expertise into the public sector system: To maximize the sustainability of the benefits accrued from the trainings, D-RASATI 2 trainings worked towards embedding knowledge and expertise in the local CERD and DOPS institutions26 not only to disseminate know-how but also to sustain the benefits of knowledge transfer after project closure. Accordingly, 161 CERD trainers and 176 DOPS counselors received TOT and coaching on ICT integration and the use and maintenance of the ICT equipment procured by D-RASATI 2, English language proficiency and teaching methodology, and ECA. Although CERD and DOPS were not involved in the LDP and SIP programs – LDP being the mandate of LU-FED, and SIP being a new program in Lebanon – D-RASATI 2 built the capacity of master trainers, trainers, and coaches specifically hired to conduct the related trainings. These trainers and coaches form now a pool of experts on which CERD and MEHE can call to implement similar programs in the future. As an added measure to increase the sustainability of the ICT integration and LDP training programs, D-RASATI 2 created 2 online Moodle platforms to facilitate networking among trainers and trainees, and trained CERD and LU-FED’s IT staff on managing the ICT online platform handed over to CERD and the LDP online platform handed over to LU-FED. Not only did D-RASATI 2 train CERD trainers and DOPS counselors, but it also developed training materials on new subjects, in line with its mission, adapted to the Lebanese curriculum and context that CERD can use to train school principals and teachers in the future. These cover the integration

26 CERD’s mandate is to train the school staff under its in-service professional development curriculum, and DOPS’ mandate is to provide support and guidance to teachers at the classroom level as they implement the Lebanese curriculum


38

of ICT in teaching and learning; a student-centered teaching methodology for English language; the institutionalization of ECA at the school level; and the implementation of the PSS approach to improve the classroom environment and learning capacity of vulnerable children. In particular, in the case of PSS, training material was also developed targeting DOPS counselors delineating the approach to provide coaching and support to teachers as they implement PSS activities in their classrooms. Securing buy-in and ownership from CERD and DOPS: Finally, D-RASATI 2 was able to secure buy-in and ownership by MEHE, CERD, and LU – through close consultations and technical advice – of several of its programs even before it closed, which is likely again to increase their sustainability: The project ensured close collaboration and coordination with MEHE to ensure that it is responding to the needs of the participants and the beneficiaries. To start with DOPS counselors and teacher mentors were included in most of the trainings delivered by D-RASATI 2; this resulted in building the capacity of DOPS counselors to better coach and support school staff in the future. As for the PSS activity, although it took MEHE more than a year to fully engage in the program, the ministry came fully on board when the project team coordinated the plan of the PSS activities with the MEHE officials and designed the training material based on the priorities of the DOPS counselors. Moreover, the ICT procurement activity was coordinated with the MEHE ICT committee which included representatives from all of MEHE’s units and directorates. MEHE took ownership of the ICT equipment procured under D-RASATI 2 and of the related 3-year warranty contract with the CBM vendor. Even before the equipment was distributed to schools in May 2015, MEHE dispatched its IT staff to the schools and CERD centers to test the equipment and resolve issues with access to internet and connection to the MEHE server. Further, CERD’s in-service professional curricula for the past 2 years included courses for teachers on the integration of ICT in teaching and learning and on English language proficiency, based on the TOTs delivered by D-RASATI 2. In 2014-2015, CERD started to roll out these courses, at CERD’s expense, whereby the CERD trainers trained under D-RASATI 2 delivered the trainings. As to PSS, in a meeting with project management in June 2016, CERD informally communicated its plan to include the PSS training in its regular curriculum starting the 2016-2017 training year and to offer it to all public schools teachers. CERD requested D-RASATI 2 to deliver an extra PSS TOT in July 2016 to a 2nd cohort of CERD trainers so that at least 70 CERD trainers are equipped to roll out the training. The extra 8-day PSS TOT was delivered on July 11-14 and 18-21, 2016. The independent project performance evaluation confirms the above as it states “MEHE and CERD had buy-in for D-RASATI 2’s plan and generally found it relevant to their priorities. This was especially true for the ICT Action Plan, English, LDP, and SIP. They also accepted the relevance of ECA and PSS, but were less fully invested at the start”. The report also states “The evidence collected shows that CERD plans and has the capacity, built through TOTs, to take D-RASATI 2 capacity development material forward as part of its official teacher trainings and likewise for MEHE. Key informant interviews also indicated that DOPS and CERD operate as separate bodies through different channels of reporting, and that D-RASATI 2 was exceptional at targeting both bodies”. The evaluation concludes that “Broadly, most D-RASATI 2 activities that involved TOT are likely to be sustained through institutionalization and adoption by MEHE and CERD. Engaging with CERD was essential to this sustainability”. Developing sound selection criteria for participants: Another element of sustainability built into the design of D-RASATI 2 relates to the selection of training beneficiaries for maximum impact on the public school system. In close consultation with MEHE, D-RASATI 2 decided to prioritize the delivery


39

of its trainings to permanent teachers as the retention rate of full-time teachers is reported to be higher than that of part-time teachers (MEHE discussions, May 2013). Training of school administrative and teaching staff who are at least 5 years away from retirement was also prioritized to allow sufficient time to apply the knowledge and skills gained through D-RASATI 2 and bring about the desired change in attitude and behavior that comes with practice.

Challenges and Responses

Several challenges were faced by the D-RASATI 2 team throughout the project life which, at times, hindered a smooth implementation of some activities. The project team was resilient and creative in their mitigation responses, at times, redesigning the activities. The impact of these hindrances and the mitigation measures taken are described below:

Delays to requests from D-RASATI 2 to give feedback or approvals due to limited resources at MEHE and CERD

These delays had substantial impact because of the short life of D-RASATI 2. As a result, the start and end of most activities were delayed, and certain deliverables could not be completed according to schedule. Delays also shortened the period in which activities could be implemented, exerting extra pressure on D-RASATI 2’s human resources and potentially reducing benefits to the targeted groups. To expedite the administrative procedures necessary for the implementation of the project activities, D-RASATI 2 provided MEHE with administrative staff support to update MEHE’s database, produce lists of beneficiaries, contact targeted trainees, and sort requests for exemptions from trainings. Following are the most salient examples of these challenges: Delays in receiving MEHE’s approval on the list of the 126 ICT beneficiary schools: It was not until February 2014 that the project received the final list of schools to benefit from the ICT procurement, this delayed the implementation of subsequent and prerequisite activities for the launching of the ICT procurement process. The original plan called for procuring ICT equipment by June 2014, so that the equipment could be distributed and installed at the schools before the start of the 2014-2015 school year. This would allow teachers to receive ICT in-school coaching and use the equipment during the same academic year. However, the delay in approving the list of schools delayed the planning for, and start of, the ICT school readiness assessment which is the 1st of several pre-requisites to ICT procurement. The assessment was planned to take place over a 3-month period from November 2013 to January 2014. The delay in approving the schools shortened the time window for implementing the assessment within the originally projected deadline. To mitigate the shortened timeframe, and to keep with internal and contractual deadlines, the project mobilized and trained a large number of volunteers who are beneficiary of another USAID-funded project, the University Scholarship Program (USP) at AUB, LAU and Haigazian universities. As a result, , the assessment in 126 schools and 6 CERD centers was completed by March 31, 2016, i.e. in one month time. In tandem with the readiness assessment, and in close consultation with MEHE, D-RASATI 2 reviewed and updated the technical specifications of the ICT equipment which were defined under D-RASATI 1, and a comprehensive school procurement model was developed and presented to MEHE, along with a proposed optimal combination of number of schools per technological tier. MEHE approved the whole process only on June 13, 2014. In anticipation of MEHE’s approval, D-RASATI 2 had finalized the Request for Quotation (RFQ) document, and thus was able to launch the bidding process on the same date. Still, this did not allow for the completion of the procurement process before the start of the 2014-2015 school year.


40

As a result of the above, the equipment was received by schools late in the school year, the ICT coaches of the ICT in-school program had very little time to train teachers on the use of the Tier Model technology, i.e. the student tablets and the classroom management LANschool software. To mitigate the impact of this delay D-RASATI 2 requested USAID on August 3, 2015 to approve a no-cost extension of the ICT in-school coaching program for 2 additional months. The objective was to provide schools with the necessary support for a smooth start of the 2015-2016 school year using the ICT tier technology. The additional coaching included the provision of TOT to CERD trainers and DOPS counselors on the operation and maintenance of the ICT Tier Model equipment, along with providing refresher training to teachers. Delays on the part of MEHE to approve the list of teachers to sit for BULATS: This delayed and intensified the efforts to administer the test within the deliverable deadline set for December 13, 2013. This caused D-RASATI 2 to allocate more resources than budgeted, and the project had to increase the number of testing sites per day which had implications on the recruitment, training and standardization of an unplanned additional number of examiners, invigilators and administrators. Delays in approving the ECA strategy and the ECA Modules: Delay to finalize the ECA strategy by the contractual deadline of March 31, 2014, which is the pre-requisite to all other ECA activities, negatively impacted the successful completion of the ECA task as per contractual agreement. The development of the ECA strategy, ECA modules, training material, planning and implementing ECA trainings could not start before a MEHE ECA Committee was formed. The committee was formed only on March 5, 2014 – 6 months into the 2-year D-RASATI 2 project –, and the 1st meeting with the D-RASATI 2 ECA team took place on March 18. The extended time taken to form the committee caused delays in beginning discussions on the content of the ECA strategy, ECA modules, and the development of ECA training materials in time to deliver ECA trainings between April and June 2014 as planned. In the meantime, D-RASATI 2 attended to all other tasks that could be implemented without requiring MEHE’s input such as drafting major sections of the ECA strategy to have it ready for discussion once the ECA committee was formed; interviewing a number of candidates in anticipation of recruiting ECA trainers; and developing school selection criteria and a list of schools to receive the ECA training. Long lapses in time between decision meetings and delays on the part of MEHE to provide feedback delayed the finalization of the ECA strategy until December 2015, 21 months behind schedule. The finalized strategy document was officially submitted to MEHE on January 15, 2016. Likewise, delays in the issuance of critical approvals delayed the start of ECA trainings for MEHE staff and school representatives to January 2015. This resulted in the necessity to condense the ECA training for schools from 4 phases delivered over 8 months into 3 phases delivered over 4 months. Unfortunately, only few of the representatives were thus able to report back to the ECA team on their experience and results of implementing ECA and evaluate their work. Nonetheless, the school representatives who completed the 3 phases of the training are considered empowered to implement ECAs using the new skills gained. As for the ECA modules, agreement on the broad themes for the ECA modules was reached in a meeting with MEHE on August 5, 2014. By December 31, 2014, first drafts for 80% of the activities under the 7 ECA modules had been completed. However, the ECA team was only granted meetings to discuss the modules with MEHE two times from January to July 2015. MEHE’s approval to share the modules with CERD was only received on July 27, 2015. Accordingly, the ECA activities were illustrated and designed, assembled into 25 booklets, and submitted on September 30 to CERD for


41

their review and approval. The pioneer nature of the ECA subject in the public school system, the lack of clear standards on how to present the ECA activities in an officially published text, and the turnover in the team of CERD reviewers led to a lengthy process of review and re-review of the text. In particular, the latter factor has resulted in a changing outlook on ECA at CERD, and hence a different set of requests for revisions. By July 26, 2016, the ECA team had re-revised all the ECA activities and resubmitted them to CERD in a 7th round. On August 26, 2016, the project received a letter from CERD approving the printing of modules after the ECA team incorporates a last round of feedback, such as adjusting the Arabic language of all modules, re-working and expanding on the objectives and steps of all activities, and re-working the illustrations of the modules to suit different age groups. The delays mentioned above culminated into:

. Concluding the ECA task through printing and distributing the ECA modules was delayed by 18 months; and

. Defaulting on the contractual conclusion of the last ECA-related deliverable, i.e. conducting regional workshops for all intermediate and secondary schools. In essence, completing the revision, design and printing of the ECA modules could not be done in time to allow their distribution in regional meetings by November 30, 2016, one month prior to project closure in December 2016, as per the financial directive for project closure.

Despite defaulting on a contractual deliverable, the project countered by redesigning the ECA distribution activity and incorporating a sustainability element in the process. The project hired a contractor to distribute the ECA modules along with an ECA stationary and art kit to each of the 1,041 schools, and the distribution of the ECA modules and kits was completed by December 23, 2016. It also implemented a TOT on the use and implementation of the ECA modules for CERD teacher trainers. In November CERD designated 33 of its trainers to receive the TOT which was delivered over 2 days on November 25 and 26. This will enable CERD to rollout ECA trainings to teachers within their future professional in-service development curricula. Delays to the Implementation of activities in hosting schools: The constant and main challenge for this task was MEHE’s continuous delay in engaging with D-RASATI 2 to implement PSS-related activities, and in providing approvals and clearances required for the component to proceed to implementation, which took 11 months. The D-RASATI 2’s contract amendment of September 2014 refined and expanded the activities targeting primary host schools. The activities included a Rapid Teacher Workforce Study (RTWS) trainings on PSS for CERD, DOPS and teachers from primary host schools, and an impact study to measure the effect of the PSS activities on reducing tensions in the classroom. During 9 months, from October 2014 to August 2015, D-RASATI 2 persistently engaged its main counterparts, MEHE and CERD, through several meetings to present the concepts of PSS and the advantages of equipping CERD trainers and MEHE staff and teachers to streamline the PSS approach in education. In December 2014, D-RASATI 2 requested MEHE and CERD’s approval to develop PSS training material. MEHE delayed its response by 7 months, until the PSS training material developed under a UNICEF-CERD project in 2014 was reviewed by DOPS. In the meantime, D-RASATI 2 attended to all other tasks that could be implemented without requiring MEHE’s input such as developing concept notes and methodologies for the PSS activities, identifying consultants, meeting with potential candidates for consultancy, and drafting a 2nd contract amendment that includes a no-cost extension of D-RASATI 2 to enable the implementation of PSS-related activities.


42

It was only on August 19, 2015, that MEHE approved D-RASATI 2 to develop its own PSS material since the material that had been developed at CERD was found not to be sufficiently comprehensive. MEHE approved D-RASATI 2 to deliver the TOT starting November 6, 2015. The delays in approving D-RASATI 2 to develop the training material until August 2015, and the start of the TOT in November resulted in a very tight timeframe to allow the development of all training modules before the start of the TOT. The late scheduling of the TOT resulted in that the 8-non-consecutive days TOT could not be completed before February 2016 due to the multiple demands on the CERD and DOPS staff. It also pushed back rolling out the training to teachers in 27 staggered regional cohorts from February to May 2016. This left only 2 months, April and May, for teachers to implement the PSS methodology in their classroom before the close of the school year. This in turn, affected the effectiveness of the PSS impact study which thus had only 2 months to measure the impact of teacher training on the classroom environment in terms of reduced tension among students. The awareness campaign: MEHE was very slow from the start to engage in discussions about the awareness campaign. It was only on April 4, 2014 that a focal point at MEHE for the awareness campaign was appointed. The 1st meeting with MEHE to discuss the broad lines of the awareness campaign took place on May 15, 2014 – 8 months into the project. The 2nd meeting with MEHE was granted 5 months later, on September 24, 2014, in which the campaign’s objectives, concept and plan of action were presented. MEHE approved the concept on December 1, two and a half months later, despite the time sensitivity of the campaign and the need for several months of preparation and groundwork for the campaign material to become ready for dissemination. The concept included filming videos to run on local TV stations in the form of Public Service Announcements (PSAs). MEHE’s final approval to start filming the videos was received on April 24, after which filming immediately started. On May 12, a MEHE committee reviewed the videos and requested minor modifications. Accordingly, the videos were revised and officially shared with MEHE on June 2 for their final approval. However, despite persistent follow-up, MEHE did not provide its official final approval of the videos or its endorsement of the awareness campaign in time to allow the implementation of the campaign by the original contractual deadline of July 2015. In response, WL granted a no-cost extension of the activity deadline to allow airing the PSAs as soon as MEHE’s approval is received. By May 30, 2016, however, MEHE’s feedback and endorsement were still awaited. Thus, on June 7, 2016, the project requested USAID to amend the contract’s language and reword the contractual deliverable and indicator from “implement the awareness campaign” to “develop the awareness campaign”.

Outdated/incomplete data at MEHE on schools and school staff

A comprehensive survey of public schools was undertaken under D-RASATI 1 in 2011, and the MEHE database was then updated accordingly. At proposal writing in 2013, the project used the available data to set its targets and select the beneficiaries of its programs. The related impact and mitigations measures taken by the project are described below: LDP and SIP programs: The impact of outdated data resulted in the project setting inflated targets for LDP and SIP which necessitated a project amendment to reduce targets and reflect the reality on the ground: . LDP program: At D-RASATI 2 proposal writing, LDP’s beneficiaries were determined to be 670

school principals. In February 2014, after cleaning the list of principals received by MEHE, only


43

409 school principals (61% of target) were found eligible to join the program due to attrition such as retirement and death. In response, and in support of MEHE’s goal to provide LDP training to all principals of the public school system, D-RASATI 2 conveyed to MEHE that it has the capacity to train 200 additional principals. These principals could be selected from a pool of 350 principals appointed after 2001 which LU – and not MEHE – had the mandate to train on LDP, by Law 73. This brought the number of school principals to join the program to 487 (73% of the original target). The project thus, facilitated a 2nd MOU between MEHE and LU to legally enable the 200 principals to receive the LDP training under D-RASATI 2. The MOU was signed by January 29, 2015. In parallel, D-RASATI 2 requested a project amendment from USAID to decrease the LDP target from 670 to 485. The amendment was approved on September 29, 2014.

. SIP program: Likewise, at D-RASATI 2 proposal writing, 244 school principals were determined to be eligible for the SIP training according to 2 selection criteria – principals having already received the LDP training in 2005 and being at least 5 years away from retirement. Applying these criteria in early 2014 to MEHE’s database turned out a list of 107 schools (44% of the target). D-RASATI 2 requested MEHE to amend the proximity to retirement criterion to two years, which turned out a total of 208 school principals (85% of target). School teams from the 208 were thus invited to receive the SIP training.

Support to host schools: At project start, MEHE shared the 2012-2013 data for the number of the public schools student population (Lebanese and non-Lebanese). This data was already largely outdated starting 2014 as the numbers of Syrian refugees in Lebanon tremendously increased over the past 2 years. On January 23, 2015, D-RASATI 2 officially requested MEHE to provide updated data on the number of students in the public schools, but MEHE’s response was never received during the life of the project.

Lower attendance to some trainings than anticipated

Outdated data on the public school staff and delays in communicating invitations to trainings via the official channels converged and resulted in low compliance with the invitation to attend D-RASATI 2’s BULATS test and some long-running trainings. BULATS placement test: Delays on the part of MEHE to approve the list of teachers to sit for BULATS in December 2013 and to communicate to them to report to the BULATS test sites resulted in only 112 teachers (21%) out of 539 invited to sit for the test. While calling the principals of the schools to which the 539 teachers report, D-RASATI 2 found that 225 teachers (42%) were no longer candidates to sit for BULATS for different reasons. Out of the 314 eligible candidates, 202 teachers on the list were repeat invitees under DR1 and repeat no-shows. D-RASATI 2’s findings were shared with MEHE to update its database. Long-running trainings: Registration to the English language, ELTM rollout, ICT TOT online, and LDP cohort 1 trainings was less than anticipated: 49%, 65%, 64%, and 60%, respectively (Table 3), despite MEHE and CERD’s confirmation of the critical need to enhance the knowledge and skills of the public school workforce on the related subjects. Table 3 – Participant registration and attendance at selected trainings

Training Invited registered % Attended 50-79%

of the training %

Attended more than 80% of the training

%

English language 5,070 2,509 49% 499 20% 1,539 61%

ELTM rollout 345 224 65% 13 6% 188 84%


44

ICT TOT online 61 39 64% N/A N/A

LDP Cohort 1 304 182* 60% N/A N/A

* Although 223 school principals registered, only 182 continued with the training and sat for the exit tests. This number is the most meaningful in this analysis

The low attendance could be attributed to the length and intensity of the training: 2 days a week for 3 hours per day over 16.5 weeks for the English language; 5 days a week for 6 hours per day over 3 weeks for the ELTM rollout; and online presence and interaction for 4 months for the ICT TOT online. Reasons given by invitees for non-compliance were many: health-related, non-eligibility such as proximity to retirement, conflict with other trainings, changes in job status, a high workload at work, working in the afternoons, and other unvoiced reasons. Another possible reason for low turnout to trainings is the delay in communicating the official invitations to teachers and school principals. Invitees were left with very short notice to make to rearrange their schedules and attend long-running trainings. Thus, large numbers of the invitees requested MEHE to exempt them from trainings, as allowed by the employee bylaws. Often, exemption approvals were issued well after the training had started. When trainees dropped attending in the middle of the training, the class dynamics and the sense of appropriateness and validity of the training were negatively affected. D-RASATI 2 promptly communicated with MEHE and CERD to keep them informed of the low turnout, and submitted lists of teachers and school principals not attending the trainings and their reasons for inability to attend. MEHE verbally communicated to D-RASATI 2 its decision to refuse all excuses given by invitees not to attend the trainings except for health reasons and proximity to retirement in the case of English language and ELTM roll out trainings; and subsequently officially informed teachers and school principals of its decision. CERD and the Regional Education Bureaus also followed up with the absentees and their school principals. In addition, the project assigned a 3-person team to help MEHE sort exemption requests, make a timely decision, and build its database. In the case of LDP Cohort 1, MEHE informed D-RASATI 2 that it is giving the choice to those school principals who are close to retirement whether they continue with the training or not. In addition, D-RASATI 2 took several measures to encourage and increase attendance: D-RASATI 2 was proactive in reinforcing MEHE’s invitation to trainings by calling school principals and teachers to give them all the detailed information on the trainings. The D-RASATI 2 team also systematically followed up with the training participants who were no-shows or skipped classes through private messages and calls to encourage them to participate. However, this placed a substantial strain on D-RASATI 2’s teams and extra unbudgeted expenses, especially for the English language training. For example, in preparation for the Y2 program, a 10-person team was assembled to make calls and send the lists of teachers to the schools by fax or email. 412 schools (95%) were reached by phone, and 312 (73%) of schools received a second notification either by fax or email. The team tried to collect as many teacher phone numbers as principals were willing to give, and more than 850 teachers were called on their phone to confirm receipt of invitation and/or to ask about timing or location preference. One-thousand one-hundred eleven (1,111) teachers received personalized phone messages to their mobile phones indicating start date, class schedule, training location address and a contact number. D-RASATI 2, in close consultation with MEHE, amended the training schedule and day length without compromising the quality of the training. With respect to English language training, the twice a week 3-hour training was scheduled after school hours. For many teachers, this conflicted with family obligations and extra work scheduled in the afternoon to supplement income. In collaboration with MEHE, the 3-hour schedule was amended so that half of it falls within normal


45

school working hours. The teachers who did not comply with the invitation to attend Y1 English training were invited again in Y2 and Y3. And an intensive and shorter duration course was offered during the summer. With respect to ELTM rollout, the timing and length of the training day was amended to fit better with the trainees’ regular school work schedule. The length of the training day was shortened from 9:30 am - 5:30 pm to 8 am – 2:30 pm by reducing the number and length of lunch and coffee breaks; and the last 2 training days were condensed into 1 and thus reducing the training from 14 to 13 days. With respect to the ICT TOT online, D-RASATI 2 took several measures to encourage the no-shows to participate in the course and motivating those who did not have a sufficient online presence to increase their online participation and interaction. The measures taken consisted of:

. Extending the period during which the absentees were allowed to join the online platform;

. Monitoring online presence and reporting to CERD, which in turn contacted the no-shows and irregular participants to encourage them to participate or increase their interaction;

. Posting engaging questions on the online discussion fora to increase interaction among trainers and trainees;

. Providing constructive feedback was provided by the subject matter specialists to appropriate contributions from the trainees; and

. Providing certification of course completion to those who successfully complete their training to boost trainee motivation and participation levels.

The efforts of D-RASATI 2’s teams resulted in a large number of the registered trainees to follow through with attendance and complete their training as shown in Table 3 above: The majority (63%) of those who registered attended more than 80% of the trainings, 61% and 84% for English language and ELTM rollout, respectively. Only 18% of the registered trainees dropped out. This could be attributed to the high value of the training to the trainees professional as well as personal lives, and the dedication of the trainers to make the training beneficial and their systematic follow-up with the trainees skipping classes.

developing rehabilitation assistance to schools and

Documents