opportunity to learn as a measure of school effectiveness in ethiopia, guatemala, honduras, and...

7/28/2019 Opportunity to Learn as a Measure of School Effectiveness in Ethiopia, Guatemala, Honduras, and Nepal

1/36

WORKING PAPER

Using Opportunity to Learn and Early GradeReading Fluency to Measure School Eectiveness inEthiopia, Guatemala, Honduras, and Nepal

Executive SummarySchool eectiveness is a concept that is dicult to dene and even more dicult tomeasure. o most educational planners, eectiveness is the measure o actors thatenhance a childs learning, irrespective o their background. While many models oschool eectiveness exist, the Five-Factor model suggests that leadership, acquisition o

basic skills, a secure environment, high student expectations, and requent perormanceassessment are critical elements o eectiveness (Scheerens, 2000).

Tis study argues that improvement in school eectiveness requires schools andeducators to concentrate on even more basic elements than those posited by the research.Tis ocus should be on providing a basic opportunity to learn (OL) by having theschool open every day, teachers present, and students present and ready to learn. Teeducational value o that basic opportunity then depends on how teachers and studentsuse the time available during the day, how much time is spent on academic activities,whether materials are present and used, and whether class sizes are reasonable. Further,the study establishes an OL prole based on 12 actors, including: 1) percentage o

days the school is open; 2) teacher attendance; 3) student attendance; 4) percentage odays let or instruction; 5) percentage o time on task; 6) percentage o equivalent dayslet or instruction; 7) percentage o students with textbooks; 8) percentage o textbookuse; 9) percentage o time spent reading; 10) Grade 3 reading fuency; 11) class size; and12) school support. Tese indicators measure the extent to which opportunity to learnwas optimized in a sample o schools in Ethiopia, Guatemala, Honduras, and Nepal.

Te study answers the ollowing questions:

How well do schools provide opportunity to learn?How does actual opportunity to learn compare to potential opportunity to learn?How does opportunity to learn vary across countries and schools?

MethodologyData or the study were collected through rst-hand eld research in Ethiopia,Guatemala, Honduras, and Nepal, made possible through collaboration with CARE,Save the Children USA, and each organizations in-country oces. Samples o schoolsin each country were drawn rom those participating in CARE and Save the Childreneducation programs based on stratication by school size and location. Controlschoolsthose not participating in the NGO-supported programswere also sampled.

Field research teams visited each school and collected data through the use o a series oinstruments including: concepts about print (CAP); Early Grade Reading Assessments


2/36

Measuring School Efectiveness: Ethiopia, Guatemala, Honduras, and Nepal

(EGRA); Stallings Classroom observation protocols; school observations; and interviewswith teachers and principals. Te sample included 24 schools in Ethiopia, 26 inGuatemala, 33 in Honduras, and 23 in Nepal. Data analysis included regression,analysis o variance, and actor analysis, along with qualitative analysis o interviews.

ContextTe CARE and Save the Children programs in each country have provided a variety osupport to the targeted communities and schools or a number o years. In Ethiopia,Guatemala, and Nepal, Save the Children ocuses on giving every child the opportunityto attend school. In Honduras, CARE works to secure educational opportunities andquality schooling to children in rural areas.

Students in the study were evenly split by gender in Ethiopia, Guatemala, andHonduras. In Nepal, there were more boys than girls. Te average age o students inthe study was 10 years. In Ethiopia, however, more than 71 percent o the sample wasolder than 10 years, compared to 40 percent, 9 percent, and 30 percent in Guatemala,Honduras, and Nepal, respectively. Students in Guatemala, Honduras, and Nepal hadhigher participation rates in kindergarten (73 percent, 86 percent, and 66 percent,respectively) than Ethiopia where only 28 percent o sampled students attendedkindergarten.

Spanish was the only language spoken in the areas visited in Honduras. O the multi-lingual countries, Ethiopia had the most linguistically homogenous set o students,

with 91 percent speaking Aan Oromo. Guatemala had the most diverse group with 34percent speaking Kiche as their mother tongue (and also speaking Spanish at home), 30percent speaking Mam, and 25 percent speaking Ixil. Another 11 percent o students inGuatemala spoke only Spanish. In Nepal, the students spoke mainly Nepali and Taru.

ResultsHow well do schools provide an opportunity to learn?Schools in Ethiopia, Guatemala, Honduras, and Nepal were open over 90 percent oschool days, and teacher and student attendance rates were airly high (between 88 and97 percent). However, time loss due to late starts, early closings, and time-o-task was

signicant in all our countries. In Guatemala, sample schools used only 72 percento the available day or instruction because o late starts, early closings, and prolongedbreaks or recess. In Guatemala and Nepal, students were on task only 59 and 60percent o the time, respectively. In Honduras, students were on task 56 percent o thetime while in Ethiopia, they were on task only 41 percent o the time.

Across the our cases, when time loss due to school closings, teacher and studentabsence, and time-o-task were combined, schools on average used less than 50 percento the equivalent available days or instruction. Expressed in terms o the number odays in the school year, this equivalent time equated to 69 days in Ethiopia, 56 days inGuatemala, 78 days in Honduras, and 87 days in Nepal. For Nepal, the value would


3/36

have been even lower i student attendance were taken into account. Ethiopias valuewould also have been lower i time-use at school were taken into account.

On average, Ethiopia and Nepal were better at providing language textbooks or Grades13 than Guatemala and Honduras. However, students were observed using thesebooks a very small percentage o the time. Students were also rarely observed reading inclass: less than 12 percent o the time in Ethiopia, Guatemala, and Nepal. Oral readingfuency in all countries was low or Grade 3, but was better or Spanish speakers inGuatemala and Honduras. Te reading fuency averages or Ethiopia and Nepal o 18and 26 words per minute (wpm), respectively, were caused by the signicant number oGrade 3 students unable to read. Average class sizes were reasonable in all our countries,and schools, on average, reported receiving a sucient number o support visits during

the year.

How does actual opportunity to learn compare with potential opportunity to learn?Te ocial school year was 203 days (812 intended instructional hours) in Ethiopia,180 days (900 intended instructional hours) in Guatemala, 200 days (1050 intendedinstructional hours) in Honduras, and 192 days (1152 intended instructional hours)in Nepal. All our countries oered more hours o intended instruction, on average,than was ound in their regions or Grades 13: 789 hours or Latin America and theCaribbean, 665 or South and West Asia, and 809 or sub-Saharan Arica (Benavot,2004).

How does opportunity to learn vary across countries and schools?Variation in opportunity to learn across schools in all our countries was airly extensive.Te smallest amount o variation was ound in three OL actors: days schools are open,teacher attendance, and student attendance. School-level variations or these actorsranged rom 70 to 100 percent, with Ethiopia having the greatest variation in days openand Guatemala having the greatest variation in teacher and student attendance.

Te variation in the percentage o time-on-task in all our countries was broad, butthe range was widest in Ethiopia and Nepal. One school in Ethiopia averaged almost70 percent student time-on-task during a lesson, while others were below 20 percent.In two schools in Nepal, students were on task 80 percent o the time, but one schoolaveraged just 33 percent. Te percentage o time spent on task in Honduras ranged rom34 to 75 percent. In Guatemala, the range was much smaller, with the lowest time-on-task at 47 percent and the highest at 71 percent. Te countries varied in how on-tasktime was used in the classroom (e.g., or copying rom the board, seatwork, discussion/debate). When teachers were on task, however, students were more likely to be on task.

Language textbooks were readily available in most o the studys schools or students inGrades 13. On average in each country, 810 schools provided over 75 percent o theirstudents with a language textbook, and 56 schools provided textbooks or all students.In Guatemala, the percentage o students with textbooks in each school varied the most

(between 0 and 100 percent). While more than 90 percent o students across the studyhad access to language textbooks, researchers rarely observed the books being used in


4/36


class. Students were rarely observed reading aloud, reading silently, or interpreting text(less than 12 percent o the time in Ethiopia, Guatemala, and Nepal and 21 percent othe time in Honduras). In addition, none o the countries had an ocial curriculum orteaching reading.

Tis lack o reading in classrooms was refected in students reading fuency scores. InNepal, the majority o students either could not read at all (44 percent) or were able toread 4160 wpm (26 percent). In Ethiopia, very ew students read more than 40 wpmand the largest percentage (36 percent) could not read at all. Not a single Ethiopianstudent was able to read more than 70 wpm. Students reading scores in Guatemalawere more evenly distributed: only 4 percent were unable to read, 8 percent read 120wpm, and 46 percent read more than 50 wpm. Students in Honduras were the strongest

readers averaging 73 wpm. Over 60 percent o Honduran students sampled were able toread at least 70 wpm and 35 percent could read above 90 wpm.

ConclusionsTe ndings in Ethiopia, Guatemala, Honduras, and Nepal suggest that: a) a great dealo instructional time was lost; b) little, i any, reading instruction took place in mostclassrooms (Honduras excepted); and (c) while textbooks existed, their eective use wassuspect. Additionally, while students had airly good knowledge o CAP, their readingfuency scores were generally below the acceptable threshold o 60 wpm, except inHonduras.

Te OL prole urther revealed that variation among schools in each country wasconsiderable and that none o the 12 OL actors correlated signicantly with learning.While sample sizes were small and likely impacted regression results, the researchersbelieve the lack o a relationship was due to the small amount o classroom readinginstruction.

Te researchers did nd some relationships. Ethiopian students working outsidethe home or money and repetition were correlated with higher reading scores.In Guatemala, the language children reported speaking at home was associated todierences in reading scores. Honduran students were observed reading more oten,

which was refected in their higher reading scores. In Nepal, teacher attendance and theavailability o textbooks were highly correlated with reading.

Te OL prole and subsequent analysis rom these cases provides insights into therelationship between schools, instruction, and learning. Unless there is a minimumamount o instruction, specically in reading, then educators should not expect arelationship between opportunity to learn and learning. Tis nding has importantpolicy implications or those implementing programs or making education sector policy.

Policy Implicationsime Loss: Schools in all our countries used less than hal o their potential opportunity

to learn. Within classrooms, less than 12 percent o the time was spent reading. Tisnding suggests a need to develop curricula that ocuses on teaching reading skills


5/36

and ensures that teachers are trained to teach reading. Tere is also a need to empowercommunities to monitor and supervise schools. Research has shown that community-managed programs that engaged parents ensured that schools were open and teachersand students were present.

extbook Availability and Use: While most students had textbooks, their use in class waslimited. Tis lack o textbook use indicates that teachers need training to use textbookseectively in class. Te training should include pedagogically sound approaches such asactivity centers that emphasize word-use and structure, silent reading o story books, andpeer reading.

Language arts books should also include stories and texts that allow students to practicereading. Tese books should engage childrens imaginations and excite them aboutlearning to read. International development organizations and developing countrygovernments also need to recognize the lack o childrens storybooks as a missingingredient in promoting literacy. Learning to read will always be an uphill battle inplaces where the environment is devoid o interesting reading material.

Linking eacher raining to Literacy Acquisition Strategies. eacher training workshopsare oten held o-site and might not introduce teachers to useul pedagogical andmanagerial concepts. As a result, a behavior change approach to teacher improvementis needed that can identiy classroom-based practices needed to engage students.Evaluation o behavior change over time needs to be conducted and tracked. Educators

and communities also need to encourage an environment that is supportive o behaviorchange.

eacher training and support should work in parallel with the provision o easels,storybooks, and other reading materials. raining should also prepare teachers tostructure classrooms and class time in ways that are conducive to reading. In this liesa more complex set o investments with unclear nancial trade-os that should becareully examined in each context.

Other policy implications rom this study include investing in remedial support orstudents unable to read, ensuring that teachers use assessment to inorm instruction,

and using school support services to help teachers implement organizational andinstructional changes.

While there are trade-os and political challenges to each implication, educators mustnd concrete ways to ensure that students learn to read. Te OL prole can serveas a useul tool or educators to gauge how students are progressing. As an evaluativetool, the prole allows ministries o education to see school variations at the regional,district, or national levels and develop appropriate interventions. Unless there is a greaterocus on instructional rather than administrative support, ocial visits to schools willcontinue to have limited impact on the opportunity to learn and learning outcomes

ound in schools.


6/36


IntroductionSchool eectiveness is a concept that is dicult to dene and even more dicult tomeasure. o most educational planners, eectiveness is the measure o actors thatenhance a childs learning, irrespective o their background. While many models oschool eectiveness exist, the Five-Factor model suggests that leadership, acquisition obasic skills, a secure environment, high student expectations, and requent perormanceassessment are critical elements o eectiveness (Scheerens, 2000).

Tis study argues that improvement in school eectiveness requires educators toconcentrate on even more basic elements than those posited by the research. Schoolsneed to ensure opportunities to learn (OL) or students, measured, in this study, bytime spent on learning activities. Opportunity to learn begins with ensuring that school

is open and that teachers and students are present. Te educational value o that basicopportunity then depends on how teachers and students use the remaining availabletime, whether materials are present and used, and whether class sizes are reasonable.

Instructional time is a multi-aceted concept. While the importance o sucientinstructional time and its impact on student achievement is well documented, thelength and ocus o time or improving student learning remains unclear (Berliner,1990; Benavot and Amadio, 2004; Abadzi, 2009). How much instructional time is lostin schools? Should educators extend the school day or year? Or, should the ocus beon improving the use o existing instructional time? I existing time is increased, will itimpact student achievement?

Tis study documented the loss o eective instructional time and argues thateducational interventions should ocus on improving the use o existing instructionaltime. Te study urther establishes a 12-actor OL prole to analyze the variationsin school eectiveness and instructional time within samples o schools in Ethiopia,Guatemala, Honduras, and Nepal. Te study answers the ollowing questions:

How well do schools provide opportunity to learn?How does actual opportunity to learn compare to potential opportunity to learn?How does opportunity to learn vary across countries and schools?

A literature review ound no studies that had examined extended allocated instructionaltime in developing countries. However, studies by Stallings (1980), Berliner (1990), andAbadzi (2007) showed that the instructional time variables, including allocated time,transition time, opportunity to learn, waiting time, and academic engagement werealterable, easy to measure, and understood by teachers. Changes in these time variablesare easy to make and quickly aect classroom perormance, as noted by Berliner(1985). While policy analysts nd greater interest in allocated time because it is easy tomanipulate, it is a weak predictor o improvement in learning.

Data collected rom schools in Ethiopia, Guatemala, Honduras, and Nepal indicated

low levels o opportunity to learn and instructional time and diered greatly rom theseschools potential opportunity to learn. Data also varied considerably between schools


7/36

across most opportunity to learn actors. In particular, the teaching o reading wasinadequate in most schools included in this study. Tese ndings highlight a series oissues related to school eectiveness and the optimization o opportunity to learn suchas ensuring that the school is open every day, that teachers and students are present, andthat students spend an increasing amount o time on task.

Defnition o an OL ProfleTe concept o opportunity to learn began in the 1960s when John Carrollacknowledged that students lagging behind could master the intended content givenmore instructional time. Te concept ocuses on the time allocated to teaching,learning, and curriculum coverage and is supported throughout the literature (Bloom,1968; Gettinger, 1984; Abadzi, 2007; Gillies and Quijada, 2008). Te OL prole

begins with the premise that student learning is a unction o time, eort, andeective instructional activities as outlined by Gillies and Quijada o the EducationalQuality Improvement Program 2 (EQUIP2). In addition to OL actors that impactthe amount and use o classroom time, this study looked at the availability and useo materials, student ability to read fuently in the language o instruction, and theprovision o support services to schools.

o construct a measurable OL prole, this study drew on Gillies and Quijadas work,using several actors verbatim and adding and modiying others to arrive at the set o 12OL actors described in able 1. A detailed discussion o each actor orms the core o

this study.

MethodologyData or the study were collected through rst-hand eld research in Ethiopia,Guatemala, Honduras, and Nepal, made possible through collaboration with CARE,Save the Children USA, and each organizations in-country oces. EQUIP2 providedthe research design and most o the data collection methodology. CARE and Savethe Children provided input to the data collection approach and recruited interns tomanage eld data collection. Both organizations country oces provided access toschools with which they worked and organized all eld logistics: CARE in Hondurasand Save the Children in Ethiopia, Guatemala, and Nepal.

Samples o schools in each country were drawn rom those participating in CAREand Save the Children education programs, based on stratication by school size andlocation. Control schoolsthose not participating in the NGO-supported programswere also sampled. Field teams then visited each school and conducted interviewswith the school director and teachers in Grades 13. One-hour observations using theStallings methodology were conducted in Grade 1, 2, and 3 classrooms. A randomsample o Grade 3 students were interviewed, were given a 10-item assessment basedon Concepts about Print (CAP, a pre-literacy evaluation o amiliarity with printedmaterials developed by Marie Clay), and completed a battery o Early Grade Reading

Assessment (EGRA) tools including letter recognition, word recognition, and readingtext.


8/36


able 1. Opportunity to Learn Factors

Factor Measure Comment

1. Percentage o daysschool is open

Te percentage o days school is openas scheduled on the academic calendar

Accounts or time lost whenschools open later or close earlierthan scheduled in the school yearand when schools are closed whenthey should be open

2. eacher attendancea Te percentage o days school is openin which teachers are present

Uses the average attendance rateor the entire sta o a school

3. Student attendancea Te percentage o days school is openin which students are present

Uses the average attendance rateor all Grade 3 students o a school

4. Percentage o theschool day available

or instruction

Te percentage o the availableclassroom time when teachers and

students are together

Accounts or time lost due to latestart or early end o the school day,

recess, and other breaks

5. Percentage o studenttime-on-taskb

Te percentage o classroom timewhen students participate ininstruction-related activities

Uses observed percentage o timeand number o students engaged ininstruction-related activities

6. Equivalentpercentage odays available orinstruction

Te percentage o equivalent daysavailable or instruction ateraccounting or time lost due to schoolclosing, teacher and student absence,daily time loss, and time-o-task

Summarizes OL Factors 15

7. Percentage ostudents with a

textbook

Te percentage o Grade 13 studentspossessing a reading or language

textbook8. Percentage o

observed textbookuse

Te percentage o observed timeduring which students use anytextbook

Uses one-hour observations inGrade 1, 2, and 3 classrooms (totalo three hours)

9. Percentage o timespent reading

Te percentage o observations duringwhich at least one student was reading

Uses one-hour observations inGrade 1, 2, and 3 classrooms (totalo three hours)

10. Grade 3 readingability

Te number o words o gradeappropriate text read correctly perminute

Uses EGRA in the language oinstruction

11. Class size Te average number o students

enrolled in Grade 3

Uses the average registered

number o children per Grade 3class in each school, not those inattendance during observations

12. School support Te number o visits by supportpersonnel during the school year

Uses reports rom principals andteachers and includes all visits byeducation ocials, NGO sta, orother support sta

a) For teacher and student attendance, data were collected or seven months o the school year in Guatemala andHonduras. In Ethiopia, rates were estimated based on one month o attendance data. Attendance data or Nepal

was not available.b) On-task activities were dened by the Stallings observation instrument and include: reading out loud,discussion/debate, demonstration/lecture, seatwork, copying, verbal instruction, practice/drill, reading text, and

interpreting text.


9/36

ContextTe our eld studies included in this research were conducted in collaboration withCARE in Honduras and with Save the Children in Ethiopia, Guatemala, and Nepal.Te CARE and Save the Children programs dened the context in which the schools inthe study operated. Te programs in each country had been operating or a number oyears and each provided a variety o supports to the targeted communities and schools.In Ethiopia, Guatemala, and Nepal, Save the Children ocuses on giving every childthe opportunity to attend school. In Honduras, CARE works to secure educationalopportunities and quality schooling to children in rural areas.

Te study drew samples o 24 schools in Ethiopia, 26 in Guatemala, 33 in Honduras,and 23 in Nepal. Te Ethiopia sample included 15 community schools and 9

government schools, o which 6 received no support rom Save the Children. InGuatemala, the sample included 20 Save the Children-supported schools and 6 controlschools. Te Honduras sample included 27 CARE-supported schools (3 in peri-urbanareas and 24 in rural districts) and 6 control schools. In Nepal, o the 23 schools in thesample, 7 were control schools.

able 2 summarizes inormation about the students included in the study. Studentswere evenly split by gender in Ethiopia, Guatemala, and Honduras. In Nepal, there weremore boys than girls. Te average age o students in the study was 10 years. In Ethiopia,however, more than 71 percent o the sample was older than 10 years, compared to 40percent, 9 percent, and 30 percent in Guatemala, Honduras, and Nepal, respectively.Students in Guatemala, Honduras, and Nepal had higher participation rates inkindergarten (73 percent, 86 percent, and 66 percent, respectively) compared toEthiopia where only 28 percent o sampled students had attended kindergarten.

Spanish was the only language spoken in the areas visited in Honduras. O the multi-lingual countries, Ethiopia had the most linguistically homogenous set o students,with 91 percent speaking Aan Oromo. Guatemala had the most diverse group with 34percent speaking Kiche as their mother tongue (and also speaking Spanish at home), 30percent speaking Mam, and 25 percent speaking Ixil. Another 11 percent o students inGuatemala spoke only Spanish. In Nepal, the students spoke mainly Nepali and Taru.

Family size and other socio-economic characteristics were similar within each countryssamples. Within each country, schools were located in areas with similar levels oeconomic development.


10/36


able 2. Characteristics o Students in Sample Schools

Ethiopia Guatemala Honduras Nepal

# o Grade 3 students tested 456 505 388 480

Boys 51% 50% 50% 54%

Girls 49% 50% 50% 48%

Mother tongue: Aan Oromo 91% Kiche 34% Spanish 100% Nepali 47%

Gurage 6% Mam 30% Taru 41%

Amharic 3% Ixil 25% Rana Taru 8%

Spanish 11% Other 3%

Average age 10.5 10 9 10

% older than 10 yrs 71% 40% 9% 30%

% who attended K 28% 73% 86% 66%

% not repeating 77% 88% 89% 94%

% live < 10 min to school 37% 51% 30% 48%

% live 10-30 min. to school 45% 40% 44% 38%

% live >30 min. to school 17% 10% 26% 15%

Average amily size 7 7 7 7

% o students work or $ 26% 29% 14% 0%

% who have at home:

Radio 82% 94% 97% 66%

oilet 77% 82% 55% 22%

Electricity 13% 79% 30% 68%

elevision 3% 63% 31% 25%

Rerigerator 2% 22% 15% 0%

> 3 o the above 11% 35%

Findings

How well do schools provide an opportunity to learn?able 3 and summarizes the average OL actor values or sample schools in eachcountry. Schools in each country were open over 90 percent o the days on the schoolcalendar and teacher and student attendance rates were airly high. Only a small portiono opportunity to learn was lost due to Factors 13. However, other actors lowered theprovision o opportunity to learn.

In Guatemala, sample schools used only 72 percent o the available day or instructiondue to late starts, early closings, teacher and student absences, and prolonged breaksin the day or recess (i.e., 30 minute recess periods ran as long as 60 minutes). In allour countries, additional opportunity to learn was lost when student time-o-task in

classrooms was taken into account. In Guatemala and Nepal, students were on taskonly 59 and 60 percent o the time, respectively. In Honduras, students were on task 56percent o the time while in Ethiopia, they were on task only 41 percent o the time.


11/361

able 3. Summary o Perormance on OL Factors

Factor

1 2 3 4 5 6 7 8 9 10 11 12

%Open

cherattdrate

Stdtattdrate

% odayused

%o

%daysinst

%w/text

% obsusingtext

%obsread

Oralread

(wpm)Classsize

#visits/

yr

Ethiopia 93% 89% 97% -- 41% 33% 83% 4% 3% 18 44 18

Guatemala 97% 88% 92% 72% 59% 33% 63% 3% 11% 46 27 7

Honduras 93% 94% 97% 82% 56% 40% 58%a 22% 21% 73 29 5

Nepal 90% 91% -- 92% 60% 45% 84% 14% 12% 26 44 3

a) Grade 3 only

In all our countries, when time loss due to school closings, teacher and student

absences, and o-task students were combined, schools used less than 50 percent o theequivalent available days or instruction (OL Factor 6). Expressed as the number odays in the school year in which children and teachers were in school at the same time,this equivalent time came to approximately 69 days in Ethiopia, 56 days in Guatemala,78 days in Honduras, and 87 days in Nepal. In Nepal, the value would have been loweri student attendance were taken into account; in Ethiopia the value would have beenlower i time-use during the day were taken into account.

On average, more schools in Ethiopia and Nepal than in Guatemala and Honduras hadlanguage textbooks or Grades 13 (provision o materials is an explicit aspect o Save

the Childrens programs). However, students were observed using those textbooks a verysmall percentage o the time (OL Factor 8). Students were also observed reading inclass similarly low percentages o time: less than 12 percent o the time in three o thecountries. Oral reading fuency in Ethiopia, Guatemala, and Nepal was low or Grade3, while students were clearly more procient readers in Honduras. Te averages inEthiopia and Nepal (18 and 26 wpm, respectively) were made lower by the signicantpercentages o Grade 3 students who could not read at all. Average class sizes werereasonable in all our countries and schools reported receiving support visits quiterequently in Ethiopia, requently in Guatemala and Honduras, and a ew times per yearin Nepal.

How does actual opportunity to learn compare to potential opportunity tolearn?o compare actual opportunity to learn with potential opportunity to learn, the studylooked at the combined eect o OL Factors 15 to determine the equivalent numbero days o instruction as compared to the total number o days in the school year. Teocial school year was 203 days in Ethiopia (812 intended instructional hours), 180days in Guatemala (900 intended instructional hours), 200 days in Honduras (1050intended instructional hours), and 192 days in Nepal (1152 intended instructionalhours). All our countries oer more hours o intended instruction, on average, thanis ound in their regions or Grades 13: 789 hours or LAC, 665 or South and WestAsia, and 809 or sub-Saharan Arica (Benavot, 2004).


12/36


From the ocial number o school days on the calendar, the number o days schoolwas closed was subtracted. In all countries, no school was open every day it shouldhave been. Days were lost when schools opened ater the ocial start date or whenschool was closed because o weather, local holidays, or school director absences, orexample. Additional days were subtracted rom the potential total to account orteacher and student absences. In all our countries, the loss o time during the schoolday in the periods or which data were available was converted into an annual estimateo equivalent days lost. Lastly, student time-o-task during observed lessons was alsoconverted into an annual estimate o equivalent days lost. Figure 1 summarizes time lostduring the school year and how this erodes the time available or opportunity to learn.

Figure 1. Equivalent Days o Schooling Available or eaching and Learning

203

180

200

192190

175

186

174168

146

175

158162

135

170

139145150

200

250

OfficialDays School

Closed Teacher

Absence

StudentAbsence DailyTimeLost Timeontask

97

69

56

78

87

0

50

100


In Ethiopia, only the equivalent o 69 out o 203 days were used or instruction (whichwould have been lower i data on time lost during the day were available). In Guatemala,instruction occurred only on the equivalent o 56 out o 180 days. In Honduras, anequivalent o only 78 out o the possible 200 days o instruction were used. In Nepal,the equivalent o 87 out o 192 days were used or instruction (which would have beenlower i student attendance data were available).

In addition to comparing the time-related OL actors to the total potential time orinstruction, data on the percentage o observations during which children were reading

(OL Factor 9) shed light on the actual versus potential opportunity to learn, or in thiscase, the opportunity to practice reading. In all our cases, the percentage o observations


13/361

in which students read material (including rom the blackboard) was very low: 21percent in Honduras, 12 percent in Nepal, 11 percent in Guatemala, and only 3 percentin Ethiopia.

Obviously, students could not have been reading all the time, but the opportunity topractice reading should be larger than such a small raction o each school day. Forexample, 3 percent o a ve-hour school day equates to 9 minutes o reading per day.Abadzi (2008) notes that achieving automized reading processes requires extensivepractice with books and phonics, particularly in languages such as English, French,Portuguese, and Spanish. Such low amounts o daily practice cannot be expected to leadto automization. It is important to note that reading scores were higher in Honduras,which could have been linked to the larger amounts o reading observed in the

classroom.

How does opportunity to learn vary across schools?Te previous section shared the average values o various OL actor across all sampleschools, disaggregated by country. However, more signicant than average values wasthe variation in opportunity to learn across schools within each country. Both the natureand quality o the opportunity to learn provided at each school varied considerably, withschools perorming better than the average on some actors while doing worse on others.While the non-control schools in each country were the objects o school improvementeorts (e.g., installation o inrastructure, provision o materials, training o teachers,ongoing nancial and technical support), the impact o these eorts was not uniormlyevident across schools.

Te ollowing section provides actor-by-actor discussion o this variation and reportswhether any signicant correlations were ound among OL actors, in particularwhether OL Factor 10 (student reading ability) correlated with any o the other actors(alone or in combination).

OL Factor 1: Te percentage o days school is openIn Ethiopia, no school in the sample was open every day. Many schools initiated theschool year ater the ocial start date and most were closed additional days. At least 2 o

the 26 schools were open less than 90 percent o the days available or instruction. Oneschool had been closed or nearly all o the rst three months o the school year.

Guatemala varied the least in the percentage o days schools were open. Some schoolswere open every day, and one school was open on 90 percent o the scheduled schooldays. When closures did occur, most were due to teacher training days or unplannedholidays. School closures in Guatemala (ve days, on average) tended to ollow a similarpattern to other countries in the region where days lost to school closings or Grade 4students ranged rom a high o 6.3 in Paraguay to a low o 1.3 in Uruguay (Zhang, etal., 2008).

In Honduras, 50 percent o interviewed principals reported beginning the school yearlate, on average ve days ater the ocial start date. Not one school in the sample was


14/36


open every day o the academic calendar. Schools most oten reported closures orteachers pay days, strikes, and teacher training. Every rural school reported closing itsdoors once per month so teachers could cash their paycheck. On the days o this studysobservations, 40 percent o the schools reported having lost over two weeks o schooldue to unocial closings.

Te same challenges were present in Nepal as in Ethiopia, where not one o the sampleschools was open every day. Tere was a great deal o variation in school closings,ranging rom a low o 4 days closed to a high o 39 days. At least 8 o the 23 schoolswere open less than 90 percent o the days available or instruction.

Figure 2. Opportunity to Learn Factors: Variation Across Sample Schools

OL Factor 2: eacher attendanceIn Ethiopia, the teacher attendance rate was obtained rom the ocial teacherattendance record o each school. eachers attended school an average o 89 percento the time. Tree schools had teacher attendance rates o 70 to 80 percent, nine hadteacher attendance rates o 80 to 90 percent, and nine had attendance rates above 90percent.

In Guatemala, teacher attendance data were dicult to obtain as many schools lackedattendance books. Absentee data were not documented in a consistent ashion and otennot collected until later in the year, i at all. Generally, an average absenteeism rate was

triangulated rom the principal log books and the teachers student absenteeism logsacross the number o months that the schools had been open. On average, across the 26


15/361

schools in Guatemala, teachers were absent two days per month, an attendance rate oroughly 88 percent. Some schools had perect teacher attendance (based on the availablerecords), while others had attendance rates as low as 68 percent.

In Honduras, teacher attendance data were collected in a manner similar to that inGuatemala. In schools with only one or two teachers, the student attendance log was theonly ocial school attendance record kept. On average, across the 33 schools, teacherswere absent 12 days per year. Only 2 schools had teacher attendance rates below 90percent while 15 had attendance rates o 95 percent and higher.

In Nepal, the researchers were unable to obtain ocial records or teacher attendance.However, principals were asked to report on teacher attendance during the week prior tothe interview. Tese attendance rates were used to estimate teacher attendance over theentire school year. On average, teachers were present 91 percent o the time. wo schoolshad teacher attendance rates just below 80 percent, and ve others had rates between 80and 90 percent. Te remaining 16 schools reported attendance rates over 90 percent.

Tese attendance rates were consistent with studies conducted by Chaudhury (2005)and Abadzi (2009) that showed teacher absence rates between 11 and 27 percent inBangladesh, Ecuador, India, Indonesia, Peru, Uganda, and Zambia.

OL Factor 3: Student attendanceI a student is not present and ready to learn, he or she cannot take advantage o the

other opportunities to learn assembled at the school. However, similar to teacherabsenteeism, ew studies have quantied student absenteeism due to scarce school andteacher attendance records. Similar to this study, available international research, suchas the work o Zhang, et al. (2008), has ocused on principal and teacher perceptionso student absenteeism, rather than actual attendance records. However, 2003 PISAdata indicated student absenteeism rates ranging rom 27 to 90 percent in Brazil, HongKong-China, Indonesia, Russia, Serbia, Tailand, and unisia. Tis study acquired andanalyzed data on student attendance or Ethiopia, Guatemala, and Honduras, but notNepal.

In Ethiopia, data obtained rom schools attendance books in the rst ew months o the

school year indicated that students in most schools were present more than 90 percento the time. Only one school had an ocial attendance rate lower than 90 percent.A specic objective o Save the Childrens program was to bring schools closer to thecommunities in which children live. Tereore, it was not surprising that the schoolsin the sample drew students rom their immediately surrounding villages. More than80 percent o students in the sample lived within a 30 minute walk to school with 37percent walking less than 10 minutes. Tis contributed to the high attendance rates.However, spot checks during classroom observations oten revealed a higher rate oabsenteeism than ocially recorded.

In Guatemala, there was considerable amount o variation in student attendance rates.Te data collected by this study indicated that students were present 92 percent o the


16/36


time. Te lowest attendance rate was 83 percent, which corresponded to a loss o 31days in that school. Since most students traveled between 15 and 18 minutes to school,the relatively high student attendance rate could have been due to the close proximity oschools to students homes, although students consistently arrived late.

Student attendance rates in Honduras were high across the schools in the sample. Onaverage, students were present 97 percent o the time. Only 2 schools had attendancerates lower than 90 percent and 14 schools had attendance rates o over 99 percent. Teadjusted school schedules in rural areas may have accounted or these high attendancerates. While city schools began class at 7:30 a.m., classes in the municipality oGujiquiro began at 8:00 a.m. to give students enough time to arrive at school. Giventhat, on average, students in the rural areas reported walking 25 minutes to school, the

delayed start may have contributed to higher attendance rates.

OL Factor 4: Percentage o the school day available or instructionTe percentage o the school day available or instruction takes into account the non-instructional components o the school day, such as recess. It also recognizes that schoolmay start late, end early, or experience interruptions in instructional time or a varietyo reasons (e.g., the teacher or students may be out o class). Tis study collected dataon the loss o instructional time during the school day in Guatemala, Honduras, andNepal.

In Guatemala, teachers and students were regularly observed arriving late to school.

Additionally, recess oten ran longer than scheduled. Te percentage o the day availableor instruction in Guatemala, Honduras, and Nepal was ound by subtracting this timelost and treating the remaining time (when teachers and students were in class together)as a percentage o the ull, ve-hour school day. On average, schools in the study usedonly 72 percent o the school day, with wide variation across schools. One school usedonly 57 percent o the day, and eight used only 67 percent o the day.

In Honduras, schools most oten started on time but experienced multiple breaks duringthe day. While most schools took both a recess and a lunch break, the length o thesebreaks varied considerably. Some schools took up to two hours or these breaks, whileothers took only 30 minutes. A number o schools did not give a lunch break due tothe governments ailure to provide student lunches. School days also ended at varioustimes due to parent meetings, bus schedules, or a lack o material to continue class. Onaverage, schools in the study used only 80 percent o the available learning time, witha wide amount o variation. One school used 100 percent o the day, compensatingor recess with an extra 30 minutes o class, while seven used only 70 percent o theavailable learning time.

Data on this OL actor or Nepal only accounted or a 30 minute recess each day,leaving 92 percent o the day available or instruction. In Ethiopia, no data werecollected on school start and end times, recess, or other interruptions. However, school

observations revealed that teachers and students were not always in class at the scheduled


17/361

time, suggesting that the use o available time or instruction was less than 100 percentin Ethiopia as well.

Little research has captured the time devoted to learning once school closures,absenteeism, and daily time loss are taken into account. However, research by Abadzi(2007a) showed that once these variables were accounted or, only 63 percent, 39percent, 71 percent, and 78 percent o the ocial days o instruction remained orlearning in Brazil (Pernambuco), Ghana, Morocco, and, unisia, respectively.

OL Factor 5: Percentage o student time-on-taskFor the purposes o this study, time-on-task reers to the time that students were payingattention to materials with instructional goals; the time spent on specic academicactivities such as reading, mathematics, or social studies; and learning time. Te StallingsObservation Instrument, used to collect the time-on-task data, measured both on-taskand o-task activities. eachers and students were considered on task when engaged inacademic activities such as reading aloud, discussion/debate, demonstration/lecture,verbal instruction, seatwork, copying, practice/drill, reading silently, or interpretingtext. O-task activities included discipline, classroom management, social interaction,student disengagement, and teachers or students being out o the classroom during anactivity.

Te overall equivalent time lost due to OL Factors 14 was small compared to the losso time when students were o task. Te variation in the percentage o time-on-task

in all our countries was broad, but the range was widest in Ethiopia and Nepal. Oneschool in Ethiopia averaged almost 70 percent o student time-on-task during a lesson,while others were below 20 percent. In Nepal, two schools were at almost 80 percento student time-on-task, but one was at 33 percent. ime spent on task in Honduranclassrooms ranged rom 34 percent to 75 percent. In Guatemala the range was muchsmaller, with the lowest student time-on-task at 47 percent and the highest at 71percent.

As able 4 demonstrates, on average, students were more likely to be engaged in non-instructional (o-task) activities than on-task activities. In all our countries, studentswere o task between 40 and 58 percent o the time. Ethiopia had the largest percentageo students observed in o-task activities at 58 percent. Most oten, these students weresocializing with others or simply not participating in the learning activity.

Concerning the on-task students, the learning activities in which students were engagedvaried between the our countries. In Ethiopia, on average, more students were engagedin demonstration and practice/drill activities while Guatemalan and Honduran studentswere observed most oten doing seatwork. In Nepal, more students, on average, wereobserved copying or doing seatwork than any other activity.


18/36


able 4. Use o ime in Class

Category o Activity Ethiopia Guatemala Honduras NepalNon-instructionala 58% 42% 44% 40%

Demonstration 11% 6% 5% 12%

Practice/Drill 10% 5% 4% 11%

Copying 7% 6% 5% 13%

Seatwork 5% 21% 25% 13%

Reading 4% 4% 6% 5%

Discussion 3% 7% 5% 2%

Verbal Instruction 2% 6% 5% 3%

a) Includes students who were interacting socially, being disciplined, involved in classroom management, orotherwise unengaged.

In addition to calculating the overall percentage o on-task students during anobservation, the researchers also looked at patterns in students behavior when theteacher was on or o task. able 5 shows that, in each country, teacher involvement ina learning activity led to greater percentages o on-task students. On average in Nepal,77 percent o students were on task when the teacher was on task, while in Guatemala,Honduras, and Ethiopia the averages were 69 percent, 62 percent, and 61 percent,respectively. However, when teachers were o task, students were much less likely to beengaged in learning activities. In both Ethiopia and Nepal, only 12 percent o students

were observed to be on task when teachers were o task. Tese numbers demonstrate theimportance o teachers leading or being involved with a lesson to increase the likelihoodthat their students will also be on task.

able 5. Use o ime in Class: eacher and Student ime-on-task Data

Percentage o students on task

Guatemala Honduras Ethiopia Nepal

Overall Average 58% 56% 42% 60%

When eacher Is on ask 69% 62% 61% 77%

When eacher Is O ask 32% 36% 12% 12%

eacherOn-ask

Grade 1 70% 63% 79%

Grade 2 73% N/A 55% 78%

Grade 3 66% 62% 74%

eacherO-ask

Grade 1 38% 14% 11%

Grade 2 29% N/A 11% 14%

Grade 3 30% 11% 10%


19/361

Researchers in Guatemala also used the Guatemalan Ministry o Education teacherpractice standards to determine eective classroom practice. While seatwork andcopying were still the predominate on-task activities, other activities were used airlyrequently to aid learning: cooperative, small group activities; debate and discussion,particularly in social studies and geography; and learning tools such as games andmanipulables. In act, teachers pedagogical skill levels rated reasonably well against theMinistry o Educations standards or time management, classroom management, anduse o instructional resources.

As with Factor 4, little research has captured the use o time in classrooms. While Brazil,Ghana, Morocco, and unisia dier in income and poverty levels rom the countries inthis study, they showed similar results. In Pernambuco, Brazil, students were engaged

in interactive learning 53 percent o the time, while students in Ghana were engagedapproximately 52 percent o the time. Rates were slightly better in Morocco and unisiaat 63 and 62 percent, respectively (Zhang, et al., 2008).

OL Factor 6: Equivalent percentage o days available or instructionTis actor measures the combined eects o OL Factors 15, all o which relate tothe amount o time available or teachers and students to engage in learning activities.o calculate the equivalent percentage o days available, researchers started with thepercentage o days school was open (the ocial number o school days minus thenumber o days a school was closed when it should have been open). eacher andstudent absence were actored in, as learning could only occur when both were present.

ime lost or breaks in the school day was then actored out. Finally researchersaccounted or time-on-task to dierentiate between classroom time used or learningactivities as opposed to non-instructional activities.

Tis study revealed that, on average, only the equivalent o 69 out o 203 days wereused or instruction in Ethiopia, 56 out o 180 days in Guatemala, 78 out o 200days in Honduras, and 87 out o 190 days in Nepal (numbers or Ethiopia and Nepalwould have been lower i data were available or time lost during the day and studentattendance, respectively). Tis study also uncovered considerable variation across schoolsin how OL Factors 15 interacted to determine an equivalent percentage o days each

school made available or instruction. Figure 3 shows this variation.

In Guatemala and Nepal, some schools had twice the equivalent number o daysavailable or instruction as compared to other schools. In Ethiopia, the schools thatmade the most time available or instruction had three times the equivalent availabledays than those with the least available time; in Honduras, some schools provided up toour times the number o equivalent instructional days than other schools.


20/36


Figure 3. Variation in the Equivalent Percentage o Available Days or Instruction

40%

60%

80%


Schoolswithgreatestamountoftime

Schoolswithleastamountoftime

0%

20%

IndividualschoolsineachcountryOL Factor 7: Percentage o students with a textbookFigure 4 shows that language textbooks were readily available or students in Grades 13in most o this studys schools. On average, up to 25 schools in each country providedover 75 percent o their students with a language textbook. In each country, ewer thansix schools had textbooks or every student. In Guatemala, six schools in the samplehad no language textbooks or students in Grades 13 and one school in both Nepaland Honduras similarly lacked this important resource. Although most Honduranclassrooms in the sample had some textbooks, less than 50 percent o the students hadexercise books, which were an integral part o the national curriculum.

In addition to language textbooks, students were asked i they had pencils andnotebooks. In all our countries, the great majority o students had all basic materials.Moreover, students were oten seen using these materials.

In Ethiopia, 18 observations revealed that students were engaged in practice and drill,which teachers taught mostly through the blackboard. During 19 observations, studentscompleted seatwork with their notebooks and 21 observations showed demonstrationtaking place in the classroom. eachers most requently used the blackboard as themedium o instruction or demonstrations. Students were observed engaging in activitiesrelated to reading in only 11 percent o observations.


21/36

Figure 4. Provision o Language Arts extbooks to Students

25%to50%

50%to74%

75%to99%

100%

geofstudentswithatextbook

Ethiopia

Guatemala

Honduras

N l

0 5 10 15 20 25 30

0%

1%to24%

Numberofschools

Percentag Nepal

In Guatemala, students were most requently observed doing seatwork in theirnotebooks (2224 observations, depending on the country). Some discussion/debateand demonstration was also observed in Guatemala, but much less requently (68observations). When these activities were taking place, the medium o instruction wasrequently the blackboard. Similar results were ound or Honduras and Nepal.

OL Factor 8: Percentage o observed textbook useA higher percentage o students were observed using textbooks in Honduras than inEthiopia, Guatemala, and Nepal. Figure 5 combines OL Factors 7 and 8 to show howtextbook availability and use were interrelated in the schools studied. Each data point inthe gure represents one school.

It is interesting to note that the majority o data points or Ethiopia, Guatemala, andNepal cluster in the lower-right side o the graph. Tis indicates that high textbookavailability in a school was associated with relatively low observed textbook use. Alsoo interest is that all outliers (schools with higher than average textbook use or a givenlevel o textbook availability) are ound in Nepal. Te results or Honduras show a morescattered use o textbooks, with more students observed possessing and actually usingtextbooks in the classroom.


22/36


Figure 5. Percentage o Students Observed Using a extbook or a Given Level oextbook Availability

Percentageo

fstudents

observed

using

any

textbook

Percentage of students with a language arts textbook

OL Factor 9: Percentage o time spent readingClassroom observations revealed that reading instruction was almost non-existent inall our countries. Students were rarely observed reading aloud, reading silently, orinterpreting text: Tis occurred less than 12 percent o the time in three countries and21 percent o the time in Honduras. Additionally, none o the countries had an ocialcurriculum or teaching reading.

While more than 90 percent o students in the study had access to language artstextbooks, researchers rarely observed textbook use in class. In Ethiopia and Nepal, the

books lacked reading passages and stories, making it dicult or researchers to developthe reading fuency test. In Guatemala, reading passages and stories were easy to locate,but were rarely in the mother tongue. In act, the only schools possessing Mayan storybooks were those supported by Save the Children, and even then these books were otenlocked in the principals oce. No structured approach to reading was observed in anyclassroom in this study. In Honduras, teachers reported waiting two to three years ortextbooks. Te district education oce reported that books would be delivered or the2008 school year, but at the time o the research visits (halway through the school year),new books had yet to arrive at these schools.

OL Factor 10: Grade 3 reading ability

Reading ability is both an outcome o opportunity to learn and a critical determinate owhether students continue to learn and advance in school. I students do not acquire an


23/36

adequate level o reading ability early in their schooling, they all urther behind. Tus,students continued opportunity to learn depends on their level o reading ability.

Given the lack o ocus on reading and reading instruction, it was not surprising tond low Grade 3 reading perormance. In Honduras, the site o the highest percentageo observed reading activities, students overall reading perormance surpassed thato students in the other three countries. While the average number o words read perminute provided a general idea o student reading levels, this number masked theenormously wide range o abilities ound in each country. As able 6 demonstrates,students in every country read at many dierent rates.

able 6. Frequency Distributions o Reading Fluency

Words perMinute Ethiopia Guatemala Honduras Nepal

Zero 36% 4% 8% 44%

1-10 15% 2% 1% 3%

11-20 9% 6% 2% 3%

21-30 13% 11% 2% 6%

31-40 12% 11% 2% 7%

41-50 9% 18% 6% 16%

51-60 3% 15% 8% 10%

61-70 2% 12% 11% 5%>70 0% 19% 60% 7%

In Nepal, the majority o students either could not read at all (44 percent) or were ableto read 4160 wpm (26 percent). In Ethiopia, very ew students read more than 40wpm and the largest percentage (36 percent) could not read at all. Not a single studentwas able to read more than 70 wpm. In addition, only 2 schools in Ethiopia and 11schools in Nepal averaged more than 25 wpm on the fuency test.

Students reading scores in Guatemala were more evenly distributed. Very ew children

could not read (4 percent), 8 percent read 120 wpm, and almost hal the sample (46percent) read more that 50 wpm. Students in the study averaged 47 wpm, but the rangeamong schools was between 25 wpm and 82 wpm.

Students in Honduras were, on average, the strongest readers in the sample with anaverage reading score o 73 wpm. Similar to Guatemala, ew children (8 percent) couldnot read and only 3 percent read at 120 wpm. Te majority o students (over 60percent) read at least 70 wpm, and 35 percent read over 90 wpm.

Neurocognitive research conducted by Abadzi (2008) suggests that all students shouldbe able to decode by the end o Grade 1; that all Grade 2 students should be able toread at least 60 wpm; and that Grade 6 students should be able to easily read 120150


24/36


wpm and provide a summary o the passage. In the Arabic script languages, all studentsshould be able to read eectively within one to two years o beginning instruction.

Identiying the causes o these reading level patterns across countries is essential inorder to improve students reading abilities. Tis study used Concepts about Printto assess whether students were acquiring skills oundational to learning to read. Asable 7 demonstrates, the majority o students in these cases had acquired pre-literacyoundation in either their mother tongue or the language o instruction. WhileEthiopian and Nepalese students had low reading scores, they knew their letters andprint concepts. Higher CAP scores were correlated with better reading perormance, butalmost all students answered most CAP questions.

able 7. Concepts about Print


# o CAPCorrect

Answers

% Studentsby CAP

scoreAvg

wpm

% Studentsby CAP

scoreAvg

wpm

% Studentsby CAP

scoreAvgwpm

% Studentsby CAP

scoreAvg

wpm

Zero 1% 9 34% 40 0% 40 1% 5

1 0% 6% 1% 2%

2 0% 3% 0% 3%

3 5% 2% 1% 3%

4 6% 2% 2% 6%

5 9% 3% 4% 8%

6 12% 9 7% 47 8% 50 8% 9

7 10% 9% 7% 11%

8 18% 24 18% 56 10% 74 20% 40

9 23% 11% 45% 29%

10 16% 5% 22% 11%

OL Factor 11: Class size

Class sizes in Grade 3 varied immensely. On average, class sizes in Ethiopia, Guatemala,Honduras, and Nepal were 46, 27, 29, and 44 students per class, respectively. At theschool level, class sizes varied rom 2877 in Ethiopia, 1040 in Guatemala, 1449 inHonduras, and 16104 in Nepal. In Ethiopia, ve schools had class sizes larger than 50students, with three containing more than 60 students per class. In Guatemala, only oneschool had a student/teacher ratio greater than 40:1 and in Honduras, only three schoolshad classes with over 45 students. Class sizes in the sample schools were generallyappropriate and, while higher than the average or OECD countries, were within therange o 2548 students per class seen in countries included in the World EducationIndicator Study (OECD, 2006).


25/36

OL Factor 12: School supportEQUIP2 research on complementary education programs ound that school supportservices were critical in allowing community schools to produce learning results similarto or better than their government school counterparts (DeSteano, et al., 2007).Tis study shows that schools in Ethiopia reported a greater number o visits than theother three countries, whether rom Save the Children sta or rom local and regionaleducation ocials (see able 3). On average, schools reported being visited almost twiceper month during the school year, with some claiming weekly visits. Tese numbers oreported visits may be overstated, however. In all our countries, the number o visitsreported by principals diered dramatically rom the number reported by teacherswithin the same school.

More important than the number o visits was whether these visits provided any benetto the school. For example, in Guatemala, visits reportedly ocused on classroommanagement or administrative support rather than instructional support to teachers.In Nepal, the number o reported visits per year was much lower than in Ethiopia andGuatemala. In all cases, school support visits did not correlate with any other studynding.

Policy ImplicationsRelationships among OTL Factors and Student Learning OutcomesResearch by Bloom (1968), Gettinger (1984), Benavot and Amadio (2004), and Abadzi

(2007) indicate that to improve students learning levels, a basic opportunity to learnmust exist. Tis opportunity to learn requires that teachers and students are present andspend most o the day engaged in learning activities. Research urther indicates that toomuch instructional time is wasted in classrooms due to poor classroom management,disciplinary action, long transition times, and teacher and student absenteeism(Stallings, (1973, 1980); and Aronsen, et al, 1998). Educators must make more ecientuse o the existing time available or instruction.

Te main ndings across Ethiopia, Guatemala, Honduras, and Nepal suggest that: a) agreat deal o instructional time was lost; b) little, i any, classroom reading instructiontook place; c) textbook use was suspect; and d) students reading abilities were low.

Given the small amount o classroom time spent on reading and the lack o a readingcurriculum, it was not surprising that students reading abilities were low. Whilestudents across all our countries knew letters and tested reasonably well on the CAP,reading fuency scores were well below the acceptable threshold o 60 wpm in Ethiopia,Guatemala, and Nepal. In act, in both Ethiopia and Nepal, large percentages ostudents in Grade 3 were unable to read at all. Teir perormance on the CAP implies,however, that even those students unable to read received some basic instruction inlower primary grades. Unortunately, this instruction was not sucient to make thetransition rom an introduction to written material to actual reading.


26/36


Te OL prole and subsequent analysis rom these country cases provides useulinsights into the relationship between schools, instruction, and learning: Unless there isa minimum amount o instruction, specically in reading, then a relationship betweenopportunity to learn and learning is unlikely. Across all our studies, schools varied inteacher and student attendance, textbook possession and use, class size, and number osupport visits by education personnel. Major variations existed across all OL actorsincluded in this studys research ramework. Despite these variations, in all countriesthere was a consistently small amount o reading instruction, little exposure to text, anda lack o opportunity or students to read (rates were higher in Honduras, however).Tis nding has important policy implications or those implementing programs ormaking education sector policy. Te ollowing discussion divides these implications intotwo categories: School Organization and Management and Classroom Environment and

Instruction.

Policy Implications: School Organization and ManagementSchool organization and management deals with areas such as school leadership,governance, accountability, and support, to which OL Factors 16 directly relate.Increasing the amount o learning time available to students is a rst step in improvingtheir overall academic perormance in reading (Snow, et al., 1998).

As noted by Aronson, et al. (1998), literature on the relationship between time andlearning dates back to the 1970s and typically ocuses on empirical, data-based research;policy reports combining educational theory and empirical research; or anecdotal,experientially-based periodical publications, usually explaining schools experiencesimplementing a certain time-related policy. Over the last several decades, school districtsthroughout the United States have studied extensions to the school day and year thatprovide more allocated time to students. However, research by Holsinger (1982), Nelson(1990), and Aronson, et al. (1998) has provided mixed ndings about the infuenceo allocated and engaged time on student learning. Research has revealed little or norelationship between allocated time and student achievement; some relationship betweenengaged time and achievement; and a larger relationship between academic time andachievement. In short, time does matter and is predictive o academic achievement iproperly used or engaged academic activities (Latham 1985 as cited in Hollowood, et

al., 1994; Brophy and Good, 1986; Greenwood, 1991). Research by Stallings (1973),Aronson, et al. (1998), and Abadzi (2007a, 2009) also show that the amount oallocated and engaged time is greatly reduced by time spent on non-academic activities,transition to classes or topics, poor classroom management, and disciplinary activities.

Schools in Ethiopia, Guatemala, Honduras, and Nepal, used, on average, less than halo the available opportunity to learn and even less o available instructional time. Tisloss o opportunity to learn was due primarily to o-task teachers and students. Further,this study ound that students being o task was directly tied to teachers being o task.Te loss o time rom teachers and students being o task in the classroom ranged rom

30 to 40 days per year. Outside o the classroom, students lost the equivalent o between


27/36

30 and 75 days o instruction due to school closures, teacher and student absenteeism,and late starts and extended recess.

Within the classroom, students spent the majority o the time doing seatwork, debatingor discussing subjects, and copying. A limited amount o reading (1020 percent oobservations) took place in the lower primary grades and an even smaller percentage oclassroom time involved students reading or analyzing text. While sucient textbooksseemed to exist in most classrooms, researchers witnessed limited use o these books. Innone o the our countries was there evidence o a reading curriculum.

When addressing time variables, the implications or policy reorm need to distinguishbetween actors that are easily addressed and those that require more complicated policyinterventions. OL actors such as school closures, absenteeism, and daily time loss aremore easily corrected than ensuring quality time-on-task, teacher training, and supportservices, because policy-makers can engage communities in solving these problems andholding schools accountable. EQUIP2 complementary education research supports theidea that community run and supported programs that engage parents help ensure thatschools are open and teachers and students are present.

Based on the results o this study, ensuring that schools are open 100 percent o theocial time, that the school day starts and ends on time, and that teachers and studentsare present would add 39 days o instruction in Ethiopia, 32 days in Guatemala, 30 daysin Honduras, and more than 50 days in Nepal. In Guatemala, ensuring that the schools

open on time and that recess is kept to 30 minutes would add an additional 49 days oinstruction. Even i the quality o instruction were held constant, it is likely that thisincrease in learning time would improve student learning outcomes.

Policy Implications: Classroom Environment and InstructionDecades o school improvement work have ocused on helping children learn throughinterventions such as teacher training and support and improved pedagogical methods.Yet, school quality still poses a challenge or educators and policy-makers alike. Studentslose important instructional time and oten lack the learning materials necessary toimprove perormance. Te answer to improving school quality and learning at times

seems elusive. Where should policy-makers and educators invest their resources? Howshould they prioritize interventions? While investments in improving actors such asschool opening, start and end times, and absenteeism can oten be addressed withimproved management o the school day, creating real improvements in the classroomand instruction is oten more dicult. Te ollowing discussion provides insights intointerventions that could improve the classroom environment and help children learn.

extbook Availability and UseAcross all our countries, 6384 percent o students had textbooks, on average. Yet, theuse o textbooks was airly limited, especially in Ethiopia and Guatemala. In Guatemala,textbooks oten sat on shelves unused and six schools had no textbooks or students.

Snow, et al. (1998) indicate that in the early grades, actors such as time, materials,and resources should support both daily independent reading o texts (selected based


28/36


on student interest) and daily assisted reading and rereading o more dicult textsthat advance students linguistic abilities. Tis approach to reading was rarely seen inclassroom observations.

Te availability and use o textbooks has important policy implications. First, as notedby Snow, et al. (1998) and Bruns, et al. (1999), textbooks should not be providedwithout helping teachers integrate these books into their instructional practice. rainingto eectively use textbooks should include pedagogically sound approaches such asactivity centers that emphasize word use and structure, silent reading o story books, andpeer reading.

Second, language arts books should include stories and texts that allow students topractice reading. Tis study ound that the content in language class textbooks inEthiopia and Nepal was limited. Examples o text that children could read were ew andailed to engage childrens imaginations and excite them about learning to read.

Finally, while most students in the studys schools had textbooks, the observed use wasairly low, and in some classrooms, essentially non-existent. Te CARE and Save theChildren programs, invested in delivering books to schools. Tis research demonstratesthat this alone alls short o ensuring that the textbooks are used in classrooms. Further,the amount o educational benet rom textbook use was unclear given their insucientnumber o reading passages, particularly in Ethiopia and Nepal.

Practice reading is critical or early grade students to learn to read. Snow, et al. (1998)points out that students should have storybooks that are below their rustration level(to encourage reading), and students should practice requently. I practice readingis essential or literacy among lower primary grade students, then books that provideinteresting stories should also be present and used in the classroom. Tere was adramatic shortage o local language story books in the three multi-lingual countries inthis study.

International development organizations and developing country governments need torecognize the lack o childrens storybooks as a missing ingredient in promoting literacy.Learning to read will always be an uphill battle in environments devoid o interesting

reading material. Projects, programs, and policy need to recognize that the creationo literate environments and the development o a culture o reading are as importantas the need to ocus on better in-school reading instruction. Projects, programs, andpolicy also need to ensure that the provision o storybooks is accompanied by theiruse, something almost never observed in this study. In Guatemala, Save the Childrendeveloped Mayan storybooks; however, these books were oten locked in the principalsoce. Tis is a prime example o a potentially useul intervention undermined becauseo a lack o knowledge and encouragement.


29/36

Linking eacher raining to Literacy Acquisition StrategiesLinked to the idea o promoting the proper use o books in classrooms is the prevailingapproach to teacher in-service training and support. Te programs included in this studyinvested considerable resources to train teachers. oo oten, that training was in the ormo o-site workshops. Tese workshops may or may not introduce teachers to useulpedagogical innovations, classroom management practices, and progressive conceptssuch as child-centered or active learning. However, independent o the content, themodel o teacher development that relies on teachers receiving instruction in a generalarea and transerring this knowledge into practice should be discredited, as noted byVillegas-Reimers (2003). Instead, a behavior change approach is needed that identiesthe specic classroom-based practices needed by teachers, and breaks down thesepractices into manageable increments o behavior change. Tis help must consist o the

chance to practice in a sae environment, clear benets to teachers exhibiting the newbehavior, consistent evaluation and eedback, and supportive organizational cultureswithin schools and communities that encourage these behaviors.

Observations in all our countries indicated airly low time-on-task and a consistentlack o engaging instruction in the early grades. In particular, the study ound a lack oinstruction tailored to the acquisition o the oundational aspects o literacy. Behaviorchanges or teachers should improve time-on-task, specically in literacy acquisition-related activities. Tis could be as simple as creating time or reading: teachers readingto students, students reading to students, and students reading by themselves. More

importantly, early primary teachers need to learn the undamental elements o teachingreading (e.g., soundletter correspondence, oral blending o sounds to read words,sight vocabulary) and need specic classroom practices that reinorce these elements.Classroom practices must link to the use o textbooks and other reading materialsto improve learning and engagement o students. Tese strategies also need to beconsistently applied by teachers in the classroom.

Te implication or policy-makers is twoold. First, classrooms must be equipped withadditional materials such as easels, storybooks, and supplies or students to createreading materials. More important is the linkage between the provision o these newmaterials and teacher training and support to ensure they are used in ways that help

students learn to read. In this lies a more complex set o investments with unclearnancial trade-os that should be careully examined in each context.

Providing Remedial or Accelerated SupportSchools also need help developing remedial strategies given the high percentages ostudents ound in this study who were unable to read or read at low levels o fuencyin Grade 3. Vaughn and Linan-Tompson (2004) write that students should learn toread by the end o their rst year in school. Many o the students in this study werenot reaching that goal. Unless specic strategies are deployed to ensure those studentslearn to read, they will all urther behind. Introducing interventions such as acceleratedlearning programs, ater-school tutoring, and special classes or students who are behind


30/36


are strategies that need to be deployed more systematically in schools similar to those inthis study.

Using Assessment to Inorm InstructionLinked to the idea o providing remedial support to students who are behind is theability to assess students literacy levels and identiy those who are acquiring the buildingblocks o literacy and those who are not. Te Ed Data II project continues to improvethe Early Grade Reading Assessment employed in this study, demonstrating its utilityin a variety o settings and languages. Such tools need to be used more systematically,not only to gauge whether learning is occurring, but also to allow teachers to spot checktheir students progress. EGRA can help teachers deploy instructional interventionsthat respond to their students needs, including identiying students needing signicant

remedial support.

Te OL prole includes reading fuency an indicator o school eectiveness. Tisprole provides a tool that could be used to contextualize the inormation aboutlearning outcomes provided by EGRA. Te OL prole provides educators, programmanagers, and policy-makers with useul, time-relevant data about variations amongschools across the 12 OL actors. Tis set o school eectiveness data can allow moretargeted support where assistance is needed. For example, one school may have hightime-on-task percentages, but high teacher absentee rates. Tis knowledge could helpa community or education ocial ocus on the specic issue at the school, in thiscase determining why teachers are consistently absent. Te OL prole would allow

individualized support to schools by allowing supervisors to collect school-relevant data.

School Support ServicesEQUIP2 complementary education research ound that school support services were acritical in the eectiveness and cost-eectiveness o complementary models. However,it is clear rom these our cases that the investments in school support were translatinginto neither improved opportunities to learn nor better learning outcomes or students.I schools receiving ongoing support are unable to implement the changes that lead tobetter use o time, then how can one expect schools to meet these challenges on theirown? Support resources need to be devoted to instruction, in particular to instruction

in reading. Unless there is a greater ocus on instructional rather than administrativesupport, visits to schools by ocials will continue to have limited impact on the actualopportunity to learn and learning outcomes obtained in schools.

ConclusionWhile there are trade-os and political challenges to each o these implications, it isimportant that educators begin to look at concrete ways to ensure that students learnto read. Te OL prole can serve as a useul tool to gauge students progress. As anevaluative tool, the prole allows ministries o education to see school variation atthe district, region, or national levels and target appropriate interventions accordingto need. For example, schools in one region may consistently start late, causing aloss o instructional time that simply requires better supervision to ensure on-time


31/36

starts. Another region may ace extreme losses due to o-task teachers and students.Understanding these nuanced dierences can help target interventions. At the schoollevel, principals and community members can compare their school to a nationalaverage, yet understand the variations in their community and region and gaugetheir progress against other schools. Finally, the OL prole can serve as a tool orcommunities to improve the accountability, governance, and management o schools.

o better support community eorts to monitor and support schools, EQUIP2 willsupport the introduction, monitoring, and evaluation o strategies to improve datacollection and use o OL actors through the development o a tool that will help localeducation ocials to easily collect OL data and track the results over time. Providingtools like these to schools and local ocials is just one o many important steps

necessary to sustainably enhance childrens learning in developing countries.


32/36


ReerencesAbadzi, Helen. 2007a.Absenteeism and Beyond: Instructional ime Loss and Consequences.Policy Research Working Paper No. 4376. Washington, D.C.: World Bank.

Abadzi, Helen. 2007b. Instructional ime Loss and Local-Level Governance. Prospects,37 (1): 1316.

Abadzi, Helen. 2009. Instructional ime Loss in Developing Countries: Concepts,Measurement, and Implications. World Bank Research Observer, 24 (2): 267290.

Anderson, Maria Elena. 2001. Guatemala: the Education Sector. Guatemala PovertyAssessment (GUAPA) Program, echnical Paper No. 2. Washington, D.C.: World Bank.

Aronson, Julia, Joy Zimmerman, and Lisa Carlos. 1998. Improving StudentAchievement by Extending School: Is It Just a Matter o ime? Paper presented at thePACE Media/Education Writers Seminar, San Francisco, April 20, 1998.

Benavot, Aaron, and Massimo Amadio. 2005.A Global Study o Intended Instructionalime and Ocial School Curricula, 1980-2000. Paper commissioned or the Educationor All Global Monitoring Report 2005, Te Quality Imperative. Geneva: UNESCOInternational Bureau o Education.

Berliner, David. 1990. Whats All the Fuss About Instructional ime? In Te Nature oime in Schools. Teoretical Concepts, Practitioner Perceptions, ed. M. Ben-Peretz and R.

Bromme. New York: eachers College Press.

Bloom, Benjamin. 1968. Learning or Mastery. UCLA Evaluation Comment, 1(2):18.

Brophy, Jere, and Tomas Good. 1986. eacher Behavior and Student Achievement.In Te Handbook o Research on eaching(3rd ed.), ed. M. C. Wittrock. New York:Macmillan.

CARE. 2009. Cares Work in Guatemala. Atlanta: CARE.

Carrol, John B. 1963. A Model o School Learning. eachers College Record, 64 (8):723733.

CIA. 2009. Te World Factbook: Guatemala. Washington, D.C.: CIA. www.cia.gov/.Viewed on November 15, 2009.

Clay, Mary. 2000. Concepts about Print: What Have Children Learned about the Way WePrint Language?Portsmouth, NH: Heinemann.

Edwards, John. 2002. Education and Poverty in Guatemala. Guatemala PovertyAssessment (GUAPA) Program, echnical Paper No. 3. Washington, DC: World Bank.

Fisher, Douglas. 2009. Te Use o Instructional ime in the ypical High SchoolClassroom. Te Education Forum, 73 (2): 168-176.


33/36

Gettinger, Maribeth. 1984. Individual Dierences in ime Needed or Learning: AReview o the Literature. Education Psychologist, 19 (1): 1519.

Gillies, John, and Jessica Jester-Quijada. 2008. Opportun

opportunity to learn as a measure of school effectiveness in ethiopia, guatemala, honduras, and...

Documents