kiribati early grade reading assessmentdocuments.worldbank.org/curated/en/1140615632511… · web...

Kiribati Early Grade Reading

Assessment (KiEGRA)

RESULTS REPORT

Kiribati Early Grade Reading

Assessment (KiEGRA)

RESULTS REPORT

Table of ContentTable of Content..................................................................................................................2List of Tables.......................................................................................................................3List of Figures......................................................................................................................5Executive Summary.............................................................................................................6

Summary of EGRA results and findings.........................................................................7Factors contributing to greater fluency and comprehension in Kiribati..........................8Recommendations............................................................................................................8

Chapter 1: Introduction......................................................................................................13Chapter 2: Instrument Design and Implementation...........................................................15

2.1. KiEGRA Survey Implementation......................................................................152.2. Sample Design...................................................................................................152.3. Development of the KiEGRA Instrument.........................................................152.4. Enumerator training...........................................................................................162.5. Data collection...................................................................................................162.6. Reliability of the Instrument..............................................................................17

Chapter 3: EGRA Results..................................................................................................183.1. Sub-test 1 – Letter Name Knowledge................................................................183.2. Sub-test 2 – Letter Sound Knowledge...............................................................203.3. Sub-test 3 – Initial Sound Identification............................................................223.4. Sub-test 4 – Familiar Word Reading.................................................................243.5. Sub-test 5 – Nonword Reading..........................................................................263.6. Sub-test 6 – Oral Passage Reading....................................................................283.7. Sub-test 7 – Reading Comprehension................................................................303.8. Sub-test 8 – Listening Comprehension..............................................................353.9. Sub-test 9 – Dictation........................................................................................373.10. Summary of assessment results.........................................................................41

Chapter 4 – Performance in reading fluency and reading comprehension........................45Chapter 5- Analysis of Student and Teacher Factors Associated with Better Reading Outcomes...........................................................................................................................50

5.1. Association of student characteristics to student reading outcomes..................505.2. Association of teacher characteristics to student reading performance.............525.3. Association of teacher expectations to student reading performance................535.4. Association of teacher training and guides to student reading performance.....555.5. Association of Classroom Environment to Student Performance.....................555.6. Association of Reading Materials to Student Performance...............................575.7. Association of Teacher Instructional and Assessment Methods to Student Performance...................................................................................................................57

Chapter 6 - Conclusions and Next Steps...........................................................................62ANNEX 1 / TABLES........................................................................................................67ANNEX 2 / INSTRUMENTS...........................................................................................79

List of TablesTable 1: EGRA Sample by Region, Year and Gender......................................................15Table 2: Reliability of the EGRA Sub-tests......................................................................17Table 3: Sub-test 1 Letter Name Knowledge: Results by Year and Gender.....................19Table 4: Letter Name Results by Region and Year...........................................................19Table 5: Sub-test 2 Letter Sound Identification: Results by Year and Gender.................20Table 6: Letter Sound Identification: Results by Region and Year...................................21Table 7: Sub-test 3 Initial Sound Identification: Results by Year and Gender.................22Table 8: Initial Sound Recognition: Results by Region and Year.....................................24Table 9: Sub-test 4 Familiar Word Reading: Results by Year and Gender.......................25Table 10: Familiar Word Reading: Results by Region and Year (without zero scores). . .25Table 11: Sub-test 5 Nonword Reading: Results by Year and Gender.............................27Table 12: Nonword Reading: Results by Region and Year (without zero scores)............27Table 13: Sub-test 6a Oral Passage Reading: Results by Year and Gender......................29Table 14: Oral Passage Reading: Results by Region and Year.........................................29Table 15: Sub-test 6b Reading Comprehension: Results by Year and Gender.................31Table 16: Correct Answers by Questions in sub-test 6b....................................................33Table 17: Distribution of Correct Answers.......................................................................34Table 18: Reading Comprehension: Results by Region and Year....................................34Table 19: Sub-test 7 Listening Comprehension: Results by Year and Gender.................35Table 20: Listening Comprehension Percentage of Correct Answers by Question..........36Table 21: Listening Comprehension Results by Region and Year....................................36Table 22: Sub-test 9 Dictation: Results by Year and Gender............................................38Table 23: Dictation: Results by Region and Year (without zero scores)..........................40Table 24: Distribution of percent correct scores for reading comprehension by sub-test

per year......................................................................................................................45Table 25: Percent of students reading with 80% or more comprehension by Year..........46Table 26: ORF score by reading comprehension score.....................................................47Table 27: Distribution of ORF Scores for students meeting and not meeting the 80%

benchmark..................................................................................................................48Table 28: Student background characteristics...................................................................50Table 29: Association of student characteristics to Oral Reading Fluency (ORF) scores.51Table 30: Profile of teachers in EGRA..............................................................................52Table 31: Association of teachers' characteristics to Oral Reading Fluency Score (ORF)

...................................................................................................................................52Table 32: Association of teacher expectations to oral reading fluency (ORF) scores.......54Table 33: Teacher training and guides...............................................................................55Table 34: Association of training and guides to student oral reading fluency (ORF) scores

...................................................................................................................................55Table 35: Average Number of Classroom Displays and Materials Observed...................55Table 36: Frequency and Type of Classroom Displays/Resources Available...................56Table 37: Association of Classroom Environment to Student ORF Scores......................56Table 38:Average Number of Reading Materials and Books in Classroom.....................57Table 39: Association of Reading Materials to Student ORF Scores................................57

Table 40: Frequency of methods used during reading instruction....................................58Table 41: Association of teacher instructional and assessment methods to student

performance...............................................................................................................58Table 42: Impact of student characteristics on Oral Reading Fluency (ORF) scores.......73Table 43: Effect of teachers' characteristics on Oral Reading Fluency Score (ORF).......74Table 44: Effect of teacher expectations on oral reading fluency (ORF) scores...............74Table 45: Effect of training and guides on student oral reading fluency (ORF) scores....75Table 46: Effect of Classroom Environment on Student ORF Scores..............................75Table 47: Effect of Reading Instructional Resources on Student ORF Scores.................75Table 48: Effect of teacher instructional and assessment methods on student performance

...................................................................................................................................76

List of FiguresFigure 1: Letter Name Results by Region and Year..........................................................20Figure 2: Letter Sound Identification: Results by Region and Year.................................22Figure 3: Initial Sound Recognition: Results by Region and Year...................................24Figure 4: Familiar Word Reading: Results by Region and Year (without zero scores)....26Figure 5: Nonword Reading: Results by Region and Year (without zero scores).............28Figure 6: Oral Passage Reading: Results by Region and Year..........................................30Figure 7: Reading Comprehension Results: Distribution of percentage of correct answers

by year.......................................................................................................................32Figure 8: Reading Comprehension Results: Distribution of percentage of correct answers

by gender...................................................................................................................32Figure 9: Reading Comprehension: Results by Region and Year.....................................34Figure 10: Listening Comprehension Results by Region and Year..................................37Figure 11: Dictation Results: Percentage of Correct Answers by Item and Year.............39Figure 12: Dictation Results: Distribution of correct answers by gender.........................39Figure 13: Dictation: Results by Region and Year (without zero scores).........................41Figure 14: Summary Results: Number correct answers for timed sub-tests by year.........42Figure 15: Summary Results: Percent correct for untimed sub-tests by year....................43Figure 16: Summary Results: Percentage of Zero Scores by Sub-test and Year...............43Figure 17: Summary Results: Percentage of Zero Scores by Sub-test and Gender...........44Figure 18: ORF distribution by reading comprehension level..........................................46Figure 19: Distribution of ORF scores for students reading with at least 80%

comprehension...........................................................................................................47Figure 20: Distribution of ORF scores for students meeting and not meeting 80%

benchmark..................................................................................................................48

KIRIBATI EARLY GRADE READING ASSESSMENTResults Report

Executive SummaryThis report summarizes the results of an Early Grade Reading Assessment (EGRA) conducted in Kiribati from September 21 to October 27, 2016. With funding from the Global Partnership for Education (GPE), the World Bank and Education Technology for Development (Et4d) carried out the assessment in collaboration with the Kiribati Ministry of Education (MOE).

The overall purpose of the EGRA was to inform education policymakers of students’ basic reading skills in years 1-3 of primary school and to identify factors that contribute to reading development in Kiribati. The findings are expected to assist policymakers with designing effective early grade reading interventions to improve school performance and literacy outcomes. This activity is part of the Pacific Early Age and Readiness Program (PEARL), which was established to improve the school readiness and literacy outcomes of children throughout the Pacific region.

The Kiribati EGRA (KiEGRA) was administered to a nationally representative sample of students enrolled in Years 1, 2 and 3. A total of 1,363 students (691 girls and 672 boys) participated in the assessment.

The EGRA tool consisted of six reading skills tests and two reading-related tests (listening comprehension and dictation). Unlike most EGRAs, which primarily test reading and listening skills, the KiEGRA included a short dictation exercise to assess early writing skills. In addition, a teacher and student survey that collected information on characteristics associated with reading outcomes was administered to identify factors contributing to reading fluency. The assessors also carried out a classroom observation in each school visited to assess the classroom environment and teaching resources available.

The analysis of KiEGRA data included descriptive statistics (means and standard deviations) to measure average levels in basic reading skills; an analysis of variance to determine statistical significance of gender and regional differences; and regression analysis to estimate the association of a given teacher, student or classroom characteristic and reading fluency outcomes.

The key findings, factors associated with reading performance, and recommendations are presented below.

Summary of EGRA results and findings Students show significant gains in word reading skills from Years 1 to 3.

There was measurable progress between Years 1-3 across all sub-tests, especially for the familiar word and oral reading fluency sub-tests. Scores on the familiar word subtest increased by 15 words between Years 1-2 (from 9 to 24 words per minute), and an additional 13 words between Years 2-3 to an average of 37 correct words per minute. On the oral reading passage sub-test, students’ scores improved by 18 words between Years 1 and 2 (from 10.9 to 29) and an additional 18 words between Years 2 and 3 (from 29 to 47). Thus, students in Year 3 are on the cusp of becoming fluent readers.

Students are not yet fluent readers: In order to understand a simple passage, students must read fast enough to retain the words in short-term memory. Research1 suggests a minimum fluency rate of 45-60 words per minute, depending on the complexity of the language. Overall, reading fluency scores averaged 29 correct words per minute. While Year 3 students read an average of 47 words per minute, only 20 percent could comprehend 80 percent of the text.

Students’ struggle with decoding. To read text fluently, students must be able to decode unfamiliar words by sounding out individual letters and syllables. Students scored low across three sub-tests that measure these skills: letter sounds, initial sounds, and nonwords. Overall, students identified an average of 27 letter sounds, 3 initial sounds, and 13 nonwords.

Reading comprehension levels are below the recommended benchmark. Over 80 percent of students in Years 1 and one-third in Year 2 have zero reading comprehension. In Year 3, comprehension improves where nearly half of students could comprehend at least 60 percent of text. However, the majority of students in Years 1-3 are reading below the recommended 80 percent comprehension benchmark. Only 14 percent of all students tested met the benchmark and about one-third of students in Year 3. The breakthrough point for reading fluency and comprehension is likely much later than Year 3 for the majority of students. An analysis of fluency scores falling in the 80 percent comprehension range revealed that students must be able to read between 40-69 words per minute with accuracy.

Girls have higher reading fluency and comprehension. Girls had a lower percentage of zero scores and scored equal to or better than boys across all sub-tests. The difference in girls’ and boys’ performance was statistically significant.

Regional differences were not statistically significant. Students in the Linnux regions performed better than other regions. Yet, the differences were insignificant.

1 Abadzi, H. (2011). Reading Fluency Measurements in EFA FTI Countries: Outcomes and Improvement Prospects. Education for all Fast Track Initiative

Factors contributing to greater fluency and comprehension in Kiribati The following factors2 were positively associated with better oral reading fluency, measured by the number of correct words read in the reading passage, per minute (cwpm):

Student characteristics: Student speaks Kiribati at home (+10 cwpm) Student likes to read (+8 cwpm) Student receives help with homework from mother (+5 cwpm) Student reads to himself/herself at home (+5 cwpm)

Teacher expectations: When teachers expect students to reach the competency later than the expected

grade level, students tend to have higher scores (+2-14 cwpm)

Classroom environment: Maintaining folder with student work and student info/student profile (+6 cwpm)

Teaching pedagogy and assessment: Teacher guided reading 1-2 days a week (+5 cwpm) Teaching reading comprehension (+2-5 cwpm) Child reading aloud or silently (+3-4 cwpm)

RecommendationsBased on the findings and conclusions, the following recommendations are presented to improve the quality of early grade reading instruction in Kiribati schools:

1. Set reading fluency and comprehension benchmarks to guide and support teacher professional development to reach reading targets: Reading benchmarks bring attention to the learning expectations both students and teachers should be held accountable to support children in becoming readers. They serve a useful function when they are coupled with strategies that will strengthen the capacity of schools and teachers to help students reach them. Without benchmarks, there is no pressure in the system to achieve learning outcomes and no fear of failure. At the same time, simply setting standards and benchmarks without evidence that schools and teachers are failing in their job and the support mechanisms to improve, benchmarks will create fear and frustration among school administrators, teachers and parents. Reading benchmarks should be established to guide the development of basic reading skills in each of the early grades both in terms of oral reading fluency and comprehension; and to guide the mechanisms that are required for schools to succeed in achieving them. As a first

2 Note that the average effects of the factors are not cumulative. In other words, students who possess more than one characteristic do not have double the effect by adding the values of both factors.

step, stakeholders should decide on the level of comprehension required to understand grade level text (e.g., 60 percent, 80 percent) and then review the fluency scores that fall within that range. Tonga, for instance, has set the reading fluency benchmark at 50 cwpm and a reading comprehension of at least 75%. Students achieving these levels can be considered to have learned to read well, have the basic reading skills needed to develop their literacy skills, and have the ability to comprehend more complex text in upper grades. If stakeholders agree with the 80 percent benchmark, then an acceptable fluency range may be 60-64 cwpm. Given that the current mean ORF score is 29 cwpm, that range may seem too high. Thus, policymakers may decide to lower the benchmark to 60 percent and the fluency benchmark to 53 cwpm. Once the benchmark is decided, the next step is to consider the targets. Currently, 14 percent of students are meeting the 80 percent reading comprehension benchmark. How many should meet the benchmark in one year or five years? Once the targets are set up, schools and educators should be supported to achieve these benchmarks. The Kiribati MEYS should then conduct a national reading campaign to sensitize all stakeholders of the new benchmarks and then monitor and report progress towards achieving the targets at all levels (national/regional MEYS, school and communities).

2. Conduct a follow-up study on teaching practices and assessment methods to identify strengths and weaknesses. Although 56 percent of teachers were trained on reading instruction in the past three years, teaching decoding skills and assessment were associated with lower ORF scores. It is important to understand why these interventions have an adverse relationship. It could be that teachers were not adequately supported to master the new teaching methodologies or that training is not aligned with best practices in reading instruction. Additionally, teachers may not understand how to accurately assess students and utilize the results for reflection and lesson planning. The analysis also showed that classrooms with teachers who maintained a record of students’ work had higher ORF scores. A follow-up study is recommended to investigate the strengths and weaknesses of current assessment and teaching methods vis-à-vis the results observed in different classrooms. It is also important to understand what types of information the teacher records and whether it is related to tracking student progress.

3. Strengthen teacher training for reading instruction with a focus on vocabulary, decoding skills and reading comprehension. Results indicate that students are weak in recognizing letter sounds and unfamiliar words, and have low reading comprehension skills. Hence, instruction in these areas should be strengthened to increase overall reading scores. In the case of Kiribati, the large shares of zero scores observed in various reading subtasks included in KiEGRA suggest inadequate instructional time in the classroom due to student absenteeism, school closures, or even inadequate use of instructional time -i.e. pedagogy or content-- during regular school days. Additionally, training should incorporate curriculum expectations and provide teachers with the specific methods, classroom activities and assessment methods required to achieve results.

4. Increase the number of books in the classroom library/reading corner and the use in teachers’ instruction. In Kiribati, 75 percent of classrooms have reading corners, but only 1.97 books are used during classroom instruction. One possible explanation is that there are not enough books in the classroom. The International Reading Association (IRA) recommends that classroom libraries start with at least seven books per child and purchase two additional new books per year. The optimal number of books in a classroom library is 300-600, depending on the grade level and number of copies3. The number of books teachers should expect children to read during the school year is 100-125 picture books by the end of Year 1 and 50-75 chapter books by the end of Year 2. There is strong evidence that reading skills grow and develop to the extent that children get actual practice reading. This is particularly difficult in countries with a modest literary tradition where the amount of titles in the local language may be limited. Recent innovations in technology now enable the production of reading materials in almost any language, provided there are guidelines for avid writers to follow. Educators in Kiribati have experience producing additional reading materials for children and their efforts should be further supported to ensure beginning readers have a reasonable variety of stories to practice and enjoy. With a corpus of reading large enough to provide sufficient reading practice, a low-cost option is to provide e-readers in low-resource countries where publishing may be expensive and there are high teacher-pupil ratios. E-readers allow students and teachers to choose from a variety of genres, it is portable so students can read from home or school, and its read aloud features provides additional support for emergent readers45. In addition to provision of an increased number of hard and soft copy books, teachers should be trained on how to better integrate materials into their instruction and on how to develop attractive reading corners6.

5. Increase time-spent reading. The more time children spend reading the better and more fluent readers they become. In order to increase students’ reading fluency skills, teachers should ensure that students are spending sufficient time reading every day through teacher-led, parent-led or self-guided reading activities. Research recommends that children read between 20-40 minutes per day. The results also showed that bringing books home on a regular basis was associated with lower ORF scores. It is important to engage parents in reading activities to ensure that students and parents are spending the recommended time reading at home.

3 Neuman, S. (undated). The importance of the classroom library. Available at: http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf 4 Adams, A. & van der Gaag, J. (2011). First Step to Literacy: Getting Books in the Hands of Children Available at: https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of-children/ 5 UNESCO (2014). Reading in the mobile era: A study of mobile reading in developing countries. http://unesdoc.unesco.org/images/0022/002274/227436E.pdf6 Neuman, S. (undated). The importance of the classroom library.

http://unesdoc.unesco.org/images/0022/002274/227436E.pdf

https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of-children/


http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf

6. Address the issue of non-readers in Years 1-2. Stakeholders should conduct classroom level assessments to identify non-readers in Years 1 and 2, diagnose the causes, and design specific activities to address deficiencies before children complete Year 2. Teachers in Years 1 and 2 have the greatest responsibility in ensuring children finish the first two grades of primary education with the necessary skills to read and understand text. In this sense, identification of struggling readers and knowledge of ways in which they need to be supported should make the core material of in-service training for Year 1 and Year 2 teachers. For instance, teachers may group students according to ability and provide remedial activities and appropriately levelled text. This “catch-up approach” is being used by UNICEF in Zambia based J-Pal’s research in India, which demonstrated that students grouped by ability is more effective than mixed-ability grouping. UNICEF will assess students in all grades and group them according to reading levels (non-readers, those who can read letter sounds, syllables, words, passages, etc.). Grouping students by level rather than grades has produced dramatic results in India, Kenya and Ghana. Teachers and school administrators should further determine whether non-readers have learning disabilities (e.g., dyslexia) and design relevant interventions for special needs students.

7. Teach students how to develop reading comprehension skills in the early grades. Given only 14 percent of students are able to comprehend 80 percent or more of grade level text, teachers should focus on increasing reading comprehension levels for all years. Research shows that teachers are often the ones to ask questions orally and that students are expected to respond, but rarely are students asked to develop their own questions as they read (think aloud) or to make predictions and then check their predictions. Students need to be empowered to make meaning from text and to use an inquiry-based approach to reading. Teachers could ask students to make predictions before, during and after reading, to retell stories, or to identify problems within the story and provide possible solutions. Teachers should use various strategies and frameworks to develop students’ reading comprehension skills from as early as kindergarten or Year 17.

8. Develop activities that specifically focus on raising boys’ performance and interest in reading. The results illustrated that boys consistently performed lower than girls. There may be cultural or gender barriers that affect boys’ interest and engagement in reading activities. In addition to designing strategies to address low competencies of boys and girls, stakeholders at all levels should discuss the

7 Sample reading comprehension activities can be found in the following guides: Ontario Ministry of Education. (2003). A Guide to Effective Instruction in Reading: Kindergarten to Grade 3 (Available at: http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf) Institute of Education Sciences (IES). (2010). Improving Reading Comprehension in Kindergarten through 3rd Grade. What Works Clearninghouse (Available at: https://education.ohio.gov/getattachment/Topics/Early-Learning/Third-Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading-Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx)

https://education.ohio.gov/getattachment/Topics/Early-Learning/Third-Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading-Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx



http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf

potential challenges specific to boys and design strategies to improve boys’ reading achievement. Successful strategies that have worked in other countries include developing gender-sensitive materials that attract boys’ attention (such as sports, science fiction, fantasy, comic books, digital text, and stories that are humorous) and integrating reading into extracurricular activities (e.g., sports, health clubs, student government).

Chapter 1: IntroductionKiribati is a country consisting of 33 coral atolls in the Pacific Ocean located about halfway between Hawaii and Australia. It is one of the smallest and most remote countries in the world with a population of 106,925 (July 2016 est.)8. Education is free and compulsory from age 6 to 14. Primary education consists of Years 1 to 6.9 Among the total official school age population (6-14 years old), 96percent were enrolled in primary school in 201410. Including those over and under-age, the gross enrollment rate was 113percent11.

Despite high enrollments and alarmingly low literacy rates throughout the region, the country has performed well on regional literacy assessments. According to the 2015 Pacific Islands Literacy and Numeracy Assessment (PILNA), 93 percent of Year 4 students are meeting or exceeding the expected level for literacy and 72 percent are performing at or above the expected level in Year 6. This is a marked improvement from 2012 where 66 percent were meeting or exceeding the benchmark in Year 4 and 43 percent in Year 6.

The government of Kiribati has instituted a new curriculum and implemented curricular reforms under the Kiribati Education Improvement Program (KEIP). The overarching goal of the KEIP is that, by 2020, all Kiribati children achieve functional literacy and numeracy after six years of basic education12. In order to achieve this goal, learning how to read in the early grades is critical.

The Pacific Early Age and Readiness Program (PEARL) was established to improve the school readiness and literacy outcomes of young children throughout the Pacific region. The World Bank, in collaboration with the Kiribati government and local education stakeholders, are collaborating in this joint effort to undertake an Early Grade Reading Assessment and to design effective interventions to improve school readiness and reading development in Kiribati. The Early Grade Reading Assessment (EGRA) measures basic reading skills in the first three grades of primary school. The EGRA tool has been administered in over 100 languages in more than 60 countries around the world, including the Pacific islands of Tonga, Vanuatu, Papua New Guinea and Timor Leste.

The EGRA tool was adapted to the Kiribati context and administered nationally from September 21 to October 27, 2016. The Kiribati EGRA (KiEGRA) aimed to achieve three objectives:

8 CIA World Factbook. (2016). Kiribati population. Available at https://www.cia.gov/library/publications/the-world-factbook/geos/kr.html9 In the Kiribati education system, grade levels are called years. Year 1 is a synonym for Grade 1. 10 World Bank. (2014). Kiribati gross enrollment ratio, primary, both sexes. Available at http://data.worldbank.org/indicator/SE.PRM.ENRR11 Ibid.12 Kiribati Education Improvement Program (KIEP) Evaluation Report. (2014). Commissioned through the Australian government Education Resource Facility.

1. To develop survey of basic reading skills and temporary reference standards to monitor reading performance in schools and system wide;

2. To build local capacity to replicate early grade reading assessments in the future; and

3. To work with local education stakeholders to interpret EGRA findings and analyze their policy and sector investment implications.

The overall purpose of the KiEGRA is to inform education stakeholders of reading skills students are struggling with and to identify factors that appear to contribute to reading development in Kiribati. The findings will help policymakers to design responsive strategies to improve early grade reading instruction and assessment. In particular, the EGRA results may assist Kiribati policymakers with building the capacity of teachers, allocating budgetary and human resources, and with developing reading indicators and benchmarks. The ultimate goal of the KiEGRA is to ensure that all children develop the skills needed to become fluent readers.

Towards this aim, the assessment was designed to respond to the following questions:1. What are the basic reading skills of students in Years 1, 2, and 3 in Kiribati?2. What is the year in which students in Kiribati “break into” reading fluently with

comprehension?3. How does student performance in Years 1, 2, and 3 compare to curriculum

expectations in Kiribati language for these grades?4. What are the main skills students struggle with? What are the main skills students

have strengthened?5. What are the differences in performance between boys and girls?6. What are the student and teacher factors associated with strong and weak early

grade reading outcomes?a. What are teacher expectations around when students should develop key

basic reading skills? How different are these from curricular expectations in the early grades?

b. What are the resources available in classrooms to support the reading development of children?

c. What are the pedagogical practices supporting reading that are prevalent in the early grades in Kiribati classrooms?

The report is divided into six chapters beginning with this introductory section in Chapter 1. Chapter 2 will discuss the KiEGRA survey design and implementation. Chapter 3 presents overall results of the reading assessment per sub-test, gender and year as well as a summary of the overall strengths and weaknesses responding to questions 1, 4 and 5. The correlation between oral reading fluency and comprehension (question #2) is discussed in Chapter 4. As there are no established standards for reading fluency, the report used 80 percent reading comprehension benchmark as the basis for determining fluency scores. Chapter 5 provides an analysis of student and teacher factors associated with improved reading outcomes, which corresponds with question #6 (6a, 6b, and 6c). Finally, the conclusions and policy recommendations to improve literacy outcomes are presented in Chapter 6.

Chapter 2: Instrument Design and Implementation2.1. KiEGRA Survey ImplementationIn order to build local capacity to replicate early grade reading assessments, Kiribati Ministry of Education staff were directly involved in the EGRA design and implementation. From August 31 to November 1, 2016, the World Bank provided in-country support to selected Ministry staff to undertake all aspects of the assessment including, sampling, adapting the KiEGRA instruments into the i-kiribati language, facilitating the training of enumerators and supervisors, coordinating survey logistics during the pilot and fieldwork, and administering the EGRA using Tangerine software.

2.2. Sample DesignWith guidance from the World Bank, the Kiribati MOE chose a nationally representative sample for students in Years 1, 2, and 3. The target population was defined as students enrolled in Years 1 to 3 in primary schools implementing the official curriculum in Kiribati. A sample of 49 schools was selected based on enrollment data from the Kiribati Education Management Information System (KEMIS). To ensure all regions had an equal probability of being selected despite their actual distribution in the country, a stratified random design with proportional allocation was applied. The final sample consisted of 1,363 students, 691 girls and 672 boys (see Table 1).

Table 1: EGRA Sample by Region, Year and Gender

Region Year 1 Year 2 Year 3 TotalGirl Boy Total Girl Boy Total Girl Boy TotalCentral 48 55 103 40 35 75 30 31 61 239Linnux 29 30 59 32 25 57 31 27 58 174North 85 68 153 72 75 147 73 65 138 438South 32 29 61 32 41 73 29 33 62 196South Tarawa

54 53 107 54 52 106 50 53 103 316

Total 248 235 484 230 228 458 213 209 422 1363

All statistical analysis perform take the complex survey design into consideration. For instance, all standard errors were adjusted using the Taylor series variance estimates adjusted for the design effects resulting from the multi-stage cluster sampling. Sampling weights were also computed to correct for the unequal probability selection resulting from the multi-stage cluster sampling. Thus all results presented are weighted results.

2.3. Development of the KiEGRA Instrument Due to differences in language, culture, and expectations about learning outcomes, the EGRA tool was adapted to the Kiribati context and piloted. From August 31 to

September 7, 2016, a team of language experts, curriculum officers and teachers attended an adaptation workshop focused on learning the research foundations and structure of the EGRA tool in order to the develop the KiEGRA instrument in the Kiribati language. Before the workshop, a language analysis was carried out to gain a good understanding of what orthography and language issues needed to be taken into account when developing the EGRA tools. Also a letter frequency lists, word frequency list and possible non-word list were developed. The letter names and letter sound grid had been drafted prior to the workshop. During the workshop, participants worked in small groups to develop a number of potential listening and reading texts with comprehension questions. Participants also translated the instruments into Kiribati. Two versions of the instruments were field-tested on September 5,2016. The results from the pre-test were shared and feedback from workshop participants and ministry staff were incorporated into the instrument. The revised instrument was piloted on September 7, 2016, and a few minor changes were applied to some of the comprehension questions.

2.4. Enumerator trainingThe Enumerator Training Workshops for KiEGRA were held in Tarawa on September 26 to 30, 2016, and on Christmas Island on October 26 to 29, 2016. Pre-selected enumerators and supervisors attended the training workshop to learn about the scope and purpose of the survey and the manipulation of tablets for data collection. Assessors were trained three days in a classroom environment, and they had the chance to administer a practice EGRA in a nearby primary school, where the instruments had been previously piloted. An Assessor Accuracy Measurement13 (AAM) was conducted three times in order to familiarize the enumerators with the process and examine their accuracy level. Results were used to select 16 enumerators and 4 supervisors. All candidates' results were above 90 percent overall.

2.5. Data collection Data collection took place in Kiribati between October 4 and November 9, 2016. Over a period of six weeks, teams of enumerators and supervisors visited sample schools in the country. Three teams based in Tarawa visited schools in the South, Central and North districts, and one team focused on schools in Linnux district. The data was uploaded at least once a week, and checked by the survey coordinators.

13 The AAM evaluates the degree of agreement among enumerators administering the same test at the same time to the same student. This type of measurement determines the trainees’ ability to accurately administer the EGRA. A script prepared beforehand, complete with deliberate errors, becomes the gold standard. After a practice exercise, the enumerators’ responses are compared against the gold standard. The benchmark is set at 90% agreement between the gold standard and the enumerator’s responses. Therefore, if the enumerators’ responses are in agreement with at least 90% of the responses on the gold standard, the assessor is considered qualified to administer the EGRA. The results are used to address any weaknesses identified during the enumerator training.

2.6. Reliability of the InstrumentCronbach’s alpha was computed for every task of the EGRA instrument to estimate the level of reliability of each sub-test. Results showed strong internal consistency for almost every sub-test except Reading Comprehension and Listening Comprehension. As a rule of thumb, an alpha coefficient of 0.80 is considered good while a 0.70 is the minimum acceptable result. The lower values of alpha coefficients for Reading and Listening comprehension could be explained by a smaller number of items (five) in those sub-tests since the number of items in a sub-test influences the value of the Cronbach’s alpha.

Table 2: Reliability of the EGRA Sub-tests

Sub-tests Number of items Number of respondent

Alpha

Letter Name Knowledge 100 1,363 0.981Initial Sound Identification 10 1,363 0.811Letter Sound Knowledge 100 1,363 0.962Familiar Word Reading 50 1,363 0.983Non Word reading 50 1,363 0.968Oral reading 60 1,363 0.988Reading Comprehension 5 1,363 0.757Listening Comprehension 5 1,363 0.606Dictation 31 1,165 0.942

Correlations between tasks are presented in Annex.

Chapter 3: EGRA ResultsThis section reports the results of the EGRA per sub-test. There were eight sub-tests comprised of the following:

1. Letter Name Knowledge2. Letter Sound Knowledge3. Initial Sound Identification4. Familiar Word Reading5. Nonword Reading6. Oral Passage Reading7. Reading Comprehension8. Listening Comprehension9. Dictation

The average scores of the above sub-tests are provided and disaggregated by year and gender. Mean scores are presented for the entire sample as well as for the sample that was able to read at least one item (minus zero scores). The overall results show the actual reading performance of the entire population, including those who can and cannot read. The results minus zero scores show the average score of the population who can read at least one item, which addresses research concerns that the inclusion of a large percentage of zero-scores could underestimate the result of the population who can read.

Results disaggregated by region are also provided at the end of each section to highlight any regional differences in average reading performance.

3.1. Sub-test 1 – Letter Name Knowledge14

The test of letter name knowledge is the most basic assessment of reading skills. It measures students’ ability to read the names of the letters of the alphabet with accuracy. Automaticity and fluency of letter name knowledge is a predictive skill for later reading success. During the EGRA, students were given a page of 100 randomly distributed upper- and lowercase letters and asked to say the names of as many letters as possible within one minute. The test included the full Kiribati alphabet and randomization was used to prevent students from reciting a memorized alphabet. The test is scored by the number of letters that students correctly name in one minute (correct letters per minute—clpm).

Table 3 shows overall scores for sub-test 1 by year, gender and minus zero scores. Overall, students were able to identify 37 letters in one minute. Excluding the zero scores, the results increase slightly to 39. Students identified 28 letters in Year 1, 44 letters in Year 2 and 41 in Year 3. The greatest improvement in letter recognition was in Year 2, where students identified 16 additional letters. Students in Year 3 scored four points lower than students in Year 2. However, assessors observed that Year 3 students were beginning to provide letter names in English rather than Kiribati. Overall, girls

14 The sub-test descriptions are based on the Early Grade Reading Assessment Toolkit (RTI, 2009).

scored higher than boys, identifying 40.3 letters compared to 34.1 for boys. Some girls exceeded the maximum score of 100 by nine points (109).

Table 3: Sub-test 1 Letter Name Knowledge: Results by Year and Gender

Sub-test 1 – Number of correct letter names identified per minute (CLPM)

N Mean SD Min Max

CLPM – overall 1,363 37.2 23.2 0 109.1CLPM – minus zero score students 1,318 38.6 22.5 1 109.1

Year

Year 1CLPM – overall 483 27.8 20.2 0 102.4CLPM – minus zero score students 457 29.2 19.7 1 102.4

Year 2CLPM – overall 458 44.0 22.7 0 107.1

CLPM – minus zero score students 448 45.2 21.9 1 107.1

Year 3CLPM – overall 422 40.5 23.4 0 109.9CLPM – minus zero score students 413 41.9 22.6 1 109.1

Gender

GirlsCLPM – overall 691 40.3 24.1 0 109.9CLPM – minus zero score students 679 41.9 23.2 1 109.9

BoysCLPM – overall 672 34.1 21.8 0 100CLPM – minus zero score students 648 35.3 21.2 2 100

The highest-ranking district for letter name scores was the Southern region. The Central, Linnux and Northern regions performed similarly. The South Tarawa region had the lowest overall mean score.

Table 4: Letter Name Results by Region and Year

Central Linnux North South South Tarawa

Year 1 32.6 30.9 28.2 28.9 25.8 Year 2 48.8 44.2 48.6 52.7 40.6 Year 3 40.6 42.9 39.1 45.6 39.9 Overall 39.7 39.2 38.4 42.9 35.4

Figure 1: Letter Name Results by Region and Year

Central

Linnux

North

South

South Tarawa

0 10 20 30 40 50 60

Year 3Year 2Year 1

Number of correct letters per minute

3.2. Sub-test 2 – Letter Sound KnowledgeIn order to read fluently, students must understand the alphabetic principle (that words are made up of sounds) and that letters (e.g., graphemes) are symbols that represent sounds. Thus, knowledge of how letters correspond to sounds is a critical skill children must master to become successful readers. The letter sound knowledge test is administered similarly to the letter name knowledge subtest. Students were provided a page of 100 randomly distributed upper- and lowercase letters of the Kiribati alphabet and asked to provide the sounds (not the names) of as many letters as they could identify within a one-minute period.

The results of this sub-test by year, gender and with and without zero scores are shown in Table 5. The overall results for the letter sound sub-test indicate that students had more difficulty with identifying letter sounds than letter names. Students were able to identify 27 letter sounds on average (10 fewer than letter names). Year 1 students accurately identified 22 letter sounds. Students in Year 2 could name an additional eight letter sounds, for a total mean score of 30. Similar to the letter name sub-test, scores decreased by two points in Year 3, from 30 letters in Year 2 to 28 in Year 3. Girls read four more letter sounds than boys, but boys achieved a higher maximum score of 95 compared to 85 for girls.

Table 5: Sub-test 2 Letter Sound Identification: Results by Year and Gender

Sub-test 2 – Number of correct letter sounds identified per minute (CSPM)

N Mean SD Min Max

CSPM – overall 1,361 26.5 16.9 0 95.1CSPM – minus zero score students 1,286 28.3 16.1 1 95.1

YearYear 1 CSPM – overall 482 22.4 15.2 0 77

CSPM – minus zero score students 443 24.4 14.3 1 77

Year 2CSPM – overall 457 29.7 16.7 0 84CSPM – minus zero score students 436 31.2 15.7 1 84

Year 3CSPM – overall 422 27.7 18.2 0 95.1CSPM – minus zero score students 407 29.2 17.4 1 95.1

Gender

GirlsCSPM – overall 689 28.8 17.3 0 85CSPM – minus zero score students 657 30.4 16.3 1 85

BoysCSPM – overall 672 24.2 16.4 0 95.1CSPM – minus zero score students 657 30.4 16.3 1 85

Student performance on the letter sound sub-test was related to the type of the letter presented. The percentage of correct answers for each of the letters tested showed that students have a good knowledge of vowel sounds but are unable to correctly identify the sounds of consonants. Sample students were able to correctly identify vowel sounds in over 80 percent of the cases. Meanwhile, performance decreased to 8 percent - 31 percent for consonants, and was particularly low for the letters w, k, t, r and b. Nasal consonants (n, m, ng) were easier to pronounce and were correctly identified in over 70 percent of the cases presented.

The Central region scored 3-5 points higher than other regions. Linnux performed the lowest averaging 25 correct letter sounds per minute (See Table 6).

Table 6: Letter Sound Identification: Results by Region and Year



Figure 2: Letter Sound Identification: Results by Region and Year

Central

Linnux

North

South

South Tarawa

0 5 10 15 20 25 30 35 40

Year 3Year 2Year 1

Number of correct letters per minute

3.3. Sub-test 3 – Initial Sound IdentificationInitial sound identification assesses phonemic awareness (the ability to hear, identify and manipulate individual sounds –phonemes- in spoken words). In this test, the assessor read aloud a word and then asked the student to identify the first sound, or phoneme, in the word. The test is comprised of 10 words and is administered orally. The test is untimed and is measured according to the number of items answered correctly.

Overall, students could identify 30 percent of initial sounds attempted (3 out of 10 items). Excluding the zero score students, the average score increases to four. Identifying initial sounds was a particularly difficult task for all students. Students identified an average of 2.3 initial sounds in Year 1, with a slight increase to 3.7 in Year 2, and practically no change (4) in Year 3. There was no difference in performance for boys and girls.

Table 7: Sub-test 3 Initial Sound Identification: Results by Year and Gender

Sub-test 3 – Number of correct initial sounds identified

N Mean SD Min Max

Overall 1,363 3.3 2.5 0 10Minus zero score students 1.033 4.3 2.0 1 10

Year

Year 1Overall 483 2.3 2.4 0 10Minus zero score students 279 3.6 2.0 1 10



Gender

GirlsOverall 691 3.5 2.6 0 10Minus zero score students 541 4.5 2.1 1 10

BoysOverall 672 3.0 2.4 0 9Minus zero score students 492 4.1 1.8 1 9

Based on the results by region in Table 8, there was very little variation in the performance of students across regions. Linnux demonstrated the highest scores on initial letter sound while the Central and Southern regions scored the lowest.

Table 8: Initial Sound Recognition: Results by Region and Year



Figure 3: Initial Sound Recognition: Results by Region and Year

Central

Linnux

North

South

South Tarawa

0 1 2 3 4 5 6

Year 3Year 2Year 1

Number of correct sounds

3.4. Sub-test 4 – Familiar Word ReadingThe familiar word sub-test measures students’ ability to read familiar words with fluency and accuracy, both of which are necessary to become fluent readers. For the familiar-word reading test, students were given a list of 50 familiar words (selected from readers available in the classroom) with instructions to read as many as they could in one minute. Familiar-word reading is a timed test scored by the number of correct words read per minute (cfwpm).

Table 9 shows the results of the familiar word reading sub-test by year, gender and with and without zero scores. The overall mean score was 23 correct words per minute and 27 excluding zero scores. The test was most difficult for Year 1 students who could only read an average of 8.6 words correctly per minute. Performance improves by 15 points in Year 2, in which students read an average of 23.7 words per minute, and another 13 points in Year 3 where the overall mean score was 37 words per minute. While these scores might be too low to lead to reading comprehension, there is steady progress towards reading fluency. Girls scored eight points above boys (27 compared to 19 words per minute for boys). The maximum score for girls was nearly 20 points higher than boys (98 compared to 78.9 respectively).

Table 9: Sub-test 4 Familiar Word Reading: Results by Year and Gender

Sub-test 4 – Number of correct familiar words read per minute (CFWPM)

N Mean SD Min Max

CFWPM – overall 1,362 22.7 20.9 0 98CFWPM – minus zero score students 1,171 26.7 20.1 1 98

Year

Year 1CFWPM – overall 482 8.6 11.9 0 88.2CFWPM – minus zero score students 338 12.4 12.6 1 88.2

Year 2CFWPM – overall 458 23.7 17.8 0 81.7CFWPM – minus zero score students 428 25.3 17.2 1 81.7

Year 3CFWPM – overall 422 37.0 21.4 0 98CFWPM – minus zero score students 405 39.8 19.5 1 98

Gender

GirlsCFWPM – overall 690 26.6 23.2 0 98CFWPM – minus zero score students 615 30.4 22.4 1 98

BoysCFWPM – overall 672 18.6 17.3 0 78.9CFWPM – minus zero score students 556 22.6 16.5 1 78.9

On the familiar word reading sub-test, regional averages were highest in the Linnux region and lowest in the Central region (See Table 10).

Table 10: Familiar Word Reading: Results by Region and Year (without zero scores)


Year 1 10.9 13.6 8.7 5.2 7.2 Year 2 23.2 30.8 23.6 26.6 22 Year 3 35.9 41.8 38.9 44.1 34.8 Overall 20.9 28.6 23.2 25.5 21.2

Figure 4: Familiar Word Reading: Results by Region and Year (without zero scores)

Central

Linnux

North

South

South Tarawa

0 5 10 15 20 25 30 35 40 45 50

Year 3Year 2Year 1

Number of correct words per minute

3.5. Sub-test 5 – Nonword ReadingTo be able to comprehend unfamiliar text, children must be able to decode new unfamiliar words. Nonword reading measures students’ ability to decode words that follow the linguistic rules of the Kiribati language but that do not exist in real life. Unlike familiar word reading in which students can read from memory or sight recognition, on the nonword reading sub-test students must sound out the words; thus, it is a purer measure of decoding ability. Students were provided with a table of 50 made-up words and instructed to read as many as they could within one-minute. The test is timed and is measured by the number of correct nonwords read per minute (cwpm).

Overall, students read an average of 13 nonwords correctly in one minute (See Table 11). When the zero scores are removed, the mean score jumps to 20 correct words per minute. The difference in these scores indicates that there is a significant proportion on non-readers on this sub-test, especially in Year 1. Year 1 readers could read an average of 13 nonwords, but when zero scores are included the mean score drops to 5 correct nonwords per minute (8 point difference). There was a 3-4 point difference between readers and non-readers in Years 2 and 3; thus, the percentage of students able to decode nonwords increases with each level. Overall mean scores do indicate progress from Years 1-3, with performance improving from 5.2 correct words in Year 1 to 14 correct in Year 2 and 22 correct in Year 3. On average, girls read five more words than boys (15.9 and 10.9, respectively), which is also evident in the higher maximum score for girls (49 versus 45).

Table 11: Sub-test 5 Nonword Reading: Results by Year and Gender

Sub-test 5 – Number of correct nonwords read per minute (CUWPM)

N Mean SD Min Max

CNWPM – overall 1,362 13.4 12.8 0 49CNWPM – minus zero score students 951 19.9 10.7 1 49

Year

Year 1CNWPM – overall 482 5.2 8.7 0 47CNWPM – minus zero score students 209 12.8 9.3 1 47



Gender

GirlsCNWPM – overall 690 15.9 13.8 0 49CNWPM – minus zero score students 512 21.7 11.7 1 49

BoysCNWPM – overall 672 10.9 11.2 0 45CNWPM – minus zero score students 439 17.8 8.9 1 45

Based on Table 12 results by region, Linnux region scored the highest with a mean score of 19.1 correct nonwords per minute and the Central region had the lowest average score (11.5 correct nonwords per minute).

Table 12: Nonword Reading: Results by Region and Year (without zero scores)


Year 1 6.1 10.8 5 2.4 4.2 Year 2 13.2 20.3 13.8 16.1 12.4 Year 3 19.1 26.4 23.3 25.1 20.5 Overall 11.5 19.1 13.7 14.7 12.3

Figure 5: Nonword Reading: Results by Region and Year (without zero scores)

Central

Linnux

North

South

South Tarawa

0 5 10 15 20 25 30

Year 3Year 2Year 1

Number of correct nonwords per minute

3.6. Sub-test 6 – Oral Passage ReadingThe best single measure of a child’s reading proficiency in the primary grades is oral reading fluency (ORF). Oral reading fluency measures students’ ability to read a short passage with sufficient speed and accuracy. It encompasses all of the previous reading skills plus the skills needed for comprehension -- the ability to translate letters into sounds, unify sounds into words, process connections, relate text to meaning, and make inferences.15 ORF has been shown to be a powerful predictor of overall reading competence and comprehension16.

The oral passage reading sub-test is a timed test. In order for students to understand a simple passage, they must be able to read it fast enough to retain the words in short-term memory. Research17 suggests a minimum fluency rate of 45-60 words per minute, depending on the complexity of the language. In this sub-test, students were asked to read a very short story comprised of 60 words in one minute. After one minute, the assessor stopped students and recorded the number of words read correctly (cwpm). If the child could not read any words correctly in the first line, the assessor stopped the test early and the child received a score of zero.

The overall mean score for this sub-test was 29 correct words per minute, well below the 45-60 cwpm standard. Among those who could read at least one word (excluding zero scores), students read 32.4 words per minute (See Table 13). While the overall mean

15 Hasbrouck, J., & Tindal, G. A. (2006). “Oral reading fluency norms: A valuable assessment tool for reading teachers.” The Reading Teacher, 59(7), 636–644. 16 Ibid17 Abadzi, H. (2011). Reading Fluency Measurements in EFA FTI Countries: Outcomes and Improvement Prospects. Education for all Fast Track Initiative

score was quite low, the bulk of the low scores were in Year 1, where students read an average of 10 correct words per minute. Students improved significantly as they progressed to Year 2, reading 29 words per minute, and Year 3, where students read an impressive 47 words per minute. Students read an additional 18-19 words each year. Girls read 11 more words than boys (34 compared to 23. The maximum score for girls were 129 words per minute while boys read a maximum of 103 words per minute.

Table 13: Sub-test 6a Oral Passage Reading: Results by Year and Gender

Sub-test 6a – Number of correct words read per minute (CWPM)

N Mean SD Min Max

CWPM – overall 1,358 28.6 25.4 0 128.6CWPM – minus zero score students 1,199 32.4 24.6 1 128.6

Year

Year 1CWPM – overall 478 10.9 13.6 0 100CWPM – minus zero score students 361 14.2 14.0 1 100

Year 2CWPM – overall 458 29.2 20.3 0 94.7CWPM – minus zero score students 428 31.2 19.5 1 94.7

Year 3CWPM – overall 422 47.2 26.4 0 128.6CWPM – minus zero score students 410 49.6 24.8 1 128.6

Gender

GirlsCWPM – overall 689 34.0 28.2 0 128.6CWPM – minus zero score students 629 37.5 27.3 1 128.6

BoysCWPM – overall 669 23.1 20.7 0 102.9CWPM – minus zero score students 570 26.8 19.9 1 102.9

Average results by region illustrated in Table 14 indicate that the Linnux region had the highest fluency rates (35.7 cwpm) while the Central region (26 cwpm) and South Tarawa region (27 cwpm) had the lowest.

Table 14: Oral Passage Reading: Results by Region and Year


Year 1 13.5 16.7 10.5 7.4 9.7 Year 2 30.1 38 29.6 33.2 26.7 Year 3 42.4 53 50.7 55.2 44.6 Overall 25.9 35.7 29.6 32.1 26.9

Figure 6: Oral Passage Reading: Results by Region and Year

Central

Linnux

North

South

South Tarawa

0 10 20 30 40 50 60

Year 3Year 2Year 1

Number of correct words per minute

3.7. Sub-test 7 – Reading ComprehensionThe reading comprehension subtask measures the ability to answer several comprehension questions based on the passage read. There are a total of five questions for this sub-test, consisting of direct, fact-based questions and at least one question requiring inference from the passage read. Students were asked questions only up to the point where they had stopped reading. For instance, if the child read the first sentence (6 words), s/he was asked one question. If s/he read half of the text (27 words), s/he was asked three questions; and, if s/he read all five sentences (54 words), s/he was asked all five comprehension questions. Similarly, if learners did not read any of the text, they were not asked any questions and received a score of zero. This sub-test is scored by the number of questions answered correctly. The scores are also calculated to determine the percentage of questions answered correctly out of a total possible (five).

The overall results are shown in Table 15. On average, students could accurately respond to 1.3 comprehension questions. Readers (excluding zero scores) comprehended the double the text of non-readers achieving 50 percent correct (2.5 out of 5 questions asked). The highest percentage of zero scores were in Year 1, where the overall mean score was 0.2. Students showed significant gains from Years 1-3. Overall mean scores improved by 30 percent from Year 1 to Year 2 with mean scores increasing from 0.2 to 1.5. From Year 2 to Year 3, scores increased nearly 20 percent from 1.5 to 2.4. Among readers (minus zero scores), scores increased an average of 11 percent each year.

Table 15: Sub-test 6b Reading Comprehension: Results by Year and Gender

Sub-test 6b – Number of questions answered correctly

N Mean SD Min Max

Number of correct answer – overall 1,363 1.3 1.5 0 5Number of correct answer – minus zero score students

757 2.5 1.2 1 5

Year

Year 1

Number of correct answer – overall 483 0.2 0.7 0 5

Number of correct answer – minus zero score students

78 1.8 1.1 1 5

Year 2



310 2.3 1.2 1 5

Year 3



369 2.9 1.2 1 5

Gender

GirlsNumber of correct answer – overall 691 1.6 1.6 0 5Number of correct answer – minus zero score students

416 2.7 1.2 1 5

BoysNumber of correct answer – overall 672 1.1 1.4 0 5Number of correct answer – minus zero score students

341 2.3 1.2 1 5

Figure 7 below shows the percentage of correct answers by Year. About 85 percent of students in Year 1 showed zero percent comprehension of the passage read. Only one percent of Year 1 students could comprehend 60 percent or more of the text. The results for Year 2 are better as the majority of the students comprehended some of the text and nearly 30 percent scored 60 percent or above. Year 3 students had more distribution across the higher percentage categories. Half of Year 3 students could comprehend 60 percent or more of the text and 27 percent could comprehend at least 80 percent of the text. Overall, the majority of students in Years 1 and 2, and half in Year 3 have reading comprehension levels below 60-80 percent.

Figure 7: Reading Comprehension Results: Distribution of percentage of correct answers by year

0% 20% 40% 60% 80% 100%0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Class1 Class2 Class3

Reading comprehension (% correct responses)

%st

ud

ents

Although girls read 11 more words in the passage than boys, the point difference between boys and girls’ reading comprehension scores was trivial. Table 15 shows that girls on average correctly responded to 0.5 more questions than boys (1.6 versus 1.1, respectively). However, the distribution of responses by percentage of correct answers illustrated in Figure 8 shows that girls had a higher percentage of correct answers in most categories and a lower percentage of zero scores compared to boys.

Figure 8: Reading Comprehension Results: Distribution of percentage of correct answers by gender

0% 20% 40% 60% 80% 100%0%

10%

20%

30%

40%

50%

60%

Girls Boys

Reading Comprehension (% correct responses)

%st

ud

ents

The average results by year and gender according to the type of question asked and the order in which it was presented is provided in Table 16. All five questions were direct factual questions; thus the answers were embedded in the text. The easiest question was Question 1, in which 38 percent of students accurately responded. Questions 2-4 had fairly equal levels of difficulty with correct answers averaging 30 percent. Question 5 was the most difficult with only five percent of students who attempted the question able to answer correctly.

Table 16: Correct Answers by Questions in sub-test 6b

Questions Year 1 Year 2 Year 3 Total

Girl Boy Total Girl Boy Total Girl Boy Total Girl Boy Total

1. Who went fishing with Tooma?

12% 6% 9% 53% 41% 47% 67% 53% 60% 43% 33% 38%

2. Where did Tooma and his father put their fishing line?

8% 6% 7% 43% 35% 39% 56% 48% 52% 35% 29% 32%

3. What kind of fish was Tooma’s catch?

6% 3% 4% 33% 22% 28% 68% 47% 57% 34% 23% 29%

4. How did Tooma feel when he hooked the catch?

6% 1% 3% 34% 26% 30% 72% 38% 55% 36% 21% 29%

5. How did Tooma’s catch get away?

1% 0% 1% 5% 1% 3% 16% 10% 13% 7% 3% 5%

The percentage and number of questions students answered correctly based on those attempted are presented in Table 17. The first column shows number and percent correct for the entire sample. Columns 0-5 show the number of questions that students attempted and scored correctly. The results indicate that the more text students read, the more they were able to comprehend. This is evident in the decreasing percentage of zero scores. Of students who did not comprehend any of the text (196), 100 percent scored zero. Meanwhile, of students who read enough to be asked two questions, nearly 50 percent scored zero. Only 7 percent of all students attempted and correctly responded to four of the questions, and 2.4 percent attempted and correctly responded to five questions.

Table 17: Distribution of Correct Answers

Number of Questions Correct

Total correct Number of questions attempted

# 0 1 2 3 4 50 606

44%196

100%184

93.4%195

48.5%13

14.6%13

4.9%5

2.4%1 195

14%13

6.6%123

30.6%19

21.4%26

9.7%14

6.6%2 198

14%84

20.9%33

37.1%52

19.5%29

13.7%3 181

13%24

26.9%94

35.2%63

29.7%4 144

10%82

30.7%62

29.3%5 39

2%39

18.4%Total (N) 1,363 196 197 402 89 267 212

Scores were similarly low on this sub-test across all regions and years. The Linnux region performed slightly better than the rest with a mean comprehension score of 1.7 and the Central and South Tarawa regions had the lowest comprehension scores (1.2).

Table 18: Reading Comprehension: Results by Region and Year



Figure 9: Reading Comprehension: Results by Region and Year

Central

Linnux

North

South

South Tarawa

0 0.5 1 1.5 2 2.5 3 3.5

Year 3Year 2Year 1

Number of questions answered correctly

3.8. Sub-test 8 – Listening ComprehensionThe purpose of the listening comprehension assessment is to measure whether the student can listen to a short passage being read aloud and then answer several questions correctly with a word or a simple statement. Poor performance on a listening comprehension tool would suggest that children simply do not have the basic vocabulary that the reading materials expect, or that they have difficulty processing what they hear. The assessor read a short fable to students and then asked five comprehension questions. Students had 15 seconds to respond to each question. As this was an untimed test, all students heard the entire story and responded to all five questions. For this reason, scores are based on the percentage of questions answered correctly.

The average mean score for listening comprehension was 46 percent correct and 52 percent without zero scores (See Table 19). The results indicate progressive improvement in listening comprehension levels by at least 6-7 percent with each Year. By the end of Year 1, students could comprehend about 40 percent of the passage read aloud. Year 2 students understood 47 percent and Year 3 students comprehended 53 percent. Boys and girls had relatively similar comprehension skills (46 percent for girls and 47 percent for boys).

Table 19: Sub-test 7 Listening Comprehension: Results by Year and Gender

Sub-test 7 – Percentage of overall listening comprehension

N Mean SD Min Max

Percentage of correct answer – overall 1,363 46.2% 28.2 0 100Percentage of correct answer – minus zero score students

1,253 51.9% 24.5 20 100

Year

Year 1Percentage of correct answer – overall 483 39.1% 27.5 0 100Percentage of correct answer – minus zero score students

422 46.7% 23.4 20 100


426 51.6% 23.1 20 100


405 57.4% 25.7 20 100

Gender

GirlsPercentage of correct answer – overall 691 45.8% 27.9 0 100Percentage of correct answer – minus zero score students

633 51.0% 24.5 20 100

BoysPercentage of correct answer – overall 672 46.7% 28.5 0 100Percentage of correct answer – minus zero score students

620 52.8% 24.4 20 100

Table 20 shows average performance per question type. All questions were fact-based questions (answers can be found within the story). While all questions were of similar level of difficulty, students performed highest on questions 1 and 3, and lowest on questions 4 and 5. It should be noted that students only heard the passage read aloud once.

Table 20: Listening Comprehension Percentage of Correct Answers by Question

Questions Year 1 Year 2 Year 3 Total

Girl Boy Total Girl Boy Total Girl Boy Total Girl Boy Total

1. Who went with her father to get some fresh air?

67% 72% 70% 82% 80% 81% 87% 84% 86% 79% 79% 79%

2. Where was a bird perched?

39% 41% 40% 49% 49% 49% 48% 47% 47% 45% 46% 45%

3. Why did Anna climb the tree?

51% 63% 57% 65% 61% 63% 71% 72% 71% 62% 65% 64%

4. What was Anna warned by her father about birds?

14% 19% 16% 27% 20% 23% 32% 36% 34% 24% 25% 24%

5. Why then was Anna happy?

13% 12% 13% 21% 16% 18% 26% 31% 28% 20% 19% 20%

In terms of regional performance, the highest scores were in the Linnux region where students correctly responded to 3.3 questions and the lowest performance was in South Tarawa where students accurately responded to 2 questions.

Table 21: Listening Comprehension Results by Region and Year


Year 1 2.2 2.8 2 1.6 1.7Year 2 2.2 3.4 2.6 2.2 2.1 Year 3 3.1 3.7 2.7 2.8 2.4 Overall 2.4 3.3 2.4 2.2 2.0

Figure 10: Listening Comprehension Results by Region and Year

Central

Linnux

North

South

South Tarawa

0 0.5 1 1.5 2 2.5 3 3.5 4

Year 3Year 2Year 1

Number of questions answered correctly

3.9. Sub-test 9 – DictationThe dictation sub-test measures students’ alphabet knowledge and ability to hear and distinguish individual letter sounds in words and to spell words correctly. For this sub-test, the assessor read aloud a short sentence of 15 words and asked students to write down what they had heard. The assessor read the sentence three times, once before students began writing and twice while they were writing. The test was untimed and scored by the percentage of items written correctly.

The dictation sub-test assessed four basic writing skills: spelling, spacing, capitalization and punctuation (use of full stop). The dictation sentence contained 10 words, consisting of a compound sentence that included a compound noun comprised of three morphemes. In pilot testing, the sentence was found to provide a good discrimination of scores between grades. The scores were calculated using weights to arrive at a maximum score of 100 percent. The spelling items received a weight of 60 percent while other components (spacing, capitalization, comma and full stop) received a weight of 40 percent. Given the focus of the EGRA is to assess reading ability, the spelling component received more weight.

Students performed exceptionally well on this writing task. The maximum score was 100 percent for all three classes. The average score for the entire sample was 71 percent and 72 percent excluding zero scores. Students’ writing skills progressively increased from 51 percent in Year 1 to 76 percent in Year 2 to finally 82 percent in Year 3. Girls scored six percentage points higher than boys (74 percent compared to 68 percent).

Table 22: Sub-test 9 Dictation: Results by Year and Gender

Sub-test 9 – Percentage of overall early writing skills)

N Mean Percent Correct

SD Min Max

Average percent of correct answer – overall

1,165 70.5% 26.4% 0% 100%

Average percent of correct answer – minus zero score students

1,153 71.7% 25.0% 2.1% 100%

Year

Year 1


332 50.5% 27.5% 0% 100%


323 52.8% 25.9% 2.1% 100%

Year 2


425 76.2% 20.4% 0% 100%


424 76.3% 20.2% 4.2% 100%

Year 3


408 82.1% 20.8% 0% 100%


406 82.8% 19.3% 4.2% 100%

Gender

Girls


608 74.2% 25.7% 0% 100%


602 75.6% 23.8% 2.1% 100%

Boys


557 66.6% 26.7% 0% 100%


551 67.6% 25.6% 4.2% 100%

As presented in Figure 11 and mentioned above, the dictation test measured spelling, spacing, capitalization and full stop. Students scored highest on spelling. The greatest improvements were with use of the full stop between Years 1 and 2 (an increase of 26 percent). However, Year 3 students struggled the most with use of the full stop scoring zero correct. Girls scored an average of 12 percent higher than boys across all writing categories (Figure 12).

Figure 11: Dictation Results: Percentage of Correct Answers by Item and Year

Spelling Spacing Capital Full stop0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Year 1 Year 2 Year 3

Writing skills

Perc

enta

ge o

f cor

rect

ans

wer

s

Figure 12: Dictation Results: Distribution of correct answers by gender

Spelling Spacing Capital Full stop0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Girls Boys

Regional results for the dictation sub-test are presented in Table 23. Students fared well across all regions with average scores between 71 percent and 77 percent correct. The Southern region scored the highest with 78 percent correct. The lowest scoring region was South Tarawa where students achieved 68 percent correct.

Table 23: Dictation: Results by Region and Year (without zero scores)


Year 1 59.5 46.4 56.3 52.9 48 Year 2 83.1 80.8 74.4 80.1 74.2 Year 3 85.5 85.6 85.3 89 78.9Overall 74.5 71.4 74.2 77.7 67.8

Figure 13: Dictation: Results by Region and Year (without zero scores)

Central

Linnux

North

South

South Tarawa

0 10 20 30 40 50 60 70 80 90 100

Year 3Year 2Year 1

Dictation precent correct score

3.10.Summary of assessment resultsThe overall EGRA results show that there is significant progress from Years 1 to 3 and that students have basic reading and writing skills. Students possess good letter recognition, familiar word reading and oral passage reading skills. There was measurable progress between Years 1-3 across all sub-tests, especially on the familiar word and oral reading fluency sub-tests. Scores on the familiar word subtest increased by 15 cwpm between Years 1-2 (from 9 to 24 words per minute), and an additional 13 between Years 2-3 to an average of 37 correct words per minute. On the oral reading passage sub-test, students’ scores improved by 18 points between Years 1 and 2 (from 10.9 to 29) and Years 2 and 3 (from 29 to 47). Thus, students in Year 3 are on the cusp of becoming fluent readers. However, students are still struggling with identifying letter sounds and decoding unfamiliar words. Listening comprehension skills were higher than reading comprehension for Years 1 and 3, but at similar levels for Year 3.

The reading comprehension sub-test had the highest percentage of zero scores across all years as well as the lowest mean scores (See Figures 16 and 17). The highest percentage of zero scores was in Year 1, where 84 percent of students could not accurately respond to one comprehension question. Students did, however, show significant improvement as they progressed throughout the levels. By Year 3, students read an average of 47 correct words per minute and could accurately respond to 47 percent of the comprehension questions asked (See Figures 14 and 15). With regards to gender, there were more female readers than male readers, and girls read equal to or better than boys in all reading skills tested. The Linnux region scored highest on six of eight reading sub-tests and lowest on Letter Sound Knowledge. Conversely, the Central Region scored lowest across five sub-tests and highest (along with the Southern region) on Letter Sound Knowledge.

Figure 14: Summary Results: Number correct answers for timed sub-tests by year

Letter

Nam

e Knowled

ge

Letter

Sound Id

entif

icatio

n

Famili

ar W

ord R

eadin

g

Nonword R

eading

Oral P

assa

ge R

eadin

g0

5

10

15

20

25

30

35

40

45

50

Year 1Year 2Year 3

Nu

mb

er o

f cor

rect

an

swer

s p

er m

inu

te

Figure 15: Summary Results: Percent correct for untimed sub-tests by year

Reading Comprehension Listening Comprehension0%

10%

20%

30%

40%

50%

60%

Year 1Year 2Year 3

Per

cen

tage

of c

orre

ct a

nsw

er

Figure 16: Summary Results: Percentage of Zero Scores by Sub-test and Year

Letter N

ame

Initi

al Sound

Letter S

ounds

Familia

r Word

Nonword

Oral P

assage R

eading

Reading C

ompre

hension

Listenin

g Com

prehensio

n

Dictatio

n0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Perc

ent o

f Zer

o Sc

ores

Figure 17: Summary Results: Percentage of Zero Scores by Sub-test and Gender

Letter Name

Initial Sound

Letter Sounds

Familiar Word

Nonword

Oral Passage Reading

Reading Comprehension

Listening Comprehension

Dictation

0% 10% 20% 30% 40% 50% 60%

Boys Girls

Chapter 4 – Performance in reading fluency and reading comprehensionAccording to the second edition of the EGRA toolkit18, there are two steps to identifying a reading fluency benchmark. The first is to identify the level of reading comprehension that is expected for the grade level. In most countries, this level is set at 80 percent or higher (4 correct answer out of 5 questions) for Years 1-3. This threshold will be used for the purposes of this EGRA study. Once the reading comprehension benchmark is set, the second step is to use EGRA data to show the range of ORF scores obtained by students able to achieve the desired level of comprehension. Because students may be able to achieve the 80 percent comprehension benchmark at different fluency rates, and there could be a wide range of scores, we have added a third step to the analysis. The third step is to identify the range of scores with the highest proportion of students meeting the comprehension benchmark. With this information, then stakeholders may decide on the value within the fluency range that should be put forward as the reading fluency benchmark.

Table 24 shows the actual distribution of percent correct scores for reading comprehension. Overall, 11 percent of students scored 80 percent correct and 3 percent achieved 100 percent. Thus, the total percentage of students achieving 80 percent or more reading comprehension is 14 percent.

Table 24: Distribution of percent correct scores for reading comprehension by sub-test per year

Comp. Score Year 1 Year 2 Year 3 Overall Mean0% 84% 32% 13% 44%20% 8% 21% 14% 14%40% 5% 19% 21% 15%60% 1% 15% 26% 13%80% 1% 11% 20% 11%100% 1% 1% 7% 3%Total 100% 100% 100% 100%

As mentioned above, as a first step, we decided to set the level of comprehension at 80 percent as an indicator that students demonstrate full understanding of the text. Table 25 shows the percentage of students by year and gender that comprehended 80 percent or more of the text read. The distribution of scores by Year is: 2 percent in Year 1, 12 percent in Year 2 and 27 percent for Year 3 students. In terms of gender, 16 percent of female students and 10 percent of male students reached the 80 percent benchmark in reading comprehension.

18 RTI International. (2016). Early Grade Reading Assessment (EGRA) Toolkit, Second Edition. Washington, DC: United States Agency for International Development.

Table 25: Percent of students reading with 80% or more comprehension by Year

80% or more reading comprehensionMean (%) SD (%) N

Overall 14% 34.1% 1363Year 1 2% 13.6% 483Year 2 13% 33.3% 458Year 3 27% 44.7% 422Female 16% 37.1% 691Male 10% 30.3% 672

For the second step, we need to look at the distribution of fluency scores of students who reached the threshold level in reading comprehension. Figure 18 shows the distribution of ORF scores for each level of reading comprehension. Globally, we can see that an increase in the level of reading comprehension is associated with an increase in ORF scores. Table 26 confirms that mean scores in ORF were higher when students had higher levels of reading comprehension. However, it also shows that some students who did not reach the 80 percent reading comprehension benchmark achieved higher ORF scores than students who did reach the benchmark. For example, students who have 80 percent in reading comprehension could have a fluency score as low as 25 correct words per minute. Meanwhile, some students who were unable to answer a single reading comprehension question correctly could read up to 6519 correct words per minute.

19 Note that ORF scores may exceed the total number of words in a passage since it is a timed test. If students could read the entire passage within the 60 seconds allowed, the fluency score would take into account the time remaining from the 60 seconds. This explains why some students present such high value as 129 correct words per minute although the passage had only 60 words.

Figure 18: ORF distribution by reading comprehension level

Table 26: ORF score by reading comprehension score

Reading Comp.Score

Mean Fluency Score

SD Min Max Sample (n)

0% 11.9 8.8 1 65 44220% 29.5 17.4 5 109 19540% 39.5 18.7 9 129 19860% 53.3 15.0 16 100 18180% 59.9 16.6 25 120 144100% 72.5 15.4 46 129 39

Distribution of ORF scores for students who reach 80 percent in reading comprehensionFigure 19 shows the distribution of ORF scores for the 183 students who reached the level of 80 percent correct in reading comprehension. The majority of the students who met the benchmark read between 40 and 69 correct words per minute. However, scores ranged between 40 and 129 words per minute. Scores above 60 indicate that students finished the paragraph in less than one minute while scores below 60 imply that students who did not finish the paragraph were able to infer enough to correctly respond to all questions.

Figure 19: Distribution of ORF scores for students reading with at least 80% comprehension

The histogram in Figure 20 compares the distribution of ORF scores for students who did and did not meet the reading comprehension benchmark. As illustrated, the number of students who reached the benchmark is quite small compared to those who did not and there is overlap between the distributions of the two groups. The benchmark set for reading fluency should ensure that a maximum number of students who did reach the 80 percent level in reading comprehension would be identified as fluent readers but it should also ensure not to include too many students who did not reach 80 percent in reading comprehension. This means that setting the benchmark value for fluency at 45 correct words per minute could include too many students who didn’t reach the 80 percent reading comprehension level while setting it at 75 correct words per minute would exclude too many students who reached the benchmark.

Figure 20: Distribution of ORF scores for students meeting and not meeting 80% benchmark

Table 27 shows that the fluency rate of 60 cwpm has a greater proportion of students meeting rather than not meeting the benchmark. Policymakers may decide on an acceptable range (e.g., 60-64 cwpm) that can be considered proficient or acceptable. If the benchmark for reading comprehension were lowered to 60 percent, then the fluency scores would also be lower (closer to 50 cwpm).

Table 27: Distribution of ORF Scores for students meeting and not meeting the 80% benchmark

Correct number of words per minute (ORF)

% meeting 80% RC benchmark

% not meeting 80% RC benchmark

45-49 36.7% 63.3%50-54 27.9% 72.1%55-59 43.1% 56.9%60-64 57.4% 42.6%65-69 47.1% 52.9%70-74 36.8% 63.2%75 and more 53.4% 46.6%

In summary, greater oral reading fluency is associated with higher levels of reading comprehension with only 14 percent of students who are identified as being able to comprehend 80 percent of what they read. More females than males reached that level of comprehension. ORF scores for students achieving the benchmark shows great variability. Stakeholders should discuss the number of correct words per minute that could qualify a student as a fluent reader. The decision should be based on the distribution of scores for students who reached the reading comprehension benchmark.

Chapter 5- Analysis of Student and Teacher Factors Associated with Better Reading Outcomes This section reports the results of regression analyses conducted to explore the effect of student and teacher factors (independent variables) on oral reading fluency scores (dependent variable). The factors are organized into seven categories: 1) student demographics; 2) teacher characteristics; 3) teacher expectations; 4) classroom environment; 5) teacher training and guides; 6) reading materials; and, 7) teacher instructional and assessment methods. The results of the analyses directly respond to research question #7.

5.1. Association of student characteristics to student reading outcomes

General background characteristics and reading activities were collected in the student questionnaire. The factors as shown in Table 28 include whether the student attended preschool, ate breakfast before arriving to school, language spoken at home, parents’ literacy, whether students receive help with homework, availability of reading materials and whether students read or are read to at home. Over 90 percent of students surveyed attended pre-school, speak Kiribati at home, eat breakfast before arriving to school and state that they like to read. Eighty-five percent of students read at home. About half of students have reading materials in Kiribati.

Table 28: Student background characteristics

Student Characteristics % of cases SE* NStudent attends preschool before Year 1 94% 0.7% 1336Student speaks Kiribati at home 96% 0.9% 1361Student eats before arriving to school 92% 0.9% 1360Student has someone who can read at home 82% 0.01% 1361Student receives help with homework from the mother

25% 1.8% 1361

Student receives help with homework from the father

17% 1.5% 1361

Student receives help with homework from the sibling

15.4% 1.1% 1361

Student receives help with homework from any other person

5.5% 0.9% 1361

Someone asks student about what he/she did in school

75% 1.3% 1316

Student tells someone at home when he/she gets good marks

77% 1.5 % 1344

Student has books, newspapers or other things to read at home

59% 2.2% 1335

Student has books, newspapers or other things to 48% 1.3% 1361

read at home in KiribatiStudent has books, newspapers or other things to read at home in English

14% 0.9% 1361

Someone reads to student at home 59% 1.4% 1361Student reads aloud to someone at home 42% 2.6% 1361Student reads to himself/herself at home 85% 1.0% 1361Student reads on a computer or mobile device at home

69% 2.3% 731

Student likes to read 96% 0.6% 1348*SE = Standard Error20

The two factors that had the greatest positive impact on ORF scores (with an increase of 7 to 10 correct words per minute) were: whether students speak Kiribati at home and whether the student likes to read. Two additional factors that resulted in an increase of about 5 correct words per minute were: whether students received help with their homework from the mother and whether students read to himself at home.

Parents’ literacy presented a positive association with ORF scores. When both parents are literate, students read an additional 8.15 words per minute. Students who had a literate father read almost twice as many words as those with a literate mother (3.86 more words compared to 1.99 more words). However, students who received help with their homework from their mother read 5 more words and the difference was statistically significant while the help received from their father or sibling was not significant.

Table 29: Association of student characteristics to Oral Reading Fluency (ORF) scores

Student Characteristics Change in ORF Score

(+/-)

SE Rsqr

Student attends preschool before Year 1 4.37 3.82 0.40Student eats breakfast before arriving to school -1.11 2.92 0.39Student speaks Kiribati at home 10.38* 2.26 0.39Student has a literate mother 1.99 2.24 0.39Student has a literate father 3.86 2.03 0.39Student has both parents literate 8.15* 3.78 0.40Student has a literate sibling 1.10 1..47 0.39Student receives help with homework from the mother

5.07* 2.36 0.40

Student receives help with homework from father

2.31 1.71 0.39

Student receives help with homework from sibling

-2.53 2.16 0.39


1.65 1.83 0.39

Student tells someone at home when he/she 0.21 1.45 0.39

20 SE is a measure of uncertainty caused by sampling error. It is used to build a confidence interval around the sample mean

gets good marksStudent has books, newspapers or other things to read at home

0.47 1.31 0.39

Student has books, newspapers or other things to read at home in Kiribati

1.60 0.93 0.39

Student has books, newspapers or other things to read at home in English

3.84 2.25 0.39

Student reads aloud to someone at home -3.29 1.65 0.39Student reads to himself/herself at home 4.93* 2.25 0.40Someone reads to student at home -1.79 1.12 0.39Student reads on a computer or mobile device at home

-0.94 1.25 0.39

Student likes to read 7.64* 2.86 0.40

5.2. Association of teacher characteristics to student reading performance

The association between teachers’ characteristics and student reading performance on the ORF sub-test was explored using data collected in the teacher interview questionnaire. Teacher profiles are presented in Table 30. Teachers in Kiribati have an average of 15 years of experience. Half of their experience seems to be on the island they are currently teaching. Nearly all primary school teachers (90 percent) are female and hold a primary school teaching certificate. Seventy-seven percent of teachers have a reading corner in the classroom and 74 percent have met with their students’ parents. Finally, less than half of teachers have not been absent from school in the last term.

Table 30: Profile of teachers in EGRA

Teacher demographics Mean (sd) NAge of the teacher 43.4 (9.5) 132Number of years of teaching experience 15.4 (9.1) 132Number of years spent teaching on the island 7.4 (7.7) 132Number of minutes from home to school 8.8 (11.9) 132

Teacher characteristics % of cases NFemale 90% 132Has a primary teaching certificate 98% 132Has a reading corner in the classroom 77% 130Has not been absent from school in the last term 43% 131Has met with parents of his/her students 74% 132

In order to identify the teacher characteristics associated with better student reading outcomes, separate regression analysis was conducted for each teacher characteristic (See Table 31). The dependent variable was the average number of words read per minute in the oral reading passage (scores from sub-test 6a) and the independent variables were the teacher characteristics described above (Table 30). Gender and year level were considered in the regression analysis to control for these two variables.

The results in Table 31 indicate that no factor had a statistically significant association with ORF scores. Whether teachers held a primary school teaching certificate or had a reading corner showed the greatest effect but those two factors are not statistically significant.Table 31: Association of teachers' characteristics to Oral Reading Fluency Score (ORF)

Teacher Characteristics Change in ORF Score

(+/-)

SE Rsqr

Has a primary teaching certificate 2.88 6.78 0.74Has a reading corner in the classroom 3.41 1.94 0.74Has not been absent from school in the last term

0.81 1.66 0.74

Has met with the parents of his/her students 1.04 1.74 0.74Age of the teacher -0.08 0.09 0.74Number of year of experience in teaching -0.01 0.10 0.74Number of year of experience in teaching on this island

0.03 0.11 0.74

Number of minute from home to school -0.11 0.07 0.74

5.3. Association of teacher expectations to student reading performance

Teachers were asked to report what grades they expected students to meet certain reading competencies (e.g., recognize letters, understand stories read aloud). Table 32 provides results of regression analyses that show the relationship of five indicators of teacher expectations on student oral reading fluency scores (number of words read correctly per minute). Each regression analysis included gender and year as control variables.

The reference level for each regression is the median expectation of all teachers for each indicator. The dash (-) represents the expected year for that competency to be reached. For example, Year 1 is the reference category in the first regression because the median expectation of teachers for students to recognize and say letter names was by the end of Year 1. The positive coefficient of 8.48 in Year 2 indicates that students whose teachers expected them to recognize and say letter names by Year 2, a year later than the median expected time, read 8.48 more words relative to students whose teachers expected them to achieve the competency by Year 1. This pattern was also observed in the second regression. Most teachers expected their students to understand stories read aloud by the teacher by the end of Year 2. Teachers who expected their students to achieve this competency by the end of Year 3 read 3.74 more words per minute, and for those with expectations for Year 4 students, the ORF score increased by 1.82 words per minute.

When teachers had higher expectations than the median teachers, reading performance decreased. This was observed for the second indicator. Students were expected to read aloud to the teacher by the end of Year 2. Teachers who expected students to achieve these competencies by the end of Year 1 saw a decrease of 2.72 correct words per minute.

Table 32: Association of teacher expectations to oral reading fluency (ORF) scores

Teacher expectations of student reading performance

Change in ORF score

(+/-)

SE Rsqr

Grade students should recognize and say letter names

Year 1 - - 0.77Year 2 8.48* 1.96Year 3 2.19 2.40Year 4 3.91 4.12

Grade students should understand stories read aloud by the teacher in the classroom

Year 1 -2.72 2.01 0.75Year 2 - -Year 3 3.74 2.10Year 4 1.82 3.29

Grade students should sound out words using phonics

Year 1 -0.56 1.72 0.78Year 2 - -Year 3 6.43* 2.14Year 4 14.25* 4.13

Grade students should read aloud to teacher and other students

Year 1 -3.61 2.72 0.76Year 2 -0.47 2.08Year 3 - -Year 4 5.86* 2.40

Grade students should understand simple texts that they read in class

Year 1 -4.40 2.60 0.75Year 2 1.09 1.99Year 3 - -Year 4 4.00 2.69

Based on the results in Table 32, students whose teachers expected them to reach the reading competency later than expected consistently read more words per minute than students who met the competency as expected. The coefficient was statistically significant when teacher expected students to recognize and say letter names in Year 2. In this situation, students were able to read 8 more correct words per minute. It is also the case for when students should sound out words using phonics or when students should read aloud to teacher and other students. Teachers who expected students to sound out words using phonics in Year 4 read 14 more cwpm, and teachers who expected students to read aloud to teacher and students in Year 4 read 5.86 more cwpm. However, teachers who expected students to reach the competency sooner than the expected grade level actually read fewer correct words per minute. When teachers expected students to read

aloud in Year 1, students read 3.6 fewer cwpm. Furthermore, when teachers expected students to understand simple texts they read in class in Year 1, they read 4.4 fewer cwpm.

5.4. Association of teacher training and guides to student reading performance

We also sought to explore the relationship between teaching materials and student reading performance. Teachers were asked whether they had a teaching syllabus and whether they had received any training on how to teach reading in the last three years. As shown in Table 33, over 60 percent of teachers had a syllabus. Eighty-nine percent of teachers reported having a teacher guide. Finally, about half of teachers have received training on reading instruction.

Table 33: Teacher training and guides

% of cases NTeacher has a syllabus 64% 132Teacher has a teacher guide 89% 132Teacher have receive training on how to teach reading in the last three years

56% 132

Results of the regression analysis for teaching resources and its relation to student reading performance, measured by a change in ORF score are presented in Table 34. Whether the teacher had a syllabus or a teacher guide did not have any effect on ORF scores. Training also did not show statistically significant association with ORF scores.

Table 34: Association of training and guides to student oral reading fluency (ORF) scores

Change in ORF score

SE Rsqr

Teacher have a syllabus -1.72 2.29 0.74Teacher has a teacher guide -2.91 2.63 0.74Teacher have receive training on how to teach reading in the last three years

1.85 1.63 0.74

5.5. Association of Classroom Environment to Student Performance

Through classroom observations, we collected data on the classroom environment and types of resources available in the classroom. Assessors recorded whether they observed the following classroom displays: the alphabet, a calendar, days of the week, numbers and the weather. Classrooms observed had an average of 3.58 classroom displays. The observation also sought to determine the types of printed materials used in instruction, such as newspapers, magazines, flashcards, food wrappers and packaging, prepaid cards, objects in treasure boxes and any other materials. There was an average of 1.10 printed material used in the classroom.

Table 35: Average Number of Classroom Displays and Materials Observed

Classroom environment Mean (sd) SE NClassroom displays 3.58 0.10 155Print materials used in instruction 1.10 0.09 155

As shown in Table 36, 67 percent of classrooms have spelling/vocabulary words displayed, 15 percent have songs/hymns/stories written on the blackboard and 54 percent have them written on charts or posters, and 69 percent have student work displayed.

Additional factors related to the classroom environment that were observed included whether there was sufficient space for organizing group activities, whether there was a reading corner in the classroom, whether teachers maintained folders with students’ work (student profiles) and the seating arrangement. The results showed that 75 percent of classrooms have a reading corner, 81 percent have sufficient space for organizing group work, 68 percent have student profiles and 77 percent of classroom students sit on the floor.

Table 36: Frequency and Type of Classroom Displays/Resources Available

Types of classroom displays/resources available % of classes SE NSpelling/vocabulary displayed 67% 0.04 155Song/hymns/stories displayed on blackboard 15% 0.03 155Song/hymns/stories displayed on charts/posters 54% 0.04 155Student work displayed 69% 0.04 155Sufficient classroom space for organized group activities

81% 0.03 155

Reading corner in the classroom 75% 0.03 155Student profiles (folder with student work and student info)

68% 0.04 155

Students sitting on floor 77% 0.04 155

Among the different classroom environment variables, the presence of student profiles had the most positive relation with ORF scores. When teachers maintained student profiles, the students read an average of 6.37 more words per minute. This relation is statistically significant. On the other hand, having student work displayed was adversely related to ORF scores. Classrooms where student work was displayed had lower overall ORF scores.

Table 37: Association of Classroom Environment to Student ORF Scores

Classroom Environment Change in ORF Score

(+/-)

SE Rsqr

Classroom displays 0.38 0.54 0.75Spelling/vocabulary displayed 1.10 1.56 0.75Song/hymns/stories displayed on blackboard 3.51 2.04 0.75Song/hymns/stories displayed on charts/posters 0.08 1.50 0.75Student work displayed -3.28* 1.55 0.76

Print materials used in instruction 0.88 0.63 0.75Sufficient classroom space for organized group activities

-2.23 1.89 0.75

Reading corner in the classroom 2.92 1.71 0.76Student profiles (folder with student work and student info)

6.37* 1.53 0.78

Students sitting on floor 0.75 1.76

5.6. Association of Reading Materials to Student PerformanceThe classroom observation further aimed to identify teaching and learning aids used for reading instruction. The instrument included 13 different reading instructional materials that could be observed in the classroom (Instructional readers, big books, posters/charts with poems, posters/charts with songs, posters/charts with phonics, A5 letter cards, small cards, mathematics charts, community studies posters, environmental science posters, healthy living charts, climate change story book, and any other reading materials). The assessors reported a mean of 8.26 different instructional resources used in the classrooms. Additionally, there was an average of 1.97 reading books observed during the lesson.

Table 38:Average Number of Reading Materials and Books in Classroom

Reading Materials Mean (sd) SE NReading instructional materials in classroom 8.26 (2.80) 0.23 152Number of reading books used during the lesson 1.97 (17.5) 1.44 148

According to the regression results in Table 39, the relation between the number of reading books used during the lesson and ORF scores of students is statistically significant. When there are more books used in the classroom, the ORF scores were higher.

Table 39: Association of Reading Materials to Student ORF Scores

Change in ORF Score

(+/-)

SE Rsqr

Reading materials in classroom 0.43 0.26 0.74Number of reading books used during the lesson

0.11* 0.04 0.74

5.7. Association of Teacher Instructional and Assessment Methods to Student Performance

The final set of regression analysis observed the relationship between instructional and assessment methods and student performance in ORF. Table 41 presents results of regressions analysis of the frequency in which students and teachers performed seventeen instructional and assessment methods within the course of the week. The dependent variable is the mean ORF score of the students and the independent variables are the instructional methods. The mean ORF score of students who were never exposed to an instructional or assessment method was compared to the mean ORF score of students

who were exposed 1-2 days, 3-4 days or 5 days a week. Results from those regression analyses were analyzed to determine if being exposed to a specific method is associated with increased or decreased ORF scores and if so, whether the effect is the same for all rates of exposure (e.g., 1-2 days, daily, etc.). All regression models include gender and Year for controlling for those two characteristics.

Table 40 shows the teaching methods used during reading instruction and the frequency in which it is applied. The instructional methods used most frequently were teaching the meaning of new words/vocabulary and listening comprehension. The most common classroom activities were shared reading, reading comprehension activities, drawing or writing sentences. Less than 10% of teachers evaluate students’ oral reading fluency on a regular basis (3-5 days a week).

Table 40: Frequency of methods used during reading instruction

Never 1 – 2 days

3 – 4 days

5 days N

Teaching of Listening Comprehension

5% 30% 19% 46% 132

Children Practice Letter Name 20% 11% 9% 61% 132Children orally retell a story that they have read

18% 42% 25% 15% 132

Children learn new letter sounds 18% 17% 15% 50% 132Children sound out unfamiliar words using knowledge of letter sounds

52% 23% 11% 14% 132

Children learning meanings of new words/vocabulary

11% 20% 33% 36% 132

Shared reading 7% 29% 39% 26% 132Group Guided reading 19% 48% 20% 13% 132Listening to a child read aloud 30% 45% 14% 11% 132Students readings on their own silently

27% 14% 7% 52% 132

Reading comprehension activities 12% 24% 30% 33% 132Children take books home to read with their parents

92% 4% 2% 2% 132

Evaluating student’s oral reading with running records or any other method

57% 36% 2% 5% 132

Teachers works on word building with students

22% 23% 20% 35% 132

Students read and draw 13% 20% 31% 36% 132Students working on spelling words in exercise books

20% 32% 20% 28% 132

Students writing sentences 10% 24% 23% 43% 132

Table 41: Association of teacher instructional and assessment methods to student performance

Teaching instructional and assessment methods

Change in ORF score

SE Rsqr

Teaching of Listening Comprehension 0.741-2 days 3.52 3.863-4 days 4.57 3.995 days 2.87 3.74

Teaching Letter Names 0.751-2 days -2.12 3.083-4 days -5.62 3.365 days -6.44* 2.50

Asking children to orally retell a story that they have read

0.74

1-2 days 0.74 2.293-4 days -1.97 2.525 days -0.85 2.82

Teaching new letter sounds 0.771-2 days -10.10* 2.683-4 days -10.16* 2.815 days -6.9* 2.36

Asking children to sound out unfamiliar words using knowledge of letter sounds

0.74

1-2 days -0.32 2.003-4 days 0.58 2.745 days -4.41 2.47

Teaching meaning of new vocabulary words

0.74

1-2 days 0.27 3.083-4 days -1.27 2.825 days -0.88 2.78

Shared reading 0.741-2 days 2.57 3.483-4 days 2.61 3.39Daily 3.01 3.59

Group Guided reading 0.741-2 days 5.20* 2.203-4 days 4.29 2.56Daily 2.75 2.92

Listening to a child read aloud 0.74

1-2 days 3.94 1.883-4 days -0.31 2.71Daily 3.55 2.83

Students readings on their own silently 0.741-2 days 2.35 2.653-4 days 4.36 3.46Daily 3.69 1.93

Reading comprehension activities 0.741-2 days 5.10 2.833-4 days 4.71 2.74Daily 2.62 2.70

Children take books home to read with their parents

0.74

1-2 days -1.27 4.293-4 days -3.32 6.69Daily 0.97 5.50


0.74

1-2 days 0.66 1.743-4 days -4.24 5.51Daily -1.57 3.76

Teacher works on word building with students

0.74

1-2 days -2.25 2.413-4 days -3.75 2.54Daily -3.36 2.30

Students read and draw 0.741-2 days -2.45 2.903-4 days -3.29 2.66Daily -5.48* 2.63

Students work on spelling words in exercise books

0.74

1-2 days 6.14* 2.253-4 days -0.01 2.48Daily 1.31 2.33

Students writing sentences 0.741-2 days -0.83 3.093-4 days 2.19 3.10Daily 0.14 2.87

Results of the regression analysis presented in Table 41 show that some instructional methods used in reading classroom are associated with higher levels of ORF scores. Reading comprehension activities were linked with higher scores between 2-5 words per minute. When students do group guided reading 1-2 days a week, they were able to read an average of 5.20 more words. Also, when students work on spelling words in exercise books 1-2 days a week, they were able to read an average of 6.14 more words.

On the other hand, several teaching methods show a negative association in students’ ORF scores. Students who learned new letters sounds 1-4 days a week read an average of 10 fewer words while students who were taught letter sounds on a daily basis read 6.9 fewer words. Additionally, when students practiced letter names, read or drew 1-5 days week, there were lower ORF scores. Evaluating students’ oral reading skills three or more days a week showed lower scores compared to evaluating once or twice a week. This could be because less time was spent on actual teaching.

Chapter 6 - Conclusions and Next StepsThe KiEGRA study shows that students are making significant gains in literacy between Years 1-3, but more improvement is needed for them to become fluent readers. Results indicate that while students have good letter recognition, familiar word reading and oral passage reading skills, they struggle with identifying letter sounds and decoding unfamiliar words. As a result, the majority of students are unable to read fluently with comprehension. The majority of students in Year 1 and one-third in Year 2 have zero reading comprehension skills. In Year 3, comprehension improves where nearly half of students could comprehend at least 60 percent of text. However, the majority of students in Years 1-3 are reading below the 80 percent comprehension benchmark. Overall, 14 percent met the benchmark. In Year 3, about 80 percent of students did not meet the benchmark.

An analysis of the relationship between oral reading fluency and reading comprehension scores to determine fluency ranges and percentage of students reaching the 80 percent reading comprehension benchmark found that greater oral reading fluency is associated with higher levels of reading comprehension. However, there is great variability in fluency ranges that fall within the 80 percent benchmark. Scores ranged from 25 to 129 words per minute. Most students meeting the benchmark read 40-69 correct words per minute. Only 14 percent of students across all years were able to comprehend 80 percent of what they read.

Across all sub-tests, girls tended to perform equal to or better than boys. More females than males met the 80 percent reading comprehension benchmark. Students in the Linnux region performed better than other regions.

The analysis identified a number of student characteristics, instructional methods, resources and teacher expectations that are associated with better reading outcomes. Students who speak Kiribati at home, have literate parents, and liked to read scored an average of 7-10 more words on the ORF sub-test. Teaching reading comprehension seemed to have a positive influence on students’ ORF scores. Guided reading was also found to be very important for increasing students’ fluency rates. When teachers maintained records of students’ work, ORF scores increased by 6 words per minute. Not surprisingly the availability of reading materials in the classroom or used in teaching instruction was positively related to ORF scores.

Interestingly, students tended to have higher ORF scores when teachers had low expectations for achievement of reading competencies. This could be because teachers provided more time for practice or reviewed lower-level skills with students. Teachers with high expectations yielded a small decrease in ORF scores. This could be because they spent less time reviewing lessons. In this analysis teacher expectations were measured according to whether they were higher or lower than the median expectation of teachers. In the future, teacher expectations should be measured against curricular standards.

Remarkably, teaching letter sounds and letter names and assessment methods were negatively related to student ORF scores. One reason could be if teachers are not closely following the teacher guide. Overemphasizing decoding skills or assessment and not utilizing results to inform instruction can hamper student performance. More research is necessary to identify the true cause behind these results.

Based on the findings and conclusions, the following recommendations are presented to improve the quality of early grade reading instruction in Kiribati schools:

1. Set reading fluency and comprehension benchmarks to guide and support teacher professional development to reach reading targets: Reading benchmarks bring attention to the learning expectations both students and teachers should be held accountable to support children in becoming readers. They serve a useful function when they are coupled with strategies that will strengthen the capacity of schools and teachers to help students reach them. Without benchmarks, there is no pressure in the system to achieve learning outcomes and no fear of failure. At the same time, simply setting standards and benchmarks without evidence that schools and teachers are failing in their job and the support mechanisms to improve, benchmarks will create fear and frustration among school administrators, teachers and parents. Reading benchmarks should be established to guide the development of basic reading skills in each of the early grades both in terms of oral reading fluency and comprehension; and to guide the mechanisms that are required for schools to succeed in achieving them. As a first step, stakeholders should decide on the level of comprehension required to understand grade level text (e.g., 60 percent, 80 percent) and then review the fluency scores that fall within that range. Tonga, for instance, has set the reading fluency benchmark at 50 cwpm and a reading comprehension of at least 75%. Students achieving these levels can be considered to have learned to read well, have the basic reading skills needed to develop their literacy skills, and have the ability to comprehend more complex text in upper grades. If stakeholders agree with the 80 percent benchmark, then an acceptable fluency range may be 60-64 cwpm. Given that the current mean ORF score is 29 cwpm, that range may seem too high. Thus, policymakers may decide to lower the benchmark to 60 percent and the fluency benchmark to 53 cwpm. Once the benchmark is decided, the next step is to consider the targets. Currently, 14 percent of students are meeting the 80 percent reading comprehension benchmark. How many should meet the benchmark in one year or five years? Once the targets are set up, schools and educators should be supported to achieve these benchmarks. The Kiribati MEYS should then conduct a national reading campaign to sensitize all stakeholders of the new benchmarks and then monitor and report progress towards achieving the targets at all levels (national/regional MEYS, school and communities).

2. Conduct a follow-up study on teaching practices and assessment methods to identify strengths and weaknesses. Although 56 percent of teachers were trained on reading instruction in the past three years, teaching decoding skills and

assessment were associated with lower ORF scores. It is important to understand why these interventions have an adverse relationship. It could be that teachers were not adequately supported to master the new teaching methodologies or that training is not aligned with best practices in reading instruction. Additionally, teachers may not understand how to accurately assess students and utilize the results for reflection and lesson planning. The analysis also showed that classrooms with teachers who maintained a record of students’ work had higher ORF scores. A follow-up study is recommended to investigate the strengths and weaknesses of current assessment and teaching methods vis-à-vis the results observed in different classrooms. It is also important to understand what types of information the teacher records and whether it is related to tracking student progress.

3. Strengthen teacher training for reading instruction with a focus on vocabulary, decoding skills and reading comprehension. Results indicate that students are weak in recognizing letter sounds and unfamiliar words, and have low reading comprehension skills. Hence, instruction in these areas should be strengthened to increase overall reading scores. In the case of Kiribati, the large shares of zero scores observed in various reading subtasks included in KiEGRA suggest inadequate instructional time in the classroom due to student absenteeism, school closures, or even inadequate use of instructional time -i.e. pedagogy or content-- during regular school days. Additionally, training should incorporate curriculum expectations and provide teachers with the specific methods, classroom activities and assessment methods required to achieve results.

4. Increase the number of books in the classroom library/reading corner and the use in teachers’ instruction. In Kiribati, 75 percent of classrooms have reading corners, but only 1.97 books are used during classroom instruction. One possible explanation is that there are not enough books in the classroom. The International Reading Association (IRA) recommends that classroom libraries start with at least seven books per child and purchase two additional new books per year. The optimal number of books in a classroom library is 300-600, depending on the grade level and number of copies21. The number of books teachers should expect children to read during the school year is 100-125 picture books by the end of Year 1 and 50-75 chapter books by the end of Year 2. There is strong evidence that reading skills grow and develop to the extent that children get actual practice reading. This is particularly difficult in countries with a modest literary tradition where the amount of titles in the local language may be limited. Recent innovations in technology now enable the production of reading materials in almost any language, provided there are guidelines for avid writers to follow. Educators in Kiribati have experience producing additional reading materials for children and their efforts should be further supported to ensure beginning readers have a reasonable variety of stories to practice and enjoy. With a corpus of reading large enough to provide sufficient reading practice, a low-cost option is to

21 Neuman, S. (undated). The importance of the classroom library. Available at: http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf

http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf

provide e-readers in low-resource countries where publishing may be expensive and there are high teacher-pupil ratios. E-readers allow students and teachers to choose from a variety of genres, it is portable so students can read from home or school, and its read aloud features provides additional support for emergent readers2223. In addition to provision of an increased number of hard and soft copy books, teachers should be trained on how to better integrate materials into their instruction and on how to develop attractive reading corners24.

5. Increase time-spent reading. The more time children spend reading the better and more fluent readers they become. In order to increase students’ reading fluency skills, teachers should ensure that students are spending sufficient time reading every day through teacher-led, parent-led or self-guided reading activities. Research recommends that children read between 20-40 minutes per day. The results also showed that bringing books home on a regular basis was associated with lower ORF scores. It is important to engage parents in reading activities to ensure that students and parents are spending the recommended time reading at home.

6. Address the issue of non-readers in Years 1-2. Stakeholders should conduct classroom level assessments to identify non-readers in Years 1 and 2, diagnose the causes, and design specific activities to address deficiencies before children complete Year 2. Teachers in Years 1 and 2 have the greatest responsibility in ensuring children finish the first two grades of primary education with the necessary skills to read and understand text. In this sense, identification of struggling readers and knowledge of ways in which they need to be supported should make the core material of in-service training for Year 1 and Year 2 teachers. For instance, teachers may group students according to ability and provide remedial activities and appropriately levelled text. This “catch-up approach” is being used by UNICEF in Zambia based J-Pal’s research in India, which demonstrated that students grouped by ability is more effective than mixed-ability grouping. UNICEF will assess students in all grades and group them according to reading levels (non-readers, those who can read letter sounds, syllables, words, passages, etc.). Grouping students by level rather than grades has produced dramatic results in India, Kenya and Ghana. Teachers and school administrators should further determine whether non-readers have learning disabilities (e.g., dyslexia) and design relevant interventions for special needs students.

7. Teach students how to develop reading comprehension skills in the early grades. Given only 14 percent of students are able to comprehend 80 percent or

22 Adams, A. & van der Gaag, J. (2011). First Step to Literacy: Getting Books in the Hands of Children Available at: https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of-children/ 23 UNESCO (2014). Reading in the mobile era: A study of mobile reading in developing countries. http://unesdoc.unesco.org/images/0022/002274/227436E.pdf24 Neuman, S. (undated). The importance of the classroom library.

http://unesdoc.unesco.org/images/0022/002274/227436E.pdf



more of grade level text, teachers should focus on increasing reading comprehension levels for all years. Research shows that teachers are often the ones to ask questions orally and that students are expected to respond, but rarely are students asked to develop their own questions as they read (think aloud) or to make predictions and then check their predictions. Students need to be empowered to make meaning from text and to use an inquiry-based approach to reading. Teachers could ask students to make predictions before, during and after reading, to retell stories, or to identify problems within the story and provide possible solutions. Teachers should use various strategies and frameworks to develop students’ reading comprehension skills from as early as kindergarten or Year 125.

8. Develop activities that specifically focus on raising boys’ performance and interest in reading. The results illustrated that boys consistently performed lower than girls. There may be cultural or gender barriers that affect boys’ interest and engagement in reading activities. In addition to designing strategies to address low competencies of boys and girls, stakeholders at all levels should discuss the potential challenges specific to boys and design strategies to improve boys’ reading achievement. Successful strategies that have worked in other countries include developing gender-sensitive materials that attract boys’ attention (such as sports, science fiction, fantasy, comic books, digital text, and stories that are humorous) and integrating reading into extracurricular activities (e.g., sports, health clubs, student government).

25 Sample reading comprehension activities can be found in the following guides: Ontario Ministry of Education. (2003). A Guide to Effective Instruction in Reading: Kindergarten to Grade 3 (Available at: http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf) Institute of Education Sciences (IES). (2010). Improving Reading Comprehension in Kindergarten through 3rd Grade. What Works Clearninghouse (Available at: https://education.ohio.gov/getattachment/Topics/Early-Learning/Third-Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading-Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx)




http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf

ANNEX 1 / TABLESAnnex 1.A: Reading skills testedAnnex 1.B: Correlations between tasksAnnex 1.C: 95 percent Confidence interval for EGRA TaskAnnex 1.D: Tuvalu Regression with Reading Comprehension as Outcome

Annex 1.A: Reading Skills Tested

Test Measured Timed

Letter Name Knowledge Ability to read alphabet letters with accuracy and fluency

Y

Letter Sound Knowledge Phonics – the ability to identify sounds of letters with accuracy and fluency

Y

Initial Sound Identification

Phonemic awareness – the ability to identify sounds in spoken words

N

Familiar Word Reading Ability to read familiar words with fluency and accuracy

Y

Nonword Reading Ability to decode linguistically sound invented words Y

Oral Passage Reading Ability to read a short passage with fluency and accuracy

Y

Reading Comprehension Ability to respond to several comprehension questions based on passage

N

Listening Comprehension Ability to comprehend a short story read aloud N

Dictation Alphabet knowledge, listening and writing skills N

Annex 1.B: Correlations between tasks

Letters Correct Per Minute

(LCPM)

Phonemic awareness

(PA)

Sounds Correct Per Minute

(SCPM)

Correct words Per Minute(CWPM)

Correct Invented Words

Per Minute(CIWPM)

Oral Reading Fluency(ORF)

Reading Comprehension

(RC)

Listening Comprehension

(LC)

Dictation(D)

LCPM 1

PA 0.483** 1

SCPM 0.688** 0.407** 1

CWPM 0.527** 0.572** 0.384** 1

CIWPM 0.477** 0.573** 0.355** 0.910** 1

ORF 0.508** 0.555** 0.359** 0.943** 0.896** 1

RC 0.455** 0.519** 0.294** 0.782** 0.735** 0.819** 1

LC 0.233** 0.331** 0.144** 0.333** 0.329** 0.306** 0.354** 1

D 0.427** 0.533** 0.306** 0.656** 0.636** 0.670** 0.607** 0.249** 1

Annex 1.C: 95 percent Confidence interval for EGRA Task

95 percent Confidence Interval for mean at Task 1: Letter Name Knowledge by Year and Gender

95% Confidence IntervalLower bound

Upper Bound

Overall 35.2 39.3Minus zero score students 36.7 40.6

Year 1 Overall 24.9 30.7Minus zero score students 26.4 32.1



Girls Overall 37.6 43.1Minus zero score students 39.5 44.3

Boys Overall 31.8 36.3Minus zero score students 33.2 37.4

95 percent Confidence Interval for mean at Task 2: Letter Sound Identification by Year and Gender


Upper Bound







95 percent Confidence Interval for mean at Task 3: Initial Sound Recognition by Year and Gender


Upper Bound







95 percent Confidence Interval for mean at Task 4: Familiar Word Reading by Year and Gender


Upper Bound







95 percent Confidence Interval for mean at Task 5: NonWord Reading by Year and Gender


Upper Bound







95 percent Confidence Interval for mean at Task6a: Oral Passage Reading by Year and Gender


Upper Bound







95 percent Confidence Interval for mean of number of items answered correctly at Task6b: Reading Comprehension by Year and Gender


Upper Bound







95 percent Confidence Interval for mean percentage of correct answer at Task7: Listening Comprehension by Year and Gender


Upper Bound







95 percent Confidence Interval for mean at Task8: Dictation by Year and Gender


Upper Bound

Overall 67.8% 73.2%Minus zero score students 69.2% 74.3%

Year 1 Overall 45.6% 55.4%Minus zero score students 48.3% 57.3%



Girls Overall 71.9% 76.5%Minus zero score students 73.7% 77.4%

Boys Overall 62.7% 70.5%Minus zero score students 63.5% 71.7%

Annex 1.D: Tuvalu – Regression Analysis with Reading Comprehension as Outcome

Table 42: Impact of student characteristics on Oral Reading Fluency (ORF) scores

Student Characteristics Change in RC score

(+/-)

SE Rsqr

Student attends preschool before Year 1 0.41* 0.10 0.36Student eats breakfast before arriving to school -0.16 0.16 0.34Student speaks Kiribati at home 0.71* 0.12 0.34Student has a literate mother 0.18 0.13 0.34Student receives help with homework from the mother

0.25* 0.10 0.34

Student receives help with homework from father

0.13 0.08 0.34

Student receives help with homework from sibling

-0.03 0.14 0.34


0.08 0.11 0.34

Student tells someone at home when he/she gets good marks

-0.05 0.08 0.34

Student has books, newspapers or other things to read at home

-0.03 0.06 0.34

Student has books, newspapers or other things to 0.05 0.05 0.34

read at home in KiribatiStudent has books, newspapers or other things to read at home in English

0.26* 0.12 0.34

Student reads aloud to someone at home -0.21* 0.09 0.34Student reads to himself/herself at home 0.38* 0.09 0.35Someone reads to student at home -0.11 0.08 0.34Student reads on a computer or mobile device at home

-0.07 0.07 0.34

Student likes to read 0.51* 0.15 0.35

Table 43: Effect of teachers' characteristics on Oral Reading Fluency Score (ORF)

Teacher Characteristics Change in RC Score

(+/-)

SE Rsqr

Has a primary teaching certificate 0.34 0.47 0.65Has a reading corner in the classroom 0.21 0.13 0.65Has not been absent from school in the last term

0.09 0.11 0.65

Has met with the parents of his/her students 0.10 0.12 0.65Age of the teacher -0.01 0.01 0.66Number of year of experience in teaching -0.01 0.01 0.65Number of year of experience in teaching on this island

0.001 0.01 0.65

Number of minute from home to school -0.01 0.01 0.65

Table 44: Effect of teacher expectations on oral reading fluency (ORF) scores

Teacher expectations of student reading performance

Change in RC score

(+/-)

SE Rsqr

Grade students should recognize and say letter names

Year 1 - - 0.68Year 2 0.39* 0.14Year 3 0.05 0.17Year 4 0.09 0.30

Grade students should understand stories read aloud by the teacher in the classroom

Year 1 -0.15 0.14 0.66Year 2 - -Year 3 -0.01 0.14Year 4 0.14 0.23

Grade students should sound out words using phonics

Year 1 -0.09 0.13 0.67

Year 2 - -Year 3 0.21 0.16Year 4 0.42 0.30

Grade students should read aloud to teacher and other students

Year 1 -0.19 0.19 0.66Year 2 0.06 0.15Year 3 -Year 4 0.19 0.17

Grade students should understand simple texts that they read in class

Year 1 -0.25 0.18 0.67Year 2 0.11 0.14Year 3 - -Year 4 -0.07 0.19

Table 45: Effect of training and guides on student oral reading fluency (ORF) scores

Change in RC score

SE Rsqr

Teacher have a syllabus 0.18 0.23 0.65Teacher has a teacher guide 0.03 0.18 0.65Teacher have receive training on how to teach reading in the last three years

0.31* 0.11 0.66

Table 46: Effect of Classroom Environment on Student ORF Scores

Classroom Environment Change in RC Score

(+/-)

SE Rsqr

Classroom displays 0.04 0.04 0.71Spelling/vocabulary displayed 0.01 0.06 0.71Song/hymns/stories displayed on blackboard 0.15 0.10 0.71Song/hymns/stories displayed on charts/posters 0.08 0.06 0.71Student work displayed 0.02 0.06 0.71Print materials used in instruction 0.02 0.13 0.71Sufficient classroom space for organized group activities

0.07 0.04 0.71

Reading corner in the classroom 0.21 0.11 0.71Student profiles (folder with student work and student info)

0.38* 0.10 0.73

Students sitting on floor 0.10 0.11 0.71

Table 47: Effect of Reading Instructional Resources on Student ORF Scores

Language use in classroom Change in RC Score (+/-)

SE Rsqr

Reading materials in classroom 0.03 0.01 0.73Number of reading books used during the lesson

0.01 0.002 0.73

Table 48: Effect of teacher instructional and assessment methods on student performance

Teaching instructional and assessment methods

Change in RC score

SE Rsqr

Teaching of Listening Comprehension 0.661-2 days 0.07 0.273-4 days 0.22 0.285 days 0.10 0.26

Teaching Letter Names 0.661-2 days 0.18 0.223-4 days -0.04 0.245 days -0.10 0.18

Asking children to orally retell a story that they have read

0.67

1-2 days -0.04 0.153-4 days -0.36* 0.17*5 days -0.23 0.19

Teaching new letter sounds 0.671-2 days -0.44* 0.19*3-4 days -0.46* 0.20*5 days -0.24 0.16

Asking children to sound out unfamiliar words using knowledge of letter sounds

0.67

1-2 days -0.05 0.143-4 days 0.30 0.195 days -0.18 0.17

Teaching meaning of new vocabulary words

0.66

1-2 days -0.12 0.213-4 days -0.24 0.195 days -0.17 0.19

Shared reading 0.671-2 days 0.11 0.243-4 days 0.27 0.23

Daily 0.36 0.24

Group Guided reading 0.671-2 days 0.28 0.153-4 days 0.16 0.18Daily -0.0001 0.20

Listening to a child read aloud 0.671-2 days 0.20 0.133-4 days 0.44* 0.19Daily 0.19 0.20

Students readings on their own silently

0.66

1-2 days -0.05 0.193-4 days 0.09 0.24Daily 0.04 0.14

Reading comprehension activities 0.671-2 days 0.36 0.193-4 days 0.43* 0.19Daily 0.23 0.19

Children take books home to read with their parents

0.66

1-2 days 0.24 0.303-4 days -0.36 0.46Daily 0.07 0.38


0.66

1-2 days 0.12 0.123-4 days -0.32 0.38Daily -0.20 0.26

Teacher works on word building with students

0.66

1-2 days 0.12 0.123-4 days -0.32 0.38Daily -0.20 0.26

Students read and draw 0.661-2 days -0.07 0.173-4 days -0.01 0.18Daily -0.16 0.16

Students work on spelling words in exercise books

0.68

1-2 days -0.31 0.20

3-4 days -0.20 0.18Daily -0.36* 0.18

Students writing sentences 0.671-2 days 0.35* 0.163-4 days -0.02 0.17Daily -0.003 0.16

ANNEX 2 / INSTRUMENTS

Annex 2.A: EGRA Instrument

Annex 2.B: Student Questionnaire

Annex 2.C: Head Teacher Questionnaire

Annex 2.D: Teacher Questionnaire

Annex 2.E: Classroom Observation

Annex 2.A: EGRA Instrument

Annex 2.B: Student Questionnaire

Annex 2.C: Head Teacher Questionnaire

Annex 2.D: Teacher Questionnaire

Annex 2.E: Classroom Observation

kiribati early grade reading assessmentdocuments.worldbank.org/curated/en/1140615632511… · web...

Documents