save the children international basic education … · in january 2013, save the children extended...
TRANSCRIPT
SAVE THE CHILDREN INTERNATIONAL
BASIC EDUCATION PROGRAM
TiANA PROJECT 2013 ENDLINE REPORT
FOR
ST ANTHONY AND NAMADIDI EDUCATION ZONES, TRADITIONAL AUTHORITY MLUMBE
ZOMBA RURAL, MALAWI
November, 2013
Contents Abbreviations .................................................................................................................................. 1
List of figures ................................................................................................................................... 2
Acknowledgement .......................................................................................................................... 3
Methodology ................................................................................................................................... 4
Measurement ................................................................................................................................ 5
Who is still in school? ..................................................................................................................... 6
Baseline and Midline TiANA Project Profile .................................................................................. 7
Average Gains in Learning .............................................................................................................. 8
Equity in gains ............................................................................................................................... 9
Learning and use of reading skills in Zomba ................................................................................. 10
Change in home literacy environment ..................................................................................... 12
Change in community literacy environment ............................................................................ 12
Teachers and Schools ................................................................................................................... 13
Teaching strengths and challenges ............................................................................................... 13
Focus on Lesson Delivery ............................................................................................................. 16
Community Strengths and Challenges .......................................................................................... 16
Youth Literacy Movement (YLM) .................................................................................................. 17
Conclusions ................................................................................................................................... 18
Appendix A: Equity models........................................................................................................... 19
Appendix B: Community Literacy Models -- Chichewa ............................................................... 20
Appendix C: Community Literacy Models -- English ................................................................... 21
Appendix D: Change in Community Literacy Models .................................................................. 22
1
Abbreviations AusAid Australia Aid DEM District Education Manager EGRA Early Grade Reading Assessment HLE Home Literacy Environment LB Literacy Boost MIE Malawi Institute of Education MSCE Malawi School Certificate of Education MTPDS Malawi Teacher Professional Development Service MUW Most Used Words ODL Open Distance Learning PEA Primary Education Advisor PTA Parents Teachers Association SCI Save the Children International SES Socio Economic Status SMC School Management Committee TA Traditional Authority TiANA Tiwerenge ndi Ana Athu USAID United States Agency for International Development YLM Youth Literacy Movement
2
List of Annexes Appendix A: Equity models Appendix B: Community Literacy Models – Chichewa Appendix c: Community Literacy Models – English Appendix C: Change in Community Literacy Models
List of figures Figure 1. Percent of children present at end of year assessment by group
Figure 2. TiANA Chichewa reading skills profile: baseline and midline
Figure 3. TiANA English reading skills profile: baseline and midline
Figure 4. Reading English single words and text by HLE and group
Figure 5. Use of Reading Skills at baseline and midline
Figure 6. Community literacy environment (CLE) predicts skill gains
Figure 7. Change in Community Learning Environment predicts skill gains
Figure 8. Revisiting Baseline Areas of Teaching Challenges -- Chichewa
3
Acknowledgement Save the Children would like to thank all partners for their support in providing insight into actions that help improve the quality of education. In particular Save the Children would like to thank the Ministry of Education, Science and Technology (MoEST), Malawi Institute of Education (MIE), District Education Manager (DEM) and Primary Education Advisors (PEAs) for Zomba Rural, Teacher Trainers from Blantyre and Machinga Teacher Training Colleges for their assistance in the data collection effort right from the development of instruments, pre testing to the actual data collection. Further, Save the Children would like to thank World Vision International (WVI), Australia AID (AusAID) and United States Agency for International Development (USAID)/Malawi Mission in a special way for their financial support and guidance throughout the implementation of the TiANA project.
Matthew Pickard Country Director
4
Introduction
In January 2013, Save the Children International extended the Basic Education program interventions in Malawi through a project called “Tiwerenge ndi Ana Athu” (TiANA) with funding from the United States Agency for International Development (USAID). TiANA is an innovation under the All Children Reading Grand Challenge (USAID, World Vision International and AusAid). The project intends to underline the role of parents and the community in making reading a reality for all children in a bid to improve learner writing and reading in the lower primary through innovative community-based production and use of teaching and learning materials. The name of the project “Tiwerenge ndi Ana Athu” translates from Chichewa as "Let us read together with our children." This project is being implemented in 10 primary schools in St. Anthony and Namadidi Zones in the area of Traditional Authority Mlumbe in Zomba District and its value adding components include: youth engagement through the Youth Literacy Movement (YLM) to champion literacy activities, incentivising community action through establishment of village savings and loans and engagement of a media house to broadcast reading camp activities. The YLM is a grouping of primary/secondary school graduates which garner support for camp reading and add intellectual capacity in implementation of the project activities. TiANA project is set out to:
• Improve community capacity to support the sustained production of relevant reading, teaching and learning materials.
• Enhance use of appropriate teaching and learning materials by teachers. • Increase parental awareness and support for children’s reading outside school
perimeter.
TiANA project seeks to strengthen children’s reading ability through both community and teacher-focused elements. While the program originally intended to conduct a baseline at the beginning of the school year in the fall of 2012, all project activities were delayed and only started in February of 2013 during which a baseline was conducted to set bench marks for implementation of the project. At the end of the school year in July 2013, the project team revisited the schools to collect end of school year, project midline data. The assessment and survey were done to ensure that the project delivers initiatives that are informed by ongoing evidence of progress, or lack thereof. The results are also meant to assess the impact of the program at different stages, thereby aiding in the identification of key intervention areas that would assist in improving teachers’ professional skills and community assistance to support learning. This report therefore examines the results of a learner background survey and reading assessment conducted in July 2013.
Methodology The baseline (February 2013) and midline (July 2013) data collection was done in order to assist in rigorously identifying the effects of the TiANA project and includes treatment plus two levels of comparison schools:
1. Ten treatment schools (TiANA) from St Anthonys’ and Namadidi education zones in TA Mlumbe;
2. Ten first level comparison schools in the current SCI’s Literacy Boost (LB) intervention schools within Zomba in Chimwalira education zone in TA Chikowi;
5
3. Ten ordinary public schools in St Michaels’ Zone in TA Mwambo within Zomba where there are no any SCI interventions.
These groups allow a comparison of TiANA versus Literacy Boost, and the normal approaches of teaching reading used in the public schools in Zomba. TiANA will use this assessment to identify gaps and measure improvements in reading skills.
The sample at baseline assessment encompassed 600 standard 3 learners, divided between 10 schools in the TiANA intervention area (n of learners = 200) and 20 comparison schools (n of learners = 400) split into Literacy Boost and comparison schools. At each school 20 (10 boys and 10 girls) learners in standard 3 were randomly selected. Curriculum-based assessments in English and Chichewa which were developed in partnership with the Malawi Institute of Education and Ministry of Education Science and Technology were then administered among the randomly selected learners.. In addition, children answered questions about their lives and homes.
The key research questions explored in this report include:
1. Who is in school at the time of the midline? 2. What has changed in the skill profile of TiANA learners across the study period? 3. How much learning progress has each group made? 4. What factors influenced greater progress in this school year and may be levers for
change in the coming year? 5. What progress have teachers made, and what challenges remain? 6. How do communities support these efforts, and what challenges remain?
By addressing these questions, this analysis and report aims to inform TiANA and Literacy Boost implementation improvements in 2013-2014.
Measurement School profile data including information on enrolment, class size and teacher background was collected through a survey of school head teachers at every school in the sample. . Teacher interviews and classroom observations of both English and Chichewa lessons were conducted. For the learners’ assessment, all learners in the sample were asked about their background characteristics (age, household possessions, household building materials, etc.) and literacy environment (availability of material, reading habits, etc.). After collecting this background data, all learners were given an emergent literacy test in both Chichewa and English composed of six components administered through five sub-tests: writing, letter awareness, single word recognition (reading of most used words), reading fluency & accuracy (words per minute read correctly and total percentage of passage words read correctly; within the same sub-test), and a set of comprehension questions linked to the fluency & accuracy passage. All instructions were given in Chichewa and children were assessed on word writing, letter identification, most used words, reading fluency, reading accuracy, and reading comprehension.
6
Who is still in school? In February 2013, the TiANA team collected data on 200 children in each of the three groups: Comparison, Literacy Boost and TiANA impact area. During the July 2013 assessment, the project found all but 90 children in comparison schools, all but 65 children in Literacy Boost intervention area and all but 30 children in TiANA impact area. Figure 1 shows how different these samples were at end line.
Differences between the children who were present and those absent, reveal that for the Comparison schools children who are missing had significantly lower baseline Chichewa reading skill scores (in four of five skills) than those who were present. In addition, they had a significantly weaker home literacy environment. For the LB intervention area school children who are missing had significantly lower baseline English reading skill scores (in two of five skills) than those present. On the contrary, children from TiANA project area who are missing had significantly poorer and weaker home literacy environments than those present.
The most likely reasons for differences among the three school groups in percentage of children missing during the end line assessment could be that the number of teachers and learners in the areas is not the same, making teacher-learner ratios quite different. TiANA project area has a teacher to learner ratio of 1 to 74; Literacy Boost intervention area has a teacher to learner ratio of 1 to 110; while the comparison schools have a teacher to learner ratio of 1 to 104. Larger class sizes may also have contributed to lower attendance of learners. In addition, the area with highest absenteeism rate is along Lake Chirwa, Malawi’s second largest lake, where a lot of money-earning opportunities such as fishing and rice growing take place. This, coupled with the fact that data collection was done in July which is the peak of rice harvesting, could have contributed to the high absenteeism in this area. In 2014, data collection will be done in June to avoid absenteeism due to rice harvesting peak period. Further, the comparison schools do not have education projects while TiANA and Literacy Boost area have innovations that motivate learners to be interested in schooling. The fact that all ten of the comparison schools are in the lake region where there are competing options for children’s time but an absence of interventions for better quality schooling gives reasons as to why this group faced a double challenge and, therefore, the lowest sample presence is not surprising.
Average baseline scores of children present in school at midline, are only different between groups for letter knowledge: TiANA project area learners have significantly lower scores on average, knowing only 11 letters out of 26; while learners in LB and comparison areas know 14 out of 26 letters, on average. While this demonstrates that the learners present were
55%
68%
85%
0%10%20%30%40%50%60%70%80%90%
Comparison Literacy Boost TiANA
Figure 1. Percent of children present at end of year assessment by group
7
comparable in most skills at baseline, it has been noted that the learners were not similar on all counts. All comparisons must be interpreted in light of the trends above: the lowest Chichewa scoring and lowest home literacy environment (HLE) comparison school learners, the lowest English scoring LB learners and the poorest and lowest HLE TiANA learners are missing. This, , therefore, makes it difficult for the project to know how they would have performed. Thus, the first recommendation of this report echoes that of Silver (2012) to look at attendance more carefully – especially among these missing groups – and work to keep children consistently in school. In addition, whether in treatment or comparison schools, every effort should be made at the end of the school year to find all children in the sample and assess their skills.
Baseline and Midline TiANA Project Profile Within TiANA schools, the children assessed in both February and July demonstrate progress across all reading skills – both in Chichewa (Figure 2) and in English (Figure 3).
While the learners are headed towards letter mastery, have doubled the number of words they can write, and read many single words common in their curriculum, they are not reading texts fluently to support full comprehension. In fact, 47 learners, or 27 percent of the remaining TiANA project sample did not read any single word of the Chichewa text correctly. While this improves upon the 47% of MTPDS Level 3, Standard 4 learners were still not able to read at endline, therefore, moving from letters and words to text should be an important focus for next year’s work in these schools.
As expected in a second language, English reading progress lags behind. Considering the Figures 2 and 3 together, greater gains were made in Chichewa than in English. Chichewa comprehension is hovering just above 50%, and very little progress was made in English comprehension. This, therefore, means that a lot of work has to be done to realize the goal of reading with comprehension. It has been observed that teachers are not always comfortable teaching in English, therefore, the basic education programme introduced content knowledge
42.3
6.1
29.2
10.1
33.0 36.0 33.3
69.2
12.1
64.2
21.6
60.4
42.0
53.9
0
10
20
30
40
50
60
70
80
letters (%) words written (#) most used words(%)
fluency (wpmc) accuracy (%) prediction (%) comprehension(%)
Figure 2. TiANA Project Chichewa reading skills profile: baseline and midline
baseline
endline
8
support for teachers to be assisted academically in English through the long term sponsorship programme.
Average midline status among the remaining children in TiANA project, Literacy Boost and comparison schools is similar across all the skills in both languages. There are no differences in treatment versus comparison groups. Without the 45% of the comparison group that is missing, it is impossible to interpret this lack of difference. Progress in each skill is, therefore, compared among the learners who were present in school in the three comparison groups.
Average Gains in Learning To generate gain scores, baseline scores are subtracted from midline scores. The resulting score represents the learning progress of the learner during the study period, in this case five months. Table 1 presents average gains for boys and girls in each reading skill for both languages and by study group. The only significant gender differences are in LB schools where girls made significantly greater progress than boys in Chichewa word reading and fluency.
Table 1. Average reading skills gains by language, sex and group
Chichewa Boys Girls English Boys Girls
Gain in Number of Words Written
Comparison 4.18 5.07 Comparison 1.95 1.38
Literacy Boost 4.87 4.71 Literacy Boost 1.12 0.55
TiANA 7.04 4.94 TiANA 2.74 3.06
Percentage Point Gain in Words Read
Comparison 31.96 28.33 Comparison 17.62 15.82
Literacy Boost 25.4 43.28 Literacy Boost 13.61 13.07
1.9
10.6
3.6 5.9
26.0
4.1 4.8
31.9
9.9
26.0
31.4
9.0
0
5
10
15
20
25
30
35
words written(#)
most used words(%)
fluency (wpmc) accuracy (%) prediction (%) comprehension(%)
Figure 3. TiANA Project English reading skills profile: baseline and midline
baseline
endline
9
TiANA 30.07 39.74 TiANA 21.23 21.36
Fluency Gain (words per minute correct)
Comparison 2.35 3.96 Comparison 6.56 5.38
Literacy Boost 6.62 12.4 Literacy Boost 4.13 5.84
TiANA 11.05 11.84 TiANA 7.71 5.05
Percentage Point Accuracy Gain
Comparison 24.65 23.4 Comparison 20.84 18.69
Literacy Boost 20.46 19.4 Literacy Boost 19.94 18.84
TiANA 22.5 32.1 TiANA 18.59 21.56
Percentage Point Comprehension Gain
Comparison 18.18 20.91 Comparison 8.18 3.64
Literacy Boost 16.04 20.59 Literacy Boost 1.12 0.74
TiANA 14.94 18.97 TiANA 3.05 4.89
The overall average gains among the learners are difficult to judge. The only national fluency data available comes from the 2010 National EGRA in Chichewa in which Standard 4 learners read 11 words per minute correctly on average and the more recent MTPDS program evaluation in which Level 3 Standard 2 learners read 9 words per minute correctly on average at endline. As TiANA project standard 3 learners still in school in this study gained 11.5 words per minute on average during the five month study period and read an average 21.6 words per minute correctly (see Figure 2), this seems a subjectively favorable outcome. Similarly, LB learners gained 9.5 words per minute on average and read 19.9 words per minute correctly. Comprehension scores too compare favorably to the 2010 and MTPDS scores in this manner. However, these comparisons, as those noted above are affected by the attrition from the samples and so no attribution to the TiANA project or LB initiatives can be made.
Equity in gains Using regression analysis, the project considered three dimensions of equity (gender, socioeconomic status, and home literacy environment) and how children at risk fared in the three comparison groups:. Home Literacy Environment (HLE) encompasses both the availability of reading materials in the home as well as how those materials are used to engage the child in reading and learning. Five dimensions of the home literacy environment relate to reading achievement in children: value placed on literacy, press for achievement, availability and use of reading materials, reading with children, and opportunities for verbal interaction. The LB Assessment operationalizes these dimensions into questions for children and synthesizes their responses into an HLE index.
10
Fitting interactions with group membership, it was found that while there were no discernable differences in the effect of being in a group on girls vs. boys or poorer vs. better off children, there was a trend across English skills of impact for children in weaker vs. stronger home literacy environments. Figure 4 shows this finding and offers further insight into the average gains presented in Table 1.
Underlying the average gains in TiANA schools are similar gains for low HLE (red) and high HLE (yellow), while in the other schools, these differences are significantly different. The high HLE learners (blue) gained greatly – even above TiANA project average gains for both groups – while the low HLE gain significantly less. This pattern also exists in English reading fluency and Chichewa writing. Its presence across these several skills suggests that TiANA project is helping all children still present at endline equally well – a strength to build upon while exploring how to ensure that the missing poorest and lowest HLE learners are in school at the end of next session so that we can continue to learn more. The absence of this pattern in Chichewa reading skills is curious – indicating that TiANA project could be doing something differently in English than in Chichewa.
Learning and use of reading skills in Zomba As relevance and use of reading skills relates to the amount of practice and progress that learners make in mastering various component skills, this section reports on whether and how the data show changes in literacy practices, and how those changes relate to learning.
11.3% 11.3% 11.3% 11.3% 6.6% 6.6% 6.6% 6.6%
22.1%
20.4%
19.8%
19.8% 10.1% 13.0%
24.1% 30.2%
0%
5%
10%
15%
20%
25%
30%
35%
40%
TiANA lowHLE
TiANA highHLE
ComparisonLow HLE
Comparisonhigh HLE
TiANA lowHLE
TiANA highHLE
ComparisonLow HLE
Comparisonhigh HLE
English word reading English accuracy
perc
enta
ge p
oint
s
Figure 4. Reading English single words and text by HLE and group
baseline TiANA low HLE gain TiANA high HLE gain Comparison Low HLE gain Comparison high HLE gain
71%
35%
10%
12%
5%
22%
6%
4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
baseline (n=382) midline (n=393)
Figure 5. Use of Reading Skills at baseline and midline
use in the kitchen
use in market
use at church/mosque
do not use
11
When asked at baseline whether they use their reading skills, 71 percent of learners answered no. Among those who answered yes, many gave further detail about where they use them: in churches/mosques, kitchens and markets. Figure 5 shows that at midline, only 35 percent of these same learners answered no, and many more report using these skills at the market, while church/mosque and kitchen remain common locations of use of reading skills.
Whether this is a reflection of the awareness of relevance of skills or an actual use is not known, but either represents a shift in thinking among these learners. The midline data suggest that reading has increasing social relevance too. At baseline, 61.8 percent of learners reported having a friend they read with; at midline among the same group of learners, 75.6 percent reported having a friend with whom they read. Further, 26 percent reported helping others to or with reading at baseline, and by midline, 45.1 percent are doing so, including helping other children read or reading things for parents, etc. Finally, while participation in reading camps increased from 20.1 percent at baseline to 37.8 percent at midline, this seems low in a sample in which 67 percent of learners are in schools in which reading camps are promoted to support reading.
Our home literacy environment index denotes a measure of materials and interactions in the home to support reading. Their values at midline are highly related to Chichewa skills and basic English skills as well. But the notions of use and relevance reported above go beyond the home, so using these three indicators helps to broaden the notion to form a community literacy index. This index combines the data on the use of skills, assisting others with reading, and reading with friends. The community literacy index score at midline significantly predicts all skill gains controlling for baseline score, age, sex, SES, HLE, and workload with the exception of gains in letters (see appendix B for full models). Figure 6 shows this trend for reading single words, accuracy and fluency across both target languages.
32.0
11.3
35.5
12.2 3.4 6.6
34.4
7.3
20.7
12.7
4.1
16.2
5.5
3.9
8.9
7.0
2.4
6.4
5.5
3.9
8.9
7.0
2.4
6.4
0
10
20
30
40
50
60
70
80
single words(%)
fluency(wpmc)
accuracy(%)
single words(%)
fluency(wpmc)
accuracy(%)
Chichewa English
Figure 6. Community literacy environment (CLE) predicts skill gains
gain CLE=3
gain CLE=2
gain CLE=1
baseline
12
Given that TiANA project and LB initiative aim to affect the home and community literacy environment, it is of interest to see whether change in these indicators is related to learning.
Change in home literacy environment Regression models revealed emerging patterns between change in HLE and learning, but nothing strong and consistent enough to present for interpretation. Whether this is due to the sample attrition, the short time of intervention or measurement error is not known, but efforts to collect data from all learners over the course of the full 2013-2014 school year of implementation will assist the project to learn more at the end of the program.
Change in community literacy environment Regression models considering change in community literacy environment, found that LB learners reporting such changes had significantly greater learning than learners in comparison groups, controlling for age, sex, initial score, SES and workload. Figure 7 shows this pattern for letters and Chichewa single word reading, while it also exists for Chichewa accuracy. If a learner in LB school began doing one (red) or two (blue) community literacy activities during the intervention period, then their predicted gain is significantly higher than the predicted gains of learners in either TiANA project or comparison schools who did so – though their gains were higher than those who did not begin such activities.
More generally across the sample, regression models link change in CLE with Chichewa and English accuracy (see Appendix C for full models). While there is hope to see this change in use and relevance of reading and its relationship to greater progress in learning in both LB and TiANA project schools, the challenges of attrition and a relatively short implementation period between baseline and midline might preclude this. The project hopes to learn more about CLE and its change associated with learning at the project endline.
52.1% 52.1% 52.1% 52.1%
32.0% 32.0% 32.0% 32.0%
22.3% 19.2%
39.2% 38.0%
26.1% 17.1%
48.4% 37.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Literacy Boost Comparisons Literacy Boost Comparisons Literacy Boost Comparisons Literacy Boost Comparisons
letters single words
Figure 7. Change in Community Learning Environment predicts skill gains
baseline gain 1 CLE change gain 2 CLE change
13
Teachers and Schools At midline, 30 teachers were interviewed and observed teaching English and Chichewa lessons. All but one spoke Chichewa at home and all but three had Malawi School Certificate of Education, or 12 years of education from primary to secondary final year; the other three had a Junior Certificate of Education that is Form 2 or 10 years of schooling and were working in TiANA project area (2 teachers) and LB schools (1 teacher).
The teachers had an average age of 31 years and had been teaching for seven years on average. Each group included teachers as young as 20 years old with 1 year of experience and those with a decade of experience as well as more than two decades of teaching. Although there are no differences among the school groups in these teacher characteristics, there are fewer female teachers in comparison schools (27%) than in LB schools (36%) or TiANA project schools (39%).
School-wise, an enrollment and attendance comparison shows that although the standard 3 enrollment is of similar size, there is a gap in attendance by school as seen in Table 2 that is mirrored in the assessment sample.
Table 2. Third grade enrolment versus attendance on day of assessment
Group Total on roll Present % Present
Comparison Schools 1,664 770 46
Literacy Boost Schools 1,417 876 62
TiANA Project Schools
1,429 1,219 85
Total 4,510 2,865 64
As noted previously, the LB and comparison school enrollment is much higher (14,102 and 13,622, respectively) than that of the TiANA project schools.
Teaching strengths and challenges The teacher observation instrument was split into two sections of Lesson Planning and Preparation and Lesson Delivery. Using this instrument, thirty standard 3 teachers from the sample were observed by trained observers who included Primary Education Advisers, Teacher Trainer from teacher Training Colleges and Save the Children Education staff. A variety of skills in English, and Chichewa reading instruction were observed and rated using the following scale: 0 = not done, 1 = Poor, 2 = Fair, 3 = Good and 4 = Outstanding. Items observed and rated using this scale included the following lesson preparation and lesson delivery areas:
• Suitability, relevance, liveliness of introduction, • Ability to cope with individual differences, • Degree of learners' participation and interaction, • Appropriate use of language, • Effectiveness of methods used,
14
• Use of teaching and learning resources, • Logical development of the lesson, • Use of continuous assessment, • Class control and organization for learners activities, • Teacher – learner and Learner-learner relationship.
Ten teachers in each group of schools were observed. Overall, Figures 8 and 9 shows that the teaching in Chichewa and English classes improved during this short period. In each of the identified areas, where the average teaching observed was poor to fair at baseline, some progress was made. The greatest progress in Chichewa teaching in Figure 8 is seen in learners asking questions to the teachers, although this remains the greatest challenge and may not be surprising in a classroom with nearly 100 learners. The least progress was made in classroom displays, although locally available resource use is up between fair to good on average.
In teaching English, in Figure 9, there was progress in the short intervention period as well, and interestingly, the greatest progress was made in the use of locally available resources. The average teacher is headed towards doing this well – which should be of assistance as all the learners take up this second language.
0
0.5
1
1.5
2
2.5
Learners initiatedialogue among
themselves
Learners initiatedialogue withthe teacher
Learners askquestions to the
teacher
Use of locallyavailable
resources
Classroomdisplays
1.6
1.2
0.4
1.7 1.7
2.0
1.6
1.3
2.4
2.1
Figure 8. Revisiting Baseline Areas of Teaching Challenges -- Chichewa
Baseline Midline
15
Again the weakest area observed was learners asking questions, an area worth following up with PEAs, mentors and SCI staff supporting teachers. Table 3 shows average group scores for eight challenging areas of teaching across both languages. These areas were identified by having overall average scores of lower than two (fair).
Table 3. Average Midline Observation Scores for Challenging Areas of Teaching by Language and Group
Chichewa
Schemes of Work Quality
Records of Work Quality
Attendance Registers
Availability
Attendance Registers Upkeep
Continuous Assessment
Records Availability
Upkeep of Continuous Assessment
Records
Learners initiate
dialogue w/teacher
Learners ask
teacher questions
TiANA 1.9 1.6 2.2 1.9 2.1 1.8 1.9 1.2
LB 2.7 2.4 2.6 2.1 2.2 1.9 1.7 2.0
Control 1.7 1.4 1.5 1.1 1.2 0.9 1.3 0.7
English
TiANA 2.1 1.6 2.2 2.0 2.0 2.0 1.7 1.5
LB 2.7 2.6 2.9 2.6 2.1 1.8 2.0 1.2
Control 1.7 1.4 1.8 1.4 1.7 1.5 0.9 0.4
It is interesting to note that of these eight, six relate to preparation, not delivery. The project should consider this an artifact of understaffing. With too few teachers to cover classes,
0
0.5
1
1.5
2
2.5
3
Learners initiatedialogue among
themselves
Learners initiatedialogue withthe teacher
Learners askquestions to the
teacher
Use of locallyavailable
resources
Classroomdisplays
1.2 1.1
0.7
1.5 1.7
2.0
1.5
1.0
2.7
2.1
Figure 9. Revisiting Baseline Areas of Teaching Challenges -- English
Baseline Midline
16
teachers have to prepare for more than one class and this affects their preparation and these six areas in particular.
The observation data also make it clear that LB teachers outperform their counterparts on average. Given that LB has been supporting these schools since 2010 and an estimated 75 percent of teachers in these schools have participated in trainings prior to the January-July 2013 time period, this is not surprising. However, it is also important to note teacher mobility is high in this area and only half of the teachers observed at baseline were also observed at midline due to this mobility. Thus, while the data suggest areas for attention, the findings cannot be attributed to change in teaching due to LB or TiANA project.
Focus on Lesson Delivery In areas of lesson delivery, there were four patterns evident at midline. First, on average, teachers everywhere were observed to do three things well: praise and correct behavior in a friendly manner, present subject matter logically, and phrase questions clearly. Second, teachers in TiANA and LB schools did four things well, on average, while comparison teachers performed fairly on average by making the introduction brief, lively, and including activities for learners, providing individual help during lessons, using situations/learners’ experiences to motivate them to learn, and simplifying the content to the level of the learners. Third, comparison school teachers observed for English lessons were fair on introduction, relevant to new subject matter, use of varied methods (i.e. pair work, group work, role play, songs and demonstration), use of varied questions (who, where, how, why etc), and distribution of questions randomly. These elements were good on average in their Chichewa lessons and in the lessons of all TiANA and LB teachers. Finally, outstanding teaching was apparent in several places:
• LB English: rewarding learners’ efforts (answering, asking questions and participating) • TiANA project Chichewa: Displays competence in subject matter that is being taught • TiANA project English: Use of question - pause – name
While these findings offer insight into progress made, they also highlight the challenges ahead in 2014. Given the identification of strengths herein, program improvement might be ignited via a discussion of what this good practice looks like, even capturing it on video, and opening discussion of how to enable more teachers to use these methods.
Community Strengths and Challenges A total of 214 community leaders were interviewed, including School Management Committee members, PTA Executive Committee members, Village heads, and Selected Parents. Table 4 shows the gender breakdown of this leadership and reveals a bias in comparison schools towards male participation in the interviews.
Table 4. Community interview participation by group and sex
Group Men Women Total
17
Literacy Boost 37 34 71
Comparison 44 19 63
TiANA 45 35 80
Total 126 88 214
The project has conducted many gender sensitization efforts in the schools it supports and this ensures that more and more women ladies are involved as is apparent in Table 4.
All Community Leaders in LB and 90% of community leaders in TiANA project schools reported talking to children and parents about dropping out from school, whereas only 60% do so in Comparison schools. They do this primarily through following up of learners through parents with the help of local leaders (43%) and encouraging the learners to go back to school (37%).
Among parents, all those interviewed in TiANA and LB schools reported assisting in teaching and learning of their children while only 40% of comparison parents do so. Parents reported supporting their children by: checking their work after school (30%), sending learners to reading camps (17%), borrowing books for their children (10%), asking/helping them to read (13%), and buying school materials for them (3%). Interestingly, there appears to be a comparison school with a reading camp – a development worth following up!
All SMCs monitor teaching and learning in the reading camps in TiANA and LB schools by looking into camp leaders’ attendance, learners’ attendance, checking availability of reading materials, and monitoring how lessons are conducted in the camps. As a result of these monitoring visits, it was noted that:
• Many learners were at least able to read better than before the camps were introduced • Many children were interested to go to the reading camps • School Management Committees requested parents to produce reading materials
Overall, changes were observed by all respondents in LB and 90% of TiANA project schools. These included: a change in number of children able to read, an increased number of children having interest in going to the reading camps, and an increased number of children reading fluently since the introduction of the camps.
Youth Literacy Movement (YLM) Youth Literacy Movement (YLM) is a group of 10 school leavers (boys and girls) per school. The YLM works alongside Bright Future Committee (BFCs) also a grouping of parents, SMC and PTA members, village heads and learners themselves who help m o n i t o r r e a d i n g c a m p a c t i v i t i e s a n d a l s o h e l p i n t h e p r o d u c t i o n o f low cost teaching and learning materials to be used by children. YLM presence is strongest in TiANA project and known in 8 of 10 LB schools as well. The youth are charged with the responsibility of encouraging children to go to reading camps, helping develop reading materials for the camps, helping supervise the reading camps, and mobilizing parents to support reading camps. While data was not collected on the extent and effectiveness of their work at midline, this could be an interesting topic for exploration at end line.
18
Conclusions There is still a great deal of ground to cover in the 2013-2014 school year. The data collection at the beginning and end of the school year will span a longer period and offer more opportunity to see the impact of the intervention. The following recommendations come from the analysis above:
• Collect data in June 2014 to avoid peak rice harvest season in the comparison area of around Lake Chirwa.
• Make every effort to gather data on all learners, regardless of enrollment status. • Review attendance records carefully and work to keep children consistently in school,
paying particular attention to: o How TiANA project can better support lowest HLE and poorest children in
school. o How LB schools can better support low English achievers.
• Redouble efforts to take Standard 3 learners beyond words reading to tackle texts and reading with comprehension.
• Promote the use of reading skills, the assistance of others as well as reading with friends to see if the project can generate greater gains associated with these changes in practice so far.
• Support teachers in areas of challenges identified at baseline that are still in need of attention as well as identified weaknesses at midline.
• Build on and spread observed teaching strengths across the system of schools supported.
• Reinforce community inputs to learning as well as attendance, and investigate the comparison school with a reading camp.
• Consider how to monitor and adapt the work of the youth thus far and integrate them into the data collection at end line.
References
Silver R. (2012): “A Qualitative Analysis of Dropout, Absenteeism, and Learner Engagement in Malawian Primary Schools” Not published Malawi Teacher Professional Development Support (MTPDS) Program (2010): “2010 Early Grade Reading Assessment”
19
Appendix A: Equity models
(2) (8) (9) (10)
VARIABLES Chichewa writing gain
English word reading gain
English fluency gain
English accuracy gain
TiANA * HLE quintiles (baseline) -0.737* -4.085** -2.231 -4.451*
(0.319) (1.147) (1.101) (1.713)
TIANA 3.076* 14.11* 5.906 11.86*
(1.339) (5.558) (3.348) (5.510)
LB -0.0167 -3.622 -0.842 0.0237
(1.281) (3.071) (2.913) (4.537)
HLE quintiles (baseline) 0.646* 3.496*** 2.206** 4.302***
(0.238) (0.701) (0.792) (1.171)
age -0.218 0.532 0.218 -0.360
(0.193) (0.893) (0.305) (0.809)
SES quintiles (baseline) -0.107 0.497 -0.196 0.303
(0.253) (1.004) (0.481) (0.881)
Sex -0.714 -0.109 -0.485 0.755
(0.815) (2.135) (1.150) (2.688)
Baseline Chichewa writing -0.359***
(0.0910)
Baseline English word reading
-0.163**
(0.0537)
Baseline English fluency
-0.105
(0.131)
Baseline English accuracy
0.466***
(0.106)
Constant 8.337** 3.432 -0.559 8.639
(2.536) (11.00) (3.937) (8.667)
Observations 407 407 407 407 R-squared 0.081 0.074 0.052 0.108 r2_a 0.0623 0.0551 0.0333 0.0899 Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05
20
Appendix B: Community Literacy Models -- Chichewa
VARIABLES
Chichewa writing gain
Chichewa single word gain
Chichewa fluency gain
Chichewa accuracy gain
Chichewa comprehension gain
community literacy environment (of 3)
1.126** 5.467* 3.872*** 8.928*** -8.571*** (0.319) (2.234) (0.895) (2.292) (2.067)
SES quintiles (baseline) -0.105 -1.490 0.448 -1.260 -0.287 (0.230) (1.021) (0.482) (1.681) (1.604)
HLE quintiles (baseline) 0.166 1.528 0.634 2.571 1.856 (0.191) (0.894) (0.599) (1.367) (0.946)
Sex -0.483 10.63* 3.444* 6.892 3.740 (0.838) (4.355) (1.591) (4.014) (4.100)
Number of chores -0.633 -7.100** -2.178 -6.416* -4.112 (0.415) (2.342) (1.166) (2.905) (2.924)
Pupil Age -0.192 0.606 -0.259 -0.849 -0.295 (0.216) (0.897) (0.500) (1.167) (1.242)
Baseline Chichewa writing
-0.411*** (0.0834) Baseline Chichewa word
reading -0.591***
(0.0456) Baseline Chichewa
fluency -0.476***
(0.0916) Baseline Chichewa
accuracy -0.521***
(0.0654)
Baseline Chichewa comprehension
-0.537***
(0.0669)
Constant
9.375** 46.73*** 10.13 43.81** 69.79*** (3.264) (11.29) (6.890) (15.10) (16.43)
Observations 407 407 407 406 588 R-squared 0.095 0.360 0.256 0.278 0.227 Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05
21
Appendix c: Community Literacy Models -- English
VARIABLES
English writing gain
English single word gain
English fluency gain
English accuracy gain
English comprehension gain
community literacy environment (of 3)
0.907*** 6.924*** 2.389** 6.326*** 3.050** (0.168) (1.179) (0.704) (1.464) (1.038)
SES quintiles (baseline)
0.118 0.485 -0.178 0.335 0.570 (0.173) (0.776) (0.449) (0.778) (0.634)
HLE quintiles (baseline)
0.437* 0.777 0.935 1.552 0.688 (0.168) (0.913) (0.599) (0.998) (0.622)
Sex -0.101 0.901 0.0193 1.923 -1.139 (0.371) (2.051) (1.103) (2.573) (1.856)
Number of chores 0.287 -2.737 -1.555* -3.252 -0.692 (0.193) (1.725) (0.705) (1.751) (1.080)
Pupil Age -0.00445 0.605 0.252 -0.275 -0.355 (0.159) (0.791) (0.288) (0.714) (0.484)
Baseline English writing
-0.375* (0.158) Baseline English word
reading -0.220***
(0.0571) Baseline English
fluency -0.145
(0.132) Baseline English
accuracy 0.381**
(0.111) Baseline English
comprehension -0.553***
(0.133)
Constant -1.007 4.016 1.195 10.90 2.976 (2.117) (11.16) (4.243) (9.738) (7.425)
Observations 406 407 407 407 407
R-squared 0.114 0.129 0.078 0.158 0.148 Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05
22
Appendix D: Change in Community Literacy Models (1) (3) (5) (16) (21)
VARIABLES Letter gain
Chichewa Single word gain
Chichewa accuracy gain
Chichewa accuracy gain
English accuracy gain
Literacy Boost school * change in CLE 1.521* 10.14* 9.956~
(0.716) (4.478) (5.704) Literacy Boost school -0.703 -8.936 -16.24
(1.005) (7.013) (10.19)
+ change in community literacy environment -0.526 -1.010 2.615 5.887* 3.802*
(0.367) (2.659) (2.723) (2.723) (1.776)
Pupil Age 0.0668 0.580 -1.086 -1.298 -0.559
(0.253) (0.879) (1.059) (1.055) (0.813)
SES quintiles (baseline) -0.0145 -0.845 -0.250 -0.479 0.867
(0.224) (1.044) (1.792) (1.780) (0.862)
Sex 0.992 10.07* 5.385 5.258 0.867
(0.820) (4.353) (4.052) (3.984) (2.826)
number of chores -1.174* -6.872** -6.255* -6.044* -3.120
(0.499) (2.361) (2.919) (2.859) (1.993)
Baseline letter score -0.459***
(0.106)
Baseline Chichewa singe word reading score
-0.528***
(0.0387) Baseline Chichewa accuracy score
-0.432*** -0.444***
(0.0576) (0.0609) Baseline English accuracy score
0.596***
(0.108)
Constant 12.61** 56.92*** 62.68*** 60.47*** 22.81
(3.481) (11.73) (14.35) (14.07) (11.29)
Observations 406 407 406 406 407 R-squared 0.372 0.351 0.252 0.234 0.098 Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05