2014 final evaluation report: teachers’ literacy knowledge,...

2014finalevaluationreport:Teachers’literacyknowledge,instructionalpractices,andtheirstudents’readingperformanceinPAQUED-supportedschoolsintheDemocraticRepublicofCongo

September,2014

SubmittedbyEducationDevelopmentCenter,Inc.Agreement#:AID-623-A-09-00010

2

ExecutiveSummaryTheProjectd’AméliorationdelaQualitédel’Education(PAQUED),fundedbytheUnitedStatesAgencyforInternationalDevelopment(USAID)andledbyEducationDevelopmentCenter(EDC),isafive-yearprogramspanning2009-2014focusedonimprovingthequalityofbasiceducationin3,000schoolsintheDemocraticRepublicofCongo(DRC).Initsinitialstages,theprojectcenteredaroundthreeobjectives:improvingthequalityofteachingandteachers’masteryofcontent,improvingstudentmasteryofsubjectcontent,andimprovingtheschoollearningenvironment.Theproject’sinterventionsincludedtheintroductionofover600InteractiveAudioInstruction(IAI)programsforreadingandmath,whichmirroredthenationalcurriculum;thetrainingofover30,000teachersinFrenchandMathcontent;theproductionanddistributionofcluster-directedprofessionaldevelopmentmodules;therehabilitationandconstructionoftrainingcenters;thedistributionofteachingandlearningkits;andthetrainingof3,000communitiesonschoolgovernanceandonimprovingtheschoollearningenvironment.InresponsetoUSAID’snewstrategy(launchedin2012withagoalof100millionchildrendemonstratingimprovementsinreadingby2015)andtoexternalmidtermreviewfindingsthatrevealedprojectactivitiestobespreadtoothinlygiventhelargeterrainandnumbersofschoolsoutlinedintheinitialprojectdesign.PAQUEDrealignedinJanuary2013tofocusprimarilyonimprovingstudentreadingoutcomes.CertaincomponentsofthePAQUEDprogramlikeIAI,self-directedtraining,communitysupport,andkitdistributionwerecontinued,andarobustexperimentalreadingprogramwasintroducedin45PAQUEDschools.Thisreadingprogramcombinedintensetraining,coaching,andtheproductionofteachingandlearningmaterials,aswellascommunitymobilizationactivitiescenteredonreading.Thisreportpresentstheresultsofacomparativeevaluationstudythatwasconductedpost-realignment,betweenMarch2013andMay2014.Thestudyfocusedonthreegroupsofteachersingrade1to6:experimentalschoolteachers,IAI-onlyteachers,andcontrolteachers.ItendeavoredtounderstandhowteacherswereusingthevariousPAQUEDinterventionsavailabletothemandhowtheirknowledgeofteachingreadingandtheirliteracyinstructionalpracticesmayhavechangedasaresultoftheseinterventions.Finally,thestudyalsosoughttounderstandwhethertherewasanydifferenceinhowgrade1and2studentsperformedinreadingasaresultoftheirteachers’participationintheinterventionsandacquisitionofliteracyknowledgeandpractice.Insummary,thefindingsfromthisstudyshowthatexperimentalteachers’knowledgeofhowtoteachreadingandwritingismorecloselyalignedwithsoundliteracyinstructionthantheirIAI-onlyandcontrolcounterparts.Experimentalteachers’practicealsochangedsignificantlywithinayearofusingthereadingprogram.Asaresult,theperformanceofthestudentsoftheseexperimentalteachersinkeyreadingskillslikeletteridentificationandfluencyshowed

3

dramaticdifferencesincomparisontotheircontrolcounterparts.Linearregressionanalysisconductedestablishessignificantlinksbetweenteachers’applicationofPAQUEDinterventionsandstudentperformance.Specifically,experimentalteachers’IAIusage,theirfidelityofimplementationofthereadingprogram,andtheirparticipationincontinuingprofessionalactivitiesandvisitsfromcoacheswereshowntocontributetochangesinteacherpractice,teacherknowledgeofliteracyinstruction,andstudentperformance.ManyofthesefindingsaresupportedbytheseparatePAQUED2014EndlineReportofEGRAandEGMAproducedbyResearchTriangleInstitute(RTI).

4

Tableofcontents

EXECUTIVESUMMARY..........................................................................................................................2

FIGURESANDTABLES...........................................................................................................................6

ACRONYMS..........................................................................................................................................8

INTRODUCTION....................................................................................................................................9

STUDYPARTICIPANTS.........................................................................................................................11

CHAPTER1:GRADE1AND2STUDENTANDTEACHERRESULTS..........................................................15GRADE2STUDENTREADINGPERFORMANCE....................................................................................................16

StudentperformanceandPAQUEDinterventions..............................................................................21Studentperformanceandteacherpractices:.....................................................................................23Studentperformanceandteacherknowledge...................................................................................25

GRADE1AND2TEACHERS’KNOWLEDGEOFLITERACYINSTRUCTION....................................................................30Phonemic,phonologicalandalphabeticawareness:.........................................................................34Fluency:..............................................................................................................................................34Vocabulary:........................................................................................................................................35Comprehension:..................................................................................................................................37Writing:..............................................................................................................................................38

GRADE1AND2TEACHERPRACTICERESULTS...................................................................................................40Phonemic,phonologicalandalphabeticawareness:.........................................................................45Fluency:..............................................................................................................................................46Vocabulary:........................................................................................................................................49Comprehension:..................................................................................................................................49Generalclassroomandliteracypractices:..........................................................................................51

CHAPTER2:GRADE3TO6TEACHERRESULTS.....................................................................................53TEACHERKNOWLEDGEOFLITERACYINSTRUCTIONFINDINGS:GRADE3,4,5&6TEACHERS....................................53

Phonemicandphonologicalawareness:............................................................................................56Fluency:..............................................................................................................................................57Vocabulary:........................................................................................................................................58Comprehension:..................................................................................................................................58Writing:..............................................................................................................................................60

GRADE3TO6TEACHERPRACTICEFINDINGS....................................................................................................62Phonemicandphonologicalawareness:............................................................................................68Fluency:..............................................................................................................................................69Vocabulary:........................................................................................................................................70Comprehension:..................................................................................................................................70Generalinstructionalandliteracypractices:......................................................................................70

RECOMMENDATIONSFORPOLICYANDPRACTICE:.............................................................................72Trainingmodalities.............................................................................................................................72

5

Materialsdevelopment:.....................................................................................................................74Communitymobilization....................................................................................................................75Researchandevaluation....................................................................................................................75InstitutionalCapacityBuilding:..........................................................................................................76

ANNEXA.METHODOLOGY.................................................................................................................78Observation(practice)andinterview(knowledge)tools:...................................................................78Readingassessment:..........................................................................................................................80

DATAANALYSIS..........................................................................................................................................81STUDYLIMITATIONS:...................................................................................................................................81

ANNEXB.TOOLS................................................................................................................................82READINGASSESSMENT:................................................................................................................................82OBSERVATION(PRACTICE)TOOLS...................................................................................................................83

Grade1and2observationtool..........................................................................................................83Grade3and4observationtool..........................................................................................................87Grade5and6observationtool..........................................................................................................91

TEACHERINTERVIEW(KNOWLEDGE)TOOLS:....................................................................................................95Grade1and2interviewtool:.............................................................................................................95Grade3and4interviewtool:...........................................................................................................104Grade5and6interviewtool............................................................................................................112

6

FiguresandTablesTable1.Frequenciesofsampledgrade1and2teachersbyprovinceandstatusatendline..............................12Table2.Frequenciesofsampledgrade3and4teachersbyprovinceandstatus................................................12Table3.Frequenciesofsampledgrade5and6teachersbyprovinceandstatus................................................12Table4.Frequenciesofsampledgrade1and2teachersbysexandstatus.........................................................12Table5.Frequenciesofsampledgrade3and4teachersbysexandstatus.........................................................12Table6.Frequenciesofsampledgrade5and6teachersbysexandstatus.........................................................12Table7.Meanclasssizedisaggregatedbysexpergrade1and2teachersampledbystatus..............................13Table8.Meannumberofstudentsdisaggregatedbysexpergrade3and4teachersampledbystatus............13Table9.Meannumberofstudentsdisaggregatedbysexpergrade5and6teachersampledbystatus............13Table10.Numberofschoolssampledbysub-division..........................................................................................13Table11.Summarydescriptivestatisticsofgrade2studentperformanceinreadingsub-testsbystatus………..17Figure1.Percentageofstudentswithzeroscoresbystatus...................................Error!Bookmarknotdefined.Table12.Summarydescriptivestatisticsofgrade2studentperformanceinreadingsub-testsbystatusomittingzeroscores.....................................................................................................................................................................18Table13.Summarydescriptivestatisticsofgrade2studentperformanceinreadingsub-testssub-testsbyprovinceandstatus..............................................................................................................................................................18Figure2.Meanscores,byprovinceandstatus.....................................................................................................20Figure3.StudentsperformanceinWCPMagainstnationalbenchmarkssetfor3rdgrade.................................21Table14.Fidelityofimplementationdataforgrade1and2teachersbyschooltreatmentstatus.....................21Figure4.Teachers’fidelityofimplementation(FOI)ofthereadingprogramandtheirstudents’meanperformanceinnumberofwordsreadcorrectly............................................................................................................................28Figure5.Teachers’fidelityofimplementation(FOI)ofthereadingprogramandtheirstudents’WCPM...........28Figures6.Experimentalteachers’applicationofyocabularypracticesatendlineandtheirstudents’WCPM....24Figure7.Experimentalteachers’applicationofP4atendlineandtheirstudents’WCPM..................................25Figure8.Experimentalteachers’totalmeanknowledgeofliteracyinstructionandtheirstudents’meanWCPM26Figure9.Experimentalteachers’totalmeanknowledgeofliteracyinstructionandtheirstudents’meanWCPM26Figure10.Experimentalteachers’responsestoQuestion3.1andtheirstudents’meanreadingaccuracy.........27Figure11.Experimentalteachers’totalmeanknowledgeofteachingwritingandtheirstudents’meanWCPM27Figures12.Correlationsbetweenteachers’responsestoQuestion1.3andstudentperformance.....................28Figure13.Experimentalteachers’responsestoQuestion5.2andtheirrstudents’meanreadingaccuracy.......29Figure14.Experimentalteachers’responsestoQuestion5.2andtheirrstudents’meanWCPM.......................30Figure15.Experimentalteachers’fidelityofimplementation(FOI)ofthereadingprogramandtheirtotalmeanknowledgeofliteracyinstructionatendline.........................................................................................................31Table15.Summaryofthegrade1and2teacherknowledgeresultsofcomparisonofmeansatendlinebetweengroups(higherpercentagesconvey“sound”knowledge).....................................................................................32Figure16.Teachers’meanknowledgeofteachingreadingbycomponentskill,atendline.................................32Table16.Itemanalysisofthegrade1and2teacherendlineknowledgeresults(means)comparisonbygroups(percentagesreflectagreement)...........................................................................................................................33Figure17.Experimentalteachers’fidelityofimplementation(FOI)ofthereadingprogramandtheirtotalmeanknowledgeofteachingfluency,atendline............................................................................................................41

7

Figure18.Experimentalteachers’fidelityofimplementation(FOI)ofthereadingprogramandtheirresponsestoQuestion2.1,atendline........................................................................................................................................41Table17.Summaryofthegrade1and2teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus..................................................................................................................4141Table18.Itemanalysisofthegrade1and2teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus......................................................................................................................41Figure19.Grade1and2teachers’changeinliteracyinstructionalpracticesfrombaselinetoendline..............42Table19.Summaryresultsoflinearregressionforthegrade1and2teacherchangeofinstructionalpracticesusingadherencetoteachersparticipationinCPDandIAIdosageaspredictors............................................................44Table20.Itemanalysisresultsoflinearregressionforthegrade1and2teachers’changeofinstructionalpracticesusingadherencetoteachersparticipationinCPDandIAIdosageaspredictors..................................................44Figure20.IAI-onlyteachers’IAIusageandtheirgainsinPractice4.....................................................................48Figure21.IAI-onlyteachers’IAIusageandtheirgainsinPractice22...................................................................48Figure22.Experimentalteachers’IAIusageandtheirgainsinfluency-buildingpractices...................................48Figure23.Experimentalteachers’IAIusageandtheirgainsinPractice9............................................................48Figure24.Experimentalteachers’IAIusageandtheirgainsinPractice10..........................................................48Figures25.IAI-onlyteachers’IAIusageandtheirgainsininstructionalpractice.................................................48Figure26.Experimentalteachers’IAIusageandtheirgainsinvocabularybuildingpractices.............................48Figure27.Experimentalteachers’IAIusageandtheirgainsincomprehensionbuildingpractices......................51Figure28.Experimentalteachers’participationinCPDandtheirgainsinPractice20..........................................52Figure29.Experimentalteachers'IAIusageandtheirgainsinPractice20(supportingstudents)......................52Table21.Summaryofthegrade3and4teacherknowledgeresultsofcomparisonofmeansatendline..........54Table22.Itemanalysisofthegrade3and4teacherknowledgeresultsofcomparisonofmeansatendlinedisaggregatedbystatus.........................................................................................................................................55Table23.Summaryofthegrade5and6teacherknowledgeresultsofcomparisonofmeansatendlinedisaggregatedbystatus...............................................................................................................................................................55Table24.Itemanalysisofthegrade5and6teacherknowledgeresultsofcomparisonofmeansatendlinedisaggregatedbystatus........................................................................................................................................56Table25.Summaryofthegrade3and4teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus.....................................................................................................................63Table26.Itemanalysisofthegrade3and4teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus......................................................................................................................63Figure30.Grade3and4teachers’changeinliteracyinstructionalpracticesfrombaselinetoendline..............64Table27.Summaryofthegrade5and6teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus......................................................................................................................65Table28.Itemanalysisofthegrade5and6teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus......................................................................................................................65Figure31.Grade5and6teachers’changeinliteracyinstructionalpracticesfrombaselinetoendline..............65Table29.Summaryresultsoflinearregressionforthegrade5and6changeinpracticeusingIAIdosageasapredictor 68Table31.Itemanalysisresultsoflinearregressionforthegrade5and6teacherobservationofinstructionalpracticesusingIAIdosageasapredictor..............................................................................................................................68Figure32.Experimentalteachers’IAIusageandtheirgainsinPractice1...........................................................65Figure33.Experimentalteachers’IAIusageandtheirgainsingeneralliteracypractices....................................71

8

AcronymsCPD ContinuingProfessionalDevelopmentCRS CatholicReliefServicesCTB CooperationTechniqueBelge(BelgianTechnicalCooperation)CWPM CorrectWordsperMinuteEDC EducationDevelopmentCenterEGRA EarlyGradeReadingAssessmentIAI InteractiveAudioInstructionMEPSP MinistèredeL’EnseignementPrimaire,SecondaireetProfessionelPAQUED Projetd’AmeliorationdelaQualitédeL’EducationRTI ResearchTriangleInstituteUSAID UnitedStatesAgencyforInternationalDevelopment

9

IntroductionIn2014,theDRCMinistryofEducation(MinistèredeL’EnseignementPrimaire,SecondaireetProfessionel,MEPSP)launchednewpolicyinitiativesintendedtoimprovethequalityofliteracyteachingandlearning.InFebruary2012,theNationalReadingCommission,establishedbytheDRCMinistryofEducation,proposednewperformancestandardsforreadingandwritingforallsixprimaryschoolgradesinFrenchandnationallanguage.Thenewperformancestandardswerepartofa“roadmap”(feuillederoute,inFrench)ofkeytaskstodevelop,implementandeffectivelymonitorprogresstowardachievingthegoalofimprovingthestateofliteracyeducationinthecountry.Thereadingroadmapincludesthedevelopmentofanewreadingcurriculumandpedagogicaltoolstosupportimplementation.Thefive-yearUSAID-fundedProjetd’AmeliorationdelaQualitédeL’Education(PAQUED)projectalignedwiththesenationalinitiativesbydevelopingandimplementingaresearch-basedreadinginstructionalapproachforGrades1and2inselectedprojectschools.Theexperimentalreadingprogramaimedtoprovideaplatformfortestingkeyinputsfromtheroadmap.Theseinputsincludethecontentstandards,anevidencebasedinstructionalsequence,andtext-levelingcriteriaandguidelines,whichtheReadingCommissionhasdevelopedaspartoftheproposednewnationalreadingcurriculum.Thetrainingapproachdesignedfortheexperimentalprogramprovidedmultipleopportunitiesforteacherstolearnandreflectuponthenewapproaches.PIEQbegantheprocessofdevelopingandtestingthenewprogrambyidentifying45experimentalschoolsinthethreeprovinceswheretheprojectoperates.16schoolswereidentifiedinBandundu,16inEquateur,and13inOrientale.Grade1and2teachersintheseschoolsbenefitedfromongoingtraining,coaching,adetaileddailylessonstructureandaccompanyingactivityguide,andleveledreadingmaterials.Theleveledreadingmaterials,developedforbothclassroomandstudentuse,drewuponthemesandcontentwithintheofficialDRClanguagecurricula(bothfornationallanguagesandforFrench).Thesetextsweredevelopedaccordingtoprovisionalbenchmarksandtext-levelingcriteriadevelopedandadoptedbytheNationalReadingCommission,whichwasestablishedlatein2012bytheMinistryofEducation.ClassroomactivitiesandstrategiesoutlinedinthesematerialsmirrortheMinistry-validatedstudentlearningstandards.ExperimentalschoolteachersalsocontinuedtobenefitfromPAQUED’sInteractiveAudioInstruction(IAI)andotherprojectinputs(e.g.,videotrainingmodules).618additionalPAQUEDprojectschools(referredtoastheIAI-onlyschools)benefitedfromIAI,projecttrainingonIAI,FrenchandMathcontentknowledge,andself-directedlearningmodulesforprofessionaldevelopment,occasionalvisitsfromaPAQUEDprojectteammember,andmaterialslikestudentkits,classroommaterials(chalk,rulers,mathkits),mp3radios,andteacherguides,butdidnotreceivetheothersupportsassociatedwiththereadingprogram.Theremaining2,382PAQUEDprojectschoolswereprovidedwithIAIprograms,training,andkitmaterialsbutwerelesslikelytoreceivevisitsfroma

10

PAQUEDagent,astheywerelargelyinaccessibleduetodistance,security,andlimitedtransportoptions.These2,382schoolswerenotincludedinthestudydescribedinthisdocument,sincetheproject’srealignedfocushadshiftedtotheexperimentalandIAI-onlyschools.AcomprehensivestudywasundertakentoidentifyhowteachersintheexperimentalandIAI-onlyschoolsevolvedoverthecourseoftheprogramintermsoftheirclassroompractices,knowledgeaboutliteracyinstruction,dispositions(i.e.,attitudes)towardliteracyandliteracyinstruction,andchangesinstudentperformance.Initialfindingsshowmarkedimprovementinteachers’knowledgeandskillsoverbaseline,aswellasimprovedstudentperformanceonletter-sound,vocabulary,andfluencymeasures.Datacollectedviaindividualinterviews,classroomobservations,andfocusgroupinterviewsofGrade1and2teachersshowimprovementsintheirknowledgeandpracticefordevelopingarangeofstudentskills,includingletter-soundknowledge,decoding/encoding,vocabulary,fluencyandcomprehension.Theseresultssuggestthatthereadingprogram,includingtheintegrateduseofIAIinstruction,positivelyimpactedteacherknowledgeandpracticeinsupportofthedevelopmentofstudents’literacyskills.Thisreportpresentstheresultsofthestudyandhighlightskeyelementsofthereadingprograminterventionthatarebelievedtohavecontributedtoresults.First,wedescribethestudysampleandthetheoryofchangeonwhichthisstudywasbased.Thereafter,thediscussionisdividedintotwosections:thefirstfocusingontheresultsforgrade1and2studentsandteachers,whoweretheprimarytargetsofthereadingprogram,andthesecondfocusingongrade3to6teachers,whoseexposuretotheprogramcamethroughclustertrainingwithgrade1and2peersandIAIliteracymaterialsforgrades3-6.Chapter1ispresentedintotwoparts:Thefirstpartpresentsstudentreadingperformancedataandthepossiblelinkagestostudents’exposuretoPAQUEDinterventionandthechangesintheirteachers’practiceandknowledge.Thesecondpartdivesdeeperintothefindingsrelatedtoteacherknowledgeofhowtoteachreadingandwritingandchangesinteachers’literacyinstructionalpracticesfrombaselinetoendline.TheseresultsarelinkedtothevariousPAQUEDinterventionsmadeavailabletoteachers.Chapter2exploresgrade3to6teachers’knowledgeofreadingandwritingandtheirchangeinpracticefrombaselinetoendline,linkingtheseresultstoPAQUEDinterventions.ThereportconcludeswithadiscussionofrecommendationsandlessonslearnedforfutureprojectsandpoliciesderivedfromadataandresultsworkshopattendedbytheDRCMinistry’sNationalReadingCommissioninAugust2014.

11

StudyparticipantsThisstudyexamined3distinctgroupsofteachers:teacherswhobenefitedfromPAQUED’sintensivedailyreadingprogram(experimentalschoolteachers)plusInteractiveAudioInstruction(IAI);teacherswhobenefitedonlyfromPAQUED’sIAIprogramming(thisgrouprepresents98%ofPAQUEDinterventionschools);andteacherswhodidnotbenefitfromthePAQUEDprogramatall(controlschoolteachers).Thestudywasdesignedasamatchedpairstudy(seeAnnexA)topermitbothlongitudinalandcross-sectionalanalysis.TeacherswithineachschoolwereselectedrandomlyfromthePAQUEDteacherdatabaseatbaselineinFebruary2012.Atbaseline,thestudyparticipantsamplesizewaspre-determinedbasedonamatched-pairdesignusingaonetail,.5effectsize(α=.025,β=.8)providingthefollowingbreakdownofteacherstobesurveyedandobserved:Takingintoaccountgeneralattrition,teachermobilityacrossgrade-levels,andsubsequentreplacementteachersselectedtoparticipateinthestudy,thedistributionschangedoverthecourseofendlineandbaseline.Thetablesandfiguresbelowprovideanoverviewofoursampledpopulationofteachersdisaggregatedbygradeleveltaught,statusandprovince,andtheiraverageclassroomsizes.ThedistributionofteachersbyprovinceandstatuswerefairlyevenlydistributedwiththeexceptionofOrientalwhereIAI-onlyteachersrepresentagreaterpercentageofthesampleacrossgradelevels.

TEACHERS Experimental IAI-only Control grade1-2 35 35 35 grade3-4 35 35 35 grade5-6 35 35 35 total 105 105 105 315

12

TeacherSamplebyprovinceandgrade

Intermsofgenderdifferencesinteacherssampled,itisinterestingtonotethedrop-offoffemaleteachersingrade5and6forIAI-onlyandcontrolschools.ThisisconsistentwiththeDRC-basedstereotypemaleteacherforolderstudents.Forexperimentalschoolsthough,thistrendwasn’taspronounced.

TeacherSamplebysexandgrade

Table1.Frequenciesofsampledgrade1and2teachersbyprovinceandstatusatendline

Status Province N

ControlBandundu 34Equateur 38Orientale 53

ExperimentalBandundu 29Equateur 30Orientale 25

IAI-onlyBandundu 30Equateur 43Orientale 56

Table2.Frequenciesofsampledgrade3and4teachersbyprovinceandstatus

Status Province N




Table3.Frequenciesofsampledgrade5and6teachersbyprovinceandstatus

Status Province N




Table4.Frequenciesofsampledgrade1and2teachersbysexandstatus

Status Sex N

ControlF 36M 53

ExperimentalF 54M 15

IAI-onlyF 66M 32


Status Sex N

ControlF 21M 53


IAI-onlyF 59M 46


Status Sex N

ControlF 8M 53


IAI-onlyF 21M 46

13

Inadditiontoteacherdemographics,itisalsoimportanttoconsiderteachers’meanclasssizesbecauselargerclasssizesareoftencorrelatedwithteacherandstudentperformance.Interestingly,thesamplerevealedslightlybiggermeanclasssizesinexperimentalschoolsversusIRI-onlyandcontrolschools.However,thisdoesnotmeanthatexperimentalschoolsnecessarilyhadhigherenrollmentrates.Thismeasurewascapturedatthebeginningofeveryclassroomobservation,whentheenumeratorwoulddrawamapoftheclassandcountthenumberofboysandgirls.Therefore,thissuggeststhatstudentattendancemaybebetterinexperimentalschoolsoverIAI-onlyandcontrolschools.

Meanclasssizebystatusandgender

Grade1to6teachersweresampledfromthe3PAQUEDinterventionprovinces.Withintheseprovinces,datawascollectedfromrandomlyselectedschoolsinthesub-divisionsasfollows:Table10.Numberofschoolssampledbysub-divisionBandundu Orientale EquateurKikwit(N=13) Kisangani(N=17) Mbandaka(N=14)Bandundu-ville(N=5) Bunia(N=9) Boende(N=5)Gungu(N=5) Isiro(N=4) Gemena(N=4)Masi-Manimba(N=5) Zongo(N=3)Kenge(N=4) Gbadolite(N=5)ApproximatelyhalfofthesamplewasdrawnfromRTI’smidlineevaluationschoolsinordertopermittriangulationofresultsbetweenstudentperformanceandteacherpracticeandknowledge.Theremaininghalfofthesamplewasselectedbasedonschoolclusterdivisions;thatis,ifanexperimentalschoolwasselectedintheRTImidlinesample,thoseschoolsthatwerealreadydesignatedas“clustered”withthoseschoolswerealsoselectedtobeexperimental.ThisisconsistentwithPAQUED’sObjective2theoryofchange,whichposits:

Qualityofteachingimprovedinreading

IncreaseinthenumberofstudentsinDRCwithimprovedreadingskills

Table7.Meanclasssizedisaggregatedbysexpergrade1and2teachersampledbystatus

Status Sex Mean Totalmean

ControlGirls 14

30Boys 16

ExperimentalGirls 19

37Boys 18

IAI-onlyGirls 17

35Boys 18

Table8.Meannumberofstudentsdisaggregatedbysexpergrade3and4teachersampledbystatus

Status Sex Mean Totalmean

ControlGirls 14

27Boys 13

ExperimentalGirls 25

44Boys 19

IAI-onlyGirls 19

39Boys 20

Table9.Meannumberofstudentsdisaggregatedbysexpergrade5and6teachersampledbystatus

Status Sex Mean TotalMean

ControlGirls 12

26Boys 14

Experimental Girls 25 40Boys 15

IAI-only Girls 16 34Boys 18

14

ThistheoryofchangeisbaseduponteacheruseandapplicationofPAQUEDtoolsandresourcesprovided.Morespecifically,under“qualityofteachingimproved”PAQUEDendeavoredtoexplorewhattypeofapproachwouldfosterthisimprovedqualityofteachingandbeyondthis,howtodiscernthepotentialofthisapproachforsustainedandinternalizedimprovementinteachingbeyondthelengthoftheprogram.

Thisstudysearchestoconfirmortodisconfirmthistheoryofchangeandexploresthefollowingquestions:

1. Howaregrade2studentsincontrolandexperimentalschoolsperforminginreadingattheendofschoolyear2013/14?

2. HowareteachersapplyingthePAQUEDinterventionsavailabletothem?3. Howdoteachers’classroompracticeslinktotheirstudents’performanceinreading?*4. Howdoteachers’understandingofeffectivereadinginstructionlinktotheirstudents’

performanceinreading?*5. Howdoteachers’useofthePAQUEDinterventionslinktotheirstudents’performancein

reading?*6. Howdidteachers’classroompracticeschangeover1.5schoolyearsANDarethesechanges

linkedtotheiruseofthePAQUEDinterventionsavailabletothem?7. WhatdoteachersunderstandabouteffectivereadinginstructionANDisthisknowledgelinked

totheiruseofthePAQUEDinterventionsavailabletothem?*Thisquestionislimitedtograde2studentsandteachersonly

Thefirstfivequestionswillbeaddressedinthefirstsectionongrade2studentreadingperformance.Thefollowingtwoquestionswillbeaddressedinthesectionsonteachers’practicesandteachers’knowledge.Forchapter2ongrade3to6teachers,onlyquestions2,6,and7willbeansweredgiventhisstudydidnotcollectreadingperformancedataforgrades3to6students.

ImprovingqualityofteachinginreadingIfwegiveteachersanexplicitreadingprogramtofollowintheirclassroomsANDWegivethemopportunitiesforlearningandreflection(includingcollectivereflection/exchangewiththeirpeers)THENTeacherswillgainanunderstandingofliteracylearningneedsandprocessesamongtheirstudentsANDwillappropriatelyapplyinstructionaltechniquesandstrategiesintheclassroom.ANDwillchangetheirdispositionsvisavisreadingandwritinginstruction

15

Chapter1:Grade1and2studentandteacherresultsPAQUEDreadingprogram:

Thereadingprogramwasdesignedtoprovidestakeholders(mostimportantly,theMinistry)witharobustmodelfortransformingteachers’instructionalpracticesandknowledgeofhowtoteachreading;therebyimprovingstudentperformance.Theprogramincludedthefollowingcomponents:

• Government-validatedStandardsandbenchmarksfromwhichallmaterialsweredesigned• Comprehensiveface-to-faceteacher-trainingoneffectivereadingstrategiesandontheuseof

instructionalmaterials• Teacheractivityguidekeyedtocurriculum,teachingstrategies,andmaterials,presentedsimple

language(French)accessibletotheteachers• Teacherread-aloudbooks(1/weekperclass)• Decodable/Leveledtexts(1/weekperclass)• 30-minuteIAIlessonsfocusedondevelopingreadingskills(1/weekperclass)• Monthlyin-classCoaching/Mentoringbyacoachtrainedinreading• Adequate,dedicateddailyteachingtimefocusedonreading• Teacher-ledweeklymeetingsinaschool-basedlearningcirclefocusedonreading.• Teacher-ledmonthlymeetingswithpeersinamultiple-schoollearningcirclefocusedonreading.• Communitysupport/participationthroughreadingclubsorEspaceCommunautaired’Eveilen

Lecture(ECEL).ThetrainingwasdesignedtolaunchwithinthePAQUEDproject’sfinalyearofoperations.Therefore,itaimedtoquicklyrespondtograde1and2teachers’needsforknowledgeandskills(i.e.,practice)developmentinthreeareas:subjectmatter(literacy),pedagogy(i.e.,thelearningprocess)andinstructionalpracticeinreadingandwriting.Thescopeanddepthofteachers’identifiedneedsatbaselinepresentedachallenge:howcouldtheprogramdevelopteachers’knowledgeandskillsquickly,followingacomprehensive,research-basedpedagogicalapproach,whileatthesametimesupportingrapidimprovementoflearners’skillsinreadingandwriting?PAQUEDaddressedthisproblembydevelopingaseriesofstructuredclassroomteachingandlearningactivitieswhichrepeatedthemselvesweekly,tohelpteachersmasterstrategiesandcontinuetopracticethem.Theseactivitiessharedabasiclessonstructure,beginningwiththedevelopmentoflearners’knowledgeoflettersandsoundsandhowtoapplythisknowledgetodecodeandencodenewwords.Theprogramfacilitatedrapidteachermasteryofinstructionalstrategiesbyrepeatingcertaininstructionalactivitiesseveraltimesduringtheweek.Thisapproachaimedtopromotethedevelopmentofteachers’understandingofliteracylearningneedsandprocessesamongearlygradelearners,ontheonehand,andtheirabilitytoeffectivelyapplyappropriateinstructionaltechniquesandstrategies,ontheotherhand.Thispractice-basedapproachdesignedtofosterteacherchangeviaongoingapplicationandreflection1wasvitaltotheprogram’ssuccess.

1TheapproachisbaseduponSchon’s(1987)“knowledge-in-action”,inwhichteachersdeveloptheknowledgeandskillsforeffectivereadingandwritinginstructionwhileapplyingresearch-basedinstructionalstrategiesintheclassroom.

16

Thetrainingcomponentoftheprogramentailedtwotrainingworkshops,regularmentoringsupportandteacherlearning(i.e.,discussion)forums/learningcircles.Inaninitialweeklongface-to-facetraining,participatingteacherslearnedthebasicstepstocorrectlyexecuteliteracylessonactivitiesvialessondemonstrationsandgroupdiscussion.Theprojectthenreinforcedteachers’skilldevelopmentthroughregularmentoringor“coaching”classroomvisitsandteacher-leddiscussionforums.Atthebeginningofeveryweek,teachersalsoparticipatedinpeer-to-peercoachingandlessonpreparation,tofurtherenhancetheircapacitytocorrectlyexecuteactivitiesandapplytechniquesandstrategiesfordevelopinglearners’skills.Asecondfive-dayfacetofacetrainingworkshopwasgivenmidwaythroughtheyeartohelpteachersbetterunderstand,improveon,andaddtotheactivitiestheyhadbecomecomfortableimplementing.Overall,thesetrainingsandongoingteachersupportcontributedtoteachers’motivationandconfidenceinimplementingthestructuredprogramintheirclassroomsandprovidedthemwithforumsforsharingtheirstudents’progressandcontinuingdifficulties.ThesectionthatfollowspresentsresultsofGrade2studentperformanceafterbenefitingfromoneyearofthereadingprogramintervention.

Grade2studentreadingperformanceAlthoughstudentreadingperformancewascapturedinRTI’sPAQUED2014EndlineofEGRAandEGMAperformance,theGrade2readingdatalargelyfocusedonpre-readingskilltesting,skillsthatwerechosenbyaMinistrycommitteeattestadaptionin2009.Inordertocapturemoreadvancedreadingskillstargetedinthegrade1and2readingprogram,ashortreadingassessmenttoolwasdevelopedbyEDCtomeasurefluency(accuracyandautomaticity)andalphabeticawareness.Thesub-testsemployedwereletteridentification,highfrequencywords,andconnected-textsubtestsadaptedfromexistingEGRAtoolsfromMali.Studentstestedwererandomly+whoparticipatedinthestudy(seesamplingandmethodologyinAnnexA).Thiswastooffertheopportunitytotriangulateteacherpractice,knowledge,andfidelityofimplementationwithstudentperformanceresults.Unfortunately,insufficientnumbersofIAI-onlystudentsweretestedinthisstudy,whichexplainstheiromissionfromthissectionofthediscussionandanalysis.Itshouldalsobenotedthatthenumberofstudentsparticipatinginthispartofthestudyremainslow.However,RTI’smoreextensivePAQUED2014EndlineofEGRAandEGMAperformancealsoshowspositivetrendsindifferentreadingsub-testsforgrade2experimentalschoolstudents.Forexample,grade2experimentalschoolstudentsprogressedsignificantlyintheiridentificationofgraphemesfrombaselinetoendline.Belowisasummaryofstudentresultsdisaggregatedbystatus(experimentalandcontrol)anddisaggregatedbyprovinceandstatus.

17

Table11.Summarydescriptivestatisticsofgrade2studentperformanceinreadingsub-testsbystatus

Sub-task Status Mean SD p-value Cohen’sD Effectsize

Numberoflettersread(outof26)

Experimental(N=169) 20.96 5.4 .000 -2.11 0.73

Control(N=82) 10.2 6.71

Numberofhighfrequencywordsread(outof8)

Experimental(N=169) 4.39 2.63 .000 -1.46 0.59

Control(N=82) 1.21 2.005

Numberofwordsreadinatext(outof26)

Experimental(N=169) 11.24 9.25 .000 -1.27 0.54

Control(N=82) 2.22 5.014

WordsCorrectlyreadPerMinute

Experimental(N=169) 9.8 13.73 .000 -1.03 0.46

Control(N=82) 1.22 3.69

Thetableaboverevealthatgrade2studentsinexperimentalschoolsperformedsignificantlybetterthantheircontrolandcounterpartsinallsub-tasks(p=.000)atendlineinMay2014.Thegraphbelowillustratesthedifferencesinzeroscoresacrosssubtests,thatis,studentswhocouldnotidentifyorreadasingleletterorword.Fornumberoflettersread,allgrade2experimentalstudentswereabletoidentifyatleastoneormoreletterswhereas1.2%ofcontrolstudentswerenotabletoidentifyasingleletter.Forhighfrequencywordreading,only7.7%ofgrade2experimentalstudentswereunabletoreadasinglewordoutofeightwhereas53.7%ofcontrolstudentswereunabletodoso.Intermsofpercentaccuracyinreadingaconnectedtext,only17.2%ofgrade2experimentalstudentswereunabletoreadasinglewordoutofeightwhereasalmost59%ofcontrolstudentscouldnotreadoneword.Figure1.Percentageofstudentswithzeroscoresbystatus

Cont

rol

Cont

rol

Cont

rol

Cont

rol

Expe

rimen

tal

Expe

rimen

tal

Expe

rimen

tal

Expe

rimen

tal

0%

20%

40%

60%

80%

100%

Number of letters read (out of 26)

Number of high frequency words read (out of 8)

Number of words read in a text (out of

26)

Words Correctly read Per Minute

1.2%

0% 7.7%

58.5%53.7%17.2%

55%

15.2%

18

Giventhelargenumbersofzeroscores,itisusefultolookatresultsomittingnon-reader’sscoresinordertocapturearealisticviewofreader’sperformance.Thetablebelowsummarizesresultsomittingthezeroscores.Overall,omittingthesescoresdrivesupmeanscoresslightlyineachsubtestwiththeexceptionofnumbersoflettersreadforexperimentalschoolstudentsforwhomnonehadzeroscoresinthatsubtest.Despiteomittingzeroscores,experimentalstudentsstillsignificantlyoutperformedtheircontrolcounterpartsacrosssubtests(p=.000).Table12.Summarydescriptivestatisticsofgrade2studentperformanceinreadingsub-testsbystatusomittingzeroscores

Sub-task Status Mean SD p-value Cohen’sD Effectsize


Experimental(N=169) 20.96 5.4 .000 -1.71 0.71

Control(N=81) 10.32 6.65


Experimental(N=156) 4.75 2.4 .000 -1.36 0.56

Control(N=38) 2.61 2.25

Numberofwordsreadinatext(outof

26)

Experimental(N=140) 13.57 8.45 .000 -1.55 0.61

Control(N=34) 5.35 6.66


Experimental(N=117) 11.56 14.22 .000 -.99 0.44

Control(N=27) 2.72 5.17

StudentperformancebyprovinceGrade2experimentalschoolstudentperformanceinvariedsignificantlyfromprovincetoprovince.Thetablebelowshowsasummaryofscoresacrossallsubtestsforeachprovince.Table13.Summarydescriptivestatisticsofgrade2studentperformanceinreadingsub-testsbyprovinceandstatus

Province Sub-task Status Mean SD p-value Cohen’sD Effectsize

BANDUNDU (N= 107)


Experimental(N=81) 19.26 6.23 .000 -1.69 0.65*

Control(N=26) 11.77 5.631


Experimental(N=81) 3.9 2.9 .000 -1.29 0.54*

Control(N=26) 1.31 2.478

Numberofwordsreadinatext(out

of26)

Experimental(N=81) 7.67 8.6 .002 -0.78 0.36

Control(N=26) 3.04 5.67

WordsCorrectly Experimental 4.75 8.6 .001 -0.75 0.35

19

readPerMinute (N=78)Control(N=17) 1.04 1.5

EQUATEUR (N=55)


Experimental(N=32) 20.69 .403 .000 -2.51 0.78*

Control(N=23) 9.61 6.31


Experimental(N=32) 4.31 1.91 .000 -1.97 0.7*

Control(N=23) 1.26 1.3


of26)

Experimental(N=32) 12.28 7.78 .000 -2.7 0.8**

Control(N=23) 1.04 1.64


Experimental(N=19) 5.77 3.51 .000 -3.14 0.84**

Control(N=18) 0.33 0.35

ORIENTALE (N=89)


Experimental(N=56) 23.57 3.42 .000 -3.2 0.84**

Control(N=33) 9.36 7.67


Experimental(N=56) 5.14 2.44 .000 -1.91 0.68*

Control(N=33) 1.09 2.07


of26)

Experimental(N=56) 15.82 8.85 .000 -1.85 0.67*

Control(N=33) 2.39 5.9


Experimental(N=41) 21.27 17.47 .000 -1.82 0.67*

Control(N=25) 1.99 5.54

*effectsizeismedium**effectsizeislargeFortheletterreadingandhighfrequencywordssub-testvariationbetweenprovincesremainedstable.However,forconnectedtextreading,inexperimentalschoolsinOrientale,studentssignificantlyoutperformednotonlytheircontrolcounterpartsinthatprovincebutalsotheirexperimentalcounterpartsinBandunduandEquateurintheirpercentaccuracyandintheirnumberofwordsreadperminute(p=.000).Thiscanbeexplainedbyseveralfactorsthatwerefoundtopositivelyandsignificantlycorrelatewithstudentresults.Theseareteachers’fidelityofimplementationofthereadingprogramandteachers’knowledgeandclassroompractices.Thesewillbediscussedfurtherbelow.

20

Figure2.Meanscores,byprovinceandstatus

*performanceforOrientaleexperimentalschoolsforWCPMisp=.000

StudentperformancerelativetobenchmarkTheDRCgovernmentsetprovisionalbenchmarksfordifferentreadingcompetenciesinFebruary2012forbothnationallanguageandFrench.Becausestudentsingrade1and2areintendedtolearntoreadinnationallanguages,nobenchmarksweresetforreadingfluencyinFrenchforgrade2.However,benchmarksweresetforgrade3.Thefigurebelowshowstheproportionofexperimentalandcontrolschoolswhoarebelowthebenchmark,atbenchmark,andabovethebenchmark.12%ofgrade2experimentalstudentsreadabovefluencybenchmarkforFrenchsetforgrade3,9%readatbenchmarkand78%readbelowthebenchmark.Incontrasttothis,only2%ofgrade2controlstudentsshowedtoreadatbenchmarkforfluencyand98%readbelowbenchmark.

10.52

14.5616.41

6.5 5.78

21.27

6.6

2

7.9

20.5

7.1

0

5

10

15

20

25Num

bero

fwords

Experimental

Control

Numberofwordsreadcorrectlyinaconnectedtext(outof26)

Numberofwordsreadcorrectlyperminute

21

Figure3.Students’performanceinWCPMagainstnationalbenchmarkssetfor3rdgrade

StudentperformanceandPAQUEDinterventions:Asmentionedearlier,analysisrevealedstudentperformancetobepositivelyandsignificantlylinkedtoseveralfactorstoteachers’fidelityofimplementationofthereadingprogram,teachers’knowledgeandteachers’classroompractices.Thedataspecificallyrevealedthatstudentperformancewassignificantlylinkedtotwofactors:ratesofIAIlistenershipandtheirteachers’fidelityofapplicationofthereadingprogram.BelowisanoutlineofthedegreetowhichteachersappliedorparticipatedincertainPAQUEDinterventionsavailabletothem.Table14.Fidelityofimplementationdataforgrade1and2teachersbyschooltreatmentstatus

ExperimentalN=69

IAI-onlyN=96

Fidelityofimplementationofreadingprogramrate

ParticipationinCPD2 IAIlistenership IAIlistenership

Mean St.Dev. Mean St.Dev. Mean St.Dev. Mean St.Dev..88 .13 .71 .15 .86 .067 .51 .28

Overall,experimentalteachersusedand/orfollowedthevariouselementsofthePAQUEDinterventionasdesigned.Despitethis,experimentalteachers’employmentofonlyoneoftheseinterventionscorrelatedsignificantlywithstudentperformance:fidelityofimplementationofthereadingprogram.Linearregressionshowedthat21%ofthevariationinmeanstudentperformanceinconnectedtextreadingcorrelatedpositivelyandsignificantly(p=.016,d=1.01,ES=0.45)withtheirteachers’fidelityofimplementationofthereadingprogram.

2CPD=Continuingprofessionaldevelopment.Thisisacompositescoresincludingratesofteacherparticipationinschool-basedandclusterbasedmeetingsavailabletothemandnumberofmonthlycoachingvisitsfromfacilitatorsorPAQUEDstaff.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Studentsreading<15WCPM

Studentsreading16to29WCPM

Studentsreadingabove29WCPM

Experimental

Control

21%ofstudentsattheendof2ndgradeinexperimentalschoolsattainedorsurpasedtheWCPMbenchmarkssetforFrenchreadingin3rdgradecomparedto2%ofstudentsincontrolschools.

78%

98%

9%

2%

12%

0%

22

Inadditiontothis,23.8%ofthevariationinstudent’smeanwordscorrectperminutecanbeexplainedbyteachers’followingthereadingprogramthewayitwasdesigned(p=.021,d=1.09,ES=0.479).

Thiscorrelationpointstotheimportanceofteachersfollowingaprogramasitisdesigned.Onaverage,teachersinexperimentalschoolsfollowed88%ofthereadingprogramactivitiesastheyweredevised.ThiswashigherforOrientaleandEquateurprovinceswhereteachersshowedtoapplymorethan90%ofthereadingprogramappropriately.Duetolownumbersofteachersasmatchedwithstudentstested,correlationsoffidelityofimplementationofthereadingprogramandmeanstudentperformancecannotbepresentedbyprovince.

00.10.20.30.40.50.60.70.80.91

0 5 10 15 20 25Meanpe

rcen

tageofreading

program

FOI

Students'meannumberofwordsreadcorrectly

Figure4.Teacher’sfidelityofimplementation(FOI)ofthereadingprogramandtheirstudents'meanperformanceinnumberofwordsreadcorrectly

R2=.21sig=.016

00.10.20.30.40.50.60.70.80.91

0 10 20 30 40Meanpe

rcen

tageofreading

program

FOI

Students'meanWCPM

Figure5.Teacher’sfidelityofimplementation(FOI)ofreadingprogramandtheirstudents'meanWCPM

R2=.238sig=.021

23

OtherPAQUEDinterventionssuchasIAIlistenershipwereassumedtohaveanimpactonstudentperformanceinreading.Thisisbecausetheprogramswerebroadcastdirectlyintotheclassroomandweredesignedtoengagethestudentsjustasmuchastheyweretoprovidecontinuoustrainingtotheteachers.Unfortunately,linearregressionanalysisfoundnosignificantcorrelationbetweenIAIusageandstudentperformance.Apossibleexplanationforthislackofsignificantcorrelationistwofold:thelackofdataonstudentabsenteeismandtimeontask,andtheverylowexposuretoIAIprogrammingthatwascalledforingrade1and2.StudentabsenteeismisalsoamajorissueintheDRCbecauseattendanceisdirectlylinkedtothestudent’spaymentoffees(examfees,enrollmentfees,etc).Whenstudentsarenotabletopaythesefees,theyarebarredfromattendingschool.Therefore,evenifteachersarepresenttolistentotheIAI,itisnotcertainthateverystudentbenefitedequally.Secondly,ingrade1and2,onlyone30-minuteprogramwasprovidedforreadingperweek.Onaverage,teachersinexperimentalschoolsshowedtousemoreoftheIAIprogramsavailabletothemthantheirIAI-onlycounterpartsand,variationofusageforthesetwogroupsofteacherswasalsomuchlowerforexperimentalschoolteachers.ThisislikelyduetoIAI’sintegrationinthereadingprogramweeklyactivitycalendar.Still,giventhelowdosageofIAIprogrammingperweek,itislittlesurprisethatIAIlistenershipwasnotsignificantlycorrelatedwithstudentreadingperformance.

However,becausethissamplesizeisfairlysmall,conclusionsaredifficulttodraw.The2014PAQUEDEGRA/EGMAreportproducedbyRTI,alargerscalestudy,establishestherelationshipbetweenstudentperformanceonthegraphemerecognitionsubtaskandPAQUEDinterventions.Thisreportshowedthatteacherparticipationincontinuingprofessionaldevelopment(CPD)activitieswasfoundtohavesubstantialimpactonstudentperformanceinthissub-task(p=0.0387).Thiscorrelationisconsistentwithteachers’assertionsinfocusgroups,whichrevealedthattheybelievedIAItobeausefultoolfororallanguagedevelopmentandengagingstudentsinnumerouspre-readingactivitieslikestretchingoutwordstohearindividualsounds,cuttingupwordsbysyllable,etc.Takingthisalltogether,thissuggeststhatIAIwhenusedregularlyandinthecontextofarobustreadingprogram,canbearpositiveresultsandprovidesoundmodelsofteachingreading.ThissuggestionisconsistentwithconclusionsdrawnbytheMinistryReadingCommission’sanalysesofdatacollectedfromvarioussources(EDC,RTI,andMukendi,2014).

Studentperformanceandteacherpractices:Inadditiontoteachers’applicationofPAQUEDinterventions,itisalsointerestingtobetterunderstandhowteachers’practiceandtheirknowledgeofteachingreadinglinkstostudentreadingperformance.AccordingtoPAQUED’stheoryofchange,improvementinteacherclassroompracticesconcerningliteracywillinfluencestudentperformance.Researchsuggeststhatteachers’explicitmodelingandinstructionofthecomponentskillsofreadingandwritingwillbenefitstudents’readingacquisition.Inthereadingprogram,activitiesofteninvolvedacombinationofreadingandwritingtodevelopskillslikephonologicalandalphabeticawareness,fluency,vocabularybuildingandcomprehension.However,linearregressionanalysisindicatedthatonlyteachers’applicationofvocabularyactivitieswasstronglycorrelatedwithstudents’readingperformanceoncertainsub-tests.Thegraphbelowshowsthat25.1%ofthevarianceinstudents’meanfluency(WPCM)canbeexplainedbyateachers’applicationofvocabularyactivitiesintheclassroom(p=0.021,d=1.13,ES=0.49).

24

Whilevocabularyactivitiesmaynotseemdirectlylinkedtoimprovingreadingfluency,thespecifictypesofvocabularyactivitiesthatcorrelatedsignificantlywithstudentperformanceexposestudentstotexttherebyprovidingopportunitiesforstudentstodevelopfamiliaritywithsightwordvocabulary.Forexample,ateacher’sapplicationofpre-readingactivitieslikemakingpredictionsanddiscussingillustrationsandnewvocabularyembeddedwithinatextexplained20.1%ofthevarianceinstudent’sfluency(WCPM)(p=.021,d=0.978,ES=0.44).Suchactivitiesinevitablyengagestudentswiththereadingofnewwords,whichcanbelinkedtodevelopingdecodingskillsnecessaryforbuildingfluency.

Studentperformanceinreadingaconnectedtextwasalsosignificantlycorrelatedwiththeirteacher’sapplicationofengagingstudentsincorrectingtheirspelling.Forexample,thegraphbelowdemonstratesthatexperimentalteachersengaginginthepracticeofaskingstudentstoengageincorrectingtheirinventedspellingscanexplain30.9%ofthevariationinstudents’meanWCPM(p=.009,d=1.3,ES=0.55).Inthereadingprogram,studentsareaskedencodewordsthatcontainaphonicspatternstudiedthatweek.Thisistohelpthemapplytheirknowledgeofletter-soundrelationships.Whenteachersaskstudentstocorrecttheirspellings,thissuggeststhatstudentsarebroughttoreinforcetheseletter-soundrelationshipsthatwillhelpthemdecodewordsthatcontainthosesamepatterns.

0

5

10

15

20

25

30

0 10 20 30 40

Meanpe

rcen

tageoftim

eallocatedto

vocabu

laryactivities

Students'meanWCPM

Figure6.Experimentalteacher’sapplicationofvocabularypracticesatendlineandtheirstudents'meanWCPM

R2=.251sig=.021

25

Generally,itwasexpectedthatanalysiswouldhaverevealedmoresignificantlinksbetweenteacherpracticeobservedatendlineandmeanstudentperformance.Thismaybeduetothelimitednumberofreadingskillstested.Still,thoselinksthatemergedfromthedatapointtotheimportanceofpre-readingactivitiesandtostudentsengagingincorrectingtheirownwriting.

Studentperformanceandteacherknowledge:PAQUED’stheoryofchangealsohypothesizesthatteachers’knowledgeofteachingreadingandwritingplayjustasimportantofaroleinpredictingstudentreadingoutcomesasclassroompracticedoes.Inthiscase,teachers’knowledgearemeasuredbyteachers’answerstoquestionsaboutspecificpracticesandtheirutilityandsuitabilityforteachingreadingandwritingtograde1and2students.ThisisconsistentwiththeteacherresultsinthefollowingsectionthatshowdirectlinksbetweenthePAQUEDinterventionsandteachers’understandingabouthowstudentslearntoread.Therefore,itisinterestingtoseewhatpredictorsofteacherknowledgeanddispositionsseemedtoexplainthevariationinstudentreadingoutcomes.

Overall,experimentalteachers’totaldemonstratedknowledgeintheendlineinterviewwereshowntobesignificantlyandpositivelycorrelatedwiththeirstudent’sperformanceinreadingofaconnectedtext(p=.045,d=1.096,ES=0.48)andtheirfluencyindoingso(p=.024,d=1.166,ES=0.5).Thisisdemonstratedinthegraphbelowshowingthat31.3%ofthevariationinstudents’meanWCPMisexplainedbytheirteachers’overallknowledgeofteachingreading.

0

0.5

1

1.5

2

2.5

3

3.5

0 10 20 30 40

Meanpe

rcen

tageoftim

eallocatedtoP4

Students'meanWCPM

Figure 7.Experimentalteachers'applicationofP4atendlineandtheirstudents'meanWCPM

R2=.309sig=.009

26

Analyzingteachers’knowledgeofteachingvariouscomponentskillsofreadingandwritingitwasfirstfoundthatteachers’knowledgeofteachingfluencycorrelatedsignificantlywithstudents’meanreadinghighfrequencywords(p=.032,d=1.1,ES=0.48),connectedtext(p=.019,d=1.22,ES=0.52)andtheirfluency(p=.000,d=2.25,ES=0.75).Thegraphbelowshowsthat57.4%ofthevariationsinstudents’meanWCPMispredictedbytheirteachers’knowledgeofhowtoteachfluency.

Thisissupportedbyitemanalysiswhichshowsexperimentalteachers’responsetoQuestion3.1(seebelow)“isitalwaysimportanttoreadforstudentssotheycanlearntoread”,wasnegativelyandsignificantlycorrelatedwithstudent’sreadingofhighfrequencywords(p=.028,d=1.09,ES=0.48)andthemeanpercentageofwordscorrectlyreadinatext(p=.019,d=1.18,ES=0.51).Thisispositiveasitdenotesthatteachers’allowanceoftheirstudentstoreadontheirowndoescorrelatewithstudents’readingperformance.Thesefindingalsosuggeststhatteachersarepassingthebatontostudents,

0102030405060708090100

0 10 20 30 40Meanpe

rcen

tageofk

nowledg

eof

literacyinstruction

Students'meanWCPM

Figure8.Experimentalteacher’stotalmeanknowledge ofliteracyinstruction andtheirstudents'meanWCPM

R2=.313sig=.024

0102030405060708090100

0 10 20 30 40Meanpe

rcen

tageofk

nowledg

eof

teaching

flue

ncy

Students'WCPM

Figure9.Experimentalteacher’stotalmeanknowledge ofteachingfluencyandtheirstudents'meanWCPM

R2=.574sig=.000

27

incorporatingthegradualreleasemodellaidoutinthereadingprogram,andallowingstudentstotakeresponsibilityfortheirownlearning.Thisisfurthersupportedbythefocusgroupfindings,whichpointtoteachers’higherexpectationsoflearners’readingcapabilitiesespeciallytoperformdecodingandotherreadingandwritingtasksindependently.

Teachers’knowledgeofteachingwritingandintegratingwritingintotheirreadinglessonswasalsopositivelyandsignificantlycorrelatedwithstudentabilitiestoreadaconnectedtext(p=.027,d=1.07,ES=0.47)andtheirWCPM(p=.015,d=1.28,ES=0.75).

0

10

20

30

40

50

60

0

2

4

6

8

10

12

Yes No

Percentageofwordsreadcorrectly

R2=.268sig=.019

Freq

uencyofte

ache

rs'respo

nses

Meanpercentageofw

ordsreadcorrectlyFigure10.Experimentalteachers'responsestoQuestion3.1andtheirstudents'mean

readingaccuracy(percentageofwordsreadcorrectlyinatext)

0102030405060708090100

0 10 20 30 40Meanpe

rcen

tageofk

nowledg

eof

teaching

writing

Students'meanWCPM

Figure 11.Experimentalteacher’stotalmeanknowledge ofteachingwritingandtheirstudents'meanWCPM

R2=.302sig=.015

28

Thisfindingisconsistentwithseveralkeyreadingprogramactivitiesthatencouragestudentstopracticewritingusingthephonicspatternsthey’velearnedordrawingandwritingtheirreactionstoaread-aloudtext.Researchalsodenotestheimportanceofstudentshavingopportunitiestoengagewithwritingasitsimultaneouslyaidsinsolidifyingtheletter-soundrelationshipsandspellingpatternsstudiedinadditiontoaidingincomprehensionofatextread.

Thislinkbetweenteachers’knowledgeofintegratingreadingandwritingintotheirlessonsissupportedbytheirresponsetoQuestion1.3.(seebelow)Itisbettertoteachreadingandwritinginthesamelessonratherthaninseparatelessons,whichwasfoundtobesignificantlycorrelatedwithstudentperformanceonallsub-tests.Forexample,thegraphbelowshowsthat33.3%ofthevariationinstudents’meanabilitytoidentifyletters(p=.006,d=1.38,ES=0.57)and39.9%ofthevariationinstudents’meanWCPM(p=.004,d=1.58,ES=0.62)ispredictedbyteachers’responsestoQuestion1.3onintegratingreadingandwriting.

Figures12.Correlationsbetweenteachers’responsestoQuestion1.3andstudentperformance

Relatedtoteachers’knowledgeofwriting,teachers’expectationsoftheirstudents’writingcapabilitiesalsorevealedtobesignificantlycorrelatedwithstudentreadingperformance.Inexperimentalschools,36.6%ofthevariationintheirstudents’readingofaconnectedtextwasexplainedbyteachers’negativeresponsetoQuestion5.2“mystudentshaveahardtimelearningtowrite.”(p=.004,d=1.48,ES=0.6).Thisrelationshipisdemonstratedinthegraphbelow.

0

5

10

15

20

25

0

2

4

6

8

10

12

14

16

Yes No

Numberoflettersread

R2=.333sig=.006

Freq

uencyofte

ache

rs'respo

nses

Meanlettersread

Experimentalteachers'responsestoQuestion1.3andtheirstudents'meanletterreading

0

2

4

6

8

10

12

14

16

0

2

4

6

8

10

12

14

Yes No

WCPM

R2=.399sig=.004

Freq

uencyofte

ache

rs'respo

nses

MeanW

CPM

Experimentalteachers'responsestoQuestion1.3andtheirstudents'meanWCPM

29

ThisfindingissupportedbyextensiveeducationresearchbyStronge(2010),whichpointstoteacherexpectationsoftheirstudentsastheprimarypredictorofstudentperformance.Thefindingsabovearealsosupportedbyfocusgroupandextensionquestionresponsesinthatexperimentalteachersexhibitedbetterunderstandingoftheimportanceofintegratingreadingandwritingactivitiescitingthedirectrelationshipbetweenencodinganddecodingandhowallowingstudentstoexperimentwithwritinghelpdevelopstudentcapacitytoreadandwriteeffectivelyandindependently.Inaclassicallyauthoritarianeducationenvironment,thisisbothprofoundandexciting.

Lastly,experimentalteachers’opinionsofthelanguageinwhichtheirstudentslearntoreadbetteralsocorrelatedsignificantlywithstudentresults.However,theycorrelatedinawaythatdoesnotcorroboratewiththeresearchasmoreexperimentalteachersassertedthattheirstudentslearntoreadmoreeasilyinFrench(asecondlanguage)ratherthanintheirmothertongueovertheirIAI-onlyandcontrolpeers.Thisisdemonstratedinthegraphbelowwhichshowsthat31.2%ofstudents’meanWCPMisexplainedbytheirteachers’negativeresponsetothequestionItiseasierformystudentstolearntoreadinFrenchratherthaninmothertongue(p=.013,d=-1.31,ES=0.55).

0

2

4

6

8

10

12

14

16

0

2

4

6

8

10

12

14

16

Yes No

Numberofwordread

R2=.366sig=.004

Freq

uencyofte

ache

rs'respo

nses M

eanwordsreadinatext

Figure13.Experimentalteachers'responsestoQuestion5.2andtheirstudents'meanreadingaccuracy

30

ItisspeculatedthatthereasonwhymoreexperimentalteachersrespondedinwaytheydidmaybeattributedthereadingprogrambeingconductedinFrench.Becausetheirstudentsbecamebetterreadersasaresultoftheprogram,teachersmaybelinkingtheirstudent’sprogressinreadingtothelanguagetheyarelearningtoreadin(theywerenotaskedtoteachinmothertonguesodonothaveapointofreferenceforstudents’abilitytodoso)..

Grade1and2Teachers’knowledgeofliteracyinstructionInadditiontolinkingteacherknowledgeandpracticetostudentperformance,thisstudyalsoendeavoredtobetterunderstandhowteachers’knowledgeofteachingreadingmightbelinkedtotheiruseofthevariouselementsofthePAQUEDinterventionsavailabletothem.Inthissection,teachers’knowledgeofreadingindifferentgroups(experimental,IAI-only,andcontrol)willbediscussedusingdatafromanextensiveface-to-faceinterviewconductedatendlineinadditiontofocusgroupdata.Theinterviewincludedquestionsonspecificreadingandwritinginstructionalpracticesandtheirutilityandsuitabilityforteachingreadingandwritingtograde1and2students.Certain“extension”questionsaskedthatteachersprovidejustificationsandaself-livedclassroomexampletosupporttheiranswer.Thiswaspartiallyusedforensuringreliabilityofteachers’answersbutalsotoobtainfurtherinsightintoteachers’responses.TheinterviewtoolscanbefoundinAnnexB.Focusgroupdatawasderivedfromaseriesoffocusgroupsconductedafterdatacollectiononteacherknowledge,practice,andstudentperformance.Focusgroupquestionsaskedexperimentalteacherstodiscusshowtheywouldintroduceanewtextorguidetheirstudentsinhowtodecodeanewword.Teachers’examplesprovidedrichinformationonhowdeeplypracticesandstrategiesembeddedwithinreadingprogramemergedfromtheirclassroomexamples.Thetablesandfiguressummarizethestatusofteachers’knowledgeatendlinegroupedbycomponentskillandpulloutspecificitemsrelativetotheteachingofcomponentskills.Theresultspresentedbycomponentskillrepresentthemeanpercentageofagreementtoagroupofquestionsclassifiedbycomponentskill.ThecompositionofquestionsbycomponentskillscanbefoundinAnnexB.Asevery

024681012141618

0

2

4

6

8

10

12

14

Yes No

WCPM

R2=.312sig=.013

Freq

uencyofte

ache

rs'respo

nses

MeanW

CPM

Figure14.Experimentalteachers'responsestoQuestion1.2andtheirstudents'meanWCPM

31

questionsposedcouldbeansweredas“yes”or“no”,themeanswerecalculatedbasedontheseresponses.Forexample,experimentalteachers’frequencyofagreementtoquestionsonhowtoteachvocabularyamountedto89%meanagreementascomparedto74%forIAI-onlyandcontrolteachers.TheresultsoftheindividualquestionsoutlinedinTable17representthepercentageofagreementforeachquestionacrossdifferentteachergroups.Thedecisiontopresentteachers’knowledgedatabycomponentskillwasdeliberate,sothatfindingsforthissectionwouldbeorganizedinthesamewayastheteacherpracticeandstudentperformanceresults.However,itshouldbenotedthatthisdivisionbycomponentskillisnotnecessarilyconsistentwithhowteachersthinkaboutteachingreadingandwriting.Thatis,teachersmaynotthinkaboutteachingvocabularyandcomprehensionseparatelynormaytheythinkaboutteachingalphabeticawarenessandphonemicawarenessseparately.Rather,focusgroupdatasuggestthattheyarecomingtothinkaboutteachingreadingandwritingastheuseofspecificactivitiesthatbuildseveralcomponentskillsinreading.Forexample,thewordstudyactivitynotonlydevelopsastudent’sabilitytodifferentiatespellingpatternsbysoundandorthographybutalsobuildstheirvocabularyastheylearnthenewwords.Overall,thefindingsbelowgenerallyrevealthatteachersinexperimentalschoolsexhibitknowledgethatisconsistentwithevidence-basedresearchonhowtoteachreadingandwritingeffectively.Inaddition,14.3%ofteachers’overallknowledgeofteachingreadingwasfoundtobepredictedbyexperimentalteachers’fidelityofapplicationofthereadingprogram(p=.047,d=0.8,ES=0.37).

Mostofthemeandifferencesinteachers’knowledgeacrossexperimentalandcontrolgroupswerealsofoundtobesignificant.Unfortunately,IAI-onlyteachersdidnotseemtodifferentiatesignificantlyintheirknowledgeofteachingcomponentskillsofreadingascomparedtotheircontrolteachercounterparts.Thesedifferencesorlackthereofarefurtherexploredinthediscussionfollowingthe

40

50

60

70

80

90

100

40 50 60 70 80 90 100

Meanpe

rcen

tageofteachersF

OI

Teachers'meanpercentagesofknowledge

Figure15.Experimentalteachers'fidelityofimplementation(FOI)ofreadingprogramandtheirtotalmeanknowledge ofliteracyinstructionatendline

R2=.143sig=.047

32

tablesandlinkstoteachers’knowledgeandvariousPAQUEDinterventionswillbeestablishedusingfidelityofimplementationdata.Table15belowprovidesasummaryoverviewofteachers’knowledgeofdifferentdomainsofreadinginstructionandindicateswhetherthedifferenceinknowledgeissignificantincomparisontothecontrolgroup.Overall,experimentalteachersseemtohavesignificantlymoreknowledgeabouthowtobestteachcertainreadingdomainsnotablyfluency,vocabulary,andcomprehensionincomparisontotheircontrolcounterparts.

Table15.Summaryofthegrade1and2teacherknowledgeresultsofcomparisonofmeansatendlinebetweengroups(higherpercentagesconvey“sound”knowledge) PAQUED

CONTROL(n=61) Experimental(n=37) IAI(n=64)Phonemic/PhonologicalandAlphabeticawareness

94% 88% 91%

Fluency 85%*** 76% 76%Vocabulary 89%** 74% 74%Comprehension 96%*** 88% 84%Writing 71% 66% 65%Total 88%*** 78% 79%**ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001Presenteddifferently,thefigurebelowillustratesthedifferencesinknowledgeofreadinginstructionacrossgroups.

0%

20%

40%

60%

80%

100%

Experimental(n=37)

IAI(n=64)

CONTROL(n=61)

Figure16.Teachers'meanknowledgeofteaching readingbycomponentskill,atendline

percen

tage

ofmeankn

owledg

e

33

Table16belowpullsoutspecificquestionsfromtheinterviewtoolwhichproducedsignificantlydifferentresponsesbetweengroups.Forexample,experimentalteachers’responsestoquestion1.1onexpectations:Mostofmystudentshaveaneasytimelearningtoread,werefoundtobestatisticallydifferentfromtheircontrolcounterparts.Table16.Itemanalysisofthegrade1and2teacherendlineknowledgeresults(means)comparisonbygroups(percentagesreflectagreement) PAQUED

CONTROL(n=61) Experimental(n=37) IAI(n=64)1.1Mostofmystudentshaveaneasytimelearningtoread

51%agree*** 30%agree 18%agree

1.2.MystudentslearntoreadmoreeasilyinmothertonguethaninFrench.

65%* 81% 85%*

2.1Beforereadinganewtext,itisusefultohaveadiscussionwiththewholeclasstodiscusswhatyourstudentsalreadyknowaboutthetext’stheme?

97% 85% 88%

2.2Itisusefultodiscussnewvocabularywithmystudentsbeforetheyreadatext.

78%** 52% 47%

3.1Itisimportanttoalwaysreadbeforemystudentssotheycanlearntoread.

63.9%*** 91% 96.8%

4.1Itisimportanttoallowstudentstotalkamongsteachotheronwhattheyhavereadtohelpthemunderstandatext.

89%* 78% 72%

4.2Afterhavingreadatext,itisimportanttoaskstudentstoexplainwhatthey’veread.

97%*** 82% 75%

4.4Itisimportanttoaskstudentsquestionsafterhavingreadatext.

100%* 92% 91%

4.5Studentsarecapableofsayingwhattheylikedordislikedaboutatextread.

91%* 82% 75%

5.2.Mystudentshavealotofdifficultylearningtowrite.

35%*** 61%* 79%

*ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.05**ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001Theresultspresentedintheabovetablesandfiguresarediscussedbydomainofreadinginstructionbelow.Attemptstoconnectteachers’knowledgeofreadinginstructiontotheiruseofthePAQUEDinterventionsavailabletothemwillalsobediscussed.Finally,teachers’responsestoextensionquestionswillalsobepresentedsoastoprovideamoreconcretevisionastohowteachersthinkabouthowtobestteachtheirstudentstoread.

34

Phonemic,phonologicalandalphabeticawareness:ThePAQUEDreadingprogramandIAIgrade1and2programplacedemphasisonthedevelopmentofphonemic,phonologicalandalphabeticawarenessinthelowergrades.Thisemphasiswasselectedtoaddressthefindingsoftheproject’sbaselineandmidlineEarlyGradeReadingAssessments,whichrevealedthatstudentsingrade2hadgreatdifficultieswithprovidinginitialsoundsinspokenwordsandwithcorrectlyidentifyingletters,skillsthatareessentialprecursorstolearninghowtodecodethewrittenword.

Experimentalschoolteachers’knowledgeofphonemicawareness,phonologicalandalphabeticawarenessatendlinedidnotdiffersignificantlyfromtheirIAI-onlyorcontrolcounterparts,andnoneofthePAQUEDinterventionswerefoundtocorrelatesignificantlywiththeseendlineteacherknowledgedata.Tobetterunderstandwhatteachersmeanwhentheyrespondto“yes”or“no”questionsontheimportanceforstudentstodeveloptheirphonemic,alphabeticandphonologicalawareness,teachers’responsestoanopen-endedfollow-upquestionarealsopresented.Whenaskedtoprovideconcreteclassroomexamplesofhowtheyhelpedtheirstudentsbuildthesecomponentskills,teachersreflectedmanyoftheactivitiesoutlinedinPAQUEDtools.Theexamplescitedincludedalphabeticawarenessactivities:

“mystudentsrecitetheletter-song(lacomptinedeslettres)whileIpoint”(N=5)“Ishowmystudentshowtodecodenewwordsbytyingtheindividualletterstotheirsounds(letter-by-letterreading)”(N=6)“Iremindmystudentstorememberthelettersoundstohelpthemreadanewword”(N=2)

andphonemicawarenessactivities:

“FromasoundthatIgive,studentscanfindotherwordsthatcontainthatsound.”(N=5)“IhelpmystudentstretchoutwordssotheycanhearallofthesoundsinthewordorIdoitbysyllable.”(N=13)

Teachersalsopointedtophonologicalawarenessactivitiesashelpfultohelpingtheirstudentswrite:

“Ifastudentknowsasoundthatalettermakes,theycanalsowriteit.Intheword‘mbenza’,ifthestudentsknowthatthebeginningsoundismadeupofm-b,theycanwriteit.”(N=3)

Theseexamplesdirectlyrelatetothetypeofactivities(suchaswordstudyandletter-soundstudy)andstrategies(suchaswordstretching)coveredintheIAIprogramsaswellasinthereadingprogramguidethatwasfollowedbyexperimentalschoolteachersonadailybasis.Suchfindingsareencouraging,astheyindicatethatteachersarebeginningtointernalizeandexplainwhattheteachingofthesebuildingblockskillslookslikeintheclassroom.Fluency:AnotherkeycomponentskillthereadingprogramandIAIprogramsaimedtodevelopisfluency.Fluencyisdefinedbyone’sabilitytoreadwithaccuracy,automaticity,andproperintonation.Fluentreadersmovebeyondletter-by-letterorsyllable-by-syllabledecoding(whichtakefocusandconcentration)torecognizechunksoftextandhencereadmorequicklyandaccurately.Researchpointstotheimportanceofdevelopingfluencyinorderforthereadertobeabletofocusmoreoncomprehendingwhats/hereadsandlessonthemechanicsofdecodingto(Rasinki,2006).Thereare

35

severalinstructionalstrategiesthatteacherscanemploytodevelopthisskill.SomethatareoutlinedinthePAQUEDreadingprogramactivitiesandIAIprogramsincludeteacherspointingtowordstohelpmovestudent’seyesfasterfromwordtoword;teachersdoingflashcardactivitieswithhighfrequencyandpreviouslystudiedwords;andteacherssimplyprovidingmoreopportunitiesforstudentstopracticereading.

Afterapplyingthesefluencyactivitiesandstrategiesintheclassroom,whatdidteachersretainas“sound”practicefordevelopingthisimportantskill?Table16summarizesthatteachers’knowledgeofdevelopingfluencywassignificantlygreaterthantheirIAI-onlyandcontrolcounterparts(p=.001,d=-0.95,ES=0.43).Thisispositiveandwasconsistentinthediscussionsundertakenwithexperimentalschoolteachers:

“Itseemslikestudentsneedmoretimetopracticereadinginordertoreadfasterandbetter.”(N=14)

Thisshowsthatteachersarebeginningtorecognizetheimportanceofpracticeforstudentstobecomebetterreaders.AlsointerestingtonoteishowPAQUEDinterventionsmayhavepredictedteachers’knowledgeofteachingfluency.Linearregressionanalysisshowedthatexperimentalteachers’adherencetothereadingprogramactivitiesexplained12.6%ofthevarianceinteachers’responsestofluencyquestions(p=.046,d=0.75,ES=0.35).

Thissignificantcorrelationsuggeststhattheexplicitfluency-buildingactivitiesinthereadingprogrammayhavecontributedtodevelopingteachers’understandingoftheimportanceofapplyingsuchactivitiestobuildtheirstudent’sreadingskills.Thisisconsistentwiththestudentperformanceresultsdiscussedabove,whichlinkteachers’knowledgeofteachingfluencytostudentreadingfluencyofaconnectedtext(p=.000,d=2.25,ES=0.75).

Vocabulary:Vocabulary(particularlyFrenchvocabularydevelopment)wasalsoakeycomponentofthereadingprogramandIAIprograms.Vocabularydevelopmentisespeciallyimportantinthecontextofsecondlanguagelearning,asisthecaseintheDRC.AsonePAQUEDIAI-onlyteacherstates:“Astudent

40

50

60

70

80

90

100

40 50 60 70 80 90 100

Meanpe

rcen

tageofteachersF

OI

Teachers'meanpercentageofknowledgeonteachingfluency

Figure 17.Experimentalteachers'fidelityofimplementation(FOI)ofreadingprogramandtheirtotalmeanknowledge ofteachingfluencyatendline

R2=.126sig=.046

36

canreadthewordsbutmaynotknowwhatthey’rereading.(N=2)”Inotherwords,onecanbeafluentreader,butifs/hedoesnothaveorallanguageskillsorvocabularyknowledgeinthelanguagebeingread,s/hewillcomprehendlittle.Consequently,masteringfluencyaloneisinsufficienttobecomingagoodreader.Toensurethatstudentsdevelopedthenecessaryvocabulary,thePAQUEDprograminterventionsallprovidedsignificantamountsoftimeforvocabularydevelopmentinFrenchthroughbrainstormingactivities(collectedesidées)aroundstorythemes;gameswithmovementsand/orillustrationstoexplainnewvocabularyrelevanttoastory;orhavingstudentsusenewvocabularylearnedinasentencetheycomposedorallyorinwriting.

Sowhatwasteachers’knowledgeonteachingvocabularyatendlineafterhavingengagedinthesevocabulary-buildingactivitieswiththeirstudents?Table16showsthatexperimentalteachers’knowledgeofhowtobestteachvocabularydifferedsignificantlyfromtheirIAI-onlyandcontrolcounterparts(p=.002,d=-1.13,ES=0.49).Thisisalsosupportedbyexperimentalteachers’responsestoQuestion2.2thatstatestheimportanceofexplainingnewvocabularybeforereadinganewtext(p=.025,t=0.48,ES=0.24)andQuestion2.1affirmingtheimportanceofdiscussingwhatstudentsknowaboutathemeisalsorelatedtovocabularydevelopment(p=.002,d=0.7,ES=0.33).Thisisnotsurprisinggiventhatthepre-readingactivityoutlinedinthereadingprogramexplicitlyinvitestheteachertodiscussthethemeofthestorywithhis/herstudentsandexplainthenewvocabularyassociatedwiththeread-aloudtextoftheweek.LinearregressionsupportsthislinkbetweenQuestion2.1andteachers’applicationofthereadingprograminthat14.4%ofthevarianceinteachers’responsetoQuestion2.1canbeexplainedbytheirapplicationofthereadingprogram(p=0.39,d=-0.8,ES=0.37).

Tofurthersupportthislinkbetweenreadingprogramapplicationandknowledgeofteachingvocabulary,theconcreteclassroomexamplesofvocabularyinstructionderivedfromtheinterviewwithexperimentalteachersallowsustobetterunderstandhowteachersareactuallyputtingthesestatementsintoclassroompractice:

0102030405060708090

0

5

10

15

20

25

30

35

Yes No

FOI

R2=.144sig=.039

Freq

uencyofte

ache

rs'respo

nses

Mean%

FOI

Figure18.Experimentalteachers'fidelityofimplementation(FOI)ofreadingprogramandtheir

responsesto Question2.1

37

“withthehelpofillustrations,Iaskquestionsthathavetodowiththethemeofthetexttobringoutthenewvocabulary,”(N=7)

”Idoabrainstormingwithmystudentsaroundthethemeandthenewvocabulary.Then,Ireadthetexttothem.”(N=6)

Bothoftheaboveexamplesareconsistentwiththestep-by-stepsequenceofhowvocabularyinstructionispresentedforthepre-readingexerciseinthereadingprogramguide.

Incontrasttotheexperimentalteachers,controlteachers’responsesandexamplesregardingvocabularyinstructionintheirclassroomsincludedsuchstatementsas:

“they[students]don’tdovocabularyatthisgradelevel,”(N=12)

“studentswillonlyunderstandthenewwordsafterthereadingofthetext.”(N=12)

Experimentalteachers’statementsareduallyreinforcedbytheirexpectationsrelatedtothelanguageinwhichstudentslearntoreadandwritemoreeasily.ExperimentalteacherssignificantlydifferedintheiropinionsofQuestion1.2,mystudentslearntoreadmoreeasilyinmothertonguethaninFrench(p=.033,d=0.56,ES=0.27).67%ofexperimentalteachersfeltthisstatementwastruewhereasalargerproportionofIAI-onlyteachers(81%)andcontrolteachers(85%)agreedwiththisstatement.Incomparisontotheircounterparts,itcanbesuggestedthatmoreexperimentalteachersmayhavedisagreedwiththisstatementbecausethereadingprogramisgiveninFrench.Hence,theymayhavefeltthatgiventherightstrategies,theirstudentscouldlearntoreadjustaseasilyinasecondlanguage.

Comprehension:Comprehensionistheultimategoalwhenreading.Ifastudentcandecodefluently,understandssufficientvocabularyinthelanguagebeingread,andisequippedwithcomprehensionstrategies,s/heiswellequippedtocomprehendatexts/hereads.Unfortunately,comprehensionisthemostdifficultskilltoacquire,becauseitrequiresthatthestudenthasacquiredthefoundationalskillslistedabove.Comprehensionisalsonotusuallythemainfocusofearlygradereadingprogramsbecausesomuchattentionisneededtohelpingstudentslearntocrackthealphabeticcodeandtodecodewithaccuracyandfluency.ThePAQUEDreadingprogramandIAIprogramsrepeatedlymodeledcomprehensionstrategiesforteacherstoapplyintheirclassrooms.Theseactivitiesincludedaskingstudentstoreflectonwhattheyalreadyknowaboutasubject;posingliteralquestions(i.e.Who?What?When?Where?Why?)aboutatexttheyhadread;askingstudentstojustifytheanswerstheygive;andshowingteachershowtoemploygraphicorganizerstostructurethinkingandinformationderivedfromatext.Comprehensionstrategiesalsoencouragedstudentstoillustrateorwritetheirreactionstotextstheyhadread;conceiveanalternativeendingtoastory;orcreatetheirownpoems,stories,orletters.

HowdidteachersinthePAQUEDinterventionschoolsdemonstratetheirknowledgeofhowtoteachcomprehensionatendline?Table16showsthatatendline,experimentalteachers’knowledgeofhowtobestteachcomprehensiondifferedsignificantlyfromtheirIAI-onlyandcontrolcounterparts(p=.000,d=0.92,ES=0.42).Thisisconsistentwithitemanalysisfor4.1,4.2and4.4,inwhichasignificantproportionofexperimentalteachersrespondedpositivelyovertheirIAI-onlyandcontrolcounterparts.

38

Statements4.1,4.2and4.4allasserthowimportantitistoaskstudentsquestionsortoexplainwhatwasreadandtoallowstudentstodiscussinformationtheyretainedfromatextwiththeirpeers.

Interviewswithexperimentalschoolteachersfurtherexplainedhowtheyorchestratereadingcomprehensionactivitiesintheirclassrooms.Theseexamplessupportteachers’responsestoquestionsontheirknowledgeofteachingreadingcomprehensionwhilealsomirroringtheapproachesoutlinedinthePAQUEDreadingprogramandIAIprograms.

“Afterreadingatext,Ialwaysaskcomprehensionquestions—sometimesinmothertongue—onthestory,”(N=17)

“Iaskmystudentstoexplainalltheysawandwhathappenedinthetext—thecharacters,theimportantevents,”(N=11)

“Iusetheillustrationstohelpstudentsanswerquestionsonthetextjustread,”(N=11)

“Iaskmystudentstotellmewhattheylikedinthetextandwhy.”(N=5)

ThesestatementsspecificallyrelatetoreadingprogramandIAIprogramactivitieswhichaskteacherstoposebothliteralandinferentialcomprehensionquestionsonthestoryread.Thisexplanation,however,wasnotconfirmedbylinearregressionanalysis,whichindicatednosignificantcorrelationsbetweenspecificPAQUEDinterventionsandateacher’sknowledgeofhowtoteachcomprehension.

Incontrast,examplesandjustificationprovidedbycontrolteachersareconsistentwiththeirresponsestothe‘knowledgeofteachingcomprehension’questions:

“Iaskmystudentstorepeattheexplanationofwhatwereadafterme.”(N=2)

“studyingatextingrade1and2istoodifficult,”(N=8)

“itismoreusefultoexplaindifficultwordsinthetextratherthanaskingstudentstosaywhattheylearned.”(N=4)

Theseexamplessuggestthatcontrolteachersarehesitanttohelptheirstudentsdevelopcomprehensionskillsormaynotknowhowtobestdevelopastudent’scomprehensionskillsbeyondroterepetition.Likelyexplanationsfortheseteacherstatementsincludeteachers’unwillingnesstoallowchildrentomakemistakesorthinkforthemselves,andperhapsalso,theirlowexpectationsoftheirstudents’abilities.

Writing:TheintegrationofwritingwasastrongcomponentofPAQUEDinstructionalmaterials,drawingontheresearchpointingtowritingasusefulfordevelopingreadingskills.Forexample,studentsconnecttheirphonemicandalphabeticawarenessskillswhentheyattempttospellwords.Likewise,astudentworkstheircomprehensionskillswhentheyareaskedtoreacttoatextinwriting,payingattentiontosentencestructure,useofvocabulary,spelling,andpunctuation.Becausewritingisaprocessthatpullstogetherseveralcomponentsskillsofreading,itallowsastudenttopracticeutilizingtheseskillstocommunicate.Itisalsoanempoweringtoolbecauseitisavisualproductionofwhatastudentifcapableofdoing.TheIAIprogramsandaccompanyingteacherguideplaceemphasisongettingstudentstoexperimentwithwritingandillustrations.Similarly,thereadingprogrampost-readingactivitiesandwordstudyactivitiesbothhavewritingcomponentsthatenablestudentstoexpresstheirideasandpreferencesbyanswering“onmyown”extensionquestionsonthestorythey

39

heardaswellasbypracticingtheirphonicsskillsthroughspelling.Convincingteacherstoallowforfreewritinglikeinventedspellingisabigstep.

ThisisbecauseatthebeginningofthePAQUEDproject,stakeholderresearchrevealedthatteachersalwaystaughtreadingandwritingseparatelybecauseitwasprescribedinthecurriculumastwoseparatesub-branches(sous-branche)andshouldthereforebetaughtindifferentlessons.Teachersalsoassertedthatstudentsshouldnotbeallowedtomakespellingmistakesandthatwritingshouldbelimitedtocopyingontheboarduntilstudentswere‘capable’ofexpressingthemselvesproperlyinwriting,askilloftenexpectedforchildreningrade4andbeyond.Basedonthesefindings,askingteacherstoallowtheirstudentstoengageinwritingactivitiesduringreadinglessonswasexpectedtobeachallenge.Endlinedataindicatingthatteachershavebeguntoallowfreewritingandinventedspellingareasignificantindicatorofprogresstowardsclassroompracticesthataredocumentedtosupportstudentachievement.

Summaryresultsonateachers’knowledgeofintegratingwritingintotheirlessonsshowedthatteachersacrossgroupsdidnotdiffersignificantly.Thewritingcompositeincludesteachers’toleranceofinventedspelling;theirperceivedimportanceofintegratingreadingandwritingintoasamelesson;andtheirperceivedimportanceofstudentshavingopportunitiestopracticewriting.However,experimentalteachers’classroomexamplesofhowtheyorchestratewritingactivitiesdoprovideevidencethat,despitethelackofdifferenceintheirknowledgeaboutteachingwriting,theyareintegratingwritingintotheirlessons:

“wepracticewritinghighfrequencywordsandfamiliarwords—theirnames,wordsinmothertongue,mom,dad,under,over,etc.”(N=10)

“WhenIteachanewletter,Iaskthestudentstofindanotherwordwiththatletterintheirbooksorintheclassroomandtowriteitontheboard,”(N=3)

“Iletthemwriteareactiontoatextandthenwecorrectitinpairs,”(N=8)

“Iaskstudentstowritetheirownsentenceswithnewwordswejustlearned.”(N=2)Teachersalsocitedusingwritingactivitiesto“motivate[their]studentstolearn,”speakingtothepowerofwritingforstudentempowermentmentionedabove.SometeachersinexperimentalandIAI-onlyschoolsstillindicatedatendlinethat:“Idowritingbywayofspellingtestsorcopyingofftheboard.”(N=16)Thisisnotsurprisinggivenhowdifficultitistoaskteacherswhooriginallyprofessedtheirlackoftoleranceforspellingmistakestoshifttheirpractices.Thispositionisechoedinassertionsbyseveralteachersincontrolschools,suchas:

“itisnotappropriateforchildrentowriteinsecondgrade”(N=7)

“Students’lackexperience.Therefore,readingandwritingmustalwaysbetaughtseparately.”(N=3)

Relativetothisdiscussionisteachers’knowledgethatintegratingreadingandwritingintothesamelessonisuseful.Althoughnosignificantdifferenceswerefoundbetweengroupsonteachers’opinionofthisitem,interviewdatashowedthatthosecontrolteacherswhothoughttheyshouldnotbetaughtinthesamelessonjustifiedtheiranswersinsaying:“studentsriskmixingreadingandwriting”(N=2)andthat“readingshouldprecedewriting”(N=4).However,thoseexperimentalteacherswhoacknowledged

40

theimportanceofintegratingreadingandwritingintothesamelessonjustifiedtheiranswersinawaythatprovidesevidenceoftheuseofthereadingprogram:

“ifastudentknowshowtowritewords,theycaneasilyreadthemandviceversa.”(N=5)“ForeachletterthatIteach,welearntheletter(grapheme),thesoundandhowtowriteit.Then,welearntoreadandwritewordswiththatletter.”(N=4)“Idospellingtestsofwordsthey[mystudents]studiedorthatcontainlettersstudied.”(N=16)

Thisshowsthatteachersseemedtodevelopanunderstandingoftherelationshipbetweendecodingandencoding,onekeytopicpresentedintheinitialreadingprogramtrainingforexperimentalteachers.Thislinkisconsistentwithlinearregressionanalysiswhichshowedthat12.1%ofthevarianceinexperimentalteachers’knowledgeofwritingcanbeexplainedbytheirapplicationofthereadingprogram(p=0.048,d=0.72,ES=0.34).

Finally,teachers’expectationsoftheirstudents’abilitiestolearntowritewerefoundtobesignificantlydifferentacrossgroups.ExperimentalteachersrespondednegativelytoQuestion5.2,mystudentshavealotofdifficultylearningtowrite(p=001,d=-078,ES=0.36).Aswritingwasmuchpracticedinthecontextofthereadingprogram,thismaysuggestthatexperimentalteachers’expectationsoftheirstudents’abilitiesmayhaveshifted.

Grade1and2TeacherpracticeresultsInadditiontounderstandingteachers’knowledgeaboutteachingreadingandwriting,itisalsoimportanttounderstandhowthesechangesmayhavetranslatedintopractice.Tomeasurechangesinteacherpractice,anobservationtoolwasadministeredingrade1and2experimental,IAI-only,andcontrolteachers’classroomsatbaselineinMarch2013andatendlineinMay2014.Sampledteacherswereaskedtoteachalessonintroducinganewletterorletter-soundrelationshiptostudentsatbothpoints,inordertoensureadegreeofcomparabilityoftheresults.Theobservationtoolcontainedarangeofspecificandobservablepracticesgroupedbythecomponentskillstheyaimedtobuild(seeAnnexBfortool).Thesepracticeswerechosentoreflectthoseoutlinedinthereadingprogram,inthenationalreadingstandards,andthoseobservedinnumerousclassroomobservationsconductedthroughouttheproject.Eachitemizedpracticewasallotted9five-minutetranchesoftime,whichcoversanaveragelessonspan.Iftheenumeratorwitnessedapractice,heorshewouldcheckofthepracticeintheappropriatetimeperiod.Thiswastoprovideasnapshotofthelessonasitprogressedandtoquantifyteachers’implementationofcertainpracticesoverothers.Itshouldbenotedthatevenifapracticewasobservedtwiceinaperiodoffiveminutes,onlyonecheckwasallowedperfive-minutetranche.Thisisapossiblelimitationofthetoolasithindersabilitytodetectsubtlechangesinteacherpractice.Still,theresultsderivedfromthetoolprovideinterestinginformationonteachers’practiceandwerefoundtobestatisticallyreliable(seeAnnexB).Anotherlimitationofthetoolisitsinabilitytoseehowteachersactuallyconductedapractice.Thougheachpracticeoutlinedisconcretelyobservable,detailsliketeachersdispositionswerenotcaptured.Forexample,foravocabularybuildingpracticewhichasksstudentstoexplainnewvocabularybeforetheyreadanewtext,thepracticeoutlinedinthetooldoesnotsayexactlyhowtheteachersgoesaboutguidingstudentsinthisactivity.

41

Table17providesanoverviewofteachers’practiceresultsgroupedbycomponentskillfromthebaselineandtotheendline,andindicateswhetherornotthesechangeswereshowntobestatisticallysignificantwithinthegroups(longitudinally)andacrossgroups(cross-sectional).Table17.Summaryofthegrade1and2teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus3

PAQUEDCONTROL(n=60)

Experimental(n=57) IAI(n=78)Baseline Endline Gainscore Baseline Endline Gainscore Baseline Endline Gainscore

Phonemic/PhonologicalandAlphabeticawareness 9.79% 8.9% -0.89% 7.25% 7.73% 0.49% 6.87% 7.45% 0.58%

Fluency 13.26% 19.68% 6.42%*** 20.39% 19.77% -.62% 18.24% 19.88% 1.64%Vocabulary 11.11% 12.86% 1.75% 6.7% 7.06% .36% 5.83% 7.5% 1.67%Comprehension 13.32% 18.91% 5.59%** 5.65% 10.82% 5.17%** 8.51% 8.39% -0.12%Generalinstructionalandliteracypractices 15.94% 14.26% -1.68% 16.61% 20.34% 3.73%* 14.96% 13.22% -1.74%

Total 15.37% 14.25% -1.12% 13.27% 15.34% 2.07%* 13.11% 12.74% -0.37%*Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.05**Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001

Thefollowingtablethatfollowspullsoutsomespecificpracticeitemsthatwereshowntochangesignificantlyacrossbaselineandendline.Thepercentagesrepresentthetotalnumberoftimesthepracticewaswitnessedoverthetotallessontime.Forexample,ifateacheraskedstudentstotrytoidentifyaspecificsoundinawordoverthreetranchesoffive-minutetimeperiodsofa40-minutelesson,theteacherwouldbeconsideredtohaveexhibitedthispracticeforapproximately37.5%oftotalinstructionaltime(3outof8).

Table18.Itemanalysisofthegrade1and2teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus

Theteacher…

PAQUED CONTROL(n=60)Experimental(n=57) IAI-only(n=78)Baseline Endline Gainscore Baseline Endline Gainscore Baseline Endline Gainscore

P15.Asksstudentstogivetheirpredictionsonthecontentofatextbyusingclues(title,illustrations,etc)

9.1% 15.13% 6.03%* 5.6% 7.93% 2.33% 6.9% 4.49% -2.41%

P16.Asksquestionsonatextread(ex.who?What?Where?How?...)

17.9% 29.14% 11.24%* 7.33% 13.71% 6.38%** 12.16% 11.74% -0.42%

P17.Solicitsideasandexperiencesfromtheirstudentsonwhattheyalreadyknowaboutasubject.

9.25% 12.24% 2.99% 6.01%* 10.77% 4.76%*** 6.31% 6.99% 0.68%

P18.Integratesreadingandwritingactivitiesintothesamelesson.

7.71% 15.94% 8.23%** 9.98% 13.94% 3.96%* 9.23% 10.86% 1.63%

P23.Encouragesstudentsinapositivemannerwhentheymakeaneffort.

42.44% 38.49% -3.95% 31.39% 35.83% 4.44%* 32.16% 33.58% 1.42%

*Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.05**Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001

3Percentageindicatestimeallocatedtotheseskillsrelativetothelesson’sentirety.

42

___=significantacrossgroups(cross-sectional)Figure19visuallyshowsthegainsinpracticethatteachersinthedifferentgroupsmadeacrossbaselineandendline.

Figure19.Grade1and2teachers’changeinliteracyinstructionalpracticesfrombaselinetoendline.

Experimental(N=57)IAI-only(N=78)Control(N=60)

-0.890.490.58

6.43-0.62

1.64

1.750.361.67

5.595.18

-0.12

-1.683.73

-1.74

-1.122.07

-0.37

0% 5% 10% 15% 20% 25%

Phonological, phonemic

and alphabetic awareness

Fluency

Vocabulary

Comprehension

General

Total

Gainscore (pct)

Negative gainscore (pct)

43

Incontrarytoteachers’knowledgeofteachingreading,grade1and2IAI-onlyteachersimprovedsignificantlyoverbaselineandendlineinthetotalliteracypracticescombined(p=.017,d=0.56,ES=0.27)andintheirapplicationofcomprehensionactivities(p=.000,d=0.87,ES=0.4)andgeneralclassroompractices(p=.003,d=0.7,ES=0.33).However,experimentalteachersimprovedsignificantlyintheinstructionofsomecomponentskills(fluencyandcomprehension)butdidnotimproveassignificantlyastheydidintheirknowledgeaboutteachingreading.Noneofthegroupsimprovedsignificantlyintheirapplicationofvocabulary-buildingandphonologicalandalphabeticawarenessactivities.

Thislackof“improvedpractice”doesnotnecessarilymeanthatteachersdidnotapplyactivitiesassociatedwiththesecomponentskillsintheendlinereadinglessonsobserved.Videosoftheseobservationsrevealthatteachersdidapplycertainpracticesthatwerenotcitedinthetool.Thetoolalsodidnotcaptureexactlyhoweffectivelyteachersappliedcertainpractices.Lastly,theabsenceofsignificantchangemayalsoberelatedtoalimitationinthetool,whichrequirestheenumeratortoonlycheckonceiftheyseethepracticewithineachfive-minutetrancheofthelesson.Becauseofthisstructure,iftheteacherhadconductedtheactivitytwiceormorewithinthatfive-minuteperiodoftime,thetoolwouldnotcaptureit.Asaresult,ateacher’sgainintermsofdemonstratedinstructionalacrossbaselineandendlinemaynotbethoroughlycapturedbythetool.Ontheotherhand,beingabletoinventoryeverytimeateacherappliesacertainpracticecanbeoverwhelmingforadatacollectoranddoingsowouldhaverequiredamorelimitedlistofpracticestoobserve.

Thefinaltwotables,below,indicatehowPAQUEDinterventionsmayhavepredictedsomeofthesechangesinteachers’practice.Interestingly,unliketeachers’knowledge,fidelityofimplementationofthereadingprogramdidnotcorrelatesignificantlywiththeirchangesinpractice.Thisisinterestingbecauseonewouldassumethatifateacherconsistentlyshowedtoapplycertainpracticesassociatedwiththereadingprogram,thatthiswouldlogicallytranslateintotheirliteracy–specificinstructionalpractices.However,experimentalteachers’listenershipofIAIdidcorrelatesignificantlywiththeirinstructionalpracticesassociatedwithallcomponentskillsexceptforphonemic,phonologicalandalphabeticawareness.Inadditiontothis,Table19revealthatexperimentalteachers’participationincontinuingprofessionaldevelopmentactivitieswassignificantlycorrelatedwiththeirapplicationofgeneralinstructionalpracticeslikewalkingaroundaclassroomtohelpstudentsinneed,askingstudentstoworkingroups,orcongratulatingstudentswhenstudentsmakeaneffort.Table20breaksdownthecorrelationsbetweenspecificpracticesappliedandIAI-onlyandexperimentalteachers’IAIlistenership.Forexperimentalteachersonly,itprovidesthecorrelationsbetweenteachers’participationinCPDandtheirapplicationofspecificinstructionalpractices.

44

Table19.Summaryresultsoflinearregressionforthegrade1and2teacherchangeofinstructionalpracticesusingadherencetoteachers’participationinCPDandIAIdosageaspredictors Experimental

IAIschools:IAIdosage ParticipationinCPD IAIdosage R2 Sig. R2 Sig. R2 Sig.Phonemic/PhonologicalandAlphabeticawareness

- - - - - -

Fluency - - .129 .007 - -Vocabulary - - .074 .043 - -Comprehension - - .083 .031 - -Generalinstructionalandliteracypractices

.183 .012 .07 .048 - -

TOTAL - - .142 .004 - -

Table20.Itemanalysisresultsoflinearregressionforthegrade1and2teachers’changeofinstructionalpracticesusingadherencetoteachersparticipationinCPDandIAIdosageaspredictors Experimental

IAIschools:IAIdosage ParticipationinCPD IAIdosage R2 Sig. R2 Sig. R2 Sig.P4.Asksstudentstocorrectbadlyspelledwordsintheirownwritingorinwritingsontheboard.

- - - - .108 .004

P9.Pointstoletters,syllables,orwordswhilehe/shereadsortoguidestudentsintheirreading.

- - .132 .006 - -

P10.Attractsattentiontopunctuationwhilestudentsread.

- - .083 .031 - -

P11A.Asksstudentstoreadaloudalone. - - - - .061 .031P11C.Asksstudentstoreadtogether - - - - .063 .028P13.Explainsorasksstudentstoexplainnewvocabularypriortoreadinganewtext.

- - .105 .015 - -

P15.Asksstudentstogivetheirpredictionsonthecontentofatextbyusingclues(title,illustrations,etc)

- - .078 .038 .068 .022

P20.Walkaroundtohelpstudentswhentheyareworkingindividuallyoringroups.

.172 .015 - - - -

P21B.Asksstudentstoworkinpairsorgroups.

- - .084 .03 - -

P22.Asksstudentstocategorizegroupsofwordsbyacharacteristic(samesound,sameletter,sametheme)

- - .086 .01 - -

P23.Encouragesstudentsinapositivemannerwhentheymakeaneffort.

- - .109 .013 - -

Thediscussionbelowfurtherexpandsonteachers’gainsininstructionalpracticesrelativetoreadingandfurtherelaboratesonthelinksbetweenthesegainsandPAQUEDinterventions.Itisbrokendownbycomponentskilltoreflecthowtheyareorganizedintheobservationtool.

45

Phonemic,phonologicalandalphabeticawareness:Asstatedearlier,thePAQUEDreadingprograminexperimentalschoolsandIAIprogramsfocusedheavilyondevelopingphonemic,phonological,andalphabeticawarenessskillsbecausetheyaretheessentialbuildingblocksfordecodinganddevelopingconceptofword.Duetothisemphasis,didteachers’tendtousemorepracticesassociatedwithbuildingphonological,phonemic,andalphabeticawareness?Whileteachersacrossgroupswereobservedapplyingseveralactivitiesassociatedwiththesecomponentskillsatendline,therewasnostatistically-significantchangeobservedinthesepracticeareas.Onaverage,experimentalteachersspentaround10%ofinstructionaltimeonexplicitlyteachingthesecomponentskills.IAI-onlyspent8%ofinstructionaltimeandcontrolteachersspent7.5%.Forthosepracticesmorecloselyassociatedwithreadingprogramactivities(i.e.helpingstudentstoidentifyletternamesandsounds),experimentalteachersspentanaverageof18%ofinstructionaltimewhereasIAI-onlyandcontrolteachersspentlesstimedoingso.Forexperimentalteachers,nospecificpracticesoutlinedinthetoolcorrelatedsignificantlywiththeiruseofPAQUEDinterventions.ForIAI-onlyteachers,10.8%ofthevarianceinteachers’changeintheirapplicationofPractice4,asksstudentstocorrectbadlyspelledwordsintheirownwritingorinwritingsontheboard,couldbeexplainedbytheirIAIusage(p=.004,d=0.69,ES=0.33),asshownbelow.

Inadditiontothis,8.6%ofIAI-onlyteachers’changeinapplicationofP22,askingstudentstocategorizegroupsofwordsbyacharacteristicwasexplainedbyIAI-listenership(p=.01,d=0.61,ES=0.29).

40

50

60

70

80

90

100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen

tageofteachersIAI-usage

IAI-onlyteachers'meanpercentageofgainsinP4

Figure20.IAI-onlyteacher’sIAIusageandtheirgainsinPractice4

R2=.108sig=.004

46

Thiscorrelationissurprising,asthisactivitywasmoreprevalentinthereadingprogramactivitiesguidethanintheIAIprograms,andtheIAI-onlyteachersdidnotreceivetheprogramactivitiesguide.However,althoughexperimentalteachersdidnotseemtoallocatemoreinstructionaltimetothisparticularpracticeacrossbaselineandendline,atendline,experimentalschoolteachersspent48%ofinstructionaltimeonthispracticewhileIAI-onlyteachersspent31%andcontrolteachersspent20%.Thisfindingisconsistentwithapplicationofreadingprogramactivitiesthatallocatesignificanttimetowordstudyandvocabularydevelopment.

Fluency:Practicesassociatedwithbuildingstudentfluencywerepartofthedailysequenceofactivitiesoutlinedinthereadingprogram.Theseactivitiesinvolvedstudentsquickreadingofhighfrequencywordsandwordsstudied;havingteacherspointtowordsforstudentstodevelopaconceptofwordandtomodelfluentreading;andallowingstudentstohavein-classtimetopracticereadingaloudinpairsorthroughchoralreading.AsaresultofPAQUEDinterventions,didteachersapplymorefluency-relatedactivitiesfrombaselinetoendline?Thecompositescoreassociatedwithfluencypracticeshownintheabovetableshowsexperimentalteachersimprovedsignificantlymoreintermsofallocatinginstructionaltimetofluencypracticesfrombaselinetoendline(p=.000,d=1.42,ES=0.58),longitudinallyandincomparisonwiththeirIAI-onlyandcontrolcounterparts.Thisissupportedbyitemanalysisshowingthatexperimentalschoolteacherssignificantlyincreasedtheirclassroomtimeallocatedtoallowingstudentstoreadinpairsoralone(p=000,d=1.43,ES=0.58).Linearregressionalsofoundthat12.9%ofthevarianceingainonthesepracticescanbeexplainedbyateacher’srateofIAIlistenership(p=.007,d=0.76,ES=0.36).

0102030405060708090100

-25 -20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen


IAI-onlyteachers'meanpercentageofgainsinP22

Figure21.IAI-onlyteacher’sIAIusageandtheirgainsinPractice22

R2=.086sig=.01

47

ThisshowsthatIAImayhavecontributedtoexperimentalteachers’increaseinapplicationoffluency-buildingactivities.Linearregressioninitemanalysisalsoshowedthat13.2%ofthevariationinexperimentalteachers’increasedapplicationofpointingtoletters,syllables,andwordstohelpguidestudentswhiletheyreadcanbeexplainedbyIAIlistenership(p=.006,d=0.77,ES=0.36).

Likewise,8.3%ofthevariationinexperimentalteachers’increasedapplicationofdrawingattentiontopunctuationtohelpstudentsreadwithintonationcanbeexplainedbyIAIlistenership(p=.031,d=0.6,ES=0.29).

0102030405060708090100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen


Experimentalteachers'meanpercentageofgainsinfluencypractices

Figure22.Experimentalteachers'IAIusageandtheirgainsinfluencybuildingpractices

R2=.108sig=.004

0102030405060708090100

-30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45

Meanpe

rcen


Experimentalteachers'meanpercentageofgainsinP9

Figure23.Experimentalteachers'IAIusageandtheirgainsinPractice9

R2=.132sig=.006

48

Still,itisimportanttonotethatdespiteIAI-onlyandcontrolteachersshowingnosignificantimprovementinapplyingfluencypracticesfrombaselinetoendline,theseteachersatendlinestillspendmoreinstructionaltimeonfluencythandotheirexperimentalteachercounterparts.ThisisperhapsbecausethereadingprogramasksteacherstofocusondevelopingseveralcomponentskillsintheirstudentswhileIAI-onlyandcontrolteachersmayonlybefocusingondevelopingtheirstudents’fluency.IAIprogramsalsofocusedheavilyondevelopingfluency.Thisisconsistentwithlinearregressionanalysiswhichfoundthat6.1%ofthevariationinthechangeofIAI-onlyteachers’gainsinaskingstudentstoreadalone(p=.031,d=0.5,ES=0.25)and6.3%ofthevariationintheirgainsinaskingstudentstoreadtogether(p=.028,d=0.52,ES=0.25)canbeexplainedbyatheirrateofIAI-listenership.Thesecorrelationsareshowninthefiguresbelow.Figures25.IAI-onlyteachers’IAIusageandtheirgainsininstructionalpractice

Duetothisfinding,itisinterestingtounderstandthepercentageoftimeteachersallocatedtobuildingfluencyskills,onaverageandacrossgroups.Teachersacrossallgroupsspentaround19to20%of

0102030405060708090100

-50 -45 -40 -35 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30

Meanpe

rcen



Figure 24.Experimentalteachers'IAIusageandtheirgainsinPractice10

R2=.083sig=.031

0102030405060708090

100

-50 -40 -30 -20 -10 0 10 20 30

Meanpe

rcen

tageofteachers'

IAI-u

sage

IAI-onlyteachers'meanpercentageofgainsinP11A

IAI-onlyteachers'IAIusageandtheirgainsinPractice11A(askingstudentstoreadaloudalone)

R2=.061sig=.031

0102030405060708090

100

-50 -40 -30 -20 -10 0 10 20 30

Meanpe

rcen

tageofteachers'

IAI-u

sage

IAI-onlyteachers'meanpercentageofgainsinP11C

IAI-onlyteachers'IAIusageandtheirgainsinPractice11C(askingstudentstoreadaloudalone)

R2=.063sig=.028

49

instructionaltimemodelingorallowingtheirstudentstopracticetheirfluency.Thosepracticesthatshowedtobemostprevalentwereteacherspointingtowordstoguidetheirstudents’reading(24%ofinstructionaltimeforexperimentalteachers,28%forIAIonly,and31%forcontrolteachers),allowingstudentstoreadaloudontheirown(16%forexperimental,25%forIAIonly,and30%forcontrolteachers)andhavingstudentsengageinchoralreading(26%ofinstructionaltimeforexperimentalteachers,25%forIAIonly,and30%forcontrolteachers).However,itshouldbenotedthatthelattertwoitemsdonotspecifywhetherornotstudentsarerepeatingaftertheteacheroriftheygenuinelyreadontheirown.Iftheyarerepeatingaftertheteacher,whichisthetraditionalinstructionalmodelinDRC,thismayexplainwhyIAI-onlyandcontrolteacherswerefoundtoexhibitthesepracticesasfrequentlyastheirexperimentalteachercounterparts.Vocabulary:Muchofday1oftheweeklyreadingprogramandtheIAIprogramswerededicatedtovocabularydevelopment,throughbrainstormingactivities,discussionofnewvocabulary,andvocabularygames.Giventhisfocus,wereteachersobservedtobeapplyingmorevocabularybuildingactivitiesatendline?Interestingly,teachers’applicationofvocabulary-buildingactivitiesintheclassroomwasnotshowntosignificantlychangefrombaselinetoendlineandacrossgroups.Still,experimentalteachersallocatedaround13%ofinstructionaltimetovocabularybuildingactivities,ascomparedtotheirIAI-only(7%)andcontrol(7.5%)counterparts.Linearregressionexplainsthisdifferenceshowingthat7.4%ofthevarianceinexperimentalteachers’applicationofvocabularyactivitiescanbeexplainedbytheirIAIlistenership(p=0.043,d=0.56,ES=0.27).

LinearregressionofspecificpracticeslikeP13showsthat10.5%ofteachers’changeinapplicationofexplainingoraskingstudentstoexplainnewvocabularypriortoreadinganewtextispredictedbyIAI-listenership(p=0.015,d=0.68,0.32).ThisfindingmakessenseasIAIprogramsmodelstrategiesforvocabularydevelopment.Comprehension:Asdiscussedearlier,instructionalpracticesaimedatbuildingstudents’readingcomprehensionarecentraltothereadingprogramsequenceofactivitiesandtheIAIprograms.

0102030405060708090100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen


Experimental teachers'meanpercentageofgainsinvocabularypractices

Figure26.Experimentalteachers'IAIusageandtheirgainsinvocabularybuildingpractices

R2=.074sig=.043

50

Interactivevocabularybuildingactivities,regularquestioningtocheckforstudentunderstanding,askingstudentstomakepredictionsusingcontextualandvisualclues,andsolicitingideasfromstudent’slifeexperiencesoastorelatetheinformationinthetexttotheirliveswereinstructionalapproachesincludedinthereadingprogramwhichsupportincreasedcomprehension.Studentreactionsinwritingtostoriesheardorread(whicharealsoconsideredcomprehensionactivities)werealsopromotedbytheprogram.GiventhePAQUEDinterventions’focusoncomprehension,didteachersspendmoreinstructionaltimeatendlineonsuchactivitiesthantheydidatbaseline?Table18abovedemonstratesthatexperimentalschoolandIAI-onlyteachersspendmoretimeengagedinreadingcomprehensionactivitieswiththeirstudentsacrossbaselineandendline(p=.005,d=0.78,ES=0.36).Fromacross-sectionalperspective,significantdifferencesbetweenexperimentalschoolandcontrolteachersandIAI-onlyandcontrolteacherswerealsofoundforthesepractices(p=.011,d=-0.53,ES=0.26).IAI-onlyteachersalsosignificantlyincreasedoverbaselineandendlineintheirapplicationofcomprehensionactivities(p=.000,d=0.87,ES=0.4).ThisisfurthersupportedbytheindividualpracticeanalysiswhichshowedexperimentalschoolteachersandIAI-onlyteacherstosignificantlyshowmoreapplicationofP17—solicitingideasorexperiencesfromstudentsonwhattheyalreadyknowaboutasubjectpriortoreading(p<.01)andaccountedforanaverageof12%ofinstructionaltimeforexperimentalteachers.Thisparticularpracticewasakeycomponentofthepre-readingactivityoutlinedinthereadingprogramandintheIAIprograms.ForP15—askingstudentstogiventheirpredictionsonthecontentofatextbyusingclues,itemanalysisshowssignificantdifferencesbetweenexperimentalandcontrolteachersandbetweenIAI-onlyandcontrolteachers(p<.05).IAI-onlyteacherstendedtomorefrequentlyaskquestionsoftheirstudentsacrossbaselineandendline(p<.01)yetoverall,experimentalschoolteachersspentthemostinstructionaltimeallocatedtoaskingquestions(29%).ThismirrorsthedesignoftheIAIprogramsinwhichcharacterscontinuouslyaskstudentsquestionstokeepthemactivelyengaged.ThisdifferencewasalsofoundtobesignificantbetweenIAI-onlyandcontrolteachers.

ThesefindingsareinterestingastheysuggestthatthepresenceofIAImayberelatedtoteachersapplyingcomprehensionactivities.Thisisconsistentwithlinearregressionshowingthat8.3%ofexperimentalteachers’changeinapplicationofcomprehensionactivitiescanbeexplainedbyIAIlistenership(p=.031,d=0.6,ES=0.29).

51

IAIlistenershipalsoexplained7.8%ofthevarianceinexperimentalteachers’applicationofP15,askingstudentstogivetheirpredictionsonthecontentofatextbyusingclues(title,illustrations,etc)(p=.038,d=0.57,ES=0.28).Generalclassroomandliteracypractices:Thepracticescontainedinthe“generalclassroomandliteracypractices”compositeincludetheincorporationofgroupwork,teachermonitoringofstudentwork,andthepresenceofpositivestudentencouragement.So,howdidteacherschangeintheirpracticesbetweenthebaselineandtheendline?IAI-onlyteachersimprovedsignificantlyacrossbaselineandendlineintheiroverallgeneralliteracyandclassroompractices(p=.003,d=0.7,ES=0.33).

IndividualpracticeanalysisshowedsomesignificantdifferencesinspecificpracticesforbothIAI-onlyandexperimentalteachers.Forexample,experimentalschoolandIAIteachersbothincreasedoverbaselineandendlineintheintegrationofreadingandwritingactivitieswithinthesamelesson,anotheroverarchingelementofthereadingprogramandtheIAIprogram(p=006,d=0.75,ES=0.35andp=.003,d=0.69,ES=0.33respectively).Thoughthispracticewasnotallocatedtoaspecificcompositepracticescore,itisinterestingtoseehowitcoincideswiththeteachers’knowledgefindingsindicatingexperimentalteachers’overallpositiveattitudestowardstheintegrationofreadingandwritingintheirlessons(43%ofexperimentalteachersagreedwiththisstatement).IAI-onlyteacherswerealsofoundtoprovidemorepositiveencouragementtotheirstudents(p=.022,d=0.53,ES=0.26)overbaselineandendline.Experimentalteachers’changeinthisparticularpracticecorrelatedsignificantlywithbothIAI-listenershipandteachers’participationincontinuingprofessionaldevelopmentactivitiesandcoachingvisits(p=.013,d=0.69,ES=0.33andp=.043,d=0.73,ES=0.34).OtheritemswerealsofoundtobesignificantlycorrelatedwithPAQUEDinterventions.Forexample,17.2%ofexperimentalteachers’changeinP20,walkingaroundtohelpstudentswhentheyareworkingindividuallyoringroupswasexplainedbyteachers’participationincontinuingprofessionaldevelopmentactivitiesandcoachingvisits(p=0.15,d=0.9,ES=0.41).

0102030405060708090100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen

tageofteachers

IAI-u

sage

Experimental teachers'meanpercentageofgainsincomprehensionpractices

Figure27.Experimentalteachers'IAIusageandtheirgainsincomprehensionbuildingpractices

R2=.083sig=.031

52

FinallyIAI-listenershipexplained8.4%ofthevariationinteachers’changeinaskingstudentstoworkinpairs(p=.03,d=0.6,ES=0.29),astrategyheavilyencouragedintheIAIprograms.

0102030405060708090100

-35 -25 -15 -5 5 15 25 35

Meanpe

rcen

tageofteachers

participationinCPD


Figure28.Experimentalteachers'participation inCPDandtheirgainsinPractice20(supporting students)

R2=.172sig=.015

0102030405060708090100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen

tageofteachers

IAI-u

sage


Figure29.Experimentalteachers'IAIusageandtheirgainsinPractice20(supporting students)

R2=.084sig=.30

53

Chapter2:Grade3to6teacherresultsThischapterpresentstheresultsofthestudyfromtheperspectiveofgrade3,4,5,and6teachers,whowereservedbyPAQUEDthroughthedistributionofIAIprograms(100lessonsperclass),accesstocontentknowledgetrainingsonFrenchandMath,thedistributionofclassroomkits,andthedistributionofaudio-videomodulestofacilitatetheirteacherlearningcircles(forumd’échange).Asthestudyaimedtofocusonreading,theresultspresentedbelowprovideinsightintoteachers’knowledgereadingandwritinginstructionandhowtheirliteracy-specificclassroompracticesmayhavechangedoverthecourseof1.5yearsfromMarch2013toMay2014.Theseresultshaveimportantimplicationsforthedevelopmentoffutureteachertrainingprogramdesign.Theseimplicationsincludetheneedforrobustreadinginterventionsatallgradesandunderstandingwhatisrequiredforthistobeeffectivelyimplemented.

ItshouldbereiteratedherethatthePAQUEDinterventioningrade3to6teacherswasnotasintenseasitwasforgrade1and2teachers.Experimentalgrade3to6teacherswerenotspecificallytargetedintheearlygradereadingprogramso,althoughtheymayhaveparticipatedinsomeschool-basedmeetings,theywerenotprovidedwiththesameintenselevelofinputsastheirgrade1and2counterparts.Furthermore,theIAIprogramsweredistributedtoteachersgraduallyoverthecourseoftheprojectastheywereproduced.Asaresult,grade1and2teachersreceivedthematthebeginningofyear2oftheprojectwhereasgrade3and4teachersreceivedtheminyear3andgrade5and6teachersreceivedtheminyear4.

Teacherknowledgeofliteracyinstructionfindings:Grade3,4,5&6teachersAtendline,thestudyaskedteachersingrade3,4,5,and6toprovideinformationabouttheirknowledgeaboutteachingreading.ThoughdisaggregatedbyexperimentalandIAI-only,manyoftheexperimentalschoolsinthispartofthesamplebenefitedfromaboutthesamelevelofinterventionastheIAI-onlyteachers.Thisisbecausethereadingprograminexperimentalschoolswasmainlytargetedatgrades1and2.Still,itwasdecidedtokeepthemasaseparategroup,asvisitstoschoolsbycoachestograde1and2teachersandparticipationbygrade3-6teachersinschool-basedteacherlearningcircles(forumd’échange)mayhaveinfluencedteachers’knowledgeoutcomes.Thisassumptionofeffectissupportedbytheresults,whichshowthatexperimentalteacherstohavemoreknowledgeontheteachingofcertaincomponentskillsovertheirIAI-onlyandcontrolcounterparts.Thetablesbelowsummarizethechangeingrade3and4andgrade5and6teacherknowledgegroupedbycompositeacrossthebaselineandtheendlineandTables21through24pulloutspecificitemsthatwereshowntobesignificantrelativetothecompositeskills.

Theresultspresentedbycomponentskillbelowrepresentthemeanpercentageofagreementtoagroupofquestionsclassifiedbycomponentskill.ThecompositionofquestionsbycomponentskillscanbefoundinAnnexB.Aseveryquestionposedcouldbeansweredas“yes”or“no”,themeanswerecalculatedbasedontheseresponses.Forexample,experimentalteachers’frequencyofagreementtoquestionsonhowtoteachfluencyamountedto83.83%meanagreement,comparedtoaround80%for

54

IAI-onlyand72%forcontrolteachers.Theresultsoftheindividualquestionsoutlinedintables22and24representthepercentageofagreementforeachquestionacrossdifferentteachergroups.

Thefindingsbelowrepresentteacherswhoparticipatedintheendlineknowledgeinterview.Overall,IAI-onlyandexperimentalteachershadmoreknowledgeabouthowtoteachreadingandwritingatendlinethandidtheircontrolcounterparts.However,thesetotaldifferenceswerenotfoundtobesignificant.Theresultsforknowledgewerealsocorrelatedwithteachers’useoftheIAIprogramsmadeavailabletothem4.Nosignificantcorrelationswerefoundbetweengrade3to6experimentalandIAI-onlyteachers’knowledgeandtheiruseofIAIprograms.Thismaybeduetotwofactors:theoveralllowmeanIAIlistenership5fortheseteacherswhichforbothexperimentalandIAI-onlyteachers,didnotexceed34%ofprogramslistenedto.Anotherreasonforlacknosignificantcorrelationsfoundmayalsobeattributedtohowknowledgewasmeasuredthroughteachers’dichotomous“yes”or“no”answers,contributingtoalackofvariabilityinresponses.Therefore,theknowledgeresultspresentedbelowwillbediscussedbycomponentskillsbutnoregressionresultswillaccompanythem.

Table21.Summaryofthegrade3and4teacherknowledgeresultsofcomparisonofmeansatendline(percentagesindicateagreement) PAQUED

CONTROL(n=54) Experimental(n=36) IAI(n=73)

PhonemicandPhonologicalawareness

94.4%*agree 89%agree 81.48%agree

Fluency 83.83%** 80.01% 72.24%

Vocabulary 72.79% 70.42% 67.59%

Comprehension 92.85% 91.67% 87.5%

Writing 77.3% 74.24% 71.6%

Integratingreadingandwriting

42% 33% 31%

Total 83.6% 81.08% 78.7%

*ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.05**ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***ThedifferenceinendlinemeansbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001

Overall,grade3and4experimentalandIAI-onlyteachersshowedtoexhibitmoreknowledgeabouttheeffectiveteachingofreadingthantheircontrolcounterparts.Thetablebelowdemonstratesthespecificquestionstowhichexperimentalteachers’answersweresignificantlydifferentfromtheircontrolcounterparts.

4Othertrainingattendance(summerinstituteparticipation)was98%acrosstheteacherpopulation.Thelackofvariabilityinattendancemakesitdifficulttolinktochangeinpracticeorendlineknowledge.5MeanIAIlistenershipforgrade3and4experimentalteacherswas39%whereasIAI-onlyteacherslistenedto32%oftheprogram.Grade5and6experimentalteachers’IAIlistenershipratewas35%andIAI-onlyteacherslistenedto32%oftheprograms.

55

Table22.Itemanalysisofthegrade3and4teacherknowledgeresultsofcomparisonofmeansatendlinedisaggregatedbystatus(percentagesindicateagreement)

PAQUEDCONTROL(n=54) Experimental(n=36) IAI(n=73)

1.4Tohelpstudentseasilyreadandwritewords,itisusefultoaskthemtocategorizewordsbycommonsounds,commonthemesorcommonendings

94%*agree 89%agree 81%agree

2.2Itisusefultotalkaboutnewvocabularywithstudentbeforereadingatext.

77%*** 50% 41%

2.4Itisbetterforstudentstolearnnewvocabularythroughastoryratherthaninlistform.

75%** 47% 47%

4.2Afterreadingatext,itisimportanttoaskstudentstoexplainwhattheyread.

92%* 86% 74%

5.1Itisokifstudentsmakespellingmistakeswhentheywriteanewwordforthefirsttime.

2.7%*** 16% 27.8%


Thetablebelowprovidesasummaryofgrade5and6teachers’knowledgeofteachingreading.Followingthesametrendasgrade3and4teachers,experimentalandIAI-onlyteachersshowedtoknowagreewithstatementsaboutteachingliteracythatwereinlinewitheffectivereadinginstruction.Table23.Summaryofthegrade5and6teacherknowledgeresultsofcomparisonofmeansatendlinedisaggregatedbystatus(percentagesindicateagreement) PAQUED



91.03%agree 84.6% 89%

Fluency 80.51% 76.61% 75.6%

Vocabulary 79.47% 77.93% 77%

Comprehension 93.88% 93.67% 89%

Writing 69.91% 69.77% 66%

Integratingreadingandwriting

50% 44.9% 44.7%

Total 82.89% 80.92% 79%


Table24pullsoutthosespecificquestionsthatwerefoundtowhichexperimentalteachersrespondedinasignificantlydifferentwaythantheircontrolcounterparts.

56

Table24.Itemanalysisofthegrade5and6teacherknowledgeresultsofcomparisonofmeansatendlinedisaggregatedbystatus(percentagesindicateagreement) PAQUED


2.1Tohelpstudentslearntoread,itisimportanttohavethemrepeatthereadingofatextafteryou.

76.92%*agree 86.57% 93.4%

3.1Beforeaskingstudentstoreadanewtext,itisusefultohaveadiscussionwiththeclasstobringoutwhattheyalreadyknowaboutthetheme.

92%* 88% 78%

3.2Itisusefultotalkaboutnewvocabularywithstudentbeforereadingatext.

74%* 52% 51%


Thedataoutlinedinthefourtablesabovewillbediscussedbycomponentskillinthefollowingsections.

Phonemicandphonologicalawareness:PAQUEDinterventionsforgrades3-6didnotfocusasintenselyondevelopingbasicdecodingskills(letter-soundrelationships,etc)astheydidforgrade1and2students.Thisisbecausestudentsingrade3,4,5,and6shouldalreadyhavedevelopedmanyoftheskillsassociatedwithlearninghowtodecodenewwords.WhileEGRAresultsdonotrevealthistoactuallybethecaseinDRC,thenationalcurriculumobjectivesassumestudentsarealreadystrongdecodersbygrade3,andtheMinistrymandatesthatdonor-fundedinterventionsalignwiththecurriculum.Therefore,itisnosurprisethatgrade5and6experimentalandIAI-onlyteachers’knowledgeaboutteachingphonemic,phonologicalandalphabeticawarenessdidnotdiffersignificantlyfromtheircontrolcounterparts.

Nevertheless,amonggrade3and4teachers,experimentalteacherstendedtoexhibitsignificantlymoreknowledgeonteachingthiscomponentskillincomparisontotheircontrolcounterparts(p=.052,d=.21,ES=.21).Thismaybeexplainedbyteachersattendingtheschoolbasedforumd’échangeandexchangingwiththeirgrade1and2experimentalcounterpartsaroundteachingtheseskills.Thisexplanationissupportedbyitemanalysisshowingexperimentalteachersingrade3and4agreedsignificantlymorewithQuestion1.4.(tohelpstudentseasilyreadandwritewords,itisusefultoaskthemtocategorizewordsbycommonsounds,commonthemesorcommonendings)(p=.052,d=.21,ES=.21)thantheircontrolcounterparts.Experimentalteachers’hightendencytorespondpositivelytothisparticularquestionisinterestingbecauseitrelatestoakeywordstudyactivityinthereadingprogramwhichwasreportedtobehighlydiscussedinschoolbasedlearningcircles(forumd’échange).

Thoughnosignificantdifferenceswerefoundacrossgroupsforthequestiononwhetherornotitwasusefulforstudentstolearntochunksofwordstoreadmorequickly,92%ofgrade3to6experimentalteachersagreedthatthiswasimportant.Someoftheclassroomexamplesteachersprovidedtosupportthisanswerwereasfollows:

57

Igiveawordtomystudentsandmystudentcutthemintosyllablesandthenreadthemquickly(N=74).Idrawmystudents’attentiontothewordfamily(root)weareworkingwith.Fromthere,theyknowthemeaningandcanreadtheword(N=9).

Othersgrade3to6teacherswhoagreedthatitisvaluabletoteachstudentstochunkwordsprovidedthefollowingjustifications:

Thedivisionofwordsintosyllableshelpsdrawoutthesoundsinthewordswhichassistsstudentsinbothreadingandwriting(N=4).Chunkingwordshelpsinstudent’sgoodpronunciationoftheword(N=5).Cuttingwordsupintosyllablesorsoundshelpsstudentsdecodedifficultwords(N=7).Ifstudentsknowthatwordsaredividedupintosyllables,theywillmoreeasilybeabletoreadit(N=8).

Experimentalgrade5and6teachers’responsesalsoreflectsomeinfluencefromthegrade1and2readingprogramstrategies,asinthisclassroomexample:

Afterareadinglesson,Ihavemystudentswriteawordthatcontainstheletterorspellingpatterntheystudied.

Grade3to6teacherswhodidnotagreethatchunkingwasusefultohelpstudentsreadquicklyjustifiedtheirresponsesinthefollowingway:

Onemustalwaysreadthewholewordwithoutcuttingitup(N=8).Grade5and6studentsdon’tneedtocutupwordstoreadthem(N=3).Notallwordshaveroots,andsyllablesareonlyusefulforslowdecoding(N=3).

Overall,theseresponsesshowthatthemajorityofinterviewedteachersseemtoplaceimportanceonsyllable-by-syllablereadingtohelpstudentsreadmorequickly.ForlearningtoreadinFrenchlanguage,thisisanappropriatestrategyandisonethatisexplicitlymodeledinthePAQUEDIAIprogramsaswellasinothertrainingsprovidedbyIFADEM.Fluency:FluencyisaskillthatPAQUEDinterventions,notablyIAI,soughttobuild.Strategiestobuildfluencyinvolvedaskingtheteachertodragtheirfingerquicklyunderwordstomovetheirstudents’eyesmorequicklyfromwordtowordorhavingstudentspracticereadingwordsinagivensentenceinorderandoutoforder.Otherstrategieswerelinkedtophonologicalawareness,likerecognitionofwordrootstohelpstudentsmorequicklychunkwordstoreadthemwhilesimultaneouslyassistingintheircomprehensionofthesewords.Finally,theIAIstoriesstrovetopresentagoodmodeloffluencyforteacherandstudentsalikeintheirread-alouds,attendingtofluidintonationandexpression.

Giventhisemphasis,whatwasteachers’knowledgeofteachingfluencyteatendline?OnlyGrade3and4experimentalteachersshowedsignificantdifferencesintheirknowledgeofteachingfluencyatendline(p=.002,d=-0.72,ES=.34)incomparisontotheircontrolcounterparts.Grade5and6experimentalandIAI-onlyteachersknewmoreaboutteachingfluencybutthedifferenceswerenotsignificant.

58

Vocabulary:BoththeIAIprogramsandtheFrenchsummerinstitutemodeledstrategiesforvocabularybuildingforteachers.Suchstrategiesincludedhavingteacherscollectideasfromstudentsaroundagiventheme,usemovementsandinstructionalmaterialsupport(illustrations,concreteobjects)todefinenewvocabulary,orhavingstudentsusenewvocabularyinsentencesorallyorbywriting.Becausestudentsarelearninginasecondlanguage,vocabularydevelopmentisvitaltoensuringreadingandlisteningcomprehension.Grade3to6experimentalandIAI-onlyteachersdidnotexhibitanysignificantdifferencesintheirknowledgeaboutteachingvocabularyatendline.However,specificquestionanalysisrevealedthatgrade3to6teachersdiddiffersignificantlyintheirresponsestocertainquestions.Forexample,75%ofGrade3and4experimentalteachersfeltisbetterforstudentstolearnnewvocabularythroughastoryratherthaninlistform(p=.005,d=.63,ES=.3).

Grade3to6teachersalsosignificantlydifferedintheiranswertothequestion:itisusefultoteachnewvocabularybeforethereadingofatextincomparisontotheircontrolcounterparts(p=.000,d=-0.82,ES=.38andp=.017,d=-0.52,ES=.25).Teachers’justificationsfortheirpositiveresponsestotheirquestionarepertinenttoexplicitstrategiesmodeledinPAQUEDinterventions:

Astudentwillbetterbeabletounderstandwhathereadsifheunderstandsthenewvocabulary.(N=43)

Suchjustificationpointstothefactthatteachersmayseethelinkbetweenvocabularyknowledgeandcomprehension.Alongthesesamelines,teachersassertedthatstudentswillbemorecuriousandattentiveintheirreadingiftheyunderstandthevocabulary.(N=9)

Teachersalsopointedtotheuseof“brainstorming”(collectedesidées)tohelpthembringoutnewvocabulary(N=14).ThisparticularactivitywasexplicitlymodeledintheFrenchsummerinstituteswhich98%ofPAQUEDteachersattended.Otherscitedtheusefulnessofaskingstudentsquestionsonthethemeofthetextinordertodeveloptheirvocabulary(N=13)whileotherindicatedthatillustrationswerehelpfulinexplainingnewvocabularypriortoreading(N=12).

Aboutaquarterofgrade3-6teachershowever,assertedthatitwasn’tusefultoteachnewvocabularybeforereadingatext.Thereasonsandexamplescitedinclude:

IalwaysstartwithreadingthetextfirstandthenIaskstudentstobringoutthedifficultwords(N=56).

“Newwordsshouldbetaughtduringthereadingofthetextandnotbefore;otherwise,thewordswillbetaughtabstractly”(N=10)

ThoughPAQUEDencouragedteacherstoteachvocabularypriortoreadinganewtext,themajorityofconcreteclassroomexamplesgivenshowedtohighlighttheimportanceofteachingofnewvocabulary.Thisispositivebecauseitmeansteachersdovaluetheteachingofnewvocabulary.

Comprehension:Asitistheultimategoaloflearningtoread,activitiesrelatedtobuildingreadingcomprehensionwerekeytothePAQUEDinterventiondesign.Theseactivitiesincludethe“questions”activitywherestudentsarealwaysaskedtoanswer“Who?What?When?Where?How?Why?”

59

(QQQOP)questionsfollowingthereadingofatext.SeveralIAIprogramswerededicatedtoshowingstudentshowtofindanswerstocertainquestionsusinglanguageandcontextclues.Attheendofeveryprogram,studentswerealwaysaskedwhattheylikedaboutthestoryorhowtheycouldrelatethestorytotheirownlives.Grade3to6experimentalandIAI-onlyteachersdidnotshowsignificantdifferencesintermsoftheirknowledgeofteachingcomprehensionascomparedtotheircontrolcounterparts.However,specificquestionanalysisrevealsthatgrade3and4experimentalteachersdifferedsignificantlyintheirresponsetoQuestion4.2afterreadingatext,itisimportanttoaskstudentstoexplainwhattheyreadortoanswercomprehensionquestions(p=.021,d=-0.51,ES=.24)incomparisontotheircontrolcounterparts.Grade5and6experimentalteachersdifferedsignificantlyintheirresponsetoQuestion3.1ontheimportanceofpre-readingactivities(p=.45,d=-0.41,ES=.2).Grade3and4teachersgaveclassroomexamplesandjustificationforwhyaskingstudentstoexplainwhattheyreadafterreadingatextwasimportantornot.Thosewhoagreedthiswasimportantcitedthattheyaskedstudentstogivethemainideaofatext(N=10).Otherssaidtheyaskedstudentscomprehensionquestions(N=21)becauseithelpedthemidentifywhetherornottheirstudentsunderstoodthetext(N=14).Teachersalsopointedtotheimportanceofaskingstudentstoexplainwhattheyreadbecauseithelpedsolidifytheinformationinthestudent’smemory(N=5).Finally,inlinewiththosewhobelievevocabularywasbesttaughtafterthereadingofatext,teachersexamplesincludedthedefinitionofnewvocabularyafterthereadingofatexttohelpstudentsexplainwhattheyread(N=10).However,somegrade3and4teachersdidnotfeelthatitwasimportanttoaskstudentstoexplainwhattheyread.Reasonsprovidedinclude:

Itisme(theteacher)whoshouldexplainthetextandthewordsread.Thestudentscanrepeatafterme.(N=9)

Studentsarenotcapableofexplainingwhattheyread.(N=5)Similarresponseswerefoundforgrade5and6teacherswhowereaskedwhetherornottheythoughtitisimportantforaskstudentstoreacttoatextorallyorinwriting.Thosewhoassertedthatitwasimportantgavethefollowingjustificationsthatpointtosomeinterestingfindings,includingteachersrecognizingstudentpreferencesandhowthey(students)liketolearn: Iaskmystudentstoreacttoatextorallyorinwritingbecauseitmotivatesthem(N=5).

Anotherexplanationshowsthatteachersvaluetextreactionbecauseithelpsthemtoevaluatestudentlearning:

Askingmystudentstoreacttoatextletsmeknowwhetherornottheyhaveunderstoodthetext(N=15)

Someteachersfeltitwasimportanttoaskstudentstoreacttoatextbecauseitwouldbringstudentstoformulatetheirpointofviewonagivensituation(N=8).

60

Thosegrade5&6controlteacherswhodisagreedwiththisstatementsaidtheydidn’tthinkstudentswerematureenoughtoanswercomprehensionquestions(N=3).Itwasalsointerestingtomentionhowthoseteacherswhoagreedwiththisstatementsaidtheyenactedthisstatementintheirclassrooms. Iaskmystudentstoreactbygivingmethemainideaofthetext(N=8). Iaskmystudentstotellmethemoralorthelessontheytookfromthestory(N=5).

Iposecomprehensionquestionstomystudentsandtheyanswerorallyorinwriting(N=21).Theseresponsesareconsistentwiththecomprehension-specificactivitiesembeddedwithintheIAIprograms.However,nocorrelationscanbeestablishedbetweenteachers’useofIAIandtheirresponsestothesequestions.Writing:WritingwasacorecomponentofthePAQUEDinterventionsdesignedforteachersandstudentsingrade3to6.Writingactivitiesinvolvedallowingstudentstoexperimentwithwritingbeyondsimplecopying,andtoengageinpair-reviewofdrafts.Forexample,everyIAIunitingrade3to6lessonscalledforstudentproductionsofparticulartextgenres(e.g.poem,letter,fable)modeledinthatunit.TheFrenchsummerinstitutesalsoincludedwritingactivities,whichmayhaveservedtoinfluenceteacherknowledgeofwritinginstruction.

Overall,grade3to6experimentalandIAI-onlyteachersdidnotshowsignificantdifferencesintheirknowledgeofteachingwritingincomparisontotheircontrolcounterparts.Still,itemanalysisrevealsthatexperimentalteachersingrade3and4didsignificantlydifferontheirtoleranceofinventedspelling:Itisokifstudentsmakespellingmistakeswhentheywriteanewwordforthefirsttime(p=.000,d=-0.87,ES=.4)incomparisontothegrade3and4controlteachers.Approximately50%ofGrade3to6experimentalteachersagreedthatitisappropriatetoteachreadingandwritinginthesamelesson,ascomparedtoapproximately40%ofcontrolteacherswhothoughtthiswasappropriate.Thoughthedifferenceacrossgroupsisnotsignificant,thereadingprogramforgrade1and2teachersdidintegratereadingandwritingintensively.Giventhis,itispossiblethatexperimentalgrade3to6teachersmayhavebeeninfluencedbytheirgrade1and2counterpartsinansweringthisquestion.

Anextensionquestiononteachers’perceivedimportanceofgivingstudentsopportunitiestowritewordsorsentencesthattheyproduceontheirownelicitedseveralinterestingresponsesthatmayshedlightonhowteachersarecomingtothinkaboutwritinginstruction.Classroomexamplesrangedfrommoreteacher-drivenactivitiestostudent-drivenactivities.Thoseexamplesprovidedforteacher-drivenactivitiesinclude:

Idospellingtests(N=27).Iaskmystudentscomprehensionquestionsonatextandtheyanswerinwriting(N=8).Studentswritewordstheysawinatextwejustread(N=26).Studentscopythetextofftheboard.(N=4).

Otherexamplesthatdemonstratedmorestudent-drivenproductionsincluded: Mystudentswritepersonalletters.(N=6)

Iaskmystudentstowritedownanywordtheyknowaroundagiventhemeorthathasaparticularspellingpattern(N=18).

61

Isendmystudentstotheboardtowritetheirideasonatext.Then,wecorrectittogether.(N=6).Usingillustrationsortheirowndrawings,studentscaneasilywritewhattheysee(N=4).

Grade3to6teacherjustificationsforprovidingwritingopportunitiesalsoemergedfromtheresponses.Inadditiontospellingtests,teachersexplainedthatwritingwasimportanttohelpthemevaluatetheirstudent’scomprehensionorlevel(N=5).Somealsosaiditfacilitatescomprehensionofthesubjectmatter(N=6)andhelpsastudentimprovetheirspelling(N=4).Writingwasalsolinkedtopromotingastudent’sinitiativeandwasconsideredtobeusefultotheirdailylives:convertingyouroralwordstothewrittenwordisneededinlife(N=9).Lastly,teacherexpectationsalsosurfacedfromgrade3to6teacherswhobothagreedanddisagreedthatprovidingopportunitiesforwritingisimportant.Forthosewhodidagree,teachersassertedthatonlythoseintelligentstudentswerecapableofwritingeventhoughtherewerestillmanyerrors(N=3).Forthosewhodidnotagreethatprovidingtheirstudentswithopportunitiestowriteisimportant,theyciteditwasbecausestudentsarenotcapableofwritinganythingthatcomesfromthem(N=6)andrather,studentshouldfirstseewhattheteacherwritesontheboardandcopy(N=2).Overall,thedataderivedfromquestionsaroundallowingstudentstopracticewriting,tomakemistakes,andtodowritingactivitieswithinthecontextofareadinglessonshowsthatteachersacrossgroupstakevariedstancesonwhatitmeanstoteachwriting.Giventhis,itisdifficulttodrawconclusionsonhowPAQUEDinterventionsmayhaveinfluencedteachers’knowledgearoundteachingwritingortheimportanceofallowingstudentstopracticewriting.

62

Grade3to6teacherpracticefindingsInadditiontoteachers’shiftsinknowledgeaboutteachingreading,itisalsoimportanttounderstandhowthesechangesmayhavetranslatedintopractice.Tomeasurechangesinteacherpractice,anobservationtoolsimilarlystructuredtothatemployedwithgrade1and2wasadministeredingrade3to6experimental,IAI-only,andcontrolteachers’classroomsatbaselineinMarch2013andatendlineinMay2014.Observationtoolsdifferedslightlybetweengrade3and4andgrade5and6teachersduetotheinevitabledifferencesinliteracypracticesassociatedwiththeselevels.

Sampledgrade3to6teacherswereaskedtoteachalessonwheretheyintroduceanewtexttostudentsatbothbaselineandendlinedatacollectioninordertoensureadegreeofcomparabilityofthelessons.Theobservationtoolcontainedarangeofspecificandobservablepracticesgroupedbythecomponentskillstheyaimedtobuild.Theseare:phonemic,phonologicalandalphabeticawareness,fluency,vocabulary,comprehension,andgeneralinstructionalpractices(seeAnnexBfortool).Thesepracticeswerechosenbasedonthoseoutlinedinthenationalstandardsandthosecommonlyobservedinnumerousclassroomobservationsconductedthroughouttheproject.Eachitemizedpracticewasallotted9five-minutetranchesoftime,whichcoversanaveragelessonspan.Iftheenumeratorwitnessedapractice,heorshewouldcheckofthepracticeintheappropriatetimeperiod.Thiswastoprovideasnapshotofthelessonasitprogressedandtoquantifyteachers’implementationofcertainpracticesoverothers.Itshouldbenotedthatevenifapracticewasobservedtwiceinaperiodoffiveminutes,onlyonecheckwasallowedperfive-minutetranche.Thisisapossiblelimitationofthetoolasithindersone’sabilitytodetectthesubtlechangesinteacherpractice.Still,theresultsderivedfromthetoolprovideinterestinginformationonteachers’practiceandwerefoundtobestatisticallyreliable(seeAnnexB).Thesummarytablesdescribethechangeingrade3and4and5and6teacherpracticesgroupedbycomponentskillacrossthebaselineandtheendline.Thetablesthatfolloweachsummarytablepresentspecificpracticesthatwereshowntochangesignificantlyovertime.Thepercentagesrepresentthetotalnumberoftimesthepracticewaswitnessedoverthetotallessontime.Forexample,ifateacheraskedcomprehensionquestionsovertwotranchesoffive-minutetimeperiodsofa35-minutelesson,theteacherwouldbeconsideredtohaveexhibitedthispracticeapproximately29%oftotalinstructionaltime(2outof7).ThelasttableshowstheresultsoflinearregressionanalysislinkingteacherchangesinpracticetotheiruseofIAIprograms.Unliketeachers’knowledge,someteachers’practicesweresignificantlycorrelatedwiththeiruseofIAIprograms.Overall,experimentalteachersimprovedsignificantlyintheinstructionofallcomponentskillsexceptforvocabularyovertime(longitudinally).Someoftheseimprovementswerefoundtobesignificantlydifferentincomparisontotheircontrolcounterparts.Inaddition,14.8%ofthevarianceingrade5and6experimentalteachers’totalchangeinpracticecanbeexplainedbytheirIAI-listenership(p=.005,d=.83,ES=.38).Finally,IAI-onlyteachersingrade5and6significantlyimprovedontheirtotalpracticesovertime(p=.009,d=.65,ES=.31).Thissectionwilldiscusstheseresults,breakingthemdownbyteachers’applicationofcomponentskills.Theanalysiswillprovidesomeinsightastowhyteachersmayhaveimprovedintheteachingofcertainskillsoverothers.

63

Table25belowprovidesasummaryofgrade3and4teachers’meanobservedliteracy-buildinginstructionalpracticesgroupedbycomponentskillcomposite.Italsoshowsthemeangainsteachersshowedtomakeoverbaselineandendlineineachcomponentskillandindicateswhenthesegainsarestatisticallysignificantbothlongitudinallyandacrossgroups.

Table25.Summaryofthegrade3and4teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus(percentageofinstructionaltimeallocated)



1.12% 3.9%2.78%***

1.39% 1.26%-0.13%

0.52% 0.89%0.37%

Fluency 13.19% 17.22% 4.03%** 19.22% 19.64% 0.42% 17.35% 17.59% 0.24%Vocabulary 5.4% 7.41% 2.01% 6.12% 7.36% 1.24% 6.2% 5.56% -0.64%Comprehension 8.47% 11.2% 2.73%** 7.56% 9.44% 1.88% 7.05% 8.31% 1.26%Generalinstructionalandliteracypractices 10.5% 13.24% 2.73%* 12.8% 13.05% 0.25% 9.76% 11.79% 2.03%Total 8.55% 9.79% 1.24%* 9.5% 10.06% 0.56% 8.18% 8.83% 0.65%

*Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.05**Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001___=significantacrossgroups(cross-sectional)

Thetablebelowpullsoutspecificpracticesoutlinedtheobservationtoolwheregrade3and4teachersshowedtomakesignificantgainsacrossbaselineandendline.Forexample,experimentalteacherstendedtointegratereadingandwritingintotheirlessons(P19)muchmoreatendlinethantheydidatbaseline.Table26.Itemanalysisofthegrade3and4teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus

PAQUEDCONTROL(n=46)

Experimental(n=48) IAI-only(n=72)Baseline Endline Gainscore Baseline Endline Gainscore Baseline Endline Gainscore

P1.Asksstudentstodecodewordsontheirownusingsound-letterassociations.

3.24% 5.09% 1.85% 1.38% 3.24% 1.85%* 1.69% 1.93% 0.24%

P4.Asksstudentstocorrectbadlyspelledwordsintheirownwritingorinwritingsontheboard.

0.23% 11.34% 11.11%*** 2.93% 1.85% -1.08% .96% 1.44% 0.48%

P13.Asksstudenttocompleteasentencewithamissingwordorallyorinwriting.

0.46% 3.24% 2.77%* 1.38% 1.7% 0.31% 1.69% 2.41% 0.72%

P18.Asksquestionsonatextread(ex.who?What?Where?How?Why?...)

20.13% 27.31% 7.17%* 16.82% 20.98% 4.17% 18.6% 21.5% 2.89%

P19.Integratesreadingandwritingactivitiesintothesamelesson. 2.31% 14.35% 12.04*** 3.54% 3.54% 0% 1.2% 3.1% 1.93%

P22A.Asksstudentstoworkindividuallyattheirdesks. 10.42% 19.21% 14.35%** 11.73% 14.97% 3.24% 7.48% 12.08% 4.58%


64

Figure30isavisualrepresentationofhowteachersspentinstructionaltimeatbaseline,endlineandtheirgainsoverbaselineandendline.Thisshowsthatgrade3and4experimentalteachersmadethelargestgainsacrossbaselineandendlineacrossmostcomponentskillsascomparedtotheirIAI-onlyandcontrolcounterparts.

Figure30.Grade3and4teachers’changeinliteracyinstructionalpracticesfrombaselinetoendline.

Experimental(N=48)IAI-only(N=72)

(N=78)Control(N=46)(N=60)

2.78-0.13

037

4.030.42

0.24

2.011.24-0.64

2.731.88

1.26

2.730.25

2.03

1.240.56

0.65

0% 5% 10% 15% 20% 25%



Fluency

Vocabulary

Comprehension

General

Total

Gainscore (pct)


65

Table27belowprovidesasummaryofgrade5and6teachers’meanobservedliteracy-buildinginstructionalpracticeacrossbaselineandendlineandtheirgainsineachcomponentskillcomposites.Thistablerevealsthatgrade5and6experimentalteachers’gainswerenotassignificantasthosemadebytheirgrade3and4counterparts.However,IAI-onlygrade5and6teachersdidshowtomakesignificantgainsintheirapplicationofphonologicalawarenessactivitieslikewordstudy.

Table27.Summaryofthegrade5and6teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus(percentageofinstructionaltimeallocated)



3.77% 6.92% 3.14%* 2.41% 5.56% 3.14%** 1.72% 2.3% .57%

Fluency 10.4 14.04% 3.64%*** 14.23% 15.57% 1.35% 13.55% 14.61% 1.05%Vocabulary 4.65% 8.35% 3.70%*** 6.09% 7.46% 1.37% 5.36% 7.09% 1.73%*Comprehension 12.0% 13.15% 1.15% 7.93% 12.0% 4.7% 7.71% 9.0% 1.29%Generalinstructionalandliteracypractices 11.7% 13.54% 1.84% 13.4% 13.62% 0.19% 7.85% 8.2% 0.35%TOTAL 9.23% 10.47% 1.24% 9.09% 10.57% 1.48% 7.45% 8.03% 0.58%


Thetablebelowpullsoutspecificinstructionalpracticesoutlinedintheobservationtoolwheregrade5and6teachersmadethemostsignificantgainsoverbaselineandendline.Forexample,grade5and6experimentalteachersseemedtointegratemoreofP2intotheirlessonswhenaskingstudentstocorrectbadlyspelledwordsusingtheCAPOT,arevisionstrategyexplicitlyproposedintheIAIprograms.Likewise,experimentalteachersalsoshowedtointegratesignificantlymorereadingandwritingactivitiesoverbaselineandendline.Table28.Itemanalysisofthegrade5and6teacherpracticeresultsofcomparisonofmeansbetweenthebaselineandtheendlinedisaggregatedbystatus


P1.Asksstudentstodecodewordsusingpartsofwordsalreadylearned(wordroots)

1.15% 2.64% 1.49% 1.46% 4.94% 3.48%*** 0.84% 1.9% 1.06%

P2.Asksstudentstocorrectbadlyspelledwordsontheboardorintheirclassmate’swritingsusingCAPOT—conjugation,accord,punctuation,andspelling.

5.56% 11.46% 5.90%* 3.9% 5.61% 1.71% 2.81% 2.34% -0.47%

P8.Asksstudentstospellhighfrequencywordsorwordsthey’vealreadystudied.

1.18% 4.59% 3.41%* 2.07% 2.37% 0.30% 1.69% 0.29% -1.40%

P11.Asksstudenttocompleteasentencewithamissingwordorallyorinwriting.

1.34% 4.4% 3.06%* 1.34% 2% 0.66% 0.98% 0.73% -0.25%

66

P12.Doesapre-readingactivitybeforereadingatext(ex.explainnewvocabulary,makepredictions)

8.92% 12.87% 3.95%* 10.98% 14.6% 3.62% 8.43% 8.91% 0.48%

P13.Asksstudentstofindsynonymsorotherwordstheyknowonagiventheme.

3.03% 10.23% 7.20%*** 5.49% 7.36% 1.87% 4.92% 9.06% 4.14%

P14.Solicitsideasandexperiencesfromtheirstudentsonwhattheyalreadyknowaboutasubject

11.44% 11.29% -0.15% 9.52% 13.1% 3.58%* 7.73% 8.18% 0.45%

P16.Asksstudentstoorderandexplainimportanteventsorinformationinatextusingagraphicorganizer.

6.5% 4% -2.5% 1% 3.1% 2.1%* 2% 2.8% .08%

P17.Guidesstudentstoformcompletesentences(orallyorinwriting)

5.89% 8.82% 2.93% 5.74% 9.36% 3.62%* 3.79% 4.82% 1.03%

P18.Integratesreadingandwritingactivitiesintothesamelesson.

2.02% 13.4% 11.38%*** 3.41% 6.86% 3.45%* 1.13% 3.07% 1.94%*

P22.Asksstudentstocategorizegroupsofwordsbyacharacteristic(samesound,sameletter,sametheme)

0.16% 1.59% 1.43%* 0.48% 1.99% 1.51% 0% 1.17% 1.17%


67

Figure31showshowgrade5and6teachersspentinstructionaltimeatbaseline,endlineandtheirgainsinthesecomponentskillcomposites.Acrossbaselineandendline,allgrade5and6teachersshowedtointegratethedirectinstructionofliteracy-specificcomponentskills.Still,meangainsweregreaterforgrade5and6experimentalteachersacrosscomponentskills.Figure31.Grade5and6teachers’changeinliteracyinstructionalpracticesfrombaselinetoendline.

ThetwofinaltablesbelowshowthesignificantcorrelationsidentifiedbetweenIAI-listenershipandteachersgainsintheapplicationofcertaincomponentskillscomposites.Table30showsthatgrade5and6experimentalteachers’gainsintheapplicationofphonologicalawarenessandgeneralinstructionalpracticeswerepositivelyandsignificantlycorrelatedwiththeiruseofIAIprograms.Forgrade5and6experimentalteachers,theonlyspecificpracticethatcorrelatedsignificantlywithIAIlistenershiprateswasP1,askingstudentstodecodewordsusingwordroots.Thispracticewasincludedinthephonologicalawarenesscomponentskillcomposite.

Experimental(N=53)

IAI-only(N=69)Control(N=58)

3.143.14

0.57

3.641.35

1.05

3.71.35

1.72

1.154.07

1.29

1.840.19

0.34

1.241.48

0.58

0% 2% 4% 6% 8% 10% 12% 14% 16% 18%



Fluency

Vocabulary

Comprehension

General

Total

Gainscore (pct)


68

Itshouldbenotedthatthesetableswerenotprovidedforgrade3and4teachersasnosignificantcorrelationsbetweenIAIusageandteacherspracticesemergedfromtheanalysis.Table29.Summaryresultsoflinearregressionforthegrade5and6changeinpracticeusingIAIdosageasapredictor Experimental

IAIdosageIAI–onlyIAIdosage

R2 Sig. R2 Sig.PhonemicandPhonologicalawareness

.102 .023 - -

Fluency - - - -Vocabulary - - - -Comprehension - - - -Generalinstructionalandliteracypractices

.112 .017 - -

Total .148 .005 - -Table30.Itemanalysisresultsoflinearregressionforthegrade5and6teacherobservationofinstructionalpracticesusingIAIdosageasapredictor Experimental

IAIdosageIAI–onlyIAIdosage

R2 Sig. R2 Sig.P1.Asksstudentstodecodewordsusingpartsofwordsalreadylearned(wordroots)

.117 .014 - -

*Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.05**Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.01***Thedifferenceinbaseline/endlinegainsbetweenPAQUEDandControlgroupteachersisstatisticallysignificantatp<.001

Thefollowingdiscussionwillfurtherexplorethedataoutlinedthetablesandfiguresaboveinordertocontextualizeteachers’gainsincertaininstructionalpracticesaroundthePAQUEDintervention.Foreaseofinterpretation,thediscussionwillbebrokendownbycomponentskillcomposites.

Phonemicandphonologicalawareness:Asmentionedpreviously,PAQUEDinterventionsdemonstratedphonologicalawarenessbuildingactivitiesinsofarastheyassistedstudentstoreadmorequicklyandefficiently.Thisisbecauseintheprimarycurriculum,itispresumedthatstudentsingrade3to6shouldhavealreadymasteredthebasicmechanicsofreading.Phonologicalawarenessactivitiesforgrades3-6includedwordanalysisforwordroots,identifyinghomonymsandhomographs,correctingspellingofwordsusingknowledgeofgrapheme-soundassociationsandgrammar,andcategorizingwordsbycommonendingand/orsound.

Acrossbaselineandendline,experimentalgrade3and4teachersimprovedsignificantlyintheirapplicationofphonologicalawarenessactivitiesintheclassroom(p=.000,d=1.26,ES=.53).Thischangewasalsosignificantlydifferentfromtheircontrolteachercounterpartswhoseemedtoshownosignificantchangeintheirapplicationofthesepractices(p=.001,d=-0.83,ES=.38).ExperimentalandIAI-onlygrade5and6teachersalsoincreasedtheirteachingofthisskillacrossbaselineandendline(p=.015,d=.7,ES=.33andp=.002,d=.79,ES=.38,respectively).Linearregressionanalysisalsoshowedthat10.2%ofgrade5and6experimentalteachers’gainsinthiscomponentskillcompositecanbe

69

explainedbytheiruseofIAIprograms(p=.023,d=.72,ES=.34).Thisisconsistentwithspecificpracticeanalysisshowingthatgrade5and6IAI-onlyteacherstoasktheirstudentstodecodewordsusingpartsofwordsalreadylearned(wordroots)(P1).11.7%ofexperimentalteachers’changeinapplicationofthispracticecanbeexplainedbytheirIAIusage(p=0.014,d=.72,ES=.34),asshownbelow.

Thoughnosignificantcorrelationswerefoundforgrade3and4teachers,IAI-onlyteacherswereshowntoapplyP1moreoftenoverbaselineandendlinebyaskingtheirstudentstodecodewordsontheirownusingsound-letterassociations(p=.022,d=.55,ES=.27).Grade3and4experimentalteachersgreatlyincreasedinP4,askingstudentstocorrectbadlyspelledwordsintheirownwritingorinwritingsontheboard(p=000,d=1.49,ES=.6).Similarly,grade5and6experimentalteachersdemonstratedanincreaseinP2,askingstudentstocorrectbadlyspelledwordsontheboardorintheirclassmate’swritingsusingCAPOT—conjugation,accord,punctuation,andspelling(p=.011,d=.73,ES=.34).Thegainscoreforthesetwoitemsforgrade3-6teacherswerealsofoundtobestatisticallydifferentfromtheircontrolcounterparts.Still,despitethissignificantincreaseintheirphonologicalawarenesspractices,bothIAI-onlyandexperimentalschoolsallocatedlessthan6%ofinstructionaltimetotheseactivitiesatendline.

Fluency:Fluencyactivitiesandinstructionalstrategiesforgrades3-6thatwereexplicitlymodeledintheIAIandtheFrenchsummerinstituteincludedhavingteachersdragtheirfingerunderwordstomovestudentseyesmorequicklyfromwordtoword,drawingstudentsattentiontovocalpausesandexaggerationswhenencounteringdifferentpunctuation,modelingfluentreading,holdingsilentreadingsessionsintheclassroom,andhavingstudentslearntoreadandwritehighfrequencywordsinFrench.Grade3to6experimentalteacherswereshowntosignificantlyincreasetheirapplicationoffluency-buildingactivitiesintheclassroom(p=.009,d=.8,ES=.37andp=.000,d=1.08,ES.48).Thesedifferencesingainswerealsostatisticallysignificantincomparisontotheircontrolcounterparts(p=.036,d=.45,ES=.22andp=.041,d=.4,ES=.19).Still,theirmeanapplicationoffluencypracticesintheclassroomremainedfairatendline,rangingfrom14to17%ofinstructionaltimeallocatedtofluency-buildingactivities.

0102030405060708090100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen

tageofteachers

IAI-u

sage

Experimental teachers'meanpercentageofgainsinP1

Figure32.Experimentalteachers'IAIusageandtheirgainsinPractice1(asktheirstudentstodecodewordsusingwordroots)

R2=.117sig=.014

70

Vocabulary:PAQUEDinterventionsfocusedonbuildingstudent’svocabularyknowledgeinFrenchthroughanarrayofpre-readingandwordstudyactivities.Morespecifically,activitiesentailedbrainstormingofwordsassociatedwithagiventheme;usingmovements,illustrationsormothertonguetodefinenewwords;employingclozetodevelopstudent’sattentiontocontextfordefiningnewwords;anddrawingattentiontosynonyms,homonyms,andhomographswhenreading.Vocabularypracticesdidnotseemtoshiftsignificantlyacrossbaselineandendlineformostteachersexceptforgrade5and6experimentalteachers,whoincreasedtheirdemonstrateduseofvocabularybuildingactivitiesintheirclassrooms(p=.000,d=1.11,ES=.49).Thisisconsistentwithspecificpracticeanalysiswhichdemonstratesthatgrade5and6experimentalteacherssignificantlyincreasedtheirapplicationofP11askingtheirstudenttocompleteasentencewithamissingwordorallyorinwriting(p=.022,d=.65,ES=.31).Otherpracticesgrade5and6experimentalteacherssignificantlyaugmentedwereP12--orchestratingpre-readingactivitybeforereadingatext(p=.017,d=.68,ES=.32)--andP13--engaginginmorewordstudyactivitiesonsynonymsordoingbrainstormingofotherwordstheyknowaroundagiventheme(p=.000,d=1.07,ES=.47).Whenlookingattimeallocatedtopracticeslikepre-readingactivities,itwasshownthatgrade3to6experimentalteachersspentanaverageof11%ofinstructionaltimeontheseactivitiesandIAI-onlyteachersspentanaverageof10%.Grade3to6controlteachersspentonly3%ofinstructionaltimeonpre-readingactivities.

Comprehension:ComprehensionstrategieswereembeddedinPAQUEDIAIandtheFrenchsummerinstitute.Activitiesforgrade3-6teacherssurroundingcomprehensionincludedteachershelpingstudentsconnecttheirpriorknowledgetonewinformationfoundinatext,askingdifferentlevelsofcomprehensionquestions,askingstudentstoorganizeinformationfoundinatexttohelpthemmakesenseofit,andaskingstudentstomakepredictionsonthecontentofatextbasedonclues.Overall,grade3and4experimentalteachersdemonstratedsignificantgainsintheapplicationofcomprehensionactivitiesintheclassroomacrossbaselineandendline(p=.007,d=.82,ES=.38)whereasIAI-onlygrade5and6teacherssignificantlyincreasedintheirapplicationofcomprehensionactivities(p=.001,d=.84,ES=.39).Specificpracticeanalysissupportsthisbyshowingthatgrade3and4experimentalteachersappliedP18moreatendline,askingtheirstudentsmorequestionsonatextread(p=.027,d=.82,ES=.38).Theseteachersspentapproximately30%ofinstructionaltimeaskingtheirstudentscomprehensionquestions.Grade5and6IAI-onlyteachersshowedmoreapplicationofP14,solicitingideasandexperiencesfromtheirstudentsonwhattheyalreadyknowaboutasubject(p=.022,d=.57,ES=,27);P16,askingstudentstoorderandexplainimportanteventsorinformationinatextusingagraphicorganizer(p=.015,d=,61,ES=.29);andP17,guidingstudentstoformcompletesentences(p=.027,d=.55,ES=.26).However,linearregressionanalysisfoundnostatisticallysignificantcorrelationsbetweenthesegainsandteachers’IAIusage.

Generalinstructionalandliteracypractices:Thepracticescontainedinthe“generalclassroomandliteracypractices”compositeincludetheincorporationofgroupwork,teachermonitoringofstudentwork,thepresenceofpositivestudentencouragement,andtheintegrationofreadingandwritingintothesamelesson.Grade3and4teachersinexperimentalschoolsimprovedsignificantlyontheirgeneralliteracyandinstructionalpracticesacrossbaselineandendline(p=.014,d=1.26,ES=.35).However,grade5and6teachersdidnotchangesignificantlyintheirapplicationofthesepractices.Still,forthesegrade

71

5and6experimentalteachers,11.2%oftheirgainsinthiscompositecouldbeexplainedbytheirIAIlistenership(p=.017,d=.7,ES=.33).

Asthe“general”compositeisfairlyvague,itisusefultoidentifywhichitemsrevealedthegreatestchangeovertime.Grade3and4experimentalteachersappliedmoreP19--integrationofreadingandwritingintothesamelesson(p=.000,d=1.52,ES=.6)--andP22A--askingstudentstoworkindividuallyattheirdesks(p=.006,d=.84,ES=.39).Grade5and6experimental,IAI-onlyandcontrolteachersalsotendedtoshiftlongitudinallytointegratereadingandwritingintothesamelesson(p=.000,d=1.26,ES=.53andp=.016,d=.6,ES=.29).Grade3-6experimentalteacherstendedtointegratereadingandwritingin13%ofthelesson,whereascontrolteachersonlyappliedwritingactivitiesin3%ofthelesson.Grade5and6teachersinexperimentalschoolsalsoshowedsignificantlymoreapplicationofwordcategorizationbycharacteristic(sound,theme,spellingpattern),andmeanapplicationtimeofthisitemintheclassroommovedfrom0%applicationto2%applicationofthispractice.Relatedtoteachers’generalpractices,itwasspecificallynotedintheobservationsthatgrade3to6teachersusedexamplesdirectlyembeddedwithinthenewmanualsdistributedbytheBelgianCooperation(CTB)in2011.Thissuggeststheyusethesemanualsintheirclassroomexamplesindicatesthattheyareusingthereadingmaterialsthatareattheirdisposal.

0102030405060708090100

-20 -15 -10 -5 0 5 10 15 20 25 30 35

Meanpe

rcen

tageofteachers

IAI-u

sage

Experimentalteachers'meanpercentageofgainsinGeneralLiteracypractices

Figure33.Experimentalteachers'IAIusageandtheirgainsinGeneralLiteracypractices

R2=.112sig=.017

72

Recommendationsforpolicyandpractice:TheresultsofthisstudyandRTI’s2014EGRAresultswerepresentedtotheDRC’sNationalReadingCommissioninAugust2014toarriveatcollectiverecommendationsforpolicyandawayforwardinreadingfortheDRCprimarygrades.Fortunately,thenewreadingandwritingstandardsandaccompanyingbenchmarkshavealreadybeendevelopedandharmonizedacrossorganizationsandvalidatedbytheMinistryofEducationinthispastyear,andthePAQUEDprogramreflectedthoseagreements.Therefore,therecommendationsreflectnotonlytheambitionsofthegovernment,butalsoexperiencefromaninitialefforttoimplementprogrammingalignedwiththosegoals.Groundedinrigorousdata,theyrepresentsoundandconstructivesuggestionsforwaystostrengthenreadingatscaleinthespecificcontextoftheDRC.

Theprimaryrecommendationderivedfromtheanalysisofthisstudyistheneedfornationalizedimplementationofarobustreadingprogramencompassingthefollowingaspects:

- Amulti-channeledteachertrainingprogramwhichincludessufficientinitialorientationandtrainingonhowtoimplementthenewreadingcurriculum;regularvisitsfromcoachesorfacilitators;andweeklyschool-based,teacher-ledmeetingsonreading.

- Appropriateandsufficientmaterialstosupporttheimplementationofthenewcurriculumincludingateacherreadingactivityguideandexamplelessonsplans;aweeklystructureforimplementation;ascopeandsequenceoutliningthetheme,newvocabulary,andphonicspatterns;accompanyingread-alouds;andappropriatelyleveledstudenttextsanddecodables.

- Acommunitytrainingcomponentthatensuresparentsandcommunitiesareinvolvedinimprovingtheirchildren’sreadingoutcomes.

Thediscussionbelowexpandsonconsiderationsnecessaryforadoptingsuchaprogram,includingtrainingmodalities,materialsdevelopment,communitymobilization,researchandevaluation,andtheneedforcontinuedinstitutionalcapacitybuilding.

Trainingmodalities:

- Continuewiththeteacher“forumd’échange”system.Asregressionanalysisshowedabove,teachers’participationincontinuingprofessionaldevelopmentactivitiesattheclusterandschoolbasedlevelcanleadtobetterteachingand,asaresult,betterstudentperformance.Thecontentdiscussedduringthesemeetingsshouldcontinuetorevolvearoundreadingandwritinginstructionandtheactivelearningstrategiesnecessaryforteacherstoactivelyengagestudentsintheirlearning.Focusgroupdatashowsthatteachers’participationintheseforumd’échangemeetingshelpedthemtofeelsupported;facilitatedtheirteachingmethodsanduseofmaterials;andallowedthemthetimetheyneededtoreflectontheirpractices,challenges,andtheirstudents’progress.

- Continuewiththecoachingmodelwhichservestoaccompanyandsupportteachersintheirapplicationofnewreadingandwritinginstructionalstrategiesandactivities.Intheexperimentalschoolexperience,coachingwasfoundtohelpteachersbuildconfidencein

73

applyingstrategiesandtomotivatethemtousethesestrategiesregularlyandsystematically.Overtime,teachersbecamelessdependentoncoachesformotivationbutcontinuedtorelyonthemtogainanunderstandingofhowtobetterapplystrategies.Thiscoachinghelpedteachersimprovetheirclassroompracticeandgainsoundknowledgeofhowtoteachreadingandwriting.Concretely,theCommissionsuggestedthatcoachesbeappointedas“trainers”intheofficialtrainingsystem.Thoughthismaybepossibleinthelonger-term,currently,‘itinerant’inspectorswhoseresponsibilitiescurrentlieinprovidingteacherspedagogicalsupportarelikelybestplacedtocarryoutthisrole.

- Reinforcethecapacityofthesystemtosupportteachers.Thoughthisdidnotcomedirectlyfromthedatapresentedabove,thereinforcementofinspectors’capacitiestosupportteachersintheapplicationofsoundteachingisessentialtoensureprogramsustainability,especiallybecauseinspectorsandschooldirectorswillultimatelyplaythe“coaching”rolepost-PAQUED.

- Usevideotoensurequalitytrainingonreadingandwritingactivities.Acascademodeloftraininginevitablyresultsinalteringtheendmessageteachersreceive.TheCommissionpointedtotheusefulnessofvideoforthoseteacherswhohaddifficultyorchestratingdifferentreadingandwritingactivitiesintheirclassroom..PAQUEDutilizedvideostohelpteachersvisualizewhatitistheyneedtodointheclassroomandshowthemhowtomakeanduselocallyfoundlowcost,no-costinstructionalmaterialsfortheirliteracyandmathlessons.Focusgroupdatarevealsthatthesevideoswereextremelyusefulforteachers,whenthetechnologyworked6.Therefore,itisrecommendedthatvideobeincorporatedintothereadingtrainingpackagetocomplementcoaching,materials,andcontinuingprofessionaldevelopmentactivities.

- Structureandsystematizethereadingprogramtodirectlyimpactimprovementinteachingpractice,knowledge,andstudentperformance.Inthedatapresentedabove,fidelityofimplementationofasystematicandstructuredweeklyreadingprogramservedtobethemostsignificantpredictorofteachers’knowledgeofteachingreadingandwritingandstudentperformance.Focusgroupdatashowedthat,duetotherepetitivenatureofactivities,teacherscametofeelmoreconfidentintheirapplicationandcouldfocusbeyondjustsimpleimplementation.Forcontinuingprofessionaldevelopment,thisisessential:reflectingonone’spracticeisshowntoleadtoimprovementonthatpractice.Furthermore,aweeklystructureprovidesaroutineforstudentsthatestablishesclearexpectationsand,especiallyinpost-conflictcontextsliketheDRC,leadstostudentwellbeing(IRC,2013).Studentsbeingabletoexpectthenextstepinalessonbuildconfidenceandgainasenseofnormalcywhichtheyrarelyexperienceoutsideoftheclassroom.TheNationalCommissionrecommendedthatthestructureofthereadingprogrambesustainedanddistributed

6Duetodelaysinthereleaseofthevideoplayersincustoms,severalvideoplayerbatteriesdiedandsubsequentlyaffecteduseofthevideoplayersinthefield.Severalbatteriesbutnotallbatterieswerereplaced.Thus,videoswerefoundtobeusefulwhenthebatterieswerefunctional.

74

beyondexperimentalschools.Thisisespeciallytimesensitiveasthenewreadingandwritingcurriculumbecomesmandatedthisschoolyear.Establishingaprogramwhichshowsteachershowtogoaboutteachingtothesenewstandardsinawaythatisnottoooverwhelmingwillbeakeyelementforthesuccessfuladoptionofthenewcurriculuminschools.

- Ensuremultipledeliverychannelstosupportteachersintheimplementationofliteracyinstructionalpractices.ThePAQUEDreadingprogramwassuccessfulbecauseitprovidedmultiplechannelsforbuildingteachers’knowledgeofteachingreading;helpingthemimplementliteracypracticesandstrategiesintheclassroom;providingthemwithaccompanyingtrainingandinstructionalmaterialsdirectlylinkedtothesestrategies;supplyingthemwithcoachingvisits,andencouragingpeer-to-peerexchangesaroundteachingreadinginbothschool-basedandschool-clusterforums.Inaddition,theprogrammirroredthenationalteachertrainingstrategy.Toensureteachers’successfuluseandapplicationofthenewreadingcurriculumintheDRC,itisthereforehighlyrecommendedthatthesemultiplechannelscontinuetobeexploited.

Materialsdevelopment:

- Makeavailablesufficientandappropriatelyleveledreadingmaterials,bothinclassroomsandforstudentstotakehometocontinuepracticingtheirreadingskills.Currently,themajorityofthebooksavailableinclassroomsarenotappropriatelyleveled,makingitdifficultforstudentstopracticereadingandforteacherstousetextsthatareatstudents’instructionallevels.ThePAQUEDreadingprogrammaterialsweredesignedtorespectthebenchmarksandlevelingcriteriadevelopedandvalidatedbytheMinistryofEducationin2013.Therefore,itwassuggestedthat,althoughthesematerialsarewritteninFrench,theycanstillserveasappropriatelyleveledreadingmaterialsforstudentstotransitionintoFrenchingrade3andshouldbewidelydistributed.Itwasalsorecommendedthattextsinnationallanguageshouldbedevelopedassoonaspossibleandappropriatelyleveledtextsforgrade3to6inFrenchshouldalsobedevelopedanddistributed.

- EnsureregularuseofIAIwithappropriatetechnologytoprovideusefulinstructionandtraining.ThePAQUEDprojectfacedsignificantchallengeswiththetechnologyselectedforthedeliveryofitsIAIprograms7.However,whenthetechnologyworkedandwhenteachersusedtheIAIprogramsregularly,datashowthattheydidcontributetoimprovementsinteachers’pedagogicalknowledgeandpractice,directlycontributingtostudentperformance.IAIprovidesauniformqualityofcontinuoustrainingandinstructiontoeveryone,whichisaparticularadvantageinavastanddiversecountryliketheDRC.

7ThedeliverymechanismselectedforIAIwasextensivelytestedatthebeginningoftheproject.Followingtesting,afinalproductwasselectedforlarge-scaleprocurement.Deliveredradiosexperiencedseveredelaysintheirreleasefromcustomswhichresultedinbatteryfailures.Thiswasonlyrealizedafterdistributionhadoccurred.Somebatterieswereimportedtoreplacethenon-functioningones.However,someradiobatteriescontinuedtofunction.Hence,whenradiosworked,theprogramswerefoundtobeusefulbyteachers.

75

Therefore,itwasrecommendedthatamarketstudybeundertakentoidentifycontext-appropriatetechnology(i.e.mobilephoneswithsolarpanels)andthatthosedevicesbeusedtodistributeIAIatalargerscaleandinsufficientnumberstomaximizeteacherandstudentuse.

Communitymobilization

- Clarifyandactivatetheroleofcommunitiesinsupportingimprovedreadingoutcomes.CommunitieshavelongbeenthebackbonetoeducationdevelopmentandpreservationintheDRC.Therefore,communityinvolvementisvitaltostudentsuccessinschooland,byextension,toreading.ItisrecommendedthatcommunityrolesandresponsibilitiesundertheCOPAsandCOGESstructuresbedefinedsothattheycancontributetoholdingtheschoolaccountableforprovidingtheeducationtheirchildrendeserveandneed.

- Trainparentsandcommunitiesinreading.Communitiesoftendon’tknowhowtheycanbesthelpimproveliteracyratesortheymaynotthinktheyhavetheresourcesormeans(financialandhumancapital)tosupportliteracy.Therefore,itisrecommendedthatcommunitiesbeprovidedwithtrainingandinformationonhowtheycancontributetobetteringtheirstudent’sliteracyrates.Trainingscanincludein-schoolandoutofschoolsupportliketheestablishmentofreadingclubs;providingparentsandsiblingswithsimpleliteracy-buildingactivitiestodowiththeirchildrenathome;orhelpingtocreateinstructionalmaterialsforliteracy(lettercards,wordcards,etc).

Researchandevaluation

- Conductresearchandevaluationtotrackprogressandkeepallactorsaccountable.Itwasrecommendedthatsufficientfinancialresourcesbeallocatedtoresearchandevaluationwithinthenationalreadingprogram.Itwasalsosuggestedthatstandardevaluationtoolstomirrornationalstandardsandbenchmarksbedevelopedandemployedtoevaluatestudentprogress.Teacherevaluationsbasedonteacherpedagogicalpracticestandardsshouldalsobedevelopedandshouldmirrorstudentevaluationssothatteachers’practicescanbealignedwithstudentlearningobjectives.Finally,theCommissionrecommendedthatastandardtoolbedevelopedtotrackcommunityactivities,astheyarecentraltoensuringstudentsuccessandattendanceinschool.

- Continuetoconductstudiessuchasthese,toinformpolicyandsupportthecontinuousimprovementoftrainingmodels.Inworkingthroughthedatapresentedinthisreport,theNationalReadingCommissioncametoappreciatethevalueofthistypeofinformationinunderstandinghowteachersteachandhowstudentsareaffectedbeteacherknowledgeandpractice.Therefore,futureresearchinitiativesareencouragedtocontinueinordertocontinuouslyinformthecommunityofpractice.

- Identifyandfurtherexplorethetrendsemergingfromexistingdataandfuturestudies.Inalldata,interestingandpertinenttrendstendtoemerge.Forexample,inRTI’s2014EndlineEGRAandEGMAreport,itemergedthatchildrenwhoseteachershad5yearsorlessofteachingexperiencesperformedbetteracrossallgroups(experimental,IAI-only,andcontrol).Sucha

76

trendbegsfurtherquestioningtobetterunderstandwhymorethanfiveyearsofteacherexperiencemaycontributetostudentsnotperformingaswell.SAsstudiesareundertakenanddeveloped,theyshouldstrivetoanswerthequestionsarisingfrompreviousresearchandevaluation.Thisalsorequiresthatstudyresultsareappropriatelyandwidelydisseminatedtolocalandinternationalstakeholders.

InstitutionalCapacityBuilding:

- Definetherolesandresponsibilitiesofallactorsinthesystem.Inorderforareadingprogramtobesuccessfullyimplemented,allactorsneedtounderstandwhattheirrolesandresponsibilitiesareandhowtheyareexpectedtocontributetoensuringitssuccess.IntheDRC,assistanceprogramsareoftencateredtothehigherpolicyechelonsoftheeducationsystemorattheschoollevel.Rarelyhaveprogramsaddressedthesystemasawhole.Therefore,itisrecommendedthatroles,responsibilities,andtrainingneedsinordertoeffectivelyexecutetheseresponsibilitiesbeputintoplaceforeveryactorfromthecentralMinistryleveltotheschooldirectorbespelledout.Forexample,thisstudyrevealedhowimportantcoacheswereinteachers’successfulapplicationandunderstandingofreadinginstruction.Sincethecoachingroleisnotcurrentlypartoftheeducationsystem,itissuggestedthatspecificrolesofinspectorsorclusterfacilitatorsincludethefunctionofareadingcoachforteachers.Thisroleneedstobedefinedindetailandtrainingandsupporthastobeprovidedtothem.Likewise,training,monitoring,andevaluationtools.Readingactivities,materials,andtrainingsalsoneedtobeharmonizedacrossexistingprojectssothatMinistryactorsacrossthesystemunderstandhowtheyfitintotheadvancementofacommongoal.

Conclusion:

Thisstudyconfirmsthatteachers’knowledgeandexpectationsofhowtoteachreadingandwritingcontributesubstantiallytostudents’readingperformance.Simplyaskingteacherstochangetheirpractices,whetherthroughgeneralinstructionsorhighlyscriptedlessonplans,ignorestheimportanceofhelpingthemunderstandthepedagogicalfoundationsofthepracticestheyareaskedtoadopt.Thereforeindesigningateachertrainingprogramonreading,itisessentialtoembedfrequentopportunitiesforteacherstoreflectinadditiontoensuringtheprogramitselfisaccessibleenoughtoallowforreflectionratherthanfrustration.

Howdoteachersimprovetheirknowledge?Thisstudysuggeststhattheylearnfromengaginginprofessionaldevelopmentactivities,includingexchangeswiththeirpeers,periodictraining,andcoaching.PAQUEDofferedarangeofprofessionaldevelopmentpathways,includingintensiveworkshops,peer-to-peercoachingandlessonpreparation,monthlyin-classcoachingandlearningcirclesformedamongneighboringschools.Furtherresearchmightexplorethecost-effectivenessoftheseandotherstrategiesforhelpingteachersimprovetheirknowledgeofreadinginstruction,astheseinvestmentsmaynotonlyhaveanimmediateimpactonstudentperformancebutmaycreatelong-termpositiveeffects,aswell.Thelessonslearnedalsocallforfurtherexperimentation,whichisalreadyatoppriorityfortheMinistryofEducationpriortothenationalroll-outofareadingprogram.

77

FurtherstudiesalsoneedtoconsiderthechallengesofconductingresearchinvastandfragilecountriesliketheDRC.Theseincludenotonlylogisticalandsecurityconsiderationsbutalsothehighratesofteacherattrition,whichmakeitdifficulttoconductlongitudinalstudies,andofstudentabsenteeism,whichputsintoquestionhowmanyofthestudentssampledwereactuallypresentformostlessons.Withthesevariablesinmind,futurestudieslikethisoneshouldsearchcaptureandcontrolforthistypeofdata.ThisisitisstillessentialtocontinuetoinformtheexcitingpolicyfrontintheDRC,decisionmakingandthedevelopmentofmaterialsandtoolsthatrespondtotheneedsandrealitiesoftheeducationsystem.

78

AnnexA.Methodology

Observation(practice)andinterview(knowledge)tools:

Toolwritingandadaptation:TheobservationtoolwasdesignedtoassesswhetherornotteacherswereusingspecificpracticesoutlinedintheexperimentalreadingprogramandembeddedwithintheIAIprograms.ThetoolwasusedadaptedfromexistingobservationinventoriesutilizedbyEDC.Timetranchesoffiveminuteswereintegratedinordertocapturetheextenttowhichpracticeswereusedandwhen.ThetoolwaspilotedintwoKinshasaschoolsbyateamoffivePAQUEDtechnicalteammembers.Post-pilot,thepracticestatementsonthetoolwererevisitedtoclarifycertainitemsthatremainedunclear,toremovethosewhichoverlapped,andtoaddessentialpracticeitemswhichappearedintheclassroombutwerenotoriginallycapturedinthetool.

TheknowledgetoolwasadaptedfromEDC’sBeliefsandInstructionalPracticesInventory(BIPI),whichwasdesignedtocaptureteachers’knowledgeandexpectationsoftheirstudentsinthedomainsofreadingandwriting.TheoriginalBIPIquestionnairewasconvertedtobeadministeredasaface-to-faceinterviewandselecteditemswereinterposedwithextensionquestionstoprovideadditionalvalidityandtoenrichteachers’simpleyesornoanswerswithjustificationsandclassroomexamples.ThetoolwaspilotedinaKinshasaschoolbyateamoffivePAQUEDtechnicalteammembers.Afterpiloting,thetoolwasadaptedtoclarifyquestionsthatwereconsideredproblematicandaddorremovequestions.

Training:Ateamoften“supervisors”weretrainedinKinshasaontooladministration.Asahigh-inferencetool,theobservationinstrumentrequiredahighdegreeofinter-raterreliability.Eachpracticeenumeratedinthetoolwasexplainedandconcreteclassroomexampleswereprovidedtotrainees.Thereafter,theywereshownmultiple15-minutevideoclipsofrealCongoleseclassroomsandgivenopportunitiestousetheobservationtooltocheckoffthepracticestheywitnessedineachfive-minutetranche.Followingeachvideoclipviewing,pairsofobserversexchangedtheirratingsofthepracticestheysaw,andwheretherewasdisagreementinwhatwasobserved,theywouldjustifytheirratingsuntilaconsensuswasreached.Asimilarprocessofconsensus-buildingwasthenappliedinaplenarysessioninwhicheachpairpresentedtheirresults.Ifotherpairsdidnotsharesimilarmarks,justificationswereprovidedandconsensuswasreachedofwhatcertainpractices“lookedlike”.Thisprocesswasrepeateduntilconsensuswasachievedamongstdatacollectors.Forthetrainingontheknowledgeinterviewtool,eachquestioninthetoolwasreadaloudandclarificationsonthequestionwereprovided.Trainingwasalsoprovidedonestablishingrapportwiththeinterviewees,emphasizingtheneedtomaketheinterviewenvironmentcalm,distantfromdistractionsandpotentialinfluenceslikeateachers’superiororpeer,andtokeeptheinterviewer’sreactionstoresponsesnon-judgmental.Trainingonhowtowritesummariesofclassroomexampleswithoutmisrepresentingteachers’opinionswasalsoprovided.Enumeratorsthenpairedoffandeachtookturnsadministeringtheinterviewforallthree“degré”levels.

38enumeratorswereselectedandtrainedonboththeobservationandinterviewtoolbythesupervisorsusingacoachingguidebasedoffthetrainingtheyhadreceivedthemselves.Followingthistraining,enumeratorswerepairedbytheirsupervisorsandsenttoschoolstobegintooladministration.Duringeachobservation,everyenumeratorwasinstructedtofillintheirobservationtoolindividually

79

accordingtowhattheysaw.Aftereachobservation,thepairswouldgettogetherandcomparetheirtools.Wheretheirobservationsdiffered,enumeratorswouldengageinthesameprocessofjustificationperformedintraining.Whenconsensuswasreached,thepairwouldfillinaconsensusobservationtoolandstapleittotheirindividualtools.Atbaselineandendlineanalysis,theseconsensustoolsandindividualtoolswerecompared.Inadditiontothis,10%ofobservationswerefilmed,scoredseparatelybytheoriginaltrainer,andcomparedtothescoresofthefieldenumerators,tomaximizeinter-raterreliability.However,nointer-raterreliabilitystudywasconducted.

Forinterviews,eachenumeratoradministeredthesameinterviewtoolface-to-faceandone-on-onewiththeteacher.Extensionquestionresponsesweresummarizedfollowingaprocessofrepeatingbacktotheteacherverbatimtheexampleprovidedandthensummarizingit.Iftheteacheragreedwiththesummary,thedatacollectorwouldnotethissummary.Ifagreementwasnotreached,theteacherwouldbeaskedtoprovideasummaryofwhattheyintendedtosayandthiswouldberecorded.

Teacherselection:Atbaseline,schoolswererandomlyselectedfromschoolsidentifiedinRTI’s“accessibleschool”samplefromtheXXXXEGRA.Classsectionswithintheseschoolswerealsorandomlyassignedbygrade-levelandbystatus(experimental,IAI-only,andcontrol).Enumeratorsweregivenalistofclasssectionstovisitineachschool.Teachers’nameswererecordedaftertheywereobservedandretainedinadatabasesothattheycouldbesimilarlyobservedattheendline.

Atendline,twoteacherswhotookpartintheobservationfromeachgrade-levelwererandomlyselectedforinterviews.

Reliabilityanalysis:Astatisticalanalysisoftestreliabilitywasusedtodescribeaninternalconsistencyofeachtool,andisbasedonthecorrelationsbetweendifferentitems(subtests).InternalconsistencyofthetestwasmeasuredwithCronbach’salphawhichistheresultofpairwisecorrelationsbetweenitems.Cronbach’salpharangesfromzeroto1,wherezerodenotesanabsenceofanycorrelationacrossitemsonthetest,and1denotesaperfectcorrelationacrossitems.AtypicalandacceptablerangeforCronbach’salphaisabove.8.Agoodinternalconsistencyofanobservationtoolmeansthatateacherwhoshowstoexhibitoneparticularfluency-buildingpracticewouldalsodemonstrateothertypesoffluency-buildingpracticesoutlinedintheobservationtool.

Atestofinternalconsistencyoftheobservationtoolsfordifferentgradefoundthattheoveralltoolreliabilitywashigh,especiallyforthegrade1and2tool(Cronbach’salpha=0.81forgrade1and2,0.7forgrade3and4and0.71forthegrade5and6tool).Theitemlevelanalysisforbothgrade3and4and5and6observationtoolsshowedthatphonologicalawarenesspracticesdidnotcorrelatewellwithotheritems.Ifweremoveitfromthetest,theCronbach’salphawillgoupto0.75.

Fortheinterview(knowledge)tool,asimilartestofinternalconsistencyfoundtheoverallreliabilitytobeaverage(Cronbach’salpha=0.62forgrade1and2,0.56forgrade3and4and0.51forthegrade5and6tool).Thisonlyincludesitemsthatrequiredyesornoanswersasextensionquestionresponsescouldnotbecapturedbytheanalysis.Therefore,whenjudgingthereliabilitymeasureonthistool,itshouldbeconsideredthatadditionalinformationbeyondthedichotomousyesornoanswersisobtainedfromthistooltherebyallowingforadegreeofvalidationtotheanswersteachersprovided.

80

Readingassessment:

Thereadingassessmentwasdesignedtoprovideasnapshotofstudent’sreadingcapabilities.Becausethegrade2EGRAadministeredbyRTIdidnotincludeafluencyassessment,itwasalsodeemednecessarytoincludeoneinthePAQUEDstudy.Thisassessmenttookanaverageof5minutestoadministerandincludedthefollowingsub-tests:

- Randomalphabetletterreadingsubtestassessedstudents’knowledgeofletternamesintheFrenchalphabet.Studentswerepresentedwith26lowercaselettersplacedoutoforderandaskedtoidentifythenamesofeachlettertheysaw.Inadditiontoletternames,lettersoundswerealsoacceptedascorrectanswers.Thesubtestwasuntimedthoughstudentsweregivenonly3secondstoidentifyeachletter.

- Highfrequency/familiarwordreadingsubtestassessedstudents’sightvocabularyknowledgeofhighfrequencyFrenchwords.Recognizingfamiliarwordsiscriticalfordevelopingreadingaccuracyandautomaticity.Inthissubtest,studentswereaskedtoidentify8wordsthatwererandomlygeneratedfromalistof580mostcommonwordsintheFrenchlanguage.Studentswereaskedtoreadeveryword.Thesubtestwasuntimedthoughstudentsweregivenonly3secondstoidentifyeachletter.

- Readingofaconnectedtextsubtestassessedstudents’readingaccuracyandautomaticityinreadinga26wordpassagealoud.Thesubtestwastimedbutnotcappedat60seconds,allowingforthestudenttoreaduntiltheend.Thisyieldedascoreofcorrectwordsperminute.

AgroupofadministratorsdrawnfromthePAQUEDtechnicalteamfromKinshasaandafewfieldagentsweretrainedontestadministrationfollowingaspecificprotocol(seeannexX).Thetestwaspilotedtoassesstheconnectedtext-levelwitharandomlyselectedgroupofgrade2classesinMbandaka,KisanganiandKikwit.Overall,90studentswerepartofthepilot.Followingthispilot,thetextwasadjustedandappropriatelyleveledinordertocapturereadingresultsfromamajorityofstudentsandtoavoidlargenumbersofzeroscores.

Sampling:InJune2014,testadministratorsreceivedrefreshertrainingandwereinstructedtoadministerthetestandrandomlysample6students(3girlsand3boys)fromeachteacherinterviewedattheendline.Studentswererandomlyselectedfromtheteachers’classlisttoassess.ResultswerethenenteredelectronicallyusingSurveyToGoinordertominimizedataentryerrors.

Reliabilityanalysis:Astatisticalanalysisoftestreliabilityisusedtodescribeaninternalconsistencyofthereadingassessment.Thetestofinternalconsistencyofthereadingassessmentfoundthattheoveralltestreliabilitywashigh(Cronbach’salpha=.871).

Reading Assessment Reliability

Subtests Item-TotalCorrelation

Cronbach'sAlphaifItemDeleted

1. alphabetletterreading .732 .8312. familiarwordreading .823 .7983. Connectedtextreading .881 .766

81

DataAnalysisAllcollecteddatawerecleanedbyEDCM&Estaffandanalyzedusingstandardstatisticaltechniques,suchasunivariateandbivariatestatistics,asneededfordifferentanalyticalpurposes.Theresultsweredisaggregatedbysexandprovince,asappropriate.Centraltendencyanalysis(e.g.mean,median)wereconductedforcontinuousdemographicvariables.Comparisonofmeansstatisticaltests(pairedandindependentsamplest-test)wereconductedtoestimatedifferencesbetweengroupssuchasprovinceandsex,whereappropriate.Bivariatestatisticalanalyses(e.g.,correlations)wereconductedtoexaminetherelationshipbetweendifferentvariables.

StudyLimitations:Thestudypresentedafewlimitationswhichmayhaveimpactedtheresultsofthestudy.First,thesamplesizeforthereadingassessmentwasquitesmalltherefore,differencesacrossgroupsweremoredifficulttodetect.Anotherpieceofdatawhichwouldhavebeenusefulinexplainingstudentperformanceresultsisstudentattendancedatainschool.Itisdifficulttoextrapolatethedegreetowhichteachers’practices,knowledge,andfidelityofimplementationofvariousPAQUEDinterventionshadimpactonstudentperformancewhenthereislackofinformationonhowoftenstudentattendedschooltobenefitfromthesefactors.Infutureresearchstudies,dataforthisvariableshouldberoutinelycollected.Secondly,teacherattritionacrossbaselineandendlinewashighacrossgradelevels(41%forgrade1and2,35%forgrade3and4and24%forgrade5and6teachers)forameanof33.7%attritionforallteacherssampled.Thoughteacherswhowerenotretrainedwerereplaced,thisreductionofmatchedsamplesizereducedthestatisticalpossibilityofdetectingdifferencesinchangeinteacherperformanceovertime.Finally,thoughinter-raterreliabilitywasaccountedforintooladministrationthroughconsensusreaching,nointer-raterreliabilitystudywasundertakenwithenumerators.

82

AnnexB.ToolsReadingassessment:

Instrumentd’appréciationdeperformanceenlectureaudegréélémentaire

Classede2èmeannéeprimaire Dated’administration:___/___/______

A. Questionàposeràl’enfant.a. Âgedel’enfant…………………..Annéescolaire…………………………………b. Classe(ex.2A)…………………………………….Ecole………………………………..

Nomdel’enseignantdel’enfant:…………………………………………..c. Est-cequesonenseignantutiliseunlivreavecdesimagesaumomentoùilleurraconteou

leurlisedeshistoires/contes?OUI NONd. Est-cequesonenseignantleurdonnedespetitslivresavecimagespourqu’ilslisentseuls?

OUINONB. Test/Appréciationdel’acquisdel’alphabet.

Consigne:combiendelettresl’enfantpeut-ilidentifiercorrectement?• Sil’enfantprendplusdetroissecondespouridentifierunelettre,demandez-luide

passeràlaprochainelettre.• L’enfantlitligneparlignedegaucheàdroite.• Acceptezlesonoulenomdelalettre.• Surcettefichederéponses,encerclezleslettresincorrectes.• Danslacaseendessous,mettezlenombredelettrescorrectementidentifiées.

k d x h r i u j b z m c sɡ o q e t ɑ n v y l w f p

C. Suivezlamêmeméthodepourl’exercicesuivant.L’enfantdoitlirerapidementcesmotsfréquemmentutilisésdanslalanguefrançaise.

le ɑvec un de moi cɑr pour est

D. Lecturedetexte:Soulignezlesmotslusincorrectement.Sil’enfantprendplusdecinqsecondesàlireunmot,demandez-luidepasseraumotsuivant.Chronométrerletempsqu’ilprendpourlelireetenregistrerletempsci-dessousensecondes.

Mon petit chat joue dans le jardin. Il a vu une souris. Il se cache. La souris est là. Il saute et il mange la souris.

/26

/8

/26

sTempsdelecture(ensecondes):

83

Observation(practice)tools

Grade1and2observationtool

Diagnosticdelaclasse:Nomdel’observateur:___________________________________ Date:______________

Classe(ex.1eC)1e___2e___ Nometprénomdel’enseignant_____________________________________

Sexedel’enseignant(encerclez):FM

Nombredefille______

Nomdel’école__________________________

L’heurede_____à_____

Nombredegarçon_____

Sousdivision____________________________________

Sujet:________________

Dessindelaclasse(fille= garçon= )

PRATIQUES DE CLASSE L’enseignant(e)…

Appliquée? (mettez un X si vous observez la pratique)

84

0 à

5 mns

5 à 10

mns

11 à 15

mns

16 à 20

mns

21 à 25

mns

26 à 30

mns

31 à 35

mns

36 à 40

mns

N/A

CONSCIENCE PHONÉMIQUE/PHONOLOGIQUE 1. Demande aux élèves d’identifier

et de compter les sons/syllabes dans un mot.

2. Demande aux élèves de dire ce qui est pareil (rime, son, prononciation) dans une liste de mots.

3. Demande aux élèves de citer tous les mots qu’ils connaissent et qui commencent par un son précis ou qui riment avec un autre.

4. Demande aux élèves de corriger des mots mal orthographiés dans ses propres écrits ou dans les écrits au tableau.

5. Demande aux élèves de remplacer le son du début d’un mot par un autre son pour former un nouveau mot (ex : mère/père).

6. Demande aux élèves d’identifier le ou les sons au début et à la fin d’un mot.

7. Montre aux élèves comment écrire les lettres de l’alphabet, les diphtongues, ou les syllabes.

8. Aide les élèves à apprendre/identifier les noms et les sons de différentes lettres.

FLUIDITÉ 9. Pointe les lettres, les syllabes ou

les mots pendant qu’il lit ou pour guider les élèves à lire.

10. Attire l’attention des élèves à la ponctuation (point, point d’interrogation…) lorsqu’ils lisent.

11. Demande aux élèves de lire à haute voix ….

tout seul

Par paire ou par banc

85

tous ensemble

12. Fait lire rapidement aux élèves des lettres, des syllabes, ou des mots fréquents au tableau ou sous forme de cartes éclairs.

VOCABULAIRE 13. Explique ou demande aux élèves

d’expliquer du nouveau vocabulaire avant la lecture d’un nouveau texte.

14. Demande aux élèves de donner le sens d’un mot avec des gestes, des dessins ou à l’aide des matériels didactiques.

COMPRÉHENSION 15. Demande aux élèves de donner

leur prédiction sur le contenu d’un texte en se servant des indices (page couverture, images, titre, contexte).

16. Pose des questions aux élèves sur un texte lu. (Ex. Qui, Quoi, Où…)

17. Sollicite les idées et expériences de ses élèves (accéder à la connaissance antérieure et faire le lien avec la vie des élèves ou d’autres matières)

GENERALES

18. Intègre des activités de lecture et d’écriture dans la même leçon (ex. les élèves écrivent le son qu’ils entendent/apprennent)

19. Veille sur la participation des élèves. (COMPTEZ ET METTEZ LE NOMBRE D’ELEVES QUI NE PARTICIPENT PAS!! Ex. 9/55 élèves)

20. Lorsque les élèves sont en groupe, en paire ou travaillent individuellement, l’enseignant circule pour aider les élèves.

21. Demande aux élèves de travailler…

86

tout seul

en groupe ou en paire

En plénière

22. Demande aux élèves de former des groupes de mots selon une même caractéristique (même son, même lettre, même thème...)

23. Encourage les élèves de manière positive lorsqu’ils fournissent un effort.

87


Diagnosticdelaclasse:

Nomdel’observateur:_________________________________________ Date:______________

Classe(ex.3eB)3e___4e___ Nometprénomdel’enseignant_____________________________________


Nombredefille______

Nomdel’école__________________________ L’heurede_____à_____


Sousdivision________________________________________

Sujet:________________


88

PRATIQUES DE CLASSE

L’enseignant(e)…

Appliquée? (mettez un X à chaque fois que vous observez la pratique) 0

à 5 mns

5 à 10

mns

11 à 15

mns

16 à 20

mns

21 à 25

mns

26 à 30

mns

31 à 35

mns

36 à 40

mns

N/A

CONSCIENCE PHONÉMIQUE/PHONOLOGIQUE

1. Demande aux élèves de décoder des mots en utilisant les associations son/lettres.

2. Demande aux élèves de dire ce qui est pareil (rime, son, prononciation) dans une liste de mots.

3. Demande aux élèves de citer tous les mots qu’ils connaissent et qui commencent par un son précis ou qui riment avec un autre.

4. Demande aux élèves de corriger des mots mal orthographiés dans ses propres écrits ou dans les écrits au tableau.

5. Demande aux élèves de remplacer le son du début d’un mot par un autre son pour former un nouveau mot (ex : mèreàpère).

FLUIDITÉ

6. Pointe les mots pendant qu’il lit ou pour guider les élèves à lire.

7. Attire l’attention des élèves à la ponctuation (point d’interrogation, exclamation, point, virgule) lorsqu’ils lisent.

8. Demande aux élèves de lire à haute voix…

tout seul

en paire ou par banc

tout ensemble

89

9. Fait un modèle de lecture avant que les élèves lisent.

10. Fait lire rapidement aux élèves des mots fréquents ou des tranches de phrases fréquentes au tableau ou sous forme de carte éclair.

VOCABULAIRE

11. Explique ou demande aux élèves d’expliquer du nouveau vocabulaire avant la lecture d’un nouveau texte.

12. Demande aux élèves de donner le sens d’un mot avec des gestes, des dessins, ou en l’utilisant dans une phrase.

13. Demande aux élèves de compléter une phrase par un mot manquant à l’oral et à l’écrit.

COMPRÉHENSION

14. Demande aux élèves de donner leurs prédictions sur le contenu d’un texte en se servant des indices (page couverture, images, titre, contexte).

15. Demande aux élèves d’ordonner des phrases (début, milieu, fin).

16. Sollicite les idées et expériences de ses élèves (accéder à la connaissance antérieure et faire le lien avec la vie des élèves)

17. Guide les élèves à former des phrases complètes (à l’oral ou à l’écrit).

18. Pose des questions aux élèves sur

90

un texte lu. (Ex. Qui, Quoi, Où, Pourquoi ?)

GENERALE

19. Intègre des activités de lecture et d’écriture dans la même leçon (ex. les élèves écrivent le mot qu’ils entendent/apprennent)


21. Lorsque les élèves sont en groupe, en paire ou travail individuellement, l’enseignant circule pour aider les élèves.

22. L’enseignant demande aux élèves de travailler….

Tout seul


En plénière

23. Demande aux élèves de former des groupes de mots selon une même caractéristique (même son, même lettre, même thème, etc.)


91


Diagnosticdelaclasse:

Nomdel’observateur:__________________________________ Date:______________

Classe(ex.6A)5e___6e____ Nometprénomdel’enseignant_____________________________________


Nombredefille______

Nomdel’école__________________________ L’heurede_____à_____


Sousdivision________________________________________

Sujet:________________



Appliquée? (mettez un X à chaque fois que vous observez la pratique)

92


Appliquée? (mettez un X à chaque fois que vous observez la pratique)

0 à

5 mns

5 à 10

mns

11 à 15

mns

16 à 20

mns

21 à 25

mns

26 à 30

mns

31 à 35

mns

36 à 40

mns

N/A

CONSCIENCE PHONÉMIQUE/PHONOLOGIQUE

1. Demande aux élèves de décoder des mots en utilisant des parties de mot déjà acquis (racines).

2. Demande aux élèves de corriger des mots mal orthographiés (au tableau ou de leur ami en utilisant le CAPOT—conjugaison, accord, ponctuation, orthographe).

FLUIDITÉ

3. Attire l’attention des élèves à la ponctuation (point d’interrogation, exclamation, point, virgule, guillemets) pour aider les élèves à lire avec un bon débit et rythme.

4. Fait lire rapidement aux élèves des mots fréquents ou des tranches de phrases fréquentes au tableau ou sous forme de carte éclair.

5. Fait un modèle de lecture avant que les élèves lisent.

6. Demande aux élèves de lire à haute voix ….

tout seul

Par paire ou par banc

tous ensemble

7. Demandez aux élèves de lire silencieusement un texte.

8. Demande aux élèves d’orthographier des mots fréquents et des mots déjà vus.

93

VOCABULAIRE

9. Demande aux élèves de donner la définition d’un mot ou d’une expression avec des gestes ou en l’utilisant dans une phrase.

10. Fait des gestes ou définit de nouveaux mots ou expressions.

11. Demande aux élèves de compléter une phrase par un mot manquant à l’oral ou à l’écrit.

12. Mène des activités de pré lecture avant de lire un texte (expliquer du nouveau vocabulaire, faire des prédictions).

13. Demande aux élèves de trouver des synonymes ou d’autres mots qu’ils connaissent sur un thème.

COMPRÉHENSION

14. Sollicite les idées et expériences de ses élèves (accéder à la connaissance antérieure et faire le lien avec la vie des élèves)

15. Pose des questions aux élèves sur un texte lu. (Ex. Qui, Quoi, Où, Pourquoi ? Comment ?)

16. Demande aux élèves d’ordonner et d’expliquer les évènements importants dans un texte (début, milieu, fin, d’autres éléments du texte, problème, solution) à l’aide d’un schéma.

17. Guide les élèves à former des phrases complètes (à l’oral ou à l’écrit).

94

GENERALE

18. Intègre des activités de lecture et d’écriture dans la même leçon (ex. les élèves écrivent un mot pour compléter une phrase, les élèves écrivent une phrase qui résume un récit)


20. Lorsque les élèves sont en groupe, en paire ou travail individuellement, l’enseignant circule pour aider les élèves.

21. L’enseignant demande aux élèves de travailler …


tout seul

En plénière

22. Demande aux élèves de former des groupes de mots selon une même caractéristique (même son, même lettre, même thème, etc.)


95

TeacherInterview(Knowledge)tools:

Grade1and2interviewtool:

Consentement:Jevaisvousposerquelquesquestionssurlalecture,l'écritureetd'autrespratiquesdeclasse.Jevouspriederépondrehonnêtementetselonvous.Iln'yapasdebonneoudemauvaiseréponse.Sivousn'avezpasd’avis,cen’estpasgrave.Sivousnecomprenezpasunequestion,s'ilvousplaîtfaiteslemoisavoir.Sivousnevoussentezpasàl'aise,vousn'avezpasàrépondre.Cen'estpasuneévaluationpourvous.Pouvons-nouscommencer? □Oui □Non

Date(jour/mois/année) |__|__|/|__|__|/|__|__|__|__|

Province(encerclez) BANDUNDUORIENTALEEQUATEURSous-Division(encerclez) KikwitKisanganiMbandaka

GunguIsiroGemenaKengeBuniaZongoMasi-ManimbaBoendeBandundu-villeGbadolite

NomdeL’Ecole

---------------------------------------------

Nomdel’enseignant ---------------------------------------------

Classeenseignée(encerclez) 1e2e

Sexedel’enseignant(encerclez) FM Nombred’année

enseigné-------------------

Nomdel’enquêteur ---------------------------------------------

Nomdusuperviseur ---------------------------------------------

Débutdel’entretien |__|__|:|__|__| H H M M

1. LalectureengénéraleCommençonsàdiscutersurcequevouspensezdel’enseignementdelalectureengénérale.Sivousavezbesoind’uneexplicationsurunequestion,n’hésitezpasàmedemander.Allons-y!

1.1. A.Pensez-vousquelaplupartdevosélèvesontbeaucoupdedifficultésàapprendreàlire?□Oui□Non□PasCertain/pasd’avis

1.2. Pensezvousquevosélèvespeuventplusfacilementapprendreàliredansleurlanguematernellequ’enfrançais?□Oui□Non□PasCertain/pasd’avis

96

1.3. Est-cequ’ilestmieuxdemenerdesactivitésdelectureetd’écritureséparément,plutôtquedanslamêmeleçon?□Oui□Non□PasCertain/pasd’avis

B.Discussion:Posezlesquestionssuivantes:

1.3.Parlonsdevotreréponseàlaquestion—“est-cequ’ilestmieuxdemenerdesactivitésdelectureetd’écritureséparément,plutôtquedanslamêmeleçon”?a) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)

1. «parcequec’estmonexpériencedanslasalledeclasse»2. «parcequec’estcequej’aiapprisdanslesformations»3. «parcequec’estcequelesenseignantsplusanciensm’ontdit»4. «parcequec’estcequemondirecteuroul’inspecteurm’adit»5. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»6. Autre(spécifier:________________________________________)888. Pasd’avis

b) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointdevue?

(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)Ecrivezunsommairedeleurexemple.

c) Discutez-vousdevosleçonsdelectureetécritureavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

1.4. Est-ilimportantpourvousdedonnerdesoccasionsauxélèvesdelireàhautevoix(toutseul,avecunami,outousensembleaveclaclasse)?□Oui□Non□PasCertain/pasd’avis

1.5. Pensez-vousque«pointerlesmotsautableaulorsquelesélèveslisent»lesaidesàlireplusrapidementetfacilement?□Oui□Non□PasCertain/pasd’avis

1.6. Pouraiderlesélèvesafacilementlireetécriredesmots,est-ilutiledeleurdemanderdecatégoriserdesmotspardessons,lettresouterminaisonscommunes?

97

□Oui□Non□PasCertain/pasd’avis

2. Lapré-lectureContinuonsàdiscutersurcequevouspensezdel’enseignementdelapré-lecturec'est-à-dire,cequiestutileetimportantàfaireavantdecommencerlalectured’unnouveautexte.

2.1.Avantdedemanderauxélèvesdelireunnouveautexte,est-ilutilepourvousd’avoirunediscussionavectoutelaclassepourressortircequ’ilssaventdéjàduthème?


2.2. Pensez-vousqu’ilestutiledeparlerdunouveauvocabulaireavecélèvesavantdelireuntexte?□Oui□Non□PasCertain/pasd’avis


2.2.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantdeparlerdunouveauvocabulaireavecélèvesavantdelireuntexte”?a) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-

LES)1. «parcequec’estmonexpériencedanslasalledeclasse»2. «parcequec’estcequej’aiapprisdanslesformations»3. «parcequec’estcequelesenseignantsplusanciensm’ontdit»4. «parcequec’estcequemondirecteuroul’inspecteurm’adit»5. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»6. Autre(spécifier:________________________________________)888. Pasd’avis

b) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointdevue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)

Ecrivezunsommairedeleurexemple.

c) Discutez-vousdecommentvousenseignélenouveauvocabulaireavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

2.3. Pensez-vousqu’ilestutilequelesélèvesseserventdesimagesd’unlivrepourlesaideràcomprendrelenouveauvocabulaire?

98


3. LedécodageContinuonsàdiscutersurcequevouspensezdel’enseignementdudécodage,c'est-à-direl’associationlettres-sonsqu’onabesoindefairepourlirelesmots.Sivousavezbesoind’uneexplicationsurunequestion,n’hésitezpasàmedemander.

3.1.Pensez-vousqu’ilestimportantdetoujourslireavantlesélèvesafinqu’ilsapprennentalire?□Oui□Non□PasCertain/pasd’avis

3.2. Pensez-vousqu’ilestimportantquelesélèvesconnaissentle(s)son(s)quefaitchaquelettredansunmotpourlelire?□Oui□Non□PasCertain/pasd’avis


3.2.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantquelesélèvesconnaissentle(s)son(s)quefaitchaquelettredansunmotpourlelire”?a) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)




c) Discutez-vousdecommentvousenseignéledécodageavecvoscollègues?

1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

3.3. Pensez-vousqu’ilestimportantquelesélèvesapprennentàlirerapidementdesmotsfréquents(ex.est,ca,les,dans,sous,des,etc.)?

99


3.4. Est-ilnécessairequelesélèvesconnaissenttoutesleslettresdel’alphabetpourlireetécrire?


3.5. Pourapprendreàlireplusrapidementunmot,pensez-vousqu’ilestimportantquelesélèves

apprennentàreconnaitreautomatiquementungroupedelettre(ex.tim-bre—ladivisiondesmotsensyllabeouenmorceau)□Oui□Non□PasCertain/pasd’avis

100

4. LacompréhensionContinuonsàdiscutersurcequevouspensezdel’enseignementdelacompréhensionc'est-à-dire,cequevouspensezestutileetimportantàfairepouraiderlesélèvesàcomprendrecequ’ilslisent.

4.1. Est-ilimportantdelaisserlesélèvesparlerentreeuxdecequ’ilsontluouécoutépourlesaideracomprendreuntexte?□Oui□Non□PasCertain/pasd’avis

4.2. Aprèsavoirluuntexte,pensez-vousqu’ilestimportantdedemanderauxélèvesd’expliquercequ’ilsontlu?□Oui□Non□PasCertain/pasd’avis


4.3. Parlonsdevotreréponseàlaquestion—“Aprèsavoirluuntexte,pensez-vousqu’ilestimportantdedemanderauxélèvesd’expliquercequ’ilsontlu”?

a) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)




c) Discutez-vousdecommentvousenseignélacompréhensionavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

4.4. Est-ilimportantdeposerdesquestionsauxélèvessuruntexteaprèsl’avoirlu?□Oui□Non□PasCertain/pasd’avis

101

4.5. Pensez-vousqu’unélèvedevraitêtrecapablededirecequ’ilaaiméoupasaimédansuntextelu?□Oui□Non□PasCertain/pasd’avis

5. L’écritureContinuonsàdiscutersurcequevouspensezdel’enseignementdel’écriturec'est-à-dire,enseignerl’orthographe,lagrammaire,lacomposition,laconventiondestextes.Ici,nousneparlonspasdelacalligraphie.5.1. Est-cegravesiunélèvefaitdeserreursd’orthographelorsqu’ilécritpourlapremièrefoisun

nouveaumotqu’iln’apasétudiéenclasse?□Oui□Non□PasCertain/pasd’avis

5.2. Est-cequevosélèvesontbeaucoupdedifficultésàapprendreàécrire?□Oui□Non□PasCertain/pasd’avis

5.3. Pensez-vousqu’ilestimportantquechaqueélèveaitdesoccasionspourécriredesmotsoudes

phrasesqu’ilentendouqu’ilconçoittoutseul?□Oui□Non□PasCertain/pasd’avis

B.DiscussionPosezlesquestionssuivantes:

5.4.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantquechaqueélèveaitdesoccasionspourécriredesmotsoudesphrasesqu’ilentendouqu’ilconçoittoutseul?”





102

c) Discutez-vousdecommentvousenseignél’écritureavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

6. VosattentesTerminonsnotrediscussionenparlantdevosattentesparrapportauxcapacitésdevosdesélèves.

6.1. Quandpensez-vousquelesélèvespeuventdécoderdenouveauxmotssansl'aidedel’enseignantenfaisantl’associationlettre-son?(LISEZLESOPTIONSAL’ENSEIGNANT)

□Apartirdu1etrimestredela1eannée□Alafindela1eannée□Alafindela2eannée□Alafindela3eannée□Cecin’estpasunecompétenceimportante

B.Discussion:

Parlonsdevotreréponseàlaquestion—“lesélèvespeuventdécoderdenouveauxmotssansl'aidedesenseignantsenfaisantl’associationlettre-sona(INSERERLAREPONSEQU’ILADONNE)»




Ecrivezunsommaireleurexemple.

c) Discutez-vousdecequevosélèvessontcapablesdefaireenlectureouenécritureavecvoscollègues?


103

6.2. Quandpensez-vouslesélèvespeuventcorrectementécriredesmotsfréquents?(LISEZLESOPTIONSAL’ENSEIGNANT)

□Apartirdu1etrimestredela1eannée□Alafindela1eannée□Alafindela2eannée□Alafindela3eannée□Cecin’estpasunecompétenceimportante

MERCIPOURVOTREPARTICIPATION!

Findel’entretien |__|__|:|__|__| H H M M

104

Grade3and4interviewtool:





NomdeL’Ecole

---------------------------------------------




enseigné-------------------





1.1. Est-cequ’ilestmieuxdemenerdesactivitésdelectureetd’écritureséparément,plutôtquedanslamêmeleçon?□Oui□Non□PasCertain/pasd’avis


1.2.Parlonsdevotreréponseàlaquestion—“Est-cequ’ilestmieuxdemenerdesactivitésdelectureetd’écritureséparément,plutôtquedanslemêmeleçon”?


7. «parcequec’estmonexpériencedanslasalledeclasse»

105

8. «parcequec’estcequej’aiapprisdanslesformations»9. «parcequec’estcequelesenseignantsplusanciensm’ontdit»10. «parcequec’estcequemondirecteuroul’inspecteurm’adit»11. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»12. Autre(spécifier:________________________________________)888. Pasd’avis



c) Discutez-vousdecommentvousenseignélalectureavecvoscollègues?5. «Oui,souvent»(1foisparsemaine)6. «Oui,parfois»(1foisparmoisoupartrimestre)7. «Seulementquandj’aiunproblème»8. «Non,jamais»

1.3 Est-ilimportantpourvousdedonnerdesoccasionsauxélèvesdelireàhautevoix(toutseul,avecunami,outousensembleaveclaclasse)?□Oui□Non□PasCertain/pasd’avis

1.4 Pensez-vousque«pointerlesmotsautableaulorsquelesélèveslisent»lesaidesàlireplus

rapidementetfacilement?□Oui□Non□PasCertain/pasd’avis

1.5 Pouraiderlesélèvesafacilementlireetécriredesmots,est-ilutiledeleurdemanderde

catégoriserdesmotspardessons,lettresouterminaisonscommuns?□Oui□Non□PasCertain/pasd’avis

3. Lapré-lectureContinuonsàdiscutersurcequevouspensezdel’enseignementdelapré-lecturec'est-à-dire,cequevouspensezestutileetimportantàfaireavantdecommencerlalectured’unnouveautexte.

2.1. Avantdedemanderauxélèvesdelireunnouveautexte,est-ilutilepourvousd’avoirunediscussionavectoutelaclassepourressortircequ’ilssaventdéjàduthème?

106


2.2. Pensez-vousqu’ilestutiledeparlerdunouveauvocabulaireavecélèvesavantdelireuntexte?



2.2.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantdeparlerdunouveauvocabulaireavecélèvesavantdelireuntexte”?d) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-


e) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointde

vue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)


f) Discutez-vousdecommentvousenseignélenouveauvocabulaireavecvoscollègues?


2.3. Pensez-vousqu’ilestutilequelesélèvesparcourentlesimagesetlisentletitred’unlivrepour

lesaideràcomprendrelenouveauvocabulaire?□Oui□Non□PasCertain/pasd’avis

2.4. Pensez-vousqu’ilestmieuxd’enseignerlenouveauvocabulairesousformedelisteplutôtquedelesapprendreàl’aided’untexteoud’unehistoire?

107


3. LedécodageetlafluiditéContinuonsàdiscutersurcequevouspensezdel’enseignementdudécodageetdelafluiditéenlecture,c'est-à-direl’associationlettres-sonsqu’onabesoindefairepourlirelesmotsetpuislafacilitéetrapiditédudécodagequ’ilfautpourdevenirunbonlecteur.Sivousavezbesoind’uneexplicationsurunequestion,n’hésitezpasàmedemander.

3.1. Pouraiderlesélèvesàapprendreàlire,est-ilimportantdefairerépéterlalecturedesmotsaprèsvous?□Oui□Non□PasCertain/pasd’avis

3.2. Pensez-vousqu’ilestimportantquelesélèvesapprennentàlirerapidementdesmotsfréquents(ex.est,ca,les,dans,sous,des,etc.)?□Oui□Non□PasCertain/pasd’avis

3.3. Pouraiderlesélèvesàdevenirbonlecteur,pensez-vousqu’ilestimportantqu’ilss’entrainentà

liredesphrasesentièresrapidementetavecintonation?□Oui□Non□PasCertain/pasd’avis

3.4. Pourapprendreàlireplusrapidementunmot,pensez-vousqu’ilestimportantquelesélèves

apprennentàreconnaitreautomatiquementungroupedelettre(ex.tim-bre—ladivisiondesmotsensyllabeouenmorceau)□Oui□Non□PasCertain/pasd’avis


3.2.Parlonsdevotreréponseàlaquestion—“Pourapprendreàlireplusrapidementunmot,pensez-vousqu’ilestimportantquelesélèvesapprennentàreconnaitreautomatiquementungroupedelettre”?d) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-

LES)7. «parcequec’estmonexpériencedanslasalledeclasse»8. «parcequec’estcequej’aiapprisdanslesformations»9. «parcequec’estcequelesenseignantsplusanciensm’ontdit»10. «parcequec’estcequemondirecteuroul’inspecteurm’adit»11. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»12. Autre(spécifier:________________________________________)

108

889. Pasd’avis

e) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointdevue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)

Ecrivezunsommairedequelquesexemplespartagé.

f) Discutez-vousdecommentvousenseignéledécodageavecvoscollègues?


4. LacompréhensionContinuonsàdiscutersurcequevouspensezdel’enseignementdelacompréhensionc'est-à-dire,cequevouspensezestutileetimportantàfairepouraiderlesélèvesàcomprendrecequ’illise.

4.1 Est-ilimportantdelaisserlesélèvesparlerentreeuxdecequ’ilsontluouécoutépourlesaideracomprendreuntexte?□Oui□Non□PasCertain/pasd’avis

4.2 Aprèsavoirluuntexte,pensez-vousqu’ilestimportantdedemanderauxélèvesd’expliquercequ’ilsontlu?□Oui□Non□PasCertain/pasd’avis


4.2Parlonsdevotreréponseàlaquestion—“Aprèsavoirluuntexte,pensez-vousqu’ilestimportantdedemanderauxélèvesd’expliquercequ’ilsontlu”?

d) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)

7. «parcequec’estmonexpériencedanslasalledeclasse»8. «parcequec’estcequej’aiapprisdanslesformations»9. «parcequec’estcequelesenseignantsplusanciensm’ontdit»10. «parcequec’estcequemondirecteuroul’inspecteurm’adit»11. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»12. Autre(spécifier:________________________________________)

109

889. Pasd’avis



f) Discutez-vousdecommentvousenseignélacompréhensionavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

4.3 Est-ilimportantdeposerdesquestionsauxélèvessuruntexteaprèsl’avoirlu?□Oui□Non□PasCertain/pasd’avis

4.4 Pensez-vousqu’unélèvedevraitêtrecapablederéagiràcequ’ilaapprisouaimédansuntextelu?□Oui□Non□PasCertain/pasd’avis

5. L’écritureContinuonsàdiscutersurcequevouspensezdel’enseignementdel’écriturec'est-à-dire,enseignerl’orthographe,lagrammaire,lacomposition,laconventiondestextes.Ici,nousneparlonspasdelacalligraphie.

5.1 Est-cegravesiunélèvefaitdeserreursd’orthographelorsqu’ilécritpourlapremièrefoisunnouveaumotqu’iln’apasétudiéenclasse?□Oui□Non□PasCertain/pasd’avis

5.2 Est-cequevosélèvesontbeaucoupdedifficultésàapprendreàécrire?


5.3 Pensez-vousqu’ilestimportantquechaqueélèvedevraitavoirdesoccasionspourécriredesmotsoudesphrasesqu’ilentendouqu’ilconçoittoutseul?□Oui□Non□PasCertain/pasd’avis

B.Discussion1. Posezlesquestionssuivantes:

110

5.4.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantquechaqueélèvedevraitavoirdesoccasionspourécriredesmotsoudesphrasesqu’ilentendouqu’ilconçoittoutseul?”

d) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)7. «parcequec’estmonexpériencedanslasalledeclasse»8. «parcequec’estcequej’aiapprisdanslesformations»9. «parcequec’estcequelesenseignantsplusanciensm’ontdit»10. «parcequec’estcequemondirecteuroul’inspecteurm’adit»11. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»12. Autre(spécifier:________________________________________)889. Pasd’avis

e) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointde

vue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)


f) Discutez-vousdecommentvousenseignél’écritureavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»


6.1 Quandpensez-vousquelesélèvespeuventécrireleurspropresidées?(LISEZLESOPTIONSAL’ENSEIGNANT)□Apartirdela1eannée□Alafindela2eannée□Alafindela3eannée□Alafindela4eannée□Alafindela5eannée□Alafindela6eannée□Cecin’estpasunecompétenceimportante

6.2 Quandpensez-vousqu’unélèvepeutlireuntexteàsonniveauetcomprendrecequ’illitsansassistancedel’enseignant?(LISEZLESOPTIONSAL’ENSEIGNANT)□Apartirdela1eannée□Alafindela2eannée□Alafindela3eannée

111

□Alafindela4eannée□Alafindela5eannée□Alafindela6eannée□Cecin’estpasunecompétenceimportante



112

Grade5and6interviewtool





NomdeL’Ecole

---------------------------------------------




enseigné-------------------





1.1 Est-cequ’ilestmieuxdemenerdesactivitésdelectureetd’écritureséparément,plutôtquedanslamêmeleçon?□Oui□Non□PasCertain/pasd’avis


1.1.Parlonsdevotreréponseàlaquestion—“Est-cequ’ilestmieuxdemenerdesactivitésdelectureetd’écritureséparément,plutôtquedanslemêmeleçon”?

d) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)

13. «parcequec’estmonexpériencedanslasalledeclasse»

113

14. «parcequec’estcequej’aiapprisdanslesformations»15. «parcequec’estcequelesenseignantsplusanciensm’ontdit»16. «parcequec’estcequemondirecteuroul’inspecteurm’adit»17. «parcequec’estcequej’aiapprisdanmoncoursdepédagogieal’école»18. Autre(spécifier:________________________________________)888.Pasd’avis



f) Discutez-vousdecommentvousenseignélalectureavecvoscollègues?


1.2 Est-ilimportantpourvousdedonnerdesoccasionsauxélèvesdelireàhautevoix(toutseul,

avecunami,outousensembleaveclaclasse)?□Oui□Non□PasCertain/pasd’avis

1.3 Pouraiderlesélèvesafacilementlireetécriredesmots,est-ilutiledeleurdemanderde

catégoriserdesmotspardessons,lettresouterminaisonscommunes?□Oui□Non□PasCertain/pasd’avis

2. LedécodageetlafluiditéContinuonsàdiscutersurcequevouspensezdel’enseignementdudécodageetdelafluiditéenlecture,c'est-à-direl’associationlettres-sonsqu’onabesoindefairepourlirelesmotsetpuislafacilitéetrapiditédudécodagequ’ilfautdevenirunbonlecteur.Sivousavezbesoind’uneexplicationsurunequestion,n’hésitezpasàmedemander.

2.1 Pouraiderlesélèvesàapprendrealire,est-ilimportantdefairerépéterlalecturedesmotsaprèsvous?□Oui□Non□PasCertain/pasd’avis

2.2 Pensez-vousquepouraiderlesélèvesaplusrapidementlirelesmots,ilestutiledeleur

demanderd’apprendreàreconnaitrelesracinesoulessyllabesdesmots?

114


115


2.2.Parlonsdevotreréponseàlaquestion—“Pensez-vousquepouraiderlesélèvesaplusrapidementlirelesmots,ilestutiledeleurdemanderd’apprendreàreconnaitrelesracinesoulessyllabesdesmots”?g) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-


h) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointdevue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)


i) Discutez-vousdecommentvousenseignéledécodageavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

2.3 Pensez-vousqu’ilestimportantdefairedesséancesdelecturesilencieuseenclasse?


2.4 Pouraiderlesélèvesàdevenirbonlecteur,pensez-vousqu’ilestimportantqu’ilss’entrainentàliredesphrasesentièresrapidementetavecintonation?


3. Lapré-lectureContinuonsàdiscutersurcequevouspensezdel’enseignementdelapré-lecturec'est-à-dire,cequevouspensezestutileetimportantàfaireavantdecommencerlalectured’unnouveautexte.

3.1. Avantdedemanderauxélèvesdelireunnouveautexte,est-ilutilepourvousd’avoirunediscussionavectoutelaclassepourressortircequ’ilssaventdéjàduthème?

116


3.2. Pensez-vousqu’ilestutiledeparlerdunouveauvocabulaireavecélèvesavantdelireuntexte?



3.2.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantdeparlerdunouveauvocabulaireavecélèvesavantdelireuntexte”?g) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-


h) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointdevue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)


i) Discutez-vousdecommentvousenseignélenouveauvocabulaireavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

3.3. Pensez-vousqu’ilestmieuxd’enseignerlenouveauvocabulairesousformedelisteplutôtquedelesapprendreàl’aided’untexteoud’unehistoire?□Oui□Non□PasCertain/pasd’avis

3.4. Pensez-vousqu’ilestimportantdedonnerdesoccasionsauxélèvesd’utiliserlesnouveauxmotsdevocabulaireouexpressionsdansdifférentesphrasesqu’ilsconçoivent?□Oui□Non□PasCertain/pasd’avis

117

4. LacompréhensionContinuonsàdiscutersurcequevouspensezdel’enseignementdelacompréhensionc'est-à-dire,cequevouspensezestutileetimportantàfairepouraiderlesélèvesàcomprendrecequ’illise.

4.1 Est-ilimportantdelaisserlesélèvesparlerentreeuxdecequ’ilsontluouécoutépourlesaideracomprendreuntexte?□Oui□Non□PasCertain/pasd’avis

4.2 Est-ilimportantpourvousdeposerdesquestionsauxélèvessuruntexteaprèsl’avoirlu?□Oui□Non□PasCertain/pasd’avis

4.3 Pensez-vousqu’ilestimportantdedemandezauxélèvesderéagiràl’oraloual’écritsurcequ’ilaapprisouaimédansuntextelu?□Oui□Non□PasCertain/pasd’avis


4.3Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantdedemandezauxélèvesderéagiràl’oraloual’écritsurcequ’ilaapprisouaimédansuntextelu”?

a. Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)


g) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotre

pointdevue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE)


h) Discutez-vousdecommentvousenseignélacompréhensionavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»

118

4. «Non,jamais»

4.4 Pensez-vousquelesschémaspeuventaiderlesélèvesaplusfacilementreprendrelesévénementsouinformationsd’untexte?□Oui□Non□PasCertain/pasd’avis

4.5 Pensez-vousqu’ilestutilequelesélèveslisentletitred’untexte,parcourentlesimagesetdedirecequ’ilspensentqu’ilsvontlireavantdelireafindelesaideràcomprendreletexte?□Oui□Non□PasCertain/pasd’avis

5. L’écritureContinuonsàdiscutersurcequevouspensezdel’enseignementdel’écriturec'est-à-dire,enseignerl’orthographe,lagrammaire,lacomposition,laconventiondestextes.Ici,nousneparlonspasdelacalligraphie.

5.1 Est-cegravesiunélèvefaitdeserreursd’orthographelorsqu’ilécritpourlapremièrefoisunnouveaumotqu’iln’apasétudiéenclasse?□Oui□Non□PasCertain/pasd’avis

5.2 Pensez-vousqu’ilestimportantquechaqueélèvedevraitavoirdesoccasionspourécriredesmotsoudesphrasesqu’ilentendouqu’ilconçoittoutseul?□Oui□Non□PasCertain/pasd’avis

B.Discussion

Posezlesquestionssuivantes:

5.2.Parlonsdevotreréponseàlaquestion—“Pensez-vousqu’ilestimportantquechaqueélèvedevraitavoirdesoccasionspourécriredesmotsoudesphrasesqu’ilentendouqu’ilconçoittoutseul?”

g) Expliquezpourquoivousavezréponducommeca.(ACCEPTEZPLUSD’UNEREPONSEETENCERCLEZ-LES)


119

h) S'ilvousplaîtdonnerunexempledevotreexpériencedeclassepoursoutenirvotrepointdevue?(GUIDEZ-LESDANSLEURREPONSEETREQUISITIONEZUNSEULEXEMPLE


i) Discutez-vousdecommentvousenseignélal’écritureavecvoscollègues?1. «Oui,souvent»(1foisparsemaine)2. «Oui,parfois»(1foisparmoisoupartrimestre)3. «Seulementquandj’aiunproblème»4. «Non,jamais»

5.3 Est-cequ’ilestimportantdecorrigertouteserreursdanslesécritsdesélèves?


5.4 Pensez-vousqu’unélèvequiécritbiennefaitpasdefautesd’orthographeoudegrammaire?


5.5 Pensez-vousquepouraiderunélèveàmieuxécrire,ilestutiledeluidemanderdecorrigerses

propresécritsoulesécritsd’unami?□Oui□Non□PasCertain/pasd’avis


6.1 Quandpensez-vousquelesélèvespeuventécrireleurspropresidées?(LISEZLESOPTIONSAL’ENSEIGNANT)□Apartirdela1eannée□Alafindela2eannée□Alafindela3eannée□Alafindela4eannée□Alafindela5eannée□Alafindela6eannée□Alafindela6eannée□Aprèsla6eannée□Cecin’estpasunecompétenceimportante

120

6.2 Quandpensez-vousqu’unélèvepeutlireuntexteàsonniveauetcomprendrecequ’illitsansassistancedel’enseignant?(LISEZLESOPTIONSAL’ENSEIGNANT)□Apartirdela1eannée□Alafindela2eannée□Alafindela3eannée□Alafindela4eannée□Alafindela5eannée□Alafindela6eannée□Aprèsla6eannée□Cecin’estpasunecompétenceimportante



2014 final evaluation report: teachers’ literacy knowledge,...

Documents