monitoring instructional effectiveness in grades 1-3 with the … · monitoring instructional...

Monitoring Instructional Effectiveness Monitoring Instructional Effectiveness in Grades 1in Grades 1--3 with the purpose of 3 with the purpose of guiding school level initiatives and guiding school level initiatives and

classroom instructionclassroom instruction

Joseph Joseph TorgesenTorgesenFlorida Center for Reading ResearchFlorida Center for Reading Research

Eastern Regional Reading First Technical Assistance CenterEastern Regional Reading First Technical Assistance Center

National Reading First Meetings, St. Louis, July, 2007National Reading First Meetings, St. Louis, July, 2007

The most important Reading First goals:The most important Reading First goals:

1. I1. Increasencrease the percentage of students reading the percentage of students reading ““at at grade levelgrade level”” each year at each grade level from each year at each grade level from kindergarten through third gradekindergarten through third grade

2. 2. DecreaseDecrease the percentage of students with serious the percentage of students with serious reading difficulties each year at each grade levelreading difficulties each year at each grade level

Our most important measures of success in doing Our most important measures of success in doing this assess student performance on reading this assess student performance on reading comprehension measures at the end of the yearcomprehension measures at the end of the year--particularly at end of third gradeparticularly at end of third grade

Whether or not we achieve these goals Whether or not we achieve these goals depends on the strength of our instruction depends on the strength of our instruction to accomplish two things during the yearto accomplish two things during the year

All students who begin the year meeting grade level All students who begin the year meeting grade level expectations continue to meet grade level expectations continue to meet grade level expectations at the end of the expectations at the end of the yearyear--they make they make expected yearly growthexpected yearly growth

All students who begin the year reading below All students who begin the year reading below grade level accelerate their development so they grade level accelerate their development so they make expected yearly growth make expected yearly growth plusplus catchcatch--up up growthgrowth

Whether or not we achieve these goals Whether or not we achieve these goals depends on the strength of our instruction depends on the strength of our instruction to do two things during the yearto do two things during the year

Insuring all students make expected Insuring all students make expected yearly growthyearly growth

Strong core reading instruction for all studentsStrong core reading instruction for all studentsEnough timeEnough time spent to meet the needs of many students spent to meet the needs of many students who do not typically receive powerful support at homewho do not typically receive powerful support at home

Enough quality so that the increased instructional time is Enough quality so that the increased instructional time is spent effectivelyspent effectively

Time X quality = yearly growth

Whether or not we achieve these goals Whether or not we achieve these goals depends on the strength of our instruction depends on the strength of our instruction to do two things during the yearto do two things during the year

Insuring students who are behind make Insuring students who are behind make expected yearly growth plus catchexpected yearly growth plus catch--up up growthgrowth

Effective differentiated instruction by classroom teacherEffective differentiated instruction by classroom teacher

Effective schoolEffective school--level systems and resources to provide level systems and resources to provide additional intensive intervention in additional intensive intervention in small enough groupssmall enough groups for for enough timeenough time, and with , and with enough skillenough skill

Time X quality = yearly growth + catch-up growth

In order to effectively prevent early reading difficulties, In order to effectively prevent early reading difficulties, we need to apply two kinds of knowledgewe need to apply two kinds of knowledge

From the From the ““science science of readingof reading””

Information about theindividual components of instruction and assessment that are most effective in raising literacy levels

From effective From effective schoolsschools

Information about leadership, organizational, and classroom practices that are most effective in raising literacy levels

Understanding, and Motivation to Apply

Lessons learned from the Kennewick, Lessons learned from the Kennewick, Washington school district:Washington school district:

Located in southeastern WashingtonLocated in southeastern Washington

Has about 15,000 students Has about 15,000 students –– 13 elementary 13 elementary schools, four middle schools, and 3 high schoolsschools, four middle schools, and 3 high schools

25% of students are ethnic minorities, and 48% 25% of students are ethnic minorities, and 48% elementary school students qualify for free or elementary school students qualify for free or reduced price lunchreduced price lunch

Lessons learned from the Kennewick, Lessons learned from the Kennewick, Washington school district:Washington school district:In 1995, the school board in Kennewick challenged In 1995, the school board in Kennewick challenged the elementary schools to have 90% of their the elementary schools to have 90% of their students at grade level in reading by the end of students at grade level in reading by the end of third grade third grade ––within 3 yearswithin 3 years

The primary responsibility for accomplishing this The primary responsibility for accomplishing this was assigned to the school principalswas assigned to the school principals


From David Montague, a principal:From David Montague, a principal:

““We thought the board and the superintendent We thought the board and the superintendent were crazywere crazy……I saw in the White Paper that I saw in the White Paper that elementary principals were responsible, and said elementary principals were responsible, and said ‘‘Why donWhy don’’t they come down to our building and t they come down to our building and see the kids that come to our school?see the kids that come to our school?’’ I mean, our I mean, our kindergarten kids seem to enter school every year kindergarten kids seem to enter school every year with lower skillswith lower skills…”…”


From David Montague, a principal:From David Montague, a principal:

““After that, the whining died down. The goal started to After that, the whining died down. The goal started to grow legsgrow legs…….Principals are messengers. When you.Principals are messengers. When you’’re re in the message business, it doesnin the message business, it doesn’’t help to criticize the t help to criticize the message. It drives a wedge. It empowers those who message. It drives a wedge. It empowers those who dondon’’t want to change. Since wet want to change. Since we’’re responsible for re responsible for implementation, it makes no sense to send a mixed implementation, it makes no sense to send a mixed message. Principals cannot play message. Principals cannot play ‘‘loyal oppositionloyal opposition’’harping against accountability and at the same time harping against accountability and at the same time provide effective leadership for growthprovide effective leadership for growth””

Lessons learned from the Kennewick, Lessons learned from the Kennewick, Washington school district:Washington school district:The District passed a bond that provided a district The District passed a bond that provided a district reading teacher for each school, and began to hold reading teacher for each school, and began to hold public meetings at a different elementary school every public meetings at a different elementary school every two weeks. two weeks.

At the schoolsAt the schools……““We began to have serious staff meetingsWe began to have serious staff meetings……we began we began going through the district White Paper and looking at the going through the district White Paper and looking at the test data to see how far behind some of our kids were. It test data to see how far behind some of our kids were. It was the first time Washington had ever had such precise was the first time Washington had ever had such precise data. In the fall of 995, 23% of our 3data. In the fall of 995, 23% of our 3rdrd graders were graders were reading at second grade level and 41% of our 3reading at second grade level and 41% of our 3rdrd graders graders were reading at a kindergarten or 1were reading at a kindergarten or 1stst grade level.grade level.

““In the United States, public schools deliver 85% or more of In the United States, public schools deliver 85% or more of their curriculum by reading textbooks, whiteboards, their curriculum by reading textbooks, whiteboards, worksheets, and computer screens. Students must read well worksheets, and computer screens. Students must read well to do well. to do well. It matters little what else they learn in elementary It matters little what else they learn in elementary school if they do not learn to read at grade level.school if they do not learn to read at grade level. Even math Even math depends on reading. There are far more words than numbers depends on reading. There are far more words than numbers in math books. As the math concepts become more in math books. As the math concepts become more complicated, the verbal descriptions become more involved complicated, the verbal descriptions become more involved as well. As Kennewick has focused on reading, our math as well. As Kennewick has focused on reading, our math scores have improved at a rate similar to our reading scores. scores have improved at a rate similar to our reading scores. The reading problem becomes more obvious in middle The reading problem becomes more obvious in middle school where students who cannot read well struggle to school where students who cannot read well struggle to absorb content in all other subjects.absorb content in all other subjects.”” P. 49P. 49

Fielding, L., Kerr, N., & Rosier, P. (2007). Fielding, L., Kerr, N., & Rosier, P. (2007). Annual growth for all students, Annual growth for all students, catchcatch--up growth for those who are behindup growth for those who are behind. Kennewick, WA: The New . Kennewick, WA: The New Foundation Press, Inc. Foundation Press, Inc.

Washington Elementary SchoolWashington Elementary School

Growth in % of 3Growth in % of 3rdrd grade students meeting grade level standardsgrade students meeting grade level standards

9595 9696 9797 9898 9999 0000 0101 0202 0303 0404 0505 0606

5757 7272 7272 6868 7878 9494 9696 9999 9494 9898 9999 9898

School YearSchool Year

Percent at Grade levelPercent at Grade level

Working harder and Working harder and more effectively at more effectively at 33rdrd gradegrade

Baseline yearBaseline year

From the Principal:From the Principal:

““By the 3By the 3rdrd year, we had exhausted our workyear, we had exhausted our work--harderharder--atat--thirdthird--grade strategygrade strategy……More of the catchMore of the catch--up gain had to be made at second and first grade. up gain had to be made at second and first grade. Our firstOur first--and secondand second--grade teachers realized that grade teachers realized that they had to become more accountable for their they had to become more accountable for their studentsstudents’’ learning. Even our kindergarten learning. Even our kindergarten teachers, who had spent most of their class time teachers, who had spent most of their class time on social activities, began the transition to teaching on social activities, began the transition to teaching phonemic awareness along with letter and sound phonemic awareness along with letter and sound recognition.recognition.””


Growth in % of 3Growth in % of 3rdrd grade students meeting grade level standardsgrade students meeting grade level standards

9595 9696 9797 9898 9999 0000 0101 0202 0303 0404 0505 0606

5757 7272 7272 6868 7878 9494 9696 9999 9494 9898 9999 9898

School YearSchool Year

Percent at Grade levelPercent at Grade level

Working harder and Working harder and more effectively at more effectively at 33rdrd gradegrade

Began testing in 2Began testing in 2ndnd

grade and focusing on grade and focusing on earlier improvementearlier improvement

Result of improvement at both 2Result of improvement at both 2ndnd and 3and 3rdrd

GradeGrade

Began providing intensive interventions in Began providing intensive interventions in the afternoon to many studentsthe afternoon to many students

Baseline yearBaseline year


School CharacteristicsSchool Characteristics55% Free/reduced lunch55% Free/reduced lunch28% Minority28% Minority85% Stability85% StabilityTeaching StaffTeaching Staff2 half2 half--day kindergarten teachersday kindergarten teachers3 classroom teachers each in 13 classroom teachers each in 1--551 District Reading Specialist1 District Reading Specialist3 Title I Teachers3 Title I Teachers1.5 Resource room/special 1.5 Resource room/special eded teachersteachers1 PE teacher1 PE teacher1 librarian, 1 Librarian secretary1 librarian, 1 Librarian secretary3 Specials teachers3 Specials teachers9 paraprofessionals9 paraprofessionals


How they get additional instructional power in first gradeHow they get additional instructional power in first grade

Small group reading during 1Small group reading during 1stst hour of the dayhour of the day

It puts 13 adults with 75 students during the first hour in It puts 13 adults with 75 students during the first hour in first gradefirst gradeStruggling students get 1:3 with most skilled instructorStruggling students get 1:3 with most skilled instructorAdvanced students get 1:7 ratios with Advanced students get 1:7 ratios with parasparas and othersand others

During the Morning Reading BlockDuring the Morning Reading Block

In the afternoonIn the afternoonMany students get additional small group or 1:1 Many students get additional small group or 1:1 instruction time as interventionsinstruction time as interventions


11stst hour (8:45hour (8:45--9:45)9:45)

Small group instructionSmall group instruction3 classroom teachers3 classroom teachers1 District Reading Teacher1 District Reading Teacher2 Title I teachers2 Title I teachersSpecials teacherSpecials teacherPE teacherPE teacher6 paraprofessionals6 paraprofessionals

The reading block for 3 first grade classroomsThe reading block for 3 first grade classrooms

22ndnd hour (9:45hour (9:45--10:45)10:45)

Whole group instructionWhole group instruction

22ndnd hour (9:45hour (9:45--10:45)10:45)

Also, during the second Also, during the second hour, hour, parasparas, Title 1, and , Title 1, and others work in small others work in small groups with 2groups with 2ndnd--55thth gradesgrades

In the afternoon, many students are provided an In the afternoon, many students are provided an additional 40additional 40--90 minutes of intervention90 minutes of intervention

From David From David MotagueMotague

””By the fifth year, I was convinced high performance reading was By the fifth year, I was convinced high performance reading was about more time and better use of that time. Students who were about more time and better use of that time. Students who were behind needed more direct instruction. Some of them started behind needed more direct instruction. Some of them started getting 60 to 90 minutes extra each day for a total of 180 to 21getting 60 to 90 minutes extra each day for a total of 180 to 210 0 minutes a day. We spent that time on the subminutes a day. We spent that time on the sub--skills they hadnskills they hadn’’t t mastered.mastered.””

““For most of KennewickFor most of Kennewick’’s high performance elementary schools, s high performance elementary schools, increasing the amount of time spent on direct reading instructioincreasing the amount of time spent on direct reading instruction n was an intuitive decision. They tried more time. It worked, andwas an intuitive decision. They tried more time. It worked, andthey kept on doing it. It was merely an extension of what they kept on doing it. It was merely an extension of what remediation was about. Principals and many teachers at these remediation was about. Principals and many teachers at these schools saw the direct connection between increasing instructionschools saw the direct connection between increasing instructional al time and increasing reading growth. Students who were a little time and increasing reading growth. Students who were a little behind needed a little more instructional time. Students who webehind needed a little more instructional time. Students who were re a lot behind needed a lot more time.a lot behind needed a lot more time.”” P. 38. P. 38.

““Growth is directly proportionate to the quality and Growth is directly proportionate to the quality and quantity of instructional time. When we looked at our quantity of instructional time. When we looked at our data student by student, we saw a painful fact with data student by student, we saw a painful fact with painful clarity. Most students who start behind stay painful clarity. Most students who start behind stay behind. Timebehind. Time--starved reading programs that rely on starved reading programs that rely on sudden growth bursts from extraordinary instruction sudden growth bursts from extraordinary instruction rarely move students from the 5thrarely move students from the 5th--30th percentiles up to 30th percentiles up to grade level.grade level.”” P. 48P. 48

““CatchCatch--up growth is driven primarily by proportional up growth is driven primarily by proportional increases in direct instructional time. Catchincreases in direct instructional time. Catch--up growth up growth is so difficult to achieve that it can be the product only is so difficult to achieve that it can be the product only of quality instruction in great quantity.of quality instruction in great quantity.””

Teacher quality x time = growth

““Quantity of instructional time can be doubled or Quantity of instructional time can be doubled or tripled in a semester. Quality of instructional time tripled in a semester. Quality of instructional time cannot. Improving quality occurs over extended cannot. Improving quality occurs over extended periods of time, at different rates for different periods of time, at different rates for different teachers in the same school, as a constant process teachers in the same school, as a constant process of arduous, intelligent labor.of arduous, intelligent labor.

Teacher quality (1) x time (1) = growth (1)Teacher quality (1) x time (2) = growth (2)Teacher quality (1) x time (3) = growth (3)

Teacher quality x time = growth

““This is why the primary and immediate strategy for This is why the primary and immediate strategy for catchcatch--up growth is proportional increase in direct up growth is proportional increase in direct instructional time. instructional time.

CatchCatch--up growth rarely occurs unless principals and up growth rarely occurs unless principals and teachers have good data, know each studentteachers have good data, know each student’’s s learning needs, and learning needs, and schedule proportional schedule proportional increases in direct instructional time.increases in direct instructional time.””

2373271054574959Sunset View3012027120171209941Lincoln4012025120101209550Vista

4312028120241209454Washington3312029120341209320Southgate429034120511209023Ridge View3312527140251206576Amistad5112033120561209260Hawthorne5512055120511209635Cascade6712055120791207680Westgate3215024135251959038Canyon V.

School % a

t Gra

de L

evel

, 20

03

% F

R L

unch

1stG

rade

Rea

ding

B

lock

1stG

rade

In

terv

entio

ns

2ndG

rade

Rea

ding

B

lock

2ndG

rade

In

terv

entio

ns

3rdG

rade

Rea

ding

B

lock

3rdG

rade

In

terv

entio

ns

Two indices from progress monitoring Two indices from progress monitoring measures that provide data about measures that provide data about effectiveness of core instruction and effectiveness of core instruction and interventionsinterventions

Effectiveness of Core InstructionEffectiveness of Core Instruction (ECI) measures (ECI) measures the percentage of students who began the school the percentage of students who began the school year (Assessment 1) on grade level (in green year (Assessment 1) on grade level (in green zone) and remained on grade level in the zone) and remained on grade level in the assessment period being reported (middle or end assessment period being reported (middle or end of year) of year) ––make expected yearly gainmake expected yearly gain

The ECI in one classroomThe ECI in one classroom1 2 3 4 5 6 7 8 9 10 11 12 1314 15 16 1719 20 21

screening

One academic year

9 of 13 students who were at grade level continue at grade level

1 23456 7 8 91011 12 1314 15161719 2021

outcome

ECI = .69ECI = .69

ECI

Green StudentsSTAY

Green StudentsFrom

Assessment 1 to Assessment 2Assessment 1 to Assessment 3

Two indices from progress monitoring Two indices from progress monitoring measures that provide data about measures that provide data about effectiveness of core instruction and effectiveness of core instruction and interventionsinterventions

Effectiveness of InterventionsEffectiveness of Interventions (EI) shows the (EI) shows the percentage of students who began the year at percentage of students who began the year at some level of risk for reading difficulties (reading some level of risk for reading difficulties (reading below grade level) but who grew rapidly enough below grade level) but who grew rapidly enough to advance to a lower level of risk, or to grade to advance to a lower level of risk, or to grade level, by the end of the year.level, by the end of the year.

The EI in one classroomThe EI in one classroom1 2 3 4 5 6 7 8 9 10 11 12 1314 15 16 1719 20 21

screening

One academic year

3 of 7 students who were at “at risk”moved to a lower risk level

1 23456 7 8 91011 12 1314 1516 1719 2021

outcome

EI = .43EI = .43

EI

RedRed oror YellowYellow StudentsStudentsIMPROVE TOIMPROVE TO

Yellow Yellow oror Green StudentsGreen StudentsFromFrom

Assessment 1 to Assessment 2Assessment 1 to Assessment 2Assessment 1 to Assessment 3Assessment 1 to Assessment 3

The EI index can be broken into two more The EI index can be broken into two more specific indicatorsspecific indicators

EIEI--I shows the percentage of students who began I shows the percentage of students who began the year at the the year at the ““intensiveintensive”” or highest level of risk or highest level of risk (red) but who grew rapidly enough to advance to (red) but who grew rapidly enough to advance to strategic (yellow) or grade level (green) by the strategic (yellow) or grade level (green) by the end of the year.end of the year.

EIEI--S shows the percentage of students who began S shows the percentage of students who began the year at the the year at the ““strategicstrategic”” or moderate level of risk or moderate level of risk (yellow) but who grew rapidly enough to advance (yellow) but who grew rapidly enough to advance to grade level (green) by the end of the year.to grade level (green) by the end of the year.

The EI in one classroomThe EI in one classroom1 2 3 4 5 6 7 8 9 10 11 12 1314 15 16 1719 20 21

screening

One academic year

1 of 3 students who were at “high risk”moved to a lower risk level

1 23456 7 8 91011 12 1314 1516 1719 2021

outcome

EIEI--I = .33I = .332 of 4 students who were at “moderate”risk moved to grade level

EIEI--I = .50I = .50

EI-I

RedRed StudentsStudentsIMPROVE TOIMPROVE TO

Yellow Yellow oror Green StudentsGreen StudentsFromFrom


EI-S

YellowYellow StudentsStudentsIMPROVE TOIMPROVE TO

Green StudentsGreen StudentsFromFrom


What They AreECI and EI are two measures, based on the Recommended Instructional Level (RIL) for students, that can be used to monitor the performance of the core reading program and the intensive intervention program(s) in schools.

Splits the reading program into two separate parts—core instruction and interventions- more analytic

What They Are Not

ECI and EI should not be considered as the primary measure in evaluating schoolwide performance in reading. These are formative assessment- meant to help focus professional development, resources, school level planning. They are not the same as outcome measures.

Cautions

Population SensitiveBased on students that remain in the same school for each assessmentSmall populations have volatile measuresCan lose all students=have no measureKindergarten and 1st grade Assessment 1 measures are different than assessment 3

Data requirementsMust have within year, longitudinally identifiable data

CautionsWhat should we expect?

With a new measure, we often don’t know what to expect, or what is “good performance”

Goal for ECI should be 100%

Goal for EI should be around 50%

Looking at trends in outcome data Looking at trends in outcome data over time as a way of thinking over time as a way of thinking

about needed changes or about needed changes or improvementsimprovements

Using School, or District, or State Level Using School, or District, or State Level Reading outcome data to make decisions Reading outcome data to make decisions

about allocation of resources and activities to about allocation of resources and activities to improve school performanceimprove school performance

Two kinds of analysisTwo kinds of analysis

1.Examining trends across measures for 1.Examining trends across measures for purposes of planning professional development, purposes of planning professional development, support and guidancesupport and guidance

2.Examining differences in performance across 2.Examining differences in performance across schools for purposes of planning and schools for purposes of planning and implementing differentiated support and oversightimplementing differentiated support and oversight

Using School, or District, or State Level Using School, or District, or State Level Reading outcome data to make decisions Reading outcome data to make decisions

about allocation of resources and activities to about allocation of resources and activities to improve school performanceimprove school performance


1.Examining trends across measures for 1.Examining trends across measures for purposes of planning professional purposes of planning professional development, support and guidancedevelopment, support and guidance

2.Examining differences in performance across 2.Examining differences in performance across schools for purposes of planning and schools for purposes of planning and implementing differentiated support and oversightimplementing differentiated support and oversight

Year to Year changes in performance on a combined measure of PA, Letter knowledge, and decoding

38 39 3835 34 35

10

20

30

40

50

60

70

% Grade Level %High Risk

2003-04 2004-05 2005-06

55

6771

2518 15

10

20

30

40

50

60

70

80

% Grade Level %High Risk

2003-04 2004-05 2005-06

Year to Year changes in measure of oral vocabulary

Kindergarten End of year

35

40

45

50

55

Year 1 Year 2 Year 3

Kinder1st2nd3rd

50

48

43

38

Year to Year improvement in % of students at “grade level” in oral vocabulary in grades Kindergarten through Third

41

38

4344

11stst Grade growth in decoding >> growth in FluencyGrade growth in decoding >> growth in Fluency

Other patterns that might be observed, and that Other patterns that might be observed, and that would have clear implications for focus and would have clear implications for focus and professional developmentprofessional development

33rdrd grade grade ---- 60% of students are at grade level in 60% of students are at grade level in reading fluency, but only 45% are at grade level in reading fluency, but only 45% are at grade level in reading comprehensionreading comprehension

22ndnd Grade growth in fluency >> growth in Reading Grade growth in fluency >> growth in Reading ComprehensionComprehension

Year to Year changes in performance on the PSF, NWF, and ORF tests on DIBELS at the end of First

Grade

57

8591

33

5970

44 4756

2030405060708090

100

PSF NWF ORF

2003-04 2004-05 2005-06

Are they really doing that much better in phonemic awareness and phonemic decoding than in reading

fluency?

When comparing performance across measures, When comparing performance across measures, it is it is very importantvery important to know that you are using to know that you are using

equivalent standards for equivalent standards for ““grade levelgrade level””performance on each measure performance on each measure -- setting bar at setting bar at

the same heightthe same height

Percentile ranks derived from DIBELS norms for Percentile ranks derived from DIBELS norms for end of year targets in PSF, NWF, and ORF at end of year targets in PSF, NWF, and ORF at end of first gradeend of first grade

PSF = 13PSF = 13thth percentilepercentileNWF = 29NWF = 29thth percentilepercentile

ORF = 35ORF = 35thth percentilepercentile

Good, et al., (2002).Good, et al., (2002).SystemSystem--wide Percentile wide Percentile Ranks for DIBELS Ranks for DIBELS Benchmark Assessment Benchmark Assessment (Tech. Report 9).(Tech. Report 9).University of Oregon.University of Oregon.

Examining performance on different measures Examining performance on different measures across grade levelsacross grade levels

Looking at year to year improvements on different Looking at year to year improvements on different measures as well as relative levels of performance measures as well as relative levels of performance on different measureson different measures

1.Examining state level trends across measures 1.Examining state level trends across measures for purposes of planning state wide professional for purposes of planning state wide professional development and supportdevelopment and support

If progress monitoring measures are available, If progress monitoring measures are available, examining growth within a year at different examining growth within a year at different grade levels grade levels –– looking for grade level looking for grade level weaknessesweaknesses

5954

36

59

4641

61

38

45

0

10

20

30

40

50

60

70

1st 2nd 3rd

R. CompORFVoc

Percent of Students at “grade level” in Reading Comprehension, Oral Reading Fluency, and Oral Vocabulary in grades 1-3, 584 schools from 3 cohorts—1st through 3rd year schools

Rcomp = SAT10ORF = DibelsVoc = PPVT

6055

43

61

48 48

63

40

50

0

10

20

30

40

50

60

70

1st 2nd 3rd

R. CompORFVoc

Percent of Students at “grade level” in Reading Comprehension, Oral Reading Fluency, and Oral Vocabulary in grades 1-3 – 318 schools after three years


There are two possible measurement problems There are two possible measurement problems associated with this comparisonassociated with this comparison

The DIBELS end of year targets, because of the The DIBELS end of year targets, because of the way they were developed, correspond to way they were developed, correspond to different percentile ranks across gradesdifferent percentile ranks across grades

11stst grade = 35grade = 35thth percentilepercentile22ndnd grade = 39grade = 39thth percentilepercentile

33rdrd grade = 40thgrade = 40ththth percentilepercentile

The DIBELS norms seem to set a higher The DIBELS norms seem to set a higher standard at the 40standard at the 40thth percentile than other percentile than other normsnorms——particularly at 2particularly at 2ndnd and 3and 3rdrd gradegrade

DIBELS normsDIBELS norms H&T normsH&T norms AimswebAimsweb normsnorms

11stst 45wpm45wpm 43wpm43wpm 45wpm 45wpm

22ndnd 91wpm91wpm 79wpm79wpm 85wpm 85wpm

33rdrd 110wpm110wpm 96wpm96wpm 102wpm 102wpm

59

50

36

59 59

41

61

54

45

0

10

20

30

40

50

60

70

1st 2nd 3rd

R. CompORFVoc

Percent of Students at “grade level” in RC, ORF, and Vocab in grades 1-3 using Hasbrook and Tindal ORF norms and 40th percentile as goal


Examining performance on different measures Examining performance on different measures across grade levelsacross grade levels

Looking at year to year improvements on different Looking at year to year improvements on different measures as well as relative levels of performance measures as well as relative levels of performance on different measureson different measures

1.Examining state level trends across measures 1.Examining state level trends across measures for purposes of planning state wide professional for purposes of planning state wide professional development and supportdevelopment and support

If progress monitoring measures are available, If progress monitoring measures are available, examining growth within a year at different examining growth within a year at different grade levels grade levels –– looking for grade level looking for grade level weaknessesweaknesses

Summary of Instructional Effects in Grades KSummary of Instructional Effects in Grades K--33

% % ““IntensiveIntensive”” % % ““on grade levelon grade level””

Kindergarten reduced 4.3Kindergarten reduced 4.3 increased 26.1 increased 26.1

11stst GradeGrade increased 6.0increased 6.0 decreased 20.2decreased 20.2

22ndnd GradeGrade increased 17.4increased 17.4 decreased 10.6decreased 10.6

33rdrd GradeGrade decreased 7.1decreased 7.1 increased 2.8increased 2.8

Some outcomes in first grade Some outcomes in first grade that led us to be worried about that led us to be worried about the quality and power of our the quality and power of our ““differentiated instructiondifferentiated instruction””

62nd

percentile

50th

percentile42nd

percentile

6863 54 66

Looking at growth in phonemic Looking at growth in phonemic decoding in 20 RF schools that had decoding in 20 RF schools that had

different success in meeting targets for different success in meeting targets for Oral Reading FluencyOral Reading Fluency

10 high performing schools10 high performing schools10 low performing schools10 low performing schools

Similar scores on NWF at beginning of Similar scores on NWF at beginning of the yearthe year

65 65 68

56

67

40

79

52

0

10

20

30

40

50

60

70

80

1 2 3 4

High ECILow ECI

Percentage of students meeting benchmarks in 1st Grade NWF for High ECI and Low ECI Reading First Schools

Download at:

http://www.fcrr.org/assessment/pdf/smallGroupAlternativeLessonStructures.pdf

Or, just go to the FCRR website (www.fcrr.org) and its listed on the home page under the new stuff

Using Reading outcome data to make Using Reading outcome data to make decisions about district and state focus to decisions about district and state focus to

improve school performanceimprove school performance


1.Examining trends across measures for 1.Examining trends across measures for purposes of planning professional development purposes of planning professional development and supportand support

2.Examining differences in performance across 2.Examining differences in performance across schools for purposes of planning and schools for purposes of planning and implementing differentiated supportimplementing differentiated support

There are at least three important ways to There are at least three important ways to examine school performance in RF schoolsexamine school performance in RF schools

Year to year changes in % of students at Year to year changes in % of students at ““grade grade levellevel”” and % of students at and % of students at ““high riskhigh risk””

Absolute levels of performance, in terms of % of Absolute levels of performance, in terms of % of students at grade level and % of students at students at grade level and % of students at ““high high riskrisk””

““Value addedValue added”” analysis that takes into consideration analysis that takes into consideration the the ““degree of difficultydegree of difficulty”” as indexed by student as indexed by student demographicsdemographics

Identifying Identifying ““Focus SchoolsFocus Schools”” in Cohorts 1 in Cohorts 1

YYGLYYGL--1313 Compares % at grade level in Year 1 with % at Grade Compares % at grade level in Year 1 with % at Grade Level in Year 3 Level in Year 3 –– positive numbers mean improvementpositive numbers mean improvement

YYHRYYHR--1313 Compares % at High Risk in Year 1 with % at High Compares % at High Risk in Year 1 with % at High Risk in Year 3 Risk in Year 3 –– positive numbers mean improvementpositive numbers mean improvement

Cohort 1 Cohort 1

Identifying Identifying ““Focus SchoolsFocus Schools”” in Cohorts 1 in Cohorts 1

Cohort 1 Cohort 1

435.73YYHR-13-C467.41YYHR-13-3794.83YYHR-13-2964.59YYHR-13-14610.10YYHR-13-K725.32YYGL-13-C469.21YYGL-13-3985.12YYGL-13-21481.40YYGL-13-13515.50YYGL-13-K

Number with 0 or less

Average

Number of schools with scores Number of schools with scores greater than zerogreater than zero on on both indices = 239, or 75% both indices = 239, or 75% -- 79 79 ““focus schoolsfocus schools””

Relationship of “school challenge” to student performance100

90

80

70

60

50

40

30

20

10

0

Level of School Challenge based on % of students qualifying for FR lunch

% o

f 1-3

Stu

dent

s Per

form

ing

At G

rade

Lev

el at

the E

nd o

f Yea

r1 2 3 4 5 6

Increasing ChallengeIncreasing Challenge72

61 5853 51

66

Decreasing Performance

Decreasing Performance

0 1 2 3 4 5 6

Average % at GL

587 RF schools in Florida

63 75 82 86 91 96

The Adult Learning and Performance Gap100

90

80

70

60

50

40

30

20

10

0

75 82 85 91

1 2 3 4 5 6

57

85

53

80

46

72

46

73

40

67

35

64

Low 15% schools

Top 15% Schools

Approx. 27%

63 96

Level of School Challenge based on % of students qualifying for FR lunch

% o

f 1-3

Stu

dent

s Per

form

ing

At G

rade

Lev

el at

the E

nd o

f Yea

r

Approx. 20%

One possible way to identify schools as One possible way to identify schools as candidates for extra supportcandidates for extra support

Not making adequate year to improvements in % Not making adequate year to improvements in % of students at of students at ““grade levelgrade level”” and reductions in % and reductions in % of students at of students at ““High riskHigh risk””

Seriously underperforming when student Seriously underperforming when student demographics are taken into accountdemographics are taken into account

Questions/DiscussionQuestions/Discussion

monitoring instructional effectiveness in grades 1-3 with the … · monitoring instructional...

Documents