english language learners and rti || determining english language learners' response to...

Hammill Institute on Disabilities

Determining English Language Learners' Response to Intervention: Questions and SomeAnswersAuthor(s): Sylvia Linan-Thompson, Paul T. Cirino and Sharon VaughnSource: Learning Disability Quarterly, Vol. 30, No. 3, English Language Learners and RTI(Summer, 2007), pp. 185-195Published by: Sage Publications, Inc.Stable URL: http://www.jstor.org/stable/30035563 .

Accessed: 17/06/2014 03:49

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Sage Publications, Inc. and Hammill Institute on Disabilities are collaborating with JSTOR to digitize,preserve and extend access to Learning Disability Quarterly.

http://www.jstor.org

This content downloaded from 195.34.79.223 on Tue, 17 Jun 2014 03:49:57 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=sage

http://www.jstor.org/stable/30035563?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


DETERMINING ENGLISH LANGUAGE LEARNERS'

RESPONSE TO INTERVENTION: QUESTIONS AND

SOME ANSWERS

Sylvia Linan-Thompson, Paul T. Cirino, and Sharon Vaughn

Abstract. Using an extant database, we examined three grade 1 criteria for identifying response to intervention (RTI) in English language learners (ELLs): (a) set benchmark criteria with a standard score above 95 (37th %ile) on both decoding and comprehension measures and a raw score of 40 or more correct words per minute (CWPM) on oral reading fluency; (b) discrepancy benchmark criteria, with performance on these measures at or above the mean of not-at-risk peers; and (c) discrepancy slope criteria, with growth during grade 1 on these measures at or above the mean of not-at-risk peers. The sample consisted of 81 students (41 intervention, 40 comparison) who were bilingual (Spanish/English) and were part of a supplemental reading instruction study during first grade. The three grade 1 criteria were evaluated in relation to a set benchmark criteria in grade 2. Results indicated that approximately 80% of the students did not meet any criteria in either year, but that the discrepancy slope criteria in grade 1 were most predictive of set benchmark criteria in grade 2. Recognizing that we applied highly stringent criteria, implications and issues are presented related to using RTI with ELLs to facilitate decision making about further intervention and referral for special education.

Sylvia Linan-Thompson, Ph.D., associate professor and fellow in the Mollie V. Davis Professorship in Learning Disabilities, The University of Texas at Austin.

Paul T. Cirino, Ph.D., research assistant professor, The University of Houston. Sharon Vaughn, Ph.D., professor and H. E. Hartfelder/Southland Corp Regents Chair, The University of Texas at Austin.

A longstanding concern in special education has been the over- and under-representation of students from linguistically diverse groups in special education due to inappropriate assessment and instruction (Donovan & Cross, 2002). These concerns have not been unfounded. Disenchantment with the use of IQ- achievement discrepancy to identify children with learning disabilities (LD) has been growing, as increasing evidence has emerged that it does not discriminate

between IQ-discrepant and nondiscrepant low achiev- ers and other subgroups of low-performing students (Fletcher et al., 1994; Stanovich & Siegel, 1994; Vellutino, Scanlon, & Lyon, 2000). Furthermore, consensus reports (Donovaan & Cross, 2002; Fletcher et al., 2002; President's Commission on Excellence in Special Education, 2002) agree that learning problems are common in schools, and that lack of appropriate instruction can lead to identification for disability in students.

Volume 30, Summer 2007 185



Minority students and English language learners (ELLs) are at greater risk since many also live below the poverty level - another factor that increases risk for academic failure. Combined these factors have contributed to the disproportionate representation of minority students in special education.

In the case of ELLs, additional factors that must be considered in making instructional and eligibility decisions are language of instruction and opportunity to learn English as well as opportunity to learn in general. Under No Child Left Behind (NCLB), there is now an emphasis on doing what is effective based on scientific research and on the academic progress of every child, including those learning English. To ensure that ELLs are making progress, students are assessed in reading and mathematics and are assessed annually to measure how well they are learning English (NCLB, 2002).

After several years of increased funding for research focusing on ELLs, the knowledge base on their instructional needs has grown exponentially. In particular, the field has made strides in identifying effective instructional and assessment practices (see Francis, Rivera, Lesaux, Kiefer, & Rivera, 2006). Therefore, we are now better poised to provide ELLs receiving English reading instruction in the early grades instruction that is research-based. Research-Based Reading Instruction and ELLs

Various intervention studies with ELLs have provided findings showing that these students benefit from instruction that includes (a) the essential components of reading, (b) features of effective instruction, and (c) development of English language skills (Denton, Anthony, Parker, & Hasbrouck, 2004; Gunn, Biglan, Smolkowski, & Ary, 2000; Gunn, Smolkowski, Biglan, & Black, 2002; Vaughn, Mathes et al., 2006). Effective interventions are comprehensive when they appropri- ately address reading components based on students' grade and learning needs: phonological awareness, phonics, fluency building, vocabulary and comprehension.

In intervention studies that provided additional reading instruction to ELLs, students who received comprehensive instruction had better outcomes than students in comparison conditions in at least one area of reading. If, on the other hand, instruction focused on just one element such as fluency building, there were no differences with comparison students (Gersten et al., 2006).

Second, instructional elements common to all the interventions included explicit and systematic instruction that incorporated elements of direct instruction. Additionally, instruction provided enough practice items so that students had multiple opportunities to

use the target skill and receive feedback as needed prior to independent practices. Instruction at the letter, word, and connected text level was fast-paced to build automaticity. It also included both decoding and encoding practice. Ensuring that students mastered instructional elements provided them a foundation of reading knowledge and skills. Finally, interventions had built-in review of all elements taught.

Third, in interventions that interspersed language support activities to build oral language improvement intervention, students outperformed comparison students on several reading measures (Gersten et al., 2006).

Assessment and ELLs There is also evidence that many of the basic reading

assessments and procedures for identifying ELLs at risk for reading disabilities are valid and reliable. The most persuasive research has addressed measures that focus on foundational literacy skills. These are the measures most useful in identifying students early who will benefit from supplemental instruction associated with the prevention of later reading difficulties. Measures of phonological awareness, alphabetic principle, and word reading are particularly useful. Identification in a prevention model does not indicate that students have a learning difficulty but that they need additional instruction to develop early literacy skills (Francis et al., 2006; Gersten et al., 2006).

Prior research with ELLs has identified reading interventions that are effective in improving reading outcomes (Denton et al., 2004; Gunn et al., 2000; Gunn et al., 2002; Vaughn, Mathes et al., 2006). In these studies, the effect of the intervention was determined by comparing differences on outcome measures between students receiving a treatment and a comparison group. Outperforming a comparison group provides relevant information about the effectiveness of the overall intervention; however, questions regarding the number of students who continue to be at risk and procedures for determining the risk status of students are understudied and not specified for ELLs. Thus, the use of RTI as a prevention model and a data source for informing identification of ELLs requires further inves- tigation.

Response to Intervention RTI as a preventive approach includes the use of stu-

dents' learning rate and level of performance to make instructional decisions. Students exhibiting difficulty in learning to read are provided intervention and given time to acquire reading skills before they are referred for special education. Under the recent reauthorization of IDEA, eligibility and identification criteria now include the following: "The LEA may use a process that

Learning Disability Quarterly 186



determines if a child responds to scientific, research- based intervention as part of the evaluation" (IDEA, 2004). However, there is no guidance on how to measure RTI, and to date little research has addressed this issue with ELLs (Linan-Thompson, Vaughn, Prater, & Cirino, 2006). How Do We Determine RTI?

Despite an examination of this question by various researchers (Fuchs, Fuchs, & Compton, 2004; Fuchs, Mock, Morgan, & Young, 2003; McMasters, Fuchs, Fuchs, & Compton, 2005), currently, there is no consensus on how to determine response or which measures are most effective for discriminating responders from nonresponders. In the absence of consensus on this issue, performance level, most often measured with a benchmark score, and growth are frequently used. The benchmark is usually set by a norm group.

In some cases these are minimal standards, meaning that if students can at least reach the benchmark, they will be at lower risk of experiencing reading difficulties. However, using benchmark set by a norming sample as criteria is particularly problematic in the case of ELLs since, with few exceptions, they are often not included in these samples. On neither performance nor growth is there convergence on the cut point to be used for determining responsiveness.

A third approach, dual discrepancy, combines performance and growth and compares students to their peers (Fuch & Fuchs, 1998; McMasters et al., 2005; Speece & Case, 2001; Speece, Case, & Molloy, 2003). Although the cut-point question is resolved, there is no consensus on which measures yield the most accurate information regarding response to intervention.

A previous study of RTI with ELLs used performance level to determine RTI (Linan-Thompson et al., 2006). The cut point was a standard score of 85 on the Word Attack and Passage Comprehension subtests of the Woodcock-Johnson Reading Battery. Using these criteria, 97% of intervention students who responded at the end of first grade maintained their status as responders through second grade compared to students in our previous studies, in which as few as 66% of the students who received 20 weeks of intervention continued to "thrive" in the general education classroom without supplemental instruction (Vaughn, Linan-Thompson, & Hickman-Davis, 2003). Furthermore, results indicated that general classroom instruction was insufficient for students at risk for reading difficulties, since 91% of the students who received the English intervention met criteria at the end of first grade compared to 41% of the control students.

These differences could be attributed to several factors. For example, the intervention provided was more

intensive both in terms of span over the year and length of each session. In addition, the use of standard scores on untimed measures rather than fluency measures may have contributed to the higher percentage of students meeting criteria.

RTI and ELLs RTI offers a promising alternative for reducing dispro-

portionate representation of culturally and linguistically diverse students in special education by identifying students at risk early and providing preventive instruction to accelerate progress. Furthermore, the emphasis on instruction and progress monitoring can potentially impact student outcomes for a population (i.e., ELL) that has historically not fared well on measures of student achievement. Yet, despite the generally positive response to RTI as a means for identifying students with learning disabilities, some researchers have questioned the validity of RTI with culturally and linguistically diverse groups (Klingner & Edwards, 2006). In particular, they emphasize the need to determine what works, for whom, and in what context. Although these questions are inherent in a multitiered RTI model, to date, with few exceptions, data have not been disaggregated by ethnicity, for students who are ELLs, or for students who have other disabilities even when these groups of students are included in the data set (Donovan & Cross, 2002). In other studies, they are excluded from participating, often because of their limited proficiency in English.

Yet, there is a growing database to suggest that many of the interventions to improve early reading for monolingual English students are used in classrooms with ELLs and that when ELLs are included they respond similarly to non-ELL students (Shanahan & Beck, 2006). As increasing numbers of schools implement RTI practices based on recommendations in the recently author- ized legislation (IDEA; December, 2004), questions about how to accurately identify EL learners in need of special education require data-based answers. To mini- mize the possibility of continuing to inappropriately identify EL learners, more research is needed on the use of RTI with EL learners.

This study examined the RTI of EL learners identified as at risk for reading problems in the fall of first grade and provided extensive and systematic intervention for seven months. To identify a viable means for determining RTI, we examined the relative number of EL learners at risk for reading difficulties who met RTI criteria after an intensive seven-month intervention using three different approaches: benchmark, growth, and discrepancy. Rather than focusing on students who did not respond to the intervention, the goal was to identify students who had responded well and were, therefore,




no longer at risk for later reading difficulties. The data were part of a larger intervention study, the results of which were provided previously (Vaughn, Cirino et al., 2006; Vaughn, Mathes et al., 2006).

METHODS Participants

Selection criteria for the original sample. Students (N = 142) were selected from two sites and four effective bilingual schools in Texas. Schools were considered effective for bilingual students because their state-level reading achievement tests at third grade indicated that 80% or more of students passed the test. To ensure that poor performance in English was not due solely to a lack of exposure to English language or print material, researchers screened students in both English and Spanish. Two tests in both Spanish and English for a total of four measures were used: (a) The Letter Word Identification (LWID) subtest from the Woodcock Language Proficiency Battery (English and Spanish, described below) and (b) the first five words from a word reading list (English and Spanish) used to assess initial word reading ability. Students who scored below the 25th percentile for the first grade on the Letter Word Identification subtest in both Spanish and English, and who were unable to read more than one word from the simple word list were eligible for the study and were ran- domly assigned to treatment or comparison conditions.

Students in the treatment condition received daily 50- minute interventions in small groups by teachers trained by the research team only during first grade. This instruction was provided in addition to the core reading instruction. The core curriculum selected by the schools for reading during the classroom reading instruction for one site was Language Enrichment (Carreker, 1999) and in the second, McGraw Hill Reading (2001).

The school selected the language of instruction for reading as English; thus, the intervention was in English to align with the school's decision. Additional details of sites, schools, teachers, and the selection process for the samples screened for inclusion, as well as findings from the intervention studies, are available in previous reports (Vaughn, Cirino et al., 2006; Vaughn, Mathes et al., 2006).

Findings for original sample. At the end of first grade, across two cohorts, the intervention group outperformed the comparison group on measures of phonological awareness, word reading fluency, reading comprehension, and spelling (Vaughn, Cirino et al., 2006; Vaughn, Mathes et al., 2006). Longitudinal findings from the English interventions revealed significant differences in favor of treatment students in oral language, decoding, spelling, fluency and, comprehension (median d = +0.41) at the end of second grade (Cirino et al., in review).

Table 1 Means and Standard Deviations by Group

Intervention N=41

Comparison N = 40

Time 1 Time 2 Time 3 Time 1 Time 2 Time 3 M SD M SD M SD M SD M SD M SD

OL 53.7 21.5 63.4 18.7 69.4 17.8 59.0 18.5

WA 85.8 8.8 100 13.6 100 19.5 83.6 6.0 93.5 11.4 93.5 12.7

PC 80.8 10.4 92.7 13.4 91.3

.6 1.9 20.2 16.4 53.3 33.5

17.2 81.4 11.8 86.4 11.9 87.4 13.0

.73 1.7 13.1 13.0 40.1 29.3

Note. Time 1 = Beginning of first grade; Time 2 = End of first grade; Time 3 = End of second grade. OL = Oral Language composite; WA = Word Attack subtest of WLPB-R; PC = Passage Comprehension subtests of WLPB-R; ORF = DIBELS Oral Reading Fluency.


ORF

62.3 17.5 65.9 15.2



Current participants. One hundred and eleven of the original 142 students (78%) were available at followup (end of grade 2) for further assessment (see Cirino et al., in review). The present study focused on students for whom all RTI criteria data were available at followup (see RTI Criteria in next section).

In all, 81 students met criteria for this study (41 treatment, 40 comparison). All students were bilingual (Spanish/English), with an average age of 8.16 (0.4 SD) at the grade 2 followup point. The sample was 44% female. At the end of grade 1, English and Spanish oracy (from the Woodcock Language Proficiency Battery- Revised, see below) were 63.41 (18.8 SD) and 64.66 (20.4 SD) for treatment students, and 62.35 (17.6 SD) and 68.18 (28.8 SD) for comparison students, respectively. Means and standard deviations at each time point (beginning of first grade, end of first grade, and end of second grade) for the three measures used for RTI criteria are provided in Table 1 as well as the means and standard deviations for oral language.

Measures Woodcock Language Proficiency Battery-Revised

(WLPB-R; Woodcock, 1991). The WLPB-R is a well-stan- dardized instrument with internal consistency values for the subtests utilized at the age range of the current study's participants ranging from .77 to .95 (median r = .89). The WLPB-R was used to assess reading decoding and comprehension outcomes. Test development, scal- ing, and the norming process for the assessment are described in detail in the WLPB-R manuals (Woodcock, 1991). The Word Attack and Passage Comprehension subtests were used, and both standard and W (a Rasch- scaled ability score) scores for these subtests were utilized, depending on the RTI criteria. In addition, an Oral Language composite was computed, based on the Picture Vocabulary, Verbal Analogies, and Listening Comprehension subtests of the WLPB-R (reported above). These measures were administered at the beginning and end of the intervention year, and at the end of the followup year.

Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002)/Indicadores Dinimicos del Exito en la Lectura (Good, Bank & Watson, 2003). The DIBELS subtest utilized was Oral Reading Fluency (ORF). To assess oral reading fluency for connected text, students were administered four measures of reading fluency requiring them to orally read a passage. Reliability was determined by correlating scores from two stories administered to a subset of the larger parent sample (N = 766) with data on these measures for students from Texas at the end of grade 1. Values were quite high, r = 0.95, for the English ver- sions. The ORF test was timed, allowing students a

maximum of 3 seconds per word and 60 seconds for each passage.

Students were administered one story at the beginning of the intervention year, two at the end of the intervention year, and four at the end of the followup year, in English. The number of passages was increased as the students' reading skills improved. Particularly at the beginning of first grade, the students were unable to read, so administering more passages would not have been appropriate. At the end of second grade, students were administered two grade 1 and two grade 2 beginning-of-year passages to determine growth on both grade-level text (second grade) and on text with level held constant over time (first grade). The dependent measures were the average of the number of words read correctly within the time limit across all stories administered at the end off each year.

RTI Criteria We were interested in the response to intervention of

the target students in first grade. These students were at risk for reading difficulties and received a supplemental reading intervention for 50 minutes a day, 5 days per week, during most of their first-grade year. The RTI of this ELL sample was examined at the end of first grade and the end of second grade.

Four different criteria were evaluated to define "responsiveness" to intervention, each in comparison to students who did not receive intervention. The benchmarks used were set benchmarks from the nor- mative sample of the WLPB-R, or expectations for the DIBELS ORF passages. These benchmarks correspond to "placement level models" (Foorman, Francis, Fletcher, Schatschneider, & Metha, 1998; O'Connor, 2000; Torgesen et al., 1999; Vellutino et al., 1996).

Discrepancy benchmarks and slopes, on the other hand, were established relative to a sample of students from the same schools as those in the intervention studies (but who were not identified as struggling readers and therefore not a part of the intervention studies). These criteria correspond to a "local norms approach" (Fuchs & Fuchs, 1998; McMaster et al., 2005; Speece & Case, 2001; Speece et al., 2003).

The longitudinal sample consisted of approximately 430 grade 1 students in immersion programs in four sites in Texas and California from 16 schools. Benchmarks were highly similar when obtained from a smaller sample (approximately 100 students) of only the four specific schools from which intervention students were selected. Therefore, the larger sample from all 16 schools was utilized. The reason for utilizing set benchmarks as well as benchmarks and slopes based on peers was that set benchmarks have not previously been used to determine adequate progress in ELLs in early




elementary school, and the discrepancy benchmarks were derived from a large sample size consistent with the intervention study students' peer group.

A preliminary examination of the methods was conducted using only a percentage of students who met criteria. In this examination, the percentage of students identified as responders were compared with and without fluency and using benchmark, growth, and dual discrepancy to extend the work that we had done previously.

When we examined the students who were identified as responders without and with the fluency benchmark as part of the criteria, the percentage who met the response criteria was 40% to 7%, respectively. Con- sidering the importance of fluency in identifying students with reading difficulties and with the goal of not over-identifying students as responders who were not, we included fluency as part of the criteria.

We also chose to use the more stringent criteria of standard scores of > 95 on both Word Attack and Passage Comprehension rather than the lower criteria we had used in the past of a standard score of > 85 on both Word Attack and Passage Comprehension. In our previous study, the less stringent criteria identified 91% of the students as responders compared to the 64% who were identified with the more stringent criteria of standard scores of > 95 on both measures.

Therefore, for this study, the first of three grade 1 RTI criteria, set benchmark, was a WLPB-R standard score at or above 95 (37th %ile) on both the Word Attack and Passage Comprehension subtests in English and a raw score of at least 40 CWPM on an appropriate DIBELS ORF passage at the end of the intervention year (grade 1). The second criterion, discrepancy benchmark criteria, was a W score at or above the mean of the longitudinal sample at the end of grade 1 on both the Word Attack and Passage Comprehension subtests in English of the WLPB-R and a raw score for words correctly read per minute on an appropriate DIBELS ORF passage that was at or above the mean of the longitudinal sample. The actual criterion values for Word Attack and Passage Comprehension were W = 481 and W = 459, respectively. The actual CWPM criterion for the ORF was 52 words.

The third criterion, discrepancy slope criteria, was a W score gain at or above the mean gain of the longitudinal sample during grade 1 on both the Word Attack and Passage Comprehension subtests in English of the WLPB-R and the student obtained a raw score gain for words correctly read per minute during grade 1 on an appropriate DIBELS ORF passage at or above the mean gain of the longitudinal sample. The actual criterion W score increases for Word Attack and Passage Compre- hension were 15 and 28, respectively. The actual CWPM

gain criteria for the ORF were 27. In each case, students who met these criteria were considered to have responded to the intervention.

These three criteria were compared to a final outcome criterion in grade 2, which was essentially the same as the set benchmark criteria from grade 1: a standard score at or above 95 (37th percentile) on both the Word Attack and Passage Comprehension subtests in English of the WLPB-R and a raw score of at least 70 words correctly read in one minute on an appropriate DIBELS ORF passage at the end of the followup year. If a student received a score below these levels on any of the three measures at the end of grade 2, this final outcome criterion was not considered to have been met.

In the remainder of this article, the three grade 1 RTI criteria are referred to as Set Benchmark 1, Discrepancy Benchmark 1, and Discrepancy Slope 1, and the grade 2 RTI outcome criteria are referred to as Set Benchmark 2.

Analyses Each of the grade 1 criteria were first evaluated against

the grade 2 criteria through simple frequency tables. For example, in comparing Set Benchmark 1 to Set Benchmark 2, the proportion of intervention students who met or did not meet criteria in both grades, only in grade 1, and only in grade 2 can be derived. In addition, struggling readers and their controls can be compared.

Next, logistic analyses were run with Set Benchmark 2 as the dependent variable, and Set Benchmark 1, Discrepancy Benchmark 1, and Discrepancy Slope 1 used as predictors, respectively. Models were also considered that included intervention status, or oral language proficiency in grade 1, as a predictor for each of the criteria. From these analyses, various measures of effectiveness of the RTI criteria were obtained (sensitivity, specificity, odds ratio, d, etc.).

RESULTS Table 2 presents the results of the frequency analyses

for each of the grade 1 criteria against the grade 2 set benchmark criteria by group (intervention or comparison). These results indicate that 88% of comparison students and 75% to 80% of intervention students did not pass any of our stringent grade 1 or grade 2 criteria. Thirteen students (16%, 8 intervention and 5 comparison) met the grade 2 criterion, and varying numbers of students met the various grade 1 criteria. Only five students (4 intervention and 1 comparison) met the Set Benchmark 1 criteria, and only one intervention student met the Discrepancy Benchmark 1 criteria. Given the low number of students who met the grade 1 benchmark criteria, and the fact that a higher number of students (9 and 12) met the grade 2 criteria in spite of not meeting the grade 1 criteria, further analyses were not conducted on the two benchmark criteria. However, 11




Table 2

Grade 1 Criteria v. Set Benchmark 2 (Grade 2) Criteria

Intervention Students N = 40

SB 1 v. SB 2

Met grade 1 but not grade 2 criteria


Met both grade 1 and grade 2 criteria

Met neither grade 1 nor grade 2 criteria

1 (2.4%)

6 (14.6%)

2 (4.9%)

32 (78.1%)

Comparison Students N=41

SB 1 v. SB 2



Met both grade 1 and grade 2 criteria

Met neither grade 1 nor grade 2 criteria

0 (0%)

3 (7.5%)

2 (4.9%)

35 (87.5%)

DB 1 v. SB 2

0 (0%)

7 (17.1%)

1 (2.4%)

33 (80.5%)

DB 1 v. SB 2

0 (0%)

5 (12.5%)

0 (0%)

35 (87.5%)

DS 1 v. SB 2

2 (4.9%)

2 (4.9%)

6 (14.6%)

31 (75.6%)

DS 1 v. SB 2

0 (0%)

2 (5.0%)

3 (7.5%)

35 (87.5%)

Note. SB2 = Set Benchmark 2 (standard score of 95 on both Word Attack and Passage Comprehension subtests of WLPB-R and reading rate of 70 CWPM on DIBELS ORF at the end of Grade 2). SB1 = Set Benchmark 1 (standard score of 95 on Word Attack and Passage Comprehension and a reading rate of 40 CWPM on DIBELS ORF at the end of Grade 1). DB1 = Discrepancy Benchmark 1 (Word Attack, Passage Comprehension, and DIBELS ORF levels at or above the mean of EL peers receiving English instruction). DB2 = Discrepancy Slope 2 (students' growth during grade 1 on the Word Attack, Passage Comprehension, and DIBELS ORF subtests are at or above the mean growth of EL peers receiving English instruction).

students (8 intervention and 3 comparison) met the Discrepancy Slope 1 criteria; these results were further explored via logistic regression.

When Discrepancy Slope 1 criteria were evaluated against the grade 2 criteria, Discrepancy Slope 1 was a significant predictor, X2(1), 21.17, p < .0001, which gen- erated a point estimate (odds ratio) of 74.2, which indicates that those who passed this grade 1 criterion were 74 times more likely to pass the grade 2 criteria than if they did not. This odds ratio corresponds to the more typical Cohen d value of +2.38, which is a very large effect. Overall classification was 93%, with a specificity of .97. Thus, passing these grade 1 criteria is a clear indi-

cator of success at the end of grade 2 (9 of 11, positive predictive value of .82); further, not meeting these criteria is a clear indicator of also not meeting grade 2 criteria (66 of 70, negative predictive value of .94).

These values are interpreted within the context of relatively few students meeting either set of criteria, such that a model predicting all failures would result in an overall classification rate of 84%. Sensitivity was .69, indicating that four of the 13 students who met grade 2 criteria did so without meeting Discrepancy Slope 1 criteria. Additional models were evaluated in which the Discrepancy Slope 1 was evaluated in the context of treatment group and oral language. In these cases, there




were no interactions of these predictors with Discre- pancy Slope 1, and in each case, only the Discrepancy Slope 1 variable had a significant effect.

DISCUSSION Despite increased attention and knowledge about

effective instructional practices related to early reading (McCardle & Chhabra, 2004), there is still much we do not know about the reading development of ELLs and the use of RTI with this population (August & Shanahan, 2006). For example, we have effective uni- versal screening measures to facilitate early identification of students at risk for reading problems, but we do not have an accurate means for determining the level of RTI needed to prevent further difficulties or to warrant more extensive interventions or services (e.g., special education). While practices for using RTI to determine student progress and to assist in decision making about further interventions is an issue that is still being examined for all students, the situation is more complex when language learning is also considered.

This study provided an initial examination of various commonly used assessment practices of RTI with ELLs and their implications with respect to the number of students who respond to intervention and their outcomes a year later. Since we are in the preliminary stages of understanding issues related to RTI and ELLs, the results of this study raise more questions than they provide answers. How Should We Determine Response to Intervention?

Of the three approaches examined here, only the Discrepancy Slope criteria at grade 1 predicted students who would meet the grade 2 criteria. This approach assessed whether growing at the same rate as typically achieving peers on measures of non-word reading, oral reading fluency, and reading comprehension resulted in a greater likelihood of meeting accepted standards in these areas one year later. It clearly did, albeit in the context of relatively few students meeting any of the criteria. In contrast, almost none of the at-risk students were able to meet either the benchmarks set by norming samples or those of their typically achieving peers.

Although not the focus of the current study, it should be noted that the benchmark for oral reading fluency of the longitudinal sample was higher, 52 CWPM, than that of the norming sample, 40 CWPM. Thus, in a large sample, ELLs not at risk who were receiving reading instruction in English were able to

perform as well as English monolingual students on measures of oral reading. Although discrepancy slope criteria best predicted the grade 2 criteria, these results clearly need replication, both in terms of these particular grade 1 and grade 2 RTI criteria and alternative RTI

criteria, in other ELL students who present with similar characteristics (i.e., significantly at risk) at the beginning of grade 1. Why Do So Few Students Meet Any of the Grade 1 or Grade 2 Criteria?

The answer to this question could be that the criteria we used were too stringent in terms of performance level as well as the number of measures. However, the performance-level criterion used for the non-word reading and comprehension subtest, the 37th %ile, was in the middle range compared to the performance level used in other studies (Al Otaiba & Fuchs, 2005). While we know from our past research (Linan-Thompson et al., 2006) that adding a fluency measure to the criteria reduces the percentage of students who meet the criteria, the benchmark of 40 CWPM is a commonly accepted end-of-first grade benchmark even for ELLs (Baker, 2003). The criteria may have been too stringent in that students needed to meet the target performance levels on every measure (i.e., rather than any single measure or any two measures). The current study emphasized meeting standards in all areas since the goal of instruction and intervention includes success in each of the areas of decoding, fluency, and comprehension.

Clearly, the choice of performance level and measures affected the results in that in an earlier study (Linan- Thompson et al., 2006), response rates were much higher, even though some of the same participants were included. Instead of a performance level of a standard score of 95, as in the current study, Linan-Thompson et al. considered a student to have met criteria with a standard score of 85. In addition, they examined only decoding and comprehension performance as opposed to also including fluency.

Another factor could be that these students started out too low and, therefore, despite an intensive intervention, were not able to reach the benchmark. Of particular note are the low scores in language skills in English and Spanish, with participants scoring, on average, two standard deviations below the norm. These students may be a group of low oracy students, and their results may not generalize to bilingual students participating in interventions with a higher level of oracy skills. The focus should be on continuing to develop interventions that will improve outcomes for ELLs so that they can meet the criteria regardless of their start-

ing point. It is reasonable to think that even with intensive interventions provided during first grade, students may require additional interventions in subsequent grades to compensate for their overall very low start and low levels of English language.

Students may have difficulty passing the grade 2 criterion, in part, for the some of the same reasons that




caused them to have difficulty meeting the grade 1 criteria: the stringency of the criteria and the low language and literacy start points prior to treatment.

A recent study by Chiappe and Siegel (2006) demon- strated that ELLs may acquire foundational literacy skills at the same rate as non-ELLs. However, they did not disaggregate the results for the lowest performing students so it may be that these students need more time. Regardless of their start point, it could be that in general ELLs need more time to solidify the skills they acquired in first grade and/or extend their English knowledge through instruction and experience. Once they have mastered basic reading skills in first grade, they have the opportunity to practice their literacy skills in second grade. Additionally, if they are reading, they have more opportunity to encounter new words and build their vocabulary and oracy. Further studies with ELLs are needed to evaluate the effectiveness at potentially earlier start points and with longer follow-up periods. It may be that students receiving intensive interventions early enough would demonstrate both improved grade 2 RTI response rates and continued improvement in later grades.

Why Don't Intervention and Comparison Students Differ in Their Grade 1 Response Rates Despite the General Effectiveness of the Intervention?

We know from previous reports of the data that intervention students outperformed comparison students immediately after intervention, as well as one year after intervention (Cirino et al., submitted; Vaughn, Cirino et al., 2006; Vaughn, Mathes et al., 2006). This occurred in a context in which it is not easy to do so, given that the participating schools were doing well according to state accountability measures. Therefore, one could expect that all students were receiving generally effective reading instruction from their classroom teacher. Though many of the comparison students also received supplemental reading instruction, the inclusion of actual supplemental instruction as a predictor did not change the fact that intervention students outperformed comparison students (Vaughn, Cirino et al., 2006).

It is important to consider that when it comes to RTI, the focus is on individual students meeting criteria and ensuring that they will continue to profit from general education reading instruction without additional intervention or referral for special education. This contrasts with studies that examine the effectiveness of the intervention overall, in which the performance of the treatment group is compared with the control providing evidence about what works but not evidence of whether it resolves all further difficulties in reading for

a given student. This is a critical distinction when decisions are made about identification and eligibility for special education. Future research might take a closer look at individual students and specific variables that might impact learning outcomes and rate of response to intervention. For example, exposure to English print outside of school or the extent to which students have opportunities to practice using English may impact their response to the intervention they are receiving at school.

If a Yearlong, Intensive Intervention Is Not Enough to Ensure Response to Intervention, Then What Is?

Notwithstanding the issues surrounding which criteria to use and which levels on which measures to define response, it is clear that based on relatively stringent criteria the rate of students' responsiveness in this study was low. An obvious answer to what might improve this responsiveness could be simply more years, but a more precise answer might be more time with high-quality instruction and opportunities to extend literacy and oracy skills. Students need more time acquiring basic foundational skills and practicing them. They need more time listening to, reading, speaking, and writing in English. They need more time developing academic language and higher order think- ing skills. However, more time does not necessarily mean more years. It might be a longer instructional day, or grouping and instructional practices that provide students more opportunities to actively use English, or it could be an extended school year.

CONCLUSIONS AND FURTHER QUESTIONS A final decision on the criteria for determining

whether ELLs' performance is sufficient for them to profit adequately from classroom instruction alone and/or to determine whether students would qualify for special education services cannot yet be made. In order to arrive at an accepted decision that resolves many of the prevailing questions about the application of RTI, further research is required. Many ELLs will profit from additional intervention, and for some it will serve as a protective factor, reducing the likelihood that they will require further intervention in the future. For others, the immediate benefit from instruction will be significant though inadequate to reduce the gap between their reading performance and grade-level expectations. For still others, when (what grade level) and how intensive (intervention for how long and under what conditions) is far less apparent.

We are aware that the findings of this study are limited to the approaches to response to intervention we selected and the criteria we established. While we considered the most prevailing practices for identifying




response to intervention to test, there are other ways to conceptualize and implement RTI that we did not address. Furthermore, we recognize that if we adjusted the criteria for RTI by lowering them, considerably more students would be classified as responders. Criteria for determining whether ELLs' performance is sufficient to profit adequately from classroom instruction alone and/or to determine whether students would qualify for special education services require additional research.

REFERENCES August, D., & Shanahan, T. (Eds.). (2006). Developing literacy in sec-

ond-language learners: Report of the national literacy panel on language-minority children and youth. Mahwah, NJ: Lawrence Erlbaum Associates.

Baker, S. (2003). Enhancing core reading instruction for English language learners in kindergarten and Grade 1. Presented at the Institute on Beginning Reading II, Salem, Oregon.

Carreker, S. (1999). Language enrichment. Bellaire, TX: Neuhaus Education Center.

Chiappe, P., & Siegel, L. S. (2006). A longitudinal study of reading development of Canadian children from diverse linguistic backgrounds. The Elementary School Journal, 107(2), 135-153.

Cirino, P. T., Vaughn, S., Linan-Thompson, S., Cardenas-Hagan, E., Fletcher, J. M., & Francis, D. J. (submitted). One-year followup outcomes of Spanish and English interventions for English language learners at-risk for reading problems.

Denton, C. A., Anthony, J. L., Parker, R., & Hasbrouck, J. (2004). Effects of two tutoring programs on the English reading development of Spanish-English bilingual students. The Elementary School Journal, 104(4), 289-305.

Donovan, M. S., & Cross, C. T. (2002). Minority students in special and gifted education. Washington, DC: National Academy Press.

Fletcher, J. M., Reid, L. G., Barnes, M., Stuebing, K. K., Francis, D. J., Olson, R. K., Shaywitz, S. E., & Shaywitz, B. A. (2002). In R. Bradley, L. Danielson, & D. Hallahan (Eds.), Identification of learning disabilities: Research to practice (pp. 185-239). Mahwah, NJ: Lawrence Erlbaum.

Fletcher, J. M., Shaywitz, S. E., Shankweiler, D. P., Katz, L., Liberman, I. Y., Steubing, K. K., et al. (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement profiles. Journal of Educational Psychology, 86(1), 6- 23.

Foorman, B. R., Francis, D. J., Fletcher, J. M., Mehta, P., & Schatschneider, C. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Pscyhology, 90(1), 37-55.

Francis, D. J., Rivera, M., Lesaux, N., Kiefer, M., & Rivera, H. (2006). Practical guidelines for the education of English language learners: Research-based recommendations for instruction and academic interventions. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13(4), 204- 219.

Fuchs, D., Fuchs, L. S., & Compton, D. L. (2004). Identifying reading disabilities by responsiveness-to-instruction: Specifying measures and criteria. Learning Disability Quarterly, 27(4), 216.

Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness to intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18(3), 157-171.

Gersten, R., Baker, S. K., Shanahan, T., Linan-Thompson, S., Chiappe, P., & Scarcella, R. (2006). Effective literacy and language instruction for English learners in the elementary grades, An IES practice guide. Washington, DC: IES, Department of Education.

Good, R. H., & Kaminski, R. A. (2002). Dynamic indicators of basic early literacy skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement.

Good, R. H., Bank, N., & Watson, J. M. (Eds.), (2003). Indicadores dindmicos del exito en la lectura. Eugene, OR: Institute for the Development of Educational Achievement.

Gunn, B., Biglan, A., Smolkowski, K., & Ary, D. (2000). The effi- cacy of supplemental instruction in decoding skills for Hispanic and Non-Hispanic students in early elementary school. The Journal of Special Education, 34(2), 90-103.

Gunn, B., Smolkowski, K., Biglan, A., & Black, C. (2002). Supplemental instruction in decoding skills for Hispanic and non-Hispanic students in early elementary school: A follow-up. Journal of Special Education, 36(2), 69-79.

Individuals with Disabilities Education Improvement Act of 2004, P.L. 108-446, 20 U.S.C. § 1400 et seq.

Klingner, J., & Edwards, (2006). Cultural considerations with Response to Intervention models. Reading Research Quarterly, 41(1), 108-117.

Linan-Thompson, S., Vaughn, S., Prater, K., & Cirino, P. T. (2006). The response to intervention of English language learners at- risk for reading problems. Journal of Learning Disabilities, 39(5), 390-398.

McCardle, P., & Chhabra, V. (Eds.). (2004). The voice of evidence in reading research. Baltimore: P. H. Brookes Pub.

McGraw Hill Reading. (2001). New York: McGraw Hill School Division.

McMaster, K. L., Fuchs, D., Fuchs, L.S., & Compton, D. L. (2005). Responding to nonresponders: An experimental field of trial of identification and intervention methods. Exceptional Children, 71(4), 445-463.

No Child Left Behind Act of 2001, Pub. L. 107-110, 20 U.S.C. § 6301 et seq. (2002)(enacted). Retrieved November 26, 2006, from http://www.ed.gov/policy/elsec/leg/esea02/107-110.pdf

O'Connor, R. E. (2000). Increasing the intensity of intervention in kindergarten and first grade. Learning Disabilities Research and Practice, 15(1), 43-54.

President's Commission on Excellence in Special Education. (2002). A new era: Revitalizing special education for children and their families. Washington, DC: Author.

Shanahan, T., & Beck, I. (2006). Effective literacy teaching for English language learners. In D. August & T. Shanahan (Eds.), Developing literacy in second-language learners: Report of the national literacy panel on language-minority children and youth (pp. 415-488). Mahwah, NJ: Lawrence Erlbaum Associates.

Speece, D. L., & Case, L. P. (2001). Classification in context: An alternative approach to identifying early reading disability. Journal of Educational Psychology, 93(4), 735-749.

Speece, D., Case, L. P., & Molloy, D. E. (2003). Responsiveness to general education instruction as the first gate to learning disabilities identification. Learning Disabilities Research and Practice, 18(3), 147-156.

Stanovich, K. E., & Siegel, L. S. (1994). Phenotypic performance profile of children with reading disabilities: A regression-based test of the phonological-core variable-difference model. Journal of Educational Psychology, 86(1), 24-53.

Torgesen, J. K., Alexander, A. W., Wagner, R. K., Rashotte, C. A., Voeller, K. K., & Conway, T. (2001). Intensive remedial instruction for children with severe reading disabilities: Immediate




and long-term outcomes from two instructional approaches. Journal of Learning Disabilities, 34(1), 33-58.

Torgesen, J. K., Wagner, R. K., Rashotte, C. K., Rose, E., Lindamood, P., Conway, T., & Garvan, C. (1999). Preventing reading failure in young children with phonological processing disabilities: Group and individual response to instruction. Journal of Educational Psychology, 91, 579-593.

Vaughn, S., Cirino, P. T., Linan-Thompson, S., Mathes, P. G., Carlson, C. D., Cardenas-Hagan, E., et al. (2006). Effectiveness of a Spanish intervention and an English intervention for English language learners at risk for reading problems. American Educational Research Journal, 43, 449-488.

Vaughn, S., Linan-Thompson, S., & Hickman-Davis, P. (2003). Response to instruction as a means of identifying students with reading/learning disabilities. Exceptional Children, 69(4), 391- 410.

Vaughn, S., Mathes, P., Linan-Thompson, S., Cirino, P., Carlson, C., Pollard-Durodola, S., Cardenas-Hagan, E., & Francis, D.J. (2006). Effectiveness of an English intervention for first-grade English language learners at risk for reading problems. The Elementary School Journal, 107, 153-180.

Vellutino, F. R., Scanlon, D. M., & Lyon, J. (2000). Differentiating between difficult-to-remediate and readily remediated poor readers: More evidence against the IQ-achievement discrepancy definition of reading disability. Journal of Learning Disabilities, 33(3), 223-238.

Vellutino, F. R., Scanlon, D. M., Sipay, E. R., Small, S. G., Pratt, A., Chen, R., & Denckla, M. B. (1996). Cognitive profiles of difficult-to-remediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of specific reading disability. Journal of Educational Psychology, 88(4), 601-638.

Woodcock, R. W. (1991). Woodcock Language Proficiency Battery- Revised. Chicago: Riverside.

Woodcock, R. W., & Johnson, M. B. (1989). Woodcock-Johnson Psycho-Educational Battery-Revised. Chicago: Riverside.

NOTE This research was supported in part by grants P01 HD39521 and R305U010001, both jointly funded by the National Institute for Child Health and Human Development and the Institute of Education Sciences. The attitudes and opinions expressed in this manuscript are those of the authors and do not necessarily reflect those of the funding agencies. The authors wish to thank their many collaborators, coworkers and students, parents, teachers, and school and district officials who made this research possible.

Please send correspondence to: Sylvia Linan-Thompson, The University of Texas, College of Education, 1 University Station, SZB D5300, Austin, TX 78712; [email protected]




english language learners and rti || determining english language learners' response to...

Documents