Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=urwl20

Download by: [University of Texas Libraries] Date: 25 May 2016, At: 07:10

Reading & Writing QuarterlyOvercoming Learning Difficulties

ISSN: 1057-3569 (Print) 1521-0693 (Online) Journal homepage: http://www.tandfonline.com/loi/urwl20

Middle School Reading Comprehension andContent Learning Intervention for Below-AverageReaders

Elizabeth Swanson, Jeanne Wanzek, Sharon Vaughn, Anna-Maria Fall, GregRoberts, Colby Hall & Veronica L. Miller

To cite this article: Elizabeth Swanson, Jeanne Wanzek, Sharon Vaughn, Anna-Maria Fall, GregRoberts, Colby Hall & Veronica L. Miller (2016): Middle School Reading Comprehension andContent Learning Intervention for Below-Average Readers, Reading & Writing Quarterly

To link to this article: http://dx.doi.org/10.1080/10573569.2015.1072068

Published online: 26 Apr 2016.

Submit your article to this journal

Article views: 30

View related articles

View Crossmark data

READING & WRITING QUARTERLY http://dx.doi.org/10.1080/10573569.2015.1072068

Middle School Reading Comprehension and Content Learning Intervention for Below-Average Readers Elizabeth Swanson,1 Jeanne Wanzek,2 Sharon Vaughn,1 Anna-Maria Fall,1 Greg Roberts,1

Colby Hall,1 and Veronica L. Miller1

1The University of Texas at Austin, Austin, Texas, USA; 2Vanderbilt University, Nashville, Tennessee, USA

ABSTRACT This study aimed to determine the efficacy of a content knowledge and reading comprehension treatment implemented by 8th-grade social studies teachers over the course of 1 school year. We randomly assigned 8th-grade students with reading difficulties to the intervention treatment condition (n à 45) or the business-as-usual comparison condition (n à 33). Students in the treatment condition scored statistically higher than students in the comparison condition on measures of knowledge acquisition (effect size [ES] à 0.35), content reading comprehension (ES à 0.59), and vocabulary recall (ES à 0.65). The small ES on standardized reading comprehension (ES à 0.10) favoring the treatment condition was not statistically significant.

In eighth grade, students are required to use the reading and comprehension skills mastered in elementary school to learn a great deal of new information in content area classes. They are faced with a variety of text sources that contain complex vocabulary and complicated syntax (Carnegie Council on Advancing Adolescent literacy, 2010; Heller & Greenleaf, 2007; Lee & Spratley, 2010). In addition, the Common Core State Standards that most states use include literacy standards integrated in middle school social studies, science, and technical subjects (National Governors Association Center for Best Practices and the Council of Chief State School Officers, 2010) based on the theory that through the use of complex text across content areas, students build discipline-specific reading ability and because of this improvement will also increase knowledge acquisition. Middle school students are expected to master skills in social studies such as (a) analyzing the author’s point of view and response to conflicting viewpoints; (b) determining word meaning and analyzing the impact of word choice; (c) integrating and evaluating content presented in diverse formats; (d) evaluating arguments and claims in text, including the validity of the reasoning and sufficiency of evidence; and (e) analyzing relationships between primary and secondary sources.

Not only is there a marked shift in learning expectations established by state and national stan-dards at the middle school level, but middle school is also a time when the role of the teacher changes—from teaching children to read and access print information as in elementary school to emphasizing content delivery. Indeed, middle school teachers view themselves as content experts and are often unable or unwilling to provide reading instruction within the content areas, citing that English language arts teachers or reading specialists should fulfill that role (Hall, 2005; Ness, 2007; O’Brien, Moje, & Stewart, 2001). These views were confirmed by the authors of a recent observation study (Swanson, Wanzek, McCulley, et al., 2015) that reported largely lecture-based social studies instruction with approximately 10% of instructional time spent reading text. Students concur with the observed state of social studies instruction, reporting largely lecture and note taking in their

CONTACT Elizabeth Swanson easwanson@austin.utexas.edu The Meadows Center for Preventing Educational Risk, The University of Texas at Austin, 1912 Speedway, D4900, Austin, TX 78712-1284, USA. Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/urwl. © 2016 Taylor & Francis

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

middle school classes (Chiodo & Byford, 2004). In a survey of social studies teachers’ opinions about effective instruction, 68% of secondary teachers identified lecture as both the most effective and the most frequently implemented instructional practice (Bolinger & Warren, 2007).

When asked why more text reading is not incorporated into social studies instruction, teachers often cite low-quality writing in textbooks and students’ inability to read (I. L. Beck, McKeown, Sinatra, & Loxterman, 1991; Youngbauer, 2015). In fact, 64% of eighth-grade students in the United States read at or below a basic level (National Center for Education Statistics, 2013) and are largely unable to access content through text or engage in complex reasoning outlined by the Common Core State Standards (National Governors Association Center for Best Practices and the Council of Chief State School Officers, 2010) without some kind of reading support in the classroom. With so many students with reading difficulties included in general education content area classrooms, it is of parti-cular importance to identify interventions that impact learning among students with a wide variety of abilities. One intervention that has been tested in general education classrooms (Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press), among students with disabilities (Swanson, Wanzek, Vaughn, Roberts, & Fall, 2015; Wanzek, Swanson, Vaughn, Roberts, & Fall, 2015), and among English language learners (Vaughn et al., in press) is called Promoting Acceleration of Comprehension and Content Through Text (PACT). Because it has been found effective across a variety of students, it stands to reason that PACT is a viable Tier 1 intervention that is poised to improve reading outcomes for the students with reading difficulties contained within general education content area classes.

The pact intervention

Evidence suggests that content approaches to reading framed within a text-processing view of comprehension that focus on content in the text as the vehicle for instruction can be effective in the general classroom (McKeown, Beck, & Blake, 2009; Vaughn, Swanson, et al., 2013; Vaughn et al., 2015) and among struggling readers included in general education social studies classes (Swanson, Wanzek, Vaughn, et al., 2015; Wanzek et al., 2015). Reading theorists and educators generally agree that proficient readers interact with the text by building a coherent representation, which involves extracting information from the text base and integrating information from multiple sources, including the text or background knowledge (Kintsch, 1998; van den Broek, 2010). This type of approach focuses on reading text to gather new information and then integrating these new ideas with previous learning (Kintsch, 1998) through discussion (Applebee, Langer, Nystrand, & Gamoran, 2003), summarization, and ques-tioning (I. L. Beck & McKeown, 2006). To this end, we designed an intervention called PACT that con-tains five components informed by the content learning model (Gersten, Baker, Smith-Johnson, Dimino, & Peterson, 2006) to improve understanding while reading text and provide opportunities for students to connect new learning to previous learning. The PACT intervention is a set of instruc-tional practices implemented daily within the social studies. Evidence suggests that when these components are presented as a unified instructional approach, students gain content knowledge and content reading comprehension (Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press). For an overview of how these components are taught over the 10-day unit; see Figure 1.

Comprehension canopy

The comprehension canopy is designed to build background knowledge and motivation. On the first day, students watch a short high-interest video. Teachers provide a purpose for viewing (e.g., “As you watch the video, write two reasons why the colonists called the First Continental Congress”) and a discussion afterward. An overarching question is introduced to unify the theme (e.g., Was the American Revolution inevitable? Why?). Each subsequent lesson begins with a review of the question and discussion of how new information informs an answer to the overarching question. At the end of the 10-day unit, students answer the comprehension canopy question during a whole-class or small- group discussion.

2 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

Essential words

A set of 4–5 high-utility, high-frequency concepts (e.g., revenue, petition) are taught and reviewed over the span of each 10-day unit. On Day 1, the teacher introduces each essential word using a stu-dent-friendly definition, visual representation, related words, sentences with the word in context, and question prompts for brief discussion of the word. Essential words are reinforced during warm-ups and are integrated into texts, comprehension checks, and knowledge application activities (all described below).

Warm-up

Throughout the unit, lessons begin with a 5-min review of an essential word using an activity that requires students to apply the meaning of the word. For example, for the word revenue, students are shown a graph listing revenues and spending for the U.S. government and asked to describe the relationship between revenue and spending over the years.

Critical reading

During critical reading sessions, students read and discuss information from primary and secondary sources of text related to the content in whole-class, small-group, paired, or individual reading arrangements. These grouping decisions are based on student need. Teachers engage students in

Figure 1. The 10-day lesson cycle. © The Meadows Center for Preventing Educational Risk, The University of Texas at Austin. Reproduced by permission of The Meadows Center for Preventing Educational Risk, The University of Texas at Austin. Permission to reuse must be obtained from the rightsholder.

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 3

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

classroom discourse and note taking that are designed to help students make connections to the com-prehension canopy, essential words, and previously learned material. In addition, essential words are reviewed when they are used in text.

Team-based learning (TBL)

We adapted an instructional feature previously implemented at the postsecondary level with medical, pharmaceutical, and business students called TBL (Michaelsen & Sweet, 2011). TBL provides students with opportunities to engage in text-based discussions and to provide text-based evidence to support ideas.

TBL comprehension checks The purposes of TBL comprehension checks are to ensure understanding of content and to provide teachers with data to guide subsequent instruction. First students complete a brief quiz individually with no access to text or notes; this quiz is turned in to the teacher for a grade. This provides an opportunity for individual accountability for content. Next students complete the check again with their heterogeneous team members. For each question, the team must agree on the answer and pro-vide evidence from the text to support the team’s decision. Scratch-off answer sheets are provided to teams for immediate feedback on accuracy. If the team scratches the correct answer a star is revealed. If the answer is incorrect, there is no star and the team returns to discuss the question using text sources to select an alternative answer. Scratch-off cards are available at http://www. epsteineducation.com/home/about/ and cost less than 20 cents each.

TBL knowledge application At the end of the unit, students work in their heterogeneous teams to complete a TBL knowledge application activity designed to apply and extend understanding of content learned from text and prior discourse by articulating new perspectives, solving problems, and presenting conclusions. For example, students may be given a set of cards with causes of the Revolutionary War. They must compare two at a time and decide which is a more compelling cause. They continue this process until they end up with the most compelling cause of the Revolutionary War. Students are then asked to write a paragraph explaining their choice.

PACT efficacy trials

The efficacy of PACT was investigated on three occasions in which eighth-grade social studies classes were randomly assigned to the PACT intervention condition or the typical practice comparison condition. Students received three 10-day units of the PACT intervention. Across all three studies, students in the treatment condition outperformed students in the typical practice comparison condition at statistically significant levels on a measure of content knowledge (Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press). Across two trials (one with English language learners), students in the PACT condition outperformed students in the typical practice condition on a measure of content reading comprehension (Vaughn, Swanson, et al., 2013; Vaughn et al., in press). Broad reading comprehension was impacted in only the first study (Vaughn, Swanson, et al., 2013).

In a study of quasi-experimental design, researchers investigated the impact of PACT on the performance of students with disabilities included in the general education classrooms in which the prior studies had taken place (Swanson, Wanzek, Vaughn, et al., 2015). Students with disabilities who received the PACT intervention outperformed students with disabilities in the typical practice comparison conditions on measures of content knowledge acquisition (effect size [ES] à 0.26) and content reading comprehension (ES à 0.34) but not standardized reading comprehension. Findings from these efficacy trials are quite promising. However, only three 10-day units of the PACT inter-vention were delivered, and with varying levels of implementation fidelity across classrooms. This

4 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

provides some indication that the effects of PACT may be more robust for students with reading dif-ficulties when implemented for a longer period of time and at higher levels of implementation fidelity.

Increased duration

In a 2013 synthesis of 19 studies investigating the effects of reading interventions delivered across 75 or more sessions for students with reading difficulties in Grades 4 through 12, ESs were small (d à 0.10) for comprehension outcomes (Wanzek et al., 2013). However, one study included in the synthesis stood out in terms of the duration of the study. Wanzek, Vaughn, Roberts, and Fletcher (2011) implemented a reading intervention for 50 min daily over 160 lessons. The intervention was characterized by reading practice, word study, vocabulary instruction, and text comprehension instruction that required students to provide text evidence to support their claims during class-wide discussions of content. Students assigned to this treatment condition outperformed students in the comparison condition at statistically significant levels on a standardized measure of reading compre-hension (ESs à 0.65–0.70). When one compares this outcome to similar interventions of shorter duration, it is possible that the duration of the intervention was one factor related to the effects.

The importance of implementation fidelity

The importance of implementation fidelity is acknowledged across a number of sources (e.g., Fogarty et al., 2014; Hulleman & Cordray, 2009; Swanson, Wanzek, Haring, Ciullo, & McCulley, 2013). How-ever, most studies to date in the field of education treat fidelity data descriptively. Because the role of implementation fidelity in reading outcomes has rarely been investigated, we look to the health sciences to provide some guidance. Results from a review of more than 500 studies (Durlak & DuPre, 2008) investigating the effects of health promotion programs for children indicated that the magnitude of the difference favoring interventions delivered with high levels of fidelity may be 2 to 3 times greater than for interventions delivered with low fidelity. This may be the case for educational interventions as well. Achieving high levels of fidelity has, in prior studies, increased the chances of program success. In one study investigating the effects of a reading strategy inter-vention among middle school students (Vaughn, Roberts, et al., 2013), researchers reported that implementation fidelity mediated reading comprehension outcomes. This finding of fidelity mediation was also reported in the most recent efficacy trial of the PACT intervention (Vaughn et al., 2015), in which increased knowledge acquisition scores were explained by increased fidelity.

Research questions

The purpose of this study was to determine the effects of PACT for students with reading difficulties when delivered for an increased duration with high levels of implementation fidelity. The following research question guided our work: What are the effects of the PACT intervention in social studies when delivered with high fidelity over the course of the academic year on eighth-grade struggling readers’ content knowledge, reading comprehension of content text, and general comprehension? We hypothesized the following: (a) Treatment students would outperform comparison students on measures of content knowledge acquisition, (b) the effects on content reading comprehension would favor the treatment group, and (c) a small effect in favor of the treatment group on a measure of general reading comprehension would be detected.

Method

Research design

We used a randomized controlled design to determine the effects of PACT on reading comprehension and content knowledge outcomes for students with reading difficulties. The school provided a

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 5

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

deidentified list of students with a scale score of 1639 or below (the state-identified passing scale score was 1556) on the state accountability measure in seventh grade. The state accountability measure is considered a low-bar measure, meaning that students who barely pass the measure may be considered students with reading difficulties as well. All incoming eighth-grade students who met the criteria were randomly assigned to the PACT treatment group or the typical practice comparison group.

Sample

Teachers Two male teachers and one female teacher participated in the study. All teachers held bachelor’s degrees and one held a degree in jurisprudence. The PACT treatment teacher (the male with the degree in jurisprudence) was hired and supervised by us. This teacher was experienced in delivering social studies content and the PACT intervention (for three 10-day units in a prior study) with high levels of implementation fidelity. We randomly assigned students to either the PACT condition or the typical practice condition. The school then placed the typical practice students into one of the two remaining social studies teachers’ classrooms (one male and one female teacher) according to school policy.

Students We randomly assigned a total of 108 students to the PACT intervention or typical practice con-ditions. Consent to participate in the study was collected after random assignment, producing a total of 78 eighth-grade students (treatment, n à 45; comparison, n à 33) whose parents provided consent for data collection. Most students were of Hispanic ethnicity, followed by White or African American. Approximately a quarter to a third of students received free or reduced-price lunch. Most students reported speaking English as their primary language at home. See Table 1 for additional information.

Measures

We examined student outcomes on the Modified Assessment of Social Studies Knowledge (MASK; Vaughn, Swanson, et al., 2013), the Gates-MacGinitie reading comprehension subtest (MacGinitie, MacGinitie, Maria, Dreyer, & Hughes, 2006), and the State of Texas Assessment of Academic Readiness (STAAR). The MASK and Gates-MacGinitie were administered to students in the treatment

Table 1. Student demographics.

Characteristic Comparison Treatment

n % n % Gender

Male 14 42.5 23 51.1 Female 18 54.5 21 46.7 Missing 1 3.0 1 2.2

Ethnicity White 5 15.2 4 8.9 African American 4 12.1 4 8.9 Hispanic 22 66.7 36 80.0 Asian 1 3.0 0 0 Missing 1 3.0 1 2.2

Free or reduced lunch 26 78.8 33 73.3 English language learners 8 24.2 0 0 Home language

English 24 72.7 33 73.4 Spanish 8 24.3 11 24.4 Missing 1 3.0 1 2.2

Special education 9 27.3 0 0

6 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

and comparison groups prior to and within 2 weeks following treatment. Trained research personnel who were uninformed of the condition (treatment or comparison) to which students were assigned administered these assessments. The STAAR was administered to students in both groups in April of the school year by trained and certified test administrators in accordance with state law.

MASK knowledge acquisition (Vaughn, Swanson, et al., 2013) The MASK knowledge acquisition assessment is a 46-item, four-option, untimed multiple-choice test that measures content knowledge covered through the intervention. With permission, items were collected from released Texas state social studies tests (Texas Assessment of Knowledge and Skills), released Massachusetts state social studies tests (Massachusetts Comprehensive Assess-ment System), and released Advanced Placement tests in social studies from the College Board. The following is a sample question from the MASK content knowledge measure:

Economic activity in the New England colonies relied heavily on trade in part because: (a) puritan beliefs pro-hibited farming for profit, (b) farmers in the region feared attacks from Native Americans, (c) a cold climate and poor soil made farming unprofitable, (d) the British monarchy provided land grants only to Southern colonies.

The MASK content knowledge measure was administered at pretest and at posttest. The MASK knowledge acquisition test was an extension of the Assessment of Social Studies Knowledge (ASK) measure used in a prior study (Vaughn, Swanson, et al., 2013). Although an extensive treatment of ASK psychometrics may be found in the original study, we note here that the alpha coefficient for the ASK content measure was .89 in a larger sample of students (Vaughn, Swanson, et al., 2013).

MASK content reading comprehension (Vaughn, Swanson, et al., 2013) The MASK content reading comprehension assessment is a 21-item, four-option, untimed multiple- choice test that measures content reading comprehension. The assessment consists of three reading passages (Lexile range à 1090–1140; word count range à 312–349), each of which is related to content covered in the year-long curriculum. Students read each passage silently and answer seven multiple- choice questions about the passage. Reading comprehension items were researcher developed and measured students’ ability to identify main ideas, understand vocabulary in context, identify cause and effect, and summarize. For example, after reading a passage titled “Salem Witchcraft Trials,” students are asked to answer the following question: “Based on the passage, which of the following words means almost the same thing as allegedly? (a) supposedly, (b) obviously, (c) fortunately, or (d) quietly.” The MASK reading comprehension test was an extension of the ASK measure used in a prior study (Vaughn, Swanson, et al., 2013). Although an extensive treatment of ASK psychometrics may be found in the original study, we note here that the alpha coefficient for the ASK comprehen-sion measure was .85 in a larger sample of students (Vaughn, Swanson, et al., 2013).

MASK vocabulary recall MASK vocabulary recall is a researcher-developed test that measures vocabulary. Students were asked to complete 30 multiple-choice vocabulary items that required students to identify the definition for the given word. At posttest they were also asked to complete 15 open-ended items for which a term was provided and students wrote a definition.

Gates-Macginitie reading comprehension subtest (fourth edition) The Gates-MacGinitie reading comprehension subtest is a group-administered, timed (35 min) assessment of reading comprehension. The assessment consists of expository and narrative passages ranging in length from three to 15 sentences. Students read each passage silently and answer three to six multiple-choice questions related to the most recently read passage. As the students progress through the assessment, items increase in difficulty. Internal consistency reliability ranges from .91 to .93, and alternate-forms reliability is reported as .80 to .87.

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 7

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

STAAR The STAAR is the accountability assessment used in the State of Texas. Tests are given to students in various content areas. The STAAR is group administered and timed (4 hr). Test items assess student progress related to the Texas Essential Knowledge and Skills and college and career readiness stan-dards. Using cut scores, students are placed into three categories of academic performance: advanced, satisfactory, and unsatisfactory. The Texas Education Agency (n.d.) reported overall alpha coefficients for the 2013 STAAR reading and social studies of .891 and .899, respectively, across the statewide sample (N à 344,892).

Intervention procedures

Social studies classes met daily for 45-min class periods. All eighth graders in both treatment and comparison groups received social studies instruction that conformed to state standards, and the same scope and sequence (i.e., unit topics and content) was followed by all three teachers, meaning that all students received the same opportunity to learn content from the standard curriculum regard-less of assignment to treatment or comparison classes. However, the intervention teacher delivered each instructional unit (eleven 8- to 15-day units) using the PACT instructional methods (i.e., com-prehension canopy, essential words, critical reading, TBL comprehension check, and TBL knowledge application).

Professional development In the year prior to implementation, the PACT teacher attended 10 hr of professional development, taught the PACT intervention for 30 days, and received coaching support as part of a prior efficacy trial (see Vaughn, Swanson, et al., 2013). Lead researchers delivered the professional development with assistance from doctoral-level research assistants and teachers who had experience delivering the intervention. In the summer prior to this study, the PACT teacher attended a second 6-hr refresher professional development session provided by the lead researcher and one research assistant. Both workshops covered (a) intervention component implementation, (b) procedures to facilitate stu-dent discourse and use of text evidence to support claims, and (c) a detailed description of the experi-mental study design with an emphasis on the importance of high implementation fidelity.

Classroom support The PACT teacher received semiscripted lesson plans and daily schedules detailing the PACT components to be delivered on each day. Students received a set of student materials that included essential word logs and copies of reading passages with embedded stopping points for discussion and note taking. All TBL comprehension check and knowledge application activity materials were provided as well. A doctoral candidate–level research assistant with extensive teaching experience and experience implementing the PACT intervention provided in-person coaching once per week. Coaches provided modeling, coteaching, monitoring of student work during teacher instruction, observation and feedback, support in planning for lessons, and ways of implementing PACT effec-tively for students who struggled in school or had disabilities (e.g., constructive feedback, text reading supports). Coaches made 42 visits throughout the school year (a little more than once per week).

Implementation fidelity

PACT intervention classes Beginning in October of the year, the six PACT class periods were audio recorded every day. The teacher did not record class periods spent in school-wide assembly or when the majority of the period was interrupted by fire drills and so on. This produced a set of 438 treatment observations. Five research assistants (one doctoral level, three doctoral candidate level, one master’s level) were trained by the lead author to use the fidelity rating scale and accompanying codebook. Coders met 90%

8 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

agreement using the gold standard method of establishing interrater reliability. To ensure that we (a) coded samples from every class period, (b) avoided selection bias when identifying the class periods, and (c) secured a sample of each PACT intervention component, we chose 20% of the recordings (n à 82) using the following steps: (a) A random list of numbers ranging from 1 to 6 (i.e., the number of class periods) was generated, and (b) this list was matched with a chronological list of the instruc-tional dates to identify the target class for each day of recording. For example, following the random list of numbers, on Day 1, Period 5 was coded; on Day 2, Period 3 was coded; on Day 3, Period 2 was coded; and so on.

Using the fidelity rating scale, trained coders listened to the audio recording of each class period to identify the level of alignment of implementation with the intended intervention. Raters scored the fidelity of each component on a Likert-type scale ranging from very low to very high alignment (1 à very low, 4 à very high). Fidelity (see Table 2) was rated as mid-high to high in a majority of observations across all intervention components. In particular, three fourths of essential word warm-ups, essential word introduction, TBL comprehension checks, and critical reading were rated as mid-high to high in implementation fidelity. Moreover, 60% or more of the comprehension canopy and TBL knowledge application components were rated as mid-high to high. These fidelity findings indicate generally high levels of intervention fidelity and provide evidence that the intervention was implemented as intended.

Comparison classes Comparison teachers identified 1 day per month when social studies instruction was expected to be uninterrupted. Doctoral candidate–level observers conducted in-person observations of all eighth- grade social studies classes taught by the teachers on the selected days. A total of 55 observations were made (27 for one comparison teacher, 28 for the other comparison teacher). The goal of these obser-vations was twofold. First, we wanted to know the extent to which we observed PACT components implemented in the comparison classes. To meet this goal, observers used the PACT treatment fidelity form described previously. Second, we wanted to know what types of instructional methods were used in these classrooms. Observers used an observation form designed for observing literacy activities in social studies content area classes (Swanson, Wanzek, McCulley, et al., 2015). This form included records for text type; number of minutes spent reading; and type of instruction, including (a) definitions in isolation, (b) definitions in context, (c) example and nonexamples of vocabulary (d) morphology, (e) context clues, (f) previewing, (g) activating or building background knowledge, (h) comprehension monitoring, (i) comprehension strategies, and (j) discussion. Interrater reliability for both forms was established using the same procedures as for the treatment classes.

Table 2 presents data collected from the PACT treatment fidelity forms completed during comparison observations. Data indicate that intervention components were minimally observed in

Table 2. Implementation fidelity and comparison group observations.

Implementation CC EW WU CR TBLC TBLK

n % n % n % n % n % n % Treatment classrooms

4 àHigh 2 22.2 3 12.5 16 66.7 21 35.0 4 23.5 3 àMid-high 4 44.4 17 70.8 2 8.3 23 38.3 12 70.6 3 60.0 2 àMid-low 3 33.3 3 12.5 5 20.8 12 20.0 1 5.9 1 à Low 1 4.2 1 4.2 4 6.7 2 40.0

Comparison classrooms 4 àHigh 4 7.3 3 àMid-high 2 àMid-low 4 7.3 17 30.9 28 50.9 14 25.5 4 7.3 1 à Low

Note. Sample sizes for treatment classrooms are as follows: CC à 9, EW à 24, WU à 24, CR à 60, TBLC à 17, TBLK à 5. Sample sizes for comparison classrooms are 55 for each implementation. CC à comprehension canopy; EW à essential words; WU àwarm-up; CR à critical reading; TBLC à team-based learning comprehension check; TBLK à team-based learning knowledge application.

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 9

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

comparison classrooms. When they were observed, the components were recognizable at the lowest level of implementation. For example, minimal overlap between PACT components and instruction in control classes was observed in essential words, warm-up, critical reading, and comprehension of text, which were all observed with mid-low implementation. TBL and knowledge application compo-nents were not observed in comparison classes. Teacher lecture with note taking was observed for 18.8% of total class time, and text reading was observed for 192 min (7.7%) across both comparison teachers. These findings are consistent with previous observation studies of social studies instruction (Chiodo & Byford, 2004; Swanson, Wanzek, McCulley, et al., 2015). Presenting definitions for words occupied 18.3% of all class time observed (range à 8.9%–27.3%), primarily in the form of copying vocabulary definitions from the textbook glossary. Quiz completion consumed 14.3% of class time, and video watching represented 6.7% of class time. Teachers were never observed providing compre-hension strategy instruction.

Analysis

To estimate treatment effects, we fit a series of regression models in Mplus 7.2. Treatment status was dummy coded (1 à treatment). Pretest scores were grand mean centered (Enders & Tofighi, 2007) and included as a covariate. ESs were estimated as the ratio between the model-derived coefficient for treatment and the pooled within-group standard deviation across conditions at posttest. Not all participants had complete data on the variables used in the regression models. From the original sample of 78 students, 14.8%, 20%, 5%, 15%, and 4.3% of the participants had missing data for MASK knowledge acquisition, MASK reading comprehension, MASK vocabulary recall, Gates-MacGinitie reading comprehension, and STAAR social studies, respectively. We addressed missing data using multiple imputation, which uses data across the full complement of variables to estimate missing values (Collins, Schafer, & Kam, 2001). A total of 1,000 data sets were imputed using the routine available in Mplus 7.2.

Results

Table 1 summarizes demographic data for students with data on at least one measurement occasion. There were significant differences across treatment conditions in the number of English learners (EL) and the number of students with special needs who provided pretest and posttest data: EL students and students with special needs were entirely unrepresented in the treatment data, although they accounted for about a quarter of the available data in the comparison sample. Students were randomly assigned to either the treatment or comparison condition, and all students participated according to their assigned condition, including EL students and students with special needs. The rates of noncon-sent (for participation in pretest and posttest data collection) were similar in both conditions (46% and 41% for treatment and comparison, respectively). However, parents of treatment-assigned EL stu-dents and students with special needs were more likely than parents of similar children in the busi-ness-as-usual comparison not to consent to the collection of study-related data (we were granted access to students’ data on the state-wide high-stakes test, the STAAR), presumably because of concerns about the experimental status of the intervention and their children’s membership in traditionally vulnerable groups. This demographic imbalance represents bias to the extent that EL students and students with special needs score lower at pretest on average than students who are not EL or who do not have special needs.

To evaluate the situation, we treated nonconsented students in both groups as instances of pre-treatment (and postrandomization) attrition and evaluated attrition-related bias using seventh-grade STAAR scores. Using two-way analysis of variance, we compared STAAR scores across treatment groups, across consent status, and across the four groups represented by the interaction of treatment and consent status. We found no significant differences for condition (p à .08), for consent status (p à .70), or for the interaction of condition and consent (p à .3). In other words, there were no

10 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

differences between the treatment groups at pretest on the STAAR, no differences between students who declined to complete the pretest battery and those who agreed, and no condition-related differences in STAAR scores for consented and nonconsented students. This pattern suggests that attrition in the groups may not have biased the sample at pretest (see Table 3).

For consented students (n à 78), Table 4 presents imputed pretest and posttest means and stan-dard deviations for MASK knowledge acquisition, MASK reading comprehension, MASK vocabulary recall, Gates-MacGinitie reading comprehension, and the STAAR social studies subtest. For the three measures included in the pretest battery, there were no statistically significant differences in mean raw scores across the two conditions. On MASK knowledge acquisition, treatment students (n à 45) averaged 26.75 items correct and comparison students (n à 33) averaged 25.62 correct responses (p à .57). The variability in the two groups’ scores was very similar (SD à 8.29 and 8.21 for treatment and comparison, respectively). On the ASK reading comprehension measure, mean raw pretest scores were 7.24 and 7.22 for the imputed data sets, and standard deviations were 2.97 and 3.26 for treatment and comparison, respectively (p à .98). Finally, average standard scores for treatment and comparison students on the Gates-MacGinitie were 90.94 and 91.94 (p à .70), respectively.

Regression analyses were used to evaluate the effect of treatment (Table 5). For MASK knowledge acquisition, MASK reading comprehension in social studies, and the Gates-MacGinitie, outcomes were modeled as a function of the intercept, the grand mean–centered pretest, and the dummy-coded term for assignment. Because MASK vocabulary recall and the STAAR eighth-grade social studies test were not included in the pretest battery, outcomes were modeled in terms of the intercept term and individual treatment status.

The effect of the intervention on MASK outcomes was statistically significant for knowledge acqui-sition (b à 4.02, p à .03), reading comprehension (b à 2.51, p à .00), and vocabulary recall (b à 1.93, p à .00). The respective ESs were 0.35, 0.59, and 0.65. On the Gates-MacGinitie, treated students scored about 1 standard score better at posttest than comparison students, all else being equal (b à 1.08, p à .55, ES à 0.10). Although this difference did not differ statistically from zero, the ES

Table 4. Imputed pretest and posttest means and standard deviations for reading outcomes.

Measure Pretest Posttest

M SD M SD MASK knowledge acquisition

Treatment (n à 45) 26.75 8.29 39.78 12.24 Comparison (n à 33) 25.62 8.21 34.61 10.31

MASK reading comprehension in social studies Treatment (n à 45) 7.24 2.97 11.19 4.70 Comparison (n à 33) 7.22 3.26 8.67 3.45

MASK vocabulary recall Treatment (n à 45) 4.25 3.30 Comparison (n à 33) 2.31 2.49

Gates-MacGinitie reading comprehension Treatment (n à 45) 90.94 10.75 94.54 11.15 Comparison (n à 33) 91.94 11.40 94.15 9.23

STAAR eighth-grade social studies Treatment (n à 45) 3353.34 289.29 Comparison (n à 33) 3179.63 497.95

Note. MASK àModified Assessment of Social Studies Knowledge; STAAR à State of Texas Assessment of Academic Readiness.

Table 3. Equivalence of groups on the seventh-grade state of texas assessment of academic readiness. Classroom Group n M SD Treatment Nonconsented 31 1535.45 95.62

Consented 36 1542.58 65.78 Comparison Nonconsented 27 1570.48 63.85

Consented 39 1553.33 63.04

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 11

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

(.10) is not inconsequential for studies of reading intervention in older samples of struggling readers (see Edmonds et al., 2009). On the STAAR eighth-grade social studies test, although group differences did not meet the .05 threshold for statistical significance, the difference met the What Works Clear-inghouse (2014) standard for practical significance (b à 173.71, p à .06, ES à 0.44). The moderate to large ES combined with the marginal p value suggests that the intervention has considerable potential for impacting knowledge-related outcomes on distal, high-stakes measures. Replication in a larger sample might yield statistical (as well as practical) significance.

Discussion

This study targeted students with reading difficulties in their general education social studies classes. Students with reading problems in middle school may struggle with the reading demands in their content area classes, in which acquiring knowledge through informational text sources is often required, putting them at risk for low academic achievement (Bulgren, Deshler, & Lenz, 2007; Lee & Spratley, 2010; Paxton, 1999; Snow, Porche, Tabors, & Harris, 2007). The previous research on PACT has demonstrated overall positive effects for the intervention in social studies classes, but the group of students with the lowest pretest scores, who have also exhibited the lowest reading scores, have generally not experienced the same growth in the PACT intervention as their peers (Swanson, Wanzek, Vaughn, et al., 2015). However, these previous studies were relatively short (6–10 weeks), fidelity of implementation was at a medium level, and the teachers were particularly challenged in the implementation of critical reading and knowledge application. Thus, in the current study, we sought to increase the duration and fidelity of implementation to determine possible outcomes for students with reading difficulties who are learning the general education social studies curriculum.

We first examined the effects of the PACT intervention on student content knowledge. Students in a PACT intervention were encouraged to engage with the content and participate in discussion and application of the content more than traditional types of instruction generally promote (D. Beck & Eno, 2012; Bolinger & Warren, 2007; Swanson, Wanzek, McCulley, et al., 2015). Previous studies with a range of students have reported benefits of the PACT treatment on knowledge acquisition (Vaughn, Swanson, et al., 2013; Vaughn et al., 2015). We hypothesized that PACT students with reading diffi-culties would also demonstrate improved content knowledge acquisition with the increased duration of intervention and high fidelity of implementation. The findings suggested that the PACT implementation yielded significantly higher social studies content knowledge for participating stu-dents than the typical instruction. The PACT study group performed significantly higher than

Table 5. Fixed effects for outcomes. Measure Predictor Estimate SE p Effect size MASK knowledge acquisition Intercept 35.28 1.30 .00

Pretest 1.02 0.09 .00 Intervention 4.02 1.86 .03 .35

MASK reading comprehension in social studies Intercept 8.67 0.61 .00 Pretest 0.45 0.17 .00

Intervention 2.51 0.94 .00 .59 MASK vocabulary recall Intercept 2.31 0.45 .00

Pretest Intervention 1.93 0.67 .00 .65

Gates-MacGinitie reading comprehension Intercept 93.76 1.31 .00 Pretest 0.68 0.08 .00

Intervention 1.08 1.80 .55 .10 STAAR eighth-grade social studies Intercept 3179.63 69.13 .00

Pretest Intervention 173.71 91.77 .06 .44

Note. MASK àModified Assessment of Social Studies Knowledge; STAAR à State of Texas Assessment of Academic Readiness.

12 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

students in typical instruction on both the MASK knowledge acquisition measure and the measure of content vocabulary. In relation to the previous work on PACT, the moderate ES of 0.35 on knowledge acquisition was larger than the ES of 0.17 noted for eighth graders in Vaughn, Swanson, et al. (2013) and comparable to the ESs of 0.32 in the Vaughn et al. (2015) replication and 0.26 for students with disabilities (Swanson, Wanzek, Vaughn, et al., 2015). Thus, student improvement was generally com-parable to the previous research, but in this study the improvements related to a full year of the social studies content as opposed to three unit topics, demonstrating a wider breadth of knowledge gains than in the previous research. Despite group differences in knowledge acquisition and vocabulary in the current study, there were no differences between groups on the state outcome measure of social studies content, suggesting that the treatment was not strong enough to help students in the PACT treatment outperform students in typical instruction on a measure used by most schools as one important metric of student learning. Nonetheless, the ES of 0.44 in favor of the treatment indicated that students in the PACT treatment did perform almost 0.5 SD higher on the state social studies test, which may be practically meaningful (What Works Clearinghouse, 2014). In further examining the data, we found that 38.1% of the treatment students scored above the passing level on the state social studies measure, whereas 25.9% of the comparison students scored above the passing level—a prac-tical metric that the schools use to make content decisions. Thus, there would be a reduction of 12% in the number of students with reading difficulties who require content remediation during the sum-mer months as a result of this increased performance in PACT.

We next asked whether the PACT intervention could improve outcomes for students in content area reading comprehension or general reading comprehension. We hypothesized that students would improve their content area reading comprehension because of the increased levels of supported text reading in the content area in the PACT intervention. We also hypothesized a small effect in favor of the treatment group on general reading comprehension. Though previous research has noted mixed results of the PACT implementation on the effects of content area reading, this study implemented PACT for a substantially longer period of time, increasing the amount of text reading that was implemented with students. In addition, the fidelity of implementation for the critical reading component was higher in this study than any of the previous studies. As hypothesized, students in the PACT classes significantly outperformed students in the typical classes on content reading comprehension. The ES of 0.59 was larger than any of the previous studies of PACT implementation for all students (Vaughn, Swanson, et al., 2013 [ES à 0.29]; Vaughn et al., 2015 [ES à 0.02]) and students with disabilities (Swanson, Wanzek, McCulley, et al., 2015 [ES à 0.34]). Thus, the implementation of PACT in this study for students with reading difficulties yielded more robust findings for content reading comprehension than the previous work. This finding may be particularly notable given that typical instruction in this study included more text reading and vocabulary instruc-tion than was noted for the comparison classes in previous studies. The increased duration and fid-elity related to the supported reading of text as well as the sustained effective vocabulary instruction and review across the school year may explain these findings. Instead of students reading connected text three times per week over a period of six to 10 weeks (Swanson, Wanzek, Vaughn, et al., 2015; Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press; Wanzek et al., 2015), students in this study read connected text daily that was followed by classroom discourse. They engaged in the vocabulary routine over 11 units of study instead of the three units in prior studies of PACT (Swanson, Wanzek, Vaughn, et al., 2015; Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press; Wanzek et al., 2015). Long-term engagement in text reading and explicit vocabulary instruction may have led to these greater effects.

We also hypothesized that improvements in content knowledge and content area reading due to PACT would not generalize to students’ general reading comprehension. PACT implementation is specific to the social studies content area and does not include components related to general reading comprehension. The ES of 0.10 on the measure of general reading comprehension (i.e., the Gates- MacGinitie) was not statistically significant. However, the ES was generally aligned with the 0.11 ES detected across 13 studies of middle school interventions on broad-scope standardized tests

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 13

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

(Lipsey et al., 2012). Our interpretation of this phenomenon is that there is some effect on reading comprehension, but the effect is small. This study may well have been underpowered to detect this small effect.

These findings are particularly poignant for schools selecting Tier 1 instruction for the content areas that improves both content knowledge and reading outcomes such as vocabulary and reading comprehension. The effects of Tier 1 interventions (e.g., Denton, Bryan, Wexler, Reed, & Vaughn, 2007) on struggling readers have been investigated in few prior studies (e.g., Swanson, Wanzek, Vaughn, et al., 2015; Vaughn et al., 2010; Wanzek et al., 2015). In one such study (Vaughn et al., 2010), content area teachers met with trainers on a regular basis to learn about effective Tier 1 vocabulary and reading comprehension instruction and then write lesson plans incorporating these practices. Some struggling readers received Tier 1 only whereas others received Tier 1 áTier 2 instruction. Students who received Tier 1 áTier 2 instruction outperformed those who received Tier 1 alone on measures of word attack, spelling, comprehension, and decoding. However, struggling readers who received Tier 1 only made gains over the course of the year. The two other studies of PACT as Tier 1 instruction for struggling readers provided evidence that Tier 1 instruction can impact content knowledge (Swanson, Wanzek, Vaughn, et al., 2015; Wanzek et al., 2015), content reading comprehension, and general reading comprehension (Swanson, Wanzek, Vaughn, et al., 2015). There are critical differences between the Vaughn et al. (2010) study and the Swanson, Wanzek, McCulley, et al. (2015) and Wanzek et al. (2015) studies. In the later two studies, Tier 1 instruction within social studies classrooms was supported in several ways. It may be that in order for Tier 1 instruction to significantly impact reading outcomes for struggling readers, the following must be provided to teachers: (a) well-developed lesson plans that teachers can follow while learning the vocabulary and comprehension instruction components, (b) coaching support so that teachers may view high-quality, expert models of the instruction, and (c) instruction that is highly aligned with content that allows teachers to move through content at an appropriate pace while implementing the reading components class wide.

In combination with previous research, the findings of this study provide evidence that the PACT implementation improves student content acquisition in the social studies. Replications of this effect have been conducted with students in general education classes (Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press), students with reading difficulties (the current study), and students with identified disabilities (Swanson, Wanzek, Vaughn, et al., 2015; Wanzek et al., 2015). In addition, the PACT implementation in this study demonstrated improvements in content area reading comprehension for students with reading difficulties, similar to the previous finding for students with disabilities (Swanson, Wanzek, Vaughn, et al., 2015). The PACT implementation in social studies classes appears to slightly improve students’ general reading comprehension. For these reasons, adopting PACT as a Tier 1 instructional package in content area classes—particularly social studies—is likely to impact outcomes for general education studies as well as struggling readers.

Limitations and future research

Limitations

These results are subject to several limitations. First, the internal validity of this study is somewhat limited by the fact that one teacher implemented the PACT intervention in social studies and two other teachers implemented business-as-usual comparison instruction in social studies. However, it should be noted that the curriculum—that is, the topics and scope and sequence of instruction— was the same across treatment and comparison classes. The only difference was the instructional practices used during the PACT intervention. There is no reason to believe that any teacher had more advanced knowledge of the social studies topics than another. In addition, the only teacher with an advanced degree was the treatment teacher. However, the advanced degree was in jurisprudence and arguably did not create a situation whereby one teacher received more college-delivered instruction in

14 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

pedagogical methods. Because our findings are not misaligned with prior findings related to the PACT intervention (e.g., Swanson, Wanzek, Vaughn, et al., 2015; Vaughn, Swanson, et al., 2013) and syntheses of reading interventions delivered to middle school students (e.g., Edmonds et al., 2009; Lipsey et al., 2012), we do not believe the effects from this study are due to teacher effects. In addition, by adding a treatment teacher at the eighth-grade level, we reduced the class size in both the treatment and comparison classrooms, thereby calling into question the feasibility of the inter-vention in intact classrooms. Classroom teachers with intact classrooms conducted all prior investi-gations of PACT and reported that the intervention was feasible. In addition, fidelity scores in the medium to high range in prior studies support the assertion that PACT can be implemented with efficacy with full classrooms. Second, the comparisons are underpowered given the small sample size. For example, although not statistically significant from zero, the standardized difference in the means for treatment/nonconsented and control/nonconsented on the STAAR eighth-grade social studies test was .44, which is a moderate-size difference. Furthermore, the greater variation (SD à 95.62) in the treatment/nonconsented group on the STAAR eighth-grade social studies test may reflect the greater prevalence of EL students and/or students receiving special education services. A final consideration is the distinction between status and growth. To the extent that students in special education or EL students respond differently to intervention compared to other groups, equivalence at pretest does not exclude the possibility of bias.

Future research

In this tightly controlled efficacy trial, the PACT intervention was delivered by a member of the research team who was hired because of his ability to implement PACT with high levels of fidelity. He was also provided with extensive coaching support. Future effectiveness studies should be conducted to deter-mine whether findings are replicated when PACT is implemented for the full school year by teachers who are trained in PACT but are school employees, thus allowing fidelity to vary across classrooms. A series of studies (Swanson, Wanzek, Vaughn, et al., 2015; Vaughn, Swanson, et al., 2013; Vaughn et al., 2015; Vaughn et al., in press) have now confirmed the difficulty teachers have implementing the knowledge application component of TBL during their social studies classes. More student-centered instructional activities for inquiry like knowledge application that include discussion, debate, role play-ing, or project/problem-based learning have been encouraged in teacher training and professional devel-opment (e.g., National Council for Social Studies, 2010); however, there is limited research on the direct outcomes for students under these instructional methods (D. Beck & Eno, 2012; Nussbaum, 2008). The challenges we have noted in implementing activities like knowledge application suggest that future research is needed to examine the direct impact of these activities on student outcomes as well as the type of support teachers need to be able to effectively implement these instructional activities in the gen-eral education classroom. Finally, because this was an investigation of the impact of a multicomponent intervention, it was not possible within the current study to determine the relative impact of the indi-vidual components. Researchers should consider future studies that answer this important question.

Funding

The research reported here was supported by the Institute of Education Sciences of the U.S. Department of Education through Grant No. R305F100013 to The University of Texas at Austin as part of the Reading for Understanding Research Initiative. The opinions expressed are our own and do not represent the views of the Institute of Education Sciences or the U.S. Department of Education.

References

Applebee, A. N., Langer, J. A., Nystrand, M., & Gamoran, A. (2003). Discussion-based approaches to developing under-standing: Classroom instruction and student performance in middle and high school English. American Educational Research Journal, 40, 685–730. doi:10.3102/00028312040003685

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 15

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

Beck, D., & Eno, J. (2012). Signature pedagogy: A literature review of social studies and technology research. Computers in the Schools, 29, 70–94. doi:10.1080/07380569.2012.658347

Beck, I. L., & McKeown, M. G. (2006). Improving comprehension with questioning the author: A fresh and expanded view of a powerful approach. New York, NY: Scholastic.

Beck, I. L., McKeown, M. G., Sinatra, G. M., & Loxterman, J. A. (1991). Revisiting social studies text from a text- processing perspective: Evidence of improved comprehensibility. Reading Research Quarterly, 26, 251–276. doi:10.2307/747763

Bolinger, K., & Warren, W. J. (2007). Methods practiced in social studies instruction: A review of public school teachers’ strategies. International Journal of Social Education, 22, 68–84.

Bulgren, J., Deshler, D. D., & Lenz, B. K. (2007). Engaging adolescents with LD in higher order thinking about history concepts using integrated content enhancement routines. Journal of Learning Disabilities, 40, 121–133. doi:10.1177/ 00222194070400020301

Carnegie Council on Advancing Adolescent Literacy. (2010). Time to act: An agenda for advancing adolescent literacy for college and career success. New York, NY: Carnegie Corporation of New York.

Chiodo, J. J., & Byford, J. (2004). Do they really dislike social studies? A study of middle school and high school students. Journal of Social Studies Research, 28(1), 16–26. doi:10.3200/tsss.97.5.187-188

Collins, L. M., Schafer, J. L., & Kam, C. M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6, 330–351. doi:10.1037/1082-989x.6.4.330

Denton, C., Bryan D., Wexler J., Reed D., & Vaughn S. (2007). Effective instruction for middle school students with reading difficulties: The reading teacher’s sourcebook. Austin: University of Texas System/Texas Education Agency.

Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350. doi:10.1007/s10464-008-9165-0

Edmonds, M. S., Vaughn, S., Wexler, J., Reutebuch, C., Cable, A., Tackett, K. K., & Schnakenberg, J. W. (2009). A synthesis of reading interventions and effects on reading comprehension outcomes for older struggling readers. Review of Educational Research, 79, 262–300. doi:10.3102/0034654308325998

Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological methods, 12, 121–138. doi:10.1037/1082-989x.12.2.121

Fogarty, M., Oslund, E., Simmons, D., Davis, J., Simmons, L., Anderson, L., … Roberts, G. (2014). Examining the effectiveness of a multicomponent reading comprehension intervention in middle schools: A focus on treatment fidelity. Educational Psychology Review, 26, 425–449.

Gersten, R., Baker, S. K., Smith-Johnson, J., Dimino, J., & Peterson, A. (2006). Eye on the prize: Teaching complex his-torical content to middle school students with learning disabilities. Exceptional Children, 72, 264–280. doi:10.1177/ 001440290607200301

Hall, L. A. (2005). Teachers and content area reading: Attitudes, beliefs and change. Teaching and Teacher Education, 21, 403–414. doi:10.1016/j.tate.2005.01.009

Heller, R., & Greenleaf, C. (2007). Literacy instruction in the content areas: Getting to the core of middle and high school improvement. Washington, DC: Alliance for Excellent Education.

Hulleman, C. S., & Cordray, D. S. (2009). Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness, 2, 88–110. doi:10.1080/19345740802539325

Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press. Lee, C. D., & Spratley, A. (2010). Reading in the disciplines: The challenges of adolescent literacy. New York, NY:

Carnegie Corporation of New York. Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., … Busick, M. D. (2012). Translating

the statistical representation of the effects of education interventions into more readily interpretable forms (NCSER 2013–3000). Washington, DC: National Center for Special Education Research.

MacGinitie, W. H., MacGinitie, R. K., Maria, K., Dreyer, L. G., & Hughes, K. E. (2006). Gates-MacGinitie reading test (4th ed.). Rolling Meadows, IL: Riverside.

McKeown, M. G., Beck, I. L., & Blake, R. G. K. (2009). Rethinking reading comprehension instruction: A comparison of instruction for strategies and content approaches. Reading Research Quarterly, 44, 218–253. doi:10.1598/ RRQ.44.3.1

Michaelsen, L. K., & Sweet, M. (2011). Team-based learning. New Directions for Teaching and Learning, 128, 41–51. doi:10.1002/tl.467

National Center for Education Statistics. (2013). A first look: 2013 mathematics and reading (NCES 2014–451). Washington, DC: U.S. Department of Education.

National Council for the Social Studies. (2010). National curriculum standards for social studies: A framework for teaching, learning, and assessment. Silver Spring, MD: Author.

National Governors Association Center for Best Practices and the Council of Chief State School Officers. (2010). Common Core State Standards initiative. Washington, DC: Author.

Ness, M. (2007). Reading comprehension strategies in secondary content-area classrooms. Phi Delta Kappan, 89, 229–231. doi:10.1177/003172170708900314

16 E. SWANSON ET AL.

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016

Nussbaum, M. E. (2008). Collaborative discourse, argumentation, and learning: Preface and literature review. Contemporary Educational Psychology, 33, 345–359. doi:10.1016/j.cedpsych.2008.06.001

O’Brien, D. G., Moje, E. B., & Stewart, R. (2001). Exploring the context of secondary literacy: Literacy in people’s everyday school lives. In E. B. Moje & D. G. O’Brien (Eds.), Constructions of literacy: Studies of teaching and learning in and out of secondary classrooms (pp. 105–124). New York, NY: Routledge.

Paxton, R. J. (1999). A deafening silence: History textbooks and the students who read them. Review of Educational Research, 69, 315–339. doi:10.3102/00346543069003315

Snow, C. E., Porche, M. V., Tabors, P. O., & Harris, S. R. (2007). Is literacy enough? Pathways to academic success for adolescents. Baltimore, MD: Brookes.

Swanson, E. A., Wanzek, J., Haring, C., Ciullo, S. P., & McCulley, L. V. (2013). Intervention fidelity in special and general education research journals. Journal of Special Education, 47(1), 3–13. doi:10.1177/0022466911419516

Swanson, E., Wanzek, J., McCulley, L., Stillman-Spisak, S., Vaughn, S., Simmons, D., … Hairrell, A. (2015). Literacy and text reading in middle and high school social studies and English language arts classrooms. Reading & Writing Quarterly. Advance online publication. doi:10.1080/10573569.2014.910718

Swanson, E., Wanzek, J., Vaughn, S., Roberts, G, & Fall, A. (2015). Improving reading comprehension and social studies knowledge among middle school students with disabilities. Exceptional Children. Advance online publication. doi:10.1177/0014402914563704

Texas Education Agency (n.d.). STAAR technical digest 2013–2014. Austin, TX: Author. Van den Broek, P. (2010, April 23). Using texts in science education: Cognitive processes and knowledge

representation. Science, 328(5977), 453–456. Vaughn, S., Cirino, P. T., Wanzek, J., Wexler, J., Fletcher, J. M., Denton, C. D., … Francis, D. J. (2010). Response to

intervention for middle school students with reading difficulties: Effects of a primary and secondary intervention. School Psychology Review, 39(1), 3–21.

Vaughn, S., Martinez, L. R., Wanzek, J., Roberts, G., Swanson, E., & Fall, A.-M. (in press). Improving content knowledge and comprehension for English language learners: Findings from a randomized control trial. Journal of Educational Psychology.

Vaughn, S., Roberts, G., Klingner, J. K., Swanson, E., Boardman, A., Stillman, S. J., … Leroux, A. (2013). Collaborative strategic reading: Findings from experienced implementers. Journal of Research on Educational Effectiveness, 6, 137–163. doi:10.1080/19345747.2012.741661

Vaughn, S., Roberts, G., Swanson, E. A., Wanzek, J., Fall, A.-M., & Stillman-Spisak, S. J. (2015). Improving middle school students’ knowledge and comprehension in social studies: A replication. Educational Psychology Review, 27(1), 31–50. doi:10.1007/s10648-014-9274-2

Vaughn, S., Swanson, E. A., Roberts, G., Wanzek, J., Stillman-Spisak, S. J., Solis, M., & Simmons, D. (2013). Improving reading comprehension and social studies knowledge in middle school. Reading Research Quarterly, 48(1), 77–93. doi:10.1002/rrq.039

Wanzek, J., Swanson, E., Vaughn, S., Roberts, G., & Fall, A-M. (2015). English learner and non-English learner students with disabilities: Content acquisition and comprehension. Exceptional Children. Advance online publication. doi: 10.1177/0014402915619418

Wanzek, J., Vaughn S., Roberts G., & Fletcher J. M. (2011). Efficacy of a reading intervention for middle school students with reading disabilities. Exceptional Children, 78, 73–87.

Wanzek, J., Vaughn, S., Scammacca, N. K., Metz, K., Murray, C. S., Roberts, G., & Danielson, L. (2013). Extensive reading interventions for students with reading difficulties after Grade 3. Review of Educational Research, 83, 163–195. doi:10.3102/0034654313477212

What Works Clearinghouse. (2014). Procedures and standards handbook (Version 3.0). Retrieved from http://ies.ed.gov/ ncee/wwc/pdf/reference_resources/wwc_procedures_v3_0_standards_handbook.pdf

Youngbauer, V. (2015). A textual analysis of social studies textbooks: How Vygotskian pedagogy could change the way social studies teachers develop curriculum with textbooks. Retrieved from http://www.academia.edu/4320348/ A_Textual_Analysis_of_Social_Studies_Textbooks_How_Vygotskian_Pedagogy_Could_Change_the_Way_Social_ Studies_Teachers_Develop_Curriculum_with_Textbooks

MIDDLE SCHOOL READING COMPREHENSION AND CONTENT LEARNING 17

Dow

nloa

ded

by [U

nive

rsity

of T

exas

Lib

rarie

s] a

t 07:

10 2

5 M

ay 2

016