dr. arthur l. petter, phd dissertation defense, dr. william allan kritsonis, dissertation committee...

Arthur L. Petterway 1

A Mixed Methods Analysis of the Impact of High Stakes Testing on

English Language Learners in Major Urban High Schools in Texas

A Dissertation Defenseby

Arthur L. Petterway

William Allan Kritsonis, PhDDissertation Committee Member

May 04, 2007


Committee Members

M. Paul Mehta, Ph.D.(Dissertation Chair)

William Allan Kritsonis, Ph.D.(Member)

Douglas Hermond, Ph.D.(Member)

David Herrington, Ph.D.(Member)

Camille Gibson, Ph.D.(Outside Member)

May 04, 2007


Dissertation Defense Format

I. Theoretical Framework

II. Purpose of the StudyIII. Quantitative

Research QuestionIV. Null HypothesesV. Methods: SubjectsVI. Methods:

InstrumentationVII. Methods:

Quantitative

VIII. Quantitative Findings

IX. Qualitative Research Question

X. Qualitative Pilot Study

XI. Qualitative Findings/Review of Literature

XII. Practical Recommendations

XIII. Recommendations for Further Study


Theoretical FrameworkA Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas

QUANTITATIVE

QUALITATIVE

% ELLs

Enrolled

In a

High School

Views/Opinions

Administrators

TeachersDistrict

Personnel

% All 10th Grade

Students

Passing

TAKS

ELA/Math

Impact of

Statewide

Testing

On

ELLs

Student

Performance

Explanatory Design

May 04, 2007


Purpose of the Study

The purpose of this study was to determine the impact of high-stakes testing on English Language Learners.

May 04, 2007


Quantitative Research Question

1. Is there a relationship between the percentage of English Language Learners enrolled in a school and the percentage of all students passing the 10th grade TAKS test in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006?

May 04, 2007


Null Hypotheses

H01: There is no statistically significant

relationship between the percentage of English Language Learners

enrolled in a school and the percentage of all students passing the 10th grade TAKS test in English Language Arts given in 2003, 2004, 2005, and 2006.

May 04, 2007


Null Hypotheses

H02: There is no statistically significant

relationship between the percentage

of English Language Learners enrolled in a school and the percentage of all students passing the 10th grade TAKS test in Mathematics given in 2003, 2004,

2005, and 2006.

May 04, 2007


Method

Subjects of the Study• Quantitative – 173 Urban High Schools in

Texas• Qualitative (N = 55)

– 6 principals– 9 assistant principals– 6 ESL district personnel– 15 ESL certified teachers – 19 Non-ESL certified teachers

May 04, 2007


Method

Instrumentation• Quantitative data were accessed and

retrieved from the TEA website regarding the major urban high schools in Texas.

• Qualitative data were derived from the on-line open-ended questionnaire, focus group, and individual interviews.

May 04, 2007


MethodQuantitative

• Descriptive Statistics

• Correlation Statistics– Pearson r Coefficient

• Simple Regression Analysis ŷ=a+b1x1

May 04, 2007


MethodQuantitative

• Predictor Variable – The percentage of English Language Learners enrolled in school.

• Outcome Variable – The percentage of all students passing the 10th grade TAKS test in English Language Arts and Mathematics.

May 04, 2007


Major Findings: Descriptive Statistics English Language Arts

Year

Percent of ELLs

Enrolled

in Schools

Percent of All Students Passing 10th Grade TAKS

2003 8.30 62.87

2004 8.29 68.28

2005 7.62 59.39

2006 6.94 78.05


Major Findings: Descriptive Statistics Mathematics

Year

Percent of ELLs

Enrolled

in Schools

Percent of All Students Passing 10th Grade TAKS

2003 8.30 61.85

2004 8.29 53.57

2005 7.62 47.68

2006 6.94 50.13


Major Findings: Correlation Pearson r

Coefficients

Year ELA Mathematics2003 -0.349** -0.293**

2004 -0.392** -0.351**2005 -0.297** -0.382**2006 -0.398** -0.356**

** Correlation is significant at the 0.05 level

May 04, 2007


Qualitative Research Question

• What are the anticipated and observed consequences of the statewide testing specifically, TAKS, on ELLs, ESL curriculum, and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?

May 04, 2007


Pilot StudyQualitative

• Developed an open-ended online questionnaire

• Two schools-Expert opinions:2 principals2 assistant principals1 district ESL personnel4 ESL certified teachers4 Non-ESL certified teachers

• Trustworthiness

May 04, 2007


The major question was explored using the following six probes:

1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this

statewide testing? (Or what has happened because of TAKS?)

3. What problems have occurred related to or because of TAKS?

4. What changes were caused by this statewide testing?

5. What are your recommendations to improve this statewide testing?

6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?

May 04, 2007


Major Findings-Probe: 1. Why is TAKS given as a statewide test? • TAKS is given as a tool to gauge

knowledge in the core areas. • TAKS is considered as a means to

determine the school’s status (Exemplary, etc.).

• TAKS is a means to assess the state curriculum or standards.

• This statewide test is mandated by law and is aligned with NCLB.

May 04, 2007


Major Findings-Review of Literature:1.Why is TAKS given as a statewide test?• Abrams & Madaus (2003)-Today’s

widespread implementation of standards-based reform and the federal government’s commitment to test-based accountability ensure that testing will remain a central issue in education for the foreseeable future. Test results can provide useful information about student progress toward meeting curricular standards.

May 04, 2007


Major Findings-Review of Literature:1.Why is TAKS given as a statewide test?

• Madaus & O’Dwyer (1999)- Current emphasis on testing as a tool of education reform continues a long tradition of using tests to change pedagogical priorities and practices.

May 04, 2007


Major Findings-Probe: 2. What are the intended consequences of this statewide testing?

• Statewide testing is intended to eventually result in ELLs performing as well as the rest of the students.

• ELLs can improve academically and eventually join the mainstream.

• TAKS is a requirement for graduation.

May 04, 2007


Major Findings-Review of Literature: 2. What are the intended consequences of this statewide testing?

• Abedi (2003)- In a positive light, valid assessment may provide diagnostic information that can inform instruction and classification.

May 04, 2007


Major Findings-Review of Literature: 2. What are the intended consequences of this statewide testing?

• Harlow & Jones (2003)- We also need to recognize that when test scores are linked to high-stakes consequences, they can weaken the learning experiences of students, transform teaching into test preparation, and taint the test itself so that it no longer measures what it was intended to measure.

May 04, 2007


Major Findings-Probe: 3. What problems have occurred related to or because of TAKS?

• Higher dropout rate

• Decrease in graduation rate for ELLs• Lower self-esteem of ELLs.

• Dismal or failing performance of ELLs

May 04, 2007


Major Findings-Review of Literature: 3. What problems have occurred related to or because of TAKS?

• Anderson (2004)- For English Language Learners, the additional requirements of an exit examination could increase dropout rates.

• Rothstein (2002)- Dropout rates are climbing throughout the United States and many researchers hold high-stakes testing at least partly to blame.

May 04, 2007


Major Findings-Review of Literature: 3. What problems have occurred related to or because of TAKS?

• Jacob (2001)- Some researchers found that dropout rates were 4 to 6 percent higher in schools with high school graduation examinations. Another study reported that students in the bottom quintile in states with high-stakes testing were 25% more likely to drop out of high school than were their peers in states without high-stakes testing.

May 04, 2007


Major Findings-Probe : 4. What changes were caused by this statewide testing?• Schools experienced the negative reality

that there is a high failure rate among ELLs.

• Low self-esteem because of low scores and tests caused frustration and exasperation on the part of ELLs.

• Pressure on the school and ELLs.

• Emphasis is placed on test performance.

• ELLs need more time to learn English

May 04, 2007


Major Findings-Review of Literature: 4. What changes were caused by this statewide testing?

• Anderson (2004)- Labeling schools can have an impact on teacher and student morale.

• Flores & Clark (2003)- Teachers have also reported that the high-stakes nature of some assessments can have a negative impact on student morale.

May 04, 2007


Major Findings-Review of Literature: 4. What changes were caused by this statewide testing?

• Hood (2003)- Although some teachers have reported that their English language learners can reach the high standards set for them, they may need more time than other students.

• Lane & Stone (2002)- Certainly, poor test scores or poorly explained assessment systems can result in decreased student motivation.

May 04, 2007


Major Findings-Probe : 5. What are your recommendations to improve this statewide testing?

• Deferment of the test, possibly a different but fair test.

• Better assistance from the school to provide for teacher collaboration.

• Modifications in teaching strategies.

• Improved and paced curriculum for ELLs.

May 04, 2007


Major Findings-Review of Literature: 5. What are your recommendations to improve this statewide testing?

• Anderson (2004)- Positive consequences that were identified included more teacher collaboration, changes in curriculum and instruction, better alignment between ESL and content area curricula and more focus on reading and writing. Negative consequences included student and teacher frustration, more teaching to the test occurring, and a narrowed curriculum.

May 04, 2007


Major Findings-Probe: 6. What needs to be done for the ELLs to improve their performance in general and specifically for this

statewide test?

• Specific interventions for ELLs.

• Quality instruction.

• Develop a more intensive English program.

• Support from home.

• Provide a more meaningful tutoring program in school.

May 04, 2007


Major Findings-Review of Literature: 6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?

• Flores and Clark (2003) found that teachers were not against accountability and viewed it as distinct from statewide testing, but also thought that an over-emphasis on testing resulted in unbalanced curriculum and inappropriate instructional decisions.

May 04, 2007


Major Findings-Review of Literature: 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test?

• Popham (2003)- In order for teachers to make specific changes to instruction, the assessments needs to be clear as to what skills are being assessed.

May 04, 2007


Practical Recommendations

• School leaders should consider that the performance of schools on high stakes testing is affected by size and the proportion of ELLs taking the test.

• School leaders should consider extended deferment of standardized tests administered in English to ELLs.

May 04, 2007


Practical Recommendations

• School leaders should consider taking a more active role with LPAC in the monitoring of ELLs in schools.

• School leaders should implement specific action plans to improve situation of ELLs.

May 04, 2007


Recommendations for Further Study

• A study could be conducted to explore additional supports needed to ensure ELLs to pass high stakes testing.

• A study could be conducted to identify what data are needed to make fair decisions about ELLs.

May 04, 2007



• A study could be conducted to determine reasons why ELLs scored lowest among student groups on TAKS.

• A study could be conducted to Explore different approaches on campuses in dealing with ELLs in terms of curriculum and instruction.

May 04, 2007



• A study could be conducted to compare the performance of ELLs vs. non-ELLs based on different objectives of TAKS.

• A study could be conducted to understand the impact of high stakes testing as viewed by parents and students.

May 04, 2007



• A study could be conducted to Explore different instruments to measure performance of ELLs.

• A study could be conducted to determine whether the provision of English Language instruction to parents of ELLS would have a significant effect on student achievement on high stakes testing.

May 04, 2007


A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in

Texas

QUANTITATIVE

QUALITATIVE

% ELLs

Enrolled

In a

High School

Views/Opinions

Administrators

TeachersDistrict

Personnel

% All 10th Grade

Students

Passing

TAKS

ELA/Math

Impact of

Statewide

Testing

On

ELLs

Student

Performance

Explanatory Design

May 04, 2007

dr. arthur l. petter, phd dissertation defense, dr. william allan kritsonis, dissertation committee...

Education