THE EFFECTIVENESS OF TITLE I TUTORING PROGRAMS IN

ELEMENTARY SCHOOLS

Eunetra Ellison-SimpsonDr. William Allan Kritsonis – Dissertation Chair

PVAMUJULY 2008

Proposal Committee Members

• Dr. William Kritsonis – Dissertation Chair

• Dr. Camille Gibson – Member• Dr. David Herrington – Member• Dr. Tyrone Tanner - Member

Chapter I

INTRODUCTION

Introduction

• Federally mandated public after-school tutoring is not always reaching the children it's intended to help, and when it does, it does not always help as much as it could (Toppo, 2008).

• Questions are being raised about the effectiveness of tutoring for underachieving students, the accountability of tutors, and academic return on this investment (Buczynski, 2008).

Introduction (con.)

• There's still a dearth of research evidence to show whether one of the federal measure's least-tested innovations--a provision that calls for underperforming schools to provide after-school tutoring--has an impact on student achievement (Evidence Thin on Student Gains, 2007).

Background of the Problem

• Not surprisingly, 40% of all urban districts are required to offer supplemental services (Ascher, 2006).

• Gewertz (2005) says that states must evaluate the effectiveness of free tutoring under the federal No Child Left Behind Act, but a lack of resources might force them to compromise on the rigor of those evaluations.

• By anecdotal accounts, most states are not far along in designing evaluations and many experts question their capacity to design and implement high-quality evaluations (Gewertz, 2005).

Statement of the Problem

• Saulny (2006) asserts that even for those students who are getting tutored, there has yet to be a scientific national study judging whether students in failing schools are receiving any academic benefit, and there is no consensus on how that progress should be judged.

Research QuestionsThe following research questions

will guide the study:

Research Question # 1

What is the relationship between tutoring effectiveness

and student achievement based on an elementary

school’s rating on the Characteristics of Effective Tutoring Scale and its TAKS Cumulative Met Standard in

reading?

Research Question #2

What do administrators report about the effectiveness of implementing Title I tutoring programs in elementary schools as indicated by their rating on the Characteristics of Effective Tutoring Scale?

Research Question #3

What do elementary school teachers report about the effectiveness of implementing Title I tutoring programs in elementary schools as indicated by their rating on the Characteristics of Effective Tutoring Scale?

Research Question #4

Is there a difference in the ratings of administrators and teachers on the Characteristics of Effective Tutoring Scale?

Null Hypotheses

• H1 - There is no relationship between tutoring effectiveness and student achievement based on an elementary school’s rating on the Characteristics of Effective Tutoring Scale and its TAKS Cumulative Met Standard in reading.

• H2- There is no difference in the ratings of administrators and teachers in their overall scores on the Characteristics of Effective Tutoring Scale.

Purpose of the Study

The purpose of the study will be to investigate the structure of tutoring programs currently employed by public schools. Faculty members will rate the effectiveness of the campus tutoring programs by completing the Characteristics of Effective Tutoring Scale. Descriptive data will be included in the study to indicate factors that may be associated with the implementation and evaluation of Title I tutoring and further analysis will determine if any significant relationship exists between the quality of a Title I campus tutoring program and its TAKS Cumulative Met Standard. The study will also investigate whether administrators and teachers differ in rating their campus tutoring program.

Significance of the Study

• Results of the study will provide an overview of the effectiveness of the tutoring methods that are utilized.

• Implications for school administrators extracted through the study may substantiate the current practices that are in place.

• On the other hand, the study may serve as an apparatus of transformational change in streamlining the content and structure of tutoring programs.

• Ensuring that compensatory education programs are effective will enable educators to advance a step further in the quest to ameliorate the achievement gap between at-risk students and students of mainstream America.

Conceptual FrameworkWhat is “tutoring effectiveness?”

According to Gordon, Morgan, Ponticell, and O’Malley (2004):• From what researchers now know, there seems to be at

least 10 components that support high-quality tutoring:• 1. More professionally prepared tutors consistently

produce significantly higher levels of student achievement than tutors with little or no special preparation.

• 2. Tutors need to use a diagnostic/developmental template to organize and implement each student's tutoring program.

• 3. The tutors must be able to track the session-to-session progress of each student in order to modify tutoring content and use student academic strengths to overcome weaknesses.

• 4. Principles of learning drawn from both cognitive and constructivist thinking seem to offer the strongest contemporary tutoring methods.

• 5. Tutors need to use continuous feedback to help students develop positive self-images as learners.

Conceptual FrameworkWhat is “tutoring effectiveness?”

• 6. Formal/informal assessment needs to be used throughout the tutoring process.

• 7. Mentoring/coaching students on learning how to learn through providing guidance on study habits, test taking, attention to school, and learning in general is a significant informal part of effective tutoring.

• 8. Mentoring/coaching each student's parents on sustaining the day-to-day learning process in the home after the tutoring ceases is an important role for effective tutors.

• 9. To facilitate the coaching of parents, it is desirable to conduct the tutoring in the student's own home outside of school hours. If this is not possible, a community center or library can be used, but an effort to provide mentoring to the parents should still be made.

• 10. Throughout the tutoring, tutors must collaborate closely with each student's classroom teacher. The final measure of the effectiveness of the tutoring is the short-term and long-term improvement of the student's day-to-day classroom achievement. Close tutor-teacher collaboration will help maximize effective tutoring.

Review of the Review of the LiteratureLiterature

The Evolution of Tutoring

• The tutor-student model is patterned after the relationship that existed between the student and sage during the age of Socrates (McDonald, 2004).

• Gordon (1990) contends that tutorial philosophy and methods were forgotten or quietly absorbed by tax-supported public schools.

The Achievement Gap

• While Americans are mixed on the urgent need to address the racial achievement gap, the school's responsibility for the gap, and the practice of disaggregation, the Bush administration and NCLB proponents have aggressively wielded the law's emphasis on achievement gaps as a political tool (Hess, 2006).

• English (2003) asserts that current methods in education have not solved the major sociopolitical and instructional problems of race or class, and they have not reduced the gap between the socioeconomic haves and have-nots.

Compensatory Education: Title I

• Title I, the best known of the Act’s six titles was included for the purpose of meeting the special educational needs of children of low-income families (Kritsonis, 2002).

• For decades, people have argued about whether Title I in itself can be praised for raising test scores among broad groups in society, or whether it should be condemned for not closing the gap between poor and rich children (Borman, Stringfield, & Slavin, 2001).

21st Century Educational Reform: No Child Left Behind

• As a result of No Child Left Behind (NCLB), substantial new federal spending is allocated for supplemental education services, which most often mean tutoring (Buczynski, 2008).

• Underlying supplemental services is the assumption that academic instruction provided outside the regular school day by public and private organizations will be able to do what schools cannot - raise the achievement of students in consistently poorly performing schools (Sunderman, 2006).

Shortcomings of Tutoring Programs

• In the 299 districts surveyed by the Center on Education Policy (CEP) in 2005-06, 20% of eligible students took advantage of supplemental services; however, extremely low percentages of eligible students enrolling in supplemental services have been reported in Houston (3%) and Philadelphia (5%) (Ascher, 2006).

Shortcomings of Tutoring Programs

Students could get better access to federally funded tutoring programs if lawmakers streamlined the sign-up process, gave states and districts money to monitor and evaluate those services, and took steps to make sure districts actually spent money for tutoring on tutoring (Borja, 2007).

Shortcomings of Tutoring Programs

In a study conducted by the U.S. Department of Education from 2003-2005, Anderson and LaGuarda (2005) found impediments regarding the state of free tutoring programs:

• Some providers were unable to describe any strategy for aligning their services with state standards.

• Student attendance at after school tutoring was a challenge.

• Provider communication with parents and teachers was seldom very effective.

Shortcomings of Tutoring Programs

• Similar findings were observed by Ascher (2006):Although NCLB mandates "highly qualified" teachers for every classroom during the school day, the law is silent about qualifications for tutors. Most tutors are certified teachers, but some are college graduates without teaching experience, and 7% are high school students. Some, but not all, providers prepare their tutors to work with their instructional programs - preparation ranges from four to 20 hours. Some, but not all, providers evaluate their tutors.

The Need for Title I Tutoring Program Evaluation

• The lack of tutorial program supervision by public or private regulatory agencies resulted in some tutors making unfair claims regarding academic improvement that unduly raised student and parent expectations (Gordon, 1990).

• As tutoring programs of various types receive attention as possible solutions to modern educational problems, it becomes necessary to evaluate them in terms of their benefits to the students involved (Von Harrison & Guymon, 1980).

• Few research studies, for example, include a control group of students, prohibiting clear conclusions on the effects of the tutoring; therefore, the long-term effects of reading interventions need to be investigated (Senesac & Silberglitt, 2008).

Investigating Tutoring Effectiveness

• The demand for proven results, extensive evaluations, and data-driven decision-making has moved the role of the superintendent from the sideline to the frontline of supporting student achievement (Peterson & Young, 2004).

• As great as the need is for similar quantitative studies from other districts, including those that follow students over more than one year, there is also need for program observations that facilitate an understanding of how supplemental services classrooms are over time and how children experience tutoring (Ascher, 2006).

• It is imperative that resources be allocated to design and implement sophisticated evaluations of tutoring efforts (Pearson, 2000).

• Tutoring programs must be evaluated rigorously and systematically in order to determine: which produce the strongest and most reliable effects on student learning, which produce negligible effects, and which produce no or even negative effects (Slavin & Calderaon, 2000).

Characteristics of Effective Tutoring Programs

• Effective programs require adequate training for tutors, whether these are college students, community volunteers, or other children; second, supervision of tutors is essential (Pearson, 2000).

Garner et al. (2002) provides a framework for effective tutoring:

• Organizational support is one of the most important aspects to a successful after-school tutoring program.

• Providing adequate space to conduct the tutoring program and appropriate materials such as books and writing supplies to implement the program, and providing a designated person or people to be in charge of implementing the program are key to program success.

• In all of the effective after-school programs reviewed, training of the tutors was a consistent factor.

• An after-school tutoring program needs to have appropriate materials to implement a high-quality program.

• Incentives are important to keep tutors and tutees involved in the program.

Summary of the Literature

• In conclusion, the literature suggests that tutors must be equipped with ample training, resources, and evaluative feedback in order to produce an effective impact on student achievement.

• Education reform measures promoting research-based programs should encompass tutoring so that this important form of education can become a more potent resource in improving student performance (Gordon et al., 2004).

MethodsMethods

Research Methods • Current conditions of tutoring programs will be

described and analyzed for relationships between tutoring effectiveness and academic achievement.

• For the purposes of establishing a relationship between variables, no logical causal ordering can be implied; student achievement is the criterion variable while level of tutoring effectiveness is the predictor variable.

• In addition, two independent variables, the rating scores indicating the perceptions of administrators and staff members will also be compared through a t-test.

Subjects of the Study

• Houston, Texas has a low enrollment of eligible students in Title I tutoring (3%) according to Ascher (2006).

• For this reason, it is the interest of the study to discover how teachers and administrators in the Houston area regard Title I tutoring programs based upon the Characteristics of Effective Tutoring.

Subjects of the Study

• The study will be conducted in Harris County, the most populous county in the state of Texas.

• Harris County includes 23 school districts; nonrandom, purposive and convenience sampling methods will be used to invite 2 school districts to participate.

• The two districts invited to participate in the study will be purposely selected based on the fact that a large number of their schools are considered Title I.

Subjects of the Study

• Schools will be invited through cluster sampling.

• Approximately 45 elementary schools reside in Districts I and II combined.

• Ten schools will be selected from the school districts (n=10) by placing all 45 elementary campus names in a hat to retrieve the desired sample size.

Subjects of the Study

• Once ten schools have been selected, teachers and administrators will be invited to participate in the study via email.

• Convenience sampling will be used in selecting administrators and teachers.

• At least two administrators and 20 teachers per campus is the desired sample size (n=220).

Instrumentation

• Thirty statements regarding program administration, program design, family involvement, and tutoring sessions are included in the survey.

• Together, these four components of the survey will be combined to elicit an overall score that will determine whether a school’s tutoring program is deemed as highly effective, efficient, emergent, or in need of improvement.

• Information about the reliability and validity of the Characteristics of Effective Tutoring Scale as a web survey will be addressed in a pilot study.

Instrumentation

• Cooperation in completing the survey is likely due to its brevity, readability of responses and the use of Likert-responses; furthermore, “if a survey request is perceived as interesting and easy, the likelihood of obtaining cooperation will increase” (Biemer & Lyberg, 2003, p. 107).

• Confidentiality of respondent’s identity and letters sent to participants to explain the significance of participating in the study may increase the response rate.

• In completing electronic surveys, “confidentialitymust be assured and guarantees must be provided thatinstallment of the communication package will not lead to virus attacks” (Biemer & Lyberg, 2003, p.201).

Instrumentation

• Additional benefits of web survey designs such as Survey Monkey include “controlled routing and embedded edits” that may “decrease measurement error and item nonresponse” (Biemer & Lyberg, 2003, p.201).

Pilot Study

• In its original form, Characteristics of Effective Tutoring is a checklist of 30 standards that effective tutoring programs may have (see Appendix C).

• Likert-type responses will be added to the checklist and will be converted into a web survey on a secure site using Survey Monkey in order to elicit responses from administrators and staff members regarding the tutoring practices of each campus.

• A pilot study will be conducted to ensure that converting the checklist by adding Likert-type responses yields a reliable and valid survey.

Pilot Study

• Through convenience sampling, five teachers and one administrator will be invited to complete the survey.

• Respondents’ answers as well as any comments regarding the flow, readability, and relevancy of the questions will be analyzed in the pilot study.

• Appropriateness of the scale will be evaluated for the purposes of the study.

• Attention to the layout of the questionnaire will also be addressed to ensure that the instrument is clear to respondents.

Procedures

• Respondents will be introduced to the Characteristics of Effective Tutoring Scale through a link provided in an email.

• All submissions will be completed online via the World Wide Web.

• Respondents will choose the location in which to complete the web survey.

• Identifying information will not be obtained. • Staff members and administrators will respond to

each item on the survey by indicating the extent to which each statement is representative of the campus.

Research Question#1

Hypothesis Criterion Variable Predictor Variable Statistical Measurement

What is the relationship

between tutoring

effectiveness and student achievement based on an elementary

school’s rating on the Characteristics of Effective

Tutoring Scale and its TAKS Cumulative

Met Standard in reading?

H1 - There is no

relationship between tutoring

effectiveness and student achievement based on an elementary

school’s rating on the Characteristics of Effective

Tutoring Scale and its TAKS Cumulative

Met Standard in reading.

Student achievement

Level of tutoring

effectiveness

Pearson’s r coefficient of correlation

CORRELATIONAL STATISTICS

• The first hypothesis, H1, involves correlational research. For each of the 10 campuses included in the study, the first variable, TAKS Cumulative Met Standard value, will be collected via AEIS reports.

• TAKS Cumulative Met Standard values for will be listed in a SPSS spreadsheet.

• Using the values emanated from the descriptive research portion of the study, the arithmetic mean of each school’s rating scale will be listed in an adjacent column in SPSS.

• The Pearson r correlation coefficient will be calculated to determine whether a significant relationship exists between a campus average score on the Characteristics of Effective Tutoring scale and its TAKS Cumulative Met Standard for TAKS reading.

Research Question#2

Statistical Measurement

What do administrators report about the effectiveness of

implementing Title I tutoring programs in elementary schools as indicated by their rating on the Characteristics of Effective

Tutoring Scale?

Descriptive statistical measures including a frequency polygon

and grouped frequency distribution will be used to

summarize the results of the survey.

Research Question#3

Statistical Measurement

What do elementary school teachers report about the

effectiveness of implementing Title I tutoring programs in

elementary schools as indicated by their rating on the

Characteristics of Effective Tutoring Scale?

Descriptive statistical measures including a

frequency polygon and grouped frequency distribution will be used to summarize the

results of the survey.

DESCRIPTIVE STATISTICS• To determine any emerging trends, a grouped frequency distribution

will be prepared for each of the 10 schools involved in the study. • Participants’ scale scores will be placed in rank order from high to

low. Visual representation of the data will be graphed in a frequency polygon in which the shape of the distribution of scores demonstrates the skewness of the data.

• Descriptive data will be useful in reporting whether respondents rate their schools on the higher end (effective) or the lower end (in need of improvement).

• Measures of central tendency will also be tabulated to summarize the data presented in the frequency distribution. The arithmetic mean will be calculated by adding up each of the respondent’s scores and dividing by the number of scores (n=220).

• For each school, the mean of all respondents’ scores will be calculated in order to determine the overall level of effectiveness of each school (highly effective, efficient, emergent, needs improvement).

• In addition, the mean of the scores from all 10 campuses will be calculated to ascertain the average level of effectiveness for Title I tutoring programs.

Research Question#4

Hypothesis Independent Variable

Independent Variable

Statistical Measurement

Is there a difference in the ratings of administrator

s and teachers on

the Characteristics of Effective

Tutoring Scale?

There is no difference in the ratings of administrator

s and teachers in their overall

scores on the Characteristics of Effective

Tutoring Scale.

Administrators’ scores on

the Characteristics of Effective

Tutoring Scale

Teachers’ scores on the Characteristics of Effective

Tutoring Scale

T-test of Independent

Means

INFERENTIAL STATISTICS

• The second hypothesis, H2, involves inferential research. Statistical procedures used in the data analysis of the research will enable one to draw conclusions regarding the impact of free tutoring.

• A t-test of independent means will be applied to determine whether there is a significant difference in the responses of administrators and staff member on the Characteristics of Effective Tutoring Scale.

• Rating scale scores of administrators and faculty members will be listed into two separate columns for analysis in SPSS.

• Using SPSS 15.0, a t-test will be calculated for the two sets of scores.

• To determine the statistical significance, the null hypothesis will be restated.

• An independent-means t-test will be applied at the standard alpha level of .05.

In Conclusion…

• Research has provided little evidence to guide policy makers and educators on the benefits of supplemental educational services, particularly in improving the education of low-income and some minority students (Sunderman, 2006).

• The study will fulfill the gap in the research to date by investigating whether administrators and teachers rate the tutoring programs implemented on their campuses as effective or in need of improvement.

• Whether the level of effectiveness of tutoring programs is related to student achievement will also be investigated.