atpe 2010 teacher quality study

8/8/2019 ATPE 2010 Teacher Quality Study

1/62

St d on t D st b t on of

Teacher QualiTy n T x s S oo s


2/62

Study on the Distributiono

Teacher Quality inTexas Schools

P r e s e n t e d b y

The Association o Texas Pro essional Educators

Ed Fuller, Ph.D.University Council or Educational Administration

The University o Texas at Austin

all 2010


3/62

2 Study on the Distribution of Teacher Quality in Texas Schools a l l 2 0 1 0

ExEcutivE Summary 4M jo i i g 4Po ic r comm io 4

introduction 8Pu po o h s u 8docum i g h di i u io o t ch Qu i 8exp o i g h C io o Compo iM u o t ch Qu i 9e i hi g r io hip b w t chQu i s u achi v m 9

r vi w o li u 9t ch Qu i 9t ch Ch c i ic a oci wi hG i i s u achi v m 10ou a o t ch Qu i 11

t ch exp i c 11t ch C i c io s u 12t ch P p io P og m Qu i

C i c io sco 12t ch s i i 12

data and mEthodology 13Co uc i g s c i g tQI r i g 13

P t ch Qu i I ic 13n w t x t ch Qu i I x 14

M ho o og O 14M ho o og two 14

componEntS o tQi ratingS 14t ch exp i c 15t ch C i c io s u 16t ch P p io P og m C i c io ex mi io sco 16t ch r io emp o m 17

SamplE o SchoolS 18t 1a: num o schoo I c ui h i s mp 19t 1b: P c g o schoo I c ui h i s mp 19

indingS 19indingS SEction i: aSSociationS bEtwEEn tQi

ratingS and StudEnt achiEvEmEnt 19

indingS SEction ii: tQi and mEaSurES otEachEr Quality 20I p i g h tQI r i g 20

igu 1: s no m di i u io 21e m schoo 21

t 2: av g t ch Qu i c io M u ac tQI r i g G oup o e m schoo t 3: P c g o novic t ch tQI r i g2t 4: P c g o t ch emp o l th o h P iv y tQI

Mi schoot 5: av g t ch Qu i c io M uac o tQI r i g G oup (Mi schoo )

High schoot 6: av g t ch Qu i c io M uac o tQI r i g G oup (High schoo )

indingS SEction iii: diStribution o taKSachEivEmEnt by tQi ratingS 26e m schoo

t 7: av g taKs Z-sco tQI r i g

o e m schoo t 8a: P c g o Hi p ic eco omicdi v g s u Oc i o tQI r i g t 8b: P c g o Hi p ic bi i gus u i P omi Hi p ic schoo Oc i o tQI r i g t 8C: av g taKs Z-sco o P omiHi p ic e m schoo tQI r i g

Mi schoo t 9: av g taKs Z-sco tQI r i go Mi schoo

High schoo t 10: av g taKs Z-sco tQI r i g

o High schoo

indingS SEction iv: diStribution o tQiratingS by School accountability ratingand StudEnt dEmographicS 30e m schoo accou i i r i g

t 11: e m schoo tQI schooaccou i i r i g

eco omic di v g s u t 12: e m schoo tQI h P c g

eco omic di v g s u t 13: e m schoo tQI h P c geco omic di v g s u o s cr gio esC

Mi o i s u t 14: e m schoo tQI h P c go Mi o i s u t 15: e m schoo tQI h P c go Mi o i s u o s c r gio esC

contEntS


4/62

3a l l 2 0 1 0 Study on the Distribution of Teacher Quality in Texas Schools

t 16: P c g o novic t ch t ch emp o G h two o h l

iv ac mic y 33Mi schoo h di i u io o tQI r i g 33accou i i r i g 33

t 17: Mi schoo tQI r i g schooaccou i i r i g 33

igu 2: s c t ch Qu i c io o ac micU cc p r cog iz Mi schoo 34

eco omic di v g s u 34t 18: Mi schoo tQI h P c g o eco omic di v g s u 34

igu 3: P c g o t ch te a igOu -o - i b h P c g o eco omicdi v g s u i h schoo 35

Mi o i s u 35t 19: Mi schoo tQI h P c go Mi o i s u 35

High schoo h di i u io o tQI r i g 35accou i i r i g 35

t 20: High schoo tQI r i g schooaccou i i r i g 36

igu 4: P c g o t ch te wi hs c Ch c i ic o h low - High -P o mi g High schoo 36

eco omic di v g s u 36t 21: High schoo tQI r i g h P c g

o eco omic di v g s u 37Mi o i s u 37

t 22: High schoo tQI r i g h P c go Mi o i s u 37

te e Q i e re ESe e ce e a e 38

igu 5: tea r gio e uc io s vic C M p 38t 23: r gio e uc io s vic CH qu loc io 38

e m schoo 39t 24: e m schoo tQI r i g di c i tQI r i g w low- High-

Pov schoo r gio e uc io s vic C 39Mi schoo 40

t 25: Mi schoo tQI r i g di ci tQI r i g w low- High-Povschoo r gio e uc io s vic C 40

High schoo 41t 26: High schoo tQI r i g di c i tQIr i g w low- High- Pov schoo r gio e uc io s vic C 41

te e Q i e me a e S d s

t 27: tQI r i g o s co schoo schooPov o iv M jo M o a t 28: av g tQI s u d mog phic olow- Pov High- Pov Mi schoo is c M jo U di ic t 29: av g tQI s u d mog phic

o low- Pov High- Pov High schooi s c M jo U di ic

concluSionS 45t 30: High schoo tQI r i g h num o tim r ac mic U cc p (2003-04

h ough 2008-09)

diScuSSion 46

policy rEcommEndationS 46s -l v Po ic di ic -l v Po ic r comm io

appEndix a 51O i l squ r g io r u

t a-1: s iz Co fci s i icsig i c c o O i l squ r g io 51

e m o h tQI r i g schoo l v 51e m schoo

t a-2 : Compo o e m schoo tQI r i g t a-2 : P opo io o s u achi v m V i cexp i e m schoo tQI r i g 53

Mi schoo t a-3 : Compo o Mi schoo tQI r i g 53t a-3 : P opo io o s u achi v mV i c exp i Mi schoo tQI r i g 54

High schoo t a-4 : Compo o High schoo tQI r i g 55t a-4 : P opo io o s u achi v mV i c exp i High schoo tQI r i g 55

appEndix b 56

t b-1: di i u io o tQI r i g oe m schoo t b-2: di i u io o tQI r i g o Mi schoo 57t b-3: di i u io o tQI r i g o High schoo 58

rE ErEncES


5/62


ExEcutivE Summary Improving teacher quality has long been a ocus o policymakers. In recent years, most re orm e orts include strategi

to improve the quality o teachers overall and/or create a more equitable distribution o teachers within schools, districts,and states. Indeed, the ederal No Child Le t Behind Act o 2001 mandated that all states assess the distribution o teach

quality, develop and implement plans to improve overall teacher quality and create a more equitable distribution o teacherquality. Yet, as noted by The Education Trust (2006), ew states have taken this mandate seriously, with most states not eveassessing the distribution o teacher quality, much less implementing plans to address the issues associated with teacher qual

This study examined the distribution o teacher quality in Texas public schools by creating several Teacher Quality Indi-ces based on di erent measures o teacher quality. Each Teacher Quality Index (TQI) provides a single measure o teachquality based on a larger number o teacher quality measures such as teacher experience, certi cation status, academic abity, or even stability at a school. Although there is ongoing debate about whether objective measures o teacher quality (suas experience, certi cation status, quality o the preparation, and teacher stability) accurately identi y teacher e ectivenin improving student achievement, recent research has ound statistically signi cant relationships between these objectivemeasures o quality and gains in student achievement. Importantly, this study ocuses on those measures ound in previouresearch to be associated with improvements in student achievement and how these measures are distributed across Texaspublic schools.

m j sThe major results o this study are as ollows:

Even after controlling for prior achievement, student demographics, and geographic location, teacher quality at theschool level is associated with student achievementespecially at the secondary level.

At the elementary-school level, teacher quality appears to be more equitably distributed than at the secondary-schoollevel, but this is more likely a result o the combination o a lack o detailed data and the greater supply o elementateachers than the result o any state or district policies to equalize teacher quality across schools.

Students in lower-performing schools have substantially less access to teacher quality than students in higher-perform-ing schools.

At all school levels, but particularly at the middle- and high-school levels, students in high-poverty and predominantlyminority schools have ar less access to teacher quality than students in low-poverty and predominantly White school

More generally, the results o this study unambiguously reveal a substantial inequitable distribution o teacher qualityacross the state at the middle- and high-school level. Clearly, students in low-per orming schools as well as in high-povertand predominantly minority schools have ar less access to the same levels o teacher quality as students in high-per ormlow-poverty, and predominantly White schools. Moreover, this nding holds true or schools that are within driving dis-tance o one anotherboth within the same district and across district lines between contiguous districts.

p re e sThe di erences in teacher quality across schools and districts in Texas are substantial. I Texas is going to increase ove

achievement and prepare a greater percentage o students to graduate rom high school and be well-prepared or li e a thigh school, we must address the inequitable distribution o teacher quality. To do so, Texas policymakers and educatorsshould explore a number of recommendations. State policy recommendations are described below as well as at the end of the report. In addition, district-level policy recommendations are included a ter the state-policy recommendations at theend o the report.

The following recommendations are targeted to state policymakers in the Governors Of ce, Legislature, State Board ofEducation, State Board for Educator Certi cation, the Texas Education Agency, and the Texas Higher Education Coordi -nating Board.


6/62


Fund and support the gathering o input rom teachers and administrators on improving teacher quality and moreequitably distributing teacher quality.

The state should hire a group o education experts (some o whom should have experience as teachers, principals, andcentral o ce administrators in Texas public schools) to travel the state and convene groups o educators in order to gathertheir input on how to best improve teacher quality and lessen the inequitable distribution o teacher quality. The resultso this e ort should drive state policy. An excellent model or such an e ort is the Teacher Leaders Network organized byDr. Barnett Berry o the Center or Teaching Quality (http://www.teacherleaders.org /). This e ort brings together teacherleaders rom across the nation in a virtual network to share best practices, provide support, and push or policy changes thatsupport teachers.

Create an annual statewide report that analyzes the aggregate TQI and the individual TQI components, and providethe overall results to the public and the individual school reports to district personnel.

Un ortunately, the state has not highlighted the distribution o teacher quality and the trends in teacher quality over time.Unless the state publicly raises the issue, the issue will remain low on the priority list o state and district policymakers.One important step that state leaders could take is to start a conversation with district administrators about the di erencebetween highly quali ed teachers and teacher quality.

According to TEA (2010), a highly quali ed teacher is a teacher that meets the ollowing requirements:

Has obtained full Texas teacher certi cation, including appropriate special education certi cation for specialeducation teachers;

Holds a minimum of a bachelors degree; and

Has demonstrated subject matter competency in each of the academic subjects in which the teacher teaches.

In this de nition, ull state certi cation does not mean that a teacher is ully certi ed in the sense that the teacher holdsa standard certi cate but rather any type of certi cate other than a permit that is granted by the State Board for Educa-tor Certi cation. The analysis in this study relied on a ull standard certi cate. In act, when serving as the Co-Directorsof Research at the State Board for Educator Certi cation, Alexander and Fuller (2004) found that Texas middle schoolmathematics teachers who had obtained a ull standard certi cate in mathematics were more e ective at increasing studentachievement than teachers with other ull state certi cates.

Further, this de nition requires teachers to demonstrate subject-matter competency through either a major in the eld o study or a passing score on a Texas state certi cation examination or a particular content area. This leaves open the pos-sibility that a person could demonstrate competency by correctly answering 70% o items correct on a certi cation examina-tion that arguably could be passed by an honors Algebra II student in a high-per orming high school in Texas. In addition,alternative certi cation teachers are only required by the state to have 12 hours o undergraduate coursework in a contentarea to be considered highly quali ed, and the counting o these hours is le t to the discretion o those working in alternativecerti cation programs.

Because these requirements are so lax, almost every teacher in Texas (and the nation) is considered highly quali ed,but this measure is not based on any empirical evidence related to student achievement. Un ortunately, district leaders rarelyacknowledge that they have a problem with their distribution o teacher quality because all schools have nearly 100% highlyquali ed teachers. The state should impress upon district leaders the need to ocus on the measures in this report ratherthan highly quali ed teacher status.

Provide monetary incentives or districts to address TQI inequities across their respective schools, and increase theexibility districts have in addressing their unique needs. This e ort holds great promise i districts design and implement programs in thought ul ways that address the issues

brought orth by this study. TEA should provide incentives or a district to speci cally address inequities in teacher qualityacross its schools. Further, TEA should allow districts greater fexibility in the spending o money to improve the distribu-tion o teacher quality across campuses.


7/62


This could be accomplished by restructuring the District Awards or Teacher Excellence (DATE) grant incentives orcreating an alternative program that exists alongside the DATE grant program. As currently con gured, the DATE programcould allow districts to create an incentive program that actually exacerbates rather than ameliorates inequities in the distrbution o teacher quality.

Adopt and und a new cost-o education index.

The cost-o -education index (CEI) was created in the early 1990s to provide unding to districts that had di culty inhiring well-quali ed teachers due to actors outside the control o district leaders, such as the percentage o economicalldisadvantaged students. The CEI has never been updated despite repeated e orts by some policymakers and researchers do so. Thus, some districts receive ar more money than they should while other districts do not receive nearly the amountthey deserve based on an updated CEI. The primary hurdle to updating the CEI is cost. The state should construct a newschool nance system with an updated CEI and implement the changes over time so that no district loses a substantialamount o money in a short period o time.

Support the creation o urban teacher academies in the 10 largest metropolitan areas across the state. Urban teacher academies provide opportunities or newly certi ed teachers to learn how to be e ective teachers o lo

per orming and economically disadvantaged students under the guidance o master teachers. These programs have beenshown to be quite e ective in building on the training provided by high-quality teacher preparation programs. For morein ormation, seehttp://www.ncate.org/documents/news/UTR_IHE_Aug122008.pd .

Support the creation o urban leader academies in the 10 largest metropolitan areas across the state. Urban leader academies would be similar to urban teacher academies but ocus on preparing newly certi ed principals

be e ective leaders attuned to increasing and equalizing teacher quality within schools. The University o Texas at Austithrough the University o Texas Collaborative Urban Leadership Program (UTCULP), has initiated a pre-service programto accomplish this goal, but the e ort is limited to three districts and ocuses only on pre-service components o trainin

Create an incentive program or preparation programs to produce teachers that meet the demand or teachers in theirlocal labor market.

Currently, there is no incentive or programs to produce a high school mathematics teacher as opposed to an elementaryteacher, even though there is a shortage o mathematics teachers and a surplus o elementary teachers.

Increase the requirements to enter teacher preparation programs in Texas, especially alternative certifcation pro-grams that tend to have lower entrance requirements than traditional university-based or post-baccalaureate programs.

Although recent additions to the accountability system or educator preparation have dramatically improved the measurused to identi y e ective preparation programs, there is still room or improvement. For example, the entrance requirem

or many alternative certi cation programs are still abysmally low. Further, some individuals can enter and complete analternative certi cation and become employed as a middle or high school teacher with as little as 12 undergraduate credithours in the subject area in which they obtained certi cation. Yet, individuals rom traditional certi cation programs mustcomplete a major in the subject area in which they obtain certi cation. Perhaps the state should require a minimum of 24hours and allow programs to decide on additional content requirements. Finally, the state should require a closely super-vised eld experience or all teachers, even those rom alternative certi cation programs.

Fund a statewide working conditions study, and encourage all schools to participate in the study.Recent research has ound that teacher working conditions have a signi cant impact on teacher e ectiveness as measu

by gains in student achievement and are the primary actor in improving teacher retention. Without addressing the issue oteacher working conditions, all other e orts to increase the degree o equity in the distribution o teachers will surely aThus, the state needs to und a high-quality working conditions survey rom an organization that can ensure valid and reliable results or schools, districts, and the state. The data should be provided back to schools and districts and training beprovided on the use o such tools to improve working conditions and the equitable distribution o teacher quality.

Improve the training o school and district leaders. Research on the relationship between working conditions and teacher turnover has consistently ound that school leade

ship behaviors are the primary actor a ecting teachers decisions to stay at or leave a particular school. These behavior


8/62


the underlying orce behind the power ul e ect that working conditions have on teacher retention. In act, leadership behav-ior is a stronger predictor o teacher retention than either student demographics or student achievement. Better training o school leadersconcomitant with other changescan increase the likelihood that more schools will have the type o schoolleader that attracts well-quali ed and e ective teachers regardless o the school characteristics.

Because district personnel have a great deal o infuence on the hiring and distribution o teachers, superintendent pro-grams should be required to teach prospective superintendents about the distribution o teacher quality and strategies toequalize teacher quality.

Improve data collection and dissemination e orts. Although Texas used to be recognized or having one o the best education data systems in the country, Texas has allen

behind a number o states. The state has invested additional money and e ort into upgrading data systems, includingmatching students to teachers in every school in the state, yet TEA has not addressed some substantial issues regardingexisting and missing data related to teacher quality. The state should bring researchers and data system experts together toidenti y weaknesses and potential solutions or improving the current data on educators. Researchers need better data andgreater access to data on the background characteristics o all teachers employed, such as undergraduate institution, grade point av-erage, SAT/ACT scores, certi cation scores, type of masters degree, and the major/minor for the undergraduate degree. Althoughthe state should acilitate the use o such data by researchers, the state also needs to ensure that such data is not available to thegeneral public in order to protect the con dentiality o individual teachers. In sum, the data should be made available to research-ers to guide policymakers but still be protected to ensure its con dentiality and the privacy o individual teachers.

Further, the state needs to invest in creating more accurate data on teacher experience and certi cation status. Some of this data is currently available through the Education Research Centers at the University o Texas at Austin, the Universityof Texas at Dallas, and Texas A&M University. However, the state could create more ef cient and effective ways to make thisdata accessible without violating the con dentiality o individuals.

Provide school-level value-added data. The general consensus o researchers on the accuracy o teacher-level value-added e orts is that such systems are gener-

ally not accurate and stable enough for use when making high-stakes decisions about teachers. However, the state shouldprovide use ul school-level value-added in ormation or each grade level and subject area or which in ormation is availableso that school and district personnel can identi y areas o strengths and weaknesses. Moreover, the state should modelappropriate use o this data and provide training on correct interpretation and appropriate use o such in ormation. I thestate does begin to provide such data, an oversight team o researchers and practitioners should be created to ensure that thedata is used and denominated in ways that meet the best practices as established by experts in the eld.

Improve the school accountability system. Currently, the states school accountability system provides a disincentive or well-quali ed and e ective teachers to move to

low-per orming schools. The primary driver o this disincentive is the absence o an accurate barometer and recognition o stu-dent growth. Teachers are o ten ear ul o seeking employment in low-per orming schools because o the increased scrutiny, theadded pressure, and the belie that they will be punished or low levels o achievement even i their students make large gains.

Although the state has implemented Required Improvement and the Texas Projection Measure in an e ort to rewardstudent growth, both measures have serious methodological faws and are clearly in erior to having an actual measure o student growth as one component o the school accountability system.

Develop a statewide campaign designed to increase the prestige o the teaching pro ession.Currently, many prospective teachers do not view the teaching pro ession as a prestigious one. A statewide campaign

coupled with more stringent entrance requirements or preparation programs that increase the overall quality o teacherscan increase the prestige o the pro ession and increase the supply o better-quali ed entrants into the pro ession.

Importantly, no one strategy will be su cient in ameliorating the inequitable distribution. Indeed, a multi-pronged, multi-year strategy at the state, district, and school levels is necessary to ensure that all students have access to a well-quali ed teacher.


9/62


introductionCurrently, there is widespread consensus among researchers and policymakers that teacher quality is the most important

school actor1 a ecting student achievement. Indeed, nearly every major policy report on education highlights the impactteacher quality has on student achievement and calls or providing a high-quality teacher or every student. Concomitant

this research have been examinations o the distribution o teacher quality across schools. Although the rst line o reseahas shown that teacher quality matters tremendously to student achievement, the second line o research has consistentlyound that children in lower-per orming schools and schools serving high proportions o poor and minority students have

lower teacher quality than higher-per orming schools and schools serving low proportions o poor and minority students(Peske & Haycock, 2006). This is not to say that there are not highly effective and well-quali ed teachers in every school,but rather to say that the proportion o well-quali ed teachers in the lower-per orming, high-poverty, and predominantlyminority schools is lower than in higher-per orming, low-poverty, and predominantly White schools. Because o this ineqtable distribution o teacher quality, the ederal No Child Le t Behind Act included a provision that required states to assthe inequity in certain teacher quali cations and implement plans to close the teacher quality gap (The Education Trust, 2006

Un ortunately, as noted by The Education Trust (2006), ew states have even conducted an analysis o distribution o teacher quality, and almost none have adopted and implemented serious plans to address the issue.2 According to the re-port, Texas reported the percentage o highly quali ed teachers in classrooms and the percentage o highly quali ed teachers in high- and low-poverty schools as well as in high- and low-minority schools. Yet, despite possessing the data to makesuch reports available, the Texas Education Agency (TEA) ailed to report on the percentage o inexperienced teachersacross the state and in high- and low-poverty schools as well as in high- and low-minority schools. Indeed, TEA has still ncomplied with this mandate (seehttp://www.tea.state.tx.us/index4.aspx?id=4662&menu_id=798).

Yet simply ocusing on the percentage o classes taught by highly quali ed teachers misses the mark (The EducationTrust, 2006, p. 2). Indeed, the authors note (2006, p.3) that this narrow ocus ignores the distribution o inexperiencedteachers and that It is possible or every teacher in a school to be highly quali ed and still have classes taught by out-o

eld teachers. Indeed, relying on highly quali ed teacher status as a measure o the distribution o teacher quality greatlyobscures the reality o the extent to which students in high-poverty, high-minority, and/or low-per orming schools havedramatically less access to well-quali ed teachers (Fuller & Carpenter, 2009, 2008; Fuller, 2009, 2008, 2007, 2003, 2002;The Education Trust, 2006).

p se e SThere are three purposes o this study. The rst purpose is to document the distribution o teacher quality across school

using more than just the percentage o highly quali ed teachers and classrooms. The second purpose is to explore di erestrategies to create a composite measure o teacher quality. The third purpose is to establish a statistically signi cant rela-tionship between composite measures o teacher quality and student achievement. Without establishing such a statisticallysigni cant relationship, the rest o the study is largely irrelevant.

Documenting the Distribution o Teacher QualityWith respect to documenting the distribution o teacher quality in Texas, Fuller and his colleagues (Fuller & Alexander,

2003; Fuller & Carpenter, 2009, 2008; Fuller & Brewer, 2005; Fuller, 2009, 2008, 2007, 2003, 2002) have repeatedly

documented the inequitable distribution o teachers in Texas in a number o studies that relied on various measures o teacher quality. However, these studies described how individual measures of teacher quality varied across schools.

1Teacher quality clearly exerts an extremely strong in uence on student achievement and must be addressed in order to improve overall student achievement and closetant to note, however, that the combined efect o actors outside o the school, such as parental level o education, amily income, and the number o books in a hoon student achievement than any school actors. Many policymakers and education writers now claim that teacher quality is the single most important actor in uencing does not validate such a claim. In terms o education policy, teacher quality is certainly where eforts need to be ocused, but eforts that address issues o poverty should 2This is consistent with the trend during the past decades to largely ignore the inputs o education, such as per-pupil expenditures and teacher quality, and ocus solelachievement and graduation rates. Recent research would suggest that we need to attend to both inputs and outputs to create a well- unctioning system.


10/62


A more concise manner or documenting the distribution o teacher quality is to create a Teacher Quality Index, or TQI. ATQI is a composite indicator o teacher quality at a school or district that is created by combining a number o individualteacher quality measures in some manner. Fuller (2008a) created a TQI or Texas schools in 2008 that documented theinequitable distribution o teachers across schools. Yet that study had two drawbacks. First, the TQI was created using a

airly simplistic methodology. Following the lead o Futernick (n.d.) in his creation o a TQI in Cali ornia, Fuller (2008a)simply ranked schools on individual measures of teacher quality and then averaged those rankings. Second, the study didnot include a statistical analysis o the association between the TQI and student achievement.

Researchers rom the Illinois Education Research Council (DeAngelis, Presley, & White, 2005; Pressley, White, &Gong, 2005) employed a more sophisticated methodology called principal components analysis (PCA). Yet, their studyhad shortcomings as well. The authors assumed that certain measures identi ed in the literature as being associated withstudent achievement were included in the TQI without assessing whether each measure was, in act, actually associatedwith student achievement in Illinois. Rather, the authors used PCA to weight each teacher-quality measure and then usedthe resulting weighted teacher-quality measure to create a TQI. Using regression analysis, the authors did nd that the TQIcreated through PCA was statistically signi cantly associated with student achievement.

Exploring the Creation o a Composite Measure o Teacher Quality

Because there was no consensus on how to construct a TQI, this study explored multiple strategies to create a TQI andreported on three di erent TQIsone based on regression results (Regress TQI), one based on an average o teacher-quality measures but excluding measures o teacher stability (Avg1 TQI), and one based on an average o teacher-qualitymeasures and including measures o teacher stability (Avg 2 TQI). Each TQI was a single numeric indicator o a number o di erent teacher-quality measures. The three di erent TQIs will be described in subsequent sections o this study.

Establishing a Relationship Between Teacher Quality and Student Achievement

Finally, in order to establish a statistically signi cant relationship between the di erent TQIs and student achievement, Iused ordinary least-squares regression analysis to examine whether each TQI was statistically signi cantly related to studentachievement a ter controlling or prior achievement, percentage o economically disadvantaged students, percentage o African American students, percentage of bilingual/English as a Second Language students, percentage of mobile students,school size, and school geographic location. The results o this analysis are discussed in Appendix A.

The remainder o this report reviews the literature on teacher quality and its relationship to student achievement, pro-

vides a description o the teacher quality measures and construction o three TQI ratings, and then examines the TQIratings across Texas public schools. A ter providing summary conclusions, the report pro ers some policy recommenda-tions or states and districts to address the inequitable distribution o teacher quality in Texas.

re e l e eTeacher Quality

Over the past decades, researchers have examined the relationship between certain observable characteristics o teach-ers (such as years o experience, certi cation status, teachers own test scores, etc.) and changes in student achievement.Traditionally, the examinations o teacher characteristics have ocused on quali cations such as the undergraduate collegeattended, certi cation status, advanced degrees, and years o teaching experience (Fabiano, 1999; Rowan, Chiang, & Miller,1997). With the advent o certi cation tests or teachers in the 1990s, researchers have also examined the relationship

between teacher test scores and student achievement (Rice, 2003). Most recently, researchers have examined the impact o teacher instability and turnover on student achievement (Ingersoll & May, 2010).

Rather than ocusing on the characteristics and quali cations o teachers, economists have turned to assessing teacherquality by directly assessing the achievement gains o the students taught by the teacher. Indeed, in just the past ve years,

3PCA is a statistical procedure that examines a relatively large number o measures related to one another and that are, perhaps, even redundantsuch as teacher quality measures. PCAtempts to identi y a smaller number o variables than the original number o variables that still explain the underlying conceptin this case, teacher quality. PCA can be used to create

or teacher quality based on how well each individual measure o teacher quality explains the overall concept o teacher quality.


11/62


a number o researchers have measured teacher quality by associating student value-added achievement gains with speciteachers over time and assigning teacher-quality ratings as a unction o student outcomes (c. ., Goldhaber et al., 2007;Hanushek, Kain, OBrien, Rivken, 2005; Sanders & Rivers, 1996). Such analyses, however, are problematic for a number oreasons. First, as Sass (2008) concluded, researchers simply have not developed a value-added system that accurately iden-ti es e ective and ine ective teachers despite large investments o time, money, and e ort. Most problematic with cuvalue-added assessments is the relatively high instability o teacher e ectiveness across years. A number o teachers, inmove rom being e ective in raising student scores to being ine ective in raising student scores and vice versa over jusyear of time. Similarly, a recent Institute of Education Sciences report found that value-added systems had relatively largeerror rates that led to a substantial percentage of teachers being incorrectly identi ed as high- or low-performing (SchocheP.Z. & Hanley, 2010). In fact, the researchers found there was about a 25% chance, if using three years of data, or a 35%chance, i using one year o data, that a teacher who was average would be identi ed as signi cantly worse than avera

Second, such analyses are rather dif cult to accomplish given current limitations of school and district information man-agement systems, which o ten do not integrate student achievement and personnel data. Third, recent evidence has oundthat a teachers peers in the school and the school context signi cantly a ect a teachers value-added assessment. In othewords, where you teach and with whom you teach impacts your e ectiveness as a teacher ( Jackson & Bruegmann, 2009)Finally, even i such assessments were accurate, they provide no use ul in ormation to districts when hiring beginning teachers othe roughly 70% o teachers who teach in subject areas and grade levels without any ormal assessment o student achievemen

Most research on teacher quality alls somewhere between the direct assessment o student achievement gains as aunction o teacher quality and more subjective observational data o teacher qualities such as caring about students and

working hard. Indeed, large-scale education policy studies tend to ocus on the relationships between easily classi ableteacher quali cations and student outcomes. In addition to the criteria o college degree, certi cation status, experience, astudent achievement, researchers have considered content area and educational preparation, teachers own test scores, andthe quality o undergraduate and graduate institutions attended (Angrist & Guryan, 2003; Coleman, 1966; Ehrenberg &Brewer, 1995; Figlio, 2002; Kersting, 2008; Lankford, Loeb, & Wyckoff, 2002).

Teacher Characteristics Associated with Gains in Student AchievementResearchers have consistently ound that particular characteristics o teachers are positively associated with student ga

in achievement. The two most consistent ndings are (a) teachers own prior test scores, especially those related to verbalability (Coleman, 1966; Ehrenberg & Brewer, 1995), and (b) the selectivity or competitiveness o the undergraduate institution attended by the teacher (Angrist & Guryan, 2003; Ehrenberg & Brewer, 1994; Figlio, 2002; Lankford et al., 2002; Re-back, 2002; Rice, 2003). Indeed, one might assume some overlap between these two characteristicsteachers rom selectinstitutions tend to do well on tests o verbal ability.

A third characteristic associated with student achievement is a teachers years o experience in education. Despite somedisagreement about the relationship between teacher experience and student achievement, there is a growing consensus thanovice teachers (those with three or ewer years o experience) and especially beginning teachers (those in their rst yearteaching) are less effective than more experienced teachers in increasing student achievement (Ehrenberg & Brewer, 1994;Ferguson, 1991; Boyd, Grossman, Hamilton, & Wyckoff, 2007; Goldhaber & Brewer, 1996; Murnane & Phillips, 1981;Rice, 2003; Ladd, Clotfelter, & Vigdor, 2007, 2010). Similarly, studies on teacher subject-matter knowledge indicate thatwhile there is a positive relationship, the relationship is not as strong as one might suspect (Wilson, Floden & Ferrini-Mun-dy, 2001); it is most important in mathematics and science (Goldhaber & Brewer, 2000; Rowan et al., 1997; Wenglinsky,

2002; Ladd, Clot elter, & Vigdor, 2007, 2010), and the e ect is more pronounced or upper than lower grades (Rice, 200Ladd, Clot elter, & Vigdor, 2007, 2010; Wayne & Youngs, 2009).

A th teacher characteristic related to student achievement is certi cation in the area in which one teaches. A surpris-ingly small amount of research has examined this relationship (Darling-Hammond, Berry, & Thoreson, 2001), althoughmore recent research rom North Carolina has addressed this issue (see Ladd, Clot elter, & Vigdor, 2007, 2010). A hand uo studies have ound that students taught by certi ed teachers have greater levels and gains in achievement than their petaught by uncerti ed teachers (typically those who have not yet obtained ull state certi cation), particularly in mathematscience, and reading (Alexander & Fuller, 2004; Boyd, et al., 2007; Laczko-Kerr & Berliner, 2002; Ladd, Clotfelter, & Vig-


12/62


dor, 2007, 2010). Surprisingly, there are even fewer studies on the relationship between teachers being assigned in- eld andstudent achievement gains. The available research has suggested that in- eld status is more closely associated with studentachievement gains than certi cation status (Wilson, 2008; Ladd, Clot elter, & Vigdor, 2007, 2010; Wayne & Youngs, 2009).For example, Alexander and Fuller (2004) found that Texas middle school teachers assigned in- eld evidenced greatergains in student achievement than teachers assigned out-o - eld. Moreover, the large e ect size o their ndings suggestedthat in- eld status has a airly substantial e ect on student achievement.

Although not a teacher characteristic, recent research has also ound that school-level employment stability o teachersis related to gains in student achievement (Ingersoll & May, 2010). This area o research is somewhat controversial, as thee ect o turnover on student achievement depends not just on the absolute level o turnover but also on the quali cationso those who leave the schools and o those who remain. I the least quali edand least e ectiveare those who leave theschool, then turnover could actually improve student achievement. However, if those who leave are the most quali ed andmost e ective, then turnover could certainly negatively impact student achievement. In their review o the business and edu-cation literature, Ingersoll and May (2010) ound that some degree o turnover is pre erable or high per ormance, but toomuch turnover has a negative impact on the per ormance o those remaining in the organization. Recently, several papershave emerged that examine the relationship between teacher turnover and student achievement (Levy, Ellis, Joy, Jablonski,& Karelitz, 2010; Meier & Hicklin 2007; Keesler 2010). In general, research in this area nds that teacher turnover tends tohave a negative e ect on student achievement, especially in schools where turnover is consistently high.

Not all research, however, nds that every one o the above characteristics is related to improvement in student achieve-ment. For example, Buddin and Zamarro (2010) ound that none o the observable characteristics o teachers (experience,certi cation status, or certi cation test scores) were associated with gains in student achievement. Some, in fact, argue thatmeasures o teacher quality such as certi cation status and licensure scores are so ine ective at predicting per ormance thatall barriers to becoming a teacher should be removed and teachers should be hired and red solely on their e ectiveness inthe classroom (c.f., Walsh, 2001; Rockoff, 2004; Kane, Rockoff, & Staiger, 2006).

However, even though the research is somewhat mixed, the preponderance of the available evidence suggests that mea-sures o teacher quality are, in act, associated with gains in student achievement. This is especially true at the secondarylevels in the areas o mathematics and science.

a e s te e QIdeally, this study o teacher quality would examine the quali cations o teachers,accurate value-added assessments o

teachers, and principals assessments o teachers. Yet, there is no value-added assessment o teachers in Texas, except in aew selected districts. Even i such data were available, the general consensus o researchers is that there is no value-added

methodology currently available that provides accurate and stable estimates of teacher effectiveness (Sass, 2008). Further,the current assessment o teachers by principals is con dential in ormation and should remain con dential because thein ormation is related to employment decisions. The availability o summary in ormation about the quality and e ective-ness o teachers as perceived by principalsin ormation that did not violate the privacy and con dentiality o individualteacherswould certainly in orm this study. Because the only data available on teacher quality in Texas is the objectivecharacteristics o teachers, this study ocuses on our broad areas o teacher characteristics that are related to studentachievement in the literature:

1. Teacher experience;

2. Teacher certi cation status;

3. Teacher preparation program quality and certi cation scores; and,

4. Teacher stability.

Teacher ExperienceTeacher experience includes two measures o teacher experience: the percentage o novice teachers and percentage o

beginning teachers. Both measures have been ound to have a negative e ect on student achievement.


13/62


Teacher Certifcation StatusTeacher certi cation status includes two primary measures: the percentage o teachers ully certi ed/not ully certi ed

and the percentage o teachers assigned in- eld/out-o - eld. Although the research on these measures is weaker than thao teacher experience, the existing research suggests that ully certi ed teachersparticularly those assigned in- eldaassociated with gains in student achievement. However, these effects are more pronounced at the secondary levels and in thsubject areas o mathematics and science.

Teacher Preparation Program Quality and Certifcation ScoresTeacher preparation program quality and certi cation scores include our measures, all based to a varying degree on

teachers scores on the Pedagogy and Pro essional Responsibilities examinations administered by the state. Certi cationscore data or the years since 2007 is no longer publicly available. Further, certi cation score data prior to 1992 is incom-plete. However, more than 50% of all teachers included in the study had pedagogy scores. While this introduces a problemwith missing data, the percentage o teachers with missing certi cation score data did not vary across school demographiBecause the missing data was equitably distributed across schools with di erent student demographics, the impact o thmissing data is reduced.

As noted above, recent research has ound that teachers who score at the top end o the distribution on such certi catiotests and tests o general ability tend to be more e ective in increasing student achievement. At the other end o the dist

tion, those scoring substantially lower than their peers on such tests tend to be less e ective in increasing student achievement than other teachers. Further, these measures serve as a proxy or teacher verbal ability because a high correlation hasbeen ound between the pedagogy tests and tests o general verbal ability.

At the preparation program level, graduates rom highly selective undergraduate programs tend to have greater verbalability and, hence, greater test scores than their peers rom less selective undergraduate programs. Thus, graduating rom selective undergraduate institution serves as a proxy or teacher verbal ability. Because the majority o newly certi ed Texteachers are rom alternative certi cation programs rather than traditional undergraduate preparation programs, in ormaton the selectivity o the undergraduate program is not available or each teacher. Indeed, the state simply does not collectin ormation on the undergraduate program attended by alternative certi cation teachers. Thus, to identi y highly selec-tive programs, this study takes a di erent approach than that typically taken by researchers. First, I identi ed all teachersas being rom a highly selective institution i they had obtained an undergraduate degree rom a Texas public institution ohigher education that was rated as a Doctoral I or II institution or a Research I or II instruction by the Carnegie Classi ca-tion system. Second, for those teachers with missing data on undergraduate institution, I used the overall pedagogy scoresfor that institution to identify highly selective institutions. Speci cally, any institution with an average pedagogy score .3standard deviations or greater than average was identi ed as a highly selective institution. For those programs starting ater 2007, the initial certi cation score passing rates rom the TEA website were used to identi y highly selective prograThus, highly selective programs were identi ed based largely on the aggregate certi cation scores o the graduates o theprograms. The underlying assumption is that programs with high scores must be highly selective in selecting individuals toenter into the program. In the analysis, these programs are not identi ed as highly selective because it is not a pure measuro undergraduate institution selectivity. Rather, I identi y such preparation programs as high-per orming.

Teacher StabilityFinally, teacher stability includes two measures o teacher retention: a measure o the percentage o teachers who have

taught in a Texas public school at least three o the past ve years, and a measure o the percentage o teachers who havetaught at least three out o the last ve years at the same schools.

The rst teacher stability measure was used to identi y the percentage o teachers who have been in classrooms teachinthe Texas Essential Knowledge and Skills (TEKS) statewide curriculum. The idea behind this measure is that teachers whoare recently familiar with teaching the current TEKS might be more effective in improving achievement than teachers whohave either never taught, never taught in Texas, or have been out o the classroom or an extended period o time and thuare not as familiar with the current TEKS as those teachers who have been in the classroom in recent years. A recent studysuggests that this measure has a research foundation. Speci cally, Ost (2009) found that teachers who consistently teach thsame subject at the same grade level are more e ective than those who do not because these teachers are intimately am


14/62


with the content and can ocus on improving instruction rather than spending time learning the variations in curriculumfrom one grade level to another or from one subject area to another. I make the same argument with respect to the TEKSthat those teachers familiar with teaching the TEKS in recent years will be more effective in improving achievement thanthose who are unfamiliar with the TEKS for whatever reason. For example, while an out-of-state teacher might have yearso teaching experience, she or he will not be as e ective in teaching Texas students as someone with the same set o skillsand experience who has been teaching in Texas because the out-o -state teacher is simply not as amiliar with teaching theTEKS as the Texas teacher.

As mentioned above, recent research has ound that greater teacher retention is associated with increased student achieve-ment, independent o the quali cations or e ectiveness o the teachers who stayed at or le t the school. Indeed, in theirreview o the literature on teacher turnover, Ingersoll and May (2010) concluded that a general consensus has emerged thatteacher turnover is a perennial problem or a large number o schools and that such turnover has a negative e ect on studentachievement, especially when turnover rates are consistently high.

The individual measures within these our areas are described in greater detail in the next section o the report. Moreover,the ollowing section o the report describes the relationship between each o the measures under the our broad areas andstudent achievement on the TAKS mathematics and reading assessments.

d me This section o the report describes how the three TQI were constructed as well as the data and methodology employed

in the construction o each o the three TQI. All the data are rom the 2008-09 academic year unless otherwise noted.

c s Se e tQi rPast Teacher Quality Indices

Theoretically, there are numerous methods one could use to construct a TQI. The most prominent TQIs have been cre-ated in Cali ornia, Texas, and Illinois and relied on two di erent methodologies. The Cali ornia and Texas methodologieswere identical. The researchers ranked each teacher-quality measure included in the study (the variables used in Cali orniaand Texas were similar, but not entirely identical) and placed schools into deciles based on the ranking or a measure. Thedeciles were averaged across all measures, and schools were then placed into nal TQI deciles based on this average.

Researchers rom the Illinois Education Research Council (DeAngelis, Presley, & White, 2005; Presley, White, & Gong,2005) employed a more sophisticated methodology called principal components analysis (PCA). PCA is a statistical pro-cedure that examines a relatively large number o measures related to one another and that are, perhaps, even redundantsuch as teacher-quality measures. PCA then attempts to identi y a smaller number o variables than the original numberof variables that still explains the underlying conceptin this case, teacher quality. So, for example, PCA could be used tocreate a single index or teacher quality based on how well each individual measure o teacher quality (certi cation status,novice teachers, teacher certi cation scores, etc.) explains the overall concept o teacher quality.

Both methodologies have drawbacks. First, both methodologies did not rely on any statistical analysis o individual teach-er-quality measures and student achievement. Rather, both methodologies relied entirely on a reading o the literature in thearea to determine the basket o teacher-quality measures used in constructing a TQI. The problem with such an approach istwo old: First, measures are o ten de ned di erently in di erent states, especially with respect to certi cation status; second,a relationship that is ound to be statistically signi cant in one state might not be statistically signi cant in another state.

Second, the California and Texas TQI methodologies were overly simplistic and never validated as being statisticallysigni cantly related to student achievement. The Illinois PCA analysis was ound to be statistically associated with studentachievement in Illinois schools. However, it is unclear whether the study controlled for prior achievement.

Third, the Illinois study did not create di erent TQIs or the di erent school levels. This could be problematic becauseteacher quality is pro oundly a ected by the supply o teachers, and there is generally a greater supply o elementary teach-


15/62


ers than secondary teachers. Speci cally, the greater the supply of teachers, the more selective schools can be. The moreselective schools can be, the greater the teacher quality o newly hired teachers and o the overall teaching orce.

One method not used to create a TQI would be to conduct a statistical analysis o the relationships between the variousmeasures o teacher quality and student achievement, then use the results o such an analysis to weight the various measuin the construction o a single TQI.

New Texas Teacher Quality IndexThis analysis explored the construction o a TQI using two di erent methods. These two methods were used to create

three separate but related TQIs. Ultimately, all three TQIs were ound to be airly highly correlated, and all were statisticalsigni cantly related to student achievement, even a ter controlling or prior levels o achievement, student demographicsand geographic location.

Methodology OneThe rst methodology employed regression analysis to identi y the individual teacher quality measures statistically sign

icantly associated with student achievement a ter controlling or prior achievement, student demographics, and geographlocation. This methodology had not previously been employed in creating a TQI, but it was suggested as the best method-ology by several researchers rom across the country. The TQI created by this methodology was called the Regress TQI.

Methodology TwoThe second methodology was similar to the Cali ornia and Texas methodology described above. This methodology

identi ed a basket o teacher-quality measures and simply averaged the measures. Variables that the literature base indicatwould have a negative relationship with student achievement were multiplied by negative one.

Two TQIs were created using this methodologyAvg 1 TQI and Avg 2 TQI. The di erence in the two TQIs was thevariables included in each TQI. In short, the Avg 1 TQI included variables in the rst three teacher-quality areas (teacherexperience, teacher certi cation status, and teacher preparation program quality/certi cation scores), while the Avg 2 TQIincluded variables in all our teacher-quality areas (teacher experience, teacher certi cation status, teacher preparation program quality/certi cation scores, and teacher stability).

c e s tQi r s This section o the report describes and de nes the various measures o teacher quali cations that were explored in thi

study. The relationship between each o the ollowing measures and student achievement were examined through a litera-ture review and through ordinary least-squares regression analysis using Texas-speci c data. Not every measure, however,was included in a nal TQI.

As shown in Appendix A, the variables included in the Regress TQI were selected based entirely on the results o the regression analysis or middle schools and high schools. For elementary schools, as explained above, only two o the

our statistically signi cant variables were included in the analysis. Three o the our statistically signi cant variables wrelated to teacher stabilitythe area o teacher quality with the smallest and least de nitive research base. The decision wmade to use only the teacher-stability variable with the strongest relationship to student achievement so that the elementarschool TQI was not just a re ection of teacher stability. However, even if all four variables had been included, the resultingTQI would have been nearly identical to the nal Regress TQI. Indeed, the correlation between the two-variable and our-variable models was .907.

For the other two TQI (Avg 1 TQI and Avg 2 TQI), the decision to include or exclude a measure was based primarily onthe results o the regression analysis. For elementary schools and middle schools, however, there was not always a statisti-cally signi cant result in each o the our areas, despite ndings in the literature that would suggest such variables to bestatistically signi cant.


16/62


Speci cally, neither school level had a statistically signi cant result in the area of teacher preparation program quality andcerti cation scores. Although the lack o statistical signi cance might indicate no relationship between the area o teacherquality and student achievement in Texas, the data available might simply not be precise enough or the analysis to identi ya relationship that actually exists. There was missing data or all our o the individual measures in this area, and, moreover,the available data were simply proxies o data used in other studies. For example, Texas does not collect or identi y theundergraduate institution o all teachers, particularly those rom alternative certi cation and out-o -state programs. Thus,rather than using actual undergraduate programs to identi y selective institutions, the study relies on a composite o under-graduate institution data and pedagogy data to identi y high-per orming preparation programs.

For elementary schools, there was also no statistically signi cant result in the area o certi cation status. Again, while thismight signal there was no statistically signi cant relationship between certi cation status and student achievement, the lacko precise data might also be the reason or the lack o a statistically signi cant relationship. Indeed, because elementaryteachers receive a composite certi cate that is based primarily on the scores rom the reading sub-test, we dont have anyidea of the distribution of teachers who did not pass the mathematics, science, social studies, or reading sub-tests. So, forexample, a teacher could be considered ully certi ed to teach all elementary subjects, but the same teacher could haveincorrectly answered all of the mathematics sub-test items. Such a teacher should be considered not fully certi ed in math-ematics. The availability o sub-score data may, in act, lead to a statistically signi cant relationship between certi cationstatus and student achievement.

In the Avg 1 TQI, the measures ound to be statistically signi cant in the regression analysis were included in the TQIbut only i they did not pertain to teacher stability. I there was no statistically signi cant measure within each o the threebroad areas o teacher quality (experience, certi cation status, and program quality/certi cation scores), then the measurewithin each o the three areas that was closest to being statistically signi cant was included. Regardless o the level o statisti-cal signi cance, none o the variables related to teacher stability were included in the Avg 1 TQI. These related measureswere not included because the literature base on the relationship between teacher stability and student achievement isrelatively small and not as consistent as or the other three areas o teacher quality. Further, some researchers argue thatteacher turnover can o ten improve student achievement and that the turnover results in the less e ective teachers leavingthe campus.

Finally, in the Avg 2 TQI, each o the measures ound to be statistically signi cant in the regression analysis were includedin the TQI. In addition, each measure selected or the Avg 1 TQI was also included in the Avg 2 TQI. Finally, any measures

in the teacher stability area that were ound to be statistically signi cantly related to student achievement in the regressionanalysis were also included in this TQI.

Ultimately, as discussed in greater detail below, each o the three TQIs were ound to be statistically signi cantly relatedto student achievement, even a ter controlling or prior achievement, student demographics, and geographic location. Thus,even though Avg 1 TQI and Avg 2 TQI were not based solely on regression results, the composite indicators were statisti-cally signi cantly associated with student achievement. Further, even though some would argue against the inclusion o teacher stability variables, the exclusion or inclusion o such variables had little e ect on the relationship between the TQIand student achievement.

The details o the construction o each TQI are included in Appendix A. Each o the variables that were identi ed in theliterature as being related to student achievement and that were available through existing data sources are described below.Again, each o these variables was entered into an ordinary least-squares regression analysis to determine i the measure had

an independent association with student achievement a ter controlling or other actors.

te e E e e e Percentage o Novice Teacher s: The percentage o a schools teachers with ewer than our years o teaching experience.

The data was modi ed to remove errors and to improve its accuracy. To do so, I used a combination o employment history,initial certi cation year, and district-reported years o experience to create a more accurate indicator o a teachers experi-ence as an educator. Because I modi ed the original data, this measure cannot be replicated with state data. (Original DataSource: Educator responsibility data, TEA; Final Data Source: Teacher experience le created by Dr. Ed Fuller).


17/62


Percentage o Beginning Teachers: The percentage o a schools teachers in their rst year o teaching. This data wasmodi ed in the same way as described for the percentage of novice teachers. (Original Data Source: Educator responsibilidata, TEA; Final Data Source: Teacher experience le created by Dr. Ed Fuller).

te e ce S s

Percentage o Teachers Fully Certifed: The percentage o a schools teachers who have obtained ull state certi cation. Tobtain ull state certi cation, a person must complete an approved preparation program, pass a pro essional responsibility(pedagogy) examination, pass a content examination, and clear a criminal background check. In most cases, the person muhold a bachelors degree rom an accredited undergraduate institution unless teaching in vocational education. A personemployed on a one-year certi cate or teachers rom out-o -state and a person employed on a probationary certi cate whenrolled in an alternative certi cation or post-baccalaureate program are not ully certi ed. It is important to note that ucerti ed in this analysis is not ully certi ed as construed by the No Child Le t Behind Act. Under NCLB, teachers onprobationary or out-of-state certi cates may be considered fully certi ed if they meet certain requirements (Data Source:Who is Teaching in Texas, 2009; SBEC).

Percentage o Teachers Out-o -Field: The percentage o a schools teachers assigned out-o - eld. A teacher is consideredassigned out-o - eld i she or he does not hold a ull state certi cation or the subject area she or he is assigned to teachmatch between the certi cate and course and subject area taught is determined by rules generated by SBEC. (Data SourceWho is Teaching in Texas, 2009; SBEC).

Percentage o Teachers Not Fully Certifed: The percentage o a schools teachers who do not hold a ull state certi cate.Teachers not holding a ull state certi cate include persons employed on a one-year certi cate or teachers rom out-o -sand persons employed on a probationary certi cate while enrolled in an alternative certi cation or post-baccalaureate pro-gram. A nal category o teachers not ully certi ed are long-term substitute teachers. It is important to note that not ucerti ed in this analysis is di erent than the not ully certi ed as de ned by the No Child Le t Behind Act. Under Nteachers on probationary or out-o -state certi cates may be considered ully certi ed i they meet certain requirements,while teachers on probationary and out-o -state certi cates are automatically considered not ully certi ed in this analysi(Data Source: Who is Teaching in Texas, 2009; SBEC).

te e p e p ce E S es Percentage o Teachers rom High-Per orming Educator Preparation Programs: The percentage o a schools teachersrom an educator preparation program classi ed as high-per orming. This variable is somewhat similar to the percentage

o teachers rom highly selective programs, but the percentage o teachers with data on undergraduate programs attendedwas too small to use it as a measure o teacher quality. Thus, the high-per orming measure was created by combining thredi erent types o data. First, teachers rom undergraduate programs ranked as Doctoral I, Doctoral II, Research I, or Re-search II institutions under the Carnegie Classi cation system were identi ed as being rom high-per orming preparationprograms. Second, teachers from preparation programs that had an average pedagogy certi cation score greater than .33standard deviations above average were identi ed as being rom high-per orming preparation programs. Third, or teachers rom preparation programs started a ter 2007, teachers rom programs with average pedagogy certi cation scores in top 10% of all preparation programs were identi ed as being from high-performing preparation programs. (Data Source:Teacher Certi cation Scores; SBEC; Higher Education Graduation Institutions and Degrees, Texas Higher Education

Coordinating Board).School-Level Average Pedagogy Certifcation Scores: This measure is the average o the pedagogy scores o teachers

employed in the school. Because pedagogy scores are from both the ExCET and TExES examinations, which have differ-ent scales, the scores were converted into z-scores. This measure had the most missing data o any measure in this study.


18/62


Teachers certi ed prior to 1987 did not take a certi cation examination, and those certi ed prior to 1993 o ten did not havea score reported even i a score existed. Further, out-o -state teachers may not have to take a certi cation exam under certaincircumstances. Finally, scores were made con dential in 2008; thus, scores could not be obtained or any teachers certi eda ter December 2007. Ultimately, about 55% o all teachers had pedagogy certi cation scores in the database. This percent-age was consistent across school levels and across schools serving di erent percentages o economically disadvantaged stu-dents. Further, an analysis o the relationship between this measure and student achievement was conducted or all schoolsin the study as well as or a sub-sample o schools that had certi cation scores or at least 75% o teachers. With both setso schools, the relationship between the aggregate pedagogy z-scores and student achievement was essentially the same.Because the missing data appeared to be spread relatively uni ormly across schools, and because restricting the sample toschools with at least 75% o teachers with certi cation score data did not signi cantly alter the relationship between scoresand achievement, this measure was included in the study. (Data Source: Teacher Certi cation Scores; SBEC).

Percentage o Pedagogy Certifcation Scores Greater than One Standard Deviation Above Average: The percentage o aschools teachers who had pedagogy scores one standard deviation above the average pedagogy score. Again, as with thepedagogy scores described above, there was the problem o missing data. The missing data was spread relatively uni ormlyacross schools, and restricting the sample o schools to those that had at least 75% o teachers with pedagogy scores did notalter the results. Thus, despite the missing data, the measure was included in the study. (Data Source: Teacher Certi cationScores; SBEC).

Percentage o Pedagogy Certifcation Scores Less than One Standard Deviation Below Average: The percentage o aschools teachers who had pedagogy scores one standard deviation below the average pedagogy score. Obviously, therewas an issue o missing data again. But, the missing data was spread relatively uni ormly across schools, and restricting thesample o schools to those that had at least 75% o teachers with pedagogy scores did not alter the results. Thus, despite themissing data, the measure was included in the study. (Data Source: Teacher Certi cation Scores; SBEC).

te e re e E e Percentage o Teachers Employed in a Texas Public School or at Least Three Years rom 2004-05 to 2008-09: The percent-

age o a schools teachers who were employed in a Texas public school at least three out o the ve academic years spanningthe academic years from 2004-05 to 2008-09. (Data Source: Educator Responsibility data, 2005, 2006, 2007, 2008, 2009, TEA).

Average One-Year Teacher Retention Rate rom 2005-06 to 2008-09: The average teacher retention rate or a school or theollowing three groups o academic years: 2005-06 to 2006-07; 2006-07 to 2007-08; and 2007-08 to 2008-09. The reten-

tion rate was calculated or each pair o years, and then the average o the three retention rates was calculated. A three-yearaverage was used because a one-year retention rate could refect the impact o opening a new school in the area rather thanteacher dissatisfaction with a school. (Data Source: Educator Responsibility data, 2005, 2006, 2007, 2008, 2009, TEA).

Percentage o Teachers Staying at the Same School rom 2007-08 to 2008-09: The one-year teacher retention rate romthe 2007-08 school year to the 2008-09 school year. Teachers were matched individually rom one year to the next usingeducator responsibility data rom TEA. This data includes the school in which each person is employed as well. Once thetwo les were matched, I determined whether a teacher had taught at a school or both years or or only the base year o 2007-08. Although, as noted above, a one-year retention rate does not always refect teacher dissatis action, the simple losso teachers and the institutional knowledge that they possess can o ten have deleterious e ects on student achievement.(Data Source: Educator Responsibility data, 2008 and 2009, TEA).

Percentage o Teachers Staying at the Same School : The percentage o a schools teachers employed in 2008-09 who hadbeen employed at the same school at least three out of ve academic years covering the span of years from 2004-05 to 2008-09. (Data Source: Educator Responsibility data, 2005, 2006, 2007, 2008, 2009, TEA).


19/62


S e S sNot all Texas public schools were included in this study. First, only schools that appear in the Texas Education Agencys

Academic Excellence Indicator System (AEIS:http://ritter.tea.state.tx.us/per report/aeis/ ) or the 2008-09 school yearwere included in the study. The numbers of schools by school level included in the 2008-09 AEIS les are represented inColumn 1 in Tables 1A and 1B.

Using this initial number o schools as a base, schools without regular accountability ratings were removed rom the lisschools. The resulting number o schools is displayed in Column 2. These schools were removed because schools withoutregular accountability ratings typically serve small populations o special types o students, such as special education students or students removed rom their regular schools because o disciplinary in ractions. Because such schools are typicsmall, the teacher quali cations measures tend not to provide an accurate picture o the quali cations o the sta .

Next, charter schools were removed rom the sample because they are not required by statute to ollow the same rulesas regular public schools with respect to teacher quali cations. Further, many are very small and serve special populationso students, particularly students who have previously dropped out. The number o schools a ter the removal o charterschools is displayed in Column 3.

Subsequently, schools that did not enroll students in both the 2007-08 and 2008-09 school years were removed from the

sample. The resulting number of schools is shown in Column 4. Such schools were removed for a number of reasons, butprimarily because one primary purpose o the study was to examine the relationship between teacher quality and studentachievement. In doing so, controlling or prior levels o achievement is a critical component o the analysis. Thus, schoothat did not exist in both years could not provide both years of TAKS scores that are necessary to complete the analysis.

Similarly, I then removed schools that did not enroll students in selected grade levels. Schools had to meet strict grade- enrollment criteria or two reasons. First, this study examined the relationship between measures o teacher quali cationand student achievement at the school level, and schools with di erent grade con gurations can have di erent levels ostudent achievement. Indeed, the addition or elimination o one grade level can dramatically alter the overall achievementpro le o a school. Thus, in order to make an apples-to-apples comparison on student achievement across schools,schools had to enroll students in the same grade levels.

Second, teacher certi cation operates differently at the different school levels. For example, elementary teachers and

some middle school teachers can obtain a generalist certi cate that allows them to be considered ully certi ed or any cosubject area at that school level, while most middle school teachers and all high school teachers obtain certi cates speci ca particular content area, such as mathematics. Only by including schools with the same grade level con gurations could Iensure that di erences in teacher quali cations were based on supply, demand, and hiring actors rather than di erencegrade con gurations. The number o schools le t in the sample a ter removal o schools not meeting the grade-level criteis shown in Column 5.

At the elementary-school level, schools had to enroll students in grades three, our, and ve but not in any higher gradesto be included in the sample. At the middle-school level, schools had to enroll students in grades six, seven, and eight andnot in any other grade levels to be included in the analysis. At the high-school level, schools had to enroll students in gradenine, ten, and eleven and not in any grades lower than grade nine to be included in the analysis.

Schools were also removed if they did not have a large enough number of students taking the TAKS test at each of the three

grade levels. At each school level, a school had to have at least 30 students take the TAKS test at each grade level in order to bincluded in the analysis. Schools with smaller numbers of test-takers were excluded from the analysis because the measuremeerror on the test scores is simply too large to provide an accurate estimate o the test per ormance o the students.

Finally, to be included, schools had to meet the TAKS testing accountability sub-population criteria of 30 students at eacgrade level in both the 2007-08 and 2008-09 school years. This criterion was employed because two years o test scoreswere needed to assess the relationship between measures o teacher quali cations and student achievement, and a sampleo 30 is the minimum number o students necessary to obtain a relatively accurate assessment o a schools achievementSchools not meeting the TAKS sample-size criteria were excluded from the analysis. The nal sample of schools is denotein Column 6.


20/62


sThe ndings are separated into four different sections. Section I describes the ndings about the association between each

TQI and student achievement. More speci cally, this section discusses whether a TQI was statistically signi cantly associatedwith student achievement on TAKS. Section II describes the relationship between the three TQI measures and the individualmeasures of teacher quality. Section III describes how TAKS achievement was distributed across schools with different TQIratings . Section IV examines the distribution of TQI ratings by school accountability rating and student demographics.

s Se i: ass be eetQi r s S e a e e e

The rst ndingand one that is important to note be ore proceeding with an analysis o the distribution o teacherquality across schoolswas that each o the three TQI ratings (Regress TQI, Avg 1 TQI, and Avg 2 TQI) were ound tobe statistically signi cantly associated with student achievement in 2008-09. Importantly, this statistically signi cant as-sociation held even a ter controlling or the e ects o prior achievement, school demographics, and geographic location o the school on student achievement in 2008-09. Because these other actors were controlled or in the analysis, we knowthat the association between each TQI and student achievement did not exist simply because a TQI was related to studentdemographics that were, in turn, related to student achievement. Rather, we know that the association between a TQI andstudent achievement was a separate association apart rom any association between student demographics and either a TQI


21/62


or student achievement.

Further, as shown in Appendix A in Tables A-2b, A-3b, and A-4b, each TQI explained a fair proportion of the variancein achievement when entered into the regression equations be ore other variables. At the elementary-school level, the TQIratings explained between 3% and 13% o the variance in student achievement. At the middle-school level, the three TQIratings explained between 10% and 24% of the variance in student achievement. At the high-school level, the three TQIratings explained between 29% and 34% of the variance in TAKS scores.

In other words, teacher quality explains between 4% and 33% of the variability or uctuation in student achievement onTAKS. Importantly, at the high school level, teacher quality explains about 30% of the variability or uctuation in studentachievement on TAKS.

When the TQI ratings were entered a ter all other control variables (student demographics and geographic location) butbe ore the prior achievement variable, the ratings explained less than 3% o the variance in scores. The di erence in theproportion o variance explained when entering the TQI ratings rst and later in the equations is due to the shared explantory power between the TQI ratings and student demographics. In other words, the TQI ratings and student demographicsshare some ability to explain student achievement. I the TQI ratings are entered rst, then the shared explanatory power iattributed to the TQI ratings. I student demographics are entered rst, then the shared explanatory power is attributed tothe student demographics.

Regardless, the results showed that TQI ratings were statistically signi cantly associated with student achievement.Further, the association appeared to be non-trivial, even a ter controlling or prior achievement, student demographics, angeographic location.

s Se ii:tQi me s es te e Q

In this section and in subsequent sections, three di erent TQI ratings are presented. The rst is the Regress TQI thatwas created by weighting measures of teacher quality based on results of OLS regression analyses as described in the met-odology section. The second is the Avg 1 TQI that was constructed by simply averaging a series o selected measures oteacher quality directly. The third ratingthe Avg 2 TQIincluded the measures employed in the Avg 1 TQI as well asmeasures o teacher retention and stability at the school.

This section o the report describes the distribution o teacher quali cation measures across the TQI ratings group-ings or each o the three school levels. The distributions or each o the three TQI ratings (Regress, Avg 1, and Avg 2) apresented. The purpose o this section is to assist the reader in comprehending how the TQI ratings refect the variousmeasures o teacher quali cations.

Be ore reporting on the relationship between the TQI ratings and individual measures o teacher quality, a guide to un-derstanding the TQI numbers is provided below.

i e e e tQi r s Be ore reporting the ndings, this section describes how to interpret a TQI rating. All three TQI ratings have been

converted into standard normal scores, or z-scores. The bene t o this conversion is that each TQI rating has a mean, oraverage, o 0 and a standard deviation o 1.0. This is shown below in Figure 1.


22/62


23/62


t ch

t ch

t ch

t ch


24/62


The two primary components o the elementary TQI ratings were the percentage o novice teachers and the percentageo teachers employed at least three o the last ve academic years. These measures were correlated (correlation coe cient =.742, p < .001), but not identical. Many teachers move into the state from other states, transfer from private schools, or havebreaks in service. Thus, both measures were use ul in explaining the overall teacher quali cations at the school.

As shown in Table 3, schools with TQI ratings o less than -1.0 had substantially greater percentages o novice teachersthan schools with TQI ratings greater than 1.0. Strikingly, at least one-third of teachers in schools with TQI ratings lessthan -1.0 had three or ewer years o teaching experience.

As shown in Table 4, schools with TQI ratings less than -1.0 had far lower percentages of teachers employed three of thepast ve years in Texas than schools with TQI ratings greater than 1.0. Indeed, the di erences are at least 23 percentagepoints. In schools with TQI ratings greater than 1.0, nearly all teachers had been employed in a Texas public school or atleast three o the past ve academic years.

In addition, schools with low TQI ratings had lower percentages o teachers ully certi ed and o teachers remaining atthe school or various measures o time as compared to schools with higher TQI ratings. For example, schools with the lowestTQI ratings had at least 14% teachers assigned out of- eld as compared to less than 5% in schools with the highest TQI ratings.

m e S sAs shown in Table 5, there is a substantial di erence in all o the teacher-quality measures between schools with the great-

est TQI ratings (> +1.5) and those with the lowest TQI ratings (< -1.5). Although all o the di erences are airly substantial,one o the most remarkable is or the percentage o teachers assigned out-o - eld. Although less than 11% o teachers inschools with the greatest TQI ratings were assigned out-of- eld, greater than 40% of teachers in the schools with the lowestTQI ratings were assigned out-o - eld.


25/62


t ch

t ch

t ch

t ch


26/62


t ch

t ch

t ch

t ch


27/62


h S sAs shown in Table 6 on page 25, there were substantial di erences in all teacher-quality measures between schools wit

low- and high-TQI ratings. Some of the most signi cant differences include:

Schools with the highest TQI ratings had nearly 100% fully certi ed teachers, while schools with the lowest TQI rat-ings had only 75% ully certi ed teachers;

In schools with the highest TQI ratings, at least 75% of teachers had been employed at the schools for at least three ofthe previous ve academic years, as compared with 50% or less or schools with the lowest TQI ratings.

Schools with the highest TQI ratings had teacher certi cation scores that were more than 0.5 standard deviationsgreater than schools with the lowest TQI ratings; and,

Although the percentage of teachers from high-performing teacher preparation programs approached 60% in theschools with the highest TQI ratings, less than 18% o teachers in schools with the lowest TQI ratings were rom higper orming prog

atpe 2010 teacher quality study

Documents