teacher effects on educational achievement

Teacher Effects on Educational Achievement . Matthew Puckett . AU ID 3665859 . May 4, 2013 Why is it Important? It is not a contentious claim that the quality of a teacher directly impacts the educational achievement of students, as this is widely accepted. What is still in question, however, is to what extent many variables can quantitatively predict these achievement metrics. Many studies have attempted to research this exact issue, to find which qualities in teaching staffs are most directly related to positive effects in student learning outcomes. This topic is an important one as it pertains to the realm of public policy implications. Especially in the current political climate with tightening budgets in public schools, it is wise for schools to use their resources more effectively and use teachers in a way that is most beneficial to their students. This memo begins by introducing the previous research, concepts, and analytical theory behind teacher effects on student achievement. Next, I explain the specific hypotheses that this analysis seeks to test, keeping in line with the theory of teacher effects. Following this is a short section describing both the data set and the variables I will use, along with descriptive statistics of these variables and how they are measured. Finally, I explain the methodology used in the OLS regression, possible issues, resolutions, and limitations of the analysis, and a results and analysis portion. To conclude, there is a short discussion of policy implications and guidance for expounded study. What do we know? The effects of teachers on their students achievement has been the subject of much study for decades, and has become increasingly relevant since our schools have been declining in terms of international performance1. Further, with tightening austerity measures and an increasing focus from policymakers on educational efficiency2, schools and school districts must use research and statistical methods to get the best bang for their buck. If specific teacher measures can be pinpointed to effectively increase graduation rates, those qualities can be focused on to improve schools. A long-held position has been that classroom size has a great deal to do with student performance3. The conventional thinking is that if a teacher has a smaller class, and therefore more time to spend individually with students, then the students will have a more productive learning experience and

Figure 1

~ Teacher Effects on Graduation Rate ~

Page 2

environment. There have been multiple studies in the past that have confirmed this assumption, while recently there have also been studies that cloud the issue4. As demonstrated in Figure 1, the effects of classroom reduction are often minimal, if they exist at all. These newer studies have challenged conventional wisdom and explained that classroom size is not nearly as important to student achievement as previously thought. For the purposes of this analysis, I assume that class size does negatively correspond to student achievement, but in light of conflicting evidence, I am curious to see what the Georgia high schools data will say. Another classical assumption is that hiring teachers with advanced degrees is a practical way to increase student performance5. This is seen is practice through school hiring standards, as teachers who do have advanced degrees are more appealing to employ, have lower unemployment rates, and often have a higher salary ceiling6. The theory behind employing is those with advanced degrees is that they will possess more knowledge of their subject matter, and in turn, be better at relaying this knowledge to students. A more recent theory has been promoted that diversity among teaching staff will also result in positive benefits to students7. The theory is, that students who are taught by a more diverse group of people will gain multiple perspectives and be more open-minded, resulting in better performance at school.

Finally, the effects of teacher salary have also been considered a useable predicting measure of student success8. As seen in Figure 2 to the left, countries with higher average teacher salaries have students with higher achievement, and this should also be true on a school-to-school basis within a country as well. The theory is that teachers

who are paid more, and more likely to excel at their jobs and take an increased interest in their students doing well a teacher who is happy in the job will be a better teacher. This theory is consistent with studies confirming that higher wages lead to more productive workers, and it is reasonable to ascertain that this will hold true for the teaching profession9. Below are four hypotheses that I will use to test some of the claims expressed in the current literature. While not a comprehensive list, by any means, the amount of data collected is enough to give a broad analysis of some of the issues mentioned above.

Figure 2


Page 3

Hypotheses HYPOTHESIS 1. Students are more likely to graduate at a higher rate when their

teachers are paid more

HYPOTHESIS 2. Students are more likely to graduate at a higher rate when their teachers have completed an advanced degree

HYPOTHESIS 3. Students are more likely to graduate when their class sizes are smaller

HYPOTHESIS 4. Students are more likely to graduate if their teachers are more diverse

Data The data being used in this analysis is from a survey of public high schools in Georgia. The dataset includes information on teaching staff, administrative staff, students, and school performance, among other variables. The units of analysis in the data are the high schools within Georgia, and the set contains 367 observations taken during the 2007 2008 school year. Variables of Interest Include Graduation Rate (Interval-Ratio)

Average teacher salary (Interval-

Ratio)

Average teacher experience (Interval-Ratio)

Percent of teachers who identify as nonwhite (Interval-Ratio)

Student/Teacher Ratio (Interval-Ratio)

Percent of teachers with an advanced degree (Interval-Ratio)

Percent of teachers who are part-time (Interval Ratio)

Table 1 - Georgia Public Schools, Variable Descriptives Range Mean Standard Deviation Low High Graduation Rate 72.0288 14.0655 0 100 Avg. Teacher Salary ($1,000s) 47.8465 3.1016 36.75 60.07 Avg. Teacher Experience 13.3139 2.5928 2.22 19.55 % of Nonwhite Teachers 0.2295 .2553 0 1 Student/Teacher Ratio 15.9822 6.3948 0.56 124 % of Teachers w/ Advanced Degree

57.5362 9.7980 0 90

% of Teachers who work part-time

7.0628 10.4296 0 70


Page 4

Method To test my hypotheses, I use ordinary least squares (OLS) linear regression to estimate coefficients of a model to forecast graduation rates based on our independent variables. Since I do not have any research to lead to a nonlinear model for any of our tested variables, a linear form is appropriate, and OLS is the best estimator of our coefficients by minimizing the sum of squared residuals. The OLS method does, however, present challenges in presenting a properly specified and accurate model. In order for OLS to be the correct method, there are five assumptions that must be met:

1. The model is correctly specified there are no omitted or irrelevant variables and the correct functional form is used,

2. There is no reverse causation between the dependent and independent variables (exogineity),

3. Error terms are not correlated (serial correlation), 4. There is constant error variance (homoskedasticity), and 5. No near or perfect multicollinearity

In putting together any regression model, efforts should be taken to test for and correct any of these violations. The most pressing issue is specification of the model, including both specification of variables in the model and the functional form used. The functional form in this case is linear, and in order to justify a different form the existing research would need to lead us to a hypothesis (i.e. logarithmic or parabolic). Since there is no evidence to suggest any of our independent variables have these characteristics, I assume a linear model is sufficient. Care was also taken to include all relevant variables from the data into my equation, while leaving out irrelevant variables. Since I am focusing solely on teacher effects, I left out variables pertaining to student achievement. Reverse causation, or exogineity, should also be considered between the dependent variable and all independent variables. In this model, the independent variable is graduation rate in schools, which does not possibly have an effect on any of our teacher measurables. While the effect of teacher salary, student-teacher ratio, etc. will likely have an impact on graduation rate, the opposite is not true meaning exogineity is not an issue. Care must also be taken to address autocorrelation in our model. To test for the presence of any serial correlation, we can plot the models residuals against their expected value. Figure 3 shows the chart visualizing the distribution of residuals. There is Figure 3 Residuals, Autocorrelation Test


Page 5

no obvious autocorrelation seen, so this assumption seems to have been met. Homoskedasticity is also a concern, especially with the data I am using. To a certain degree, there will always be violations of this assumption. The data set is cross-panel, a red flag for homoskedasticity issues, and the range of our variables is also considerably wide, which furthers the problem. An easy way to correct for this violation is to use robust standard errors, which adjust for homoskedasticity issues by ensuring hypothesis tests are not too confident. By increasing the standard errors in the model, any homoskedasticity is accounted for. Finally, multicollinearity will increase the variances and standard errors of the estimated coefficients should it exist in a model. To check for multicollinearity, the first step is to discern if there are any unexpected signs in the model (i.e. a positive correlation is shown when research suggests the correlation is negative). Further, collinearity can be checked through the variance inflation factor (VIF). The results of this test are below.

Since none of the VIF values are over ten (the threshold for further inspection), there appears to be no issues with multicollinearity. Since all the assumptions of OLS have either been met or accounted for, the model seems to be correct.

Results & Analysis After running the regression model for forecasting graduation rate (dependent variable) against the six independent variables, the coefficients of each are an indicator of their effects. It is also necessary to run hypothesis tests on each variable to determine whether the coefficient is statistically significant within the model. The coefficient of determination, or R squared, is just 0.1966, which is very low. This means that 19.66% of the variance in graduation rate can be accounted for through the model. While a low R squared does not in and of itself mean a model is not good, this models R low R squared likely tells us we are missing some variables that explain variation in graduation rate. This also makes the overall significance of the model shaky, since over 80% of the variation in the dependent variable is not taken into account and would likely change the models coefficient significantly. For each of the hypotheses stated earlier, a determination must be made about the significance of individual variables. To test the significance of each variable, I use a 95% confidence level, comparing the P > | t | value to 0.05. The four variables from the regression model that will be used to test the original hypotheses, and the results of each T-tests:


Page 6

1. Average Teacher Salary ($1,000s) The P-value of the teacher salary variable is 0.038, which gives it statistical significance at the 95% confidence level. 2. Percent of Teachers with an Advanced Degree The P-value of the advanced degree variable is 0.407, which means that the variable is not of statistical significance. 3. Student Teacher Ratio The P-value of the student/teacher ratio is 0.046, which gives it statistical significance at the 95% confidence level. 4. Teacher Diversity The P-value of the nonwhite teacher percent variable is ~0.000, which gives it statistical significance at the 95% confidence level.

Table 3 Regression Results for Graduation Rate Constant 31.523

(18.121)

Avg. Teacher Salary ($1,000s) 0.797** (0.382)

Avg. Teacher Experience -1.388**

(0.372)

% of Nonwhite Teachers -15.320** (3.607)

Student/Teacher Ratio 1.116**

(.558)

% of Teachers w/ Advanced Degree 0.090 (0.109)

% of Teachers who work part-time 0.188**

(0.058)

R-squared 0.1966 No. Observations 353 Robust standard errors are reported in parenthesis. **indicates significance at 95% confidence.

By combining the coefficient estimates in the above table with our determinations of statistical significance, the hypotheses described at the beginning of the memo can be tested.


Page 7

HYPOTHESIS 1. Students are more likely to graduate at a higher rate when their teachers are paid more

H0: Teachers salary is not correlated to higher graduation rates. Ha: Teachers salary is correlated to higher graduation rates.

To test this hypothesis, I use the teachers salary variable in the model, and statistical significance was found meaning that ! 0. Taking this into account, we can reject the null hypothesis and conclude that teachers salary is correlated to higher graduation rates. The regression model estimates that, for every increase in teachers salary of $1,000, graduation rate is expected to rise 0.797%, on average. HYPOTHESIS 2. Students are more likely to graduate at a higher rate when their teachers have completed an advanced degree

H0: Percent of teachers who have an advanced degree is not correlated to higher graduation rates.

Ha: Percent of teachers who have an advanced degree is correlated to higher graduation rates.

To test this hypothesis, I use the percent of teachers who have an advanced degree variable in the model, but since statistical significance was not found, we fail to reject the null. The regression model estimates that, for every increase of one percent in amount of teachers who have advanced degrees, graduation rate is expected to rise 0.09%. But again, since the coefficient is not statistically significant, we cannot reject the null hypothesis. HYPOTHESIS 3. Students are more likely to graduate when their class sizes are smaller

H0: A lower student-teacher ratio is not correlated to higher graduation rates. Ha: A lower student-teacher ratio is correlated to higher graduation rates.

To test this hypothesis, I use the student-teacher ratio variable, measured in interval-ratio form. Since statistical significance was found, we can conclude that ! 0. Taking this into account, we reject the null hypothesis and conclude that a lower student-teacher ratio is correlated to higher graduation rates. The regression model estimates that, for every one-pupil decrease in student-teacher ratio, graduation rate is expected to rise 1.116%, on average. HYPOTHESIS 4. Students are more likely to graduate if their teachers are more diverse

H0: A more diverse teaching staff is not correlated to higher graduation rates. Ha: A more diverse teaching staff is correlated to higher graduation rates.

To test this hypothesis, I use the percent of teachers who are nonwhite variable in the model. Based on research and intuition, it was expected that more diversity would lead to higher graduation rates, but the model found significance in the variable the other way


Page 8

(i.e. more diversity led to worse graduation rates). It is most likely that this coefficient is negative due to the low R squared value and poor overall power of the model. It is also possible that unobserved variables contributed to this result, or even that there actually is a negative correlation between diversity and graduation rate. To any extent, this result means that we cannot reject the null hypothesis. Implications & Discussion To conclude, if our regression coefficients are a valid model to estimate graduation rate, there are policy implications to be pondered. As expected, a higher average teacher salary increases graduation rates in schools, which is a strong case to raise teacher salaries. This fact is supported by our previous research, and is likely due to the fact that paying teachers more attracts more talent to the field. These more talented teachers help their students graduate and even though budgets are being tightened, investing in teacher pay seems to be a worthwhile endeavor. We could not conclude that teachers who have advanced degrees have better performing students, and if this is the case it would be wise to cut back on hiring these teachers, especially because they often have higher salary demands. We were able to conclude, however, that a lower student-teacher ratio helped graduation rates. While there has been recent evidence to the contrary, this result is in line with years of research. Reducing class size could be an effective tool for learning outcomes. Finally, we could also come to no firm conclusion regarding diversity of teaching staffs. The model is full of shortcomings, however, and is unlikely to be a reliable prediction tool. The conclusions drawn from the analysis are still subject to the very low R squared value, which compromises the results. Further study regarding these variables would likely need to incorporate many more descriptive variables, and possible use of multi-level data to hold for unseen variation. Further research would be very beneficial on this subject as it relates to school-level decisions, and should be focused on explaining much more of the graduation rate variation.


Page 9

Citations 1. Elliot, Phillip. "US Schools Still Lagging Other Nations." The Boston Globe, 25 Apr. 2013. Web. . 2. Ginn, Jennifer. "Why Johnny Cant Ride the Bus." The Council of State Governments, n.d. Web. . 3. Nye, Barbara. "How Large Are Teacher Effects." Educational Evaluation and Policy Analysis 26.3 (2004): 237-57. Web. 4. Khimm, Suzy. "Study: Class Size Doesnt Matter." The Washington Post, 28 Jan. 2012. Web. . 5. Wright, Melanie. "Should Teachers Pursue Master's Degrees?" Princeton-Brookings, The Future of Children, 20 Aug. 2009. Web. . 6. Carnevale, Anthony. "Not All College Degrees Are Created Equal." Georgetown GPPI, n.d. Web. . 7. WISELI. "Benefits and Challenges of Diversity in Academic Settings." University of Wisconsin-Madison, n.d. Web. . 8. RAND Education. "Effect of Teacher Pay on Student Performance." RAND Corporation, 2006. Web. . 9. Abowd, John. "High Wage Workers and High Wage Firms." Econometrica 67.2 (1999): 251-333. Web. Figure 1 Data from California SAT-9 Tests and reading level, compiled by supportingevidence.com Figure 2 Data from United Nations education survey, compiled by thestranger.com

teacher effects on educational achievement

Documents