using research on employees' performance to study the effects of teachers on students'...

Using Research on Employees' Performance to Study the Effects of Teachers on Students'AchievementAuthor(s): Brian Rowan, Fang-Shen Chiang and Robert J. MillerSource: Sociology of Education, Vol. 70, No. 4 (Oct., 1997), pp. 256-284Published by: American Sociological AssociationStable URL: http://www.jstor.org/stable/2673267 .

Accessed: 05/07/2014 13:40

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

American Sociological Association is collaborating with JSTOR to digitize, preserve and extend access toSociology of Education.

http://www.jstor.org

This content downloaded from 86.20.100.12 on Sat, 5 Jul 2014 13:40:26 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=asa

http://www.jstor.org/stable/2673267?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Using Research on Employees' Performance to Study the Effects of Teachers on Students'

Achievement Brian Rowan

University of Michigan Fang-Shen Chiang

National Chi Nan University, Taiwan Robert J. Miller

University of Michigan

The study reported here used general ideas about employees' performance to develop and test a model of teachers' effects on students' achievement in mathematics using data from the longitudinal files of the National Education Longitudinal Study of 1988 (NELS:88). A general model of employees' performance suggests that the effects of teachers on students' achievement can be explained by three general classes of variables: teachers' ability, motivation, and work situation. This article dis- cusses how these general classes of variables can be operationalized in the NELS:88 data set and presents estimates of models of the combined effects of these classes of variables on students' achievement. The analyses revealed that teachers' knowledge of subject matter and expectancy motivation have direct effects on students' achievement in mathematics and that the size of these effects depends on the average levels of ability of students in a school.

D ecades of research on teaching suggest that after relevant characteristics of students are con-

trolled, teachers' effects on students' achievement can be attributed to three general classes of variables: teaching ability, defined in terms of teachers' knowledge of subject matter and teaching strategies; teachers' motivation, usually defined by such constructs as teachers' efficacy, locus of control, and outcome expectancies; and the school and classroom situations in which teachers work, including such factors as class size, instructional grouping arrangements, time allocations, and the extent to which schools have been "restructured" to provide teachers with appropriate control over working conditions and/or support from colleagues.

Although the importance of these general classes of variables to effective teaching is widely recognized, educa-

tional researchers frequently study the effects of each class of variables on students' achievement in isolation from the others. As a result, little is known about how teachers' ability, motivation, and work situations combine to produce students' achievement. In this article, we use a general model of employees' performance, drawn from the literature on organizational-industrial (I-0) psychology, to address this issue. This model views employees' performance (measured in terms of some specific outcome) as a complex function of employees' ability, motivation, and work situations. At its simplest, the model suggests that ability, motivation, and work situations have direct and additive effects on employees' performance. In a more complex form, however, it would suggest that the three general classes of variables just men- tioned interact to produce employees'

256 SOCIOLOGY OF EDUCATION 1997, VOL. 70 (OCTOBER): 256-284



Effects of Teachers on Students' Achievement 257

performance (for a discussion, see Campbell 1990).

Multiplicative models of employees' performance have intuitive appeal. For example, if the performance of tasks is a function of ability times motivation ([P = flA * M)]), employees with high ability would not perform well unless they were also highly motivated, and highly motivated employees would not perform well unless they also had high levels of ability. A multiplicative model would also view the effects of ability and motivation on job performance as varying across work situations (P = flA * SI or P = fIM * SI). For example, ability and motivation may have particularly large effects when tasks are complex and uncertain or when employees (rather than superiors) control work processes.

In our study, we applied these general ideas about employees' performance to the case of teaching. Clearly, teaching performance is a multidimen- sional construct. However, a long line of research in I-0 psychology suggests that there is little payoff to searching for a single or "ultimate" criterion that sum- marizes an employee's performance across all dimensions of a job (P. Smith 1976). As a result, we examined teaching performance in terms of a single outcome: students' achievement. In particular, we posed the following research questions:

1. How have educational researchers conceptualized and measured the general constructs of teachers' ability and motivation?

2. Are there direct effects of teachers' ability and motivation on students' achievement?

3. Is there any evidence that teachers' ability and motivation interact to produce students' achievement? That is, are the relationships of ability and motivation to students' achievement additive (P = flA + MI) or multiplicative (P = flA *

MI)? 4. How do work situations affect

teaching performance? Are the effects additive (P =JIA + M + S]), or do different situations make teachers' ability and motivation more salient or less salient to

job performance (P = JIA * SI, P = JIM *

S])? We examined these questions with

reference to previously conducted large- scale, quantitative research on the effects of teachers and teaching on students' achievement. In this article, we first review the different ways educational researchers in this research tradition have conceptualized and measured the constructs of teachers' ability, motivation, and work situations. We then discuss the issue of additive versus multiplicative effects in models of employees' performance, paying particular attention to the way this problem has been addressed in arguments about school restructuring. Finally, we present our study, which used data from the longitudinal data files of the National Education Longitudinal Study of 1988 (NELS:88) to investigate the effects of teachers' ability, motivation, and work situations on loth-grade students' mathematics achievement.

PREVIOUS RESEARCH

There are striking parallels between research on teachers' performance and research on employees' performance in general.

Teaching Ability

Consider research on the relationship of teaching ability to students' achievement. In I-0 psychology, a large body of research on personnel selection and assessment has examined the relationship of employees' ability to job performance (for reviews, see Campbell 1990; Hunter and Hunter 1984; M. Smith and George 1992). In this literature, researchers have usually represented abilities either as some general factor- often a measure of general intelligence, for example, IQ-or in terms of one or more specific factors. An emerging theme in personnel psychology is the idea that specific abilities, such as those assessed in job-sample tests, have a greater power to predict job performance than do more general measures of ability, including general measures of



258 Rowan, Chiang, and Miller

intelligence (Prediger 1989; M. Smith and George 1992).

Research on teaching effectiveness also has tended to focus on specific abilities related to teaching performance (Darling-Hammond, Wise, and Klein 1995; Shulman 1986; Sternberg and Horvath 1995). Although many specific factors have been identified, we focus on two specific factors in this article: teachers' knowledge of subject matter and teachers' use of appropriate teaching strategies.

Subject-matter knowledge. Beginning in the late 1980s, research on teaching increasingly began to emphasize the importance of teachers' subject-matter knowledge to teaching performance (see Shulman 1986). The rationale for this emphasis is straightforward: A deep knowledge of the subject being taught can support teachers in both the planning and interactive phases of teaching. In planning for instruction, subject-matter knowledge supports the development of good lesson structures that orga- nize and sequence instruction around important concepts and related opera- tions. Moreover, subject-matter knowledge supports the interactive phases of instruction, acting as a resource as teachers formulate explanations and examples, both in lectures and in response to students' questions (Byrne 1990; Leinhardt and Smith 1985). Thus, Sternberg and Horvath (1995:11) argued: "First, and most obviously, expert teachers must have content knowledge-knowledge of the subject matter to be taught."

Many researchers are attempting to link teachers' subject-matter knowledge to teaching effectiveness in qualitative studies of small samples of teachers (see, for example, the studies cited in Wilson and Wineburg 1993:733). Although we applaud these in-depth analyses, the data analysis reported here was built on an older line of research grounded in production-function analyses of school effectiveness (for a review, see Hanushek 1986). This literature has used crude measures to assess the effects of teachers' subject-matter knowledge on students' achievement,

but it has done so using large-scale, often nationally representative, data sets and therefore offers a useful balance to the small-scale studies being conducted in research on teaching.

The production-function research reviewed here includes two lines of research. First, beginning with the Coleman report (Coleman et al. 1966), a number of studies have examined the relationship of teachers' verbal ability to students' achievement. Hanushek's (1986) review of production-function research showed that many (but not all) of these studies found a statistically significant, positive relationship between the verbal ability of teachers and the achievement of their students. To some extent, it makes sense to view verbal ability as a general cognitive ability affecting the performance of teachers, and this appears to be the reason that this measure of teachers' ability was included in many production-function studies.

In our study, however, we focused the analysis on more specific abilities, in particular, the knowledge teachers have of the specific subject matter they are teaching. For example. we examined the effects of teachers' mathematics knowledge on students' mathematics achievement using a crude test of teachers' mathematics knowledge included in NELS:88. The emphasis on measures of highly specific, job-relevant knowledge is consistent with recent research in personnel psychology that has emphasized the role of specific factors in job performance. It is also consistent with much recent research on teaching.

A second line of production-function research has attempted to measure teachers' job-relevant knowledge by examining their educational attainment, particularly the type of degrees they earned in college and/or graduate school. Here too, the literature points to the power of specific measures of subject-matter knowledge in predicting teachers' effectiveness. For instance, early production-function studies often examined whether students of teachers with master's degrees had higher levels of achievement than students of




teachers with bachelor's degrees. The general assumption guiding this line of research was that higher levels of education would endow (or select for) teachers with higher levels of knowledge and skills relevant to the work of teaching (Hanushek 1989; Hedges, Laine, and Greenwald 1994).

Recent research on teachers' education, however, has turned from measuring teachers' education in terms of degree levels toward a more detailed analysis of the subjects teachers studied in their educational programs. This research demonstrates that teachers who have taken more courses in the subject matter they are teaching tend to have students with higher levels of achievement, an indication that teachers' specific, subject-matter knowledge is related to teaching performance (Chaney 1995; Monk 1994).

In our study, we followed the ideas developed in I-0 psychology and recent production-function research in examining the effects of teachers' subject- matter knowledge on students' achievement. Following I-0 psychology, we hypothesized that a teacher's specific subject-matter knowledge and specific educational preparation have effects on students' performance.

To assess this specific knowledge, we developed two measures of teachers' subject-matter knowledge. The first was a crude index of teachers' knowledge in mathematics (the subject area studied), drawn from the one-item math quiz included in the NELS teacher survey. Second, we sorted teachers into two groups, those who majored in mathematics at the undergraduate and/or graduate level and those who never majored in mathematics. Although these are crude measures of teachers' job-relevant, subject-matter knowledge, we nevertheless developed the following hypothesizes:

Hypothesis la: Students whose mathematics teachers scored higher on the math quiz will have higher levels of achievement on the NELS math test than will students whose teachers scored lower.

Hypothesis lb: Students whose teachers majored in mathematics at the undergrad-

uate and/or graduate level will have higher levels of mathematics achievement than will students whose teachers did not.

Teaching strategies. In I-0 psychology and research on teaching, assessments of job-relevant abilities extend beyond measures of job-related knowledge. In the case of teaching, for example, one of the most important skills that can affect job performance is the particular teaching strategy a teacher uses. One way to discuss teaching strategies is to draw on the distinction between "direct" and "indirect" approaches to instruction made in early research on teaching (for a discussion, see Darling- Hammond and Snyder 1992).

Direct instruction is a teacher- centered instructional strategy that emphasizes large-group instruction; recitation and drill; and opportunities for controlled, independent practice (Good, Grouws, and Ebmeier 1983; Rosenshine and Stevens 1986). Much evidence suggests that direct instruction works best in teaching curricular content that is well structured, highly sequenced, and focused on tasks that demand rote memorization and/or the repeated application of low-level cognitive routines (Darling-Hammond and Snyder 1992). In contrast, indirect instruction is less teacher centered and more student directed. It can be charac- terized by the use of open-ended and divergent questions, student-initiated discussions, and attention to students' personal experiences. Apparently, indirect instructional strategies are more effective than direct instructional strategies in teaching students higher-order thinking skills that demand deep conceptual knowledge of a subject and the ability to solve complex, multistep problems (Darling-Hammond and Snyder 1992:65).

This brief characterization of two different teaching strategies has important implications for research on teaching effectiveness. In particular, it suggests that the effectiveness of any particular teaching strategy depends on the type of student outcomes used to judge teaching effectiveness. When teaching




effectiveness is judged by students' performance on basic skills tests, a great deal of evidence suggests that direct instructional strategies are effective (for reviews, see Brophy and Good 1986; Rosenshine and Stevens 1986). On the other hand, when teaching effectiveness is judged by students' performance on tasks that demand higher-order thinking, indirect teaching strategies may be more effective.

The contingent relationship between teaching strategies and student outcomes is illustrated in some recent research using NELS:88 data. In an analysis of the content validity of the NELS:88 achievement tests used in our study, Kupermintz, Ennis, Hamilton, Talbert, and Snow (1995) showed that the NELS mathematics tests strongly emphasize higher-order thinking (for example, over half the items on the tests emphasize what the authors called inferential reasoning). Kupermintz et al. also found that when teachers placed more emphasis on higher-order thinking in instruction, their students had higher scores on the NELS mathematics test, even after students' prior mathematics achievement was controlled.

In our study, we built on this analysis by developing a measure of teaching strategy that we called higher-order thinking (HOT) instruction (cf. Raudenbush, Rowan, and Cheong 1993). Our measure assesses the extent to which teachers emphasize a deep conceptual understanding of mathematics and encourage students to think in open-ended ways about the subject. The measure does not completely capture the many dimensions of teaching denoted by the concept of indirect instruction, but it does assess some of them. Moreover, this form of instruction should be an important job-related skill, given the nature of the mathematics tests we used as the outcome variable. As a result, the second hypothesis for our study was this:

Hypothesis 2: Students whose mathematics teachers place more emphasis on HOT instruction will have higher levels of achievement on the NELS mathematics

test than will students whose teachers place less emphasis on HOT instruction.

Teachers' Motivation

In addition to studying various dimensions of teaching ability, educational researchers have also examined the effects of teachers' motivation on students' achievement. In I-0 psychology, many alternative theoretical perspectives on employees' motivation have been developed and tested (for a review, see Kanfer 1990). Currently, 1-0 psychologists (like psychologists in general) are most interested in cognitive theories of motivation, particularly theories derived from Bandura's (1993) work on self-efficacy, attribution theo- rists' discussions of internal and external locus of control (see Rotter 1966; Weiner 1986), and different versions of expectancy theory (see Kanfer 1992; Vroom 1964).

All these perspectives suggest that higher motivation increases the performance of tasks, but each proposes a different source of motivation. In Bandura's (1986) theory, the source is employees' beliefs that they can mount a particular type of performance; in expectancy theories, the source is a belief that performances will lead to val- ued outcomes; and in attribution theories, the source is beliefs about the relative influence of external versus internal sources of performance.

Educational researchers have tended to blend these three, distinct approaches to the study of motivation into a single line of work on teacher efficacy (for a detailed discussion of this point, see Chiang 1996). A careful examination of the many items used in this research, reviewed by Guskey and Passaro (1994), suggests that the items tend to measure both efficacy expectations and outcome expectations. For example, the item, "When I really try, I can get through to the most difficult students" seems to ask teachers to judge whether they are capable of mounting a particular kind of performance ("when I really try," an efficacy expectation) and whether such a performance will affect students'




outcomes ("I can get through to students," an outcome expectation).

A factor analysis by Guskey and Passaro (1994) also showed that the items typically used to measure teacher efficacy load onto two dimensions, which they labeled teachers' sense of self-efficacy and teachers' sense of teaching efficacy. In our view, these dimensions appear to assess teachers' beliefs about locus of control. For instance, an item measuring self-efficacy is, "When the student does better than expected, many times it is because I exert effort." An item measuring teaching efficacy is, "When it comes right down to it, a teacher really can't do much because a student's home envi- ronment is a large influence on his/her achievement." The first item seems to suggest an internal locus of control and the second, an external locus of control.

Given the ambiguities in items typically used to assess teachers' motivation, we assumed that past research on teacher efficacy assessed teachers' general motivational force with respect to the task of classroom instruction. That is, we assumed that teachers who respond positively to the usual array of efficacy items have generally high efficacy and outcome expectations and a high internal locus of control. Efficacy theory, expectancy theory, and attribution theory would all argue that these teachers have higher levels of motivation and should be more effective in teaching. Unfortunately, there is little strong evidence that teacher efficacy of this sort actually has positive effects on students' achievement. The best study of this issue, for example, by Ashton and Webb (1986), examined the effects of teacher efficacy on students' achievement after controlling for measures of teaching strategies. It found that support for the proposition that teacher efficacy affects students' achievement was mixed.

An alternative approach to the study of teachers' motivation has better empirical support-the line of research on teachers' educational expectations for students. Beginning with the seminal work of Rosenthal and Jacobson (1968),

teachers' expectations for students have been studied extensively (for a review, see Brophy 1983). In brief, this research suggests that teachers devote more effort and use different and more effective teaching strategies when they teach students whom they expect to learn more. This view is, of course, consistent with general ideas about outcome expectancies (that outcome expectancies are associated with job performance) and even demonstrates how motivational states based on outcome expectancies get translated into successful outcomes through employees' behaviors and cognitions.

It is worth noting that the research on teachers' expectations for students dif- fers in one important respect from the research on teacher efficacy, which also assesses teachers' outcome expectations. As we discussed earlier, items used to measure teacher efficacy appear to measure generalized expectations about teaching outcomes (outcome expectations aggregated across multiple students and/or classes), whereas studies of teachers' expectations tend to focus on teachers' expectations with respect to specific students. There is evidence that high school teachers' expectations and motivational states vary across the students and classes they teach (Raudenbush et al. 1992) and that, as a result, teachers use different instructional strategies in different classes and with different students (Brophy 1983; Raudenbush et al. 1993). Thus, it may be better to assess teachers' outcome expectancies for particular students, rather than in general. In fact, the specificity of measurement in research on teachers' expectations may account for its better empirical support.

In summary, we discussed two approaches to studying work motivation in research on teaching. One approach, found in the literature on teacher efficacy, appears to have developed measures of teachers' general state of motivation with respect to teaching; the other assesses teachers' specific outcome expectations for particular students. On the basis of these approaches,




we developed the following hypotheses for our study:

Hypothesis 3a: Students whose teachers have generally high efficacy and outcome expectations and an internal locus of control will have higher levels of mathematics achievement than will students whose teachers have generally low efficacy and outcome expectations and an external locus of control.

Hypothesis 3b: Students whose teachers have higher outcome expectations for them will have higher levels of mathematics achievement than will students whose teachers do not hold such expectations for them.

Effects of Work Situations

To this point, we have argued that variables measuring employees' ability and motivation can have important effects on job performance. However, there has been much debate in the behavioral sciences about the relative importance of personal characteristics like ability and motivation versus situational variables in the deter- mination of employees' performance (Peters and O'Connor 1980; Pfeffer 1991; Staw 1985). To address this debate, researchers must incorporate measures of work situations into models of employees' performance. The simplest way to do so is to treat the situations in which employees work as having additive effects on employees' performance, that is, to assume that employees' performance is an additive function of employees' ability, motivation, and some vector of organizational or situational variables.

This is the treatment that one usually finds in research on organizational effectiveness. In this literature, researchers simply assume that in addition to having motivated and skilled employees, organizations perform better when they use more efficient technologies or have better organizational designs. A similar treatment can be found in the literature on school effectiveness, in which variables measuring the organizational structure and climate of schools are assumed to work in

addition to variables like teachers' expectations in producing students with high levels of achievement.

An emerging line of research in I-0 psychology, however, examines how work situations interact with ability and motivation to affect job performance. For example, Kanfer (1990) described a number of experiments in I/O psychology that were designed to examine whether motivational effects on performance were more salient under some task conditions than under others.

Research on work restructuring in business and industry has pursued a similar line of investigation. In this area, studies have found that when organizations delegate decision making and coordination to line employees, employees' performance comes to depend more heavily on employee's skills and motivation than when technologies are more mechanistic and organizational designs are more centralized. In his discussion of "high commitment" work systems, for example, Walton (1980) stated that the effects of ability and motivation on employees' performance are seen to vary across situations. When tasks are complex and organizations are decentralized and collegial, differences in employees' motivation and ability are thought of as having especially strong effects on performance. However, when tasks are simple and organizations emphasize centralized planning and control, these differences are considered to have weaker effects.

A similar line of argument has developed in recent discussions of school restructuring. Although many researchers and policy analysts assume that school restructuring will have direct effects on students' achievement, another theme in this literature is that school restructuring will make better use of the available expertise and motivation of teachers. One way to interpret this latter theme is as an argument about the multiplicative effects of teachers' ability, motivation, and school restructuring on students' achievement. In the multiplicative argument, teaching is seen as a complex task that requires high levels of skills and motivation for




successful performance (cf. Rowan 1994), and an effective form of school organization is one that capitalizes on teachers' abilities and motivation to improve their performance. In recent years, a consensus has begun to emerge that bureaucratic and centralized controls over teaching suppress the effects of teachers' skill and motivation on students' achievement, while organizational designs that feature decentralized decision making, high levels of staff cooperation and interaction, and supportive leadership allow skilled and motivated teachers to be more effective (for a review of these arguments, see Rowan 1990).

To date, two general forms of this argument have appeared. The first bor- rows heavily on the idea that teaching is a complex and professionalized line of work that is best managed through 6"organic" forms of management (Rowan 1990). This line of work tends to emphasize a form of school restructuring that promotes teachers' control over decision making, staff collegiality and collaboration, and supportive leadership (Rowan, Raudenbush, and Kang 1991). In most cases, these managerial processes are seen as having direct effects on student outcomes. But it is also possible to argue that they enhance the capacity of able and motivated teachers to perform their work well, in which case, the argument implies that the effects of teachers' ability and motivation are increased as organizations decentralize decision making and develop collegial and collaborative forms of work and supportive patterns of administrative leadership.

The second line of argument dis- cusses the organizational conditions under which complex forms of teaching (such as indirect teaching strategies focused on higher-order thinking) can be made more effective. It advances the idea that an emphasis on deep conceptual understanding, student- centered forms of classroom instruction, and open-ended forms of classroom dis- course cannot be effective under current school conditions. Indeed, the effectiveness of these innovative forms of instruction is seen to be constrained by

the lockstep schedule of schools, by dis- ciplinary specialization, and by the lack of opportunity for teachers to interact with one another about effective ways to teach. In this argument, reformers call for school restructuring that involves more flexible scheduling, the greater use of interdisciplinary teams, and common planning periods. Although these forms of school restructuring may have direct effects on students' achievement (see, for example, Lee and Smith 1994), they are also assumed to make indirect forms of instruction more effective (see, for example, Elmore 1995).

In our study, we tested these lines of argument in several ways. First, following previous research, we developed two kinds of measures of school restructuring. The first set of measures have been used extensively by Rowan and colleagues to assess the extent to which teachers have control over instructional decision making, cooperate with staff to solve instructional problems, and experience support from the principal and other administrators (Raudenbush et al. 1992, 1993; Rowan et al. 1991; Rowan, Raudenbush, and Cheong 1994). The second set of measures come from Lee and Smith's (1994) study of restructuring practices in high schools. Using NELS:88 data, Lee and Smith listed organizational practices that were rare in high schools, which they labeled restructuring practices. In our study, we examined these restructuring practices, among others-the use of common planning periods, interdisciplinary teams, and flexible scheduling-and investigated the following hypotheses about them:

Hypothesis 4a: Students in schools that are engaged in restructuring have higher levels of mathematics achievement than do students who are not in such schools.

Hypothesis 4b: In schools that are engaged in restructuring, the effects of teachers' ability, motivation, and HOT instruction on students' achievement are greater than in schools that are not engaged in school restructuring.




DATA AND PROCEDURES

We examined the hypotheses just developed with public-use data from the NELS:88 first follow-up data files (Ingels, Scott, Lindmark, Frankel, and Meyers 1992a, 1992b), restricting the analysis to data on students' achievement in mathematics, where the data required to test our arguments were available. The NELS first follow-up data sets allow a longitudinal analysis of growth in students achievement from the 8th to the loth grade. The files also include summary data on relevant student, teacher, and school characteristics that may affect students' achievement during this period.

Sample

We selected the student sample for the analysis after we applied five data filters. First, we examined only data on students' achievement in mathematics. Second, we dropped all student cases that lacked scores on both the 8th- and 10th-grade NELS mathematics tests. Third, we included only students for whom data were collected from their mathematics teachers. Fourth, we dropped all student cases from schools with fewer than five NELS-sampled students. Finally, given the small number of non-Catholic private schools in the sample, we dropped all student cases from these schools. This process left us with a sample of 5,381 students attending 410 schools (382 public schools and 28 Catholic schools).

In a series of analyses, we compared the characteristics of students and schools that were excluded from the analysis with the those that were included. In the original first follow-up sample, 6,913 students could have been included in a longitudinal analysis of mathematics achievement. The original sample also included data on 1,698 schools. Compared to the excluded students, the students who were included in the study were from slightly higher socioeconomic (SES) backgrounds and had slightly higher levels of achievement in the 8th- and 1oth- grades. However, the included and

excluded students were equally likely to be female and to come from minority backgrounds. In the analyses reported here, we adjusted the NELS sample weights in light of missing data, and all regression analyses reported here use these weights.

Compared to the original sample of 1,698 schools, the study sample of 410 schools contained greater proportions of larger schools and higher percentages of public and Catholic schools. As expected, the schools also had higher average levels of achievement and average SES. Since no sample weights were provided for schools in the NELS first follow-up data, we did not adjust for the likelihood of schools being selected in the sample in our analyses.

Measures

In studying the effects of teachers' ability, motivation, and work situations on students' achievement, we controlled for a large number of student- and school-level variables. With these variables controlled, we then focused on a longitudinal analysis of students' achievement, adjusting 1 Oth-grade achievement in mathematics for prior achievement in mathematics and other academic subjects. With prior achievement and other variables controlled, we examined the effects of our various measures of teachers' ability, motivation, and work situations on students' achievement. The variables used in the analysis are discussed next. Appendix A presents more details on the variables, and Appendix B shows the means and standard deviations of the variables, as well as correlations among variables.

Measures of students' achievement. The outcome variable in the analysis was students' IRT-adjusted score on the NELS 1Oth-grade mathematics achievement test. In the analysis, we used standard scores, with a mean of 0 and a standard deviation (SD) of 1. The NELS 10th-grade mathematics test had three forms (high, medium, and low), each with 40 items. Each form gave different emphases to the content areas of arithmetic, algebra, geometry, data-




probability, and advanced topics and to items requiring skill-knowledge, understanding-comprehension, and problem solving (Ingels et al., 1992a, 1992b). In general, the tests contained more items that required understanding, comprehension, and problem solving than those that required simple skills and knowledge. They also placed greater emphasis on arithmetic and algebra than on other topics. The analyses reported in Kupermintz et al. (1995) suggest that higher scores on the NELS tests reflect the mastery of higher-order skills, such as conceptual understanding and problem-solving ability. The 10th-grade mathematics test had a reported reliability of .93 (Ingels et al. 1992a, 1992b).

In the analysis, we controlled for students' prior achievement in two ways. First, we included students' IRT- adjusted score on the 8th-grade NELS mathematics test, a single, 40-item mathematics test with a reported reliability of .89 (NCES 1995). We used a standard score for this variable. In addition, we included a measure that we called initial academic status, which was a standard score constructed from the mean of a student's IRT-adjusted scores on three other 8th-grade tests- reading, science, and social studies. The use of both prior mathematics achievement and prior achievement in other academic subjects gave us a stringent set of controls on students' ability and prior achievement in the prediction of students' achievement in the 10th grade.

Students' course taking and track placement. Our analysis also controlled for students' opportunity to learn the content tested on the NELS mathematics achievement test. To control for opportunity to learn, we first developed two measures of students' course taking in mathematics in the 9th- and loth- grades-high math and low math-both of which were based on students' reports. The high math variable is a count of the number of courses students reported taking from among Algebra I, Algebra II, geometry, and trigonometry, and the low math variable is a count of the number of courses students reported

taking from among prealgebra, business math, general math, and other math. The high- and low math variables each had a possible range of 1 to 4 courses, and scores on the two scales are negatively correlated (r = -.45).

Another measure of students' opportunity to learn was drawn from teachers' reports about the track assignments of the mathematics courses students were taking at the time of the first follow-up survey. The original item had five possible categories: advanced-honors track, academic track, general track, vocational-technical-business track, and other. To conserve cases, we included students whose teachers' failed to report the track assignments of classes. In the analysis reported later, we included four dummy-coded variables to represent a student's track placement (advanced-honors, alcademic, missing, and other). The reference category in these analyses combines students who were assigned to the general and vocational-technical-business tracks.

Other student background variables. We also included several other measures of students' background characteristics: students' sex (coded 0 = female, 1 = male) and students' SES, a standard score combining data on parents' education, parents' jobs, and family income. In addition, we controlled for different levels of motivation among students by including students' educational expectations, a dummy-coded variable (coded 0 = student does not expect to go to college and 1 = student expects to go to college). In the study sample, 73 percent of the students expected to go to college.

Measures of teachers' ability. Three measures of teachers' ability were taken from the teacher data files and then assigned to all the students taught by a given mathematics teacher. The first measure was a teacher's score on the math quiz included in NELS:88. The NELS teacher questionnaire contained a single item that was designed to measure teachers' mathematical knowledge (see Appendix A). In the analysis, we coded teachers' responses as 0 = incorrect and 1 = correct. Among the 2,077 mathematics teachers who




answered this item, 69 percent answered it correctly.

Clearly, this one-item measure is a less-than-adequate representation of teachers' content knowledge in mathematics for a number of reasons. It does not broadly sample teachers' knowledge in this domain and in all likelihood lacks reliability. Nevertheless, there is some evidence of the measure's validity. In a preliminary analysis, we found that 56 percent of the teachers who had never majored in mathematics as undergraduate or graduate students, compared to 75 percent of those who did major in mathematics, answered the item correctly.

The second measure of teachers' ability was a dummy-coded variable indicating whether or not a teacher had majored in mathematics in undergraduate and/or graduate school. In an earlier analysis, Chiang (1996) found that teachers who majored in mathematics at the undergraduate and/or graduate level made a significant difference in students' achievement in the 10th-grade sample. However, teachers with graduate degrees in mathematics did not have students with higher levels of achievement than their colleagues with bachelor's degree in mathematics. Therefore, in our study, the variable was coded 0 = never majored in mathematics and 1 = majored in mathematics at the undergraduate and/or graduate level.

The final measure of teachers' ability focused on mathematics teachers' use of particular instructional strategies. We created a variable called teachers' emphasis on HOT instruction. This variable, taken from the teacher data file, is a standardized scale score constructed from the average score of five items asking teachers to report the amount of emphasis they placed on (1) understanding the logical structure of mathematics; (2) understanding the nature of proof; (3) knowing mathematical facts, principles, and algorithms; (4) thinking about what a problem means and ways it may be solved; and (5) understanding mathematical concepts. Kupermintz et al. (1995) found that the items in this scale were

correlated with students' scores on items in the NELS achievement test that assess the mastery of higher-order skills, and Raudenbush et al. (1993) found that teachers were more likely to emphasize these aspects of instruction when they were teaching high-track, rather than low-track, mathematics classes. Thus, overall, previous studies suggested that a scale containing these items is a valid measure of HOT instruction. In our study, we assigned the score obtained on this scale by a given mathematics teacher to all the students whom the teacher taught.

Measures of teachers' motivation. We developed two measures of teachers' motivation for this study using data from the teacher files. The first measure, which we called teachers' general force of motivation, was based on teachers' responses to 12 items that are typically used to measure teachers' sense of efficacy. As we discussed earlier, the items appear to measure three closely related constructs-teachers' efficacy expectations, teachers' outcome expectations, and teachers' locus of control. A factor analysis showed that all the items loaded on a single factor that accounted for 26 percent of the variance in the items, and a scale combining these 12 items had an internal consistency (Cronbach's alpha) of .73. In our analysis, the scale is a standardized score based on the average response of teachers to the 12 items. We assigned the scale score for a given mathematics teacher to each of the students whom the teacher taught.

The second measure of teachers' motivation was teachers' expectations for students. This measure was taken from an item in the teacher questionnaire asking teachers to report whether or not they thought a particular student would go to college. The original response choices were "yes,99 "no,99 or "don't know." After we assigned the teachers' responses to the appropriate students, we found that teachers expected 75 percent of the students in the sample to attend college. In our view, this variable is a useful measure of a teacher's expectancy motivation with




respect to a particular student. That is, it reflects teachers' beliefs about the likelihood of given students' learning from teaching. However, it is possible that the measure also reflects teachers' assessments of given students' financial ability to attend college.

School restructuring variables. We developed two sets of variables to measure school restructuring. The first set was based on teachers' responses to a series of questionnaire items used previously by Rowan and his colleagues (Raudenbush et al. 1991; Rowan et al. 1991, 1993). These items assess the extent of teachers' control over decision making, staff collaboration, and supportive leadership in teachers' work environ- ments. Each of these constructs was assessed by a multi-item scale (for the specific items in each scale, see Appen- dix A). Since previous psychometric analyses showed that there is significant within-school variation in these scales, we included these scales at two levels of analysis. First, mathematics teachers' scale scores on these variables were assigned to each of the students they taught to represent within-school variation in teachers' perceptions of teachers' control, staff cooperation, and school leadership. Second, the scale scores for all teachers in a given school were averaged, and a school-level score was included in the analysis. In our study, both the student-level scores and the school-level scores were standardized.

A second set of variables were drawn from principals' responses about restructuring practices in NELS schools. Using the data reported in Lee and Smith (1994), we included four school characteristics from their larger list of school restructuring practices: (1) the use of interdisciplinary teams, (2) the presence of a school-within-a-school structure, (3) the use of flexible class scheduling, (4) and the use of common planning periods for teams of teachers. Each of these variables could take on one of three values-the practice was present in a school, the practice was absent in a school, or data on the presence or absence of the practice were missing. In the analysis, we included

two dummy-coded variables for each practice, one indicating that the practice was present, the other that data were missing. The reference category was cases in which the practice was absent. A possible shortcoming of these measures is that they do not refer to specific practices that occur in mathematics departments. Instead, they are measures of whether or not restructuring practices occur anywhere in a school.

Other school-level control variables. Finally, the analysis included three variables measured at the school level: school sector (public or Catholic), average student ability (the average score on the NELS eighth-grade reading, science, and social studies tests for all the NELS-sampled students in a school), and school enrollment (a measure of student enrollment broken down into nine categories).

Procedures

The data were analyzed using the statistical computing package HLM/2L (Bryk, Raudenbush and Congdon 1994). Thus, the statistical model we used was a two-level HLM with data on students included at Level 1 and school- level data included at Level 2 (for a discussion of HLMs, see Bryk and Raudenbush 1992). An analysis of NELS data using a two-level HLM analysis is less than perfect for estimating teachers' effects on students' achievement. In the NELS first follow-up study, teachers were included because one or more of the students they taught were in the NELS sample. As a result, our study did not include data on a representative sample of teachers, a representative sample of the classes taught by a given teacher, or a representative sample of students in these classes. Rather, we were forced to estimate teachers' effects on students' achievement by assigning the responses of teachers about their own motivation, ability, and work situations to each of the students they taught. Because a given teacher frequently taught more than one NELS student in a school, the student data cannot be assumed to be statistically




independent, a situation that violates a basic assumption of regression analysis. However, this violation is not severe in these data. The modal mathematics teacher in this study taught only one student, and 78 percent of the teachers in the sample taught three or fewer students. Nevertheless, the reader is advised that the lack of statistical inde- pendence at the student level has the potential to shrink the standard errors of estimates of teachers' effects on students' achievement artificially, thus biasing the results toward finding statistically significant effects.

At the same time, the data also have the potential to underrepresent the overall magnitude of teachers' effects on students' achievement, mainly because of the period in which the data were collected. NELS first follow-up data were collected during the second half of the students' sophomore year in high school. However, the statistical model used here assesses the growth in students' achievement over two years, from the end of the 8th grade to the end of the loth grade. In the NELS design, the teacher of a given NELS student may have been teaching him or her for less than a semester, but our study estimated the growth in achievement across approximately four semesters. Under these conditions, it is not unrealistic to assume that we underrepresented teachers' effects on students' achievement, since our measure assesses the effect of only one of several mathematics teachers that a student could have studied with during the period of the study.

Despite these shortcomings, the data and procedures we used have a number of advantages for studying teachers' effects on students' achievement. First, the NELS data allowed us to develop a fairly complete set of controls for students' social background, prior achievement, and opportunity to learn. As a result, our estimates of teachers' effects on students' achievement should not be biased by poor model specification. Second, by using NELS data, we were able to conduct one of the few studies that included multiple measures of teachers' ability and teachers' motiva-

tion within the framework of the same analysis. Third, by estimating teachers' effects on students' achievement within the framework of a two-level HLM, we avoided the problems that Bryk and Raudenbush (1992) argued are present in purely between-students and purely between-schools analyses of data on students' achievement. Moreover, the hierarchical analysis allowed us to model directly hypotheses about the varying effects of teachers' ability and motivation on students' achievement in different schools and thus to incorporate sys- tematically an analysis of school restructuring into research on teachers' effects on students' achievement.

RESULTS

The analysis of data proceeded in several stages. In the exploratory stage, we used simple correlations and ordinary least- squares (OLS) regression analyses to build an adequate model of teachers' effects on students' achievement. After completing these preliminary analyses, we used the statistical program HLM/2L (Bryk et al. 1994) to estimate a two-level HLM and test our hypotheses about the effects of teachers' ability, motivation, and school restructuring on students' achievement. In the third and final stage, we trimmed this model so it included only variables that had statistically significant effects in the first round of HLM analyses and used this trimmed model to explore the various interaction effects of interest to the study, particularly interactions among school restructuring variables and variables measuring teachers' ability and motivation.

Preliminary Analysis

We followed previous "school process" models in developing the analytic model for this study (for a review, see Rowan 1995). These models investigate the different educational treatments received by students from different social and academic backgrounds and the effects of such treatments on student outcomes. As in previous research, we found a moderate correlation between students' SES and




students' achievement before the students entered high school (r = .41, p < .000). Students' subsequent treatment in high schools, however, appeared to be more highly correlated with prior achievement than with SES. For example, 8th-grade mathematics achievement had a correlation of .38 with placement in the advanced mathematics track in loth grade, a correlation of .51 with taking high-math courses, and a correlation of -.51 with taking low-math courses. In contrast, correlations between SES and these variables were much lower, usually in the range .20 to .30 (see Appendix B).

Initial HLM1,2L Model

After conducting these preliminary analyses, we developed a two-level HLM that included all the measures developed for the study. In this phase, we were interested in testing the broad hypothesis that teachers' ability, teachers' motivation, and school restructuring have direct effects on students' achievement, when students' prior achievement, track placement, previous course taking, and other student- and school-level variables are controlled.

The results of the HLM analysis are presented in Table 1. In the analysis, only the intercepts in the Level 1 model are assumed to vary across schools; the other student-level coefficients are fixed. All the student-level variables were centered around their respective grand means, and all analyses were weighted at the student (but not the school) level.

Effects of student-level control variables. As Table 1 shows, prior student achievement in mathematics and in other subjects had strong positive effects on students' mathematics achievement in the loth grade. The effects of the various opportunity-to-learn variables (the tracking and course-taking variables) were also strong and statistically significant. In fact, the t-statistics show that the effects of prior achievement and opportunity-to-learn variables were quite large, especially in comparison to other effects in the model.

Other characteristics of students also had significant effects on mathematics achievement. Boys, for example, had more growth in mathematics achievement during the first two years of high school than did girls, and students who expected to go to college outperformed those who did not. Among the student- level control variables, only students' SES failed to have a statistically significant effect on achievement.

Effects of variables measuring teachers' abilities. The results also indicate that variables that measure teachers' content knowledge affect students' achievement. As Table 1 shows, students whose teachers answered the math quiz item correctly had higher levels of achievement than did those whose teachers answered the question wrong, although the difference in achievement across these groups was only about .02 SD in the test score. In addition, students who were taught by a teacher with a degree in mathematics had higher levels of achievement in mathematics, although here, too, the effect was quite small-about .015 SD for students whose teachers majored in mathematics.

The results in Table 1 do not confirm the hypothesis that teachers' emphasis on HOT instruction affects students' mathematics achievement. But there are reasons to be cautious about this finding. The zero-order correlation of HOT instruction with students' achievement in 10th-grade mathematics is 0.35, which indicates that current mathematics achievement and HOT instruction covary as predicted. In exploratory analyses, however, we found that once the high- and low- math variables were entered into an OLS regression model, the estimated relationship between HOT instruction and students' achievement became statistically insignificant.

One possible explanation for this finding is that HOT instruction and taking high- and low-math courses measur- e the same underlying construct- opportunity to learn the content on the NELS achievement tests. Alternatively, since course work and instructional strategies are correlated, course-work variables may present a better record of




Table 1. Preliminary Two-Level HLM Regression Analysis: lOth-Grade Mathematics as the Dependent Variable (N= 410 schools, 5,381 students)

Regression Independent Variable Coefficient SE t-ratio p-value

School-Level Model School = public -0.074344 0.041107 -1.089 0.070 School enrollment 0.018010 0.009626 1.871 0.061 Average SES -0.010525 0.013294 -0.792 0.429 Average ability 0.134136 0.012777 10.498 0.000 Supportive leadership -0.007727 0.01223 -0.631 0.528 Teacher control 0.020161 0.011080 1.82 0.068 Staff cooperation 0.007930 0.011959 0.663 0.507 Common planning time 0.064060 0.025477 2.514 0.012 Missing data for CPT 0.001458 0.092867 0.016 0.988 Flexible scheduling -0.003876 0.031779 -0.122 0.903 Missing for FS -0.000955 0.075671 0.013 0.990 Interdisciplinary teams -0.022692 0.021208 -1.070 0.285 Missing for IT 0.054328 0.163856 0.332 0.740 School-within-a-school -0.011544 0.024910 -0.463 0.643 Missing for SWAS -0.067566 0.130460 -0.518 0.604

Student-Level Model Grade 8 math score 0.485152 0.011866 40.886 0.000 Educational background 0.121262 0.010802 11.226 0.000 Student SES -0.012522 0.008629 -1.451 0.147 Sex = male 0.022986 0.006871 3.345 0.001 Student expectation 0.023688 0.009050 2.618 0.009 Advanced track 0.078278 0.013768 5.685 0.000 Academic track 0.059801 0.008675 6.893 0.000 Other track -0.044946 0.018878 -2.381 0.017 Track missing 0.071528 0.016225 4.409 0.000 Number of high-math classes 0.063560 0.009180 6.923 0.000 Number of low-math classes -0.077492 0.008359 -9.270 0.000 Teachermath quiz score 0.019628 0.008070 2.432 0.015 Degree in mathematics 0.015010 0.007799 1.925 0.054 HOT instruction -0.007751 0.007719 -1.004 0.316 General force of motivation -0.005925 0.007380 -0.803 0.422 Teacher expectation 0.067727 0.008656 7.825 0.000 Supportive leadership 0.014361 0.009778 1.469 0.142 Staff cooperation -0.006628 0.009432 -0.671 0.502 Teacher control 0.007106 0.008339 0.852 0.394

students' cumulative histories of instruction than does a onetime measure of HOT instruction taken during the second semester of high school. In our preliminary analyses, for example, we found that high-achieving students who had taken more advanced mathematics courses were more likely to have teachers who emphasized HOT instruction, and from this perspective, it is possible that the course-taking variables measure a student's cumulative exposure both to curricular content and HOT instruction.

Effects of variables measuring teachers' motivation. The findings presented in Table 1 also provide mixed support

for hypotheses about the effects of teachers' motivation on students' achievement. On the one hand, the data show that teachers' general force of motivation had no effect on students' achievement in mathematics. On the other hand, teachers' expectations for specific students did have a statistically significant effect on students' achievement. As Table 1 shows, this effect was small. Students whose mathematics teachers expected them to go to college outperformed students whose teachers did not expect them to go to college by about .07 SD.

It is possible that the effects of teachers' expectations on students'




achievement are artifactual. For example, the variable teachers' expectations may simply reflect teachers' accurate assessments of students' ability and thus be a surrogate measure for students' ability. This argument is made less plau- sible, however, by the stringent controls for prior achievement used in the study.

The alternative interpretation is that this is a real effect of teachers' expectancy motivation on students' achievement. To the extent that this interpretation is valid, the finding suggests that a teacher's student-specific expectancy motivation has a greater effect on students' achievement than a teacher's general force of motivation. This interpretation makes good sense. A teacher's general force of motivation would be based on the teacher's expectancies averaged across all the students with whom he or she had instructional contact, and this average may not represent well a teacher's force of motivation with respect to any given student.

Effects of school-restructuring variables. Table 1 also shows the effects of the two kinds of restructuring variables on students' achievement. Only 2 of the 10 restructuring variables have statistically significant effects on students achievement, and both variables appear in the school-level model. At this level of analysis, the data show that students who attended schools where teachers reported more control over decision making and shared common planning periods had higher levels of mathematics achievement. These effects, although not particularly strong, do make sense. Rowan (1990) argued that two key fea- tures of school restructuring are decentralized decision making and opportunities for teachers to collaborate with one another over instruction. The two restructuring variables that have significant effects in the current analysis seem to tap these dimensions of organizational structure and process. As a result, despite the mixed findings and small effects, we think the data provide some support for the idea that students achieve more in restructured schools.

Effects of other school characteristics on students' achievement. Table 1

shows that other school-level variables also affect students' achievement. The largest effect results from the average ability of students in a school. The positive effect of average ability on students' achievement occurred even after stringent controls for ability-related variables were introduced in the within- school model and even after other school-level variables were controlled. Table 1 also indicates that larger high schools and Catholic high schools had small, positive effects on students' achievement.

Analysis of Interaction Effects

In the final stage of the analysis, we examined interaction effects. We began this stage by searching for interactions among measures of teachers' ability and teachers' motivation to test the general idea that P = fiA * M). In the analysis, we created a complete set of interaction terms by multiplying each measure of teachers' ability (math quiz score, degree, and HOT instruction) by each measure of teachers' motivation (general force of motivation and teachers' expectations for specific students) and then entered these interaction terms into our regression model. None of the interaction terms was statistically significant, so we stayed with a model in which the effects of ability and motivation on students' achievement are additive.

In the next stage of the analysis, we explored potential interactions among the school-restructuring variables and teachers' ability and motivation. In this stage, we focused only on the school- restructuring variables that were measured at the school level and examined whether these school-level variables could be used in a "slopes as outcomes" analysis, in which the "slopes" of interest represented the effects on students' achievement of three dummy- coded variables-teachers' quiz scores, degrees, and expectations for students. Using the exploratory procedures rec- ommended by Bryk et al. (1994:47-50), we first examined the hypothesis that the effects of these dummy-coded variables varied across the schools in the




sample. The analysis demonstrated that there was heterogeneity in the effects of interest and that most other slopes in the model did not vary.

As a next step, we examined the correlations of the effects of the dummy- coded variables with several school- level variables, including all the school- restructuring variables, as well as measures of school size and average ability. In this analysis, we found that the school-restructuring variables generally had small and statistically insignificant correlations with univari- ate estimates of the effects of teachers' ability and motivation on students' achievement. On the other hand, the average ability of students attending a school was highly correlated with these effect measures. As a result, we decided to abandon the idea that school restructuring conditions the effects of teachers' ability and motivation on students' achievement and to stay with a model in which the effects of school restructuring are additive. However, we decided to allow the effects of teachers' ability and motivation to vary and to model these effects using the average ability of a school as an independent variable.

Table 2 shows the results of this analysis. In this table, we trimmed away all the variables that had insignificant effects in Table 1 and then re-estimated our basic HLM model, this time adding a series of cross-level interactions in which the effects of teachers' quiz scores, degrees, and expectations on students' achievement were assumed to vary with the average ability of students in a school. In Table 2, we further assumed that after these new effects were taken into account, there was no additional random variation in within- school coefficients. Hence, like Table 1, the data reported here are from a fixed- effects model in which only the intercepts are assumed to vary randomly.

The results reported in Table 2 are only slightly different from those reported in Table 1. At the school level, the most important change is in the effect of school size on students' achievement. This effect is no longer

statistically significant, while other effects, including the effects of the two school-restructuring variables, remain unchanged. At the student level, the results for most variables also remain unchanged. However, the effects of teachers' quiz scores, degree, and expectations for students are now affected by the average level of ability of students in a school. In Table 2, the findings now suggest that as the average ability of students rises, the effects of teachers' ability and motivation on students' achievement decline. These findings are not surprising. Much previous research suggested that low-achieving students are more sensitive to variations in the qual- ity of teachers than are high- achieving students (Hanushek 1986), and this observation is consistent with the findings in Table 2.

DISCUSSION

The findings reported here provide preliminary support for the broad hypothesis that teaching performance is a function of various dimensions of teachers' ability, motivation, and work situation. The effect sizes of variables measuring these broad constructs were not large in this study (always less than .10 SD in students' achievement). Moreover, the effects were uneven across various indicators of the constructs. However, the small effects found in this study, as well as the unevenness of the results, could easily result from unreliability in measurement.

Although the effects we found are generally small, two additional consid- erations suggest that they are important. First, as we discussed previously, students in the NELS first follow-up sample were only briefly exposed to the teachers on whom data were gathered. As a result, the data show the effects of just one of the teachers that a student was exposed to during the two years in which growth in achievement was studied. Moreover, the effects of teaching variables on students' achievement are conditioned by the ability context in which teachers work. As a result, a proper estimate of the effect sizes of




Table 2. Final Two-Level HLM Regression Analysis: lOth-Grade Mathematics as the Dependent Variable (N= 410 schools, 5,381 students)

Regression Independent Variable Coefficient SE t-ratio p-value

School-Level Model School = public -0.070446 0.039589 -1.779 0.075 School enrollment 0.009750 0.008929 1.092 0.275 Average ability 0.124222 0.009143 13.586 0.000 Teacher control 0.020047 0.009430 2.126 0.033 Common planning time 0.069080 0.024757 2.790 0.006

Student-Level Model Grade 8 math score 0.488311 0.011793 41.406 0.000 Educational background 0.118423 0.010659 11.110 0.000 Sex = male 0.022510 0.006820 3.301 0.001 Student expectation 0.021738 0.008907 2.441 0.015 Advanced track 0.076923 0.013402 5.740 0.000 Academic track 0.056926 0.008474 6.718 0.000 Other track -0.052620 0.018586 -2.831 0.005 Track missing 0.061782 0.016049 3.850 0.000 Number of high-math classes 0.056551 0.009046 6.251 0.000 Number of low-math classes -0.076132 0.008214 -9.269 0.000 Teachermath quiz score 0.018296 0.007968 2.296 0.022 Effect of average ability on

teacher math quiz score slope -0.045544 0.008275 -4.175 0.000 Degree in math 0.015742 0.007707 2.042 0.041 Effect of average ability on

degree in mathematics slope -0.016690 0.008361 -1.996 0.046 Teacher expectation 0.066796 0.008461 7.894 0.000 Effect of average ability on

teacher expectation slope -0.028330 0.007663 -3.697 0.000

teaching variables must take into account not only the additive effect of a given variable measuring teachers' ability or motivation, but the multiplicative effect of teaching in different ability contexts. When it does, the effect sizes of the teaching variables on students' achievement appear to be both statistically significant and substantively significant. This is especially the case for students in schools with high proportions of low-achieving students.

One way to demonstrate the substantive importance of teachers' ability and motivation is to use the estimated regression equation to predict the achievement scores for students in schools with different ability composi- tions. Table 2, for example, shows that in a school at the grand mean of ability composition, the effect on students' achievement of having a teacher who answered the NELS math quiz item correctly is .018 (.018[math quiz = 1] - .045[math quiz = 1 * average ability in

school = 0]). In contrast, in a school that is 1 SD below the mean in average student ability, the total effect of having a teacher who answered the math quiz item correctly is .063 (.018[1] - .045[1 * - 1]). Since the effect on achievement of being in a school 1 SD below the grand mean in average ability is -.12, the results suggest that a student in a low- ability school who was assigned to a mathematics teacher who correctly answered the NELS math quiz would experience only half the disadvantage of such an assignment than would a similar student in a similar school who was assigned to a mathematics teacher who did not answer the quiz score correctly.

The effect is less dramatic, but similar, for having a mathematics teacher with a degree in mathematics. In schools at the grand mean of average ability, the effect of having a teacher with a degree in mathematics is .016. However, for a student in a school 1 SD below the grand mean in average ability,




the effect increases to .033 (making up about one-third the disadvantage of being in a low-ability school experi- enced by students whose teachers do not have degrees in mathematics). Clearly, the results suggest that teachers' ability is substantively important in schools with high percentages of low- achieving students.

Another way to illustrate the substantive significance of teachers' effects on students' achievement is to think in terms of the cumulative effects of teacher variables on students' achievement in low-ability schools. The NELS data show that roughly 20 percent of the students in schools 1 standard deviation below the mean in average ability were assigned to mathematics teachers who did not have degrees in mathematics, did not answer the math quiz correctly, and did not expect the students to go to college. For these students, the combined effects of being assigned to such teachers would be .18 SD units on the NELS math test. As Table 2 shows, this cumulative effect is much larger than the effect of being in the advanced or academic track and/or the effect of taking additional college preparatory mathematics courses.

Although we are optimistic about the substantive importance of teachers' effects on students' achievement, especially in schools with high percentages of low-achieving students, we see a real need to develop much more refined measures of the basic constructs discussed in this article in future research on teaching. As we discussed earlier, the design of the NELS study is less than ideal for assessing the effects of teaching on students' achievement, and the measures of teachers' ability, motivation, and work situation we examined have many flaws. Given these shortcomings, we are compelled to emphasize the ten- tative and exploratory nature of the findings presented here. Clearly, much more refined studies of teachers' effects on students' achievement are needed in future research.

Still, our data suggest that such research is worth pursuing and provide several insights into how to proceed

with this kind of research in the future. First, the findings indicate that teachers' ability and motivation combine in additive (rather than multiplicative) fashion to affect students' achievement. This finding vastly simplifies the task of building models of teaching performance and analyzing data in quantitative research teaching.

Second, the findings suggest that the effects of teachers' ability and motivation vary across different work situations. Although we expected such variation to be associated with the extent of school restructuring, we did not find it to be so. Instead, we found that teaching ability and motivation had larger effects in schools where students entered with lower levels of achievement. Third, the finding that teachers' effects on students' achievement vary depending on ability contexts suggests that it may be important to search for other work situations that condition the effects of teachers' ability and motivation on student outcomes. The finding also has practical implications. Many schools with low-achieving students have trouble recruiting talented teachers and maintaining high morale among their teachers. Yet teachers' high levels of motivation and talent appear to have the biggest effects on students' achievement in just these settings. Clearly, attention to recruiting and retaining teachers in schools that serve low- achieving students must continue to be a priority for educational policy makers.

Finally, our findings suggest an explanation for the uneven results of previous studies on teachers' motivation and some directions for future research on this topic. Our study found that a specific measure of teachers' expectations for students had a larger effect on students' achievement than did a general measure of motivational force that was based on items used in past research on teacher efficacy. One explanation for the failure of the general measure of teachers' motivation to have effects on students' achievement is that high school teachers teach several different classes and adapt their levels of efficacy, expectancy, and locus of control to the specific conditions




in each of these classes (cf. Raudenbush et al. 1992). Thus, it makes sense to expect that a specific measure of expectancy is more predictive of performance than a more general measure. In the future, therefore, we recommend that

studies of teachers' motivation should develop highly specific indicators of teachers' motivation in particular classes or with respect to particular students, rather than rely on general measures of teachers' motivation.

APPENDIX A

NELS Survey Items Used in the Studya

Teacher Variables 1. Teacher Math Quiz Score

FlT2M29: Your students have been learning how to write math statements expressing proportions. Last night you assigned the following:

A one pound bag contains 50 percent more tan M&Ms than green ones. Write a mathematical statement that represents the relationship between the tan(t) and green(g) M&Ms, using t and g to stand for the number of tan and green M&Ms.

Here are some responses you get from students: Kelly: 1.5 t = g Lee: .50 t = g Pat: .5 g= t Sandy: g + 1/2 g = t

Which of the students has represented the relationship best? (MARK ONE) All of them (1) Kelly (2) Lee (3) Pat (4) Sandy (5) None of them. It should be . (6) Don't know (7)

The correct answer is 5; therefore, response 5 was recoded 1; all others were recoded 0.

2. Degree in Mathematics

* FlT3_10: What were your major and minor fields of study for your bachelor's degree?

* FlT3_11: What were your major and minor fields of study for your high- est graduate degree?

3. Higher-Order Thinking Instruction (5 items); alpha reliability = .69 (restandardized)

How much emphasis do you give to each of the following objectives? Responses on a four-point scale: 1 (none), 2 (a little), 3 (moderate), and 4 (heavy).

* FlT2M19A: Understanding the logical structure of mathematics ? FlT2M19B: Understanding the nature of proof * FlT2M19E: Knowing mathematical facts, principles, and algorithms




* F1T2M19G: Thinking about what a problem means and ways it might be solved

* F1T2M19J: Understanding mathematical concepts

4. General Force of Motivation (12 items); alpha reliability = .73 (restandardized). Responses on a six-point scale, from 1 (strongly disagree) to 6 (strongly agree).

* F1T4_1D: My success or failure in teaching students is due primarily to factors beyond my control rather than to my own effort and ability. (reversed)

* F1T4_1E: The level of student misbehavior (e.g., noise, horseplay, or fight- ing in the halls, cafeteria, or student lounge) in this school interferes with my teaching. (reversed)

* F1T4_1I: Many of the students I teach are not capable of learning the material I am supposed to teach them. (reversed)

* FlT4_1M: The amount of student tardiness and class cutting in this school interferes with my teaching. (reversed)

* F1T4_2A: Routine duties and paperwork interfere with my job of teaching. (reversed)

* F1T4_2J: I sometimes feel it is a waste of time to try to do my best as a teacher. (reversed)

* FlT4_20: The level of student drug or alcohol use in this school interferes with my teaching. (reversed)

* F1T4_5A: If I try really hard, I can get through even to the most difficult or unmotivated students.

e F1T4_5C: If some students in my class are not doing well, I feel that I should change my approach to the subject.

* F1T4_5D: By trying a different method, I can significantly affect a studen- t's achievement.

* F1T4_5E: There is really very little I can do to ensure that most of students achieve at a high level. (reversed)

* F1T4_5F: I am certain I am making a difference in the lives of my students.

5. Teacher's Expectation

* F1T1_4: Will this student probably go to college?

Possible responses were "yes," "no," or "don't know. "Yes" was coded 1, and no and don't know were coded 0.

6. Teacher Control (9 items); alpha reliability = .74 (restandardized). Responses for FlT2_17A to FlT2_17D on a six-point scale, from "no control" to "complete control." Responses for FlT4_9A to F1T4_9D on a five-point scale, from ''no influence" to ''a great deal of influence."

* FlT2_17A: Respondent's control over texts/materials * FlT2_17B: Respondent's control over content taught * FlT2_17C: Respondent's control over teaching techniques * FlT2_17D: Respondent's control over disciplining * F1T2_17E: Respondent's control over amount of homework * FlT4_9A: Teacher's influence over discipline policy * FlT4_9B: Teacher's influence over in-service programs * F1T4_9C: Influence grouping students by ability * F1T4_9D: Influence over establishing curriculum




7. Staff Cooperation (6 items); alpha reliability = .85 (restandardized). Responses on a six-point scale, from 1 (strongly disagree) to 6 (strongly agree).

* FlT4_1B: Can count on staff members to help out * FlT4_1C: Colleagues share beliefs about mission * FlT4_2C: Teachers at this school are continually learning * F1T4_2E: Great deal of cooperative effort among staff e F1T4_2F: Broad agreement among faculty about mission * F1T4_2H: School seems like a big family

8. Supportive Leadership (12 items); alpha reliability = .92 (restandardized). Responses on a six-point scale, from 1 (strongly disagree) to 6 (strongly agree).

* FlT4_1F: Principal poor at getting resources. (reversed) * F1T4_1G: Principal deals with outside pressures * FlT4_1H: Principal makes plans and carries them out * F1T4_1J: Goals-priorities for the school are clear * FlT4_1L: Staff members are recognized for a job well done * F1T4_10: Principal knows what kind of school he/she wants * FlT4_1P: Administration knows problems faced by staff * F1T4_1Q: Encouraged to experiment with teaching * F1T4_2I: Principal lets staff know what's expected * F1T4_2K: Principal is interested in innovation * F1T4_2L: Rules for student behavior are enforced ? F1T4_2M: Principal consults staff before making decisions




APPENDIX B

Bivariate Correlations for Math Achievement Model (N = 5,381)

Variable 31 30 29 28 27 26 25 24

1. Grade 10 math score -.028* .078** .035* .072** .001 .128** .007 .408**

2. Grade 8 math score -.025 .079** .033* .051** -.014 .126** .002 .371**

3. Educational background -.027* .067** .033* .019 -.010 .112** .002 .367**

4. Student SES .010 .090** .024 .062** .007 .054** .008 .364**

5. Sex = male .006 -.025 -.039** -.001 .025 -.004 -.010 .013

6. Student expectation -.016 .041** .013 .018 .010 .030* .047** .189**

7. Advanced track .002 .021 .004 .029* -.003 -.016 .033* .101**

8. Academic track -.003 .003 .026 .009 -.029* -.001 -.041** .081**

9. Other track .005 -.009 -.006 -.024 -.102 .014 .016 -.025

10. Track missing .023 .028* .023 .035* .020 -.009 .030* .006

11. General track -.009 -.027 -.038** -.036** .029* .010 .005 -.149**

12. Number of high-math classes -.024 .110** .018 .036** -.012 .060** .022 .222**

13. Number of low-math classes .025 -.047 -.012 .005 -.010 -.035* -.050** -.193**

14. Teacher's math quiz score .018 -.008 .036** -.025 .001 .063** .000 .118**

15. HOT instruction .021 .014 .008 .006 .041** .025 .039** .126**

16. General force of motivation .042** .059** .032* .027 -.002 -.026 -.001 -.046**

17. Teacher expectation -.018 .051** .035* .039** .034** .029* .011 .218**

18. Supportive leadership -.049** -.007 .004 -.020 .533** .148** .593** -.006

19. Staff cooperation .006 .084** .172** -.029* .474** .301** .621** .089**

20. Teacher control -.018 .006 .016 -.036** .326** .555** .135** .142**

21. School= Catholic -.055** .069** .133** -.073** .122** .189** .127** .122**

22. School enrollment .044** .038** -.019 .067** -.067** -.426** .011 -.034*

23. Average SES .000 .167** .070** .114** .034* .118** .016 .709**

24. Average ability -.052** .173** .092** .126** .061** .240** -.051**

25. Supportive leadership (school) -.053** .037** .051** -.054** .276** .228**

26. Teacher control (school) .012 .100** .025 .013 .186**

27. Staff cooperation (school) -.009 .008 .103** -.002

28. Common planning time .079** .146** .020

29. Flexible scheduling -.014 .197**

30. Interdisciplinary teams .070**

31. School-within-a-school

* p<.o5. ** p<.Ol.




APPENDIX B (continued)


Variable 23 22 21 20 19 18 17 16

1. Grade 10 math score .315** .000 .038** .103* .061** -.002 .538** -.044**

2. Grade 8 math score .299** -.007 .025 .099* .055** -.007 .482** -.041**

3. Educational background .275** -.019 .049** .070* .054 -.004 .432* * -.043**

4. Student SES .517** .104** .069** .049* .052** .006 .376** -.012

5. Sex = male .016 .028* .001 .021 -.007 .002 -.058** .007

6. Student expectation .200** .030* .071** .030* .045** .023 .369** -.007

7. Advanced track .099** .052** .055** .063* -.003 .037** .252** -.013

8. Academic track .061** -.044* -.049** .005 -.018 -.054** .207 .007

9. Other track -.010 .023 .047** .010 .023 -.043** -.120** -.012

10. Track missing .009 .041** -.027* -.181* .033* .047** -.027* -.001

11. General track -.132** -.011 .013 .021 .000 .029* -.340** .006

12. Number of high-math classes .194** .010 .145** .056* .056** -.010 .451** -.027*

13. Number of low-math classes .152** .009* -.097** -.034* -.057** -.018 -.376** .052**

14. Teacher's math quiz score .095** .015 -.003 .053* .014 .001 .087** -.047**

15. HOT instruction .109** .030* .001 .111* .029* .059** .320** .084**

16. General force of motivation -.027* .100* -.056** -.036* -.028* .014 .007

17. Teacher expectation .242** .048** .050** .060 .037** .024

18. Supportive leadership .024 -.005 .100* * .354* .349**

19. Staff cooperation .100** -.081* .201 .168*

20. Teacher control .078** -.236** .149* *

21. School = Catholic .140** -.115**

22. School enrollment .159**

23. Average SES

24. Average ability

25. Supportive leadership (school)

26. Teacher control (school)

27. Staff cooperation (school)

28. Common planning time

29. Flexible scheduling

30. Interdisciplinary teams


* p<.05. ** p<.01.






Variable 15 14 13 12 11 10 09 08

1. Grade 10 math score .363** .163** -.515** .555** -.447** -.013 -.179** .240**

2. Grade 8 math score .326** .136** -.459** .508** -.391** -.018 -.150** .178**

3. Educational background .285** .114** -.424** .444** -.343** -.002 -.131** .172**

4. Student SES .204** .116** -.266** .302** -.249** .007 -.077** .135**

5. Sex = male -.025 -.042** .031* -.031 .035* -.004 .024 -.026

6. Student expectation .222** .076** -.287* * .354** -.272** -.020 -.074** .188* *

7. Advanced track .193** .064** -.196* * .263** -.229** -.068** -.051* * -.358**

8. Academic track .234** .125** -.274* * .290* * -.707* * -.209 -.156**

9. Other track -.126** -.056** .114** -.136** -.100** -.030*

10. Track missing .030* -.217** .018 -.023 -.134**

11. General track -.346** -.070** .382** -.433**

12. Number of high-math classes .374** .145** -.455**

13. Number of low-math classes -.31* * -.124**

14. Teacher's math quiz score .119**

15. HOT instruction

16. General force of motivation

17. Teacher expectation

18. Supportive leadership

19. Staff cooperation

20. Teacher control

21. School = Catholic

22. School enrollment

23. Average SES

24. Average ability








* p<.05. ** p<.O1.





Bivariate Correlations for Math Achievement Model (N= 5,381)

Variable 07 06 05 04 03 02

1. Grade 10 math score .379** .365** .021 .412** .743** .871**

2. Grade 8 math score .384** .318** .030* .397** .760**

3. Educational background .302** .325** -.020 .402**

4. Student SES .190** .316** .025

5. Sex = male -.020 -.092**

6. Student expectation .153**

7. Advanced track

8. Academic track

9. Other track

10. Track missing

11. General track

12. Number of high-math classes

13. Number of low-math classes

14. Teacher's math quiz score

15. HOT instruction

16. General force of motivation

17. Teacher expectation

18. Supportive leadership

19. Staff cooperation

20. Teacher control

21. School = Catholic

22. School enrollment

23. Average SES

24. Average ability








* p <.05. ** p< .o1.




REFERENCES

Ashton, Patricia T. and Rodman B. Webb. 1986. Making a Difference: Teachers' Sense of Efficacy and Student Achievement. New York: Longman.

Bandura, Albert. 1986. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice Hall.

. 1993. "Perceived Self-efficacy in Cognitive Development and Functioning." Educational Psychologist 28:117-48.

Brophy, Jere E. 1983. "Research on the Self- fulfilling Prophecy and Teacher Expectations." Journal of Educational Psychology 75:631-61.

Brophy, Jere E. and Thomas Good. 1986. "Teacher Behavior and Student Achievement." Pp. 328-75 in Handbook of Research on Teaching (3rd ed.), edited by M. C. Wittrock. New York: Macmillan.

Bryk, Anthony S. and Stephen W. Raudenbush. 1992. Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage.

Bryk, Anthony S., Stephen W. Raudenbush, and Richard T. Congdon. 1994. HLM: Hierarchical Linear Modeling with the HLM/2L and HLM/3L Programs. Chicago: Scientific Software.

Byrne, Colin J. 1990. Teachers' Knowledge and Teachers' Effectiveness in Teaching Mathematics (Report R 000 23 1202). London: Economic and Social Research Council.

Campbell, John P. 1990. "Modeling the Performance Prediction Problem in Industrial and Organizational Psy- chology." Pp. 687-732 in Handbook of Industrial and Organizational Psychology (2nd ed., Vol. 1), edited by M. D. Dunnette and L. H. Leatta. Palo Alto, CA: Consulting Psychologists Press.

Chaney, Bradford. 1995. Student Outcomes and the Professional Preparation of Eighth-grade Teachers in Science and Mathematics: NSF/NELS:88 Teacher Transcript Analysis. Rockville, MD: Westat.

Chiang, Fang-Shen. 1996. "Teacher's Ability, Motivation and Teaching Effectiveness." Unpublished doctoral dissertation, University of Michigan, Ann Arbor.

Coleman, James S., Ernest Q. Campbell, Carol J. Hobson, James McPartland, Alexander M. Mood, Frederic D. Weinfeld, and Robert L. York. 1966. Equality of Educational Opportunity. Washington, DC: U.S. Government Printing Office.

Darling-Hammond, Linda and Jon Snyder. 1992. "Curriculum Studies and the Traditions of Inquiry: The Scientific Tradition." Pp. 41-78 in Handbook of Research on Curriculum, edited by P. W. Jackson. New York: Macmillan.

Darling-Hammond, Linda, Arthur E. Wise, and Stephen P. Klein. 1995. A License to Teach: Building a Profession for 21st- Century Schools. Boulder, CO: Westview Press.

Elmore, Richard F. 1995. "Teaching, Learning, and School Organization: Principles of Practice and the Regularities of Teaching." Educational Admin- istration Quarterly 31:355-74.

Good, Thomas L., Douglas A. Grouws, and Howard Ebmeier. 1983. Active Mathe- matics Teaching. New York: Longman.

Guskey, Thomas R. and Perry D. Passaro. 1994. "Teacher Efficacy: A Study of Construct Dimensions." American Educational Research Journal 31:627-43.

Hanushek, Eric A. 1986. "The Economics of Schooling: Production and Efficiency in Public Schools." Journal of Economic Literature 14:1141-77.

. 1989. "The Impact of Differential Expenditures on School Performance." Educational Researcher 19:45-51, 62.

Hedges, Larry V., Richard D. Laine, and Robert Greenwald. 1994. "Does Money Matter? A Meta-Analysis of Studies of the Effects of Differential School Inputs on Student Outcomes." Educational Researcher 23:5-14.

Hunter, John E. and Ronda F. Hunter 1984. "Validity and Utility of Alternative Predictors of Job Performance." Psy- chological Bulletin 96:72-97.

Ingels, Steven J., Leslie A. Scott, Judith T. Lindmark, Martin R. Frankel, and Sharon L. Meyers. 1992a. National Education Longitudinal Study of 1988. First Follow- up: Student Component Data File User's Manual. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

1992b. National Education Longitudinal Study of 1988. First Follow- up: Teacher Component Data File User's Manual. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

Kanfer, Ruth. 1990. "Motivation Theory and Industrial and Organizational Psy- chology." Pp. 75-170 in Handbook of Industrial and Organizational Psychology (2nd ed., Vol. 1), edited by M. D. Dunnette and L. H. Leatta, Palo Alto, CA: Consulting Psychologists Press.




* 1992. "Work Motivation: New Directions in Theory and Research." pp. 1-53 in International Review of Industrial and Organizational Psychology (Vol. 7), edited by C. L. Cooper and L. T. Robertson. New York: John Wiley & Sons.

Kupermintz, Haggai, Michele M. Ennis, Laura S.Hamilton, Joan E. Talbert, and Richard E. Snow. 1995. "Enhancing the Validity and Usefulness of Large-scale Educational Assessment: NELS: 88 Mathematics Achievement." American Educational Research Journal 32:525-54.

Lee, Valerie E. and Julia B. Smith. 1994. Effects of High School Restructuring and Size on Gains in Achievement and Engagement for Early Secondary Students. Madison: Center on Organ- ization and Restructuring of Schools, University of Wisconsin.

Leinhardt, Gaeg and Donald A. Smith. 1985. "Expertise in Mathematics Instruction: Subject Matter Knowledge." Journal of Educational Psychology 77:247-71.

Monk, David H. 1994. "Subject Area Preparation of Secondary Mathematics and Science Teachers and Student Achievement." Economics of Education Review 13:125-45.

Peters. Lawrence H. and Edward J. O'Connor. 1980. "Situational Constraints and Work Outcomes: The Influences of a Frequently Overlooked Construct." Academy of Management Review 5:391-97.

Pfeffer, Jeffrey. 1991. "Organizational Theory and Structural Perspectives on Management." Journal of Management 17:789-903.

Prediger, Dale J. 1989. "Ability Differences Across Occupations: More than G." Journal of Vocational Behavior 34:1-27.

Raudenbush, Stephen W., Brian Rowan, and Yuk-Fai Cheong. 1992. "Contextual Effects on the Self-perceived Efficacy of High School Teachers." Sociology of Education 65:150-67.

. 1993. "The Pursuit of Higher Order Instructional Goals in Secondary Schools: Class, Teacher and School Influences." American Educational Research Journal 30:523-53.

Rosenshine, Barak and Robert Stevens. 1986. "Teaching Functions." Pp. 376-91 in Handbook of Research on Teaching (3rd ed.), edited by M. C. Whittrock. New York: Macmillan.

Rosenthal, Robert and Lenore Jacobson. 1968. Pygmalion in the Classroom: Teacher Expectation and Pupil's

Intellectual Development. New York: Holt, Rinehart & Winston.

Rotter, Julian B. 1966. "Generalized Expectations for Internal Versus External Control of Reinforcement." Psychological Monographs 80:1-28 (Whole No. 609).

Rowan, Brian. 1990. "Commitment and Control: Alternative Strategies for the Organizational Design of Schools." Pp. 353-89 in Review of Research in Education, edited by C. Cazden, Washington, DC: American Educational Research Association.

. 1994. "Comparing Teachers' Work with Work in Other Occupations: Notes on the Professional Status of Teaching." Educational Researcher 23:4-17, 21.

1995. "Research on Status Attainment and School Effects." Pp. 33-40 in Continuity and Contradiction: The Futures of the Sociology of Education, edited by W. T. Pink and G. W. Noblit, Norwood, NJ: Ablex.

Rowan, Brian, Stephen W. Raudenbush, and Sang-Jin Kang. 1991. "Organizational Design in High Schools: A Multilevel Analysis." American Journal of Edu- cation 99:238-66.

Rowan, Brian, Stephen W. Raudenbush, and Yuk-Fai Cheong. 1994. "Teaching as a Nonroutine Task: Implications for the Management of Schools." Educa- tional Administration Quarterly 29: 479-500.

Shulman, Lee S. 1986. "Those Who Under- stand: Knowledge Growth in Teaching." Educational Researcher 15:4-14.

Smith, Mike and Dave George. 1992. "Selection Methods." Pp. 55-97 in International Review of Industrial and Organizational Psychology (Vol. 7), edited C. L. Cooper and L. T. Robertson. New York: John Wiley & Sons.

Smith, Patricia. 1976. "Behaviors, Results, and Organizational Effectiveness: The Problem of Criteria." Pp. 745-75 in Handbook of Industrial and Organ- izational Psychology, edited by M. D. Dunnette. Chicago: Rand McNally.

Staw, Barry M. 1985. "Repairs on the Road to Relevance and Rigor: Some Unexplored Issues in Publishing in Organizational Research." Pp. 96-107 in Publishing in Organizational Sciences, edited by L. L. Cummings and P. Frost. Homewood, IL: Richard D. Irwin.

Sternberg, Robert J. and Joseph A. Horvath. 1995. "A Prototype View of Expert Teaching." Educational Researcher 24:9-17.




Vroom, Victor H. 1964. Work and Motivation. New York: John Wiley & Sons.

Walton, Richard E. 1980. "Establishing and Maintaining High Commitment Work Systems." Pp. 208-90 in The Organiza- tional Life Cycle, edited by J. R. Kimberly and R. H. Miles. San Francisco: Jossey- Bass.

Weiner, Bernard. 1986. An Attribution Theory of Motivation and Emotion. New York: Springer-Verlag.

Wilson, Suzanne M. and Samuel S. Wineburg. 1993. "Wrinkles in Time and Space: Using Performance Assessments to Understand the Knowledge of Teachers." American Educational Research Journal 30:729-71.

Brian Rowan, Ph.D., is Professor and Associate Dean for Research, School of Education, University of Michigan, Ann Arbor. His main fields of interest are organization theory and analysis of school effectiveness.

Fan-Shen Chiang, Ed.D., is Associate Professor, Graduate Institute of Comparative Education, National Chi Nan University, Puli, Taiwan. Dr. Chiang's main fields of interest are comparative education, educational policy, and organization theory.

Robert J. Miller, MA, is a doctoral student, Department of Educational Admin- istration and Policy, University of Michigan, Ann Arbor. His main fields of interest are educational policy and effects of teaching on students' achievement.

This article is a revised version of a paper presented at the annual meeting of the American Educational Research Association, New York City, April 1996. Work on the paper was partially supported by Grant R308A60003 from the U.S. Department of Education, Office of Educational Research and Improvement, to the Consortium for Policy Research in Education (CPRE). The authors wish to thank Valerie Lee and Bob Croninger for their help at various points in the research, as well as colleagues who participated in the CPRE Policy Forum on Incentives and Systemic Reform, where these ideas were first presented. Address all correspondence to Dr. Brian Rowan, School of Education, University of Michigan, 610 East University, 1110 SEB, Ann Arbor, MI 48109-1259, or by E-mail at [email protected].



using research on employees' performance to study the effects of teachers on students'...

Documents