raising the bar and equity? effects of state high school

Educational Evaluation and Policy AnalysisFall 2003, Vol. 25, No. 3, pp. 299-318

Raising the Bar and Equity? Effects of State High SchoolGraduation Requirements and Accountability Policies on

Students' Mathematics Course Taking

Kathryn S. SchillerState University of New York at Albany

Chandra MullerUniversity of Texas at Austin

In response to the national push to raise academic perfornance of all students, most states haveadopted policies designed to raise academic standards, monitor progress toward those standards, andhold schools and students responsible for attaining them. Given the complex nature of the educationalprocess, these policies are likely to have mixed effects on both general levels of attainment and strat-ification based on race or ethnicity and social class. Using nationally representative longitudinal dataand hierarchical linear modeling, this article explored the association between students' mathemat-ics course work and states' high school graduation requirements and assessment or accountabilitypolicies. We found that students in states with more graduation requirements tended to enroll in higherlevel mathematics courses as freshmen and persist to take more advanced level courses. Similar trendswere also found for students in states that link test perfornance to consequences for schools. Exten-sive testing, however, had little effect on course taking except to increase differences based on socio-economic status. In contrast, differences between racial or ethnic groups tended to be smaller in stateswhere test perfornance was linked to consequences for students.

Keywords: accountability, educational stratification, equity, graduation requirements, mathematicsachievement, opportunities to learn, race and ethnicity, social class, sociology of education

THE No Child Left Behind Act of 2001 broughtsweeping changes in the role of the federal gov-ernment in elementary and secondary schoolingthrough, among other reforms, increased man-dated testing and school accountability. The lawrequires states to almost immediately start admin-istering mathematics and reading examinationsbased on established state curriculum standardsto all students in grades 3-12. In addition to over-

all progress toward meeting state standards, thelaw also calls for monitoring the progress withineach school of students who are economically dis-advantaged, from racial or ethnic minority groups,have disabilities, or have limited English profi-ciency. Schools that fail to make state-defined ad-equate progress toward meeting the state standardswill be subjected to increasingly severe sanctionsover five years culminating with restructuring,

This research was supported by a grant to the first author from the American Education Research Foundation, which receivesfunds for its "AERA Grants Program" from the National Science Foundation and the National Center for Education Statistics(U.S. Department of Education) under NSF Grant RED-9452861. It was also supported by funding from the National Institute ofChild Health and Human Development under grant R01 HD40428-02 (Chandra Muller, PI) and the National Science Foundationunder grant REC-0126167 (Chandra Muller, PI) to the Population Research Center, University of Texas at Austin. We thank thereviewers for their comments and suggestions. Opinions reflect those of the authors and do not necessarily reflect those of thegranting agencies.

299

Schiller and Muller

such as state takeover or conversion to a charterschool. The goal of this legislation is to not onlyraise academic standards and, thus, performancefor American school children, but also to decreasegaps in achievement between socially advantagedand disadvantaged groups.

The provisions in No Child Left Behind were acontinuation of efforts over the past 40 years byeducational policymakers and practitioners to raisestandards in mathematics and science. These re-forms have included not only raising expecta-tions for students' mastery of these subjects, butalso requiring that all students have exposure toa core curriculum incorporating these standards.Approximately 20 years ago, state policy makersalso began implementing examination systemsto hold schools accountable for students' aca-demic progress (McDonnell, 1994). Both re-form efforts-raising expectations and increas-ing external accountability-redefine what a highschool graduate should know and provide in-centives for all students to acquire a minimumlevel of achievement in order to earn a diploma.

In response to these reforms, schools haveraised graduation requirements and restructuredtheir academic programs. Between 1980 and1993, the average number of credits in core aca-demic subjects that schools required for earninga high school diploma increased by over 1.6 years(Stevenson & Schiller, 1999). Over two thirdsof this change was in requirements for additionalcourses in mathematics and science. Anotherdramatic change was the softening, if not officialelimination, of formal academic tracking sys-tems in favor of a standards-based core curricu-lum in which tracks are more subject specific andbased on timing of course enrollments (Lucas,1999). During the 1980s, public high schoolsincreased the size of their academic tracks by14% to enroll an average of 46% of their sopho-more cohorts while vocational track enrollmentsdropped by 12% to an average of less than 19%of their sophomore cohorts (Stevenson & Schiller,1999). This shifting of students into the academictrack was most dramatic in states requiring testscore results to be widely disseminated to policymakers, the media, and parents.

Complexity of the educational process, how-ever, means that these efforts to improve students'educational experiences and academic achieve-ment have had mixed results. High-stakes ex-aminations for students, for example, have been

300

related to higher rates of dropping out for at-riskstudents but do not appear to affect levels ofachievement (Jacob, 2001). Greater school ac-countability appears to increase the number of ad-vanced mathematics credits high school studentseam, but does not affect their probability of earn-ing a diploma (Muller & Schiller, 2000). In addi-tion, these state accountability policies also seemto exacerbate the attainment gap between studentsof low- and high-socioeconomic backgrounds,suggesting that poor students may be negativelyimpacted by holding schools responsible fortheir academic progress (Muller & Schiller, 2000).Thus, neither high-stakes examinations for stu-dents nor school accountability are a panaceafor helping all students reach higher academicstandards.

Developing effective policies requires under-standing how proposed reforms may influencestudent achievement at different stages of the ed-ucational process, with thoughtful considerationof potential negative effects. This study exploreswhether students' mathematics course enrollmentsas freshmen and in high school overall variedas a function of states' high school graduationrequirements and assessment or accountabilitypolicies. Drawing from a nationally represen-tative longitudinal sample of U.S. high schoolstudents in the early 1990s, we used hierarchi-cal linear modeling (HLM) to examine varia-tion across states in both the level of mathemat-ics courses students tended to take and differencesin course enrollments related to race or ethnic-ity and social class. We focused on mathematicsbecause students placements in this highly struc-tured and sequential subject creates key turningpoints in their opportunities to learn (Schneider,Swanson, & Riegle-Crumb, 1998; Stevenson,Schiller, & Schneider, 1994). The mathematicscourses students take in high school affect theiracademic achievement and their admission tocompetitive postsecondary schools and prepro-fessional programs.

Opportunities for Learningand State Policies

A core goal of schooling has always beento promote students' development of skills andknowledge important for success as adults throughcourses of study providing them with basic op-portunities for learning. Since the Cold War andA Nation at Risk (National Commission on Ex-

Raising the Bar and Equity?

cellence in Education, 1983), U.S. high schoolshave been criticized for failing to produce grad-uates prepared for the demands of higher educa-tion and the workforce. Of particular concern isthat U.S. high school students continually lagbehind their counterparts in other industrializednations in mathematics and science (Stedman,1997). The former is considered especially prob-lematic because understanding basic mathemat-ical principals taught in algebra and geometryare important for students' success in science(Schmidt et al., 2002). Graduates who are weakin these two subjects are considered unpreparedfor entry into medicine, engineering, and othertechnology fields. These concerns have focusedpolicy makers' attention on what courses studentstake in high school and whether they master thematerial to which those courses are supposed toexpose them.

In response to policymakers' mounting con-cerns about both academic quality and educationalinequity, educational reforn efforts since the mid-1980s have encouraged "de-tracking" by requir-ing all students to complete a common core cur-riculum (Wells & Oakes, 1996). These effortswere fueled by sociological research revealinggreat variation in the academic experiences ofadolescents, with some exposed to challengingcurriculum in the college preparatory track whileothers received only basic instruction in the gen-eral track (Gamoran, 1987; Oakes, 1985). Whileintended to allow matching of students' talentsand interests to course content, high school trackassignments were often based on non-academiccriteria such as social class and ethnicity (Oakes& Guiton, 1995). Even in schools without formaltracking, students' opportunities for learning areoften constrained by systems of prerequisites, es-pecially in highly structured subjects like mathe-matics, that create sequences of opportunitiesfor learning that can span both grade levels andschools (Stevenson et al., 1994). Where studentsare placed as freshmen creates a positional advan-tage for gaining access to advanced level courses,which are related to greater gains in academicachievement and entry into postsecondary school-ing (Schneider et al., 1998). Thus, curricular struc-tures create defacto tracking in that freshmencourse enrollments determine to a great extentstudents' academic trajectories in high school. Inthis article we explored whether states' efforts toraise standards and increase accountability were

related to the level of mathematics courses fresh-men take and how far students progressed in thesubject during high school.

To what extent policymakers can change stu-dents' course enrollments, and thus achievement,is questionable because many individual factors in-fluence the types and number of courses they take.Children of college educated parents are morelikely to enroll in algebra in 8th grade, allowingthem to move on to geometry as high school fresh-men, compared to their classmates whose parentsonly attended high school (Stevenson et al., 1994;Useem, 1991). Children of more educated parentsnot only receive a head start in the high schoolmathematics curriculum, but also tend to persistin taking courses including exposure to advancedalgebra and calculus. While this situation appearsto be changing, girls and minority students havebeen traditionally under-represented in advancedlevel mathematics courses (Oakes, 1985). One ofthe central concerns of our analyses was to deter-mine whether state policies were related to differ-ences based on social class and ethnicity in fresh-man mathematics course enrollments as well asaccumulation of advanced course credits in thesesubjects.

During the early 1980s, regulatory changesfocused on raising academic standards by increas-ing the number of credits required in academicsubjects compared to earlier diploma holders, ineffect altering the definition of a high school grad-uate (Chaney, Burgdorf, & Atash, 1997; Clune& White, 1992; Stevenson & Schiller, 1999).The logic behind such changes was that requir-ing students to take more courses in core acad-emic subjects increases their opportunities forlearning key skills and results in higher levels ofacademic achievement. Basic assumptions behindthese policies were that many high school studentsare motivated to take only the minimum numberof required courses, that the additional coursesthey take will be academically rigorous, and thatthey are able to do the work required to pass thesecourses. Research finds at least partial supportfor arguments linking high school graduationrequirements to increased mathematics coursetaking and academic achievement of students,especially for those who are marginal in theirmotivation and skills (Chaney et al., 1997; Clune& White, 1992). This study was designed to com-pare differences in mathematics trajectories ofsimilarly able students in states with differingcourse requirements for high school graduation.

301

Schiller and Muller

Although increased graduation requirementsappeared to raise enrollment in academic courses,many policy makers questioned whether coursetitles accurately reflect their content or that studentsmay be given passing grades without leamingthe material (McDonnell, 1994). These concernsfueled efforts since the late 1980s to establishperformance standards and increase externalmonitoring of students' progress toward thosestandards. Initially, mandating extemal examina-tions was a mostly "persuasive" reform strategyintended to provide information indicating whichstudents need remediation, to establish commonacademic goals for students and teachers, and topromote community grassroots movements sup-porting academic excellence (McDonnell, 1994).The assumption underlying these policies wasthat regular monitoring of students' academicprogress would improve their preparation for ad-vanced level work and increase the demand forrigorous course offerings. Critics of these poli-cies, however, raise concerns that standardizedtesting creates self-fulling prophecies that limitthe opportunities for leaming of academicallyand socially disadvantaged students (Wells &Oakes, 1996). In our study we examined whethermore extensive testing of high school students inacademic subjects was associated with enrollmentin higher level mathematics courses throughouthigh school for all students.

Many states also have established formal sys-tems of rewards and sanctions linked to perfor-mance on mandated examinations that are de-signed to hold students and schools accountablefor attaining at least minimal academic standards.Although controversial, initial reform efforts es-tablished high stakes examinations linking testperformance to consequences for individual stu-dents such as track placement, grade promotion,and high school graduation (Heubert & Hauser,1999). Critics expressed concern that suchpolicies structurally limit socially and academ-ically disadvantaged students' access to oppor-tunities for learning by allocating them to reme-dial courses and encouraging them to dropoutof school (Catterall, 1989; Jacob, 2001). Someresearch, however, suggests that motivated low-achieving students may benefit from an increasedemphasis on academic achievement and supportfrom teachers (Muller, 1998; Roderick & Engel,2001; Schiller & Muller, 2000). Because theseexaminations are usually held early in high school,

302

the policy is more likely to impact the coursestaken by freshmen and sophomores to preparefor the tests and might potentially discouragestudents from taking more advanced level coursesas juniors and seniors. We explored in these analy-ses whether states' high stakes accountability poli-cies affected students' tendencies to enroll in oravoid higher level mathematics courses at twokey points in their high school careers.

State policies promoting institutional account-ability by linking tangible consequences forschools to aggregate measures of student perfor-mance started becoming common in the 1990s.Rather than directly regulating instructional ac-tivities, these state policies set academic excel-lence as the goal while giving schools the freedomto determine the best way to help their studentsreach the state standards (Elmore, Abelman, &Fuhrman, 1996). One way schools might chooseto raise student performance is to increase thenumbers enrolled in courses that prepare themfor higher level work in key academic subjects likemathematics. However, some schools might alsomarginalize poor-performing, particularly minor-ity and poor, students to avoid accountability fortheir expected failure on the state assessments(Schiller & Muller, 2000). We explored whethergreater school accountability was related to eq-uity in opportunities to learn mathematics acrosssocial classes or racial and ethnic groups.

The decentralized nature of the nation's schoolsystems means that states vary greatly in the strate-gies and policies they have adopted at a given time.Although most states raised academic courserequirements for a high school diploma duringthe 1980s, by 1990 only three states had adoptedthe National Commission on Excellence in Edu-cation's recommendation that all students takeat least three years of mathematics (Chaney et al.,1997). In 1993, states on average required 2.4years of mathematics (Stevenson & Schiller,1999). By this time, most states had also imple-mented some sort of mandated testing program,although how often and in how many subjectsstudents were tested as well as the consequencesfor test performance varied greatly across states(Schiller & Muller, 2000).

In the analyses for this article we examined theimpact of greater course requirements for highschool graduation, more frequent mandated test-ing, and implementation of sanctions and rewardsfor either students or schools linked to test per-


formance on high school students' course takingin mathematics. Using longitudinal data from anationally representative cohort of 8th graders,we focused on two key stages of students' aca-demic careers: (a) where they entered the highschool mathematics curriculum and (b) how farthey progressed through the curriculum. We usedHLM to test the extent to which these policiesaimed at raising levels of attainment and increas-ing equity were related to differences based onsocial class and race or ethnicity in opportunitiesfor learning mathematics in high school. Con-trolling for other aspects of students' social back-grounds and middle school mathematics classesand grades, we also examined how the relation-ship between freshman mathematics course place-ments and students' persistence in the subjectvaried among states with different policies.

Data and Method

The sample

The analyses in this article required the use oftwo data sets, one to provide longitudinal infor-mation on students' social backgrounds and aca-demic experiences, and the other to provide infor-mation on states' assessment and accountabilitypolicies. Both of the studies we used were con-ducted in the early 1990s.

The National Education Longitudinal Studyof 1988-92 (NELS:88-92) followed a nation-ally representative sample of 8th graders in 1988through their high school careers and beyond(Ingles, Scott, Lindmark, Frankel, & Myers, 1992).The panel used for these analyses consisted of10,046 public school students who participatedin the first three waves of data collection (1988,1990, and 1992) and for whom high school tran-scripts were collected. All 50 states and the Dis-trict of Columbia are represented in NELS:88-92,with an average of 196 students and 22 highschools per state.' For these analyses, the samplewas weighted to take into account the complexsample design and nonresponse rates so that theresults would be representative of those for the1988 8th-grade cohort.

Information on states' assessment and account-ability policies was obtained from the NationalLongitudinal Study of Schools (NLSS). One pur-pose of NLSS was to examine the impact of statepolicies on changes in school practices (Levine& Stevenson, 1997; Stevenson & Schiller, 1999).In 1993, state departments of education were

asked through the National Cooperative Educa-tion Statistics System to answer a lengthy ques-tionnaire concerning their testing and account-ability policies. Responses were received fromall 50 states and the District of Columbia.

Mathematics course enrollments

The measures of students' mathematics coursetaking were constructed from the NELS:88-92course-level transcript file, which includes indica-tors of the topic and when it was taken for everycourse a student took during high school. Basedon the standard sequences of mathematics coursesmost students take, courses were classified intoone of the following groups, in hierarchical order:(0) no math, (1) remedial math, (2) general math,(3) pre-algebra, (4) Algebra I, (5) geometry,(6) Algebra II, (7) advanced math, (8) pre-calculus,(9) calculus (Schiller & Hunt, 2001). The firstanalysis in this article predicted the highest levelmathematics course students took as freshmen,indicating where they entered the high schoolmathematics curriculum. The second analysis pre-dicted the number of Camegie units earned inhigher level mathematics courses (geometry andabove), which are commonly required for admis-sion to a competitive college or postsecondaryacademic program. Due to a highly structuredsequence of prerequisites, how many Carnegieunits students accumulated in advanced levelmathematics courses is a good indicator of howfar they progressed toward calculus. Becausewhere students started in the sequence was likelyto affect how far they progressed, freshman courseplacement was also used as a predictor of thesecond dependent variable.

Student-level variables

This study focused on differences across statesnot only in students' freshman mathematicscourses and number of advanced credits, but alsoin the variation in these outcomes related to so-cioeconomic status and race or ethnicity. In theseanalyses, family socioeconomic status (SES) wasa measure of students' financial and social re-sources from outside the school based on a com-posite of parents' education, income, and occupa-tion created by NCES. Using HLM, we evaluatedwhether the relationship between SES and math-ematics course taking varied across states withdiffering graduation requirements or assessmentpolicies.

303

Schiller and Muller

Instead of the usual four-group classificationof students' race or ethnicity, our HLM modelsonly included indicators for African Americanand Latino/a with the comparison group beingwhite or Asian American. We chose not to distin-guish between white and Asian American studentsbecause of the latter group's small sample size andsparse distribution in many states that resulted inunreliable HLM coefficients. Our results concern-ing the effects of race or ethnicity on mathematicscourse taking and their variation across states werenot significantly affected by this decision.

To control for other individual characteristicsthat might have influenced mathematics courseenrollments, we included other indicators ofstudents' background characteristics (gender andfamily structure) and prior academic achieve-ment (middle school mathematics grades and8th-grade mathematics course enrollments). Weused grades, rather than test scores, because theymeasure how well students met the expectationsof their middle school teachers in their classesand are frequently used by high schools to placestudents in freshman courses.2 We also includedindicators for whether students attended an urbanor rural public high school, with suburban as thecontrast category. For a description of these vari-ables, see Appendix A.

State policy measures

Our approach to analyzing the effects of statepolicies was to develop indicators of strategies,or policy levers, that states adopted to raise expec-tations and increase accountability for students'academic progress. Efforts to characterize statepolicies have ranged from broad general charac-terizations of the policy environment (Lee, 1998)to analyses of specific policies such as requiringstudents to pass an examination to graduate (Cat-terall, 1989; Jacob, 2001).3 Analyses using theformer are difficult to interpret because distinc-tions in the purposes of various policies are lost.The latter often fail to find significant effects ofpolicies unless the analyses focus on the sub-populations most likely subject to the policies.Our goal was to develop "mid-level" indicatorsof state policies that reflect the various strategiesstates used to raise expectations and accountabil-ity as well as the extent to which a type of policylever was employed. The measures used in ouranalyses were based on state policies reported in1993, the year during which most NELS studentsgraduated from high school.

304

In this study we included an indicator of states'academic course requirements for high schoolgraduation, the oldest strategy for raising aca-demic standards by requiring students to takemore courses. In the NLSS questionnaire, stateswere asked to report the number of Carnegieunits in various subjects that students were re-quired to complete to be eligible for a high schooldiploma. The variable used in this analysis wasthe total number of credits required in the fourcore academic subjects of English, social studies,mathematics and science. (See Appendix B for afull description of the state variables.) Only threestates (Colorado, Massachusetts, and Wyoming)reported setting no course requirements for highschool graduation, while the remaining states re-quired an average of 9.96 credits in these subjectsto earn a diploma.

Our measure of the extensiveness of states'testing programs in 1993 was based on their re-ports of the grade levels and major academic sub-jects in which mandated tests were administeredto students during high school. Only seven statesreported no mandated testing of high schoolstudents in the major subjects of English, socialstudies/history, mathematics, and science. Theremaining states gave on average four tests to highschool students, although two states (Minnesotaand Virginia) reported testing students in all foursubjects every year. The extensiveness of a state'stesting program is an indicator of whether exter-nal examinations were used on a regular basisto monitor students' progress through the estab-lished curriculum, usually with the intention ofraising overall levels of achievement.

Although most states tested high school stu-dents, they varied in the extent to which perfor-mance on those tests carried meaningful conse-quences for students or schools. Our measure ofconsequences for students based on test perfor-mance was the sum of states' reports of whethertest scores were recommended or required for pur-poses such as placement in remedial or advancedplacement programs, promotion to the next grade,or award of a high school diploma. Almost twothirds of the states had guidelines or mandatorypolicies specifying how test scores should be usedto determine some aspect of students' academicprogram or success. Those states with suchpolicies linked test scores to an average of threeor four consequences for students. Fewer stateslinked students' performance on mandated tests


to rewards and sanctions for schools in 1993. Thesurvey asked about eight types of consequences,such as financial rewards for meeting standardsor sanctions like loss of accreditation for failureto do so. Over two thirds of the states reportedthat they either did not set performance standardsor did not provide incentives for meeting thosestandards. The remaining third of the states linkedaggregate test scores to an average of three or fourconsequences for schools.

In preliminary analyses, the four measures ofstates' policies appeared to reflect distinct strate-gies for increasing standards and accountability.The four measures of state policies were onlymoderately related to each other with correla-tions all less than .36. The strongest correlationreflected that the number of consequences forschools and for students linked to test perfor-mance was related to states having a testing pro-gram. However, the extensiveness of testing andhow results were used to deterrnine sanctions orrewards tended to be unrelated.

Analysis technique

The questions of whether state testing policieswere related to students' mathematics courses inhigh school required a multilevel analytic strat-egy. We were concerned not only with variationin students' mathematics course taking acrossstates with differing policies (direct effects), butalso with whether the associations of students'outcomes with their social backgrounds variedacross states (interaction effects). A commontechnique for analyzing hierarchical data (in thiscase, students nested within states) and cross-level effects is HLM, which allows simultane-ous consideration of factors from two levels ofanalysis (Bryk & Raudenbush, 1992; Rauden-bush & Bryk, 1986).4

The same student and state policy variableswere used for analyses of students' freshmanmathematics course level and the number of ad-vanced mathematics credits earned, except fresh-man course level was also used to predict thelater outcome. The student-level model is shownin Equation 1, where ij was the value for a givenstudent in a given state and Bkj was the coefficientfor students' SES, race and ethnicity, or the con-trol variables in each state. The effects for someof the student-level factors, such as race or eth-nicity, were expressed by several coefficients, forexample B21 for Latino/a and B3j for African-

American. The term ev- was a measure of the ran-dom error, which included unmeasured sourcesof variation in a particular student's outcome. Inour analyses, all the student-level variables werecentered around their grand means for the sam-ple, which allowed the intercept (Boj) to be inter-preted as the mean outcome for each state adjustedfor the characteristics of students in that state(Bryk, Raudenbush, & Congdon, 1996; Willms& Raudenbush, 1989).

Yij = oj + B,j (SES, )

+ B2 -31(Race/Ethnicityij)

+ B4jio1 (Controls) + e0 (1)

Preliminary analyses indicated that, in our sam-ple, the associations between the student-levelcontrol variables and mathematics courses eitherdid not vary significantly across states or thosevariations were not related to state testing poli-cies. Either situation meant that assuming theassociations were constant across states did notsubstantially affect the results for SES and raceor ethnicity. Thus, for the analyses presentedhere, the coefficients for the student-level controlvariables were set to be "fixed effects" and ourstatistical model assumed that the relationshipsbetween these student characteristics and mathe-matics course enrollments were the same for allstates (Bryk & Raudenbush, 1992).

The state-level analyses, in essence, examinedthe extent to which variation in the coefficientsfor the intercept, SES, and race or ethnicity wererelated to states' graduation and accountabilitypolicies. Equation 2 shows the general modelused for estimating the effects of these state poli-cies.5 Each of the policy variables was centeredaround its grand mean, which meant that yko wasthe average effect of variable k across states andthe other coefficients were adjustments to thosecoefficients, or interaction effects, for states thatdiffered in their testing policies. The effect of astudent-level variable was increased when thecoefficient for a state policy variable was in thesame direction (plus or minus) as the interceptfor the student-level variable and reduced whenthe two coefficients were in the opposite direc-tion. The term ukj was the error term for estima-tion of the student-level coefficient for each state.

Bkj = YkO + Ykl-k 4 (State Policiesj )+ Ukj (2)

305

Schiller and Muller

The combined HLM model is shown inEquation 3.

y, = [700 + 7 oio 4 (State Policiesj)+uoj

+ [y1 + 711 o4 (State Policiesj ) + u,1 ]* SES

[7 2-30 + 722-34 (State Policiesj )+ U2_3]

* Race/Ethnicityi0

+74-100 * Control Variablesij +eij (3)

Results

States vary in their approaches to raising highschool students' academic attainment and pro-moting equality of opportunities for learning, butthe extent to which these policies impact students'educational careers is uncertain. The purpose ofour study was to examine variation across statesadopting different strategies for raising standardsand establishing accountability in two critical as-pects of students' mathematics course enrollments:where they started as freshmen, and the amount ofadvanced-level course work completed by gradu-ation. The goal was to determine whether thesestate policies were related to students' mathemat-ics course placements as freshmen and their per-sistence in advanced mathematics as well as dif-ferences based on SES and race or ethnicity.

Freshman Mathematics Course Placements

The results for students' freshman mathematicscourse enrollments are shown in Table 1. The toppanel of the table contains the coefficients for theintercept and independent variables modeled onthe state level. The first column shows the Level-2 intercept, or average effect, for the student-levelvariables of interest. The other four columnsshow the coefficients for state policy variables.The lower panel contains the coefficients for thestudent-level controls, which were assumed to beconstant across states.

All of the student-level control variables excepturbanicity were significant predictors of students'mathematics course placements as freshmen. Stu-dents tended to enroll in a higher level course ifthey were female, lived with both natural parents,had higher mathematics grades in middle school,and enrolled in Algebra as an 8th grader. 6 Eighthgraders who took remedial mathematics tendedto be placed in lower level courses as freshmen,

306

even taking into account their social backgroundsand middle school mathematics grades. Freshmancourse enrollments appeared to have been similaracross urban, suburban, and rural locations.

The first row of Table 1 indicates that states'graduation and accountability policies were re-lated to differences in where freshmen enter thehigh school mathematics sequence. On aver-age, freshmen tended to enroll in pre-algebra(coded as 3). In states requiring more academiccourse credits for graduation, freshmen tended totake slightly higher level mathematics courses.Although statistically significant, this effect wassmall at less than 7% of a course level per stan-dard deviation change in the number of coursesrequired (.077 = .024 * 3.203). This difference,however, was only slightly smaller than thoserelated to gender or family structure. Extensive-ness of testing was also significantly related tofreshman course enrollments, with students instates with more extensive testing tending to en-roll in slightly lower level freshman mathematicscourses (-.059 = -.017 * 3.461). Neither of thestate accountability policy variables were signif-icantly related to freshman course enrollments.

Extensive testing was also significantly relatedto a somewhat stronger effect of socioeconomicstatus on freshman mathematics course level. Astandard deviation increase in the number of testswas related to almost a 20% increase in the effectof SES [.197 = (.018 * 3.461)/.314]. These re-sults indicate the gaps between poor and rich stu-dents were larger in states that test high schoolstudents more frequently and in more subjects.The stronger effect of SES in states with moreextensive testing was consistent with analyses ofother academic outcomes such as earning a highschool diploma (Muller & Schiller, 2000).

Our results identified no overall differencesbased on race or ethnicity after controlling forprior academic performance and SES. However,state policies were related to significant differencesin freshman mathematics course enrollmentsbetween African American and white students.The tendency for African American students toenroll in somewhat lower level courses com-pared to similar whites was stronger in states re-quiring more academic courses for graduationor linking test performance to consequences forstudents. The latter policy strategy more thandoubled the effect of being African Americanfor each additional consequence. Conversely, the

U; I

C OFoo C-C-

(I 0. It

o 4f C; C

.eQ

-. CC, -C

rto

, .' °- C , U X D= E0lU

0~~~~~~~~~~~~~~~~~~~~~CC0~~~~~~~~~~~~~~r ~ 0 - . 0 c

.-0 ~ ~ ~ c*

O~~ Cl '~~~~0 C'r C D0 ~ ~ 0-' C'CC-- C~~~~~ o\O O

0 fCC~~~~~~~CV 0 0 0 0 '

307

Schiller and Muller

gap between African American and white studentswas significantly smaller in states linking testscores to consequences for schools.

Number of AdvancedMathematics Course Credits

The analysis of the number of advanced math-ematics course credits students accumulated inhigh school included the same student-level vari-ables as the previous analyses with the level offreshman mathematics course included as an inde-pendent variable. Because of the sequential natureof the high school mathematics curriculum, whatmathematics courses freshmen took was a strongpredictor of how far they progressed in the sub-ject before graduation. One of the main purposesof this analysis was to explore how state gradua-tion requirements and accountability policies wererelated to students' course taking patterns in math-ematics over time.

As before, Table 2 shows the coefficients forthe intercept and slopes modeled on the statelevel in the top panel, and the coefficients forthe student-level control variables in the bottompanel. Focusing on the control variables first,students who lived with both parents, had highermathematics grades in middle school, and tookAlgebra in 8th grade tended to earn more ad-vanced mathematics credits in high school. Gen-der and urbanicity were not significantly related tothe number of credits students earned in coursessuch as Geometry, Algebra II, or Trigonometry.These results indicate that most factors relatedto where students entered the mathematics se-quence continued to similarly influence theirlater trajectories through the high school math-ematics curriculum.

The top panel of Table 2 shows that state poli-cies were related to students' accumulation of ad-vanced mathematics course credits, both directlyon the average number of credits earned andthrough interactions with students' race or ethnic-ity, socioeconomic status, and freshman courseplacements. On average, students tended to earn1.6 credits in advanced mathematics, approxi-mately equivalent to a year of Geometry and overa half year of Algebra II. Even though they enteredthe mathematics sequence at a slightly higherlevel than students in other states, students instates requiring more academic courses to earn ahigh school diploma tended to earn fewer ad-vanced mathematics course credits. This result

308

may reflect that increasing core course require-ments could discourage or prevent students fromtaking advanced courses in mathematics as theyare forced to satisfy requirements in other sub-jects. Although linking test performance to con-sequences for schools did not appear to impactfreshman course placements, students in stateswith more consequences tended to earn a some-what greater number of advanced mathematicscredits. Again, however, these statistically sig-nificant effects were fairly small at 4% (-.042 =-.013 * 3.203) of a credit decrease per standarddeviation increase in number of academic coursesrequired and about 9% (.089 = .046 * 1.929) of acredit more per standard deviation increase inthe number of consequences for schools. Whilefreshmen in states with more extensive testingtended to take slightly lower level courses, stu-dents did not earn significantly different numbersof advanced mathematics credits in states withless testing when their initial placements weretaken into account. Overall, students' trajectoriesthrough high school mathematics, as well as theirinitial placements, appeared to have been shapedto some extent by states' graduation and account-ability policies.

These state policies were also related to dif-ferences in the number of advanced mathematicscredits based on students' SES or race and eth-nicity. As with initial placements, the effect ofSES tended to be slightly stronger in states withmore extensive testing programs. The effect ofSES on the number of advanced mathematicscredits earned increased by 14% [.139 = (.013 *3.461)/.323] per standard deviation change inthe number of mandated tests. These results sug-gest that extensive testing had a persisting effecton social stratification not only through initialplacements but also through increasing the gapsbetween poor and rich students over their highschool careers.

States' graduation requirements and account-ability policies were also related to differences inthe number of advanced mathematics require-ments earned by racial and ethnic groups. Latinola students tended to earn on average slightlyfewer credits in advanced mathematics than whiteor Asian American students, with this differencebeing almost a third smaller (.312 =.029/.093) foreach additional required course. Taking into ac-count their lower freshman mathematics courses,African American students tended to accumulate

4.goO C,)Cn

4)4)U . cqo -t

0nm 00 C,* r0 t Cl 0*Cl

o $

4)Oj

04)*** * XX*

Y~~~~~~~~C o

:1s- e " < ~Cl D M ;C i

00 c CC

309

Schiller and Muller

a slightly greater number of advanced math-ematics credits compared to white students instates with higher graduation requirements. Theeffects of state policies specifying consequencesfor students showed a different pattern, withAfrican American students tending to earn feweradvanced mathematics credits in high school. Bothof these interaction effects were larger than theoverall average difference between African Amer-icans and whites. One possible explanation forthese patterns is that requiring more courses en-couraged African American students, who tendedto start out in lower courses as freshmen, to per-sist in taking mathematics longer in order to ac-cumulate the necessary number of course credits.In contrast, greater consequences for studentsmight have create self-fulfilling prophecies thatdiscouraged these minority students from enter-ing more advanced level courses.

Finally, states' graduation requirements andaccountability policies appear to have influencedstudents' trajectories through the high schoolmathematics curriculum. On average, studentstended to accumulate .448 of a Carnegie unit morein advanced mathematics for each level higherthey were placed as freshmen. The link betweenfreshman course placements and accumulatedcourse credits was stronger in states with a greaternumber of course requirements, more conse-quences for schools linked to test performance,and with a fewer number of consequences forstudents. These interaction effects were statisti-cally significant but modest, amounting to about9% to 12% change in the effect of freshmancourse placements per standard deviation increasein a given state policy measure. However, theseresults suggest that some state policies were as-sociated with not only freshman starting in moredifficult courses, but also helping these studentspersist in taking more advanced level courses.

Overall and cumulative effectsof state graduation requirementsand accountability policies

As noted above, state graduation requirementsand accountability policies had complex rela-tionships with students' mathematics course en-rollments in high school. For example, studentsin states with a greater number of academic coursesrequired for high school graduation tended to beplaced in higher courses as freshmen, to earn feweradvanced mathematics credits and the influence

310

of their freshman course placements was strongerin these states. In addition, this state policy wasrelated to overall higher freshman course place-ments and a stronger tendency for African Amer-icans to be placed lower than white students. Toexamine the overall and cumulative effects ofstate graduation requirements and accountabilitypolicies, we used the HLM equations to estimatethe expected number of advanced mathematicscredits for students with identical social back-grounds and middle school experiences but whodiffered on the key variables of interest for theseanalyses.

Table 3 shows the overall effects of a state pol-icy on students' advanced mathematics courseenrollments by estimating the expected numberof credits earned by students who differed in theirfreshman course placements but were otherwiseidentical (i.e., had the same social backgroundsand middle school mathematics experiences).Low (or few) or high (or many) values for any ofthe variables were one standard deviation below(or the lower bound) or above the mean for thatvariable. For freshman mathematics course en-rollments, the low and high placements wereroughly equivalent, respectively, to Pre-Algebraand Geometry.

Clearly shown in Table 3 is the strong effect offreshman course placements on advanced math-

TABLE 3Expected Number of Advanced Mathematics CreditsEarned in High School by Freshman CourseEnrollment and State Policy

Level of FreshmanMath Coursea

State Policy Low High

Graduation requirementaFew 1.161 2.186Many .966 2.211

Extensiveness of testingaFew 1.024 2.217Many 1.103 2.181

Consequences for schoolsNone 1.051 2.106Manyb 1.084 2.356

Consequences for studentsNone 1.042 2.303Manyb 1.086 2.088

Note.± I standard deviation from the mean.I standard deviation above the mean.


ematics courses, with the lower placed studentexpected to earn about one credit compared totwo credits for the otherwise identical higherplaced student. However, these gaps clearly var-ied between states with different policies, withthe overall effect of the state policies rangingfrom .12 to .26 of a Carnegie unit, or approxi-mately 1.5 months to half a semester of a year-long mathematics course. The smaller changewas in the gap between low and high students instates differing in amounts of testing, which wasdue to a weak effect of this policy on freshmancourse placements.

Both graduation requirements and linking testperformance to consequences for schools wererelated to an increasing gap between high andlow students, but by affecting different types ofstudents. The impact of graduation requirementswas strongest for the students placed in lowerfreshman classes, with those in states with manygraduation requirements expected to earn lessthan one credit in advanced mathematics. In con-trast, more consequences for schools were ex-pected to increase the number of Carnegie unitsin advanced mathematics earned by both types ofstudents, but the expected increase was dramati-cally larger for the high student (.250 comparedto .033).

Similarly, the effect of linking test performanceto consequences for students differed based ontheir freshmen course placements. In states withmore consequences for students, the gap betweenlow and high students closed by more than aquarter of a Carnegie unit. However, this trendwas mostly due to the high student being expectedto take fewer advanced level courses than an iden-tical student in a state with fewer consequences.

In summary, the only policy that seemed tosignificantly reduce the gap based on freshmancourse placements was linking test performanceto consequences for students, but only by possi-bly discouraging students on track toward takingadvanced level courses from doing so. Linkingtest performance to consequences for schools ap-peared to encourage freshmen in higher coursesto stay on track toward advanced mathematics,but the policy also seemed to increase inequitybetween these students and freshmen placed inlower courses.

Using a similar approach, we examined whetherstate graduation requirements and accountabil-ity policies might have mitigated or exacerbated

differences in how far students progressed in ad-vanced mathematics based on social class andrace or ethnicity. To show the cumulative effectof these state policies, we took into account ini-tial differences between groups by using the ex-pected freshman mathematics course placementfor a particular group of students in estimatingthe expected number of advanced mathematicscredits for that type of student. Table 4 shows theestimates calculated for those state policies thathad significant coefficients for the independentvariable of interest-extensiveness of testing forSES and the other three state policies for raceor ethnicity. These estimated differences wererelated to both direct effects of these policies andtheir indirect effects through freshman course en-rollments, indicating how state graduation require-ments and accountability policies may have influ-enced the process of educational stratification.

The stronger advantage of students from moreaffluent families in the accumulation of advancedmathematics credits in states with more exten-sive testing is clearly shown in the top portionof Table 4. In a state with extensive testing, astudent from a high-SES family (defined as onestandard deviation above the mean) would havebeen expected to earn over 2 Carnegie units ofadvanced mathematics credits compared to 1.210credits by an otherwise similar low-SES student.This difference was 30% larger than that betweensimilar rich and poor students in states with lesstesting, which was just under .6 of a credit. Agood portion of the effect of extensive testingwas through the freshman mathematics courseplacements such that, when freshman mathemat-ics course placements were held constant, the dif-ference between rich and poor students in stateswith extensive testing was about half a creditsmaller (not shown). Thus, monitoring students'academic progress through state mandated testingappears to have exacerbated initial differences inacademic placements based on social class.

The estimated differences in advanced mathe-matics credits earned between racial or ethnicgroups varied between states such that AfricanAmerican, and sometimes Latino/a, studentswould have been expected to eam more advancedmathematics credits than similar white studentsin some states. In a state mandatingfew academiccourses for graduation (one standard deviationbelow the mean for all states), a white studentwould have been expected to accumulate 1.756

311

TABLE 4Expected Number of Advanced Mathematics Credits Earned in High School by State Policy, Social Class andRace/Ethnicity

Extensiveness of Testinga

Student Characteristic Low High

Socioeconomic statusaLow 1.341 1.210Average 1.638 1.607High 1.934 2.003

High School Graduation Requirementsa

Race or ethnicity Few Many

African American 1.571 1.746Latino/a 1.479 1.626White 1.756 1.631

Consequences for School

Race or ethnicity None Manyb


Consequences for Student

Race or Ethnicity None Manyb


± 1 standard deviation from the mean.I 1 standard deviation from the mean.

Note. The estimated number of credits reresent the cumulative effects of social background and a given state policy by taking intoaccount expected freshman course placements.

credits in advanced mathematics compared to1.571 and 1.479 credits for a similar AfricanAmerican and Latino/a students, respectively. Instates mandating students complete many aca-demic courses to graduate, African Americanswould have been expected to accumulate slightlymore (.11) advanced mathematics credits thanwhite or Latino/a students with similar socialbackgrounds and middle school academic expe-riences. A similar but weaker pattern is also seenfor state policies linking test performance to con-sequences for schools, with African Americanand Latino/a students being expected to earn fewercredits than similar whites in states with no con-sequences for schools and slightly more credits instates with many consequences. The apparent ad-vantage of minority status reflects that these stu-dents exceeded expectations based on their socialbackground and prior academic performance and

312

that these states may have had greater equity inopportunities for learning.

In contrast, the effect of linking test perfor-mance to fewer consequences for students resultedin a greater number of advanced mathematicscredits for African American and Latino/a studentscompared to similar white students. All racial andethnic groups would be expected to earn more ad-vanced mathematics credits in states with fewerconsequences for students, with the differences be-tween groups being slightly smaller in these states.In states with many consequences for students, thedifference between Whites and similar Latino/aswas almost a quarter of a credit and almost .20 ofa credit between Whites and African Americans.These results suggest, like extensive testing, link-ing test performance to consequences for studentsmay have exacerbated differences in opportunitiesfor learning based on social background.

Discussion and ConclusionsNo Child Left Behind has the stated dual goals

of both increasing academic standards and de-creasing inequality between social and economicgroups by, among other reforms, increased use ofstandardized testing and accountability. Usingdata from the early 1990s, our results suggest thatthese policies are likely to have mixed effects onstudents' opportunities to learn mathematics inhigh school. Given the strong link between coursework and learning, these policies' effects on stu-dent achievement and inequality also will be prob-ably mixed. These findings, however, were notunexpected given the complex nature of students'academic careers over time.

Overall, our results indicated that state grad-uation requirements and accountability policieshad small but statistically significant effectson students' mathematics course taking in highschool, both on the types and number of coursestaken and on stratification related to social classand race or ethnicity. The relatively small size ofthe effects was not surprising because individualstudents' course taking patterns were influencedby many factors, such as their educational aspira-tions and schools' class schedules (Oakes, 1985;Useem, 1991). In addition, we may have under-estimated the total effect of these policies onadvanced mathematics course taking due to con-trolling on students' freshman course placements.Regardless, even small differences between stateswere important because they reflect the experi-ences of large numbers of students.

Our results indicated that increasing school ac-countability for student test performance was theonly strategy that seemed to increase all students'opportunities for learning mathematics in highschool, but especially for minority students andthose who took higher level courses as freshmen.While students in states linking test performanceto a greater number of consequences for schoolstended to enroll in similar freshman mathematicscourses as those in other states, they appeared topersist in taking the advanced level courses de-sired by competitive colleges and necessary forentry into health care, science, and technologyoccupations. One possible explanation for thiseffect is that school accountability focused stu-dents and teachers on a common goal of aca-demic excellence, with those students whoshowed potential being particularly encouraged toprogress further in the mathematics curriculum

(Muller, 1998; Roderick & Engel, 2001; Rosen-holtz, 1987). Alternatively, higher rates of ad-vanced mathematics course taking could also bethe result of at-risk students being more likely todrop out of school in these states (Schiller &Muller, 2000). More research is needed intowhether school accountability provides incen-tives to not only invest in academically able stu-dents, but also encourage students who are at-risk of failing to leave school early.

State high school graduation requirements hadmixed effects on course placements and differ-ences in enroilment based on students' social back-grounds. Students in states with more graduationrequirements tended to enter the high school math-ematics curriculum at a higher level as freshmen.However, students placed in lower level fresh-men courses in these states were less likely thanthose in other states to take advanced level coursesin mathematics. While African American studentsalso tended to take lower level courses as freshmanin states with more requirements, they appearedto overcome this disadvantage to earn more ad-vanced level credits compared to similar stu-dents in other states. These results were con-sistent with findings from similar studies in the1980s (Chaney et al., 1997; Clune & White,1992) suggesting that requiring students to takemore academic courses may promote both equityand excellence. However, our results suggest onecaveat to this pattern, students who fall behind inthe curriculum may be unable to gain access toadvanced level courses in these states.

In contrast, holding students accountable fortheir test performance tended to depress the numn-ber of advanced mathematics credits they earned,especially for freshmen who took higher levelcourses such as geometry, and among minorities.Because student accountability is not directly re-lated to course enrollments, the results for higherlevel freshman courses might be due to few in-centives for students in these states to pursuemore advanced level courses after mastering thebasics necessary to perform adequately on thestate mandated tests (Jacob, 2001; Muller, 1998).African American and Latino/a students may beparticularly adversely affected by this policybecause they are more likely to have difficultieswith standardized tests and thus subject to sanc-tions that prevent them from taking more advancedlevel courses (Catterall, 1989). As a strategy in-dependent of the others, increasing student ac-

313

Schiller and Muller

countability appears to promote neither excel-lence nor equity.

Similarly, extensive testing was related togreater stratification based on students' socio-economic status both in their freshman mathe-matics courses as well as the number of advancedmathematics credits they earned. Low-SESstudents tended to earn more advanced mathe-matics credits in states with fewer mandatedtests, while high-SES students varied very littlebetween states. Frequent testing may reinforce,rather than alleviate, academic problems throughself-fulfilling prophecies concerning the acade-mic ability of poorer children (Muller & Schiller,2000). The weaker impact of this policy on afflu-ent children may be a function of the additionalacademic support and encouragement they prob-ably received from parents, teachers, and peersfocused on college attendance. Interestingly, fre-quent testing slightly reduced the gap in the num-ber of advanced course credits earned related tofreshman course placements, but only by regres-sion toward the mean for both higher and lowerlevel students. Thus, extensive testing does notappear to have increased students' opportunitiesfor learning and tended to exacerbate social classinequality.

Our analyses suggest some general ways inwhich various state strategies for increasing stan-dards and accountability may impact students'opportunities for learning mathematics in highschool. Additional research is needed to clarifythe mechanisms through which state reform ini-tiatives shape students' academic experiences.For example, subject specific analyses might ex-amine whether students' achievement growth inmathematics or another subject is highest in theyears in which they are tested in that subject.Multiple cohort studies would also allow explo-ration of the impact of policy changes over time.Additional research should also explore whetherthese policies also have differential impacts withinstates based on school characteristics, which woulddeepen our understanding of how educationalstratification within states develops. Althoughwe did not find any differences based on schoollocation, schools serving disadvantaged popula-tions may respond differently to various statepolicies than those with more affluent students.While almost all schools offer at least some ad-vanced level courses, more affluent schools mayhave the resources to expand their offering of ad-

314

vanced level courses as demand for them increasesin response to greater school accountability. Incontrast, schools serving socially disadvantagedpopulations may have to invest their resources inbringing poor performing students up to a mini-mal level of proficiency. Both analyses of specificpolicies and changes in schools over time requirelongitudinal data, which is often expensive to col-lect but critical for understanding education as aprocess.

With increased public scrutiny of the U.S. ed-ucation system since the 1980s, state policy mak-ers started taking a more active role in adoptingreforms intended to improve students' academicachievement by raising standards and creatingaccountability systems (Timar, 1997). Instead ofstudents selecting from a plethora of courses andgood behavior rewarded as much as learning inthe "shopping mall high school," all students areexpected to take a rigorous set of core academiccourses and both schools and students are to berewarded for meeting state performance stan-dards. Our results suggest that such policies in-fluence the educational process, but are likely tohave mixed and unintended effects. Raising thebar through graduation requirements and holdingschools accountable for student achievementmay benefit all students. However, increasedamounts of student testing are likely to exacer-bate gaps in achievement between rich and poorstudents.

NotesThe NELS sample design limits the number of

transfer students in the transcript sample because onlya subsample of schools with small numbers of panelparticipants were included in the follow-up and tran-script studies (Ingles et al., 1995). About 2% of thetranscript panel sample changed states between thefirst and second follow-ups, which means that the vastmajority of the students' high school experiences wereshaped by one state's policies. No clear differences inoutcomes were found between those students who didor did not change states.

2 Although achievement tests were administered tostudents during each wave of NELS, previous analyses(Schneider & Coleman, 1993) indicate an odd relation-ship between grades and test scores in NELS that makescomparison of African Americans and whites tricky.Our preliminary analyses indicated similar odd race andethnic differences when examining course placementsover time and test scores. For example, African Amer-icans tended to be placed in higher level courses asfreshman than white students with similar 8th-grade


math test scores. More substantively, the tests given inNELS were not designed to reflect mastery of materialcovered in the students' particular courses, were notrelated to state curriculum or performance standards,and were not provided to teachers or schools. Becausegrades are used for these purposes, we chose to usethose in our models instead of test scores as indicatorsof mathematics achievement prior to high school.

3 In preliminary analyses, we included Lee's (1997)indicators of policy trends and coherence in our mod-els but dropped them when none of the coefficients forthese variables were statistically significant. Prelimi-nary analyses, in contrast, indicated that the number ofmathematics credits required by states was stronglyrelated to the number of advanced mathematics cred-its students earn but not freshman course placements.However, we were more interested in the general strat-egy of raising overall standards by increasing aca-demic course requirements, which might create con-flicting demands on students' time and negativelyimpact their likelihood of taking advanced level coursesin even a key subject like mathematics. In addition,composite indicators of course requirements and man-dated testing in the four core academic subjects wereparallel measures with those for student and school ac-countability, which could not be linked to performancein a particular subject or test. Substantively, the com-posite measures were likely to be more reliable indica-tors of general policy strategies than the adoption of aspecific policy, possibly in isolation. Another advantageof this approach was that indicators of general strategies(e.g., an emphasis on testing) were likely to be more sta-

ble than specific policies (e.g., the test used). For exam-ple, 80 states reported no increase or a moderate in-crease in testing or uses of testing from 1980. Thus,while the particular tests may have changed, the ten-dency to use them is more stable. Similarly, over 75%of states implemented testing or accountability policiesin 1989 or earlier such that they were in effect over theNELS cohort's high school careers.

4In preliminary analyses we ran some 3-level modelsnesting students within schools within states but decidedagainst using them for this article because (a) we werenot interested in school effects per se, and (b) the 3-levelmodels were substantively similar but dramaticallyless robust than the 2-level models. The latter was aresult of the number of schools per state and studentsper school in NELS being too small for estimation ofrobust 3-level models.

I The limited number of states constrains the numberof variables that can be included in the state-level mod-els. Supplementary analyses, however, indicated nosignificant differences across states in the dependentvariables or the effects of the independent variablesbased on geographic region and indicated no strong in-teraction effects between the state policy variables.

6 Analyses of NELS and other data sets collectedduring the 1990s indicated that gender differencesin mathematics course enrollments have either dis-appeared or show an advantage for girls over boys(Campbell, Hombo, & Mazzeo, 2000). Also, supple-mentary analyses indicated statistically significant vari-ation in the coefficient for gender across states, but thisvariation was not related to our state policy measures.

315

Appendix A

Source, Coding, and Descriptive Statistics for Student-Level Variables

Sample

Student-level Variables M Std Source and Coding*

Level of freshman 3.53 1.27 Obtained from high school transcripts.mathematics course.

Advanced mathematics 1.68 1.29 Obtained from high school transcripts, number of Carnegie unitscredits in algebra II, geometry, trigonometry, pre-calculus, and calculus.

Socioeconomic status .01 .73 NCES constructed variable in the Second Follow-up Student file.

Race/ethnicityAfrican-American .11 .32 Constructed from an NCES variable based on student reportsLatino/a .08 .28 with European Americans and Asian Americans as the base

category, and Native Americans excluded from analysis.

Male .49 .50 NCES variable based on student report. Coded 1 = male, and0 = female.

Living with both Parents .68 .47 Composite based on parents' and students' reports of adults inthe household. Coded 1 = both parents, and 0 = other.

Middle school 4.04 .97 8th graders' report of mathematics grades "from the sixth grademathematics grades until now."

8th-grade mathematics coursesRemedial mathematics .06 .24 8th grader's report of which mathematics courses they attendedAlgebra .37 .48 that year.

UrbanicityUrban .20 .40 Constructed from an NCES variable with the suburban as theRural .36 .48 base category.

Note. *AIl information is obtained from the National Educational Longitudinal Study of 1988-94.

Appendix B

Sources, Coding, and Descriptive Statistics for State-Level Variables

State TestingPolicy M Std Source and Coding*

High school 9.960 3.203 "Please indicate your State's high school graduation requirements for thegraduation class of 1992." The variable is the sum of the number Carnegie units requiredrequirements for a "regular diploma" in English, mathematics, science, and social studies.

Extensiveness 4.020 3.461 "At what high school grades, and in which content areas does current Stateof testing policy require that student performance be assessed?" Summed across math,a = .8949 reading, science, and history/social studies for 9th through 12th grades.

Consequences 2.066 2.189 "Does your State currently require or set guidelines for high school studentfor students testing for any of the following purposes?" Coded 2 = "state has mandatorya = .5976 policy"; 1 = "state has guidelines"; and 0 = "neither." Summed across high

school graduation, placement in remedial (compensatory education) programs,diploma eligibility, and promotion.

Consequences 1.137 1.929 "Does State policy set standards for high school's performance based on studentfor schools test results?" Coded 0 if "describes neither acceptable nor unacceptable results."cx = .8485 Otherwise, number of rewards for meeting standards (financial incentives,

official recognition/publicity, accreditation, waivers from testing or report-ing requirements, waivers from other regulations or deregulation) and sanc-tions for failing to meet standards (negative publicity, loss of accreditation,loss of control to higher educational authority).

Note. *AIl the variables in this table were responses to the questionnaire sent to State Departments of Education as part of theNational Longitudinal Study of Schools.

316

References

Bryk, A. S., Raudenbush, S., & Congdon, R. (1996).

HLM: Hierarchical linear and nonlinear modeling

with the HLM/2L and HLM/3L programs. Chicago,IL: Scientific Software International.

Bryk, A. S., & Raudenbush, S. W. (1992). Hierar-

chical linear models: Applications and data

analysis methods (Vol. 1). Newbury Park, CA:Sage Publications.

Campbell, J. R., Hombo, C. M., & Mazzeo, J. (2000).Trends in academic progress: Three decades of stu-

dent performance. Washington, DC: U.S. Depart-ment of Education.

Catterall, J. S. (1989). Standards and school dropouts: A

national study of tests required for high school grad-

uation. American Journal of Education, 98(1), 1-34.

Chaney, B., Burgdorf, K., & Atash, N. (1997). Influ-

encing achievement through high school graduationrequirements. Educational Evaluation and Policy

Analysis, 19(3), 229-244.Clune, W. H., & White, P. A. (1992). Education reform

in the trenches: Increased academic course taking in

high schools with lower achieving students in stateswith higher graduation requirements. Educational

Evaluation and Policy Analysis, 14(1), 2-20.

Elmore, R. F., Abelman, C. H., & Fuhrman, S. H.

(1996). The new accountability in state education

reform: From process to performance. In H. F. Ladd

(Ed.), Holding schools accountable: Performance-based reform in education (pp. 65-98). Washing-

ton, DC: Brookings Institution Press.Gamoran, A. (1987). The stratification of high school

learning opportunities. Sociology of Education, 60,

61-77.Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High

stakes: Testing for tracking, promotion, and gradu-

ation. Washington, DC: National Academy Press.Ingles, S. J., Dowd, K. L., Taylor, J. R., Bartot, V. H.,

Frankel, M. R., & Pulliam, P. S. (1995). National

Education Longitudinal Study of 1988 second

follow-up: Transcript component data file user's

manual. Washington, DC: U.S. Department of

Education.Ingles, S. J., Scott, L., Lindmark, J., Frankel, M. R., &

Myers, S. (1992). National Education LongitudinalStudy of 1988 first follow-up: Student componentdatafile user's manual. Washington, DC: U.S. De-

partment of Education.Jacob, B. (2001). Getting tough? The impact of high

school graduation exams. Educational Evaluation

and Policy Analysis, 23(2), 99-121.Lee, J. (1997). State activism in education reform:

Applying the Rasch model to measure trends and

examine policy coherence. Educational Evaluationand Policy Analysis, 19(1), 29-43.

Lee, J. (1998). State policy correlates of the achieve-ment gap among racial and social groups. Studies in

Educational Evaluation, 24(2), 137-152.Levine, R., & Stevenson, D. L. (1997). Changes in

America's high schools, 1980-1993: The NationalLongitudinal Study of Schools. Washington, DC:U.S. Department of Education.

Lucas, S. R. (1999). Tracking inequality: Stratifica-tion and mobility in American high schools. New

York: Teachers College Press.McDonnell, L. M. (1994). Assessment policy as per-

suasion and regulation. American Journal of Edu-cation, 102(4), 394-420.

Muller, C. (1998). The minimum competency examrequirement, teachers' and students' expectationsand academic performance. Social Psychology of

Education, 2(2), 199-216.Muller, C., & Schiller, K. S. (2000). Leveling the play-

ing field? Students' educational attainment andstates' performance testing. Sociology of Education,

73(3), 196-218.National Commission on Excellence in Education.

(1983). A nation at risk: The imperative for edu-

cational reform. Washington, DC: U.S. Govern-ment Printing Office.

Oakes, J. (1985). Keeping track: How schools struc-ture inequality. New Haven, CT: Yale University

Press.Oakes, J., & Guiton, G. (1995). Matchmaking:

The dynamics of high school tracking decisions.American Educational Research Journal, 32(1),

3-33.Raudenbush, S., & Bryk, A. S. (1986). A hierarchical

model for studying school effects. Sociology of Ed-

ucation, 59, 1-17.Roderick, M., & Engel, M. (2001). The grasshopper and

the ant: Motivational responses of low-achieving stu-dents to high stakes testing. Educational Evaluation

and Policy Analysis, 23(3), 197-227.Rosenholtz, S. J. (1987). Education reform strategies:

Will they increase teacher commitment? American

Journal of Education, 95(4), 534-562.Schiller, K. S., & Hunt, D. (2001, April). Leading

up or out: Paths through high school mathemat-

ics. Paper presented at the annual meeting of the

American Educational Research Association,Seattle, WA.

Schiller, K. S., & Muller, C. (2000). External exami-nations and accountability, educational expecta-tions, and high school graduation. American Journal

of Education, 108(2), 73-102.Schmidt, W. H., McKnight, C. C., Houang, R. T.,

Wang, H., Wiley, D., Cogan, L. S., et al. (Eds.).(2002). Why schools Matter: A cross-national com-

parison of curriculum and learning: New York:

Jossey-Bass.

317

Schiller and Muller

Schneider, B., & Coleman, J. S. (Eds.). (1993). Parents,their children, and schools. Boulder, CO: WestviewPress.

Schneider, B., Swanson, C. B., & Riegle-Crumb, C.(1998). Opportunities for learning: Course sequencesand positional advantages. Social Psychology ofEducation, 2, 25-53.

Stedman, L. C. (1997). International achievementdifferences: An assessment of a New Perspective.Educational Researcher, 26(3), 4-15.

Stevenson, D. L., & Schiller, K. S. (1999). State edu-cation policies and changing school practices: Evi-dence from the National Longitudinal Study ofSchool, 1980-1993. American Journal of Educa-tion, 107, 261-288.

Stevenson, D. L., Schiller, K. S., & Schneider, B.(1994). Sequences of opportunities for learning.Sociology of Education, 67, 184-198.

Timar, T. B. (1997). The institutional role of stateeducation departments: A historical perspective.American Journal of Education, 105(3), 231-260.

Useem, E. L. (1991). Student selection into coursesequences in mathematics: The impact of parentalinvolvement and school policies. Journal of Researchon Adolescence, 1, 231-250.

Wells, A. S., & Oakes, J. (1996). Potential pitfalls ofsystemic reform: Early lessons from research ondetracking. Sociology of Education, 135-143.

Willms, J. D., & Raudenbush, S. W. (1989). A longi-tudinal hierarchical linear model for estimatingschool effects and their stability. Journal of Educa-tional Measurement, 26(3), 209-232.

Authors

KATHRYN S. SCHILLER is Assistant Professor,SUNY, The University at Albany, Department of Ed-ucational Administration and Policy Studies, Albany,NY 12222. [email protected]. Her areas of spe-cialization are sociology of education, organizations,and opportunities to learn.

CHANDRA MULLER is Associate Professor, Uni-versity of Texas at Austin, Department of Sociologyand Population Research Center, 1 University StationG1800, Austin, TX 78712, [email protected] areas of specialization are the effects of curricu-lum on adolescents' achievement, health and social re-lationships.

Manuscript Received August 2, 2002Revision Received May 21, 2003

Accepted July 25, 2003

318

COPYRIGHT INFORMATION

TITLE: Raising the Bar and Equity? Effects of State HighSchool Graduation Requirements and AccountabilityPolicies on Students’ Mathematics Course Taking

SOURCE: Educ Eval Policy Anal 25 no3 Fall 2003WN: 0328803465004

The magazine publisher is the copyright holder of this article and itis reproduced with permission. Further reproduction of this article inviolation of the copyright is prohibited. To contact the publisher:http://www.aera.net/

Copyright 1982-2003 The H.W. Wilson Company. All rights reserved.

raising the bar and equity? effects of state high school

Documents