american education reform: lessons in the last decade · figure 3: expenditure per student and...

American Education Reform:

Lessons in the Last Decade

Parag A. PathakMIT

1/55

Introduction

National Center for Education Statistics (2010); 2007-08 CPI adjusted dollars

2/55

Introduction

Figure 3: Expenditure per student and student achievement across countries

350

400

450

500

550

0 10000 20000 30000 40000 50000 60000 70000 80000

Math performance in PISA 2003

Cumulative educational expenditure per student

Mexico

Belgium

Iceland

FranceSweden

SwitzerlandDenmark

AustriaNorway

USA

ItalyPortugal

Spain

Korea

GermanyIreland

Czech Rep.

HungaryPoland

Slovak Rep.

Greece

Finland

NetherlandsCanada

Japan

Australia

R 2 = 0.15

R 2 = 0.01

Notes: Association between average math achievement in PISA 2003 and cumulative expenditure on educational institutions per student between age 6 and 15, in US dollars, converted by purchasing power parities. Dark line: regression line for full sample. Light line: regression line omitting Mexico and Greece. Source: Woessmann (2007a).

3/55

Introduction

4/55

Introduction

Math, 17 year olds, National Assessment of Educational Progress

from Nation’s Report Card

5/55

Introduction

Black-White Achievement Gap on NAEP in 2010

6/55

Introduction Schools and Gaps

Background: Achievement Gaps

Whatever pathology may exist in Negro families is far exceededby this social pathology in the school system that refuses toaccept a responsibility that no one else can bear and thenscapegoats Negro families for failing to do the job . . . The jobof the school is to teach so well that family background is nolonger an issue. – Martin Luther King (1968, Where Do We Gofrom Here?)

Black and Hispanic students still score substantially lower thanWhites on achievement tests in all grades

Similar gaps by family background

Public schools still struggle to close these gaps

Question: Can schools alone can ever close the achievement gap?

Coleman Report Equality of Educational Opportunity 1966; schoolfunding trumped by student and family background

7/55


Schools and Gaps

Three skeptical views

X Hernstein and Murray’s (1994) Bell Curve reflects genetic determinism;social scientists typically see this as discredited

X Rothstein (2004) argues for a kind of social determinism:

. . . there is nothing illogical about a belief that schools, ifwell-operated, can raise lower-class achievement without investing inhealth, social, early childhood, after-school, and summer programs.But while the belief is not illogical, it is implausible, and the manyclaims made about instructional heroes or methods that close that gapare, upon examination, unfounded.

X Heckman (2005) lays out a more sophisticated critique of theschool-centered approach:

Late remediation, no matter how extensive, cannot restore childrenfrom disadvantaged environments to the level of performance theywould have attained had they received economically efficient earlyinterventions that compensate for disadvantage in the early years.

8/55


Is US education broken?

Two views

1) Need more inputs

2) Need to radically rethink current system: incentives, vouchers,charters, accountability

Example: class size debate“It would not be prudent to radically restructure the U.S. educationsystem out of misplaced frustration that the current system has failedmiserably or out of an unsupported presumption that progress cannotbe made in the context of existing system.” - Alan Krueger

9/55

Introduction

Two areas of major progress in last decade

1) Student assignment mechanisms: ways to assign students fairly andtransparently

2003: New York City adopts a new centralized mechanism2005: Boston changes the rules of their existing centralized mechanism2007: England bans ‘First Preference First’ mechanisms nationwide2009: Chicago abandons mechanism midstream2012: Denver and NOLA adopt new mechanisms, combining charterand district school admissions in one process2013: Washington DC

2) Understanding school effects

Charter schoolsSelective exam schoolsSmall schoolsSchool takeovers

10/55

Student Assignment Overview

Residence-based school assignment vs. school choice

Residence-based assignment⇒ residential choice determine schools, so housing market “prices” access

School choice⇒ separate link between residential choice and school access, allow some

to leave neighborhood in effort to equalize access

Brown vs. Board (1954): “separate educational facilities areinherently unequal”

1970s: court-supervised forced busing

X Boston in 1974: If school more than 50% nonwhite,then it was racially imbalanced

X Judge Garrity’s 14-year court supervision of BostonPublic Schools, longest anywhere

11/55


“The Soiling of Old Glory” by Stanley J. Forman1977 Pulitzer Prize for Spot Photography

12/55


1980-90s: introduce element of choice into busing plans

Rise of open enrollment or school choice

X Already present since families can “vote with their feet”X As of 2008, greater numbers in open enrollment than in charters and

vouchersX PICS vs. Seattle/Louisville (2007): “best way to stop discriminating

on the basis of race is to stop discriminating on the basis of race”

Cited rationales include equity considerations; desire to break linkbetween housing market and school options; introduce quasi-marketforces into education

Active debates about rationing oversubscribed schools

X E.g., zone geographies, proximity set-asides, sibling and family-linkpolicies

X Major fault-line of debate: pro-neighborhood vs. pro-choice

13/55


Important Ingredients

Primitives

1. a set of students I = {i1, ..., in},2. a set of schools S = {s1, ..., sm},3. a capacity vector q = (qs1 , ..., qsm),

4. a list of strict student preferences P = (Pi1 , ...,Pin), and

5. a list of strict school priorities π = (πs1 , ..., πsm).

matching µ : I → S is a function from the set of students to the set ofschools such that no school is assigned to more students that its capacity.

mechanism: systematic way to compute a matching for each problem.

14/55


Example of a mechanism

Gale and Shapley (1962) defined the following deferred acceptancealgorithm for the marriage problem. We can adapt it for our many-to-onesetting:

Step 1: Each student proposes to her first choice. Each schoolrejects any unacceptable student, and holds the most preferred set ofapplicants up to capacity.

In general, at

Step k: Each student who was rejected in the previous step proposesto her next choice. Each school “holds” her most preferredacceptable offers to date, and rejects the rest.Algorithm terminates after a step where no more rejections.

Properties: produces best stable matching for students, truthful reportingis dominant strategy for students

15/55

Student Assignment Boston

Student Assignment in Boston (from 1999-2005)

Over 60,000 students from grades K-12 in almost 140 schools, dividedinto three zones: East, West, and North.

Main new school entry points are K2, 6th and 9th grade: about 3,300entering Kindergarten, 5,400 entering grade 6, and about 6,300entering grade 9.

In January, students asked to rank at least three schools in order ofpreference.

For elementary and middle school, parents are asked to considerschools in their zone plus five schools open to all neighborhoods.High school admissions are citywide.

16/55


3-zone Map

!

!

!

!

!!

!!

!!!!

!

!

!!!!!!

!!

! !

!!

!!!!

!!!

!

!

!!!!!

!

!!

!

!!

!!

!!

!!!!

!!

!!

!!!!!

!

!!

!!! !!!

!!

!!

!

!

!!!

! !!

!

!!

!

!!!!

!!

!

!!!!

!

!!

!

!!

!!

!!

!!

!!

!!

!

!

!

!!

!

!!

!!

!!

!

!

!!

!!

!

!!

!!!!

!!

!!

!!!!

!

!

!

!!

!

!!

!! !!

!!

!!!!

!

!!!!

!!

!!!!

!!

!!

!!

!

!!

!!

!!

!!!

!

!!

!

!

!!

!!

!

!!!!

!

!!

!!

!!

S. Boston

NORTH ZONE

Kennedy HCA

ECC at Fifield

Boston Latin School 7-12

Kennedy HCA

Dudley St NCS

WEST ZONE

EAST ZONE

Allston-Brighton

W. Roxbury

E. Boston

Roxbury

Hyde Park

Roslindale S. DorchesterMattapan

Jamaica Plain

S. End

N. Dorchester

Fenway-Kenmore

Charlestown

ACC

BATA

Grew

CASH

Hale

Orchard Gardens K-8

BDEA

Clap

Otis

Dever

Bates

Haley Kenny

Ellis

Tynan

Mason

Adams

Guild

Condon

Mozart

Sumner

Conley

Holmes

Taylor

Mather

Bradley

Everett

Gardner K-8

Winship

Manning

Mendell

Holland

Trotter

Russell

Perkins

Burke HS

Lyon K-8

Channing

Chittick

Marshall

King K-8

Winthrop

S. Boston EC:Excel HS

Green Academy

Umana Academy

Lyon 9-12

Arts Academy HSFenway HS

Tobin K-8

Irving MS

Rogers MS

Philbrick

Mattahunt

Henderson

Harbor MS

Perry K-8

Eliot K-8

McKay K-8

O'Donnell

Edison K-8

Lyndon K-8

JF Kennedy

English HS

TechBoston

Murphy K-8

Haynes EEC

Timilty MS

Gavin Complex:UP AcademyMS Academy

Blackstone

Hurley K-8

Snowden HS

EdwardsMS

PJ Kennedy

Baldwin ELPA

Brighton HS

McKinley MS

Dearborn MS

McCormack MS

E. Greenwood

Frederick MS

Harvard/Kent

West Zone ELCHennigan

Greater Egleston HSHernández K-8

Carter Center

Community Acad

Newcomers Acad/BIHS

Charlestown HS E. Boston HS

Ohrenberger 3-8

Mildred Ave 4-8Ellison/Parks EES

Dorchester Acad

O'Bryant 7-12

Madison Park HS

East Boston EEC

S. Greenwood K-8

Jackson/Mann K-8Horace Mann K-12

Beethoven (K1-3)

McKinley Prep HS

W. Roxbury EC:W. Roxbury Academy

Boston Latin

Academy7-12

Urban Science Academy

Kilmer K-8(Upper)

Lee Academy (K0-1)Lee School (2-6)

Quincy (K-5)

Kilmer K-8 (Lower)

Curley K-8

Young Achievers

K-8

HigginsonLewis K-8

Quincy Upper (6-12)

Warren/Prescott K-8

BTU Pilot K-8

McKinley S. End Acad & K-5

Roosevelt K-8(Upper)

Roosevelt K-8(Lower)

Milton

Quincy

Newton

Dedham

Brookline

Cambridge

Belmont

Braintree

Somerville

Watertown

EverettChelsea

Revere

Westwood

Arlington

Winthrop

Medford

Canton

BPS Strategic PlanningUpdated August 10, 2012

Boston Public SchoolsSY 2012 - 2013 µ

0 0.5 10.25Miles

LegendELC/EECK-5K-8Middle School

High SchoolSpecial

!!

!!

!!

!!

!!

!!

6/7-12!!

Muñiz AcademyMission Hill K-8

Hyde Park EC:New Mission HS

BCLA

17/55


Boston Mechanism (before 2006)

X For each school a priority ordering is determined according to thefollowing hierarchy:

1) First priority: sibling and walk zone2) Second priority: sibling3) Third priority: walk zone4) Fourth priority: other students

Students in the same priority group are ordered based on an evenlottery.

X Each student submits a preference ranking of the schools (with noconstraint)

X The final phase is the student assignment based on preferences andpriorities:

18/55


Round 1: In Round 1 only the first choices of the students are considered.For each school, consider the students who have listed it as their firstchoice and assign seats of the school to these students one at a timefollowing their priority order until either there are no seats left or there isno student left who has listed it as her first choice.

In general, at

Round k: Consider the remaining students. In Round k only the kth

choices of these students are considered. For each school with stillavailable seats, consider the students who have listed it as their kth choiceand assign the remaining seats to these students one at a time followingtheir priority order until either there are no seats left or there is no studentleft who has listed it as her kth choice.

19/55


Performance of Boston mechanism

Typical year in Boston (2001-02):

K2 6 9

1st choice 2,598 4,157 5,4972nd choice 301 415 4283rd choice 131 294 1004th choice 61 61 425th choice 33 26 11Unassigned 202 476 302

Roughly 80% get their top choice, 8% get 2nd choice, ..., 5-9%unassigned

Similar patterns across years

Issue: Are these true preferences?

20/55


Nov 2003-June 2005: Ongoing discussion in Boston about schoolchoice, one aspect was assignment mechanism

July 2005: Boston School Committee voted to change their studentassignment mechanism to the student-optimal stable mechanism

X Stability: no student and school who want to block the allocationX Student-optimal stable mechanism based on student-proposing deferred

acceptance algorithm from marriage market lecture

Superintendent Thomas Payzant’s Report to School Committee(5/11/2005): “A strategy-proof algorithm levels the playing field bydiminishing the harm done to parents who do not strategize or do notstrategize well.”

21/55


Parent comments at public hearing

I find the current system of maximizing first choice to be insidious anddestructive. I urge each school committee member to vote enthusiasticallyfor this new algorithm proposal. [...] My wife and I take dozens of phonecalls around choice time in Dorchester. We have to tell people that itdoesn’t make sense to choose our children’s elementary school. And thatis absurd. And the people who get that advice get very angry. [...]Because to get into the O’Hearn you need to be luckier than megabucks.So I have to say [to these parents], don’t make your first choice your firstchoice. That’s enraging. It is at the bottom of the anger that you [theSchool Committee] get from West Roxbury.

It angers the parents who figure it out because they are told not to maketheir first choice the first one. And it hurts those who don’t figure it outbecause they choose a popular school and end up in the administrativeassignment bin.

22/55


Information of Participants

Why might parents understand?

BPS School Guide (2004, p3, quotes in original):

For a better chance of your “first choice” school...consider choosing less popular schools.

Advice from the West Zone Parent’s Group:Introductory meeting minutes, 10/27/03

One school choice strategy is to find a school you like that isundersubscribed and put it as a top choice, OR, find a school thatyou like that is popular and put it as a first choice and find a schoolthat is less popular for a “safe” second choice.

⇒ Evidence of sophisticated behavior among some players,unsophisticated behavior by others.

23/55

Student Assignment Chicago

Chicago Sun-Times November 12, 20098th-graders’ shot at elite high schools better

Poring over data about eighth-graders who applied to the city’s elitecollege preps, Chicago Public Schools officials discovered an alarmingpattern.

High-scoring kids were being rejected simply because of the order inwhich they listed their college prep preferences.

“I couldn’t believe it,” schools CEO Ron Huberman said. “It’s terrible.”

CPS officials said Wednesday they have decided to let anyeighth-grader who applied to a college prep for fall 2010 admissionre-rank their preferences to better conform with a new selection system.

Previously, some eighth-graders were listing the most competitivecollege preps as their top choice, forgoing their chances of getting intoother schools that would have accepted them if they had ranked thoseschools higher, an official said.

Under the new policy, Huberman said, a computer will assignapplicants to the highest-ranked school they quality for on their list.

“It’s the fairest way to do it.” Huberman told Sun-Times.24/55


Chicago Public Schools

9 selective high schools

Applicants: Any current 8th

grader in Chicago

Composite test score:entrance exam + 7th gradescores

Up to Fall 2009, systemworked as follows:

Take admissions test

Rank up to 4 schools

©2010 Google - Map da

chools

20

il.us/

IENCE ACADEMY HS

PARATORY HIGH SCHOOL

EPARATORY HIGH SCHOOL

H SCHOOL

GH SCHOOL

E PREPARATORY ACADEMY HS

ATORY HIGH SCHOOL

RSS Print Send Link

To see all the details that are visible on thescreen, use the "Print" link next to the map.

Selected Enrollment High Schools - Google Maps http://maps.google.com/maps/ms?hl=en&ie=UTF8&msa=0&m...

1 of 2 9/23/10 12:42 PM

25/55


Chicago Selective Enrollment Mechanism

Round 1: Only the first choices of the students are considered. Foreach school, consider the students who have listed it first. Assignschool seats to these students following their composite test scoreuntil either there are no seats left or there is no student left listing itas her first choice.

In general, for k = 2, ..., 4

Round k: For the remaining students, only the kth choices areconsidered. For each school with still available seats, consider thestudents who have listed it as their kth choice. Assign the remainingschool seats to these students following their composite test scoreuntil either there are no seats left or there is no student left listing itas her kth choice.

26/55

Student Assignment England

English context

Forms of school choice for decades

2003 School Admissions Code

� “National Offer Day”: coordinated admissions nationwide, underauthority of Local Education Authority; 800,000 students given offer

2007 School Admissions Code

� Strengthened enforcement of admissions rules

Section 2.13: In setting oversubscription criteria the admission authorities for

all maintained schools must not:

give priority to children according to the order of other schools named

as preferences by their parents, including ’first preference first’

arrangements.

27/55


A first preference first system is any “oversubscription criterionthat gives priority to children according to the order of other schoolsnamed as a preference by their parents, or only considers applicationsstated as a first preference” (School Admissions Code, 2007, Glossary,p. 118).

Best-known first preference first system is Boston mechanism(pre-2005)

Rationale given by Dept. for Ed & Skills (2007):

“‘first preference first’ criterion made the system unnecessarily complex to

parents who had to play an ‘admissions game’ with their children’s future”

Echoes themes from our 2003-05 Boston discussion, wherepolicymakers said the new mechanism (BPS 2005):

“adds transparency and clarity to the assignment process, by allowing for

clear and straightforward advice to parents regarding how to rank schools.”

28/55


Ban of FPF Mechanism in 2007

2007 Admissions Code outlaws FPF at more than 150 Local EducationAuthorities (LEAs) across the country; continued through 2012

Some LEAs abandoned earlier:

Pan London Admissions Authority adopted an ‘equal preference’system in 2005 (=student-optimal stable mechanism)

designed to “eliminate the need for tactical preferences and make the

admissions system fairer”; it will “create a level playing field for school

admissions”

cf. June 2005 comments by Boston superintendent that newalgorithm

“levels the playing field by diminishing the harm done to parents who do not

strategize or do not strategize well.”

In 2006, Coldron report: 101 LEAs used equal preference, 47 usedfirst preference first

29/55

School assignment: some lessons

Central divide: neighborhoods and choice

X Many policy instruments used: zones, priorities, etc

Given these rules, choice systems often designed poorly

X Single vs. multiple offer systems; congestionX Decision-making aids, fairs, outreachX Incentive issues, disadvantage the unsophisticatedX Rules need to be transparent, available for auditing

Try as much as possible to set policies first, and use good mechanismto implement those policies

30/55

School Effectiveness

School Effectiveness Research

31/55

School Effectiveness

Measuring School Effectiveness

The challenge in measuring the effect of attending a school is thatstudents who attend that school differ in observed & unobserved waysfrom those who do not

X Attributing achievement differences to a student’s school may beconfounded by differences between students

The lottery method finds students for whom the random numberdetermines assignment, holding everything else equal

Since lottery winners are more likely to enroll in particular schools,when we compare subsequent outcomes of winners vs. losers, theseidentify the causal effect of attending the school

X Method allows us to adjust for any unobserved differences inmotivation or background

X Instrumental variables handles partial compliance

Its also possible to look for things that approximate lotteries, likesharp cutoffs (‘local’ randomization)

32/55

School Effectiveness Charters

Charter Schools

Charter schools are publicly funded, but operate with minimalsupervision

X Nonprofits, universities, teachers, or parents can open charters; nofor-profit in Massachusetts

X Charters are granted by the state DOEX Each Charter runs as its own districtX Charters often adhere to a formula; most of in Boston are “No

Excuses,” similar to KIPP, a national franchise

State Charters are funded through tuition paid by sending districts

X Tuition ≈ senders’ average per-pupil expenditureX Since 1999, senders’ tuition is partially reimbursed by state (determined

by growth in costs)

33/55


Key Charter Features

State Charters are outside local collective bargaining agreements

X State Charters hire, fire, and have loose work rules much like privateschools

X Charter teachers need not be certified, but must pass the state ed testin first year of work

Charter schools are meant to be accountable

X A charter is subject to periodic review; may be suspended, revoked, ornon-renewed

X Accountability criteria: success of academic program; organizationalviability; faithfulness to a charter

X As of 2009, of 75 charters granted in Mass., 9 have been lost

34/55


0ptFigure 1: Complier Distributions for MCAS Scaled Scores

Notes: This figure plots smoothed MCAS scaled score distributions for treated and untreated compliers. The sample is restricted to lottery applicants who are projected to graduate between 2006 and 2013 assuming normal academic progress from baseline. Dotted vertical lines at scaled score 220 mark MCAS needs improvement thresholds, 240 for MCAS proficiency thresholds, and 260 for MCAS advanced thresholds.

Angrist, Cohodes, Dynarski, Pathak and Walters (2014)

35/55


0ptFigure 2: Complier Distributions for SAT Scores

Notes: This figure plots smoothed SAT score distributions for treated and untreated compliers. The sample is restricted to lottery applicants who are projected to graduate between 2007 and 2012.

Angrist, Cohodes, Dynarski, Pathak and Walters (2014)

36/55


0pt

Mean Effect Mean Effect

(1) (2) (3) (4)

Any 0.484 0.063 0.601 0.115

(0.072) (0.084)

Two-year 0.121 -0.106** 0.183 -0.058

(0.051) (0.064)

Four-year 0.363 0.170** 0.418 0.173**

(0.070) (0.079)

Four-year Public 0.135 0.154*** 0.145 0.195***

(0.059) (0.070)

Four-year Private 0.228 0.016 0.273 -0.022

(0.076) (0.094)

Four-year Public In MA 0.116 0.116** 0.121 0.146**

(0.054) (0.063)

N 2599 1887

Enrolled ImmediatelyEnrolled Immediately or One-

year Later

Table 8: Lottery Estimates of Effects on College Enrollment

Panel A: Attendance at Any NSC-Covered School

Angrist, Cohodes, Dynarski, Pathak and Walters (2014)37/55


0ptFigure 2: School-specific Treatment Effects

A. Observational Estimates

B. Lottery Estimates

Notes: These figures plot school-specific math effects against school-

specific ELA effects. Middle and high-school estimates are pooled to

create the figures.

←↩

42/42

Angrist, Pathak and Walters (2013)

38/55

School Effectiveness Small schools

Small schools in NYC

Attrition Adjustment

Regents Subject(1) (2) (3) (4) (5) (6)

Math 0.458*** 0.062** 0.136** 0.131** 0.132*** 0.119***(0.058) (0.030) (0.063) (0.066) (0.040) (0.040)

N 7,494 7,494 7,494 7,493 7,448 10,058

English 0.775*** 0.083** 0.107** 0.107** 0.080 0.072(0.102) (0.041) (0.053) (0.054) (0.051) (0.049)

N 7,224 7,224 7,224 7,223 7,148 10,037

Living Environment 0.449*** 0.079*** 0.176*** 0.169*** 0.189*** 0.178***(0.061) (0.023) (0.059) (0.057) (0.050) (0.045)

N 6,976 6,976 6,976 6,975 6,940 10,058

Global History 0.672*** 0.074** 0.111* 0.108* 0.100* 0.112*(0.070) (0.036) (0.060) (0.059) (0.052) (0.059)

N 7,145 7,145 7,145 7,144 7,071 10,037

US History 0.785*** 0.066*** 0.085** 0.085** 0.066** 0.059**(0.139) (0.021) (0.035) (0.035) (0.029) (0.029)

N 6,656 6,656 6,656 6,655 6,584 10,037

*significant at 10%; ** significant at 5%; *** significant at 1%.

Note: This table reports 2SLS estimates of the effect of years spent in a small school on Regents test outcomes. The instrument is an indicator for obtaining a lottery offer. All models include year of test and risk set controls. Column (4) adds demographic controls, while column (5) adds baseline test scores. Demographic controls include birthdate and dummies for female, black, Hispanic, white, asian, Manhattan, Brooklyn, Bronx, and Queens. We use baseline Math for Math and Living Environment, and baseline Reading for English, Global History, and US History. In column (6), for any students who with missing outcome score, we impute the student's baseline score. Math baseline score is used for Regents Math and Living Environment. English baseline score is used for Regents English, Global History, and US HIstory. In columns (7) and (8), we report estimates of upper and lower bounds without dummies of test year and risk sets, demographic and baseline controls following Lee (2009). Robust standard errors custered by school and year of test are in parenthesis.

Table 4. Lottery Results Basic Controls 2SLS w/ Additional Controls

First StageReduced Form 2SLS

Demographics Demographics + Baselines

Treat missing test score as baseline score

Abdulkadiroglu, Hu and Pathak (2014)

39/55

School Effectiveness Small schools

0pt

Mean for Non-‐Offered All Small Schools(1) (2)

Ever attend college 0.365 0.071**(0.028)

N 4,633 8,573

Attend within four years 0.283 0.069***(0.026)

N 4,633 8,573

Attend within six years 0.379 0.064**(0.032)

N 3,740 7,095

Ever attend a 2 year college 0.196 0.036(0.028)

N 4,633 8,573

Ever attend a 4 year college 0.205 0.060***(0.020)

N 4,633 8,573

Ever attend a 4-‐year public college 0.135 0.066***(0.021)

N 4,633 8,573

Ever attend a 4-‐year private college 0.087 0.007(0.019)

N 4,633 8,573

Ever attend CUNY 0.247 0.070**(0.028)

N 4,633 8,573

At least 2 academic semesters attempted 0.243 0.049*(0.025)

N 4,633 8,573

At least 4 academic semesters attempted 0.157 0.035(0.023)

N 3,740 7,095

Table 7. Lottery Estimates on College Choice, Enrollment, and Persistence

Panel A. College Choice and Enrollment

Panel B. College Persistence

Notes: This table reports 2SLS estimates of small school attendance on measures of credits per year and attendance, where the endogenous variable is attend small school in 9th grade and instrument is an indicator for offer. We drop applicants in the 2003-‐04 cohort because we have few observations in the NSC file for this cohort. We code "attend college" as 1 if students could be matched with the NSC file and their college name and enrollment dates are not missing. An academic semester is enrollment between January and May or enrollment between August and December. The attend within six years and persistence results for 4 academic semesters are only for application cohorts from 2004-‐05, 2005-‐06, and 2006-‐07. Robust standard errors clustered on 9th grade school are reported in parenthesis. *significant at 10%; **significant at 5%; ***significant at 1%.

Abdulkadiroglu, Hu and Pathak (2014)

40/55

School Effectiveness Exam schools

What’s an Exam School?

A public school with a competitive admissions exam

A magnet school, focused on academics, with selective admissions

Boston and New York run the most famous and longstanding examprograms; Found throughout US

Often seen as the flagship of a public school system, vigorouslydefended by teachers and alumni

Selective exam schools admissions are controversial, generatingconcerns about elitism, segregation, and racial achievement gaps

As at elite colleges and universities, exam school students are clearlyhigher-achieving than most public school peers

Our question:

What’s the causal effect of Boston and New York exam schoolattendance on student achievement?

RD provides the answer

41/55


Boston Exam Schools

Three traditional exam schools spanning grades 7-12

X Boston Latin School (1635)X Boston Latin Academy (1877)X John D. O’Bryant School of Math and Science (1893)

BLS is America’s oldest public high school

X Roots of public high school movement (Goldin and Katz 2008);Imitators include Brooklyn Latin, opened in 2006

X BLS boasts that Harvard was created for its graduatesX BLS appears in U.S. News and World Report top 20

Racial preferences in exam school admissions

X Began with Judge Garrity in 1974X After McLaughlin vs. Boston School Committee in 1996, exam school

admissions made race-blind

Exams offer more advanced courses, but have larger classes

42/55


New York Exam Schools

Three original academic exam schools, opened early 20th CenturyX Stuyvesant High SchoolX The Bronx High School of ScienceX Brooklyn Technical High School

Three new exam schools opened in 2002; 1 more opening and 1conversion sinceX New exams are much smaller, younger, some too young for us to have

good coverage, and in any case, arguably a different animalX Our NYC design parallels Boston in focusing on the 3 most established

schools (similar metrics)

Stuyvesant is most competitive (Stuy students 2σ above NYC),followed by Bronx Science, then Brooklyn Tech; these schools offer arich and advanced curriculumAdmissions controversiesX 1972 Hecht-Calandra Act legislated exam-only admitsX Specialized High School Institute (SHSI) opened in 1995 to prep

minority applicants

43/55


What’s the Exam School Treatment?

1) Peer Effects

Exam school students study with peers who have similarly high levelsof achievement

2) Tracking

Advanced courses, links with local scientists, math and sciencecompetitions, other academic enrichment activities

3) Resources

Some have modern science labs and well-equipped athletic facilitiesAlumni networks and donationsMore senior teaching staff (though not necessarily the best teachers)On other hand, exam classes are larger (21 vs. 15 in Boston; 31 vs. 27in NYC), exam schools sometimes said to be crowded

Because the resource comparison is mixed, we see the exam schoolexperiment as most informative about a combination of peer andtracking effects

44/55


0.2

.4.6

.81

−20 −10 0 10 20

O’Bryant

0.2

.4.6

.81

−20 −10 0 10 20

Latin Academy

0.2

.4.6

.81

−20 −10 0 10 20

Latin School

Figure 3. Enrollment at Any Boston Exam School for 7th Grade Applicants (1997-2008)

45/55


−.5

0.5

11.

52

2.5

−20 −10 0 10 20

O’Bryant

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin Academy

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin School

Figure 5. Average Baseline Math Scores of Peers for

7th Grade Applicants (1997-2008) in Boston

46/55


−.5

0.5

11.

52

2.5

−20 −10 0 10 20

O’Bryant

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin Academy

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin School

Figure 8. 7th (2006-2009) and 8th (1999-2009) Grade Math Scores for

7th Grade Applicants (1997-2007 / 2005-2008) in Boston

47/55


−.5

0.5

11.

52

2.5

−20 −10 0 10 20

O’Bryant

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin Academy

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin School

Figure 9. 7th (2001-2009) and 8th (2006-2009) Grade English Scores for

7th Grade Applicants (2000-2008 / 2004-2007) in Boston

48/55


−.5

0.5

11.

52

2.5

−20 −10 0 10 20

O’Bryant

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin Academy

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin School

Figure 10. 10th Grade Math (2003-2009) Scores for

7th (1999-2005) and 9th (2001-2007) Grade Applicants in Boston

49/55


−.5

0.5

11.

52

2.5

−20 −10 0 10 20

O’Bryant

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin Academy

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin School

Figure 11. 10th Grade English (2003-2009) Scores for

7th (1999-2005) and 9th (2001-2007) Grade Applicants in Boston

50/55


−.5

0.5

11.

52

2.5

−20 −10 0 10 20

O’Bryant

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin Academy

−.5

0.5

11.

52

2.5

−20 −10 0 10 20

Latin School

Figure 16. SAT Scores for 7th (2000-2005) and 9th (2001-2006) Grade Applicants in Boston

51/55


−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Brooklyn Tech

−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Bronx Science

−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Stuyvesant

Figure 22. Average Baseline Math Score of Peers for

9th Grade Applicants (2004-2007) in NYC

52/55


−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Brooklyn Tech

−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Bronx Science

−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Stuyvesant

Figure 24. Advanced Math Regents Scores for 9th Grade Applicants (2004-2007) in NYC

53/55


−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Brooklyn Tech

−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Bronx Science

−.5

0.5

11.

52

2.5

−10 −5 0 5 10

Stuyvesant

Figure 25. English Regents Scores for 9th Grade Applicants (2004-2007) in NYC

54/55

School effectiveness: summary

Lots of evidence that “No Excuses” charters in urban areas producelarge achievement effects for disadvantaged children

X Children who benefit the most, least likely to apply

Mostly mixed evidence on alternative models

X NYC’s small school effect size comparable to charters, but elsewheresmall schools not as successful ⇒ autonomy and brand new school

Value-added at Boston and New York’s selective exam schools seemsentirely driven by selection

All studies found at seii.mit.edu

55/55

american education reform: lessons in the last decade · figure 3: expenditure per student and...

Documents