Unraveling of markets

Al Roth

Market Design

2

Stages and transitions observed in various markets

Stage 1: UNRAVELINGOffers are early, dispersed in time, exploding…no thick market

Stage 2: UNIFORM DATES ENFORCEDDeadlines, congestion

Stage 3: CENTRALIZED MARKET CLEARING PROCEDURES

3

Avery, Christopher, Christine Jolls, Richard A. Posner, and Alvin E. Roth, “The Market for Federal Judicial Law Clerks”, U.

of Chicago Law Review, 68, 3, Summer, 2001, 793-902.

• The market for clerkships starting e.g. in 2003 cleared in September 2001, i.e. at the very beginning of the first semester of the second year of law school; more than a year and a half before the start of employment.

• This introduces several potential causes of inefficiency. – because the market cleared before information became available

(e.g. students’ second and third year grades, law review articles, etc.) that can help produce efficient matches of clerks and judges.

– because competition among judges to hire earlier than their competitors made the market fast, chaotic, and thin, many students and judges had little opportunity to consider a wide range of options, but rather had to transact quickly, before options could be developed.

• (The market for papers in law reviews is pretty interesting too, incidentally…

4

The proximate cause of this study:Memo from the Administrative Office of the United States Courts, to All United States

Judges, October 7, 1998.

“At its September 15, 1998 session, the Judicial Conference of the United States rescinded its September 1993 policy recommending to all judicial officers that March 1 of the year before a clerkship begins be the benchmark starting date for law clerk interviews. This action was taken because the policy on law clerk interviews has not been universally followed...”

5

We tracked the market as it unraveled, through interviews and surveys and records

of final transactions.• The basic chronology of these markets (in which

offers became earlier from year to year) is:– interviews lead very quickly to offers – offers produce very quick responses – responses are generally acceptances; and– many scheduled interviews are canceled as a result.

• Thus, students and judges tend to pair off quickly with those with whom they have early interviews. As a result, – many students limit the judges to whom they apply to

avoid being paired off early with a less preferred judge

• We also witnessed complex but binding verbal contracts.

6

Interviews lead very quickly to offers (1999, 2000) Time between first offer and

interview with the offering judge

Percent of

responding

students

Offer made at end of interview 34%

1-2 days elapsed between interview

and offer

23%

3-4 days elapsed between interview

and offer

9%

5-7 days elapsed between interview

and offer

15%

1-2 weeks elapsed between interview

and offer

8%

More than 2 weeks elapsed between

interview and offer

11%

Total number of responses: 101

7

First offers from early interviewsInterviewing producing first

offer

Percent of

responding

studentsa

First interview produced first

offer

36%

Second interview produced first

offer

23%

Third interview produced first

offer

19%

Fourth interview produced first

offer

8%

Fifth or subsequent interview

produced first offer

15%

Total number of responses: 102

8

Offers lead to quick replies

9

Responses are generally “yes”…Offer information Percent of

responding students

Cumulative percentage

First offer was accepted

Yes No

73% 27%

73% 100%

Number of offers 1 2 3 4 5 6

68% 25% 3% 2% 1% 1%

68% 93% 96% 98% 99%

100%

10

…even when the first offer isn’t the first choice

11

Form of offersMany Varieties of Exploding Offers• “[A particular judge] has a policy of often making offers on the spot and expecting

an immediate response. He told me this when contacted for the interview.” • "The judge offered me the clerkship (unofficially) on 2/26 (during the interview)

provided I would guarantee I would accept when formally offered on 3/1. I asked for 24 hrs. to consult my wife, but he said he couldn’t give me 24 hrs. I guaranteed him I would accept."

• "[The judge] offers to as many people as … would be good clerks, then the first 2 who accept get the spots. You can call … at any time to ask how many offers are outstanding."

Conditions of Offers• “I told [a particular judge] I would want to make one phone call before accepting

an offer from him (should he choose to make me an offer), even though he warned me it would greatly decrease the likelihood he would make me an offer.”

• "The first judge I interviewed with gave me a sort of 'veiled' offer. He said that if I had any other offers that I couldn’t refuse to let him know as soon as possible. …

• The next judge I interviewed with (in the same building) said he really wanted me to work with him, but that he would not make me an offer if the first judge already had."

12

Expectation of Affirmative Responses Subtle Pressures• "I received an offer from another judge who I knew was having trouble

finding his last clerk. Although it wasn’t explicit, the offer was made in such a way that I felt I had no choice but to accept."

• "I understood that according to “protocol” I should only inform a judge (J2) that I had another offer (from J1) if I was certain that I would accept a position with J2 over the offer that I had."

Explicit Pressures• “[A particular judge] [m]entioned how, if he were to give offer to someone

and they didn’t immediately accept, it would make him wonder if he had made the right choice and ‘almost’ made him want to withdraw it—but he said he didn’t do that, said he might give a little time.”

• "I have an interview scheduled with my most preferred judge (Judge X) on [later date]. Judge Y calls and wants me to interview on [earlier date]. I ask Judge Y when she would be making her offers, and she says, “I am going to wait until after I finish all the interviews, talk with my clerks and then decide--so [after the later date of the Judge X interview]. [Judge Y] calls on [date prior to later date of Judge X interview] with an offer. … I decline saying I would like to interview further before making my decision. Judge Y gets fairly offended and says, “you know, students should withdraw right after the interview if they are not going to accept an offer.”

13

Strategic Scheduling

Students Attempt to Stagger Interviews• "I chose to withdraw my application from a 7th circuit judge because I was told

he would make lightning offers and expect me to make no phone calls before accepting."

• "I was offered an early interview by one judge who, though I knew I would be happy clerking for, was not my top choice. I was led to believe he might offer a position at the interview. I had a difficult time deciding whether to go to the interview (and possibly foreclose other options) or cancel (and possibly lose the bird in the hand). I went. Got an offer. Accepted."

• "… not answering the telephone for fear of being trapped into a less-than-ideal interview early on, and trying to arrange interviews strategically were also difficult procedures."

Judges Respond Strategically• "Judge A -- in order to get early time slot, wanted commitment to come to

interview (not accept another offer before it)."

• An Intricate Dance Can Ensue• " (magistrate SDNY)--called very early and wanted me to interview before Feb

1, on condition I be ready to accept on the spot. I scheduled the interview, then postponed once I got more calls. They postponed even further when I told them I wouldn’t be able to given them an answer that day …"

14

Students limit their application pools

• Survey: “Did you limit the number of judges to whom you applied based on a concern that some of your less-preferred judges would offer you interviews or positions before you had heard back from your more-preferred judges?” – 1999, 42% “yes”– 2000, 55% “yes”

15

Summary: 1999, 2000

• interviews led very quickly to offers • offers produced very quick responses • responses were generally acceptances; and• many scheduled interviews were canceled as a result.

Thus, students and judges tended to pair off quickly with those with whom they have early interviews. As a result,

• many students limited the judges to whom they apply to avoid being paired off early with a less preferred judge

• We also witnessed complex but binding verbal contracts.

• Offers became earlier from year to year (moving back from February to September of the second year).

Avery, Christopher, Jolls, Christine, Posner, Richard A. and Roth, Alvin

E., "The New Market for Federal Judicial Law Clerks" . University of Chicago Law Review,

74, Spring 2007, 447-486.

17

Proximate cause of this new, descriptive study

UNITED STATES COURTS OF APPEALS

March 11, 2002Dear Dean:

We are pleased to report that the federal appellate judges have voted overwhelmingly in favor of a new Plan for Law Clerk Hiring. The Plan includes: (1) a moratorium on law clerk hiring during the Fall of 2002, (2) an arrangement ensuring that the hiring of law clerks will not be done earlier than the Fall of the third year of law school, and (3) an agreement that the focus of law clerk hiring will be on third year law students and law graduates. The precise terms of the new Plan are set forth in the attached "Summary," and the history leading to the Plan's adoption appears in the appended "Background" statement. More than two-hundred federal appellate judges considered the Plan. Ninety-two percent (92%) of the judges either supported the Plan or indicated that they would not oppose it. Thus, the "substantial consensus" requirement that was needed to put the Plan into operation was easily satisfied.

18

Judges’ memo, cont.“There are numerous advantages to a law clerk hiring system that

focuses on third year students and law graduates. Law clerk candidates will be able to present more information with their applications and be judged more fairly. Thus, for example, applicants who are in their third year of law school will be able to offer

• a transcript showing performance during four semesters of school in a good variety of courses;

• information on law journal selection, journal publications, and election to a journal editorial board;

• writing samples from seminar courses;• information regarding experience gained in clinical courses and

public interest endeavors;• references from law professors for whom the student has worked as

a research assistant or a teaching assistant;• recommendations associated with judicial internships;• significant recommendations from Summer employers;• information on moot court competitions; and• information on selection or election to positions in student

government.

19

Initial FAQ (accompanying judges’ memo)

Q Does the Plan endorse Summer interviewing?A No. Many judges would have opposed the Plan had it endorsed

Summer interviewing. There was a concern that Summer interviews would be very inconvenient for many people. The reasons are manifold: many judges are away on vacation during the Summer; law clerk applicants are otherwise occupied with Summer jobs, vacations, foreign travel, and bar examinations (for recent graduates); law professors often are away on vacation and thus unavailable to furnish references; and law schools do not release grades on any uniform schedule, so official student transcripts from some law schools are not available until near September. However, the Plan does not forbid a law student who, say, is from Virginia and working in Tulsa during the Summer from talking with a judge who is otherwise available to chat. This has happened under existing hiring arrangements and the judges saw no reason to prohibit it under the new Plan. The main point, however, is that the formal hiring process will take place in the Fall when applications will be submitted and materials and references from the law schools will be sent to the judges.

20

Initial FAQ, cont.

Q How is "Fall" determined under the Plan?A There is no fixed definition of Fall, nor is there any

fixed starting date for the hiring season. Under existing arrangements, some judges do their hiring in September, some in October, and others do it even later. The Plan does not change this.

Q Are judges forbidden from making "exploding offers," i.e., offers that require an applicant to respond promptly to an offer?

A The Plan does not purport to address how an offer is given by a judge. This is for each judge to determine. However, no applicant is obliged to act on an offer if the terms are unacceptable, nor is an applicant obliged to accept the first offer that he or she receives.

21

Subsequent refinements of rules

• From year to year, changes have been made to– Put precise start dates for applications (after

Labor Day—already specified by start of market in 2003)

– Specify later precise dates for• Scheduling interviews• Conducting interviews and making offers

– No rules regarding exploding offers

22

2004 modifications

The critical dates under the Revised Law Clerk Hiring Plan are as follows:

• After Labor Day: Third year law students and law school graduates may submit law clerk applications and letters of reference may be submitted on their behalf.

• From the Day After Labor Day Through the Second Sunday After Labor Day (September 7 - 19 in 2004): Reading period.

• Beginning on the first Monday after Labor Day (September 13 in 2004): Judges may schedule interviews to be held after the reading period.

• Beginning on the Second Monday After Labor Day (September 20 in 2004): Judges may conduct interviews and extend offers.

Subsequent modification of dates in 2005.

23

Data

We have repeated the surveying of both federal appellate judges and applicants that we did in our prior study.

• Federal appellate judges surveyed in fall of 2004 and fall of 2005.

• Third year law students at the four law schools that provide the greatest number of clerks surveyed in fall of 2004, fall of 2005, and fall of 2006.

• About a 50% response rate from both federal appellate judges and students. On the key measure of judge reports of adherence versus nonadherence to the start dates, if anything selection bias should lead us to underestimate the level of nonadherence.

24

Overview of conclusions

• Hiring is later than before the reform, and judges value the later hiring highly.

• There is a large amount of nonadherence to the specified start dates.

• Exploding offers and market congestion remain very significant problems in the law clerk market, so that the market isn’t thick (despite the fact that most of the hiring is done in a short time).

• Nevertheless, the market has managed to hold on to some gains for four years.

• We think there are some prospects that the law clerk market might be able to sustain a long-run equilibrium in which there is significant nonadherence in some sectors and general adherence in other sectors.

25

Hiring is later than before the reform, and judges value the later hiring highly.

Fall of 2004 Fall of 2005 Number and (in parentheses) percentage of responding judges who prefer current system

94 (84%)

68 (84%)

Number and (in parentheses) percentage of responding judges who prefer system in effect prior to 2002

17 (15%)

11 (14%)

Number and (in parentheses) percentage of responding judges who are indifferent between the two systems

1 (1%)

2 (2%)

Total number of responses to question about former and current law clerk hiring regimes

112 81

26

Some representative quotes: value of information

Survey Comment 2004 Survey #8 We much prefer hiring 3rd yr students. 2004 Survey #50 The present system is working—it is far better than when we were

fighting to hire as soon as law review selections were made. 2004 Survey #64 [T]he system as a whole was improved by a single factor: moving

the interview/hiring year from 2L to 3L. 2004 Survey #75 [M]oving hiring from the 2nd to 3rd year makes [the new process]

all worthwhile. 2004 Survey #85 The new system is much better than the old. I had just sent a

memo to all the judges on my court saying I was going to hire only 3Ls and graduates when the new rules were proposed.

2004 Survey #92 The plusses of interviewing/hiring in 3d year are substantial and worth resisting the cheaters and playing by the rules.

2005 Survey #34 [T]he additional information is great. 2005 Survey #56 We greatly like waiting until 2nd year grades [and] law review

positions [are] determined. 2005 Survey #99 It is a vast improvement to consider 3rd year applicants.

27

Violations of rules: Some representative quotes

• “It’s sad (pathetic?) that judges aren’t obeying their own rules. [It] flies in the face of the whole notion of ‘law and order.’”

• “One of [Judge X's] clerks even chastised me for ‘overly stringent adherence to this timeline they have’ and noted that other students from my school were willing to interview ahead of schedule. It was a real conflict for me. I felt like I had to choose between cheating and (potentially) not getting a clerkship.”

• “It’s very disheartening to see so many Federal judges—the ostensible paragons of rules and fair play—breaking their own rules and scheduling interviews before the agreed-upon date in the law clerk hiring plan. I expected better.”

• “Many people/professors brazenly cheated. … For a group of individuals nominally committed to the rule of law, this behavior strikes me as unethical and hypocritical.”

28

Adherence to start dates (2004 judge survey; gray areas = nonadherence)

Number and (in parentheses) cumulative percentage of responding judges

Before September 7

September 7–12

September 13–19

September 20–26

After September 26/Not yet

Date of first interview

11 (9%)

6 (15%)

22 (33%)

66 (91%)

11 (100%)

Date of first offer

5 (4%)

9 (12%)

11 (21%)

67 (78%)

26 (100%)

29

Judges’ perceptions: 2004

Number and (in parentheses) cumulative percentage of responding judges Overall Within

Circuit

Relatively few judges adhered

To responding judge’s knowledge, many judges adhered but a substantial number did not

To responding judge’s knowledge, almost all judges adhered

To responding judge’s knowledge, all judges adhered

At least one judge in Circuit did not adhere

Start date for scheduling interviews

3 (3%)

34

(36%)

41 (75%)

26 (100%)

36 (69%)

Start date for conducting interviews and making offers

5 (5%)

43

(46%)

40 (85%)

16 (100%)

36 (73%)

30

Judges’ perceptions: 2005Number and (in parentheses) cumulative percentage of responding judges

Overall Within Circuit

Relatively few judges adhered

To responding judge’s knowledge, many judges adhered but a substantial number did not

To responding judge’s knowledge, almost all judges adhered

To responding judge’s knowledge, all judges adhered

At least one judge in Circuit did not adhere

Start date for scheduling interviews

5

(6%)

40

(52%)

27 (84%)

14 (100%)

35 (87%)

Start date for conducting interviews and making offers

4

(5%)

44

(58%)

23 (86%)

12 (100%)

34 (81%)

31

Student reports: 2004 (gray areas = nonadherence)

Number and (in parentheses) cumulative percentage of responding students

Before September 7

September 7–12

September 13–19

September 20–26

After September 26/Not yet

Date of first scheduling of interview

8

(5%)

39

(31%)

94

(92%)

8

(97%)

5

(100%)

Date of first interview

9

(6%)

7

(11%)

18

(23%)

101

(91%)

13

(100%)

Date of first offer

4

(3%)

3

(5%)

8

(12%)

84

(77%)

29

(100%)

32

Student reports: 2005 (gray areas = nonadherence)

Number and (in parentheses) cumulative percentage of responding students

Before September 6

September 6–14

September 15–21

September 22–28

After September 28/Not Yet

Date of first scheduling of interview

12

(9%)

35

(35%)

80

(95%)

1

(96%)

5

(100%)

Date of first interview

8

(6%)

10

(13%)

24

(31%)

83

(93%)

9

(100%)

Date of first offer

3

(3%)

7

(9%)

5

(13%)

89

(89%)

13

(100%)

33

Student reports: 2006 (gray areas = nonadherence)

Number and (in parentheses) cumulative percentage of responding students

Before Sept. 5

Sept. 5-13 Sept. 14-20

Sept. 21-27 After Sept. 27/Not Yet

Date of first scheduling of interview

13

(11%)

31

(38%)

68

(97%)

3

(99%)

1

(100%)

Date of first interview

13

(11%)

6

(16%)

18

(32%)

77

(97%)

3

(100%)

Date of first offer

10

(9%)

3

(12%)

10

(22%)

69

(87%)

14

(100%)

34

Exploding offers: Some representative quotes

• “I received the offer via voicemail while I was in flight to my second interview. The judge actually left three messages. First, to make the offer. Second, to tell me that I should respond soon. Third, to rescind the offer.It was a 35 minute flight.”

• “I had 10 minutes to accept.”• “I asked for an hour to consider the offer. The judge

agreed; however thirty minutes later [the judge] called back and informed me that [the judge] wanted to rescind my offer.”

35

Exploding and short-fuse offers: judges’ reported deadlines

1998-1999 and 1999-2000 markets

2004-2005 and 2005-2006 markets

Within one day 23%

34%

Within two days 36%

42%

Within a week 67%

76%

Number of responding judges

193

163

36

Fast market clearing: student responses

Fall of 2004 Fall of 2005 Fall of 2006

First offer received on start date for interviewing and making offers

38

51 45

First offer received after start date for interviewing and making offers

59

52 38

Of first offers received on start date for interviewing and making offers, percentage accepted on start date

42% 63%

62%

So, for a nontrivial proportion of students, the market ends after the first interview (and some offers come even before the start date). Similarly for judges, the market is not thick.

37

Recap

• The market has now successfully held the late date (fall of third year of law school) for four years in a row (2003-6). – This yields an advantage due to better information– This certainly helps the (non-complying) judges who

make the earliest offers, and apparently helps many of the complying judges.

• But congested interviewing, exploding offers are still a problem– This means that for many participants, the market still

isn’t in fact thick.• What are the prospects for the future?

38

Stages and transitions observed in various other markets with timing problems

Stage 1: UNRAVELINGOffers are early, dispersed in time, exploding…

Stage 2: UNIFORM DATES ENFORCEDDeadlines, congestion

Stage 3: CENTRALIZED MARKET CLEARING PROCEDURES

39

Uniform start dates in the law clerk market

Stage 1: UNRAVELING

Stage 2: UNIFORM DATES ENFORCED

1983, 1986, 1989, 1990, 1993, 2002

1983: Sept 15 of 3rd year, abandoned in ’84

1986: April 1 of 2nd year

1989: March 1, not adopted

1990: May 1, 12:00 Noon

1993: March 1 (not formally abandoned until 1998)

40

Thoughts on the future evolution of the clerk market

• It has cycled between stage 1 and stage 2 multiple times

• Comparison to other markets,– Stage 2 markets

• Psych—25 years in stage 2.• Japanese universities

• The current arrangement has held together as long as the previous longest (1993-97) stage 2 arrangement.

41

Prospects for holding the current dates?

• Adherence to start dates – equilibrium with partial adherence? – Psychological break point – very unlikely.– Geographic or political segmentation – more

likely.– Concern that some apparent adherence may

be formal only.

42

Prospects/challenges for a centralized match?

43

Haruvy, Ernan, Alvin E. Roth, and M. Utku Ünver, “The Dynamics of Law Clerk Matching: An Experimental and Computational Investigation of Proposals for Reform of

the Market,” Journal of Economic Dynamics and Control, 30, 3 , March 2006, Pages 457-486..

Decentralized Centralized

- idealized

Centralized

- coerced

Without

Announcements 8 cohorts

10 cohorts

8 cohorts

With Announcement 10 cohorts

11 cohorts

10 cohorts

44

Success of match

0

0.5

1

1.5

2

2.5

Mkts 1-5 Mkts 6-10 Mkts 11-15 Mkts 16-20

Market

Nu

mb

er g

oin

g t

hro

ug

h m

atch

central-idealized

central-coerced

45

Changing market culture?

• We have a little experience with this in designing the policies around the recently re-established Gastroenterology match (Niederle, Proctor, and Roth 2006).