8/10/2019 7. Unraveling
1/118
Unraveling
8/10/2019 7. Unraveling
2/118
Northwestern Law School: July 2010 News Release
"CHICAGO --- Northwestern University School of Law and
the global law firm Jones Day announced today July 26that the firm will conduct its on-campus interviews for2011 summer associates in September instead of duringthe law school's official on-campus interviewing (OCI)program, which begins Aug. 11. In a move benefiting bothstudents and law firms, Jones Day will conduct interviewson behalf of its 14 U.S. offices on Monday, Sept. 13.
"Jones Day joins Northwestern Law in the belief that the
current recruitment system has created a competitive raceamong law schools and law firms to conduct on-campusinterviews earlier. The result is an inefficient system thatdoes not serve employers or student applicants well,
according to the law school and law firm. 2
8/10/2019 7. Unraveling
3/118
Whats going on?
The announcement refers to the 2010interviewing and hiring of summerassociatesfor the summer of 2011, after
they will have finished only their secondyear of law school. That is, the competitionto find employment and employees for the
summer after the second year of lawschool has now moved a year in advance,to the summer after the first year.
3
8/10/2019 7. Unraveling
4/118
Why are summer hiresimportant?
Because at the biggest law firms, much ofthe hiring of full time associates, who jointhe firm after the completion of their third
year of law school, has unraveled into thesummer associate market; i.e. thecompetition for summer associates is
fierce because summer associates largelybecome full time employees.
4
8/10/2019 7. Unraveling
5/118
Why might this possibly beinefficient?
One reason, made clear by the recent economiccrisis (which reduced corporate demand foroutside legal services) is that hiring more than a
year before the start of employment makes itdifficult for law firms to forecast their demand.Thousands of summer associates at large firmswho accepted permanent offers shortly after
their second year summer associateships inAugust 2008 (for permanent jobs in 2009), hadthem rescinded or deferred.
5
8/10/2019 7. Unraveling
6/118
6
Stages and transitions observed in various other marketswith timing problems
Stage 1: UNRAVELING
Offers are early dispersed in
time exploding
Stage 2: UNIFORM DATES
ENFORCED
Deadlines, congestion
Stage 3:
CENTRALIZED MARKET
CLEARING PROCEDURES
8/10/2019 7. Unraveling
7/118
Roth, A.E. and X. Xing, "Jumping the Gun: Imperfectionsand Institutions Related to the Timing of MarketTransactions,"American Economic Review, 84,
September, 1994, 992-1044. American and British Doctors Federal Court Clerkships
American Law Firms
Postseason college football bowls
Business School Markets: New MBA's and Marketing Professors
Japanese University Graduates
Clinical Psychology Internships
Fraternities and Sororities
Canadian Lawyers Athletes
Marriage
Academic markets for mathematicians, biologists, and chemists
Medieval and Modern Commodity Markets7
http://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdf8/10/2019 7. Unraveling
8/118
8
Unraveling (Roth and Xing 94)
Stage 1begins when the market comes into being (e.g. when a few
hospitals begin offering internships, or when federal court clerkships arecreated by legislation) and the relatively few transactions are made withoutovert timing problems. By the middle of stage 1 the market has grown, andsome appointments are being made rather early, with some participantsfinding that they don't have as wide a range of choices as they would like--students have to decide whether to accept early job offers or take a chanceand wait for better jobs, and some employers find that not all of the students
they are interested in are available by the time they get around to makingoffers. The trade journals start to be full of exhortations urging employers towait until the traditional time to make offers, or at least not to make themany earlier next year than this year. Towards the end of stage 1, the rate ofunraveling accelerates, until sometimes quite suddenly offers are beingmade so early that there are serious difficulties distinguishing among thecandidates. There is no uniform time for offers to be made nor is there acustomary duration for them to be left open, so participants find themselvesfacing unnaturally thin markets, and on both sides of the market a variety ofstrategic behaviors emerge, many of which are regarded as unethicalpractices. Various organizations concerned with the market may haveproposed guidelines intended to regulate it, without notable success. Asstage 1 ends, influential market participants are engaged in a vigorousdebate about what can and should be done. From beginning to end, stage 1may have covered a period of more than fifty years, or fewer than ten.
8/10/2019 7. Unraveling
9/118
9
Causes and cures of unraveling
Unraveling is one of the kinds of market failurethat has often been corrected with appropriateclearinghouse design. But theres still a lot that
we dont understand about its causes.
Unraveling results from massively multiplecauses: you can only use time strategically intwo directions (earlier and later), so there isgoing to be more than one reason to go early.
Avoiding congestion is one reason; early hiringmay even be efficiency enhancing if there isnt
enough time to hire at an otherwise desirable latetime. (This is one attraction of clearinghouses.)
8/10/2019 7. Unraveling
10/118
10
Forestalling: unraveling in
Medieval markets
"Thus at Norwich no one might forestallprovisions by buying, or paying 'earnest money'for them before the Cathedral bell had rung for
the mass of the Blessed Virgin; at Berwick-on-Tweed no one was to buy salmon betweensunset and sunrise, or wool and hides except atthe market-cross between 9 and 12; and at
Salisbury persons bringing victuals into the citywere not to sell them before broad day.
8/10/2019 7. Unraveling
11/118
11
Unraveling in Space "Besides injury through mere proximity, and anticipation in
time, there might be damage due to interception of traffic... "In 1233 Eve de Braose complained that Richard fitz- Stephen
had raised a market at Dartmouth to the injury of hers atTotnes, as ships which ought to come to Totnes were stoppedat Dartmouth and paid customs there. No decision wasreached, and eight years later Eve's husband, William de
Cantelupe, brought a similar suit against Richard's sonGilbert. The latter pleaded that his market was on Wednesdayand that at Totnes on Saturday; but the jury said that themarket at Dartmouth was to the injury of Totnes, becauseDartmouth lies between it and the sea, so that ships touchedthere and paid toll instead of going to Totnes; and also that
cattle and sheep which used to be taken to Totnes marketwere now sold at Dartmouth; the market at Dartmouth wastherefore disallowed.
Both quotes: Salzman, as quoted in Roth and Xing (1994)
8/10/2019 7. Unraveling
12/118
12
Market Failure by Unraveling, in Medicine,Football, Gastroenterology, Law, and in the Lab
Kagel, John H. and A.E. Roth, "The dynamics of reorganization in matchingmarkets: A laboratory experiment motivated by a natural experiment,"Quarterly Journal of Economics, February, 2000, 201-235.
Frchette, Guillaume, Alvin E. Roth, and M. Utku nver,"Unraveling YieldsInefficient Matchings: Evidence from Post-Season College Football Bowls,Rand Journal of Economics, Winter 2007, 967-982.
Niederle, Muriel and Alvin E. Roth, "Unraveling reduces mobility in a labormarket: Gastroenterology with and without a centralized match," Journal ofPolitical Economy, 111, 6, December 2003, 1342-1352.
Avery, Christopher, Christine Jolls, RichardA. Posner, and Alvin E. Roth, "TheMarket for Federal Judicial Law Clerks" , University of Chicago Law
Review, 68, 3, Summer, 2001, 793-902.Haruvy, Ernan, Alvin E. Roth, and M. Utku nver, The Dynamics of Law ClerkMatching: An Experimental and Computational Investigation of Proposalsfor Reform of the Market, Journal of Economic Dynamics and Control, 30,3 , March 2006, Pages 457-486.
Avery, Christopher, Christine Jolls, Richard A. Posner, and Alvin E. Roth, "TheNew Market for Federal Judicial Law Clerks, University of Chicago LawReview, 74, Spring 2007, 447-486.
Niederle, Muriel, and Alvin E. Roth, "Market Culture: How Rules GoverningExploding Offers Affect Market Performance,"American Economic Journal:Microeconomics, 1, 2, 2009, 199-219.
Niederle, Muriel, Alvin E. Roth, and M. Utku Unver, Unraveling Results fromComparableDemand and Supply: An Experimental Investigation, May, 2009.
Roth, Alvin E, "Marketplace institutions related to the timing of transactions, andreply to Priest (2010)" October, 2010. (Here is the shorter version that isforthcoming in Journal of Labor Economics, maybe in April 2012)
http://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdf8/10/2019 7. Unraveling
13/118
13
Organizations concerned with the timing of when offers are made,accepted, rejected
Council of Graduate Schools (CGS): graduate admissions, National Association for College Admission Counseling (NCAC):
undergraduate admissions, (early action, early decision) National Resident Matching Program (NRMP): entry level medical
residencies, (also Canadian Resident Matching ServiceCaRMSandvarious regional matches in Britain)
Specialty Matching Services (SMS): advanced medical residencies andfellowships,
Association of Psychology Postdoctoral and Internship Centers (APPIC):clinical psychology positions, National Association for Law Placement (NALP) for positions in law firms, Judicial Conference of the United States and various ad hoc committees
of judges for federal judicial clerkships, Provincial Law Societies in Canada.
National Association of Colleges and Employers (NACE) for US collegeundergraduates, NCAA: formerly for postseason college football bowls, now regulated by
the Bowl Championship series (BCS), NCAA for recruitment of college athletes, and various drafts National Panhellenic Conference for sorority matching
The Japan Federation of Employers Associations (Nikkeiren) for Japaneseuniversity graduates
8/10/2019 7. Unraveling
14/118
14
1990 College Football Bowl Games: Ratings
8/10/2019 7. Unraveling
15/118
15
Probability of top 2 teams endingthe season as top 2 in AP Poll
1 Week Prior 2 Weeks Prior 3 Weeks Prior 4 Weeks Prior
Probability .690 .586 .310 .345Standard Error .086 .091 .086 .088
Number of Obs. 29 29 29 29
8/10/2019 7. Unraveling
16/118
16
Summary of College Bowl arrangements after the NCAAabandoned its attempt to control the market after 1991
Rose Bowl Fiesta Bowl Orange Bowl Sugar Bowl Cotton Bowl
Starting Year 1902 1971 1935 1934 1937
First
Team
Since 1947 -Champion of
Big TenConference
Until 1978Champion of
WesternConferenceStarting 1978
At Large
Team
Champion ofBig Eight
(Twelve)Conference
Champion ofSoutheastern
Conference
Champion ofSouthwest
Conference
Matchups prior to
Bowl Coalition
Era
( - 1992)
Second
Team
Since 1947 -Champion of
Pacific Ten
(Coast orEight
previously)
Conference
At LargeTeam
At LargeTeam
At Large Team At Large Team
BC
Bowl?
No Yes Yes Yes Yes
First
Team
Champion ofBig Ten
Conference
At LargeTeam possibly
to create 12matchup
Champion ofBig Eight
(Twelve)Conference
Champion ofSoutheastern
Conference
Champion ofSouthwest
Conference
Matchups in Bowl
Coalition Era(1992-1994)
Second
Team
Champion of
Pacific TenConference
At Large
Team possiblyto create 12
At Large
Team possiblyto create 12
At Large Team
possibly tocreate 12
At Large Team
possibly tocreate 12
8/10/2019 7. Unraveling
17/118
17
BA
Bowl?
No Yes Yes Yes No
First
Team
Champion of
Big TenConference
First Team from
BigTwelveConference notgoing to BABowls
Matchups in Bowl
Alliance Era
(1995-1997)
Second
Team
Champion ofPacific TenConference
2 At Large Teams, ACC, Big East, BigTwelve,
Southeastern conference championspossibly to create 1-2 matchup in one of thesebowl games
A team fromPacific Ten orWesternConferences out
of BA BowlsBCS
Bowl?
Yes Yes Yes Yes No
First
Team
First team fromBigTwelveConference outof BCS Bowls
Matchups in Bowl
Championship
Series Era
(1998-)
Second
Team
ACC, Big East, Big Twelve, Big Ten, Pacifi c Ten,
Southeastern conf erence champions, up to 2 highly ranked
other conference or at large teams (with Notre Dame having
priori ty) always to create always 1-2 matchup in BCS rank ings
in one of these bowl games A comparable
team fromSoutheasternConference
During this time the membership of the variousconferences also changed, and a number of
independents joined conferences.
d i d h ffi i f h k
8/10/2019 7. Unraveling
18/118
18
and increased the efficiency of the market:
Average Normalized Nielsen Ratings in BCS Bowls
BC
BC
BC
BABA
BA
BCSBCS
BCS
BCS
BCSBCS
BCS
BCS
2
6
8
10
12
AverageRegularS
easonRatings
2
6
8
10
12
1985 1992 1995 1998 2005Year
Average minus Regular Coalition Regular
8/10/2019 7. Unraveling
19/118
How do clearinghouses helpstop unraveling?
They can establish a thick market at anefficient time.
Weve conjectured that stability plays a
role in helping them do this successfully.
19
8/10/2019 7. Unraveling
20/118
Priority matching (an unstable system)
Edinburgh, 1967 No longer in use
Birmingham 1966, 1971, 1978 " " " "
Newcastle 1970's " " " "
Sheffield 196x " " " "
In a priority matching algorithm, a 'priority' is defined for eachfirm-worker pair as a function of their mutual rankings. Thealgorithm matches all priority 1 couples and removes themfrom the market, then repeats for priority 2 matches, priority 3, etc.
E.g. in Newcastle, priorities for firm-worker rankings wereorganized by theproductof the rankings, (initially) as follows:
1-1, 2-1, 1-2, 1-3, 3-1, 4-1, 2-2, 1-4, 5-1...
8/10/2019 7. Unraveling
21/118
Priority matching (an unstable system)
This can produce unstable matchings -- e.g. if a desirable firm and
worker rank each other 4th, they will have such a low priority(4x4=16) that if they fail to match to one of their first three choices,it is unlikely that they will match to each other. (e.g. the firm mightmatch to its 15th choice worker, if that worker has ranked it first...)
After 3 years, 80% of the submitted rankings were pre-arranged 1-1 rankings without any other choices ranked. This worked to thegreat disadvantage of those who didn't pre-arrange their matches.
8/10/2019 7. Unraveling
22/118
22
What makes a clearinghousesuccessful or unsuccessful?
A matching is stable if there arent a doctor and
residency program, not matched to each other,who would both prefer to be.
Hypothesis: successful clearinghouses producestable matchings.
How to test this?
8/10/2019 7. Unraveling
23/118
23
Market Stable Still in use (halted unraveling)
NRMP yes yes (new design in 98) Edinburgh ('69) yes yes
Cardiff yes yes Birmingham no no Edinburgh ('67) no no Newcastle no no Sheffield no no Cambridge no yes
London Hospital no yes Medical Specialties yes yes (~30 markets, 1 failure) Canadian Lawyers yes yes (Alberta, no BC, Ontario) Dental Residencies yes yes (5 ) (no 2) Osteopaths (< '94) no no Osteopaths (> '94) yes yes
Pharmacists yes yes Reform rabbis yes (first used in 97-98) yes Clinical psych yes (first used in 99) yes
So stability looks like an important feature of a centralized labormarket clearinghouse.
8/10/2019 7. Unraveling
24/118
24
The need for experiments
How to know if the difference betweenstable and unstable matching mechanismsis the key to success?
There are other differences between e.g.Edinburgh and Newcastle
The policy question is whether the newclearinghouse needs to produce stablematchings (along with all the other things itneeds to do like handle couples, etc. ) E.g. rural hospital question
8/10/2019 7. Unraveling
25/118
25
A matching experiment(Kagel and Roth, QJE 2000)
6 firms, 6 workers (half "High productivity" half "low productivity") It is worth $15 plus or minus at most 1 to match to a high It is worth $5 plus or minus at most 1 to match to a low There are three periods in which matches can be made:-2, -1, 0. Your payoff is the value of your match, minus $2 if made in
period -2, minus $1 if made in period -1 Decentralized match technology : firms may make one offer at any
period if they are not already matched. Workers may accept atmost one offer. Each participant learns only of his own offers andresponses until the end of period 0.
After experiencing ten decentralized games, a centralized matching
technology was introduced for period 0 (periods -2 and -1 wereorganized as before). Centralized matching technology: participants who are still
unmatched at period 0 submit rank order preference lists, and arematched by a centralized matching algorithm.
Experimental variable: Newcastle (unstable) or Edinburgh (stable)
algorithm.
8/10/2019 7. Unraveling
26/118
26
Average Cost of Early Markets
0
2
4
6
8
10
1-5 6-10 11-15 16-20 21-25
Markets #
Cost(Dollars)
Decentralized Markets Markets with a Clearinghouse
Stable
(Deferred
Acceptance)
algorithm
New castle
Priority
algorithm
8/10/2019 7. Unraveling
27/118
27
Offers and acceptances
8/10/2019 7. Unraveling
28/118
8/10/2019 7. Unraveling
29/118
29
The Roth-Peranson algorithm is also used inother market clearinghouses since 1998
Postdoctoral Dental Residencies in the United States
Psychology Internships in the United States andCanada
Neuropsychology Residencies in the United Statesand Canada
Osteopathic Internships in the United States
Pharmacy Practice Residencies in the United States
Articling Positions with Law Firms in Alberta,Canada
8/10/2019 7. Unraveling
30/118
30
NRMP also runs Fellowship matches through the Specialties MatchingService (SMS) (using the Roth-Peranson algorithm since 1998)
Abdominal Transplant Surgery (2005)
Child & Adolescent Psychiatry (1995) Colon & Rectal Surgery (1984) Combined Musculoskeletal Matching
Program (CMMP) Hand Surgery (1990)
Medical Specialties Matching Program(MSMP) Cardiovascular Disease (1986)
Gastroenterology (1986-1999;rejoined in 2006)
Hematology (2006) Hematology/Oncology (2006) Infectious Disease (1986-1990;
rejoined in 1994) Oncology (2006)
Pulmonary and Critical Medicine (1986) Rheumatology (2005)
Minimally Invasive and GastrointestinalSurgery (2003)
Obstetrics/Gynecology
Reproductive Endocrinology (1991) Gynecologic Oncology (1993) Maternal-Fetal Medicine (1994) Female Pelvic Medicine &
Reconstructive Surgery (2001)
Ophthalmic Plastic & ReconstructiveSurgery (1991)
Pediatric Cardiology (1999)
Pediatric Critical Care Medicine (2000) Pediatric Emergency Medicine (1994) Pediatric Hematology/Oncology (2001) Pediatric Rheumatology (2004) Pediatric Surgery (1992) Primary Care Sports Medicine (1994) Radiology
Interventional Radiology (2002) Neuroradiology (2001) Pediatric Radiology (2003)
Surgical Critical Care (2004) Thoracic Surgery (1988) Vascular Surgery (1988)
8/10/2019 7. Unraveling
31/118
31
Gastroenterologists Niederle, Murieland Alvin E. Roth, "Unraveling reduces mobility in a
labor market: Gastroenterology with and without a centralized
match," Journal of Political Economy, 1. 111, no. 6, December 2003,1342-1352. .
Niederle, Murieland Alvin E. Roth, "The Gastroenterology FellowshipMatch: How it failed, and why it could succeed onceagain,"Gastroenterology, 127, 2, August 2004, 658-666.
McKinney, C. Nicholas, Niederle, Murieland Alvin E. Roth, "Thecollapse of a medical labor clearinghouse (and why such failures arerare),"American Economic Review, 95, 3, June, 2005, 878-889.
Niederle, Murieland Alvin E. Roth, "The Gastroenterology FellowshipMarket: Should there be a Match?,"American Economic Review,Papers and Proceedings, 95,2, May, 2005, 372-375.
Niederle, Muriel, Deborah D. Proctor and Alvin E. Roth, What will beneeded for the new GI fellowship match to succeed?,Gastroenterology,130, January 2006, 218-224.
Niederle, Muriel, Deborah D. Proctor and Alvin E. Roth, ''TheGastroenterology Fellowship Match: The First Two
Years,''Gastroenterology , 135, 2 (August), 344-346, 2008.
http://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/http://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://www.stanford.edu/~niederle/8/10/2019 7. Unraveling
32/118
32
Timing of Interviews: Unraveling
Interviews for positions starting in 05 for specialties in Match.
Proportion of hospitals that started interviewing
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
aug sept oct nov dec jan feb mar apr
Match Start 05
8/10/2019 7. Unraveling
33/118
33
Timing of Interviews: Unraveling
Interviews are happening earlier and earlier
Proportion of hospitals that started interviewing
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.91
aug sept oct nov dec jan feb mar apr
Start 05
Start 03
Match Start 05
8/10/2019 7. Unraveling
34/118
34
Timing of Interviews: Unraveling
Interviews are happening earlier and earlier
Proportion of hospitals that started interviewing
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
aug sept oct nov dec jan feb mar apr
Start 07 Survey
Start 06 Survey
Start 05
Start 03
Match Start 05
8/10/2019 7. Unraveling
35/118
35
Timing of Offers
For each of 44 programs:
Use date of first and last offer, andassume that last offer is the longest openoffer: Overestimate length of time they areon the market.
Each program is represented by one line.
Dates during which fellowship programs were making offers. Each
8/10/2019 7. Unraveling
36/118
36
a es du g c e o s p p og a s e e a g o e s acprogram is represented by one of the horizontal lines, indicating the(maximal) dates during which it could have had outstanding offers(2005 survey data, n=44).(As of November 15, 11 (27%) programs had already finished making
offers, 12 (25%) had not yet started, and 21 (48%) were in the midst.)
8/10/2019 7. Unraveling
37/118
37
Market is very dispersed in time
0
0.1
0.2
0.3
0.4
0.5
0.6
1-Au
g
16-Aug
1-Se
p
16-Sep
1-Oc
t
16-Oct
1-No
v
16-Nov
1-De
c
16-Dec
1-Jan
16-Jan1-Fe
b
15-Feb
1-Ma
r
havingoutstandingoffers
8/10/2019 7. Unraveling
38/118
8/10/2019 7. Unraveling
39/118
Eff t f M t h M bilit
8/10/2019 7. Unraveling
40/118
40
Effects of a Match: Mobility
With a Match, fellows are more mobile.
This effect is more pronounced for large hospitals.
(Data: 9180 of 15,187 gastroenterologists who completed residency andfellowship in US after 1977.)
Share of mobility of GI fellows for each year
before match match after match
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1977 1980 1983 1986 1989 1992 1995 1998 2001
Hosp
City
State
8/10/2019 7. Unraveling
41/118
41
Transition to a later market In May 2005, the American Gastroenterological
Association (AGA), the American College of
Gastroenterology (ACG), the American Society forGastrointestinal Endoscopy (ASGE) and the
American Association for the Study of LiverDiseases (AASLD) decided to reintroduce a GIfellowship match, starting in 2006, for positionsbeginning in July 2007.
How to manage the transition? Rates of participation
Concern among programs about whether their chief competitorswill participate.
Change of dates to June 2006 (from as early as July2005)
How to reassure programs that other programs will wait
for Match? (Without a congested transition like 1945-50)
8/10/2019 7. Unraveling
42/118
42
In June 2005, Debbie Proctor, thegastroenterologist who took the lead inreorganizing the match, sent us an email saying,in part
Im answering 3-4 emails per day especially onthis issue. I want to make sure MY competitionis in the match and that they dont cheat. Well,this is another way of saying that if they cheat,then I will too!...Have you ever seen this before?
The distrust amongst program directors? I find ithard to believe that we are unique. Maybe this is[a] social science phenomenon?
8/10/2019 7. Unraveling
43/118
43
Which markets are unraveled? Marketdesign
It appears that markets in which transactions aremade at early, uncoordinated times are marketsin which there are both
Exploding offers Binding commitments
Example: Graduate admissions has a nicelydesigned set of market rules, sent to all admitted
students, that suppresses exploding offers (inpart by facilitating reneges of premature
acceptances).
8/10/2019 7. Unraveling
44/118
44
Students are under no obligation to respondto offers of financial support prior to April 15;earlier deadlines for acceptance of such
offers violate the intent of this Resolution. Inthose instances in which a student acceptsan offer before April 15, and subsequentlydesires to withdraw that acceptance, the
student may submit in writing a resignationof the appointment at any time through April15.
8/10/2019 7. Unraveling
45/118
45
The need for experiments
Note that a simple experimental environment will bequite different from the markets in the table, and from thegastroenterology market.
The laboratory environment, because it is so simple, is
different from each of these markets in more transparentways than they are different from one another. while it is always somewhat risky to draw inferences about the
effect of a rule change in one market from the effects in adifferent market, the inferences may be clearer when one of themarkets is simple.
And in the experiment, the rules are an exogenousexperimental variable, so that their influence can bereadily observed.
8/10/2019 7. Unraveling
46/118
46
An experiment allows us to view differentoffer regimes in a controlled environment
5 firms, 6 applicants, 9 periods. In each period, a firm may make an offer to at most one applicant. Firmsmake offers, applicants decide upon the offers they receive.
Firms and applicants are assigned qualities.
If firm of quality x hires an applicant of quality y, both firm and applicantwill receive a payoff of xy points each.
Firms qualities are simply their assigned participant number, 1,2,3,4,5. Uncertainty about applicants quality is only resolved over time:
In periods 1, 4 and 7, each applicant receives a signal, an integerbetween 1 and 10, each equally likely.
In period 7, the relative ranking of the sum of the 3 signals determinesthe applicants quality. The applicant with the highest sum of 3 signals
has a quality of 6 Ties are broken randomly. Firms see all signals, applicants see only their own signals (as they
become available over time) and their ranking in period 7.
(Note that in this experiment the cost of early matching is bad
matches due to uncertainty about quality.)
E i t l t t t 3 k t lt
8/10/2019 7. Unraveling
47/118
47
Experimental treatments: 3 market cultures
Exploding offers: Firms can make exploding offers andacceptances are binding.
Renege: Firms can make exploding offers, but applicantscan renege on their acceptance, for a small fee (1 point).
Open offers: Firms can only make open offers.
(Many equilibria:One in which all matches are agreed upon inefficiently early.
All environments have a perfect equilibrium with efficient latematching.But the late matching equilibrium is more fragile when offers areexploding and acceptances are binding.)
Ti i f fi l ff ( ff th t t d d t
8/10/2019 7. Unraveling
48/118
48
Timing of final offers (offers that were accepted and notreneged upon for the renege treatment) in terms of the
number of signals that were observed.
Timing of final offers
1
1.5
2
2.5
3
1-5 6-10 11-15 16-20Markets
N
umberofsign
als
available
Exploding
Open
Renege
8/10/2019 7. Unraveling
49/118
8/10/2019 7. Unraveling
50/118
50
Resolution Regarding Gastroenterology Fellowship Applicants,
Including all Research and Clinical Fellowship Applicants and Positions
This resolution concerns the conditions surrounding gastroenterology fellowship offers toapplicants, acceptance by applicants of such offers, and participation by applicants andprograms in the gastroenterology fellowship Match. The general spirit of this resolution isthat each applicant should have an opportunity to consider all programs before making adecision and be able to participate in the Match. The intention of this resolution is to ensureuniformity so that everyone participates fairly and to establish the principle that all positions should be
filled through the Match or after Match Day. It therefore seeks to create rules that give bothprograms and applicants the confidence that applicants and positions will remain availableto be filled through the Match and not withdrawn in advance of it.
This resolution addresses the issue that some applicants may be persuaded or coerced tomake commitments prior to, or outside of, the Match.Early offers and acceptances, and offersoutside of the Match, are violations of the rules and of this resolution and are not condoned.Anyapplicant may participate in the matching processby registering for the Match to interview and
consider match-participating programs; however, an applicant who accepts a position prior to, or outsideof, the Match must comply with the National Resident Matching Program/Specialty Matching Services(NRMP/SMS) Match Participation Agreement by either resigning the accepted position if he/shewishes to submit a rank order list of programs or by withdrawing from the Match prior to therank order list certification deadline, which is the first week in June. In addition, no program maywithdraw a position from the Match after the quota change deadline to offer that position outside thematching process. The spirit of this resolution is to make it unprofitable for programdirectors to press applicants to accept early offers, and to give applicants an opportunity toconsider all offersas well as to provide uniform and widely acceptable rules that protect bothapplicants and fellowship programs.
8/10/2019 7. Unraveling
51/118
8/10/2019 7. Unraveling
52/118
52
Another proposal: (What do you think of it?)
8/10/2019 7. Unraveling
53/118
Another proposal: (What do you think of it?)Uniform notification date forpostdoctoralpositions in
professional psychology
53
O th di h d th bl
8/10/2019 7. Unraveling
54/118
54
Orthopedic surgery had the same problem
But aspiring surgeons may not be able to turn downearly offers after accepting them, even if the orthopedic
organizations were to adopt a policy like thegastroenterologists'.
However, while the (15) orthopedic professionalorganizations also cannot directly prevent employers
from making early offers, unlike the gastroenterologists,they feel they can effectively punish employers whomake early offers, by not allowing them to presentpapers at professional meetings.
Harner, Christopher D., Anil S. Ranawat, Muriel
Niederle, Alvin E. Roth, Peter J. Stern, Shepard R.Hurwitz, William Levine, G. Paul DeRosa, Serena S. Hu,"Current State of Fellowship Hiring: Is a universal matchnecessary? Is it possible?," Journal of Bone and JointSurgery, 90, 2008,1375-1384.
http://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdf8/10/2019 7. Unraveling
55/118
55
Federal judges hiring law clerks have a similar
8/10/2019 7. Unraveling
56/118
56
Federal judges hiring law clerks have a similarproblem
This market has also unraveled. Like orthopedic surgeons, law students can't change their
minds no matter how early the offer (law students are not ina position to break promises to federal judges).
Like the gastroenterology organizations, the judicialconferences have no way to prevent judges from hiring
early, or from making exploding offers, or punishing thosewho do.
There are now discussions underway among judges andlaw schools about the possibility of instituting aclearinghouse like the ones used by doctors.
But until some way is found to address the issue of earlyand exploding offers before a clearinghouse, the chances ofsuccess are small.
It appears that judges may have to become willing to dosome "community enforcement" of norms against early
exploding offers before a clearinghouse will work for them.
8/10/2019 7. Unraveling
57/118
57
Law Clerks
Avery Christopher Christine Jolls Richard A Posner and
8/10/2019 7. Unraveling
58/118
58
Avery, Christopher, Christine Jolls, Richard A. Posner, andAlvin E. Roth, The Market for Federal Judicial Law Clerks,University of Chicago Law Review, 68, 3, Summer, 2001, 793-902.
The market for clerkships starting in 2003 cleared in the September2001, i.e. at the very beginning of the first semester of the secondyear of law school; nearly two years before the start of employment.
This introduces several potential causes of inefficiency.
because the market clears so early, it clears before informationbecomes available (e.g. students second and third year grades, lawreview articles, etc.) that can help produce efficient matches ofparticular clerks and judges.
because competition among judges to hire earlier than theircompetitors makes the market fast, chaotic, and thin, many studentsand judges have little opportunity to consider a wide range ofoptions, but rather have to transact quickly, before options can bedeveloped.
8/10/2019 7. Unraveling
59/118
59
The proximate cause of that study:
Memo from the Administrative Office of the UnitedStates Courts, to All United States Judges,October 7, 1998.
At its September 15, 1998 session, the JudicialConference of the United States rescinded itsSeptember 1993 policy recommending to all
judicial officers that March 1 of the year before aclerkship begins be the benchmark starting date
for law clerk interviews. This action was takenbecause the policy on law clerk interviews hasnot been universally followed...
8/10/2019 7. Unraveling
60/118
60
Over the next few years, we observed
interviews led very quickly to offers
offers produced very quick responses responses were generally acceptances; and many scheduled interviews were canceled as a result.
Thus, students and judges tended to pair off quickly withthose with whom they have early interviews. As a result,
many students limited the judges to whom they apply toavoid being paired off early with a less preferred judge
We also witnessed complex but binding verbal contracts.
Offers became earlier from year to year (moving backfrom February to September of the second year).
An initial modest reform:
8/10/2019 7. Unraveling
61/118
61
An initial, modest reform:Ninth Circuit Court Announced Positions for
Sept, 2003, as of August, 2001
Haruvy Ernan Alvin E Roth and M Utku nver The
8/10/2019 7. Unraveling
62/118
62
Haruvy, Ernan, Alvin E. Roth, and M. Utku nver, TheDynamics of Law Clerk Matching: An Experimental andComputational Investigation of Proposals for Reform of
the Market, Journal of Economic Dynamics andContro l, 30, 3 , March 2006, Pages 457-486..
Decentralized
Centralized
- idealized
Centralized
- coerced
Without
Announcements
8 cohorts 10 cohorts8 cohorts
With Announcement 10 cohorts11 cohorts 10 cohorts
Experimental Design: 3 period market, in which information aboutmatch quality becomes fully available in the form of accumulatedgrades only in year 3.
8/10/2019 7. Unraveling
63/118
63
Judges have qualities 1 to 4, known from the beginning of the market. In each of three years, students grades are independently drawn
from {0,1,2} with a discrete uniform density. Ties between studentsare broken arbitrarily only after year 3 grades become available (butbefore the application process starts in year 3). After all ties arebroken, the applicant with the highest cumulative grade in year 3 hasquality 4, the applicant with the second highest cumulative grade hasquality 3, the applicant with the second to last grade has quality 2,and the applicant with the lowest grade has quality 1.
The payoff to each subject is the product of his own quality and that of
the subject to whom he is matched Subjects experienced 20 markets, with the roles (judge or applicant)
remaining the same, judges qualities remaining fixed, and applicantsgrades determined randomly in each year of each market. So, offersmade in year 1 can be contingent only on year 1 grades, while offersmade in year 3 are made after all grades, and hence students final
qualities are known. Applicants decide, at the beginning of each year, to which judges, ifany, to submit applications. Applicants can apply to as manyavailable judges as they wish, and no judge may make an offer to anapplicant who has not applied to him. However, when a judgemakes an offer, the applicant must accept, unless a better judgemade an offer simultaneously.
8/10/2019 7. Unraveling
64/118
64
In all six treatments, in each year (1-2 in the centralized treatments and1-3 in the decentralized):
Applicants send applications to available firms.
Firms may choose any one applicant from the pool of applicantswho have applied in a given year, and they are matched to thisapplicant unless a higher quality firm also chooses that applicant.(i.e. applicants cant refuse an offer unless a better one arrives atthe same time)
In the centralized-idealized treatment, year 3:
Firms and applicants not matched by the end of year 2 are allmatched in year 3, at the unique stable matching among thoseremaining in the market. (Notice that even in the centralized-idealized treatment when an applicant applies to a firm and receivesan offer in years 1 and 2, she cannot decline this offer).
In the centralized-coerced treatment, year 3:
Firms and applicants who were not matched by the end of thesecond year were placed via the central match in year 3. But to beeligible for matching to a particular firm following year 2, an applicantneeds to have sent an application to that firm in either year 1 or year2.
8/10/2019 7. Unraveling
65/118
65
Average Group Welfaredashed lines represent treatments with announcements.
23
24
25
26
27
28
29
30
mkts 1-5 mkts 6-10 mkts 11-15 mkts 16-20
Market Block
Welfare
decentralized(NA) central-coerced(NA) central-idealized (NA)
8/10/2019 7. Unraveling
66/118
66
Success of late match
0
0.5
1
1.5
2
2.5
Mkts 1-5 Mkts 6-10 Mkts 11-15 Mkts 16-20
Market
Num
bergoingthro
ughmatch
central-idealized
central-coerced
Proximate cause of our 2007 study
8/10/2019 7. Unraveling
67/118
67
Avery, Christopher, Christine Jolls, Richard A. Posner, and Alvin E. Roth,
The New Market for Federal Judicial Law Clerks, University of Chicago
Law Review, 74, Spring 2007, 447-486
UNITED STATES COURTS OF APPEALS
March 11, 2002Dear Dean:
We are pleased to report that the federal appellate judges havevoted overwhelmingly in favor of a new Plan for Law Clerk Hiring.The Plan includes: (1) a moratorium on law clerk hiring duringthe Fall of 2002, (2) an arrangement ensuring that the hiring oflaw clerks will not be done earlier than the Fall of the third yearof law school, and (3) an agreement that the focus of law clerkhiring will be on third year law students and law graduates.The precise terms of the new Plan are set forth in the attached"Summary," and the history leading to the Plan's adoption appearsin the appended "Background" statement. More than two-hundredfederal appellate judges considered the Plan. Ninety-two percent(92%) of the judges either supported the Plan or indicated that theywould not oppose it. Thus, the "substantial consensus" requirement
that was needed to put the Plan into operation was easily satisfied.
Judges memo cont
8/10/2019 7. Unraveling
68/118
68
Judges memo, cont.There are numerous advantages to a law clerk hiring system that
focuses on third year students and law graduates. Law clerkcandidates will be able to present more information with theirapplications and be judged more fairly. Thus, for example,applicants who are in their third year of law school will be able tooffer
a transcript showing performance during four semesters of school ina good variety of courses;
information on law journal selection, journal publications, andelection to a journal editorial board;
writing samples from seminar courses; information regarding experience gained in clinical courses and
public interest endeavors;
references from law professors for whom the student has worked asa research assistant or a teaching assistant; recommendations associated with judicial internships; significant recommendations from Summer employers; information on moot court competitions; and information on selection or election to positions in student
government.
8/10/2019 7. Unraveling
69/118
69
Initial FAQ (accompanying judges memo)
Q Does the Plan endorse Summer interviewing?A No. Many judges would have opposed the Plan had it endorsed
Summer interviewing. There was a concern that Summer interviewswould be very inconvenient for many people. The reasons aremanifold: many judges are away on vacation during the Summer;law clerk applicants are otherwise occupied with Summer jobs,
vacations, foreign travel, and bar examinations (for recentgraduates); law professors often are away on vacation and thusunavailable to furnish references; and law schools do not releasegrades on any uniform schedule, so official student transcripts fromsome law schools are not available until near September. However,the Plan does not forbid a law student who, say, is fromVirginia and working in Tulsa during the Summer from talking
with a judge who is otherwise available to chat. This hashappened under existing hiring arrangements and the judges sawno reason to prohibit it under the new Plan. The main point,however, is that the formal hiring process will take place in the Fallwhen applications will be submitted and materials and referencesfrom the law schools will be sent to the judges.
8/10/2019 7. Unraveling
70/118
70
Initial FAQ, cont.
QHow is "Fall" determined under the Plan?AThere is no fixed definition of Fall, nor is there any
fixed starting date for the hiring season. Underexisting arrangements, some judges do their hiring inSeptember, some in October, and others do it even later.
The Plan does not change this.QAre judges forbidden from making "exploding offers," i.e.,
offers that require an applicant to respond promptly to anoffer?
AThe Plan does not purport to address how an offer is
given by a judge. This is for each judge to determine.However, no applicant is obliged to act on an offer if theterms are unacceptable, nor is an applicant obliged toaccept the first offer that he or she receives.
8/10/2019 7. Unraveling
71/118
71
Subsequent refinements of rules
From year to year, changes have beenmade to
Put precise start dates for applications (after
Labor Dayalready specified by start ofmarket in 2003)
Specify later precise dates for
Scheduling interviews
Conducting interviews and making offers
No rules regarding exploding offers
8/10/2019 7. Unraveling
72/118
72
2004 modifications
The critical dates under the Revised Law Clerk Hiring Planare as follows:
After Labor Day: Third year law students and law schoolgraduates may submit law clerk applications and lettersof reference may be submitted on their behalf.
From the Day After Labor Day Through the SecondSunday After Labor Day (September 7- 19 in 2004):Reading period.
Beginning on the first Monday after Labor Day(September 13in 2004): Judges may schedule
interviews to be held after the reading period. Beginning on the Second Monday After Labor Day
(September 20in 2004): Judges may conduct interviewsand extend offers.
Subsequent modification of dates in 2005.
Critical Dates under the Law Clerk Hiring Plan for 2011
8/10/2019 7. Unraveling
73/118
Critical Dates under the Law Clerk Hiring Plan for 2011http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/CriticalDates?OpenDocument
73
Offers may be made as soon as interviews are permitted under
http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/CriticalDates?OpenDocumenthttp://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/CriticalDates?OpenDocument8/10/2019 7. Unraveling
74/118
74
Offers may be made as soon as interviews are permitted underthe Plan. Generally, it is for the judge to determine the termsupon which an offer is extended. However, judges areencouraged not to require an applicant to accept an offerimmediately without reasonable time to weigh it against otherviable options that remain open to the applicant. This wouldnot prohibit an applicant from accepting an offer on the spot.
When setting up an interview with a clerkship applicant, ajudge should make clear to the applicant his or her interviewand offer policies or practices. For example, a judge may havea policy or practice of making offers and entirely filling his orher clerkship slots, even if more interviews are scheduled for
that day. The applicant should be told this in a timely fashion,so that the applicant's decision to accept or decline theinterview is appropriately informed. Applicants should also beinformed if the judge will ask them to make a decision on thespot.
http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/SUMMARY?OpenDocument
http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/SUMMARY?OpenDocumenthttp://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/SUMMARY?OpenDocument8/10/2019 7. Unraveling
75/118
75
Data
We have repeated the surveying of both federal appellatejudges and applicants that we did in our prior study.
Federal appellate judges surveyed in fall of 2004 and fallof 2005.
Third year law students at the four law schools thatprovide the greatest number of clerks surveyed in fall of2004, fall of 2005, and fall of 2006.
About a 50% response rate from both federal appellatejudges and students. On the key measure of judge
reports of adherence versus nonadherence to the startdates, if anything selection bias should lead us tounderestimate the level of nonadherence.
Adherence to start dates (2004 judge
8/10/2019 7. Unraveling
76/118
76
Adherence to start dates (2004 judgesurvey; gray areas = nonadherence)
Number and (in parentheses) cumulative percentage of
responding judges
Before
September
7
September
712
September
1319
September
2026
After
September
26/Not yetDate of first
interview
11
(9%)
6
(15%)
22
(33%)
66
(91%)
11
(100%)
Date of first
offer
5
(4%)
9
(12%)
11
(21%)
67
(78%)
26
(100%)
Judges perceptions: 2004
8/10/2019 7. Unraveling
77/118
77
Judges perceptions: 2004
Number and (in parentheses) cumulative percentage of responding judges
Overall Within
Circuit
Relativelyfew judges
adhered
Toresponding
judges
knowledge,
many judges
adhered buta substantial
number didnot
Toresponding
judges
knowledge,
almost all
judgesadhered
Toresponding
judges
knowledge,
all judges
adhered
At leastone judge
in Circuit
did not
adhere
Start date
for
scheduling
interviews
3(3%)
34
(36%)
41(75%)
26(100%)
36(69%)
Start date
for
conducting
interviews
and making
offers
5(5%)
43
(46%)40
(85%)16
(100%)36
(73%)
8/10/2019 7. Unraveling
78/118
78
Judges perceptions: 2005Number and (in parentheses) cumulative percentage of responding judges
Overall Within
Circuit
Relatively
few judges
adhered
To
responding
judges
knowledge,
many judges
adhered but
a substantialnumber did
not
To
responding
judges
knowledge,
almost all
judges
adhered
To
responding
judges
knowledge,
all judges
adhered
At least
one judge
in Circuit
did not
adhere
Start date
for
scheduling
interviews
5
(6%)
40
(52%)27
(84%)
14
(100%)
35
(87%)
Start date
for
conducting
interviews
and making
offers
4
(5%)
44
(58%)23
(86%)
12
(100%)
34
(81%)
Student reports: 2004
8/10/2019 7. Unraveling
79/118
79
Student reports: 2004(gray areas = nonadherence)
Number and (in parentheses) cumulative percentage ofresponding students
Before
September
7
September
712
September
1319
September
2026
After
September
26/Not yet
Date of first
scheduling ofinterview
8(5%)
39
(31%)94
(92%)8
(97%)5
(100%)
Date of first
interview
9
(6%)
7
(11%)
18
(23%)
101
(91%)
13
(100%)
Date of first
offer
4
(3%)
3
(5%)
8
(12%)84
(77%)
29
(100%)
8/10/2019 7. Unraveling
80/118
Student reports: 2006
8/10/2019 7. Unraveling
81/118
81
Student reports: 2006(gray areas = nonadherence)
Number and (in parentheses) cumulative percentage of
responding students
Before
Sept. 5
Sept. 5-13 Sept. 14-
20
Sept. 21-27 After Sept.
27/Not Yet
Date of first
scheduling ofinterview
13(11%)
31
(38%)68
(97%)3
(99%)1
(100%)
Date of first
interview
13
(11%)
6
(16%)
18
(32%)
77
(97%)
3
(100%)
Date of first
offer
10
(9%)
3
(12%)
10
(22%)69
(87%)
14
(100%)
Exploding offers remain a problem:
8/10/2019 7. Unraveling
82/118
82
Exploding offers remain a problem:Some representative quotes
I received the offer via voicemail while I was in flight tomy second interview. The judge actually left threemessages.First, to make the offer.Second, to tell me that I should respond soon.
Third, to rescind the offer.It was a 35 minute flight.
I had 10 minutes to accept.
I asked for an hour to consider the offer. The judgeagreed; however thirty minutes later [the judge] calledback and informed me that [the judge] wanted to rescindmy offer.
8/10/2019 7. Unraveling
83/118
8/10/2019 7. Unraveling
84/118
84
Fast market clearing: student responses
Fall of 2004 Fall of 2005 Fall of 2006
First offer received on start date
for interviewing and making
offers
38 51 45
First offer received after startdate for interviewing and making
offers
59 52 38
Of first offers received on start
date for interviewing and making
offers, percentage accepted onstart date
42% 63% 62%
So, for a nontrivial proportion of students, the market ends after the firstinterview (and some offers come even before the start date). Similarly forjudges, the market is not thick.
R
8/10/2019 7. Unraveling
85/118
85
Recap
The market has now successfully held the latedate (fall of third year of law school) for severalyears in a row (2003- ). This yields an advantage due to better information
This certainly helps the many non-complying judgeswho make the earliest offers, and apparently alsohelps many of the complying judges.
But congested interviewing, exploding offers arestill a problem
This means that for many participants, the market stillisnt in fact thick.
What are the prospects for the future?
Stages and transitions observed in various other marketsith ti i bl
8/10/2019 7. Unraveling
86/118
86
with timing problems
Stage 1: UNRAVELING
Offers are early, dispersed in
time, exploding
Stage 2: UNIFORM DATESENFORCED
Deadlines, congestion
Stage 3:
CENTRALIZED MARKET
CLEARING PROCEDURES
U if t t d t i th l l k k t
8/10/2019 7. Unraveling
87/118
87
Uniform start dates in the law clerk market
Stage 1: UNRAVELING
Stage 2: UNIFORM
DATES ENFORCED
1983, 1986, 1989, 1990, 1993, 2002
1983: Sept 15 of 3rdyear, abandoned in 84
1986: April 1 of 2ndyear
1989: March 1, not adopted
1990: May 1, 12:00 Noon
1993: March 1 (not formally abandoned until 1998)
Thoughts on the future evolution of the
8/10/2019 7. Unraveling
88/118
88
Thoughts on the future evolution of theclerk market
It has cycled between stage 1 and stage 2multiple times
Comparison to other markets, Stage 2 markets
Psych25 years in stage 2. Japanese universities
The current arrangement has formally heldtogether longer than the previous longest (1993-97) stage 2 arrangement. But theres lots of
non-compliance. Discussions are underway about market
design
Would unraveling occur if
8/10/2019 7. Unraveling
89/118
gsalaries were competitive?
And what are competitive salaries? Recallthe case of law firms
89
8/10/2019 7. Unraveling
90/118
90http://www.nalp.org/reported_salaries_by_city
http://www.nalp.org/reported_salaries_by_cityhttp://www.nalp.org/reported_salaries_by_cityhttp://www.nalp.org/reported_salaries_by_city8/10/2019 7. Unraveling
91/118
91
salaries of
$160,000accounted for25% of reportedsalaries.
salaries of$160,000accounted
for about18% ofreportedsalaries
Law Firms
8/10/2019 7. Unraveling
92/118
Law Firms
In an attempt to regulate the market, theNALP, formed in 1971, issued regulationsintended to govern the timing and duration
of offers by law firms. E.g. a regulationadopted in 1988 specified that offersshould remain open for at least two weeks,
and others specified dates until whichoffers should remain open.
92
8/10/2019 7. Unraveling
93/118
But Roth and Xing (1994, p1007) note that
some law firms began to give offers which
met the letter of [these regulations], but
which structured the compensation so thatthe offer was competitive because itincluded a signing bonus which could
only be collected if the offer was acceptedmuch more promptly.
93
2010 rules (NALP 2010c),
8/10/2019 7. Unraveling
94/118
All offers to law student candidates(candidates) should remain open for at least
two weeks after the date of the offer letter Q.May employers offer signing bonuses that
decline or evaporate according to the date ofacceptance of the offer of employment?A.This type of signing bonus violates the letterand spirit of the Principles & Standards.
Signing bonuses or other benefits that require a
decision in advance of the dates in Part V or thatvary according to the date of acceptance areconsidered special inducements and violate thisprovision.
94
Which markets are unraveled?
8/10/2019 7. Unraveling
95/118
95
Supply and demand.
Li and Rosen, Li and Suen, insurance Halaburda
Niederle, Roth and Unver In some markets unraveling is attributed to an
imbalance of supply and demand
But to get unraveling you need two things: Firms have to be willing to make early offers
Workers have to be willing to accept them
This is most likely to occur when both firms andworkers can plausibly think of themselves as being onthe long side of the marketa common state of mindin the markets we study
Simple supply and demand hypothesis
8/10/2019 7. Unraveling
96/118
p pp y yp Excess s upp ly or d emand ( in the relevant part of the market)
increases competi t ion, wh ich causes ineff ic ient ly ear lytransact ions.
Menard (book review, 2003), on college admissions:"There are many reasons that college admission has become so complex,
but the main one is demand..(In 1932 Yale admitted 72% of applicants,now 13%).
Avery, Fairbanks, and Zeckhauser (book, 2003, p32) quote a 1990U.S. News and World Report story.
"Many colleges, experiencing a drop in freshman applications as thepopulation of 18-year-olds declines, are heavily promoting early-acceptance plans in recruiting visits to high schools and in campus toursin hopes of corralling top students sooner."
Roth (JPE 1984about the market for new physicians around 1900,"The number of positions offered for interns was, from the beginning,
greater than the number of graduating medical students applying for
such positions, and there was considerable competition amonghospitals for interns. One form in which this competition manifesteditself was that hospitals attempted to set the date at which they wouldfinalize binding agreements with interns a little earlier than their principalcompetitors."
96
Comparable supply and demand
8/10/2019 7. Unraveling
97/118
p pp yhypothesis
Early transactions require both that firmsshould want to make early offers, andworkers should want to accept them.
So too much imbalance in supply anddemand should notbe associated withunraveling.
97
In many markets, both sides feel they are on the long side
8/10/2019 7. Unraveling
98/118
Many positions available
at unselective colleges
Fewer positions at elite
colleges than students
eager to enroll.
College admissions
Fewer top teams than
topbowls
More teams than bowlsPostseason College
Football Bowls (before
BCS)
Fewer top and research
oriented applicants than
positions
More board-certified
applicants than positions
Medical subspecialties
Many more interested
foreign medical graduatesthan American positions
Many more first year
positions than graduatesof American med schools
American medical
residents
Fewer Law Review
editors than Federal
appellate judges
Many more law grads
than judges
Federal court clerkships
Many positions available
at unselective colleges
Fewer positions at elite
colleges than students
eager to enroll.
College admissions Fewer positions at elite
colleges than students
eager to enroll.
College admissions
Fewer top teams than
topbowls
More teams than bowlsPostseason College
Football Bowls (before
BCS)
More teams than bowlsPostseason College
Football Bowls (before
BCS)
Fewer top and research
oriented applicants than
positions
More board-certified
applicants than positions
Medical subspecialties More board-certified
applicants than positions
Medical subspecialties
Many more interested
foreign medical graduatesthan American positions
Many more first year
positions than graduatesof American med schools
American medical
residents
Many more first year
positions than graduatesof American med schools
American medical
residents
Fewer Law Review
editors than Federal
appellate judges
Many more law grads
than judges
Federal court clerkships Many more law grads
than judges
Federal court clerkships
98
8/10/2019 7. Unraveling
99/118
Supply and demand model
8/10/2019 7. Unraveling
100/118
nFfirms, nFhof high quality
Firm quality is common knowledge fromtime 1
nAapplicants, of which nAhwill eventually
become high quality
Only the proportion nAh/nAis known until the
first period of the latehiring stage, when thehigh quality individuals are realized
Rules: there are multiple early and lateperiods: each unmatched firm may make anoffer to at most one worker in each period,
acceptances are binding100
Rules
8/10/2019 7. Unraveling
101/118
Rules At the beginning of each period, each available firm
can extend an offer to an applicant of its choice. After the firms make offers, the applicants observe all
the offers made by the firms and each applicant canaccept one of the offers she got in the period or rejectthem all.
Acceptances are binding; once an applicant acceptsan offer, firms cannot make her further offers. Each offer is valid only in the period it is extended (it is
an exploding offer; although workers can considermultiple offers if made in the same period )
After the period is over, all decisions made in theperiod become public information.
101
Payoffs
8/10/2019 7. Unraveling
102/118
y Remaining unmatched earns zero. Let EU(i,j) be the expected
payoff of a quality i agent from being matched with a quality jagent. EU(i,j) is increasing in both arguments andsupermodular so that
EU(h,h)-EU(h,)>EU(,h)-EU(,)0 At the beginning of the late stage, a tie-breaking lottery e is
drawn to rank high quality agents among each other and lowquality applicants among each other from the best to the
worst. The lottery results become public information. The payoff of an agent a with quality i being matched with a
partner b of quality j is given byua(b)=EU(i,j)+eb
where eb is the lottery draw of the agent b. E[e]=0 and support
of e is small enough so that being matched with a high qualitypartner is always better than being matched with a low qualitypartner
102
Efficiency and inefficiency due to unraveling
8/10/2019 7. Unraveling
103/118
Well focus on big (and visible) inefficiencies in
which high quality agents are unnecessarilymatched with low quality, i.e. in which thematching isnt as assortative as it could be.
Qual ityw ise Eff ic iency = Sum of the Expected
Payoffs are maximized (without taking tie-breakerinto account) SupermodularityA matching is qualitywise efficient
if and only if it is qualitywise assortative i.e. high typeapplicants are matched with high firms as long as it is
possible and remaining agents are matched with lowquality partners as long as it is possible.
103
Many Nash equilibria
8/10/2019 7. Unraveling
104/118
Many Nash equilibria
But well be able to say something aboutsubgame perfect equilibria.
104
8/10/2019 7. Unraveling
105/118
Subgame perfect equilibria:
8/10/2019 7. Unraveling
106/118
lemmas
Late stage: Any subgame perfect equilibriumproduces assortative matching among thefirms and applicants still unmatched at the
beginning of the late hiring stage. From the beginning: In the cases of excess
supply or demand, nAhnF(Case 1), and
nFnA>nFh(Case 3) the unique subgame
perfect equilibrium outcome is late,assortative matching.
106
What causes late and assortative matchingwhen supply and demand are unbalanced?
8/10/2019 7. Unraveling
107/118
when supply and demand are unbalanced?
Excess supply (case 1): nAh
nFApplicants would accept early offers but firms
prefer to wait and guarantee a high qualityapplicant.
Excess demand (case 3): nFnA>nFh
Applicants wont accept low quality early
offers, since they can always get a low quality
offer late.Applicants would accept high quality early
offers
At equilibrium high quality firms prefer to wait
107
Unraveling that doesnt harm
8/10/2019 7. Unraveling
108/118
efficiency
Lemma: In the case of nFhnA (Case 4,excess demand), the outcome of anysubgame perfect equilibrium is qualitywiseefficient. That is, high quality firms may make early
offers that are accepted, but this doesnt harm(qualitywise) efficiency, since not all highquality firms can be matched anyway. (i.e. ifall applicants are matched, the outcome isqualitywise efficient.)
108
Necessary conditions for inefficient
8/10/2019 7. Unraveling
109/118
unraveling at SPE
Theorem: A qualitywise inefficient earlymatching is an outcome of a subgame perfectequilibrium only if the market is one ofcomparabledemand and supply (Case 2):
nA> nF>nAh
. Inefficiency results at equilibria in which highquality firms wait for the late stages (whenthey can always hire someone), but lowquality firms make early offers which areaccepted (since some applicants will be leftunmatched). But these necessary conditions arent sufficient.
109
Sufficient conditions for full unraveling
8/10/2019 7. Unraveling
110/118
In the case of comparable demand and supply
nA> nF>nAh(Case 2), if0 > nF
huhh+(nF-nFh-nA
h)uh-(nA-nAh)u
and nFh nA
h,
then all low quali ty f i rms h ire in the ear lyhir ing s tage at any subgame perfectequi l ibr ium, leading to a qualitywiseinefficient matching with positive probability.
So we can choose experimentalparameters in which the SPE predictsunraveling.
110
8/10/2019 7. Unraveling
111/118
Experimental design: 4 firms with 1 or 2 position(s)each and 6 or 12 applicants Half of the positions are high quality and
8/10/2019 7. Unraveling
112/118
112
each and 6 or 12 applicants. Half of the positions are high quality andone third of the applicants are high quality.
Treatments6 Appli cants
2 H igh Quali ty Applicants
12 Appli cants
4 H igh Quality Applicants
4 Firm Positi ons
2 H igh Quality
Positions
Baseline Thin
comparable supply/demand
market treatment(Case 2)
Treatment with
excess supply (Case 1)
8 Firm Positions(with f irm quota 2)
4 H igh Quality
Positions
Treatment with
excess demand(Case 3)
Thick comparable
supply/demand market
treatment(Case 2)
Prediction (Theorem) A qualitywise inefficient early matching isan outcome of a subgame perfect equilibrium only if themarket is one of comparable demand and supply (case 2).(Inefficiency results when low quality firms make early offersthat are accepted.)
(SPE) Predictions
8/10/2019 7. Unraveling
113/118
( )
113
SPEnA 6
nAh 2
nA 12
nAh 4
nF 4
nF
h
2
THIN
COMPARABLE
Low quality
firms hire earlyqualitywise-inefficient
outcome
EXCESS
SUPPLY
Late and
assortativematching
efficient outcome
nF 8
nFh 4
EXCESS
DEMAND
Late and
assortative
matching
efficient outcome
THICK
COMPARABLE
Low quality
firms hire early
qualitywise-inefficient
outcome
87.50%
100.00%
Median % High Type Firms Hiring Early
8/10/2019 7. Unraveling
114/118
114
0.00%
12.50%
25.00%37.50%
50.00%
62.50%
75.00%
87.50%
Markets 1-5 Markets 6-10 Markets 11-15 Markets 16-20
THIN COMPARABLE - 6 App - 4 Firm EXCESS SUPPLY -12 App - 4 Firm
EXCESS DEMAND - 6 App - 8 Firm THICK COMPARABLE - 12 App - 8 Firm
0.00%
12.50%
25.00%
37.50%
50.00%
62.50%
75.00%
87.50%
100.00%
Markets 1-5 Markets 6-10 Markets 11-15 Markets 16-20
Median % LowType Firms Hiring Early
THIN COMPARABLE -6 App - 4 Firm EXCESS SUPPLY -12 App - 4 Firm
EXCESS DEMAND - 6 App - 8 Firm THICK COMPARABLE - 12 App - 8 Firm
What have we learned so far?
8/10/2019 7. Unraveling
115/118
115
What have we learned so far? Unraveled markets (with early, dispersed,
exploding offers) produce different outcomes inpredictable ways. Less thick, unsafe, suffer from congestion, inefficient Loss of mobility in the gastro market
Unraveling involves both Market design: e.g. how exploding offers are treated Supply and demand: both firms and workers have to
be willing
Centralized markets can solve these problems Centralized matches are preceded by (potential)
decentralized markets. Even well-designed (stable) matches can unravel Moving from an early to a late equilibrium can be hard
Open questions
8/10/2019 7. Unraveling
116/118
Open questions
what factors make a market prone tounraveling?
Which markets are adequately organized
without a coordinated marketplace(coordinated in time, or space, or both)?
For which markets is a centralizedmarketplace desirable?
For which markets does a centralizedclearinghouse (possibly computer-assisted)offer additional benefits?
116
Experiments have multiple roles to play
8/10/2019 7. Unraveling
117/118
117
Experiments have multiple roles to play
They allow us to investigate questions that thefield data cannot answer E.g. why did the GI match fail in 96?
They allow us to investigate hypotheses
suggested by the field data E.g. does the ability to renege on early acceptancesreduce exploding offers and unraveling?
What conditions of supply and demand promoteunraveling?
They play a role in the considerable amounts ofdiscovery, demonstration, and persuasion thatare necessary to coordinate market participantsto move from one equilibrium to a better one.
Conclusions:
8/10/2019 7. Unraveling
118/118
Markets dont always spring up likeweeds, some of them are hothouseorchids that need care and attention.
We need to understand how marketswork in enough detail so we can fix themwhen theyre broken.