program report pldi 2010
DESCRIPTION
Program Report PLDI 2010. Alex Aiken. Thanks!. Ben Zorn The Program Committee The ERC and other reviewers The PLDI steering committee. The Process. The Program Committee. 22 people Papers received >= 3 PC reviews Except for PC submissions Met for 1.5 days in January - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/1.jpg)
Program ReportPLDI 2010
Alex Aiken
![Page 2: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/2.jpg)
Thanks!
• Ben Zorn
• The Program Committee
• The ERC and other reviewers
• The PLDI steering committee
![Page 3: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/3.jpg)
The Process
![Page 4: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/4.jpg)
The Program Committee
• 22 people
• Papers received >= 3 PC reviews– Except for PC submissions
• Met for 1.5 days in January– Every PC member attended– 11.5 hours of discussion
![Page 5: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/5.jpg)
The External Review Committee (ERC)
• 57 people
• Every paper received >= 1 ERC review– Except for PC submissions
• Discussion via – conference software– email– phone calls
![Page 6: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/6.jpg)
Why Have an ERC?
• Deal with double-blind reviewing conflicts– More on this shortly
• Organize and recognize ERC– We have an ERC whether we call it that or not
![Page 7: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/7.jpg)
Significant Changes This Year
• Allowed PC submissions
• Changes to double-blind reviewing guidelines
![Page 8: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/8.jpg)
PC Submissions
• The problem– PC accept rates declining– Becoming burdensome to staff the PLDI PC– #1 reason: the PC submission policy
• The PC has been banned from submitting for many years
• Consensus– A real problem– Needed to be fixed
![Page 9: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/9.jpg)
PC Submissions (Cont.)
• Design a mechanism for evaluating PC submissions with no conflicts of interest
• Principle– No PC member participates in evaluation or dicussion
of any PC paper
• Solution– The ERC reviews and decides PC submissions
• Consensus required to accept a PC paper• i.e., all reviewers must agree the paper is acceptable
![Page 10: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/10.jpg)
Double-Blind Reviewing Issues
• Only the PC chair knows reviewers and authors– Bottleneck in, e.g., detecting conflicts of interest– Single point of failure
• Double-blind reviewing not always desirable– Potential for authors to use it to withhold information– Lack of context for reviewers
• Unclear rules for authors– No bright line between what is and what is not permitted in
submissions• See past efforts to write double-blind submission policies. . .
![Page 11: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/11.jpg)
Double-Blind Reviewing Changes
• Simplified policy– Only two rules:– No author names/institutions in the paper– Use 3rd person to describe own prior work
• Author identities revealed before feedback– Made feedback direct– Gave reviewers time to consider context
“First-read double-blind”
![Page 12: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/12.jpg)
Feedback
• On PC submissions– Positive
• On double-blind reviewing– Positive from authors & PC members– Some negative feedback from double-blind
champions• But some of these people become more agreeable
after hearing about the problems the simplified policy addresses
![Page 13: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/13.jpg)
The Papers
![Page 14: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/14.jpg)
The Bottom Line
• 206 submissions• 41 papers accepted• 20% accept rate
• Accept rate was the same for PC papers
![Page 15: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/15.jpg)
Submissions By Topic
![Page 16: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/16.jpg)
Accept Rate by Topic
“Error bars” show ccept rate +/- 2 papers
Key:Red line: overall accept rateRed topics: below average accept rateBlue topics: above average accept rateTopics sorted by decreasing # of submisisons
![Page 17: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/17.jpg)
And Now For Something Completely Different . . .
The Health of the Field
![Page 18: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/18.jpg)
A Conversation
• “I worry that programming languages is dying as a research field.”– A respected colleague
• My perception is just the opposite!
• But, is the statement true?
• How can we measure the health of a field?
![Page 19: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/19.jpg)
Measures
• Standard measures– Papers, citations, conferences– All look healthy– But these are lagging indicators
• People– Is the field attracting new talent?– Predictive of activity in the future
![Page 20: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/20.jpg)
Year of 1st Paper for PLDI’10 AuthorsGraph shows number of PLDI’10 authors who published their first scientific paper (in any forum according to DBLP) in a given year.
![Page 21: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/21.jpg)
Cumulative Plot
2007-2010likely
current graduate students
2001-200648 people
1980-200148 people
Conclusion: The field’s pipeline is healthy!
![Page 22: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/22.jpg)
Log-Log Plot
![Page 23: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/23.jpg)
What Does This Mean?
• The log-log plot is very nearly linear– Only significant deviation is in the region where
there is very little data anyway– Strongly suggests the number of active
researchers follows a power law– A polynomial whose degree is the slope of the
line in the log-log plot
• Conclusion: People drop out of research at a steady rate at all ages
![Page 24: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/24.jpg)
Possible Explanations
• Mentors– Senior people work with junior people
• Deadwood– Senior people publish fewer papers
• Moving on– There is life after research
![Page 25: Program Report PLDI 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816790550346895ddcc112/html5/thumbnails/25.jpg)
Conclusions
• Good news– Lots of young people entering the field– Just like always
• Melancholy news– Half of us won’t be here in 5-7 years– Applies almost equally to all levels of seniority