writing acceptable patches: an empirical study of open source project patches

Download Writing acceptable patches: an empirical study of open source project patches

Post on 02-Jul-2015

151 views

Category:

Software

0 download

Embed Size (px)

DESCRIPTION

Presented at ICSME2014 http://www.icsme.org/

TRANSCRIPT

  • 1. Writing Acceptable Patches: An EmpiricalStudy of Open Source Project PatchesYida Tao*, DongGyun Han and Sunghun Kim**The Hong Kong University of Science and TechnologyKAIST Institute for IT Convergence, Korea Advanced Institute of Science and Technology

2. Developers submit many patchesbut not all of them can be acceptedCode Reviewer2 3. Developers submit many patchesbut not all of them can be acceptedPatch ReviewerITS A NO FROM ME!3 4. 70%60%50%40%30%20%10%0%Firefox(Baysal et al.WCRE12)Apache(Rigby andGerman, 2006)FLAC,OpenAFS(Weigerber et al. MSR08)Patch rejection& resubmission56%40%60%4 5. Patch rejection & resubmission = ?increased interruption and workload5 6. Patch rejection & resubmission =increased interruption and workload6 7. We hope our patches to beaccepted right after the first orearly rounds of code review7 8. How to writeacceptablepatches?8 9. How to writeacceptablepatches? Learn from past failures Q1: Why are patches rejected? Understand the priority Q2: Which reasons are more decisive to reject a patch? Q3: Which reasons are more difficult to judge? Analyze primary stakeholders Q4: Do patch reviewers and writers have unanimouspatch evaluation criteria?9 10. How to writeacceptablepatches? Learn from past failures Q1: Why are patches rejected? Understand the priority Q2: Which reasons are more decisive to reject a patch? Q3: Which reasons are more difficult to judge? Analyze primary stakeholders Q4: Do patch reviewers and writers have unanimouspatch evaluation criteria?10 11. How to writeacceptablepatches? Learn from past failures Q1: Why are patches rejected? Understand the priority Q2: Which reasons are more decisive to reject a patch? Q3: Which reasons are more difficult to judge? Analyze primary stakeholders Q4: Do patch reviewers and writers have unanimouspatch evaluation criteria?11 12. Learn from past failures Q1: Why are patches rejected? Understand the priority Q2: Which reasons are more decisive to reject a patch? Q3: Which reasons are more difficult to judge? Analyze primary stakeholders Q4: Do patch reviewers and writers have unanimouspatch evaluation criteria?Patch extraction& inspectionDeveloper surveyLiterature12 13. Patch Extraction & Inspection13 14. Extracting Rejected Patches Projects Eclipse and Mozilla Bug tracking system Bugzilla Criterion for rejected patch Patches with explicit review- flag Number of patches 300 randomly inspected 162 Eclipse + 138 MozillaPatch attachments for Eclipse bug#233156spell checking performance test (patch)2009-11-23no flagsperformance test (5.69 KB, patch)2009-11-30no flagsperformance test + fix (12.95 KB, patch)2009-12-01review-fix(7.63 KB, patch)2010-01-04review-performancetest (6.53KB, patch)2010-01-04review+fix (10.08 KB, patch)2010-01-14review+14 15. Patch Inspection Open coding Review comments of 300 rejected patches # open codes: 637The patch fixes CCE butintroduces a new bug: Also the parameter shouldbe renamed to Open code: new bug introduced bad parameter name15 16. Patch InspectionOpen code:malformed patchOpen code:wrong indentationTheme:Violating coding styles Open coding16 Review comments of300 rejected patches # open codes: 637 Axial coding 17. Patch InspectionOpen code:malformed patchOpen code:wrong indentationTheme:Violating coding styles Open coding17 Review comments of300 rejected patches # open codes: 637 Axial coding 12 themes why patchesare rejected 18. Theme Frequency24.6%Suboptimal solution18 19. Theme Frequency24.6%19Incomplete fix22.3% 20. Theme Frequency24.6%2022.3%11.9% Including unnecessary changes 21. Theme FrequencyViolating coding styles9.7%24.6%2122.3%11.9% 22. Bad NamingInconsistent, misleadingdocumentationMissing documentationTest FailureIntroducing New BugsCompilation errorsDuplicate code5.5%Theme Frequency5.7%6.1%6.3%2224.6%22.3%11.9%9.7% 23. Suboptimalsolution, 24.6%Incomplete fix,22.3%IncludingunnecessaryIntroducing newbugs, 4.1%Test failures,5.5%Missingdocumentation,5.7%Inconsistent,misleadingdocumentation,6.1%Bad naming,6.3%Violating coding change, 11.9%style guidelines,9.7%Compilationerrors, 1.9%Duplication, 1.6%Patch size toolarge, 0.3%12 patch rejection reasons Developer surveyfrom manual inspection Decisive score of each reason246 / 1785 (14%) Difficulty to judge each reasonEclipse and Mozilladevelopers23 24. Q2: How decisive each rejection reason is?24 25. Patch reviewers have very low tolerance for patchesthat fail compilation or tests, or introduce new bugs25 26. Inconsistent or misleading documentation is highlyunacceptable and way worse than missing documentation26 27. Inconsistent or misleading documentation is highlyunacceptable and way worse than missing documentation27 28. Reviewers reject a large patch not solely because of its size,but mainly because the underlying reasons that induce itslarge size28 29. Reviewers reject a large patch not solely because of its size,but mainly because the underlying reasons that induce itslarge size29 30. Q3: Which patch-rejection reasonsare more difficult to judge?30 31. Very easy Very difficult to judgeNot decisive Very decisive to reject a patch 31 32. Inconsistent, misleadingdocumentationTest failuresCompile errorsVery easy Very difficult to judgeNot decisive Very decisive to reject a patch 32 33. Very easy Very difficult to judgeDuplicationUnnecessarychangesNot decisive Very decisive to reject a patch33 34. Very easy Very difficult to judgeIncomplete fixSuboptimal solutionIntroducenew bugsNot decisive Very decisive to reject a patch34 35. Q4: Do patch reviewers andwriters have unanimous patchevaluation criteria?35 36. Distinguish patch reviewers and patch writersMethod Patch writer: write 1 patch Patch reviewer: review 1 patch or both Responses 246 respondents 131 patch writers, 115 patch reviewers36 37. Avg. Decisive Score2 3 4 537Compilation errorsTest FailuresIntroducing new bugsInconsistent DocumentationSuboptimal solutionIncomplete fixDuplicationUnnecessary changesViolating coding stylesBad namingMissing documentationPatch size too largePatch reviewer Patch writer 38. Avg. Decisive Score2 3 4 538Compilation errorsTest FailuresIntroducing new bugsInconsistent DocumentationSuboptimal solutionIncomplete fixDuplicationUnnecessary changesViolating coding stylesBad namingMissing documentationPatch size too largePatch reviewer Patch writer 39. Avg. Decisive Score2 3 4 539Compilation errorsTest FailuresIntroducing new bugsInconsistent DocumentationSuboptimal solutionIncomplete fixDuplicationUnnecessary changesViolating coding stylesPatch reviewers tend to have stricter criteriaon patch quality Bad namingcompared to patch writersMissing documentationPatch size too largePatch reviewer Patch writer 40. Why are patches rejected?40 41. Why are patches rejected?Patch inspection41 42. Why are patches rejected?+Patch inspection Developer survey42 43. Why are patches rejected?+ +Patch inspection Developer survey Literature43 44. Why are patches rejected?+ +Patch inspection Developer survey Literature30 patch rejection reasons in 5 categories44 45. 1. Problematic implementation or solution2. Difficult to read or maintain3. Lack of communication or trust4. Deviating from the project focus or scope5. Affecting the development schedule30 patch rejection reasons in 5 categories45 46. 1. Problematic implementation or solution Compilation & test failures Incomplete Suboptimal Solution too aggressive orhostile for end users46 47. 2. Difficult to read or maintain Coding style violation Inconsistent documentation Using deprecated API No accompanied test cases47 48. 3. Lack of communication or trust No discussion before patch submission Patch writers are not responsive4. Deviating from the project scope or focus Irrelevant or obsolete5. Affecting the development schedule Freeze the progress of other features48 49. Please check our paper for moredetails on patch rejection reasons49 50. SummaryWriting acceptable patches50 51. Checklist* Compile error* Test failure* Documentation* Duplication* Unnecessary changes51 52. Checklist* Compile error* Test failure* Documentation* Duplication* Unnecessary changes52Communication* Discussion prior topatch submission* Timely response toreview comments 53. Checklist* Compile error* Test failure* Documentation* Duplication* Unnecessary changes53Communication* Discussion prior topatch submission* Timely response toreview commentsCall for industrial &research efforts* Introduce new bugs* Suboptimal solutions* Incomplete fix