the design of bug fixes

The Design of Bug Fixes Emerson Murphy-Hill

North Carolina State University

Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan

Microsoft Research , USA

ICSE 2013

About the paper Empirical study performed in Microsoft Investigated two research questions:

What are the different ways that bugs can be fixed? (Design space)

What factors influence which fix an engineer chooses? (Decisions / Navigating design space)

Contribution The first systematic characterization of the

design of bug fixes

Overview

Methodology 1. Opportunistic Interviews

Randomly went to 4 product groups Asked 8 engineers per group about his last fixed

bug for 10-15 min

Methodology 2. Firehouse Interviews

Observe a just-completed fix in a bug tracker “Rush to the scene” 8 engineers from a different product group

Methodology (cont.) 3. Triage Meetings

6 meetings Little data could be gathered: Lack of context

4. Survey Multiple-choice, estimated 15-20 minutes 324 out of 2000 responsed (16%)

Results: Design Space

Results: Design Space Data propagation

Layered infrastructure with four layers, top most being the UI

Top: expecting megabytes Bottom: only accept kilobytes

Results: Design Space (cont.)

Error Surfacing A crash when a user is deleting file Engineer: catch exception and handle it Whether or not to notify the user

Results: Design Space (cont.)

Behavioral Alternatives In a kind of mobile device When first load, back botton can not work

simultaneously with other features Choose a certain behavior

Results: Design Space (cont.)

Functionality Removal Rather than fixing, just remove the whole

feature 75% of respondents said they had removed

features from their software to fix bugs in the past

Results: Design Space (cont.)

Refactoring

Results: Design Space (cont.)

Internal vs External A testing framework Changing the framework (internal) Changing other team’s code (external)

Results: Design Space (cont.)

Accuracy A web browser is not printing well Accurate: print driver retrieve the fonts Less accurate: use a heuristic to produce a

better but not optimal printing

Results: Design Space (cont.)

Hardcoding Receive database queries, with some queries

malformed Hardcoded, using a fixed set of queries

Results: Navigating the Design Space

Results: Navigating the Design Space (cont.)

Risk Management by Development Phase

Results: Navigating the Design Space (cont.)

Risk Management by Development Phase

Results: Navigating the Design Space (cont.)

Interface Breakage

Results: Navigating the Design Space (cont.)

Consistency

Results: Navigating the Design Space (cont.)

User Behavior

Results: Navigating the Design Space (cont.)

User Behavior

Results: Navigating the Design Space (cont.)

Cause Understanding 18% indicated that they have not had “time to

figure out why the bug occurred”

Results: Navigating the Design Space (cont.)

Social Factors

Implications Additional Factors in Bug Prediction and

Localization Limits of Bug Prediction and Localization Refactoring while Fixing Bugs Usage Analytics Fix Reconsideration

Limitations Generalizability beyond the population Expectancy bias Memory bias Unable to collect contextual information Misunderstanding