how do software engineers understand code changes?
DESCRIPTION
Presented at FSE2012 http://www.sigsoft.org/fse20/TRANSCRIPT
![Page 1: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/1.jpg)
How Do Software Engineers Understand Code Changes? An Exploratory Study in Industry
Yida Tao (HKUST), Yingong Dang (MSRA), Tao Xie (NCSU)
Dongmei Zhang (MSRA), Sunghun Kim (HKUST)
![Page 2: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/2.jpg)
2
![Page 3: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/3.jpg)
3
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
![Page 4: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/4.jpg)
4
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
![Page 5: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/5.jpg)
5
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
![Page 6: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/6.jpg)
6
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char by char, javascript has awesome string
methods… ”
![Page 7: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/7.jpg)
7
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char by char, javascript has awesome string
methods… ”
![Page 8: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/8.jpg)
8
> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()
“Why this change here? This is the only one that doesn’t seem to make sense for
me…”
>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;
“What is this used for, I can’t spot it in use anywhere and every component and service
seems to have it set to false.”
> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from
this patch, aren’t they?”
>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;
“I’m not sure why you walk this char by char, javascript has awesome string
methods… ”
“…”
“…”
“…”
![Page 9: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/9.jpg)
9
“How do software engineers understand code changes?”
![Page 10: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/10.jpg)
10
Research Questions RQ1: How frequent is code change
understanding practiced and in which development tasks it is required?
RQ2: What are engineers’ information needs and difficulty for understanding code changes?
RQ3: How to improve the effectiveness and efficiency of the practices in understanding code changes?
![Page 11: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/11.jpg)
11
Study Methodology
Literature Review
• Potential information needs
Questionnaire Design
• Investigate RQ1, RQ2
Pilot Interview
• Question is relevant & clear
![Page 12: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/12.jpg)
12
Study Methodology
Literature Review
• Potential information needs
Questionnaire Design
• Investigate RQ1, RQ2
Pilot Interview
• Question is relevant & clear
Online Survey
• 16% response rate (180 respondents)
Follow-up Interview
• Investigate RQ3
Analysis
• Answering RQs
![Page 13: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/13.jpg)
13
Survey Participants
Dev55%
Test31%
PM14%
Role Distribution Product Team
OS
Desktop App
Web App
Mobile App
Service
Others
![Page 14: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/14.jpg)
14
RQ1 • Frequency ?• Development tasks ?
RQ2 • Information needs ?• Difficulty ?
RQ3 • Improvement ?
RQs
![Page 15: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/15.jpg)
15
RQ1: Frequency of Understanding Code Changes
How often do you need to understand code changes?o Several times each houro About once an houro Several times each dayo About once a dayo Several times each weeko About once a weeko Rarelyo Never
![Page 16: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/16.jpg)
16
RQ1: Frequency of Understanding Code Changes
Several times each hour
About once an hour
Several times each day
About once a day
Several times each week
About once a week
RarelyNever
05
101520253035404550
DevTestPM
Abso
lute
# o
f re
sponse
s
![Page 17: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/17.jpg)
17
RQ1: Tasks Requiring Code Change Understanding
“Select the top three tasks that most often require you to understand code changes” [Design/Planning] Refactoring [Implementation] Developing new feature [Implementation] Fixing bug [Integration] Resolving merge conflict [Verification] Reviewing others code changes [Verification] Reviewing my own code changes [Verification] Writing & updating test cases Other, please specify
![Page 18: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/18.jpg)
18
RQ1: Tasks Requiring Code Change Understanding
Reviewing others' changes
Fixing bug
Developing new feature
Reviewing my own changes
Writing/updating test cases
Refactoring
Resolving merge conflict
0% 15% 30% 45% 60% 75%
121
100
89
73
48
34
30
Percentage of participants who select the task
![Page 19: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/19.jpg)
19
RQ1• Frequently practiced• Major development
tasks
RQ2 • Information needs ?• Difficulty ?
RQ3 • Improvement ?
Answers to RQs
![Page 20: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/20.jpg)
20
Potential Information Needs Literature review (code-change analysis and
management) 180 articles in 10 SE venues over the past decade
![Page 21: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/21.jpg)
21
Potential Information Needs Literature review (code-change analysis and
management) 180 articles in 10 SE venues over the past decadeReasoning & assessing the
change• Completeness• Clones• Design• …Exploring the change’s context & impact• Risk• Consistency• Tests• ...
Evaluating the change history• Change-proneness• Defect-proneness
![Page 22: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/22.jpg)
22
Survey Questions “Rate the importance & difficulty of each
information need (formulated as question) in a change understanding task”
Very Importa
nt
Important
Somewhat
ImportantNot
Important
3
2
1
0
![Page 23: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/23.jpg)
23
Survey Questions “Rate the importance & difficulty of each
information need (formulated as question) in a change understanding task”
Very Importa
nt
Important
Somewhat
ImportantNot
Important
Very Difficult
Difficult
Relatively Easy
Straightfor-ward
3
2
1
0
![Page 24: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/24.jpg)
24
Survey Questions “Rate the importance & difficulty of each
information need (formulated as question) in a change understanding task”
Does this change introduce code clones?
Does this change break any code elsewhere?
Which tests should be run to verify this change?
Is this changed location a hotspot for past fixes?……
Very Importa
nt
Important
Somewhat
ImportantNot
Important
Very Difficult
Difficult
Relatively Easy
Straightfor-ward
3
2
1
0
![Page 25: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/25.jpg)
25
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
![Page 26: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/26.jpg)
26
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
RiskConsiste
ncy
Completeness
Design
![Page 27: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/27.jpg)
27
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
RiskConsiste
ncy
Completeness
Design
Rationale
![Page 28: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/28.jpg)
28
RQ2: Information Needs
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
RiskConsiste
ncy
Completeness
Design
Rationale
Change-proneness
Defect-proneness
![Page 29: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/29.jpg)
29
RQ1• Frequently practiced• Major development
tasks
RQ2• Risk & Quality are
important but difficult to know
RQ3 • Improvement ?
Answers to RQs
![Page 30: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/30.jpg)
30
RQ3: Interview Items
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
Risk
Rationale
Change-proneness
Defect-proneness
![Page 31: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/31.jpg)
31
Determining a Change’s Risk
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
Risk
![Page 32: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/32.jpg)
Current Practice on Determining a Change’s Risk
32
Manual Code Review• Error-prone• Cross-components• Unclear interface• Hidden assumptions• …
Unit & Regression Testing• Time consuming• Depends on how thorough the tests are• …
![Page 33: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/33.jpg)
33
Support Determining a Change’s RiskM
anual co
de
revie
w • Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
![Page 34: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/34.jpg)
34
Support Determining a Change’s RiskM
anual co
de
revie
w • Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
Diff
Code Analy
sis
Navigation in diff
“…miss a level of understanding object
relationships”
![Page 35: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/35.jpg)
35
Support Determining a Change’s RiskTe
stin
g • which code must be retested as it is dependent upon the change?
• who owns testing that dependency?
• which tests must be run?
“An ‘Intelli-sense’ for updating these (affected) tests would be nice as well.”
![Page 36: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/36.jpg)
36
Discussion
1 2 30
1
2
3
Importance
Diffi
cu
lty o
f acq
uir
ing
th
e
info
rmati
on
Rationale
Change-proneness
Defect-proneness
![Page 37: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/37.jpg)
Discussion
37
• Availability & Quality of commit message• “It’s entirely up to the dev making the change as to
how hard or easy it is for someone else to figure out why the change was made.”
Why is understanding the rationale of a change easy?
• Developers• Here and now• Short-term issue• Own Knowledge
• Testers & PMs• Historical metrics might be good to reflect
bugginess and complexity of a specific area
Why are historical metrics not that important?
![Page 38: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/38.jpg)
38
Other Information Needs
“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”
![Page 39: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/39.jpg)
39
Other Information Needs
“Can this change be broken into smaller discreet
changes?”
“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”
![Page 40: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/40.jpg)
40
Composite Code Change
![Page 41: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/41.jpg)
41
Understanding a Composite Code Change
![Page 42: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/42.jpg)
42
Decomposing a Composite Code Change
![Page 43: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/43.jpg)
43
RQ1• Frequently practiced• Major development
tasks
RQ2• Risk & Quality are
important but difficult to know
RQ3• Determining a
change’s risk• Decomposing a
composite change
Answers to RQs
![Page 44: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/44.jpg)
44
Summary Evidence
Understanding code changes is a fundamental practice that happens frequently in major development tasks
![Page 45: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/45.jpg)
45
Summary Evidence
Understanding code changes is a fundamental practice that happens frequently in major development tasks
Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change
![Page 46: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/46.jpg)
46
Summary Evidence
Understanding code changes is a fundamental practice that happens frequently in major development tasks
Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change
Practical Needs Navigation in diff Change decomposition Available & informative commit message
![Page 47: How do software engineers understand code changes?](https://reader036.vdocuments.mx/reader036/viewer/2022062707/55864794d8b42a3c348b51a0/html5/thumbnails/47.jpg)
47
Acknowledgment
All participants of survey / interview
Miryung Kim, Robin Moeur, Thomas Zimmermann, Jacek Czerwonka, and Kathryn McKinley