![Page 1: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/1.jpg)
An Empirical Study of Perceptions of Task Difficulty
1
Rafael Leano, Souti Chattopadhyay (Rini) and Anita Sarma
Department of Electrical Engineering & Computer Science
Oregon State University
What Makes A Task Difficult?
VLHCC 2017, Raleigh, North Carolina
![Page 2: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/2.jpg)
2
Task Difficulty
Project
Managers
Task
Developer
Software Engineering
![Page 3: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/3.jpg)
3
Task Difficulty
Project
Managers
Task
Developer
Software Engineering
Time ?Effort ?
![Page 4: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/4.jpg)
4
Task Difficulty
Project
Managers
Task
Developer
Software Engineering
COCOMOSLIMSEER
?
“... software engineers are in great need for techniques for accurate effort estimation, and they are not necessarily knowledgeable about techniques they can use to meet their
needs.” [ Ivanov et al. , FSE2017 ]
![Page 5: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/5.jpg)
5
When and why do developers estimate task difficulty,AND
if they use metrics (and which).
RQs:1. Why do developers perform task estimation?2. How comfortable are developers when performing estimations?3. What project metrics do developers find useful when estimating task
difficulty?
Goal
![Page 6: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/6.jpg)
6
InterviewCard
Sorting Analysis
Survey
Methodology
33 responses2 companies
15 participants34 minute (avg.)
557 cards23 categories
![Page 7: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/7.jpg)
7
Participants thought of:
- One difficult task
- One easy task
Rate the metrics :
Exploratory SurveyCat. Metric
COD
E
LOC - Lines of Code
CCOM - Cyclomatic complexity
CTCR - Comment To Code Ratio
TASK
CMTT - Task Comments
FILT - Task Files
REOP - Reopens
DEVT - Developers associated with task
PRO
CESS FCOM – # commits of the file
CENT - Centrality
DEVF - Developers associated with file
Essential
Worthwhile
Unimportant
Unwise
![Page 8: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/8.jpg)
8
Follow-up Interview
• Task Assignment• Task Management• Metric Ratings (survey)• Identifying difficult tasks
![Page 9: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/9.jpg)
9
RQ1: Why do developers perform task estimation?
Prioritizing tasks7 out of 15
Ensuring optimal task assignments11 out of 15
![Page 10: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/10.jpg)
10
RQ1: Why do developers perform task estimation?
Every developer finds it essential to perform task estimation.
Some do it without acknowledging it as a separate task.
Prioritizing tasks7 out of 15
I choose the difficult task first, …” [P14]
... I follow the one which are easier to solve first.” [P7]
Ensuring optimal task assignments11 out of 15
... [if] I am given a task which is relatively simple to me, then the team is losing out on my experience.” [P12]
![Page 11: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/11.jpg)
RQ2: How comfortable are developers when performing estimations?
11
Comfortable Unsure or Unable
Informal“guesstimation” Unsure
6 out of 15
6 out of 9
9 out of 15
3 out of 9
![Page 12: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/12.jpg)
RQ2: How comfortable are developers when performing estimations?
12
there is no rule ... to say that if [this happens] ... then [the task] is difficult”. [P5]
Developers are either unaware of various metrics.
Or performing estimation with those metrics is difficult.
Usually by the time you get [the task] you have a good idea of how difficult it is going to be …” [P3]
Comfortable Unsure or Unable
Informal“guesstimation” Unsure
6 out of 15
6 out of 9
9 out of 15
3 out of 9
![Page 13: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/13.jpg)
13
RQ3: What metrics do developers find useful when estimating task difficulty?
Useful Metrics (Survey)
Developers claim that some of the existing metrics are useful.
Additional Metrics (Interview)
There are additional, non-traditional metrics that developers find important.
P11 says prioritizing work “... requires looking at the code.”
![Page 14: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/14.jpg)
14
Useful MetricsRank Metric E W UI UW W+%
1 DEVT (T) 9 17 2 5 78.82 FILT (T) 6 18 2 7 72.73 DEVF (P) 8 15 3 7 69.73 FCOM (P) 6 17 8 2 69.74 LOC (C) 3 19 4 7 66.75 CCOM (C) 7 14 4 8 63.65 REOP (T) 4 17 5 7 63.65 CENT (P) 4 17 5 7 63.66 CMTT (T) 6 14 6 7 60.67 CTCR (C) 6 10 7 10 48.5
![Page 15: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/15.jpg)
15
Useful Metrics
# Metric E W UI UW W+%
1 DEVT (T) 9 17 2 5 78.82 FILT (T) 6 18 2 7 72.73 DEVF (P) 8 15 3 7 69.7
3 FCOM (P) 6 17 8 2 69.7
![Page 16: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/16.jpg)
16
FCOM (Process)
Useful Metrics
DEVT (Task)
DEVF (Process)
FILT (Task)1
33
2Reflect the amount of work required to complete the task.
These reflect the evolution (history) of the file.
Traditionally, Code metrics used in models.Developers find Task and Process metrics more useful.
![Page 17: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/17.jpg)
17
Additional Metrics
Category Metric
Collaboration Communication, Coordination, Clients, Access
Project History Task Lifetime, Defects
Task Character Priority, Dependencies
Task Type Bug Reproduction, Framework,Testing
![Page 18: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/18.jpg)
18
IMPLICATIONSFormal Models for Task Estimation
Develop formal models that developers can use themselves.
Task Metrics
Including task-related metrics in formal task estimation models.
Measuring Additional Metrics
Find ways to measure additional metrics that developers find useful.
![Page 20: What Makes A Task Difficultweb.engr.oregonstate.edu/~chattops/presentations/task-difficulty.pdfFormal Modelsfor Task Estimation Develop formal models that developers can use themselves](https://reader034.vdocuments.mx/reader034/viewer/2022052020/603439d0dda98026b414fb79/html5/thumbnails/20.jpg)
20
IMPLICATIONSFormal Models for Task Estimation
Develop formal models that developers can use themselves.
Task Metrics
Including task-related metrics in formal task estimation models.
Measuring Additional Metrics
Find ways to measure additional metrics that developers find useful.