imws2014 requirements engineering quality revealed (sylvie trudel - monette)
TRANSCRIPT
REQUIREMENTS ENGINEERING QUALITY REVEALED THROUGH FUNCTIONAL SIZEMEASUREMENT:
AN EMPIRICAL STUDY IN AN AGILE CONTEXT
Jean-François Dumas-Monette
Dept. of Computer ScienceSylvie Trudel
Dept. of Computer Science
2© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
“Every line is the perfect length if you don't measure it.” ― Marty Rubin
3
Content Functional Size Measurement : COSMIC Method Agile : Approach and Requirements Engineering Case Study Design Case Study Description Measurement and Observation Conduct Results and Other Findings Recommendations and conclusion
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
4
Functional Size Measurement : COSMIC Method
2nd generation of functional sizing
Cofounded in Canada
Japan’s national standard since 2003
Translated or being translated in 12 languages
Free measurement guide (v3.0) http://www.cosmicon.com
Measurement unit : CFP (Cosmic Function Point)
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
5
Software to measure
Boun
dary
COSMIC : Overview
Functional process 1
Matériel de stockage
Write (W)
Read (R)
Functional process 2
Functional process n
…
Users or
IO Hardware
Entry (E)
Exit (X)
Entry (E)or
FunctionalUser
Exit (X)or
Othersystems
Data
‘Interfaces’ ‘Infrastructures’
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
6© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Agile Approach and Requirements Engineering Software development philosophy defined in the « Agile Manifesto »
4 Values and 12 Principles
Methods implementing this approach : Scrum TDD Kanban XP …
7© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Requirements Engineering in Agile Documentation in the form of « User Stories »
1 US = 1 Functionality 1 Functionality = n US
Described in 3 parts
+ List of test cases
Themes
Features
Epics
User Stories
TasksExpected hierarchy
8© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Case Study Design
WHAT WAS OUR PLAN ?
9© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Case Study Design – Research Assumptions
A1. Measurement Productivity
A2. Development Productivity
A3. Extensibility
© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM 10
Case Study Design
11© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Case Study Description
WHO IS AXON INTEGRATION AND DEVELOPMENT ?
12© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Axon’s context Center of expertise in software development, integration and
modernization Specialists in test automation Small organization : ~25 employees In operation since 1998 (16 years)
Continuous software process improvement Agile development approach : Scrum & Lean Awards and distinctions
2013 : Business Solution, Askida 2011 : Innovation, MoneyCell 2011 : Best project, MoneyCell
13
Measurement used for software process improvement
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
14
Continuous improvement
Qualitative
Quantitative
Best practice models i.e.: CMMI
Retrospectives
Project functional size Productivity rate Estimation models
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
15
Their questions:
1. What estimation model(s) can we derive from our projects ?
2. What improvements can we bring to the requirement engineering process in Agile ?
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
16© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Measurement and observation conduct
WHAT REALLY HAPPENED ?
© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Measurement and Observation Conduct
17
Potential Projects
2310
Unsuitable Projects
5Measured Projects
138
18© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Results and other findings
WHAT DID WE UNCOVER ?
19© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Results – Software development relative effort
Project # US Project Effort
(Staff-Hours)# FP Project Size
(CFP)Relative Effort
(hours/CFP)Quality Rating
D 70 4417 33 352 12,5 a
C 51 7595 33 303 25,1 a
A 440 8610 70 715 12,0 b
E 41 2289 30 129 17,7 b
B 97 5565 27 177 31,4 c
A2. Development Productivity
20© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Results – Software development relative effort
21© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Results – Measurement relative effort
Project Measurement Effort
(Staff-Hours)
Project Size
(CFP)
Relative Measurement effort
(min./CFP)
QualityRating
C 26 303 5,1 a
D 37 352 6,3 a
E 18 129 8,4 b
A 164 715 13,7 b
B 45 177 15,3 c
A1. Measurement Productivity
22© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Results – Software measurement relative effort
23© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Results – Identification of defects in the requirements
A3. Extensibility
Inadequate functional decompositionInconsistent terminology for data groups/objectsInconsistent story namingUnavailability of data modeling artefactsUser story inner inconsistencyInconsistent RE process
24© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Other findings
Measurement challenges Unavailable requirement information Unavailable key personnel
Estimation model Evaluation of potential business opportunities Comparison to expert judgement
25© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM
Recommendations and conclusion
WHAT DID WE LEARN ?
26© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Recommendations
Train functional analysts as measurers
Perform and record data modeling
Organize the software improvement projects as any other project
Include a measurer during sprint reviews and retrospectives
27© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM
Conclusion and future work
Quality of requirements Project relative effort predictor?
Quality of requirements Measurement effort predictor!
Functional sizing Functional decomposition Terminology Inconsistencies
© DUMAS-MONETTE & TRUDEL 2014 IWSM MENSURA 2014 - ROTTERDAM 28
29
Thank you !
© Dumas-Monette & Trudel 2014 IWSM MENSURA 2014 - ROTTERDAM