@andivogelsang is re research relevant for practitioners? · • low perceived relevance can, in...
TRANSCRIPT
Is RE Research Relevant for Practitioners?
First Results from the RE-pract Study
GI FG RE Treffen
Andreas Vogelsang, Daniel Méndez, Xavier Franch
30.11.2018
RE Research 2010 – 2016
International RequirementsEngineering Conference (RE)
International Conference onSoftware Engineering (ICSE)
Intl. Working Conference on RE: Foundation for Software Quality (REFSQ)
International Symposium on Empirical Software Engineering and Measurement (ESEM)
435papers on RE
22
113
39
245
Foundations of Software Engineering (ESEC/FSE)
16
What about RE?
The RE-pract Project
435papers on RE
4351-sentence summaries
154participants
15summaries
2,164ratings
“A method for using defect taxonomies in order to improve requirements reviews and testing.”“A [method] for [using defect taxonomies] in order to [improve requirements reviews and testing].”
“A [literature survey] on [requirements elicitation techniques and a roadmap of research] in order to [improve the elicitation of tacit knowledge].”
Essen-tial
Worth-while
Unim-portant
Un-wise
Don’tknow
The Study Participants
The Study Participants
The Study Participants
First Results: Overview
Rating RE SE ESEM
Essential 24.2% 20.0% 17.6%
Worthwhile 46.3% 51.0% 48.8%
Unimportant 25.3% 25.6% 29.2%
Unwise 4.2% 3.4% 4.6%
Results: Academic vs. Practitioner-Driven Research
Results: The Value of Professional Evaluations
Results: Relevant Content – Documentation
Results: Relevant Content – People
Results: Relevant Content – Quality
Results: Relevant Content – Phase
Results: Relevant Content – Process
Conclusions and Next Steps
• If you aim for research that is perceived as relevant• Aim for evaluation with professional practitioners • Aim for the industry track• Aim for quality assurance, shared understanding, V&V
• But… • Perceived relevance != relevance• Correlation != causation (the above recipe is no guarantee for relevance)• Short-term relevance = long-term relevance?
(Our scope was 6 years)• Context (domain, company size, type of systems) may have a strong impact• Low perceived relevance can, in fact, indicate a need for research
• To do• Analyze Inter-Rater Agreement (controversial vs. non-controversial research)• Analyze impact of context factors• Analyze development over time• Correlate relevance and citations• Identify needed research topics
