technical and empirical software engineering: bridging the chasm
DESCRIPTION
Joost Schalken. Johan Hoorn. Henk Koning. Hans van Vliet. Technical and Empirical Software Engineering: Bridging the Chasm. IMSE meeting. Outline. Problem background Why the chasm between technicians and empiricists? Towards a common goal Towards a common vocabulary Conclusions. - PowerPoint PPT PresentationTRANSCRIPT
Technical and Empirical Software Engineering:
Bridging the ChasmIMSE meeting
Hans van Vliet
Henk Koning
Joost Schalken
Johan Hoorn
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
2
Outline• Problem background• Why the chasm between technicians and
empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
3
Outline• Problem background• Why the chasm between technicians and
empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
4
Problem background
IntroductionAt typical IMSE meetings a presentation…• explains details of a single research project• no relations with other research are discussed• no clarification is give how the project helps
advance computer science or software engineering (practice) as a whole
Within Information Management and Software Engineering:
• very diverse research topics (see next slide…)• not a lot of cross-disciplinary research
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
5
Problem background
IMSE Staff Research Statements• Software asset
management• Software architecture• Software metrics• Program transformation • Interaction Design• Requirements
Engineering
• Object-oriented software development
• Mental models of users• Distributed logic
programming• Intelligent multimedia• Logical disks
Information Management and Software Engineering Section: Scientific Staffurl: http://www.cs.vu.nl/imse/staff-en.html (accessed 9th June, 2004)
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
6
Problem background
Wouldn’t it be great if we could…• get inspiration for research from our colleagues?• better understand the research from colleagues?• see how all research projects combined help
advance science and help IT practitioners?• prevent reinventing the wheel twice, but instead
rely on the existing knowledge of our colleagues?• perform research together with our colleagues,
finding new niches on the borders of our disciplines?
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
7
Problem background
Potential improvementsPotential for improvement:• more co-operation• more idea sharing• more communication between sub-disciplines
– bridge the chasm between engineering oriented andempirically oriented researchers
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
8
Problem background
Examples of Joined Research• Does program transformation help increase the
productivity of software maintenance?• How does interface design affect software
architecture?• How can we best reverse engineer the
architecture of a system to improve human understanding?
• What is the impact of object orientation on program transformation?
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
9
Outline• Problem background• Why the chasm between technicians and
empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
10
My hypothesis
Language barriers cause other barriersThe chasm between technologists & empiricists
caused by:• different vocabularies/languages• different research methods
• lack of clarity of how we work towards a common goal
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
11
My hypothesis
Improved co-operation by better understandingWe hope to bridge the chasm between the
researchers by:• identifying our common goals• mutual understanding of our research methods
– explaining the differences between technological andempirical research
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
12
Outline• Problem background• Why the chasm between technicians and
empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
13
Towards common goals
Difference between Engineering and Science• “Scientists extend our knowledge of the laws of
nature while engineers apply those laws of nature to build useful artefacts, under a number of constraints.”
A. Abran & J.W. Moore (eds.), SWEBOK: Guide to the Software Engineering Body of Knowledge.IEEE Press, 2004.
• Scientists want to understand the world.• Engineers want to understand how to improve the
world.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
14
Towards common goals
What sets up apart from Computer ScienceSoftware Engineering vs. Computer Science:• [For software engineering] “the content is
markedly different from Computer Science.Just as electrical engineering is based upon the science of physics, software engineering should be based, among others, upon computer science.In both cases, though, the emphasis is necessarily different.”
A. Abran & J.W. Moore (eds.), SWEBOK: Guide to the Software Engineering Body of Knowledge.IEEE Press, 2004.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
15
Towards common goals Definition of Software Engineering (I)• The application of a systematic, disciplined,
quantifiable approach to the development, operation, and maintenance of software;that is, the application of engineering to software.
Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries(IEEE Std 610-1990). IEEE Press, 1990.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
16
Towards common goals Definition of Software Engineering (II)• “Software engineering is often used to describe
the activities, methods and tools of large scale software development.”
C. Lewerentz & H. Rust, Are software engineers true engineers?.In: Annals of Software Engineering, vol. 10, 2000, pp. 311-328.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
17
Towards common goals Definition of Software Engineering ResearchDiscover, improve and evaluate…• methods, and• tools
… to help practitioners develop large scale software.
Software Engineering Research entails both• discovery of new/enhanced methods and tools• evaluation of methods and tools
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
18
Outline• Problem background• Why the chasm between technicians and
empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
19
Towards a common vocabulary
Engineering and Empirical Research Questions• Engineers improve solutions to problems in the
real worldTheir core (design) problem is an action problem.– knowledge is needed as far as it contributes to solving
problems and understanding the solutions.
• Empirical scientists advance understanding of real world phenomena.Their core problem is a knowledge problem.– all knowledge is sought for, regardless of its applicability.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
20
Towards a common vocabulary
Typical Action and Knowledge Problems• Engineering research questions (action problems)
– Can we construct a new tool U?– Can we invent a new method X?– Can we improve method Y?– Can we improve tool Z?
• Empirical research questions (knowledge problems)– How much does the efficiency increase when tool X
instead of tool Y is used?– Why does method Y help reduce requirements ambiguity?– Under what circumstances does design X improve
modifiability?
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
21
Towards a common vocabulary Different Questions different Paradigms• Type of problem (action/knowledge) has influence
on the research methods used.• Empirical scientists typically put more emphasis
on research methodology up front.• Engineers typically explain their design or solution
using a framework or design methodology afterwards.
D.L. Parnas & P.C. Clements, A Rational Design Process, How and Why to Fake It.In: IEEE Transactions on Software Engineering, vol 12(2) feb. 1986.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
22
Towards a common vocabulary Engineering Methodology (I): the Action Cycle
C. Mitcham, Thinking Through Technology: The Path Between Engineering and Philosophy. University of Chicago Press, 1994.
actionaction plan torealize a goal
change in situation: effect
reason causes
control
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
23
Towards a common vocabulary Engineering Methodology (II): OKT
T.J. Plomp, Onderwijskundige technologie: Enige Verkenningen.Enschede: Universiteit Twente, 1982.
Educational technological model(in Dutch: OnderwijsKundig Technologisch model)
pilot study
design
realisation/construction
test, evaluation, revision
implementation
implem
entation
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
24
Towards a common vocabulary
Empirical Methodology: The empirical cycle
A.D. de Groot,Methodologie:Grondslagen van onderzoek en denken in de gedragswetenschappen.‘s-Gravenhage: Mouton, 1961.
observation
induction
deduction
testing
evaluation
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
25
Towards a common vocabulary
Empirical Methodology: Some remarks (I)With empirical research attention is paid to:• Exact statement of research goals / hypothesis• Design of the experiments
– which subjects receive which treatments?
• Operationalization– how do you measure the attributes
• Data collection
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
26
Towards a common vocabulary
Empirical Methodology: Some remarks (II)With empirical research attention is paid to (continued):• Analysis of data
– quantitative/statistical methods– qualitative methods
• Validity of the experiment– internal validity– construct validity– external validity– statistical validity
T.D. Cook & D.T. Campbell, Quasi-Experimentation: Design & Analysis Issues for Field Settings.Chicago: Rand McNally College Publishing Company, July 1979.
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
27
Outline• Problem background• Why the chasm between technicians and
empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
28
Working together makes sense!• Without new methods and tools from engineering
research no empirical software engineering research!– What could be investigated?
• Without empirical software engineering research the usefulness of new methods and tools is unclear!– Why would practitioners trust your tool or method that
has only been validated on trivial problems?
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
29
Conclusions• Co-operation can be beneficial
– interesting research areas at the borders of sub-disciplines
– certain types of research follow others types of research
• Co-operation only possible through mutual understanding
21-04-23
Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)
30
And now its your turn…• Clarifications• Questions• Comments• Discussion• …