technical and empirical software engineering: bridging the chasm imse meeting hans van vliet henk...

Technical and Empirical Software Engineering:

Bridging the ChasmIMSE meeting

Hans van Vliet

Henk Koning

Joost Schalken

Johan Hoorn

18-04-23

Technical and Empirical Software Engineering: Bridging the Chasm (v0.3)

2

Outline• Problem background• Why the chasm between technicians and

empiricists?• Towards a common goal• Towards a common vocabulary• Conclusions

18-04-23


3



18-04-23


4

Problem background

IntroductionAt typical IMSE meetings a presentation…• explains details of a single research project• no relations with other research are discussed• no clarification is give how the project helps

advance computer science or software engineering (practice) as a whole

Within Information Management and Software Engineering:

• very diverse research topics (see next slide…)• not a lot of cross-disciplinary research

18-04-23


5

Problem background

IMSE Staff Research Statements• Software asset

management• Software architecture• Software metrics• Program transformation • Interaction Design• Requirements

Engineering

• Object-oriented software development

• Mental models of users• Distributed logic

programming• Intelligent multimedia• Logical disks

Information Management and Software Engineering Section: Scientific Staffurl: http://www.cs.vu.nl/imse/staff-en.html (accessed 9th June, 2004)

18-04-23


6

Problem background

Wouldn’t it be great if we could…• get inspiration for research from our colleagues?• better understand the research from colleagues?• see how all research projects combined help

advance science and help IT practitioners?• prevent reinventing the wheel twice, but instead

rely on the existing knowledge of our colleagues?• perform research together with our colleagues,

finding new niches on the borders of our disciplines?

18-04-23


7

Problem background

Potential improvementsPotential for improvement:• more co-operation• more idea sharing• more communication between sub-disciplines

– bridge the chasm between engineering oriented andempirically oriented researchers

18-04-23


8

Problem background

Examples of Joined Research• Does program transformation help increase the

productivity of software maintenance?• How does interface design affect software

architecture?• How can we best reverse engineer the

architecture of a system to improve human understanding?

• What is the impact of object orientation on program transformation?

18-04-23


9



18-04-23


10

My hypothesis

Language barriers cause other barriersThe chasm between technologists & empiricists

caused by:• different vocabularies/languages• different research methods

• lack of clarity of how we work towards a common goal

18-04-23


11

My hypothesis

Improved co-operation by better understandingWe hope to bridge the chasm between the

researchers by:• identifying our common goals• mutual understanding of our research methods

– explaining the differences between technological andempirical research

18-04-23


12



18-04-23


13

Towards common goals

Difference between Engineering and Science• “Scientists extend our knowledge of the laws of

nature while engineers apply those laws of nature to build useful artefacts, under a number of constraints.”

A. Abran & J.W. Moore (eds.), SWEBOK: Guide to the Software Engineering Body of Knowledge.IEEE Press, 2004.

• Scientists want to understand the world.• Engineers want to understand how to improve the

world.

18-04-23


14

Towards common goals

What sets up apart from Computer ScienceSoftware Engineering vs. Computer Science:• [For software engineering] “the content is

markedly different from Computer Science.Just as electrical engineering is based upon the science of physics, software engineering should be based, among others, upon computer science.In both cases, though, the emphasis is necessarily different.”

A. Abran & J.W. Moore (eds.), SWEBOK: Guide to the Software Engineering Body of Knowledge.IEEE Press, 2004.

18-04-23


15

Towards common goals Definition of Software Engineering (I)• The application of a systematic, disciplined,

quantifiable approach to the development, operation, and maintenance of software;that is, the application of engineering to software.

Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries(IEEE Std 610-1990). IEEE Press, 1990.

18-04-23


16

Towards common goals Definition of Software Engineering (II)• “Software engineering is often used to describe

the activities, methods and tools of large scale software development.”

C. Lewerentz & H. Rust, Are software engineers true engineers?.In: Annals of Software Engineering, vol. 10, 2000, pp. 311-328.

18-04-23


17

Towards common goals Definition of Software Engineering ResearchDiscover, improve and evaluate…• methods, and• tools

… to help practitioners develop large scale software.

Software Engineering Research entails both• discovery of new/enhanced methods and tools• evaluation of methods and tools

18-04-23


18



18-04-23


19

Towards a common vocabulary

Engineering and Empirical Research Questions• Engineers improve solutions to problems in the

real worldTheir core (design) problem is an action problem.– knowledge is needed as far as it contributes to solving

problems and understanding the solutions.

• Empirical scientists advance understanding of real world phenomena.Their core problem is a knowledge problem.– all knowledge is sought for, regardless of its applicability.

18-04-23


20


Typical Action and Knowledge Problems• Engineering research questions (action problems)

– Can we construct a new tool U?– Can we invent a new method X?– Can we improve method Y?– Can we improve tool Z?

• Empirical research questions (knowledge problems)– How much does the efficiency increase when tool X

instead of tool Y is used?– Why does method Y help reduce requirements ambiguity?– Under what circumstances does design X improve

modifiability?

18-04-23


21

Towards a common vocabulary Different Questions different Paradigms• Type of problem (action/knowledge) has influence

on the research methods used.• Empirical scientists typically put more emphasis

on research methodology up front.• Engineers typically explain their design or solution

using a framework or design methodology afterwards.

D.L. Parnas & P.C. Clements, A Rational Design Process, How and Why to Fake It.In: IEEE Transactions on Software Engineering, vol 12(2) feb. 1986.

18-04-23


22

Towards a common vocabulary Engineering Methodology (I): the Action Cycle

C. Mitcham, Thinking Through Technology: The Path Between Engineering and Philosophy. University of Chicago Press, 1994.

actionaction plan torealize a goal

change in situation: effect

reason causes

control

18-04-23


23

Towards a common vocabulary Engineering Methodology (II): OKT

T.J. Plomp, Onderwijskundige technologie: Enige Verkenningen.Enschede: Universiteit Twente, 1982.

Educational technological model(in Dutch: OnderwijsKundig Technologisch model)

pilot study

design

realisation/construction

test, evaluation, revision

implementation

implem

entation

18-04-23


24


Empirical Methodology: The empirical cycle

A.D. de Groot,Methodologie:Grondslagen van onderzoek en denken in de gedragswetenschappen.‘s-Gravenhage: Mouton, 1961.

observation

induction

deduction

testing

evaluation

18-04-23


25


Empirical Methodology: Some remarks (I)With empirical research attention is paid to:• Exact statement of research goals / hypothesis• Design of the experiments

– which subjects receive which treatments?

• Operationalization– how do you measure the attributes

• Data collection

18-04-23


26


Empirical Methodology: Some remarks (II)With empirical research attention is paid to (continued):• Analysis of data

– quantitative/statistical methods– qualitative methods

• Validity of the experiment– internal validity– construct validity– external validity– statistical validity

T.D. Cook & D.T. Campbell, Quasi-Experimentation: Design & Analysis Issues for Field Settings.Chicago: Rand McNally College Publishing Company, July 1979.

18-04-23


27



18-04-23


28

Working together makes sense!• Without new methods and tools from engineering

research no empirical software engineering research!– What could be investigated?

• Without empirical software engineering research the usefulness of new methods and tools is unclear!– Why would practitioners trust your tool or method that

has only been validated on trivial problems?

18-04-23


29

Conclusions• Co-operation can be beneficial

– interesting research areas at the borders of sub-disciplines

– certain types of research follow others types of research

• Co-operation only possible through mutual understanding

18-04-23


30

And now its your turn…• Clarifications• Questions• Comments• Discussion• …

technical and empirical software engineering: bridging the chasm imse meeting hans van vliet henk...

Documents

empirical software engineering

software engineering

software engineering

chasm v0

engineering oriented

disciplinary research

outline problem background

problem background examples