a gqm plan for the evaluation of the trustworthiness of open-source software

A GQM plan for the evaluation of the trustworthiness A GQM plan for the evaluation of the trustworthiness of open-source softwareof open-source software

Michele Chinosi, Vieri Del Bianco, Luigi Lavazza, and Michele Chinosi, Vieri Del Bianco, Luigi Lavazza, and Davide TaibiDavide Taibi

Università degli Studi dell’InsubriaUniversità degli Studi dell’InsubriaDipartimento di Informatica e ComunicazioneDipartimento di Informatica e Comunicazione

1st International Workshop on Trust in Open Source Software - Limerick, Ireland, June 14, 20071st International Workshop on Trust in Open Source Software - Limerick, Ireland, June 14, 2007 - - 22 - -

Qualipso objectivesQualipso objectives

Software adoption/rejection depends on trustworthiness

Trustworthiness of OSS products regarded by some as not as guaranteed

viewed as more difficult to assess

industrial organizations interested to assess the trustworthiness assess the trustworthiness

To help foster the adoption, use, and production of OSS products, it is therefore important that the real goals and needs of software organizations be given top priority when investigating assessment methods, techniques, and indicators for OSS products.


Qualipso approachQualipso approach

QualiPSo project trustworthiness

three-step approach:

1. Information about the corporate goals of the European software industry with respect to open source trustworthiness was acquired via several interviews.

The variety of goals and needs of different organizations emerged.

2. The factors and measures for trustworthiness that are relevant in the context of those needs and goals are defined

the Goal/Question/Metric paradigm is used.

3. The metrics defined will be validated by means of measures on the field and by carrying out studies on several real-life projects.


The G/Q/MThe G/Q/M

Goal(object, purpose, quality, viewpoint, environment)

definition

Q1 Q2 Q3 Q4

Implicitmodel

M1 M2 M3 ...

interpretation


The role of OSS in SW organizationsThe role of OSS in SW organizations

An organization perceives the trustworthiness of the OSS on the basis of the role that the OSS plays with respect to the organization itself.

OSS producer of OSS

User

Customizer or value adder

etc.

Since the role of OSS is fundamental to determine the trustworthiness criteria, we define a GQM plan for each relevant case of OSS role.

The identification of the different types of usage of OSS in software organizations is still ongoing

Since the analysis of the information collected buy means of questionnaires is not yet completed

The GQM goals presented here must be regarded as a preliminary hypothesis of work.


The GQM goalsThe GQM goals

Goal 1: Analyse OSS for the purpose of evaluating the trustworthiness from the point of view of users (people who use the SW as is).

Goal 2. Analyse OSS for the purpose of evaluating the trustworthiness from the point of view of developers (e.g., organizations involved in development, organizations whose business involves OSS development)

Goal 3. Analyse OSS for the purpose of evaluating the trustworthiness from the point of view of integrators (e.g., organizations that integrate SW into their products, especially if modifications are required).

Goal 4. Analyse the OSS development process for the purpose of evaluating the relation between the trustworthiness of OSS and the characteristics of the process.

Goal 5. Analyse the OSS usage process for the purpose of evaluating the relation between the (perception of) trustworthiness of OSS and the characteristics of the process.


Goal 1Goal 1

Analyse OSS for the purpose of evaluating the trustworthiness from the point of view of users (people who use the SW as is).

Quality foci:Software quality

– direct evaluation of functional and non-functional aspects– direct or indirect evaluation of internal qualities, e.g., code qualities– functional and non functional qualities evaluated indirectly, e.g., considering

the reputation of developers, the opinions of the users community, etc.– indirect evaluation, through evaluations, certifications and assessments

made by an external organizationEconomic factorsAvailability and quality of support, guidance, documentation, etc.Availability and quality of SW maintenance in the short term Perspective of support in the futurePerspective of SW maintenance in the futurePerspective of SW evolution in the future

Variation factors:Problem domain Software criticality


Goal 2.Goal 2.

Analyse OSS for the purpose of evaluating the trustworthiness from the Point of view of developers

Trustworthiness is perceived as the quality that the organization wants to build in the OSS to be released.

If development is carried out cooperatively Trustworthiness affected by external factors

Quality foci:

Software qualitydirect evaluation of functional aspects

direct evaluation of non-functional aspects

direct evaluation of internal qualities

Perspective of support in the future

Customer related issues

Variation factors:

Economic factors

Business model


Goal 3. Goal 3.

Analyse OSS for the purpose of evaluating the trustworthiness from the point of view of integrators

Combine the points of view of the first two goals.

Quality foci :

How the internal code quality affects the trustworthiness of the final product.

Economic factors

Legal (mainly license-related) issues.

Availability of support, guidance, documentation...

Availability of technical support, guidance, documentation...

Availability of SW maintenance in the short term

Perspective of support in the future by other organizations involved in the development

Perspective of SW maintenance in the future

Issues related to customers


Goal 4Goal 4. .

Analyse the OSS development process for the purpose of evaluating the relation between the trustworthiness of OSS and the characteristics of the process.

Quality foci:

Trustworthiness of products (as defined in previous goals)

Maturity of the software development process, includingdevelopment release planning

creation and maintenance of product documentation

process modelling

Etc...

Qualification and experience of developers

Quality assurance practices used in the development process

Development process retrospectives

Variation factors:

Business model of development


Goal 5Goal 5. .

Analyse the OSS usage process for the purpose of evaluating the relation between the perception of trustworthiness of OSS and the characteristics of the process.Quality foci:

Trustworthiness of products Adaptations Extensions Functionality usedUser interfaces used

Variation factors:Type and experience of the userType of usageEffort and durationMethodology and tools employedStatus of the OSS projectPlatform and language


Future workFuture work

Refine the definition of goals according to classes of use.

Each class should be characterized by a unique combination of values of the following attributes:

OSS is used only within the organization [yes|no (it is released outdside)]

OSS is modified [yes|no (it is used ‘as is’)]

OSS is integrated with other SW [yes|no]

Additionally, evaluation criteria should be taken into consideration:

E.g., ethical vs. commercial


AcknowledgmentsAcknowledgments

The research presented was partially funded by the Qualipso IST project , sponsored by the EU in the 6th FP (IST-034763).

http://www.qualipso.eu

a gqm plan for the evaluation of the trustworthiness of open-source software

Economy & Finance