Software testing and development for intended

quality

Tero Pesonen

Objective

To show how software development, software testing and intended quality depend on one another.

The research area includes, in addition to traditional software development and service models, also the emerging XaaS (Everything as a Service) architectures, technologies and service models.

The project results help the participating companies in improving the efficiency of their quality management and software testing and hence the efficiency of their software development as a whole. Testing techniques Testing as a service

Research Problem

Assessment framework through which each OU is evaluated

Intended Software Quality

ISO/IEC 25010, Software Product Quality

Software Testing

ISO/IEC 29119, Testing Spice, IEEE Std 1012 StaaS

Software Development

• Software Products

ISO/IEC 12207ISO/IEC 15504• New

ServicesSaaS

Preliminary Findings

In the software developing OU, requirements management plays an important role in achieving intended quality. Requirements act as the basic communication

medium through which the customer/user, developers and testers concretize different aspects of quality.

Requirements are never throughout, and different stakeholders interpret them differently. Requirements change.

Successful requirements management contributes positively to achieving the targeted quality by reducing overhead caused by doing wrong work.

Preliminary Findings

Building ”good” quality is different to building targeted type and level of quality as efficiently as possible.

For many organizations, concretizizing intended quality attributes (also non-functional) is not yet as well implemented as actual development and testing activities are.

Many organizations are also developer centric, and here QA emphasizes functional quality.

Intended quality appears therefore to be also a knowledge transfer issue within the development process, not just a testing or software development challenge, as often perceived.

Preliminary Findings

Agile methods favour direct, short development cycles that produce immediate feedback on quality for testers, developers and users. Agility establishes an efficient communication channel. Non-agile methods seem to require stricter, more

formal quality management approaches for similar level of success.

Large, complex, multistakeholder projects do not benefit from agility to the same extend, as the communication overhead is rather increased than reduced in such a context. Well-defined waterfall-like processes may fare better.

Observations from the data

Developers and testers need to be able to communicate directly, face-to-face, without effort; physical co-location is seen very important for mutual productivity.

Developers should not review their own code; one becomes blind to one’s own mistakes. That time is better spent reviewing a colleague’s work.

Requirements management solutions (e.g. Shared management centers) seem a good starting point in improving the efficiency of the overall development process. Note: How is non-functional quality communicated in the solution?

The same organization may need to apply different processes to different types of development work, not just have all development governed by one procedure. Rapid, explorative development work benefits from agility. Maintenance work can be more strictly phased and documented.

Observations from the data

In projects were testing is done as a service, it is paramount that no communication obstacles are created between testers, managers and developers, despite them being part of different organizations. Knowledge transfer between stakeholders should be considred

already in project planning. Develoeprs and testers should be communicated the formal

aspects and be inquired for feedback; they observe, notice and draw conclusions – and frustrate when things do not work.

Cloud architectures support very quick development for targeted quality with module re-use and well-defined API’s But force vendor lock-in. Offer poor portability to non-cloud platforms if software grows.

Publication 1 / STX

Research question: How do developing organizations build intended quality? Sub questions: Do different size and type of OU’s build quality differently? Do cloud environments affect how quality is built? What is the role of testing methods and tools for quality?

Conference paper.Contents:1. Introduction2. Related research3. Research process

Qualitative, Grounded Theory method. 18 STX interviews + 10 interviews of supplementary data. xx sheets of transcripted data.

Publication 1 / STX

4. Results4.1 Quality is communicated as requirements, which are translated

into architecture and then into software code. Testing ensures the requirements are, first, correct, and, second, being addressed correctly.

4.2 For most organizations, the potential in improved requirements management seems currently greater than that provided by better testing techniques or other kinds of testing tools.

4.3 Process improvement seems to follow improved requirements management.

4.4 Service-oriented OU’s view quality less formally than product-oriented ones. Knowledge transfer plays a role in organization’s growth, and has to be reckoned with.

4.5 Cloud environments facilitate certain kinds of development work and aid in achieving targeted quality efficiently, but not without major compromises.

5. Discussion and Summary.

Research: Next Steps

Assessment of testing On-going work:

Master’s Thesis by Aoke Li on the status of standardization in China.

Due to February 2012. A new Master of Science Thesis study for STX

will be started this spring. Research topics:

Software product lines (ISO/IEC 2655x.) Distributed global software development. Cloud laboratory (external project with China.) Quantitative survey.