CEN 5076 Software Testing

Instructor: Peter Clarke

http://www.cs.fiu.edu/~clarkep/courses/Fall2005/cen5076

Fall 2005

CEN 5076 Class 1 - 08/29 2

Course Introduction

• Syllabus

• Project

• STRG

• Questions

CEN 5076 Class 1 - 08/29 3

Software Engineering

• Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software; that is, the application of engineering to software.  (5/5 points)

(Source: IEEE Std 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology)

CEN 5076 Class 1 - 08/29 4

Software Process

• S/w Specification – requirements elicitation (func. & non-func.) and analysis.

• S/w Development – systems design, detailed design (OO design), implementation.

• S/w Validation – validating system against requirements (testing).

• S/w Evolution – meet changing customer needs and error correction (maintenance).

CEN 5076 Class 1 - 08/29 5

Software Specification

Functionality of the software and constraints (non-functional requirements) on its operation must be defined.

Involves:Requirements elicitation• the client and developers define the purpose of the

system. • Output - description of the system in terms of actors

and uses cases.• Actors include roles such as end users and other

computers the system needs.

CEN 5076 Class 1 - 08/29 6

Software Specification cont

• Uses cases are general sequences of events that describe all possible actions between actor and the system for a given piece of functionality.

Analysis• Objective: produce a model of the system that is

correct, complete, consistent, unambiguous, realistic, and verifiable.

• Developers transform the use cases into an object model that completely describes the system.

• Model is checked for ambiguities and inconsistencies. • Output: Object model annotated with attributes,

operations, and associations.

CEN 5076 Class 1 - 08/29 7

Software Development

Software to meet the specification is produced.

System Design• Goals of the project are defined.• System decomposed into smaller subsystems

(architectural model).• Strategies to build system identified e.g., h/w and s/w

platform, data management, control flow, and security.• Output: model describing subsystem decomposition

and system strategies.

CEN 5076 Class 1 - 08/29 8

Software Development cont

Object Design• Bridges the gap between analysis model and the

strategies identified in the system design.Includes:– describing object and subsystem interfaces,– selecting off-the-shelf components,– Restructure object model to attain design goals e.g.,

extensibility, understandability, and required performance.

• Output: detailed object model annotated with constraints and supporting documentation.

CEN 5076 Class 1 - 08/29 9

Software Development cont

Implementation• Translation of the object model into source code.• No general process followed.• There are tools to assists the programmer such as

CASE tools.

CEN 5076 Class 1 - 08/29 10

Software Validation (or V&V)

Ensures the software does what the customer want, i.e., that the software conforms to its specification and meets the expectations of the customer.

Validation: ‘Are we building the right product?’Ensures the software meets the expectations of the

customer.

Verification: ‘Are we building the product right?’Ensures the software conforms to the specification.

CEN 5076 Class 1 - 08/29 11

Software Validation (V&V) cont• Techniques

1. Software inspections (static): analyze and check system representations e.g., requirements documents, design diagrams, and program source code.

2. Software testing (dynamic): executing an implementation of the software with test data and examining the outputs against expected results.

• V&V process establishes the existence of defects.

• Debugging is a process that locates and corrects these defects.

CEN 5076 Class 1 - 08/29 12

Software Evolution

Software must evolve to meet the customer needs.

Software maintenance is the process of changing a system after it has been delivered. Reasons for maintenance:– to repair faults,– to adapt the software to a different operating

environment, and– to add to or modify system’s functionality.

CEN 5076 Class 1 - 08/29 13

Attributes of Good Software

• Maintainability– Ease of changing the software to meets the changing

needs of the customer.

• Dependability– Reliability, security and safety.

• Efficiency– Responsiveness, processing time, and memory

usage.

• Usability– Appropriate user interface and adequate

documentation.

CEN 5076 Class 1 - 08/29 14

Software Process Models

• Waterfall

• V-Model

• Evolutionary

• Boehm’s Spiral Model

• Incremental

• Unified Software Development Process Model (USDP)

CEN 5076 Class 1 - 08/29 15

Waterfall Model (Royse 1970)

Requirements Definition

System and software design

Implementation and unit testing

Integration and system testing

Operation and maintenance

CEN 5076 Class 1 - 08/29 16

V-Model (Jensen & Tonies 1979)

Requirements Specification

System design

Detailed Design

Implementation

Unit Test

System and integration test

Acceptancetest

Horizontal lines denoteThe information flow between activities at the same abstraction level.

CEN 5076 Class 1 - 08/29 17

Evolutionary Model

Development

Specification

Validation

Concurrentactivities

Intermediate Versions

Intermediate Versions

OutlineDescription

OutlineDescription

InitialVersion

InitialVersion

FinalVersion

FinalVersion

CEN 5076 Class 1 - 08/29 18

Boehm’s Spiral Model (Boehm 1987)

Requirementsplan

Concept ofoperation

Riskanalysis

Prototype1

Prototype2

Prototype3

Riskanalysis

Riskanalysis

S/wReqs.

Reqs.Validation

DevelopmentPlan

IntegrationPlan

DesignValidation

AcceptanceTest

Sys.ProductDesign

Integration & Test

Unit Test

Code

DetailedDesign

Design objectives,alternatives, & constraints

Plan next phase

Evaluate alternatives,identify & resolve risks

Develop & verifynext level product

Not shown in detail

CEN 5076 Class 1 - 08/29 19

Incremental Development (Mills et al. 1980)

Define outlinerequirements

Assign requirementsto increments

Design systemarchitecture

Develop systemincrement

Validateincrement

Integrateincrement

Validatesystem

System incomplete

Finalsystem

CEN 5076 Class 1 - 08/29 20

Unified Software Development Process

Analysis model

Use case model

Design model

Deployment model

Implementation model

Test model

specified by

realized by

distributed by

verified by

implemented by

Process is use case driven!

Requirementscaptured as a set of use cases.

All models are relatedthrough traceability dependencies.

System Development

CEN 5076 Class 1 - 08/29 21

View of UDSP from a Testing Perspective

Analysis model

Use case model

Design model

Deployment model

Implementation model

Test model

specified by

realized by

distributed by

Verified and/orvalidate

implemented by

Process is use case driven!

Requirementscaptured as a set of use cases.

All models are relatedthrough traceability dependencies.

System Development

CEN 5076 Class 1 - 08/29 22

What is software testing?

• Software testing is the process of operating software under specified conditions, observing or recording the results and making an evaluation of some aspect of the software.

(IEEE/ANSI std 610.12-1990)

CEN 5076 Class 1 - 08/29 23

What is software testing?

• Software testing is the process of uncovering evidence of defects in software systems.A defect can be introduced during any phase of development or maintenance and results from one or more “bugs” – mistakes, misunderstandings, omissions or even misguided intent on the part of the developers.

(McGregor & Sykes 2001)

CEN 5076 Class 1 - 08/29 24

Testing Terminology

The following defns are taken from Binder 2000 and McGregor & Sykes 2001.

• Failure – is the manifested inability of a system or component to perform a required function within specified limits e.g. abnormal termination, or unmet time and space constraints of the software.

• A fault is missing or incorrect code.• An error is a human action that produces a fault.

CEN 5076 Class 1 - 08/29 25

Testing Terminology cont

• A bug is an error or a fault.• Debugging is the process of tracking down the

source of failures (errors and/or faults) and making repairs.

• A test case specifies the pretest state of the component under test (CUT) and its environment, the test inputs and conditions, and the expected results.

• A test point is a specific value for test case input state variables.

CEN 5076 Class 1 - 08/29 26

Testing Terminology cont

• A test suite is a collection of test cases, typically related to some testing goal or criterion.

• A test driver is a class or utility program that applies test cases to a CUT.

• A stub is a partial, temporary implementation of a component.

• A test harness is a system of test drivers and other tools to support test execution.

CEN 5076 Class 1 - 08/29 27

Testing Approach (McGregor & Sykes)

• Approach– Analyze a little – Design a little– Code a little– Test what you can

• Kinds of testing– Model testing– Class testing, replaces unit testing– Interaction testing, replaces integration testing

CEN 5076 Class 1 - 08/29 28

Testing Approach (McGregor & Sykes)

• Kinds of testing cont.– System (and subsystem) testing– Acceptance testing– Deployment/self testing

• Motto: Test early, Test often, Test enough