Cairo UniversityFaculty of Computers and Information

CS251 – Software Engineering

Lecture 20: SW Testing

Presented by:

Mohammad El-Ramly, PhD

http://www.acadox.com/join/75UDWT

2

Outline

Definition of Software Testing Testing as a Quality Assurance Technique Testing Terminology Levels and Types of Testing

3

4

What is a computer program?

5

What is a house?

6

Any better definition?

The focus is on•Stakeholders

– (for people)•Purpose

– (to live in)

Stakeholder:Any person affected by:•success or failure of a project, or •actions or inactions of a product, or•effects of a service.

7

Apply this definition to prog.

8

Apply this definition to prog.

A computer program is• a communication• among several humans and

computers• who are distributed over space

and time,• that contains instructions that

can be executed by a computer.

9

Software Quality

10

Software Error (Bug)

11

Software Error (Bug)

An attribute of a software product that reduces its value to a favored

stakeholder or increases its value to a disfavored

stakeholderwithout a sufficiently large countervailing

benefit.

An error may or may not be a coding error, or a functional error. Design errors are bugs too.

12

Software Testing

• is an empirical• technical • investigation• conducted to provide stakeholders• with information • about the quality• of the product or service under test

13

Software Testing

14

Software Testing is Search for Information

1. Find important bugs

2. Assess the quality of the product

3. Help managers assess the progress of the

project

4. Help managers make release decisions

5. Block premature product releases

6. Help predict and control product support costs

7. Check interoperability with other products

15

Software Testing is Search for Information

8. Assess conformance to specifications

9. Certify the product meets a particular

standard

10.Ensure the testing process meets

accountability standards

11.Minimize the risk of safety-related lawsuits

12.Help clients improve their processes

13.Evaluate the product for a third party

16

Fact of testing

Testing does not guarantee

the absence of defects

it gurantees their presence

17

18

Assume we want to test Font Increase functionality in Word

CalibriFont size

from 7 to 13.5

19

Assume we want to test Font Increase functionality in Word

ArialFont size

from 7 to 13.5

20

Assume we want to test Font Increase functionality in Word

Times New Roman

Font size from 7 to 13.5

21

Assume we want to test Font Increase functionality in Word

Courier NewFont size from 7 to 13.5

22

23

So, what should we test?

24

So, what should we test?

What to cover? Every font size (to a tenth of a point). Every character in every font. Every method of changing font size. Every user interface element associated with font size

functions. Interactions between font sizes and other contents of a

document. Interactions between font sizes and every other feature

in Word. Interactions between font sizes and graphics cards &

modes. Print vs. screen display.

A program can fail in many ways

System under

test

Program state

Intended inputs

System state

Configuration andsystem resources

From other cooperating processes, clients or servers

Monitored outputs

Program state, including uninspected outputs

System state

Impacts on connected devices / system resources

To other cooperating processes, clients or servers

Unit testing with JUnit 26

27

28

Why?

Why testing? Improve software quality Increase customer satisfaction and reduce cost Finish faster and on budget Catch errors and bugs

In short, to Produce Better Code

29

What should be tested ?

Test for boundary conditions Test for both success and failure Test for general functionality Test for configuration and resources Test for interoperability Etc..

30

When to start testing

Software quality and is a life-cycle process

Testing can start as early as

the project starts

Software Quality

Landscape

Plannin

g

Requir

emen

ts

Design

Develo

pmen

t

Testin

g

Guidelines,

Standards a

nd

Templates

QA Planning

and Budgetin

g

Testing

Debugging

Reviews

Refactorin

g

Requirements

V & V

Design R

eview

Collabora

tive

Development /

Reviews

Metrics

Change and Configuration Management

Bug Tracking Surveys

Deploy

ment &

Mainten

ance

M a n a g e m e n t

Contract

Review

Software Quality

Landscape

Plannin

g

Requir

emen

ts

Design

Develo

pmen

t

Testin

g

Guidelines,

Standards a

nd

Templates

QA Plannin

g

and Budgetin

g

Testing

Debugging

Reviews

Refactorin

g

Requirements

V & V

Design R

eview

Collabora

tive

Development /

Reviews

Metrics

Change and Configuration Management

Bug Tracking Surveys

Deploy

ment &

Mainten

ance

M a n a g e m e n t

Contract

Review

Software Quality

Landscape

Plannin

g

Require

men

ts

Desig

n

Develo

pmen

t

Testin

g

Guidelines,

Standards a

nd

Templates

QA Plannin

g

and Budgetin

g

Testing

Debugging

Reviews

Refactorin

g

Requirements

V & V

Design R

eview

Collabora

tive

Development /

Reviews

Metrics

Change and Configuration Management

Bug Tracking Surveys

Deploy

ment &

Mainten

ance

M a n a g e m e n t

Contract

Review

04/22/23 CPSC-4360-01, CPSC-5360-01, Lecture 11

34

Testing Organization (V-Model)

Produce ValidateRequirements specification

System Specification

System Design

Module Design

Coding

UnitTesting

IntegrationTesting

System Testing

Acceptance Testing

UserSpecification

SystemSpecification

SoftwareSpecification

ModuleSpecification

Produce Validate

Produce Validate

Produce Validate

Testing Terminology

What is test case

36

A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly

1

37

Good test case design

An good test case satisfies the following criteria: Reasonable probability of catching an error Does interesting things Doesn’t do unnecessary things Neither too simple nor too complex Not redundant with other tests Makes failures obvious Mutually Exclusive, Collectively Exhaustive

Unit testing with JUnit 38

Test case design technique

Test case design techniques can be broadly split into two main categories

Black box (functional)

White box (structural)

2

3

39

Black Box tests

Targeted at the apparent simplicity of the software Makes no assumptions about implementation Good for testing component interactions

Tests the interfaces and behavior

Input Output

2

40

White Box tests

Targeted at the underlying complexity of the software Intimate knowledge of implementation Good for testing individual functions

Tests the implementation and design

Input Output

3

Black box testing: Test against the specifications of the component Tests the interface of the component.

White box testing: Test design and implementation of the

component. Test cases exercise specific sets of condition,

loops, etc.

Black box vs. White box

CPSC-4360-01, CPSC-5360-01, Lecture 11

Test to specifications: Also known as black-box, data-driven, functional, or input/output

driven testing. Ignore the code — use the specifications to select test cases.

Test to code: Also known as glass-box, logic-driven, structured, or path-

oriented testing. Ignore the specifications — use the code to select test cases.

Neither exhaustive testing to specifications nor exhaustive testing to code is feasible.

Another way to classifytest cases

43

Unit testing with JUnit 44

Test case writing example

Suppose we have two parameters we want to cover in a set of tests. Parameters are as follows..

Operating system Win98 Win2k Winxp

Printers HP 4100 HP 4200

How We should write test case for this ??

Unit testing with JUnit 45

Types (Levels) of Tests

Unit Testing Individual classes or types

Component Testing Group of related classes or

types

Integration Testing Interaction between classes

4

5

6

46

Types (Levels) of Tests

System Testing Testing to confirm that all code

modules work as specified, and that the system as a whole performs adequately on the platform on which it will be deployed.

Acceptance Testing Final testing usually involving

users to ensure their requirements are met.

7

8

Testing that Involves Users Alpha testing:

In-house testing. By a test team or end users.

Beta testing: By users or selected subset of actual customers in a normal

work environment. Product is very close to completion. Open beta release: Let public carry out the beta testing.

Acceptance testing: By users to check that system satisfies requirements to

decide whether to accept the system based on the test result.

9

10

48

49

What is a testing framework?

A test framework provides reusable test functionality which: Is easier to use (e.g. don’t have to write the same

code for each class) Is standardized and reusable Provides a base for regression tests

11

Regression testing

New code and changes to old code can affect the rest of the code base ‘Affect’ sometimes means ‘break’

We need to run tests on the old code, to verify it works – these are regression tests

Regression testing is required for a stable, maintainable code base

12

51

Why use a testing framework?

Each class must be tested when it is developed

Each class needs a regression test Regression tests need to have standard

interfaces Thus, we can build the regression test when

building the class and have a better, more stable product for less work

52

Example: Old way vs. new way int max(int a, int b) {

if (a > b) { return a; } else { return b; }}

void testMax() { int x = max(3, 7); if (x != 7) { System.out.println("max(3, 7) gives " + x); } x = max(3, -7); if (x != 3) { System.out.println("max(3, -7) gives " + x); }}

public static void main(String[] args) { new MyClass().testMax();}

@Testvoid testMax() { assertEquals(7, max(3, 7)); assertEquals(3, max(3, -7));}

53

JUnit

JUnit is a framework for writing unit tests A unit test is a test of a single class

A test case is a single test of a single method A test suite is a collection of test cases

Unit testing is particularly important when software requirements change frequently Code often has to be refactored to incorporate the

changes Unit testing helps ensure that the refactored code

continues to work

13

14

54

JUnit..

JUnit helps the programmer: Define and execute tests and test suites Formalize requirements and clarify architecture Write and debug code Integrate code and always be ready to release a

working version

55

Test suites Obviously you have to test your code to get it working in

the first place You can do ad hoc testing (testing when something happens), or You can build a test suite (set of tests that can be run at any time)

Disadvantages of writing a test suite It’s a lot of extra programming

True—but use of a good test framework can help quite a bit You don’t have time to do all that extra work

False—Experiments repeatedly show that test suites reduce debugging time more than the amount spent building the test suite

Advantages of having a test suite Your program will have many fewer bugs It will be a lot easier to maintain and modify your program