1

Software Testing and Quality Assurance

Lecture 15 - Planning for Testing (Chapter 3, A Practical Guide to Testing Object-

Oriented Software)

2

Lecture Outline Adequacy of test cases. Testing Roles Testing Planning

3

A testing process: dimensions of software testing (cont...)

How much testing is adequate? No testing, exhaustive testing (consider the

expected lifetime of the software), is the software life-critical,

Coverage: is a measure of how completely a test suite exercises the capabilities of a piece of software. For example,

If every LOC executed at least once, the number of requirement that is checked by the test suite.

4

A testing process: adequacy of test cases

Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to.

5

A testing process: adequacy of test cases Adequacy can be measured based on

coverage: Coverage based on the requirements:

based on what the software is supposed to do how many of the requirements called out in the

specification are tested Coverage based on the code: based on how

the software actually works ― how much of the software was executed as a

result of running test suites

6

A testing process: adequacy of test cases (cont...)

Functional testing (specification-based or black box testing): test cases are constructed based solely on the

software’s specification and not on how the software is implemented.

Structural testing (implementation-based or white box testing): test cases are constructed based on the code

that implements the software.

7

A testing process: adequacy of test cases (cont...)

Rule of Thumb Use some combination of both

approaches.

8

A testing process: adequacy of test cases― risk analysis (cont...)

Risk analysis in the testing process is applied to determine the level of details and amount of time to dedicate to testing a component.

A reasonable scale of increasing risk for components is: Prototype components Production components Library components

Ris

k in

crea

ses

9

Roles in the testing process One or more people can assume each

role. One person can assume multiple roles. Roles must be separate: Class tester: test individual classes as

they are programmed. Integration tester: test a set of objects

that are being brought together from different development resources.

10

Roles in the testing process System tester: responsible for

independently verifying that the completed application satisfies the system requirements.

Test manager: managing the test process.

11

Planning activities: scheduling testing activities Class tests:

Scheduled based on the developer’s judgment as they become useful or necessary.

Useful during coding Necessary when adding a component to

the code base.

12

Planning activities: scheduling testing activities

Integration tests: Scheduled as specific intervals, usually at

the end of major iteration that signal the completion of an increment

System tests: Performed on major deliverables at

specified intervals throughout the project, usually specified in the project plan.

13

Planning activities: estimation Estimation: estimating resources ―cost,

time and personnel Factors should be considered for

estimation: Level of coverage: estimate the amount of

effort for one use case then you can construct the estimate for the whole system

Domain type: logic intensive vs. data intensive.

14

Planning activities: estimation Equipment required: environment as

close as possible to the deployment environment. Costs of equipment for simulation must also be considered.

Organizational model: buddy approach (two developers swap code with each other and test) or different approach.

Testing effort estimate: effort for testing all classes: total number of classes times the effort per class.

15

Planning activities: document templates

Project test plan Component test plan (one per important

component/class). Use case test plan (one per use case) Integration test plan (one per increment) System test plan (only one)

16

Planning activities: document templates— Project test plan

The purpose of project test plan is to summarize the testing strategy that is to be employed for the project: The steps in the development process at

which testing will occur The frequency of testing with which the

testing should occur Who is responsible for the activity

17

Planning activities: document templates— Project test plan

The project test plan may be an independent document or it may be included in either the overall

project plan or the project’s quality assurance plan.

18

Planning activities: document templates— Component test plan

The purpose of the component test plan is to define the overall strategy and specific test cases that will be used to test a certain component.

Two types of guiding information: Project criteria: standards that have been

agreed upon as to how thoroughly each component will be tested (e.g. 100% of the critical components will be tested)

19

Planning activities: document templates— Component test plan

Project procedure: identify techniques that have been agreed upon as the best way to handle a particular task (e.g. constructing a PACT class for each component that will be tested)

20

Planning activities: document templates— Use case test plan

The purpose of the use case test plan is to describe the system-level tests to be derived from a single use case

Three levels of use cases: High level: abstract use cases that are the basis

for being extended to end-to-end system level use case

End-to-end system

21

Planning activities: document templates— Use case test plan

Functional sub-use cases: aggregated into end-to-end system-level use cases.

Types of use cases: Functionality use cases: modified the

data maintained by the system in some way

Report use cases: accessed information, summarized it, and format it for the user.

22

Planning activities: document templates— Integration test plan

Integration test plan is important in an iterative development environment.

Specific sets of functionality will be delivered before others.

Out of these increments the full system slowly emerges.

23

Planning activities: document templates— Integration test plan

Since small, localized behavior should have already been tested. Integration tests should be more complex and more comprehensive than the typical component tests.

Individual test plans are combined to form the integration test plan for a specific increment.

Integration test depends on other test cases, so no template is provided.

24

Planning activities: document templates— System test plan

The system test plan is a document that summarizes the individual use case test plans and provides information on additional types of testing that will be conducted at the system level.

25

Key points Adequacy of test cases: test the software enough to

be reasonably sure that the software works as it is supposed to.

Roles in the testing process: class tester, integration tester, system tester, and test manager.

Factors should be considered for estimation: level of coverage, domain type, equipment required, organizational model, and testing effort estimate.

Document templates