8/8/2019 Domain 1 Sept2005

1/33

Test Principles and Concepts

Domain 1

8/8/2019 Domain 1 Sept2005

2/33

Key Components

Why do we test software

Understanding the testing challenge

Structural versus functional techniques Quality principles

Life cycle testing

The people challenges of software testing

8/8/2019 Domain 1 Sept2005

3/33

Why Do We Test Software?

What is a defect?

What is quality software?

ITs view - means meeting requirements

Users of software view - means fit for use

The two software quality gaps

User gap

IT gap

What is a defect to a software tester?

8/8/2019 Domain 1 Sept2005

4/33

Why Does a Development Process

Produce Defects? Variability is the enemy of quality the

concepts behind maturing a software

development process is to reducevariability.

What does it mean for a process to be in

or out of control?

Special causes of variation

Common causes of variation

8/8/2019 Domain 1 Sept2005

5/33

Reducing the Frequency of Defects

in Software Development

Five levels of maturity

Ad hoc

Control Core competency

Predictable

Innovative

Testers need to understand process

maturity

8/8/2019 Domain 1 Sept2005

6/33

An Effective Development Process

that Minimizes Defects

Incorporates

The concept of quality

Reducing variability

Reducing or eliminating defects

8/8/2019 Domain 1 Sept2005

7/33

PDCA View of a Process

P - Devise a plan

D - Execute (do) the plan

C - Check the results A - Take the necessary action

8/8/2019 Domain 1 Sept2005

8/33

8/8/2019 Domain 1 Sept2005

9/33

Workbench Components

Inputs

Procedures

Deliverables Standards

Tools

8/8/2019 Domain 1 Sept2005

10/33

Why do we test software The

Answer We test software because the processes

to build software are defect-prone

Traditional way of software testing identifieswhere the specification has been incorrectly

implemented, incompletely implemented or is

missing

Software testing compensates for the fact thatthe software development process does not

identify the true needs of the user

8/8/2019 Domain 1 Sept2005

11/33

Software testing can add significant value

to the IT organization.

Finding defects early costs considerably less Remove defects before going into production

Identify weaknesses in development process

so that those processes can be improved

Mature processes produce software moreeffectively and efficiently

8/8/2019 Domain 1 Sept2005

12/33

Understanding the Testing

Challenge Test Objectives

A testing Goal

Enable test manager and project manager togauge testing progress and success

Enhance communication within and without

project team by defining scope of test effort

Statement of the objective & expected results Created from system requirements document

Use a checklist to ensure process is followed

8/8/2019 Domain 1 Sept2005

13/33

Strategic Risks

A risk is a condition that can result in a

loss

We cannot eliminate risks, but we canreduce their occurrence and/or impact of

the loss

One of the most effective methods to

reduce computer system strategic risk is

testing

8/8/2019 Domain 1 Sept2005

14/33

Economics of Testing

Most problems associated with testing

occur from one of the following causes:

Failing to define testing objectives Testing at the wrong phase of the life cycle

Using ineffective test techniques

8/8/2019 Domain 1 Sept2005

15/33

Testing Policy

Definition of testing

Testing system

Evaluation Standards

8/8/2019 Domain 1 Sept2005

16/33

What Are You Testing For?

Why are defects hard to find?

Not looking

Looking, but not seeing Corrected condition causes another defect

Life cycle testing

8/8/2019 Domain 1 Sept2005

17/33

Verification and Validation

Verification ensures that the system (software,

hardware, documentation and personnel)

complies with the organizations standards and

processes Answers question Did we build the right system?

Validation physically ensures that the system

operates according to plan by executing the

system functions through a series of tests that

can be observed and evaluated

Answers question Did we build the system right?

8/8/2019 Domain 1 Sept2005

18/33

Functional and Structural Testing

Functional testing Advantages

Simulates actual system usage

Makes no system structure assumptions

Disadvantages Potential of missing logical errors in software

Possibility of redundant testing

Structural testing Advantages

You can test the softwares structure logic

You test code that you wouldnt use if you preformed only functionaltesting

Disadvantages Does not ensure that youve met user requirements

Its tests may not mimic real-world situations

8/8/2019 Domain 1 Sept2005

19/33

Testing Techniques

White-box testing Statement coverage

Decision coverage

Condition coverage

Decision/condition coverage

Multiple condition coverage

Black-box testing Equivalence partitioning

Boundary analysis

Error guessing

Incremental testing Top-down

Bottom-up Thread testing

Independent testing

Regression testing

8/8/2019 Domain 1 Sept2005

20/33

Reviews and Inspections

Three types of reviews

In-process

Decision-point or phase-end Post implementation

Three classes of reviews

Informal or peer

Semiformal or walkthroughs

Formal or inspections

8/8/2019 Domain 1 Sept2005

21/33

Structural System Testing

Techniques

Stress

Execution

Recovery Operations

Compliance (to process)

Security

8/8/2019 Domain 1 Sept2005

22/33

Functional System Testing

Techniques

Requirements

Regression

Error handling Manual support

Intersystem

Control Parallel

8/8/2019 Domain 1 Sept2005

23/33

Quality Principles

What is Quality?

Five perspectives of quality

Transcendent I know it when I see it

Product-based Possesses desired features

User-based Fitness for use

Development- and manufacturing-based

Conforms to requirements

Value-based At an acceptable cost

8/8/2019 Domain 1 Sept2005

24/33

Definitions of Quality

Frequently defined as meeting the customers

requirements the first time and every time

Conformance to a set of customer requirements

that, if met, result in a product that is fit for itsintended use

Much more than the absence of defects

R

equires controlled process improvement Can only be seen through the eyes of the

customers

8/8/2019 Domain 1 Sept2005

25/33

Why Quality

Cost of poor quality

Quality philosophies

Deming philosophy

Michael Tveites regrouping of Demings 14 points

Jurans ten steps to quality improvement

Cost of quality

Prevention costs

Appraisal costs

Failure costs

8/8/2019 Domain 1 Sept2005

26/33

Quality Assurance versus Quality

Control

Definitions

Quality assurance is a planned and systematic set of

activities necessary to provide adequate confidence

that products and services will conform to specifiedrequirements and meet user needs. Staff function.

Quality control is the process by which product quality

is compared with applicable standards, and the action

taken when nonconformance is detected. Line

function

8/8/2019 Domain 1 Sept2005

27/33

Life Cycle Testing

Requirements

Design

Program (build/construction) Test process

Installation

Maintenance

8/8/2019 Domain 1 Sept2005

28/33

Quality Factors

Correctness

Authorization

File Integrity Audit Trail

Continuity of Processing

Service Levels

8/8/2019 Domain 1 Sept2005

29/33

8/8/2019 Domain 1 Sept2005

30/33

Type of Development Methodology

Impacts the Test Plan

Traditional system development (and most

major changes to existing software

systems

Iterative development/prototyping/CASE

System maintenance

Purchased/contracted software

8/8/2019 Domain 1 Sept2005

31/33

People Challenges of

SoftwareTesting Negative view of testing and testers

Testers hold up implementation

Top ten people challenges Training in testing

Relationship building with developers

Using tools

Getting managers to understand testing

Communicating with users about testing

Making the necessary time for testing

Testing over the wall software

Trying to hit a moving target

Fighting a lose-lose situation

Having to say no

8/8/2019 Domain 1 Sept2005

32/33

Raise Management Awareness of

the Importance of Testing Calculate costs of testing and ensure the costs are

understood by management

Show management ways that the testing costs can bereduced by using more effective techniques

Relay other benefits of testing

Show that test training is built into your own personalgoals and objectives, and discuss your training needswith your immediate supervisor

Collect and distribute articles about testing Inquire about the budget for testing

Use creative means to enlighten management

8/8/2019 Domain 1 Sept2005

33/33

Model for Test Process

Improvement Examine the organizations needs and business

goals

Conduct assessment

Initiate process improvement Analyze assessment output and derive actionplan

Implement improvements

Confirm improvements

Sustain improvement gains

Monitor performance