Download - Software Testing Practices 1
-
8/6/2019 Software Testing Practices 1
1/53
# 1 Entity Solutions Limited 2004-05
Software Testing
Practices
By
Prof. V K GargM Tech, CISA, CISM, PMPR
10th & 11th March 2006
Mumbai
Organized by:
Iducate Managers
404, Sai prasad, Road No. 7, MIDC, Near Akruti Trade Centre, Andheri
East Mumbai 400093, phone: 28263207, email: [email protected]
Prasanta Mohanty
-
8/6/2019 Software Testing Practices 1
2/53
# 2 Entity Solutions Limited 2004-05
Representing View of Trainer
I am here for representing the view of Prof.
V.K.Garg.
I will lead your kind attention to theconcepts which are new to us and, if
agreed, can be implemented with help of
senior management.
-
8/6/2019 Software Testing Practices 1
3/53
# 3 Entity Solutions Limited 2004-05
Why Testing?
HOW CAN IT ADD VALUE?
-
8/6/2019 Software Testing Practices 1
4/53
# 4 Entity Solutions Limited 2004-05
Testing
Can be described as a process used for
Twin Terms Testing & Debugging in
software and to establishing that thesoftware has attained a specific degree of
quality with respect to SELECTED
ATTRIBUTES.
Group of procedures carried out to
evaluate some aspect of a piece of
software
-
8/6/2019 Software Testing Practices 1
5/53
# 5 Entity Solutions Limited 2004-05
Debugging
Is the process of locating the fault or
defect, repairing the code, and retesting
the code. Debugging process begins AFTER testing
has been carried out and the tester has
noted that the software is not behaving as
specified.
-
8/6/2019 Software Testing Practices 1
6/53
# 6 Entity Solutions Limited 2004-05
Software Development Process
-
8/6/2019 Software Testing Practices 1
7/53
# 7 Entity Solutions Limited 2004-05
Validation
Is the process of evaluating a software
system or component during, or at the end
of, the Development cycle in order to
determine whether it satisfies specified
requirements
Is usually associated with Traditional
Execution-based Testing that exercisingthe code with Test cases
-
8/6/2019 Software Testing Practices 1
8/53
# 8 Entity Solutions Limited 2004-05
Verification
Is the process of evaluating a software
system component to determine whether
the products at a given development
phase satisfy the conditions imposed at
the start of the phase
Is usually associated with activities such
as Inspections and Reviews of Softwaredeliverables.
-
8/6/2019 Software Testing Practices 1
9/53
# 9 Entity Solutions Limited 2004-05
The Extended / Modified V Model
-
8/6/2019 Software Testing Practices 1
10/53
# 10 Entity Solutions Limited 2004-05
Basic Definitions -1 Errors
An Error is a mistake , misconception ormisunderstanding on the part of a softwaredeveloper
Faults (Defects) A Fault (defect) is introduced into the software as the
result of an error
It is an Anomaly in the software that may cause it toBehave Incorrectly, and not according to its
specifications. Failures
A Failure is the inability of a software system orcomponent to Perform its Required functions withinspecified performance requirements.
-
8/6/2019 Software Testing Practices 1
11/53
# 11 Entity Solutions Limited 2004-05
Test Cases
Are procedures that examine thesoftware and produce results that will
lead to Acceptance, Modification orRejections of the Software.
Contains the following information
A set of Test Inputs (Set of Data Items)Execution Condition (Conditions
required for running the test)
Expected Outputs (Specified results to
be produced by the code under test)
Basic Definitions -2
-
8/6/2019 Software Testing Practices 1
12/53
# 12 Entity Solutions Limited 2004-05
ExampleTest
case #
Input Description Input Data Expected
Results
Actual
Results
1 Search for a book
by specifying a
valid author name
Author= Post a list of
books written
by this author
2 Search for a book
by specifying a
valid auther name
but non-existent
title
Auther=
Title=
Title name not
found.
List all books
written by this
author
3 Have multiple users
conduct the samequery
simultaneously
Search
Criteria=No.of
Users=
Accurate
results arepresented to all
users, with an
acceptable
response time
-
8/6/2019 Software Testing Practices 1
13/53
# 13 Entity Solutions Limited 2004-05
Test
Is a group of Related Test Cases or a group ofrelated Test cases and Test Procedures
(steps needed to carry out a test) Test Bed
Is an Environment that contains all theHardware, and software needs to test asoftware component or a software system
Test Report
Is a Document or piece of software that allowstesters to determine whether test has passedor failed.
Basic Definitions -3
-
8/6/2019 Software Testing Practices 1
14/53
# 14 Entity Solutions Limited 2004-05
Software Quality Software Quality related to the degree to which a
system, system component, or process meetsspecified requirements
Quality relates to the degree to which a system,system component, or process meets customer oruser needs or expectations.
Reviews
A review is a group meeting (process) whose
purpose is to evaluate a software artifact or a setof software artifacts
Are a type of static testing techniques that can beused to evaluate the quality of a software Artifactsuch as Requirements Document, A Test Plan, A
Design Document or a code component.
Basic Definitions -4
-
8/6/2019 Software Testing Practices 1
15/53
# 15 Entity Solutions Limited 2004-05
Defects Hypotheses and Tests - 1
-
8/6/2019 Software Testing Practices 1
16/53
# 16 Entity Solutions Limited 2004-05
Successful test is one that reveals the presence ofa Hypothesized defects
Role of a Tester is like that of a Doctor, who is inthe process of constructing a diagnosis for an ill
patient. The doctor develops Hypotheses aboutPossible illness using his knowledge of possiblediseases, and the patients symptoms
Test are made in order to make the correctdiagnosis. One could view defective software as the
ill patient. Testers like doctors need to have knowledge about
possible defects (illness in order to develop defecthypotheses)
Defects Hypotheses and Tests - 2
-
8/6/2019 Software Testing Practices 1
17/53
# 17 Entity Solutions Limited 2004-05
TEST TYPES
1. User Interface Tests
2. Functional Tests
3. Server side Tests4. Database Tests
5. Help Tests
6. Installation Tests7. Configuration and Compatibility Tests
8. Security Tests
9. Performance Tests
-
8/6/2019 Software Testing Practices 1
18/53
# 18 Entity Solutions Limited 2004-05
Software Testing Principles-1
1. Testing is the process of Executing a softwarecomponent using a selected set of test cases with theintent of
1. Revealing defects
2. Evaluating Quality2. A good test case is one that has a high probability ofrevealing a yet undetected defect(s)
3. A test case must contain the expected output or result
4. Test Cases should be developed for both valid and
invalid input conditions.
5. Test results should be inspected meticulously.
-
8/6/2019 Software Testing Practices 1
19/53
# 19 Entity Solutions Limited 2004-05
6. Testing should be carried out by agroup that is independent of thedevelopment group.
7. Tests must be repeatable andreusable.
8. Testing should be planned.
9. Testing activities should be integratedinto the software lifecycle.
Software Testing Principles-2
-
8/6/2019 Software Testing Practices 1
20/53
# 20 Entity Solutions Limited 2004-05
Testing Challenge
Achieve testing goals by using Finite Number of Test
Cases
Design Effective test cases
Greater probability of detecting defects
More efficient use of organizational resources
Higher probability for Test Reuse
Close adherence to Testing and Project
Schedules and budgets Possibility for delivery of High Quality software
product.
-
8/6/2019 Software Testing Practices 1
21/53
# 21 Entity Solutions Limited 2004-05
Smart Testing
-
8/6/2019 Software Testing Practices 1
22/53
# 22 Entity Solutions Limited 2004-05
Basic Testing Strategies
Both the strategies should be used to design Test cases
-
8/6/2019 Software Testing Practices 1
23/53
# 23 Entity Solutions Limited 2004-05
Black Box Testing - 1
Tester has knowledge of what the system
under test Does. No knowledge of its Inner
Structure, how it works.
Can be used for any size of the softwareunder test such as Simple Module, Member
function, Object cluster, subsystem or a
complete software system
Only considers software Behavior and
Functionality, is often called Functional or
Specification Based Testing
-
8/6/2019 Software Testing Practices 1
24/53
# 24 Entity Solutions Limited 2004-05
Black Box Test strategy considers onlyInputs and Outputs as a basis forDesigning Test Cases. Main issue is How
do we choose a suitable set of Inputs fromall possible valid and Invalid Inputs
Highly effective in Testing software with awell defined set of Inputs and Outputs thatare described in specification.
Black Box Testing - 2
-
8/6/2019 Software Testing Practices 1
25/53
# 25 Entity Solutions Limited 2004-05
Black Box Testing Methods
Random Testing
Equivalence Class Partitioning
Boundary Value Analysis
Error Guessing
-
8/6/2019 Software Testing Practices 1
26/53
# 26 Entity Solutions Limited 2004-05
Random Testing
Every Software module / system has an Input Domain fromwhich Test Input data is selected. If a Tester randomly selectsinputs from the domain, this is called Random Testing
Main Issues:
Are these values adequate
Are there any Input values other than those selected more
likely to reveal defects e.g. values at the beginning or at theEnd of the domain.
Should any values outside the domain be used as Test input
Tools that generate Random Test data are available, theygenerate random test inputs according to a statistics distributionassociated with pattern of usage
Very useful at the system level for stress testing.
-
8/6/2019 Software Testing Practices 1
27/53
# 27 Entity Solutions Limited 2004-05
Equivalence Class Partitioning - 1
Equivalence Class partitioning results in a
partitioning of the Input or output domain of
the software under test
The Finite Number of partitions orequivalence classes that result, allow the
tester to select a given number, of an
equivalence class as a Representative of that
class.
A Test value in a particular class is equivalent
to a test value of any other member of that
class.
-
8/6/2019 Software Testing Practices 1
28/53
# 28 Entity Solutions Limited 2004-05
The tester must consider both valid
and invalid equivalence classes
Equivalence classes may also beselected for output conditions (not a
common usage)
The derivation of equivalent classesis a heuristic process
Equivalence Class Partitioning - 2
-
8/6/2019 Software Testing Practices 1
29/53
# 29 Entity Solutions Limited 2004-05
If Input condition is
specified as..
Equivalence Class selection
Range of Values Select one valid equivalence class that covers the allowed range
Select two invalid equivalence classes, one outside each end of
the range
Number of Values Select valid equivalence class that includes the allowed number of
values
Two invalid equivalence classes that are outside each end of the
allowed numbers
Set of valid inputs Select one valid equivalence that combines al the numbers of the
setSelect one invalid equivalence class for any value outside the set
Must be condition Select one valid equivalence class to represent the must be
condition
One invalid class that does not include the must be condition
Equivalence Class Partitioning - 3
-
8/6/2019 Software Testing Practices 1
30/53
# 30 Entity Solutions Limited 2004-05
Boundary Value Analysis
The test cases developed based on
Equivalent class partitioning can be
strengthened by use of Boundary ValueAnalysis
-
8/6/2019 Software Testing Practices 1
31/53
# 31 Entity Solutions Limited 2004-05
If Input condition is specified
as
Test Case Selection
Range of Values Develop valid test cases for the end of the range
Invalid test cases for possibilities just above and below the ends of the
range
Number of Values Develop valid test cases for the minimum and maximum numbers
Develop invalid test cases that include one lesser and one greater than
the maximum and minimum
Ordered set (e.g. table or
linear list)Develop tests that focus on the first and last element of the test
Boundary Value Analysis
-
8/6/2019 Software Testing Practices 1
32/53
# 32 Entity Solutions Limited 2004-05
White Box Testing Strategy W
hite Box Testing Also called glass box testing
Is a complementary approach to Black box testing
The Testers Goal is to determine if all the logical and dataelements in the software unit are functioning properly
Tester needs knowledge of Internal logic structure of thesoftware under test. This knowledge becomes available tothe tester only during detailed design phase of development.
Follows Black box design
Can be most useful when testing small components, due to
details required for test design Focus is on Structural Elements such as statements and
Branches.
-
8/6/2019 Software Testing Practices 1
33/53
# 33 Entity Solutions Limited 2004-05
Coverage Analysis Is important for security point of view
Example - If a Tester sets a goal for a unit specifyingthe Requirement for complete or 100% statementcoverage then the test cases developed must ensurethat all the statements in the unit are executed at least
once. The logical elements most commonly considered for
coverage are:
Program Statements
Decision / Branches Conditions
Combinations of Decisions / Conditions
Paths (Node Sequence)
-
8/6/2019 Software Testing Practices 1
34/53
# 34 Entity Solutions Limited 2004-05
Levels of Testing - 1
Execution based Software testing
specially for large systems is usually
carried out at different levels.
-
8/6/2019 Software Testing Practices 1
35/53
# 35 Entity Solutions Limited 2004-05
Level What is Tested Goal of Testing
Unit Test A single component Detect functional and structural defects in
the unit
Integration Test Several components
as a group
Component Interaction
System Test System as a whole Performance evaluation such as usability,reliability, and other quality related
attributes
Acceptance Test Custom made s/w for
individual client
Show that s/w meets all the customer
requirements
Alpha Test S/w developed for the
Mass market
Bring potential users to the developers
site to use (test) the s/w
Beta Test S/w developed for the
Mass Market
Send the s/w out to potential user to use it
under real world conditions and report
defects to the developing organization
Levels of Testing - 2
-
8/6/2019 Software Testing Practices 1
36/53
# 36 Entity Solutions Limited 2004-05
Test Planning
-
8/6/2019 Software Testing Practices 1
37/53
# 37 Entity Solutions Limited 2004-05
IEEE Definition of Test Plan
A document describing the scope,
approach, resources, and schedules of
intended testing activities. It identifies test
items, the features to be tested, the testing
tasks, who will do each task, and any risk
requiring contingency planning
-
8/6/2019 Software Testing Practices 1
38/53
# 38 Entity Solutions Limited 2004-05
Test Plan Template (as per IEEE
Standards) Contents1. Introduction
2. Items to be Tested
3. Features to be Tested
4. Approach
5. Pass/Fail Criteria6. Suspension and Resumption Criteria
7. Test Deliverables
8. Testing Tasks
9. The Testing Environment
10. Responsibilities
11. Staffing and Training Needs12. Scheduling
13. Risks and Contingencies
14. Approvals
-
8/6/2019 Software Testing Practices 1
39/53
# 39 Entity Solutions Limited 2004-05
INTRODUCTION
Summarize the software items and software features to betested. The need for each item and its history may be included.
References to the following documents, when they exist, are
required in the highest level test plan:
a) Project authorization;
b) Project plan;c) Quality assurance plan;
d) Configuration management plan;
e) Relevant policies;
f) Relevant standards.
-
8/6/2019 Software Testing Practices 1
40/53
# 40 Entity Solutions Limited 2004-05
Test Items
Identify the test items including their version/revision level.
Supply references to the following test itemdocumentation, if it exists:
a) Requirements specification;
b) Design specification;
c) Users guide;
d) Operations guide;
e) Installation guide.
Reference any incident reports relating to the testitems.
Items that are to be specifically excluded fromtesting may be identified.
-
8/6/2019 Software Testing Practices 1
41/53
# 41 Entity Solutions Limited 2004-05
Features to be tested
Identify all software features and
combinations of software features to be
tested. Identify the test design
specification associated with each feature
and each combination of features.
-
8/6/2019 Software Testing Practices 1
42/53
# 42 Entity Solutions Limited 2004-05
Features not to be tested Identify all features and significant
combinations of features that will not be
tested and the reasons.
-
8/6/2019 Software Testing Practices 1
43/53
# 43 Entity Solutions Limited 2004-05
Approach
Describe the overall approach to testing. For each major group offeatures or feature combinations
Specify the approach that will ensure that these feature groups areadequately tested.
Specify the major activities, techniques, and tools that are used to testthe designated groups of features.
The approach should be described in sufficient detail to permitidentification of the major testing tasks and estimation of the timerequired to do each one.
Specify the minimum degree of comprehensiveness desired. Identifythe techniques that will be used to judge the comprehensiveness of thetesting effort (e.g., determining which statements have been executedat least once).
Specify any additional completion criteria (e.g., error frequency). Thetechniques to be used to trace requirements should be specified.
Identify significant constraints on testing such as test item availability,testing resource availability, and deadlines.
-
8/6/2019 Software Testing Practices 1
44/53
# 44 Entity Solutions Limited 2004-05
Item pass/fail criteria
Specify the criteria to be used to
determine whether each test item has
passed or failed testing.
-
8/6/2019 Software Testing Practices 1
45/53
# 45 Entity Solutions Limited 2004-05
Suspension criteria and resumption
requirements
Specify the criteria used to suspend all or
a portion of the testing activity on the test
items associated with this plan.
Specify the testing activities that must be
repeated, when testing is resumed.
-
8/6/2019 Software Testing Practices 1
46/53
# 46 Entity Solutions Limited 2004-05
Test deliverables Identify the deliverable documents. The following documents
should be included:
a) Test plan;
b) Test design specifications;
c) Test case specifications;
d) Test procedure specifications;
e) Test item transmittal reports; (item submitted to test eng.)f) Test logs;
g) Test incident reports;
h) Test summary reports.
Test input data and test output data should be identified asdeliverables.
Test tools (e.g., module drivers and stubs) may also beincluded.
-
8/6/2019 Software Testing Practices 1
47/53
# 47 Entity Solutions Limited 2004-05
Testing tasks
Identify the set of tasks necessary to
prepare for and perform testing. Identify all
inter-task dependencies and any special
skills required.
-
8/6/2019 Software Testing Practices 1
48/53
# 48 Entity Solutions Limited 2004-05
Environmental needs
Specify both the necessary and desired properties of the testenvironment. This specification should contain the physical
characteristics of the facilities including the hardware, the
communications and system software, the mode of usage (e.g.,
stand-alone), and any other software or supplies needed to support
the test.
Specify the level of security that must be provided for the test
facilities, system software, and proprietary components such as
software, data, and hardware.
Identify special test tools needed. Identify any other testing needs
(e.g., publications or office space). Identify the source for all needs
that are not currently available to the test group.
-
8/6/2019 Software Testing Practices 1
49/53
# 49 Entity Solutions Limited 2004-05
Responsibilities
Identify the groups responsible for managing,
designing, preparing, executing, witnessing,
checking, and resolving.
Identify the groups responsible for providingthe test items identified and the
environmental needs identified.
These groups may include the developers,
testers, operations staff, user representatives,
technical support staff, data administration
staff, and quality support staff.
-
8/6/2019 Software Testing Practices 1
50/53
# 50 Entity Solutions Limited 2004-05
Staffing and training needs
Specify test staffing needs by skill level.
Identify training options for providing
necessary skills.
-
8/6/2019 Software Testing Practices 1
51/53
# 51 Entity Solutions Limited 2004-05
Schedule Include test milestones identified in the
software project schedule as well as all
item transmittal events.
Define any additional test milestonesneeded. Estimate the time required to do
each testing task.
Specify the schedule for each testing task
and test milestone. For each testingresource (i.e., facilities, tools, and staff)
specify its periods of use.
-
8/6/2019 Software Testing Practices 1
52/53
# 52 Entity Solutions Limited 2004-05
Risks and contingencies
Identify the high-risk assumptions of the
test plan. Specify contingency plans for
each (e.g., delayed delivery of test items
might require increased night shift
scheduling to meet the delivery date).
-
8/6/2019 Software Testing Practices 1
53/53
# 53 Entity Solutions Limited 2004-05
Thanks