software testing basics
TRANSCRIPT
Software Testing
Prepared by:Raghav
Target Audience
Beginners Intermediate
What do we cover?
Basics of testing Levels of Testing Testing techniques
Software testing
25
50
75
100
125
Relative costper defect
Req Design Code Test Production
Software testing
Points to Ponder: 80% of all defects occur in 20% of the work Test strategies impact test costs Conversion of testing from craft to process can
give extensive benefits
Software Testing
What hinders effective testing?
Aspects that hinder Effective testing
Optimism Belief that the system works Negative attitude towards effective testing Conflict between testers and developers Testing is expensive Delivery commitments
The V Model
What is V model?
The V ModelThe V Modelvalid
ation
Customer Requirements
Software Requirements
Architecture (HLD)
Detailed Design (LLD)
Acceptance
System testing
Integration testing
Unit testing
Code
Testing v/s Life Cycle
Stage Test Activity
Requirements Acceptance Test ,System Test
Design (HL) Integration Test
Design (LL) Unit Test
Code Unit test / Debug
Integration Integration test / debug
System test / debug
Acceptance Acceptance test / debug
Why Test?
Developers not fallible Bugs in compilers, languages, DBs, OS Certain bugs easier to find in testing Don’t want customers to find bugs Post release debugging is expensive
Test team
Why separate testing team?
Is Testing everything?
There are other approaches possible to create good software
Effective methods are•Inspection•Design style•Static analysis•Language checks•Development environment
Can Testing be replaced?
NO!! Even if we use other methods, cannot do away
with testing Review, inspect, read, walkthrough, better
methodologiesAnd then TEST
TESTING
Definition: Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements. (IEEE 83a)
Testing objective
Process of executing a program with an intent of finding an error
A good test case is one that has a high probability of finding an as yet undiscovered error
A successful test is one that uncovers an as yet undiscovered error
Testing Involves
•Test plan•Generate test conditions, cases•Create required test environment•Execute tests by
Initiating application under testApplying inputs that are specified in the already generated test case
Compare actual results with expected resultsResults: Test passed or failed!
A ‘FAILED’ Test
The term ‘Test Failed’ is used when the program has failed to give the expected result
Actually, the test has succeeded in discovering a fault!
Definitions
•Defect
A deviation from specification or standard
Anything that causes customer dissatisfaction
Verification
All QC activities throughout the life cycle that ensure that interim deliverables meet their input specification
Validation
The ‘test phase’ of the life cycle which assures that the end product meets the user’s needs
Debugging
Debugging is the act of attempting to
determine the cause of the symptoms
of malfunctions detected by testing or
by frenzied user complaints. (Beizer)
Testing: static and dynamic
• Static Testing: Verification performed without executing the system’s code
•Code Inspection•Reviews
• Dynamic Testing: Verification or validation performed by executing the system’s code
Testing Techniques
• Black Box• White Box• Basis path testing
Black box testing
What is black box testing?
Black box testing
Also called Behavioural testing, focuses on functional requirement of the software
Black box testing
Detects following categories of errors
Incorrect or missing functions Interface errors Errors in data structures and external database
access Behaviour or performance errors Initialisation and termination errors
Types of Black box testing
Boundary value analysisEquivalence partitioningSpecial value
BVA(Boundary Value Analysis)
What is BVA?
Boundary value analysis (two variables)Boundary value analysis (two variables)
a<=x1 <=b
c<=x2 <=d
X2
X1
d
c
a b
BVA
Great number of errors tends to occur at the boundaries of the input domain rather than in the ‘center’
BVA leads to a selection of test cases that exercise bounding values
Equivalence partitioning
What is equivalence partitioning?
Equivalence partitioning testingEquivalence partitioning testing •EC partitions the data to partition
of a set• Partition refers to collection of
mutually disjoint subsets whose
union is the entire set.•Choose one data element from
each partitioned set.• EC based testing allows
- to have a sense of complete
testing.
- helps avoid redundancy• The KEY is the choice of
equivalence relation.
Special value
What is special value testing?
Special value testingSpecial value testing
• Select test data on the basis of features of a function to be computed
• Tester uses her/his domain knowledge, experience with similar programs
• Ad-hoc/seat of pants/skirt testing
• No guidelines, use best engineering judgment
• Special test cases/Error guessing
• Is useful – don't discount effectiveness !
Levels of testingLevels of testing
What does each level of testing uncover?
• Unit testing– implementation deficiencies– Code / LLD
• Integration testing– interface deficiencies– architecture / HLD
• System testing– requirement deficiencies– SRS
• Regression testing– fix has not broken existing
functionality - product / code
• Alpha / Beta testing• Acceptance testing
– customer expectations– notion of stability (time)
• Certification testing – external compliance– third party
Unit Testing
What is Unit testing?
Unit Testing
Unit- Smallest testable piece of software A unit can be compiled / assembled / linked /
loaded; and put under a test harness Unit testing is done to show that the unit does not
satisfy the functional specification and/or its implemented structure does not match the intended design structure
Integration Testing
What is integration testing?
Integration Testing
Integration is the process of aggregating components to create larger components.
Integration testing done to show that even though components were individually satisfactory, the combination is incorrect or inconsistent
Integration problems
What are the general integration problems?
Types of Integration problems
Configuration / version control I/O format, protocol mismatches Conflicting data views/ usage Data integrity violated Wrong parameters
Integration test approachIntegration test approach
• Top down
• Bottom up
• Sandwich
M23
M12
M22
M31
M0
M11
M21
M13
Top down integrationTop down integration
• Modules are integrated by moving downward
• Depth-first or breadth-first manner
• Verifies major control or decision points early in the test process
• Stubs replace low-level modules
M23
M12
M22
M31
M0
M11
M21
M13
Bottom – up integrationBottom – up integration
• Construction and testing with atomic modules
• Stubs are NOT needed
• Drivers are needed
Integration testing – some thoughtsIntegration testing – some thoughts
• How to choose integration order – incremental
• May be based on criticality of interfaces/module
• Risk reduction principle may be used
• Critically based on ..
- Module used by lot of other modules
- Module has high cyclomatic complexity
- Has special non-functional (e.g, performance) requirements
What to check for ….
• Interface integrity
• Functional validity
• Information content
• Performance
System testing
What is system testing?
System Testing
A system is the big component System testing is aimed at revealing bugs that
cannot be attributed to a component as such, to inconsistencies between components or planned interactions between components
Concern: issues, behaviours that can only be exposed by testing the entire integrated system (e.g., performance, security, recovery)
System testing is…System testing is…
• Functionality Testing
• Stress testing
• Usability testing
• Security testing
• Performance testing
• Storage testing
• Internationalization testing
• Compatibility testing
• Installation testing
• Reliability testing
• Recovery testing
• Supportability testing
• Compliance testing
• …
Acceptance Testing
Aims at uncovering ‘implied requirements’ Aims at evaluating ‘fitness for use’ Should not find bugs which should have been
found in earlier testing phases
Regression Testing
Testing after changes been made to ensure that no unwanted changes were made
Test Strategy
This defines the scope and general directions for testing for a project
Should be developed for each project Should be reviewed and approved with the
project plan
Test Plan
Test plans specify the test conditions, features, functions that will be tested for a specific level of testing. Plans also contain other information about resources, schedules, etc
Test plans should be prepared as soon as the corresponding document in the development life cycle is produced
Test specification
What is test specification?
Test Case
A sequence of steps, sub steps, and other actions, performed serially, in parallel, or in some combination of consecution, that creates the desired test conditions that the test case is designed to evaluate
Is complete testing possible?
To prove that a program is free of bugs is•Practically impossible•Theoretically a mammoth exercise
Stop Criteria
When to stop testing?
STOP Criteria
Time runs out – Poor criteria A certain number of errors found Require a certain test coverage Stop when testing becomes unproductive
Software testing
ExercisePrepare a checklist for testing
IV & V
What is Independent Verification & Validation?