software testing 1
DESCRIPTION
abcTRANSCRIPT
LOGO
Nguyễn Đạt ThôngFaculty of Mathematics and InformaticsUniversity of Science, VNU-HCM
Software TestingSoftware Testing
Course
�Name: Software Testing 1�Lecturer: Nguyễn Đạt Thông�Credit: 3�Duration:
� Presentation: 13 weeks� On-the-fly practice: 02 weeks� Lab practice: 10 weeks
�Prerequisites:� Software Engineering
Nguyễn Đạt Thông Software Testing 2
Content
1• Introduction
2• Test Principles
3• Test Types
4• Test Techniques
5• Test Methodologies
6• Test Automation
7• Test Management
8• Test Tools
9• Test a Software Project
3Software TestingNguyễn Đạt Thông
Practice
Test Analysis
Test Case
Test Report
Tester
Case
Bug Report
Bug Tracking
Report
Nguyễn Đạt Thông Software Testing 4
Evaluate
�Mid-term exam: 30%� Type: Writing� Time: Up to chapter 6
�Lab practice: 30%� Type: On lab� Time: 15th week
�Final exam: 50%� Type: Multiple choice� Time: (Depend)
Nguyễn Đạt Thông Software Testing 5
Reference
1. Internet Source. Keyword:Software Testing, Test Methodology, Test Principles, Test Tool, Test Technique.
2. Dorothy Graham, Erik van Veenendaal, 2. Dorothy Graham, Erik van Veenendaal, Isabel Evans, Rex Black“Foundations of Software Testing”
3. Software Testing Help http://www.softwaretestinghelp.com
4. The UML Testing Profile http://www.omg.org
Nguyễn Đạt Thông Software Testing 6
Reference (2)
6. Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki
Nguyễn Đạt Thông Software Testing 7
Introduction
Chapter 1
Nguyễn Đạt Thông Software Testing 8
Content
1• Introduction
2• Test Principles
3• Test Types
4• Test Techniques
5• Test Methodologies
6• Test Automation
7• Test Management
8• Test Tools
9• Test a Software Project
9Software TestingNguyễn Đạt Thông
Introduction
�What is a bug?�Quality and Reliability�Why is testing necessary?�Purposes of testing�No bugs found…�How much testing is enough?�Test approaches�Which factors influence test?�Independence in test�Who are Testers?
10Software TestingNguyễn Đạt Thông
What is a bug?
�Error: a human action that produces an incorrect result
�Fault: a manifestation of an error in software� also known as a defect or bug� also known as a defect or bug� if executed, a fault may cause a failure
�Failure: deviation of the software from its expected delivery or service � (found defect)
Nguyễn Đạt Thông Software Testing 11
What is a bug? (2)
A person makes
an error...
… that creates abug in the
Nguyễn Đạt Thông Software Testing 12
bug in thesoftware
...
… that can causea failure
in operation
Quality and Reliability
�Software quality:� Reliability: the probability that software will not
cause the failure of the system for a specified time under specified conditions.
� Efficiency: the ability to ensure high � Efficiency: the ability to ensure high performance once the application is in run-time mode.
� Security: the ability to encounter critical vulnerabilities that damage the business.
� Maintainability: includes adaptability, portability and transferability
Software Testing 13Nguyễn Đạt Thông
Quality and Reliability (2)
�The less failures system causes, the more reliable the system is.
�No system is bug-free. So a system can be reliable but still have bugs.
�Bugs occur in software since:�Bugs occur in software since:� Software is written by human beings
• Know something, but not everything• Have skills, but are not perfect• Make mistakes
� Software is developed under increasing pressure and strict deadlines
Nguyễn Đạt Thông Software Testing 14
Why is testing necessary?
�Software failures cost:� Huge sums
• Ariane 5: $7,000,000,000• Mariner space probe to Venus: $250,000,000• American Airline: $50,000,000• American Airline: $50,000,000
� Very little or nothing at all• Minor inconvenience• No visible
�Software is not “linear”:� Small input may have very large effect
Software Testing 15Nguyễn Đạt Thông
Why is testing necessary? (2)
�Software failures can cause death or injury:� Radiation treatment kills patients� Train driver killed� Aircraft crashes (Airbus & Korean Airlines)� Aircraft crashes (Airbus & Korean Airlines)� Bank system overdraft letters cause
suicide
Software Testing 16Nguyễn Đạt Thông
Why is testing necessary? (3)
�Testing is necessary:� because software is likely to have bugs� because failures can be very expensive� to learn about the reliability of the software� to avoid being sued by customers� to avoid being sued by customers� to stay in business
�Testing is not:� to prove that the software has no bugs� to fill the time before the release date� because testing is included in the project plan
Software Testing 17Nguyễn Đạt Thông
Purposes of testing
�Purposes of testing are to:� find bugs/faults � build confidence� demonstrate conformance to requirements� reduce costs� show system meets user needs� assess the software quality� improve the software quality: reliability,
efficiency, security, usability, maintainability, reusability, testability, etc.
Software Testing 18Nguyễn Đạt Thông
No bugs found…
ConfidenceBugs found
�No bugs found = confidence?
Nguyễn Đạt Thông Software Testing 19
Time
No bugs found… (2)
ManyBugs
FewBugs
You thinkyou are here
High
Low High
Nguyễn Đạt Thông Software Testing 20
FewBugs
FewBugs
You may be here
TestQuality
Low
Software Quality
Low High
How much testing is enough?
system has20 screens
Average 4 menus3 options / menu
Average: 10 fields / screen2 types input / field(date as Jan 3 or 3/1)(number as integer or decimal)
� Total for 'exhaustive' testing:
�� 20 x 4 x 3 x 10 x 2 x 100 = 20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests480,000 tests
� 1 sec/test-> 8000 mins ~ 133 hrs ~ 17.7 days
(not counting finger trouble, bugs or retest)
� 10 secs-> 34 wks, 1 min-> 4 yrs, 10 min-> 40 yrs
Software Testing 21Nguyễn Đạt Thông
(number as integer or decimal)Around 100 possible values
We cannot test everything
How much testing is enough? (2)
�Exhaustive testing:� is exercising all combinations of inputs and
preconditions� consumes impractical amount of time
�We do not run an exhaustive test�We do not run an exhaustive test� not enough resources� not enough time� not necessary
Software Testing 22Nguyễn Đạt Thông
How much testing is enough? (3)
�it’s never enough�when you have done what you planned�when your customer/user is happy�when you have proved that the system
works correctlyworks correctly�when you are confident that the system
works correctly�it depends on the risks for your system
Software Testing 23Nguyễn Đạt Thông
How much testing is enough? (4)
�So little time vs. So much to test:� test time will always be limited� base on RISK, we have to determine:
• what to test first• what to test most• what to test most• how thoroughly to test each item• what not to test (this time)
� we have to allocate the time available for testing by prioritizing testing
Software Testing 24Nguyễn Đạt Thông
How much testing is enough? (5)
�Other factors that influence testing� contractual requirements� legal requirements� industry-specific requirements
• e.g. pharmaceutical industry (FDA), compiler standard tests, safety-critical or safety-related such as railroad switching, air traffic control
Nguyễn Đạt Thông Software Testing 25
How much testing is enough? (6)
�Most important principles:
It is difficult to determinehow much testing is enoughbut it is not impossible
Software Testing 26Nguyễn Đạt Thông
Prioritize testsso that,
whenever you stop testing,you have done the best testing
in the time available.
Test approaches
�Traditional testing approach� Show that the system:
• does what it should• doesn't do what it shouldn't
� Easy test cases� Easy test cases
Software Testing 27Nguyễn Đạt Thông
Goal: show working
Success: system works
Result: bugs left in
Test approaches (2)
�A better testing approach� Show that the system:
• does what it shouldn't• doesn't do what it should
� Difficult test cases� Difficult test cases
Software Testing 28Nguyễn Đạt Thông
Goal: find bugs
Success: system fails
Result: fewer bugs left in
Test approaches (3)
�The testing paradox� Purpose of testing is to destroy confidence:
• Purpose of testing is to try to find bugs• Finding bugs destroys confidence
� Purpose of testing is to build confidence� Purpose of testing is to build confidence
Software Testing 29Nguyễn Đạt Thông
The best way to build confidence
is to try to destroy it
Independence in test
�Test your own work?� find 30% - 50% of your own bugs� same assumptions and thought processes� see what you meant or want to see, not
what is therewhat is there� emotional attachment
• don’t want to find bugs• actively want NOT to find bugs
�Independence in test is very important.
Software Testing 30Nguyễn Đạt Thông
Independence in test (2)
�Levels of independence� Tests designed by the person who wrote
the software� Tests designed by a different person� Tests designed by someone from a � Tests designed by someone from a
different department or team (e.g. test team)
� Tests designed by someone from a different organisation (e.g. agency)
� Tests generated by a tool (low quality tests?)
Software Testing 31Nguyễn Đạt Thông
Who are Testers?
�Testers are the persons who:� Are usually not welcomed,� Do a boring repeated work,� Do a destructive process,� Bring bad news,� Describe bugs to authors,� Work under worst (time) pressure,� Need to take a different view,� Need a different mindset,
Software Testing 32Nguyễn Đạt Thông
Who are Testers? (3)
�Testers have responsibilities to:� follow the test plans, script, etc. as
documented � report bugs objectively and factually, no abuse� check tests are correct before reporting bugs� check tests are correct before reporting bugs� remember it is the software, not the
programmer, that you are testing� assess risk objectively� prioritise what you report � communicate the truth
Software Testing 33Nguyễn Đạt Thông
Who are Testers? (2)
�Testers have the rights to:� be regarded as a professional,� request specifications and test plans,� access to accurate progress and changes,� be trained in working domain,� find bugs,� report bugs seriously (non-reproducible),� make predictions about future bug levels,
Software Testing 34Nguyễn Đạt Thông
Questions
The important thing is not to stopquestioning. Curiosity has its ownreason for existing.
(Albert Einstein)
35Software TestingNguyễn Đạt Thông