cen 5070 – software v&v what is software testing © 2001-8, dr. e.l. jones
TRANSCRIPT
![Page 1: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/1.jpg)
CEN 5070 – Software V&V
What is Software Testing
© 2001-8, Dr. E.L. Jones
![Page 2: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/2.jpg)
9/2003 Testing Concepts 2
Purpose
This module presents the basic concepts of software testing. We present several aspects of software testing, and a give a generic lifecycle.
"… arbitrarily selected test set ... results in inefficient testing, leaving
some functions untested while performing redundant testing of others."Darlene Mackay, Quality Consultants Unlimited
![Page 3: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/3.jpg)
9/2003 Testing Concepts 3
Agenda
• What is Testing
• Testing Lifecycle
• A Framework for Testing
• A Complete Example
![Page 4: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/4.jpg)
9/2003 Testing Concepts 4
WHAT IS TESTING?
• Executing software to determine whether it meets its specification
• Executing a software element to discover defects or anomalies
• Executing software to evaluate its behavior in different environments
• Inspecting software element code to discover defects or anomalies.
![Page 5: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/5.jpg)
9/2003 Testing Concepts 5
TESTING IS HARD
• Exhaustive testing is impossible
• Software is complex
• Specifying requirements is hard
• Testing is a fallible, human process
• Consequences of failure not always understood in advance
![Page 6: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/6.jpg)
9/2003 Testing Concepts 6
WORKING SMART
• Test small pieces
• Use proven techniques
• Use tools to advantage
• Start early
• Cost of errors grows exponentially with time they go undetected
![Page 7: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/7.jpg)
9/2003 Testing Concepts 7
Agenda
• What is Testing
• Testing Lifecycle
• A Framework for Testing
• A Complete Example
![Page 8: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/8.jpg)
9/2003 Testing Concepts 8
A TESTING LIFECYCLE
Analysis
Design
Implementation
Execution
Evaluation
Specification
Test Strategy/Plan
Test Script, Data, Driver
Defect DataProblem Reports
Test Results
Test Cases
![Page 9: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/9.jpg)
9/2003 Testing Concepts 9
TESTING vs DEVELOPMENT
• Work broken into stages
• Require similar skills
• Require management oversight
• Produce artifacts used to assess quality of the work
• Test artifacts are overhead, not part of delivered system.
![Page 10: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/10.jpg)
9/2003 Testing Concepts 10
TEST ARTIFACTS
• Strategy/Plan (must know techniques)
• Test cases – central artifacts
• Test machinery to apply test cases
• Data showing results
• Reports that interpret test results
• List of problems/defects to be fixed.
![Page 11: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/11.jpg)
9/2003 Testing Concepts 11
TESTING IS HIERARCHICAL
• Unit testing – smallest components
• Aggregate testing – assemblies of units
• Integration testing – combinations of assemblies
• System testing – system built from assemblies
![Page 12: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/12.jpg)
9/2003 Testing Concepts 12
TESTING OBJECTIVES
• To demonstrate conformance
• To find defects
• To measure behavior in specific environments• Load / stress testing
• Performance testing
• To measure trustworthiness/reliability
![Page 13: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/13.jpg)
9/2003 Testing Concepts 13
Agenda
• What is Testing
• Testing Lifecycle
• A Framework for Testing
• A Complete Example
![Page 14: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/14.jpg)
9/2003 Testing Concepts 14
REQUIRED ELEMENTS OF A TESTING METHODOLOGY?
• Specification for the software
• Process for designing test cases
• Repeatable process for designing, running and evaluating tests
• Accountable artifacts of testing activities
• Economical use of human, time and computing resources
![Page 15: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/15.jpg)
9/2003 Testing Concepts 15
Agenda
• What is Testing
• Testing Lifecycle
• A Framework for Testing
• A Complete Example
![Page 16: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/16.jpg)
9/2003 Testing Concepts 16
BLACK-BOX TESTING
• Testing based on the specification rather than the implementation.
• Specification defines the expected response(s) to stimuli
SoftwareunderTest
Stimuli Response(s)
![Page 17: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/17.jpg)
9/2003 Testing Concepts 17
BLACK-BOX TECHNIQUES
• Functional testing -- tests the behavior of the software.
• Boundary testing -- tests behavior at the lower/upper bounds of input values
• Random testing -- tests using randomly generated stimuli
• Intuitive (ad hoc) testing -- error guessing
![Page 18: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/18.jpg)
9/2003 Testing Concepts 18
FUNCTIONAL TEST DESIGN METHODOLOGY
• Specification
• Identify behaviors
• Develop test cases
• Write test script
![Page 19: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/19.jpg)
9/2003 Testing Concepts 19
EXAMPLE A(1) Specification
• Compute pay for an hourly employee, given the number of hours worked and the hourly pay rate. Compute overtime at 1.5 times hourly rate for hours in excess of 40.
SoftwareunderTest
Hours Pay
Rate
![Page 20: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/20.jpg)
9/2003 Testing Concepts 20
EXAMPLE A(2) Identify Behaviors
• Case 1: No overtime (Hours <= 40)• Expect Pay = Hours * Rate
• Case 2: Overtime (Hours > 40)• Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)
![Page 21: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/21.jpg)
9/2003 Testing Concepts 21
EXAMPLE A(3) Create Test Cases
• Case 1: No overtime (Hours <= 40)• Use Rate = 10, Hours = 30
• Expect Pay = Hours * Rate = 300
• Case 2: Overtime (Hours > 40)• Use Rate = 10, Hours = 50
• Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)
= 550
![Page 22: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/22.jpg)
9/2003 Testing Concepts 22
EXAMPLE A(4) Write Test Script
StepStimuli Expected Response
Hours Rate Pay =
1
2
30
50 10
10 300
550
![Page 23: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/23.jpg)
9/2003 Testing Concepts 23
A MORE COMPLEX EXAMPLE (B)
• Increased number of behaviors
• Use of decision table to document
behaviors
• Test case generation from decision table
![Page 24: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/24.jpg)
9/2003 Testing Concepts 24
EXAMPLE B(1) Specification
• Compute pay for employee, given the number of hours worked and the hourly pay rate. For hourly employees (rate < 30), compute overtime at 1.5 times hourly rate for hours in excess of 40. Salaried employees (rate >= 30) are paid for exactly 40 hours.
![Page 25: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/25.jpg)
9/2003 Testing Concepts 25
EXAMPLE B(2) Identify Behaviors
• Case 1: Hourly AND No overtime • (Rate < 30) & (Hours <= 40)• Expect Pay = Hours * Rate
• Case 2: Hourly AND Overtime • (Rate < 30) & (Hours > 40)• Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)
• Case 3: Salaried (Rate >= 30) • Expect Pay = 40 * Rate
![Page 26: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/26.jpg)
9/2003 Testing Concepts 26
DECISION TABLE
Condition
c1: Rate < 30 | Y Y N N
c2: Hours <= 40 | Y N Y N
Action
a1: Pay = Straight time | X
a2: Pay = Overtime | X
a3: Pay = Professional | X X
Columns defineBehaviors
![Page 27: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/27.jpg)
9/2003 Testing Concepts 27
EXAMPLE B(3) Create Test Cases
• One test case per column of decision table
• Case 1: Hourly, No Overtime
• Case 2: Hourly, Overtime
• Case 3: Salaried, No Extra Hours
• Case 4: Salaried, Extra Hours
• Order the test cases by column
![Page 28: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/28.jpg)
9/2003 Testing Concepts 28
EXAMPLE B(4) Write Test Script
StepStimuli Expected Response
Hours Rate Pay =
1
2
30
50 10
10 300
550
3 30 40 1600
4 50 40 1600
![Page 29: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/29.jpg)
9/2003 Testing Concepts 29
RULES -- DECISION TABLES
Condition
c1: Rate < 30 | Y Y N N
c2: Hours <= 40 | Y N Y N
Action
a1: Pay = Straight time | X
a2: Pay = Overtime | X
a3: Pay = Professional | X X Use X to select action(s)
Elementaryconditions
Use 'Y', 'N', '-' or space
![Page 30: CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56649f335503460f94c50635/html5/thumbnails/30.jpg)
9/2003 Testing Concepts 30
NEXT TIME
• Test Case Generation Strategies
• Documenting the Test
• Specification Techniques