Степан Максимчук: “effective test design techniques”
TRANSCRIPT
![Page 1: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/1.jpg)
Effective Test Design Techniques
Stepan Maksimchuk,Manual QC Analyst, Edgar-Online
![Page 2: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/2.jpg)
Test Design techniquesTesting
Static
Static Analysis Review
Dynamic
Black-box
Functional Non-Functional
White-box Experience-based
Defect-based
Dynamic Analysis
![Page 3: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/3.jpg)
Black-box techniquesEquivalence PartitioningBoundary Value AnalysisDecision TablesUse CasesState-based testingPairwise testing
![Page 4: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/4.jpg)
Subset A
Subset A 1
Subset A 2
Subset A 3
Equivalence Partitioning
Set Subset A
Subset B
Test Case 1
Test Case 2
Test Case 3
![Page 5: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/5.jpg)
Equivalence Partitioning
X Equivalence Partitioning
X1
X2
Y Equivalence Partitioning
Y1
Y2
Y3
TC1 TC2 TC3
X1 X2 X1
Y1 Y2 Y3
Selecting Test Cases
![Page 6: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/6.jpg)
4999 50001499 1500499 5000 1
Boundary Value Analysis
EP1 EP2 EP3 EP4 EP5
Boundary Values
![Page 7: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/7.jpg)
Conditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Real Account?
Active Account?
Within Limit?
Location?
Actions
Approve?
Call Cardholder?
Call Vendor?
Conditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y Y N N N N Y Y Y Y N N N N
Within Limit? Y Y N N Y Y N N Y Y N N Y Y N N
Location? Y N Y N Y N Y N Y N Y N Y N Y N
Actions
Approve?
Call Cardholder?
Call Vendor?
Decision TablesConditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y Y N N N N Y Y Y Y N N N N
Within Limit? Y Y N N Y Y N N Y Y N N Y Y N N
Location? Y N Y N Y N Y N Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N N N
Call Cardholder? N Y Y Y N Y Y Y N N N N N N N N
Call Vendor? N N N N Y Y Y Y Y Y Y Y Y Y Y Y
![Page 8: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/8.jpg)
Decision Tables - CollapsingConditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y Y N N N N Y Y Y Y N N N N
Within Limit? Y Y N N Y Y N N Y Y N N Y Y N N
Location? Y N Y N Y N Y N Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N N N
Call Cardholder? N Y Y Y N Y Y Y N N N N N N N N
Call Vendor? N N N N Y Y Y Y Y Y Y Y Y Y Y Y
Conditions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y Y N N N N Y Y Y Y N N N N
Within Limit? Y Y N N Y Y N N Y Y N N Y Y N N
Location? Y N Y N Y N Y N Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N N N
Call Cardholder? N Y Y Y N Y Y Y N N N N N N N N
Call Vendor? N N N N Y Y Y Y Y Y Y Y Y Y Y Y
Conditions 1 2 3 5 6 7 8 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y N N N N Y Y Y Y N N N N
Within Limit? Y Y N Y Y N N Y Y N N Y Y N N
Location? Y N ~ Y N Y N Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N N
Call Cardholder? N Y Y N Y Y Y N N N N N N N N
Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y Y
Conditions 1 2 3 5 6 7 8 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y N N N N Y Y Y Y N N N N
Within Limit? Y Y N Y Y N N Y Y N N Y Y N N
Location? Y N ~ Y N Y N Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N N
Call Cardholder? N Y Y N Y Y Y N N N N N N N N
Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y Y
Conditions 1 2 3 5 6 7 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y N N N Y Y Y Y N N N N
Within Limit? Y Y N Y Y N Y Y N N Y Y N N
Location? Y N ~ Y N ~ Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N
Call Cardholder? N Y Y N Y Y N N N N N N N N
Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y
Conditions 1 2 3 5 6 7 9 10 11 12 13 14 15 16
Real Account? Y Y Y Y Y Y N N N N N N N N
Active Account? Y Y Y N N N Y Y Y Y N N N N
Within Limit? Y Y N Y Y N Y Y N N Y Y N N
Location? Y N ~ Y N ~ Y N Y N Y N Y N
Actions
Approve? Y N N N N N N N N N N N N N
Call Cardholder? N Y Y N Y Y N N N N N N N N
Call Vendor? N N N Y Y Y Y Y Y Y Y Y Y Y
Conditions 1 2 3 5 6 7 9
Real Account? Y Y Y Y Y Y N
Active Account? Y Y Y N N N ~
Within Limit? Y Y N Y Y N ~
Location? Y N ~ Y N ~ ~
Actions
Approve? Y N N N N N N
Call Cardholder? N Y Y N Y Y N
Call Vendor? N N N Y Y Y Y
![Page 9: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/9.jpg)
Combining Decision Tables with EPConditions 9
Real Account? N
Active Account? ~
Within Limit? ~
Location? ~
EP
Three mismatch
Number/Name
Number/Expiry
Number/CSC
Two mismatch
Two mismatch
Two mismatch
Result: +7 Test Cases
![Page 10: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/10.jpg)
Combining Decision Tables with EP and BVAConditions 1 2 3 5 6 7
Real Account? Y Y Y Y Y Y
Active Account? Y Y Y N N N
Within Limit? Y Y N Y Y N
Location? Y N ~ Y N ~BVA
EP EP
Zero before limit
Normal after
transaction
At limit after
transaction
Just over limit after
transaction
At limit before
transaction
Max after transaction
Result: +1 Test Case
0 limit limit + 0.01 max
![Page 11: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/11.jpg)
Use CasesE-commerce purchase: Normal Workflow
1. Customer places one or more Items in shopping cart2. Customer selects checkout3. System gathers address, payment, and shipping information from Customer4. System displays all information for confirmation5. User confirms order to System for delivery
Exceptions: Customer attempts to checkout with empty shopping cart; System gives error
message Customer provides invalid address, payment, or shipping information; System
gives error messages as appropriate Customer abandons transaction before or during checkout; System logs
Customer out after 10 minutes of inactivity
![Page 12: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/12.jpg)
Use Cases – creating test case for normal workflow# Test Step Expected Result
1 Place 1 item in cart Item in cart
2 Click checkout Checkout screen
3 Input valid US address, valid payment using American Express, and valid shipping method information
Each screen displays correctly and valid inputs are accepted
4 Verify order information Shown as entered
5 Confirm order Order in system
6 Repeat steps 1-5, but place 2 items in cart, and pay with Visa, and ship international
As shown in 1-5
7 Repeat steps 1-5, but place the maximum number of items in cart, and pay with Mastercard
As shown in 1-5
8 Repeat steps 1-5, but pay with PayPal As shown in 1-5
![Page 13: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/13.jpg)
Use Cases – creating test case for exceptions# Test Step Expected Result
1 Do not place any items in cart Cart empty
2 Click checkout Error message
3 Place item in cart, click checkout, enter invalid address, then invalid payment, then invalid shipping address
Error messages, can’t proceed to next screen until resolved
4 Verify order information Shown as entered
5 Confirm order Order in system
6 Repeat steps 1-3, but stop activity and abandon transaction after placing item in cart
User logged out exactly 10 minutes after last activity
7 Repeat steps 1-3, but stop activity and abandon transaction on each screen
As shown in 6
8 Repeat steps 1-4; do not confirm order As shown in 6
![Page 14: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/14.jpg)
State-Based testing
0-switch
A1 A2 A9
B10 B8 B3
C14 C11 C4
D13 D12 D5
F6 F7
1-switch0-switch
A1 A2 A9
B10 B8 B3
C14 C11 C4
D13 D12 D5
F6 F7
1-switch
A1A1 A1A2 A1A9
0-switch
A1 A2 A9
B10 B8 B3
C14 C11 C4
D13 D12 D5
F6 F7
1-switch
A1A1 A1A2 A1A9
0-switch
A1 A2 A9
B10 B8 B3
C14 C11 C4
D13 D12 D5
F6 F7
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
0-switch
A1 A2 A9
B10 B8 B3
C14 C11 C4
D13 D12 D5
F6 F7
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 1: A1A1A2
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 2: A9B8A1A9B8A2
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 3: A9B10C14C14C4
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 4: A9B10C11D13D13D5
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 5: A9B10C11D12F7A2
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 6: A9B10C11D12F7A1A9B10C11D12F7A9B3
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 7: A9B8A9B10C4
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 8: A9B10C14C11D5
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
Test Case 9: A9B10C11D13D12F6
1-switch
A1A1 A1A2 A1A9 A9B10 A9B8 A9B3
B10C14 B10C11 B10C4 B8A1 B8A2 B8A9
C14C14 C14C11 C14C4 C11D13 C11D12 C11D5
D13D13 D13D12 D13D5 D12F6 D12F7
F7A1 F7A2 F7A9
![Page 15: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/15.jpg)
State-Based testing – test casesTest Case 1: A1A1A2
Test Case 2: A9B8A1A9B8A2
Test Case 3: A9B10C14C14C4
Test Case 4: A9B10C11D13D13D5
Test Case 5: A9B10C11D12F7A2
Test Case 6: A9B10C11D12F7A1A9B10C11D12F7A9B3
Test Case 7: A9B8A9B10C4
Test Case 8: A9B10C14C11D5
Test Case 9: A9B10C11D13D12F6
![Page 16: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/16.jpg)
Pairwise testing – orthogonal arrayFactors
Test A B
1 0 0
2 0 1
3 1 0
4 1 1
Factors
Test A B C
1 0 0 0
2 0 1 1
3 1 0 1
4 1 1 0
AB: 00, 01, 10, 11
Factors
Test A B C
1 0 0 0
2 0 1 1
3 1 0 1
4 1 1 0
Factors
Test A B C
1 0 0 0
2 0 1 1
3 1 0 1
4 1 1 0
Factors
Test A B C
1 0 0 0
2 0 1 1
3 1 0 1
4 1 1 0
AC: 00, 01, 11, 10 BC: 00, 11, 01, 10
![Page 17: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/17.jpg)
Pairwise testing – orthogonal arrayRules for choosing orthogonal array:
There must be at lease as many column as factors (drop any extra columns).
There must be at least enough numbers in the columns to hold the option for each factor (spare numbers change to ‘~’ which is referred as “tester’s choice”).
There must be at least as many rows as the product of the two largest numbers of options (scrutinize all rows, two at a time, to see if they could be compressed using “tester’s choice”).
![Page 18: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/18.jpg)
Pairwise testing – orthogonal arrayRules for applying orthogonal array:
Drop any extra columns.Map factors to the columns by adding columns headings.Select one column at a time and map the options for that
factor onto the numbers.If you have to many rows:
Drop any extra rows with no interesting single options or pairs of options and compress rows.
Fill the tildes with any option that you like or leave it for tester to decide during execution.
![Page 19: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/19.jpg)
Pairwise testing - exampleFactors:
Connections speed: Dial-Up and BroadbandOperating System: Mac, Linux, Windows 7 and 8Security: Native OS, Symantec, Trend, McAfeeBrowser: Firefox, IE, Opera
![Page 20: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/20.jpg)
Pairwise testing - exampleFactor
Test Speed OS Security Browser
1 Dial-Up Mac OS Firefox
2 Dial-Up Linux Symantec IE
3 Dial-Up Win7 Trend Opera
4 Dial-Up Win8 McAfee ~
5 Broadband Mac Symantec Opera
6 Broadband Linux OS ~
7 Broadband Win7 McAfee Firefox
8 Broadband Win8 Trend IE
![Page 21: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/21.jpg)
Pairwise testing - exampleFactor
Test Speed OS Security Browser
9 ~ Mac Trend ~
10 ~ Linux McAfee Opera
11 ~ Win7 OS IE
12 ~ Win8 Symantec Firefox
13 ~ Mac McAfee IE
14 ~ Linux Trend Firefox
15 ~ Win7 Symantec ~
16 ~ Win8 OS Opera
![Page 22: Степан Максимчук: “Effective Test Design Techniques”](https://reader038.vdocuments.mx/reader038/viewer/2022103105/587831ed1a28abef5d8b6c79/html5/thumbnails/22.jpg)
Thank you