Topic 7 : TestingSoftware Development Techniques

Learning Objectives

• Develop test cases

• Make use of black box testing

• Make use of white box testing

• Make use of unit testing

• Make use of integration testing

Why Test?

• No matter how well software has been designed and coded, it will definitely still contain defects

• Testing is the process of executing a program with the intent of finding faults (bugs)

• Testing can increase the confidence that a program “works”.

Formal Testing

• Testing helps us determine if a program is correct.

• But what do we mean by ‘correct’?

• For computer programs, there is a range of criteria we can apply to determine correctness:

• It must handle all the required functionality for all valid data.

• It must not generate unexpected behaviour.

• It must respond appropriately to all anticipated data.

Testing

• We need a structured approach to testing.

• We decide on a set of test data.

• We run that test data through the algorithm.

• We record the expected results and the actual results.

• If all the tests in a set pass, then the program is considered correct.

• If any fail, you go back to the algorithm, identify and fix the problem, and then run all the tests again.

Test Sets

• A test set consists of a series of customized test cases.

• Each of these consist of a set of input values for a function.

• The number of cases in a set will vary depending on:• Function complexity

• Amount of confidence of correctness required• Feasibility

• Running a test set involves:• Desk-checking the function with each set of inputs.

• Recording actual results versus expected results.

Test Strategies

• Two techniques exist for handling the comparing of test data versus expected results.

• Black-box testing

• White-box testing

Black Box Testing

• Black Box Testing, also known as Behavioral Testing, is a software testing method in which the internal structure / design/ implementation of the item being tested is not known to the tester.

• This method is named so because the software program, in the eyes of the tester, is like a black box; inside which one cannot see.

Black Box Testing

• This method of attempts to find errors in the following categories:

1. Incorrect or missing functions

2. Interface errors

3. Behavior or performance errors

• Example: A tester, without knowledge of the internal structures of a website, tests the web pages by using a browser; providing inputs (clicks, keystrokes) and verifying the outputs against the expected outcome.

Black Box Testing

• In black-box testing, we do not consider how a result was obtained. All we consider are the inputs and the outputs.

• To perform a black-box test, we do not even look at the code within a function.

• We just pass in the input and then record the results.

• It is impossible to thoroughly test a program, so we must choose data that is representative of other data.

• But, we must also consider special cases where errors are possible.

White Box Testing

• also known as Clear Box Testing, Open Box Testing, Glass Box Testing, Transparent Box Testing, Code-Based

• It is a software testing method in which the internal structure / design / implementation of the item being tested is known to the tester.

• The tester chooses inputs to exercise paths through the code and determines the appropriate outputs.

White Box Testing

• Programming know-how and the implementation knowledge is essential.

• White box testing is testing beyond the user interface and into the fundamentals of a system.

• This method is named so because the software program, in the eyes of the tester, is like a white/transparent box; inside which one clearly sees.

White Box Testing

• Example

• A tester, usually a developer as well, studies the implementation code of a certain field on a webpage, determines all legal (valid and invalid) AND illegal inputs and verifies the outputs against the expected outcomes, which is also determined by studying the implementation code.

White Box Testing

• It is used to evaluate the flow of logic through a program.

• White box testing requires working through the code.

• Test cases though are chosen to test all possible flows of execution through a program.

• The philosophy behind white-box testing is that errors are just as likely on ambiguous branches of logic that may not be stressed by black-box testing

Black Box Testing and White Box Testing

• These are complementary techniques - you do not choose to use one over the other, you use both to prove correctness in different areas.

• The two techniques together give great confidence about a program’s correctness.

• Black box testing shows that for a given set of test data, the right results will come out.

• White box testing shows that for data on all possible flows of execution, the right results come out.

Documenting a Test Case

• In both types, each test case should be recorded in a table like so:

• For each parameter, include a column for the input:

Case Input 1 Input 2 Expected Actual

1 10 20 30

2 20 5 25

3 -10 -20 -30

4 -5 100 95

How to Decide on Test Cases

• Deciding on test-cases can be complex - there is a strategy to it. Choosing 1, 2, 3, 4, and 5 gives you five test cases, but not necessarily good ones.

How to Decide on Test Cases

• Test cases should be aimed at the areas that are most likely to generate problems.

• Although you also need to make sure that there are enough to cover a wide range of possible inputs.

• To begin with, we handle equivalence partitions.

• For the range of your possible inputs, partition it into categories.

• Positive and negative, for example.

• Pick two or three test cases that cover each partition.

Equivalence Partitioning

Case Input 1 Input 2 Expected Actual

1 20 40 60

2 30 60 90

3 -10 -15 -25

4 -72 -8 -80

5 52 -10 42

6 28 -17 11

7 -13 25 12

8 -80 50 -30

How to Decide on Test Cases

• Next, we want test data that stresses areas where bugs disproportionately cluster:

Boundaries

Maximums and minimums

Invalid input

Boundary Testing

• Boundaries are those areas where errors are very likely to creep in because of the transition between sets.

• Between positive and negative

• Between valid and invalid

• Between a condition being true and false.

• These areas are often fruitful tests, because programmers often make errors in boundary checking.

Maximums and Minimums

• These do not apply as confidently to pseudo code as to real programs.

• Real programs run with real, finite limits to how they can represent numbers.

• A good program will handle this.

• If the maximum whole number the system can hold is 1000, what happens if you try to add 999 and 999?

• You should have test cases that determine whether or not that is true.

Unit Testing

• If everything is contained within a single program, you must have test cases to handle every possible combination of every possible set of input for every possible calculation.

• That is a lot of testing.

• However, we can test functions in isolation.

• We just plug inputs into them and record the output.

• This is called unit testing.

• We generate test sets and desk-check each function.

Unit Testing

• Unit testing is a software development process in which the smallest testable parts of an application, called units, are individually and independently examined for proper operation.

• Unit testing is often automated but it can also be done manually.

• Unit testing does not give us confidence that the program is working.• But it gives us confidence that a particular function is working.

Unit Testing

• We test each function in isolation as part of our testing.

• We do not test the whole program, just its various functions.

• Only when we are sure each function works by itself do we link them up.

Integration Testing

• Integration Testing is the phase in software testing in which individual software modules are combined and tested as a group.

• It occurs after unit testing and before validation testing.

• Integration testing is used to test the link between our main program and each of its functions.

• It maps the inputs of the program onto the outputs of the function, and how that output is then fed into other functions.

Integration Testing

• If we know our functions are correct, then we know that any errors that are encountered are in the communication between the functions.

• In integration testing, we test functions in ever increasing chains.

• We make sure function A and function B work together, and then we make sure A and B and C work together.

• If the program stops working, it was as a result of the last thing you added.

Integration Testing

• For integration testing, we run the test program for a series of test cases just as with white- and black- box testing.

• Making a note of expected outputs and actual outputs.

Unit and Integration Testing

• Unit testing lets us determine if a single function works correctly.

• Integration testing lets us test the communication between functions.

• Unit testing happens first, and then integration testing is used to make sure each function communicates properly with the rest of the program.

• Used together, they give us confidence that when placed in proper context, the program will work correctly.

• And if it does not, we know what is causing the problem.

Conclusion

• Algorithms can be very complicated.

• And making sure they work is just as complicated.

• We need an effective strategy for testing to help manage this.

• Otherwise, we have no way of telling what is causing problems other than sitting and staring.

• Black-box and white-box testing let us test our units.

• Integration testing lets us test how our units communicate.

Terminology

• Test Case

• A set of input data that is mapped onto both an expected and an actual result from a test.

• Test Set• A series of test cases

• Black-box testing• Testing the results that come from a unit

Terminology

• White-box testing

• Testing the flow of execution through a unit

• Unit testing

• Testing of a single function in isolation

• Integration testing

• Chain together working functions to test their communication.

References

• http://softwaretestingfundamentals.com/black-box-testing/

• http://softwaretestingfundamentals.com/differences-between-black-box-testing-and-white-box-testing/

End of Topic 7Software Development Techniques