development testing

Development testing Yury Kisliak @ EIS Group

- What is testing- Kinds of testing- Kinds of coverage- Asserts

What is testing?

What is testing?

What is testing?

What is testing?

What is testing?

What is testing?

What is testing?

Good testing consist of selecting a good set of test inputs and designing a good acceptability checks.

What is going on when we test?

What is going on when we test?

-A lot!

What is going on when we test?

What is going on when we test?

What is going on when we test?

What is going on when we test?

Why testing?

- Testing shows presence of defects (but cannot prove that there are no defects).

- Appropriate testing reduces the probability for defects.

- What is testing- Kinds of testing- Kinds of coverage- Asserts

Types of testinghttps://en.wikipedia.org/wiki/Software_testing#Testing_types

Types of testinghttps://en.wikipedia.org/wiki/Software_testing#Testing_types

Testing Levels

Testing Levels

Testing Levels

Testing Methods

Testing Methods

Static

- Reviews, walkthroughs, inspections.

- Static program analysis.

Testing Methods

Static

- Reviews, walkthroughs, inspections.

- Static program analysis.

Dynamic

- Executing programmed code with a given set of test cases

Testing Methods

Static

- Reviews, walkthroughs, inspections.

- Static program analysis.

Dynamic

- Executing programmed code with a given set of test cases

Black box

- Examining functionality without any knowledge of internal implementation.

White box

- Internal perspective of the system, as well as programming skills, are used to design test cases.

Testing Methods

Static

- Reviews, walkthroughs, inspections.

- Static program analysis.

Dynamic

- Executing programmed code with a given set of test cases

Black box

- Examining functionality without any knowledge of internal implementation.

White box

- Internal perspective of the system, as well as programming skills, are used to design test cases.

- What is testing- Kinds of testing- Kinds of coverage- Asserts

Coverage? What coverage?

Coverage? What coverage?

Coverage? What coverage?

Coverage? What coverage?

Coverage? What coverage?

Equivalence partitioning

Equivalence partitioning

Equivalence partitioning

Equivalence partitioning

Test Coverage

Test Coverage

Test Coverage

Test Coverage

Test Coverage - is proportion of a program, exercised during testing.

+ gives objective score

+ when coverage is <100%, we are given a meaningful task

- not god in finding “errors of omission”

- difficult to interpret scores <100%

- 100% coverage does mean that all bugs are found

Coverage metrics

Coverage metricsHow many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

Coverage metricsHow many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Coverage metricsStatement (line) coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Coverage metricsStatement (line) coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

100%

Coverage metricsStatement (line) coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

100%

Coverage metricsStatement (line) coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

100% 50%

Coverage metricsBranch coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Coverage metricsBranch coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

50%

Coverage metricsBranch coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

50% 50%

Coverage metricsBranch coverage: How many coverage metrics

are out there?- A lot!

http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

50% 50%+ = 100%

Coverage metrics ✓ Line coverage Branch coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Loop coverage:

Execute each loop zero, one and more than one times.

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

a, !b, c => true

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

a, !b, c => true!a, !b, c => false

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

a, !b, c => true!a, !b, c => false!a, b, c => true

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

a, !b, c => true!a, !b, c => false!a, b, c => true!a, b, !c => false

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Modified condition/decision coverage:

MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision

a, !b, c => true!a, !b, c => false!a, b, c => true!a, b, !c => false

MC/DC is used in avionics software to ensure adequate testing of the most critical (flight and landing related) parts.

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Path coverage:

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Path coverage:

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Path coverage:

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

Path coverage:

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓ Path coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

What else:

Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓ Path coverage✓

How many coverage metrics are out there?

- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)

How many of them do we need to care about?

- Very few.

What else:

- boundary value coverage- synchronization coverage- … other 94 and more coverages

How to use coverage

How to use coverage

How much testing is enough?

Tools

Tools

- What is testing- Kinds of testing- Kinds of coverage- Asserts

Assertions

Assertions

- makes code self-checking, leading to more effective testing

- make code fail early

- document assumptions, preconditions, postconditions and invariants.

- are not for error handling

- must not have side effects

Assertions

- makes code self-checking, leading to more effective testing

- make code fail early

- document assumptions, preconditions, postconditions and invariants.

- are not for error handling

- must not have side effects

Use error handling code for conditions you expect to occur. Use assertions for conditions that should never occur.

Assertions

- makes code self-checking, leading to more effective testing

- make code fail early

- document assumptions, preconditions, postconditions and invariants.

- are not for error handling

- must not have side effects

Use error handling code for conditions you expect to occur. Use assertions for conditions that should never occur.

The end.

Testing by itself does not improve software quality. Test results are an indicator of quality, but in and of themselves they don’t improve it. Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. What you eat before you step on to the scale determines how much you will weigh, and the software-develompent techniques you use determine how many errors testing will find. If you want to lose weight, don’t buy a new scale; change your diet. If you want to improve your software, don’t just test more; develop better.

-Steve McConnell