Evaluation of Safety Critical Software

by D. Parnas, A. J. van Schouwen, and P. Kwan

Presented by:

Hatem Halaoui

Presentation Outline Paper

Introduction Why is software used? Software controllers and other controllers Software concerns Modular Structure Reliability Assessment for Safety-critical software Conclusions

Uncovered issues Statistical software crisis information due to bad software

engineering

Introduction The failure of programmable computer applications could

be life threatening. Computers now have safety-critical functions.

To reduce application failures these questions need some answers: what standards must a software product satisfy? What documentation is needed? How much testing is needed? How should software be structured?

Why is software a special concern? Many tragedies were caused by software failures. Software systems do not work well until they have been

used and have failed repeatedly Software products may fail in their first use because

situations that were anticipated by the programmers were also overlooked by the test planners

The use of randomly generated data reduces the likelihood of shared oversight

Why is software used?

Software makes it practical to build more logic into the system. They can distinguish a large number of situations and provide suitable outputs.

Logic implemented in software is easier to change than that implemented in hardware

Software provide more information in more suitable form

How are software controllers like other controllers?

Similar to other controllers (hardware), software controllers has the following properties:

1. Inputs can be described as mathematical functions

2. Outputs can be described as mathematical functions of the inputs

3. Values of the variables can also be described as mathematical function of the controller output

4. Relations between variables can be described

How are software controllers different from other controllers?

Complexity: Programs need more precise documentation which might fill a book case programs need much more time (years ) before being trusted

Error Sensitivity: software is much more sensitive to small errors. Hard to test:

Huge number of testing cases in software what about points between the testing points

Correlated failures: Assumptions for hardware design are invalid for software: like assuming that a hardware

failures are not strongly correlated In contrast to hardware, duplicating software components does not imply higher reliability

Lack of professional standards No professional software engineers No agreement on the skills and knowledge that a software engineer should have

Software Testing Concerns We cannot test software for correctness: large number of states It is difficult to make accurate predictions of software reliability and

availability It is not practical to measure the trustworthiness of software: a

software is trustworthy if the probability of having a potentially catastrophic flaw is acceptably low.

There is a role for testing: some scientist argue that one should test rather than spending this time review and does mathematical verifications

There is a need for an independent validation agency: it is difficult to test one’s own design in an unbiased way

Software Reviewability Concerns

Why is the reviewability concern for software?Before, there were not too much care about software,

its documentation, and its trustworthinessNow, much more software controllers in big

equipments and industryReviews are very important to increase the correctness

of softwareEngineers are required to do documentations

What Reviews are needed Review for the correct intended function Review for maintainable, understandable, well

documented structure. Review each module to verify the algorithm and date

structure design are consistent with the specified behavior Review the code for consistency with the algorithm and

data structure design. Review test adequacy: was the testing sufficient to

provide sound confidence in the software functioning.

What documentation is required to review functional requirements?

Should be done by engineers and scientists who understand the situation to be monitored and devices to be controlled.

The functional requirement can be stated by giving three mathematical functions:

1. The required values of the controlled variables in terms of values of the relevant observable environmental parameters

2. The computer inputs in terms of those observable environmental variables

3. The values of the controlled environmental variables in terms of the computer output

This can be given as a set of tables and formulae

What documentation is required to review software structure?

Documents that describe the breakdown of the program into modules (each module is a unit which could be a set of programs).

The purpose of this review is to make sure that:1. The structure allow independent development and change2. All programs need are included once and only once3. Module interfaces are defined4. Modules are compatible to each other and meet the requirements

Three types of documents are needed:1. Requirements and specifications2. Informal document describing responsibilities of each module3. Module specification (black box description)

What documentation is required to review module’s internal design?

Design documentation which include description of two types of mathematical functions: program functions and abstraction functions

Programs must be described in a hierarchical way where each small program or subprogram is described

This needs software engineers and specialist in the areas that the software is dealing with

What documentation is required to review the code?

Algorithms and data structures The reviewers should be experienced users of the

hardware and compilers

What documentation is required to review the test plan review?

Mathematical verification of the code The test plan should be described in a document

not available for designers Random testing is required Should be reviewed by specialists in software

testing and application area

Reviewing relationships between these documents

Relationship between these documents should be verified Main issues about documents relationships

Module guide should show no ambiguity about module’s responsibilities

Each module design document should show arguments that verify the module specifications

The module design document, which describes the algorithms should be mapped on the code

Test plans must show how the tests are derived and how they cover the requirements

Why is configuration management essential for rigorous reviews

All documents must be kept consistent when changes made.

If document is changed, all related documents must be reviewed and possibly change.

Designers and reviewers should be notified of changes and have the latest versions of documents

Modular Structure Software should be organized in accordance with the

modern principles of “information hiding”, “Object Orientation”, “Separation of Concerns”, “Encapsulation”, “Data Abstraction”, etc.

Large programs should be organized in smaller and simpler assignments known as modules.

Software engineers should be trained to use abstract mathematical specifications

Details that change a lot should be hidden

Reliability Assessments for Safety-Critical Software

Some engineers argue that the correctness of a software is not probabilistic

Not true!! Software appears to exhibit stochastic properties

We can never say that a software is 100% reliable

What should be measuring? Encountering the sequence of input that lead to failures Different kinds of software needs different measuring Example

In Safety-critical applications errors are not acceptable

We cannot predict the failure rate from failure rates of individual lines or subprograms (a failure in a subprogram might lead into a failure in others)

The finite state machine model of programs

Software is a machine that is described by two functions: Next State and Output

Software can be viewed as a finite state machine described by large tables

Loading a program in the machine selects a terminal submachine consisting of all states can be reached from the initial state

It helps to know the number of tests we need to perform

Use of hypothesis testing

We need to confirm that failure probability is below upper bound

Assume the upper bound probability of a failure is 1/h => reliability is 1-1/h

For N random tests the probability there will be no failure during testing is M=(1-1/h)N

Hypothesis testing example

Reliability estimates of programs Three classes of programs:

1. Memoryless batch programs (each run is independent of others)2. Batch programs with memory (a run might depend on previous one)3. Real time programs (run continuously)

Reliability of memoryless batch programs : tests consist of single run for each set of inputs

Reliability of batch programs: test consist of single run of input data and internal state

Reliability of Real time programs: when testing the concept of inputs must be replaced with a multidimensional

trajectory. Each trajectory gives the input values as continuous functions of time Length of trajectories should be estimated

Conclusions Information hiding should be used Documentation should be complete, precise and making

use of mathematical notations rather than natural language Mathematical verification techniques must be used An independent agency should test the software using

valid random testing to get estimates of the reliability of the software at critical situations.

Hypothesis testing should be used to allow us to know the probability that the software meet the requirements

Uncovered Issues

One of the most important issues that lead into software crisis is the errors resulting from discreatizing continuous data

Such data will not be accurate Operations applied on multiple of such data will

lead to high loss of precision

Software Crisis Examples Source: http://www.unt.edu/benchmarks/archives/1999/july99/crisis.htm Software crisis:

The cost of owning and maintaining software in the 1980’s was twice as expensive as developing the software.

During the 1990’s, the cost of ownership and maintenance increased by 30% over the 1980’s. In 1995, statistics showed that half of surveyed development projects were operational, but

were not considered successful. The average software project overshoots its schedule by half. Three quarters of all large software products delivered to the customer are failures that are

either not used at all, or do not meet the customer’s requirements. Main reasons

Software development is seen as a craft, rather than an engineering discipline. The approach to education taken by most higher education institutions encourages that "craft"

mentality.