what makes software dependable? martyn thomas cbe freng independent consultant / expert witness

What makes software dependable?

Martyn Thomas CBE FREngindependent consultant / expert witness

http://www.thomas-associates.co.uk

Software projects are not dependable Standish “Chaos Chronicles” (2004 edition):

18% of projects “failed”; (cancelled before completion) 53% of projects “challenged” (operational, but over budget

and/or over time with fewer features or functions than initially specified…)

Typical Standish figures: Cost overruns on 43% of projects; and Time overruns on 82% of projects.

Oxford University/Computer Weekly 2003 study:

10% of UK projects “failed”; and 75% of UK projects “challenged”.

Software products are often not dependable Security vulnerabilities Safety-critical faults Requirements errors Programming mistakes

Security vulnerabilities Usually poor programming, such as

allowing a program to read over-length data and overwrite itself. Weak languages such as C allow such

errors. The Code Red and Slammer worms

exploited this sort of error in Microsoft products, and did more than $3B of damage worldwide.

Safety Critical Faults Erroneous signal de-activation. Data not sent or lost Inadequate defensive programming

with respected to untrusted input data Warnings not sent Display of misleading data Stale values inconsistently treated Undefined array, local data and output

parameters

More safety critical faults-Incorrect data message formats-Ambiguous variable process update-Incorrect initialisation of variables-Inadequate RAM test-Indefinite timeouts after test failure-RAM corruption-Timing issues - system runs backwards-Process does not disengage when required-Switches not operated when required-System does not close down after failure-Safety check not conducted within a suitable time frame-Use of exception handling and continuous resets-Invalid aircraft transition states used-Incorrect aircraft direction data-Incorrect Magic numbers used-Reliance on a single bit to prevent erroneous operation

Errors found in C130J software after certification.

Source: Andy German, Qinetiq. Personal communication.

Requirements Errors what happened Airbus A320, Warsaw 1993 aircraft landed on wet runway aquaplaned, so brakes didn’t work pilot applied reverse thrust, but disabled why airborne ⇔ disabled airborne ⇔ not WheelPulse ⇔ disabled simplified; for full analysis, see [Ladkin 96]

Coding Errorstype Alert is (Warning, Caution, Advisory);function RingBell(Event : Alert) return Boolean-- return True for Event = Warning or Event = Caution, -- return False for Event = Advisoryis Result : Boolean;begin if Event = Warning then Result := True; elsif Event = Advisory then Result := False; end if; return Result;end RingBell;-- C130J code: Caution returns uninitialised (usually TRUE, as

required).

What makes a Project dependable?

BUSINESS ISSUES The business requirements must be

properly understood The business changes and risks must be

planned, budgeted and managed competently

Requirement changes must be kept under control and budgets and timescales must be adjusted to reflect any changes

Stakeholder conflicts must be resolved before the computing project starts

What makes a Project dependable?

REQUIREMENTS ISSUES

The computing requirements must be correctly derived from the business, formalised and analysed for omissions and contradictions Most requirements changes are NOT changes to the

business needs, but things that were overlooked when the contract was placed.

Every requirements change adds cost and delay, and transfers risk back from the supplier to the customer.

Beware Output Based Specifications and Agile Methods. They should only be used where the risks are well understood and accepted by the customer.

What makes a Project dependable?

COMPUTING ISSUES

The computing development and risks are planned and managed competently

Strong software engineering methods and tools are used to develop the software and show that it has the required properties.

Testing is used to show that the assumptions are correct not that the software is correct!

Testing software tells you that the tests work – not that the software works

Continuous behaviour means you can interpolate between test results

Discrete behaviour means that you can’t!

What makes a product dependable? Explicit, unambiguous requirements, with the

critical properties clearly identified Development using strong, science-based

software engineering Rigorous quality management Auditable evidence that the critical

requirements have been met Software built around COTS components will

often fail these criteria Cheap, dependable and a good fit to your

requirements. Pick two.

How do you get the right technical solution to a business requirement?

USE AN ARCHITECT!

See the Royal Academy of Engineering report on complex IT Systems.

Role of the Systems Architect Help the customer to understand the business

requirements and possibilities Propose appropriate and technically feasible high-

level solutions (architectures) Help resolve stakeholder conflicts and agree

requirements and architecture Complete and FORMALISE the technical specification

This will eliminate most requirements risk. Manage supplier selection Manage the supply contract for the customer Manage requirement changes Manage the user acceptance phase

Then use Correct by Construction development

SPARKImplementation

INFORMEDDesign

Formal Design

FormalSpecification

SecurityProperties

Proof of SecurityProperties

(Z)

Proof of FormalSpecification

(Z)

Refinement Proofof Formal Design

(Z)

Proof of SecurityProperties

(SPARK Proof)

Proof ofFunctionalProperties

(SPARK Proof)

Static Analysis

AssuranceActivity

KeySystem Test

System TestSpecification

Highly productive

Ada Source Lines

Spark annotations

LOC/day

Core 9,939 16,564 38

Support 3,697 2,240 88

Other Statistics from NSA Tokeneer experiment Total effort 260 man days Total cost – $250k Total schedule – 9 months Team – 3 people part-time Testing criterion – 99.99% reliability

with 90% degree of confidence Total critical failures – 0 [Yes, zero!]

WHY use Correct by Construction S/W Engineering ?

“Meets Common Criteria and ITSEC security requirements for EAL5 +”

“Produces code more quickly and reliably and at lower cost than traditional methods”

“Is commercially supported (ORA Canada, Praxis HIS, Pyrrhus Software, SPRE Inc.)”

“Reasonable learning curve” “C by C is proven and practical”

These are NSA quotes.

CONCLUSIONSoftware projects can be dependable and can deliver dependable software.But this depends on a robust specification and strong correct by construction software engineering.Plan, budget and manage the business changesUse an independent system architectPrefer proof above testing for critical properties

Useful referenceshttp://www.praxis-his.com/publications/

http://www.raeng.org.uk/news/publications/list/reports/Complex_IT_Projects.pdf

http://www7.nationalacademies.org/CSTB/project_dependable.html

http://www.ukcrc.org.uk/resource/reports/index.cfm