verification and validation · 26.11.14 b. wolff - gla - resume 10 some relevant standards (recall)...

2014-2015

Verification and ValidationPart 15 : A Resume

Burkhart WolffDépartement Informatique

Université Paris-Sud / Orsay

26/11/14 B. Wolff - GLA - Resume 2

What is Software Engineering ?

" Methods, techniques and tools for# design: requirement analysis, models,

specifications# development: programmation, integration# validation: prototypes, testing# verification: formal proof of required properties# maintenance: reusability, improvements

" A slightly longer answer:



" ... slightly longer answer:

The discipline of software engineering was created to address poor quality of software, get projects exceeding time and budget under control, and ensure that software is built systematically, rigorously, measurably, on time, on budget, and within specification. [Wikipedia [en]]

" Or much shorter:

SE addresses the problems of

« Development in the Large » ... so for teams with 100 or 1000 of developers, and budgets of sometimes billions of dollars.



" ... slightly longer answer:

Reminder: What is it, when I talk about " Or much shorter:

SE addresses the problems of

« Development in the Large » ... so for teams with 100 or 1000 of developers, and budgets of sometimes billions of dollars.



" ... slightly polemically:

Reminder: What is it, when I talk about Software Engineering ? Writing:

! “Write-once, throw away” programs ?! Programs written by a small team with 50 KLoCs?! Our LRI Website-Service

OR:

! programs in industrial context with large institutions (states, companies) commissioning it ?


Answers to the Problem of Software Engineering

" Development Processes

" Standards on Processes, Documentation, and organization

" Process-support# Version management

(automatic merge problem)# Configuration management# Build Management# Analysis-Techniques of the merges

26.11.14 B. Wolff - GLA - Resume 7

2013-14 L2-GL - Intro 12

One way to view Software Engineering Project

MANAGEMENT PROCESS

Development Mgt. Risc Management

ConfigurationManagement

« PeopleWare »(Staff, Sub-contractors)

PROCESSUS QUALITE

QUALITYASSURANCE

QUALITYCONTROL

METRICS

FEASIBILITY STUDIES (BEFORE PROJET)

SPECIFICATION & DESIGN

PRODUCTION

INTEGRATION & VALIDATION

EXPLOITATION

TECHNICAL PROCESS

Development

Maintenance & Support


How can software be «built systematically»?" A Variant: The VPM3-Model (Daimler)






(automatic merge problem)# Configuration management# Build Management# Analysis-Techniques of the merges# Verification and Validation Techniques

26.11.14 B. Wolff - GLA - Resume 10

Some Relevant Standards (Recall)

" Many attempts to control development processes andsoftware products by standards (norms)

" Some attempts to assure and certify software quality.# Most serious and relevant (in France):# DO 178B (Avionics)# ISO 29119 (Software Test) # ISO 15408 « Common Criteria» for computer

security certification requiring formal models as well as proof techniques for EAL 6 and EAL 7.






(automatic merge problem)# Configuration management# Build Management# Analysis-Techniques of the merges# Verification and Validation Techniques


Concepts of Central. Version Control(Recall)

" Working copises(in user space)

" Repository(on the server-side)

" update:syncing with therepository

" commit: creating a new revision of a document(involves new registration, inclusion in documents, consistency checks)

" operations lock, checkout, import, ...


Build Management: A Build-Server (Recall)

"


The Problem of Software Engineering

" In a large process, automatic support formerges of documents and code becomes vital

" This raises the problem of consistency of products:documents and code

" Answers we know of:# . . .


Answers to the Problem of SE

" Answers we know of:# more Formal Modeling (in Analysis and Design)# Static Analysis-Techniques of the merges

(from type-checking in UML and code,to MISRA code style checkers)

# Verification and Validation Techniques! Black box tests! White box tests! Deductive Verification


A first Glance to an Example: Bank

Opening a bank account. Constraints:" there is a blacklist" there is a present of 15 euros in the initial account" account numbers must be distinct.


A first Glance to an Example: Bank (2)

" context Compte:inv unique: Compte.allInstances->isUnique(c | c.no)

context Banque::ouvrirCompte(nomC: String) : Integer

pre: Personne.allInstances->forall(p | p.nom <> nomC)post: Personne.allInstances->isUnique(p | p.nom = nomC

and p.oclIsNew() and )

and Compte.allInstances ->isUnique(c | c.titulaire.nom = nomC and c.oclIsNew() and c.solde = 15)and Personne.allInstances ->includesAll(Personne.allInstances@pre)and Compte.allInstances ->includesAll(Compte.allInstances@pre)


Static Analysis (Recall)

" Modern Static Analysis Methodsare typically a combination of the following techniques# Data-Flow, Control-flow, Patterns, (“Classics”)# Symbolic Execution (“forward analysis”)# Deductive Verification (wp like, “backw. ana.”)# Model-Checking# Abstract Interpretation # Predicate Abstraction,# Interpolant Construction


Testing Methods (Recall)

" A taxonomy on types of tests# Static Test / Dynamic (Runtime) Test # Structural Test / Functional Test# Statistic Tests

" Functional Test; Link to UML/OCL# Dynamic Unit Tests, Static Unit Tests,

# Coverage Criteria

" Structural Tests# Control Flow and Data Flow Graphs

# Tests and executed paths. Undecidability.

# Coverage Criteria



" It is an approximation to full verification (for ex. by proof)" Main emphasis: finding bugs early,

# either in the model# or in the program# or in both

" A systematic test is:# process programs and specifications

and to compute a set of test-cases under controlled conditions.

# ideally: testing is complete if a certain criteria,the adequacy criteria is reached.

26/11/14 B. Wolff - Ingé. 2 - Proof-Based Verification II 27

Hoare – Logic: A Proof System for Programs

" Hoare revisited (i):

" ... this part is actually highly deterministic


Hoare – Logic: A Proof System for Programs

" Hoare revisited (ii):

" ... this part needs some work, and some new ideas.


Tools: gwhy and Squareroot


Resume : Test vs. Proof

Test

# Requires Testability of Programs (initializable,reproducible behaviour, sufficient control over non-determinism)

# Can be also Work-Intensive !!!

# Requires Test-Tools, whose details can be very complicated

# Requires a Formal Specification

# Makes Test-Hypothesis, which may be hard to justify !


Resume : Test vs. Proof

Deductive Verification

# Needs knowledge / assumptions on the precise execution of the language and the the environment(may be difficult in practice ... gcc -o3 !!!)

# Is work-intensive and intellectually expensive !!!

# Requires Tools, whose details can be complicated

# Requires a Formal Specification

# Currently restricted to abstract models or realistic C with code-size 10kloc.)


An Example for an Extended Static Checking (Combi-) Method: SAL Annotations

# Due to heavy machinery(heuristics, patterns, abstract interpretation ...)

the annotation of loops with invariantsis not necessary by the user for the SALlanguage.

The programmer has just to provide contracts.

Since 2006, Microsoft annotates the entireWindows and Word code-base with SAL.


An Example for an Extended Static (Combi-) Checking Method: SAGE

# ... a white box fuzz - tester.


Static Analysis : (recall)

" Lots of Methods with the same Goal:

More automation, better prediction of “real errors”

(i.e. less false-positives)better explanation of “real errors”

(very difficult)less constraints on the supported language subset

(“real” C, “real” Java) trend to “dirty languages”such as JavaScript


Global Summary

" Current Research and Trends:

# Combination of Methods

# Test and Proof have a lot in common

# A lot of theory needs to combined with heavy technology

verification and validation · 26.11.14 b. wolff - gla - resume 10 some relevant standards (recall)...

Documents