1 jan tretmans university of nijmegen [email protected] © jan tretmans university of nijmegen...
Post on 20-Dec-2015
215 views
TRANSCRIPT
![Page 1: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/1.jpg)
1
Jan TretmansUniversity of Nijmegen
© Jan Tretmans University of Nijmegen
Model Based Testing
Property Checking for Real
![Page 2: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/2.jpg)
2 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 3: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/3.jpg)
3 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 4: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/4.jpg)
4 © Jan Tretmans University of Nijmegen
Testing
Testing:
to check the quality of an objectby performing experimentsin a controlled wayw.r.t. a specification
Software Testing :
testing the quality of a software product
mn
n+
mtester
IUT
![Page 5: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/5.jpg)
5 © Jan Tretmans University of Nijmegen
Paradox of Software Testing
Testing is:
important
much practiced
30% - 50% of project effort
expensive
time critical
not constructive(but sadistic?)
But also:
ad-hoc, manual, error-prone
hardly theory / research
no attention in curricula
not cool :“if you’re a bad programmer you might be a tester”
Attitude is changing: more awareness more professional
Improvements possible
with formal methods !?
![Page 6: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/6.jpg)
6 © Jan Tretmans University of Nijmegen
Types of Testing
unit
integration
system
performancerobustness
functional behaviour
white box
black box
Level of detail
Accessibility
Characteristics
usability
reliability
module
stress
![Page 7: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/7.jpg)
7 © Jan Tretmans University of Nijmegen
Testing
IUT
specification
property
IUTcorrect w.r.t.specification
tester
pass fail
test cases
scenarios
![Page 8: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/8.jpg)
8 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 9: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/9.jpg)
9 © Jan Tretmans University of Nijmegen
Formal methods: proving properties research sound theories “clean”
Testing : trial & error practice heuristics “dirty hands”
Testing & Formal MethodsA Perfect Couple ?
“Testing is not necessary after formal verification”
“Testing can only detect the presence of errors, not their absence”
“Formal methods are toys for boys”
“Formal methods are only for toy problems"
![Page 10: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/10.jpg)
10 © Jan Tretmans University of Nijmegen
Formal Testing
Why testing with a formal specification:
improves the testing process
precise and unambiguous basis for testing
allows automatic generation of tests
Why no formal verification (theorem proving, model checking, …) :
code/structure of system not accessible (black-box)
system too complex
verification only on model of implementation
prove evidence to customer/user
no formalization possible
![Page 11: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/11.jpg)
11 © Jan Tretmans University of Nijmegen
Development Process
informalrequirements
specification
realization
design
code
formalizable
![Page 12: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/12.jpg)
12 © Jan Tretmans University of Nijmegen
Goal:
Testing functional behaviour
of black-box implementation
with respect to specification
in formal language
based on formal definition
of conformance
Specification BasedFunctional Testing
with Formal Methods
implementationunder test
formal testing
Model assumed to be
correct
specifications
![Page 13: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/13.jpg)
13 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 14: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/14.jpg)
14 © Jan Tretmans University of Nijmegen
test
executionpass / fail
Formal Testing
test
generation
test suite TS
specification S
implementation i
correctness criterion
implementation
relation
i passes Ts
i ioco s
soundexhaustiv
e
![Page 15: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/15.jpg)
15 © Jan Tretmans University of Nijmegen
Formal Testing : Conformance
s SPECS SpecificationIUT Implementation under Test
IUT is concrete, physical object
Model the physical world
But IUT is black box ! ?
Assume that model iIUT exists
specification S
implementation IUT
correctness criterion
IUT correct wrt s
![Page 16: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/16.jpg)
16 © Jan Tretmans University of Nijmegen
Formal Testing : Test Hypothesis
Test hypothesis :
IUT IMPS . iIUT MODS .
t TESTS . obs ( t, IUT ) = obs ( t, iIUT )
IUT iIUT
test t test t
![Page 17: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/17.jpg)
17 © Jan Tretmans University of Nijmegen
Formal Testing : Conformance
specification S
implementation iIUT
formalcorrectness
criterioniIUT imp s
s SPECS Specification
iIUT MODS model of IUT
Test hypothesis : each concrete IUT can be modelled
by some iIUT MODS
Correctness : iIUT imp s
iIUT is not known ;
testing to learn about iIUT
![Page 18: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/18.jpg)
18 © Jan Tretmans University of Nijmegen
Completeness
exhaustive
sound
Completeness of test suite Ts on models:
i MODS .
t Ts . obs ( t, i ) = pass i imp s
![Page 19: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/19.jpg)
19 © Jan Tretmans University of Nijmegen
Testing for Conformance
IUT passes Ts def t Ts . IUT passes t
IUT passes t def obs ( t, IUT ) = pass
Test hypothesis :
t TESTS . obs ( t, IUT ) = obs ( t, iIUT )Proof obligation :
i MODS .
( t Ts . obs ( t, i ) = pass ) i imp s
IUT passes Ts IUT correct wrt s ?
Definition : IUT correct wrt sIUT correct wrt s
iIUT imp s
t Ts . obs ( t, iIUT ) = pass
t Ts . obs ( t, IUT ) = pass
t Ts . IUT passes t
IUT passes Ts
![Page 20: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/20.jpg)
20 © Jan Tretmans University of Nijmegen
Formal Testing
exec : TESTS IMPS (OBS)
der : SPECS (TESTS)
Ts TESTS
s SPECS
IUT IMPS
imp
iIUT MODS
obs : TESTS MODS (OBS)
Proof soundness and exhaustivess:iMODS .
tTs . obs(t,i) = pass i imp s
Test hypothesis :
IUTIMPS . iIUT MODS .
tTESTS . obs(t,IUT) = obs(t,iIUT)
pass / fail
Then:
IUT is correct iff it passes the test
![Page 21: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/21.jpg)
21 © Jan Tretmans University of Nijmegen
Approaches to Formal TestingInstantiations of Formal
Framework
Programs as functions (pre/post-conditions)
Finite State Machine / methods with state
Labelled Transition Systems
Abstract Data Type testing
![Page 22: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/22.jpg)
22 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 23: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/23.jpg)
23 © Jan Tretmans University of Nijmegen
Simple Input/Output Programs
Specification: s X Y
Test hypothesis: implementation i :: X Y
Tests: T X
Passing a test t T : ( t, i(t) ) s
Tools: QuickCheck, GST
Problem/challenge: select "good" subset T X:classical techniques: equivalence partitioning, boundary analysis,
……
IUTx: X y: Y
![Page 24: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/24.jpg)
24 © Jan Tretmans University of Nijmegen
Input/Output Program: Example
Specification:
propSqrt(x,y) = x 0 |y y - x| 0.00001
Possible test set: T = { 0, 0.00232, 1, 1.87392, 2738894 }
Tools like QuickCheck, GST easily generate thousands
of test cases
But still: what is a "good" set ?
IUT
i(x) = x
x: real
pre: x 0
y: real
post: |y y - x| 0.00001
![Page 25: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/25.jpg)
25 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 26: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/26.jpg)
26 © Jan Tretmans University of Nijmegen
Formal Testing with Transition Systems
exec : TESTS IMPS (OBS)
der : LTS (TTS)
Ts TTS
s LTS
IUT IMPS
ioco
iIUT IOTS
obs : TTS IOTS (traces)
Proof soundness and exhaustivess:
iIOTS .
( tder(s) . t(obs(t,i)) = pass )
i ioco s
Test hypothesis :
IUTIMPS . iIUT IOTS .
tTTS . exec(t,IUT) = obs(t,iIUT)
pass / fail
![Page 27: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/27.jpg)
27 © Jan Tretmans University of Nijmegen
Labelled Transition Systems
Labelled Transition System S, L, T, s0
?coin
?button
!alarm ?button
!coffee
states
actions transitionsT S (L{}) S
initial states0 S
![Page 28: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/28.jpg)
28 © Jan Tretmans University of Nijmegen
i ioco s =def Straces (s) : out (i after ) out (s after )
Implementation Relation ioco
Correctness expressed by implementation relation ioco:
p p = !x LU {} . p !x
out ( P ) = { !x LU | p !x , pP } { | p p, pP }
Straces ( s ) = { (L{})* | s }p after = { p’ | p p’ }
![Page 29: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/29.jpg)
29 © Jan Tretmans University of Nijmegen
i ioco s =def Straces (s) : out (i after ) out (s after )
Implementation Relation ioco
Correctness expressed by implementation relation ioco:
Intuition:
i ioco-conforms to s, iff
• if i produces output x after trace , then s can produce x after
• if i cannot produce any output after trace , then s cannot produce any output after ( quiescence )
![Page 30: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/30.jpg)
30 © Jan Tretmans University of Nijmegen
?dub
!choc
?kwart
!tea
!coffee
?dub?kwart
?dub?kwart
?dub?kwart !choc
?dub
!tea
ioco ioco
Implementation Relation ioco
!coffee
?dub
!tea
s
ioco
![Page 31: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/31.jpg)
31 © Jan Tretmans University of Nijmegen
Algorithm
To generate a test case from transition system specification s0
compute T(S), with S a set of states, and initially S = s0 after ;
1 end test case
pass
For T(S), apply the following recursively, non-deterministically:
2 supply input
!a
T( S after ?a )
Test Generation Algorithm
allowed outputs or : !x out ( S
)
forbidden outputs or : !y out ( S )
3 observe output
fail
T ( S after !x )
fail
allowed outputsforbidden outputs ?y
?x
![Page 32: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/32.jpg)
32 © Jan Tretmans University of Nijmegen
Completeness of Test Generation
For every test t generated with algorithm we have:
Soundness :t will never fail with correct implementation
i ioco s implies i passes t
Exhaustiveness :each incorrect implementation can be detectedwith a generated test t
i ioco s implies t : i fails t
![Page 33: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/33.jpg)
33 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 34: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/34.jpg)
34 © Jan Tretmans University of Nijmegen
A Test Tool : TorX
On-the-fly test generation and test execution
Implementation relation: ioco
Specification languages: LOTOS, Promela, FSP, Automata
TorX IUTobserveoutput
offerinput
nextinput
specification check
output
passfailinconclusive
user:manualautomatic
![Page 35: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/35.jpg)
35 © Jan Tretmans University of Nijmegen
TorX
![Page 36: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/36.jpg)
36 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 37: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/37.jpg)
37 © Jan Tretmans University of Nijmegen
TorX Case Studies
Conference Protocol
EasyLink TV-VCR protocol
Cell Broadcast Centre component
‘’Rekeningrijden’’ Payment Box protocol
V5.1 Access Network protocol
Easy Mail Melder
FTP Client
“Oosterschelde” storm surge barrier-
control
academi
c
Philips
CMG
Interpay
Lucent
CMG
academi
c
CMG
![Page 38: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/38.jpg)
38 © Jan Tretmans University of Nijmegen
Interpay ‘’Rekeningrijden’’Payment Box Protocol
![Page 39: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/39.jpg)
39 © Jan Tretmans University of Nijmegen
“Rekeningrijden”
Characteristics :
Simple protocol
Parallellism :
many cars at the same time
Encryption
Real-time issues
System passed traditional testing
phase
![Page 40: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/40.jpg)
40 © Jan Tretmans University of Nijmegen
PaymentBox
(PB)Road SideEquipment
OnboardUnit
UDP/IPWireless
‘’Rekeningrijden’’ Highway Tolling System
![Page 41: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/41.jpg)
41 © Jan Tretmans University of Nijmegen
spec
PB
TorX PaymentBox
‘’Rekeningrijden’’: Test Architecture
PCO
![Page 42: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/42.jpg)
42 © Jan Tretmans University of Nijmegen
spec
PB+
ObuSim+
TCP/IP+
UDP/IP
PaymentBox
TorX
‘’Rekeningrijden’’: Test Architecture
SUT
Test Context
ObuSim
TCP/IP UDP/IP IAPPCO
![Page 43: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/43.jpg)
43 © Jan Tretmans University of Nijmegen
‘’Rekeningrijden’’: Issues
Parallellism :
very easy
Encryption :
Not all events can be synthesized :
Leads to reduced testing power
Real-time :
How to cope with real time constraints ?
Efficient computation for on-the-fly testing ?
Lack of theory: quiescence vs. time-out
![Page 44: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/44.jpg)
44 © Jan Tretmans University of Nijmegen
‘’Rekeningrijden” : Results
Test results :
1 error during validation (design error)
1 error during testing (coding error)
Automated testing :
beneficial: high volume and reliability
many and long tests executed ( > 50,000 test
events )
very flexible: adaptation and many configurations
![Page 45: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/45.jpg)
45 © Jan Tretmans University of Nijmegen
Overview
Introduction Testing
Formal methods and Testing
Formal Testing Framework
Pre/post-condition program testing
Transition system testing ioco test theory
a test tool
an application: "Rekeningrijden"
Conclusions
![Page 46: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/46.jpg)
46 © Jan Tretmans University of Nijmegen
Concluding ……
Testing can be formal, too (M.-C. Gaudel, TACAS'95)
Testing shall be formal, too
But testing can never be complete
it increases confidence in correctness
Testing can be the first step:
counter-example finding before expensive formal verification
Or the last:
to show that not only the model is correct
but also the real system
![Page 47: 1 Jan Tretmans University of Nijmegen tretmans@cs.kun.nl © Jan Tretmans University of Nijmegen Model Based Testing Property Checking for Real](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d535503460f94a2f4a2/html5/thumbnails/47.jpg)
47 © Jan Tretmans University of Nijmegen
Thanks to ...
University of Twente:Ed Brinksma, Lex Heerink, Axel Belinfante,Jan Feenstra, René de Vries, Machiel van der Bijl, ……
Eindhoven University of Technology:Nicolae Goga, Loe Feijs, Sjouke Mauw
University of Nijmegen: Pieter Koopman
Philips Research Labs Eindhoven: Lex Heerink, Ron Koymans
KPN Research: Erik Kwast, Henri Dol
Lucent Technologies - R&D Centre Twente: Arjan de Heer
Project Pampa, IRISA, Rennes
(Logica)CMG Information Technology:Peter Christian, Robert Spee, Wouter Geurts, ……
Interpay B.V.: Cornel van Mastrigt, Rommert Jorritsma
Ordina Utopics: Pim Kars
………