1 jan tretmans [email protected] embedded systems institute eindhoven, nl radboud university...
TRANSCRIPT
![Page 1: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/1.jpg)
1
Jan Tretmans
Embedded Systems Institute
Eindhoven, NL
Radboud UniversityNijmegen, NL
Model-Based Testing
with Labelled Transition
Systems
![Page 2: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/2.jpg)
2 © Jan Tretmans
Testing
Testing:
checking or measuring some quality characteristics
of an executing object
by performing experiments
in a controlled way
w.r.t. a specification
IUT
tester
specification
![Page 3: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/3.jpg)
3 © Jan Tretmans
Types of Testing
unit
integration
system
efficiency
maintainability
functionality
white box
black box
Level of detail
Accessibility
Characteristics
usabilityreliability
module
portability
![Page 4: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/4.jpg)
4 © Jan Tretmans
Automated Model-Based TestingModel-Based
Automated
Testing
model
IUT
IUTconftomodel
TTCNTTCNtestcase
s
pass fail
testtool
testgeneration
tool
testexecution
tool
IUT passes tests
IUT confto model
![Page 5: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/5.jpg)
5 © Jan Tretmans
Towards Model Based Testing
Increase in complexity, and quest for higher quality software
testing effort grows exponentially with complexity
testing cannot keep pace with development
More abstraction
less detail
model-based development
Checking quality
practice: testing - ad hoc, too late, expensive, lot of time
research: formal verification - proofs, model checking, . . . .
with disappointing practical impact
![Page 6: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/6.jpg)
6 © Jan Tretmans
Towards Model Based Testing
Model based testing has potential to combine
practice - testing
theory - formal methods
Model Based Testing :
testing with respect to a (formal) model / specification
state model, pre/post, CSP, Promela, UML, Spec#, . . . .
promises better, faster, cheaper testing:
algorithmic generation of tests and test oracles : tools
formal and unambiguous basis for testing
measuring the completeness of tests
maintenance of tests through model modification
![Page 7: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/7.jpg)
7 © Jan Tretmans
A Model-Based Development Process
informalrequirements
specification
realization
design
code
formalizablevalidation
formalverification
testing
model-based
informal world
world of models
real world
![Page 8: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/8.jpg)
8 © Jan Tretmans
formal world
real world
Formal Verification
model m
of i
sat
m modcheck s
m sat s
completesound
modelchecker
m modcheck s
m modcheck s
assumption:m is valid model of i
sat
specifications
implementationi
![Page 9: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/9.jpg)
9 © Jan Tretmans
formal world
real world
Formal Testing
specifications
implementationi
conftoIUT passes Ts
IUT confto s
correctness of
test generation ?
IUT passes Ts
IUT fails Ts
test
execution
test
generation
test suite TS
implementationi
IUT
![Page 10: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/10.jpg)
10 © Jan Tretmans
Approaches to Model-Based
Testing
Several modelling paradigms:
Finite State Machine
Pre/post-conditions
Labelled Transition Systems
Programs as Functions
Abstract Data Type testing
. . . . . . .
Labelled Transition Systems
![Page 11: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/11.jpg)
11 © Jan Tretmans
Model-Based Testing for LTS
test execution
testgeneration
tests
model
IUT
confto
Involves:• model / specification
• implementation IUT + models of IUTs• correctness• tests• test generation• test execution• test result analysis
pass / fail
![Page 12: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/12.jpg)
12 © Jan Tretmans
Labelled Transition System S, L, T, s0
?coin
?button
!alarm ?button
!coffee
states
actions transitionsT S (L{}) S
initial states0 S
Models of Specifications:Labelled Transition Systems
![Page 13: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/13.jpg)
13 © Jan Tretmans
?dub
!choc
?kwart
!tea
!coffee
?dub?kwart
?dub?kwart
?dub?kwart
!choc
?dub
!tea
Example Models
!coffee
?dub
!tea
?dub
!coffee
?dub
(Input-Enabled) Transition Systems
![Page 14: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/14.jpg)
14 © Jan Tretmans
i ioco s =def Straces (s) : out (i after ) out (s after )
Intuition:
i ioco-conforms to s, iff
• if i produces output x after trace , then s can produce x after
• if i cannot produce any output after trace , then s cannot produce any output after ( quiescence )
CorrectnessImplementation Relation ioco
![Page 15: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/15.jpg)
15 © Jan Tretmans
i ioco s =def Straces (s) : out (i after ) out (s after )
CorrectnessImplementation Relation ioco
p p = !x LU {} . p !x
out ( P ) = { !x LU | p !x , pP } { | p p, pP }
Straces ( s ) = { (L{})* | s }p after = { p’ | p p’ }
![Page 16: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/16.jpg)
16 © Jan Tretmans
?dub
!choc
?kwart
!tea
!coffee
?dub?kwart
?dub?kwart
?dub?kwart
!choc
?dub
!tea
ioco ioco
Implementation Relation ioco
!coffee
?dub
!tea
s
iocoioco
?dub
!coffee
?dub
![Page 17: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/17.jpg)
17 © Jan Tretmans
Test Cases
‘quiescence’ label
tree-structured finite, deterministic final states pass and fail
from each state pass, fail : either one input !a
or all outputs ?x and
?coffee
!dub
!kwart
?tea
?coffee?tea
!dub
pass
failfail
failpass
Model of a test case = transition system :
![Page 18: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/18.jpg)
18 © Jan Tretmans
Algorithm
To generate a test case from transition system specification s0
compute T(S), with S a set of states, and initially S = s0 after ;
1 end test case
pass
For T(S), apply the following recursively, non-deterministically:
2 supply input
!a
T( S after ?a )
ioco Test Generation Algorithm
allowed outputs or : !x out ( S
)
forbidden outputs or : !y out ( S )
3 observe output
fail
T ( S after !x )
fail
allowed outputsforbidden outputs ?y
?x
![Page 19: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/19.jpg)
19 © Jan Tretmans
?coffee ?tea
passfail fail
?coffee
passfail
?tea
Test Generation Example
s
?dub
!coffee
?dub
test
!dub
![Page 20: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/20.jpg)
20 © Jan Tretmans
Test Execution Example
Two test runs :
t i dub pass i'
pass i't i dub coffee i passes t
?coffee ?tea
passfail fail
?coffee
passfail
?tea
test
!dub
i?dub
!coffee
?dub
?dub
?dub
?dub
![Page 21: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/21.jpg)
21 © Jan Tretmans
Test Result AnalysisCompleteness of ioco Test Generation
For every test t generated with algorithm we have:
Soundness :t will never fail with correct implementation
i ioco s implies i passes t
Exhaustiveness :each incorrect implementation can be detectedwith a generated test t
i ioco s implies t : i fails t
![Page 22: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/22.jpg)
22 © Jan Tretmans
Formal Testing with Transition Systems
exec : TESTS IMPS (OBS)
gen : LTS (TTS)
Ts TTS
s LTS
IUT IMPS
ioco
iIUT IOTS
passes : IOTS TTS
{pass,fail
}
Proof soundness and exhaustiveness:
iIOTS .
( tgen(s) . i passes t )
i ioco s
Test hypothesis :
IUTIMP . iIUT IOTS .
tTTS . IUT passes t iIUT passes t
pass / fail
=
![Page 23: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/23.jpg)
23 © Jan Tretmans
Variations on a Theme
i ioco s Straces(s) : out ( i after ) out ( s after )
i ior s ( L {} )* : out ( i after ) out ( s after )
i ioconf s traces(s) : out ( i after ) out ( s after )
i iocoF s F : out ( i after ) out ( s after )
i mioco smulti-channel ioco
i uioco s universal ioco
i wioco s non-input-enabled ioco
i sioco s symbolic ioco
i (r)tioco s (real) timed tioco (Aalborg, Twente, Grenoble,
Bordeaux,. . . .)
i iocor s refinement ioco
i hioco shybrid ioco
. . . . . .
![Page 24: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/24.jpg)
24 © Jan Tretmans
? money
? button1 ? button2
! coffee! tea
test case
fail
! money
! button2
? tea
fail
? coffee
pass
n: int
[ n 35 ] -> [ n 50 ] ->
with data model
and hybrid and time
c := 0
c < 10 c < 15
[ c 5 ] ->
c := 0
d Vt / dt =
3d Vc / dt =
2
Vc := 0
[Vc = 10 ] ->
Vt := 0
[Vt = 15 ] ->
?
Testing Transition Systems: StatusExtensions
?
coin1
?
coin3
?
coin2and actionrefinement
![Page 25: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/25.jpg)
25 © Jan Tretmans
!x
?ok
!err
?but
?but
?but
!err
?but
!ok
?ok
?ok?err
!x
?err
!y
?ok?err
?ok?err
?ok?err
?but
!x
i1 ioco s1
i2 ioco
s2
ioco s1||s2i1||i2
okerr
but
Xy
okerr
but
Xy
Component Based Testing
?but
?but
?but
!y
?but
![Page 26: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/26.jpg)
26 © Jan Tretmans
Compositional TestingComponent Based Testing
i1
i2 s2
s1
ioco
i1 ioco s1
i2 ioco s2
s1 || s2i1 || i2
If s1, s2 input enabled - s1, s2 IOTS - then ioco is preserved !
![Page 27: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/27.jpg)
27 © Jan Tretmans
Variations on a Theme: uioco
?a?a
?b
!z
?b?a
!y !x
i ioco s Straces(s) : out ( i after ) out ( s0 after )s0
s1 s2
out ( s0 after ?b ) =
but ?b Straces(s) : under-
specification :
anything allowed after ?bout ( s0 after ?a ?a ) = { !x }
and ?a ?a Straces(s)
but from s2 , ?a ?a is under-specified :
anything allowed after ?a ?a ?
![Page 28: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/28.jpg)
28 © Jan Tretmans
Variations on a Theme: uioco
?a?a
?b
!z
?b?a
!y !x
s0
s1 s2
i uioco s Utraces(s) : out ( i after ) out ( s0 after )
Now s is under-specified in s2 for ?a :
anything is allowed.
Utraces(s) =
{ Straces (s) | 1 ?a 2 = ,
s': s 1 s' s'
?a }
ioco uioco
![Page 29: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/29.jpg)
29 © Jan Tretmans
Variations on a Theme: uioco
LILU
LI
?a
?a
?b
!z
?b?a
!y !x
s0
s1
?b
?a
s2
i uioco s Utraces(s) : out ( i after ) out ( s0 after )
Alternatively, via chaos process for under-specified inputs
![Page 30: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/30.jpg)
30 © Jan Tretmans
Testing Components
methodinvocations
IUTcomponent
||
IUTcomponent
methodinvocations
methodsinvoked
methodcall
IUTcomponent
methodreturned
methodcalled
methodreturn
IUTcomponent
methodinvocation
![Page 31: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/31.jpg)
31 © Jan Tretmans
Testing Components
tester
methodcall
IUTcomponent
methodreturn
methodcall
methodreturn
LI = offered methods calls used methods returns
LU = offered methods returns used methods calls
specifications
LTS(LI, LU)
![Page 32: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/32.jpg)
32 © Jan Tretmans
Testing Components
tester
methodcall
IUTcomponent
method
return
method
call
methodreturn
Input-enabledness: s of IUT, ?a LI : s
?a ?
No ! ?
![Page 33: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/33.jpg)
33 © Jan Tretmans
i uioco s =def Utraces (s) : out (i after ) out (s after )
CorrectnessImplementation Relation wioco
in (s after ) = { a? LI | s after must a? }
i wioco s =def Utraces (s) : out (i after ) out (s after )
and in (i after ) in (s after )
s after must a? = s’ ( s s’ s’ a? )
![Page 34: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/34.jpg)
34 © Jan Tretmans
Formal Testing with Transition Systems
exec : TESTS IMPS (OBS)
gen : LTS (TTS)
Ts TTS
s LTS
IUT IMPS
ioco
iIUT IOTS
passes : IOTS TTS
{pass,fail
}
Proof soundness and exhaustiveness:
iIOTS .
( tgen(s) . i passes t )
i ioco s
Test hypothesis :
IUTIMP . iIUT IOTS .
tTTS . IUT passes t iIUT passes t
pass / fail
=
![Page 35: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/35.jpg)
35 © Jan Tretmans
Test Assumption
IUTinput a?
LI output x? LU
quiescence
Sequencing of inputs, outputs, and :Input-enabledness: s of IUT, ?a LI :
IUT
s ?a
IUT behaves as an IOTS (input-enabled LTS)
![Page 36: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/36.jpg)
36 © Jan Tretmans
Comparing Transition SystemsTesting Equivalences
S1 S2S1 S2
environment environment
Suppose an environment interacts with the systems:
the environment tests the system as black box
by observing and actively controlling it;
the environment acts as a tester;
Two systems are equivalent if they pass the same tests.
![Page 37: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/37.jpg)
37 © Jan Tretmans
Formal Testing : Test Assumption
Test assumption :
IUT . iIUT MOD.
t TEST . IUT passes t iIUT passes t
IUT iIUT
test t test t
![Page 38: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/38.jpg)
38 © Jan Tretmans
Completeness of Formal Testing
IUT passes Ts def t Ts . IUT passes t
Test hypothesis :
t TEST . IUT passes t iIUT passes tProof obligation :
i MOD .
( t Ts . i passes t ) i imp s
IUT passes Ts IUT confto s ?
Definition : IUT confto sIUT confto s
iIUT imp s
t Ts . iIUT passes t
t Ts . IUT passes t
IUT passes Ts
![Page 39: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/39.jpg)
39 © Jan Tretmans
!choc
?dub
!tea
ioco
Test Assumption
!coffee
?dub
!tea
s
More tests may be needed, starting in initial state:
meta-assumption: reliable restart
![Page 40: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/40.jpg)
40 © Jan Tretmans
?dub
!choc
?dub
!tea !choc
?dub
!tea
Alternative Test Assumption
ioco
ioco
Test: 1. do ?dub2. make core dump3. make many copies of core dump4. continue test with each copy
An “Ambramsky”-test can distinguish them
![Page 41: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/41.jpg)
41 © Jan Tretmans
Alternative Test Assumption
ioco
ioco ?kwart
?dub
?kwart
!tea !choc
?dub
?kwart
?dub
?kwart
!tea !choc
With test ?dub.?kwart.undo you can distinguish them
![Page 42: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/42.jpg)
42 © Jan Tretmans
Concluding
Testing can be formal, too (M.-C. Gaudel, TACAS'95)
Testing shall be formal, too
A test generation algorithm is not just another algorithm :
Proof of soundness and exhaustiveness
Definition of test assumption and implementation relation
For labelled transition systems :
ioco for expressing conformance between imp and spec
a sound and exhaustive test generation algorithm
tools generating and executing tests:
TGV, TestGen, Agedis, TorX, . . . .
![Page 43: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/43.jpg)
43 © Jan Tretmans
Model based formal testing can improve the testing process :
model is precise and unambiguous basis for testing
design errors found during validation of model
longer, cheaper, more flexible, and provably correct tests
easier test maintenance and regression testing
automatic test generation and execution
full automation : test generation + execution + analysis
extra effort of modelling compensated by better tests
Perspectives
![Page 44: 1 Jan Tretmans jan.tretmans@esi.nl Embedded Systems Institute Eindhoven, NL Radboud University Nijmegen, NL Model-Based Testing with Labelled Transition](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649e115503460f94afcb37/html5/thumbnails/44.jpg)
44 © Jan Tretmans
Thank You