testing telecoms software with quickcheck
DESCRIPTION
Testing Telecoms Software with QuickCheck. Thomas Arts John Hughes Chalmers/ITU. Joakim Johansson Ulf Wiger Ericsson. Session Border Gateway. Monitors signalling traffic; opens and closes pinholes. 2x1G Ethernet Linux ”blade” processor. Media Gateway Controller. - PowerPoint PPT PresentationTRANSCRIPT
Testing Telecoms Software with QuickCheck
Thomas Arts
John Hughes
Chalmers/ITU
Joakim JohanssonUlf Wiger
Ericsson
Session Border Gateway
Media Proxy
Opens media pin-holes for calls in
progress
2x10G Ethernet
Hardware for packet forwarding
Media Gateway Controller
Monitors signalling
traffic; opens and closes pinholes
2x1G Ethernet
Linux ”blade” processor
Session Border Gateway
Media Proxy
Media Gateway Controller
ITU standard protocol H.248
(Megaco) 150KLOC Erlang
1.5MLOC C
A QuickCheck Property
prop_reverse() -> ?FORALL(Xs,list(int()), lists:reverse(lists:reverse(Xs)) == Xs).
6> eqc:quickcheck(rev:prop_reverse()).....................................................................................................OK, passed 100 teststrue
A Wrong Property
prop_reverse_wrong() -> ?FORALL(Xs,list(int()), lists:reverse(Xs) == Xs).
10> eqc:quickcheck(rev:prop_reverse_wrong())......................Failed! After 22 tests.[-2,4,0]Shrinking......(6 times)[1,0]false
Testing the Media Proxy Control Software
Media Proxy
QuickCheckSend
random command sequences
Check the responses
Generate random messages in a random command sequence
But not completely random!
MediaDescriptor ::= SEQUENCE{
termStateDescr TerminationStateDescriptor OPTIONAL,streams CHOICE{
oneStream StreamParms,multiStream SEQUENCE OF StreamDescriptor
} OPTIONAL,…
}
Message Example: ASN.1 Description
• A media descriptor contains a list of streams
…with IWD Restrictions
• A media descriptor contains a list of streams
MediaDescriptor ::= SEQUENCE{
termStateDescr TerminationStateDescriptor OPTIONAL,streams CHOICE{
oneStream StreamParms,multiStream SEQUENCE OF StreamDescriptor
} OPTIONAL,…
}
QuickCheck Generator for Erlang Records
mediadescriptor(Streams) when Streams=/=[]-> {mediaDescriptor, #'MediaDescriptor'{ streams = case Streams of [{Id,Mode}] -> oneof([{oneStream,streamParms(Mode)}, {multiStream,[stream(Id,Mode)]}]); _ -> {multiStream, [stream(I,M) || {I,M}<-Streams]} end}}.
stream(I,Mode) -> #'StreamDescriptor'{ streamID = I, streamParms = streamParms(Mode)}.
Generators can be embedded in any data
structure
Message constructionLogic from the IWD
QuickCheck
Local Control DescriptorsAdd Request
Desc. Desc. Desc. Properties, Package M Comment
Media M
Stream M Multiple Stream descriptors can be included.
Localcontrol
O LocalControl will be included in all cases except when no media (m-line) is defined in the remote SDP.
mode O The default value of the mode property is “Inactive”, the property is not mandatory if the wanted value is “Inactive”.
StreamParms ::= SEQUENCE{
localControlDescriptor LocalControlDescriptor OPTIONAL,localDescriptor LocalRemoteDescriptor OPTIONAL,remoteDescriptor LocalRemoteDescriptor OPTIONAL,…,statisticsDescriptorStatisticsDescriptor OPTIONAL
}
streamParms(Mode) -> ?LET(RemoteMediaDefined, probably(), if RemoteMediaDefined ->
#'StreamParms'{ localControlDescriptor = localControl(Mode),
localDescriptor = localDescriptor(RemoteMediaDefined),
remoteDescriptor = remoteDescriptor(RemoteMediaDefined)}; true -> …… end).
Random boolean with
high probability
Two Cases: With and Without Remote Media
Megaco Commands
ContextContext
Megaco Commands
Context
ContextContext
Termination
Add
Returns Context ID &
Termination ID
Megaco Commands
Context
ContextContext
Termination
Termination
Add
Uses context ID
Megaco Commands
Context
ContextContext
Termination
TerminationStream
StreamModify
Uses termination ID
Megaco Commands
Context
ContextContext
Termination
TerminationStream
Stream
Subtract
Megaco Commands
Context
ContextContext
Termination
StreamStream
Subtract
Megaco Commands
ContextContext
Symbolic Test Cases
• Generated wrt an abstract state machine tracking contexts, terminations & streams
[{set,1,…,send_add, [?choose_cxt_id,Streams,Req]}, {assert,…,check_add,[{var,1}], {set,2,…,send_add, [{call,…,get_reply_cxt,[{var,1}]}, Streams2,Req2]}, {assert,…,check_add,[{var,2}]}]
Encode/Decode Error
• Recall the stream parameters…
StreamParms ::= SEQUENCE{
localControlDescriptor LocalControlDescriptor OPTIONAL,localDescriptor LocalRemoteDescriptor OPTIONAL,remoteDescriptor LocalRemoteDescriptor OPTIONAL,…,statisticsDescriptorStatisticsDescriptor OPTIONAL
}
Add/Modify Error
Add
Modify
Add/Subtract Error
• There one day, gone the next…
Add
Subtract
Add/Add/Modify Error
• If the terminations have different numbers of streams
Add
Modify
Add
Add/Add/Sub/Add… Error
• Extracted from a test case 160 commands long
Add
Subtract
Add
Add
Add
Subtract
Subtract