02157FunctionalProgram-
ming
Michael R. Hansen02157 Functional ProgrammingProperty-based testing: An appertizer
Michael R. Hansen
1 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Overview
QuickCheck: A Lightweight Tool for Random Testing of HaskellPrograms, Claessen and Hughes, 2000
• Random generation of values of arbitrary types• Properties are expressed as Boolean-valued functions
let rec sort xs = .....let rec ordered xs = ...
// Test that: for all lists xs: ordered(sort xs)let sortProp (xs: int list) = ordered(sort xs)
let _ = Check.Quick sortPropOk, passed 100 tests.
The tool has been ported to many languages. We look at FsCheckfor the .Net platform
Consult https://fscheck.github.io/FsCheck/ concerninginstallation and resources
2 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Testing for correctness wrt. a reference model (I)
#r ".......FsCheck.dll"open FsCheck
let rec sumA xs acc = match xs with| [] -> 0| x::xs -> sumA xs (x+acc);;
Correctness property wrt. the built-in function: List.sum:
for all xs: List.sum xs = sumA xs 0
let sumRefProp xs = List.sum xs = sumA xs 0;;let _ = Check.Quick sumRefProp;;Falsifiable, after 2 tests (2 shrinks) (StdGen ...... :Original:[-2; -1]Shrunk:[1]
• uses built-in generators for lists• tool provides a short counterexample
3 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Testing for correctness wrt. a reference model (II)
let rec sumA xs acc = match xs with| [] -> acc| x::xs -> sumA xs (x+acc);;
Correctness property wrt. the built-in function: List.sum:
for all xs: List.sum xs = sumA xs 0
let sumRefProp xs = List.sum xs = sumA xs 0;;let _ = Check.Quick sumRefProp;;Ok, passed 100 tests.
• default is 100 random tests• can be configured
4 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Testing for correctness wrt. a reference model (III)
Test is exposed using Check.Verbose as follows:
let sumRefProp xs = List.sum xs = sumA xs 0;;let _ = Check.Verbose sumRefProp;;
0:[-2].....99:[-1; 0; -1; -1; 2; 1; -1; 0; 0; 5; -1; 1; -1; 0; 0; -1; 2; -1; -2; 0; 1; 0; -1;1; -1; 1; -1; 0; -1; -1; -1; -1; 1; -1; 1; 1; 1; 0; -2; 1; 2; -2; 1; 0; 0; -2;1; 0; -1; 0; -1; -1; -2; 2; 0; 1; -1; -1; 1; 1; 0; 0; -1; 0; 1; 0; 0; 1; 1; 1;0]Ok, passed 100 tests.
5 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Checking invariants: Search trees
Type for search trees with associated insertion function:
type Tree = | Lf | Br of Tree*int*Tree;;
let rec insert i t =match t with| Lf -> Br(Lf,i,Lf)| Br(t1,j,t2) -> match compare i j with
| 0 -> t| n when n<0 -> Br(insert i t1 ,j,t2)| _ -> Br(t1,j, insert i t2);;
The function insert should preserve the search tree invariant:
for all trees t, integers 3: invariant(t) implies invariant(insert i t)
where invariant is a Boolean-valued function see handout
6 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Testing invariants: Search trees
The property can be checked as follows:
let insertInvProp i t = not (invariant t)|| invariant(insert i t);;
let _ = Check.Quick insertInvProp;;Ok, passed 100 tests.
Observe:• The invariant for insert i t is only tested when the invariant holds
for t short-circuit semantics of ||• Many randomly generated trees do not satisfy the invariant, and
hence more that 100 tests may be needed
7 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Configuring the number of tests
Suppose we want 400 random tests:
let _ = Check.One ( Config.Quick with MaxTest = 400; ,insertInvProp)
Ok, passed 400 tests.
8 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019
02157FunctionalProgram-
ming
Michael R. Hansen
Summary
Property-based testing supports testing at a high level of abstraction• Focus is on fundamental properties – not on concrete test cases• You write programs for properties – not concrete test cases• Properties are tested automatically• Short counterexamples are found — when properties are
falsified
The examples given here are just appetizers.
Next week we will have a look at• more kinds of properties• user-defined random generators• ...
9 DTU Compute, Technical University of Denmark Property-based testing: An appertizer MRH 07/11/2019