property-based testing - qconqconsf.com/.../property_based_testing_qcon_2016.pdf · property based...
TRANSCRIPT
![Page 1: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/1.jpg)
Property-Based Testing
Matt Bachmann @mattbachmann
![Page 2: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/2.jpg)
Testing is Important
![Page 3: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/3.jpg)
Testing is Important
![Page 4: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/4.jpg)
Testing is Important
![Page 5: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/5.jpg)
Testing is Hard
![Page 6: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/6.jpg)
Testing is Hard
![Page 7: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/7.jpg)
Testing is Hard
![Page 8: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/8.jpg)
Capture the Important Cases
Minimize The Coding Overhead
![Page 9: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/9.jpg)
Sorting a list of integers
![Page 10: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/10.jpg)
def test_sort_empty(xs):assert quicksort([]) == []
def test_sorted(xs):assert quicksort([1, 2, 3]) == [1, 2, 3]
def test_sort_unsorted(xs):assert quicksort([5, 4]) == [4, 5]
![Page 11: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/11.jpg)
Property Based Testing● Describe the arguments● Describe the result● Have the computer try to
prove your code wrong
![Page 12: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/12.jpg)
● The Arguments○ List of Integers
● Properties of the Result○ List○ All elements preserved○ Results are in ascending
order
![Page 13: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/13.jpg)
HypothesisInspired by Haskell’s QuickCheck
Fantastic Docs
Offers training/contracting
http://hypothesis.works/
![Page 14: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/14.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 15: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/15.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 16: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/16.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 17: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/17.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 18: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/18.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 19: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/19.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 20: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/20.jpg)
@given(st.lists(st.integers()))def test_sort(xs): sorted_xs = quicksort(xs) assert isinstance(sorted_xs, list) assert Counter(xs) == Counter(sorted_xs) assert all( x <= y for x, y in
zip(sorted_xs, sorted_xs[1:]) )
![Page 21: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/21.jpg)
@given(st.lists(st.integers()))
test_sort([])
test_sort([0])
test_sort([93932932923, 82883982983838])
test_sort([9999,77,2,3,3,100,3,39,3993])
![Page 22: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/22.jpg)
![Page 23: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/23.jpg)
Check Out Other LanguagesHaskell - QuickCheck
Scala - ScalaCheck
Java - QuickTheories
![Page 24: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/24.jpg)
[-3223, -99999999999, 32]
![Page 25: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/25.jpg)
[-3223, -99999999999, 32]
[-3223, -99999999999]
![Page 26: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/26.jpg)
[-3223, -99999999999, 32]
[-3223, -99999999999]
[1, 0]
![Page 27: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/27.jpg)
@given
![Page 28: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/28.jpg)
@given(text())def demonstrate(s): print s
“10111�2”“1120莭┓Ksdjkjadasdamaslk8891h2ne d-1dni2hd”
![Page 29: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/29.jpg)
@given(integers())def demonstrate(s): print s
123123120-99423
![Page 30: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/30.jpg)
@given(lists(integers()))def demonstrate(s): print s
[][1,3,23,42][-1]
![Page 31: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/31.jpg)
@given(dictionaries(
integers(), list(text()))
)def demonstrate(s): print s
{323 : [“321312321”]}{8382: [“”, “!1”, “asdas"}{111: [“saodjad”]}
![Page 32: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/32.jpg)
@given(builds(
func, integers()
))def demonstrate(s): print s
func(0)func(31321)func(-21)
![Page 33: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/33.jpg)
@given(sampled_from(
[‘A’, ‘B’, ‘C’])
)def demonstrate(s): print s
‘A’‘A’‘C’
![Page 34: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/34.jpg)
@given(recursive(
booleans()lists
))def demonstrate(s): print s
True[True][True, [[False, True]]]
![Page 35: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/35.jpg)
@given( st.builds( Dog, breed=st.text(), name=st.text(), height=st.floats(), weight=st.floats() ))
![Page 36: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/36.jpg)
@given( st.builds( Dog, breed=st.sampled_from(KNOWN_BREEDS), name=st.text(), height=st.floats(), weight=st.floats() ))
![Page 37: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/37.jpg)
@given( st.builds( Dog, breed=st.sampled_from(KNOWN_BREEDS), name=st.text(min_size=5), height=st.floats(), weight=st.floats() ))
![Page 38: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/38.jpg)
@given( st.builds( Dog, breed=st.sampled_from(KNOWN_BREEDS), name=st.text(min_size=5), height=st.floats(
min_value=1, max_value=6, allow_nan=False, allow_infinity=False ),
weight=st.floats(min_value=1, max_value=300, allow_nan=False, allow_infinity=False
), ))
![Page 39: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/39.jpg)
@given( st.builds( Dog, breed=st.sampled_from(KNOWN_BREEDS), name=st.text(min_size=5), height=st.floats(
min_value=1, max_value=6, allow_nan=False, allow_infinity=False ),
weight=st.floats(min_value=1, max_value=300, allow_nan=False, allow_infinity=False
), ))@example(Dog(breed="Labrador", name="Spot", height=2.1, weight=70))
![Page 40: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/40.jpg)
Potentially infinite cases
Nothing hardcoded
Very little code
![Page 41: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/41.jpg)
![Page 42: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/42.jpg)
Slow Tests Make Their Voices Heard
![Page 43: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/43.jpg)
![Page 44: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/44.jpg)
![Page 45: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/45.jpg)
Not Super Friendly to TDD
![Page 46: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/46.jpg)
![Page 47: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/47.jpg)
![Page 48: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/48.jpg)
Properties Require More Thought
![Page 49: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/49.jpg)
Pattern 1:The code should not
explode
![Page 50: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/50.jpg)
![Page 51: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/51.jpg)
/batputer/criminals/aliases/{id}?sort={sort}&max_results={max}
● JSON response● Expected response codes
○ 200 - OK○ 401 - Forbidden○ 400 - Invalid data○ 404 - Not Found
![Page 52: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/52.jpg)
✓
![Page 53: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/53.jpg)
![Page 54: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/54.jpg)
@given(st.integers(), st.text(), st.integers()))
def test_no_explosion(id, sort, max): response = requests.get( BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in
[200, 401, 400, 404])
![Page 55: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/55.jpg)
@given(st.integers(), st.text(), st.integers()))
def test_no_explosion(id, sort, max): response = requests.get( BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in
[200, 401, 400, 404])
![Page 56: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/56.jpg)
@given(st.integers(), st.text(), st.integers()))
def test_no_explosion(id, sort, max): response = requests.get( BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in
[200, 401, 400, 404])
![Page 57: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/57.jpg)
@given(st.integers(), st.text(), st.integers()))
def test_no_explosion(id, sort, max): response = requests.get( BATPUTER_URL.format(id, sort, max) ) assert response and response.json() assert (response.status_code in
[200, 401, 400, 404])
![Page 58: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/58.jpg)
@given(st.integers(), st.text(), st.integers()))
def test_no_explosion(id, sort, max): my_cool_function(id, sort max)
![Page 59: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/59.jpg)
@given(st.integers(), st.text(), st.integers()))
def test_no_explosion(id, sort, max): try: my_cool_function(id, sort max) except ValueError: pass
![Page 60: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/60.jpg)
Pattern 2:Reversible Operations
![Page 61: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/61.jpg)
Encoding
Undo Operations
Serialization
![Page 62: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/62.jpg)
Encoding
Undo Operations
Serialization
![Page 63: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/63.jpg)
class HistoricalEvent(object): def __init__(self, id, desc, time): self.id = id self.desc = desc self.time = time
![Page 64: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/64.jpg)
class EventEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, HistoricalEvent):
return {
"id": obj.id
"description": obj.description
"event_time":obj.event_time.isoformat()
}
return json.JSONEncoder.default(self, obj)
![Page 65: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/65.jpg)
def fromJson(json_str):
dct = json.loads(json_str)
return HistoricalEvent(
dct['id'],
dct['description'],
dateutil.parser.parse(
dct['event_time']
)
)
![Page 66: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/66.jpg)
![Page 67: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/67.jpg)
@given(st.integers(), st.text(), datetimes(timezones=['UTC']))
def test_to_from_json(id, desc, date):
original_event = HistoricalEvent(id, desc, date)
encoded_event = json.dumps(event, cls=EventEncoder)
decoded_event = json.loads(
encoded_event,
object_hook=HistoricalEvent.fromJson
)
assert decoded_event == original_event
![Page 68: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/68.jpg)
@given(st.integers(), st.text(), datetimes(timezones=['UTC']))
def test_to_from_json(id, desc, date):
original_event = HistoricalEvent(id, desc, date)
encoded_event = json.dumps(event, cls=EventEncoder)
decoded_event = json.loads(
encoded_event,
object_hook=HistoricalEvent.fromJson
)
assert decoded_event == original_event
![Page 69: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/69.jpg)
@given(st.integers(), st.text(), datetimes(timezones=['UTC']))
def test_to_from_json(id, desc, date):
original_event = HistoricalEvent(id, desc, date)
encoded_event = json.dumps(event, cls=EventEncoder)
decoded_event = json.loads(
encoded_event,
object_hook=HistoricalEvent.fromJson
)
assert decoded_event == original_event
![Page 70: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/70.jpg)
@given(st.integers(), st.text(), datetimes(timezones=['UTC']))
def test_to_from_json(id, desc, date):
original_event = HistoricalEvent(id, desc, date)
encoded_event = json.dumps(event, cls=EventEncoder)
decoded_event = json.loads(
encoded_event,
object_hook=HistoricalEvent.fromJson
)
assert decoded_event == original_event
![Page 71: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/71.jpg)
@given(st.integers(), st.text(), datetimes(timezones=['UTC']))
def test_to_from_json(id, desc, date):
original_event = HistoricalEvent(id, desc, date)
encoded_event = json.dumps(event, cls=EventEncoder)
decoded_event = json.loads(
encoded_event,
object_hook=HistoricalEvent.fromJson
)
assert decoded_event == original_event
![Page 72: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/72.jpg)
![Page 73: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/73.jpg)
Pattern 3:Testing Oracle
![Page 74: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/74.jpg)
Optimizing
Refactoring
Emulating
![Page 75: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/75.jpg)
Leave It Alone
![Page 76: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/76.jpg)
=
![Page 77: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/77.jpg)
@given(st.integers(), st.text())
def test_against_legacy(arg1, arg2):
assert (
new_hotness(arg1, arg2)
==
legacy_system(arg1, arg2)
)
![Page 78: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/78.jpg)
Compare Against Brute
Force
![Page 79: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/79.jpg)
@given(st.lists(st.integers()))
def test_against_brute_force(input):
assert (
easy_but_inefficent(input)
==
optimized(input)
)
![Page 80: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/80.jpg)
The System
![Page 81: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/81.jpg)
The System
![Page 82: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/82.jpg)
The System
![Page 83: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/83.jpg)
The System
![Page 84: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/84.jpg)
Pattern 4:Invariants
![Page 85: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/85.jpg)
![Page 86: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/86.jpg)
![Page 87: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/87.jpg)
![Page 88: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/88.jpg)
Stateful Tests● Define a state● What operations can happen in
what conditions?● How do operations affect the
state?● What must be true for each step?
![Page 89: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/89.jpg)
https://commons.wikimedia.org/wiki/File:Max-Heap.svg
http://hypothesis.works/articles/rule-based-stateful-testing/
![Page 90: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/90.jpg)
Invariant● No matter what series of
operations is performed the head of the tree must be the max element
![Page 91: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/91.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
__init__pushpop
merge
![Page 92: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/92.jpg)
![Page 93: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/93.jpg)
FIND. ME. BUGS.
![Page 94: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/94.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
Integers
Heaps
![Page 95: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/95.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
__init__ Integers
Heaps
![Page 96: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/96.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
push Integers
Heaps
![Page 97: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/97.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
merge Integers
Heaps
![Page 98: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/98.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
pop Integers
Heaps
assert result actually max
![Page 99: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/99.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
class HeapMachine(RuleBasedStateMachine):
Heaps = Bundle('heaps')
@rule(target=Heaps)
def new_heap(self):
return Heap()
@rule(heap=Heaps, value=integers())
def heap_push(self, heap, value):
push(heap, value)
![Page 100: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/100.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
class HeapMachine(RuleBasedStateMachine):
Heaps = Bundle('heaps')
@rule(target=Heaps)
def new_heap(self):
return Heap()
@rule(heap=Heaps, value=integers())
def heap_push(self, heap, value):
push(heap, value)
![Page 101: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/101.jpg)
class HeapMachine(RuleBasedStateMachine):
Heaps = Bundle('heaps')
@rule(target=Heaps)
def new_heap(self):
return Heap()
@rule(heap=Heaps, value=integers())
def heap_push(self, heap, value):
push(heap, value)http://hypothesis.works/articles/rule-based-stateful-testing/
![Page 102: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/102.jpg)
class HeapMachine(RuleBasedStateMachine):
Heaps = Bundle('heaps')
@rule(target=Heaps)
def new_heap(self):
return Heap()
@rule(heap=Heaps, value=integers())
def heap_push(self, heap, value):
push(heap, value)http://hypothesis.works/articles/rule-based-stateful-testing/
![Page 103: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/103.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
…@rule(target=Heaps, heap1=Heaps, heap2=Heaps)def merge(self, heap1, heap2): return heap_merge(heap1, heap2)
@rule(heap=Heaps.filter(bool))def pop(self, heap): correct = max(list(heap)) result = heap_pop(heap) assert correct == result
![Page 104: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/104.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
…@rule(target=Heaps, heap1=Heaps, heap2=Heaps)def merge(self, heap1, heap2): return heap_merge(heap1, heap2)
@rule(heap=Heaps.filter(bool))def pop(self, heap): correct = max(list(heap)) result = heap_pop(heap) assert correct == result
![Page 105: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/105.jpg)
FIND. ME. BUGS.
![Page 106: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/106.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
@rule(heap=Heaps.filter(bool)) def pop(self, heap): correct = max(list(heap)) result = heap_pop(heap)> assert correct == resultE AssertionError: assert 1 == 0
![Page 107: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/107.jpg)
http://hypothesis.works/articles/rule-based-stateful-testing/
v1 = new_heap()
push(heap=v1, value=0)
push(heap=v1, value=1)
push(heap=v1, value=1)
v2 = merge(heap2=v1, heap1=v1)
pop(heap=v2)
pop(heap=v2)
![Page 108: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/108.jpg)
![Page 109: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/109.jpg)
Property Based Testing● Describe the arguments● Describe the result● Have the computer try to
prove your code wrong
![Page 110: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/110.jpg)
![Page 111: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/111.jpg)
![Page 112: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/112.jpg)
![Page 113: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/113.jpg)
![Page 114: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/114.jpg)
![Page 115: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/115.jpg)
Thanks!Matt Bachmannhttps://github.com/Bachmann1234
Twitter: @mattbachmann
Slides: https://goo.gl/LbelRE
![Page 116: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/116.jpg)
More about Hypothesis
● http://hypothesis.works/
More On Property Based Testing In General
● http://www.quviq.com/● https://fsharpforfunandprofit.com/posts/property-based-testing-2/
Testing Eventual Consistency with RIAK
● https://www.youtube.com/watch?v=x9mW54GJpG0
![Page 117: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/117.jpg)
Images
https://upload.wikimedia.org/wikipedia/commons/8/83/Jan_Fabre's_%22Searching_for_Utopia%22.jpg
https://www.flickr.com/photos/wonderlane/27784474073
https://upload.wikimedia.org/wikipedia/commons/3/36/Under_Floor_Cable_Runs_Tee.jpg
https://www.flickr.com/photos/versageek/493800514versageekhttps://creativecommons.org/licenses/by-sa/2.0/
https://upload.wikimedia.org/wikipedia/commons/3/39/Chip-pan-fire.jpg
https://upload.wikimedia.org/wikipedia/commons/4/4e/The_Thinker_by_Rodin_at_the_Cantor_Arts_Center_of_Stanford_University.JPG
https://www.flickr.com/photos/t0fugurl/2507049701Picture (without crossout) by Leanne Poonhttps://creativecommons.org/licenses/by-nc-nd/2.0/
https://upload.wikimedia.org/wikipedia/commons/thumb/1/19/Dtjohnnymonkey-Stopwatch-no-shading.svg/2000px-Dtjohnnymonkey-Stopwatch-no-shading.svg.png
https://commons.wikimedia.org/wiki/File:HK_Jockey_Club_Creative_Arts_Centre_JCCAC_Old_Machine_Printers.JPG
https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/Speed_bump,_Segev%C3%A5ng,_Malm%C3%B6.jpg/640px-Speed_bump,_Segev%C3%A5ng,_Malm%C3%B6.jpg
![Page 118: Property-Based Testing - QConqconsf.com/.../property_based_testing_qcon_2016.pdf · Property Based Testing Describe the arguments Describe the result Have the computer try to prove](https://reader035.vdocuments.mx/reader035/viewer/2022081612/5f0f32277e708231d442f60c/html5/thumbnails/118.jpg)
Images
https://upload.wikimedia.org/wikipedia/commons/6/62/Bird%27s_Rat%27s_nest.jpg
https://upload.wikimedia.org/wikipedia/commons/a/ac/Iceberg.jpg
https://upload.wikimedia.org/wikipedia/commons/4/49/Sleepy_Kit_(6969930840).jpg
http://imgs.xkcd.com/xk3d/303/compiling_4.png
https://pixabay.com/static/uploads/photo/2015/08/18/20/33/blueprints-894779_960_720.jpg
https://upload.wikimedia.org/wikipedia/commons/thumb/6/6b/Jenga_distorted.jpg/686px-Jenga_distorted.jpg
https://upload.wikimedia.org/wikipedia/commons/5/5c/MSC_Tomoko_in_the_Santa_Barbara_Channel_-_IMO_9309461_(3898801499).jpg
https://upload.wikimedia.org/wikipedia/commons/thumb/e/e0/Git-logo.svg/2000px-Git-logo.svg.png
https://upload.wikimedia.org/wikipedia/commons/c/c3/Podlaskie_-_Czarna_Bia%C5%82ostocka_-_Puszcza_Knyszy%C5%84ska_-_G%C3%B3ry_Czuma%C5%BCowskie_-_E_-_v-NW.JPG
https://www.flickr.com/photos/petsadviser-pix/8126550573