survey of unit-testing frameworks - szakmeister · survey of unit-testing frameworks by john...
TRANSCRIPT
![Page 1: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/1.jpg)
Survey of Unit-Testing Frameworks
by John Szakmeister and Tim Woods
![Page 2: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/2.jpg)
Our Background
• Using Python for 7 years
• Unit-testing fanatics for 5 years
![Page 3: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/3.jpg)
Agenda
• Why unit test?
• Talk about 3 frameworks:
‣ unittest
‣ nose
‣ py.test
![Page 4: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/4.jpg)
Why bother?
• Confidence!
• Automation
• To make failures more obvious
• To prevent regressions
• Aids debugging
![Page 5: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/5.jpg)
Commonalities
• All the frameworks:
‣ Follow the xUnit mentality
‣ Verify via asserts
![Page 6: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/6.jpg)
unittest
• In the standard library
• The de-facto standard unit test framework
![Page 7: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/7.jpg)
Writing some tests...
• Need to subclass TestCase
import unittest
class TestFoo(unittest.TestCase): …
![Page 8: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/8.jpg)
Write some tests...
• The simple way: test methods start with ‘test’
• Use assertions validate your results
![Page 9: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/9.jpg)
Examples...
def testIsInt(self): self.assertTrue(foo.isInt(0)) self.assertFalse(foo.isInt("not an int"))
def testMakeList(self): self.assertEquals([1,2,3], foo.makeList(1, 2, 3))
def testDivide(self): self.assertEquals(0, foo.divide(0, 1)) self.assertRaises( ZeroDivisionError, foo.divide, 1, 0)
![Page 10: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/10.jpg)
Test Fixtures
• Useful if test cases need more “infrastructure”
• setUp() is called before each test method
• tearDown() is called after each test method
![Page 11: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/11.jpg)
Example...
def setUp(self): # ... Any *common* set up required for your # test cases can go here self.db = create_db_connection() def tearDown(self): # Clean up the fixture here self.db.close()
![Page 12: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/12.jpg)
Running the Tests...
• Executing all tests within the file is easy
• Add this to the file:
if __name__ == '__main__': unittest.main()
• Collects all test cases, and executes them using the default console test runner
![Page 13: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/13.jpg)
Example output
• On the command line, run:
:: PYTHONPATH=. python tests_unittest/test_foo.py....---------------------------------------------------Ran 4 tests in 0.000s
OK
![Page 14: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/14.jpg)
Collecting All Tests...
• Need a suite to help collect the test cases
• In each file, do the following:
def suite(): testSuite = unittest.TestSuite() loader = unittest.TestLoader()
for testCase in [TestFoo, TestFooBar]: testSuite.addTest( loader.loadTestsFromTestCase(testCase)) return testSuite
![Page 15: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/15.jpg)
(cont)
• Pull all the suites together in the __init__.py:
import unittestimport test_foo
def collectAll(): allSuite = unittest.TestSuite(test_foo.suite())
# If you have others, you can add them by # doing: # allSuite.addTest(test_bar.suite())
return allSuite
![Page 16: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/16.jpg)
Finally, run all the tests
• Need to launch unittest.main(), but this time, tell it how to find the full test suite:
#!/usr/bin/env pythonimport unittestimport tests_unittest
unittest.main( defaultTest='tests_unittest.collectAll')
![Page 17: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/17.jpg)
setuptools
• You can launch your tests via setuptools:
from setuptools import setup
setup(..., test_suite = 'tests_unittest.collectAll' )
![Page 18: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/18.jpg)
Pros
• It’s in the standard library
• Writing the actual tests is easy
![Page 19: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/19.jpg)
Cons
• Tedious to collect test suites, especially for a large code base
‣ Michael Foord is working on this(check out the discover package)
• Not as easily extensible as other frameworks
‣ Others have done it though. See the testtools project in Launchpad.
![Page 20: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/20.jpg)
Future Improvements
• unittest is vastly improved in Python 2.7
‣ Test discovery
‣ Skipping
‣ Expected Failures
‣ assertRaises using ‘with’ statement
‣ New assert methods, and much more
![Page 21: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/21.jpg)
nose
• Written by Jason Pellerin
![Page 22: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/22.jpg)
Install nose
• Available from http://somethingaboutorange.com/mrl/projects/nose/
• Use easy_install: easy_install nose
![Page 23: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/23.jpg)
Just write tests
• Finds test files, functions, classes, methods
‣ test or Test on a word boundary
‣ Customize with regular expression
• No need to write suites
• Use package structure to organize tests
‣ Add __init__.py to directory
![Page 24: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/24.jpg)
Write a test
• Create a file, test_foo.py:from nose.tools import *
import foo
def test_isInt(): assert_true(foo.isInt(0))
![Page 25: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/25.jpg)
Use assertions
• Provided in ‘nose.tools’
• Same asserts as unittest, in PEP 8 style
• Also provides:
‣ ok_ => assert
‣ eq_ => assert_equals
![Page 26: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/26.jpg)
Use assertions
def test_isInt(): assert_true(foo.isInt(0)) assert_false(foo.isInt("not an int"))
def test_makeList(): assert_equals([1,2,3], foo.makeList(1, 2, 3))
def test_divide(): eq_(0, foo.divide(0, 1))
![Page 27: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/27.jpg)
Check exceptions
• Verify that test function raises exception
• Use raises decorator:@raises(ZeroDivisionError)def test_divide_by_zero(): foo.divide(1, 0)
![Page 28: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/28.jpg)
Test fixtures
• Use with_setup decorator
• Can be used for simple setup
• More complex cases should probably use a test class
• Fixtures per package, module, class
![Page 29: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/29.jpg)
Test fixtures_db = None
def setup_func(): global _db _db = create_db_connection()
def teardown_func(): global _db _db.close()
@with_setup(setup_func, teardown_func)def test_with_fixture(): ok_(isinstance(_db, DummyConnection))
![Page 30: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/30.jpg)
Test classes
• Define class that matches test regexp
• Define test methods in the class
• Optionally define setup and teardown
![Page 31: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/31.jpg)
Test classesclass TestFoo():
def setup(self): # Fixture setup self.db = create_db_connection()
def teardown(self): # Fixture teardown self.db.close()
def test_isInt(self): ok_(foo.isInt(0))
def test_dbConn(self): ok_(isinstance(self.db, DummyConnection))
![Page 32: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/32.jpg)
Generative tests
• Each yield results in a test casedef test_generator(): for i in xrange(0, 20, 2): yield check_even, i
def check_even(even_number): assert_true(foo.isEven(even_number)) assert_false(foo.isEven(even_number + 1))
![Page 33: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/33.jpg)
Attributes
• Add attribute tag to testsfrom nose.plugins.attrib import attr
@attr('nothing')def test_zero_equals_zero(): assert 0 == 0
• Can set a specific value@attr(speed=‘slow’)
![Page 34: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/34.jpg)
Attributes
• Select attributes at runtime (-a/--attr)
‣ nothing
‣ speed=slow
• Python expression (-A/--eval-attr)
‣ “not nothing”
‣ “(speed==‘slow’ and not nothing)”
![Page 35: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/35.jpg)
Skip tests
• Raise exception to report test skippedfrom nose.plugins.skip import SkipTest
def test_skipme(): raise SkipTest
![Page 36: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/36.jpg)
Runs unittest tests
• Loads tests from unittest.TestCase subclasses
• Easily used as a front-end for legacy tests
![Page 37: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/37.jpg)
Running tests
• Type nosetests at the top level
• Run a subset of tests
‣ cd package; nosetests
‣ nosetests package/tests/module.py
‣ nosetests package.tests.module:name
![Page 38: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/38.jpg)
Other features
• setuptools integration
• Plugin system
‣ Debug, code coverage and profiling
‣ Doctest runner
‣ XUnit XML output
![Page 39: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/39.jpg)
Pros
• No tedious mucking about with suites
• Can be used with legacy unittest tests
• Generative tests
• Plugins
![Page 40: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/40.jpg)
Cons
• Not in standard library
• No official release supporting Python 3.x
‣ py3k branch exists
![Page 41: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/41.jpg)
py.test
• Written by Holger Krekel
![Page 42: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/42.jpg)
Install py.test
• Actually part of pylib
• Download from http://pytest.org/
• Use easy_install: easy_install py
![Page 43: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/43.jpg)
Easy to get started..
• Just create a test file
‣ Make sure it starts with ‘test_’
• Start adding test methods, or test classes:
import fooimport py.test
def test_isInt(): pass
![Page 44: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/44.jpg)
Uses ‘assert’
def test_isInt(): assert True == foo.isInt(0) assert False == foo.isInt("not an int")
def test_makeList(): assert [1,2,3] == foo.makeList(1, 2, 3)
![Page 45: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/45.jpg)
Checking exceptions...
• Use py.test.raises()
def test_divide(): assert 0 == foo.divide(0, 1)
# Dividing 1 by 0 should raise # ZeroDivisionError py.test.raises( ZeroDivisionError, foo.divide, 1, 0)
![Page 46: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/46.jpg)
“Test” classes
Just start the class with “Test”:
class TestFoo(): def test_isInt(self): assert True == foo.isInt(0) assert False == foo.isInt("not an int")
![Page 47: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/47.jpg)
Test Fixtures
• With test classes, use:
def setup_method(self, method): # Fixture setup self.db = create_db_connection()
def teardown_method(self, method): # Fixture teardown self.db.close()
![Page 48: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/48.jpg)
Funcargs
• Helps you to create arguments instead
def pytest_funcarg__db(request): db = create_db_connection() request.addfinalizer(lambda: db.close()) return db
def test_db(db): # ... do something … assert isinstance(db, DummyConnection)
• Funcargs are the preferred way in py.test
![Page 49: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/49.jpg)
Funcargs
• Helps you to create arguments instead
def pytest_funcarg__db(request): db = create_db_connection() request.addfinalizer(lambda: db.close()) return db
def test_db(db): # ... do something … assert isinstance(db, DummyConnection)
• Funcargs are the preferred way in py.test
![Page 50: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/50.jpg)
Generative tests...
• Similar to nose…
def test_generative(): for i in xrange(0,20,2): yield check_even, i
def check_even(even_number): assert True == foo.isEven(even_number) assert False == foo.isEven(even_number+1)
• This has been deprecated though
![Page 51: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/51.jpg)
Generative tests...
• New style test generators
‣ Still experimental
‣ Encompasses several ways of doing:
• Parameterized tests
• Scenario tests
![Page 52: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/52.jpg)
“Marking” tests...
• Can mark tests as “expected failure”
@py.test.mark.xfaildef test_xfail(): assert 0 == 1
• Or with keywords
# A test marked with keyword "nothing"@py.test.mark(nothing=True)def test_zero_equals_zero(): assert 0 == 0
![Page 53: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/53.jpg)
“Marking” tests...
• Some keywords are added automatically
‣ Filename
‣ Class names
‣ Function names
![Page 54: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/54.jpg)
Skipping Tests
• Skip if an import fails bogus = py.test.importorskip("bogus")
‣ Skip all when done at module level
‣ Skips an individual test when done inside a test function
• Skip for some other reason py.test.skip("A short message")
![Page 55: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/55.jpg)
Disabling a test class
• Great for disabling platform-specific tests:
class TestWin32Only: disabled = sys.platform != 'win32' # Test cases follow…
• Also good for disabling tests that require a specific software/hardware setup
‣ Need a decent way to test for it
‣ Keywords are useful for this too
![Page 56: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/56.jpg)
Executing Tests
• Use the py.test command:
py.test [/path/to/file/or/dir] […]
• Automatically collects tests
• Begins executing tests immediately
![Page 57: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/57.jpg)
Executing Tests
• Tests with a specific keyword:
py.test -k keyword
• Tests that don’t have a specific keyword:
py.test -k “-keyword”
![Page 58: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/58.jpg)
Distributed Testing
• Two modes
‣ Local
‣ Remote
![Page 59: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/59.jpg)
Locally
• Take advantage of multiprocessors machine
• Significant speedups if tests are IO bound
• To run tests in ‘<num>’ processes, use: py.test -n <num>
![Page 60: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/60.jpg)
Remotely
• Two strategies
‣ Load balancing - every test one run once py.test --dist=load py.test -d
‣ Multiplatform - run every test on each platform py.test --dist=each
![Page 61: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/61.jpg)
Remote Mechanisms
• Use ssh py.test -d --tx ssh=user@host//chdir=/tmp \ --rsyncdir pkg
• Use a gateway py.test -d --tx socket=ip:port//chdir=/tmp \ --rsyncdir pkg
• Gateway launched via a small script
‣ Does not require installing pylib
![Page 62: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/62.jpg)
conftest.py
• Captures command-line args in a config file
• Great for storing remote test configuration
pytest_option_tx = ["ssh=localhost"]pytest_option_dist="load"# Paths are relative to conftest.pyrsyncdirs = [".", "../foo.py"]
• Can also configure plugin options
![Page 63: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/63.jpg)
Other features
• Can drop into PDB on error
• Has a plugin system
‣ Execute unittest-style TestCases
‣ Coverage reporting (via figleaf)
‣ Pylint integration
‣ And many more
![Page 64: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/64.jpg)
Pros
• Don’t need to collect the tests
• Generative tests
• Marking tests
• Distributed testing
• Excels with large test suites
• More to py.test than this small example
![Page 65: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/65.jpg)
Cons
• Default output is a bit sloppy
• Funcargs and generative == breakage!
• No Python 3 support yet
• Distributed tests can hang on an internal failure
![Page 66: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/66.jpg)
We have source!
http://www.szakmeister.net/misc/unit-testing.zip
![Page 67: Survey of Unit-Testing Frameworks - szakmeister · Survey of Unit-Testing Frameworks by John Szakmeister and Tim Woods. Our Background •Using Python for 7 years •Unit-testing](https://reader033.vdocuments.mx/reader033/viewer/2022042307/5ed3188c444dc013b77eeb9e/html5/thumbnails/67.jpg)
Questions?