system testing with pytest and docker-py...a lot of things! very slick! @bobcatwilson @mtomwing...
TRANSCRIPT
![Page 1: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/1.jpg)
System Testing with pytest and docker-py
By Christie Wilson (@bobcatwilson)& Michael Tom-Wing (@mtomwing)
github.com/keeppythonweird/pytest-dockerpy
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
![Page 2: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/2.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Hello!● Christie Wilson
○ Senior Developer @ Demonware○ Team Lead: Test Tools
● Michael Tom-Wing○ Software Engineer in Test @ Demonware○ Focus on automation and quality
● We’re from Canada!
![Page 3: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/3.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
● Video Game Industry
● Online services for games
● Come see us in the vendor area!
![Page 4: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/4.jpg)
Once upon a system test
Commander McFluffles and the Quest for Quality
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 5: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/5.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Overview
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 6: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/6.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Testing at Demonware 2011-2016
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 7: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/7.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
What is testing?
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 8: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/8.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Best Practices for System Testing
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 9: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/9.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
pytest fixtures and docker-py
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 10: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/10.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Takeaways for Dev and Ops
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 11: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/11.jpg)
Once upon aSystem test
![Page 12: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/12.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Testing at Demonware 2011-2016
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 13: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/13.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Demonware - 2011
![Page 14: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/14.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Demonware - RIP other testing methods
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 15: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/15.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Demonware - 2016
![Page 16: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/16.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Demonware - 2016
● Test Tools team● Variety of tests
○ Unit tests○ Integration tests○ System tests
![Page 17: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/17.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
What is testing?
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 18: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/18.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
What is testing? Why do we test?
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 19: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/19.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Why do we test?● To increase confidence in our software● Avoid regressions● Document behaviour
![Page 20: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/20.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Why don’t we test?● NOT to find all the bugs in our software● Will never find all bugs
@bobcatwilson @mtomwing
![Page 21: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/21.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Testing + Software Quality● Testing does not increase the quality of our software● By the time our tests run our software is already buggy
○ Introduce quality through requirements + design!
● But...
![Page 22: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/22.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Testing + Software Quality● Tests provide metrics to let us reason about quality
○ E.g. coverage, timing, # of logs
● Untested software is always VIEWED as lower quality● Less information about the quality
![Page 23: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/23.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Example
SERVICE DATABASECLIENT LIB
![Page 24: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/24.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
How to test it● Unit tests● Integration tests● System tests
![Page 25: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/25.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Unit Tests● Unit tests: ~100% coverage● Integration tests● System tests
SERVICE DATABASECLIENT LIB
![Page 26: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/26.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Integration Tests● Unit tests: ~100% coverage● Integration tests: service <-> DB● System tests
SERVICE DATABASECLIENT LIB
![Page 27: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/27.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
System TestsTest the entire system
Pros● Most valuable● Most likely to find bugs
Cons● Slowest● Hardest to maintain
![Page 28: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/28.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
System Tests● Unit tests: ~100% coverage● Integration tests: service <-> DB● System tests: happy path, few simple failures
DATABASE
CLIENT LIB
SERVICE
![Page 29: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/29.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Best Practices for System Testing
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 30: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/30.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
BEST PRACTICES● Use fresh state between tests● Will help avoid dependencies between tests
○ e.g. test ordering shouldn’t affect their outcomes
● Docker helps make this very easy!
![Page 31: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/31.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
BEST PRACTICES● Ensure tests can run on build servers and locally
○ Ease the burden for writing and running tests
● Restrict the test environments you’ll support○ e.g. Linux, OSX, toaster
![Page 32: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/32.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
BEST PRACTICES● Tests should clean up after themselves● Fail fast● Fail informatively
![Page 33: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/33.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Glue Code
![Page 34: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/34.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
pytest fixtures and docker-py
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 35: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/35.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
PyTest● Python testing library● v.s. unittest = less boilerplate● More batteries included
○ e.g. fixtures, plugins
![Page 36: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/36.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
PyTest Fixtures● Provide setup and teardown for tests● Pytest will ensure that the setup
and teardown always happen○ And in that order!
● System tests generally set up a lot of things!
● Very slick!
![Page 37: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/37.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
PyTest Fixtures
SETUP
TEARDOWN
YOUR TEST
![Page 38: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/38.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
PyTest Fixtures
SETUP TEARDOWNTEST #1
SETUP TEARDOWNTEST #2
SETUP TEARDOWNTEST #3
![Page 39: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/39.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
PyTest Fixtures - ScopeSETUP
TEARDOWN
TEST #1
TEST #2
TEST #3
![Page 40: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/40.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
PyTest Fixtures - ScopeSETUP
TEARDOWN
TEST #1SETUP TEARDOWN
TEST #2SETUP TEARDOWN
![Page 41: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/41.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
DockerDOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER
![Page 42: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/42.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Docker● Challenging service setup = docker● Setup + teardown: bash scripts
![Page 43: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/43.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Docker-py● Python library for using docker● Interface is 1:1 with the REST
interface○ Can be a bit clunky
![Page 44: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/44.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Docker-py1. Create client2. Pull image3. Create container4. Start container5. Remove container
![Page 45: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/45.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Create client
![Page 46: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/46.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pull image
![Page 47: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/47.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pull image - errors
Thanks Steven Erenst!
![Page 48: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/48.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Create container
![Page 49: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/49.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Start container
![Page 50: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/50.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Kill and remove container
![Page 51: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/51.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pytest Fixtures + Docker-py
CREATE CONTAINER DELETE CONTAINERTEST #1
CREATE CONTAINER DELETE CONTAINERTEST #2
CREATE CONTAINER DELETE CONTAINERTEST #3
![Page 52: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/52.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pytest Fixtures + Docker-py
RUN percona:5.6 DELETE percona:5.6TEST #1
TEST #2
TEST #3
RUN httpd:2.4
DELETE httpd:2.4
RUN redis DELETE redis
RUN percona:5.6 DELETE percona:5.6
RUN redis DELETE redis
RUN percona:5.6 DELETE percona:5.6
RUN redis DELETE redis
![Page 53: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/53.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pytest Fixtures + Docker-py
![Page 54: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/54.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pytest Hooks● pytest magic!
![Page 55: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/55.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pytest Fixtures - Docker Logs
![Page 56: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/56.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Pytest Fixtures - Docker Logs
![Page 57: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/57.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
What about docker-compose?● Works well when the deployment is static between tests
○ Not as well suited when deployment is different for each test
● Integrates with pytest fixtures!○ e.g. Use a fixture to run docker-compose up
● docker-py can help
![Page 58: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/58.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
What about docker-compose?
![Page 59: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/59.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Gotchas● Wait for the service to start (backoff)● Maximize container startup speed!
![Page 60: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/60.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Takeaways for Dev and Ops
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
![Page 61: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/61.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
What do I do with this?● Developers● Ops
![Page 62: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/62.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Developers● When to write tests and when not to● Try some TDD: start with a system test
![Page 63: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/63.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Developers - Introduce System Tests● Add one system test to each piece of software you own● Make sure tests can run:
○ With as little setup as possible○ As quickly as possible
● Add the system tests to your CI
![Page 64: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/64.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Developers - Already Have System Tests● Do you need all the tests you have?
○ Can you replace with integration or unit tests?○ How many retest functionality?○ Can some of the tests be removed?
● Can the tests be faster?
![Page 65: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/65.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Ops● One off scripts: don’t need system tests● Scripts and automation that will be used
in the future need system tests○ What is one bare minimum system test you can
add?
● Use automation to regularly run your tests○ e.g. Travis CI
![Page 66: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/66.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Ops - Tests for tools● Tools that use services you can run:
○ Use something like pytest + docker-py
● Tools that use services you can’t run (e.g. AW$):○ Can you run a short system test, e.g. once per week?○ Is it going to cost you a lot?○ Make sure the tests clean up after themselves
![Page 67: System Testing with pytest and docker-py...a lot of things! Very slick! @bobcatwilson @mtomwing goo.gl/JkzmYJ PyTest Fixtures SETUP TEARDOWN YOUR TEST @bobcatwilson @mtomwing goo.gl/JkzmYJ](https://reader033.vdocuments.mx/reader033/viewer/2022042418/5f33bbbfafc52a781c2581cb/html5/thumbnails/67.jpg)
@bobcatwilson @mtomwinggoo.gl/JkzmYJ
Overview github.com/keeppythonweird/pytest-dockerpy@bobcatwilson@mtomwing