the test pyramid
DESCRIPTION
How to balance your automated test suite to have a good feedback, from units to end-to-end tests.TRANSCRIPT
How to balance your test suite to have a good feedback, from units to end-to-end tests.
Thiago Ghisi@thiagoghisi
The Test
Pyramid
2002 2007 2010
Tests, seriously? It’s a mobile app...[ ]
Shared codebases108
Devs, QAs and Ops 144
Brian Marick@marick
January, 2009
http://www.slideshare.net/jezhumble/creating-maintainable-automated-acceptance-tests
Mike Cohn@mikewcohnOctober, 2009
http://www.slideshare.net/jezhumble/creating-maintainable-automated-acceptance-tests
http://www.slideshare.net/jezhumble/creating-maintainable-automated-acceptance-tests
Jez Humble@jezhumbleAugust, 2010
Feedback
Perfect!!! Let’s follow the test pyramid guidelines…[ ]
20% of code coverage. :-(
really hard to isolate the dependencies. :-(
easy. :-)
FunctionalTests • UI/End-to-end testing [with full
stack]
• Tests user flow
• BDD, ATDD
• Feedback after a coffee
business likes that ;)
sounds perfect
Cucumber feature file
Honeydew steps
Cucumber
Honeydew
wraps UIAutomator
step definitions in Ruby
+=]
☺
https://github.com/amplify-education/honeydew
why cucumber?https://speakerdeck.com/nicholaspufal/why-bdd-is-misunderstood
Cool.. write as much user-like tests as you can… PERFECT!![ ]
UI test results history (1)
UI test results history (2)
UI test results history (3)
Flaky / Fragile
Slow
Costy
Feedback
the build monkey
http://watirmelon.com/tag/software-testing-pyramid/
Few UI tests
OK… let’s do it right now…[ ]
Unit Test
• Verify small pieces individually
• Run fast
• Stable
• Run a lot of times during development
• Feedback instantly
Unit tests results history
Feedback
Android? It’s Java, let’s TDD.
Android TDD
java.lang.RuntimeException: Stub!
Android SDK classes are not trivial
eyes on docs
huge API
test object
result()
assert(expected, obj.result())
Humm, I’ve seen that..
The ‘2 + 2 = 4’ example!
the object u’re looking at
dependencies
getData()
saveSomething()
getData()
saveSomething()
when(stub.getData()).thenReturn(data)
verify(mock).saveSomething()
http://martinfowler.com/articles/mocksArentStubs.html
mockstubdoublefake
dummy
Unit Testswith
Dependency injection
Roboguice
Robolectric
jUnit
DI
Mocks
Android Shadows
@Test
Lots of Unit tests
Where are you?
http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
http://fabiopereira.me/blog/2012/03/18/introducing-depth-of-test-dot/
external services
backend
libs
API contracts validation
PACTO
https://github.com/thoughtworks/pactohttps://speakerdeck.com/marcosccm/evoluindo-ecossistemas-com-consumer-driven-contracts
Service/Integration tests instead of end-to-end tests
Annnnd.. the manual tests?[ ]
Session based exploratory/usability
tests
http://blog.kalistick.com/Agile/harden-testing-pyramid/
http://watirmelon.com/2011/06/10/yet-another-software-testing-pyramid/
http://blog.softed.com/2013/08/26/1883/
UserJourney
UnitTests of single methods without dependencies
IntegrationTests of multiple integrated method
UI
Test automation
Testing on the level of the users - incl. browser, network, databases, external systems, …
Connection of UI tests to a persona like flow through
the system
$$$slowerfewer
$fastermore
closer to the user
closer to the developer
https://my.thoughtworks.com/docs/DOC-21002
http://www.slideshare.net/nashjain/inverting-the-testing-pyramid
http://www.slideshare.net/guestc8adce/pivotal-labs-open-view-presentation-quality-assurance-and-developer-testing-presentation
http://blog.codeclimate.com/blog/2013/10/09/rails-testing-pyramid/
Feedback
Feedback
http://herdingcats.typepad.com/my_weblog/2011/02/a-radical-book.html
