Regression Testing with SymfonyTools and techniques for building your test suite

Introducing the speaker

Joachim Unger@joeywannabe

Symfony Explorer since 2.0Web Developer since 2010Former scientist at ABB Robotics

Software Developer @LittleBird GmbHBerlin

Part 1: Exploring the techiques

Regression Testing Test granularities in web projects Basic design patterns Symfony test components Behaviour Driven Development Continuous Integration Lessons learned

Regression Tests

Discover new software bugs (regression) Verify bugfixes on existing functionality Validate changes against other components Include rerunning previously run tests Run on each code change Ensure product quality Minimize risks

Types of Regression Tests

Progressive Regression Tests Specification has changed Code has been changed

> Add new test cases

Correcting Regression Tests Specification has not changed Test code changes against existing test cases

Regression Tests in web projects

Defined test strategy Test-Driven Programming

Tests for all code changes (e.g. bugfixes) Tests have to be up-to-date Continuous Refactoring

Automated Test Suite Run on each code change Smoke, Acceptance and Unit Tests Continuous Integration

Software Quality

ORM/DBAL

Model

Controller

Utility

View

Browser

Structural Quality Design and structure of code Loose coupling of components Ensures extensibility of code

Functional Quality Error-free execution of application Strongly influences user acceptance

Unit Tests

Verify functionality of single component Allow safe refactoring of code parts Must run successfully on each change Test is written with/before code

Assert that the result of foobar(“foo“) is “bar“.

Acceptance Tests

Verify the functional requirements Ensure the overall application usability Test specification provided by stakeholder

Check that a click on button„Put on shopping cart“ puts the item

into the shopping cart

Smoke Tests

Verify the overall application functionality Good to identify crucial bugs

Check that the url of a product pageis really showing a product page

Separation Of Concerns

Separation of task in independent blocks Loose coupling between components Heavy use of dependency injection

MVC as an example Model: Exchange and storage of data View: Representaton of data Controller: Application Logic

Decoupling of unit tests

ORM/DBAL

Model

Controller

Utility

View

Browser

Inversion Of Control

Removal of static assignments Framework only works with abstraction of components Object graph is instantiated dynamically

Separation of execution and implementation

Unit Testing in PHP

phpunit: Status quo for testing in PHP Created by Sebastian Bergmann Originated as port of Junit Supports various features / extensions

Framework for embedding tests in application Execution as test suite for an project Mocking of injected objects Code & coverage analysis

Various functions for test assertions

Test Components in Symfony

Unit Tests Enforce usage of dependency injection Testing of DBAL, model and relations Testing of components, utility classes Mocking of dependencies

Functional Tests Testing page reachability Testing of page content and links Easy utilization for smoke tests

http://symfony.com/doc/current/book/testing.html

Behaviour Driven Development

Enhancement to Test-Driven Development Introduced by Dan North in 2006 Tests act as functional specification Missing link between developer and stakeholder

Justify functional requirements at beginning Software developed against functional specification Specified in a formal language (no experts needed) Clear focus on functionality

BDD Test Specification

Scenario: Put item in shopping cartGiven the case, the customer wants to buy a movie

And his shopping cart is emptyAnd there are 2 movies in stock

When he puts the item in the shopping cartThen the item should show up in the shopping cart

And the shopping cart counter should show 1And 1 movie should be in stock

Gherkin

Mink

BDD Testing Bundles

Behat

Formal language to describe test cases

Encapsulation of different browser drivers

Test Engine for executing Gherking test case

Integration of Mink into SymfonyMinkBundle

BehatBundle

Integration of Behat into Symfony

Other Testing Bundles

DoctrineFixturesBundle Preparation of test data sets

LiipFunctionalTestBundle Simplifies preparation and execution Utilizes DoctrineFixturesBundle

PSSMockeryBundle (Symfony2MockerExtension) Mocking of services in DIC Demarcation of tests

Continuous Integration

Continuous process to apply quality control Various CI Systems available

Bamboo, Jenkins, CruiseControl Successfull development process for CI

Direct feedback -> Short execution times Frequent checkins Commits to trunk Automated test chain Automated deploy -> Continuous delivery

Automated Build Smoke Tests Unit Tests Syntax Tests Coding Standards Copy/Paste Detection Documentation

Continuous Integration

Periodical Build Smoke Tests Unit Tests Syntax Tests Coding Standards Copy/Paste Detection Documentation+ Acceptance Tests+ Other intensive tests

Favoured Test Suite

Test-Driven Development Testing influences whole development process Loose coupling of components

Test chain according to granularities Smoke Tests: Routes, Firewalls (Reachability) Unit Tests: Modular components, Utility classes DBAL Tests: Entity, Relations and Repository BDD Tests: User Interface (Usability)

Continuous Integration System Triggers test chain on each commit Verifies state before final deploy

Lessons learned

Commitment to TDD needed Testing influences software design Testing takes its time Quality can hardly be achieved later on Full test coverage almost unachievable Regression tests need automated test suite Testing only way to ensure software quality

Questions

Questions regarding the practicalusage probably covered in part 2