functional testing the_good_the_bad_and_the_ugly

Automated Functional Testing

John Ferguson Smart

ConsultantTrainerMentorAuthorSpeakerCoder

Technical Agile Practices

Test Driven Development

Continuous Integration/Delivery

Automated

Acceptance Criteria

Refactoring

Pair

Progra

mming

“Without these technical prac1ces, your agile adop1on is hollow”-‐ Ma6 Wynne

Technical Agile Practices

Test Driven Development

Continuous Integration/Delivery

Automated Acceptance Criteria

Refactoring

Pair

Progra

mming

“Without these technical prac1ces, your agile adop1on is hollow”-‐ Ma6 Wynne

A good func<onal test tells a story

PracticesTools

QA loses its potential if only done at the end of the project

The Three Amigos

Living Documentation - test results that serve everyone

So show me the tools!

The Record-Replay Scam

in theory

in practice

The Record-Replay Scam

What do ogres and good functional tests have in common?

(This test is not like an ogre)

Remember...good automated acceptance tests have layers

Start off with the business objectives

Business-‐level requirements

Describe the business flow

High-‐level steps (business flow)

Describe the page interactions

Page-‐level interac<ons

Page implementation details

HTML details go here

Business Rules

Business Flow

Page/Component interac<ons

Page/Component details

Good automated acceptance tests have layers

Come on - can we see the tools now?

Unit tes<ng BDD tools (39%)

Given/When/Then (40%)

Tabular (13%)

WebDriver / Selenium 2

Mature browser automa<on

Wide adop<on

Cross-‐browser support

Cross-‐browser support

...or use Selenium Hub

appium

Uses the WebDriver API

Can run on SauceLabs (with some limita<ons)

WatirWeb Application Testing in Ruby

browser.goto 'http://bit.ly/watir-example'browser.text_field(:name => 'entry.0.single').set 'Watir'browser.checkbox(:value => 'Ruby').set

Ruby DSL for web tes<ng

WebDriver-‐based

Fluent and readable

No na<ve support for Page Objects?

http://bit.ly/watir-example'




GebGroovy Browser Automation

Browser.drive { go "http://myapp.com/login" assert $("h1").text() == "Please Login" $("form.login").with { username = "admin" password = "password" login().click() } assert $("h1").text() == "Admin Section"}

Groovy DSL for web tes<ng

WebDriver-‐based

Fluent and readable

Supports Page Objects

http://myapp.com/login

http://myapp.com/login

28

High level BDD

Many tools are available

29

Narrative:In order to increase sales of advertised articlesAs a sellerI want buyers to be able to easily find ads for articles they want to buy

Scenario: Searching by keyword and location

Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale.

search_by_keyword_and_location.story

Business-‐friendly specifica<ons

An example in JBehave

30


Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale.



Given Sally wants to buy a <present> for her son When she looks for '<present>' in the '<category>' category Then she should obtain a list of ads for <expected> for sale.

Examples:|present |category |expected||puppy |Pets & Animals | puppies||kitten |Pets & Animals | kittens||seiko |Jewellery & Watches| watch |

Tabular examples

31


Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of puppy ads


public class SearchAdsSteps { @Steps BuyerSteps buyer;

@Given("Sally wants to buy a $present for her son") public void buyingAPresent(String present) { buyer.opens_home_page(); }

@When("she looks for $keyword in the $category category") public void adSearchByCategoryAndKeyword(String category, String keyword) { buyer.chooses_category_and_keywords(category, keyword); buyer.performs_search(); }

@Then("she should obtain a list of $keyword ads") public void shouldOnlySeeAdsContainingKeyword(String keyword) { buyer.should_only_see_results_with_titles_containing(keyword); }}

Step implementa<ons in your favorite language

32

From web tests to living documentation

33

(Think “Two-CDs”)

34

2 Automate your acceptance criteria

1 Discover your acceptance criteria

4 Execute your acceptance tests

3 Implement your acceptance criteria

35


Feature: Browse CatalogIn order to find items that I would like to buyAs a customerI want to be able to browse through the catalog

Story: Browse by categoryIn order to find items more easilyAs a customerI want to be able to browse through the product categories

Acceptance CriteriaSee all the top-level categoriesBrowse through the category hierarchyShould display the correct products for each categoryEach category should have the correct sub-categories

Define acceptance criteria for each story

36



Scenario: See all top-level categoriesGiven I want to browse the catalogWhen I am on the home pageThen I should see the following product categories: Clothing, Accessories, Shoes

Clarify the acceptance criteria with examples

37


Story: Browse by categoryIn order to find items more easilyAs a customerI want to be able to browse through the product categories


Scenario: See all top-level categoriesGiven I want to browse the catalogWhen I am on the home pageThen I should see the following product categories: Clothing, Accessories, ShoesNarrative:

In order to find items more easilyAs a customerI want to be able to see what product categories exist


We now have executable specifica<ons

38


...but they are reported as ‘pending’

39

3 Implement your acceptance criteria

Narrative:In order to find items more easilyAs a customerI want to be able to see what product categories exist


40


What tests have been run

41

What requirements have been tested?


42


How did we show that a par<cular requirement

worked?

43


How did we show that a par<cular requirement

worked?

Demo time!

hYp://wakaleo.com

hYp://thucydides.info

Want to learn more?

http://thucydides.info




John Ferguson Smart

Thank you

functional testing the_good_the_bad_and_the_ugly

Technology