web acceptance testing with behat
Post on 08-May-2015
6.507 Views
Preview:
DESCRIPTION
TRANSCRIPT
AUTOMATEDWEB ACCEPTANCE TESTINGWITH BEHAT
Fabian KissYMC AG | May 2012
Picture by viZZZual.com, licensed under the CC BY 2.0
TESTINGIN AGILE
Pict
ure
by S
heff
ield
Tig
er, l
icen
sed
unde
r the
CC
BY 2
.0
MIKE COHN'S AGILE TESTING PYRAMID
Automated
tests
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
UI
Service
Unit
MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
UI
Service
Unit
least investment
most investment
MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
UI
Service
Unit
least investment
most investment
BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
Manual + automated tests
BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
AcceptanceTesting
...
Unit Testing...
Usability Testing
...
Performance Testing
…
BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
AcceptanceTesting
...
Unit Testing...
Usability Testing
...
Performance Testing
…
Business-facing
Technology-facing
Supp
ort p
rogr
amm
ing Critiqu e pro duct
BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
AcceptanceTesting
...
Unit Testing...
Usability Testing
...
Performance Testing
…
Business-facing
Technology-facing
Supp
ort p
rogr
amm
ing Critiqu e pro duct
Manual + automated tests
Manual + automated tests
BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
AcceptanceTesting
...
Unit Testing...
Usability Testing
...
Performance Testing
…
BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
AcceptanceTesting
...
Unit Testing...
Usability Testing
...
Performance Testing
…
BDD
TDD/BDD
BDD
BDD
Specification-oriented BDD
Scenario-oriented BDD
BDD
Specification-oriented BDD
Scenario-oriented BDD
RSpec
BDD
Specification-oriented BDD
Scenario-oriented BDDCucumber
Behat
RSpec
BDD
Specification-oriented BDD
Scenario-oriented BDDCucumber
Behat
Gherkin DSL
RSpec
BDD
Specification-oriented BDD
Scenario-oriented BDDCucumber
Behat
Gherkin DSL
RSpec
BDD
Specification-oriented BDD
Scenario-oriented BDDCucumber
Behat
Gherkin DSL
RSpec
...
.. .
GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action Then some testable outcome is achieved And something else we can check happens too
GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action Then some testable outcome is achieved And something else we can check happens too
GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action Then some testable outcome is achieved And something else we can check happens too
GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action Then some testable outcome is achieved And something else we can check happens too
GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action Then some testable outcome is achieved And something else we can check happens too
GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action Then some testable outcome is achieved And something else we can check happens too
GHERKIN EXAMPLE
http://docs.behat.org/guides/1.gherkin.html
Scenario: Wilson posts to his own blog Given I am logged in as Wilson When I try to post to "Expensive Therapy" Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog Given I am logged in as Wilson When I try to post to "Greg's antitax rants" Then I should see "Hey! That's not your blog!"
GHERKIN EXAMPLE
http://docs.behat.org/guides/1.gherkin.html
Scenario: Wilson posts to his own blog Given I am logged in as Wilson When I try to post to "Expensive Therapy" Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog Given I am logged in as Wilson When I try to post to "Greg's antitax rants" Then I should see "Hey! That's not your blog!"
Feature
STEPS & STEP DEFINITIONS
http://docs.behat.org/quick_intro.html#basic-usage
Given some precondition
/** * @Given /^some precondition$/ */public function somePrecondition(){ //...}
STEPS & STEP DEFINITIONS
http://docs.behat.org/quick_intro.html#basic-usage
Given some precondition
/** * @Given /^some precondition$/ */public function somePrecondition(){ //...}
Step
Step definition
STEPS & STEP DEFINITIONS
http://docs.behat.org/quick_intro.html#basic-usage
Given some precondition
/** * @Given /^some precondition$/ */public function somePrecondition(){ //...}
Step
Step definition
Matching
Run
STEP ARGUMENTS
http://docs.behat.org/quick_intro.html#some-more-behat-basics
When I try to post to "Expensive Therapy"
/** * @When /^I try to post to "([^"]*)"$/ */public function iTryToPostTo($site){ //...}
STEP ARGUMENTS
http://docs.behat.org/quick_intro.html#some-more-behat-basics
When I try to post to "Expensive Therapy"
/** * @When /^I try to post to "([^"]*)"$/ */public function iTryToPostTo($blog){ //...}
Argument
SCENARIO BACKGROUNDS
http://docs.behat.org/guides/1.gherkin.html#backgrounds
Background: Given I am logged in as Wilson
Scenario: Wilson posts to his own blog When I try to post to "Expensive Therapy" Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog When I try to post to "Greg's antitax rants" Then I should see "Hey! That's not your blog!"
SCENARIO BACKGROUNDS
http://docs.behat.org/guides/1.gherkin.html#backgrounds
Background: Given I am logged in as Wilson
Scenario: Wilson posts to his own blog When I try to post to "Expensive Therapy" Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog When I try to post to "Greg's antitax rants" Then I should see "Hey! That's not your blog!"
SCENARIO OUTLINES
http://docs.behat.org/guides/1.gherkin.html#scenario-outlines
Scenario Outline: Wilson posts to a blog Given I am logged in as Wilson When I try to post to "<blog>" Then I should see "<authorization>"
Examples: | blog | authorization | | Expensive Therapy | Your article was published. | | Greg's antitax rants | Hey! That's not your blog! |
SCENARIO OUTLINES
http://docs.behat.org/guides/1.gherkin.html#scenario-outlines
Scenario Outline: Wilson posts to a blog Given I am logged in as Wilson When I try to post to "<blog>" Then I should see "<authorization>"
Examples: | blog | authorization | | Expensive Therapy | Your article was published. | | Greg's antitax rants | Hey! That's not your blog! |
x 2
STEP TABLES
http://docs.behat.org/guides/1.gherkin.html#tables
Scenario: Wilson posts to a blog Given I am logged in as Wilson When I try to post to: | blog | | Expensive Therapy | | Greg's antitax rants | Then I should see an authorization outcome
STEP TABLES
http://docs.behat.org/guides/1.gherkin.html#tables
Scenario: Wilson posts to a blog Given I am logged in as Wilson When I try to post to: | blog | | Expensive Therapy | | Greg's antitax rants | Then I should see an authorization outcome
x 2
ACCESSING THE WEB
Pict
ure
by M
isse
rion,
lice
nsed
und
er th
e CC
BY
2.0
MINK API
Driver object
$client = new \Selenium\Client($host, $port);$driver = new \Behat\Mink\Driver\SeleniumDriver( 'firefox', 'base_url', $client);
Session object
$session = new \Behat\Mink\Session($driver);$session>start();$session>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session>getPage();$elementByCss = $page>find('css', 'h3 > a');$elementByCss>click();
http://mink.behat.org/
MINK API
Driver object
$client = new \Selenium\Client($host, $port);$driver = new \Behat\Mink\Driver\SeleniumDriver( 'firefox', 'base_url', $client);
Session object
$session = new \Behat\Mink\Session($driver);$session>start();$session>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session>getPage();$elementByCss = $page>find('css', 'h3 > a');$elementByCss>click();
http://mink.behat.org/
MINK API
Driver object
$client = new \Selenium\Client($host, $port);$driver = new \Behat\Mink\Driver\SeleniumDriver( 'firefox', 'base_url', $client);
Session object
$session = new \Behat\Mink\Session($driver);$session>start();$session>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session>getPage();$elementByCss = $page>find('css', 'h3 > a');$elementByCss>click();
http://mink.behat.org/
Sahi, Goutte, Zombie.js
MINK API
Driver object
$client = new \Selenium\Client($host, $port);$driver = new \Behat\Mink\Driver\SeleniumDriver( 'firefox', 'base_url', $client);
Session object
$session = new \Behat\Mink\Session($driver);$session>start();$session>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session>getPage();$elementByCss = $page>find('css', 'h3 > a');$elementByCss>click();
http://mink.behat.org/
Sahi, Goutte, Zombie.js
MINK API
Driver object
$client = new \Selenium\Client($host, $port);$driver = new \Behat\Mink\Driver\SeleniumDriver( 'firefox', 'base_url', $client);
Session object
$session = new \Behat\Mink\Session($driver);$session>start();$session>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session>getPage();$elementByCss = $page>find('css', 'h3 > a');$elementByCss>click();
http://mink.behat.org/
Sahi, Goutte, Zombie.js
BEHAT REPORT
~/Behat$ behat features/blogpost.feature
Feature: Blog posting As a blogger I want to post an article to a blog
Scenario: Wilson posts to his own blog # features/blogpost.feature:5 Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs() When I try to post to "Expensive Therapy" # FeatureContext::iTryToPostTo() Then I should see "Your article was published." # FeatureContext::iShouldSee()
Scenario: Wilson fails to post to somebody else's blog # features/blogpost.feature:10 Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs() When I try to post to "Greg's antitax rants" # FeatureContext::iTryToPostTo() Blog "Greg's antitax rants" does not exist Then I should see "Hey! That's not your blog!" # FeatureContext::iShouldSee()
2 scenarios (1 passed, 1 failed)6 steps (4 passed, 1 skipped, 1 failed)
BEHAT REPORT
~/Behat$ behat features/blogpost.feature
Feature: Blog posting As a blogger I want to post an article to a blog
Scenario: Wilson posts to his own blog # features/blogpost.feature:5 Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs() When I try to post to "Expensive Therapy" # FeatureContext::iTryToPostTo() Then I should see "Your article was published." # FeatureContext::iShouldSee()
Scenario: Wilson fails to post to somebody else's blog # features/blogpost.feature:10 Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs() When I try to post to "Greg's antitax rants" # FeatureContext::iTryToPostTo() Blog "Greg's antitax rants" does not exist Then I should see "Hey! That's not your blog!" # FeatureContext::iShouldSee()
2 scenarios (1 passed, 1 failed)6 steps (4 passed, 1 skipped, 1 failed)
default formatter (“pretty”)
all formatters:http://docs.behat.org/guides/6.cli.html#format-options
format pretty
CHALLENGES...
DEVELOPERS WRITESCENARIOS
Pict
ure
by M
arja
n Kr
ebel
j, lic
ense
d un
der t
he C
C BY
2.0
DEVELOPERS WRITESCENARIOS
AN ARTIFACT ASKED FOR?UNDERSTANDABILITY?
Pict
ure
by M
arja
n Kr
ebel
j, lic
ense
d un
der t
he C
C BY
2.0
Pict
ure
by C
onor
Law
less
, lic
ense
d un
der t
he C
C BY
2.0
SCENARIO ABSTRACTION LEVEL
Pict
ure
by C
onor
Law
less
, lic
ense
d un
der t
he C
C BY
2.0
SCENARIO ABSTRACTION LEVELWhen I fill the form
When I fill my first name in
to field “first name”
When I fill my last name int
o field “last name”
When I fill ...VS.
REDUNDANCY
Pict
ure
by R
ob F
aulk
ner,
licen
sed
unde
r the
CC
BY 2
.0
REDUNDANCY
LIMITED: GHERKIN'S REUSE CAPABILITIESLIMITED: UNDERSTANDING OF THOSE
Pict
ure
by R
ob F
aulk
ner,
licen
sed
unde
r the
CC
BY 2
.0
TIME-CONSUMING REFACTORING
Pict
ure
by E
arls
37a,
lice
nsed
und
er th
e CC
BY
2.0
TIME-CONSUMING REFACTORING
EXECUTION IN REAL-TIME
Pict
ure
by E
arls
37a,
lice
nsed
und
er th
e CC
BY
2.0
COMPLEX INFRASTRUCTURE
Pict
ure
by D
omen
ico
Nar
done
, lic
ense
d un
der t
he C
C BY
2.0
COMPLEX INFRASTRUCTURE
Pict
ure
by D
omen
ico
Nar
done
, lic
ense
d un
der t
he C
C BY
2.0
Behat Mink Driver Browser Emulator Browser
FURTHER REFERENCES: http://www.softwarequalityconnection.com/2011/08/how-to-improve-communication-between-qa-and-development/
http://dannorth.net/introducing-bdd/
http://www.infoq.com/news/2011/02/BDD-ATDD
http://www.slideshare.net/headrevision/the-concept-of-behaviordriven-development
http://www.slideshare.net/everzet/bdd-in-symfony2
SOME BEST PRACTICES: http://www.agileengineeringdesign.com/2012/01/7-deadly-sins-of-automated-software-testing/
http://elabs.se/blog/15-you-re-cuking-it-wrong
http://www.engineyard.com/blog/2009/15-expert-tips-for-using-cucumber/
http://eggsonbread.com/2010/09/06/my-cucumber-best-practices-and-tips/
top related