automated acceptance testing from scratch
Post on 15-Jul-2015
242 Views
Preview:
TRANSCRIPT
• Daniel Davis
• Software Developer for 8 years
• Fun Fact:
– I recently went to see “Laser Cat” in DC
Who Am I?
Really though, who are you?
• Came from Java world
• Python developer for 2 years
• DevOps
– Lots of work with automation and quality
• First introduced to AAT about 2 years ago
– Now I use it at my client!
• Go to the website, click around, make sure
things look right…
• Problem: Scaling
– What happens as the site continues to grow?
– What happens as we deploy more frequently?
• We need a way to test the functionality of
the site
Simple!
• Tests written at the functional/business
level
• Can be read by non-technical people
• Doesn’t have to be automated to provide
value!!!
What is Acceptance Testing
Automated Acceptance Testing
• Great for covering a lot of ground
• Not so good for very specific things
• Java/JVM world: Cucumber-JVM/JBehave
• Ruby: Cucumber-Ruby
• Ruby on Rails: Cucmber-Rails
• Node.js/Javascript: Cucumber-Javascript
– AngularJS: Protractor not the same
• C#/.NET: SpecFlow
• Python: Behave
Cucumber
16
• Cucumber’s syntax and style is virtually
the same across all the platforms!
• Tonight’s examples will be in Python
Good News Everyone…
17
• One example, from start to finish
– This is how we solve problems
• Learning Objectives:
– Write an acceptance test
– Automate that test using Selenium
– Incorporate that test into your Continuous Integration
• Stay high level
– Not enough time to go deep
Let’s get started!
18
• Objective: Want to test Upvote/Downvote
functionality
• Follow Along:
– https://github.com/Ooblioob/aat_demo/
theCatAPI.com
• Install your cucumber tool of choice
• Directory structure:
project-root/
|-- features/
|-- steps/ (we’ll talk about this soon)
|-- pages/ (we’ll talk about this soon)
Project Setup
23
• A Human-readable syntax describing the
test
• Consists of a main “feature” and scenarios
describing the feature
Writing Gherkin
Feature: Title (one line describing the story/feature)
As a [role]
I want [feature]
So that [benefit]
Scenario: Title1 (for Behaviour 1)
Given [context or setup]
And [some more context]...
When [event occurs]
Then [expected outcome]
And [another outcome]...
Scenario: Title2 (for Behaviour 2)
…
Writing Gherkin
26
TheCatAPI.com
27
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/
features/upvote.feature
• Value even when not automated
– Clear, concise way to manually test
– Developer friendly
• Not extremely specific…
– What does it mean “a new cat image should be loaded…”?
– Declarative vs Imperative
– Don’t get bogged down in precision/verboseness
Observations
• Create an “environment.py” file in the
features directory
– Defines the test runner setup
• (optional) Create an environment.cfg file in
the features directory
– Holds editable configurations for your setup
• Create a “steps.py” file in the steps directory
– Contains the implementation of your steps
Project Setup
31
Environment.py
32
Full Example:
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/fe
atures/environment.py
• Setting up Selenium
– Select a driver, like Chromedriver, PhantomJS
or SauceLabs
• Setting up a logger
• Initializing the context object
• Set defaults
What goes in Environment.py?
33
• Parses each line of Gherkin
• Decide how to implement
Glue code (steps)
home.feature
steps/steps.py
Writing your implementation
Example:
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/featu
res/steps/steps_home.py
• Making references to context
– Way of passing information between steps
• What is “Home” in the context?
– Class representing the homepage
– Part of “Page-object model”
Observations
36
• For each page element:
– Selectors: Id, CSS, XPATH, etc
– get() method
– commonly used attributes
• Define navigation to the page
– go()
• Define other actions / helpers
– Storing info, waits, etc.
Writing your pages
• Need to setup the web driver
– Let’s use ChromeDriver
• https://sites.google.com/a/chromium.org/chromedri
ver/getting-started
Environment.py revisited
45
• Use page-object model to organize code
• Write simple, reusable methods to
implement pages
• Installed Chromedriver
Recap
• Now that we have tests
– Validate site is up and running
– Validate deployments
– Ensure new code doesn’t break existing
functionality (regression)
Why Continuous Integration?
• PhantomJS
– Headless browser
– Based on Webkit
– Fast
• Works with Selenium
Solving the problem
51
• Black Box
– Can’t see problems when they happen
• Can’t handle certain things
– e.g. alerts
• Buggy, but getting better
Problems with PhantomJS
52
At this time, I don’t recommend it…
• Test different browser / os configurations
– Don’t need to own that hardware
• Integration with many CI tools
– Jenkins, Travis, CircleCI, Bamboo, Teamcity…
• Video recording/playback
– No longer a black box!!!
Advantages
54
• Sign Up for an account:
– https://saucelabs.com/signup/trial
• Receive an API key
Sauce Labs Setup
• Modify your environment.py file to use
Sauce instead of Chrome
– Provide username and api key
– Select a browser and OS to use
Sauce Labs Setup
59
Sauce Labs Setup
60
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/f
eatures/environment.py
Sauce Labs Setup
61
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/f
eatures/environment.py
• Can only test what’s available through the
browser
• Selenium can be painful at times
– May not wait for things to load
– Debugging can be hard
Limitations of AAT
• Data
– Not a great choice for data-heavy applications
– Not impossible, but trickier:
• Launch a separate test server?
• Create a “testing mode” where transactions are
rolled back
– Might be better covered by Integration Testing
Limitations of AAT
65
• https://github.com/Ooblioob/aat_demo
• https://cukes.info/
• http://jenisys.github.io/behave.example/
• https://docs.saucelabs.com/reference/sauc
e-connect/
Resources
66
top related