dave haeffner's proven method to grading the quality of selenium tests
TRANSCRIPT
How To Grade Your Selenium Tests
Let’s talk about tests
What Makes a Bad Test?
• Fails For No Good Reason
• Changes in AUT, Need to Update Multiple Tests
• Slow
• Unreliable
• Hard to Understand and Maintain
What Makes a Good Test?• Written for BDD or xUnit test framework
• Test one thing (atomic)
• Each test can be run independently (autonomous)
• Anyone can understand what it is doing
• Group similar tests together
• Centralized setup/teardown
• Uses Page Objects
A Testing Rubric (Each test starts with 100 points, deduct as necessary)
Item Belong in tests? Score
Selenium Commands No -3 per (max of -9)
Locators No -2 per (max of -8)
Selenium setup/teardown* No -20
Hard-coded sleeps No -5 per (max of -20)
Implicit wait calls No -10
Explicit Wait calls No -3
Conditionals No -5 (max of -20)
Calls to Page Objects Yes N/A
Assertion(s)* Yes N/A
Let’s talk about Page Objects
What Makes a Bad PO?
• Gigantic in size (e.g., hundreds or thousands of lines in code)
• Contain overly complicated logic
• Return not enough or too much information to the test (leaky abstraction)
• Assertions happening in the PO instead of the test
What Makes a Good PO?• Contains State (e.g., locators)
• Contains Behavior (e.g., methods to interact with the page)
• Returns some information about the page (e.g., new page object, text from the page, a boolean result, etc. — never a WebElement)
• Verifies page ready state as part of initialization
A Page Object Rubric (Each Page Object starts with 100, deduct points as necessary)
Item Belong in POs? Score
> 200 lines of code No -5 per 50 lines over
Assertions No* -5 per (max of -20)
Hard-coded sleeps No -5 per (max of -20)
Implicit wait calls No -10
Explicit waits Yes N/A
Verify page ready state* Yes -8 if not verifying an element
Locators* Yes N/A
http://se.tips/se-waiting-jim-evans
Let’s talk about locators
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Start with IDs and Classes
Locator Strategies• Class
• CSS selectors
• ID
• Link Text
• Partial Link Text
• Tag Name
• XPath
Good locators are: • unique • descriptive • unlikely to change
That rules a few of these out
Start with IDs and Classes
Use CSS or XPath (with care)
Which is better CSS or XPath?
A Locator Rubric (Each locator gets a grade)
Item Letter Grade E.g.,
Dynamic locators D- User account specific or tied to page render
Tied to page layout D XPath: / / / CSS: > > >, etc.
Text on the page C Link text, page copy
Reasonable traversal B Parent to child w/in an element node
Using semantic name B+ Input labels (name='username')
Semantic ID A Unique, descriptive, unlikely to change
Non-unique locator? -2 full letter grades
So…
What’s your testing GPA?Topic Weight
Tests 20%
Page Objects 30%
Locators 50%
http://seleniumconf.co.uk
http://se.tips/master-visual-testing