Domain Specific Languages for Selenium tests

Emily Bache, jfokus 2010

What is selenium?• Selenium is a suite of

tools to automate web application testing

• Includes Selenium RC (Remote Control) & Selenium IDE

Selenium RC

• Write test in Java (etc)

• Run your web app in a browser

Selenium IDE

• Firefox plug-in for recording tests

Why choose selenium over other tools?

• Run tests in a real browser– Use real brower's javascript

interpreter• Much of selenium itself is written in

Java• Comparatively large user base• Actively maintained – recent alpha

release of version 2.0

Selenium 2.0

• new WebDriver API• DefaultSelenium API unchanged

from 1.0• “bridge” class to ease migration.

Demo: Ad Orchestrator

Media Agency

Publisher

ConsumerCreate Ad Campaign

Provide Ad inventory

Receive Ads

Sign Up

Screencast: Create Publisher

• url of screencast demo

file://SEGGPS00.IBS.NET/USERS$/seemibac/My Documents/ericsson/saranac/testing aop/create_publisher_test.htm

@Test public void createNewPublisher() { selenium.click("menuItemPNewPublisher"); selenium.waitForPageToLoad("30000"); selenium.type("name", "Disney"); selenium.type("address", "24 Stars Ave"); selenium.type("zipCode", "91210"); selenium.type("city", "Hollywood"); selenium.type("country", "USA"); selenium.type("contactPersonName", "Emily"); selenium.type("contactPersonEmail", "emily@drutt.com"); selenium.type("contactPersonPassword", "123456"); selenium.type("contactPersonRetypePassword", "123456"); selenium.click("save"); selenium.waitForPageToLoad("30000"); verifyEquals(selenium.getTable("Users.1.1"), "Emily"); }

Wait a minute, why are we doing this?

• Agile Tests: – support developers– critique the product

Agile Testing Quadrants

Q2 Q3Q4Q1

Business Facing

Technology Facing

Critique P

rodu ct

Sup

porti

ng th

e te

am

Original Idea by Brian Marick

Agile Testing Quadrants - examples

Q2 Q3Q4Q1

Business Facing

Technology Facing

Critique P

rodu ct

Sup

porti

ng th

e te

am

Automated Functional tests

AutomatedUnit tests

Manual Exploratory testing

Performance tests

From “Agile Testing” by Crispin/Gregory

Selenium tests are usually Q2

• Help developers understand the app from the user's point of view

• Identify regression bugs

Q2 Q3Q4Q1

Business Facing

Technology Facing

Critique

Product

Sup

porti

ng

the

team

Bug insertion

CI Server

cost

time

End of iteration

Release tests

At a customer

Before check in

Good tests have these properties:

Readability Robustness

SpeedCoverage

So Selenium tests should:

• Be run often, every build, every checkin, every day– But they are often rather slow to execute

• Clearly explain what features the app has– But the IDE produces low abstraction widget-

oriented script• Support refactoring

– But the tests are often rather fragile, eg xpaths break when the html changes

Prefer ids to xpaths

selenium.selectFrame("mainFrame");selenium.click("//a[@id='menuItemPNewPublisher']/div");

selenium.click("menuItemPNewPublisher");

Robustness

Readability

Avoid sleep(), prefer waitFor()

selenium.waitForCondition( "selenium.getValue(\"id='userTableStatus'\") == \"loaded\";", "3000");

Robustness

Speed

try { Thread.sleep(3000); } catch (InterruptedException e) { }

public static void waitForElementForSeconds(Selenium selenium, String elementId, int seconds) { for (int halfSecond = 0; halfSecond < seconds * 2; halfSecond++) { if (selenium.isElementPresent(elementId)) { return; } Util.sleep(500); } throw new RuntimeException("Time out - Element not found: " + elementId);}

OR:

Check for correct page and absence of error messages

assertTrue(selenium.isElementPresent("loginButton"), "Not on login page");

//... do login ...

assertFalse(selenium.isVisible("badPassword"), "Failed to log in: bad password");assertTrue(selenium.isElementPresent("logoutLink"), "Failed to log in: no logout link shown");

Readability

Coverage

In @Before, remove alerts and frame selections

@Beforepublic void clearAlertsAndFrameSelections() { while (selenium.isAlertPresent()) { selenium.getAlert(); } selenium.selectFrame("relative=top"); while (selenium.isAlertPresent()) { selenium.getAlert(); }}

Robustness

Selenium tests are slow

Speed

• Develop tests in Firefox • Use TestNG instead of JUnit• Choose a representative subset to run at

each checkin, < 10 mins to run• Use Selenium Grid to parallelize• Use WebDriver instead of DefaultSelenium

Selenium 2.0 : WebDriver vs DefaultSelenium• WebDriver

– is faster to execute tests

– has clean API and support for PageObject pattern

– has immature support for Selenium IDE and Grid

• DefaultSelenium – runs in any broswer

(almost)– has “kitchen sink” API

Organize your test code

• Data structs for form data• PageObjects OR Scenario classes for

domain actions

Coverage

Robustness

Readability

Data structs for form data

Robustness

Readability

PublisherData newPublisher = new PublisherData("Disney") .withAddress( new AddressData("24 Stars Ave", "91210", "Hollywood", "USA")) .withContactPerson( new UserData("Emily", "emily@test.com", "123456"));

Data structs for form data (2)

Robustness

Readabilitypublic class PublisherData {

public String name; public AddressData address; public UserData user;

public PublisherData(String name) { this.name = name; }

public PublisherData withAddress(AddressData addressData) { this.address = addressData; return this; }

PageObject pattern • The page structure of your app helps

structure your test code• WebDriver (Selenium 2.0) has built-in

support for it

Coverage

Robustness

Readability

PageObject example

Coverage

Robustness

Readability public class NewPublisherPage implements Page {

private Selenium selenium;

public NewPublisherPage(Selenium selenium) { this.selenium = selenium; selenium.click("menuItemPNewPublisher"); selenium.waitForPageToLoad("30000"); }

//...

PageObject example (2)

Coverage

Robustness

Readabilitypublic class NewPublisherPage implements Page { //... public void enterPublisherDetails(PublisherData pd) { selenium.type("name", pd.name); selenium.type("address", pd.address.firstLine); selenium.type("zipCode", pd.address.zipCode); selenium.type("city", pd.address.city); selenium.type("country", pd.address.country); selenium.type("contactPersonName", pd.user.name); selenium.type("contactPersonEmail", pd.user.email); selenium.type("contactPersonPassword", pd.user.password); selenium.type("contactPersonRetypePassword", pd.user.password); }

//...

PageObject example (3)

Coverage

Robustness

Readabilitypublic class NewPublisherPage implements Page {

//...

public Page save() { selenium.click("save"); selenium.waitForPageToLoad("30000"); if (selenium.isVisible("errors")) return this; return new ViewPublisherPage(selenium); } //...

Test code with PageObject

TestCase Page Object

Encapsulate actions possible on a page.

Data Struct

Encapsulate form data

Controls what happens:• which pages • which data • what to assert

@Test public void createNewPublisher() { selenium.click("menuItemPNewPublisher"); selenium.waitForPageToLoad("30000"); selenium.type("name", "Disney"); selenium.type("address", "24 Stars Ave"); selenium.type("zipCode", "91210"); selenium.type("city", "Hollywood"); selenium.type("country", "USA"); selenium.type("contactPersonName", "Emily"); selenium.type("contactPersonEmail", "emily@test.com"); selenium.type("contactPersonPassword", "123456"); selenium.type("contactPersonRetypePassword", "123456"); selenium.click("save"); selenium.waitForPageToLoad("30000"); verifyEquals(selenium.getTable("Users.1.1"), "Emily"); }

What the test looked like last time

CreatePublisher test with Page Object

@Test public void createNewPublisher() { PublisherData newPublisher = new PublisherData("Disney") .withAddress( new AddressData("24 Stars Ave", "91210", "Hollywood", "USA")) .withContactPerson( new UserData("Emily", "emily.bache2@drutt.com", "123456"));

NewPublisherPage page = new NewPublisherPage(selenium); page.enterPublisherDetails(newPublisher); ViewPublisherPage viewPage = (ViewPublisherPage)page.save(); viewPage.verifyPublisherDetails(newPublisher); }

Scenario classes for domain actionspublic class PublisherScenarios { Selenium selenium; public PublisherScenarios(Selenium selenium) { this.selenium = selenium; } public void createPublisher(PublisherData pd) { openNewPublisherPage(); enterPublisherDetails(); save(); }

public void openNewPublisherPage() { selenium.click("menuItemPNewPublisher"); selenium.waitForPageToLoad("30000");

} //...

}

Robustness

Readability

@Test public void createNewPublisher() { PublisherData newPublisher = new PublisherData("Disney") .withAddress( new AddressData("24 Stars Ave", "91210", "Hollywood", "USA")) .withContactPerson( new UserData("Emily", "emily@test.com", "123456"));

PublisherScenarios scenarios = new PublisherScenarios(selenium); scenarios.createPublisher(newPublisher); scenarios.verifyPublisherDetails(newPublisher); }

Refactored test with Scenario and Data Structs

Test code in layers

TestCase Scenarios Data Struct

Methods for domain actions.

Encapsulates form data.

Controls what happens:• which actions • which data • what to assert

Demo: Ad Orchestrator

Media Agency

Publisher

ConsumerCreate Ad Campaign

Provide Ad inventory

Receive Ads

Sign Up

Testing Business Rules in Ad Orchestrator

@Test public void testGenderTargeting() throws Exception { ConsumerData male = new ConsumerData("000005") .withGender(Gender.MALE); consumerScenarios.signUp(male); ConsumerData female = new ConsumerData("000006") .withGender(Gender.FEMALE); consumerScenarios.signUp(female); //Add campaign targeting males String maleName = "GenderMale"; CampaignData maleCampaign = new CampaignData(mediaAgency, maleName); maleCampaign.targetData = new TargetData().withGender(Gender.MALE); maleCampaign.addDefaultWapBanner(); campaignScenarios.addCampaign(maleCampaign);

Testing Business Rules in Ad Orchestrator (2) //Add campaign targeting females String femaleName = "GenderFemale"; CampaignData femaleCampaign = new CampaignData(mediaAgency, femaleName); femaleCampaign.targetData = new TargetData().withGender( Gender.FEMALE); femaleCampaign.addDefaultWapBanner(); campaignScenarios.addCampaign(femaleCampaign); // Verify each consumer gets correct campaign checkAdForTelephoneNumberChoosesCampaign("000005", maleName, inventory); checkAdForTelephoneNumberChoosesCampaign("000006", femaleName, inventory); }

Increasing Readability with a DSL

Q2 Q3Q4Q1

Business Facing

Technology Facing

Critique

Product

Sup

porti

ng

the

team

• Business people can't usually read Java

Selenium is only one part of your test strategy

Q2 Q3Q4Q1

Business Facing

Technology Facing

Critique

Product

Sup

porti

ng

the

team

• Tests against a UI are relatively expensive to maintain

Conclusions

• You can test your web application using Selenium• There are lots of things you can do to increase

test:–––––

• Organize test code using domain scenarios or PageObjects

ReadabilityRobustnessSpeedCoverage

Questions?

Images attribution: DaPino Webdesign, Lebreton, Asher Abbasi, Woothemes, Asher, Iconshock, Andy Gongea, from www.iconspedia.com

• http://seleniumhq.org/ • http://www.xeolabs.com/portal/articles/selenium-and-extjs

• http://emilybache.blogspot.com• http://jsolutions.se

I would like to thank Kamilla Asp from Ericsson for allowing me to use Ad Orchestrator to illustrate my talk with examples.

http://seleniumhq.org/http://www.xeolabs.com/portal/articles/selenium-and-extjshttp://emilybache.blogspot.com/http://jsolutions.se/

Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40Slide 41Slide 42Slide 43Slide 44