K2  Keynote  5/4/16  10:00              

Open  Source  Test  Automation:  Riding  the  Second  Wave  

 Presented  by:  

 

David  Dang  

Zenergy  Technologies    

Brought  to  you  by:        

   

   

350  Corporate  Way,  Suite  400,  Orange  Park,  FL  32073    888-­‐-­‐-­‐268-­‐-­‐-­‐8770  ·∙·∙  904-­‐-­‐-­‐278-­‐-­‐-­‐0524  -­‐  info@techwell.com  -­‐  http://www.stareast.techwell.com/      

 

       

David  Dang  Zenergy  Technologies    For  more  than  seventeen  years,  David  Dang  has  been  a  leader  in  the  test  automation  industry.  As  VP  of  automated  solutions  for  Greensboro,  NC-­‐based  Zenergy  Technologies,  David  spearheads  the  development  of  advanced  frameworks  that  emphasize  reusability  and  reduce  maintenance  efforts.  He  is  an  expert  in  all  major  commercial  automation  tools  as  well  as  open  source  tools  such  as  Selenium  and  Jenkins.  On  the  mobile  front,  David  uses  advanced  concepts  to  design  optimal  frameworks  using  mobile  automation  toolsets  including  Perfecto  and  Appium.  In  addition  to  his  high-­‐level  consulting  engagements  for  Zenergy’s  clients,  David  is  in  high  demand  as  a  presenter  at  major  software  quality  assurance  and  testing  conferences.  Read  more  about  David  and  Zenergy  at  zenergytechnologies.com.      

   

Open Source Test Automation: Riding the

Second Wave

StarEast 2016 Presented By: David Dang

A Testing Partner You Can Trust

Overview

What is an open source test automation framework/tool?

Why is it gaining in popularity?

First wave of open source frameworks/tools

Current open source frameworks/tools

Strategies for success

Should you catch the second wave?

Q & A

What is Open Source?

Software for which the original source code is made freely available and may be redistributed and modified

Open source code is meant to be shared within the community. Enhancements, bug fixes, branching of the code are made by the community

Currently on www.opensourcetesting.org/functional.php, there are around 135 open source test automation tools/frameworks

Why is Open Source Test Automation Gaining Popularity?

FREE! Companies have to do more with less

More applications are being converted to web-based

Development teams are pushing for open source test automation

Industry trends include more open source in all areas (Eclipse, Java, Jenkins, OpenStack, Drupel, etc.)

First Wave of Open Source Test Automation Tools/Frameworks

FitNesse: Operates below the user interface level. The user provides various inputs to the application and determines if the correct results are returned

Ruby Watir: Ruby libraries used to automate web browsers. It allows users to automate clicks, fill in forms, press buttons, etc.

Selenium RC: Writes automated web application UI tests. RC comes in two parts: server acts as an HTTP proxy for web, and client libraries interact with the web elements

Why the First Wave Tapered Off

Tools were too fragmented, difficult to integrate into complete solutions.

Level of instrumentation increased the technical challenge and decreased reliability

Learning curve tended to be higher due to overly complex and less refined solutions

IT landscape wasn’t ready to embrace open source at the time

Why the First Wave Tapered Off

FitNesse Does not run at the UI level; runs at the source code level

Fixtures must directly access the source code

Watir Only available for Ruby

Each browser had a different library

Selenium RC: Required heavy instrumentation

Had a large, clunky API

Effectiveness and reliability limited by use of Javascript injection

Current Open Source Test Automation Tools/Frameworks

Selenium WebDriver: Drives a browser directly using its built in support for automation

Cucumber: Runs automated acceptance tests written in a behavior-driven development (BDD) style. It uses Ruby to create step definition

Robot Framework: Generic test automation framework based on keyword-driven testing approach. The core framework is implemented using Python

Sample of Selenium WebDriver

Sample of Cucumber

Sample of Robot Framework

Why Second Wave Tools are More Popular

WebDriver laid foundation for numerous additional tools

Increased support from well-known organizations WebDriver

Developed by Thought Works

Microsoft is developing a driver for their new Edge browser

W3C has adopted it as the standard for browser automation

Appium – Sauce Labs

Supports popular methodologies (Ex. Cucumber BDD)

Cleaner abstractions and more concise APIs

Less instrumentation required

Better integration

Selenium RC vs Selenium WebDriver: Architecture

Selenium RC Server

Test Code (e.g. Java, C#, Python,

etc.)

Web Browser

Selenium RC

Selenium Core

Application Client

Application Server

Browser Launcher

HTTP Proxy

1

2

3

4

5

Selenium WebDriver

Selenium WebDriver

Test Code (e.g. Java, C#, Python,

etc.)

Web Browser (IE, Firefox,

Chrome, etc.) Application Server

Selenium RC vs Selenium WebDriver: API

Selenium RC Selenium WebDriver

WebDriver: The Foundation of the Second Wave

As the industry standard tool for

browser automation, countless tools are being created that extend or

integrate with Selenium WebDriver

Tools like Appium and Protractor

extend WebDriver for use in other technologies

Tools like Cucumber integrate with WebDriver so it can be used with

methodologies such as BDD

Application Under Test

Selenium WebDriver

Selenium vs Packaged Tools: Feature Comparison

Feature Packaged Tools Selenium

Environment Integrated Development IDE (Eclipse, IntelliJ, RubyMine)

Browser Compatibility Limited (e.g., Firefox, IE, Chrome)

Firefox, IE, Chrome, Safari, Opera, Edge, HtmlUnit, PhantomJS, and others

Language support Limited (e.g., VBScript only or .NET only)

Java, Javascript, C#, Python, Ruby, and others

Reporting Included Separate package (ReportNG, Allure) or custom

Test Management Integrated Custom

Selenium vs Packaged Tools: Feature Comparison

Feature Packaged Tools Selenium

Continuous Integration Integrated Separate package (Jenkins, Bamboo)

Support 800 number The Community…

Object Management Included Separate package or custom

Test Control Included Separate package (TestNG, phpUnit, RSpec)

Strategies for Success

People

Requires more technical resources

Resources must be comfortable with object-oriented programming

Resources must understand framework/tool configuration and environment setup

Resources must be able to diagnose and fix technical issues without vendor support

May have to identify and even fix bugs in the tool

May need to contribute to the community with new features

May need to work more closely with development team

Strategies for Success

Technology

Open source requires more time and effort to get started and maintain

No official support, only the community

Components are not built-in; they must use other libraries or be developed from scratch

Some technologies are not well supported by open source tools

Some technologies have features that lack support from otherwise useful open source tools

Strategies for Success

Process

Requires more dedicated resources

Tool should fit into your existing process

For e.g., if you are a .NET shop, you will often find less of a learning curve with a tool that offers .NET as an option

A tool like Cucumber will have a greater return if it is adopted early in the SDLC

Requires a detailed plan for implementation and maintenance

Framework will be required including standards and page object models

Framework Development Process

Gather Information Architecture

Design

Core Framework

Implementation

Ongoing

Development

Ongoing Maintenance

Should You Catch the Second Wave?

People Is your team comfortable with object-oriented programming?

Are they comfortable dealing with technical challenges with minimal support?

Are they able to contribute to the tool with bug fixes and enhancements?

Do you have a plan in place for training and mentoring?

Should You Catch the Second Wave?

Process Do you have the resources to dedicate toward ensuring

automation is successful?

Do you have a plan for creating a stable, maintainable framework?

Does open source make sense from a budget standpoint?

Is open source a good fit for your overall IT strategy?

Should you Catch the Second Wave?

Technology Have you chosen a tool that is appropriate for your platform?

How active is the community? Is there somewhere to turn for help?

Is the tool updated frequently and responsive to platform changes?

Does investing in open source fit into your organization’s technology road map?

Are you comfortable supporting multiple tools sets?

Summary

WebDriver is the foundation of the current wave of open source automation

People, process and technology are the keys to success Moving to open source is not a decision to take lightly

Questions?