get max from automation

05/02/20231

GET MAX FROM YOUR AUTOMATED TESTSNitin Mukhija

- Director Of Engineering In Test PayU India Pvt. Ltd.

05/02/20232

FYI

05/02/20233

Hopefully this will not happen, so lets get started…

05/02/20234

Why Automate?

• Manual is slow• Faster feedback on Continuous Integration –

24*7 testing• Repetitive manual regression tests are

boring and make testers lose focus• Manual testers focus on testing new features• Allows for faster upgrades and refactoring• Consistency and Documentation• Higher test coverage – multiple data

combinations

05/02/20235

Let’s see an example of poorly written automation…

05/02/20236

Automation Design• Example of a bad code Hardcoded

Browser

Hardcoded Environment

Hardcoded Timeout

05/02/20237

Automation Design• Example of a bad code

Hardcoded Object

Identifiers

Hardcoded Test Data

Hardcoded Selenium

Constructs

Repeated Business

Logic

05/02/20238

Automation Design• Non-Actionable Report – High Failure Investigation Cost

Manually run again to repro. Can’t log bugs from report

No Screenshot / Page HTML

Insufficient Logging

05/02/20239

Unplanned Automation

05/02/202310

Let’s see the things you can do to get MAX from your automation…

05/02/202311

Tip #1:- Choose Right Design• Example – Layered Architecture• Hotel Book Web App

05/02/202312

Layered Architecture• Automation Libs• Framework Core• Page Objects• Business Logic Helpers• Test Cases

05/02/2023

Page Object

Action On Page

Mandatory to Return next

Page after click

UI Elements of Page

Page Name

13

05/02/202314

Layered ArchitectureSample layered design:-• Automation Libs• Framework Core• Page Objects• Business Logic Helpers• Test Cases

05/02/202315

Business Logic Helpers

Combine Page

Actions to

build business

Logic

Login

Search

Book

05/02/202316

Layered Architecture• Automation Libs• Framework Core• Page Objects• Business Logic Helpers• Test Cases

05/02/202317

Neat Test Case

Extensively use Business Logic

Helpers

Easy to read & interpret manual

test case

TC Description

05/02/202318

Layered Architecture• Automation Libs• Framework Core• Page Objects• Business Logic Helpers• Test Cases

05/02/202319

Automation Libs• Selenium, QTP, etc.• All external third party jars

Framework Core• Wrapper around various

external libraries

Class DescriptionLog Log reportingHelper Various utility methodsDataBase Methods to interact with database Popup Methods to handle popups, alerts on the pageTestData Reader

Methods to read from test data sheet

Browser Methods to do operations on your Browser/PageElement Methods to do operations on Elements of the Page

05/02/202320

Tip #1:- Right Design- Benefits• Reduce Maintenance cost• Business logic at one place• Reusable code• Localized changes – Data and Object Repo• Easy to understand test cases• Easy to change underlying automation lib

05/02/202321

Tip #2:- Configurable• Don’t hardcode – Load at runtime

– Environment URL’s– Browser Name– Platform– ObjectWaitTime– Database credentials– Any other param which drives the execution

05/02/202322

Tip #3:- Data Driven• Run same test case with different data combinations

Read login credentials from Excel

Test Data Excel Row Number

05/02/202323

Tip #4:- Parallel Execution

• Write non-interfering atomic test cases

• Standard Win7 can run 12 parallel browsers – execution time reduced to 1/12th

Faster Execution = Faster Feedback

05/02/2023 24

05/02/202325

Tip #5:- Rich Reports

Hyperlinks of Page Screenshot

& HTML

Uniform logging of TC steps in

Green/Red/Black logs

Failure Page URL

Log bugs by just reading logs, even manual tester can do!

TC Description

05/02/202326

Allure Reporting Failure Grouping

05/02/202327

Custom Listeners

• Logging of unhandled exceptions• Cleanup tasks• Soft Asserts

05/02/202328

Tip #6:- Customized email results• Host Test Results on a web server - Reports

history• Email Results - Customized Subject

Pass Percentage

Area Owner

05/02/202329

Consolidated Results

05/02/202330

Tip #7:- Measure Automation Accuracy

# of test cases failed due to Dev code issue Vs

# of test cases failed due to automation code issue

05/02/202331

Tip #8:- Measure Code Coverage

Sample PHP Code

Coverage Report

05/02/202332

Tip #9:- Automated Deployments

• Using Docker - Build production like, testing environment on the fly

• Jenkins job to:-– Deploy application code automatically– Run Automation

• A Step towards Continuous Delivery

05/02/202333

TestLink - TCM

Test Cases

Test Plans

Test Builds

Reports

Execution Engine

Fetch Git Code & Compile

TestLink Integration Module

Get Test Cases To Run

Launch & Drive Execution

Retrieve Results/Exceptions

Push Test Results & Email (ReportNG)

Layered Automation Suite

Test Cases (TestNG)

Business Logic

Page Objects

Framework Core

Selenium Webdriver

Tip #10:- Continuous Integration

05/02/202334

That’s strange, not even a single Oxygen mask dropped!!

05/02/202335

Questions & Answers

05/02/202336

Thank You!!!

https://in.linkedin.com/in/nitinmukhija@nitin_mukhija