performance & automation testing

INDIUM SOFTWAREAn Independent Software Testing Firm

Indium SoftwarePerformance & Automation

Testing Services

“Business Assurance through Quality Assurance”

© 2017 Indium Software

Performance Testing Service Overview

2

Focused Performance Testing practice – over past 10 yearsOver 50 Performance test engagements executed.Expertise in HP Mercury, Microfocus, IBM Rational & Open Source tools.Strong focus on Independent, Objective and unbiased analysis.

Performance Test Centre Services:• Load / Stress / Volume Testing• Fail-Over testing• Scalability Testing• Performance Benchmark/Baseline Testing• Performance Test Tool and Process Consulting

Differentiators – Simple, Flexible and Scalable• Process (with toll gate rigor) focus• Transaction based pricing (per script model)• High ROI on QA Investments• Innovative engagement models• Just in time resourcing model – core and flexible

teams.

Validate Architecture | Validate Response times | SLAs | Validate Scalability

© 2017 Indium Software3

Proficiency on Tools• Commercial - HP Loadrunner, MS VSTS, OATS, MF-Silk

Performer, IBM’s RPT• Open Source – Jmeter, Badboy, OpenSTA, Grinder

Key Deliverables• Overall Performance Roadmap• Test Plan for all major Releases• Assess high risk aspects of design w.r.t Technology• Comprehensive Test Summary with Observations ,

Recommendations and Extrapolations

Focus Areas - • Application Benchmarking & Predictive Usage Pattern• WAN simulation Using Cloud based load testing• Infrastructure Upgrade – Validation• Mobile application based Performance Testing• Maturity based Performance Engineering Models 

Value Additions• Load Balancer / Memory Bottleneck• Processor Bottleneck in DB & App. Server • Disc I/O constraints / Network Latency• Server Configuration / Network Bandwidth • DB Deadlock / SQL response time / Indexing issues• JVM / JDBC Connections Constraints• Timeout / TCP Connection stability

Performance Test Approach | Overview

© 2017 Indium Software

SMART Performance Test Strategy

Prioritization Matrix for Business Scenarios

# of Concurrent UsersFrequency of transaction(s) usage Customer Facing vs Financial Impact

Define High level scenarios

Business Processes• Sub-Processes

• Transactions or Process Steps

Define Test Data

Volume of Test Data needed (Static vs Volatile)Mining of historical data for quick test data preparation

Performance Test Environment

VPN connectivity – Configure Static IP address (Offshore) within client firewallApplication access – Test ID creation, user privilegesTools – Depend on the feasibility

Readiness to Start

People – Performance Testers with multiple test tool experienceProcess – Performance Testing process, methodology, scripting, execution, Analysis & ReportingTools – Templates, Jump-Start-Kits

4

SMART stands for Script, Model, Analyse, Report and Tune

© 2017 Indium Software

Performance Test Schema

5

PC 1 Application / Production Server

PC 3 PC 4

Performance Test Lab:

To conduct load test, monitor performance, analyze and report, remotely.

Load Generators: For hosting Controller & Load Generators for load generation via LAN/VPN onto Application Server. These could be at same location or at different geographies.

PC 2

VPN/LAN

Virtual UsersLoad

PCOffshore (Indium)

PC

CUSTOMER Infrastructure

© 2017 Indium Software

Performance Test Metrics

6

• OS Resource Utilizations CPU Usage Memory Usage Disk IO Utilization

• Network Utilization Bandwidth Latency Traces

• Servers Utilization (Web /App) Heap Usage & GC JVM JDBC Threads Application Logs

• SLA compliance• Certify the build for production deployment

• Trend analysis• SLA compliance

Metrics

•Test Tool Results•Monitors Data•Diagnostics tool data•Data from Support Teams

Analysis Report

•Execution & Monitors data analysis•Log analysis

•Diagnostics report•Bottlenecks and Root causes

Execution Report

•Response times•Server resource usage statistics•Defect tracking and comparison

Test Summary Report (TSR)

•Consolidated report of al the tests conducted•Recommendations

© 2017 Indium Software7

Recommendations

Usage distribution – Response time

Server utilization

Performance Test metrics

Deep Dive Performance Dashboard

Artefacts (Samples) – Performance Tests

© 2017 Indium Software

Test Automation Portfolio

8

• Functional Test Automation, Test Data Generation, Testing Tools Administration, Regression automation

• Multi Browser / Multi OS• Multi Device – PC / Mobile / Tablets

• Usage of reusable functions to ease coding • Reusable Automation Frameworks with proven track record in effort reduction • Process , checklists and templates to act as jumpstart kit

• Web Technologies - Java, .NET, LAMP• Enterprise Apps – SAP, Oracle, Peoplesoft, Salesforce.com• BI / DW, Big Data• Mobile, Social Media, Cloud, Analytics

Services

Jump Start

Technologies

© 2017 Indium Software

Tools• Commercial - HP QTP, IBM Rational

Functional Tester, Silk Test, TestComplete, Ranorex

• Open Source - Selenium RC, SOAPUI, Watir

Key Deliverables• Test Strategy / Plan• Test Automation Framework• Automation Suite• Test Automation Execution SummaryFocus Areas – <<Customer Name>> : • Define the process for the automation

suite management and maintenance Lorem Ipsum

• Implement Test Automation Framework and develop Test Automation Suites for Smoke , Sanity & Regression

• Automation suite should ensure that , the application build is stable across all major / minor releases which could include defect fixes or new feature enhancements and extend good test coverage

Value Additions• Reporting alerts over email / SMS• Cloud ready• Interface with multiple tools• Hybrid automation framework

9

Test Automation Approach | Overview

© 2017 Indium Software

Indium’s Selenium Automation Framework Enhanced (iSAFE) Framework

10

Framework Layer

Mobile A

pplication Under Test

Automation Suite

Automation

Scripts

Generic

Libraries

Input Data

Driver Script

Common

Libraries

Object Repository

Config and Global Variables Reporting Engine

Scheduler

Start up/Clean Up

Properties

Execution

© 2017 Indium Software

iSAFE Capabilities

11

iSAFE Capabilities

Test Reporting in multiple formats

with historical graphs

Ability to work with multiple OS /

Browsers / Devices

Ease of Setup and Execution resulting in

minimum learning

Fully Configurable Tests using

Spreadsheets

Support for API and DB Testing

Email/SMS Text

Status/Voice calls to

Stakeholders

Target based & parallel

execution using Selenium

Grid

Exception Handling and Screen Shot

Management