test automation trends & tco - anztbtco080813.pdf · test automation engineer testing tools...

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Test Automation Trends & TCO Kate Droukman Minesh Hirani Presales Consulting, Hewlett-Packard Australia

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2

Introduction

Kate Droukman

Education

Bachelor of Software Engineering

Master of Business Administration

Work Experience

Test Automation Engineer

Testing Tools Administrator

Developer

Dev Lead

Presales Consultant

Minesh Hirani

Education

Bachelor of Software Engineering

Work Experience

Developer

Test Analyst

Project Manager

Presales Consultant

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

Major IT Shifts

Mobility Big Data Agile

Social Media Cloud Consumerisation

Sources: HP “Enterprise 20/20” Ebook ; “8 Trends Driving the Future of Information Technology” IT Business Edge; “The Big Five IT Trends of the next half decade”, ZDNet

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4

• Multiple Devices • Mobile Lab management • Carriers & Locations

Test Automation Trends

Mobile

• Test Automation as Design Specification • Build Verification Testing • Lower level technical testing • Continuous Integration

Agile

Cloud • Continuous Delivery • SaaS and 3rd Party Application Releases

Other • Model Based Frameworks • Framework Accelerators

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5

What’s Next?

UI Driven Development Test Automation as a

Service Mobility beyond

Smartphones

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6

Functional Test Automation Landscape

36

GUI Test Automation

Tools

436

Unit Test Frameworks

11

BDD Frameworks

9

Service Test Tools

Vendor

Open Source

Sources: wikipedia.org; github.com; infoq.com

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7

Functional Test Automation Technologies

Graphical User Interface Technologies

Object Recognition Object Interaction Object Definition

Communication Protocol Technologies

Protocol Support Communication Flow

Code Interaction Technologies

Programming Language Support

IDE, SCCM, and Build Integration

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8

Calculating Test Automation TCO

Is it difficult to use?

Does it support our applications?

How much does it cost?

How quickly will I learn it?

What do I need to automate?

Do we have required skills?

What’s available out of the box?

How will I manage test results?

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9

Learning Curve

Available in-house skills

Time to learn new tool

Time to learn new framework

Pre-requisite technical skills

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10

Implementation

Time to install the tool

Integration of the new tool with existing systems

Setting up processes and frameworks

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11

Creating Tests

Average time taken to create new tests

Available validation options

Available flow control options

Parameterization

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12

Maintaining Tests

Handling of dynamic scenarios

Rate of change for AUT

Identification of tests affected by application change

Introduce new controls and interactions

Average time taken to change a test

Expected lifespan of the test

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13

Analysis

Time taken to find root cause of failed result

Variety of analysis options

Auditing and Logging

Integrations with Defect Management system

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14

Strategic Decisions

Are we acquiring for project or for organization?

What is the expected lifespan of the application?

Are we focusing on one application or a portfolio?

How long will we require the tool for?

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15

Test Automation TCO Formula

Technology Support + Acquisition Cost + Installation and Implementation + Available Resources * Skill (and Salary) + Training + Test Creation Time + Test Maintenance Effort + Supported Technologies + Analysis of Results + Tool Maintenance and Support + Integration + Duration of use

= TCO

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you