automation 2.0 taking test automation beyond regression testing · taking test automation beyond...

8
d Automation 2.0 Taking test automation beyond regression testing 401-408, A-Wing, Pride Silicon Plaza, S.B. Road, Shivaji Nagar, Pune -411006, Maharashtra, INDIA Email: [email protected] Tel: +91-20-41020202 Global Delivery Centre:

Upload: others

Post on 31-May-2020

24 views

Category:

Documents


0 download

TRANSCRIPT

d

Automation 2.0

Taking test automation beyond

regression testing

401-408, A-Wing, Pride Silicon Plaza, S.B. Road, Shivaji Nagar, Pune -411006,

Maharashtra, INDIA

Email: [email protected] Tel: +91-20-41020202

Global Delivery Centre:

Introduction

Whenever Test Automation is a point of discussion, it has always been linked with regression

testing. Traditionally, it has been believed that test automation can only be used to support

regression testing by taking the existing manual tests & converting them into automated test

suites which are then rerun as part of regression testing to assess the quality of the software.

Although there is no denying in the fact that test automation is best suited for regression testing,

however there are few more facts attached to this. Regression tests of this sort miserably fail to

detect new defects. Such automated regression suites are the simplest form of test automation

which is highly expensive to maintain. It is fragile and breaks easily and hardly utilizes the full

potential of test automation. These regression suites merely automate what the manual testers

have been doing, in fact many a times, losing many of the advantages which manual testing has

for finding defects.

Through this document, Nitor intends to go much beyond the regular regression testing and try

to throw light on what all can be achieved by leveraging the strengths and possibilities of test

automation. Although there are limitless possibilities which one can achieve through test

automation, this documents restricts to only few of those key possibilities, which would help in

effectively extending the reach of test automation much beyond traditional regression testing

Automation beyond regression

Below mentioned are some of the key possible areas, wherein test automation can be utilized to

leverage and fully utilize the immense potential of test automation

Building a BVT (Build Validation Test) suite

Creating test data for heavy data dependent scenarios

Continuous Integration, thereby facilitating end to end (Build-Deploy-Test) testing

Automated Unit Testing

Technical testing such as Web Service, SOA and API testing

Test Automation leveraging

Selenium Web Driver

BVT (Build Validation Test) Suite

With extensive adaption and implementation of agile methodology, there has been a drastic shift

when it comes to test execution cycles. Unlike the traditional Waterfall methodology, wherein

the QA team gets a prolonged test execution cycle, in Agile, the test execution cycles have

become much shorter & a repetitive effort. Within a sprint cycle (of 2 or 3 weeks), there are

multiple test execution cycles which are carried out to test the intended functionality being

developed and released.

The test execution cycles usually begin with a Smoke test cycle, wherein the basic

functionalities of the application are checked, just to ensure whether the build is stable enough

& testable enough so as to proceed further with a comprehensive functional testing cycle. This

smoke testing suite usually consists of a fixed set of test scenarios/test cases designed using a

‘shallow & wide’ approach covering the basic aspects & key functionalities of the entire

application. These tests are subset of tests cases that verify main functionalities. As this smoke

test suite is executed for every build cycle (almost on a daily basis), it does make sense to have

it automated. This automated smoke test suite is nothing but the ‘Build Verification Test’ suite

which is a set of test cases run on each new build to verify that the build is testable.

Benefits of automating the BVT suite –

Saves the manual testing effort in executing the same set of test cases manually for

every build cycle

Helps to surface out critical defects early

Helps to save the efforts of QA team in setup and testing of the build, when major

functionality is broken

Test Data Generation

One of the most important aspects in any testing activity is the usage of effective test data. Test

data plays a very important role in success of testing cycles. The test data drives the test cases.

However, generating the test data becomes a humungous task, especially for data driven/menu

driven applications which are heavily data dependent. In such cases, generating the test data

takes away valuable time of the QA resources. At times, when the customer is ready to share

his live/production data the life becomes less miserable, however using customer’s data is

never a wise approach and at times, if such live data is exposed to unauthorized people, it not

only puts the credibility at stake, but eventually leads to loss of business.

To overcome such situations and to generate effective and quality test data without spending

too much of manual effort, generating the test data through automation scripts is the most viable

and feasible solution. There are several test automation and automated tools available which

can be utilized for creating a comprehensive and effective set of test data.

Continuous Integration

Continuous Integration (CI) is a practice/process in which the developer integrates the code into

a shared repository frequently, which can even be several times a day. This arises the need to

test the quality of the build for each commit or each check-ins. Carrying out such repetitive tests

manually consumes a lot of manual efforts. Moreover, usually as it is the same test of scripts

which needs to be tested every time, it makes more sense to have these tests automated rather

than testing them manually. This is where executing automated test scripts in a CI environment

is beneficial.

In CI, each of the integrations or check-in builds are verified by an automated build, allowing

teams to detect integration and other errors/problems early. CI allows the team to run

automates test scripts after each commit & thus convey the results to the development team

quickly and help to locate the issues more easily.

The below diagram depicts the CI process:

All the above mentioned phases/activities of CI can be automated, right from compiling the build

to publishing/deploying the build to triggering the unit testing suite, automated BVT suite,

functional testing suite to reporting the results to the relevant stakeholder, everything can be

made possible through a single test automation framework

Using test automation in CI process effectively can yield multiple benefits. Some of these have

been mentioned below:

Helps in unearthing the issues/defects early and also helps in identifying the location of

those defects

Debugging time is reduced, hence developer can spend more time on their core

activities

Better confidence on the quality of the build

Risks are reduced

Automated Unit Testing

There is no denying in the fact that, it’s virtually impossible to write a bug free code. We humans

are bound to make mistakes. Hence, testing of the code been written by the developer is so

very imperative in the overall development cycle to ensure a thorough quality check. This is

where Unit testing comes into picture.

A unit test is a piece of code that tests another piece of code. This unit testing can be

automated by using several unit testing tools. There are several commercial and open source

tools available such as N-Unit, J-Unit, Struts, TestNG etc: Based on the platform and the

technology for which the code needs to be tested, the appropriate automation tool and

framework can be used.

An automated unit test suite has a number of important and tangible advantages:

Unit tests find problems early in the development cycle.

An automated unit test suite test your code in two dimensions:

o Time dimension because once you’ve written a unit test, it ensured that the code

works well for now and in the future

o Space dimension because unit tests written for other features ensures that the

new code did not break them

The development process becomes more flexible

Reduces the need and effort for manual testing

All of the above leads to making software more predictable, robust, repeatable and

better quality.

Technical testing – API testing, SOA testing and Web service testing

Although technical testing can be carried out manually, however there are several limitations

and challenges in doing so.

Complexities and dependencies needs to be addressed

Lack of required technical skills

Human errors leading to erroneous results

These limitations can be easily overcome by implementing test automation for carrying out

these technical testing activities. There are several tools (both commercial & open source)

currently available in the market which can be used & by leveraging the inherent benefits of

automated testing tools; the technical testing assignments can be made much more effective,

reliable, faster & error-free.

Conclusion

Test Automation can do much more than traditional regression testing. Hence, limiting test

automation for performing functional testing or regression testing would not only do injustice to

test automation, but would also be like utilizing just 10% of its overall capabilities. Leveraging

the benefits of test automation and utilizing it effectively in tasks which are complex, tedious,

time & energy consuming and difficult to carry out manually is the what the current time &

situation demands.

About Nitor Test Services

Nitor Test Center of Excellence (TCoE) is an independent test practice with well-defined testing

processes and profound knowledge of tools and techniques to cater to the testing needs of

varied products/applications. Our professional team of test analysts has worked on products of

varying complexity in terms of technology and functionality, projects and solutions across focus

areas such as Business Intelligence, Portals & Collaboration and Enterprise Mobility.

Nitor offers end to end testing services at aid our partners in continuous business growth through our

niche technology focus and diverse domain expertise. Our TCoE offering include

Highlights of Test Center of Excellence (TCoE)

ISTQB Gold Certified test partner

SSAE Soc- II Type-1 accredited organization

Technology and platform agnostic. NitorTCoE is technology /software / tool independent

70+ strong team of certified test engineers

Seven years of profound QA experience

Strong knowledge base with expertise in diverse methodologies and tools

Contact Us

NitorInfotech

401-408, A-Wing, Pride Silicon Plaza

S.B. Road

Pune -411006; Maharashtra, INDIA

Email: [email protected]