NTT DATA welcomed Dell Services into the family in 2016. Together, we offer one of the industry’s most comprehensive services portfolios designed to modernize business and technology to deliver the outcomes that matter most to our clients.

Testing in DevOps: An NTT DATA Services Point of View

Introduction In this evolving IT world, innovation and adaptability is the key to remaining competitive. Testing in DevOps is a practice that follows next-generation, smart automation methodologies using proven processes, tools and subject matter experts that help companies achieve their quality assurance (QA) objectives. At NTT DATA, we’re focused on reducing delays and redundancy, and utilizing automation to implement transformation like rapid development across the entire value chain — a discipline of DevOps. We believe that test automation solutions must provide continuous feedback to development and operations teams and enable rapid and continuous integration, ensuring bug-free build deployment at a push of a button.

Our approach to agile testing in DevOpsTesting and QA follows a collaborative and continuous approach. In our practice, we use shift-left testing for test automation with integrated processes and continuous feedback to DevOps agile project teams. This not only helps improve early testing but also ensures increased test coverage in general. By automating the testing solution, we’re able to utilize functional testing while also supporting integration and non-functional testing like performance, database, security and mobility testing. The solution also takes care of other dependencies like service virtualization, test data management and dynamic environment provisioning, ultimately meeting DevOps testing objectives.

Introduction 1

Table of contents

Enabling continuous feedback inside the agile team 3

Solving the test environment dilemma 4

The NTT DATA way of automation 5

Client success story

Written by:Saurabh Pandey

About the author:Saurabh is the center of excellence capability lead, ensuring organization-wide competency, and developing frameworks and best practices as well as supporting next-generation process and technology transfer/adoption for our customers’ IT ecosystem. He is an MBA and engineer with more than 15 years of rich experience. As an agile and DevOps evangelist, he provides consulting for automation, QA testing, independent verification and evaluation, and has helped organizations on transform their testing processes for the digital age.

Saurabh developed the SDET Competency Development framework and a certification program widely accepted not only within Dell Services but also by our customers to help them jump-start the DevOps transformation journey.

6

2

What we offer 6

Saurabh Pandey

3

Finished workSprint end date

and deliverables do not change

Sprint review

Product backlog

Sprint planning

Release planningRanked list of

what is required

1–4 week sprint cycle

Daily scrum meeting

Focus on small, valuable, vertical chunks of work to release value earlier in the process

Product owner The teamGathers input from executives, the team, stakeholders and users

The teamPrepares the release backlog, selecting the highest priority stories for the sprint

EveryoneScrum masterMeasures progress with burn down/up charts

• Enable continuous delivery of highest priority business value

• Allow changes throughout lifecycle, as prioritized by customers

• Automate labor-intensive and error-prone activities, such as builds, regression and deployments

Figure 1: Inside the agile team — from sprint planning to the finished product

Steer

Develop/test

Continuousbusiness planning

Collaborativedevelopment

Continuoustesting

Continuous release and deployment

Continuous monitoring

Continuous customer feedbackand optimization

Accelerate software delivery for faster time to value

Balance speed, cost, quality and risk for increased capacity to innovate

Reduce time to customer feedback for improved customer satisfaction experience

Automation/regression

Agilecontinuousfeedback

Deploy

Steer

Develop/test

Automation/regression

Deploy

Agilecontinuousfeedback

Figure 2: Enabling continuous feedback and processes in a testing environment with agile

4

People

• Software development engineers in test (SDETs): qualified resources that hold agile certifications and have expertise in development

• Cross-functional experts: resources with multiple areas of expertise working toward a common goal to achieve increased performance and produce a faster, higher quality product

Process

• Automation across value chain• Behavior-driven development:

collaboration between developers, QA and non-technical participants

• Test-driven development: technique using automated unit tests to drive the design of software and force decoupling of dependencies

• Acceptance test-driven development: methodology based on communication between the business customers, the developers and the testers

• Continuous Performance

Technology

• Automation framework/tools: for functional and non-functional testing

• Service virtualization: when dependent applications are unavailable or even before they have been built

• Environment provisioning view: a centralized, automated system that triggers application deployment process

• Mobility test framework: guidelines for testing on various commercially available mobile testing frameworks like pCloudy, Perfecto Mobile and DeviceAnywhere

• Test data management: Leveraging IPs and industry-leading tools

In-house tools: smart performance testing framework, test data management, environment provisioning view, agnostic automation accelerator

Third-party tools: Parasoft SOAtest, Delphix, TestNow, BuildNow, Maven, Gradle, TestNG, Jenkins, Cucumber, Geb, Selenium

Figure 3: Integrating people, process and technology for end-to-end testing

The NTT DATA way of automationSDETs commonly develop frameworks that support DevOps principles ensuring rapid and continuous testing. They design these frameworks and scripts before the development builds are created to streamline the entire software development lifecycle. At NTT DATA,

we use single-click deployment, which reduces manual effort and increases value across unit testing, functional testing, integration testing and user acceptance testing. Gone are the days where return on investment was the only measurable parameter for automation. We can help organizations maintain their

return on investment with fast, easy-to-use automation testing processes. More importantly, automation is planned within the sprint, and the speed is maintained across the release.

Best practices in DevOps and agile testingInclude testers and QA managers in the agile development team

• Promote the importance of testers and encourage continuous feedback with the programmers and the customer

• Ensures SDETs actively participate in meetings to define the main business flows and work on short iteration activities alongside developers

• Utilize the specialized skills of test-driven development, including unit testing and continuous integration at the unit level

• Leverage automation testing as a key method of regression testing

Benefits of agile testing in DevOpsAgile and continuous delivery can improve software quality and reduce risk by taking advantage of cloud computing and automated deployment. Throughput can also be improved while increasing testing coverage and ensuring high quality. Typical agile testing activities include:

• Collaborative meetings: Testing requirements are discussed and refined as a team (during stand-up meetings/scrums) allowing the combined team to better address the business and technical aspects of the requirement. This enables overall alignment and helps prevent misunderstandings.

• SDET engagement: SDETs participate in the overview sprint planning, ensuring automated tests are fully leveraged to implement regression testing.

• SDET and developer sync: Because the SDET is working with the developers on the same team, they are able to give immediate feedback, which lets the developer quickly and easily fix the bug

• Implementation of continuous integration: Because testing is done continuously throughout the software development lifecycle, it can’t be cut. In waterfall projects, testing is often cut in order to meet deadlines. This results in defective software being

Agile challenges for testing engineers in DevOpsSince requirements and code flux steadily during a sprint, ensuring full test coverage is not always straight forward. Requirements churns effect test coverage and can also lead to support of daily build tests. When there’s a tight turnaround, a tendency to defer certain types of testing to later sprints can result in larger problems that are harder to fix when so close to final delivery. Pushing performance, API, compatibility and stress testing to later in the software development lifecycle will create a high-stress environment for the test team.

5

Test data management

In DevOps, the agile test organization and testers need continuous access to test data. With agile development, continuous integration and delivery is also critical as frequent testing is required. Our test data management software supports the agile method by streamlining access to test data and providing up-to-date insights.

Service virtualization

• Simulates the behavior of an entire application or system during testing

• Runs on commodity hardware, private cloud and public cloud

• Provides each developer with their own test environment

• Allows developer and testers to continue to use their existing testing tools (such as manual, web, performance and user interface test automation)

Deployment automation

• Orchestrates and automates the deployment of applications, middleware configuration and database changes into development, test and production environments

• Enables automated, consistent deployments and rollbacks of applications

• Ensures clear visibility• Integrates with middleware,

provisioning and service virtualization

Figure 4: You can enable a dynamic, test-driven infrastructure by automating deployment, virtualizing test environments and integrating agile into test data management.

What we offerQA/test team transformation consulting: We can help you transform your organization to fit an agile/DevOps model and adopt DevOps principles and processes in-house. We offer a wide range of services including defining test/QA transformation strategies, helping you plan your DevOps implementation, training sessions for your in-house staff and suggesting best-in-class professional services to help get you started.QA/testing and automation services: If you’re ready to implement a DevOps approach, our experienced subject matter experts and engineers can deploy, manage and support all of your environments 24x7x365 — so your team can focus on the activities that grow your business.

Further reading“Part Dev, Part Ops = DevOps,” infographic by Dustin Whittle. App Dynamics. July 8, 2014. https://blog.appdynamics.com/devops/part-dev-part-ops/http://searchsoftwarequality.techtarget.com/definition/Scrum-sprint

6

NTT DATA partners with clients to navigate the modern complexities of business and technology, delivering the insights, solutions and outcomes that matter most. We’re a top 10 global IT services and consulting provider that wraps deep industry expertise around a comprehensive portfolio of infrastructure, applications and business process services.

© 2016 NTT DATA, Inc.  All rights reserved. November 2016 | 1108_Agile_Testing_Point_of_view_paper.indd | Rev. 1.0

Visit nttdataservices.com to learn more.

Business need

One of the largest global hospitality companies in the world managing a portfolio of 3,897 hotels and resorts across six continents. In order to improve their testing efforts, they wanted to automate regression packs and reduce time and effort.

Solution

• Shifted testing left by performing early stage testing and adapting lean functional testing

• Focused on areas that could be automated using Selenium and in-house utilities

• Implemented component-based test automation approach using Selenium lean framework

• Created lean test abilities to reduce the manual repetitive effort

• Automated API Regression Test Suite 100% using SOAP UI (Groovy Scripts)

Benefits

• Increased productivity and defect-finding capacity

• Reduced effort by 50%

• Delivered release on time

• Enabled reusable business components to streamline during development and maintenance

AwardsWinner of “Digital Accelerator Award” in the “Top API program” contest from Apigee.

Client success story: