reinventing testing devops
Post on 27-Oct-2021
4 Views
Preview:
TRANSCRIPT
Reinventing Software Testing for DevOps
and Modern Application Delivery
By Wayne Ariola
v
Tricentis | Reinventing Software Testing | 1
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
Think back just 5 years ago. In 2014…
• The seminal DevOps book—Gene Kim’s The Phoenix Project—was one year old
• Gartner predicted that 25% of Global 2000 enterprises would adopt DevOps to some extent by 20161
• "Continuous Testing” just started appearing in industry publications and conferences2
• Many of today’s popular test frameworks were brand new—or not yet released
• The term “microservices” was just entering our lexicon
• QC/UFT and ALM were still sold by HP (not even HPE yet)
• Only 30% of enterprise software testing was performed fully “in house”3
• There was no GDPR restricting the use of production data for software testing
• Packaged apps were typically updated on an annual or semi-annual basis and modern platforms like
SAP S/4HANA and Salesforce Lightning hadn’t even been announced
Times have changed—a lot. If the way that you’re testing hasn’t already transformed dramatically, it will soon.
And the pace and scope of disruption will continue to escalate throughout the foreseeable future.
It’s becoming increasingly clear that the path forward requires testing to deliver fast, continuous insight into
an application’s business risk and “release readiness.” However, the scope of what’s needed to achieve that
varies broadly. It might be a matter of correlating data across your best-of-breed test automation toolset to
deliver actionable, business-focused insights on whether a given release is ready to progress into production.
You might need to transform people, processes, and technologies to speed up a decades-old manual testing
approach. Or, perhaps you need to extend your testing practices to cover all the web/mobile UIs, APIs,
mainframes, packaged apps, and other technologies involved in modern business transactions.
This paper explains what’s required across every testing team, explores some of the primary challenges to
delivering the expected level of insight, then provides a quick introduction to how 1600+ organizations
overcome those challenges with Tricentis.
1 Gartner, “Market Trends: DevOps — Not a Market, but a Tool-Centric Philosophy That Supports a Continuous Delivery Value Chain"
2 Alan Zeichick, “Forget ‘Continuous Integration’—the buzzword is now ‘Continuous Testing’,” SD Times
3 Capgemini, Sogeti, HP, “World Quality Report 2014-2015”
v
Tricentis | Reinventing Software Testing | 2
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
Continuous Testing: Does the Release Candidate Have an Acceptable Level
of Risk?
“Digital transformation” has elevated software delivery to a CXO conversation—and that includes software
testing. Today’s executives face relentless pressure to deliver innovative software faster than competitors. On
the one hand, testing is all-too-often the roadblock that stands between highly-accelerated Dev processes
and highly-automated Ops-driven delivery processes. But on the other hand, testing is essential for ensuring
that the release doesn’t place the business at risk—undermining the very “customer experience” that digital
transformation is dedicating to enhancing.
How can businesses achieve the optimal balance between speed and risk to deliver engaging customer
experiences faster than competitors? Enter Continuous Testing. Continuous Testing is the process of
executing automated tests as part of the software delivery pipeline in order to obtain feedback on the
business risks associated with a software release as rapidly as possible.
Continuous Testing really boils down to providing the right feedback to the right stakeholder at the right time.
For decades, testing was traditionally deferred until the end of the cycle. At that point, testers would provide
all sorts of important feedback…but nobody really wanted to hear it then. It was too late, and there was little
the team could feasibly do, except delay the release. With Continuous Testing, the focus is on providing
actionable feedback to people who really care about it and when they are truly prepared to act on it.
Contrary to popular belief, Continuous Testing can happen at any point in the application delivery lifecycle. At
some point, the concept of Continuous Testing was inappropriately conflated with the trend of “Shift Left.” To
deliver the right feedback to the right stakeholder at the right time, Continuous Testing needs to occur
throughout the software delivery lifecycle—and even beyond that to production (e.g., monitoring information
from production and feeding that back from the quality perspective). Just as the name indicates, Continuous
Testing involves testing continuously. Simply starting and finishing testing earlier is not, by definition,
Continuous Testing.
Of course, testing continuously in a way that provides the right feedback to the right stakeholders at the right
time is no easy feat. The short answer to why it’s so challenging is that the time available to test is constantly
decreasing while the complexity of what we need to test is increasing. But it’s more than that. To better
understand why “reinventing testing” is now so essential, let’s take a quick look at some of the core forces
influencing modern application delivery.
Decentralization of Architectures and Teams
System architectures have oscillated from a centralized to a much more decentralized set of integrated
systems. From the monolithic mainframe applications of the past, we’ve evolved into a set of integrated
systems that are becoming significantly more distributed. Cloud-native applications, the advent of
microservices which require an extreme degree of orchestration—this is all part of the decentralization trend
in system architectures.
v
Tricentis | Reinventing Software Testing | 3
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
The Development team structure has traditionally followed the system architecture trends. Development
team structure went from large centralized waterfall type teams to much smaller and more decentralized
teams.
v
Tricentis | Reinventing Software Testing | 4
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
As Development teams decentralize and grow more fragmented, it becomes increasingly difficult to maintain
control over epics and deliverables. Just imagine: instead of unified groups working on monolithic
applications, we shifted to many small autonomous groups each working on smaller parts of a highly-
distributed system. How do you ensure a coordinated and compelling user experience across the
overarching application?
Now, teams are oscillating back a bit: they’re not necessarily looking for ways to centralize the Dev team
structure, but rather to collaborate more effectively within the decentralized Dev team structure.
Then there’s the Test team structure—following the same general pattern, but lagging a bit behind.
The Test team structure has moved from a highly-centralized group responsible for all testing, to testing
centers of excellence, to digital testing centers of excellence, to testers being embedded within Agile Dev
team structures. Now, Test teams (like Dev teams) are shifting toward more of a mixed model that’s largely
decentralized, but has centralized components available to test end-to-end interactions.
v
Tricentis | Reinventing Software Testing | 5
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
Proliferation of Tools
One impact of this team decentralization is that different teams select different tools to drive their quality
processes. That selection of independent tools might fulfill the needs of individual teams—but what happens
when you need to make a fast (often automated) release decision regarding the broader application, which
probably involves multiple teams and toolsets?
Such a decision requires an accurate assessment about how the latest changes impact the user experience.
However, the data required to understand that change impact is scattered across different teams and tools—
open source, custom built, and commercial.
Highly Interdependent Applications
Next, let’s shift to the extreme interdependency that inevitably stems from decentralized system
architectures. Obviously, this interdependency is going to be a much more pressing pain for a QA team
responsible for end-to-end business transactions that involve SAP, legacy apps, mainframes, web UIs, APIs,
etc. than it is for a DevTest team building microservices for cloud-native apps. For the former, you can’t just
grab an open source tool, play around with it, and succeed. You need to understand all the different
technologies well enough to automate them, you need a way to feed secure, reliable compliant data into (and
across) the technologies, and you need on-demand access (real or simulated) to all the different components
associated with each of those transactions.
However, implementing test automation is only the start. Assume you have a series of transactions that might
involve Applications A through E—and you have automated regression tests built out for these core
transactions. Now imagine all the times that one of the many associated application components changes.
For each change, you need to review and potentially update all of the associated regression tests.
It’s just a matter of time before the results become ridden with false positives and the test suite fails to detect
critical failures or negative change impacts. As the application components continue to evolve, technical debt
accumulates and teams feel that an oppressive amount of work is required just to catch up (let alone keep
up). At this point, there’s certainly a temptation to fall back on manual testing. Yet, no matter how many
people you throw at the problem, there’s simply no way that manual testing can deliver the right feedback to
the right stakeholder at the right time in a modern delivery process.
v
Tricentis | Reinventing Software Testing | 6
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
Decentralization of Performance Testing
Finally, let’s look at one more impact of decentralized architectures and teams: decentralization of
performance testing. This shift has naturally led to the proliferation and expansion of APIs, web services, and
microservices. A performance issue in any of these distributed components could have a ripple effect across
the entire application. And now that new functionality is being released weekly, daily, or hourly, each
decentralized team needs instant insight into whether their incremental changes negatively impact
performance.
In-depth load testing by performance testing specialists remains critical—but it doesn’t provide the level of
fast, on-demand load testing that’s critical for Agile and DevOps. Developers and testers need a way to
expose critical performance issues before new functionality progresses through the delivery pipeline. To
achieve this, they must be able to easily create load tests that provide fast feedback on the functionality
they’re evolving. Beyond that, they must also be able to execute and scale those load tests as needed—
without the exorbitant costs and efforts traditionally required to establish, configure, and maintain a
performance test lab.
Tricentis, the Cloud’s #1 Continuous Testing Platform
Every team is different. Some might be focused on automating traditionally manual processes while others
might be wrestling with the orchestration and correlation of all the various test automation tools they’ve
come to master. The challenge is getting to the point where you can report on whether an overarching
application or project involving all these different teams—with different cadences, architectures, tool stacks,
structures, and challenges—has an acceptable level of risk.
That’s why Tricentis provides a Continuous Testing platform which is optimized for all the various challenges
that modern QA and DevTest teams face. Start by tackling your most pressing pains, then easily expand as
expectations shift and new challenges emerge.
Tricentis helps enterprise testing teams:
• Expose change impacts in minutes with advanced, resilient test automation optimized for 150+
technologies
• Modernize testing across SAP and packaged apps with the most comprehensive testing solution for
the intelligent enterprise
• Accelerate release cycles by orchestrating and scaling testing efforts across your teams, projects,
applications, and tools (including open source)
• Reduce risks with centralized visibility/traceability, predictive analytics, and “release readiness”
dashboards
v
Tricentis | Reinventing Software Testing | 7
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved
Tricentis features several core components that can be applied individually or in concert across your teams
and projects:
If you’re intrigued and would like to learn more about the platform and its specific components, we
encourage you to read Gartner’s independent analysis, get a free trial, or sign up for a quick demo.
top related