extreme automation: software quality for the next generation enterprise

K4

Keynote

5/8/2014 8:30:00 AM

Extreme Automation: Software

Quality for the Next Generation

Enterprise

Presented by:

Theresa Lanowitz

voke, inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Theresa Lanowitz

voke, inc.

Founder of voke, inc. Theresa Lanowitz is recognized as a strategic thinker in the application lifecycle, virtualization, cloud computing, and convergence markets. From 1999â€“2006 Theresa was a research analyst with Gartner where she pioneered the application quality ecosystem, championed the application security space, and identified new and emerging companies to watch. At Gartner, she was the founder and chairperson of the highly successful Application Development conference. Theresa’s professional career began with McDonnell Douglas where she worked on the C-17 transport plane. At Borland Software, Theresa shipped the Java development tool JBuilder. At Sun Microsystems, she was responsible for the strategic marketing of the Jini project.

4/29/2014

1

© 2014 voke media, llc. All Rights Reserved. Reproduction and distribution are prohibited.

◉ Software Quality for the

Next Generation Enterprise

Theresa Lanowitz, voke Founder

STAREAST | Orlando, FL | 8 May, 2014


◉ Agenda

• About voke

• Global Trends

• Next Generation Enterprise

• Extreme Automation

- Lifecycle Virtualization

- Development Testing

- Mobility

• Embracing Extreme Automation

• Net/Net

4/29/2014

2


๏About voke


◉ About voke

• Independent industry analyst firm

• Founded in 2006 by Theresa Lanowitz

• Firm focuses on the application lifecycle and its transformation

including virtualization, cloud computing, embedded systems,

mobile and device software

• Provides strategic, independent, and impartial advice and market

observations through quantitative and qualitative research

4/29/2014

3


๏Global Trends


◉ Current Reality

4/29/2014

4


◉ Goldman Sachs – August 20, 2013

An internal system that Goldman Sachs

uses to help prepare in meeting market

demand for equity options inadvertently

produced orders with inaccurate price

limits and sent them to exchanges.

Goldman Sachs placed four senior

technology specialists on

administrative leave after a

programming error caused the

investment bank to send faulty

stock-options orders.


◉ NASDAQ – August 22, 2013

A “computer malfunction”

froze thousands of

securities listed on the

Nasdaq Stock Market for

three hours. Many of the

country’s most-traded

shares ground to a

standstill as brokers were

unable to execute orders.

4/29/2014

5


◉ Standard & Poor’s Warning – September 20, 2013

A rash of electronic breakdowns

could trigger cuts in credit

ratings for stock exchanges,

warns ratings agency Standard

& Poor’s.

The ratings agency expects

exchanges to maintain

enough liquid capital to cover

unexpected losses caused by

electronic disruptions.


◉ More Software Failures Daily

4/29/2014

6


◉ Why Are Software Failures Making Headlines?

• Economy

- Lack of investment during Global Financial Crisis (GFC)

• Faster, faster, faster

- Faster is not equal to better

• Old technology

- Most organizations are not up to date on current software platforms

- Most organizations are not leveraging the power of modern solutions


YouTube reported a peak of more than 8

million concurrent livestreams of Felix

Baumgartner’s 24-mile/38.6 km fall to Earth.

◉ Red Bull Stratos & YouTube – October 14, 2012

4/29/2014

7


๏Next Generation Enterprise


4/29/2014

8



4/29/2014

9


◉ Parity – A Basic Requirement


◉ Testing Roadmap – Critical for Business Success

Chaos

Establish

Testing

Role

Dev and

Testing

Parity

IT &

Business

Alignment

Change

Agent

4/29/2014

10


◉ Next Generation Enterprise


๏Extreme Automation

4/29/2014

11


The concept of leveraging modern tools to break

through barriers and overcome historic

challenges to mitigate risk and manage

complexity throughout the software supply chain.

Source: voke – February 2014

◉ Extreme Automation – Definition


◉ Factors Driving Extreme Automation Need

1. Composite applications

2. Software supply chain

3. Consumerization of IT

4. Demand for quality

5. Mobile

6. Cloud

7. Competitive business differentiation through software


4/29/2014

12


◉ Classic Challenge – Deliver All Three

Cost

Quality Schedule


◉ People, Process, Technology – Building Blocks

People Process Technology

• Internal skills

• Trusted partners

and providers

• Customer advocate

• Relationship

manager

• Process designation

• Communication

• Collaboration

• Optimized

• Testing platform

?

4/29/2014

13


◉ Minimize Risk – Move to Extreme Automation

How Development Ensures

Working Software

Technology

Companies

Enterprise Companies

Continuous Integration 61% 47%

Code reviews 60% 64%

Automated unit tests 63% 38%

Manual unit tests 46% 56%

Automated functional testing 51% 40%

Manual functional testing 59% 64%

Performance testing 58% 42%

Source code analysis tools used 29% 11%

Code refactoring 39% 20%

Database refactoring 20% 9%

Design patterns 21% 16%

Domain-driven design 14% 4%

Source: voke Market SnapshotTM Report: Agile Realities – July 11, 2012


๏Lifecycle Virtualization

4/29/2014

14


Server Desktop Lifecycle(Service and Lab)

Market Adoption

Phase

Mature Trending Evolving

Technology

Phase

Stable and full

featured

Stable with

emerging

features

Stable and evolving

Primary Users Data center Service

center

Dev, test, and

operations

Vendors VMware,

Microsoft, Citrix

Citrix,

Microsoft,

VMware

CA, HP, IBM,

Microsoft, Parasoft,

Skytap, Tricentis

◉ Evolution of Virtualization


Service Virtualization – Enabling development and test

teams to simulate and model their dependencies of

unavailable or limited services. Removes constraints and

wait times frequently experienced by development and

test teams to access components, architectures,

databases, mainframes, etc.Source: voke Market SnapshotTM Report: Service Virtualization – December 2012

Virtual Lab Management – The creation of a virtual instance

of a physical environment to enable anyone to access a

virtual equivalent of any environment on demand at any

time without the need to wait for a physical machine set-up

or additional hardware.Source: voke Market SnapshotTM Report: Virtual Lab Management – March 2010

◉ Lifecycle Virtualization – Definitions

4/29/2014

15


◉ Access to Required Systems

33Average number of

systems needed

for dev or test

18Average number

of systems with

unrestricted

access for dev

or test

Source: voke Market SnapshotTM Report: Service Virtualization – December 2012


◉ Lab Access – Wait Time

96%Wait for

access

Wait Time to Access Lab Participant PercentagesNone – immediate access 4%

1 hour or less 7%

2-4 hours 14%

5-8 hours 11%

2-4 days 38%

1 week 11%

2-4 weeks 14%

10 weeks 1%


4/29/2014

16


◉ Lab Access – Restrictions

69%Time

restrictions

Lab Access Restrictions Participant Percentages4 hours or less 31%

5-7 hours 6%

1-2 days 22%

1 week 6%

4-6 weeks 4%

No restrictions 31%



Category Metric

Reduced Wait Times 64% -achieved a 50% – 100% reduction in

wait time

Greater Availability of

Services

75% - achieved a 50% or greater increase in

availability of services

Decreased Software Cycles 55% - achieved a 25% - 50% reduction in

overall software cycle

Increased Test Coverage 59% - achieved 50% - 90% increase in test

coverage

Shorter Test Cycle Times Average 58% decrease in test cycle length

Improved Time-to-Market Average 24% faster time-to-market

Fewer Production Defects Average 22% reduction of production

defects


◉ ROI Service Virtualization – Summary

4/29/2014

17


๏Development Testing


◉ Development Testing – Definition

Complement traditional blackbox functional testing.

Transform the software testing activity from reactive to proactive by

moving testing upstream to identify and remediate defects earlier in

the lifecycle.


4/29/2014

18


◉ Development Testing – Overview

Primary User Tool Types Function VendorsDevelopers Static code analysis Defect prevention

• Memory leaks

• Deadlocks

• Race conditions

Coverity

Parasoft

Developers Dynamic code analysis Detect dependencies

based on system execution

Coverity

Parasoft

Developers Unit testing Smallest testable part of an

application to ensure code

meets its design and

behaves as intended

Coverity

Microsoft

Parasoft

Developers

QA professionals

Application security testing Identify security

vulnerabilities through

scanning and penetration

testing

Coverity

HP

IBM

Parasoft

Development Managers Policy Management Define consistent coding

standards across teams

Coverity

Parasoft


◉ Development Testing – Benefits

• Test at every phase of the application lifecycle

• Collaborate and communicate with development to increase test

coverage

• Remediate defects earlier in the lifecycle to save time and money

• Reduce traditional testing cycle by focusing testing on areas that

require more thorough testing (increase code coverage)

• Reduce the number of defects entering production

4/29/2014

19


๏Mobility


◉ Evolution of Complexity – Testing Challenges

Mainframe Client/Server Web Mobile

Co

mp

lexit

y

4/29/2014

20


◉ Pillars of Testing – Adding Mobile

Does it work? Does it perform? Is it secure?

Functional Performance Security

Lo

ad

Stre

ss

En

du

ran

ce

Ne

two

rks

Pla

tform

s

>

Req

uire

men

ts

Man

ual

Reg

ressio

n

Exp

lora

tory

Ne

two

rks

Pla

tform

s

Us

er E

xp

erie

nc

e

>

Scan

s

Pen

etra

tion

Sta

tic A

naly

sis

Devic

e

Data

Co

mp

lian

ce

>= Additive for mobile

Infra

stru

ctu

re


◉ Mobile Apps – Best Practices

• Mobile testing IS application testing

• Extend enterprise best practices to incorporate mobile

uniqueness

- Gestures

- Location

• Leverage existing skills in QA

- Platform tooling

- Automation

- Performance

- Security

• Include new team members with specific mobile skills

- Hardware

- Infrastructure

- OS

- Networking

4/29/2014

21


๏ Embracing Extreme Automation


◉ Extreme Automation – Benefits

• Goal – Extreme Automation

• Automation is critical

- Traditional applications

- Mobile

• Reduces risk associated with multi-platform delivery

- Mobile demand

• Reduces overall software cycle time

• Increases code coverage

• Assists in managing the dynamic nature of mobility

• Enhances visibility into the software supply chain and reduces

risk

• Enhances communication and collaboration throughout the entire

lifecycle

4/29/2014

22


◉ Extreme Automation – Getting There

People Process Technology

Leverage

relationships with

vendors

Select pilot projects

for new technology

Get current and stay

current on existing

tools

Leverage

relationships with

professional service

providers

Form a CoE for new

technology

Evaluate new tools

offered by current

suppliers

Update internal skills

to focus on achieving

extreme automation

Leverage a CoE

model to go from pilot

project to department

/ enterprise use

Explore innovation

from other vendors


◉ Embrace New Technology – InnovateTechnology Market Vendor/Product Benefit to Quality

Automated Testing • HP Sprinter

• Microsoft Visual Studio Test Professional

• Tricentis Tosca Testsuite

• Dynamic creation of test cases

• Increase testing across platforms

Lifecycle Virtualization • CA Lisa

• HP Service Virtualization

• IBM Rational Test Virtualization Server

• Microsoft Visual Studio Lab Manager

• Parasoft Virtualize

• Skytap

• Tricentis Tosca Virtualize

• Reduce wait times

• Increase availability of services

• Reduce defects entering production

• Reduce overall software cycle time

• Reduce test cycle time

• Increase code coverage

Development Testing • Coverity Development Testing Platform

• Microsoft Visual Studio

• Parasoft Development Testing

• Prevent defects from leaving development

• Increase predictability

• Increase visibility across the lifecycle

• Increase collaboration

Continuous Integration • Electric Cloud Electric Commander


• Prevent broken builds from entering QA

• Automate build/test/deploy process

Requirements • HP Requirements Management

• IBM Rational Requisite Pro


• Provide a single view across lifecycle

• Trace requirements across lifecycle

• Prevent requirements defects from

entering production

Performance • HP Performance Center

• IBM Rational Performance Tester


• Neotys NeoLoad

• Prevent and manage non-functional

defects

• Retain customer engagement

Application Security • Coverity Security Advisor

• HP Security Solution

• IBM Security AppScan

• Parasoft Application Security Solution

• Reduce security vulnerabilities from

entering production

• Manage security risks of applications

4/29/2014

23


๏ Net/Net


◉ Net/Net

• Software and the brand are linked

• Testing organization is a customer advocate and brand protector

• Embrace people, process, AND technology

• Deliver on extreme automation- Reduce the risk of failure in production

• Lifecycle virtualization is real and proven - Increase time-to-market

- Reduce defects entering production

- Reduce software cycle

- Reduce test cycle time

- Increase test coverage

• Move testing upstream with development testing- Increase code coverage

• Mobile testing IS application testing- Identify YOUR mobile strategy

4/29/2014

24


◉ Learn More at www.vokeinc.com

• Become a Research client at www.vokeinc.com

- voke Market Snapshot: Service Virtualization

- voke Category Snapshot: Lifecycle Virtualization

- voke Market Snapshot: Virtual Lab Management

- voke Market Snapshot: Agile Realities

- voke Strategic Brief: Cost of Rework

- voke Market Mover Array: Testing Platforms

- voke Market Mover Array: Application Lifecycle Management

• Follow voke on Twitter: @vokeinc

• Take our surveys


Thank You!!!

4/29/2014

25


๏ Q&A


◉ voke Contact

voke, inc.

2248 Meridian Boulevard

Suite H

Minden, NV 89423

P: +1.866.895.9045

W: www.vokeinc.com

T: @vokeinc

extreme automation: software quality for the next generation enterprise

Technology

voke media

founder of voke

theresa lanowitz voke

voke global trends

voke founder stareast

software quality

theresa lanowitz firm

software failures