webinar: going agile? don't forget about testing

September 25, 2015

Introduction to QASymphony for [INSERT COMPANY NAME]

#AgileTransformation

Agile Transformation: People, Process and Tools to Make

Your Transformation Successful


The audio for this webinar is delivered through your computer. There is no dial-in number. Make sure your speakers are turned up or use a pair of headphones.

Going Agile? Don’t Forget About Testing


HOW TO WATCH THIS WEBINAR

• Audio for this webinar is delivered through your computer. Make sure your speakers are turned up or use a set of headphones.

• If your audio quality is choppy, it could be your internet connection.

• You can customize your webinar viewing experience by increasing, decreasing or minimizing the size of the widgets on your screen.

• If you have questions, enter them in the widget on the right.

2


• This webinar will be recorded and available on-demand tomorrow. You will get an email when it is available.

• Join the conversation on Twitter using the hashtag #AgileTransformation

• Use the Q&A widgets to ask questions during the webinar.

• At the end of the webinar, you will be asked to take a short survey.

HOUSEKEEPING


NEXT WEBINAR

#AgileTransformationRobust test management platform purpose-built to help agile

teams accelerate software testing

ABOUT QASYMPHONY


ABOUT GALLOP

Analyst Recognitions

Awards

Thought Leadership

Certifications

Partnership with Leading Tools

Vendor

2,000+ resources worldwide

FY2015-16 turnover of USD 85mn that grew at a CAGR of ~58% over

the last 3 years

Currently serving 80 of the Forbes

Global 2000 Companies

Project Cignificance aims to empower lives with free

and quality education

CSR Initiatives

Independent Testing Services

Company in North America

Publicly Listed & CMMi L5 certified

5 continents, 10+ countries

Global presence

500+ Satisfied

Customers

58%+ Revenue Growth

Gallop Thought Leadership

Total Quality Management

5 Patents*

Testing IP

400+ Testers in North America1200+ Testers at the Global Delivery

Center in India

Certifications : ISTQB, HP, IBM, CSQA, CEH, CISSP and more

Reduced Cost of Quality

*Pending


OTHER WEBINARS & RESOURCES

WWW.QASYMPHONY.COM/RESOURCES


PRESENTERS

Kevin DunneVP, Strategy and Business Development

QASymphony@KevinDunneQA

Sujit NairAssociate Director with the Enterprise Solutions Group

Gallop@Sujit_N


WHY COMPANIES ARE GOING AGILE

There are a number of microeconomic and macroeconomic trends pushing teams towards adopting agile, some of the most prominent include:

App EconomyInstant feedback and

“needed it yesterday”mentality

Cloud AdoptionAbility (and expectation)

for frequent releases

Best of BreedIntegrating several tools,

increasing complexity

Startup CompetitionFast moving startups with

funding disrupting markets

Open SourceAbility to build quickly on

existing components

Digital DisruptionShift from traditional

channels to mobile & web platforms


WHAT DOES AGILE LOOK LIKE?

Agile differs from Waterfall in creating shorter, more integrated cycles that generate code more quickly, allowing teams to react to changes as they come and reduce risk and waste

Source: http://www.kenwayconsulting.com/wp-content/uploads/2012/03/Agile-v-Waterfall.jpg

http://www.kenwayconsulting.com/wp-content/uploads/2012/03/Agile-v-Waterfall.jpg


Making a successful Agile transformation is not as simple as selecting an agile process and moving forward, it requires thoughtful planning across process, as well as people and tools

WHAT DO WE NEED TO CONSIDER?

Process

● What works well in my current process? What is not working well?

● What do I value - time to market, cost, quality, something else?

● What restrictions are in place to keep us from adopting agile? Which can we move?

People

● What valuable skills do my current employees have? Which are they lacking?

● Is my organization equipped to deal with uncertainty?

● Can my people adopt new skillsets needed by agile? Mindsets?

Tools

● What tools are critical to my process? Which can be replaced to assist in transformation?

● What new tools and categories of tools will be required?

● How can I empower smaller teams to use and leverage tools?

Thank You


PROCESS


WHAT AGILE MEANS FOR TESTING

Agile development means more frequent development cycles, often with more regular code delivery.

What does this mean for testing?1. Testers must be involved and aligned with the entire

development process2. Testing will happen much more regularly, in smaller

increments3. Testing will follow development more closely, in tandem

with coding


HOW TO OPTIMIZE TESTING FOR AGILE

Get Testers Involved EarlyEmbed testers within product or scrum teams, making sure they have a seat at the table in

planning discussions and provide insights around quality and testing considerations

Embrace Frequent, if not Continuous, TestingAs agile promotes frequent code merges, builds, and deployments, employ a process that allows

the most vital testing to be identified and executed efficiently

Decrease the Time from Design to TestWith 2-3 agile sprints, we do not have weeks (or even days) to define and automate test cases,

adopt a process for manual or automated tests that allows tests to closely follow designs


GALLOP & AGILE TESTINGS

DLC

Requirements Gathering

Application Design Development Testing

Test Requirement Analysis

Test Planning & Strategy Test Design Text

Execution

Analyse PlanDesign

BuildTest

Deploy

Analyse PlanDesign

BuildTest

Deploy

Peo

ple

Development Lead QA Lead

Project ManagerDevelopers Testers

Functional Automation

Developer ArchitectSCRUM Master

Sprint Team

Pro

cess

Tool

s

Product Owner

Pro

cess

Test Management Defect Management Automation

Service Test

High Focus on Compliance Focus on Outcomes

Process Heavy Project Based Delivery Lean Processes Product Based Delivery

Waterfall to Agile Transformation

http://findicons.com/icon/13483/user?id=295109

http://findicons.com/icon/13483/user?id=295109


Quality Assurance to Quality Engineering – The Transition

Manual Testing Skills

Specific Testing Skills

Regression Automation

Performance Testing

Manual Test Cases

Black Box Testing

Prolonged Cycle Time

UI Testing

Standard Third Party Tool Sets

Transform to QE Skill-Set

Optimization by Cross Training

Early Functional Automation

Performance Engineering

Automated Test Case Generation

White Box Testing

Reduced execution cycles

Database, Services & Component Testing

Open Source Tools & Commercial Tools

Quality Assurance

(From)

Quality Engineering

(To)

Tran

sition

Courtesy by Lisa Crispin/Brian Marick

Agile Testing Quadrants


GALLOP’s AGILE ASESSMENT FRAMEWORK

Discover and Document

Analyze and Benchmark

Roadmap & Implementation

Approach

Goals, Constraints, Timelines, Risks, Key

Focus Areas

Applications, Technology, Platform,

Automation Framework,

Automated scripts

Agile Processes, Metrics, Tools Strategy, Test

Assets, Skills, Org Map, Functional Testing, Test Environments, Test Data

Performance Goals, Scripts & Reports

Assess Performance Test Strategy, Performance

Engineering, Monitoring and related processes

Evaluate functional testing assets, maturity

of QA processes & benchmark based on agile maturity model

Analyze Tool Utilization, Licensing,

Integration opportunities

Review Test Automation Strategy, Framework maturity,

test scripts & coverage

Design the roadmap based on the target

operating model and prioritized key focus

areas

Identify & Prioritize the key focus areas

Review the analysis and findings with PoC & Key

Stakeholders

Design the Target Operating Model

Deliverable

s

Assessment Plan Test Assets Inventory

Target Operating Model

Agile QA Test Maturity

Implementation Roadmap

Interim Test Assessment Report

Final Test Assessment Report


Discover and Document

Create Test Assets Inventory and Review with the POC

Elicit the existing processes, tools, test assets related to functional testing, test automation, performance testing and also the team skills to understand the AS-IS state and provide comprehensive documentation

Understand and Document Test Assets

Understand Organization Setup and Day in the life

of QA

Understand the existing processes

and test assets

Collect process and project artifacts

S.No Test Assets Inventory Overview Asset Type Details

1 Agile / SCRUM Guide Guidelines for working with SCRUM methodology, techniques

Process Docs 1

2 Test Management Tool Test Management Tool Tools 13 Automation Tool QTP Tool Overview Tools 14 Performance Testing Performance engineering scripts Non-functional Scripts 2005 Manual Test Cases for App1 Functional Test Cases for App1 Functional Test Case 12406 Manual Test Cases for App2 Functional Test Cases for App2 Functional Test Case 6737 Automated Test Scripts Automated Functional Test Scripts for App1 & 2 AutomatedTestScripts 350

8 Environments Environments - Dev, QA, Pre-Production, Production, Demo, Training

Environments 5

9 Test Org Roles Team structure and R&R Roles List 1

Organization Map Template

Test Assets InventoryTest Assets

Inventory Template

Resource Skills Inventory Template


Analyze and Benchmark

Design Target Operating Model

Overall Agile Test Maturity

Customized Assessment Framework built using industry standard models such as SAFe, TPI, Agile & DevOps, ITIL, TMMi that helps to benchmark and design target operating model for transformation.

Conduct AssessmentMap to the Industry Benchmarks and Standards

Analyze the Test Assets Inventory from the

previous phase

Conduct interviews with

stakeholders

Analysis of defects and artifacts

Conduct gap analysis

Assessment Questionnaire


Develop Roadmap

Deliverable

s

Based on the score, the recommendations are categorized into Quick wins, and Medium & Long Term initiatives

Design the roadmap aligned to client’s objectives, that helps in prioritizing the initiatives and balances the risks & impact, along with timelines

Focus Areas Reccomendations Deliverables Transformation Score Ease of Implementation

ROI

Agile Processes Define Key Agile ProcessesAgile FrameworkSCRUM Kit

Deploy in wave 1 6 2 3

Tools Integration Integrate JIRA with ZenDesk Integrate workflows in tools Deploy in wave 2 2 1 2

Agile Test CoEEstablish a Flexible Org. design with specialized and generalists roles in QA

TCoE Org. DesignTeams fully aligned to TCoE 3 1 3

Performance TestingEstablish Performance Capability for prioritized applications

Application Performance Analysis ReportPerformance Scripts


Enabler Establish effective Process feedbackProcess Improvement ProcessProcess Improvement guidelinesProcess Improvement template

Deploy Process ClinicsIdenitfy Process Implementation Champions

6 2 3

Test Management Tool Estabslish Test Management ToolTool Feasibility Analysis reportTool Implementation Guide


Test Automation Establish Test Automation Capability Automation Feasibility AnalysisTest Suite, Scripts, data is made ready

6 2 3

Analyze the recommendations based on ease of Implementation and ROI


Case Study

Deliverable

s» Client is a leading provider of face to face marketing services

ranging from Pre-Event Planning, On-Site Execution, Extending the Experience to Audio Visual Solutions with a presence in North America, Europe and Asia

» Key Requirement: Functional, Regression and Compatibility Testing of four applications

» Application Environment: SharePoint and .NET Technology» Testing Types: Functional, Regression and Compatibility Testing

» Delayed receipt of user stories in every sprint» Lack of clarity in user story requirements» Parallel sprints, each with a rigid schedule» Lack of standardized QA processes and best practices» Unstable Test Environment with configuration and access issues» Critical defects leaked into post-production after regression

testing

Key Challenges

» Reduced RAI (Requirement Ambiguity Index) from 35% in sprint 1 to 12% in sprint 3 by improving the clarity of user stories

» Increased test coverage by leveraging Gallops Requirements Testing Framework

» Reduced post-production defects by 40%» Met aggressive sprint schedules with the help of reusable

process templates» Introduced industry standard Agile and QA best practices to

ensure that every sprint stays on schedule

Key Benefits» Leveraged Gallop Requirement Testing Framework to improve

requirements review process and hence the quality of user stories

» Mandated the use of sprint demos to hasten understanding of functionality being added in every sprint

» Worked with client team to resolve test environment issues» Introduced reusable process templates to accelerate and

standardize QA processes across applications and sprints» Published detailed test execution reports for the compatibility

matrix defined to enable easier and fast sprint sign-off

Solutions Provided

About the Client

Thank You


TOOLS


A NEW BREED OF TOOLS

Automation is undergoing even more dramatic changes than manual testing, with a new breed to tools:

● Open source: to provide lower cost and allow more members to contribute automation

● Cross Platform: to accommodate for multi channel experience across web, mobile, & desktop

● DevOps Capable: reliable, with ability to be run quickly and in parallel to allow for fast, frequent builds


OPEN SOURCE AUTOMATION

Every category of automation tool is undergoing pressure from open source options:

Unit Testing

● jUnit● nUnit/xUnit

UI Testing

● Selenium● Protractor

Load/Performance

● jMeter● Gatling

Build

● Jenkins● Travis

Orchestration

● Puppet● Chef

Configuration Management

● Docker● Selenium Grid


Early Automation Approach in Agile

Write Acceptance Test Cases

Decision to Automate

Test

Define Acceptance Test Criteria

Yes

Prototype Rinse & Refine Integrate Testing

Iterative design

Design/Update proto types for the Business

Components at UI and API level

Develop/Update the code, User Interface controls and

Unit test cases

Unit testing, integration testing and do bug fixes

No

User Story

Perform Manual Testing

Coordinate each other for developing Automation suite

Execute API level automation scripts on

continuous integration

environment till Acceptance criteria is

met

Execute end-to-end automation scripts on

developed UI

Design Pseudo codes/Classes for API

Level Automation

Build API Level automation scripts with negative and

positive scenarios

Execute API level automation scripts on every

component of developed code

Design Keywords for UI Level automation

Build automation scripts using Keywords for UI Level

automation

Capture UI Objects properties and integrate

with UI automation scripts

API Level Automation Testing

UI Level Automation Testing

Application Code Development

Perform goal based Exploratory testing

Adopt Hybrid Risk Based Testing for Test

optimization

Thank You


PEOPLE


MOVING FROM SILOS TO EMBEDDED TESTERS

Siloed Testers Embedded Testers


A NEW BREED OF TESTER

Testers in the past were able to work independently, focusing on discrete issues with less collaboration

Agile testers do not have that luxury, they must be much more:1. Collaborative, as they might be a single tester working

inside a product team full of developers & analysts2. Adaptable, as they might be required to wear 4 or 5 hats

in any given day3. Well versed, as they may be reassigned to many different

projects within one organization


HOW TO DEVELOP TESTERS FOR AGILE

Testers in Agile should focus on learning a broader set of skills, rather than going deep in one specific area:

Team Skills

● Collaborating in diverse teams

● Managing conflict across teams

● Hiring and retaining top talent in fast growing companies

● Creating a team culture of trust and buy in

Technology Skills

● Understanding and contributing to application code

● Understanding and contributing to automation (functional, unit testing)

● Expanding knowledge of additional testing types (performance, security, etc.)

● Expanding knowledge of build & release pipeline

Managerial Skills

● Motivating teams under high pressure

● Estimating and planning in agile organizations

● Managing agile project on both time and budget


Q&A

Questions?

Thank You


PLEASE TAKE OUR SHORT SURVEY

Thank you

webinar: going agile? don't forget about testing

Software