Quality Contamination in Agile Scrum Teams & the RemediesThe QA Strategy Sumeet Gupta @sumeetgupta1982

@srijan #SrijanWW

Learning Outcome

● How to perform QA on Agile Scrum Projects

● Defining efficient QA strategies for Agile Projects

● Agile/QA best practices

@srijan #SrijanWW

Quality Contaminations in Scrum Teams

● No QA strategy/Plan, checklist and milestones defined for Project

● No Gates/Checkpoints defined

● No Metrics defined, measured and tracked

● No guidelines for Defect severity/priority clearly defined

● No Defect Root Cause Analysis(RCA) and Quality Improvement Plan

(QIP) prepared.

● Not able to fit QA in same sprint as Development

● Don’t think before Test- No upfront test planning or

defining/understanding of Test cases/scenarios

● QA not part of Development Team- QA efforts not estimated, QA

resources/Testers considered separate hanging entity in scrum team only

responsible for QA

● No Technical Debt assessment and Reduction Plan

@srijan #SrijanWW

The Remedies and recommendations

⦿ Do Occurrence/Impact analysis for respective

Projects using the matrix and plan to address the

Issues/challenges progressively in order or

effective retrospectives. Don’t try to fix

everything together.

⦿ Keep on continuously refining and improving.

⦿ Inspect and Adapt alternatives/solutions

proposed in coming slides for each of the Quality

contaminations/challenges based on project

specific applicability and suitability.

Impact (low -> high)

Occ

urr

ence

(lo

w->

hig

h)

@srijan #SrijanWW

No QA strategy/Plan or checklist or milestones defined for Project

⦿ Define QA strategy/Plan, checklist to address the below categories as per

Deployment strategy and Project constraints.

⦿ Define milestones (like Feature complete, Code Freeze, UAT …) and add

placeholder user stories into the backlog/Sprints, if not possible to meet DONE

every User Story/Sprint(ideal case with CI/CD)

@srijan #SrijanWW

No QA strategy/Plan or checklist or milestones defined for Project

⦿ Review the checklist against status (Meeting, On-track, Off-track, Not started,

exception) at regular cadence(ideally every sprint), Highlight the Risk to Ship

if any, to the stakeholders and take corresponding resolutions based on Risk

assessment.

Code Quality Functional Testing System and Solution Testing

Release Requirement

Code Review Feature testing Load Testing Documentation

Unit/Component testing Regression testing Longevity Testing Automation

Static and complexity Analysis Gorilla Testing Solution/Integration Testing EFT/Alpha/Beta

Performance Testing Training

Security

No Gates/Checkpoints defined⦿ Define strict Definition of Done(DOD) checklist or Workflows for

UserStory/Sprint/Release and review before acceptance (sample Sprint DOD as

below)

⦿ Define 360 Quality review feedback forums and review projects at regular cadence

⦿ Raise Flag based on Project QA checklist defined in previous slide

@srijan #SrijanWW

Can’t Fit it all in Story or Sprint Done?

@srijan #SrijanWW

⦿ Mix in Release Done along the way so you don't build up a huge

debt to complete in the final Release Done

No Metrics defined, measured and tracked

MetricsDefect count grouped by severity/priority

Defect Incoming vs Outgoing trend

Defect Resolution Time

Defect Density

Code coverage

@srijan #SrijanWW

⦿ Measure and Track various QA metrics/trends (few mentioned below)

⦿ Review Metrics in QA review forums, raise flag incase of metrics

going beyond Project defined thresholds or degrades and take

corrective actions based on RCA. For e.g. Increase in Defect

Density per module/Function point might require module

refactoring

No guidelines for Defect severity/priority clearly defined

⦿ Define clear Defect guidelines for severity and Priority

⦿ Ensure Team raises bugs/Defects following guidelines by timely auditing projects or Bug scrubbing at regular intervals

⦿ Do regular bug scrubbing in respective projects based on Severity/Priority, Product Roadmap and track defect metrics For e.g. Linking duplicate defects, de-prioritizing or closing defects related to Feature de-prioritized or moved out of the Backlog scope.

@srijan #SrijanWW

No Defect RCA and Quality Improvement Plan (QIP) prepared

⦿ Identify and update component or EPIC while raising defect

⦿ Do Defect RCA at regular cadence(may be for high defect density components) to

identify focus areas or collaterals or source of defect

⦿ Based on RCA, define action plan and Quality improvement Plan like

enhancing/improving test suites, improving story quality, some re-factoring or

re-designing etc. … and add respective stories into the backlog. Based on the

Return on Investment(ROI), we can get them prioritized accordingly

@srijan #SrijanWW

Not able to fit QA in same sprint as Development

⦿ Don’t follow out-of-cycle testing or mini waterfall within sprint, with QA related work getting accumulated towards end of the sprint

⦿ As QA estimates are not done or out-of-cycle testing, there is always a backlog of QA work coming from previous sprint or DONE not being met every sprint, which initiates chain reaction of Spills every sprint and low velocity of the team

⦿ During the initial phase of the sprint QA resource is busy clearing backlog of previous sprint and during end again new items have accumulated from the current sprint items leading to no time for defining acceptance test or Test Planning – inefficient and optimal utilization of QA resources.

⦿ Meet DONE every sprint, by completing User-Stories end-to-end, instead of doing multiple half-done Issues

⦿ Test Automation should be focused for in-cycle QA

@srijan #SrijanWW

Don’t think before Test- No upfront test planning or defining/understanding of Test cases/scenarios

⦿ Estimate and groom stories efficiently so that acceptance test and test planning can be done

upfront or techniques like ATDD/TDD can be applied

⦿ Define and identify various Use cases/scenarios upfront, which might also help in clearing

scope of User Story or defining more clear Acceptance Criteria

⦿ Target maximum test coverage defining both positive and negative use-cases/test-cases and

tracking the coverage metrics

@srijan #SrijanWW

QA not part of Development Team- QA efforts not estimated, QA resource considered separate hanging entity in scrum team only responsible for QA

⦿ Estimate QA effort while estimating for User Stories and Tasks

⦿ Involve QA team members during User Story grooming and estimations for

better understanding of User Stories and refining Acceptance

⦿ QA is not designated person’s responsibility, it collective team responsibility

(necessary for self-organizing and cross-functional)

⦿ Team should review Test-cases/use cases defined by QA resource for better

understanding of requirements and identifying gaps/refining scope of User

Stories or adding additional tests.

@srijan #SrijanWW

No Technical Debt assessment and Reduction Plan

⦿ Assess Debt and develop plan to reduce over weeks,

months, releases, etc. based on Return of

Investment(ROI) and get Stories added to backlog

⦿ Build a practice that manages and reviews all debt

and gets it to zero over “n” weeks, months or

releases.

⦿ Funding Technical Debt Reduction

▪ Use interns or new joinees bandwidth

▪ Reserve capacity each sprint or Release (may be

10-20%) in agreement with Product Owner

▪ Show the ROI to Product Owner like ability to

deliver new features faster (refactor example) or

as an Opportunity asking for additional capacity

⦿ Review the Technical Debt Plan regularly (may be at

Quality review )

Debt Examples▪ Defect Backlog▪ Automation—manual tests▪ Code Complexity mgmt.▪ Static Analysis Violations▪ Refactoring not part of

practice▪ Limited/inaccurate test use

case▪ Low unit coverage with high

complexity & defect density▪ Lack of or out of date

Documentation▪ Poor architecture, usability,

reliability ▪ Build issues—slow or

breaks

@srijan #SrijanWW

No Technical Debt assessment and Reduction Plan

@srijan #SrijanWW

“Software testing proves the existence of bugs not their absence.” – Edsger W. Dijkstra

“Quality is not an act, it is a habit” – Aristotle

@srijan #SrijanWW

Thank You!

Take this conversation online by tweeting using the hashtag #SrijanWW

Sumeet GuptaAgile Coach