continuous testing in the agile age
TRANSCRIPT
![Page 1: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/1.jpg)
Continuous Testing in the Agile AgeTesting as part of the Continuous Delivery process
785 Castro Street, Mountain View, CA 94041 | 1.855.445.2285 | [email protected]
BlazeMeter
ITAY MENDEL Devop
MICHAEL REDMANCustomer Success
ANDREY POKHILKOChief Scientist
ALON GIRMONSKYFounder & CEO
R
![Page 2: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/2.jpg)
Waterfall Agile
2015 CURRENT STATE OF AFFAIRSMoving from Waterfall to Agile
![Page 3: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/3.jpg)
“Continuous Delivery (CD) is rapidly emerging as the ‘new normal’ in software development, with approximately 80% of SaaS companies and 51% of non-SaaS companies adopting this practice”
Perforce Software Study
![Page 4: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/4.jpg)
QA wasn’t invited to the party
THE TESTING BOTTLENECK
![Page 5: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/5.jpg)
TESTING IS COMPLICATED!Why?
![Page 6: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/6.jpg)
Developers• Unit
• Troubleshooting
CI/DevOps QA Operations• API
• Functionality
• Performance
• Regression
• Load
• Soak
• Regression
• Acceptance
• Load
• Monitoring
• Discrete VS continuous
• Frequent code commits
• Domain expertise required
• Multiple test points
ON THE VERGE OF CONTINUOUS DELIVERYTesting is a Challenge
![Page 7: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/7.jpg)
TIM
E TO
REL
EASE
RELEASE NUMBER / TIME
20 40 60 80 100 120 140 160 180 200 220 240
2wks
1wks
3wks
4wks
5wks
THE OUTCOMEAny Agile Process is Most Likely to Fail
![Page 8: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/8.jpg)
Test Early
Test Often
Test Everywhere
Automate it
CONTINUOUS TESTING PROCESS
![Page 9: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/9.jpg)
Test Early
Test Often
Test Everywhere
Automate it(and get everyone on the same toolset)
CONTINUOUS TESTING PROCESS
![Page 10: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/10.jpg)
CONTINUOUS TESTING
REDUCED TIME-TO-TEST
INCREASED # OF ITERATIONS
AUTOMATION
TIME-TO-RELEASE QUALITY
TIM
E TO
REL
EASE
RELEASE NUMBER / TIME
20 40 60 80 100 120 140 160 180 200 220 240
2wks
1wks
3wks
4wks
5wks
Before
After
CONTINUOUS TESTING FACILITATES AGILE
![Page 11: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/11.jpg)
Developer
Devops
QA
OPs
OPs
Commit Code +
Test Representation
Configure
Test
Acceptance Test
Configure
Version Control Repository
Comprehensive Testing
Post-Production Monitoring
Continuous Deployment
Continuous Integration
PEOPLE
Red Arrow = Test Failed Green Arrow = Test Succeeded Blue Arrow = Sent to Testing
SYSTEMS TESTING ALERT
CONTINUOUS TESTING FACILITATES AGILE
Leve
rage
Pre
vio
us
Test
Rep
rese
nta
tio
n
Co
nti
nu
ou
s T
es
tin
g
![Page 12: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/12.jpg)
IS POST-PROD MONITORING A PART OF TESTING?
![Page 13: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/13.jpg)
Developers CI/DevOps QA Operations• GitHub
• JSON tests
• Jenkins
• JSON tests
• + JMeter tests
• + New Relic
• JSON tests
• JMeter tests
• New Relic
• Load & Soak
• Deploy
• Test
• Monitor
CD STACKWhat Toolset Will We Use?
![Page 14: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/14.jpg)
Demo?We suggest you watch this 7 minute movie to see BlazeMeter in action: https://www.youtube.com/watch?v=6Oih-v-LICE
Please view in HD quality.
![Page 15: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/15.jpg)
• Testing starts @ Dev
• The module’s test configuration is maintained by the developer in a JSON format
• Version control friendly
• Devop configures CI
• Tightly integrated with CI environments
Alert on Failure + Artifacts✖
ENTERPRISE CONTINUOUS TESTINGUse Case #1
![Page 16: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/16.jpg)
PERFORMANCE LAB IN THE CLOUDUp to 1M users, multi-GEO, 100% Apache JMeter compatible
![Page 17: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/17.jpg)
ON PREMISE + CLOUD TRAFFIC GENERATIONFull hybrid model
• Allocate as many on-premise servers as needed
• Group them into clusters (e.g. R&D, QA, OPs)
• Assign permissions
• Have each cluster appear as yet another GEO location
![Page 18: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/18.jpg)
ANY SCRIPT, ANY SCENARIOComprehensive scripting at your fingertips
JMeter is the most commonly used open source web, mobile and API backend testing solution. BlazeMeter is 100% compatible with JMeter
HTTP / HTTPS
Login / Signup
Form POSTing / Credit card transactions
Rest / Web services / JSON
Parameters extraction ( JSON / RegEx / Xpath )
Mobile
Mongo, MSSQL, Oracle SQL, MySQL
Multiple user profiles
![Page 19: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/19.jpg)
PRODUCT: JUST A TASTEO
n-P
rem
ise
A JSON Test Representation VS GUI
![Page 20: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/20.jpg)
On
-Pre
mis
e
PRODUCT: JUST A TASTEA JSON Test Representation VS GUI
![Page 21: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/21.jpg)
THRESHOLDS Setting global and per URL thresholds (Automation)
![Page 22: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/22.jpg)
THRESHOLDS Setting global and per URL thresholds (UI)
![Page 23: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/23.jpg)
CONTINUOUS INTEGRATIONConnect with leading CI applications
![Page 24: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/24.jpg)
JENKINSLeverage inherent reporting
![Page 25: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/25.jpg)
NEWRELIC INTEGRATION
Capture Monitoring DataCapture Load Testing Data
Ask the Test Data any Question
![Page 26: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/26.jpg)
AWS CLOUDWATCHFetch CW related data associated with selected AWS resources
![Page 27: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/27.jpg)
QUICKLY RECORD ANY SCENARIO
• Easily Record:
Web traffic
Mobile web or apps
HTTP/HTTPS
APIs
• No JMeter knowledge or installation required
![Page 28: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/28.jpg)
REST API AND DSL SCRIPTINGBuilds into any continuous delivery process
![Page 29: Continuous Testing in the Agile Age](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a865841a28abf93f8b4642/html5/thumbnails/29.jpg)
THANK YOU FOR YOUR TIME!
Q & A
For more information:
BlazeMeter.com/blog/ [email protected] @BlazeMeter