combining automated functional and load testing
TRANSCRIPT
![Page 1: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/1.jpg)
IntegratingRanorex and NeoLoad
Time to make peace between functional and non-functional testing
![Page 2: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/2.jpg)
Welcome to the Ranorex-Neotys Webinar
Christian Breitwieser, RanorexInnovation Manager
Henrik Rexed, NeotysTechnical Evangelist
![Page 3: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/3.jpg)
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
![Page 4: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/4.jpg)
• Functional test automation for desktop, web and mobile applications
• Cross-technology end-to-end testing• Tool for testers and developers
![Page 5: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/5.jpg)
• Functional test automation for desktop, web and mobile apps
• Cross-technology end-to-end testing
![Page 6: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/6.jpg)
Tool for testers and developers
Code Module
Action Based Modules
Ranorex Studio
![Page 7: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/7.jpg)
• Deliver faster applications faster through continuous performance validation
- Who We Are
![Page 8: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/8.jpg)
- Who We AreMission: Deliver faster applications faster through continuous performance validation
![Page 9: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/9.jpg)
- Who We AreMission: Deliver faster applications faster through continuous performance validation
![Page 10: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/10.jpg)
User Demands Process Revolutions
We’re Driven by Your Challenges
Technology Transitions
![Page 11: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/11.jpg)
User Demands Process Revolutions
We’re Driven by Your Challenges Technology Transitions
![Page 12: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/12.jpg)
User Demands Process Revolutions
We’re Driven by Your Challenges Technology Transitions
![Page 13: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/13.jpg)
User Demands Process Revolutions
We’re Driven by Your Challenges Technology Transitions
Docker
![Page 14: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/14.jpg)
Load and Performance Testing
24x7 Performance Monitoring
Continuous Performance Validation Solution
![Page 15: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/15.jpg)
Load and Performance Testing
24x7 Performance Monitoring
Continuous Performance Validation Solution
![Page 16: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/16.jpg)
Continuous Performance Validation Solution
DevelopmentContinuous
Integration & Component Testing
![Page 17: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/17.jpg)
Continuous Performance Validation Solution
DevelopmentContinuous
Integration & Component Testing
![Page 18: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/18.jpg)
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
DevelopmentContinuous
Integration & Component Testing
![Page 19: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/19.jpg)
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
DevelopmentContinuous
Integration & Component Testing
![Page 20: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/20.jpg)
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
DevelopmentContinuous
Integration & Component Testing
![Page 21: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/21.jpg)
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
Production
Proactive Monitoring
DevelopmentContinuous
Integration & Component Testing
![Page 22: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/22.jpg)
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
Production
Proactive Monitoring
DevelopmentContinuous
Integration & Component Testing
![Page 23: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/23.jpg)
Continuous Performance Validation Solution
Pre-productionSystem-wide
Testing
Production
Proactive Monitoring
DevelopmentContinuous
Integration & Component Testing
![Page 24: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/24.jpg)
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
![Page 25: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/25.jpg)
User Experience is Crucial• 57% of users use their mobile devices to access
applications
• 40% of users move towards a competitor after a bad user experience
• 44-61% of users share their feelings on social networks
![Page 26: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/26.jpg)
What isUser Experience? Usability
Ergonomics
Performance
Accessibility
Functionality
![Page 27: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/27.jpg)
User Experience Impacts the Business
![Page 28: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/28.jpg)
Case StudyImpact of the response time on the conversion rate:
![Page 29: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/29.jpg)
![Page 30: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/30.jpg)
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
![Page 31: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/31.jpg)
Why Functionalityis important Usability
Ergonomics
Performance
Accessibility
Functionality
![Page 32: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/32.jpg)
Otherwise…
![Page 33: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/33.jpg)
Test AutomationPyramid
Unit
Integration
Functional
Manual
![Page 34: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/34.jpg)
Manual & Integration TestingManual testing• Time-consuming• Error-prone• Cannot „run“ overnight
Integration testing• No black box• Hard to cross technology
borders• Good coding skill required
![Page 35: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/35.jpg)
Benefits of Automated Functional Testing• Quick feedback• Parallel test execution• Scalable
• Codeless functional testing• Increased test coverage
![Page 36: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/36.jpg)
Numbers & FactsTomTom – Case Study:
• Testing time: from 10 days down to 4h• Over 90% of time saved with Ranorex automated tests
Siemens – Case Study:• Due to the use of C#, new test cases can be implemented
more effectively• Only 60% of the time needed with Ranorex compared to
previously used tool
![Page 37: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/37.jpg)
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load Testing
• Live Demonstration
![Page 38: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/38.jpg)
Separated Functional & Performance Testing
• Functional testing ensures functionality… but not when the AUT is under load
• Load/Performance testing ensures a fluent user experience… but does not guarantee a functional UI
![Page 39: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/39.jpg)
Ranorex Customer Experience Survey No Idea
Yes No54% 37%
9%
Do you combine Ranorex test automation with non-functional tests?
In addition to functional test automation you perform…
Other
Load Testing
Database Testing
Web-Service-Testing
Performance Testing
Unit Testing
4%
19%
22%
27%
31%
33%
![Page 40: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/40.jpg)
Functional – Performance Tests
Golden Rule: Automated performance testing should not take place if functional regression tests failed.
Performance tests require that the application or part of the system is:
• Operational: No systematic errors or exceptions
• Representative : Enough data available in the test environment to determine the behavior under load conditions
![Page 41: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/41.jpg)
How Can We Increase Risk Coverage?
100 test cases
40 test casescovering 80% of the risknot necessarily selected for performance testing
We prioritize test cases based on risk. These test cases are validated through :- Automated regression testing- Exploratory testing
![Page 42: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/42.jpg)
Validate Critical Business Case Under Load
This guarantees:• The stability of the crucial use cases under real usage
conditions• High user experience• The identification of potential bottle-necks, even across
technology borders
![Page 43: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/43.jpg)
The Initial Workflow…Automated functional
testing
Exploratory testing
Load testing
Exploratory testing
under load
…
![Page 44: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/44.jpg)
Applying the Ranorex-NeoLoad Integration…
00:00:00 00:00:30 00:01:00 00:01:30
Load
NeoLoad
Application Server
App Error Rate
Virtual Users Load
Automated functional
tests
UserExperience
![Page 45: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/45.jpg)
… You‘ll Find Bugs Earlier!Automated functional
testing
Automated functional
testing under load
Exploratory testing
Exploratory testing under
load
…
![Page 46: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/46.jpg)
Agenda• Quick Company and Tool Intro• Performance Matters in User Experience• Benefits of Automated Functional Testing• Benefits of Combining Automated Functional and Load
Testing• Live Demonstration
![Page 47: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/47.jpg)
Further Resources
Combining Automated Functional and Load Testinghttp://www.ranorex.com/blog/combining-functional-and-load-testing
How to Combine Ranorex and NeoLoad Testshttp://www.ranorex.com/blog/combine-ranorex-and-neoload-tests
![Page 48: Combining Automated Functional And Load Testing](https://reader036.vdocuments.mx/reader036/viewer/2022070521/58f022c11a28ab553e8b4583/html5/thumbnails/48.jpg)
Questions?
Henrik Rexed: [email protected]
Download NeoLoad Free Edition
Visit our neotys.com
Follow me on Twitter @hrexed
Christian Breitwieser: [email protected]
Download free Ranorex trial
Visit www.ranorex.com
Follow us on Twitter @Ranorex