are you ready for more visitors cognizant gomez jan20
DESCRIPTION
To tell us more about the steps that you need to take to ensure that your applications, transactions and web pages perform well, when it matters most, under load, or at peak times, we are joined today by two experts in the field – Hari Ramachandran from Cognizant and Colin Mason from Gomez. Hari has 10 years of experience in IT industry in Software Development and Performance Management. He has lead performance testing initiatives for large mission critical applications for several Fortune 500 companies across Insurance, Banking/Finance, Travel, Retail, Manufacturing and Logistics. Hari is a Mercury LoadRunner Certified Product Consultant, and he currently heads the Non Functional Testing Center of Excellence at Cognizant. Colin is the Product Manager for Reality Load, the web load and performance testing solution at Gomez. He has 10 years of performance testing experience, and has overseen hundreds of load tests for companies all over the world. He has presented at several industry conferences, including STAR, and has authored industry-acclaimed papers on performance testing web applications. In today’s presentation Hari and Colin will discuss the challenges of delivering Web applications today, why performance testing is critical, and they will share with you their recommendations for successful performance testing. You will also learn about Gomez’s Reality Load, a new way of realistically load testing your applications from the Internet, where your customers are, so you can find and resolve more problems inside and outside the firewall than ever before.TRANSCRIPT
Are you ready for more Website visitors?Best Practices to realistically load test your applications beyond the firewall
Hari Ramachandran, Project Manager, CognizantColin Mason - Product Manager, Gomez
Cognizant Testing Services
Need for Positive User Experience
Internet users has increased from 360 Million in 2000 to 1.8 billion in 2009.
Around Twenty Five per cent of the world’s population now use the Internet. North America (74%) has the highest penetration.
Internet users has increased from 360 Million in 2000 to 1.8 billion in 2009.
Around Twenty Five per cent of the world’s population now use the Internet. North America (74%) has the highest penetration.
User Expectations Not Met at Peak Times Impact
the Business67% of online consumers expect consistently good experiences at peak times regardless of website traffic, but their expectations are not met
Poor web experiences at peak times impacts revenue, brand & loyalty•78% of consumers have gone to a competitor site due to poor
performance at peak times• After a poor experience
•42% have discussed it with family, friends, peers or online•47% left with a less positive perception of the company•88% of online consumers are less likely to return to a site
Total Sample (N=1538)
Many of the industry’s largest merchants suffered site slowdowns or other problems in 2009
Non Functional Requirement’s are not properly defined and tested
Web server/App Server/Middleware/Database server are not properly configured
Database is not properly tuned
Application is not designed to scale for huge volumes
Infrastructure is not properly sized
3rd party components or services components didn’t scale under load
Non Functional Requirement’s are not properly defined and tested
Web server/App Server/Middleware/Database server are not properly configured
Database is not properly tuned
Application is not designed to scale for huge volumes
Infrastructure is not properly sized
3rd party components or services components didn’t scale under load
•Make informed business decisions about capacity, third-party components & services, or future expansions plans•Verify response-time and end-user experience under load•Find problems and bottlenecks before real end-users are impacted
•Make informed business decisions about capacity, third-party components & services, or future expansions plans•Verify response-time and end-user experience under load•Find problems and bottlenecks before real end-users are impacted
•Mitigate risk when launching promotions, rolling-out new applications or features, upgrading or virtualizing infrastructure, or anticipating seasonal spikes in web traffic
•Mitigate risk when launching promotions, rolling-out new applications or features, upgrading or virtualizing infrastructure, or anticipating seasonal spikes in web traffic
Protect online revenue when it matters most, at peak times Ensure performance and availability of key transactions
across geographies, for expected user load and beyond
Protect online revenue when it matters most, at peak times Ensure performance and availability of key transactions
across geographies, for expected user load and beyond
Why Performance Testing is Critical?
Ensure success of critical Web-based business initiatives
Validate speed, scalability and stability
Deploy applications, new features & infrastructure with confidence
The Challenge of Delivering Web Applications
The Web Application Delivery Chain
MajorISP
Local ISP
Mobile Carrier
Internet
Content DeliveryNetworks
3rd Party/Cloud Services
Browsers and devices UsersUsers
Storage
Web Servers
App Servers
DB Servers
Mainframe
Load Balancers
Mobile Components
Network
Response time should be less than 2 seconds
Application should be available 24*7
Application should support X concurrent users
Infrastructure should be shared
CDNs, external 3rd party components & services outside the firewall should also meet SLAs
Response time should be less than 2 seconds
Application should be available 24*7
Application should support X concurrent users
Infrastructure should be shared
CDNs, external 3rd party components & services outside the firewall should also meet SLAs
Performance Testing – Maturity Levels
When to Performance Test in Development Lifecycle
Requirement Analysis
• Gather the Non Functional requirements – Involve all stakeholders• Convert the business requirements into measurable performance
Development
• Early Performance Testing/ Component Level Performance Testing• Application Profiling
Design
• Architectural Benchmarking• Software benchmarking• Hardware benchmarking
Integration & Testing
• Pre Production Performance Testing – Load Testing, Stress Testing, Spike Testing, Endurance Testing, Volume Testing, Real End User Testing
Implementation
•Application Performance Monitoring•Performance Benchmarking
Recommendations For Successful Performance Testing
Start early in the project lifecycle
Gather Application Usage Statistics - Workload Analysis, Business critical transactions, User locations , Resource utilization
Define Performance Testing objectives
Identify the right tool for Performance testing Tool
Inside the firewall – HP Loadrunner, Micro focus SilkPerformer, IBM Rational Performance Tester, Microsoft VSTS, Radview Webload, Neotys NeoLoad, OpenSTA, Jmeter
Outside the firewall – Gomez Reality Load
Have Production like environment
Scripting – Parameterization, Correlation, Error validation, Smoke Tests
Deploy Monitors to collect server side statistics
Execute Baseline tests, Plan for iterative Performance testing
Performance Testing & Problem Diagnosis
Design Phase : Architecture Performance Benchmarking
Excessive Threads Spawned
High Virtual Memory Usage
After implementing Thread Pooling & Fix Patches From Third-Party Components
Application prototype was becnhamarked on the planned production environment Early identification of design issues and uncovered major performance risk posed by the third-party products saving huge cost and effort
Company: Real Estate and Relocation Services
• Building a new online real-estate transaction management system
• Perform End to End performance Testing
Company: Real Estate and Relocation Services
• Building a new online real-estate transaction management system
• Perform End to End performance Testing
Performance Testing & Problem Diagnosis
Development Phase: Early Performance Testing
It is seen that the available memory goes down as the number of loaded assemblies increases
Number of loaded assemblies keeps increasing throughout the test, although the same transactions are repeatedly executed
Heap dump data showing many dynamically loaded assemblies in the heap
Memory dumps show that there are many dynamically generated XMLSerializer assemblies present in memoryCache the XMLSeraliazer objects after creation, so they are not created every time. This avoids multiple creation of assembliesWith the fix, the number of assemblies load stabilizes 5 minutes after the test, while without the fix, it grows continuously
Early identification of critical code-level performance issues resulted in significant cost and effort saving for all stakeholders
After fixing, the application code was stable and enabled increased scalability and better performance
Performance Testing & Problem Diagnosis
Integration & Testing Phase: Pre-Production Performance Testing
Sudden drop in throughput after 70 users indicating a bottleneck.
CPU Utilization also peaks indicating the server side bottleneck
Application Server configurations where optimized for optimal settings after iterative test execution cycle. This eliminated high CPU utilization of the systems
Reduced the CPU utilization of the application server from 95% to around 20% Availability of the system was improved from 87% to around 99%
Web applications increasingly assembled at browser level
3rd party performance from ads, videos, shopping cart vary widely under load
Content Delivery Network configuration and effectiveness impact end-user experience
Web Application Delivery In A Web 2.0 World
MajorISP
Local ISP
Mobile Carrier
Internet
Content DeliveryNetworks
3rd Party/Cloud Services
Browsers and devices UsersUsers
Storage
Web Servers
App Servers
DB Servers
Mainframe
Load Balancers
Mobile Components
Network
The Web Application Delivery Chain
• Network peering problems
• Outages
• Inconsistent geo performance• Bad performance under load• Blocking content delivery• Incorrect geo-targeted content
• Configuration issues• Oversubscribed POP• Poor routing optimization• Low cache hit rate
• Network peering problems
• Bandwidth throttling
• Inconsistent connectivity
• Configuration errors
• Application design issues
• Code defects• Insufficient
infrastructure
• Poorly performing JavaScript
• Browser/device incompatibility
• Page size too big
• Too many objects
• Low cache hit rate
• Network resource shortage
• Faulty content transcoding
• SMS routing / latency issues
The Challenge of Ensuring Quality Web Experiences
Zone of customer expectationZone of customer expectation
Systems management
tools: “OK”…user is NOT happy
Zone of customer expectationZone of customer expectationTraditional zone of controlTraditional zone
of control
MajorISP
Local ISP
Mobile Carrier
Internet
Content DeliveryNetworks
3rd Party/Cloud Services
Browsers and devices Users
Storage
Web Servers
App Servers
DB Servers
Mainframe
Load Balancers
Mobile Components
Network
Gomez Load Testing 2.0 : On-Demand Realistic Load Testing
High volume load (HTTP, Browser)
Find infrastructure breaking pointsDefine capacity headroom
Real-world load
Find user experience breaking pointsAccurately measure response time
Gomez Network: The World’s Most Comprehensive Performance and Testing Network
Web Performance Management and Load Testing
100+ locations
Backbone
Cross-Browser Testing 500+ browser/OS combo’s5,000+ supported devices
Virtual Test Bed
Web Performance Management and Load Testing 100,000+ locations
Gomez Last Mile
Company: Online Retailer
• Several 3rd Parties now involved in serving up key content
• Goal was to validate performance of entire application
Company: Online Retailer
• Several 3rd Parties now involved in serving up key content
• Goal was to validate performance of entire application
Load Testing 2.0 Detects 3rd-party External Errors
2
MajorISP
Local ISP
Mobile Carrier
Internet
Content DeliveryNetworks
3rd Party/Cloud Services
Browsers and devices Users
Storage
Web Servers
App Servers
DB Servers
Mainframe
Load Balancers
Mobile Components
Network
The load increases throughout the test
The transaction rate increases and then falls off as response times climb
Errors are seen, all on a 3rd party object
Load Testing 2.0 Detects 3rd-party External Errors cont
• 3rd party hardware was insufficient for overall demands on application
• Based on SLAs 3rd party had to improve performance to get paid
• 3rd party hardware was insufficient for overall demands on application
• Based on SLAs 3rd party had to improve performance to get paid
Company: Regional Online News Source
• Began testing for the 2008 election season
• Goal was to validate overall performance focusing in 2 key regions
Company: Regional Online News Source
• Began testing for the 2008 election season
• Goal was to validate overall performance focusing in 2 key regions
Load Testing 2.0 Uncovers Regional Response Time Discrepancies under Load - Case Study
MajorISP
Local ISP
Mobile Carrier
Internet
Content DeliveryNetworks
3rd Party/Cloud Services
Browsers and devices Users
Storage
Web Servers
App Servers
DB Servers
Mainframe
Load Balancers
Mobile Components
Network
3
By traditional Load Test 1.0 standards the test passed
Page response times stayed under 4 seconds, outside of one brief blip
There was only 1 page error and 11 errors total out of 60000+ transactions
Increase and hold load and not exceed response times of 4 seconds and Success Rate of 99%
No Performance Issues Detected From Datacenter
Last Mile data showing substantial number of measurements greater than 4 seconds
Performance Issues Detected When Using Real User Desktops
Key geographies for this customer are New York and Pennsylvania
The response time never met the 4 second average goal.
Availability was Less than 99%.
Top Revenue-generating geographies wereNew York and Pennsylvania.
By Load Testing 2.0 standards the test failed.
Load Testing 2.0 Uncovers Poor Response Time for Top Revenue-Generating Regions
Load Testing 2.0 versus Load Testing 1.0
Capability Load Test 1.0•HTTP traffic behind the firewall
Load Test 2.0•Real-Desktops Last Mile•High Volume Datacenters
Accuracy of End-User Response Time
Incomplete Most Accurate
Ability to drive large load volume
Limited- requires too much hardware
Better - SaaS model
Understand CDN Impact No Yes
Understand 3rd Party (ads, feeds, shopping carts, analytics, cloud providers..)
No Yes
Real-world load to uncover geo discrepancies
No Yes
Visibility behind the firewall Better Good
Pre-production Testing Better Used closer to deployment --to test under realistic conditions
Client-Server Testing Yes No
Find Out How Gomez + Cognizant Can Help You
Sign Up Now For Your Free Load Testhttp://www.gomez.com/gomez-load-testing-trial/
Reality Load XF
• Test from browser to datacenter, with drill down to internal & external application components
• SaaS with no investment or maintenance costs and rapid payback
• Full browser testing across globally distributed geographies
Reality Load XF
• Test from browser to datacenter, with drill down to internal & external application components
• SaaS with no investment or maintenance costs and rapid payback
• Full browser testing across globally distributed geographies
Cognizant
• Testing Services to complement your Reality Load investment.• Business and technical side expertise to provide accelerated
results.• Over 500+ dedicated Performance Testing Professionals to
deliver services you need from across the globe.
Cognizant
• Testing Services to complement your Reality Load investment.• Business and technical side expertise to provide accelerated
results.• Over 500+ dedicated Performance Testing Professionals to
deliver services you need from across the globe.