real user measurement: the secret weapon for quality
TRANSCRIPT
1© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Test Faster.
Release Sooner.
Measure What Matters.
2© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Real User Measurement: The Secret Weapon for Quality
o Introductions
o Audience Poll
o The SOASTA Platform
o Introduction to Real User Measurement
o Using mPulse and CloudTest together
o Customer case study
o mPulse Demo
Agenda
3© 2014 SOASTA CONFIDENTIAL - All rights reserved.
IntroductionsWho we are…
o Buddy Brewer
o VP mPulse Business Development
o Cliff Crocker
o VP mPulse Product Management
5© 2014 SOASTA CONFIDENTIAL - All rights reserved.
SOASTA Core DNA
Contextual & Actionable
Intelligence
SOASTA Global Test Platform
SOASTA Analytic Engine
In-Memory Performance Analytics(Patent Filed)
SOASTA Global Test CloudLargest Test Automation Platform in The world
SOASTA Command & Control
Test & Measurement Automation Grid(Patent)
mPulseReal User
Measurement
Production Dev/Test Feedback Loop
CloudTestPerformance
(Patent)
Pre-ProductionTest Build & Editing
TouchTest
Functionality(Patent Filed)
6© 2014 SOASTA CONFIDENTIAL - All rights reserved.
The New Load Testing Standards
o Frequent testing from development through production
o Realism and accuracy from the user’s view to the last server
o Real time analytics and integrated monitoring
o End to end optimization from app to infrastructure
o Test cycle acceleration and optimization
Better tests, faster, more often
7© 2014 SOASTA CONFIDENTIAL - All rights reserved.
mPulse
8© 2014 SOASTA CONFIDENTIAL - All rights reserved.
What is RUM?Real User Measurement (traditional definition)
RUM 101Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user.
• Involves instrumentation of the page with JavaScript
• Measurements are fired across the network to a collection point through a small request object (beacon)
<JS>
MySite.com
<beacon>
Before
During After
Increase test accuracy and relevance
Gain operational visibility, feedback
Decrease production risk
When is mPulse Useful?Applications before, during, and after testing
10© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Before: Improve test accuracyMatch test characteristics to real user behavior
11© 2014 SOASTA CONFIDENTIAL - All rights reserved.
During: Watch Customers While TestingRemove risk from testing in production
Safely test production
Watch performance
Watch key metrics
12© 2014 SOASTA CONFIDENTIAL - All rights reserved.
After: Find Real-world Performance IssuesFind areas of focus for next round of development/testing
+ = 2 seconds
= 8 seconds
+
Home +
Product +
= 5 seconds
= 2.5 seconds
13© 2014 SOASTA CONFIDENTIAL - All rights reserved.
After: Set Meaningful Performance GoalsDerive web performance goals from business targets
3.5 sec
Instrument what you want to manage:
Business Target: 30% Bounce RatePerformance Goal: 3.5 Second Load Time
• Conversion Rate• Items Sold• Comments• Signups
14© 2014 SOASTA CONFIDENTIAL - All rights reserved.
2014 Winter OlympicsSucceeding with testing and real user measurement
15© 2014 SOASTA CONFIDENTIAL - All rights reserved.
2014 Winter Olympics
o World wide traffic
o Regional browsers
o Event-driven traffic
Managing challenges before and after launch
o Feb 7-23, 2014
o 414 Million Page Views
o 124.3 Million Unique Visits
16© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Impact of Speed on Visitor Loss
11% @ 1.35 sec
33% @ 5.4 sec
Thank You
Contact SOASTA:www.soasta.com/cloudtest/
Follow us:
twitter.com/cloudtest
facebook.com/cloudtest
RESOURCES
Case studies: soasta.com/customers
Upcoming webinars: soasta.com/knowledge-center- “Getting Started with mPulse” – April 4th 10am- “Agile Load Testing in the Real World” – March
Need a test, fast? CloudTest On-Demand.- email [email protected]
www.SOASTA.com