![Page 1: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/1.jpg)
Cliff Crocker @cliffcrocker SOASTA VP Product, mPulse
![Page 2: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/2.jpg)
1. How fast am I? 2. How fast should I be?
3. How do I get there?
![Page 3: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/3.jpg)
Synthe'c vs. RUM
![Page 4: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/4.jpg)
Synthetic 101 Synthetic monitoring (for purposes of this discussion) refers to the use of automated agents (bots) to measure your website from different physical locations. • A set ‘path’ or URL is defined • Tests are run either adhoc or scheduled and data is collected
![Page 5: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/5.jpg)
RUM 101 Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user.
• Involves instrumentation of your website via JavaScript
• Measurements are fired across the network to a collection point through a small request object (beacon)
<JS> <beacon>
![Page 6: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/6.jpg)
RUM Cast a wide net • Identify key areas of concern • Understand real user impact • Map performance to human behavior & $$
Synthetic Diagnostic tool • Identify issues in a ‘lab’/remove variables • Reproduce a problem found with RUM
h0p://www.flickr.com/photos/84338444@N00/with/3780079044/
h0p://www.flickr.com/photos/ezioman/
![Page 7: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/7.jpg)
The Early Days of RUM
• Round-trip time • Start/stop timers via JavaScript • Early contributors:
• Steve Souders/Episodes • Philip Tellis/Boomerang.js • Both widely in use today
![Page 8: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/8.jpg)
Navigation Timing
![Page 9: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/9.jpg)
Browser Support for Navigation Timing
!
![Page 10: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/10.jpg)
10 © 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation Timing
DNS: Domain Lookup Time
function getPerfStats() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; }
![Page 11: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/11.jpg)
11 © 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation Timing
TCP: Connection Time to Server
function getPerfStats() { var timing = window.performance.timing; return { connect: timing.connectEnd - timing.connectStart}; }
![Page 12: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/12.jpg)
12 © 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation Timing
TTFB: Time to First Byte
function getPerfStats() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; }
![Page 13: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/13.jpg)
13 © 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation Timing
Base Page
function getPerfStats() { var timing = window.performance.timing; return { basePage: timing.responseEnd - timing.responseStart}; }
![Page 14: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/14.jpg)
14 © 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation Timing
Front-end Time
function getPerfStats() { var timing = window.performance.timing; return { frontEnd: timing.loadEventStart - timing.responseEnd}; }
![Page 15: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/15.jpg)
15 © 2014 SOASTA CONFIDENTIAL - All rights reserved.
Navigation Timing
Page Load Time
function getPerfStats() { var timing = window.performance.timing; return { load: timing.loadEventStart - timing.navigationStart}; }
![Page 16: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/16.jpg)
Measuring Assets
• Strength of synthetic • Full visibility into asset performance
• Images • JavaScript • CSS • HTML
• A lot of which is served by third-parties • CDN!
![Page 17: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/17.jpg)
Object Level RUM
![Page 18: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/18.jpg)
Browser Support for Resource Timing
![Page 19: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/19.jpg)
CORS: Cross-Origin Resource Sharing
Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"
Start/End time only unless Timing-Allow-Origin HTTP Request Header defined
![Page 20: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/20.jpg)
Resource Timing
var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’; var me = performance.getEntriesByName(url)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart }
![Page 21: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/21.jpg)
Resource Timing
• Slowest resources • Time to first image • Response time by domain • Time a group of assets • Response time by initiator type (element type) • Cache-hit ratio for resources
For examples see: http://www.slideshare.net/bluesmoon/beyond-page-level-metrics
![Page 22: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/22.jpg)
Resource Timing
• PerfMap - https://github.com/zeman/perfmap • Mark Zeman
• Waterfall.js - https://github.com/andydavies/waterfall • Andy Davies
![Page 23: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/23.jpg)
1. How fast am I? 2. How fast should I be?
3. How do I get there?
![Page 24: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/24.jpg)
Picking a Number
• Industry benchmarks? • Apdex? • Analyst report? • Case studies?
![Page 25: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/25.jpg)
“Synthetic monitoring shows you how you relate to your competitors, RUM shows you how you relate to your customers.”
– Buddy Brewer
![Page 26: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/26.jpg)
Benchmarking
![Page 27: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/27.jpg)
Benchmarking
• http://soasta.com/mpulseUX
![Page 28: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/28.jpg)
Benchmarking
• Page construction • Requests • Images • Size
• Other important metrics • Speedindex • Start Render • PageSpeed/Yslow scoring
![Page 29: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/29.jpg)
Performance is a Business Problem"
"
![Page 30: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/30.jpg)
Yahoo! - 2008
Increase of 400ms causes 5-9% increase in user abandonment
http://www.slideshare.net/stubbornella/designing-fast-websites-presentation
![Page 31: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/31.jpg)
Shopzilla - 2009
A reduction in Page Load time of 5s increased site conversion 7-12%!
http://assets.en.oreilly.com/1/event/29/Shopzilla%27s%20Site%20Redo%20-%20You%20Get%20What%20You%20Measure%20Presentation.ppt
![Page 32: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/32.jpg)
Walmart - 2012
http://minus.com/msM8y8nyh#1e
SF WebPerf – 2012 Up to 2% conversion drop for every additional second
![Page 33: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/33.jpg)
So What?"""
![Page 34: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/34.jpg)
SIMULATION
![Page 35: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/35.jpg)
![Page 36: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/36.jpg)
![Page 37: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/37.jpg)
How Fast Should You Be?
• Use synthetic measurement for benchmarking your competitors
• Understand how fast your site needs to be to reach business goals/objectives with RUM
• You must look at your own data
![Page 38: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/38.jpg)
1. How fast am I? 2. How fast should I be?
3. How do I get there?
![Page 39: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/39.jpg)
Real users are not normal
![Page 40: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/40.jpg)
![Page 41: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/41.jpg)
Page Load Times
2.76s – Median
10.45s – p95
17.26s – p98
![Page 42: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/42.jpg)
Platform Median 95th Percentile
98th Percentile
Mobile 3.62s 12.53s 20.04s
Desktop 2.44s 9.31s 15.86s
Page Load Times
![Page 43: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/43.jpg)
Page Load Times
OS Median 95th Percentile
98th Percentile
Windows 7 2.41s 9.29s 15.89s
Mac OS X/10 2.30s 8.11s 13.45s
iOS7 3.27s 10.64s 15.79s
Android 4 4.06s 14.30s 27.93s
iOS8 3.53s 11.54s 19.72s
Windows 8 2.67s 10.75s 18,74s
![Page 44: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/44.jpg)
Other Factors
• Geography • User Agent • Connection Type • Carrier/ISP • Device Type
![Page 45: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/45.jpg)
Not All Pages are Created Equal
For a typical eCommerce site, conversion rate
drops by up to 50% when “browse” pages increase
from 1 to 6 seconds
![Page 46: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/46.jpg)
Not All Pages are Created Equal
However, there is much less impact
to conversion when “checkout” pages degrade
![Page 47: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/47.jpg)
How Do I Get There?
• Focus on the highest value opportunities/demographics • Identify key pages that have the most impact on your
KPIs • Prioritize based on reducing friction within the funnel or
critical path
![Page 48: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/48.jpg)
Thank You!"""
![Page 49: Edge 2014: A Modern Approach to Performance Monitoring](https://reader033.vdocuments.mx/reader033/viewer/2022051514/5480b3555806b5fc108b4583/html5/thumbnails/49.jpg)
Images
1. Modern (Title Slide): https://www.flickr.com/photos/looking4poetry/3477854720/in/photolist-6ijV3q-9t3sjb-cj8VWE-MNjPA-4yBqug-2rk5he-4nZVzJ-4yE45J-9t3Fvj-2rptr5-69ymdr-6XVTH-6QZ4dg-9t3HN5-8geUkB-6oEXs5-eH9zS-H2XTt-cKdJvb-7GNKWx-cKdKZd-5Rw3qt-jfT5Dx-Jfiuy-bTop6R-2m8kAB-Jfiuu-7E8eMf-9jqD6-9hYAvd-Jfius-gsqr7U-7KiNAZ-8kzG9V-euXdb-44qJNN-47jBuY-nsxZrZ-7cGjQL-4cfHKq-cGzbSC-aQtWPR-8y3sR3-6okjAW-5A7BSC-6aRsvh-eSMHkX-kxvdyK-9t3udu-c8EkYu
2. Man vs. Machine: https://www.flickr.com/photos/eogez/3289851965/in/photolist-61HmnB-9nkQu3-dfinF1-4GjK24-4GjL6Z-4GoV6C-dibVCU-4GoTXo-4GjF3k-4GjH7Z-4GoR6h-ek6eT9-ek6eUU-4fADUx-4fEDV9-65JA6W-65Jsxu-65JDnj-4fAE12-65JDTh-65JBZu-65Ek9M-65JtDu-65JuiN-65JzFS-65EfiH-65Jv63-65JxX7-65Enyc-65Eh4c-65Jxjm-65JwmU-65EhUP-65JCML-65EjxP-65JBQf-65Ee4Z-65JD4b-65JBF1-65Ec1r-65EkPB-65Jy4o-65Eg96-65JAjb-65Emfe-65Efoz-65JBam-65Eniv-65JCrU-65Eed2
3. https://www.flickr.com/photos/bradybd/2818154005/
4. https://www.flickr.com/photos/perspective/149321089/
5. Dartboard: http://en.wikipedia.org/wiki/Darts#mediaviewer/File:Darts_in_a_dartboard.jpg
References:
1. http://calendar.perfplanet.com/2011/a-practical-guide-to-the-navigation-timing-api/
2. http://www.slideshare.net/bluesmoon/beyond-page-level-metrics