godaddy at velocity 2016 new york
TRANSCRIPT
Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
How GoDaddy radically improved web performance and increased customer engagementJim Pierson
Velocity NY - Sept 22, 2016
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.2Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
The Money slide• On June 4th, 2015, GoDaddy deployed its
fastest India home page.
• The next day, user purchases shot up several percentage points. Soon after, we rolled out the solution to all of our sales sites.
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.3Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Agenda• How GoDaddy matured as a Performance organization
• The Tools and Best Practices that worked
• How Big Data collection and Dashboards changed the game
• Anomaly detection in Production and Testing, it’s a math problem.
• The need to Join forces
Performance Engineering Maturity Level 1 - Novice
Novice
Proactive
Reactive
• Initial :• Firefighting, chasing• Blind, apathy, lacking TRUST• Hands-On learning of the tools
• Eventual:• Investigative approach
• Trench coat vs Lab coat• Look at every angle
• Tool Proficiency• Patterns evolve into Best Practices
• Exit:• Measure Everything -> RUM, APM
• Relapse:- too many new toys
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.5Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Top Fixes – Deferring JS• Defer JavaScript:
• UXCORE
• Tag Mangers
• LIVE CHAT
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.6Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Top Fixes – Deferring JS
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.7Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Top Fixes – File size optimization• File size reduction:
• Combine JS/CSS
• Minify JS/CSS
• Compress everything!
• Review/optimize:• Images
• Fonts
• CSS
• JS
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.8Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Top Fixes – Head Flushing• HEAD flushing
(chunking)• Fetch CSS
immediately!
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.9Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
UglyNot so good PLT• Onload: 5s in Phx, 13s in HK
• VisComplete: 4.8s in Phx, 6.3s in HK
• 13s in Hong Kong
• 133 items
• Nothing deferred
• No CDN Domain
http://wpt.intranet.gdg/result/160712_W4_256/1/details/
....
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.10Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Nice!A good site PLT• Onload: 1.6 seconds in Phx, 1.7 in HK
• VisComplete: 2.5s in Phx, 3.8s in HK
• 14 items b4 onload
• Faster Server Response Time – Head Flushing
• JS Deferred
• Tags Deferred
• Initial images limited
• Minimized css, Fonts
• Oops, one css after Onload, reflow
http://wpt.intranet.gdg/result/160712_4Q_255/1/details/ (Phx)http://www.webpagetest.org/result/160712_AT_18DC/1/details/ (HK)
Goal oriented
Novice
Performance Engineering Maturity Level 2 – Goal oriented
Proactive
Reactive
• Initial :• Aspirational goals• Limited storage, visualization• Data not JOIN’d and not defined
• Eventual:• Storing everything• Everyone is using the dashboard
• Exit:• Regular Reviews and faster troubleshooting
• Relapse:• Too lenient on ship goals• Not watching the changes
Measure Everything
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.12Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Real User Monitoring• Browser based
• Hadoop, Elastic
Search, Kibana (ELK)
• Test in production
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.13Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
RUM Scorecard for Execs
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.14Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
CDN Measurement with RUM – Resource TimingCedexis uses a RUM tag added to our page to measure CDN perf
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.15Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Application Performance MonitoringNew Relic APM agentFaces 2 directions: (transactions, and external transactions)
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.16Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Using APM to make comparisons
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.17Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Keep-Alives We are reopening TCP on every transaction
8 RTs x 60ms = 480ms
Goal oriented
Novice
Performance Engineering Maturity Level 3 – Under Control
Proactive
Reactive
• Initial :• 3 steps forward, 1 step back• Cant get at the data fast enough (hours)
• Eventual:• Dynamic threshold math• Inline anomaly detection for production• Catching bad perf before production
• Exit:• Cross-Org communications
• Relapse:- Alerting complexities- Improper hand-off of Alerts
Measure Everything
AutomateEverything
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.19Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Anomaly detection - Change Index
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.20Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Change Index - Rank
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.21Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Real User Monitoring - Alerting
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.22Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Perf Tests that can stop the Build• Assert on perf
• Using RUM baseline
• View results in ELK
dashboard
Goal oriented
Novice
Performance Engineering Maturity Level 4 – Best of Breed
Proactive
Reactive
• Initial :• Everyone is involved, no one owns it• Super complex• Putting off for years major architectural issues
• Eventual:• Everyone bought into the big event • and in lock-step
• Exit:• 3-5 year arc per product group
• Relapse:- New products
Measure Everything
AutomateEverything
Communicate With Everyone
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.24Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Team work and influences
platform
Design/PO
editing
Mid tier
Backend
Legacy
CDN
• Legacy owner can make simple but quick changes
• Platform needs lots of coordination
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.25Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Pealing back the Onion• Many teams• Many priorities• No clear owner
Cart
Front End
Ads
Auth
Backend
API
CDN
Performance Engineering Maturity Levels - Recap
L4) Joining Forces, Driving Architecture refactoring
Goal oriented
Novice
Measure Everything
CommunicateWith Everyone
Proactive
Reactive
Automate Everything
L2) Low hanging fruit, Best Practices,setting goals, basic monitoring
L1) Firefighting, following the trail, discovery
L3) Anomaly detection - Operational AlarmsControlling regression with CICD
CONFIDENTIAL. COPYRIGHT © 2016 GODADDY INC. ALL RIGHTS RESERVED.27Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Q & A