meetup performance
DESCRIPTION
TRANSCRIPT
![Page 1: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/1.jpg)
Meetup PerformanceGreg Whalin, CTO Meetup (@gwhalin), Justin Cataldo, Lead UI Engineer (@jcataldo), Will Howard, Lead UI Engineer
![Page 2: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/2.jpg)
Meetup
Platform for local groups
Mission is MEME (Meetup Everywhere About Most Everything)
~6.2m members~70k Groups~500k group joins every month~5 million Meetups have happened~53 million RSVPs
![Page 3: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/3.jpg)
General Architecture and Back-end Performance
(just a tiny bit - this could and should be another presentation)
![Page 4: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/4.jpg)
DataMySQL (any RDBMs store) biggest pain
replication use InnoDB!smart indexing (take advantage of clustered indexes)server side conn pools and short timeouts on connectionssmart about fs choice on Linux (we use XFS after benchmarking)
Hardware - relatively expensive boxes
multi-core (8 or 16) Opteronlots of ram (32/64GB)fast drives (lots of spindles, RAID10)
Cache, cache, cache!
innodb buffer cachememcachelocal app server memory
Shrink data when possiblearchive unused datacustom serialization when serializingdata partitioning/sharding
![Page 5: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/5.jpg)
Storage
Over half a million photos uploaded to Meetup every month
Scaled and processed into 4 different sizes (plus original)
![Page 6: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/6.jpg)
Storage solutions
Options for growth include NAS, SAN, or something else
NAS and SAN are single point of failure and possibly $$$
Only postpones problem
![Page 7: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/7.jpg)
MogileFSdeveloped by Brand Fitzpatrick (i.e. Memcached) OSS distributed filesystem (built in Perl)any hard drive on network can easily be added to clusterscales easily and cheaply
![Page 8: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/8.jpg)
Much much much more going on
but...
![Page 9: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/9.jpg)
UI Performance(much of our focus here)
![Page 10: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/10.jpg)
Why does performance matter?
![Page 11: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/11.jpg)
Why does performance matter?
Slow site
![Page 12: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/12.jpg)
Why does performance matter?
Slow site
Bad User Experience
![Page 13: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/13.jpg)
Why does performance matter?
Slow site
Bad User Experience
Drop in Member Activity
![Page 14: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/14.jpg)
Why focus on front end performance?
Back end only accounts for 10-15% of the response timeLess time and resourcesCosts less
http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html
![Page 15: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/15.jpg)
Case Study: Event Details
![Page 16: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/16.jpg)
Load time = 6.321s
www.webpagetest.org
Event Details: Load Time
![Page 17: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/17.jpg)
Lots of javascript being loaded
Event Details: Requests
![Page 18: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/18.jpg)
How do we improve performance?
![Page 19: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/19.jpg)
3 Steps to improving performance
1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
![Page 20: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/20.jpg)
3 Steps to improving performance
1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
![Page 21: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/21.jpg)
Why externalize scripts
Prevents blockingInline scripts prevent asynchronous downloadsDownloads must wait for the script to be executed
CachingInline JavaScript is downloaded every timeExternal scripts are cached by the browserReduced overall page size
ReusableCan use the same code somewhere else on the site easily
![Page 22: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/22.jpg)
Pull out inline script
<script type="text/javascript">if(typeof Meetup.EventDetails == 'undefined') Meetup.EventDetails = {}; (function(){ var self = Meetup.EventDetails;...})();</script>
> EventDetails.js
![Page 23: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/23.jpg)
3 Steps to improving performance
1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
![Page 24: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/24.jpg)
Web:scriptCustom tag built in houseMoves inline and external script to the bottom of the pageAllows UI engineers to not have to worry about where they place scriptsCompresses inline script using YUICompressor
/***** Load External Script *****/ <web:script src="/script/Meetup/packed/EventDetails.js" />
/***** Load Inline Script *****/ <web:script> Meetup.Copy.noMembersMarkedAttended = "<trn:message key="event.attendance.noMembersMarkedAttended">No members have been marked attended</trn:message>"; Meetup.Copy.noMembersMarkedAttendedDynam = '<trn:message key="event.attendance.noMembersMarkedAttendedDynam"><trn:param name="GROUPING">__GROUPING__</trn:param>No members in "{GROUPING}" have been marked attended</trn:message>'; Meetup.Copy.noMembersMarkedAbsent = "<trn:message key="event.attendance.noMembersMarkedAbsent">No members have been marked absent</trn:message>";</web:script>
![Page 25: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/25.jpg)
3 Steps to improving performance
1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
![Page 26: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/26.jpg)
Reduce Requests
Concatenate as much as possibleLoad only what we need upfront
![Page 27: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/27.jpg)
Concatenation using Sprockets
Sprockets (www.getsprockets.com)Created by 37SignalsRuby library that preprocesses and concatenates JavaScript filesBaked into our build process
![Page 28: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/28.jpg)
Concatenation using Sprockets (cont.)
EventDetails.jsCommentDeleteConfirm.jsBubbleTips.jsPlacesManager.jsMaxCharactersEnforcer.js > EventDetails.js
![Page 29: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/29.jpg)
Using Sprockets/******* Sprockets Directives *******/ //= require <CommentDeleteConfirm>//= require <DomDeco/BubbleTips>//= require <DomDeco/PlacesManager>//= require <DomDeco/MaxCharactersEnforcer> /******* Begin Event Details Code *******/ if(typeof(Meetup.EventDetails) == 'undefined') Meetup.EventDetails = {}; (function(){ var self = Meetup.EventDetails;...........})();
/******* Build Process *******/ <exec executable="sprocketize" failonerror="true" output="${image.dir}/script/Meetup/packed/EventDetails.js"> <arg value="-I"/> <arg path="${image.dir}/script/Meetup/"/> <arg path="${image.dir}/script/Meetup/EventDetails.js"/></exec>
![Page 30: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/30.jpg)
Lazy Loading
Defer loading of javascript files until they are neededReduces the initial upfront requestsHelps reduce blocking by downloading files asynchronouslyPrecaching
![Page 31: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/31.jpg)
Lazy Loading: How it works
Inserts scripts into the head dynamically
Meetup.Script.include("http://static2.meetupstatic.com/script/Meetup/DomDeco/LinkDecorator.js",callback);
var scriptNode = function(src) { return createDOM("script", { "type": "text/javascript", "src": src }); }
var load = function(id, src, n) { var script = scriptNode(url); head.appendChild(script); script.onload = script.onreadystatechange = function() { if (!this.readyState || this.readyState == "loaded" || this.readyState == "complete") { script.onload = script.onreadystatechange = null; } } }
![Page 32: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/32.jpg)
Did it make a difference?
![Page 33: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/33.jpg)
Javascript requests cut by 50%
Reduced requests
![Page 34: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/34.jpg)
Event Details Page Load After Old Load Time = 6.321s New Load time = 4.643s
![Page 35: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/35.jpg)
Event Details Page Load After Old Load Time = 6.321s New Load time = 4.643s
Load time decreased by
27%!
![Page 36: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/36.jpg)
But that's not all
![Page 37: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/37.jpg)
Execute when the dom is ready
Execute early
![Page 38: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/38.jpg)
Execute early: DOMReadyLibraries that have a DOM ready solution:
jQueryYUIPrototypePretty much every modern JS library (not MochiKit)
Meetup uses MochiKit, so we rolled our own.
![Page 39: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/39.jpg)
Execute early: DOMReady
Meetup.DOMReady.ready(function(){Meetup.EventDetails.init();if(Meetup.EventDetails.isCanceled != 4 && Meetup.EventDetails.rsvp != 0){deletePopup = new Meetup.CommentDeleteConfirm();deletePopup.pagerOffsetFieldName = "p_commentsList";deletePopup._decorate();}});
And by rolled our own, I mean we're using the Dean Edwards/Matthias Miller/John Resig implementation. http://dean.edwards.name/weblog/2006/06/again/#comment5338
With a few changes.
![Page 40: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/40.jpg)
Execute early: Even earlier
Do you need to wait for the DOM to be ready?
If you aren't manipulating the DOM, there's no reason to wait until it's ready.
![Page 41: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/41.jpg)
Automated Image Optimization
./filescan.sh /usr/local/meetup/static/img/ 'smusher -q @' 'jpg,png'&
Using smush.ithttp://developer.yahoo.com/yslow/smushit/
Smusher Ruby gemhttp://github.com/grosser/smusher (gem install smusher)
BASH script that watches our image directories for changes and executes smusher.
![Page 42: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/42.jpg)
Event Delegation
Run less JavaScript up front
![Page 43: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/43.jpg)
Event Delegation
From: http://www.quirksmode.org/js/events_order.html
But first, a little bit about event bubbling...
![Page 44: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/44.jpg)
Event DelegationPros
Much faster on load (not connecting DOM elements)
No need to disconnect / reconnect with AJAX calls
Fewer memory leaks
Cons�
�Does not work well with nested elements
Doesn't work with all events
Slight performance hit with execution
A lot of JS libraries already have plug-ins for event delegation (jQuery, YUI, prototype).
But, it's pretty easy to write your own (we did).
![Page 45: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/45.jpg)
Event Delegation: Meetup.Dispatcher<div id="C_page"> ...
<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>
var mdp = Meetup.Dispatcher.init("C_page", "onmouseover");
mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver);
Meetup.UI.infoHover.mouseOver = function(e) {topicId = _getTopicId(e.target());if (!topicId || topicId == "") return;_primeCache(topicId);var activeEl = _getActiveEl(e.target());var pos = getElementPosition(activeEl);
...}
<div id="C_page"> ...
<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>
![Page 46: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/46.jpg)
Event Delegation: Meetup.Dispatcher
![Page 47: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/47.jpg)
Event Delegation: Meetup.Dispatcher<div id="C_page"> ...
<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>
// Inits a new instance of dispatcher// Connects a mouseover event to the parent container "C_page"var mdp = Meetup.Dispatcher.init("C_page", "onmouseover");
// Calls Meetup.UI.infoHover.mouseOver() when target element has "topic-info-hover" class.mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver);
Meetup.UI.infoHover.mouseOver = function(e) {topicId = _getTopicId(e.target());if (!topicId || topicId == "") return;_primeCache(topicId);var activeEl = _getActiveEl(e.target());var pos = getElementPosition(activeEl);
...}
<div id="C_page"> ...
<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>
![Page 48: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/48.jpg)
Speeding up DOM crawling with Sizzle
Internet Explorer 7
MochiKit: 6623.94ms
Sizzle: 306.03ms
Firefox 3.5MochiKit: 210.524ms
Sizzle: 111.553ms
sizzlejs.com
![Page 49: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/49.jpg)
Where do we go from here?
More concatenation and lazy loading where it makes sense Defer image loading where it makes senseReduce DOM elementsReduce CSS and improved selector efficiencyand more
![Page 50: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/50.jpg)
Deployment and ServingAs it pertains to css/html/js
![Page 51: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/51.jpg)
Launches
Launch multiple times a day (sometimes)
Need launches to be quick / no downtime
Optimize static resources only at deploy time and only if modified
![Page 52: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/52.jpg)
Deployment of static content
Sprockets (reduce requests)
YUICompressor for js (local mod to speed up optimizing multiple files) Pre-compress css and jspSet cache-control to be fresh for over a year (indefinite)
All links on site generate programatically and versioned
![Page 53: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/53.jpg)
Link generation
Custom JSTL page functionhttp vs https domain sharding and cookie free domaincontent versioning
<link rel="stylesheet" href="${mfn:staticUrl( "/style/meetup.css", state.context.isSecure )}" type="text/css" /> <img src="${mfn:imgUrl( '/img/noPhoto_50.gif', state.context.isSecure )}" alt=""class="noPhoto" />
<link rel="stylesheet" href="http://static1.meetupstatic.com/050991196173395491322880/style/meetup.css" type="text/css" /> <img src="http://img1.meetupstatic.com/39194172310009655/img/noPhoto_50.gif" alt="" class="noPhoto"/>
![Page 54: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/54.jpg)
Versioning static content
MD5 checksum of contents of fileRun w/ each launchStore versions in db tied to release
mysql> select * from resource_version where filename = '/style/base.css'; | 394328 | /style/base.css | 39020083689267241 | 567 | | 398052 | /style/base.css | 8487620432388779772669 | 568 | | 401776 | /style/base.css | 357470606563045379 | 569 | | 405506 | /style/base.css | 3068234199748867 | 571 | | 409240 | /style/base.css | 024745310801291061590 | 572 | | 412974 | /style/base.css | 024745310801291061590 | 573 | | 416708 | /style/base.css | 09972542737049101325 | 574 |
![Page 55: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/55.jpg)
Static content serving Served off CDN (reverse proxy)Anycast DNS for hostname resolution Origin servers running lighttpdStrip versioning from url using rewrite rulesCache set to 12 months outCompress
expire.url = ( "/" => "access plus 12 months") compress.cache-dir = "/var/cache/lighttpd/"compress.filetype = ("text/plain", "text/html", "text/css", "application/x-javascript", "text/javascript") url.rewrite-once = ( "^/script/(.*/)?[0-9]+/(.+).js$" => "/script/$1$2.js", "^/style/(.*/)?[0-9]+/(.+).css$" => "/style/$1$2.css", "^/img/(.*/)?[0-9]+/(.+).(gif|png|jpg)$" => "/img/$1$2.$3", "^/\d+/script/(.+).js" => "/script/$1.js", "^/\d+/style/(.+).css" => "/style/$1.css", "^/\d+/img/(.+).(gif|jpg|png)" => "/img/$1.$2", "^/photos/(([^\/]+)/.+/(.+)\.jpeg)(?:\?.*)?$" => "/cgi-bin/photos.fcgi?type=$2&key=$3", "^/photos/([^\/]+)/([^\/]+)\.jpeg(?:\?.*)?$" => "/cgi-bin/photos.fcgi?type=$1&key=$2", "^/file.*" => "/")
![Page 56: Meetup Performance](https://reader034.vdocuments.mx/reader034/viewer/2022051514/549a3c13b479596a4d8b5887/html5/thumbnails/56.jpg)
Questions?Also, we need help! Hiring for:
Linux Systems AdministratorSoftware EngineersUI EngineersQA EngineersCommunity SpecialistPR RenegadeSponsorship SalesAccount Coordinator
http://www.meetup.com/jobs/