cvcc performance tuning
TRANSCRIPT
Web Performance
John McCaffreyRailsperformance.blogspot.com
John McCaffrey: Its all about me
• Doing Java since 2000, Rails since 2007• Presented at WindyCityRails
– 2008: Advanced Firebug and JS unit testing– 2009: PDF Generation in Rails– 2010: Rails Performance Tuning
• Addicted to Performance Tuning• railsperformance.blogspot.com
• Feedback: http://spkr8.com/t/4961
@J_McCaffrey
(the slides will be available, with additional references)
What are we gonna cover?
• Performance touches a lot of layers• We have a very diverse audience
Measuring Page Load
Fixing page load problems
Analyzing Server Performance
Database issues
Ruby issues Q&A
Starting off on the right foot
It all started...
def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0}end
def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0}end
def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end
def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0}end
def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0}end
def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end
It all started...
2811 queries!
Down to 17!!
And then...
Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50
How common are ROI calculations on software projects?
53626 * $4.50 = $241,317 (just in the last year)
Our theme of the day: Leverage
to achieve the greatest value
Performance and Business metrics: Bing.com
Slow site = revenue drop1 sec = 2.8% 2 sec = 4.3%
Performance and Business metrics: Speed matters
Amazon: 100 ms delay caused a 1% drop in revenue.
Google: 400 ms delay caused a 0.59% decrease in search requests per user.
Yahoo!: 400 ms delay caused a 5-9% decrease in traffic.
Bing: 2 sec delay caused a 4.3% drop in revenue per user.
Mozilla made their download page 2.2 seconds faster and saw an increase of 15.4% in downloads.
Google Maps reduced the file volume by 30% and observed a 30% increase in map requests.
Netflix enabled gzip on the server; pages became 13-25% faster and saved 50% of traffic volume!
Performance and Business metrics: Shopzilla.com
Went from 6 secdown to 1.2sec
http://www.phpied.com/the-performance-business-pitch/
Fred Wilson: Speed is the most important feature
10 Golden Principles of Successful Web Apps Fred Wilson, Venture Capitalist www.avc.com
•Speed• Instant Utility• Software is media• Less is More• Make it programmable• Make it personal• Make it RESTful• Discoverability• Clean• Playful
OK, so where do we begin?
Give yourself a fighting chance, make a plan
Alois Reitbauer, dynatrace
Anti-patterns• Thinking Scalability something you just sprinkle on later• Guessing, not testing• Ad-hoc, unstructured• Waiting until it hurts, not proactively planning• Thinking that any early planning is Premature optimization
Video on Parleys.com
Premature optimization
"Premature optimization is the root of all evil"
-Donald Knuth
Premature optimization
Some people really like Donald Knuth
Premature optimization, with DJ DK
like, really, really like him
Premature optimization
"Premature optimization is the root of all evil'
-Donald Knuth
"Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out"@J_McCaffrey
Don't waste time on making optimization changes until you KNOW they are necessary
If you can not measure it, you can not improve it.
— Lord Kelvin
Let's do this the right way
Measure:• Understand the tools to use, at each layer, to measure
and monitor performance
Create repeatable tests:• Be able to invoke the system and easily test• A/B, before/after tests, long-term monitoring
Isolate your changes:• One thing at a time
That's it: Measure, Test, and Isolate your changes
Terminology
Load:How much work is there?
Utilization:How much of the resources are in use?
Scalability:How well can we handle the load?
Throughput:How many tasks can be done per unit of time?
Concurrency:How many tasks can we do at once?
Capacity:How big can our throughput go before things fall apart?
Response time (avg) == Latency
Requests per second == Throughput
Terminology
Performance:• The resources and Time for a single request• Measured in Response Time
Throughput:• The number requests that can be handled per/time• Measured in Requests per second
Scalability:• Ability to grow and handle more requests, without
significantly degrading the experience
Performance != Scalability
Terminology
Let's say: 1 worker that can compute a job in .5 sec
Throughput = 2 jobs per sec Latency = .5 sec per job
Performance != Scalability
Adding more workers will not improve the latency
But not having enough workers can increase the response time, due to Queuing and delay
There are performance opportunities at each layer
Waterfall: Twitter.com
Waterfall
Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100%
When the html is downloaded
When the rest of it is loaded
Waterfall
When the backend is actually doing work
Walmart: Waterfall
Waterfall
Total time: 5520 ms Dynamic: 388 ms (7%)Static: 4580 ms (~82%)
Firebug Netpanel
Simple and repeatable way to measure page load
Usually, less than 20% of the time is spent in fetching the HTML
So how do we optimize the other 80%?
YSlow
• Created by Steve Souders• Observes the netpanel data, applies rules and gives you a
score.• Deep understanding of how the browser works• Its the best way to have a repeatable, measurement of your
site's load performance
http://developer.yahoo.com/yslow
YSlow
• Minimize HTTP Requests• Use a Content Delivery Network• Add an Expires or a Cache-Control Header• Gzip Components• Put StyleSheets at the Top• Put Scripts at the Bottom• Avoid CSS Expressions• Make JavaScript and CSS External• Reduce DNS Lookups• Minify JavaScript and CSS• Avoid Redirects• Remove Duplicate Scripts• Configure ETags• Make AJAX Cacheable• Use GET for AJAX Requests• Reduce the Number of DOM Elements• No 404s• Reduce Cookie Size• Use Cookie-Free Domains for Components• Avoid Filters• Do Not Scale Images in HTML• Make favicon.ico Small and Cacheable
Yslow: Created by Steve Souders
http://stevesouders.com/cuzillion/
Google Page speed
• Similar to YSlow• includes paint events• separates out ads, trackers from your content• better recommendations and optimizations• 1 click to see optimized version of your files
http://code.google.com/speed/page-speed
Google Page speed
Google Page speed
Make it repeatable
http://www.showslow.com/ track your site, or your competitors will check every 24hrs for you
• webpagetest.org• gtmetrix.org• zoompf.com• loadimpact.com• gomez.com
Summary
• Page load is the place to start• Read everything in the Yslow and PageSpeed rules• Encourage other devs to play with YSlow, PageSpeed• Show Webpagetest.org, and zoompf.com to the boss
Now that we have a way to measure page performance, we can think about making changes
Measuring Page Load
Fixing page load problems
Analyzing Server Performance
Database issues
Ruby issues Q&A
Starting off on the right foot
Isn't everyone already doing this?
Analysis of the Alexa top 1000 sites found:
• 42% don't gzip• 44% have more than 2 css files• 56% serve css from a cookied domain• 62% don't minify• 31% have more than 100k size css
Top 1000 retail sites 50% aren't doing both keep-alive and compression (the 2 easiest things!!)
Compress with Gzip
• Can save you 50% to 80% of your bandwidth• Simplest performance improvement you can make• Beware differences in browsers• File types: html, js, css, xml, rss, txt, font, json
Probably the simplest and smartest thing you can do for your app!
Combine and minify javascript and css
Rails• javascript_include_tag :all, :cache => "all"• Asset_packager, Jammit
Load common js libraries from googlehttp://github.com/rpheath/google_ajax_libraries_api
Jsmin or YUICompressorClosure compiler
Let's check it out!
Combine and minify javascript and css
Sprites and image optimization• spriteme.org• JqueryUI already gives you sprited images
Image optimization• Smush.it
Expires and Browser Caching
Setting a far future expires tells the browser not to ask for the file
Expires:Mon, 02 Sep 2030 21:23:08 GMT
/images/logo.png?1234567890
Using the query string lets us break the caching by having the browser ask for a new file
There is a problem with using a URL query string this way http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/
/images/logo-fp-839b180ff39a24f8d6e0ee70e4c40fed.png
Rails Caching
Take your dynamic content, and make it static• Page• Action• Fragment
Full length tutorials• http://railslab.newrelic.com/• http://railscasts.com/episodes?search=caching• Greg Pollack Aloha on Rails http://vimeo.com/10860860
Measuring Page Load
Fixing page load problems
Analyzing Server Performance
Database issues
Ruby issues Q&A
Starting off on the right foot
The Stack
You are here
Request-log-analyzer
• http://github.com/wvanbergen/request-log-analyzer• Request distribution per hour• Most requested• HTTP methods• HTTP statuses returned• Rails action cache hits• Request duration• View rendering time• Database time• Process blockers• Failed requests
Rack::Bug
Rack::Bug
Commercial tools
New Relic• Developer plugin is a good start• Free version doesn’t get you much (Bronze version with heroku)• Works for .NET
• Scoutapp.com• Similar to new Relic• Large selection of community plugins
Testing tools
Apache bench ab -n 10 -c 2 http://www.somewhere.com/
Httperfhttperf --server localhost --port 3000 --uri / --num-conns 10000
Jmeteryes, its ugly, but its powerful
Measuring Page Load
Fixing page load problems
Analyzing Server Performance
Database issues
Q&A
Starting off on the right foot
Database issues
Common database issues• Bad queries• Not utilizing explain• Inadequate indexes• N+1 queries• Selecting more data than is needed• Inconsistent queries for the same data
Query_reviewer
Common database issues• Bad queries• Not utilizing explain and slow query log• http://github.com/dsboulder/query_reviewer • Runs explain on all of your queries, outputs to div in
page
Rails_indexes
http://github.com/eladmeidar/rails_indexes
Foreign key indexes
• Columns that need to be sorted
• Lookup fields
• Columns that are used in a GROUP BY
• Rake tasks to find missing indexes.
New one I haven’t tried yet
http://github.com/samdanavia/ambitious_query_indexer
Bullet plugin
Bullet plugin
• http://github.com/flyerhzm/bullet
Help you reduce the number of queries
with alerts (and growl).
Slim_scrooge
Be more specific in your select• http://github.com/sdsykes/slim_scrooge• Instruments your code• Observes the usage pattern• Suggests/changes your select statement• When invoked within the exact same context
Ruby Issues
Most are due to memory problems• Slow GC• Not release references• Capturing scope
• Profiling will reveal what’s going on• http://guides.rubyonrails.org/performance_testing.html
If you want to learn everything there is to know about profiling • Aman Gupta & Joe Damato• http://timetobleed.com/ • http://memprof.com/ • They’ve already identified and fixed several memory leaks
Ruby issues: Faster Ruby Libraries
C Extension++
• XML parser http://nokogiri.org/
• JSON parser http://github.com/brianmario/yajl-ruby/
• CSV parser http://www.toastyapps.com/excelsior/
• HTTP client http://github.com/pauldix/typhoeus
Date
http://github.com/jeremyevans/home_run
Take home points
Performance• Performance affects the bottom line• The biggest performance win is usually in improving the load time• Continue to monitor and test your apps performance over time• Gzip, combine and minify to get a big boost• Lack of indexes is likely to be one of your biggest backend issues• Try to stay up to date with Libraries and patches• A well tested codebase is easier to tune!• Upgrading to ruby 1.9 will give you a huge performance boost
Links: great info on performance
Performance• Scaling Rails series http://railslab.newrelic.com• RailsCasts.com• Velocity Conf http://en.oreilly.com/velocity2010 • Google io conf http://code.google.com/events/io/2010/• dynatrace site http://ajax.dynatrace.com/pages/• http://www.igvita.com• http://www.mysqlperformanceblog.com
Wait, I made it through that?
Designing for performance• Watch how they really use it• Multiple tabs?• Back and forth between list and detail view
• Great presentation at WindyCityRails2009 by David Eisinger • Optimizing perceived performance• Make it feel fast
Wait, I made it through that?
Architectural musings
•Background long running tasks• Email, image processing, reports, feeds•Leverage Rack, Sinatra or Padrino•Parallel processing, Event Machine•Learn from heroku, scalable from the start•Revaluate your cloudy-ness• Great RailsConf presentation by James Golick
Beyond ySlow: improving page performance outside the initial load
MySql and Postgres Database tuning
Heroku: Building Scalable apps from the start
Tuning performance for Mobile devices
Performance Improvements coming in Rails 3, and 3.1
Speeding up your tests
Questions for you
Architectural musings
• Moving to or already on Ruby 1.9?• Serving your content from a separate domain• Cookieless?