cloudflare and drupal - fighting bots and traffic peaks

47
CLOUDFLARE HANDLIN G BOTS A ND T R AFFI C PEA KS Łukasz Klimek : www.softinn.eu

Upload: lukasz-klimek

Post on 23-Aug-2014

637 views

Category:

Internet


7 download

DESCRIPTION

Overview of Cloudflare platform with integration with Drupal CMS; DrupalCamp Wrocław http://goo.gl/0YS0kB

TRANSCRIPT

  • ukasz Klimek : www.softinn.eu
  • PLAN 1. Introduction 2. Cloudflare basics 3. Performance 4. Security 5. Show me the results! 6. Cloudflare and Drupal 7. Questions / discussion 2
  • 3
  • DRUPAL HOSTING NEEDS Shared hosting Cloud / dedicated server Complex infrastructure 4
  • THE PROBLEM Spam bots Comments User registrations Worms, viruses, trojans Traffic peaks Event websites 5
  • FIGHTING SPAM Captcha-style (Captcha / reCAPTCHA) Already cracked. By Google themselves ;-) Mollom captcha text analysis user reputation 6
  • 7
  • PERFORMANCE ISSUES We still process our PHP scripts! Huge CPU utilization Memory consumption DoS in case of multiple concurrent connections 8
  • INCREASING PERFORMANCE APC memcache boost Minimize number of requests Combine & minify CSS / JS Website code refactoring 9
  • NOT ENOUGH? Separate DB server Separate host for static content Reverse proxy (Varnish) 10
  • SO WE GET 11
  • 12
  • ADDING REDUNDANCY 13
  • LOOKS COMPLEX? And thats just the beginning No development/staging servers No shared storage between servers No backups No monitoring No Internet connection redundancy Issues with bandwidth consumption 14
  • 15
  • 99.9% uptime Defend against bots & spam Handle traffic peaks Decrease server load Minimize bandwidth usage Minify CSS and JS LETS SUMMARIZE THE NEEDS 16
  • 17
  • 18
  • WHAT IS CLOUDFLARE? Content Delivery Network (CDN) Web Application Firewall Code optimizer Traffic statistics Application platform 19
  • WHAT IS CLOUDFLARE? (2) 20
  • CLOUDFLARE NETWORK 21
  • 22
  • CLOUDFLARE AS A CDN Works like reverse proxy Caching of static files Caching of dynamic (generated) pages for anonymous users No bandwidth limits / fees 23
  • PERFORMANCE SETTINGS Caching level: Aggressive: http://softinn.eu/pic.jpg?with=query Simplified: http://softinn.eu/pic.jpg?ignore=this-query-string Basic: http://softinn.eu/pic.jpg 24
  • RULES Ability to customize performance & security settings based on URLs Up to 3 rules in Free plan, 20 in Pro plan IMO the most important tool in Cloudflare 25
  • CODE OPTIMIZATIONS Auto Minify - remove unnecessary characters JS CSS HTML Rocket Loader Loads JS asynchronously (after window.onload) Can have some side-effects Website Preloader Detects most often used static resources Fetches these resources to browsers cache 26
  • ROCKET LOADER 27
  • IMAGES Mirage 2 Asynchronous image loading All images in a single request Polish - image otimization Lossless Remove metadata Average reduction of size: about 21% Lossy Additional lossy compression Average reduction of size: 48% 28
  • MIRAGE 2.0 29
  • 30
  • SECURITY OPTIONS E-mail address obfuscation Server side exclude (SSE) Browser integrity check HTTP headers inspection (incl. User-agent) Visitor reputation Hotlink protection HTTP Referers that are not in-zone and not blank will be denied access Hotlink-ok mechanism (eg. http://softinn.eu/hotlink- ok/img.gif SSL support 31
  • THREAT CONTROL 32
  • SUSPICIOUS VISITORS Captcha Ability to blacklist / whitelist IPs Drupal module: Cloudflare 33
  • WEB APPLICATION FIREWALL Set of security rules to address most common threats OWASP TOP 10 Cloudflare-designed: PHP, WHCMS, Joomla, Wordpress, No Drupal-specific rules 34
  • ALWAYS ONLINE Limited version of your site is always online Only the most popular pages No POST and SSL support Crawler-based - crawling every 7, 3 or 1 day Triggers: HTTP status 502 or 504 Connection timeout, SSL errors etc. 35
  • 36
  • EXAMPLE STATISTICS 37
  • NOT A SILVER BULLET Logged-in users Cache invalidation Performance of non-cached pages 38
  • CACHE INVALIDATION There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton (after http://martinfowler.com/bliki/TwoHardThings.html) 1. Cloudflare stores copy of a page in the cache 2. User changes this page 3. How can Cloudflare know that the page has changed? 39
  • 99.9% uptime Defend against bots & spam Handle traffic peaks Decrease server load Minimize bandwidth usage Minify CSS and JS DOES IT SOLVE OUR NEEDS? 40
  • 41
  • PREPARING TO DEPLOY CLOUDFLARE 1. Cache expiration policy 2. Plan your URLs / pathauto config http://www.site.com/can-cache/... 3. Views expiration settings (Views Content Cache?) 4. Apache configuration (proper expiration of static content) 42
  • Expire monitors content updates Expire invokes hook_expire_cache() (cfpurge_expire_cache()) Cloudflare API: zone_file_purge https://drupal.org/project/expire https://drupal.org/project/cfpurge Define Cache everything rule on Cloudflare CFPurge still needs some work; only 16 installs Lack of Views integration 43 CACHE INVALIDATION: EXPIRE + CFPURGE
  • CLOUDFLARE + DRUPAL: QUICK START Review Cloudflare performance settings (Auto Minify, Caching Level, Mirage, Polish, ) Review Cloudflare security settings (obfuscation, hotlink protection, ) Whitelist important IP addresses (monitoring, APIs, ) Create Cloudflare Rules (/admin/*, /user/*, ) Handle remote (client) IP address correctly Install & configure modules (cloudflare, CFPurge, expire) Change DNS delegation Create Cloudflare account 44
  • DNS CONFIGURATION 45
  • TO DO TASKS FOR COMMUNITY 502 / 504 on errors (compatibility with Cloudflare Always Online) https://drupal.org/node/2268487 Views expiration Expire all views that use CT https://drupal.org/node/2146797 (wont fix ) Integrate Expire with Views Content Cache https://drupal.org/node/1786436 (wont fix ) Integrate blacklists with antispam modules (Mollom etc.) 46
  • THANK YOU! 47 ukasz Klimek E-mail: [email protected] Mobile: +48 66 999 2096 Skype: casatm | Twitter @lklimek http://tinyurl.com/lklimek http://goo.gl/2dEgs7 Software Inn www.softinn.eu