launch with confidence
DESCRIPTION
Launch with Confidence. Improving Website Performance and Scalability with Memcached. Presented by: Shawn Smiley [Lead Architect]. About Me. Shawn S. Smiley Lead Architect at Achieve Internet 15+ years experience in Internet software and hosting 5+ years with Drupal Recently married. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/1.jpg)
Launch with Confidence
Improving Website Performance and Scalability with Memcached
Presented by: Shawn Smiley [Lead Architect]
![Page 2: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/2.jpg)
About Me • Shawn S. Smiley• Lead Architect at
Achieve Internet• 15+ years
experience in Internet software and hosting
• 5+ years with Drupal• Recently married
LinkedIn: www.linkedin.com/in/shawnsmiley/Twitter: @shawnsmiley
3
![Page 3: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/3.jpg)
AgendaAbout Achieve
At a Glance Key Clients Achieve Services
Intro to Caching How it Works?
What is Memcached?When and Why You Should Use It?
Performance and StatsMinimal Setup
Components of a Memcache Solution PECL Memcache vs Memcached Commands
Verifying and Troubleshooting Minimal Install Verification
Distributed Configurations When to Use It Types Configuration Tuning Tips
Are There Alternatives?ConclusionQ&A
3
![Page 4: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/4.jpg)
Achieve at a Glancelegal name
founded
headquarters
leadership
mission
primary contact info
Achieve Internet, Inc.
1998
1767 Grand Avenue, Suite #2 | San Diego, CA 92109 | U.S.A.
Ron Huber, Chief Executive Officer
Achieve leverages best-in-class Open Source web application development experience, and tools to create optimized platforms for enterprise clients.
800.618.8777 | [email protected]
Achieve is a leading software development company that leverages Drupal to develop advanced enterprise web and mobile solutions for an array of clients in varying
verticals. Achieve has a singular goal: To help you launch your advanced digital platforms with confidence.
Our ethos is “it’s not magic, it’s excellence,” and our all-star developers deliver on that promise with every new project we take on by consistently exceeding client
expectations.
4
![Page 5: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/5.jpg)
Key Clients: Media & Entertainment
clients
5
![Page 6: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/6.jpg)
Additional Key Clients
clients
6
![Page 7: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/7.jpg)
Achieve ServicesAchieve builds custom enterprise IT management systems to solve the most complex business challenges.
Achieve makes integration easy so you can focus on what is most important to your business – your customers.
Optimize the performance of your web site or application to increase productivity and streamline operations.
Achieve has vast experience in creating global web solutions that far out perform the competition.
‘Going Mobile’ is no longer an option. With Achieve you won’t have to compromise design for utility.
Application Development
Platform Integration
Infrastructure Services
Globalization Services
Mobility & End User Services
7
![Page 8: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/8.jpg)
Intro to Caching
8
![Page 9: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/9.jpg)
Intro to Caching• A cache is a system that stores the results of
some processing for repeated later use.
• Example in Drupal is the page cache.
• Utilized via the cache_set() and cache_get() Drupal API calls.
• Caches in Drupal are Key/Value pairs.
9
![Page 10: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/10.jpg)
Intro to Caching: How it Works?
REQUEST CHECK CACHE CACHE_GET() QUERY CACHE
RETURN CACHED
DATA
10
![Page 11: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/11.jpg)
Intro to Caching: How it Works?
REQUEST CHECK CACHE CACHE_GET() QUERY CACHE
GENERATE DATA
CACHE_SET() SAVE CACHE
RETURN CACHED
DATA
11
![Page 12: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/12.jpg)
What is Memcached?
12
![Page 13: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/13.jpg)
What is Memcached?• “Memcached is an in-memory key-value store for
small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.” – memcached.org
• A system to cache data in RAM rather than in database.
• Replaces Drupal’s default database cache.
• Just one component of your high performance scalable architecture.
13
![Page 14: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/14.jpg)
When & Why Should You Use It?
14
![Page 15: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/15.jpg)
Why Use Memcached?• Improves cache read/write performance
• Reduces the number of database queries
• Reduces database load
• Can improve page load performance*
• Allows distributing the cache load to separate servers
• Moves the threshold for performance degradation to a higher number of concurrent users.
* By itself, Memcache.d won’t provide a significant performance improve on low traffic sites. The page load performance boost comes mostly from offloading work from your database server so that the system can handle more requests.
15
![Page 16: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/16.jpg)
Performance Stats• Performance & Scalability
• Cache read performance: 4X Faster• Cache write performance: 60X Faster
• Actual statistics* (Drupal core vs. Memcached)• 1000 Cache write operations: 12 seconds vs. 0.2 seconds• 1000 Cache read operations: 0.4 seconds vs. 0.1 seconds• DB queries: 3028 vs. 13
* Cache Read/Write stats are for performing 2,000 operations (1,000 write, 1,000 read)* Your results will vary, tests performed on a single AWS instance with default D7 install
16
![Page 17: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/17.jpg)
Performance Stats
17
![Page 18: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/18.jpg)
Performance Degradation Threshold
18
NOTE: This is an idealized graph of the average change in response time across several projecs.
![Page 19: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/19.jpg)
When to (or not) use Memcached• Use Memcached when:
• Significant portion of site traffic is authenticated users or when static page caching isn’t an option.
• Have adequate server resources to allocate to Memcached without negatively impacting other services.
• Have enough traffic to make a difference.• Don’t use Memcached when:
• Site is completely anonymous traffic • Static caching is better for this (such as Varnish or Boost)
• Low traffic sites that are not straining server resources • Probably won’t see any noticeable benefit and have
increased the complexity of the system.• Severely resource constrained single server hosting
• Can negatively impact performance/scalability in this case.
19
![Page 20: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/20.jpg)
Setup
20
![Page 21: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/21.jpg)
Components of Memcached Solution
• Memcached Daemon (v 1.4.x)• PHP Extension
• PECL Memcache (> 3.x) • PECL Memcached (> 2.x)• TIP: Use the setting “memcache_extension” to select
which library to use if you have both installed.• Memcache Module• Drupal 8, Drupal 7.9+, Pressflow 6, or
Drupal 6 with core patch
21
![Page 22: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/22.jpg)
PECL Memcache vs. MemcachedPECL Memcache PECL MemcachedStable, not as frequently updated Newer, more featuresEasy to install More complicated install with
multiple dependenciesLower PHP memory footprint in some cases.
Can be slightly faster in some cases
Slightly better for single server Memcached daemon installations.
Slightly better for distributed or clustered Memcached daemon installations.igbinary support
* Generally, we default to PECL Memcache unless a project can take advantage of the additional features provided by PECL Memcached.
22
![Page 23: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/23.jpg)
Minimal Install: Commands (Ubuntu)
• Enable apt multiverse repositories
• sudo apt-get install php-pear build-essential
• sudo apt-get install memcached
• sudo apt-get install php5-memcache
• drush dl memcache devel
23
![Page 24: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/24.jpg)
Minimal Install: Daemon Configuration
/etc/memcached.conf (or /etc/default/memcached)•Configuration file for the memcached daemon•Important settings:
• -m 64• Amount of memory in MB to allocate.
• -p 11211• TCP/IP Port to listen for cache requests on.
• -l 127.0.0.1• IP Address to listen for cache requests on.
24
![Page 25: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/25.jpg)
Determining Memcached size• Look at the data size of all cache* tables in your DB.• Use this handy query to get the total size:
SELECT count(*) TABLES, sum(table_rows) ROWS, concat(round(sum(data_length)/(1024*1024),2),'MB') DATA, concat(round(sum(index_length)/(1024*1024),2),'MB') idx, concat(round(sum(data_length+index_length)/(1024*1024),2),'MB') total_sizeFROM information_schema.TABLESWHERE TABLE_SCHEMA=’dbname' AND TABLE_NAME LIKE "cache%"
• Total memory allocated to Memcached should be at least the size returned in the “DATA” column.
• I usually like to add at least 50% to this size to start with.
25
![Page 26: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/26.jpg)
Note about Memcached security• Basically there are no security controls in
Memcached.• Any user or application that can establish a TCP
connection to the Memcached daemon can access all data stored within Memcached.
• Security Best Practices:• Always set the daemon to listen on a specific IP
address/network interface.• Use 127.0.0.1 if there is only 1 Memcached server and it
is running on the same server as the web server.• Use iptables or other firewall to restrict access to the
Memcached TCP/IP Port(s).
26
![Page 27: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/27.jpg)
Minimal Install: Drupal Settings.php
// Required minimum settings.$conf['cache_backends'][] = 'sites/all/modules/contrib/memcache/memcache.inc';$conf['cache_default_class'] = 'MemCacheDrupal';$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
// Optional, but recommended settings.$conf['memcache_key_prefix'] = 'ai_'; // Remember about 250 byte key size limit.$conf['lock_inc'] = 'sites/all/modules/contrib/memcache/memcache-lock.inc';$conf['memcache_stampede_protection'] = TRUE;
// The following are required only if Memcached is on a different server.$conf['memcache_servers’] = array(’10.11.12.13:11211' => 'default');$conf['memcache_bins’] = array('cache' => 'default');
// Only if both PECL Memcache and Memcached are installed.$conf['memcache_extension'] = 'Memcache';
27
![Page 28: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/28.jpg)
Minimal Install: Drupal Configuration
Drupal Performance Settings
28
![Page 29: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/29.jpg)
Verifying & Troubleshooting
29
![Page 30: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/30.jpg)
Minimal Install: Verifying• Memcache Admin Module
• Adds configuration page at Configuration => Memcached
• Enable the option “Show Memcache statistics” to see a log of all Memcached calls on each page.
• Devel Module• Enable the Query Logging option.
• Memcache Admin UIs:• http://livebookmark.net/journal/2008/05/21/
memcachephp-stats-like-apcphp/ • https://code.google.com/p/phpmemcacheadmin/
30
![Page 31: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/31.jpg)
Minimal Install: Verifying - Memcache
• Enable the module “Memcache Admin”• Config page at Configuration => Memcache• Enable the option “Show Memcache statistics” to see
a log of all Memcached calls on each page.
31
![Page 32: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/32.jpg)
Minimal Install: Verifying - Devel• In Devel Settings, enable “Display query log”• Reload pages, you’ll see a list of all queries executed• Ensure there are no calls to any “cache*” tables (other
than cache_form)• Ensure there are no calls to the “semaphore” table
(lock.inc)
32
![Page 33: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/33.jpg)
Minimal Install: Verifying – Admin UIs
Demo
•Drupal Memcache Statistics
•memcache.php
•Memcache Admin
33
![Page 34: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/34.jpg)
Distributed Configurations
34
![Page 35: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/35.jpg)
Distributed Install: Overview
• When to use
• Types of distributed installs
• Configuration Steps
35
![Page 36: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/36.jpg)
Distributed Install: When To Use It• High traffic sites making more cache requests
than a single instance can efficiently handle. • The default config of a Memcached daemon will
support up to 1,024 simultaneous connection requests.• Large caches where it makes sense to separate
out cache data by type or update frequency (e.g. cache_menu is usually pretty static whereas the default cache could be constantly changing)
• I start looking at this as an option when the memory allocation for Memcached starts getting over 128MB.
36
![Page 37: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/37.jpg)
Distributed Install: Types• Multi-server/Clustered
• Distributes cache load across multiple Memcached servers.• Allows for handling larger numbers of cache operations.
Useful for high traffic sites.• NOTE: Caches are not replicated between multiple servers
by default.• Multi-bin
• Segments different types of caches to different servers.• Useful for very large caches with varying characteristics
(e.g. segmenting mostly read-only caches from caches that are frequently updated)
• Hybrid• You can combine both of the above approaches (e.g. have
multiple bins with each bin spread across multiple servers).
37
![Page 38: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/38.jpg)
Distributed Install: Configuration
• Server configuration is same as the basic install, just on multiple servers.
• Make sure the Memcached daemon is configured to listen on each servers external IP address (not 127.0.0.1).
• All daemons within a cluster should be configured with the same memory allocation.
• You can have multiple Memcached daemons running on a single server. Just use different ports.
38
![Page 39: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/39.jpg)
Distributed Install: ConfigurationAdditional settings needed in the site settings.php file// Must be exactly the same on all web servers.$conf['memcache_servers' = array( '10.11.12.13:11211' => 'default', '10.11.12.14:11211' => 'default', '10.11.12.15:11211' => 'my_readonly_bin',);$conf['memcache_bins’] = array( 'cache' => 'default', // must specify if setting this variable. 'cache_bootstrap' => 'my_readonly_bin', 'cache_menu' => 'my_readonly_bin', 'cache_path' => 'my_readonly_bin',);// If using PECL Memcached.$conf['memcache_options'] = array( Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT,);
39
![Page 40: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/40.jpg)
Tuning Tips• Watch the “Available Memory”, “Evictions”
(==0), and “reclaimed” (==0) stats to determine if memory allocation needs to be adjusted.
• Watch “Max Connection Errors” (==0), “Accepting Connections” (==1), “Connection Yields” (==0), and “Listen Disabled Number” (==0) to determine when you need to move from a single memcached server to multiple servers.
Tip: See this page for a description of various stat counters:
http://www.pal-blog.de/entwicklung/perl/memcached-statistics-stats-command.html
40
![Page 41: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/41.jpg)
Alternatives
41
![Page 42: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/42.jpg)
Are There Alternatives?Redis (https://drupal.org/project/redis)
• Great for persistent or replicated caches.• Supports additional caching options such as structured
data caching.• More efficient support for selective cache delete/expiration.• Nice comparison to Memcache:
http://noblogdeffound.blogspot.com/2013/04/memcached-vs-redis-comparison-name.html
MongoDB Cache (https://drupal.org/project/mongodb)• Great for extremely large caches or when a large amount
of data must be stored in a single cache key.
42
![Page 43: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/43.jpg)
Conclusion
By adding Memcached to our hosting solution we:• Reduced the number of database queries.• Reduced the load on our database servers.• Reduced the amount of time the site spends accessing
cached data.• Reduced the rendering time for page requests.• Improved the overall scalability of the site.
43
![Page 45: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/45.jpg)
Connect with AchieveIf you have any further questions or would like to get in touch with Achieve you can do so on these various
channels:
45
linkedin.com/company/Achieve-Internet
plus.google.com/101805819007866218919
@AchieveInternet
facebook.com/pages/Achieve-Internet
![Page 46: Launch with Confidence](https://reader035.vdocuments.mx/reader035/viewer/2022081422/56814ac0550346895db7d466/html5/thumbnails/46.jpg)
Q & A
46