Queues, Pools, Caches

Download Queues, Pools, Caches

Post on 26-Jan-2015




1 download

Embed Size (px)


Transaction processing systems are generally considered easier to scale than data warehouses. Relational databases were designed for this type of workload, and there are no esoteric hardware requirements. Mostly, it is just matter of normalizing to the right degree and getting the indexes right. The major challenge in these systems is their extreme concurrency, which means that small temporary slowdowns can escalate to major issues very quickly.In this presentation, Gwen Shapira will explain how application developers and DBAs can work together to built a scalable and stable OLTP system - using application queues, connection pools and strategic use of caches in different layers of the system.


<ul><li> 1. Queues, Pools,and CachesPresented by: Gwen shapira, Senior consultant</li></ul> <p> 2. About Myself 13 years with a pager Oracle ACE Oak table member Senior consultant for Pythian @gwenshap http://www.pythian.com/news/author/shapira/ shapira@pythian.com2 2009/2010 Pythian 3. Why PythianRecognized Leader: Global industry-leader in remote database administration services and consulting for Oracle,Oracle Applications, MySQL and SQL Server Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion andWestern Union to help manage their complex IT deploymentsExpertise: One of the worlds largest concentrations of dedicated, full-time DBA expertise. Employ 7Oracle ACEs/ACE Directors Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata,Oracle GoldenGate &amp; Oracle RACGlobal Reach &amp; Scalability: 24/7/365 global remote support for DBA and consulting, systems administration, specialprojects or emergency response 3 2009/2010 Pythian 4. 2009/2010 Pythian 5. WHY?5 2009/2010 Pythian 6. OLTP:High ConcurrencyLow LatencyLow Variance6 2009/2010 Pythian 7. 7 2009/2010 Pythian 8. Our mission:Use modern application design tocontrol database concurrency to itsmaximum throughput, lower latencyand make the system morepredictable.8 2009/2010 Pythian 9. Nobody expects modernapplication design!9 2009/2010 Pythian 10. Our Chief weapons are: Connection Pools Queues Caches And fanatical monitoring And ruthless capacity planning And nice red uniforms!10 2009/2010 Pythian 11. Connection Pools1 12. The ProblemOpening a database connectionis high latency operation.OLTP systems cant afford this latencyfor every user request 2009/2010 Pythian 13. The Solution 2009/2010 Pythian 14. Application Business LayerApplication Data Layer JNDIDataSource InterfaceDataSource Connection JDBC DriverPool14 2009/2010 Pythian 15. New Problems CPU + Latch Slow Resp. contention TimeRun out of Add More! connections15 2009/2010 Pythian 16. 5000 connectionsin pool ?But only 12 cores?16 2009/2010 Pythian 17. 17 2009/2010 Pythian 18. 18 2009/2010 Pythian 19. And thats not all How long does it take to open 5000 connections to the database?19 2009/2010 Pythian 20. New Solutions 1.Load test 2.Limit connection pool 3.???? 4.Low latency!!!20 2009/2010 Pythian 21. Objection!1. But I dont use all connections at once2. Our connection pool grows and shrinks automatically 2009/2010 Pythian 22. "Dynamic connection pools are a loaded gun pointed at your system. Feeling lucky, punk?" Graham Wood, Oracle22 2009/2010 Pythian 23. Objection!Problem: We cant load testSolution: Guesstimate1. How many cores are available?2. How much time is spent squatting a connection without running database queries?3. How much workload is IO-bound? 2009/2010 Pythian 24. Objection!Problem: We have 5000 application serversSolution: Data Layer1. Separate servers running data layer2. Fewer servers3. Load balance based on capacity 2009/2010 Pythian 25. Queues2 26. The ProblemWe have more user requests thandatabase connections 2009/2010 Pythian 27. What do we do? 1. Your call is important to us 2. Show them static content 3. Distract them with funny cat photos 4. Prioritize them 5. Acknowledge them and handle therequest later 6. Jump them to the head of line 7. Tell them how long the wait is27 2009/2010 Pythian 28. Solution Message QueueMsg 1 Msg 2 Msg 3... Msg N28 2009/2010 Pythian 29. Application Business LayerMessage QueueApplication Data Layer DataSource Interface JNDI DataSource Connection JDBC DriverPool29 2009/2010 Pythian 30. New Problems Stuff developers say about message queues: 1. It is impossible to reliably monitor queues 2. Queues are not necessary if you do propercapacity planning 3. Message queues are unnecessarilycomplicated.30 2009/2010 Pythian 31. 31 2009/2010 Pythian 32. Do Monitor:1. Service time2. Arrival rate3. Queue size4. Utilization32 2009/2010 Pythian 33. Capacity Planning Myth: With proper capacity planning, queues are not necessary Fact: Over-provisioning is not proper capacity planning Fact: Queue theory is capacity planning tool. Fact: Introduction of a few well defined and well understood queues will help capacity planning.33 2009/2010 Pythian 34. Complex Systems 1. Queues are simple 2. Enterprise messagequeues are complex 3. Match solution to problemrequirements34 2009/2010 Pythian 35. Caches3 36. The ProblemOLTP scales best when working set iscached in RAMRDBMS have limitedmemory scalability 2009/2010 Pythian 37. The Solution - Memcached App App App App Memcached Memcached Memcached Memcached App App37 2009/2010 Pythian 38. How is it awesome? 1. Less DB access 2. Less disk access 3. Distributed 4. Simple KV store 5. Free memory 6. Latency and availability resilience38 2009/2010 Pythian 39. Amazon ElastiCache Memcached cluster of any size in Amazon cloud Unfortunately only accessible from EC2 9 cents per node per hour!39 2009/2010 Pythian 40. Linear Scalability?40 2009/2010 Pythian 41. More Numbers 1. 0.007ms latency on my desktop 2. 2ms latency on cloud 3. 60K gets a second 4. All from the smallest possible servers at 38cents per hour.41 2009/2010 Pythian 42. Application BusinessMessageLayer Queue Memcached Application Data Layer DataSource InterfaceJNDIDataSource Connection JDBC Pool Driver42 2009/2010 Pythian 43. New Problems Does not apply automatically How to use it effectively? How to monitor it? How big?43 2009/2010 Pythian 44. Use Case - Select function get_username(int userid) { /* first try the cache */ name = memcached_fetch("username:" + userid); if (!name) { /* not found : request database */name = db_select("SELECT username FROM users WHERE userid = ?", userid); /* then store in cache until next get */memcached_add("username:" + userid, username); } return data; }44 2009/2010 Pythian 45. Use Case - Update function update_username(int userid, string username) { /* first update database */ result = db_execute("Update users set username=? WHERE userid=?", userid,username);if (result) {/* database update successful: update cache */memcached_set("username:" + userid, username);}45 2009/2010 Pythian 46. Usage Advice 1. Use the ASH 2. More memory, fewer cores 3. DB is for durable writes 4. Warm-up the cache 5. Store nulls 6. Updates are tricky 7. Backward compatible schema46 2009/2010 Pythian 47. How Big? Cluster: As big as you can Node: Not too big to fail.47 2009/2010 Pythian 48. What will we gain by adding 1G cache?1. You cant calculate2. Log all cache hits and misses, by key3. Or sample4. Run cache simulator5. Predict avg. latency48 2009/2010 Pythian 49. 1ms * 0.95 + 5ms * 0.05 = 1.2ms49 2009/2010 Pythian 50. Monitor 1. Number of items, gets, setsand misses 2. Number of evictions andeviction time. 3. Low hit rate and higheviction rate? 4. Swapping 5. Average response time 6. Number of connections50 2009/2010 Pythian 51. Reminder: 1. Use Connection Pools 2. Limit the number of connections 3. Use queues to handle the excessiveload 4. Use caches to make everythingfaster51 2009/2010 Pythian 52. Thank you and Q&amp;A To contact us sales@pythian.com 1-866-PYTHIAN To follow us http://www.pythian.com/news/ http://www.facebook.com/pages/The-Pythian-Group/ http://twitter.com/pythian http://www.linkedin.com/company/pythian52 2009/2010 Pythian </p>