mysql performance tuning at coscup 2014
DESCRIPTION
Presentation used at COSCUP 2014 in Taiwan.TRANSCRIPT
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
MySQL Performance Tuning
MySQL Global Business UnitSales Consulting Senior Manager, JAPAC梶山 隆輔 / Ryusuke Kajiyama
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
MAKE MySQLa Better MySQL
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
MySQL 5.5MySQL Cluster 7.3
MySQL Enterprise Monitor 2.3MySQL Enterprise Backup
SecurityScalabilityHAAudit
MySQL 5.6MySQL Workbench 6.0
MySQL Utilities
MySQL Applier for Hadoop
MySQL Workbench 5.2
MySQL Enterprise Oracle Certifications
Over 3,5 Years of MySQL Innovation
MySQL Cluster ManagerWindows installer & Tools
MySQL Cluster 7.2MySQL Cluster 7.1
MySQL Migration Wizard
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
MySQL: Everywhere and GrowingPlatform of Choice for Next Generation Web, Cloud & Embedded Applications
World’s Most Popular Open
Source Database
Database Platform for
Next Gen Web Apps
Leading Database for
Web Applications
Leading Database in the Cloud
Integrated with Hadoop in
majority of Big Data platforms
#1 Linux Career IT skill: MySQL *
* Source: Linux Career IT Skills Watch update July 2013
Downloads & Sales Growth
Best of both Worlds SQL+NoSQL
LAMP + Python +Node.js + Go +...
Awards & Social Media Momentum
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
MySQL 5.6: Best Release Ever!
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
MySQL 5.6: Scalability
Users can fully utilize latest generations of hardware and OS Scales as data volumes and users grow
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
MySQL 5.5
MySQL 5.6
MySQL 5.6 SysBench Benchmarks
Up to 234% Performance Gain
Replication Binary Log Group Commit • Days of losing 50%+ master
performance are gone!• 10% overhead when configuring
sync_binlog=0• Additional 5% when configuring
sync_binlog=1
https://blogs.oracle.com/MySQL/entry/mysql_5_6_replication_performance
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Same app can leverage: Key-value access to InnoDB via
familiar Memcached API SQL for rich queries, JOINs,
FKs, etc. Fully transactional
MySQL 5.6: InnoDBNoSQL Key Value Access to InnoDB
Up to 9x performance boost for updates
Great for fast data ingestion in Big Data pipeline
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
MySQL 5.6: Optimizer
• Simple word but many meanings• Main objective:
• Users (direct or indirect) should be satisfied• Most typical performance metrics
• Throughput• Latency / Response time• Scalability • Combined metrics
Starvation
transactions/time
Defining Performance
• Multi User applications • Request waits in queue before being processed • User response time = queueing delay + service time
• Non high tech example – support call center.• “Hockey Stick” - queuing delay grows rapidly when
system getting close to saturation• Need to improve queueing delay or service time
to improve performance• Improving service time reduces queuing delay
Connections
Response T
ime
Queuing Theory
• Main Question – where does service time comes from ?• network, cpu, disk, locks...
• Direct Measurements• Sum up all query times from web pages
• Indirect measurements• CPU usage• Disk IO latency• Network traffic• Load Average• Number of running queries• etc.
Service Time: Key to the hotspot
Benchmark Tests
• Great tool to:• Quantify application performance• Measure performance effect of the changes• Validate Scalability• Plan deployment
• But• Can be very misleading if done wrong• Need to be able to read results correctly
• Typical Errors• Testing with 1GB size with 100G in production• Using uniform distribution
• “Harry Potter” ordered as frequent as Zulu dictionary• Testing in single user scenario
Benchmark Tools for MySQL• Creating own benchmark tool
– Generating from SQL statements in General Log– Fetching SQL statements with MySQL Proxy or TCP Dump
• DBT2– http://osdldbt.sourceforge.net/ – http://nippondanji.blogspot.com/2009/03/dbt-2.html
• mysqlslap MySQL 5.1 +– http://dev.mysql.com/doc/refman/5.6/en/mysqlslap.html
• SysBench– http://sysbench.sourceforge.net/
• Performance costs money, whatever road you take• Investigate different possibilities
• Better hardware could be cheaper than a major rewrite• How much performance/scalability/reliability do you need ?
• 99.999% could be a lot more expensive than 99.9% • Is peak traffic requirements 100x average or just 3x ?
• Take a look at whole picture• Is this the largest risk/bottleneck ?
• Identify which optimizations are critical for business• Optimization of “everything” is often waste of resources• What is the cost of suboptimal performance ?
Business side of optimization
Basic1: Checking Server Configurations
• The MySQL server is controlled by “System Variables”
• Set via:• Option File: my.cnf / my.ini• Temporary change: SET [GLOBAL] <vriable>=<value>
• Can be per connection (LOCAL) or server wide(GLOBAL)
Basic2: Checking Status of Server Monitor system performance using “Status Variables”
• Knowing internal commands of one queryMysql> FLUSH STATUS; <run query>; SHOW STATUS;
• Checking status periodically shell> mysqladmin -u -p ... ex -i 15 -r | grep -v ‘ 0 ‘
http://dev.mysql.com/doc/refman/5.6/en/server-status-variables.html
Client2 ClientN
Connection Thread Pool
Parser Query 101101
Client1
Optimizer
Storage Engines InnoDB MyISAM MERGE MEMORY ARCHIVE
Query Cache
MySQL Server
MySQL Server Architecture
• max_connections (151)• number of connections server will
allow. May run out of memory if too high, because of per connections memory usage
• thread_cache_size (8)• Keep up to this amount of threads
“cached” after disconnect• Typical setting
max_connections/3
Client2 ClientN
Connection Thread Pool
Client1
mysql> show status;• Max_used_connections
• check if it matches max_connections, too low value or sign of overload
• Threads_created• thread_cache misses• should be low.
Server Connections & Threads
• sort_buffer_size (2M)• Memory to allocate for sort. Will use
disk based sort for larger data sets. Often fine at 512K or 1M
• other buffers, read, read_rnd, etc… smaller defaults often OK
• You can change dynamically if large sort operation is needed in batch operation etc
Client2 ClientN
Connection Thread Pool
Client1
mysql> show status;• Sort_merge_passes -
• number of passes made during file merge sort.
• check if file sort needs to be done at all
• use index if possible
Connection Thread Work Buffers
• query_cache_size (0)• Amount of memory to use for query
cache• Typically 32M is fine, some databases
need 128M• query_cache_type (ON)
• Worst case performance overhead is about 15%-20%
• Favors servers with higher SELECT/WRITE ratios
• Best Practice• Set to DEMAND• Add SQL_CACHE to appropriate
queries
Connection Thread Pool
Parser Query 101101
Query Cache
mysql> show status;• Qcache_hits, Qcache_inserts
• hits/inserts cache hit ratio, if small, maybe disable query cache
• Qcache_lowmem_prunes• times older queries were removed due to
low memory, need to increase query_cache_size if high
Server Query Cache
• innodb_buffer_pool_size • 80% of memory on Innodb only system• caches data & indexes unlike MyISAM
• innodb_log_file_size• A key parameter for write performance• Recovery time is no more an issue.• Bigger is better for write QPS stability
• innodb_flush_log_at_trx_commit• 1 (slow) will flush (fsync) log at each commit. Truly
ACID• 2 (fast) will only flush log buffer to OS cache on commit,
sync to disk once/sec.• 0 (fastest) will flush (fsync) log every second or so
• innodb_file_per_table• Always good choice to distribute i/o• Default ON from 5.6
Storage Engines InnoDB MyISAM MERGE MEMORYARCHIVE
mysql> SHOW ENGINE INNODB STATUS;
Great way to see what is going on inside InnoDB, hard to parse
• File IO• Buffer Pool • Log activity • Row activity
InnoDB Performance Tips
• innodb_flush_method = O_DIRECT• Not to consume OS cache
• innodb_buffer_pool_instances (5.5+)• To avoid mutex contention• 2 or more in can
• innodb_io_capacity (5.5+)• Enlarge if you have fast disks• Default 200 is good for 2 disks striped
• innodb_read_io_threads (5.5+)• innodb_write_io_threads (5.5+)
• Enlarge if you have fast disks• Default 4 is usually good enough
Storage Engines InnoDB MyISAM MERGE MEMORYARCHIVE
mysql> SHOW ENGINE INNODB STATUS;
Great way to see what is going on inside InnoDB, hard to parse
• File IO• Buffer Pool • Log activity • Row activity
InnoDB Performance Tips (Cont.)
DBT-2 (W200) Transactions per Minute %user %iowait
Buffer pool 1G 1125.44 2% 30%
Buffer pool 2G 1863.19 3% 28%
Buffer pool 5G 4385.18 5.5% 33%
Buffer pool 30G(All data in cache) 36784.76 36% 8%
Semi-in-memory Database with InnoDB
• DBT-2 benchmark (write intensive)• 20-25GB hot data (200 warehouses, running 1 hour)• Nehalem 2.93GHz x 8 cores, MySQL 5.5.2, 4 RAID1+0 HDDs• RAM size affects everything. Not only for SELECT,
but also for INSERT/UPDATE/DELETE• INSERT: Random reads/writes happen when inserting into indexes in random order• UPDATE/DELETE: Random reads/writes happen when modifying records
• Normalization• good for OLTP, writes• data redundancies eliminated• join performance penalty• smaller total data set• E/R diagram clean translation
• Denormalization• good for OLAP, reporting• data redundancies across tables for
better indexing• maybe eliminate joins
• Data types– use tinyint, smallint, mediumint,
save space!– join columns same data type– varchar(64) instead of char(64)– declare NOT NULL if true– varchar(64) instead of varchar(255)– INT not DECIMAL(9)
• Indexing– multi-column– ordered BTREE– index prefixes– covering
Schema Design
#Time: 08073101 16:25:24#User@Host: root[root] @ localhost [127.0.0.1]#Query_time: 8 Lock_time: 0 Rows_sent: 20 Rows_examined: 243661SELECT part_num FROM `inventory`.`parts` WHERE(`ven` = "foo") ORDER BY `delivery_datetime` DESC LIMIT 100;
Monitoring Queries - Slow Query Log
Cons• Requires MySQL restart (5.0 and lower)• Growth must be managed using FLUSH LOGS• Entries must be parsed/sorted for relevance• mysqldumpslow helps, but still tedious, takes time
Pros• Logs queries that took longer than X (user defined)• Logs queries that do not use indexes (5.0 and higher)• Includes data needed to trace offending queries
Monitoring Queries – SHOW PROCESSLIST;
mysql> SHOW FULL PROCESSLIST\G ******** 1. row *****************Id: 1User: MyUserHost: localhostdb: inventoryCommand: QueryTime: 1030455State: Sending Data Info: SELECT part_num from ‘inv’;
…..2 rows in set (0.00 sec)
Pros• Shows current processes• Shows status of executing
queries• Includes data needed to trace
offending queries
ConScripting needed to:• automate, • integrate with Slow Query Log,• Aggregate/parse results for
analysis,• notify DBA of problem
Fixing Problem Queries – EXPLAIN; EXPLAIN SELECT part_num FROM `inventory`.`parts` WHERE (`ven` = "foo") ORDER BY `delivery_datetime` DESC LIMIT 100;\G ******** 1. row ************* ID: 1 select_type: SIMPLE table: parts type: refpossible_keys: ven, part# key: ven key_len: 3 ref: null rows: 872 Extra: Using WHERE1 row in set (0.00 sec)
Analyze• How indexes are being used (or
not…)• required filesorts• What tables, columns are being
queried
Fix/Tune - involves iterations of:• Add/alter indexes• Alter tables, columns, datatypes• Alter query structure• Test, 10 GOTO 10 until done
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1230
MySQL Enterprise Monitor
Real-time MySQL performance and availability monitoring
Visually find & fix problem queries
Disk monitoring for capacity planning
Cloud friendly architecture (no agents)
Start monitoring MySQL in 10 minutes
Remote agent option provides OS monitoring
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1231
MySQL Query Analyzer
Real-time query performance
Visual correlation graphs
Find & fix expensive queries
Detailed query statistics
Query Response Time index (QRTi)
“With the MySQL Query Analyzer, we were able to identify and analyze problematic SQL code, and triple our database performance. More importantly, we were able to accomplish this in three days, rather than taking weeks.”
Keith Souhrada Software Development Engineer Big Fish Games
MySQL Enterprise ScalabilityMySQL Thread Pool
• MySQL default thread-handling – excellent performance, can limit scalability as connections grow
• MySQL Thread Pool improves sustained performance/scale as user connections grow
• Thread Pool API
New!
MySQL Enterprise Scalability
ConfigurationMySQL 5.6.11Oracle Linux 6.3, Unbreakable Kernel 2.6.324 sockets, 24 cores, 48 ThreadsIntel(R) Xeon® E7540 2GHz CPUs512GB DDR RAM
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.34
Performance Schema in MySQL 5.5 & MySQL 5.6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.35
Performance Schema in MySQL 5.5
Laid out the framework for instrumentationFocused early on low level instruments
– Prove it could handle load, build from ground up
Not turned on by default, had some overhead issues
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.36
Performance Schema in MySQL 5.5
17 Tables222 Instruments
Instrument Event ClassFile IO wait/io/file/%Mutexes wait/synch/mutex/%Read/Write Locks wait/synch/rwlock/%
Conditions wait/synch/cond/%
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.37
Performance Schema in MySQL 5.5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.38
Performance Schema in MySQL 5.6
Fixed lots of performance issuesFocused on bringing the instrumentation to the DBA / Developer
Statements / Statement DigestsExecution StagesObject tracking (table / index IO, table locks)Network IO
Turned on by default, with a subset of instrumentation disabled
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.39
Performance Schema in MySQL 5.6
52 Tables (+35)545 Instruments (+323)
Instrument Type Event Class
Statements statement/%Stages stage/%Table IO wait/io/table/%Table Locks wait/lock/table/%Network IO wait/io/socket/%Idle Timing idle
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.40
Performance Schema in MySQL 5.6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.41
Improvements made to date in MySQL 5.7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12
Insert Picture Here
42
Memory UsageMetadata LockingReplication Configuration & StatusPrepared StatementsTransactionsStored Programs
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.43
Improvements made to date in MySQL 5.7
75 Tables (+23)784 Instruments (+239)
Instrument Type Event Class
Transactions transaction
Memory memory/%
Metadata Locks wait/lock/metadata/%
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.44
Improvements made to date in MySQL 5.7
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45
DMRs: Development Milestone Releases
Fully Functional Release Candidate Quality
Two to Four per Year
Early Community Testing, Use, & Feedback
Accelerates Rate of Enhancements
Improves Quality
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46
MySQL 5.7
InnoDB for better transactional throughput, availability Replication for better scalability and availability Utilities for dev/ops automation Performance Schema for better performance metrics Optimizer for better EXPLAINing query performance
Available Now! Get it here:
MySQL 5.7.2 builds on MySQL 5.6 by improving:
dev.mysql.com/downloads/mysql/
New!
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47
MySQL 5.7 Sysbench BenchmarksSysbench Point Select
Intel(R) Xeon(R) CPU X7560 x86_644 sockets x 8 cores-HT (64 CPU threads)2.27GHz, 256G RAMOracle Linux 6.2
95% Faster than MySQL 5.6 172% Faster than MySQL 5.5
500,000 QPS
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.48
RO In-Memory @MySQL 5.7
Sysbench OLTP_RO 8-tables, 32cores-HT :
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.49
RO In-Memory @MySQL 5.7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.50
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.51
第一次台灣MySQL社群聚會
我們將在 7月 22日下午 6:00到 8:00借用台北市忠孝西路一段 35樓甲骨文的會議室,舉辦第一次台灣MySQL社群聚會,歡迎您抽空參加。在聚會中我們將會中提供簡單的晚餐,同時分享MySQL的最新動態,包括:
– 展示如何用新發表的MySQL Fabric支持上 PB級資料的應用系統,做資料庫分表分庫,高可用和讀寫分離
– MySQL 5.7.4 DMR的新功能和産品發展路線也非常歡迎您分享您的心得和意見如果您打算參加 22日的聚會,或打算分享您的心得
1st TWMUG meeting