mysql webinar series 4/4 - manage & tune

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

How To Manage & Tune MySQL

Mark Swarbrick Principle Presales Consultant

MySQL Webinar Series 4/4


Safe Harbor Statement The following is intended to outline our general product direcFon. It is intended for informaFon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcFonality, and should not be relied upon in making purchasing decisions. The development, release, and Fming of any features or funcFonality described for Oracle’s products remains at the sole discreFon of Oracle.

3


Manage & Tune MySQL

Tuning ConsideraFons + Go-‐live factors to consider

Planning for outages

Upgrading MySQL

4

1

2

3


Tuning + Go-‐Live ConsideraFons

5


Performance Tes0ng – How to measure success

Monitoring – Make sure you are monitoring the correct metrics

Backup Strategy – Will your backups be suitable for purpose?

High Availability – Does the up;me requirement match the design?

Failover – Have you got the policies and procedures in place to failover to a backup data center or server?

Growth – Have you planned for growth of data and throughput?

Tuning – How to tune and monitor for performance

6


MySQL Performance Schema •  IdenFfy performance boXlenecks •  IdenFfy problemaFc queries • Get real Fme insight into locks •  See exactly what is happening within MySQL

• Get real Fme insight into MySQL internals

• Get real Fme insight into query execuFons

7

mysql> select * from host_summary_by_stages; +-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | host | event_name | total | wait_sum | wait_avg | +-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | hal | stage/sql/Opening tables | 889 | 1.97 ms | 2.22 us | | hal | stage/sql/Creating sort index | 4 | 1.79 ms | 446.30 us | | hal | stage/sql/init | 10 | 312.27 us | 31.23 us | | hal | stage/sql/checking permissions | 10 | 300.62 us | 30.06 us | | hal | stage/sql/freeing items | 5 | 85.89 us | 17.18 us | | hal | stage/sql/statistics | 5 | 79.15 us | 15.83 us | | hal | stage/sql/preparing | 5 | 69.12 us | 13.82 us | | hal | stage/sql/optimizing | 5 | 53.11 us | 10.62 us | | hal | stage/sql/Sending data | 5 | 44.66 us | 8.93 us | | hal | stage/sql/closing tables | 5 | 37.54 us | 7.51 us | | hal | stage/sql/System lock | 5 | 34.28 us | 6.86 us | | hal | stage/sql/query end | 5 | 24.37 us | 4.87 us | | hal | stage/sql/end | 5 | 8.60 us | 1.72 us | | hal | stage/sql/Sorting result | 5 | 8.33 us | 1.67 us | | hal | stage/sql/executing | 5 | 5.37 us | 1.07 us | | hal | stage/sql/cleaning up | 5 | 4.60 us | 919.00 ns | +-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+


MySQL Workbench Overview

• MySQL Database IDE • Millions of Downloads • Windows, OS X, Linux


Workbench 6.0

New Look, Redesigned GUI, Simplified Workflow •  Home Screen, Managing ConnecFons, Edit & Admin, Color Scheme, Icons

For Developers •  Schema Inspector, Maintenance, Cascading Delete, VerFcal Results, Table Data Search, Context SensiFve Help, Table Templates

For MigraFon •  New Sybase Anywhere, SQLite

For Administrators •  Server Status / Server Data, Backup GUI, Audit Log Inspector

For Design •  Improved SynchronizaFon, Improved model prinFng to PDF files

What’s New


Performance Tasks In Workbench

Monitor

Assess

OpFmize Tune

Configure


Start by looking at Server Status

§  Management Tab – Server Status


Basics

•  Enabled/Disabled Features

§ Directory Setup § On best disks? § Adequate space? §  Slow Query Log On?


Performance Tes0ng – How to measure success

Monitoring – Make sure you are monitoring the correct metrics

Availability

Use a decent query

Logs

Logs filling up disk space

Slow query log filling space

Disk Space

Disks geing full is the most common problem

Ideally alarm on how soon disk space changes rather than absolute disk space

13


Performance Tes0ng – How to measure success Monitoring – Make sure you are monitoring the correct metrics

Stalls / Spikes

Table Locks

CPU Spikes

Memory Paging

Connec0ons

Are you reaching max_connecFons limit?

Can the applicaFon connect?

Processes

Long running Processes

14



Transac0ons Long running or long idle transacFons Queued transacFons (size of trans queue) (show innodb status) Replica0on Is ReplicaFon running Whats the replicaFon lag

15



Query Performance Top Queries to opFmise – (interacFve monitor not generate alerts on) Response Fme outliers Queries not using index Queries using full/ parFal scans / table scans Queries that return errors / warning Opera0onal Aspects Server restarts Server config change events Messages in the error Log

16


Backup Strategy – Will your backups be suitable for purpose

MySQLdump -‐ It can be used to dump a database or a collecFon of databases for backup or transfer to another SQL server (not necessarily a MySQL server). The dump typically contains SQL statements to create the table, populate it, or both. However, mysqldumpcan also be used to generate files in CSV, other delimited text, or XML format.

MySQL Enterprise Backup -‐ "Hot" Online Backups -‐ Backups take place enFrely online, without interrupFng MySQL transacFons

High Performance -‐ Save Fme with faster backup and recovery

Incremental Backup -‐ Backup only data that has changed since the last backup

ParFal Backup -‐ Target parFcular tables or tablespaces

Full Instance Backup -‐ Backs up data, as well as configuraFon and other informaFon to easily create a complete "replica"

17

MySQL Backup Types: Comparison mysqldump LVM Snapshots MySQL Replica0on MySQL Enterprise Backup

Full Backup

✔ ✔ ✔ ✔

Incremental Backups ✖ ✔ ✖ ✔

Par0al Backups ✔ ✖ ✖ ✔

Compression Support ✖ ✖ ✖ ✔

Allows updates ✖ ✖ ✔ ✔

Point in Time -‐ Consistent ✖ ✔ ✔ ✔

Backup Speed Poor Good Very Good Very Good

Recovery Speed Very Poor Good Very Good Very Good

Par0al Restore ✔ ✖ ✖ ✔

LZ4 Compression ✖ ✔

AES 256 Encryp0on ✖ ✔

Corrup0on Detec0on ✔ ✖ ✖ ✔

Meets Regulatory Archive Req.

✔ ✖ ✖ ✔

Supports DDL ✔ ✖ ✖ ✔


High Availability – Does the up;me requirements match the design?

See Planning for outages sec;on ;-‐) But always good to refer back to the original design spec!

19


Failover – Have you got the policies and procedures in place to failover to a backup data center or server?

Most HA strategies fail not due to the hardware or sooware, but to the lack of awareness and tesFng of policies and procedures in place!

20


Growth – Have you planned for growth of data and throughput?

21


Tuning – How to tune and monitor for performance

22

Hardware

Config OpFons

Schema

InnoDB

Performance Monitoring


Hardware

Schema Changes Data Growth

Indexes

SQL

90% of Performance Problems

Source of Database Performance Problems

23

Hardware: The Perfect MySQL Server

•  The more cores the beXer (especially for 5.5 and later) •  x86_64 -‐ 64 bit for more memory is important –  The more the beXer •  Fast HD (10-‐15k RPM SATA) or NAS/SAN…… –  RAID 10 for most, RAID 5 OK if very read intensive –  Hardware RAID baXery backed up cache criFcal! – More disks are always beXer! -‐ 4+ recommended, 8-‐16 can increase IO •  …Or SSD (for higher throughput) –  Intel, Fusion-‐IO good choices; good opFon for Slaves •  At least 2 x NICs for redundancy •  Slaves should be as powerful as the Master

Config OpFons The MySQL server is controlled by “System Variables”

•  Set Via: –  my.cnf / my.ini –  SET [GLOBAL] <variable>=<value> –  client, i.e mysql •  Can be local (session) or global

The World’s Most Popular Open Source Database Copyright 2010 Oracle


Schemas •  Size = performance, smaller is beXer –  Size right! Do not automaFcally use 255 for VARCHAR •  Temp tables, most caches, expand to full size

•  Use “procedure analyse” to determine the opFmal types given the values in your table –  hXp://dev.mysql.com/doc/refman/5.1/en/procedure-‐analyse.html –  mysql> select * from tab procedure analyse (64,2000) \G

•  Consider the types: –  enum : hXp://dev.mysql.com/doc/refman/5.1/en/enum.html –  set : hXp://dev.mysql.com/doc/refman/5.1/en/set.html

•  Compress large strings –  Use the MySQL COMPRESS and UNCOMPRESS funcFons


InnoDB Tuning

•  Unlike MyISAM -‐ InnoDB uses a single cache for both index and data –  innodb_buffer_pool_size -‐ should be 70-‐80% of available memory. –  It is not uncommon for this to be very large, i.e. 34GB on a system with 40GB of memory – Make sure its not set so large as to cause swapping! –  mysql>show status like 'Innodb_buffer%' ; •  InnoDB can use direct IO on systems that support it -‐ Linux, FreeBSD, and Solaris –  innodb_flush_method = O_DIRECT •  For more InnoDB tuning see –  hXp://dev.mysql.com/doc/refman/5.1/en/innodb-‐tuning-‐troubleshooFng.html

A Virtual MySQL DBA Assistant!

§  Web-‐based, global view of MySQL/Cluster applicaFons (on-‐premise and Cloud deployments)

§  Automated, rules-‐based monitoring and alerts (SMTP, SNMP enabled)

§  Query capture, monitoring, analysis and tuning, correlated with Monitor graphs

§  Real-‐Fme ReplicaFon Monitor with auto-‐discovery of master-‐slave topologies

§  Integrated with Oracle Support

MySQL Enterprise Monitor


Planning For Outages

29

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.




Copyright © 2012, Oracle and/or its affiliates. All rights reserved. September 29th, 2012

The Lowdown on Down0me…. Maintenance Failures Disasters Operator / User

Revenue Customer Sat Employee Regulatory

CAUSE

EFFECT

IMPACT






HA Considera0ons

•  SLA requirements to support business objecFves •  OperaFonal capabiliFes •  Service agility & Fme to market •  Budgetary constraints






Not Everything Needs 99.999% Up0me

•  Recovery Time ObjecFve –  Maximum length of downFme before there is break in “business conFnuity”

•  Recovery Point ObjecFve –  Point in Fme to which data must be recovered when service is re-‐established

•  Tier 1, Mission-‐criFcal services •  Tier 2, Business-‐criFcal services •  Tier 3, Task-‐criFcal services •  Tier 4, Non-‐criFcal services






High Availability Solu0ons

MySQL CGE with Geo-‐ReplicaFon

MySQL Carrier Grade EdiFon (aka MySQL Cluster)

Shared storage cluster

DRBD with clustering sooware

MySQL ReplicaFon with clustering sooware

MySQL ReplicaFon

Best PracFce AdministraFon






Mapping Up0me to Availability

%"9" 9" 9" 9"9"35 days 4 days 50 mins 5 mins 8 hours

Cos

t & C

ompl

exity

SPs & Line of

Business

Web & Cloud

Services

eCommerce Telecoms Military

.

Shared-Nothing, Geo-Replicated

Clusters

Clustered & Virtualized Systems

Replicated Systems






HA Comparison Chart Overall

HA *1

Manual failover

autom. f/o

Active/ Active

RPO *1 RTO *1 No SPOF DR

MySQL 90.000% ü *2 ? ?

Best Practice 95.000% ü *2 1d 2h

MySQL Replication 98.000% ü ü *3 ü *4 1m 30m

MySQL semi-sync Repl. 98.000% ü ü *3 ü *4 0s 30m

DRBD 99.000% ü ü 0s 2m ü DRBD + Geo Repl. 99.500% ü ü 0s 2m ü ü Shared Storage 99.900% ü ü 0s 1m ü *5

Shared Storage + Geo Repl. 99.950% ü ü 0s 1m ü *5 ü MySQL Cluster 99.990% ü ü ü 0s 0,1s ü MySQL Cluster + Geo Repl. 99.999% ü ü ü 0s 0,1s ü ü

*1 Only guidelines *3 Requires addiFonal cluster framework s/w *5 Requires special hardware *2 Requires backup/restore *4 Slaves can be used read-‐only


Upgrading MySQL

37

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 38

MySQL Repo’s YUM -‐ MySQL Yum repository supports the following Linux Distros: Red Hat Enterprise Linux 5,6,7 / Oracle Linux 5,6,7 Fedora 18, 19, and 20 MySQL APT -‐ The MySQL APT repository provides a simple and convenient way to install and update MySQL products with the latest sooware packages using Apt. The APT repository supports the following Linux Distros: Debian – 7, Ubuntu -‐ 12.04 LTS, Ubuntu -‐ 14.04 LTS, Ubuntu -‐ 14.10 MySQL SUSE Repo -‐ The MySQL repository for SUSE Linux provides a simple and convenient way to install and update MySQL products with the latest sooware packages using the Zypper package management tool. The repository at this Fme supports the following distro versions: SLES 12 for x86-‐64 SLES 11.3 for x86-‐64 edelivery.oracle.com – All Commerical binaries for all Oracle Sooware. RPM’s and tarball’s available for most popular distro’s dev.mysql.com – All GPL binaries for MySQL including source tarballs.


Windows Installer


Upgrade Concepts

•  Back up your databases, including the mysql database that contains the grant tables.

•  Read all the notes in the upgrade package. These notes enable you to idenFfy upgrade issues that apply to your current MySQL installaFon. Some incompaFbiliFes discussed in that secFon require your aXenFon before upgrading. Others require some acFon aoer upgrading.

•  Read the Release Notes as well, which provide informaFon about features that are new in MySQL 5.6 or differ from those found in earlier MySQL releases.

•  Aoer upgrading to a new version of MySQL, run mysql_upgrade. This program checks your tables, and aXempts to repair them if necessary. It also updates your grant tables to make sure that they have the current structure so that you can take advantage of any new capabiliFes. (Some releases of MySQL introduce changes to the structure of the grant tables to add new privileges or features.)


Upgrade Best Prac0ces

•  Use virtualizaFon snapshots wherever possible for easy failback •  You can run mulFple versions of MySQL on the same server in different directories, just make a copy of the data files for easy fallback

•  Try not to skip versions ie upgrade from 5.0 -‐> 5.1 > 5.5 > 5.6 etc •  Read the release notes •  Have some sample queries and record the output before and aoer an upgrade.

•  For absolute data integrity you can dump the data out of the old version and then reimport into the new version


Thank You!

42

mysql webinar series 4/4 - manage & tune

Technology

oracle andor

hal stagesqlinit

hal stagesqlcleaning

hal stagesqlstatistics

hal stagesqlpreparing

hal stagesqloptimizing

hal stagesqlend

hal stagesqlexecuting