collaborate 2012 - administering mysql for oracle dbas

Administering MySQL for Oracle DBAs

Eng. Nelson Calero, OCPUYOUG

About me:

http://www.linkedin.com/in/ncalero

Working with Oracle tools and Linux environments since 1996

DBA Oracle (since 2001) & MySQL (since 2005)

Oracle University Instructor since 2011

Co-founder and President of the Oracle user Group of Uruguay (UYOUG) since 2009

Computer Engineer. OCP DBA 10g

Administering MySQL for Oracle DBAs

Uruguay

1 – Brief Intro to main MySQL architectural components2 – Comparison Oracle vs MySQL3 – Installation and Upgrades4 – Security & auditing5 – Performance management6 – Backup & Recovery7 – Designing for High Availability

AGENDA

6/54 http://dev.mysql.com/doc/refman/5.5/en/pluggable-storage-overview.html

1 - Introduction to the MySQL architecture

MySQL server • comes in a Community and Enterprise Edition• has many databases• no schemas• two OS processes when running: mysqld and mysqld_safe• listen on port tcp/ip 3306 by default

• local connections use sockets (similar to beq in Oracle)• creates one thread per each client connection

• thread pool is a commercial feature of 5.5.16 (Oracle shared server) • cost based optimizer• replication is not transactional • cluster is another product, with shared nothing architecture

binaries• installed in /usr/bin if using RPM installation on most *nix• if using binary distribution (.tar), can be placed in custom directories

• Is the way of having different versions on same server

No OFA or similar.

Startup script (/etc/init.d/mysql) has some defaults, changed by configuration file (/etc/my.cnf)

datadir: /var/lib/mysqlbindir: /usr/bin

Internal components

•memory caches { query | innodb buffer | innodb log | myisam key | user | dictionary } cache

• InnoDB undo log - rollback segments •splitted as insert undo and update undo buffer •part of the system tablespace by default•since MySQL 5.6 can be moved: innodb_undo_tablespaces

Storage engines: pluggable architecture that enables different algorithms for handling data.

InnoDB – Transactional. Default from 5.5 MyISAM – non transactional. Default before 5.5 Memory – stored only in memory, not persisted to disk NDB – used by the Cluster Archive/Blackhole/Merge/Federated - other specific use engines

Many third parties implementations XtraDB – from Percona. InnoDB improved

1 - MySQL architectureInternal components

Storage engine files

• table.frm – table definition, regardless of storage engine used ($datadir/dbname)

• InnoDB logs – transactional log, used for crash recovery (ib_logfile1)

• 2 by default. Circular rotation. No archiving

• InnoDB system tablespace – ibdata1

• Stores all data and metadata by default

• Can be changed to external tablespaces: innodb_file_per_table in my.cnf

• Stores each table data in one file named table.ibd

• other tablespaces – used by NDB similarly as Oracle.

Binary logs

• All changes done on the server (called events), in the order they were executed.

• Can be inspected easily with the command mysqlbinlog

• Configurable format: Statement, Row or Mixed

• Needed for replication and point in time recovery

• Have a purge (EXPIRE_LOGS_DAYS) and rotation (MAX_BINLOG_SIZE) policy

More data types, more design choices:

• 9 numeric. Oracle only has one• INT is 4 bytes, TINYINT 1 byte• UNSIGNED: double possible values for autoincrements

•DATE (3 bytes), DATETIME (8 bytes) and TIMESTAMP (4 bytes)

Choosing a data type has more impact on the space used• int(N) – similar to Oracle, does not affect bytes used to store

1 - MySQL architectureDatatypes

Data dictionary

• information_schema: metadata about objects

• performance_schema: (5.5.3) dynamic metrics about server usage (as v$ views)

mysql> SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES -> WHERE TABLE_SCHEMA = 'performance_schema';+----------------------------------------------+| TABLE_NAME |+----------------------------------------------+| cond_instances || events_waits_current || events_waits_history || events_waits_history_long || events_waits_summary_by_instance || events_waits_summary_by_thread_by_event_name || events_waits_summary_global_by_event_name |...

Server variables •Similar to Oracle initialization parameters, change functionality

•Some can be changed dynamically

•Can be persistent changed with startup options or parameter file

•Global/session scope. Global do not affect session who changed it.

mysql root@localhost.information_schema>show variables like 'query%' limit 4;+------------------------------+---------+| Variable_name | Value |+------------------------------+---------+| query_alloc_block_size | 8192 || query_cache_limit | 1048576 || query_cache_min_res_unit | 4096 || query_cache_size | 0 |+------------------------------+---------+

Status variables

•Dynamic counters about server activity

mysql root@localhost.information_schema>show status like 'qc%';+-------------------------+-------+| Variable_name | Value |+-------------------------+-------+| Qcache_free_blocks | 0 || Qcache_free_memory | 0 || Qcache_hits | 0 || Qcache_inserts | 0 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 0 || Qcache_queries_in_cache | 0 |+-------------------------+-------+

Slow query log: can be enabled to record log running queries long_query_time=N (0 to 10)

General log: can be enabled to record all server activity.general_log = 'ON'

Error log: start, stop and error messages.file hostname.err by defaultcan be changed with log_error=file.log in my.cnf

1 - MySQL architectureLog Files

Uses standard GRANT/REVOKE commands

Privileges per host/user/database/table/columngrant select on employees.* to you@'desktop' identified by pwd;

Data stored on mysql database: user/db/*_priv tables

1 - MySQL architectureSecurity

Lock handling: specific of the storage engine

InnoDB – row level

MyISAM – table level

Transactions?

• autocommit enabled by default outside transactions

• default tx_isolation is REPEATABLE-READ

• READ_COMMITED does not work with binlog mode STATEMENT

1 - MySQL architectureLocking and transactions

What happens if the server crashes?• InnoDB has auto recovery (using ib_logfiles)

• MyISAM does not, need manual intervention (myisamchk, repair table)

Some parameters needs to be adjusted if using InnoDB and replication:• InnoDB_flush_log_at_trx_commit

• sync_binlog

• innodb_support_xa

1 - MySQL architectureLocking and transactions

AGENDA

Which database flavor?• Standard? Enterprise? Exadata? Big data appliance?• There are different implementation decisions in every detail

• Initially created for different purposes

Focus on Standard Edition, some picks:• Processes handling• Optimizer features• Locking• Memory management

Pick one Oracle functionality, look for that in MySQL.

2 - Comparison with Oracle architecture

Some SQL differences:•No need for dual. Select without FROM clause works.

•Dual exists just for compatibility•No sequences. autoincrement clause at column definition

•last_insert_id() can show the autoincrement value of last insert•procedures, but no packages and user defined types•multi-record insert•insert delayed•select into outfile / load data file•drop table if exists•partial indexes (column(N)) looks like function based index, but they do not exists

2 - Comparison with OracleSQL

Case insensitive char comparison

mysql root@localhost.employees>select * from employees where first_name='JAANA' limit 3;

+--------+------------+------------+-----------------+--------+------------+| emp_no | birth_date | first_name | last_name | gender | hire_date |+--------+------------+------------+-----------------+--------+------------+| 52681 | 1956-03-28 | Jaana | Besselaar | M | 1986-09-26 || 53990 | 1960-05-26 | Jaana | Cunliffe | M | 1995-07-09 || 54450 | 1954-02-24 | Jaana | Ranon | F | 1988-08-23 |+--------+------------+------------+-----------------+--------+------------+8 rows in set (0.01 sec)

2 - Comparison with Oracle SQL

Silent conversions

mysql root@localhost.employees>create table pru (name varchar(10));Query OK, 0 rows affected (0.19 sec)

mysql root@localhost.employees>insert into pru values ('Jhon'),('Lindenbaumgreen');Query OK, 2 rows affected, 1 warning (0.16 sec)Records: 2 Duplicates: 0 Warnings: 1

mysql root@localhost.employees>select * from pru;+------------+| name |+------------+| Jhon || Lindenbaum |+------------+2 rows in set (0.00 sec)

with SQL_MODE this can be changed:

Mysql> set SQL_MODE=strict_all_tables; Query OK, 0 rows affected (0.00 sec)

Mysql> insert into pru values ('Jhon'),('Lindenbaumgreen'); ERROR 1406 (22001): Data too long for column 'name' at row 2

NOTE: this works because SET uses SESSION scope by default

Many other behaviors can be changed to something Oracle-like:

• SQL_MODE=ORACLE Equivalent to PIPES_AS_CONCAT, ANSI_QUOTES, IGNORE_SPACE, NO_KEY_OPTIONS,

NO_TABLE_OPTIONS, NO_FIELD_OPTIONS, NO_AUTO_CREATE_USER.

• SQL_MODE=TRADITIONAL Equivalent to STRICT_TRANS_TABLES, STRICT_ALL_TABLES, NO_ZERO_IN_DATE,

NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER, and NO_ENGINE_SUBSTITUTION.

• Warning: changing SQL_MODE can hurt tools that expect classic behavior

• Great reference for going deeper on this: "MySQL Idiosyncrasies that BITE" by Ronald Bradford.

2 - Comparison with OracleSQL

Console– mysql– mysqladmin– mysqldump– InnoDB Hot Backup: licensed

– Third partiesPercona Toolkit (former maatkit): many useful script utilitiesOpenArk: morePercona XtraBackup: OpenSource non locking transactional backup

– http://forge.mysql.com/tools/

MySQL Tools

GUI tools

• MySQL Workbench• Data modeling, SQL development, database management, forward and reverse engineering, change management.

• Merge and improvement of old tools from MySQL AB.

• Community edition under GPL

• MySQL Enterprise Monitor• Real time alerting and monitoring solution

• With support contract

MySQL Tools

AGENDA

• Fresh install: rpm -Ivh mysql-server

• Customization?•defaults are for a single installation, single instance per server.

/var/lib/mysql/etc/my.cnf/var/log/mysql/mysqld.log...

• We can create and use our own custom deploy, just to not miss OFA: /u01? /u02/mysql/data

3 - Installation and Upgrade

Default installation•no root user password. Should be used mysql_secure_installation

Autocommit is enabled. If we want to change it:mysql root@localhost.employees>set autocommit=off;Query OK, 0 rows affected (0.00 sec)

3 - Installation

Do not forget about GLOBAL/SESSION variables:

mysql root@localhost.employees>set global autocommit=off; Query OK, 0 rows affected (0.00 sec)

3 - Installation

Also, variables are not persistent when changed with SET:

oraculo:/var/lib/mysql # service mysql restartRestarting service MySQL Shutting down service MySQL doneStarting service MySQL done

mysql root@localhost.(none)>show global variables like '%autocommit%';+---------------+-------+

=> It needs to be changed through startup options, in my.cnf file

3 - Installation

Just run rpm -Uvh?

• First on development environments

• Review changes in the new version, looking for:

• new reserved words that could be in use by our existing tables

• parameters in use deprecated/renamed

• data conversion needed for some of our columns?

• known issues

3 - Upgrade

Effect of upgrading binaries with RPM -Uvh (minor version)oraculo:~ # service mysql startWill update MySQL now, if you encounter any problems, please read following file: /usr/share/doc/packages/mysql-community-server/README.SuSELog files inconsistency, please merge following files manually: /var/log/mysql/mysqld.log /var/lib/mysql/mysqld.logRunning protected MySQL... Upgrading MySQL... Looking for 'mysql' as: /usr/bin/mysqlLooking for 'mysqlcheck' as: /usr/bin/mysqlcheckRunning 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/myRunning 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/mySUELDOS.PARAMETROS_REPORTES OK...Running 'mysql_fix_privilege_tables'...OKStarting service MySQL done

3 - Upgrade

Version after the upgrade:

oraculo:~ # mysql --versionmysql Ver 14.14 Distrib 5.5.21, for Linux (x86_64) using readline 6.1

Worked without issues during last versions (on OpenSUSE 11.4)

5.5.18-73.15.5.18-74.15.5.20-75.15.5.20-785.5.20-80.15.5.21-81.1

3 - Upgrade

AGENDA

• Privilege ALL includes SUPER, which allows to administer the MySQL server. Follow least needed principle, and also avoid using %:

GRANT SELECT (col1), INSERT (col1,col2) ON employees.employee TO 'you'@'desk';

GRANT select on employees.* to you@'%' identified by pwd; GRANT select on *.* to you@'%'; GRANT ALL on *.* to you@'%';

• Each user/host combination defines a unique user

• Vanilla is not possible to block connections to specific users

• Log analysis to have proper auditing in place?• heavily used servers should use replica, TCP or OS mechanisms.

4 - MySQL Security & auditing

Auditing

• No built in functionality

• Can be implemented with triggers, sames as with Oracle

• TIMESTAMP datatype has automatic updating and initialization, no triggers needed

col_name TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP

4 - MySQL Security & auditing

AGENDA

NOs:• way of modify/cheat optimizer statistics as in Oracle• historical repository like AWR – Enterprise Monitor with support contract• limits on server CPU/IO usages• ability to rewrite queries on the fly

Can:• configure many internal memory areas and number of client threads • use hints to force index usage• use external home made solution for query rewrite

5 - Performance Management

• classical unix tools outside the databasevmstat / oprofile / strace / topgdb – poor man's profiler

• inside database• mytop / innotop utilities• explain / explain extended

• before MySQL 5.6.3, subqueries in the FROM clause are executed • status variables

• com_*, innodb_*, connections• information_schema• show engine status / processlist• profiles

mysql >set profiling=1;Query OK, 0 rows affected (0.00 sec)

mysql >select count(1) From employees;+----------+| count(1) |+----------+| 10000 |+----------+1 row in set (0.21 sec)

mysql >show profile for query 1;+----------------------+----------+| Status | Duration |+----------------------+----------+| starting | 0.000142 || checking permissions | 0.000017 || Opening tables | 0.140542 || System lock | 0.000039 || init | 0.000022 || optimizing | 0.000008 || statistics | 0.000011 || preparing | 0.000009 || executing | 0.000005 || Sending data | 0.075795 || end | 0.000018 || query end | 0.000007 || closing tables | 0.000012 || freeing items | 0.000020 || logging slow query | 0.000003 || cleaning up | 0.000005 |

AGENDA

mysqldump – logical backup, engine independent full, database or table

locking based on storage engine used

XtraBackup – open-source hot backup / non-locking tool from Percona

Hotbackup with InnoDB hotbackup: mysqldump --single-transaction --master-data XtraBackup: innobackupex /data/backups

Needs an extra step to prepare prior to use in recovery www.percona.com/doc/percona-xtrabackup/

6 - Backup & Recovery

AGENDA

Replication – built in:• Transfer and apply binary log from master to slaves.• Simple to setup. • Flexible to create cascade configurations.• Can be partial, filtering by DB, tables, and combined.• Asynchronous. Semi-sync in 5.5.• Easy to break. Needs periodical consistency checks.• No conflict resolution. Needs manual intervention when detected.• Not automated failover for HA.• Apply single threaded until 5.6.• Can be configured as circular, but it needs application level coding to avoid inconsistencies.

7 - Designing for High Availability

MySQL Cluster• Different distribution and binaries from MySQL 5.5• Shared nothing architecture: Data nodes, SQL nodes and Manager.• Data is stored redundant among Data nodes• Online operations: backup, upgrade, node provisioning• Memory usage tied to data handled and #nodes in the cluster• 7.2 recent in production with many improvements

Other solutions• SAN/DRBD: Protection from server failures• Pacemaker: Cluster resource manager. Automation of HA among servers.• Galera: Synchronous replication as server plugin. Community and Enterprise.• Tungsten: MultiMaster replication at application level. Community and Enterprise.

7 - Designing for High Availability

• MySQL addresses the same problems as Oracle Database.

• Do not look for same functionality, but ACID and performance.

• Some specific task can be easier (example: Partitioning).

• Need to develop custom scripts for admin tasks, using standard OS tools.

• Big community of users.

• Being FOSS software, source code is available. This allows to overcome lack of specialized tools for specific issues, and depending on your skills you can fix your own problems, and benefit the community.

• Many improvements by other companies (as Percona and Facebook).

Last remarks

Questions?

nelson.calero@gmail.com

Command line could be tuned similar to sqlplus with glogin script?

oracle@oraculo:~> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Tue Feb 28 22:09:00 2012Copyright (c) 1982, 2011, Oracle. All rights reserved.Connected to:Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

22:09:01 SYS@XE>

oracle@oraculo:~> tail $ORACLE_HOME/sqlplus/admin/glogin.sqlset pagesize 200set linesize 120SET TIME ON TIMING ONSET SQLPROMPT '&_user@&_connect_identifier>'

Tip: Better command line prompt

Add PROMPT parameter to /etc/my.cnf

[mysql] no-auto-rehash prompt=mysql \u@\h.\d>

oraculo:~ # mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.5.21-log Source distribution Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql root@localhost.(none)>use information_schema Database changed mysql root@localhost.information_schema>

Tip: Better command line prompt

collaborate 2012 - administering mysql for oracle dbas

mysql architecturehttp

mysql architecturebinaries

server activity mysql

mysql architecturemysql

dba oracle

mysql fororacle dbaseng

oracle tools

oracle dbasabout

Technology

weblogic for dbas 1.0h

a dbas toolbox - doag

mercado de trabalho para dbas

powerpivot for dbas

oracle for sql server dbas

basic mysql troubleshooting for oracle dbas

hadoop databases for oracle dbas

01 the oracle dbas world

oracle rac deep dive for dbas

beginning mysql for non-dbas

exadata management for dbas

storage latency for oracle dbas

dbas, developers and analysts - everyone can benefit...

mysql replication troubleshooting for oracle dbas

postgresql wal for dbas

essential linux commands for dbas

weblogic for dbas

regulatory compliance not just for dbas

future of dbas seminar ireland

a postgresql dbas toolbelt · 2017. 3. 26. · a postgresql...