mysql-performance schema- what's new in mysql-5.7 dmrs

36
MySQL Performance Schema New enhancements in MySQL 5.7 (DMRs) Mayank Prasad Principal Member Technical Staff Oracle, MySQL October 1, 2014 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Upload: mayank-prasad

Post on 23-Jun-2015

405 views

Category:

Software


1 download

DESCRIPTION

I presented these slides in Oracle Open World, 2014. It lists down new features/enhancements done in MySQL-Performance Schema in MySQL 5.7 (DMRs).

TRANSCRIPT

Page 1: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

MySQL Performance SchemaNew enhancements in MySQL 5.7 (DMRs)

Mayank PrasadPrincipal Member Technical StaffOracle, MySQLOctober 1, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Page 2: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2

Page 3: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The Journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

3

4

5

6

3

Page 4: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The Journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

3

4

5

6

4

Page 5: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Why Performance Schema?

SystemLow throughput?

?

5

DBA

Hot table?

Network

High traffic on link?

Storage

Too much disk spin?App Developer

Slow application?

MySQL Developer

Code contention?

End User

Stuck session?

Page 6: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

History and Origin

“The Performance Schema is a mechanism to give user an insight of what is happening behind the scenes when MySQL server is running.”

Development was started by Marc Alff in 2008. He is Chief Architect and Lead of Performance Schema Team (Christopher Powers and me) at Oracle-MySQL.

Introduced in MySQL 5.5.• New storage engine : Performance Schema• New Database : performance_schema• Statistics stored in tables (hard-coded DDLs). • Non-persistent data.• SQL user interface.

6

Page 7: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

MySQL 5.7 Performance Schema : Design

MySQLServer

Instrumentation points(P_S hooks)

P_S Internal Buffers

P_SStorageEngine

StorageEngine

InterfaceStatisticsReport

FetchData

SQL Query

Collect DataStoreData

P_STables

7

Page 8: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

3

4

5

6

8

Page 9: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

The Journey : Then and now..

• MySQL 5.5

• Only waits events

– Mutex waits

– Conditions waits

– R/W locks waits

– File I/O waits

• Setup_(timers/consumers/instruments) tables

• Few summary tables

• 17 Tables and 218 Instruments

• MySQL 5.6

• Statements and stages

• Table, Index and network I/O. Table locks

• Host Cache

• User, account, host

• Setup_(actor/objects)

• Multiple summary tables

• Events’ hierarchy

• 52 Tables and 550Instruments

• MySQL 5.7 (DMRs)

• Instrumentation:

– Transactions

– Metadata locks

– Prepared statements

– Stored programs

– Memory usage

• User variables

• Replication summary tables

• 76 Tables and 885Instruments

• …9

Page 10: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The Journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

4

3

5

6

10

Page 11: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Transaction Instrumentation

• Exposes all aspects of Transactions within server

• Implicit and Explicit

• Quantitative (counts, timing) and Qualitative (isolation level) statistics

• Aggregated across several dimensions (user, account, session etc.)

• Statements to non-transactional tables are ignored (MyISAM)

11

Page 12: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Transaction Instrumentation (cont.)Transactions stats at runtime

* Units in PICOSECONDs

EVENTS_TRANSACTIONS_SUMMARY_BY_THREAD_BY_EVENT_NAME

THREAD_ID 23

EVENT_NAME Transaction

COUNT_STAR 3

SUM_TIMER_WAIT 18558993066000*

… …

COUNT_READ_WRITE 3

SUM_TIMER_READ_WRITE 18558993066000*

… …

EVENTS_TRANSACTIONS_CURRENT

THREAD_ID 23

EVENT_ID 11

EVENT_NAME Transaction

STATE COMMITTED

TIMER_WAIT 22073180000*

ACCESS_MODE READ WRITE

ISOLATION LEVEL REPEATABLE READ

AUTOCOMMIT NO

12

Page 13: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Event Hierarchy

Session

Transaction

Statement*

Stage(closing tables, preparing, optimizing)

Wait(sync, lock, i/o)

* Transactional tables only.

13

Page 14: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Event Hierarchy (cont.)An example

START TRANSACTION;INSERT INTO test.t1 VALUES(‘3’);COMMIT;

14

Page 15: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Event Hierarchy (cont.)An example

START TRANSACTION;INSERT INTO test.t1 VALUES(‘3’);COMMIT;

15

Page 16: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Metadata Locks Instrumentation

METADATA_LOCKS

OBJECT_TYPE: TABLE

OBJECT_SCHEMA: test

OBJECT_NAME: tab1

LOCK_TYPE: SHARED_READ

LOCK_DURATION: TRANSACTION

LOCK_STATUS: GRANTED

OWNER_THREAD_ID: 23

OWNER_EVENT_ID: 41

16

wait/lock/metadata/sql/mdl

• What are MDLs?

• Exposes metadata lock dependencies between session.

• Which session holds MDL

• Which session waiting MDL

• Lock request status

– PENDING, GRANTED, KILLED, TIMEDOUT

Page 17: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Stored Programs Instrumentation

• New objects types in setup_objects.

– PROCEDURE

– FUNCTION

– TRIGGER

– EVENT

• Can be configured individually.

• Nesting is maintained.

Triggers, Events, Functions, Procedures

OBJECT_TYPES

OBJECT_SCHEMA

OBJECT_NAME

ENABLED TIMED

TABLE test tab1 YES NO

PROCEDURE test proc1 NO NO

TRIGGER db1 trg1 YES YES

FUNCTION % % NO NO

EVENT % % NO NO

SETUP_OBJECTS

17

Page 18: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Stored Programs Instrumentation (contd..)Example : create trigger test.trg before insert on test.tab1 for each row

insert into test.tab2 values ('2');

create procedure test.proc ()BEGIN

select count(*) from test.tab1;insert into test.tab1 values('123');

END

call test.proc();

EVENTS_STATEMENTS_SUMMARY_BY_PROGRAM

OBJECT_TYPEOBJECT_SCHEMA

OBJECT_NAME

COUNT_STAR

SUM_TIMER_WAIT

COUNT_STATEMENT

SUM_STATEMENTS_WAIT

SUM_ERRORS

SUM_ROWS_AFFECTED

PROCEDURE test proc 1 21233759000 2 21126786000 0 1

TRIGGER test trg 1 2001201000 1 1976505000 0 1

18

Page 19: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Stored Programs Instrumentation (contd..)Example : (Nesting)

19

EVENTS_STATEMENTS_HISTORY

EVENT_ID 461

SQL_TEXT insert into d1.tab1 values('123')

OBJECT_TYPE PROCEDURE

OBJECT_NAME proc

NESTING_EVENT_ID 442

EVENTS_STATEMENTS_HISTORY

EVENT_ID 469

SQL_TEXT insert into d1.tab2 values ('2')

OBJECT_TYPE TRIGGER

OBJECT_NAME trg

NESTING_EVENT_ID 461

Page 20: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Prepared Statements Instrumentation

• What are Prepared Statements?

• Exposes Prepared Statements in use by server.

• Covers

– PREPARE <stmt>

– mysql_stmt_prepare()

SQLCOM_PREPARE, COM_STMT_PREPARE

20

Page 21: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Prepared Statements Instrumentation (contd.)Example prepare stmt1 from “select * from employee where emp_id=?”;

Set @emp_id=12345;execute stmt1 USING @emp_id;

EVENTS_STATEMENTS_HISTORY_LONG

THREAD_ID: 20

EVENT_ID: 4

EVENT_NAME: statement/sql/prepare_sql

SQL_TEXT: prepare stmt1 from "select * from employee where emp_id=?’;"

PREPARED_STATEMENTS_INSTANCES

STATEMENT_ID: 1

STATEMENT_NAME: stmt1

SQL_TEXT: select * from employee where emp_id=?

OWNER_THREAD_ID: 20

OWNER_EVENT_ID: 4

OWNER_OBJECT_TYPE: NULL

OWNER_OBJECT_NAME: NULL

COUNT_EXECUTE: 1

21

Page 22: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

User Variables

• Exposes user-defined variables

• Max name limit 64

USER_VARIABLES_BY_THREAD

Set @var_1=12345;--------Set @var_2=67890;

USER_VARIABLES_BY_THREAD

THREAD_ID VARIABLE_NAME VARIABLE_VALUE

23 var_1 12345

24 var_2 67890

22

Page 23: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Memory Usage Instrumentation

• Aspects covered :

– Memory size

– Operation Count (1M bytes once vs. 1 byte 1M times)

– Low/high watermarks (spikes)

• Aggregation

– Type of memory (various cache, internal buffers)

– Thread/account/user/host

• Memory behavior of code/choice of design

Who is taking what, how much and how many times.

23

Page 24: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Memory Usages Instrumentation (contd..)MEMORY_SUMMARY_GLOBAL_BY_EVENT_NAME

EVENT_NAME memory/sql/TABLE

COUNT_ALLOC 1381

COUNT_FREE 924

SUM_NUMBER_OF_BYTES_ALLOC 2059873

SUM_NUMBER_OF_BYTES_FREE 1407432

CURRENT_COUNT_USED 457

HIGH_COUNT_USED 461

CURRENT_NUMBER_OF_BYTES_USED 652441

HIGH_NUMBER_OF_BYTES_USED 669269

• Say :

• T1 : 1MB-2MB

• T2 : 10MB-12MB

• Together :

11MB - 14MB

24

Page 25: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Replication Summary TablesTables for replication statistics

25

• SHOW SLAVE STATUS (Limitations)

– Lots of information packed together

– No logical division of information

– No cherry picking (difficult for automation)

– Difficult to scale (more new fields)

• Why Performance Schema Tables?

– SQL Interface. Fetch what is required

– Split logically into different tables

– Easier to extend

Page 26: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Replication Summary TablesTables for replication statistics

26

Replication status

Connection Information Execute Information

Connection Configuration Connection Status

Execute Configuration Execute Status

Coordinator/SQL Thread

Worker Thread

replication_connection_configuration

replication_connection_status

replication_execute_status_by_coordinator

replication_execute_status_by_worker

replication_execute_status

replication_execute_configuration

Page 27: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Replication Summary TablesTables for replication statistics

Table Information

replication_connection_configuration (Host, Port, User etc.)

replication_connection_status (Server UUID, Receiver thread ID, Service State etc.)

replication_execute_configuration (Desired Delay)

replication_execute_status (Remaining Delay)

replication_execute_status_by_coordinator (Thread Id, Service State, Last Error info.)

replication_execute_status_by_worker (WID, WTID, Service State, Last error Info.)

27

Page 28: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The Journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

3

5

4

6

28

Page 29: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Configuration

• Build

• Startup

– Configuration file

– Command line

[mysqld]

performance_schema_consumers_event_waits_history = ON.

performance_schema_events_waits_history_size = 1000.

performance_schema_instruments=‘statement/sql/% = COUNTED’

--performance_schema_consumers_event_waits_history = ON.

--performance_schema_events_waits_history_size = 1000

cmake . -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 -DDISABLE_PSI_MUTEX=1

29

Page 30: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Configuration (contd..)

• Runtime

• Auto tune (value=-1)

– Use heuristics to compute• Max number of objects (mutex, rwlocks, files, sockets, threads etc.)

• Memory for statistical data (history, history_long, digest etc.)

• NOT ALL VARIABLES ARE AUTO TUNED.

– Aligned with server tuning• Use hints from server variables

update setup_consumers set ENABLED=‘NO’ where NAME=‘global_instrumentation’;.

update setup_instruments set ENABLED=‘YES’ where NAME=‘statement/sql/%’;

30

max_connections

table_open_cache

table_definition_cache

open_files_limit

Page 31: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The Journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

3

4

6

5

31

Page 32: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

An Example

• Lets say thread, with thread id T0, is trying to alter a table tab1 and is hung.

• Now, what T0 is waiting for

– LOCK_STATUS=PENDING and OBJECT_NAME=‘tab1’

• Lets see who has taken MDL on table tab1.

– Thread T1 (column OWNER_THREAD_ID) is holding MDL.

• Is T1 waiting for some MDL?

SELECT * FROM metadata_locks WHERE THREAD_ID = T0;

SELECT * FROM metadata_locks WHERE OBJECT_NAME=‘tab1’, LOCK_STATUS=GRANTED;

SELECT * FROM metadata_locks WHERE OWNER_THREAD_ID = T1, LOCK_STATUS=PENDING;

Diagnose a stuck session

32

Page 33: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

An Example (contd…)

• If T1 is not waiting for a MDL, then ?

• What T1 is waiting for

– Say T1 is waiting for mutex_A (column OBJECT_INSTANCE_BEGIN)

• Lets see who has taken this mutex_A

– Ok, so thread T2 is holding mutex_A (column LOCKED_BY_THREAD_ID)

• Find out what thread t2 is waiting for

SELECT * FROM events_waits_current WHERE THREAD_ID = T1;

SELECT * FROM mutex_instances WHERE OBJECT_INSTANCE_BEGIN = mutex_A;

SELECT * FROM events_waits_current WHERE THREAD_ID = T2;

Diagnose a stuck session

33

Page 34: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

References

34

• Blogs:

– http://marcalff.blogspot.com/

– http://www.markleith.co.uk/

– http://mayankmysql.blogspot.in/

• Performance Blog:

– http://dimitrik.free.fr/

• MySQL documentation:

– http://dev.mysql.com/doc/refman/5.7/en/performance-schema.html/

Page 35: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Program Agenda

Need and Origin

The Journey : Then and now …

New in MySQL-5.7 (DMRs)

Configuration

An example

Q&A

1

2

4

5

6

3

35

Page 36: MySQL-Performance Schema- What's new in MySQL-5.7 DMRs

Thank You!

Q&A ?

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |