tokudb troubleshooting - percona.com · tokudb troubleshooting march, 22, 2017 sveta smirnova,...
TRANSCRIPT
![Page 1: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/1.jpg)
TokuDB Troubleshooting
March, 22, 2017
Sveta Smirnova, George O. Lorch III, Vlad Lesin
![Page 2: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/2.jpg)
•What you need to know about TokuDB?•Data Corruption and Inconsistencies•Locks•Performance
Query PerformanceHigh Concurrency Performance
•Instruments
Table of Contents
2
![Page 3: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/3.jpg)
What you need to know about TokuDB?
![Page 4: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/4.jpg)
• Transactional• ACID• MVCC• Row-level locking• XA• Transaction Isolation Levels• No Foreign Keys
Full featured engine
4
![Page 5: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/5.jpg)
• Fast inserts• Bulk loader• Compression• Fractal Tree
Optimized for Writes
5
![Page 6: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/6.jpg)
• Clustered secondary indexes• Indexes can include up to 32 columns• Indexes do not age
Optimizations for Retrievals
6
![Page 7: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/7.jpg)
• OPTIMIZE TABLE• Index creations
• DROP INDEX is offline• Hot column operations
• Addition• Deletion• Expansion• Rename
• Backup
Online Operations
7
![Page 8: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/8.jpg)
• Slave-only feature• Requires RBR on master
• Fast slaves• Must be read-only• Tables must have UNIQUE key• You can use any engine on master
Read Free Replication
8
![Page 9: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/9.jpg)
• SHOW ENGINE TOKUDB STATUS• Detailed lock information• Tables in Information Schema• No Performance Schema Support
• Runtime statistics in SHOW ENGINE TOKUDBSTATUS
• Will be added in future versions
Powerful Diagnostic Tools
9
![Page 10: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/10.jpg)
Data Corruption and Inconsistencies
![Page 11: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/11.jpg)
• Handled automatically at startup• Fixes known inconsistencies• Real corruptions are not reparable!
• You interfere this process only slightly• Backup restoration required when corruption
cannot be cured automatically• Disable write cache for safety• Verify your backups!
How TokuDB Deals with Data Corruption?
11
![Page 12: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/12.jpg)
• Handled automatically at startup• You interfere this process only slightly
• There is no option, similar toinnodb force recovery
• Feature Request FT-733• tokudb strip frm data removes embedded .frm
metadata from status filesShould be used with great care!
• Backup restoration required when corruptioncannot be cured automatically
• Disable write cache for safety• Verify your backups!
How TokuDB Deals with Data Corruption?
11
![Page 13: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/13.jpg)
• Handled automatically at startup• You interfere this process only slightly• Backup restoration required when corruption
cannot be cured automatically
• Disable write cache for safety• Verify your backups!
How TokuDB Deals with Data Corruption?
11
![Page 14: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/14.jpg)
• Handled automatically at startup• You interfere this process only slightly• Backup restoration required when corruption
cannot be cured automatically• Disable write cache for safety
• Verify your backups!
How TokuDB Deals with Data Corruption?
11
![Page 15: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/15.jpg)
• Handled automatically at startup• You interfere this process only slightly• Backup restoration required when corruption
cannot be cured automatically• Disable write cache for safety• Verify your backups!
How TokuDB Deals with Data Corruption?
11
![Page 16: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/16.jpg)
• What are Symptoms of Real Corruption?• Assertions about bad checksums• Assertions while unpaking row data• Not recoverable
• What are Symptoms of Inconsistency?• Different query results for Primary vs secondary
key access?
Corruptions vs Inonsistencies
12
![Page 17: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/17.jpg)
• What are Symptoms of Real Corruption?• Assertions about bad checksums• Assertions while unpaking row data• Not recoverable
• What are Symptoms of Inconsistency?• Different query results for Primary vs secondary
key access?
Corruptions vs Inonsistencies
12
![Page 18: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/18.jpg)
• Are inconsistency limited to know key range?• Recoverable
• Is it not possible to limit inconsistency toknown range?
• There can be some data ”loss” or ”gain”
Automatic Inconsistencies Recovery
13
![Page 19: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/19.jpg)
• Are inconsistency limited to know key range?• Is it not possible to limit inconsistency to
known range?• Not recoverable• Use dump-reload
• There can be some data ”loss” or ”gain”
Automatic Inconsistencies Recovery
13
![Page 20: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/20.jpg)
• Are inconsistency limited to know key range?• Is it not possible to limit inconsistency to
known range?• There can be some data ”loss” or ”gain”
• If Primary key misses data• If Primary key still contains ”deleted” rows
Automatic Inconsistencies Recovery
13
![Page 21: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/21.jpg)
• TokuDB file formats are not the same acrossMySQL variants
• Percona Server 5.6 and 5.7• MariaDB 10.x• Builds, released by Tokutek
On Oracle MySQLOn MariaDB
• Migrating will require logical dump and reload• Upgrade from Percona Server 5.6 to Percona
Server 5.7 can be in place
File Formats
14
![Page 22: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/22.jpg)
• TokuDB file formats are not the same acrossMySQL variants
• Percona Server 5.6 and 5.7• MariaDB 10.x• Builds, released by Tokutek
On Oracle MySQLOn MariaDB
• Migrating will require logical dump and reload• Upgrade from Percona Server 5.6 to Percona
Server 5.7 can be in place
File Formats
14
![Page 23: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/23.jpg)
• Can cause InnoDB corruptions withinnodb use native aio=1
• Cannot be used to create new slave• Work in progress to solve this
TokuDB Hotbackup
15
![Page 24: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/24.jpg)
• Can cause InnoDB corruptions withinnodb use native aio=1
• Cannot be used to create new slave• Work in progress to solve this
TokuDB Hotbackup
15
![Page 25: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/25.jpg)
Locks
![Page 26: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/26.jpg)
• Row-level• No Auto Increment lock tuning• Table locks for table definition modifications
Locks in TokuDB
17
![Page 27: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/27.jpg)
• Lock wait timeout• session1> START TRANSACTION;
• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 28: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/28.jpg)
• Lock wait timeout• session1> START TRANSACTION;• session1> UPDATE t SET f=’foo’ WHERE id=12345;
• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 29: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/29.jpg)
• Lock wait timeout• session1> START TRANSACTION;• session1> UPDATE t SET f=’foo’ WHERE id=12345;• session2> UPDATE t SET f=’bar’ WHERE id=12345;
• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 30: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/30.jpg)
• Lock wait timeout• Deadlocks
• session1> START TRANSACTION;
• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 31: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/31.jpg)
• Lock wait timeout• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;
• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 32: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/32.jpg)
• Lock wait timeout• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;
• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 33: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/33.jpg)
• Lock wait timeout• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;
• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 34: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/34.jpg)
• Lock wait timeout• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;
• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 35: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/35.jpg)
• Lock wait timeout• Deadlocks
• session1> START TRANSACTION;• session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;• session2> START TRANSACTION;• session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;• session1> UPDATE t SET f=’foo’ WHERE id=54321;• session2> UPDATE t SET f=’foo’ WHERE id=12345;
Which Issues Will We Discuss?
18
![Page 36: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/36.jpg)
• Information Schema tables• TOKUDB TRX• TOKUDB LOCKS• TOKUDB LOCK WAITS
• Variable tokudb lock timeout debug• Variable tokudb last lock timeout
• Contains JSON document, described latest lockconflict for the session
Locks Diagnostics
19
![Page 37: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/37.jpg)
• Information Schema tables• Variable tokudb lock timeout debug
• Controls how lock timeouts and deadlocks arereported
0 - Nothing reported1 - JSON in tokudb last lock timeout variable2 - JSON in error log3 - JSON in tokudb last lock timeout and error log
• Session variable!
• Variable tokudb last lock timeout• Contains JSON document, described latest lock
conflict for the session
Locks Diagnostics
19
![Page 38: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/38.jpg)
• Information Schema tables• Variable tokudb lock timeout debug• Variable tokudb last lock timeout
• Contains JSON document, described latest lockconflict for the session
Locks Diagnostics
19
![Page 39: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/39.jpg)
• Sessions
• Error log2017-03-12T13:18:17.758148Z 336 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":336, "dbname":"./test/t-main", "requesting_txnid":48, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:18:17.758206Z 336 [ERROR] TokuDB:
requesting_thread_id:336 q:UPDATE t SET f=’bar’ WHERE id=12345
By example: lock wait timeout
20
![Page 40: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/40.jpg)
• Sessionssession1> START TRANSACTION;
session1> UPDATE t SET f=’foo’ WHERE id=12345;
• Error log2017-03-12T13:18:17.758148Z 336 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":336, "dbname":"./test/t-main", "requesting_txnid":48, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:18:17.758206Z 336 [ERROR] TokuDB:
requesting_thread_id:336 q:UPDATE t SET f=’bar’ WHERE id=12345
By example: lock wait timeout
20
![Page 41: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/41.jpg)
• Sessionssession1> START TRANSACTION;
session1> UPDATE t SET f=’foo’ WHERE id=12345;
session2> UPDATE t SET f=’bar’ WHERE id=12345;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
• Error log2017-03-12T13:18:17.758148Z 336 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":336, "dbname":"./test/t-main", "requesting_txnid":48, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:18:17.758206Z 336 [ERROR] TokuDB:
requesting_thread_id:336 q:UPDATE t SET f=’bar’ WHERE id=12345
By example: lock wait timeout
20
![Page 42: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/42.jpg)
• Sessionssession1> START TRANSACTION;
session1> UPDATE t SET f=’foo’ WHERE id=12345;
session2> UPDATE t SET f=’bar’ WHERE id=12345;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
session1> select @@tokudb_last_lock_timeout\G
*************************** 1. row ***************************
@@tokudb_last_lock_timeout: NULL
1 row in set (0.00 sec)
• Error log2017-03-12T13:18:17.758148Z 336 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":336, "dbname":"./test/t-main", "requesting_txnid":48, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:18:17.758206Z 336 [ERROR] TokuDB:
requesting_thread_id:336 q:UPDATE t SET f=’bar’ WHERE id=12345
By example: lock wait timeout
20
![Page 43: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/43.jpg)
• Sessionssession1> START TRANSACTION;
session1> UPDATE t SET f=’foo’ WHERE id=12345;
session2> UPDATE t SET f=’bar’ WHERE id=12345;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
session1> select @@tokudb_last_lock_timeout\G
*************************** 1. row ***************************
@@tokudb_last_lock_timeout: NULL
1 row in set (0.00 sec)
session2> select @@tokudb_last_lock_timeout\G
*************************** 1. row ***************************
@@tokudb_last_lock_timeout: {"mysql_thread_id":336, "dbname":"./test/t-main",
"requesting_txnid":16, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
1 row in set (0.00 sec)
• Error log2017-03-12T13:18:17.758148Z 336 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":336, "dbname":"./test/t-main", "requesting_txnid":48, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:18:17.758206Z 336 [ERROR] TokuDB:
requesting_thread_id:336 q:UPDATE t SET f=’bar’ WHERE id=12345
By example: lock wait timeout
20
![Page 44: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/44.jpg)
• Sessions• Error log
2017-03-12T13:18:17.758148Z 336 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":336, "dbname":"./test/t-main", "requesting_txnid":48, "blocking_txnid":13,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:18:17.758206Z 336 [ERROR] TokuDB:
requesting_thread_id:336 q:UPDATE t SET f=’bar’ WHERE id=12345
By example: lock wait timeout
20
![Page 45: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/45.jpg)
• Sessionssession1> START TRANSACTION;
session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;
• Error log2017-03-12T13:45:00.551794Z 2688 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:45:00.551827Z 2688 [ERROR] TokuDB:
requesting_thread_id:2688 q:UPDATE t SET f=’bar’ WHERE id=12345
2017-03-12T13:45:03.863714Z 2685 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80,
"key_left":"ff31d40000", "key_right":"0131d40000"}
2017-03-12T13:45:03.863771Z 2685 [ERROR] TokuDB:
requesting_thread_id:2685 q:UPDATE t SET f=’foo’ WHERE id=54321
By example: Deadlock
21
![Page 46: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/46.jpg)
• Sessionssession1> START TRANSACTION;
session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;
session2> START TRANSACTION;
session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;
• Error log2017-03-12T13:45:00.551794Z 2688 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:45:00.551827Z 2688 [ERROR] TokuDB:
requesting_thread_id:2688 q:UPDATE t SET f=’bar’ WHERE id=12345
2017-03-12T13:45:03.863714Z 2685 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80,
"key_left":"ff31d40000", "key_right":"0131d40000"}
2017-03-12T13:45:03.863771Z 2685 [ERROR] TokuDB:
requesting_thread_id:2685 q:UPDATE t SET f=’foo’ WHERE id=54321
By example: Deadlock
21
![Page 47: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/47.jpg)
• Sessionssession1> START TRANSACTION;
session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;
session2> START TRANSACTION;
session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;
session1> UPDATE t SET f=’foo’ WHERE id=54321;
• Error log2017-03-12T13:45:00.551794Z 2688 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:45:00.551827Z 2688 [ERROR] TokuDB:
requesting_thread_id:2688 q:UPDATE t SET f=’bar’ WHERE id=12345
2017-03-12T13:45:03.863714Z 2685 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80,
"key_left":"ff31d40000", "key_right":"0131d40000"}
2017-03-12T13:45:03.863771Z 2685 [ERROR] TokuDB:
requesting_thread_id:2685 q:UPDATE t SET f=’foo’ WHERE id=54321
By example: Deadlock
21
![Page 48: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/48.jpg)
• Sessionssession1> START TRANSACTION;
session1> SELECT * FROM t WHERE id=12345 FOR UPDATE;
session2> START TRANSACTION;
session2> SELECT * FROM t WHERE id=54321 FOR UPDATE;
session1> UPDATE t SET f=’foo’ WHERE id=54321;
session2> UPDATE t SET f=’foo’ WHERE id=12345;
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
session2> select @@tokudb_last_lock_timeout\G
*************************** 1. row ***************************
@@tokudb_last_lock_timeout: {"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78, "key_left":"ff39300000", "key_right":"0139300000"}
1 row in set (0.00 sec)
• Error log2017-03-12T13:45:00.551794Z 2688 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:45:00.551827Z 2688 [ERROR] TokuDB:
requesting_thread_id:2688 q:UPDATE t SET f=’bar’ WHERE id=12345
2017-03-12T13:45:03.863714Z 2685 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80,
"key_left":"ff31d40000", "key_right":"0131d40000"}
2017-03-12T13:45:03.863771Z 2685 [ERROR] TokuDB:
requesting_thread_id:2685 q:UPDATE t SET f=’foo’ WHERE id=54321
By example: Deadlock
21
![Page 49: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/49.jpg)
• Sessionssession2> UPDATE t SET f=’foo’ WHERE id=12345;
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
session2> select @@tokudb_last_lock_timeout\G
*************************** 1. row ***************************
@@tokudb_last_lock_timeout: {"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78, "key_left":"ff39300000", "key_right":"0139300000"}
1 row in set (0.00 sec)
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
session1> select @@tokudb_last_lock_timeout\G
*************************** 1. row ***************************
@@tokudb_last_lock_timeout: {"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80, "key_left":"ff31d40000", "key_right":"0131d40000"}
1 row in set (0.00 sec)
• Error log2017-03-12T13:45:00.551794Z 2688 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:45:00.551827Z 2688 [ERROR] TokuDB:
requesting_thread_id:2688 q:UPDATE t SET f=’bar’ WHERE id=12345
2017-03-12T13:45:03.863714Z 2685 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80,
"key_left":"ff31d40000", "key_right":"0131d40000"}
2017-03-12T13:45:03.863771Z 2685 [ERROR] TokuDB:
requesting_thread_id:2685 q:UPDATE t SET f=’foo’ WHERE id=54321
By example: Deadlock
21
![Page 50: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/50.jpg)
• Sessions• Error log
2017-03-12T13:45:00.551794Z 2688 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2688, "dbname":"./test/t-main",
"requesting_txnid":80, "blocking_txnid":78,
"key_left":"ff39300000", "key_right":"0139300000"}
2017-03-12T13:45:00.551827Z 2688 [ERROR] TokuDB:
requesting_thread_id:2688 q:UPDATE t SET f=’bar’ WHERE id=12345
2017-03-12T13:45:03.863714Z 2685 [ERROR] TokuDB: lock timeout
{"mysql_thread_id":2685, "dbname":"./test/t-main",
"requesting_txnid":78, "blocking_txnid":80,
"key_left":"ff31d40000", "key_right":"0131d40000"}
2017-03-12T13:45:03.863771Z 2685 [ERROR] TokuDB:
requesting_thread_id:2685 q:UPDATE t SET f=’foo’ WHERE id=54321
By example: Deadlock
21
![Page 51: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/51.jpg)
information schema> SELECT * FROM TOKUDB_TRX;
+--------+---------------------+----------+
| trx_id | trx_mysql_thread_id | trx_time |
+--------+---------------------+----------+
| 126 | 4135 | 70 |
| 128 | 4141 | 61 |
+--------+---------------------+----------+
2 rows in set (0.00 sec)
All TokuDB Transactions
22
![Page 52: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/52.jpg)
information schema> SELECT * FROM TOKUDB_LOCKS\G
*************************** 1. row ***************************
locks_trx_id: 126
locks_mysql_thread_id: 4135
locks_dname: ./test/t-main
locks_key_left: ff39300000
locks_key_right: 0039300000
locks_table_schema: test
locks_table_name: t
locks_table_dictionary_name: main
*************************** 2. row ***************************
locks_trx_id: 128
locks_mysql_thread_id: 4141
locks_dname: ./test/t-main
locks_key_left: ff31d40000
locks_key_right: 0031d40000
locks_table_schema: test
locks_table_name: t
locks_table_dictionary_name: main
2 rows in set (0.00 sec)
All TokuDB Locks
23
![Page 53: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/53.jpg)
information schema> SELECT * FROM TOKUDB_LOCK_WAITS\G
*************************** 1. row ***************************
requesting_trx_id: 146
blocking_trx_id: 128
lock_waits_dname: ./test/t-main
lock_waits_key_left: ff31d40000
lock_waits_key_right: 0131d40000
lock_waits_start_time: 1489327019006
lock_waits_table_schema: test
lock_waits_table_name: t
lock_waits_table_dictionary_name: main
1 row in set (0.00 sec)
TokuDB Lock Waits
24
![Page 54: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/54.jpg)
• Not supported• No error returned• Clause silently ignored!• Behaves as regular read lock for SELECT• Bug #1672532
LOCK IN SHARE MODE
25
![Page 55: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/55.jpg)
• TokuDB assumes that if some transaction isrunning lock retry, others do not need to run it
• They wait for timeout instead• With low number of connections you will sleep
when you may do useful job• Slow performance• Pull request to fix it
Lock races
26
![Page 56: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/56.jpg)
• TokuDB assumes that if some transaction isrunning lock retry, others do not need to run it
• They wait for timeout instead
• With low number of connections you will sleepwhen you may do useful job
• Slow performance• Pull request to fix it
Lock races
26
![Page 57: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/57.jpg)
• TokuDB assumes that if some transaction isrunning lock retry, others do not need to run it
• They wait for timeout instead• With low number of connections you will sleep
when you may do useful job
• Slow performance• Pull request to fix it
Lock races
26
![Page 58: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/58.jpg)
• TokuDB assumes that if some transaction isrunning lock retry, others do not need to run it
• They wait for timeout instead• With low number of connections you will sleep
when you may do useful job• Slow performance
• Pull request to fix it
Lock races
26
![Page 59: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/59.jpg)
• TokuDB assumes that if some transaction isrunning lock retry, others do not need to run it
• They wait for timeout instead• With low number of connections you will sleep
when you may do useful job• Slow performance• Pull request to fix it
Lock races
26
![Page 60: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/60.jpg)
Performance
![Page 61: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/61.jpg)
PerformanceQuery Performance
![Page 62: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/62.jpg)
create clustering index birth date on employees(birth date);
alter table employees add clustering index (birth date);
• Contains copy of data• Practically ordered copy of the table• Covered index for any kind of query• Cannot be unique
Clustering Secondary Indexes
29
![Page 63: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/63.jpg)
• Contains copy of data
• Practically ordered copy of the table• Covered index for any kind of query• Cannot be unique
Clustering Secondary Indexes
29
![Page 64: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/64.jpg)
• Contains copy of data• Practically ordered copy of the table
• Covered index for any kind of query• Cannot be unique
Clustering Secondary Indexes
29
![Page 65: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/65.jpg)
• Contains copy of data• Practically ordered copy of the table• Covered index for any kind of query
• ”Regular” indexsession1> create index birth_date on employees(birth_date);
Query OK, 0 rows affected (4.99 sec)
Records: 0 Duplicates: 0 Warnings: 0
session1> select first_name, last_name, birth_date from employees
-> where birth_date > ’1960-01-01’ order by birth_date desc;
...
117075 rows in set (1.27 sec)
• Cannot be unique
Clustering Secondary Indexes
29
![Page 66: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/66.jpg)
• Contains copy of data• Practically ordered copy of the table• Covered index for any kind of query
• Clustering indexsession1> create clustering index birth_date on employees(birth_date);
Query OK, 0 rows affected (5.96 sec)
Records: 0 Duplicates: 0 Warnings: 0
session1> select first_name, last_name, birth_date from employees
-> where birth_date > ’1960-01-01’;
...
117075 rows in set (0.69 sec)
• Cannot be unique
Clustering Secondary Indexes
29
![Page 67: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/67.jpg)
• Contains copy of data• Practically ordered copy of the table• Covered index for any kind of query• Cannot be unique
Clustering Secondary Indexes
29
![Page 68: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/68.jpg)
• Prior to 5.6.27-76.0• ANALYZE TABLE in foreground
• Variable tokudb analyze in background• Controlled by variable tokudb analyze mode• How ANALYZE operation is aggressive• Variable tokudb auto analyze• Variable tokudb cardinality scale percent
Index Statistics
30
![Page 69: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/69.jpg)
• Prior to 5.6.27-76.0• Variable tokudb analyze in background
• ANALYZE TABLE in background• Immediate return
• Controlled by variable tokudb analyze mode• How ANALYZE operation is aggressive• Variable tokudb auto analyze• Variable tokudb cardinality scale percent
Index Statistics
30
![Page 70: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/70.jpg)
• Prior to 5.6.27-76.0• Variable tokudb analyze in background• Controlled by variable tokudb analyze mode
• TOKUDB ANALYZE CANCEL• TOKUDB ANALYZE STANDARD• TOKUDB ANALYZE RECOUNT ROWS
• How ANALYZE operation is aggressive• Variable tokudb auto analyze• Variable tokudb cardinality scale percent
Index Statistics
30
![Page 71: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/71.jpg)
• Prior to 5.6.27-76.0• Variable tokudb analyze in background• Controlled by variable tokudb analyze mode• How ANALYZE operation is aggressive
• tokudb analyze throttle: maximum number ofkeys to visit per second
• tokudb analyze time: number of seconds tospend on each index when calculating cardinality
• Variable tokudb auto analyze• Variable tokudb cardinality scale percent
Index Statistics
30
![Page 72: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/72.jpg)
• Prior to 5.6.27-76.0• Variable tokudb analyze in background• Controlled by variable tokudb analyze mode• How ANALYZE operation is aggressive• Variable tokudb auto analyze
• Fully automatic ANALYZE• Similar to innodb stats auto recalc• Enabled in thread context which triggered
analysis
• Variable tokudb cardinality scale percent
Index Statistics
30
![Page 73: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/73.jpg)
• Prior to 5.6.27-76.0• Variable tokudb analyze in background• Controlled by variable tokudb analyze mode• How ANALYZE operation is aggressive• Variable tokudb auto analyze• Variable tokudb cardinality scale percent
• Tells server how index is unique• Hardcoded in InnoDB: 50%
Index Statistics
30
![Page 74: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/74.jpg)
• Not all ICP conditions are implemented
• No mechanism to pass information about ICPusage back to the server
• You will not see ”Using index condition” forTokuDB even if optimization used
• Descending range scans were pain before5.6.35/5.7.17
•SELECT id FROM t1 WHERE id BETWEEN 5 AND 5000 ORDER BY id DESC
• Bug #1672871
Index Condition Pushdown
31
![Page 75: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/75.jpg)
• Not all ICP conditions are implemented• No mechanism to pass information about ICP
usage back to the server• You will not see ”Using index condition” for
TokuDB even if optimization used
• Descending range scans were pain before5.6.35/5.7.17
•SELECT id FROM t1 WHERE id BETWEEN 5 AND 5000 ORDER BY id DESC
• Bug #1672871
Index Condition Pushdown
31
![Page 76: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/76.jpg)
• Not all ICP conditions are implemented• No mechanism to pass information about ICP
usage back to the server• You will not see ”Using index condition” for
TokuDB even if optimization used• Descending range scans were pain before
5.6.35/5.7.17•
SELECT id FROM t1 WHERE id BETWEEN 5 AND 5000 ORDER BY id DESC
• Bug #1672871
Index Condition Pushdown
31
![Page 77: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/77.jpg)
• Two block sets• Crash safe
• OPTIMIZE TABLE changes nodes• Nodes are flushed into new block set• Old block set still exists• Size of index doubles• These bytes will be re-used
• Kind of pre-allocation
OPTIMIZE TABLE
32
![Page 78: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/78.jpg)
• Two block sets• Crash safe
• OPTIMIZE TABLE changes nodes
• Nodes are flushed into new block set• Old block set still exists• Size of index doubles• These bytes will be re-used
• Kind of pre-allocation
OPTIMIZE TABLE
32
![Page 79: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/79.jpg)
• Two block sets• Crash safe
• OPTIMIZE TABLE changes nodes• Nodes are flushed into new block set
• Old block set still exists• Size of index doubles• These bytes will be re-used
• Kind of pre-allocation
OPTIMIZE TABLE
32
![Page 80: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/80.jpg)
• Two block sets• Crash safe
• OPTIMIZE TABLE changes nodes• Nodes are flushed into new block set• Old block set still exists
• Size of index doubles• These bytes will be re-used
• Kind of pre-allocation
OPTIMIZE TABLE
32
![Page 81: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/81.jpg)
• Two block sets• Crash safe
• OPTIMIZE TABLE changes nodes• Nodes are flushed into new block set• Old block set still exists• Size of index doubles
• These bytes will be re-used• Kind of pre-allocation
OPTIMIZE TABLE
32
![Page 82: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/82.jpg)
• Two block sets• Crash safe
• OPTIMIZE TABLE changes nodes• Nodes are flushed into new block set• Old block set still exists• Size of index doubles• These bytes will be re-used
• Kind of pre-allocation
OPTIMIZE TABLE
32
![Page 83: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/83.jpg)
• Special syntaxmysql> SET tokudb_create_index_online=on;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE INDEX index ON table (field_name);
• ALTER implies offline index creation• Watch progress in SHOW PROCESSLIST
session2> show processlist\G
*************************** 1. row ***************************
Id: 424
User: root
Host: localhost:40422
db: employees
Command: Query
Time: 7
State: Adding of indexes about 98.4% done
Info: create index title on titles(title)
Rows_sent: 0
Rows_examined: 0
Online CREATE INDEX
33
![Page 84: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/84.jpg)
• Special syntax• ALTER implies offline index creation
• Watch progress in SHOW PROCESSLISTsession2> show processlist\G
*************************** 1. row ***************************
Id: 424
User: root
Host: localhost:40422
db: employees
Command: Query
Time: 7
State: Adding of indexes about 98.4% done
Info: create index title on titles(title)
Rows_sent: 0
Rows_examined: 0
Online CREATE INDEX
33
![Page 85: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/85.jpg)
• Special syntax• ALTER implies offline index creation• Watch progress in SHOW PROCESSLIST
session2> show processlist\G
*************************** 1. row ***************************
Id: 424
User: root
Host: localhost:40422
db: employees
Command: Query
Time: 7
State: Adding of indexes about 98.4% done
Info: create index title on titles(title)
Rows_sent: 0
Rows_examined: 0
Online CREATE INDEX
33
![Page 86: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/86.jpg)
• Read progresssession2> show processlist\G
*************************** 1. row ***************************
Id: 424
User: root
Host: localhost:40422
db: employees
Command: Query
Time: 1
State: Queried about 300000 rows
Info: select title, count(*) from titles group by title
Rows_sent: 0
Rows_examined: 0
• Write progresssession2> show processlist\G
*************************** 1. row ***************************
Id: 424
User: root
Host: localhost:40422
db: employees
Command: Query
Time: 6
State: Queried about 443308 rows, Updated about 53000 rows
Info: update titles set title=’Senior Cat’ where title=’Senior Engineer’
Rows_sent: 0
Rows_examined: 496933
Query progress
34
![Page 87: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/87.jpg)
• Read progress• Write progress
session2> show processlist\G
*************************** 1. row ***************************
Id: 424
User: root
Host: localhost:40422
db: employees
Command: Query
Time: 6
State: Queried about 443308 rows, Updated about 53000 rows
Info: update titles set title=’Senior Cat’ where title=’Senior Engineer’
Rows_sent: 0
Rows_examined: 496933
Query progress
34
![Page 88: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/88.jpg)
PerformanceHigh Concurrency Performance
![Page 89: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/89.jpg)
• Checkpointing
• Read heavy workloads• Fractal Tree map file fragmentation• Concurrent queries on single table
Main Factors which Affect Performance
36
![Page 90: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/90.jpg)
• Checkpointing• Read heavy workloads
• OS cache with high compression levels may workwell
• TokuDB is write optimized!
• Fractal Tree map file fragmentation• Concurrent queries on single table
Main Factors which Affect Performance
36
![Page 91: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/91.jpg)
• Checkpointing• Read heavy workloads• Fractal Tree map file fragmentation
• Block allocator algorithm• Improved in newer versions
Old versions: linear searchNew versions: MHS algorithmDetails
• Concurrent queries on single table
Main Factors which Affect Performance
36
![Page 92: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/92.jpg)
• Checkpointing• Read heavy workloads• Fractal Tree map file fragmentation• Concurrent queries on single table
Main Factors which Affect Performance
36
![Page 93: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/93.jpg)
• Flushing down dirty pages in background
• Fuzzy checkpointing (as in InnoDB)• Sharp checkpoining (as in TokuDB)
TokuDB Sharp Checkpointing
37
![Page 94: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/94.jpg)
• Flushing down dirty pages in background• Fuzzy checkpointing (as in InnoDB)
• Flushes dirty pages when server is idle• Or if innodb max dirty pages pct reached
• Sharp checkpoining (as in TokuDB)
TokuDB Sharp Checkpointing
37
![Page 95: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/95.jpg)
• Flushing down dirty pages in background• Fuzzy checkpointing (as in InnoDB)• Sharp checkpoining (as in TokuDB)
• It will checkpoint at some time period no matterwhat the current workload is
• May lead to visible stalls in user activity• Variables tokudb checkpointing period and
tokudb cache size control how oftencheckpointing will happen
TokuDB Sharp Checkpointing
37
![Page 96: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/96.jpg)
• Fractal Tree design• Fractal Tree buffers modifications in intermediate
locations• These buffered modifications should be
eventually flushed into index files• While they are not flushed read operations have
to search through all not applied stages
• Write locks have higher priority than read locks• Trx 1 blocks a row with read lock
• Trx 2 requests write lock on this row• Trx 3 requests read lock on the row• TokuDB: Trx 3 will wait when Trx 2 releases lock
Read Heavy Workloads
38
![Page 97: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/97.jpg)
• Fractal Tree design• Write locks have higher priority than read locks
• Trx 1 blocks a row with read lock
• Trx 2 requests write lock on this row• Trx 3 requests read lock on the row• TokuDB: Trx 3 will wait when Trx 2 releases lock
Read Heavy Workloads
38
![Page 98: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/98.jpg)
• Fractal Tree design• Write locks have higher priority than read locks
• Trx 1 blocks a row with read lock• Trx 2 requests write lock on this row
• Trx 3 requests read lock on the row• TokuDB: Trx 3 will wait when Trx 2 releases lock
Read Heavy Workloads
38
![Page 99: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/99.jpg)
• Fractal Tree design• Write locks have higher priority than read locks
• Trx 1 blocks a row with read lock• Trx 2 requests write lock on this row• Trx 3 requests read lock on the row
• TokuDB: Trx 3 will wait when Trx 2 releases lock
Read Heavy Workloads
38
![Page 100: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/100.jpg)
• Fractal Tree design• Write locks have higher priority than read locks
• Trx 1 blocks a row with read lock• Trx 2 requests write lock on this row• Trx 3 requests read lock on the row• InnoDB: Trx 3 receives access immediately
• TokuDB: Trx 3 will wait when Trx 2 releases lock
Read Heavy Workloads
38
![Page 101: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/101.jpg)
• Fractal Tree design• Write locks have higher priority than read locks
• Trx 1 blocks a row with read lock• Trx 2 requests write lock on this row• Trx 3 requests read lock on the row• TokuDB: Trx 3 will wait when Trx 2 releases lock
Read Heavy Workloads
38
![Page 102: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/102.jpg)
Instruments
![Page 103: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/103.jpg)
• TokuDB trxsession2> select * from TokuDB_trx\G
*************************** 1. row ***************************
trx_id: 4022
trx_mysql_thread_id: 424
trx_time: 7
1 row in set (0.00 sec)
Tables in Information Schema
40
![Page 104: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/104.jpg)
• TokuDB lockssession2> select * from TokuDB_locks \G
*************************** 1. row ***************************
locks_trx_id: 4022
locks_mysql_thread_id: 424
locks_dname: ./employees/titles-main
locks_key_left: 00a16304000a53656e696f722043617479840f
locks_key_right: 00a16304000a53656e696f722043617479840f
locks_table_schema: employees
locks_table_name: titles
locks_table_dictionary_name: main
...
Tables in Information Schema
40
![Page 105: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/105.jpg)
• TokuDB lock waitsmysql> select * from information_schema.tokudb_lock_waits\G
*************************** 1. row ***************************
requesting_trx_id: 4115
blocking_trx_id: 4022
lock_waits_dname: ./employees/titles-main
lock_waits_key_left: -infinity
lock_waits_key_right: +infinity
lock_waits_start_time: 1489526456023
lock_waits_table_schema: employees
lock_waits_table_name: titles
lock_waits_table_dictionary_name: main
1 row in set (0.00 sec)
Tables in Information Schema
40
![Page 106: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/106.jpg)
• TokuDB background job statussession1> select * from information_schema.TokuDB_background_job_status\G
*************************** 1. row ***************************
id: 35
database_name: employees
table_name: titles
job_type: TOKUDB_ANALYZE_MODE_RECOUNT_ROWS
job_params: TOKUDB_ANALYZE_THROTTLE=0;
scheduler: USER
scheduled_time: 2017-03-15 00:37:10
started_time: 2017-03-15 00:37:10
status: recount_rows employees.titles counted 230230 rows and 0 deleted in 0 seconds.
1 row in set (0.00 sec)
Tables in Information Schema
40
![Page 107: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/107.jpg)
• TokuDB file mapsession1> show create table departments\G
*************************** 1. row ***************************
Table: departments
Create Table: CREATE TABLE ‘departments‘ (
‘dept_no‘ char(4) NOT NULL,
‘dept_name‘ varchar(40) NOT NULL,
PRIMARY KEY (‘dept_no‘),
UNIQUE KEY ‘dept_name‘ (‘dept_name‘)
) ENGINE=TokuDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
Tables in Information Schema
40
![Page 108: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/108.jpg)
• TokuDB file mapmysql> select * from TokuDB_file_map\G
*************************** 1. row ***************************
dictionary_name: ./employees/departments-key-dept_name
internal_file_name: ./employees/departments_key_dept_name_5ca_1_1d_B_1.tokudb
table_schema: employees
table_name: departments
table_dictionary_name: key-dept_name
*************************** 2. row ***************************
dictionary_name: ./employees/departments-main
internal_file_name: ./employees/departments_main_5ca_1_1d_B_0.tokudb
...
table_dictionary_name: main
*************************** 3. row ***************************
...
table_dictionary_name: status
Tables in Information Schema
40
![Page 109: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/109.jpg)
• TokuDB fractal tree block mapmysql> select * from TokuDB_fractal_tree_block_map where dictionary_name
-> like ’./employees/departments%’\G
*************************** 1. row ***************************
dictionary_name: ./employees/departments-key-dept_name
internal_file_name: ./employees/departments_key_dept_name_5ca_1_1d_B_1.tokudb
checkpoint_count: 1
blocknum: 0
offset: NULL
size: 0
table_schema: employees
table_name: departments
table_dictionary_name: key-dept_name
Tables in Information Schema
40
![Page 110: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/110.jpg)
• TokuDB fractal tree infomysql> select * from TokuDB_fractal_tree_info where dictionary_name
-> like ’./employees/departments%’\G
*************************** 1. row ***************************
dictionary_name: ./employees/departments-key-dept_name
internal_file_name: ./employees/departments_key_dept_name_5ca_1_1d_B_1.tokudb
bt_num_blocks_allocated: 4
bt_num_blocks_in_use: 4
bt_size_allocated: 12360
bt_size_in_use: 584
table_schema: employees
table_name: departments
table_dictionary_name: key-dept_name
Tables in Information Schema
40
![Page 111: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/111.jpg)
• Internal information about TokuDB enginemysql> SHOW ENGINE TOKUDB STATUS;
+--------+-----------------------------------------------------------+--------------------------+
| Type | Name | Status |
+--------+-----------------------------------------------------------+--------------------------+
| TokuDB | disk free space | more than 10 percent... |
| TokuDB | time of environment creation | Sun Mar 12 15:58:05 2017 |
| TokuDB | time of engine startup | Tue Mar 14 22:44:31 2017 |
| TokuDB | time now | Wed Mar 15 01:05:31 2017 |
| TokuDB | db opens | 891 |
| TokuDB | db closes | 881 |
| TokuDB | num open dbs now | 10 |
| TokuDB | max open dbs | 23 |
| TokuDB | period, in ms, that recovery log is automatically fsynced | 0 |
| TokuDB | dictionary inserts | 3522300 |
...
SHOW ENGINE TOKUDB STATUS
41
![Page 112: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/112.jpg)
• Internal information about TokuDB engine| TokuDB | le: max committed xr | 2 |
...
| TokuDB | checkpoint: period | 60 |
...
| TokuDB | cachetable: miss | 23 |
...
| TokuDB | locktree: memory size | 0 |
...
| TokuDB | ft: dictionary updates | 0 |
...
| TokuDB | hot: operations ever started | 0 |
...
| TokuDB | txn: begin | 4436 |
| TokuDB | txn: begin read only | 2 |
| TokuDB | txn: successful commits | 4419 |
...
SHOW ENGINE TOKUDB STATUS
41
![Page 113: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/113.jpg)
• Internal information about TokuDB engine| TokuDB | logger: next LSN | 4620325 |
...
| TokuDB | indexer: number of indexers successfully created | 1 |
...
| TokuDB | loader: number of loaders successfully created | 8 |
...
| TokuDB | memory: number of malloc operations | 0 |
...
| TokuDB | filesystem: ENOSPC redzone state | 0 |
| TokuDB | filesystem: threads currently blocked by full disk | 0 |
...
| TokuDB | context: tree traversals blocked by a full fetch | 0 |
...
| TokuDB | handlerton: primary key bytes inserted | 67840548 |
+--------+----------------------------------------------------+----------+
322 rows in set (0.05 sec)
SHOW ENGINE TOKUDB STATUS
41
![Page 114: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/114.jpg)
• Inner workings of TokuDB storage engine• Headers, similar to ENGINE TOKUDB
STATUS
Status Variables
42
![Page 115: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/115.jpg)
• Help to tune TokuDB
• Several are really status variables• tokudb backup last error[ string]• tokudb last lock timeout
• Can trigger operations• tokudb backup dir
Server Variables
43
![Page 116: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/116.jpg)
• Help to tune TokuDB• Several are really status variables
• tokudb backup last error[ string]• tokudb last lock timeout
• Can trigger operations• tokudb backup dir
Server Variables
43
![Page 117: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/117.jpg)
• Help to tune TokuDB• Several are really status variables
• tokudb backup last error[ string]• tokudb last lock timeout
• Can trigger operations• tokudb backup dir
Server Variables
43
![Page 118: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/118.jpg)
Summary
![Page 119: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/119.jpg)
• TokuDB is fully functional engine• Reach lock diagnostic• Reach progress diagnostic• Engine runtime information
• SHOW ENGINE TOKUDB STATUS• Status variables
• Write optimized
Summary
45
![Page 120: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/120.jpg)
• Documentation in the user manual
• TokuDB posts at Percona Blog
• Old Tokutek posts in Percona Blog
More information
46
![Page 121: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/121.jpg)
???
Time for questions
47
![Page 122: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/122.jpg)
http://www.slideshare.net/SvetaSmirnova
https://twitter.com/svetsmirnova
Thank you!
48
![Page 123: TokuDB Troubleshooting - percona.com · TokuDB Troubleshooting March, 22, 2017 Sveta Smirnova, George O. Lorch III, Vlad Lesin](https://reader030.vdocuments.mx/reader030/viewer/2022040704/5dd0f780d6be591ccb6391fc/html5/thumbnails/123.jpg)
DATABASE PERFORMANCEMATTERS