how to design indexes, really
DESCRIPTION
MySQL users commonly ask: Here's my table, what indexes do I need? Why aren't my indexes helping me? Don't indexes cause overhead? This talk gives you some practical answers, with a step by step method for finding the queries you need to optimize, and choosing the best indexes for them.TRANSCRIPT
![Page 1: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/1.jpg)
How to Design Indexes, Really
Bill Karwin, Percona Inc.
Tuesday, December 4, 2012
![Page 2: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/2.jpg)
www.percona.com
It’s About Performance
•What’s the most frequent recommendation in database performance audits?
•What’s the easiest way to speed up SQL queries, without schema changes or code changes?
•What’s a commonly neglected part of database development and maintenance?
indexes.Tuesday, December 4, 2012
![Page 3: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/3.jpg)
www.percona.com
Indexing Mistakes
Tuesday, December 4, 2012
![Page 4: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/4.jpg)
www.percona.com
Index Shotgun
“Indexes improve performance, so I index every column.”
Tuesday, December 4, 2012
![Page 5: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/5.jpg)
www.percona.com
Index Shotgun
•Many such indexes are never used.• Indexes occupy space on disk in memory buffers.•Each index needs to be modified during INSERT,
UPDATE, DELETE.•Query optimizer considers indexes during every
query, more indexes makes this work harder.
Tuesday, December 4, 2012
![Page 6: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/6.jpg)
www.percona.com
Index Aversion
“Indexes have overhead, so I never create any indexes.”
Tuesday, December 4, 2012
![Page 7: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/7.jpg)
www.percona.com
Index Aversion
• The right indexes are crucial to speed up queries.•Most workloads are read-heavy, so the net benefit
of indexes outweighs the overhead of updating.• Indexes are compact, so they are a more efficient
use of buffer memory.
Tuesday, December 4, 2012
![Page 8: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/8.jpg)
www.percona.com
SQL Aversion
“I use a NoSQL database because it just works.” *
(* after I create the right indexes.)
Tuesday, December 4, 2012
![Page 9: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/9.jpg)
www.percona.com
SQL Aversion
•Many non-relational databases require you to define indexes explicitly too.
• The ANSI/ISO SQL standard doesn’t specify anything about indexes.
•So if you want to use NoSQL, just use indexes! :-)
Tuesday, December 4, 2012
![Page 10: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/10.jpg)
www.percona.com
Naive Questions
“Here’s my schema...
what indexes do I need?”
Copyright 2012 Gene Ontology Consortium http://www.geneontology.org/
Tuesday, December 4, 2012
![Page 11: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/11.jpg)
www.percona.com
Naive Questions
• Index choice depends on...•What tables and columns you need to query•What JOINs you need to perform•What GROUP BY’s and ORDER BY’s you need
• Index design is not implicit from table design.
Tuesday, December 4, 2012
![Page 12: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/12.jpg)
www.percona.com
Lesson #1
relationalschema design is based on data.
index design is based on queries.
but
Tuesday, December 4, 2012
![Page 13: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/13.jpg)
www.percona.com
Where Are the Queries?
Slow query logGeneral query logApplication logging
TcpdumpBinary log
Process listTuesday, December 4, 2012
![Page 14: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/14.jpg)
www.percona.com
Collecting Log of Queries
•Enable slow-query log:mysql> SET GLOBAL slow_query_log = ON;
• Include all queries, not just those that are slow:mysql> SET GLOBAL long_query_time = 0;
•Percona Server can log more information:mysql> SET GLOBAL log_slow_verbosity = 'full';
Tuesday, December 4, 2012
![Page 15: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/15.jpg)
www.percona.com
Where to Start?
pt-query-digest
http://www.percona.com/doc/percona-toolkit/pt-query-digest.html
Tuesday, December 4, 2012
![Page 16: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/16.jpg)
www.percona.com
pt-query-digest
•Analyzes your query log.•Reports the queries accounting for greatest load.• Slow queries.• Quick queries that run frequently.
•Outputs a ranked list of top query patterns.$ pt-query-digest \/var/lib/mysql/mysql-slow.log \> ~/pqd.txt
Tuesday, December 4, 2012
![Page 17: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/17.jpg)
www.percona.com
Ranked Queries# Profile# Rank Query ID Response time Calls R/Call Apdx V/M Item# ==== ================== =============== ===== ======= ==== ===== =======
# 1 0xA8D2BBDE7EBE7822 4932.2992 28.8% 78 63.2346 0.00 5.22 SELECT person_info# 2 0xFE25DAF5DBB71F49 4205.2160 24.6% 130 32.3478 0.00 3.47 SELECT title
# 3 0x70DAC639802CA233 1299.6269 7.6% 14 92.8305 0.00 0.17 SELECT cast_info# 4 0xE336B880F4FEC4B8 1184.5101 6.9% 294 4.0289 0.36 2.29 SELECT cast_info
# 5 0x60550B93960F1837 905.1648 5.3% 60 15.0861 0.05 1.33 SELECT name# 6 0xF46D5C09B4E0CA2F 777.2446 4.5% 16340 0.0476 1.00 0.17 SELECT char_name
# 7 0x09FCFFF0E5BC929F 747.4346 4.4% 130 5.7495 0.53 7.69 SELECT name# 8 0x9433950BE12B9470 744.1755 4.4% 14368 0.0518 1.00 0.18 SELECT name
# 9 0x4DC0E044996DA715 448.5637 2.6% 130 3.4505 0.65 8.31 SELECT title# 10 0x09FB72D72ED18E93 361.1904 2.1% 78 4.6306 0.28 1.89 SELECT cast_info title
Tuesday, December 4, 2012
![Page 18: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/18.jpg)
www.percona.com
Slow Queries# Profile# Rank Query ID Response time Calls R/Call Apdx V/M Item# ==== ================== =============== ===== ======= ==== ===== =======
# 1 0xA8D2BBDE7EBE7822 4932.2992 28.8% 78 63.2346 0.00 5.22 SELECT person_info# 2 0xFE25DAF5DBB71F49 4205.2160 24.6% 130 32.3478 0.00 3.47 SELECT title
# 3 0x70DAC639802CA233 1299.6269 7.6% 14 92.8305 0.00 0.17 SELECT cast_info# 4 0xE336B880F4FEC4B8 1184.5101 6.9% 294 4.0289 0.36 2.29 SELECT cast_info
# 5 0x60550B93960F1837 905.1648 5.3% 60 15.0861 0.05 1.33 SELECT name# 6 0xF46D5C09B4E0CA2F 777.2446 4.5% 16340 0.0476 1.00 0.17 SELECT char_name
# 7 0x09FCFFF0E5BC929F 747.4346 4.4% 130 5.7495 0.53 7.69 SELECT name# 8 0x9433950BE12B9470 744.1755 4.4% 14368 0.0518 1.00 0.18 SELECT name
# 9 0x4DC0E044996DA715 448.5637 2.6% 130 3.4505 0.65 8.31 SELECT title# 10 0x09FB72D72ED18E93 361.1904 2.1% 78 4.6306 0.28 1.89 SELECT cast_info title
this query executed a few times, but each takes over 1 minute
Tuesday, December 4, 2012
![Page 19: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/19.jpg)
www.percona.com
Quick Queries# Profile# Rank Query ID Response time Calls R/Call Apdx V/M Item# ==== ================== =============== ===== ======= ==== ===== =======
# 1 0xA8D2BBDE7EBE7822 4932.2992 28.8% 78 63.2346 0.00 5.22 SELECT person_info# 2 0xFE25DAF5DBB71F49 4205.2160 24.6% 130 32.3478 0.00 3.47 SELECT title
# 3 0x70DAC639802CA233 1299.6269 7.6% 14 92.8305 0.00 0.17 SELECT cast_info# 4 0xE336B880F4FEC4B8 1184.5101 6.9% 294 4.0289 0.36 2.29 SELECT cast_info
# 5 0x60550B93960F1837 905.1648 5.3% 60 15.0861 0.05 1.33 SELECT name# 6 0xF46D5C09B4E0CA2F 777.2446 4.5% 16340 0.0476 1.00 0.17 SELECT char_name
# 7 0x09FCFFF0E5BC929F 747.4346 4.4% 130 5.7495 0.53 7.69 SELECT name# 8 0x9433950BE12B9470 744.1755 4.4% 14368 0.0518 1.00 0.18 SELECT name
# 9 0x4DC0E044996DA715 448.5637 2.6% 130 3.4505 0.65 8.31 SELECT title# 10 0x09FB72D72ED18E93 361.1904 2.1% 78 4.6306 0.28 1.89 SELECT cast_info title
another query executed frequently, each call is quick
Tuesday, December 4, 2012
![Page 20: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/20.jpg)
www.percona.com
Understanding Indexes by Analogy
Tuesday, December 4, 2012
![Page 21: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/21.jpg)
www.percona.com
This is a Telephone Book
•A telephone book is like an index sorted by last name first, first name last.
•Also includes the person’s phone number.
Thomas Sienckihttp://en.wikipedia.org/wiki/File:Telefonbog_ubt-1.JPG
Tuesday, December 4, 2012
![Page 22: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/22.jpg)
www.percona.com
Implementation
• In MySQL syntax, this would look like the following:CREATE INDEX phone_idx ON TelephoneBook (last_name, first_name, phone_number);
Tuesday, December 4, 2012
![Page 23: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/23.jpg)
www.percona.com
Simple Searches
•Since the index is pre-sorted, it benefits queries for last name:! SELECT * FROM TelephoneBook WHERE last_name = 'Smith';
•By benefit, we mean that it helps to narrow down the search quickly, because the entries are already in sorted order.
Tuesday, December 4, 2012
![Page 24: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/24.jpg)
www.percona.com
Compound Searches
• The sort order benefits us further if we search on last name and first name together:! SELECT * FROM TelephoneBook WHERE last_name = 'Smith' AND first_name = 'John';
Tuesday, December 4, 2012
![Page 25: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/25.jpg)
www.percona.com
When the Index Fails
•But if we search only on first_name, the sort order doesn’t help:! SELECT * FROM TelephoneBook WHERE first_name = 'John';
• The ‘John’ entries occur throughout the book, in an unpredictable distribution. Anyone’s first name can be ‘John’. We have to search every page of the book.
Tuesday, December 4, 2012
![Page 26: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/26.jpg)
www.percona.com
Order Matters
• From this, we conclude that the order we define the columns in an index matters.
•An index with the columns declared in the opposite order would benefit a search for first name alone, but then it wouldn’t help a search for last name alone.CREATE INDEX phone_idx2 ON TelephoneBook (first_name, last_name, phone_number);
Tuesday, December 4, 2012
![Page 27: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/27.jpg)
www.percona.com
Both Indexes Can Be Helpful
• If both criteria use equality comparisons, then the order of columns shouldn’t matter.
•Either index would benefit a query that searches for a specific value in both columns.! SELECT * FROM TelephoneBook WHERE last_name = 'Smith' AND first_name = 'John';
Tuesday, December 4, 2012
![Page 28: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/28.jpg)
www.percona.com
Range Comparisons
•Searching for multiple names is another case. •An index still helps us to narrow the search even if
we search for all names that match a pattern:! SELECT * FROM TelephoneBook WHERE last_name LIKE 'S%';
• The book groups all the names starting with ‘S’ together, so the index can help us to avoid reading the whole book.
Tuesday, December 4, 2012
![Page 29: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/29.jpg)
www.percona.com
Compound Range Comparisons
• If we search by a range of last names, but a specific first name, then the first names are again distributed in an unpredictable way among the matching last names:! SELECT * FROM TelephoneBook WHERE last_name LIKE 'S%' AND first_name = 'John';
• The ‘John’ entries are not grouped together among all ‘S’ names, so we have to search manually every ‘S’ name to find the ones we want.
Tuesday, December 4, 2012
![Page 30: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/30.jpg)
www.percona.com
Order Matters
• From this we see that once we do a range comparison on a column in an index, any criteria we have for subsequent columns cannot benefit from the index.CREATE INDEX phone_idx ON TelephoneBook (last_name, first_name, phone_number);
this column in the index helps the range search
but then subsequent columns do not help searching
Tuesday, December 4, 2012
![Page 31: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/31.jpg)
www.percona.com
Sorting by Index
•Since the telephone book is pre-sorted, we can rely on this if we need to read through the entries in that order. ! SELECT * FROM TelephoneBook WHERE last_name = 'Smith'ORDER BY first_name;
• There’s no extra work to do here; we can simply read the entries in the order they are naturally written in the book. • So the ORDER BY is a no-op.
Tuesday, December 4, 2012
![Page 32: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/32.jpg)
www.percona.com
When the Index Can’t Help Sorting
•Sorting by any other column is another story.! SELECT * FROM TelephoneBook WHERE last_name = 'Smith'ORDER BY phone_number;
• The book is already ordered by last name and then by first name, but not numerically by phone number.
• To get the result sorted in the manner above, we could copy the ‘Smith’ entries onto sticky notes, then re-order the sticky notes by phone number manually.
Tuesday, December 4, 2012
![Page 33: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/33.jpg)
www.percona.com
Order Matters
• From this, we conclude that an index helps if we sort by the column immediately following the columns in the search criteria.
•But if we need to sort by another column later in the index, or by a column that isn’t in the index at all, we have to do it the hard way.
Tuesday, December 4, 2012
![Page 34: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/34.jpg)
www.percona.com
Index-Only Searches
•Once we find the matching entries, we may need other fields of information:! SELECT phone_number FROM TelephoneBook WHERE last_name = 'Smith' AND first_name = 'John';
•Because the phone number is included in the index entry, we get it for free. The entry includes all three fields, even if our query didn’t search for the phone number explicitly.
Tuesday, December 4, 2012
![Page 35: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/35.jpg)
www.percona.com
When the Index Can’t Help
• If we need other information that isn’t included in the index, we may need to do extra work.! SELECT business_hours FROM TelephoneBook WHERE last_name = 'Smith Plumbing';
• In this case, we’d have to use the phone number of the business, and call them to ask their hours.• That’s the extra work.
Tuesday, December 4, 2012
![Page 36: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/36.jpg)
www.percona.com
Columns Matter
• From this, we conclude that putting columns in an index can be very important for certain queries, even if that query doesn’t use the columns for searching or sorting.
• This is called a covering index.
Tuesday, December 4, 2012
![Page 37: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/37.jpg)
www.percona.com
Lesson #2
rate your indexes
using the star system.
Tuesday, December 4, 2012
![Page 38: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/38.jpg)
www.percona.com
The Star System
•Rate an index with respect to a given query by three “stars” which describe effectiveness.
• This technique is covered in Relational Database Index Design and the Optimizers by Tapio Lahdenmäki and Michael Leach.• http://www.wiley.com/WileyCDA/WileyTitle/
productCd-0471719994.html
Tuesday, December 4, 2012
![Page 39: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/39.jpg)
www.percona.com
The Star System
★First star: Rows referenced by your query are grouped together in the index.
★Second star: Rows referenced by your query are ordered in the index the way you want them.
★Third star: The index contains all columns referenced by your query (covering index).
Tuesday, December 4, 2012
![Page 40: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/40.jpg)
www.percona.com
mysql> EXPLAIN SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC\G
id: 1 select_type: SIMPLE table: cast_info type: ALLpossible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 24008907 Extra: Using where; Using filesort
Look at this Terrible Query
uses no index
scans whole table
scans a lot of rows
sorts the hard way
Tuesday, December 4, 2012
![Page 41: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/41.jpg)
www.percona.com
★ First Star
★Pick all columns from equality predicates. Define the index with these columns first.
! SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC;
ALTER TABLE cast_info ADD INDEX (movie_id, role_id);
equality predicates
Tuesday, December 4, 2012
![Page 42: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/42.jpg)
www.percona.com
★ Second Star
★Add the column(s) in the GROUP BY or ORDER BY clause, if the query has one.
! SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC;
ALTER TABLE cast_info ADD INDEX (movie_id, role_id, nr_order);
sorting column
Tuesday, December 4, 2012
![Page 43: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/43.jpg)
www.percona.com
★ Third Star
★Add any remaining columns referenced in the SELECT list.
! SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC;
ALTER TABLE cast_info ADD INDEX (movie_id, role_id, nr_order, person_id, person_role_id);
select-list columns
Tuesday, December 4, 2012
![Page 44: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/44.jpg)
www.percona.com
Explain Thatmysql> EXPLAIN SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC\G
id: 1 select_type: SIMPLE table: cast_info type: refpossible_keys: movie_id key: movie_id key_len: 8 ref: const,const rows: 57 Extra: Using where; Using index
★1: effective index
★3: covering index
★2: no filesort
Tuesday, December 4, 2012
![Page 45: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/45.jpg)
www.percona.com
Complications
•Can’t achieve First-Star indexes when:• WHERE clause has more complex expressions
including OR predicates (disjunction).! SELECT * FROM TelephoneBookWHERE last_name = 'Smith' OR first_name = 'John'
disjunction
Tuesday, December 4, 2012
![Page 46: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/46.jpg)
www.percona.com
Complications
•Can’t achieve Second-Star indexes when:• WHERE clause includes range predicates
(>, <, !=, NOT IN, BETWEEN, LIKE, etc.).• Query includes both GROUP BY and ORDER BY
referencing different columns.• Query must sort by multiple columns over different
tables (can’t make a single index span tables).• Query includes ORDER BY in a different order than
the order in which rows are accessed.
Tuesday, December 4, 2012
![Page 47: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/47.jpg)
www.percona.com
Complications
•Can’t achieve Third-Star indexes when:• Query requires more columns than the maximum of
16 columns per index.• Query requires BLOB, TEXT, or VARCHAR columns
longer than the maximum of 1000 bytes per index (utf8 counts three bytes per character).
Tuesday, December 4, 2012
![Page 48: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/48.jpg)
www.percona.com
Example Query
Tuesday, December 4, 2012
![Page 49: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/49.jpg)
www.percona.com
Example Querymysql> EXPLAIN SELECT STRAIGHT_JOIN n.* FROM char_name AS c INNER JOIN cast_info AS i ! ON c.id=i.person_role_id INNER JOIN name AS n ! ON i.person_id=n.id WHERE c.name = 'James Bond'ORDER BY n.name\G
char_name namecast_info
Tuesday, December 4, 2012
![Page 50: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/50.jpg)
www.percona.com
Not Optimized*************************** 1. row *************************** id: 1 select_type: SIMPLE table: c type: ALLpossible_keys: PRIMARY key: NULL key_len: NULL ref: NULL rows: 2402968 Extra: Using where; Using temporary; Using filesort*************************** 2. row *************************** id: 1 select_type: SIMPLE table: i type: indexpossible_keys: person_id key: person_id key_len: 9 ref: NULL rows: 21676661 Extra: Using where; Using index; Using join buffer*************************** 3. row *************************** id: 1 select_type: SIMPLE table: n type: eq_ref
2.4 million × 21.7 million = 52 trillion comparisons
Tuesday, December 4, 2012
![Page 51: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/51.jpg)
www.percona.com
Add Indexes★ First-star index:
mysql> ALTER TABLE char_name ADD INDEX n (name(20));
★ Third-star index:mysql> ALTER TABLE cast_info ADD INDEX pr_p (person_role_id, person_id);
char_name namecast_info
Tuesday, December 4, 2012
![Page 52: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/52.jpg)
www.percona.com
Optimized*************************** 1. row *************************** id: 1 select_type: SIMPLE table: c type: refpossible_keys: PRIMARY,n key: n key_len: 62 ref: const rows: 1 Extra: Using where; Using temporary; Using filesort*************************** 2. row *************************** id: 1 select_type: SIMPLE table: i type: refpossible_keys: pr_p key: pr_p key_len: 5 ref: imdb.c.id rows: 108383 Extra: Using where; Using index*************************** 3. row *************************** id: 1 select_type: SIMPLE table: n type: eq_ref
improved by a factor of 480 million!
Tuesday, December 4, 2012
![Page 53: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/53.jpg)
www.percona.com
Why Not Second Star?
•Not possible to create a second-star index here.• The order of the join column isn’t necessarily the
same as the order of the sort column:+---------+--------------------------+| id | name |+---------+--------------------------+| 127817 | Connery, Sean || 201693 | Lazenby, George || 213877 | Moore, Roger || 228614 | Dalton, Timothy || 581060 | Brosnan, Pierce || 707448 | Craig, Daniel |+---------+--------------------------+
Tuesday, December 4, 2012
![Page 54: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/54.jpg)
www.percona.com
Lesson #3
tidy up indexes
you don’t need.
Tuesday, December 4, 2012
![Page 55: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/55.jpg)
www.percona.com
Indexes You Don’t Need?
create indexes
that help your queries
drop indexes
that don’t help your queries
and
Tuesday, December 4, 2012
![Page 56: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/56.jpg)
www.percona.com
Redundant Indexes
pt-duplicate-key-checker
http://www.percona.com/doc/percona-toolkit/pt-duplicate-key-checker.html
Tuesday, December 4, 2012
![Page 57: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/57.jpg)
www.percona.com
Redundant Indexes
mysql> create table foo (a int, b int, key (a), key (b), key (a,b));
$ pt-duplicate-key-checker h=localhost# ############################################################## test.foo # #############################################################
# a is a left-prefix of a_2# Key definitions:# KEY `a` (`a`),# KEY `a_2` (`a`,`b`)# Column types:#! `a` int(11) default null#! `b` int(11) default null# To remove this duplicate index, execute:ALTER TABLE `test`.`foo` DROP INDEX `a`;
Tuesday, December 4, 2012
![Page 58: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/58.jpg)
www.percona.com
Unused Indexes
pt-index-usage
http://www.percona.com/doc/percona-toolkit/pt-index-usage.html
Tuesday, December 4, 2012
![Page 59: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/59.jpg)
www.percona.com
Unused Indexes$ sudo pt-index-usage /var/lib/mysql/mysql-slow.log/var/lib/mysql/mysql-slow.log: 12% 01:07 remain/var/lib/mysql/mysql-slow.log: 25% 00:57 remain/var/lib/mysql/mysql-slow.log: 37% 00:49 remain/var/lib/mysql/mysql-slow.log: 49% 00:41 remain/var/lib/mysql/mysql-slow.log: 61% 00:31 remain/var/lib/mysql/mysql-slow.log: 73% 00:21 remain/var/lib/mysql/mysql-slow.log: 84% 00:12 remain/var/lib/mysql/mysql-slow.log: 97% 00:02 remain
. . .ALTER TABLE `imdb`.`cast_info` DROP KEY `movie_id`;
-- type:non-unique. . .
Tuesday, December 4, 2012
![Page 60: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/60.jpg)
www.percona.com
Index Usage Statistics
Index Statisticsin Percona Server
http://www.percona.com/doc/percona-server/5.5/diagnostics/user_stats.html
Tuesday, December 4, 2012
![Page 61: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/61.jpg)
www.percona.com
Index Usage Statistics
• INFORMATION_SCHEMA.INDEX_STATISTICS•Enable with:
mysql> SET GLOBAL userstat = ON;
•Statistics are cleared when you restart mysqld.•Negligible performance impact:
• http://www.mysqlperformanceblog.com/2012/06/02/how-expensive-is-user_statistics/
Tuesday, December 4, 2012
![Page 62: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/62.jpg)
www.percona.com
Index Usage Statistics
• Find indexes that were never used:mysql> SELECT CONCAT(
! 'ALTER TABLE `', s.table_schema, '`.`', s.table_name,! ' DROP KEY `', s.index_name, '`;') AS ddl FROM INFORMATION_SCHEMA.STATISTICS AS s LEFT OUTER JOIN INFORMATION_SCHEMA.INDEX_STATISTICS AS i ! USING (table_schema, table_name, index_name) WHERE s.table_schema = 'imdb' AND i.rows_read IS NULL;
+---------------------------------------------------------+| ddl |+---------------------------------------------------------+| ALTER TABLE `imdb`.`cast_info DROP KEY `movie_id`; || ALTER TABLE `imdb`.`cast_info DROP KEY `person_id`; || ALTER TABLE `imdb`.`comp_cast_type DROP KEY `kind`; || ALTER TABLE `imdb`.`company_type DROP KEY `kind`; |. . .
Tuesday, December 4, 2012
![Page 63: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/63.jpg)
www.percona.com
Lesson #4
make it a habit.
Tuesday, December 4, 2012
![Page 64: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/64.jpg)
www.percona.com
Review Queries Regularly
•Repeat the query analysis periodically:• As new application code introduces new queries.• As the volume of data grows, making trivial queries
more expensive.• As site traffic changes, making some queries more
frequent.
Tuesday, December 4, 2012
![Page 65: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/65.jpg)
www.percona.com
Review Queries Regularly
• Track query types you have reviewed before:• pt-query-digest --review
Saves query types to a table, so you can add notes to them, mark them as reviewed, omit them from future reports.
• pt-query-digest --review-history Saves query performance statistics to a table, so you can analyze trends in the data over time.
Tuesday, December 4, 2012
![Page 66: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/66.jpg)
www.percona.com
Review Indexes Regularly
•Run pt-duplicate-key-checker periodically:• After schema changes.• You can run this on a development or test instance.
Tuesday, December 4, 2012
![Page 67: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/67.jpg)
www.percona.com
Review Indexes Regularly
•Run pt-index-usage or review INDEX_STATISTICS periodically:• On a regular schedule (e.g. weekly).• Especially after application code changes.
Tuesday, December 4, 2012
![Page 68: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/68.jpg)
www.percona.com
Lessons Redux
make it a habit.tidy up.
identify queries.
rate your indexes.
Tuesday, December 4, 2012
![Page 69: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/69.jpg)
www.percona.com
Percona Training for MySQL
Senior Industry ExpertsIn-Person and Online Classes
Custom Onsite Traininghttp://percona.com/training
Tuesday, December 4, 2012
![Page 70: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/70.jpg)
www.percona.com
Visit Our Sponsors
diamond
platinum
exhibitors
and the rest!
Tuesday, December 4, 2012
![Page 71: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/71.jpg)
www.percona.com
SQL Antipatterns
http://www.pragprog.com/titles/bksqla/
Tuesday, December 4, 2012
![Page 72: How to Design Indexes, Really](https://reader036.vdocuments.mx/reader036/viewer/2022082310/5550992cb4c90590208b4790/html5/thumbnails/72.jpg)
www.percona.com
License and Copyright
Copyright 2012 Bill Karwinwww.slideshare.net/billkarwin
Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/
You are free to share - to copy, distribute and transmit this work, under the following conditions:
Attribution. You must attribute this work
to Bill Karwin.
Noncommercial. You may not use this work for commercial purposes.
No Derivative Works. You may not alter, transform,
or build upon this work.
Tuesday, December 4, 2012