![Page 1: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/1.jpg)
© 2019 Percona. 1
Peter Zaitsev, CEO, PerconaMorgan Tocker, Senior Product and Community Manager, TiDB
Horizontally Scale MySQL with TiDBWhile avoiding Manual Sharding
May 1st, 2019Percona Technical Webinars
![Page 2: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/2.jpg)
© 2019 Percona. 2
In This Presentation
Scaling MySQL
Why and when Sharding is Needed
Problems to Consider
Solutions TiDB offers
![Page 3: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/3.jpg)
© 2019 Percona. 3
MySQL Scalability (Single Instance)
![Page 4: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/4.jpg)
© 2019 Percona. 4
Single MySQL Instance Can Do
Hundreds of Thousands of Queries/Sec
Tends of Thousands of Updates/Sec
Traverse Tens of Millions of Rows/Sec
Comfortably Handle Several TB Database size
![Page 5: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/5.jpg)
© 2019 Percona. 5
Lets Do Some Math100.000 QPS
10 Queries per User Interaction
10.000 User Interactions/sec
864.000.000 User Interactions/Day
30 User Interactions/User Avg
28.000.000 Daily Active Users Possible
15M of Daily Active Users counting time of day skew
![Page 6: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/6.jpg)
© 2019 Percona. 6
Is it Enough ?
More than Enough for Small-Medium Size Applications
Not enough for next Uber or
![Page 7: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/7.jpg)
© 2019 Percona. 7
Additional Worries
Single Thread Query Execution means no scalability for complicated queries
Huge instances are painful especially in the age of Cloud, Containers, Kubernetes
![Page 8: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/8.jpg)
© 2019 Percona. 8
Solution
Sharding – Splitting the data across multiple instances by some
criteria
![Page 9: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/9.jpg)
© 2019 Percona. 9
Approach to Sharding
•Application manually places data in right location
Manual Sharding
•Sharding implemented on database engine level
Automatic Sharding
![Page 10: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/10.jpg)
© 2019 Percona. 10
Sharding Pains
Manual Sharding
•Increases Application Complexity
•Reduces Development Velocity
Automated Sharding
•More Complicated Database Engine
•Danger of relying on Magic
![Page 11: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/11.jpg)
© 2019 Percona. 11
Sharding Problems to Consider
Picking Right Sharding Key
Query Routing
Cross Shard Query Execution
Schema Maintenance
Consistent Backups
Cluster Scaling and Shard Balancing
![Page 12: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/12.jpg)
© 2019 Percona. 12
Automating Sharding
Application Level
• Manual Sharding Done Right
• Custom API • Pain for smaller
group of backend Developers
Proxy
• Use Existing MySQL backend
• Easy Compatibility for Routed Queries
• Hard to handle distributed queries optimally
Custom Distributed Engine
• Can be designed to solve all the Sharding problems
• Compatibility with MySQL is harder
![Page 13: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/13.jpg)
© 2019 Percona. 13
Examples for MySQL
• Hibernate ShardsApplication Level
• ProxySQL• VitessProxy
• TiDBCustom Engine
![Page 14: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/14.jpg)
© 2019 Percona. 14
Beyond MySQL
• Analytical Workloads to Spark, Hadoop, ClickHouse, RedShift
• Full Text Search workloads to Elastic and Solr
• Document and Key Value Workloads to MongoDB and Cassandra
Have been moving certain
workloads off MySQL
![Page 15: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/15.jpg)
© 2019 Percona. 15
Polyglot Persistence
Great
•Allows to use the best tool for the job
Not So Great
•Increases complexity in development and operations
![Page 16: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/16.jpg)
© 2019 Percona. 16
TiDB
Allow to horizontally scale MySQL workloads
and limit technology sprawl
![Page 17: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/17.jpg)
© 2019 Percona. 17
Thank You!
![Page 18: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/18.jpg)
How to horizontally scale MySQL with TiDB while avoiding sharding issues
May 2019
Morgan Tocker, PingCAP (@morgo)
![Page 19: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/19.jpg)
Agenda
● History and Community● Technical Walkthrough● Use Cases● MySQL Compatibility● Benchmarks
![Page 20: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/20.jpg)
History and Community
Founded in 2015 in ChinaTi = Titanium Apache 2.0 LicensedStorage layer (TiKV) a CNCF project since 2018US Office since 2018
Quick Numbers:700+ Annual Conference Attendees300+ Production Deployments250+ GitHub Contributors (the TiDB server alone)
![Page 21: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/21.jpg)
Agenda
● History and Community● Technical Walkthrough● Use Cases● MySQL Compatibility● Benchmarks
![Page 22: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/22.jpg)
Introduction
TiDB is a distributed database that speaks the MySQL protocolIt is not based on the MySQL source codeIt is an ACID/strongly consistent databaseThe inspiration is Google Spanner/F1
It separates SQL processing and Storage into separate componentsBoth of them are independently scalable
The SQL processing layer is statelessIt is designed for both Transaction and Analytical Processing (HTAP)
![Page 23: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/23.jpg)
TiDBTiDB
Region 1 L
TiKV Node 1
Region 2
Region 3
Region 4
Region 2 L
TiKV Node 3
Region 3
Region 4 L
Region 1
Region 4
TiKV Node 2
Region 3 L
Region 2
Region 1
TiKV Cluster
PD Cluster
![Page 24: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/24.jpg)
Row + Column Storage (Announced Jan 2019)
Spark Cluster
TiDBTiDB
Region 1
TiKV Node 1
Region 2
Region 3
Region 4
Region 2
TiKV Node 3
Region 3
Region 4
Region 1
Region 4
TiKV Node 2
Region 3
Region 2
Region 1
TiFlash Node 2
TiFlash Extension Cluster TiKV Cluster
TiSparkWorker
TiSparkWorker
TiFlash Node 1
![Page 25: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/25.jpg)
TiDB: The SQL Layer
Node1 Node2 Node3 Node4
MySQL Network ProtocolSQL Parser
Cost-based OptimizerDistributed Executor (Coprocessor)
ODBC/JDBC MySQL ClientAny ORM which supports MySQL
TiDB
TiKV
![Page 26: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/26.jpg)
TiKV: The Storage Foundation
RocksDB
Raft
Transaction
Txn KV API
Coprocessor API
RocksDB
Raft
Transaction
Txn KV API
Coprocessor API
RocksDB
Raft
Transaction
Txn KV API
Coprocessor API
Raft Group
Client
gRPC
TiKV Instance TiKV Instance TiKV Instance
gRPC gRPC
PD Cluster
![Page 27: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/27.jpg)
Migration (in and out of TiDB)
DMMySQL Binlog
SQL Dump File
Lightning
TiDB Binlog
MySQL Instances
![Page 28: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/28.jpg)
Agenda
● History and Community● Technical Walkthrough● Use Cases● MySQL Compatibility● Benchmarks
![Page 29: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/29.jpg)
Use Cases
1. Approaching the maximum size for MySQL on a single server. Debating whether or not to shard.
2. Already sharded MySQL, but having a hard time doing analytics on up-to-date data.
![Page 30: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/30.jpg)
Mobike + TiDB
● 200 million users● 200 cities● 9 million smart bikes● ~30 TB / day
![Page 31: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/31.jpg)
Agenda
● History and Community● Technical Walkthrough● Use Cases● MySQL Compatibility● Benchmarks
![Page 32: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/32.jpg)
pingcap.com/docs/sql/mysql-compatibility/
![Page 33: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/33.jpg)
Summary
● Compatibility with MySQL 5.7:○ Joins, subqueries, DML, DDL etc.○ All SQL Modes
● On the 3.0 roadmap:○ Views, Window Functions
● Not Planned:○ Stored Procedures, Triggers, Events, Fulltext
![Page 34: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/34.jpg)
Nuanced
● Some features work differently○ Auto Increment○ Optimistic Locking
● TiDB works better with smaller transactions○ Recommended to batch updates, deletes, inserts to
5000 rows
![Page 35: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/35.jpg)
Tools
● Mydumper, ProxySQL work○ We maintain a branch of mydumper○ We are looking to push upstream
● Innotop.. won’t work● MySQL Workbench works!
![Page 36: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/36.jpg)
Agenda
● History and Community● Technical Walkthrough● Use Cases● MySQL Compatibility● Benchmarks
![Page 37: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/37.jpg)
Benchmarks (TiDB 2.1) Thank you Alexander Rubin!
https://www.percona.com/blog/2019/01/24/a-quick-look-into-tidb-performance-on-a-single-server/
![Page 38: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/38.jpg)
Benchmarks (TiDB 2.1) Thank you Alexander Rubin!
https://www.percona.com/blog/2019/01/24/a-quick-look-into-tidb-performance-on-a-single-server/
![Page 39: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/39.jpg)
Benchmarks (TiDB 2.1) Thank you Alexander Rubin!
https://www.percona.com/blog/2019/01/24/a-quick-look-into-tidb-performance-on-a-single-server/
![Page 40: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/40.jpg)
Benchmarks (3.0 Alpha)
Same Benchmark
Different Test Conditions:3 node clusterEach node with 1 TiDB + 1 TiKVLoad balanced by HA ProxyHigher thread counts
Different hardware:Original = m4.16xlarge (64 CPU cores)This Test = 40 vCPUs XEON E5-2630 v4 @ 2.20GHz
https://github.com/pingcap/docs/blob/master/benchmark/sysbench-v4.md
February 2019, Internal Test
![Page 41: Horizontally Scale MySQL with TiDB - Percona · In This Presentation Scaling MySQL Why and when Sharding is Needed Problems to Consider Solutions TiDB offers ... Technical Walkthrough](https://reader035.vdocuments.mx/reader035/viewer/2022062604/5fb4914102ddd760a16eabfe/html5/thumbnails/41.jpg)
Thank you!
Full Day TiDB Track!