proxysql - high performance and ha proxy for mysql

Download ProxySQL - High Performance and HA Proxy for MySQL

Post on 25-Jan-2017

2.033 views

Category:

Data & Analytics

7 download

Embed Size (px)

TRANSCRIPT

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    ProxySQLHigh Performance & High AvailabilityProxy for MySQL

    Ren Cannarene@sysown.com

    http://proxysql.com/

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    GAStable and production readyversion 0.2

    http://proxysql.com/

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Genesis: main motivations

    rewrite queriescache frequent reads

    solving/finding the answer forDEVs vs DBAs

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Open Source proxies for MySQL

    MySQL Proxy

    HAProxyhttp://tinyurl.com/ptjmzde

    Now also MaxScale

    http://proxysql.com/https://github.com/sysown/proxysqlhttp://tinyurl.com/ptjmzde

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    MySQL Proxy

    Pros:customizableexpandableacceptable performance

    Cons:CPU intensiveNot really reliableNot maintained

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    HAProxy

    Pros:very stablehigh performancemature software

    Cons:Layer 7 proxy only for HTTPDoesnt understand the MySQL ProtocolLayer 4 for other services

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Proxy for rewrite and caching?

    MySQL Proxy with:

    Lua scriptExternal caching

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    ProxySQLNetwork database proxy that sits transparently between the application and the database to:

    empower the DBAsimprove operationunderstand and improve performance

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    The right way!

    Designed by a DBA for DBAsExperience in a variety of setupsExperience in very large production environments

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    FeaturesSome of the most interesting features:

    on-the-fly rewrite of queriescaching reads outside the database serverconnection pooling and multiplexingcomplex query routing and read/write splitload balancingreal time statisticsMonitoringHigh Availability and ScalabilitySeamless failoverFirewallQuery throttlingquery timeoutruntime reconfiguration

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Hostgroups and Query Routing

    All backends are grouped into hostgroups

    Hostgroups have logical functionalities

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Hostgroups example #1

    HostGroup0 (HG0): Write masters

    HostGroup1( HG1): Read slaves

    Read/Write split

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Hostgroups example #2

    HG0: main write mastersHG1: main read slavesHG2: reporting slavesHG3: ad-hoc queries slavesHG4: data warehouse write masters HG5: data warehouse read slaves HG6: remote site serversHG7: test servers

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Support for heterogeneous setups

    Hostgroups have logical functionalities

    No replication relationship between hostgroups

    ProxySQL is not replication aware by design, yet it monitors replication

    A query is routed to a single hostgroup

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Basic designHG0

    HG2

    HG1

    ProxySQLAPP

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Basic design , example #2HG0

    HG2

    HG1

    ProxySQL

    APP1

    APP2

    APP3

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Internals diagramHG0

    HG1

    APP1 ProxySQL

    Queries cache

    Queries Processor

    APP2

    APP3 Thr

    ead

    #1

    Thr

    ead

    #2

    Thr

    ead

    #3

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Queries Processor

    Rewrite queriesDefines what to cacheDefines the hostgroup target

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Queries rules

    Complex rules to match incoming traffic.Regex on queryusernameschemaname(a lot more in the roadmap)

    Rules can be chained

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Queries cache

    Caching on the wireInternal key/value storageIn memory onlyPattern basedExpired by timeout

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Queries Cache Benchmark

    Benchmark on a 4 cores server

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Queries Rewrite

    Rewrite on the wire

    Regex match/replace

    Optionally cached

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Queries Rewrite Benchmark

    Ref: http://tinyurl.com/oan4528

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Users AuthenticationHG0

    HG1

    APP1 ProxySQL

    Queries cache

    Query Processor

    APP2

    APP3 Thr

    ead

    #1

    Thr

    ead

    #2

    Thr

    ead

    #3Users Auth

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Users Authentication

    Credentials stored in the proxy

    User login always possible (even without backends)

    Max connections

    Security enhancement in roadmap:different credentials on backends and frontendsinternal mapping

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Hostgroup Managerand Connections Pool

    HG0

    HG1

    APP1 ProxySQL

    Queries cache

    Query Processor

    APP2

    APP3 Thr

    ead

    #1

    Thr

    ead

    #2

    Thr

    ead

    #3Users Auth

    ConnectionPool

    HostgroupManager

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Hostgroups Manager

    Management of servers

    Track servers status

    Tightly integrated with the connections pool

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Connections Pool

    Reduced the overhead of creating new connections, and are recycled when not in use

    One to many connectionsMultiplexing & maximum connectionsAuto-reconnect and automatic re-execution of queriesFailover management

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Multiplexing

    Reduce the number of connections against mysqld (configurable)

    Many clients connections (tens of thousands) can use few backend connections (few hundreds)

    Order by waiting time

    Few edge cases not handled correctly: open a feature request if you hit any

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Auto-reconnect and re-execution

    Automatic detection of failuresGraceful handlingAuto-reconnect when possiblePause until a backend becomes available

    Re-execution of queries

    http://proxysql.com/https://github.com/sysown/proxysql

  • @proxysqlhttp://proxysql.com

    https://github.com/sysown/proxysql/sysown.com

    Failover management

    Seamless switchover:http://www.proxysql.com/2015/09/proxysql-tutorial-seamless-replication.html

    Managed by external processSwitchover in less than 1 second

    http://proxysql.com/https://github.com/sysown/proxysqlhttp://www.proxysql.com/2015/09/proxysql-tutorial-seamless-replication.htmlhttp://www.proxysql.com/2015/09/proxysql-tutorial-seamless-replication.html

  • @proxysqlhttp://proxysq