Database highload solutions

Observing. Comparing. Conclusion.

Only free products.

INTROWhat we need?

Demands

• High performance • Scalability• Durability• Complex analytic/statistics queries

Products typesNonSql DatabasesPro:

High performance High scalability Document oriented

Contra: Impossibility to perform typical

web queries

SQL DatabasesPro:

Sql language Well known Any complex queries

Contra: Not enough performance Scalability*

*For observed database

CompetitorsNON Sql Databases

• MongoDB• Redis, Memcache*

SQL Databases

• MySql and Clones• Postgres

* Key-value storage like memcacheDB

SCALING STRATEGYTheory

Scaling Strategy

o Scale back and Scale up.

1. Partitioning : split your tables.

2. Replications: Extend your servers farm and split ‘Insert/update/delete’ and ‘select ’ queries to Master/Slave(s) .

3. Sharding : split your databases.

Scale Back and Scale UpWhat?Scale Back – remove/archive unused data.Scale Up – add more power(RAM,CPU) , disks.When?• When you can.• Your database get benefits from that.Changes in application:• No

PartitioningWhat?Partitioning – split tables to logical parts by same criteria's. When?• You have big table(s) which utilize the biggest part of traffic.• Your queries ask only some parts of information.How?• Database do it work for you clearly for your application.Changes in application:• No

Example:You can have millions commentaries but in 99% you show only latest of them.

ReplicationsWhat?Copying(sync or async) data from “master” server – to “slaves”(or other Master). The idea – read from slave, write to master.When?• You have much more “read” traffic then “write” one.How?• Databases have tools/solutions for replication.Changes in application:• Not big

ShardingWhat?Split database into many smaller databases.There are:• Vertical sharding• Horizontal sharding

Vertical shardingWhat?Split database into many database(servers) by functionalityWhen?• You have tables which functionalities do not cross.How?• You will split them manually.Changes in application:• Not so bigLacks• No more “JOINS” between these tables

Example:Commentaries table utilize 90% of traffic. Shard it vertically and move the table onto a

separate server.

Horizontal shardingWhat?Split a big table into many serversWhen?• You have a huge table and it too big for one server.How?• You will split them manually.Changes in application:• A lotLacks• No more “JOINS” between this table and other. No more ‘dumb’

queries for the table.

Example:You have billions users. Split the table into few servers. Each server will contains same

table but with different data.

SOLUTIONSPropositions

Mysql

Mysql Clone: Percona

Background: Pecona company.

Mysql Clone: MariaDB

Background: Michael “Monty” Widenius – founder of Mysql.

Mysql vs Clones

Postgres

Background: Scape.

The Best Mysql vs Postgres

MongoDB

Background: Scape.

Key-value storages

Background: Scape.