Cassandra Delivering Christmas gifts
in France since 2012
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Puppet
Me, myself and I
2
Software
Cassandra Matthieu Nantern
@mNantern
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Context
Delivering parcels at La Poste
Why ?
Why do we need another system ?
Project’s constraints and alternatives
How ?
How we build our solution. Using C*
with PHP and managing TTL
Go Live!
Provisioning servers. Performance and resilience testing
41 2 3
OverviewFrom Mysql to Cassandra
3
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Mail services
Parcels distribution
More than 250 000 employees
70 000 postmen delivering parcels
La Poste
4
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Scan Print
Scan Distribute
La PostePostman’s tour
5
(1) A postman scans every parcels
(2) He prints his list of parcels
(3) He distributes the parcels
(4) He scans every remaining parcels
Why ?
From MySQL to C*
1
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Why do we need another system ?
7
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Max 13ms per call
15 days of retention for data
Easy to deploy and operate
Easy to scale for years to come
Project’s constraints
8
How ?
From MySQL to C*
2
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Data Modeling
10
Id Data key1 key2 key3
1 <Xml> k11 k21 k31
2 <Xml> k12 k22 k32
3 <Xml> k13 k23 k33
Id Data Search1 Search2
1 <Xml> k11k31 k21k11
2 <Xml> k12k32 k22k12
3 <Xml> k13k33 k23k13
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Using Cassandra with PHP
11
$pool = new ConnectionPool("Keyspace1", array("cass:9160"));
$column_family = new ColumnFamily($pool, 'ColumnFamily1');
For dummies
$column_family->insert('row_key', array('name1' => 'val1', 'name2' => 'val2'));
$column_family->get('row_key');
$row1 = array('name1' => 'val1', 'name2' => 'val2');$row2 = array('foo' => 'bar');$column_family->batch_insert(array('row1' => $row1, 'row2' => $row2);
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Using Cassandra with PHP
12
Level 2
$index_exp=CassandraUtil::create_index_expression($indexName, $indexValue,$operator);
$index_clause = CassandraUtil::create_index_clause($indexArray);
$rows = $columnFamily->get_indexed_slices($index_clause);
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
TTL in C*
13
$cf->insert('row_key', array('col1' => 'col_val1','col2' =>'col_val2'),null,TTL);
$columnsTTL = array('col1'=>3, 'col2'=>4);
$cf->insert('row_key', array('col1' => 'val1','col2' =>'val2'),null, $columnsTTL);
Go Live!
From MySQL to C*
3
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Provisioning C* with Puppet
12 servers with Ubuntu 10.04
15
package { 'cassandra': ensure => installed}
initial_token: <%= ((2**127) * (current_cdb_server.to_i-1) / cdb_servers_number.to_i ) %>
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Provisioning C* with Puppet
16
package { 'cassandra-data-model': ensure => latest}
Managing the data model
Current Migration Id
2
migration_1.txtmigration_2.txtmigration_3.txtmigration_4.txt
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Testing our cluster
17
Mysql performance
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Testing our cluster
18
Cassandra performance
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Testing our cluster
19
Resilience
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Migration plan
20
t=0
WriteRead
t+15d
Read Write
Final words
From MySQL to C*
4
YOUFor watching
THANK
Merci!
www.xebia.fr | http://blog.xebia.fr | #CassandraEUTelephone : 33 (0)1 53 89 99 99 | email : [email protected] | 156 bd Haussmann - 75008 Paris|
Thanks !
23
Thank you for the support !
Thank you for the opportunity to work at La Poste !
Thank you for trusting me on Cassandra (and everything else)!
