Couchbase@ViberFebruary 2014

Amir Ish-ShalomSystem Architect

About Viber

The Viber service• Free, cross platform text messaging• Free, cross platform VoIP calls

(voice and video)• Photo, video and location sharing• Stickers and Emoticons• Group communication platform

(up to 100 participants)• Push To Talk

Monetization

Viber out Sticker market

Simplicity and User Experience• No registration needed• User ID = your mobile number• Automatic friends detection

(no add a friend)

• Always on. No battery impact• 32 languages• Multiple devices experience:

Mobile, Tablets and Desktop

Viber in numbers

Viber in numbers• Hundreds of millions of user accounts• Almost 1 million added daily• Billions of messages every month• Billions of talking minutes every month

Viber Growth

2013-12

2013-11

2013-10

2013-09

2013-08

2013-07

2013-06

2013-05

2013-04

2013-03

2013-02

2013-01

2012-12

2012-11

2012-10

2012-09

2012-08

2012-07

2012-06

2012-05

2012-04

2012-03

2012-02

2012-01

2011-12

2011-11

2011-10

2011-09

2011-08

2011-07

2011-06

2011-05

2011-04

2011-03

2011-02

In 2013 vs. 2012 there was:• Over 3x growth in talking minutes• Over 5x growth in messages• Over 12x growth in group messages

Mes

sage

s per

mont

h

Viber DB Architecture

Viber DB Architecture – 1st Generation

In-housein-memory DB

Application ServersViberClients

Viber DB Architecture – 2nd Generation

Redis Cache MongoDB Cluster

Redis Cluster

Application ServersViberClients

Redis Sharder

Redis Sharder

2nd generation DB architecture advantages• Got us this far• Never lost data from MongoDB• Redis performance

2nd generation DB architecture problems

• MongoDB performance• MongoDB does not scale well with many application servers• Redis – In-memory database with no sharding• Redis Sharder – Not manageable and robust enough

3rd generation DB architecture requirements• High performance• Large data sets• Scalable• Robust• Backed-up• Always on• Easy to monitor• Prefer single DB solution

Solution:

Viber DB Architecture – 3rd Generation

Application ServersViberClients

CouchbaseBackupClusterCouchbase

Clusters

XDCR

Back-end servers• Over 300 application servers• 2nd generation DB architecture:• MongoDB – 150 servers (master + 2 slaves)• Redis – Over 100 servers (master + 1 slave)

• 3rd generation DB architecture:• 6 Couchbase clusters (up to 50 nodes each)• 0 – 2 replicas, XDCR & external backup• Total of 100-120 Couchbase servers

Increased performance using

less than ½ of the DB servers!

Migrating from 2nd to 3rd generation DB’s• Migrate a live system• Zero downtime• No data loss• Consistent data

How did we migrate?• Stage 1: Add new CB cluster in parallel to existing cluster

Only delete keys from CB• Stage 2: Read only from MongoDB

Write/Delete to both CB & MongoDB• Stage 3: Background process that copies all data from MongoDB

to CB (if it doesn’t exist)• Stage 4: Validate data (both DB’s should be identical)• Stage 5: Read only from CB

Write/Delete from both CB & MongoDB• Stage 6: Remove MongoDB and use only CB

Where are we now?• Completed migration of 3 clusters to Couchbase• Currently migrating 3 additional clusters to Couchbase

Obstacles we had to overcome with Couchbase1. No server-side document updates

Solution: CAS2. No internal data structures (maps, sets, lists, etc.)

Solution 1: “Appendable” JSON’s{“key”:”123”,”data”:”abc”}; {“key”:”456”,”data”:”efg”};-{“key”:”123”}

Solution 2: Break-up key to several keys3. No secondary indexes / range queries

Solution 1: Process daily backups (sqlite3 files) Solution 2: Use views

Couchbase Cluster with 10 nodes

Future improvements• Complete database migration to use only Couchbase• Upgrade to CB v2.5 to distribute replica across EC2 AZ’s• More extensive use of views• Integration with our Big Data analytics Hadoop architecture• Integration with Elastic Search using XDCR

Questions?

Thank you