couchbase chennai meetup 2 - big data & analytics

Couchbase & BigData-Analytics

Kadhambari AnbalaganSoftware Architect, RedBlackTree Technologies Pvt. Ltd.

©2014 Couchbase Inc.

Couchbase + Big Data

New Data Stream Merged View

All DataPrecompute

Views (Map Reduce)

Process Stream

Incremental Views

Partial Aggregat

e

Partial Aggregat

e

Partial Aggregat

e

Real-Time Data

BatchRecompute

Batch Views

Real-Time Views

Real-TimeIncrement

Merge

Batch Layer

Serving Layer

Speed Layer

Couchbase HadoopConnector


Use Case – Cookie Store


Cookie Store - The Problem


A Central Cookie Repo – The Solution


Infrastructure Requirements

Had to operate at high volume Caching and Persistence Low latency – 3 to 5 ms Flexible data structure High availability Scalability Fault tolerant Disaster Recovery


Migration Strategy

Analyze

Regression

Dual-ModeMigrate

Monitor


Functional View

CookieService

Couchbase DC A Couchbase DC B

ApplicationCookie Libraries

Couchbase Client

Customers

Front Tier

Mid Tier

Data Tier


Deployment Model

CookieService

CookieService

CookieService

XDCR

Active

WriteRead

Birdirectional Unidirectional

Active Passive

9


Cookie Analytics


Use Case – User Activity Tracking

12©2014 Couchbase Inc.

User activity tracking and real-time analytics

Objectives & ChallengesProvide business users with real time reports and visualizations of user interaction data Collect web and mobile clickstream in real time Integrate with other big data technologies (Hadoop and

Storm) Provide views of data across multiple dimensions (e.g., time,

location, browser and device types)

130m+ active accounts, in 190+ countries, 25 currencies

10TB data 1B documents

SolutionDeploy Couchbase Server to capture, store, and process real time web data Ingests data (via Storm) from multiple inputs, including

mobile, web, and other services, storing data as JSON documents

Integrates with Hadoop to pass data for additional offline analytics

Generates views for business users in under 1 minute, based on 10-minute data collection intervals

The Couchbase AdvantageReal time performance, easy integration with Storm and Hadoop


User activity tracking and real time analytics


User activity tracking and real-time analytics

14

Couchbase Solution Couchbase Server deployed to capture, store, and

process real time web data Ingests data (via Storm) from multiple inputs, including

mobile, web, and other services, storing data as JSON documents

Integrates with Hadoop to pass data for additional offline analytics

Results Consistent low latency (sub 10-msec response) High availability enabled by distributed caching and

XDCR Views for business users are generated in under 1

minute, based on 10-minute data collection intervals

Couchbase & Fraud Detection

16©2014 Couchbase Inc.

Fraud Detection with FICOObjective & ChallengesCapture transactions, store card / account profiles, customer profiles & user defined variables with sub-msec latency and high throughput Growing number of accounts, cards and customers means

more data needs to be tracked and faster latencies are required

Relational systems unable to scale to the required throughput HA / DR solutions not streamlined – need custom

development

Falcon #1 Fraud Detection

platform in the world

65% of worlds credit / debit cards scored by Falcon Solution

Use Couchbase as the “profiling store” and replace relational database Each Falcon customer has 100’s of millions of card and / or

account profiles that can easily be stored and updated based on consumer’s real time activity

Neural networking algorithms run on Couchbase and access data as key-value pairs. Memory-first architecture allows <1ms responses.

Complete HA / DR solution delivers 24x365 application uptime

The Couchbase AdvantageMemory-first architecture means high throughput, all with click-button scalability


Fraud Detection with Couchbase at Wells Fargo

17


Fraud Detection with Couchbase at Wells Fargo

18

Couchbase Solution Use Couchbase as the “profiling store” and replace

relational database Each Falcon customer has 100’s of millions of card and /

or account profiles that can easily be stored and updated based on consumer’s real time activity

Results Complete HA / DR solution delivers 24x365 application

uptime Memory-first architecture allows <1ms responses. Neural networking algorithms run on Couchbase and

access data as key-value pairs

IOT with Couchbase


Internet of Things @ Verizon

20

Objective & ChallengesEnable new service offering for Verizon enterprise customers to manage all devices connected to the company’s network Collect and store data in real time from 10K’s-100K’s of

devices on a single customer’s network Analyze data for usage statistics and patterns Provide near real-time insights and reports into device

usage

New enterprise offering

Enable enterprises to manage all non-cellphone devices on their network

Provide near real time insights and views on devices

SolutionDeploy Couchbase Server to store data and serve reports on connected devices Couchbase Server ingests data at high speed, from any kind

of connected device: alarms, locking systems, modems, solar panels, cash registers

Stream-based indexing enables fast views and reports JSON data model easily handles any data type, new data

types

The Couchbase AdvantageMassive speed and scale that’s easy to manage


Verizon - Architecture

couchbase chennai meetup 2 - big data & analytics

Technology