how we leverage cloud, big data, social data at qmining (salon bi nov 6 2012)

How we leverage the cloud, big-data, social data to do BI?

Salon BI 6 November 2012

francis@qmining.com Founder & CEO

● Story● QMining● BI landscape - Issues ● How QMining is leveraging:

○ Open source○ Cloud○ Hadoop for Big data○ Social data

● Innovation○ QMiner○ QMarketing

Hidden Agenda

Story

Products and Consulting

Big Data Knowledge Action

Big Data

Any data includingweb traffic, social data,

e-commerce transactions

Patterns/clusters

Predictive Analysis

Knowledge

Action

BI Landscape - Issues

● Frustrating○ not simple/not fast/costly

● Increasing data & sources● User want more power (bypass IT)

QMining

● Simple/fast/cost effective● Scalable● More power to the users (bypass IT)

Why?

Examples

Context Enterprise QMining

Big Player (content) 18 mois = failure 3 months = success

online marketing 180+ days = not done

11 days = success

online KPI 30+ days investigations

2 days (bots)

IBM SPSS 15 days 1 day

... ...

Where?

Applications

QMiningIT Application Managementex: performance

monitoring & troubleshooting

Online Marketingex: Marketing

Management Platform

paymentE-commerce

ex: merchant retention

IT Service Management

ex: business process automation

How?

Principles:● Open source● Minimize Abstraction● Cloud

Tools:● Python● Hadoop (for Big Data)

Methodology

Open SourceLeverage collaboration and openness to move faster

-71% of clients don't use open source

All abstractions leakAbstract -> Procrastinate!

http://www.aleax.it/pycon_abst.pdf (Alex Martelli : "Abstraction as a Leverage" )

Minimize A Tower of AbstractionSimplify & lower the layer of abstraction

Examples:

● Work on file not BD if possible● Low level linux command lines (cut, grep, sed etc.)● High level languages : python● Minimize # of languages; ex: Hadoop streaming

(map/reduce=python)● Prototype on a laptop (atom N450)

CloudLeverage IAAS & PAAS infrastructure to build new SaaS

Cloud - PaaS vs IaaS

Pros ● framework● deployment● multi-tenant● https● log & data

visualisation

● less limits● Scaling● Pricing

General Score +20 -20

Recommendations:

The best of both world might be heroku a Cloud application platform but we haven't tried itmontreal python slides on a comparison (appengine vs amazon, pris & cons for a startup)

Prototype

Production

QMining in Cloud

Principles:

● Easy deployment of environments ● Scalable by design● Multi-tenant by design (don't manage multiple versions/instances)

● REST API by design

the power of matlab for prod

example coradiant : 100X speedup

What Howpowerof

PaaS (Platform as a Service)

Web framework

Big Data Leverage cloud to process lot of data (hadoop)

We use amazon EMR (hadoop, EC2 & S3)

Innovation

Why enterprises fail at innovation? (blog)

1. Salon BI 2011: Philippe Nieuwbourg M&A2. Boston CXO Forum 24 October : Best Practice on Global

Innovation (IBM, EMC, P&G, Intuit) Exploit vs Explore 3. Brad Feld (Managing Director at Foundry Group) Startup community 29 October Notman house MTL Hierarchy vs network

QMiner

1. Simplify real user management2. Simplify business view management3. Simplify Persona management

QMiner Simplify real user management

10% of your online users are experiencing slowness

● 100% are located in China ● 75% are using mobile device● 90% are using iphone

http://qmining.qminer.com

QMiner & Social Graph Leverage the social graph apis

Facebook Social Graph:http://developers.facebook.com/docs/reference/api/

QMiner Example: http://fr.qmining.com/facebook/

QMinerSimplify business view management

Your are losing $10K/hour:● 100% are Male● 75% are using mobile device● 90% are using iphone

http://qmining.qminer.com/api/clients/qmining/apps/fbtest/ruleset/

QMiner Simplify persona management

None buying customers are:93% male67% single

87% high educated

Persona fit = 25%

QMarketingSimplify online marketing management

Investing in online marketing and not getting the expected

ROI?

You can't find a global solution in this fragmented market?

QMarketing - Marketing Management

Optimize your online Marketing investments

Hadoop for Channel Optimization

Non sales Patterns with social data

Summary

● Story● QMining● BI landscape - Issues (not simple, slow, costly)

● How QMining is leveraging:○ Open source with linux, python, github

○ Cloud with amazon, google appengine

○ Big data with hadoop

○ Social data with facebook api

● Innovation○ QMiner Simplify real user, business view & persona management ○ QMarketing simplify online marketing management

QUESTIONS

info@qmining.com