analyzing complex behavior graphs in hadoop at scale

Analyzing Complex Behavior Graphs in Hadoop at Scale

Today’s speakers

Sanjeev Srivastav@ssrivastav

Joy Thomas@JoyAThomas1

© 2014 Apigee – For Public display

Apigee social channels

3

YouTubehttp://youtube.com/apigee

Slideshare

http://slideshare.com/apigee

© 2014 Apigee – For Public display

Agenda

• Customer behavior graphs

• GRASP: an event model for customer interactions

• GRASP for– the customer journey– predictive modeling

Customer behavior graphs

© 2014 Apigee – For Public display

Why do we need behavior graphs?

© 2014 Apigee – For Public display

Customer view: a journey

7

© 2014 Apigee – For Public display 8

Understand each customer’s journey

customer journeysiloed view

© 2014 Apigee – For Public display 9

Identify common interactions and influences

common interactions & influencescustomer journey

© 2014 Apigee – For Public display

Customer behavior

10

Behavior graph• sequence of events:

– actions experienced and taken

Social graph• links between people & activities

– at a particular point in time

Behavior graph

Social graph

© 2014 Apigee – For Public display

Why do we need new technology for behavior graphs?

© 2014 Apigee – For Public display 12

Challenges with current technologies

• SQL is not efficient for sequences of events and quick counting of customer journeys

• Custom algorithms are expensive

• Various graph systems are oriented toward social graphs and computations of neighborhoods, “friend of a friend,” etc.

Event model for customer interactions: GRASP

© 2014 Apigee – For Public display© 2014 Apigee Confidential – All Rights Reserved 14

• Graph and sequence processing

- time-sequenced graph analytics on Hadoop

GRASP

© 2014 Apigee – For Public display

Model for user behavior

Users act on nodes in a temporal sequence of events

1

3 52

2 50

0

USER PROFILEUserID: U56Gender: MGeo: San FranciscoInterests: bikes, fashion

USER PROFILEUserID: U57Gender: FInterests: news, financeAge: 35-40

NODE PROFILEType: ContentPageID: P100Category: product reviewSubCat: mountain bike

NODE PROFILEType: CreativeID: Creative95Category: VideoAdAdvertiser: BikePros

EVENTType: PageViewUserID: U56PageID: P100TimeSpent: 180 sec. Scrolls: 3

EVENTType: AdViewUserID: U56AdID: Creative95PlayTime: 30 sec.Rewinds: 1

© 2014 Apigee – For Public display

Aggregated behavior graph

0

1

2

3

5

0

1

3 52

2 5Impressions: 1TimeSpent: 20Clicks: 1

0

0Impressions: 4TimeSpent: 10Clicks: 0

Impressions: 5TimeSpent: 30Clicks: 1

Combine

© 2014 Apigee – For Public display

Event streams

17

Event stream

Combined event streamPurchase

health insurance

Offer for Health

Insurance

GRASP merges event streams and normalizes time relative to responses

Store visits

Emails

Phone calls

Purchases

F, 35, Married

M,25, Married

© 2014 Apigee – For Public display

Data representation • Data model: events & dimensions• Data structure: aggregated

behavior • Graphs for events, not tables• Data access: API

GRASP: graphs vs. tables

18

Event facts are represented as graphs, not tables

SQL is not effective for sequence queries

1

2

3

40

Userdimension

Nodedimension

Events

Data storage & computation• Distributed data structure on

Hadoop• Computation using map reduce

Dual use of GRASP: customer journey & predictive models

© 2014 Apigee – For Public display

How has this been used for solving customer engagement problems?

© 2014 Apigee – For Public display© 2013 Apigee Confidential – All Rights Reserved 21

Dual use of GRASP for analytics

GRASP

GQM

Data

Profiles

Text

Events

Feature extraction

Events

Data scientist

Adaptive applications

Developer

Custom app

Business user

Segments manager, GQM

DeveloperDeveloper

Predictive

Descriptive

A

Y

X

C D

B

Predictive & descriptive analyticsGraph and sequence processing (GRASP)

© 2014 Apigee – For Public display

Visualize customer journeys across all interactions

• Examples: telco, healthcare, retail

© 2014 Apigee – For Public display© 2013 Apigee Confidential – All Rights Reserved

• Machine learning using path sequences

Build behavior prediction models

© 2014 Apigee – For Public display

Relevant problems to solve

24

• Multi-channel customer event data for understanding customer journeys and predicting behavior

• Modeling evolving customer behavior using updates to behavior graphs

Questions?

Sanjeev Srivastav@ssrivastav

Joy Thomas@JoyAThomas1

© 2014 Apigee – For Public display

Thank you