technical appendix catch the pink flamingo analysis acquiring, … · 2016-08-16 · technical...

Technical Appendix Catch the Pink Flamingo Analysis

Produced by: Greg B Watson

Acquiring, Exploring and Preparing the Data The purpose of this appendix is to ensure that the analysis of the game “Catch the Pink Flamingo” produced for Eglence can be repeated, either by other teams to verify our findings, or by the same team in the future to continue the analysis. In the following sections, we used the following analysis tools: Splunk data exploration and data set overview Knime Categorization with a decisiontree pySpark Cluster Analysis Neo4j Graph Analysis Data Exploration (Splunk) Data Set Overview The table below lists each of the PinkFlamingo game log files available for analysis with a short description of what is found in each one.

File Name Description Fields

adclicks.csv A line is added to this file when a player clicks on an advertisement in the Flamingo app

timestamp – when the click occurred

txID – a unique id (within adclicks.log) for the click

userSessionid – the id of the user session for the user who made the click

teamid – the current team id of the user who made the click

userid – the user id of the user who made the click

adID – the ID of the ad clicked adCategory – the category/type of

the ad clicked

buyclicks.csv A line is added to this file when a player makes an inapp purchase in the Flamingo app

timestamp – when the purchase was made

txID – a unique id (within buyclicks.log) for the purchase

userSessionId – the id of the user session for the user who made the purchases

team – the current team of the user who made the purchase

userid – the user id of the user who made the purchase

buyID – the id of the item purchased

price – the price of the item purchased

gameclicks.csv A line is added to this file each time a user performs a click in the game

time – when the click occurred clickid – a unique id for the click userid – the id of the user

performing the click usersessionid – the id of the

session of the user when the click is performed

isHit – denotes if the hit is on a flamingo (value is 1) or missed the flamingo (value is 0)

teamId – the id of the team of user teamLevel – the current level of the

team of the user

levelevents.csv A line is added to this file each time a team starts or finishes a level in the game

time – when the event occurred eventid – a unique id for the event teamid – the id of the team level – the level started or

completed eventType – the type of event,

either start or end

teamassignments.csv

A line is added to this file each time a user joins a team. One user can be a member of at most one team at one time.

time – when the user joined the team

team – the id of the team userid – the id of the user assignmentid – a unique id for this

assignment

team.csv This file contains one line for each team terminated in the game

teamid – the id of the team name – the name of the team teamCreationTime – the timestamp

when the team was created teamEndTime – the timestamp

when the last team member left the team

currentLevel – the last level achieved by the team

usersession.csv

Each line in this file describes a user session, which denotes when one user started and stopped playing the game. Additionally, when a team goes to the next level in the game, the session is ended for each user in the team and a new session is started.

userSessionId – a unique id for the session

assignmentId – the team assignment id for the user to the team

startTimeStamp – a timestamp denoting when the session started

endTimeStamp – a timestamp denoting when the session ended

teamlevel – the level of the team during the session

platformType – the type of platform of the user during the session

users.csv This table contains one line for each user playing the game

timestamp – when the user first played the game

id – the userid assigned to the user nick – the nickname chosen by the

user twitter – the twitter handle of the

user dob – the date of birth of the user country – the twoletter country

code where the user lives

Aggregation (Splunk) Amount spent buying items 21407

# Unique items available to be purchased 6

A histogram showing how many times each item is purchased:

A histogram showing how much money was made from each item:

Filtering A histogram showing how many times each category of advertisement was clickedon:

The following table shows the user id, platform, and hitratio percentage for the top three buying users:

Rank User Id Platform HitRatio (%)

1 2229 iPhone 12%

2 12 iPhone 92%

3 471 IPhone 15%

Data Classification Analysis (DecisionTree with Knime) Data Preparation Analysis of combined_data.csv Sample Selection

Item Amount

# of Samples 4619

# of Samples with Purchases 1411

Attribute Creation A new categorial attribute was created to enable analysis of players as broken into 2 categories (HighRollers and PennyPinchers). A screenshot of the attribute follows:

The attribute “avg_price” indicates if the average prices of ingame purchases exceeds $5 ($avg_price$ > 5 => “HighRoller”) or is $5 or less ($avg_price$ <= 5 => “PennyPincher”) The creation of this new categorical attribute was necessary because our assigned task is to identify the attributes that can predict if a player will purchase items worth more than $5.

● The decisiontree algorithm will fail if we target a continuous variable ● Our algorithm will run faster if we store the category as part of the dataset rather than

evaluate the category as part of building the decision tree ● For the decision we week to make (choosing the approach for advertising ingame

purchases to gamers), predicting a category is sufficient. Attribute Selection The following attributes were filtered from the dataset for the following reasons:

Attribute Rationale for Filtering

avg_price We excluded rows where avg_price is missing because these rows cannot train or test our model for predicting avg_price_binned. We excluded the column, avg_price, because the target, “avg_price_binned_ is calculated from it.

userSessionId This is a madeup foreign key used to merge user and session records. We are better to match on the raw data from each record

userId userId is a madeup foreign key. Any relationship we found with respect to a madeup key is accidental and hard to interpret. For example, if the existing keys have been created in sequence, then the numeric value of the userId may vary with the number of months for which the user has been a member. This would be an accidental relationship since we do not know of a rule that says userId’s have to be created in the sequence in which users join. If we want to check for a relationship between monthsasmember and dollar value of purchases, we are better to calculate “monthsasmember” from the user createdate.

Data Partitioning and Modeling The data was partitioned into train and test datasets. The “train” data set was used to create the decision tree model.

The trained model was then applied to the “test” dataset. This is important because… we avoid overfitting (building a model that exactly fits the data used to build the model, but is less accurate than possible in predicting new instances) and can calculate a level of confidence in predicting the outcome of new instances of game players. When partitioning the data using sampling, it is important to set the random seed because… other analysts who repeat the same steps will get the same answer (i.e. “results are reproducible”). Other researchers are thereby able to verify that we have done the steps correctly. A screenshot of the resulting decision tree can be seen below:

Evaluation A screenshot of the confusion matrix can be seen below:

As seen in the screenshot above, the overall accuracy of the model is 88.5% = (308 + 192) / (308 + 38 + 27 + 192) x 100 Which is also shown in the screenshot of “accuracy statistics”, below

The cells in the confusion matrix have the following meaning:

● 308 time the decisiontree predicted the player would be a “PennyPincher” and the player was, in fact, a PennyPincher

● 38 times the decisiontree predicted the player would be a “PennyPincher” and the player was, in fact, a HighRoller

● 27 times the decisiontree predicted the player would be a “Highroller” and the player was, in fact, a PennyPincher

● 192 times the decisiontree predicted the player would be a “HighRoller” and the player was, in fact, a HighRoller.

Analysis Conclusions The final KNIME workflow is shown below:

What makes a HighRoller? iPhone users tend to be HighRollers. All other users tend to pinch pennies.

Specific Recommendations to Increase Revenue

1. Display ads for highprice items to iPhone users.

2. For Mac and Android users, start with displaying ads for low price items, but add occasional ads for highprice items the longer they play.

3. For Windows and Linux users, start with displaying ads for low price items, but an occasional ad for an iPhone as they spend more months with the game.

Clustering Analysis (pySpark) Attribute Selection Goal: To maximize the value of the game to Eglence, vendors of ingame purchases, and advertisers by encouraging the users who make purchases to stay engaged with the game for more hours per day, and stay active as members for more months.

Attribute Rationale for Selection

timeOfDay Description Extract time of day from the timestamp for the start of each session. Rationale 1) Eglence can easily price advertising by timeofday charging more

for the timeofday that provides the most sales. This will provide an early revenue stream with which to finance further development.

2) The logic for this measure can be leveraged to calculate other measures going forward, such as the ratio of buyclicks to gameclicks. Thus the logic we deploy now, will provide a foundation for further calculations going forward

Months As Member

Description Calculate the number of months that the user has been a member of “Catch the Pink Flamingo”, then count the number of purchases per session for each member Rationale Players will follow a lifecycle with use initially rising as they gain experience, and then dropping off as they tire with the game. Explicitly monitoring the lifecycle will help Eglence design the game to engage users faster and keep them longer

TeamSz Description Count the number of members in each team Rationale Social interaction can make the game more interesting and promote member retention. Bad behavior can also cause players to leave sooner. By specifically monitoring team size, Eglence can develop an understanding of features that promote positive social interaction among members

PlatformCnt

Description Count the number of different platform types a user uses Rationale Developing for a platform that generates few purchases (like Windows mobile and Linux) may seem unprofitable. However, before we reduce the development budget for Windows and Linux, we will check to see if users mix platforms.

Training Data Set Creation The training data set used for this analysis is shown below (first 5 lines):

Dimensions of the training data set (rows x columns) : 590 x 5

# of clusters created: 6

Cluster Centers

Cluster # Cluster Center

1 monthsAsMember=11, timeOfDay=5, teamSz=130, platformCnt=1.02






These clusters can be differentiated from each other as follows: Cluster 1 is different from the others in that… Players start playing at 5:00 am, and have been members for 11 months

Cluster 2 is different from the others in that… Players have been members for 20 months, have teams of 111 users (smallest team size of the clusters), and start playing midmorning Cluster 3 is different from the others in that… Players have been members for 9 months, have teams of 149 users (largest team size of the clusters), and start playing midmorning Cluster 4 is different from the others in that… Players have been members for 34 months, have teams of 143 users, and start playing midmorning Cluster 5 is different from the others in that… Players have been members for 39 months, have teams of 118 users, and start playing midmorning Cluster 6 is different from the others in that… Players have been members for 11 months, start playing in the evening, and have teams of 130 users.

Recommended Actions Goal: Keep users engaged in the game so they play regularly and remain active longer. Predictability and consistency makes the community attractive to advertisers and vendors.

Action Recommended Rationale for the action

Drop Windows Mobile and Linux

Most users never changed platform and the number of users a user used does not appear to have a significant impact on clustering. I suggest Eglence not spend money developing for a platform that attracts few users, vendors, and advertisers.

Differentiate users by number of months as active members

There seems to be significant clustering around 9, 11, 20, and 34+ months. I suggest Eglence offer recognition/promotions for users (and advertise the promotions to all users) who reach 6 months, 1 year, 2 years, 3 years of active play, and make sure that these users find each other by offering special teams and competitions. This will encourage users to stay engaged

Differentiate users by timeofday of play

There seems to be significant clustering around 5:00am, midmorning, and 5:00pm. Suggest Eglence conduct live events around these hours to encourage users to interact with each other as this will keep users engaged in the game.

Consider realworld physical activities to appeal to older players

The reported dateofbirth shows an even distribution of players aged from 18 to 70 with 20% being over the age of 55. The pattern of engagement… active use lasting years and play starting midmorning with a cluster at 5:00am… suggests retired

people. This is older than the DOB in users.csv, but who doesn’t want to be perceived as younger than they really are? I suggest Eglence provide activity guides that caregivers for seniors can download from the Eglence website for physical realworld activities to accompany playing the game online, and write blog posts on the mental benefits of game play. This would make more seniors aware of the game and encourage players to stay engaged.

Graph Analytics Analysis (neo4j) Modeling Chat Data using a Graph Data Model

The image below shows the nodes and relationships that Neo4j will contain after the chat data

is loaded. The table explains the relationships and nodes.

Entity Description Relationships

TeamChatSession

A session in which team members can see each others chats

(:user)[:CreateSession]>(:TeamChatSession) (:TeamChatSession)<[:OwnedBy](:Team) (:user)[:Joins]>(:TeamChatSession) (:user)[:Leaves]>(:TeamChatSession)

ChatItem One instance of a chat text

(:user)[:CreateChat]>(:ChatItem) (:ChatItem)[:PartOf]>(:TeamChatSession) (:ChatItem)[:Mentioned]>(:user) (:ChatItem)[:RespondsTo]>(:ChatItem)

Team

User (:user)[:InteractsWith]>(:user)

Creation of the Graph Database for Chats

Describe the steps you took for creating the graph database.

1. The table below describes the schema of the csv files that contain the chat data

CSV File Description Fields

chat_create_ team_chat.csv

A line is added to this fiel when a player creates a new chat session for their team. Each record creates a node labeled, “TeamChatSession”, and edges labeled, “CreateSession” from user “OwnedBy” to Team

userId: id for the user who initiated the session

teamId: id for the team teamSessionId: id for the

session Timestamp: create

date/time for the session

chat_item_ team_chat.csv

A line is added to this file when a user adds a chat item to a team chat session. This record creates the node labeled “chatItem” and the edges labeled “createChat” from user, “PartOf” to teamChatsession, and “Joins” from user to teamChatSession

userId: id for the user who created the chat item

teamChatSessionId: id for the chat session in which the chat item was created

chatItem: id the cat item Timestamp: create

date/time for the chat item

chat_join_ team_chat.csv

A line is added to this file when a player joins a team chat session. This record creates an edge labeled “joins” from User to teamChatSession

Userid: id of the user who joined the team chat session

TeamChatSessionId: id of the team chat session

Timestamp: data/time user left

chat_leave_ team_chat.csv

A line is added to this file when a player leaves a team chat session. This record creates an

Userid: id of the user who left the team chat session

edge labeled “leaves” from User to teamChatSession

TeamChatSessionId: id of the team chat session

Timestamp: data/time user left

chat_mention_ team_chat.csv

A line is added to this file when a player mentions another player in a chat post. The record creates an edge labeled “Mentioned” from chatItem to User.

chatItemId: id of chat item

userId: id of user mentioned in chat

timestamp: date/time of mention

chat_respond_ team_chat.csv

A line is added to this file when a player responds to a chat post. The record creates an edge labeled, “ReponseTo” from ChatItem to ChatItem

chatItemId: source chat item

chatItemId: chat responding to source chat item

Timestamp: date/time of response

2. To load the chat data, execute the following steps

● Download the data from the designated repository ● Unzip the chat data to the neo4j default import folder

D:\Users\Greg.B.Watson\Documents\Neo4j\default.graphdb\Import You should have the six files described in the table above

● Execute the following commands in Neo4j to load the csv files to the graph database.

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_create_team_chat.csv" AS row MERGE (u:User {id: toInt(row[0])}) MERGE (t:Team {id: toInt(row[1])}) MERGE (c:TeamChatSession {id: toInt(row[2])}) MERGE (u)[:CreatesSession{timeStamp: row[3]}]>(c) MERGE (c)[:OwnedBy{timeStamp: row[3]}]>(t)

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_join_team_chat.csv" AS row MERGE (u:User {id: toInt(row[0])}) MERGE (s:TeamChatSession {id:

toInt(row[1])}) MERGE (u)[:Joins{timeStamp: row[2]}]>(s)

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_leave_team_chat.csv" AS row MERGE (u:User {id: toInt(row[0])}) MERGE (s:TeamChatSession {id: toInt(row[1])}) MERGE (u)[:Leaves{timeStamp: row[2]}]>(s)

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_item_team_chat.csv" AS row WITH row LIMIT 0 MERGE (:User {id: toInt(row[0])})[:CreateChat{timeStamp: row[3]}]>(:ChatItem {id: toInt(row[2])}) MERGE (:ChatItem {id: toInt(row[2])})[:PartOf{timeStamp: row[3]}]>(:TeamChatSession {id: toInt(row[1])}) MERGE (:User {id: toInt(row[0])})[:Joins{timeStamp: row[3]}]>(:TeamChatSession {id: toInt(row[1])})

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_item_team_chat.csv" AS row MERGE (u:User {id: toInt(row[0])}) MERGE (s:TeamChatSession {id: toInt(row[1])}) MERGE (c:ChatItem {id: toInt(row[2])}) MERGE (u)[:CreateChat{timeStamp: row[3]}]>(c) MERGE (c)[:PartOf{timeStamp: row[3]}]>(s) MERGE (u)[:Joins{timeStamp: row[3]}]>(s)

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_mention_team_chat.csv" AS row MATCH (c:ChatItem {id: toInt(row[0])}) MATCH (u:User {id: toInt(row[1])}) MERGE (c)[:Mentioned{timeStamp: row[2]}]>(u)

USING PERIODIC COMMIT 1000 LOAD CSV FROM "file:///chat_respond_team_chat.csv" AS row MATCH (a:ChatItem {id: toInt(row[0])}) MATCH (b:ChatItem {id: toInt(row[1])}) MERGE (a)[:ResponseTo{timeStamp: row[2]}]>(b)

The screenshot below was created with the following query that displays every node and relationship in the database. Teams (purplishpink) create teamchatsessions (green) and users (orangish pink) create a cloud of chatitems (grey)

The image below shows the same query with the limit set to 100 so that we can see the relationships.

Finding the longest conversation chain and its participants

Report the results including the length of the conversation (path length) and how many unique users were part of the conversation chain. Include an image of the graph with the longest conversation chain.

Ten (10) chats were in the longest conversation chain.

As shown in the screenshot below, the 10 chats involved 5 users.

Analyzing the relationship between top 10 chattiest users and top 10 chattiest teams

Include your table containing the top 3 chattiest users and teams below, and report whether or not any of the chattiest users are part of any of the chattiest teams.

Chattiest Users

Users Number of Chats Userid 394 (team 63) 115 Userid 2067 (team 7) 111 Userid 209 (team 7) 109

Chattiest Teams

Teams Number of Chats 82 1324 185 1036 112 957

Userid 999 belongs to team 52. Both user 999 and team 52 were in the top 10 chattiest users and teams respectively. User 999 is the only one of the top 10 chattiest users to belong to one of the top 10 chattiest teams.

How Active Are Groups of Users?

Report the top 3 most active users in the table below.

Steps to rank the activeness of groups of users:

1. Construct ‘neighborhoods’ of users who either mention each other or respond to each other: A user is a member of neighborhood if the user has an 'interactsWith' relationship with anyone in the neighborhood.

The number of distinct users in the neighbourhood is k. k(k1) is the maximum number of 'interactsWith' relationships one user can have. We subtract 1 from k because the user cannot chat with himself. We multiply because it does not matter which user initiated the interaction.

The number of distinct users one user actually interactsWith is n

MATCH (u1:User)[:CreateChat]>(:ChatItem)[:Mentioned]>(u2:User) CREATE UNIQUE((u1)[:InteractsWith]>(u2))

MATCH (u1:User)[:CreateChat]>(:ChatItem)[:ResponseTo]>(:ChatItem)<[:CreateChat](u2:User) CREATE UNIQUE((u1)[:InteractsWith]>(u2))

2. Remove any ‘InteractsWith’ relationships where the user interacts with himself or herself. An ‘InteractsWith’ relationship exists when a ‘ChatItem’ either ‘Mentioned’ another user or ‘RepondedTo’ a ‘ChatItem’ from another user.

Since it is technically possible for a user to respond to his/her own chat items, and we are not interested in users influencing themselves, we have to remove the self loops.

MATCH (u1)[r:InteractsWith]>(u1) DELETE r

3. Calculate the clustering coefficient. The ‘clustering coefficient’ ranges from 0 (an isolated user) to 1 (a user in a group where every user is connected to every other user)

The ratio n/k(k1) is the cluster coefficient

ToDo: Find the number of users in the same neighborhood that each user DID NOT chat with, add that to the number each did chat with to get the number of neighbors

The query to find the top 3 most active users is as follows. Note that to make this query efficient, we assume that the top 3 most active users are among the top 10 chattiest: We hardcode the top 10 list and check that the users are in that list before calculating the coefficient.

MATCH (u1:User)[r:InteractsWith](u2:User) WHERE u1.id IN [394,2067,209,1087,554,516,1627,999,668,461] WITH u1 AS u, COLLECT(DISTINCT(u2.id)) AS neighbors MATCH (u1:User),(u2:User) WHERE u1.id IN neighbors AND u2.id IN neighbors RETURN u.id,

TOFLOAT(SUM(CASE WHEN (u1)(u2) THEN 1 ELSE 0 END)) /(SIZE(neighbors)*(SIZE(neighbors)1)) AS coefficient

ORDER BY coefficient DESC

Most Active Users (based on Cluster Coefficients)

User ID Coefficient 461 1 394 1 516 / 209 0.952

Recommended Actions

Finally, make recommendations to Eglence, Inc. and include examples of how your findings support

them. Include this information in Slide 6 of your final presentation.

● Give promotions to the users with the highest chat coefficient in each timeofday cluster: High chatcoefficient users are likely to influence the largest number of other users to buy,

and are most likely to influence users who regularly log on at the same time of day as they do.

● Analyze further which users get chat “mentions” from other teams and why: We can

improve game play by discouraging behavior that puts off players, and encourage players with

whom players on other teams

want to play and/or want to imitate