inmar givoni - product optimization through big data analytics and machine learning
TRANSCRIPT
Product optimization through Big Data Analytics
and Machine Learning
Inmar Givoni,
Ph.D. (Machine Learning)
VP, Big Data
About Me
● Ph.D. Computer Science (Machine Learning)
from University of Toronto ● Microsoft Research-Cambridge, SV ● VP of Big Data @ Kobo ● Multiple publications, patents ● Public speaking, outreach ● inmarg.net for more
2009 2012
About Kobo
About Kobo
>5M eBooks >350K
The Kobo catalog
Where do people read?
Where do people read?
Where do people read?
Data Sources
Using Data for
Product Design:
4 Case Studies
The importance of looking at data
The importance of looking at data
The importance of testing everything
You are not your average user!
The importance of testing everything
Gut feelings are often wrong
But post-hoc analysis can lead to great insights
The importance of testing everything
● Personal market research is useful but...
● Sample biases ● Do users always ‘know’ what they want? ● Why ask when you can get the answer?
Hypothesis (A/B) Testing
random splits
right test
statistical
significance
minimal
detection rate
sample size
Hypothesis (A/B) Testing
Hypothesis (A/B) Testing
• The right way to make business
decisions
• Put reliable numbers on products
– cannibalism, time of x effects
• Can be integrated via 3rd party
technologies
– but do A/A test!
Personalization &
Targeting Products
Personal Recos - devices & apps
Personal Recos - web
Personal Recos
● Important driver of sales & engagement
● Continuous analysis of user behaviour patterns
● Use domain specific knowledge
● Continuously improve
Personal Recos - web
Personal Recos Page - Visits
Personal Recos - weekly email
Personal Recos - triggered email
get one when done reading a book
Personal Recos - triggered email
high conversion
how to improve?
Personal Recos - triggered email
Big Data analytics - how far along?
ML - detecting the real end
Personal Recos - triggered email
Home Page Layout Optimization
Home Page Layout Optimization
2
1
4 3
5
6
2
1
4 3
5
6
2’
1
4 3
5
6
Home Page Layout Optimization
From Analytics to Data Products
● Email time of day
analysis →
trigger individually
● BI on best website assets
→
automatically optimizing
platform
Thank You!