"is data science inhibiting product evolution?", steven lemm, chief data officer at...
TRANSCRIPT
![Page 1: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/1.jpg)
Is data science inhibiting product development?
Steven LemmChief Data Officer @ Outfittery GmbH
![Page 2: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/2.jpg)
[email protected] | Data Natives 20162
WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)
![Page 3: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/3.jpg)
[email protected] | Data Natives 20162
WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)
Example e-commerce platformFrontend products: registration/authentication, check-out, recommender engine, article search, …Backend products: inventory control, order engine, logistic processes, payment service, payment control, …
![Page 4: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/4.jpg)
[email protected] | Data Natives 20162
WHAT’S A PRODUCTDefinitiona product is something that is created and then made available to external or internal customers (eg. a software application)
Regis-tration
article search
article details
check-out
payment control
Example e-commerce platformFrontend products: registration/authentication, check-out, recommender engine, article search, …Backend products: inventory control, order engine, logistic processes, payment service, payment control, …
For the rest of the talk consider a simplified platform
![Page 5: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/5.jpg)
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
PRODUCT DEVELOPMENT
3
![Page 6: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/6.jpg)
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
PRODUCT DEVELOPMENT
3
Regis-tration
article search
article details
check-out
payment control
How to measure additional benefits?
![Page 7: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/7.jpg)
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
KPI: search rank
KPI: revenue
PRODUCT DEVELOPMENT
3
Regis-tration
article search
article details
check-out
payment control
How to measure additional benefits?
![Page 8: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/8.jpg)
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
KPI: revenue
article searchVer. B
KPI: search rank
PRODUCT DEVELOPMENT
3
How to measure additional benefits? A/B testing!
![Page 9: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/9.jpg)
[email protected] | Data Natives 2016
Definitionmodification of products with new or different characteristics that offer new or additional benefits to the customer.
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
KPI: revenue
article searchVer. B
KPI: search rank
PRODUCT DEVELOPMENT
3
How to measure additional benefits? A/B testing!
![Page 10: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/10.jpg)
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article search
article details
check-out
payment control
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score
payment control
![Page 11: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/11.jpg)
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article search
article details
check-out
payment control
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
![Page 12: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/12.jpg)
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article search
article details
check-out
payment control
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
Conversion
PnL
![Page 13: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/13.jpg)
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
![Page 14: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/14.jpg)
[email protected] | Data Natives 2016
Example smart payment control
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
![Page 15: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/15.jpg)
[email protected] | Data Natives 2016
Conversion
Example smart payment control
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT CHANGES WITH DATA SCIENCE?
4
ext. solvency score+ cust. engagement
![Page 16: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/16.jpg)
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Conversion
![Page 17: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/17.jpg)
[email protected] | Data Natives 2016
Hint 1: create a thorough monitoring of the data inputs, e.g.
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Conversion
![Page 18: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/18.jpg)
[email protected] | Data Natives 2016
Hint 1: create a thorough monitoring of the data inputs, e.g.
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Hint 2: make payment control dependency on article search explicit
Conversion
![Page 19: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/19.jpg)
[email protected] | Data Natives 2016
Hint 1: create a thorough monitoring of the data inputs, e.g.
Regis-tration
article details
check-out
payment control
article searchVer. B
HOW TO PROTECT AS PAYMENT?
5
ext. solvency score+ cust. engagement
Hint 3: document the dependencies and educate product managersHint 2: make payment control dependency on article search explicit
Conversion
![Page 20: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/20.jpg)
[email protected] | Data Natives 2016
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
![Page 21: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/21.jpg)
[email protected] | Data Natives 2016
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
Step 0: connect with product manager of payment control first
![Page 22: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/22.jpg)
[email protected] | Data Natives 2016
Regis-tration
article searchVer. A article
detailscheck-
outpayment control
Revenue
article searchVer. B
search rank
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score
payment control
Step 0: connect with product manager of payment control first
![Page 23: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/23.jpg)
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score
payment control
Step 2: deploy article search and collect training data for new payment control logic
Step 0: connect with product manager of payment control first
![Page 24: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/24.jpg)
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
Step 2: deploy article search and collect training data for new payment control logicStep 3: train and deploy an updated payment control logic
Step 0: connect with product manager of payment control first
![Page 25: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/25.jpg)
[email protected] | Data Natives 2016
Regis-tration
article details
check-out
payment control
article searchVer. B
Step 1: make payment control independent on article search and run A/B test
WHAT TO DO AS SEARCH?
6
ext. solvency score+ cust. engagement
Step 2: deploy article search and collect training data for new payment control logicStep 3: train and deploy an updated payment control logic
Step 0: connect with product manager of payment control first
THIS MAKES PRODUCT DEVELOPMENT VERY EXPENSIVE
![Page 26: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/26.jpg)
[email protected] | Data Natives 2016
… Excecutives:• do not underestimate the costs of data science - there is no free lunch
•prioritize features according to , where total costs cover development, testing, data acqusition, data science
•bring together product management and data science as close as possible•centralise all A/B testing & personalisation efforts in a data department
… Product manager• map out all and document all data dependencies
… Data scientist• build a thorough monitoring & alerting systems for all inputs to your logics• prefer explicit over implicit data dependencies• create sparse solutions, i.e. limit the dependencies don’t take everyone’s data• educate product management
RECOMMENDATIONS TO …
7
![Page 27: "Is Data Science Inhibiting Product Evolution?", Steven Lemm, Chief Data Officer at Outfittery GmbH](https://reader031.vdocuments.mx/reader031/viewer/2022021507/58f16e2e1a28ab6a578b4581/html5/thumbnails/27.jpg)
Thank you