pragmatic deep learning for image labelling - an ...iterative building of a recommender system...
TRANSCRIPT
![Page 1: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/1.jpg)
Pragmatic Deep Learning for image labelling - An application to a travel recommendation engine
主讲人:Dataiku数据科学家 Alexandre Hubert
![Page 2: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/2.jpg)
Introduction and Context
Iterative building of a recommender system
Labeling ImagesPragmatic deep learning for dummies
Post Processing AKA: Image for BI on steroids
OutlineResultsMore images !
![Page 3: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/3.jpg)
Dataiku
• Founded in 2013 • 90 + employees, 100 + clients • Paris, New-York, London, San
Francisco, Singapore
Data Science Software Editor of Dataiku DSS
DESIGN
Load and prepareyour data
PREPAREBuild your
models
MODELVisualize and share
your work
ANALYSE
Re-execute your workflow at ease
AUTOMATEFollow your production
environment
MONITORGet predictions
in real time
SCOREPRODUCTIO
N
![Page 4: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/4.jpg)
E-business vacation retailerNegotiate the best prize for their clientsDiscount luxury
•Key Figures
Sale Image is paramountPurchase is impulsive
18 Millions of clients. Hundreds of sales opened everyday
![Page 5: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/5.jpg)
Specificities
Highly temporary sales
-> Classical recommender system fail
-> Time event linked (Christmas, ski, summer)
Expensive Product-> Few recurrent buyers-> Appearance counts a lot
![Page 6: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/6.jpg)
Iterative Building of a Recommender System
![Page 7: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/7.jpg)
Basic Recommendation Engines
![Page 8: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/8.jpg)
Other Factors
![Page 9: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/9.jpg)
One Meta Model to Rule Them All
Recommendersasfeatures
Machinelearningtooptimizepurchasingprobability
Combine
Recommend
Describe
![Page 10: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/10.jpg)
Cleaning, combining and enrichment of data
Recommendation Engines
Optimization of home display
the application automatically runs and
compiles heterogeneous data
Generation of recommendations based on
user behaviour
Every customer is shown the 10 sales he is the most likely to buy
Customer visitsPurchases
Sales Images
Metal model combine recommendations to
directly optimize purchasing probability
Meta Model
Recommender system for Home Page Ordering
+7% revenue
Sales information
(A/B testing)
Batch Scoring every night
![Page 11: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/11.jpg)
Why use Image ?
We want do distinguish
« Sun and Beach »
« Ski »
A picture is worth a thousand words
![Page 12: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/12.jpg)
Sales Images
Integrating Image Information
Labeling Model
Pool + Palm Trees Hotel+ Mountains
Pool + Forest + Hotel + Sea
Sea + Beach +Forest + Hotel
Sales descriptions vector
CONTENTBASED
Recommender System
![Page 13: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/13.jpg)
Image Labelling For Recommendation EnginePragmaticDeeplearningfor“Dummies”
![Page 14: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/14.jpg)
Using Deep Learning modelsCommon Issues
“I don’t have GPUs server”
“I don’t have a deep leaning expert”
“I don’t have labelled data” (or too few) “I don’t have the time to wait for model training ”
I don’t want to pay to pay for private apis” / “I’m afraid their labelling will change over time”
![Page 15: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/15.jpg)
“I don’t have (or few) labelled data” -> Is there similar data ?
Solution 1 : Pre trained models
PLACESDATABASEUS SUNDATABASE
205categories2.5Mimages
307categories110Kimages
![Page 16: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/16.jpg)
tower: 0.53skyscraper: 0.26
swimming_pool/outdoor: 0.65inn/outdoor: 0.06
Solution 1 : Pre trained models If there is open data, there is an open pre trained model !• Kudos to the community• Check the licensing
ExamplewithPlaces(CaffeModelZoo):
![Page 17: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/17.jpg)
Solution 2 : Transfer Learning
Credit:Fei-FeiLi&AndrejKarpathy&JustinJohnsonhttp://cs231n.stanford.edu/slides/winter1516_lecture11.pdf
![Page 18: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/18.jpg)
PLACESDATABASE OURDATASUNDATABASE
Training(optional)
Pre-trainedmodelVGG16
tower: 0.53skyscraper: 0.26
Re-Training
TransferredData:Lastconvolutionallayerfeatures
Re-trainedmodelTensorFlow
2fullyconnectedlayers
CaffeModelZoo
GPU
CPU
GPU
Leverage existing knowledge !
Solution 2 : Transfer Learning
Accuracy:72%,Top-5Acc:90%>stateoftheartondatasetalone
![Page 19: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/19.jpg)
Post Treatment & Results
(Or how we transfer the labelling information)
UsingImagesinformationforBIonsteroids
![Page 20: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/20.jpg)
Labels post-processing
Complementary information Redondant information
Issue with our approach:
Solution : NMF Matrix Factorization
DimensionReduction ExplicabilitySparsity Balancedness
![Page 21: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/21.jpg)
Image content detectionTopic scores determine the importance of topics in an image
TOPIC TOPICSCORE(%)
Golfcourse– Fairway– Puttinggreen 31
Hotel– Inn– Apartmentbuildingoutdoor 30
Swimmingpool– LidoDeck– Hottuboutdoor
22
Beach– Coast- Harbor 17
TOPIC TOPICSCORE(%)
Tower– Skyscraper– Officebuilding 62
Bridge– River– Viaduct 38
![Page 22: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/22.jpg)
Results ? 1) Visits :
• France and Morocco• Pool displayed
2) First Recommendation • Mostly France & Mediterranean • Fails to display pools
3) Only Images recommendation • Pool all around the world• Does not respect budget
4) Third column = Right Mix
1) 2) 3) 4)
![Page 23: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/23.jpg)
Conclusion
Do iterative data science !Start simple and grow Evaluate at each stepsImage labelling = BI on steroids
Transfer LearningKick-start your projectGain time and moneyAny Data Scientist can do it
Deep Learning Don’t start from scratch !Is there existing data ? Is there a pre-trained model ?
![Page 24: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/24.jpg)
Learned along the way
What’s next ?
Attractiveness=%visitswithtag/%saleswithtag
Forskisales,indoorpicturesperformsbetter
![Page 25: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/25.jpg)
What’s Next ?
Kenya
Prague
Berlin
Cambodia
![Page 26: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/26.jpg)
What’s Next ? Customize the Image !
Kenya
Prague
Berlin
Cambodia
![Page 27: Pragmatic Deep Learning for image labelling - An ...Iterative building of a recommender system Labeling Images Pragmatic deep learning for dummies Post Processing ... I don’t want](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601eec06bebb786457527bfa/html5/thumbnails/27.jpg)
Thank you for your attention !