ad serving at spotify scale · ad server decision delivery ad exchanges campaign management...
TRANSCRIPT
![Page 1: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/1.jpg)
Ad Serving at Spotify ScaleA journey of incremental full stack overhaul
Kinshuk Mishra, Director of [email protected]@_kinshukmishra
![Page 2: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/2.jpg)
A lucky mistake
![Page 3: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/3.jpg)
Expected consequences
![Page 4: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/4.jpg)
Sarcastic empathy
![Page 5: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/5.jpg)
Some valuable feedback
![Page 6: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/6.jpg)
The unintended consequence
Artist engagement for exposed users went up
![Page 7: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/7.jpg)
The unintended consequence
Promising insights about content promotion use-case
![Page 8: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/8.jpg)
The unintended consequence
Confirmation that the ad server is a powerful
messaging platform
![Page 9: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/9.jpg)
Why should you care?
![Page 10: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/10.jpg)
Introduction
Ad technology
stack
Architecture Evolution
![Page 11: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/11.jpg)
Introduction
Ad technology
stack
Architecture Evolution
![Page 12: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/12.jpg)
What I do● Founded ads engineering team at Spotify in 2011
● Build all things ads engineering - team & software
● Major focus areas :
○ Ad delivery (Backend and Web)
○ Multi-platform native ads (Client Platform)
○ Ad performance (ML and Data)
![Page 13: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/13.jpg)
3 noteworthy things
![Page 14: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/14.jpg)
Full stack refactor
Evolution at scale
Pragmatic choices
![Page 15: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/15.jpg)
100,000,000+ MAU
![Page 16: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/16.jpg)
50,000,000+Subscribers
![Page 17: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/17.jpg)
30,000,000+ Songs
![Page 18: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/18.jpg)
2,000,000,000+ Playlists
![Page 19: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/19.jpg)
$5,000,000,000+ Revenue paid to rightsholders
![Page 20: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/20.jpg)
60 Markets
![Page 21: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/21.jpg)
Platform Ubiquity
![Page 22: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/22.jpg)
Freemium business model
![Page 23: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/23.jpg)
Ad
![Page 24: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/24.jpg)
![Page 25: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/25.jpg)
![Page 26: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/26.jpg)
![Page 27: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/27.jpg)
Introduction
Ad technology
stack
Architecture Evolution
![Page 28: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/28.jpg)
Beauty of Ad Server
Relevancy Pacing Unique View Sequence Optimization
![Page 29: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/29.jpg)
Complexity of Ad tech ecosystem
![Page 30: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/30.jpg)
In essence it is pretty simple
Client
User Profile database
Ad Server
Campaign Management Portal
Billing/Reporting
Ad campaign database
Data CollectionSystem
![Page 31: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/31.jpg)
Spotify Ads infrastructure in 2011
EdgeServiceDesktop
LogDelivery HDFS
User Profile
Batch
Basic Ad Server
CampaignManagement
Billing/Reporting
![Page 32: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/32.jpg)
Spotify Ads infrastructure in 2017
iOS
EdgeService
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
Optimization
ModelingSelf-Serve Portal
Creative Generation Payments
Billing/Reporting
![Page 33: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/33.jpg)
Multi-platform clients
iOS
EdgeService
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
Optimization
ModelingSelf-Serve Portal
Creative Generation Payments
Billing/Reporting
![Page 34: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/34.jpg)
Data collection
iOS
EdgeService
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
Optimization
ModelingSelf-Serve Portal
Creative Generation Payments
Billing/Reporting
![Page 35: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/35.jpg)
Intelligence
iOS
EdgeService
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
Optimization
ModelingSelf-Serve Portal
Creative Generation Payments
Billing/Reporting
![Page 36: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/36.jpg)
Ad Delivery
iOS
EdgeService
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
Optimization
ModelingSelf-Serve Portal
Creative Generation Payments
Billing/Reporting
![Page 37: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/37.jpg)
Demand fulfillment
iOS
EdgeService
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
Optimization
ModelingSelf-Serve Portal
Creative Generation Payments
Billing/Reporting
![Page 38: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/38.jpg)
Now you know too
Ad server is a powerful messaging platform
![Page 39: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/39.jpg)
Introduction
Ad technology
stack
Architecture Evolution
![Page 40: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/40.jpg)
Architecture overhaul is hard
● While keeping the business running
● While innovating on new products
● When you should have done it yesterday
![Page 41: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/41.jpg)
Why did Spotify evolve Ads architecture?
![Page 42: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/42.jpg)
Future needs●
● Growth in scale
● Emergence of new client platforms
● Cheap cloud computing
● New products to meet business objectives
● Technical debt
![Page 43: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/43.jpg)
The 3 stories
![Page 44: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/44.jpg)
Fixing the legacy mess
Story 1
![Page 45: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/45.jpg)
Original ad server designEdge Service
Routerhash(userid)
Ad server ring with partitions
Ad server instance
Memcache
Memcache
Memcache
Memcache
Campaign DB
User DBDesktop
Rendering Ad trigger decisioning
Ads Ranking
Ads Caching
Ad batching & fetch communication
![Page 46: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/46.jpg)
Problems
![Page 47: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/47.jpg)
Stateful service with faulty persistence
![Page 48: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/48.jpg)
Cache as a data store
![Page 49: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/49.jpg)
Service cluster as a hashed ring
![Page 50: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/50.jpg)
Ad decisioning in Client
![Page 51: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/51.jpg)
Batch Client-Server Calls
![Page 52: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/52.jpg)
Fix strategy
![Page 53: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/53.jpg)
Fix strategy tactic
![Page 54: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/54.jpg)
Isolate refactor to one system at a time
![Page 55: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/55.jpg)
The ad server transition
EdgeService
LogDelivery HDFS
User Profile
Batch
Smart Ad Server
CampaignManagement
Billing/Reporting
Ad Server Proxy
(routing) Basic Ad Server
Gradual transition from basic to smart ad serving
Desktop
Rendering Ad trigger decisioning
Ads Ranking
Ads Caching
Ad batching & fetch communication
![Page 56: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/56.jpg)
After the ad server transition
Proxy Service
LogDelivery HDFS
User Profile
Batch
CampaignManagement
Billing/Reporting
Smart Ad Server
Desktop
Rendering Ad trigger decisioning
Ads Ranking
Ads Caching
Ad batching & fetch communication
![Page 57: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/57.jpg)
Lean, mean and fast
Story 2
![Page 58: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/58.jpg)
Division of responsibilities
Desktop iOS
Android
Ads SDK
Desktop
Web
Rendering Ad trigger decisioning
Ads Ranking
Ads Caching
Ad batching & fetch communication
Ad decisioning
Ad fetch orchestration
Client context
Ad Trigger & Render
Before After
![Page 59: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/59.jpg)
Problems
![Page 60: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/60.jpg)
Thick Clients
![Page 61: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/61.jpg)
Logic duplication
![Page 62: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/62.jpg)
Tightly coupled monolith
![Page 63: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/63.jpg)
Fix strategy
![Page 64: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/64.jpg)
Reduce State Management
![Page 65: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/65.jpg)
Break monolith into services
![Page 66: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/66.jpg)
Isolate platform independent logic into a lib
![Page 67: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/67.jpg)
Fix tactic
![Page 68: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/68.jpg)
Design your systems to be master of one thing
![Page 69: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/69.jpg)
Remember division of responsibilities?
Desktop iOS
Android
Ads SDK
Desktop
Web
Rendering Ad trigger decisioning
Ads Ranking
Ads Caching
Ad batching & fetch communication
Ad decisioning
Ad fetch orchestration
Client context
Ad Trigger & Render
BAD GOOD
![Page 70: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/70.jpg)
Multiplatform Client design
iOS
Proxy Service
Android
Ads SDK
Desktop
Web
Chromecast/Playstation/
FireTV
Ad Aggregation
Service
LogDelivery GCS
User Profile
Targeting Service
DMP
Stream Batch
Ad Server
Decision Delivery Ad Exchanges
CampaignManagement
ModelingSelf-Serve Service
Creative Generation Payments
Billing/Reporting
![Page 71: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/71.jpg)
Knowledge is power, Unreliable data is your enemy
Story 3
![Page 72: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/72.jpg)
Event Stream Historical
ETL1 ETL2 ETL3
UserEntity1(attribute1, attribute2) UserEntity1(attribute1, attribute3) UserEntity1(attribute1, attribute3’)
![Page 73: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/73.jpg)
![Page 74: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/74.jpg)
Problems
![Page 75: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/75.jpg)
Duplicate, undiscoverable and fragmented datasets
![Page 76: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/76.jpg)
Metric inaccuracy
![Page 77: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/77.jpg)
Overloaded Data Infra
![Page 78: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/78.jpg)
Fix strategy
![Page 79: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/79.jpg)
Focus on reliable and timely log delivery
![Page 80: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/80.jpg)
![Page 81: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/81.jpg)
Data engineering with SLA
![Page 82: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/82.jpg)
Dataset canonicalization
![Page 83: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/83.jpg)
Some useful lessons learnt from architectural overhaul
![Page 84: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/84.jpg)
Test with minimal impact radius
![Page 85: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/85.jpg)
Mistakes are inevitable
![Page 86: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/86.jpg)
Speed up build decisions
![Page 87: Ad Serving at Spotify Scale · Ad Server Decision Delivery Ad Exchanges Campaign Management Optimization Modeling Self-Serve Portal Creative Generation Payments Billing/ Reporting](https://reader034.vdocuments.mx/reader034/viewer/2022043009/5f9aa94f5d8d891a2f313e07/html5/thumbnails/87.jpg)
Think for tomorrow, Solve for today