soundcloud's toolbox for microservices
TRANSCRIPT
SoundCloud’s Toolbox for Microservices
SoundCloud is the largest online audio distribution platform
12 HOURS OF AUDIOevery minute
Bora Tunca
Joined SoundCloud 2.5 years ago
soundcloud.com front-end
client specific apis
core-engineering
@grandbora
Mic
rose
rvic
es
0
20
40
60
80
100
120
2013 2014 2015 2016
Architecture
Productivity
Monitoring
FINAGLE
Finagle
Future
Services
Filters
https://monkey.org/~marius/funsrv.pdfYour Server as a Function
Service
Req Future[Rep]
Req => Future[Rep]Service
Service
Req Future[Rep]ReqA Future[RepA]
(ReqA, Service) => Future[RepA]Filter
Filter
Service
Req Future[Rep]ReqA Future[RepA]
(ReqA, Service) => Future[RepA]Filter
Filter + Service
val service = new ExceptionFilter andThen new HeaderFilter andThen new HttpTimeoutFilter(Duration(1, TimeUnit.SECONDS)) andThen new RestService()
Toolbox
Finagle
CLIENT SPECIFIC APIS
MOTHERSHIP
Public API
messages
likes
Api-v2 Api-mobile Api-embedded Public API
Api-v2 Api-mobile Api-embedded Public API
messages
MOTHERSHIPlikes
search
stats
images
Backend For Fronted
Toolbox
Client Specific ApisFinagle
STRANGLER PATTERN
Api-v2 Api-mobile Api-embedded Public API
messages
MOTHERSHIPlikes
search
stats
images
Api-v2 Api-mobile Api-embedded Public API
messages
MOTHERSHIPlikes
search
stats
images
Api-v2 Api-mobile Api-embedded Public API
messages
MOTHERSHIPlikes
search
stats
images
MOTHERSHIP
Legacy Public API
MOTHERSHIP
Legacy Public API
Public API Strangler
MOTHERSHIP
Legacy Public API
Public API Strangler
messages
likes
search
MOTHERSHIP
Legacy Public API
Public API Strangler
messages
likes
search
stats
images
MOTHERSHIP
Legacy Public API
Public API Strangler
messages
likes
search
stats
images
Toolbox
Strangler PatternClient Specific ApisFinagle
CRITICAL PATH
BFF
BFF authentication geo location features
1
BFF
messages
MOTHERSHIPlikes
search
stats
images
2
authentication geo location features
1
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
BFF
messages
MOTHERSHIPlikes
search
stats
images
2
authentication geo location features
1
Microservices
BFFs
Microservices
Client ApplicationsBFFs
Microservices
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
BFF
BFFBFF
BFFBFF
messages
MOTHERSHIP
BFF
likes
search
stats
images
authentication geo location features
BFF
BFFBFF
BFFBFF
Critical Path
On critical path…
Handle failures gracefully
On critical path…
Handle failures gracefully
Be ready for traffic bursts
On critical path…
Toolbox
Critical Path
Client Specific ApisStrangler Pattern
Finagle
SERVICE LAYERING
Public API
messages
likesMOTHERSHIP
messages
MOTHERSHIP
BFF
likes
search
stats
images
messages
MOTHERSHIP
BFF
likes
search
stats
images
BFF
messages
MOTHERSHIP
BFF
likes
search
stats
images
BFF
messages
MOTHERSHIP
BFF
likes
search
stats
images
tracks
BFF
images userstracks
Foundation Layer
images userstracks
Foundation Layer
playlistsstream profiles
Value Added Layer
images userstracks
Foundation Layer
playlistsstream profiles
Value Added Layer
images userstracks
Foundation Layer
playlistsstream profiles
Value Added Layer
images userstracks
Foundation Layer
playlistsstream profiles
Value Added Layer
BFF BFF
Edge Layer
images userstracks
Foundation Layer
playlistsstream profiles
Value Added Layer
BFF BFF
Edge Layer
images userstracks
Foundation Layer
playlistsstream profiles
Value Added Layer
MOTHERSHIP
BFF BFF
Edge Layer
Toolbox
Service Layering
Client Specific ApisStrangler PatternCritical Path
Finagle
HUMANE REGISTRY
Mic
rose
rvic
es
0
20
40
60
80
100
120
2013 2014 2015 2016
Humane Registry
Oriented around people
Humane Registry
Oriented around people
Don't rely on people to stay up to date
Humane Registry
Oriented around people
Don't rely on people to stay up to date
Expect humans to read and contribute
Humane Registry
SERVICES DIRECTORY
Github
PagerDuty
Distributed Tracing
Prometheus
Deployment Pipeline
Eng-doc
Live Configs
Toolbox
Humane Registry
Client Specific ApisStrangler PatternCritical PathService Layering
Finagle
PROMETHEUS
Prometheus
Scalable data collection
Operational simplicity
Multi-dimensional data
Powerful query language
https://developers.soundcloud.com/blog/prometheus-monitoring-at-soundcloudhttp://prometheus.io/
Prometheus
My Service
Prometheus
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
My Service
jvm client
val requestCount = telemetry.counter("outgoing_http_requests_total", "A counter for the HTTP requests", "service", "method", "status", "statusClass")
send
requestCount.labels( serviceName, request.getMethod, response.getStatus, response.getStatusClass).inc()
val requestCount = telemetry.counter("outgoing_http_requests_total", "A counter for the HTTP requests", "service", "method", "status", "statusClass")
send
requestCount.labels( serviceName, request.getMethod, response.getStatus, response.getStatusClass).inc()
query SUM(rate(outgoing_http_requests_total{statusClass="5xx"}[10m])) by (service, method, status)
Toolbox
Client Specific ApisStrangler PatternCritical PathService LayeringHumane RegistryPrometheus
Finagle
SERVICE LEVEL OBJECTIVES
ERROR_RATE=0.0001
ERROR_RATE=0.0001P99_LATENCY=0.500
ERROR_RATE=0.0001P99_LATENCY=0.500AVAILABILITY_TARGET=0.99999
ERROR_RATE=0.0001P99_LATENCY=0.500AVAILABILITY_TARGET=0.99999
Toolbox
Service Level ObjectivesPrometheus
Client Specific ApisStrangler PatternCritical PathService LayeringHumane Registry
Finagle
RECAP
Toolbox
Service Level ObjectivesPrometheus
Client Specific ApisStrangler PatternCritical PathService LayeringHumane Registry
Finagle
backstage.soundcloud.com
@pcalcado - How we ended up with microservices
@starcoffe - Why SoundCloud stopped using its own public API
InfoQ - Moving from a Monolith to Microservices at SoundCloud
@SoundCloudDev
InfoQ - Microservices Evolution at Soundcloud
Sam Newman - Backends for Frontends
https://soundcloud.com/jobs
Thank you!
@grandbora
