![Page 1: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/1.jpg)
Wednesday, March 6, 13
![Page 2: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/2.jpg)
Applying Evolutionary Architecture on a
Popular API
phil calçado@pcalcado
http://philcalcado.com
QCon London Session Code: 1603Wednesday, March 6, 13
![Page 3: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/3.jpg)
Wednesday, March 6, 13
![Page 4: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/4.jpg)
Wednesday, March 6, 13
![Page 5: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/5.jpg)
> 50K registered client apps
~ 180 million users / month
~5 years old
Wednesday, March 6, 13
![Page 6: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/6.jpg)
Wednesday, March 6, 13
![Page 7: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/7.jpg)
API controllers UI controllers
domain logic
Wednesday, March 6, 13
![Page 8: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/8.jpg)
http://bit.ly/building-the-next-soundcloud
Wednesday, March 6, 13
![Page 9: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/9.jpg)
API controllers UI controllers
domain logic
Wednesday, March 6, 13
![Page 10: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/10.jpg)
API controllers
UI controllers
domain logic
internet
Wednesday, March 6, 13
![Page 11: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/11.jpg)
a new platform while keeping the old one
running?
Wednesday, March 6, 13
![Page 12: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/12.jpg)
WARNING:THIS IS
WORK IN PROGRESS
http://bit.ly/work-at-soundcloud and you can help us ;)
Wednesday, March 6, 13
![Page 13: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/13.jpg)
DO NOT BREAK CLIENT APPS
Wednesday, March 6, 13
![Page 14: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/14.jpg)
separate application
from domain
Wednesday, March 6, 13
![Page 15: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/15.jpg)
A set is a collection of
your own sounds
ClassicSoundCloud
NextSoundCloud
A set is a collection of
your own sounds
Wednesday, March 6, 13
![Page 16: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/16.jpg)
A set is a collection of
your own sounds
ClassicSoundCloud
NextSoundCloud
A set is a collection of
your own sounds
A set is a collection of
sounds
SoundCloudAPI
Wednesday, March 6, 13
![Page 17: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/17.jpg)
API controllers
UI controllers
domain logic
internet
Wednesday, March 6, 13
![Page 18: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/18.jpg)
API controllers
UI controllers
domain logic
internet
domain logic domain logic
Wednesday, March 6, 13
![Page 19: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/19.jpg)
applications enforce their own business rules
domain enforces “global” rules
Wednesday, March 6, 13
![Page 20: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/20.jpg)
applications enforce their own business rules
domain enforces “global” rules
(accidents happen...)
Wednesday, March 6, 13
![Page 21: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/21.jpg)
how to add/change features on the fly?
Wednesday, March 6, 13
![Page 22: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/22.jpg)
attempt #1: identify “official” apps
- too easy to spoof- abused by spammers- Android and iOS can’t easily rotate secrets
Wednesday, March 6, 13
![Page 23: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/23.jpg)
attempt #2: namespace endpoints
- “experimental” endpoints would link to regular ones but never be linked back- toyed with “lifted” hyperlinks, not enough time for launch
Wednesday, March 6, 13
![Page 24: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/24.jpg)
attempt #3: just use feature flags
- works ok for logged in users
- system is a little monster now, and ruby is not very good with numbers. rewriting in scala
Wednesday, March 6, 13
![Page 25: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/25.jpg)
SPLITTING THE MOTHERSHIP
Wednesday, March 6, 13
![Page 26: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/26.jpg)
define internal protocols
Wednesday, March 6, 13
![Page 27: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/27.jpg)
micro-services- couple thousands lines of code
- mostly JVM-based, JRuby, Clojure and Scala
- more about this at the SoundCoud pub night tomorrow: http://lanyrd.com/2013/soundcloudpub/ (Walrus Pub Waterloo)
Wednesday, March 6, 13
![Page 28: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/28.jpg)
HTTP+JSON & AMQP for most stuff
- moving AMQP usage for event broadcasting, as opposed to messaging
- no internal HTTP caching (yet?)
Wednesday, March 6, 13
![Page 29: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/29.jpg)
make it easy to get stuff in prod
- heroku-like deployment system
- tending towards twitter stack (Finagle) for I/O
Wednesday, March 6, 13
![Page 30: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/30.jpg)
URNs, not IDs
soundcloud:sounds:123
Wednesday, March 6, 13
![Page 31: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/31.jpg)
URNs, not IDs- everything addressable has a URN, some have URLs
- Mapping depends on context, e.g. given object may have a public URL and an internal URL
Wednesday, March 6, 13
![Page 32: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/32.jpg)
<3 hypermedia
- can easily remove things from the main app without breaking clients, just some HTTP 303s
Wednesday, March 6, 13
![Page 33: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/33.jpg)
OPTIMISE ALL THE THINGS!!11!
Wednesday, March 6, 13
![Page 34: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/34.jpg)
twitter moving back to server rendering...?
Wednesday, March 6, 13
![Page 35: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/35.jpg)
not too bad, actually
- besides load balancers vs. increase in number of requests per page
http://backstage.soundcloud.com/2012/08/evolution-of-soundclouds-architecture/
Wednesday, March 6, 13
![Page 36: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/36.jpg)
the actual problems
- number of requests from Next keeps increasing, page load affected- very different behaviour depending on the client app
Wednesday, March 6, 13
![Page 37: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/37.jpg)
API controllers
UI controllers
domain logic
internet
domain logic domain logic
Wednesday, March 6, 13
![Page 38: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/38.jpg)
API controllers
internet
UI controllers
Wednesday, March 6, 13
![Page 39: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/39.jpg)
attempt #1: optmising proxy for clients
UI controllers
domain logic
internet
domain logic domain logic
API controllers
Proxy
Wednesday, March 6, 13
![Page 40: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/40.jpg)
attempt #1: optmising proxy for clients
- still leverage public API- API has to know about how things behave in different clients
Wednesday, March 6, 13
![Page 41: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/41.jpg)
attempt #2: specialised back-end for official
clients
UI controllers
domain logic
internet
domain logic domain logic
Specialised Back-end API controllers
Wednesday, March 6, 13
![Page 42: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/42.jpg)
attempt #2: specialised back-end for official
clients
- each official app has its own back-end facade- no rendering, just data- public API becomes just one of them- front-end heavy teams can just script a common core using JRuby
Wednesday, March 6, 13
![Page 43: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/43.jpg)
NEXT STEPS
Wednesday, March 6, 13
![Page 44: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/44.jpg)
what we are looking at
- dataflow programming for the specialised back-ends
- ProtocolBuffers for Android
- make the mothership just a back-end service, not exposed to the Internet
Wednesday, March 6, 13
![Page 45: Applying Evolutionary Architecture on a Popular API](https://reader033.vdocuments.mx/reader033/viewer/2022052823/55530ca3b4c905533f8b4e99/html5/thumbnails/45.jpg)
phil calçado
http://philcalcado.com@pcalcado
www.soundcloud.com
Wednesday, March 6, 13